Before

Foreword

"API on Rails 6" is based on "APIs on Rails: Building REST APIs with Rails". It was initially published in 2014 by Abraham Kuri under the licenses MIT and Beerware.

The first version was not maintained and was initially planned for Ruby on Rails version 4, which does not receive more than security updates. I wanted to update this excellent book, adapting it to new versions of Ruby on Rails. Therefore, this book is available for Ruby on Rails versions 5.2 and 6.0 (the one you are currently reading).

Note
This book is also available in the Molière language (It means french).

About the author

My name is Alexandre Rousseau, and I am a Rails developer with more than 4 years of experience (at the time of writing). I am currently a partner in a company (iSignif) to build and maintain a SAAS product using Rails. I also contribute to the Ruby community by producing and maintaining some gems that you can consult on my Rubygems.org profile. Most of my projects are on GitHub, then don’t hesitate to follow me.

This book’s source code is available in Asciidoctor format on GitHub. Feel free to fork the project if you want to improve it or fix mistakes I didn’t notice.

This book is provided on MIT license. All the book’s source code is available on Markdown format on GitHub

MIT license

Copyright 2019 Alexandre Rousseau

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

This book’s cover picture uses a beautiful photo shot by Yoann Siloine who published it on Unsplash.

Thanks

A big "thank you" to all Github contributors who keep this book alive. In alphabetical order:

Introduction

Welcome to API on Rails 6, a tutorial on steroids to learn the best way to build your next API with Rails. The purpose of this book is to provide a comprehensive methodology to develop a RESTful API following best practices.

As soon as you finish this book, you will be able to create your own API and integrate it with any client, such as a web browser or mobile application. The generated code is built with Ruby on Rails 6.0, which is the current version.

The purpose of this book is not only to teach you how to build an API with Rails but rather to teach you how to build an evolutive and maintainable API with Rails. That is, improve your current knowledge with Rails. On this journey, you will learn to:

  • Use Git for version control

  • Building JSON responses

  • Test your end-points with unit and functional tests

  • Set up authentication with JSON Web Tokens (JWT)

  • Use JSON:API specification

  • Optimize and cache the API

I strongly recommend you follow all the steps in this book. Try not to skip chapters because I will give you some tips and tricks to improve your skills throughout the book. You can consider yourself the main character of a video game that gains a level in each chapter.

In the first chapter, I will explain how to configure your environment (if you don’t already have it). Then we will create an application called market_place_api. I will ensure that I teach you the best practices I have learned during my experience. This means that we’ll start using Git just after initializing the project.

We’ll build the application following a simple working method that I use daily in the next chapters. We will develop the entire application using Test Driven Development (TDD). I will also explain the interest of using an API for your next project and choosing a suitable response format such as JSON or XML. Further on, we will get our hands on the code and complete the application’s basics by building all the necessary roads. We will also secure access to the API by building authentication by exchanging HTTP headers. Finally, in the last chapter, we will add some optimization techniques to improve the server’s structure and response times.

The final application will scratch the surface of being a market place where users will be able to place orders, upload products, and more. There are plenty of options out there to set up an online store, such as Shopify, Spree, or Magento.

Conventions on this book

The conventions in this book are based on the ones from Ruby on Rails Tutorial. In this section, I’ll mention some that may not be so clear.

I’ll be using many examples using command-line instructions. I won’t deal with windows cmd (sorry guys), so all the examples use Unix-style command line prompt, as follows:

$ echo "A command-line command"
A command-line command

I’ll be using some guidelines related to the language. What I mean by this is:

  • Avoid means you are not supposed to do it

  • Prefer indicates that from the 2 options, the first it’s a better fit

  • Use means you are good to use the resource

If for any reason you encounter some errors when running a command, rather than trying to explain every possible outcome, I recommend you to `google it', which I don’t consider a bad practice or whatsoever. But if you feel like you want to grab a beer or have some trouble with the tutorial, you can always email me.

Development environments

One of the most painful parts for almost every developer is setting everything up, but as long as you get it done, the next steps should be a piece of cake and well rewarded. So I will guide you to keep you motivated.

Text editors and Terminal

There are many cases in which development environments may differ from computer to computer. That is not the case with text editors or IDE’s. I think for Rails development an IDE is way too much, but some other might find that the best way to go, so if that it’s your case I recommend you go with RadRails or RubyMine, both are well supported and come with many integrations out of the box.

  • Text editor: I personally use vim as my default editor with janus, which will add and handle many of the plugins you are probably going to use. In case you are not a vim fan like me, there are a lot of other solutions such as Sublime Text which is a cross-platform easy to learn and customize (this is probably your best option), it is highly inspired by TextMate (only available for Mac OS). A third option uses a more recent text editor from the guys at GitHub called Atom. It’s a promising text editor made with JavaScript. It is easy to extend and customize to meet your needs. Give it a try. Any of the editors I present will do the job, so I’ll let you decide which one fits your eye.

  • Terminal: If you decided to go with kaishi for setting the environment, you would notice that it sets the default shell to zsh, which I highly recommend. For the terminal, I’m not a fan of the Terminal app that comes out of the box if you are on Mac OS, so check out iTerm2, which is a terminal replacement for Mac OS. If you are on Linux, you probably have a nice terminal already, but the default should work fine.

Browsers

When it comes to browsers, I would say Firefox immediately, but some other developers may say Chrome or even Safari. Any of those will help you build the application you want. They come with a nice inspector not just for the DOM but also for network analysis and many other features you might know already.

Package manager

  • Mac OS: There are many options to manage how you install packages on your Mac, such as Mac Ports or Homebrew, both are good options, but I would choose the last one, I’ve encountered fewer troubles when I install software, and I manage it. To install brew, just run the command below:

$ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
  • Linux: You are all set! It really does not matter if you are using apt, pacman, yum as long you feel comfortable with it, and you know how to install packages so you can keep moving forward.

Git

We will be using Git a lot, and you should use it too, not just for the purpose of this tutorial but for every single project.

  • on Mac OS: $ brew install git

  • on Linux: $ sudo apt-get install git

Ruby

There are many ways in which you can install and manage ruby, and by now, you should probably have some version installed if you are on Mac OS. To see which version you have, just type:

$ ruby -v

Rails 6.0 requires the installation of version 2.5 or higher.

I recommend using Ruby Version Manager (RVM) or rbenv to install it. We will use RVM in this tutorial, but it doesn’t matter which of these two options you use.

The principle of these tools is allowing you to install several versions of Ruby on the same machine, in an environment that is airtight to a possible version installed on your operating system, and to be able to switch from one to the other easily.

To install RVM, go to https://rvm.io/ and install the GPG footnote key:[The GPG key allows you to verify the identity of the author of the sources you download.]. Once that’s done:

$ gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
$ \curl -sSL https://get.rvm.io | bash

Next, it is time to install ruby:

$ rvm install 2.6

Now it is time to install the rest of the dependencies we will be using.

Gems, Rails & Missing libraries

First, we update the gems on the whole system:

$ gem update --system

In some cases, if you are on a Mac OS, you will need to install some extra libraries:

$ brew install libtool libxslt libksba openssl

We then install the necessary gems and ignore documentation for each gem:

$ gem install bundler
$ gem install rails -v 6.0.0

Check for everything to be running nice and smooth:

$ rails -v
Rails 6.0.0

Database

I highly recommend you install Postgresql to manage your databases. But here, we’ll be using SQlite for simplicity. If you are using Mac OS, you should be ready to go. In case you are on Linux, don’t worry. We have you covered:

$ sudo apt-get install libxslt-dev libxml2-dev libsqlite3-dev

or

$ sudo yum install libxslt-devel libxml2-devel libsqlite3-devel

Initializing the project

Initializing a Rails application may be pretty straightforward for you. If that is not the case, here is a super quick tutorial.

There is the command:

$ mkdir ~/workspace
$ cd ~/workspace
$ rails new market_place_api --api
Note
The --api option appeared in version 5 of Rails. It allows you to limit the libraries and Middleware included in the application. This also avoids generating HTML views when using Rails generators.

As you may guess, the commands above will generate the bare bones of your Rails application.

Versioning

Remember that Git helps you track and maintain your code history. Keep in mind that the source code of the application is published on GitHub. You can follow the project on GitHub.

Ruby on Rails initialized the Git directory for you when you used the rails new command. This means that you do not need to execute the git init command.

However, it is necessary to configure the information of the author of commits. If you have not already done so, go to the directory and run the following commands:

$ git config --global user.name "Type in your name"
$ git config --global user.email "Type in your email"

Rails also provide a .gitignore file to ignore some files that we don’t want to track. The default .gitignore file should look like the one shown below:

.gitignore
# Ignore bundler config.
/.bundle

# Ignore the default SQLite database.
/db/*.sqlite3
/db/*.sqlite3-journal

# Ignore all logfiles and tempfiles.
/log/*
/tmp/*
!/log/.keep
!/tmp/.keep

# Ignore uploaded files in development.
/storage/*
!/storage/.keep
.byebug_history

# Ignore master key for decrypting credentials and more.
/config/master.key

After modifying the .gitignore file, we just need to add the files and commit the changes, the necessary commands are shown below:

$ git add .
$ git commit -m "Initial commit"
Tip
I have found that committing a message starting with a present tense verb, describing what the commit does and not what it did, helps when you are exploring the history of the project. I find it is more natural to read and understand. I’ll follow this practice until the end of the tutorial.

Lastly and as an optional step, we setup the GitHub (I’m not going through that here) project and push our code to the remote server: We first add the remote:

$ git remote add origin git@github.com:madeindjs/market_place_api_6.git

Then we push the code:

$ git push -u origin master

As we move forward with the tutorial, I’ll be using the practices I follow daily. This includes working with branches, rebasing, squash and some more. For now, you don’t have to worry if some of these don’t sound familiar to you. I walk you through them in time.

Conclusion

It’s been a long way through this chapter. If you reach here, let me congratulate you and be sure that things will get better from this point. So let’s get our hands dirty and start typing some code!

The API

In this section, I’ll outline the application. By now, you should have read the previous chapter. If you did not read it, I recommend you to do it.

You can clone the project until this point with:

$ git checkout tags/checkpoint_chapter02

To summarize, we simply generated our Rails application and made our first commit.

Planning the application

As we want to go simple with the application, it consists of five models. Don’t worry if you don’t fully understand what is going on. We will review and build each of these resources as we move on with the tutorial.

Schema of links betweens models

In brief, the user will be able to place many orders, upload multiple products which can have many images or comments from other users on the app.

We will not build views for displaying or interacting with the API, so not to make this a huge tutorial, I’ll let that to you. There are plenty of options out there like javascript frameworks (Angular, Vue.js, React.js).

By this point, you must be asking yourself:

all right, but I need to explore or visualize the API we will be building?

That’s fair. Probably if you google something related to API exploring, an application called Postman will pop. It is great software, but we won’t be using that anyway because we’ll use cURL, allowing anybody to reproduce requests on any computer.

Setting the API

An API is defined by wikipedia as an application programming interface (API) specifies how some software components should interact with each other. In other words, the way systems interact with each other through a common interface, in our case, a web service built with JSON. There are other kinds of communication protocols like SOAP, but we are not covering that here.

As the Internet media type standard, JSON is widely accepted, readable, extensible, and easy to implement. Many of the current frameworks consume JSON APIs by default (Angular or Vue.js, for example). There are also great libraries for Objective-C too like AFNetworking or RESTKit. There are probably good solutions for Android, but I might not be the right person to recommend something because of my lack of experience in that development platform.

All right. So we are building our API with JSON. There are many ways to achieve this. The first thing that comes to mind would be just to start adding routes defining the endpoints. This may be bad because they may not have a URI pattern clear enough to know which resource is being exposed. The protocol or structure I’m talking about is REST which stands for Representational State Transfer and by Wikipedia definition

aService.getUser("1")

And in REST you may call a URL with a specific HTTP request, in this case with a GET request: http://domain.com/resources_name/uri_pattern

RESTful APIs must follow at least three simple guidelines:

  • A base URI, such as http://example.com/resources/.

  • An Internet media type to represent the data is commonly JSON and is commonly set through header exchange.

  • Follows the standard HTTP Methods such as GET, POST, PUT, DELETE.

    • GET: Reads the resource or resources defined by the URI pattern

    • POST: Creates a new entry into the resources collection

    • PUT: Updates a collection or member of the resources

    • DELETE: Destroys a collection or member of the resources

This might not be clear enough or may look like a lot of information to digest, but hopefully, it’ll get a lot easier to understand as we move on with the tutorial.

Routes, Constraints and Namespaces

Before start typing any code, we prepare the code with git. We’ll be using a branch per chapter, upload it to GitHub and then merge it on master branch. So let’s get started. Open the terminal, cd to the market_place_api directory and type in the following:

$ git checkout -b chapter02
Switched to a new branch 'chapter02'

We will only be working on the config/routes.rb, as we are just going to set the constraints and the default response format for each request.

config/routes.rb
Rails.application.routes.draw do
  # ...
end

First of all, erase all commented code that comes within the file. We are not gonna need it. Then commit it, just as a warm-up:

$ git add config/routes.rb
$ git commit -m "Removes comments from the routes file"

We are going to isolate the API controllers under a namespace. With Rails, this is fairly simple: you just have to create a folder under the app/controllers named API. The name is important because that’s the namespace we’ll use for managing the controllers for the API endpoints.

$ mkdir app/controllers/api

Then we add that namespace into our routes.rb file:

config/routes.rb
Rails.application.routes.draw do
  # API definition
  namespace :api do
    # We are going to list our resources here
  end
end

By defining a namespace under the routes.rb file. Rails will automatically map that namespace to a directory matching the name under the controllers folder, in our case the api/` directory.

Rails media types supported

Rails can handle up to 35 different media types, you can list them by accessing the SET class under de Mime module:

$ rails c
2.6.3 :001 > Mime::SET.collect(&:to_s)
 => ["text/html", "text/plain", "text/javascript", "text/css", "text/calendar", "text/csv", "text/vcard", "text/vtt", "image/png", "image/jpeg", "image/gif", "image/bmp", "image/tiff", "image/svg+xml", "video/mpeg", "audio/mpeg", "audio/ogg", "audio/aac", "video/webm", "video/mp4", "font/otf", "font/ttf", "font/woff", "font/woff2", "application/xml", "application/rss+xml", "application/atom+xml", "application/x-yaml", "multipart/form-data", "application/x-www-form-urlencoded", "application/json", "application/pdf", "application/zip", "application/gzip"]

This is important because we are going to be working with JSON, one of the built-in MIME types accepted by Rails, so we just need to specify this format as the default one:

config/routes.rb
Rails.application.routes.draw do
  # API definition
  namespace :api, defaults: { format: :json }  do
    # We are going to list our resources here
  end
end

Up to this point, we have not made anything crazy. What we want to generate is a base_uri which includes the API version. But let’s commit changes before going to the next section:

$ git add config/routes.rb
$ git commit -m "Set the routes constraints for the API"

Api versioning

At this point, we should have a nice route mapping using a namespace. Your routes.rb file should look like this:

config/routes.rb
Rails.application.routes.draw do
  # API definition
  namespace :api, defaults: { format: :json }  do
    # We are going to list our resources here
  end
end

Now it is time to set up some other constraints for versioning purposes. You should care about versioning your application from the beginning since this will give a better structure to your API. When changes need to be made, you can give developers who are consuming your API the opportunity to adapt to the new features while the old ones are being deprecated. There is an excellent railscast explaining this.

To set the version for the API, we first need to add another directory under the API we created:

$ mkdir app/controllers/api/v1

This way we can namespace our api into different versions very easily, now we just need to add the necessary code to the routes.rb file:

config/routes.rb
Rails.application.routes.draw do
  # Api definition
  namespace :api, defaults: { format: :json }  do
    namespace :v1 do
      # We are going to list our resources here
    end
  end
end

By this point, the API is now scoped via the URL. For example, with the current configuration, an endpoint for retrieving a product would be like http://localhost:3000/v1/products/1.

Common API patterns

You can find many approaches to set up the base_uri when building an API following different patterns, assuming we are versioning our API:

  • api.example.com/: In my opinion, this is the way to go, gives you a better interface and isolation, and in the long term can help you to quickly scalate

  • example.com/api/: This pattern is very common, and it is actually a good way to go when you don’t want to namespace your API under a subdomain

  • example.com/api/v1: it seems like a good idea by setting the version of the API through the URL seems like a more descriptive pattern, but this way you enforce the version to be included on URL on each request, so if you ever decide to change this pattern, this becomes a problem of maintenance in the long-term

There are some practices in API building that recommend not to version the API via the URL. That’s true. The developer should not be aware of the version he’s using. For simplicity, I have chosen to set aside this convention, which we will be able to apply in a second phase.

It is time to commit:

$ git commit -am "Set the versioning namespaces for API"

We are at the end of our chapter. Therefore, it is time to apply all our modifications to the master branch by making a merge. To do this, we place ourselves on the master branch and we merge chapter02:

$ git checkout master
$ git merge chapter02

Conclusion

I know it’s been a long way, but you made it, don’t give up this is just our small scaffolding for something big, so keep it up. In the meantime, and if you feel curious, some gems handle this kind of configuration:

I’m not covering those in this book since we are trying to learn how to implement this kind of functionality, but it is good to know. By the way, the code up to this point is here.

Presenting users

In the last chapter, we manage to set up the bare bones for our application endpoints configuration.

In the next chapter, we will handle user authentication through authentication tokens and setting permissions to limit access for, let’s say, signed-in users. In the coming chapters, we will relate products to users and give them the ability to place orders.

You can clone the project until this point with:

$ git checkout tags/checkpoint_chapter03

As you can already imagine, there are a lot of authentication solutions for Rails, AuthLogic, Clearance, and Devise.

These solutions are turnkey libraries, i.e., they allow you to manage many things like authentication, password forgetfulness, validation, etc…​ Nevertheless, we will use bcrypt gem to hash the user’s password.

This chapter will be complete. It may be a long one, but I will try to cover as many topics as possible. Feel free to have a coffee, and let’s go. At the end of this chapter, you will have built all the user logic and validation and error management.

It is a good time to create a new branch:

$ git checkout -b chapter03
Note
Just make sure you are on the master branch before checking out.

User model

Generation of the User model

We will start by generating our User model. This model will be really simple and will contain only two fields:

  • email which will be unique and will allow it to connect to the application

  • password_digest which will contain the hashed version of the password (we will discuss this later in this chapter)

We generate our User model using the generate model command provided by Ruby on Rails. It is very easy to use:

$ rails generate model User email:string password_digest:string
invoke  active_record
      create    db/migrate/20190603195146_create_users.rb
      create    app/models/user.rb
      invoke    test_unit
      create      test/models/user_test.rb
      create      test/fixtures/users.yml
Note
The model is the element containing the data and the logic related to the data: validation, reading, and recording.

This command generates a lot of files! Don’t worry, we’ll review them one by one.

The migration file contained in the db/migrate folder contains the migration that describes the changes that will be made to the database. This file should look like this:

db/migrate/20190603195146_create_users.rb
class CreateUsers < ActiveRecord::Migration[6.0]
  def change
    create_table :users do |t|
      t.string :email
      t.string :password_digest

      t.timestamps
    end
  end
end
Note
The inserted date at the beginning of the migration file name should be different for you since it corresponds to the migration creation date.

We will make a small change to the migration to add some database validations. With Rails, it is common practice to make these verifications directly in the Ruby model. It is good practice to do so also in the database schema.

We will, therefore add two additional constraints:

  • email is mandatory: we use the property null: false.

  • email must be unique: we add an index for the email column with property unique: true.

  • password is mandatory: we use the property null: false.

The migration thus becomes:

db/migrate/20190603195146_create_users.rb
# ...
create_table :users do |t|
  t.string :email, null: false
  t.index :email, unique: true
  t.string :password_digest, null: false
  # ...
end

We can run changes once the migration is complete with the following command:

db/migrate/20190603195146_create_users.rb
$ rake db:migrate
== 20190603195146 CreateUsers: migrating ======================================
-- create_table(:users)
   -> 0.0027s
== 20190603195146 CreateUsers: migrated (0.0028s) =============================
Note
This command will convert our migration into a SQL query that will update the SQlite3 database in the db folder.

Model

So we defined our database schema. The next step is to update our model to define validation rules. These rules are defined in the template located in the app/models folder.

Ruby on Rails provides a complete validation mechanism found at their official documentation. In our case, we want to validate only three things:

  1. the email must have a valid format

  2. the email must be unique

  3. the password must be filled in

These three rules are defined by the following code:

app/models/user.rb
class User < ApplicationRecord
  validates :email, uniqueness: true
  validates_format_of :email, with: /@/
  validates :password_digest, presence: true
end

There you go. Rails use a straightforward syntax, and the code is very readable.

Email validation

You may notice that the email validation uses a simplistic validation by only checking for the presence of a @.

That’s normal.

There are infinite exceptions to the email address so well that even Look at all these spaces!@example.com is a valid address. Therefore, it is better to favor a simple approach and confirm the email by a validation email.

Unit tests

We end with the unit tests. We use here the Minitest test framework, which is provided by default with Rails.

Minitest is based on Fixtures, which allows you to fill your database with predefined data. Fixtures are defined in YAML files in the tests/fixtures folder. There is one file per template.

We must, therefore start by updating our tests/fixtures.

Note
fixtures are not designed to create all the data your tests need. They just allow you to define the basic data your application needs.

So we will start by creating a fixture defining a user:

test/fixtures/users.yml
one:
  email: one@one.org
  password_digest: hashed_password

So we can now create three tests:

  • 1. Check that a user with valid data is valid:

test/models/user_test.rb
# ...
test 'user with a valid email should be valid' do
  user = User.new(email: 'test@test.org', password_digest: 'test')
  assert user.valid?
end
  • 2. Check that a user with an invalid email address is not valid:

test/models/user_test.rb
# ...
test 'user with invalid email should be invalid' do
  user = User.new(email: 'test', password_digest: 'test')
  assert_not user.valid?
end
  • 3. Check that a new user with a duplicate email is not valid. So we use the same email as the fixture we just created.

test/models/user_test.rb
# ...
test 'user with taken email should be invalid' do
  other_user = users(:one)
  user = User.new(email: other_user.email, password_digest: 'test')
  assert_not user.valid?
end

There you go. We can verify that our implementation is correct just by simply running unit tests we have just created:

$ rake test
...
3 runs, 3 assertions, 0 failures, 0 errors, 0 skips

I think it’s time to do a little commit to validate our progress:

$ git add . && git commit -m "Create user model"

Password hash

We have previously implemented the storage of user data. We still have a problem to solve: the storage of passwords is in clear text.

If you store user passwords clearly, then an attacker who steals a copy of your database has a giant list of emails and passwords. Some of your users will only have one password — for their email account, for their banking account, for your application. A simple hack could escalate into massive identity theft. - source - Why you should use bcrypt

So we will use the bcrypt gem to hash the password.

Note
Hash is the process of transforming a character string into Hash. This Hash does not allow you to find the original character string. However, we can easily use it to determine if a given character string matches the hash we have stored.

We must first add the Bcrypt gem to the Gemfile. We can use the bundle add command. This one will:

  1. add the gem to the Gemfile by retrieving the current version

  2. launch the command bundle install which will install the gem and update the file Gemfile.lock which "locks" the current version of the gem

Therefore, issue the following command:

$ bundle add bcrypt

Once the command is executed, the following line is added at the end of the Gemfile:

Gemfile
gem "bcrypt", "~> 3.1"
Note
Version 3.1 of bcrypt is the current version at the time of writing. It may therefore vary for your case.

Active Record offers us a method ActiveModel::SecurePassword::has_secure_password that will interface with Bcrypt and hack the password for us very easily.

app/models/user.rb
class User < ApplicationRecord
  # ...
  has_secure_password
end

has_secure_password adds the following validations:

  • The password must be present when creating.

  • The password length must be less than or equal to 72 bytes.

  • The confirmation of the password using the attribute password_confirmation (if sent)

This method will also add a User#password attribute that will be automatically hashed and saved in the User#password_digest attribute.

Let’s try this right now in the Rails console. Open a console with rails console:

2.6.3 :001 > User.create! email: 'toto@toto.org', password: '123456'
 =>#<User id: 1, email: "toto@toto.org", password_digest: [FILTERED], created_at: "2019-06-04 10:51:44", updated_at: "2019-06-04 10:51:44">

You can see that when you call the User#create! method, the password attribute is hashed and stored in password_digest. We can also send a password_confirmation attribute that ActiveRecord will compare to password:

2.6.3 :002 > User.create! email: 'tata@tata.org', password: '123456', password_confirmation: 'azerty'
ActiveRecord::RecordInvalid (Validation failed: Password confirmation doesn t match Password)

Everything is working as planned! Let’s now make a committe to keep the history concise:

$ git commit -am "Setup Bcrypt"

Build users

It’s time to make our first entry point. We will begin by building the show action, which will respond with a single user in the JSON data format. The steps are:

  1. generate the users_controller.

  2. add the corresponding tests

  3. build the real code.

Let’s first focus on generating the controller and functional tests.

To respect the viewing of our API, we will cut our application using modules. The syntax is, therefore, as follows:

$ rails generate controller api::v1::users

This command will create users_controller_test.rb file. Before going further, there are two things we want to test for an API:

  • The JSON structure returned by the server

  • The HTTP response code returned by the server

Common HTTP codes

The first digit of the status code specifies one of the five response classes. The bare minimum for an HTTP client is that it uses one of these five classes. Here is a list of commonly used HTTP codes:

  • 200: Standard response for successful HTTP requests. This is usually on GET requests

  • 201: The demand was met and resulted in the creation of a new resource. After the POST requests

  • 204: The server has successfully processed the request but does not return any content. This is usually a successful DELETE request.

  • 400: The request cannot be executed due to bad syntax. It can happen for any type of request.

  • 401: Similar to 403, but specifically for use when authentication is required and has failed or has not yet been provided. It can happen for any type of request.

  • 404: The requested resource could not be found but may be available again in the future. Usually concerns GET requests

  • 500: A generic error message, given when an unexpected condition has been encountered, and no other specific message is appropriate.

For a complete list of HTTP response codes, see Wikipedia article.

We will therefore implement the functional test that verifies access to the Users#show method,

test/controllers/api/v1/users_controller_test.rb
# ...
class Api::V1::UsersControllerTest < ActionDispatch::IntegrationTest
  setup do
    @user = users(:one)
  end

  test "should show user" do
    get api_v1_user_url(@user), as: :json
    assert_response :success
    # Test to ensure response contains the correct email
    json_response = JSON.parse(self.response.body)
    assert_equal @user.email, json_response['email']
  end
end

Then simply add the action to our controller. It is extremely simple:

app/controllers/api/v1/users_controller.rb
class  Api::V1::UsersController < ApplicationController
  # GET /users/1
  def show
    render json: User.find(params[:id])
  end
end

If you run the tests with rails test you get the following error:

$ rails test

...E

Error:
UsersControllerTest#test_should_show_user:
DRb::DRbRemoteError: undefined method `api_v1_user_url` for #<UsersControllerTest:0x000055ce32f00bd0> (NoMethodError)
    test/controllers/users_controller_test.rb:9:in `block in <class:UsersControllerTest>`

This type of error is very common when you generate your resources manually! Indeed, we have totally forgotten the route. So let’s add them:

config/routes.rb
Rails.application.routes.draw do
  namespace :api, defaults: { format: :json } do
    namespace :v1 do
      resources :users, only: [:show]
    end
  end
end

Tests should now pass:

$ rails test
....
4 runs, 5 assertions, 0 failures, 0 errors, 0 skips

As usual, after adding one of the features we are satisfied with, we make a commit:

$ git add . && git commit -m "Adds show action to the users controller"

Test our resource with cURL

So we finally have a resource to test. We have several solutions to test it. The first one that comes to mind is cURL, which is integrated into almost all Linux distributions. So let’s try it:

First, initialize the rails server on a new terminal.

$ rails s

Then switch back to your other terminal and run:

$ curl http://localhost:3000/api/v1/users/1
{"id":1,"email":"toto@toto.org", ...

We find the user we created with the Rails console in the previous section. You now have a user registration API entry.

Create users

Now that we have a better understanding of building entry points, it is time to extend our API. One of the most important features is to let users create a profile on our application. As usual, we will write tests before implementing our code to extend our test suite.

Ensure that your Git directory is clean and that you do not have a file in staging. If so commit them so that we can start over.

So let’s start by writing our test by adding an entry to create a user on the file users_controller_test.rb:

test/controllers/users_controller_test.rb
# ...
class Api::V1::UsersControllerTest < ActionDispatch::IntegrationTest
  # ...
  test "should create user" do
    assert_difference('User.count') do
      post api_v1_users_url, params: { user: { email: 'test@test.org', password: '123456' } }, as: :json
    end
    assert_response :created
  end

  test "should not create user with taken email" do
    assert_no_difference('User.count') do
      post api_v1_users_url, params: { user: { email: @user.email, password: '123456' } }, as: :json
    end
    assert_response :unprocessable_entity
  end
end

That’s a lot of code. Don’t worry I’ll explain everything:

  • In the first test, we check the user’s creation by sending a valid POST request. Then, we checked that an additional user exists in the database and that the HTTP code of the response is created (status code 201)

  • In the second test, we check that the user is not created using an email already used. Then, we check that the HTTP code of the response is unprocessable_entity (status code 422)

At that point, the tests must fail (as we expected):

$ rails test
...E

So it’s time to implement the code for our tests to be successful:

app/controllers/api/v1/users_controller.rb
class Api::V1::UsersController < ApplicationController
  # ...

  # POST /users
  def create
    @user = User.new(user_params)

    if @user.save
      render json: @user, status: :created
    else
      render json: @user.errors, status: :unprocessable_entity
    end
  end

  private

  # Only allow a trusted parameter "white list" through.
  def user_params
    params.require(:user).permit(:email, :password)
  end
end

Remember that each time we add an entry in our API we must also add this action in our routes.rb file.

config/routes.rb
Rails.application.routes.draw do
  namespace :api, defaults: { format: :json } do
    namespace :v1 do
      resources :users, only: %i[show create]
    end
  end
end

As you can see, the implementation is quite simple. We have also added the private method user_params to protect mass attribute assignments. Now our tests should pass:

$ rails test
......
6 runs, 9 assertions, 0 failures, 0 errors, 0 skips

Yeah! Let’s commit changes and continue to build our application:

$ git commit -am "Adds the user create endpoint"

Update users

The user update scheme is very similar to the one at creation. If you are an experienced Rails developer, you may already know the differences between these two actions:

  • The update action responds to a PUT/PATCH request.

  • Only a connected user should be able to update his information. This means that we will have to force a user to authenticate. We will discuss this in Chapter 5.

As usual, we start by writing our tests:

test/controllers/users_controller_test.rb
# ...
class Api::V1::UsersControllerTest < ActionDispatch::IntegrationTest
  # ...
  test "should update user" do
    patch api_v1_user_url(@user), params: { user: { email: @user.email, password: '123456' } }, as: :json
    assert_response :success
  end

  test "should not update user when invalid params are sent" do
    patch api_v1_user_url(@user), params: { user: { email: 'bad_email', password: '123456' } }, as: :json
    assert_response :unprocessable_entity
  end
end

For the tests to succeed, we must build the update action on the file users_controller.rb and add the route to the file routes.rb. As you can see, we have too much-duplicated code. We will redesign our tests in chapter 4. first, we add the action the file routes.rb:

config/routes.rb
Rails.application.routes.draw do
  # ...
  resources :users, only: %i[show create update]
  # ...
end

Then we implement the update action on the user controller and run our tests:

app/controllers/api/v1/users_controller.rb
class Api::V1::UsersController < ApplicationController
  before_action :set_user, only: %i[show update]

  # GET /users/1
  def show
    render json: @user
  end

  # ...

  # PATCH/PUT /users/1
  def update
    if @user.update(user_params)
      render json: @user, status: :ok
    else
      render json: @user.errors, status: :unprocessable_entity
    end
  end

  private
  # ...

  def set_user
    @user = User.find(params[:id])
  end
end

All our tests should now pass:

$ rails test
........
8 runs, 11 assertions, 0 failures, 0 errors, 0 skips

We do a commit Since everything works:

$ git commit -am "Adds update action the users controller"

Delete the user

So far, we have built many actions on the user controller with their tests, but it is not finished. We just need one more, which is the action of destruction. So let’s create the test:

test/controllers/users_controller_test.rb
# ...
class Api::V1::UsersControllerTest < ActionDispatch::IntegrationTest
  # ...

  test "should destroy user" do
    assert_difference('User.count', -1) do
      delete api_v1_user_url(@user), as: :json
    end
    assert_response :no_content
  end
end

As you can see, the test is straightforward. We only respond with a status of 204, which means No Content. We could also return a status code of 200, but I find it more natural to answer No Content in this case because we delete a resource, and a successful response may be enough.

The implementation of the destruction action is also quite simple:

app/controllers/api/v1/users_controller.rb
class Api::V1::UsersController < ApplicationController
  before_action :set_user, only: %i[show update destroy]
  # ...

  # DELETE /users/1
  def destroy
    @user.destroy
    head 204
  end

  # ...
end

Don’t forget to add the action destroy in the file routes.rb:

config/routes.rb
Rails.application.routes.draw do
  # ...
  resources :users, only: %i[show create update destroy]
  # ...
end

Tests should pass if everything is correct:

$ rails test
.........
9 runs, 13 assertions, 0 failures, 0 errors, 0 skips

Remember that after making some changes to our code, it is good practice to commit them to keep a well-cut history.

$ git commit -am "Adds destroy action to the users controller"

And as we get to the end of our chapter, it is time to apply all our modifications to the master branch by doing a merge:

$ git checkout master
$ git merge chapter03

Conclusion

Oh, there you are! Well done! I know it was probably a long time, but don’t give up! Make sure you understand each piece of code, things will improve. In the next chapter, we will redesign our tests to make the code more readable and maintainable. Then stay with me!

Authenticating user

It’s been a long time since you started. I hope you enjoy this trip as much as I do.

In the previous chapter, we set up user resource entries. If you have skipped this chapter or have not understood everything, I strongly recommend looking at it. It covers the first bases of the tests and is an introduction to JSON answers.

You can clone the project up to this point:

$ git checkout tags/checkpoint_chapter04

In this chapter, things will get very interesting because we will set up our authentication mechanism. In my opinion, it’s one of the most interesting chapters. We will introduce many new terms, and you will end with a simple but powerful authentication system. Don’t feel panic. We will get to that.

First things first (and as usual when starting a new chapter), we will create a new branch:

$ git checkout -b chapter04

Stateless session

Before we go any further, something must be clear: an API does not handle sessions. If you don’t have experience building this kind of application, it might sound a little crazy but stay with me. An API should be stateless, which means by definition is one that provides a response after your request and then requires no further attention.. This means no previous or future state is required for the system to work.

The flow for authenticating the user through an API is effortless:

  1. The client request for sessions resource with the corresponding credentials (usually email and password)

  2. The server returns the user resource along with its corresponding authentication token.

  3. For every page that requires authentication, the client has to send that authentication token

Of course, this is not the only 3-step to follow, and even on step 2, you might think, well, do I really need to respond with the entire user or just the authentication token? It really depends on you, but I like to return the entire user. This way, I can map it right away on my client and save another possible request from being placed.

This section and the next will focus on building a Sessions controller and its corresponding actions. We’ll then complete the request flow by adding the necessary authorization access.

JWT presentation

When it comes to authentication tokens, there is a standard: the JSON Web Token (JWT).

JWT is an open standard defined in RFC 75191. It allows the secure exchange of tokens between several parties. - Wikipedia

Overall a JWT token is composed of three parts:

  • a header structured in JSON contains, for example, the validity date of the token.

  • a payload structured in JSON can contain any data. In our case, it will contain the identifier of the "connected" user. A signature allows us to verify that our application has encrypted the token and is therefore valid.

These three parts are each encoded in base64 and then concatenated using points (.). This gives us something like that:

A valid JWT token
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c

Once decoded, this token gives us the following information:

The JWT token header
{ "alg": "HS256", "typ": "JWT" }
The payload of the JWT token
{ "sub": "1234567890", "name": "John Doe", "iat": 1516239022 }
Note
For more information about JWT tokens I invite you to visit jwt.io

This has many advantages, such as sending information in token’s payload. For example, we may choose to integrate user’s information into the payload.

Setting up the authentication token

The JWT standard has many implementations in various languages and libraries. Of course, there is a Ruby gem on this subject: ruby-jwt.

So let’s start by installing it:

$ bundle add jwt

Once completed the following line is added to your Gemfile:

gem "jwt", "~> 2.2"

The library is very simple. There are two methods: JWT.encode and JWT.decode. Let’s open a terminal with console rails and run some tests:

2.6.3 :001 > token = JWT.encode({message: 'Hello World'}, 'my_secret_key')
2.6.3 :002 > JWT.decode(token, 'my_secret_key')
 => [{"message"=>"Hello World"}, {"alg"=>"HS256"}]

In the first line, we encoded a payload with the secret key my_secret_key. So we get a token we can simply decode. The second line decodes the token, and we see that we find our payload well.

We will now include all this logic in a JsonWebToken class in a new file located in lib/. This will allow us to avoid duplicating the code. This class will just encode and decode the JWT tokens. So here is the implementation.

lib/json_web_token.rb
class JsonWebToken
  SECRET_KEY = Rails.application.credentials.secret_key_base.to_s

  def self.encode(payload, exp = 24.hours.from_now)
    payload[:exp] = exp.to_i
    JWT.encode(payload, SECRET_KEY)
  end

  def self.decode(token)
    decoded = JWT.decode(token, SECRET_KEY).first
    HashWithIndifferentAccess.new decoded
  end
end

I know that’s a lot of code, but we’re going to review it together.

  • the method JsonWebToken.encode takes care of encoding the payload by adding an expiration date of 24 hours by default. We also use the same encryption key as the one configured with Rails

  • the method JsonWebToken.decode decodes the JWT token and gets the payload. Then we use the HashWithIndifferentAccess class provided by Rails, which allows us to retrieve a value of a Hash with a Symbol or String.

There you go. To load the file into our application, you must specify the lib folder in the list of Ruby on Rails _autoload_s. To do this, add the following configuration to the application.rb file:

config/application.rb
# ...
module MarketPlaceApi
  class Application < Rails::Application
    # ...
    config.eager_load_paths << Rails.root.join('lib')
  end
end

And that’s it. Now it’s time to make a commit:

$ git add . && git commit -m "Setup JWT gem"

Token’s controller

We have, therefore, set up the system for generating a JWT token. It’s now time to create a route that will generate this token. The actions we will implement will be managed as RESTful services: the connection will be managed by a POST request to the create action.

We will start by creating the controller of and method create in the namespace /API/v1. With Rails, one order is sufficient:

$ rails generate controller api::v1::tokens create

We will modify the route a little to respect the REST conventions:

config/routes.rb
Rails.application.routes.draw do
  namespace :api, defaults: { format: :json } do
    namespace :v1 do
      # ...
      resources :tokens, only: [:create]
    end
  end
end

We will build functional tests before going any further. The desired behavior is the following:

  • I receive a token if I send a valid email/password pair

  • otherwise, the server responds with a forbidden response

The tests therefore materialize as follows:

test/controllers/api/v1/tokens_controller_test.rb
require 'test_helper'

class Api::V1::TokensControllerTest < ActionDispatch::IntegrationTest
  setup do
    @user = users(:one)
  end

  test 'should get JWT token' do
    post api_v1_tokens_url, params: { user: { email: @user.email, password: 'g00d_pa$$' } }, as: :json
    assert_response :success

    json_response = JSON.parse(response.body)
    assert_not_nil json_response['token']
  end

  test 'should not get JWT token' do
    post api_v1_tokens_url, params: { user: { email: @user.email, password: 'b@d_pa$$' } }, as: :json
    assert_response :unauthorized
  end
end

You may be wondering: "but how can you know the user’s password?". Simply use the BCrypt::Password.create method in the fixtures of users:

test/fixtures/users.yml
one:
  email: one@one.org
  password_digest: <%= BCrypt::Password.create('g00d_pa$$') %>

At this precise moment, if you run the tests, you get two errors:

$ rake test

........E

Error:
Api::V1::TokensControllerTest#test_should_get_JWT_token:
JSON::ParserError: 767: unexpected token at ''


Failure:
Expected response to be a <401: unauthorized>, but was a <204: No Content>

That’s normal. It’s now time to implement the logic to create the JWT token. It is effortless.

app/controllers/api/v1/tokens_controller.rb
class Api::V1::TokensController < ApplicationController
  def create
    @user = User.find_by_email(user_params[:email])
    if @user&.authenticate(user_params[:password])
      render json: {
        token: JsonWebToken.encode(user_id: @user.id),
        email: @user.email
      }
    else
      head :unauthorized
    end
  end

  private

  # Only allow a trusted parameter "white list" through.
  def user_params
    params.require(:user).permit(:email, :password)
  end
end

That’s a lot of code, but it’s straightforward:

  1. We always filter parameters with the method user_params.

  2. We retrieve the user with the method User.find_by_email (which is a "magic" method of Active Record since the field email is present in the database), and we retrieve the user.

  3. We use the method User#authenticate (which exists thanks to the gem bcrypt) with the password as a parameter. Bcrypt will hash password and check if it matches the attribute password_digest. The function returns true if everything went well, false if not.

  4. If the password corresponds to the hash, a JSON containing the token generated with the class JsonWebToken is returned. Otherwise, an empty response is returned with an unauthorized header.

Are you still here? Don’t worry, it’s over! Now your tests must pass.

$ rake test

...........

Finished in 0.226196s, 48.6304 runs/s, 70.7351 assertions/s.
11 runs, 16 assertions, 0 failures, 0 errors, 0 skips

Very good! It’s time to make a commit that will contain all our changes:

$ git add . && git commit -m "Setup tokens controller"

Logged user

We implemented the following logic: API returns the authentication token to the client if credentials are correct.

We will now implement the following logic: we’ll find the corresponding user of the authentication token given into the HTTP header. We’ll need to do so each time this client requests an entry point that requires permission.

We will use the HTTP header Authorization, which is often used for this purpose. We may also use a GET parameter named apiKey but I prefer to use an HTTP header because it gives context to the request without polluting the URL with additional parameters.

We will therefore create a current_user method to meet our needs. It will find the user thanks his authentication token, which is sent on each request.

When it comes to authentication, I like adding all the associated methods in a separate file. Then simply include the file in the ApplicationController. In this way, it’s straightforward to test in isolation. Let’s create the file in the controllers/concerns directory with a current_user method that we will implement right after:

app/controllers/concerns/authenticable.rb
module Authenticable
  def current_user
    # TODO
  end
end

Then, let’s create a concerns directory under tests/controllers/ and an authenticable_test.rb file for our authentication tests:

$ mkdir test/controllers/concerns
$ touch test/controllers/concerns/authenticable_test.rb

As usual, we start by writing our tests. In this case, our current_user method will search for a user by the authentication token in the HTTP header Authorization. The test is quite basic:

test/controllers/concerns/authenticable_test.rb
# ...
class AuthenticableTest < ActionDispatch::IntegrationTest
  setup do
    @user = users(:one)
    @authentication = MockController.new
  end

  test 'should get user from Authorization token' do
    @authentication.request.headers['Authorization'] = JsonWebToken.encode(user_id: @user.id)
    assert_equal @user.id, @authentication.current_user.id
  end

  test 'should not get user from empty Authorization token' do
    @authentication.request.headers['Authorization'] = nil
    assert_nil @authentication.current_user
  end
end

You may be wondering, "Where does the MockController come from?". It is a Mock, i.e., a class that imitates another’s behavior to test a behavior.

We can define the MockController class just above our test:

test/controllers/concerns/authenticable_test.rb
# ...
class MockController
  include Authenticable
  attr_accessor :request

  def initialize
    mock_request = Struct.new(:headers)
    self.request = mock_request.new({})
  end
end
# ...

The MockController class simply includes our Authenticable module that we will test. It contains a request attribute that contains a simple Struct that mimics the behavior of a Rails request by containing a headers attribute of the type Hash.

Then we can implement our two tests right after

test/controllers/concerns/authenticable_test.rb
# ...
class AuthenticableTest < ActionDispatch::IntegrationTest
  setup do
    @user = users(:one)
    @authentication = MockController.new
  end

  test 'should get user from Authorization token' do
    @authentication.request.headers['Authorization'] = JsonWebToken.encode(user_id: @user.id)
    assert_not_nil @authentication.current_user
    assert_equal @user.id, @authentication.current_user.id
  end

  test 'should not get user from empty Authorization token' do
    @authentication.request.headers['Authorization'] = nil
    assert_nil @authentication.current_user
  end
end

Our tests must fail. So let’s implement the code so that it can be passed:

app/controllers/concerns/authenticable.rb
module Authenticable
  def current_user
    return @current_user if @current_user

    header = request.headers['Authorization']
    return nil if header.nil?

    decoded = JsonWebToken.decode(header)

    @current_user = User.find(decoded[:user_id]) rescue ActiveRecord::RecordNotFound
  end
end

There you go! We get the token from the header Authorization and we look for the corresponding user. Nothing very witchcraft.

Now our tests must pass:

$ rake test
.............
13 runs, 18 assertions, 0 failures, 0 errors, 0 skips

All we have to do is include the Authenticable module in the ApplicationController class:

app/controllers/application_controller.rb
class ApplicationController < ActionController::API
  # ...
  include Authenticable
end

And now it is time to commit our changes:

$ git add . && git commit -m "Adds authenticable module for managing authentication methods"

Authentication with the token

Authorization plays an important role in constructing applications because it helps us define what the user is allowed to do.

We have a route to update the user, but there is a problem: anyone can update any user. This section will implement a method that will require the user to be logged in to prevent unauthorized access.

Authorize actions

It is now time to update our users_controller.rb file to refuse access to certain actions. We will also implement the current_user method on the update and destroy action to ensure that the user who is logged in will only be able to update his data and only delete (and only) his account.

Therefore, we will split our test should update user and should destroy user into two tests.

Let’s start by updating the should update user test.

test/controllers/api/v1/users_controller_test.rb
# ...
class Api::V1::UsersControllerTest < ActionDispatch::IntegrationTest
  # ...
  test "should update user" do
    patch api_v1_user_url(@user),
      params: { user: { email: @user.email } },
      headers: { Authorization: JsonWebToken.encode(user_id: @user.id) },
      as: :json
    assert_response :success
  end

  test "should forbid update user" do
    patch api_v1_user_url(@user), params: { user: { email: @user.email } }, as: :json
    assert_response :forbidden
  end
end

You can see how we have to add a header Authorization for the user’s modification action. We want to receive a forbidden response if we don’t.

We can imagine about the same thing for the test should forbid destroy user:

test/controllers/api/v1/users_controller_test.rb
# ...
class Api::V1::UsersControllerTest < ActionDispatch::IntegrationTest
  # ...
  test "should destroy user" do
    assert_difference('User.count', -1) do
      delete api_v1_user_url(@user), headers: { Authorization: JsonWebToken.encode(user_id: @user.id) }, as: :json
    end
    assert_response :no_content
  end

  test "should forbid destroy user" do
    assert_no_difference('User.count') do
      delete api_v1_user_url(@user), as: :json
    end
    assert_response :forbidden
  end
end

Theses tests should fail for the moment as you might expect:

$ rails test test/controllers/api/v1/users_controller_test.rb
..F

Failure:
Expected response to be a <2XX: success>, but was a <403: Forbidden>

..F

Failure:
"User.count" didn t change by -1.
Expected: 0
  Actual: 1

The solution is quite simple. We will add a before_action, which will call the check_owner method for the update and destroy actions. This way, we will check that the user corresponding to the JWT token is the same as the user who needs to be updated.

Here is the implementation:

app/controllers/api/v1/users_controller.rb
class Api::V1::UsersController < ApplicationController
  before_action :set_user, only: %i[show update destroy]
  before_action :check_owner, only: %i[update destroy]
  # ...

  private
  # ...
  def check_owner
    head :forbidden unless @user.id == current_user&.id
  end
end

There you go! The implementation is straightforward. It is, therefore, time to commit:

$ git commit -am "Restrict actions for unauthorized users"
$ git checkout master
$ git merge chapter04

Conclusion

Yeah! You made it! You are halfway done! Keep up the good work. This chapter was a long and hard one, but it is a great step toward setting a solid mechanism for handling user authentication. We even scratch the surface for simple authorization rules.

In the next chapter, we will be focusing on customizing the JSON output for the user with fast_jsonapi gem and adding a product model to the equation by giving the user the ability to create a product and publish it for sale.

User’s products

In the previous chapter, we implemented the authentication mechanism we will use throughout the application.

We have a straightforward implementation of the User model, but the moment of truth has come. We will customize the JSON output and add a second resource: the user’s products. These are the elements that the user will sell in the application and will be directly linked.

If you are familiar with Rails, you may already know what I am talking about. But for those who don’t know, we will associate the User model with the Product model using the has_many and belongs_to methods of Active Record.

In this chapter, we will:

  • build the Product model from scratch

  • associate it with the user

  • create the necessary entries so any customer can access the information.

You can clone the project up to this point:

$ git checkout tags/checkpoint_chapter05

Before we start and as usual, when starting new features, we need to create a brand new branch:

$ git checkout -b chapter05

The product model

We will first create a Product model. Then we’ll add some validations and finally associate it with the User model. Like the User model, the Product will be fully tested and automatically deleted if the user is deleted.

The foundations of the product

The Product template will need several fields:

  • a price attribute for the product price

  • a published Boolean to know if the product is ready to be sold or not

  • a title to define a sexy product title

  • a user_id to associate this particular product to a user

    As you may guess we generate it with the command `rails generate`:
$ rails generate model Product title:string price:decimal published:boolean user:belongs_to
Running via Spring preloader in process 1476
      invoke  active_record
      create    db/migrate/20190608205942_create_products.rb
      create    app/models/product.rb
      invoke    test_unit
      create      test/models/product_test.rb
      create      test/fixtures/products.yml
Note
We used the belongs_to type for the attribute user. This is a shortcut that will create an user_id column of type int and add a foreign key to the users.id field. user_id will also be defined as an index. This is a good practice for association keys because it optimizes database queries. It is not mandatory, but I highly recommend it.

Migration file should look like bellow:

db/migrate/20190608205942_create_products.rb
class CreateProducts < ActiveRecord::Migration[6.0]
  def change
    create_table :products do |t|
      t.string :title
      t.decimal :price
      t.boolean :published
      t.belongs_to :user, null: false, foreign_key: true

      t.timestamps
    end
  end
end

We now just have to initiate migration:

$ rake db:migrate

A test must fail at this point:

$ rake test
....E

Error:
Api::V1::UsersControllerTest#test_should_destroy_user:
ActiveRecord::InvalidForeignKey: SQLite3::ConstraintException: FOREIGN KEY constraint failed

rails test test/controllers/api/v1/users_controller_test.rb:43

You certainly said:

What?! But I didn’t touch the users!.

What I have seen in the code of other developers, when they work with associations, is that they forget about destroying dependencies between models. What I mean by that is if a user is deleted, so should the user’s products.

We need a user with one of the products to test this interaction between the models. Then we will delete this user in the hope that the products will disappear with him. Rails have already generated this for us. Take a look at the fixture of the products:

test/fixtures/products.yml
one:
  title: MyString
  price: 9.99
  published: false
  user: one
# ...

You can see this fixture does not use the attribute user_id but user. This means that the one product will have an user_id attribute corresponding to the one user ID.

Therefore, it is necessary to specify a cascading deletion to delete the one product when the one user is deleted. Let’s start with the unit test:

test/models/user_test.rb
# ...
class UserTest < ActiveSupport::TestCase
  # ...
  test 'destroy user should destroy linked product' do
    assert_difference('Product.count', -1) do
      users(:one).destroy
    end
  end
end

You just have to modify the User model and specify the has_many relationship with the depend: :destroy option. We will see later what this method does in more detail.

app/models/user.rb
# ...
class User < ApplicationRecord
  # ...
  has_many :products, dependent: :destroy
end

And that’s it. Now make a commit:

$ git add . && git commit -m "Generate product model"

Product validations

Validations are an important part when building any kind of application. This will prevent any junk data from being saved onto the database. In the product, we have to make sure that the price is a number and that it is not negative.

Also, an important thing about validation is to validate that every product has a user. In this case, we need to validate the presence of the user_id. You can see what I’m talking about in the next code snippet.

test/models/product_test.rb
# ...
class ProductTest < ActiveSupport::TestCase
  test "should have a positive price" do
    product = products(:one)
    product.price = -1
    assert_not product.valid?
  end
end

Now we need to add the implementation to make the tests pass:

app/models/product.rb
class Product < ApplicationRecord
  validates :title, :user_id, presence: true
  validates :price, numericality: { greater_than_or_equal_to: 0 }, presence: true
  belongs_to :user
end

Tests are now green:

$ rake test
................

We have a bunch of good quality code. Let’s commit it and keep moving:

$ git commit -am "Adds some validations to products"

Products endpoints

It is now time to start building the products endpoints. For now, we will just build five REST actions. In the next chapter, we will customize the JSON output by implementing the fast_jsonapi.

First we need to create the products_controller, and we can easily achieve this with the command below:

$ rails generate controller api::v1::products
      create  app/controllers/api/v1/products_controller.rb
      invoke  test_unit
      create    test/controllers/api/v1/products_controller_test.rb

The above command will generate a lot of files that will allow us to start working quickly. What I mean by that is that it will generate the controller and test files already scoped to version 1 of the API.

As a warmup, we will start nice and easy by building the show action for the product.

Show action for products

As usual, we begin by adding some product show controller specs. The strategy here is straightforward: we just need to create a single product and make sure the server’s response is what we expect.

test/controllers/api/v1/products_controller_test.rb
# ...
class Api::V1::ProductsControllerTest < ActionDispatch::IntegrationTest
  setup do
    @product = products(:one)
  end

  test "should show product" do
    get api_v1_product_url(@product), as: :json
    assert_response :success

    json_response = JSON.parse(self.response.body)
    assert_equal @product.title, json_response['title']
  end
end

Then we add the code to make the test pass:

app/controllers/api/v1/products_controller.rb
class Api::V1::ProductsController < ApplicationController
  def show
    render json: Product.find(params[:id])
  end
end

Wait! Don’t run the tests yet. Remember we need to add the resource to the routes.rb file:

config/routes.rb
Rails.application.routes.draw do
  namespace :api, defaults: { format: :json } do
    namespace :v1 do
      resources :users, only: %i[show create update destroy]
      resources :tokens, only: [:create]
      resources :products, only: [:show]
    end
  end
end

Now we make sure the tests are nice and green:

$ rake test
.................

As you may notice already, the specs and implementation are straightforward. They behave the same as users.

Products list

Now it is time to output a list of products (which could be displayed as the marketplace product catalog). This endpoint should be accessible without credentials. That means we don’t require the user to be logged in to access the data. As usual, we will start writing some tests:

test/controllers/api/v1/products_controller_test.rb
# ...
class Api::V1::ProductsControllerTest < ActionDispatch::IntegrationTest
  setup do
    @product = products(:one)
  end

  test "should show products" do
    get api_v1_products_url(), as: :json
    assert_response :success
  end

  test "should show product" do
    get api_v1_product_url(@product), as: :json
    assert_response :success

    json_response = JSON.parse(self.response.body)
    assert_equal @product.title, json_response['title']
  end
end

Let’s move into the implementation, which for now is going to be a simple index method:

app/controllers/api/v1/products_controller.rb
class Api::V1::ProductsController < ApplicationController
  def index
    render json: Product.all
  end
  #...
end

Don’t forget to add the corresponding route:

config/routes.rb
Rails.application.routes.draw do
  namespace :api, defaults: { format: :json } do
    namespace :v1 do
      # ....
      resources :products, only: %i[show index]
    end
  end
end

We are done for now with the public product endpoints. In the next sections, we will focus on building the actions requiring a user to be logged in to access them. Said that we are committing these changes and continue.

$ git add . && git commit -m "Finishes modeling the product model along with user associations"

Creating products

Creating products is a little more complex because we will need an additional configuration. The strategy we will follow is to assign the created product to the user who owns the JWT token provided in the HTTP header Authorization.

So let’s start by the products_controller_test.rb file:

test/controllers/api/v1/products_controller_test.rb
# ...
class Api::V1::ProductsControllerTest < ActionDispatch::IntegrationTest
  # ...

  test 'should create product' do
    assert_difference('Product.count') do
      post api_v1_products_url,
           params: { product: { title: @product.title, price: @product.price, published: @product.published } },
           headers: { Authorization: JsonWebToken.encode(user_id: @product.user_id) },
           as: :json
    end
    assert_response :created
  end

  test 'should forbid create product' do
    assert_no_difference('Product.count') do
      post api_v1_products_url,
           params: { product: { title: @product.title, price: @product.price, published: @product.published } },
           as: :json
    end
    assert_response :forbidden
  end
end

Wow! We added a lot of code. If you remember the previous section, tests are pretty similar to those for about user creation. Except for some minor changes.

In this way, we can see the user and create a product associated with them. But wait! There’s something better.

If we adopt this approach, we can increase the scope of our authorization mechanism. We built the logic to get a logged user from the header Authorization and assigned him a method current_user. It is therefore quite easy to set up by simply adding the authorization header to the request and retrieving the user from it. So let’s do it:

app/controllers/api/v1/products_controller.rb
class Api::V1::ProductsController < ApplicationController
  before_action :check_login, only: %i[create]
  # ...

  def create
    product = current_user.products.build(product_params)
    if product.save
      render json: product, status: :created
    else
      render json: { errors: product.errors }, status: :unprocessable_entity
    end
  end

  private

  def product_params
    params.require(:product).permit(:title, :price, :published)
  end
end

As you can see, we protect the create action with the check_login method. We also build the product by associating the current user. I added this very simple method to the concern authenticable.rb:

app/controllers/concerns/authenticable.rb
module Authenticable
  # ...
  protected

  def check_login
    head :forbidden unless self.current_user
  end
end

One last thing before you do your tests: the necessary route:

config/routes.rb
Rails.application.routes.draw do
  namespace :api, defaults: { format: :json } do
    namespace :v1 do
      # ...
      resources :products, only: %i[show index create]
    end
  end
end

Now tests should all pass:

$ rake test
....................

Updating products

Hopefully, by now, you understand the logic to build the upcoming actions. This section will focus on the update action, which will work similarly to the create one. We just need to fetch the product from the database and update it.

We first adding the action to the routes so we don’t forget later:

config/routes.rb
Rails.application.routes.draw do
  namespace :api, defaults: { format: :json } do
    namespace :v1 do
      # ...
      resources :products, only: %i[show index create update]
    end
  end
end

Before we start dropping some tests, I just want to clarify that similarly to the create action we will scope the product to the current_user. In this case, we want to ensure the product we are updating is owned by the current user. So we will fetch that product from the user.products association provided by Rails.

Let’s add some specs:

test/controllers/api/v1/products_controller_test.rb
require 'test_helper'

class Api::V1::ProductsControllerTest < ActionDispatch::IntegrationTest
  # ...

  test 'should update product' do
    patch api_v1_product_url(@product),
          params: { product: { title: @product.title } },
          headers: { Authorization: JsonWebToken.encode(user_id: @product.user_id) },
          as: :json
    assert_response :success
  end

  test 'should forbid update product' do
    patch api_v1_product_url(@product),
          params: { product: { title: @product.title } },
          headers: { Authorization: JsonWebToken.encode(user_id: users(:two).id) },
          as: :json
    assert_response :forbidden
  end
end
Note
I have added a fixture corresponding to a second user to verify that the second user cannot modify the first user’s product.

Tests may look complex but take a second peek. They are almost the same we built for users.

Now let’s implement the code to make our tests pass:

app/controllers/api/v1/products_controller.rb
class Api::V1::ProductsController < ApplicationController
  before_action :set_product, only: %i[show update]
  before_action :check_login, only: %i[create]
  before_action :check_owner, only: %i[update]

  # ...

  def create
    product = current_user.products.build(product_params)
    if product.save
      render json: product, status: :created
    else
      render json: { errors: product.errors }, status: :unprocessable_entity
    end
  end

  def update
    if @product.update(product_params)
      render json: @product
    else
      render json: @product.errors, status: :unprocessable_entity
    end
  end

  private
  # ...

  def check_owner
    head :forbidden unless @product.user_id == current_user&.id
  end

  def set_product
    @product = Product.find(params[:id])
  end
end

Implementation is quite simple. We will simply retrieve the product from the connected user and simply update it. We have also added this action to the before_action to prevent any unauthorized user from updating a product.

Now tests should pass:

$ rake test
......................

Destroying products

Our last stop for the product endpoints will be the destroy action. You might now imagine how this would look like. The strategy in here will be pretty similar to the create and update actions: we’ll get the logged user with JWT token and then fetch the product from the user.products association and finally destroy it, returning a 204 code.

Let’s start again by adding the route name to the routes file:

config/routes.rb
Rails.application.routes.draw do
  namespace :api, defaults: { format: :json } do
    namespace :v1 do
      resources :users, only: %i[show create update destroy]
      resources :tokens, only: [:create]
      resources :products
    end
  end
end

After this, we have to add some tests as shown on this code snippet:

test/controllers/api/v1/products_controller_test.rb
# ...
class Api::V1::ProductsControllerTest < ActionDispatch::IntegrationTest
  # ...

  test "should destroy product" do
    assert_difference('Product.count', -1) do
      delete api_v1_product_url(@product), headers: { Authorization: JsonWebToken.encode(user_id: @product.user_id) }, as: :json
    end
    assert_response :no_content
  end

  test "should forbid destroy product" do
    assert_no_difference('Product.count') do
      delete api_v1_user_url(@product), headers: { Authorization: JsonWebToken.encode(user_id: users(:two).id) }, as: :json
    end
    assert_response :forbidden
  end
end

Now we simply add the necessary code to make tests pass:

app/controllers/api/v1/products_controller.rb
class Api::V1::ProductsController < ApplicationController
  before_action :set_product, only: %i[show update destroy]
  before_action :check_login, only: %i[create]
  before_action :check_owner, only: %i[update destroy]

  # ...

  def destroy
    @product.destroy
    head 204
  end

  # ...
end

As you can see the four lines implementation does the job. We can run tests to make sure everything is good and then we will commit the changes as we added a bunch of new code. Also, make sure you hook this action to the before_action callback as with the update action.

$ rake test
........................

Let’s commit the changes:

$ git commit -am "Adds the products create, update and destroy actions"

Feed the database

let’s fill the database with fake data before continuing with more code. We will use seeds to do so.

With the file db/seeds.rb, Rails gives us a way to easily and quickly provide default values for a new installation. It is a simple Ruby file that gives full access to all classes and methods of the application. So you don’t need to enter everything manually with the Rails console but you can simply use the file db/seeds.rb with the command rake db:seed.

So let’s start by creating a user:

db/seeds.rb
User.delete_all
user = User.create! email: 'toto@toto.fr', password: 'toto123'
puts "Created a new user: #{user.email}"

And now you can create the user by simply executing the following command:

$ rake db:seed
Created a new user: toto@toto.fr

It works. I don’t know about you, but I like to have dummy data that correctly fills my test database. Only I don’t always have the inspiration to give meaning to my seed, so I use the gem faker. Let’s set it up there:

$ bundle add faker

Now we can use it to create five users at once with different emails.

db/seeds.rb
User.delete_all

5.times do
  user = User.create! email: Faker::Internet.email, password: 'locadex1234'
  puts "Created a new user: #{user.email}"
end

And let’s see what happens:

$ rake db:seed
Created a new user: barbar@greenholt.io
Created a new user: westonpaucek@ortizbotsford.net
Created a new user: ricardo@schneider.com
Created a new user: scott@moenerdman.biz
Created a new user: chelsie@wiza.net

There you go. But we can go further by creating products associated with these users:

db/seeds.rb
Product.delete_all
User.delete_all

3.times do
  user = User.create! email: Faker::Internet.email, password: 'locadex1234'
  puts "Created a new user: #{user.email}"

  2.times do
    product = Product.create!(
      title: Faker::Commerce.product_name,
      price: rand(1.0..100.0),
      published: true,
      user_id: user.id
    )
    puts "Created a brand new product: #{product.title}"
  end
end

There you go. The result is amazing. In one order, we can create three users and six products:

$ rake db:seed
Created a new user: tova@beatty.org
Created a brand new product: Lightweight Steel Hat
Created a brand new product: Ergonomic Aluminum Lamp
Created a new user: tommyrunolfon@tremblay.biz
Created a brand new product: Durable Plastic Car
Created a brand new product: Ergonomic Leather Shirt
Created a new user: jordon@torp.io
Created a brand new product: Incredible Paper Hat
Created a brand new product: Sleek Concrete Pants

Let’s commit changes:

$ git commit -am "Create a seed to populate database"

And as we get to the end of our chapter, it’s time to apply all our modifications to the master branch by making a merge:

$ git checkout master
$ git merge chapter05

Conclusion

I hope you have enjoyed this chapter. It’s a long one but the code we put together is an excellent base for the core app.

In the next chapter, we will focus on customizing user and product models' output using the gem fast_jsonapi. It will allow us to easily filter the attributes to display and manage associations such as embedded objects.

Building JSON

In the previous chapter, we added products to the application and built all the necessary routes. We have also associated a product with a user and restricted some of the actions of products_controller.

Now you should be satisfied with all this work. But we still have a lot of work to do. Currently, we have a JSON output that is not perfect. JSON output looks like this:

{
  "products": [
      {
          "id": 1,
          "title": "Tag Case",
          "price": "98.7761933800815",
          "published": false,
          "user_id": 1,
          "created_at": "2018-12-20T12:47:26.686Z",
          "updated_at": "2018-12-20T12:47:26.686Z"
      },
    ]
}

However, we want an output that does not contain the fields user_id, created_at, and updated_at.

An important (and difficult) part when creating your API is to decide the output format. Fortunately, some organizations have already faced this kind of problem and have established some conventions you will discover in this chapter.

You can clone the project up to this point with:

$ git checkout tags/checkpoint_chapter06

Let’s start a new branch for this chapter:

$ git checkout -b chapter06

Presentation of JSON:API

An important and difficult part of creating your API is deciding the output format. Fortunately, some conventions already exist. Certainly, the most used of them is JSON:API.

The JSON:API documentation gives us some rules to follow regarding the JSON document formatting.

Thus, our document must contain these keys:

  • data: which must contain the data we send back

  • errors which must contain an array of errors that have occurred

  • meta which contains a meta object

The content of the data key is also quite strict:

  • it must have a type key corresponding to kind of the JSON model (an article, a user, etc…​)

  • properties of the objects must be placed in an attributes key

  • links of the objects must be placed in a relationships key

In this chapter we will customize the JSON output using jsonapi-serializer gem (fork of Netflix’s fast_jsonapi gem) . Luckily for us, it already implements all JSON:API specifications.

So let’s install the gem jsonapi-serializer:

$ bundle add jsonapi-serializer

You should be ready to continue with this tutorial.

Serialize user

JSON:API Serializer uses serializers. Serializers represent Ruby classes responsible for converting a model into an Hash or a JSON.

So we need to add a user_serializer.rb file. We can do it manually, but the gem provides a command-line interface to do it:

$ rails generate serializer User email
      create  app/serializers/user_serializer.rb

This has created a file called user_serializer.rb under the app/serializers directory. The new file should look like the following file:

app/serializers/user_serializer.rb
class UserSerializer
  include JSONAPI::Serializer
  attributes :email
end

This serializer will allow us to convert our User object to JSON, which implements all JSON:API specifications. Because we specified email as attributes we retrieve it in data array.

Let’s try all this in the Rails with rails console console:

2.6.3 :001 > UserSerializer.new( User.first ).serializable_hash
=> {:data=>{:id=>"25", :type=>:user, :attributes=>{:email=>"tova@beatty.org"}}}

There you go. As you can see, this is easy. Now we can use our new serializer in our controller:

app/controllers/api/v1/users_controller.rb
class Api::V1::UsersController < ApplicationController
  # ...
  def show
    render json: UserSerializer.new(@user).serializable_hash.to_json
  end

  def update
    if @user.update(user_params)
      render json: UserSerializer.new(@user).serializable_hash.to_json
    else
      # ...
    end
  end

  def create
    # ...
    if @user.save
      render json: UserSerializer.new(@user).serializable_hash.to_json, status: :created
    else
      # ...
    end
  end

  # ...
end

Quite easy, isn’t it? However, we should have a test that fails. Try it for yourself:

$ rake test

Failure:
Expected: "one@one.org"
  Actual: nil

For some reason, the answer is not quite what we expect. This is because the gem modifies the response we had previously defined. So to pass these tests, you just have to modify it:

test/controllers/api/v1/users_controller_test.rb
# ...
class Api::V1::UsersControllerTest < ActionDispatch::IntegrationTest
  # ...
  test "should show user" do
    # ...
    assert_equal @user.email, json_response['data']['attributes']['email']
  end
  # ...
end

If you do so test now should pass:

$ rake test
........................

Let’s commit to these changes and keep moving forward:

$ git add . && git commit -am "Adds user serializer for customizing the json output"

Serialize products

Now that we understand how the serialization gem works, it’s time to customize the product output. The first step is the same as what we did in the previous section. We need a product serializer. So let’s do it:

$ rails generate serializer Product title price published
      create  app/serializers/product_serializer.rb

Now let’s add attributes to serialize the product:

app/serializers/product_serializer.rb
class ProductSerializer
  include JSONAPI::Serializer
  attributes :title, :price, :published
end

There you go. It’s no more complicated than that. Let’s change our controller a little bit.

app/controllers/api/v1/products_controller.rb
class Api::V1::ProductsController < ApplicationController
  # ...
  def index
    @products = Product.all
    render json: ProductSerializer.new(@products).serializable_hash.to_json
  end

  def show
    render json: ProductSerializer.new(@product).serializable_hash.to_json
  end

  def create
    product = current_user.products.build(product_params)
    if product.save
      render json: ProductSerializer.new(product).serializable_hash.to_json, status: :created
    else
      # ...
    end
  end

  def update
    if @product.update(product_params)
      render json: ProductSerializer.new(@product).serializable_hash.to_json
    else
      # ...
    end
  end
  # ...
end

And we’re updating our functional test:

test/controllers/api/v1/products_controller_test.rb
# ...
class Api::V1::ProductsControllerTest < ActionDispatch::IntegrationTest
  # ...
  test 'should show product' do
    # ...
    assert_equal @product.title, json_response['data']['attributes']['title']
  end
  # ...
end

You can check that tests pass but they should. Let’s commit these small changes:

$ git add .
$ git commit -m "Adds product serializer for custom json output"

Serialize associations

We have worked with serializers, and you may notice that it is straightforward. In some cases difficult decision is naming your routes or structuring the JSON output. When working with associations between models on an API there are many approaches you can take.

We don’t have to worry about this problem in our case: JSON:API specifications did it for us!

To summarize, we have a has_many type association between users and products.

app/models/user.rb
class User < ApplicationRecord
  has_many :products, dependent: :destroy
  # ...
end
app/models/product.rb
class Product < ApplicationRecord
  belongs_to :user
  # ...
end

It is a good idea to integrate users into the JSON outputs of products. This will make the output more cumbersome, but it will prevent the API client from executing other requests to retrieve user information related to the products. This method can save you a huge bottleneck.

Theory of the injection of relationships

Imagine a scenario where you go to the API to get the products, but you have to display some of the user information in this case.

One possible solution would be adding the attribute user_id to the product_serializer to get the corresponding user later. This may sound like a good idea, but if you are concerned about performance or your database transactions are not fast enough, you should reconsider this approach. You must understand that for each product you retrieve, you will have to retrieve its corresponding user.

Faced with this problem, there are several alternatives.

Integrate into a meta attribute

The first solution (a good one, in my opinion) is to integrate identifiers of linked users to products in a meta attribute. So we obtain a JSON like below:

{
  "meta": { "user_ids": [1,2,3] },
  "data": [

  ]
}

So that the client can retrieve these users from these user_ids.

Incorporate the object into the attribute

Another solution is to incorporate the user object into the product object. This may make the first request a little slower, but in this way, the client does not need to make another additional request. An example of the expected results is presented below:

{
  "data":
  [
    {
        "id": 1,
        "type": "product",
        "attributes": {
          "title": "First product",
          "price": "25.02",
          "published": false,
          "user": {
            "id": 2,
            "attributes": {
              "email": "stephany@lind.co.uk",
              "created_at": "2014-07-29T03:52:07.432Z",
              "updated_at": "2014-07-29T03:52:07.432Z",
              "auth_token": "Xbnzbf3YkquUrF_1bNkZ"
            }
          }
        }
    }
  ]
}

The problem with this approach is we have to duplicate the `User' objects for each product that belongs to the same user:

{
  "data":
  [
    {
        "id": 1,
        "type": "product",
        "attributes": {
          "title": "First product",
          "price": "25.02",
          "published": false,
          "user": {
            "id": 2,
            "type": "user",
            "attributes": {
              "email": "stephany@lind.co.uk",
              "created_at": "2014-07-29T03:52:07.432Z",
              "updated_at": "2014-07-29T03:52:07.432Z",
              "auth_token": "Xbnzbf3YkquUrF_1bNkZ"
            }
          }
        }
    },
    {
        "id": 2,
        "type": "product",
        "attributes": {
          "title": "Second product",
          "price": "25.02",
          "published": false,
          "user": {
            "id": 2,
            "type": "user",
            "attributes": {
              "email": "stephany@lind.co.uk",
              "created_at": "2014-07-29T03:52:07.432Z",
              "updated_at": "2014-07-29T03:52:07.432Z",
              "auth_token": "Xbnzbf3YkquUrF_1bNkZ"
            }
          }
        }
    }
  ]
}

Incorporate the relationships into `include

The third solution (chosen by the JSON:API) is a mixture of the first two.

We will include all the relationships in an include key that will contain all the previously mentioned objects' relationships. Each object will also include a relationship key that defines the relationship, and that must be found in the included key.

A JSON is worth a thousand words:

{
  "data":
  [
    {
        "id": 1,
        "type": "product",
        "attributes": {
          "title": "First product",
          "price": "25.02",
          "published": false
        },
        "relationships": {
          "user": {
            "id": 1,
            "type": "user"
          }
        }
    },
    {
        "id": 2,
        "type": "product",
        "attributes": {
          "title": "Second product",
          "price": "25.02",
          "published": false
        },
        "relationships": {
          "user": {
            "id": 1,
            "type": "user"
          }
        }
    }
  ],
  "include": [
    {
      "id": 2,
      "type": "user",
      "attributes": {
        "email": "stephany@lind.co.uk",
        "created_at": "2014-07-29T03:52:07.432Z",
        "updated_at": "2014-07-29T03:52:07.432Z",
        "auth_token": "Xbnzbf3YkquUrF_1bNkZ"
      }
    }
  ]
}

Do you see the difference? This solution drastically reduces the size of the JSON and therefore, the bandwidth used.

Application of the injection of relationships

So we will incorporate the user object into the product. Let’s start by adding some tests.

We will simply modify the Products#show test to verify that we are recovering:

test/controllers/api/v1/products_controller_test.rb
# ...
class Api::V1::ProductsControllerTest < ActionDispatch::IntegrationTest
  # ...
  test 'should show product' do
    get api_v1_product_url(@product), as: :json
    assert_response :success

    json_response = JSON.parse(response.body, symbolize_names: true)
    assert_equal @product.title, json_response.dig(:data, :attributes, :title)
    assert_equal @product.user.id.to_s, json_response.dig(:data, :relationships, :user, :data, :id)
    assert_equal @product.user.email, json_response.dig(:included, 0, :attributes, :email)
  end
  # ...
end

We are now checking three things on the JSON that has been returned:

  1. it contains the title of the product

  2. it contains the user ID of the user linked to the product

  3. the user data is included in the include key

Note
You may have noticed that I have chosen to use the method Hash#dig. It is a Ruby method allowing you to retrieve elements in a nested Hash by avoiding errors if an element is not present.

To pass this test we will start by including the relationship in the serializer:

app/serializers/product_serializer.rb
class ProductSerializer
  include JSONAPI::Serializer
  attributes :title, :price, :published
  belongs_to :user
end

This addition will add a relationship key containing the user’s identifier:

{
  "data": {
      "id": "1",
      "type": "product",
      "attributes": {
          "title": "Durable Marble Lamp",
          "price": "11.55",
          "published": true
      },
      "relationships": {
          "user": {
              "data": {
                  "id": "1",
                  "type": "user"
              }
          }
      }
  }
}

This allows us to correct our first two assertions. We now want to include attributes of the user who owns the product. To do this, we simply need to pass an option :include to the serializer instantiated in the controller. Then let’s do it:

app/controllers/api/v1/products_controller.rb
class Api::V1::ProductsController < ApplicationController
  # ...
  def show
    options = { include: [:user] }
    render json: ProductSerializer.new(@product, options).serializable_hash.to_json
  end
  # ...
end

There you go. Now, this is what the JSON should look like:

{
  "data": {
    ...
  },
  "included": [
    {
      "id": "1",
      "type": "user",
      "attributes": {
          "email": "staceeschultz@hahn.info"
      }
    }
  ]
}

Now all tests should pass:

$ rake test
........................

Let’s make a commit to celebrate:

$ git commit -am "Add user relationship to product serializer"

Retrieve user’s products

Do you understand the principle? We have included user information in the JSON of the products. We can do the same by including product information related to a user for the /api/v1/users/1 page.

Let’s start with the test:

test/controllers/api/v1/users_controller_test.rb
# ...
class Api::V1::UsersControllerTest < ActionDispatch::IntegrationTest
  # ...
  test "should show user" do
    get api_v1_user_url(@user), as: :json
    assert_response :success

    json_response = JSON.parse(self.response.body, symbolize_names: true)
    assert_equal @user.email, json_response.dig(:data, :attributes, :email)
    assert_equal @user.products.first.id.to_s, json_response.dig(:data, :relationships, :products, :data, 0, :id)
    assert_equal @user.products.first.title, json_response.dig(:included, 0, :attributes, :title)
  end
  # ...
end

serializer:

app/serializers/user_serializer.rb
class UserSerializer
  include JSONAPI::Serializer
  attributes :email
  has_many :products
end

And to finish controller:

app/controllers/api/v1/users_controller.rb
class Api::V1::UsersController < ApplicationController
  # ...
  def show
    options = { include: [:products] }
    render json: UserSerializer.new(@user, options).serializable_hash.to_json
  end
  # ...
end

There you go. We obtain a JSON like the following:

{
  "data": {
    "id": "1",
    "type": "user",
    "attributes": {
      "email": "staceeschultz@hahn.info"
    },
    "relationships": {
      "products": {
        "data": [
          { "id": "1", "type": "product" },
          { "id": "2", "type": "product" }
        ]
      }
    }
  },
  "included": [
    {
      "id": "1",
      "type": "product",
      "attributes": {
        "title": "Durable Marble Lamp",
        "price": "11.5537474980286",
        "published": true
      },
      "relationships": {
        "user": {
          "data": {
            "id": "1",
            "type": "user"
          }
        }
      }
    },
    {
        ...
    }
  ]
}

It was straightforward. Let’s make a commit:

$ git commit -am "Add products relationship to user#show"

Search for products

This last section will continue to strengthen the Products#index action by setting up a straightforward search mechanism allowing any customer to filter the results. This section is optional as it will have no impact on the application modules. But if you want to practice more with the TDD I recommend that you complete this last step.

I use Ransack or pg_search to build advanced search forms extremely quickly. But since the goal is learning and searching, we are going to do very simple. I think we can build a search engine from scratch. We simply have to consider the criteria by which we will filter the attributes. Hang on to your seats it’s going to be a tough trip.

We will, therefore, filter the products according to the following criteria:

  • By title

  • By price

  • Sort by creation date

It may seem short and easy, but believe me, it will give you a headache if you don’t plan it.

The keyword by

We will create a scope to find records that match a particular character pattern. Let’s call it filter_by_title.

We will start by adding some fixtures with different products to test:

test/fixtures/products.yml
one:
  title: TV Plosmo Philopps
  price: 9999.99
  published: false
  user: one

two:
  title: Azos Zeenbok
  price: 499.99
  published: false
  user: two

another_tv:
  title: Cheap TV
  price: 99.99
  published: false
  user: two

And now we can build some tests:

test/models/product_test.rb
# ...
class ProductTest < ActiveSupport::TestCase
  # ...
  test "should filter products by name" do
    assert_equal 2, Product.filter_by_title('tv').count
  end

  test 'should filter products by name and sort them' do
    assert_equal [products(:another_tv), products(:one)], Product.filter_by_title('tv').sort
  end
end

The following tests ensure that the method Product.filter_by_title will correctly search for products according to their title. We use the term tv in lowercase to ensure that our search will not be case sensitive.

app/models/product.rb
class Product < ApplicationRecord
  # ...
  scope :filter_by_title, lambda { |keyword|
    where('lower(title) LIKE ?', "%#{keyword.downcase}%")
  }
end
Note
scoping allows you to specify commonly-used queries that can be referenced as method calls on models. With these scopes you can also link with Active Record methods like where, joins, and includes because a scope always returns an object ActiveRecord::Relation. I invite you to take a look at Rails documentation

Implementation is sufficient for our tests to pass:

$ rake test
..........................

By price

To filter by price, things can get a little more delicate. We will break the logic of filtering by price in two different methods: one that will look for products larger than the price received and the other that will look for those below that price. This way, we will keep some flexibility, and we can easily test the scope.

Let’s start by building the tests of the scope above_or_equal_to_price:

test/models/product_test.rb
# ...
class ProductTest < ActiveSupport::TestCase
  # ...
  test 'should filter products by price and sort them' do
    assert_equal [products(:two), products(:one)], Product.above_or_equal_to_price(200).sort
  end
end

Implementation is very, very simple:

app/models/product.rb
class Product < ApplicationRecord
  # ...
  scope :above_or_equal_to_price, lambda { |price|
    where('price >= ?', price)
  }
end

This is sufficient to convert our tests to green:

$ rake test
...........................

You can now imagine the behavior of the opposite method. Here are the tests:

test/models/product_test.rb
# ...
class ProductTest < ActiveSupport::TestCase
  # ...
  test 'should filter products by price lower and sort them' do
    assert_equal [products(:another_tv)], Product.below_or_equal_to_price(200).sort
  end
end

And implementation.

app/models/product.rb
class Product < ApplicationRecord
  # ...
  scope :below_or_equal_to_price, lambda { |price|
    where('price <= ?', price)
  }
end

For our sake, let’s do the tests and check that everything is beautiful and green:

$ rake test
............................

As you can see, we haven’t had many problems. Let’s just add another scope to sort the records by date of the last update. If the owner of the products decides to update some data, he will surely want to sort his products by creation date.

Sort by creation date

This scope is very easy. Let’s add some tests first:

test/models/product_test.rb
# ...
class ProductTest < ActiveSupport::TestCase
  # ...
  test 'should sort product by most recent' do
    # we will touch some products to update them
    products(:two).touch
    assert_equal [products(:another_tv), products(:one), products(:two)], Product.recent.to_a
  end
end

And the implementation:

app/models/product.rb
class Product < ApplicationRecord
  # ...
  scope :recent, lambda {
    order(:updated_at)
  }
end

All our tests should pass:

$ rake test
.............................

Let’s commit our changes:

$ git commit -am "Adds search scopes on the product model"

Search engine

Now that we have the basis for the search engine we will use in the application, it is time to implement a simple but powerful search method. It will manage all the logic to retrieve the product records.

The method will link all the scope that we have previously built and return the result. Let’s start by adding some tests:

test/models/product_test.rb
# ...
class ProductTest < ActiveSupport::TestCase
  # ...
  test 'search should not find "videogame" and "100" as min price' do
    search_hash = { keyword: 'videogame', min_price: 100 }
    assert Product.search(search_hash).empty?
  end

  test 'search should find cheap TV' do
    search_hash = { keyword: 'tv', min_price: 50, max_price: 150 }
    assert_equal [products(:another_tv)], Product.search(search_hash)
  end

  test 'should get all products when no parameters' do
    assert_equal Product.all.to_a, Product.search({})
  end

  test 'search should filter by product ids' do
    search_hash = { product_ids: [products(:one).id] }
    assert_equal [products(:one)], Product.search(search_hash)
  end
end

We have added a lot of code, but I assure you that the implementation is straightforward. You can go further and add some additional tests but, in my case, I didn’t find it necessary.

app/models/product.rb
class Product < ApplicationRecord
  # ...
  def self.search(params = {})
    products = params[:product_ids].present? ? Product.where(id: params[:product_ids]) : Product.all

    products = products.filter_by_title(params[:keyword]) if params[:keyword]
    products = products.above_or_equal_to_price(params[:min_price].to_f) if params[:min_price]
    products = products.below_or_equal_to_price(params[:max_price].to_f) if params[:max_price]
    products = products.recent if params[:recent]

    products
  end
end

It is important to note that we return the products as an object ActiveRecord::Relation so that we can chain other methods if necessary or page them as we will see in the last chapters. Simply update the Product#index action to retrieve the products from the search method:

app/controllers/api/v1/products_controller.rb
class Api::V1::ProductsController < ApplicationController
  # ...
  def index
    @products = Product.search(params)
    render json: ProductSerializer.new(@products).serializable_hash.to_json
  end
  # ...
end

We can run the entire test suite to ensure that the application is in good health so far:

$ rake test
.................................
33 runs, 49 assertions, 0 failures, 0 errors, 0 skips

Let’s commit all these changes:

$ git commit -am "Adds search class method to filter products"

And as we get to the end of our chapter, it is time to apply all our modifications to the master branch by making a merge:

$ git checkout master
$ git merge chapter06

Conclusion

Until now, it was easy thanks to the gem jsonapi-serializer. In the coming chapters, we will start building the Order model to involve users in the products.

Placing Orders

In the previous chapter, we handle associations between products and users and serialize them to scale fast and easy. Now it is time to start placing orders, which is going to be a more complex situation. We will handle associations between these three models. We have to be smart enough to handle the JSON output we are delivering.

In this chapter, we will make several things, which I list below:

  • Create an Order model with its corresponding specs

  • Handle JSON output association between the order user and product models

  • Send a confirmation email with the order summary

So now everything is clear, we can get our hands dirty. You can clone the project up to this point with:

$ git checkout tags/checkpoint_chapter07

Let’s create a branch to start working:

$ git checkout -b chapter07

Modeling order

If you remember the associations model, Order model is associated with users and products simultaneously. Actually, It is straightforward to achieve this in Rails. The tricky part is when it comes to serializing this object. I’ll talk more about this in the next section.

Let’s start by creating the older model with a special form:

$ rails generate model order user:belongs_to total:decimal

The command above will generate the order model, but I’m taking advantage of the references method to create the corresponding foreign key for the order to belong to a user. It also adds the belongs_to directive into the order model. Let’s migrate the database.

$ rake db:migrate

Now it is time to write some tests into the order_test.rb file:

test/models/order_test.rb
# ...
class OrderTest < ActiveSupport::TestCase
  test 'Should have a positive total' do
    order = orders(:one)
    order.total = -1
    assert_not order.valid?
  end
end

The implementation is fairly simple:

app/models/order.rb
class Order < ApplicationRecord
  belongs_to :user
  validates :total, numericality: { greater_than_or_equal_to: 0 }
  validates :total, presence: true
end

Don’t forget to add the orders relationship to our users by specifying cascading deletion:

app/models/user.rb
class User < ApplicationRecord
  # ...
  has_many :products, dependent: :destroy
  has_many :orders, dependent: :destroy
  # ...
end

Tests should pass:

$ rake test
..................................

And commit all this:

$ git add . && git commit -m "Generate orders"

Orders and Products

We need to set up the association between the order and the product, built with a has-many-to-many association. As many products will be placed on many orders, and the orders will have multiple products. So, in this case, we need a middle model that will join these two other objects and map the appropriate association.

Let’s generate this model:

$ rails generate model placement order:belongs_to product:belongs_to

Let’s run the migration on the database:

$ rake db:migrate

Implementation is like so:

app/models/product.rb
class Product < ApplicationRecord
  belongs_to :user
  has_many :placements, dependent: :destroy
  has_many :orders, through: :placements
  # ...
end
app/models/order.rb
class Order < ApplicationRecord
  has_many :placements, dependent: :destroy
  has_many :products, through: :placements
  # ...
end

If you have been following the tutorial so far, the implementation is already there because of the references type we pass on the model command generator. We should add inverse_of option to the placement model for each belongs_to call. This gives a little boost when referencing the parent object.

app/models/placement.rb
class Placement < ApplicationRecord
  belongs_to :order
  belongs_to :product, inverse_of: :placements
end

Let’s run the models spec and make sure everything is green:

$ rake test
..................................

Now that everything is nice and green, let’s commit the changes and continue.

$ git add . && git commit -m "Associates products and orders with a placements model"

Expose the user model

It is now time to prepare the order controller to expose the right orders. If you remember the previous chapters where fast_jsonapi was used, you should remember that it was straightforward.

Let us first define what actions we will take:

  1. An indexing action to retrieve current user orders

  2. A show action to retrieve a particular order from the current user

  3. A creation action to actually place the order

Let’s start with the action index. First, we have to create the order controller:

$ rails generate controller api::v1::orders

Up to this point and before start typing some code, we have to ask ourselves:

Should I leave my order endpoints nested into the UsersController or should I isolate them?

The answer is straightforward: it depends on the amount of information you want to expose to the developer.

In our case, we will not do this because we will retrieve the user orders from the /orders route. Let’s start with some tests:

test/controllers/api/v1/orders_controller_test.rb
# ...
class Api::V1::OrdersControllerTest < ActionDispatch::IntegrationTest
  setup do
    @order = orders(:one)
  end

  test 'should forbid orders for unlogged' do
    get api_v1_orders_url, as: :json
    assert_response :forbidden
  end

  test 'should show orders' do
    get api_v1_orders_url,
      headers: { Authorization: JsonWebToken.encode(user_id: @order.user_id) },
      as: :json
    assert_response :success

    json_response = JSON.parse(response.body)
    assert_equal @order.user.orders.count, json_response['data'].count
  end
end

If we run the test suite now, both tests should fail as you may expect. This is because they have not even set the correct routes nor actions. So let’s start by adding the routes:

config/routes.rb
Rails.application.routes.draw do
  namespace :api, defaults: { format: :json } do
    namespace :v1 do
      resources :orders, only: [:index]
      # ...
    end
  end
end

Now it is time for the orders serializer implementation:

$ rails generate serializer Order

And let’s add relationships:

app/serializers/order_serializer.rb
class OrderSerializer
  include JSONAPI::Serializer
  belongs_to :user
  has_many :products
end

It is now time to implement the controller:

app/controllers/api/v1/orders_controller.rb
class Api::V1::OrdersController < ApplicationController
  before_action :check_login, only: %i[index]

  def index
    render json: OrderSerializer.new(current_user.orders).serializable_hash.to_json
  end
end

And now all of our tests should pass:

$ rake test
....................................
36 runs, 53 assertions, 0 failures, 0 errors, 0 skips

We like our commits very atomic, so let’s commit these changes:

$ git add . && git commit -m "Adds the index action for order"

Render a single order

As you can already imagine, this route is straightforward. We only have to set up a few configurations (routes, controller action), and this section will be over. We will also include products related to this order in the output JSON.

Let’s start by adding some tests:

test/controllers/api/v1/orders_controller_test.rb
# ...
class Api::V1::OrdersControllerTest < ActionDispatch::IntegrationTest
  # ...
  test 'should show order' do
    get api_v1_order_url(@order),
        headers: { Authorization: JsonWebToken.encode(user_id: @order.user_id) },
        as: :json
    assert_response :success

    json_response = JSON.parse(response.body)
    include_product_attr = json_response['included'][0]['attributes']
    assert_equal @order.products.first.title, include_product_attr['title']
  end
end

As you can see, the second part of the test verifies the product is included in the JSON.

Let’s add the implementation to run our tests. On the routes.rb file, add the show action to the order routes:

config/routes.rb
# ...
Rails.application.routes.draw do
  # ...
  resources :orders, only: %i[index show]
  # ...
end

And implementation should look like this:

app/controllers/api/v1/orders_controller.rb
class Api::V1::OrdersController < ApplicationController
  before_action :check_login, only: %i[index show]
  # ...
  def show
    order = current_user.orders.find(params[:id])

    if order
      options = { include: [:products] }
      render json: OrderSerializer.new(order, options).serializable_hash.to_json
    else
      head 404
    end
  end
end

Our tests should be all green:

$ rake test
.....................................
37 runs, 55 assertions, 0 failures, 0 errors, 0 skips

Let’s commit the changes and move onto the create order action:

$ git commit -am "Adds the show action for order"

Placing an order

It is now time to allow the user to place some orders. This will add complexity to the application but don’t worry, we’ll do it one step at a time.

Before launching this feature, let’s take the time to think about the implications of creating an order in the application. I’m not talking about setting up a transaction service like Stripe or Braintree but things like:

  • management of out-of-stock products

  • decrease in product inventory

  • add some validation for order placement to ensure that there are enough products at the time the order is placed

It seems like there’s still a lot to do but believe me: you’re closer than you think, and it’s not as hard as it looks. For now, let’s keep it simple and assume that we still have enough products to place any number of orders. We’re just concerned about the server’s response at the moment.

If you remember order model we need three things:

  • a total for the order

  • user who places the order

  • products for the order

Based on this information we can start adding some tests:

test/controllers/api/v1/orders_controller_test.rb
# ...
class Api::V1::OrdersControllerTest < ActionDispatch::IntegrationTest
  setup do
    # ...
    @order_params = { order: {
      product_ids: [products(:one).id, products(:two).id],
      total: 50
    } }
  end

  # ...

  test 'should forbid create order for unlogged' do
    assert_no_difference('Order.count') do
      post api_v1_orders_url, params: @order_params, as: :json
    end
    assert_response :forbidden
  end

  test 'should create order with two products' do
    assert_difference('Order.count', 1) do
      post api_v1_orders_url,
        params: @order_params,
        headers: { Authorization: JsonWebToken.encode(user_id: @order.user_id) },
        as: :json
    end
    assert_response :created
  end
end

As you can see, we are creating a order_params variable with the order data. Can you see the problem here? If not I’ll explain it later. Let’s just add the necessary code to make this test pass.

First, we need to add the action to the resources on the routes file:

config/routes.rb
# ...
Rails.application.routes.draw do
  # ...
  resources :orders, only: %i[index show create]
  # ...
end

Then the implementation which is easy:

app/controllers/api/v1/orders_controller.rb
class Api::V1::OrdersController < ApplicationController
  before_action :check_login, only: %i[index show create]
  # ...

  def create
    order = current_user.orders.build(order_params)

    if order.save
      render json: order, status: 201
    else
      render json: { errors: order.errors }, status: 422
    end
  end

  private

  def order_params
    params.require(:order).permit(:total, product_ids: [])
  end
end

And now our tests should all be green:

$ rake test
.......................................
39 runs, 59 assertions, 0 failures, 0 errors, 0 skips

Ok, so we have everything nice and green. Now we should move on to the next chapter, right? Let me stop you right there. We have some serious errors on the app, and they are not related to the code itself but on the business part.

Not because the tests are green, it means the app is filling the business part of the app. I wanted to bring this up because in many cases, that’s super easy just receiving params and building objects from those params thinking that we are always receiving the correct data. In this particular case, we cannot rely on that, and the easiest way to see this is that we are letting the client set the order total, yeah crazy!

We have to add some validations or a callback to calculate the order total and set it through the model. This way we don’t longer receive that total attribute and have complete control over this attribute. So let’s do that.

We first need to add some specs for the order model:

test/models/order_test.rb
# ...
class OrderTest < ActiveSupport::TestCase

  setup do
    @order = orders(:one)
    @product1 = products(:one)
    @product2 = products(:two)
  end

  test 'Should set total' do
    order = Order.new user_id: @order.user_id
    order.products << products(:one)
    order.products << products(:two)
    order.save

    assert_equal (@product1.price + @product2.price), order.total
  end
end

We can now add the implementation:

app/models/order.rb
class Order < ApplicationRecord
  # ...
  def set_total!
    self.total = products.map(&:price).sum
  end
end

We can now hook the set_total! method to a before_validation callback to ensure it has the correct total before it is validated.

app/models/order.rb
class Order < ApplicationRecord
  before_validation :set_total!
  # ...
end

We are making sure the total is always present and bigger or equal to zero. This means we can remove those validations and remove the specs. I’ll wait. Our tests should be passing by now:

$ rake test

...........F

Failure:
OrderTest#test_Should_have_a_positive_total [/home/arousseau/github/madeindjs/market_place_api/test/models/order_test.rb:14]:
Expected true to be nil or false


rails test test/models/order_test.rb:11

............................

Finished in 0.542600s, 73.7191 runs/s, 110.5786 assertions/s.

Oops! We get a failure on our previous test Should have a positive total. This is logical since the order total is calculated dynamically. So we can simply remove this test that has become obsolete.

Our tests must continue to pass. Let’s commit our changes:

$ git commit -am "Adds the create method for the orders controller"

Send order confirmation email

The last section for this chapter will be to sent a confirmation email for the user who just placed it. If you want to skip this and jump into the next chapter, go ahead. This section is more like a warmup.

You may be familiar with email manipulation with Rails so I’ll try to make this fast and simple. We first create the order_mailer with an email named send_confirmation:

$ rails generate mailer order_mailer send_confirmation

Now we can add some tests for the order mails we just created:

test/mailers/order_mailer_test.rb
# ...
class OrderMailerTest < ActionMailer::TestCase

  setup do
    @order = orders(:one)
  end

  test "should be set to be delivered to the user from the order passed in" do
    mail = OrderMailer.send_confirmation(@order)
    assert_equal "Order Confirmation", mail.subject
    assert_equal [@order.user.email], mail.to
    assert_equal ['no-reply@marketplace.com'], mail.from
    assert_match "Order: ##{@order.id}", mail.body.encoded
    assert_match "You ordered #{@order.products.count} products", mail.body.encoded
  end

end

I simply copied/pasted tests from the documentation and adapted them to our needs. We must now ensure that these tests pass.

First, we add the method OrderMailer#send_confirmation:

app/mailers/order_mailer.rb
class OrderMailer < ApplicationMailer
  default from: 'no-reply@marketplace.com'
  def send_confirmation(order)
    @order = order
    @user = @order.user
    mail to: @user.email, subject: 'Order Confirmation'
  end
end

After adding this code we must add corresponding views. It is a good practice to include a text version in addition to the HTML version.

<%# app/views/order_mailer/send_confirmation.text.erb %>
Order: #<%= @order.id %>
You ordered <%= @order.products.count %> products:
<% @order.products.each do |product| %>
  <%= product.title %> - <%= number_to_currency product.price %>
<% end %>
<!-- app/views/order_mailer/send_confirmation.html.erb -->
<h1>Order: #<%= @order.id %></h1>
<p>You ordered <%= @order.products.count %> products:</p>
<ul>
  <% @order.products.each do |product| %>
    <li><%= product.title %> - <%= number_to_currency product.price %></li>
  <% end %>
</ul>

Now, our tests should pass:

$ rake test
........................................
40 runs, 66 assertions, 0 failures, 0 errors, 0 skips

And now, just call the method OrderMailer#send_confirmation in the creation action on the order controller:

app/controllers/api/v1/orders_controller.rb
class Api::V1::OrdersController < ApplicationController
  # ...
  def create
    order = current_user.orders.build(order_params)

    if order.save
      OrderMailer.send_confirmation(order).deliver
      render json: order, status: 201
    else
      render json: { errors: order.errors }, status: 422
    end
  end
  # ...
end

To make sure we didn’t break anything, let’s run all the tests:

$ rake test
........................................
40 runs, 66 assertions, 0 failures, 0 errors, 0 skips

Let’s commit to everything we’ve just done to complete this section:

$ git add . && git commit -m "Adds order confirmation mailer"

And as we get to the end of our chapter, it is time to apply all our modifications to the master branch by making a `merge':

$ git checkout master
$ git merge chapter07

Conclusion

That’s it! You did it! You can applaud yourself. I know it’s been a long time but believe me, it’s almost over.

In the next chapters, we will continue working on the order template to add validations when placing an order. Some scenarios are:

  • What happens when products are not available?

  • Decrease the quantity of the product in progress when placing an order

The next chapter will be short, but it is essential for the health of the application. So don’t skip it.

Improving orders

In the previous chapter, we extended our API to place orders and send a confirmation email to the user (just to improve the user experience). This chapter will take care of some validations on the order model, just to make sure it is placeable, just like:

  • Decrement the current product quantity when an order is placed

  • What happens when products are not available?

We’ll probably need to update the JSON output for the orders a little, but let’s not spoil things up.

So now that we have everything clear, we can get our hands dirty. You can clone the project up to this point with:

$ git checkout tags/checkpoint_chapter08

Let’s create a branch to start working:

$ git checkout -b chapter08

Decrementing product quantity

On this first stop, we will update the product quantity to make sure every order will deliver the actual product. Currently, the product model doesn’t have a quantity attribute. So let’s do that:

$ rails generate migration add_quantity_to_products quantity:integer

Wait, don’t run migrations now. We’ll be making a small modification to it. As a good practice, I like adding default values for the database just making sure I don’t mess things up with null values. This is a perfect case!

Your migration file should look like this:

db/migrate/20190621105101_add_quantity_to_products.rb
class AddQuantityToProducts < ActiveRecord::Migration[6.0]
  def change
    add_column :products, :quantity, :integer, default: 0
  end
end

Now we can migrate the database:

$ rake db:migrate

And let’s not forget to update the fixtures by adding the quantity field (I chose the value 5 totally randomly).

test/fixtures/products.yml
one:
  # ...
  quantity: 5

two:
  # ...
  quantity: 5

another_tv:
  # ...
  quantity: 5

It is now time to reduce the product’s quantity once the Order has been passed. The first thing probably coming to mind is to do it in the Order model. This is a common mistake.

When you work with Many-to-Many associations, we completely forget the join model, which is Placement. The Placement is a better place to manage this because we have access to the order and the product. This way, we can easily reduce the stock of the product.

Before we start implementing the code, we need to change how we manage the creation of the order because we now have to accept a quantity for each product. If you remember, we are waiting for a table of product identifiers. I will try to keep things simple and send a Hash table with the keys product_id and quantity.

A quick example would be something like that:

product_ids_and_quantities = [
  { product_id: 1, quantity: 4 },
  { product_id: 3, quantity: 5 }
]

This is going to be tricky so stay with me. Let’s first build some unit tests:

test/models/order_test.rb
# ...
class OrderTest < ActiveSupport::TestCase
  # ...

  test 'builds 2 placements for the order' do
    @order.build_placements_with_product_ids_and_quantities [
      { product_id: @product1.id, quantity: 2 },
      { product_id: @product2.id, quantity: 3 },
    ]

    assert_difference('Placement.count', 2) do
      @order.save
    end
  end
end

Then into the implementation:

app/models/order.rb
class Order < ApplicationRecord
  # ...

  # @param product_ids_and_quantities [Array<Hash>] something like this `[{product_id: 1, quantity: 2}]`
  # @yield [Placement] placements build
  def build_placements_with_product_ids_and_quantities(product_ids_and_quantities)
    product_ids_and_quantities.each do |product_id_and_quantity|
      placement = placements.build(product_id: product_id_and_quantity[:product_id])
      yield placement if block_given?
    end
  end
end

And if we run our tests, they should be all nice and green:

$ rake test
........................................
40 runs, 60 assertions, 0 failures, 0 errors, 0 skips

The build_placements_with_product_ids_and_quantities will build the placement objects, and once we trigger the save method for the order everything will be inserted into the database. One last step before committing this is to update the orders_controller_test along with its implementation.

First we update the orders_controller_test file:

test/controllers/api/v1/orders_controller_test.rb
# ...
class Api::V1::OrdersControllerTest < ActionDispatch::IntegrationTest
  setup do
    @order = products(:one)
    @order_params = {
      order: {
        product_ids_and_quantities: [
          { product_id: products(:one).id, quantity: 2 },
          { product_id: products(:two).id, quantity: 3 },
        ]
      }
    }
  end

  # ...

  test 'should create order with two products and placements' do
    assert_difference('Order.count', 1) do
      assert_difference('Placement.count', 2) do
        post api_v1_orders_url,
          params: @order_params,
          headers: { Authorization: JsonWebToken.encode(user_id: @order.user_id) },
          as: :json
      end
    end
    assert_response :created
  end
end

Then we need to update the orders_controller:

app/controllers/api/v1/orders_controller.rb
class Api::V1::OrdersController < ApplicationController
  # ...

  def create
    order = Order.create! user: current_user
    order.build_placements_with_product_ids_and_quantities(order_params[:product_ids_and_quantities])

    if order.save
      OrderMailer.send_confirmation(order).deliver
      render json: order, status: :created
    else
      render json: { errors: order.errors }, status: :unprocessable_entity
    end
  end

  private

  def order_params
    params.require(:order).permit(product_ids_and_quantities: [:product_id, :quantity])
  end
end

Note that I also modified the OrdersController#order_params method.

Finally, we need to update the factory product file to assign a high quantity value to have at least a few products in stock.

Let’s commit this changes and keep moving:

$ git add .
$ git commit -m "Allows the order to be placed along with product quantity"

Did you notice we are not saving the quantity for each product anywhere? There is no way to keep track of that. This can be easily fixed by just adding a quantity attribute to the Placement model. So this way for each product we save its corresponding quantity. Let’s start by creating the migration:

$ rails generate migration add_quantity_to_placements quantity:integer

As with the product quantity attribute migration we should add a default value equal to 0. Remember this is optional but I do like this approach. Migration file should look like:

db/migrate/20190621114614_add_quantity_to_placements.rb
class AddQuantityToPlacements < ActiveRecord::Migration[6.0]
  def change
    add_column :placements, :quantity, :integer, default: 0
  end
end

Then run migrations:

$ rake db:migrate

Let’s add the attribute quantity in the fixtures:

test/fixtures/placements.yml
one:
  # ...
  quantity: 5

two:
  # ...
  quantity: 5

Now we just need to update the build_placements_with_product_ids_and_quantities to add the quantity for the placements:

app/models/order.rb
class Order < ApplicationRecord
  # ...

  # @param product_ids_and_quantities [Array<Hash>] something like this `[{product_id: 1, quantity: 2}]`
  # @yield [Placement] placements build
  def build_placements_with_product_ids_and_quantities(product_ids_and_quantities)
    product_ids_and_quantities.each do |product_id_and_quantity|
      placement = placements.build(
        product_id: product_id_and_quantity[:product_id],
        quantity: product_id_and_quantity[:quantity],
      )
      yield placement if block_given?
    end
  end
end

Now our tests should pass:

$ rake test
........................................
40 runs, 61 assertions, 0 failures, 0 errors, 0 skips

Let’s commit the changes:

$ git add . && git commit -m "Adds quantity to placements"

Extending the Placement model

It is time to update the product quantity once the order is saved, or more accurate once the placement is created. To achieve this, we will add a method and then hook it up to an after_create callback.

test/models/placement_test.rb
# ...
class PlacementTest < ActiveSupport::TestCase
  setup do
    @placement = placements(:one)
  end

  test 'decreases the product quantity by the placement quantity' do
    product = @placement.product

    assert_difference('product.quantity', -@placement.quantity) do
      @placement.decrement_product_quantity!
    end
  end
end

Implementation is fairly easy as shown bellow:

app/models/placement.rb
class Placement < ApplicationRecord
  # ...
  after_create :decrement_product_quantity!

  def decrement_product_quantity!
    product.decrement!(:quantity, quantity)
  end
end

Let’s commit our changes:

$ git commit -am "Decreases the product quantity by the placement quantity"

Validate quantity of products

Since the beginning of the chapter, we have added the attribute quantity to the product model. It is now time to validate the quantity of product is sufficient for the order to be placed. To make things more interesting, we will do this using a custom validator.

Note
You can consult documentation.

First, we need to add a validators directory under the app directory (Rails will pick it up for so we do not need to load it).

$ mkdir app/validators
$ touch app/validators/enough_products_validator.rb

Before we drop any code line, we need to add a spec to the Order model to check if the order can be placed.

test/models/order_test.rb
# ...
class OrderTest < ActiveSupport::TestCase
  # ...

  test "an order should not claim too much product than available" do
    @order.placements << Placement.new(product_id: @product1.id, quantity: (1 + @product1.quantity))

    assert_not @order.valid?
  end
end

As you can see on the spec, we first make sure that placement_2 is trying to request more products than are available, so in this case, the order is not supposed to be valid.

The test by now should be failing, let’s turn it into green by adding the code for the validator:

app/validators/enough_products_validator.rb
class EnoughProductsValidator < ActiveModel::Validator
  def validate(record)
    record.placements.each do |placement|
      product = placement.product
      if placement.quantity > product.quantity
        record.errors[product.title.to_s] << "Is out of stock, just #{product.quantity} left"
      end
    end
  end
end

I manage to add a message for each of the products out of stock, but you can handle it differently. Now we just need to add the validator to the Order model like so:

app/models/order.rb
class Order < ApplicationRecord
  include ActiveModel::Validations
  # ...
  validates_with EnoughProductsValidator
  # ...
end

Let’s commit changes:

$ git add . && git commit -m "Adds validator for order with not enough products on stock"

Updating the total

Did you realize that the total is being miscalculated? Currently, it is just adding the price for the products on order regardless of the quantity requested. Let me add the code to clarify the problem:

Currently, in the order model we have this method to calculate the amount to pay:

app/models/order.rb
class Order < ApplicationRecord
  # ...
  def set_total!
    self.total = products.map(&:price).sum
  end
  # ...
end

Instead of calculating the total by just adding the product prices, we need to multiply it by the quantity. So let’s update the spec first:

test/models/order_test.rb
# ...
class OrderTest < ActiveSupport::TestCase
  # ...

  test "Should set total" do
    @order.placements = [
      Placement.new(product_id: @product1.id, quantity: 2),
      Placement.new(product_id: @product2.id, quantity: 2)
    ]
    @order.set_total!
    expected_total = (@product1.price * 2) + (@product2.price * 2)

    assert_equal expected_total, @order.total
  end
end

And the implementation is fairly easy:

app/models/order.rb
class Order < ApplicationRecord
  # ...
  def set_total!
    self.total = self.placements
                     .map{ |placement| placement.product.price * placement.quantity }
                     .sum
  end
  # ...
end

And the specs should be green:

$ rake test
..........................................
42 runs, 63 assertions, 0 failures, 0 errors, 0 skips

Let’s commit the changes and wrap up.

$ git commit -am "Updates the total calculation for order"

And as we get to the end of our chapter, it is time to apply all our modifications to the master branch by making a merge:

$ git checkout master
$ git merge chapter08

Conclusion

Oh, you’re here! Allow me to congratulate you! That’s a long way from the first chapter. But you’re one step closer. In fact, the next chapter will be the last. So try to make the most of it.

The last chapter will focus on optimizing the API using paging, caching, and background tasks. So buckle up, it’s going to be a hectic ride.

Welcome to the last chapter of the book. It has been a long way, but you are only a step away from the end. In the previous chapter, we completed the modeling of the order model. We could say that the project is now finished, but I want to cover some important details about optimization. The topics I will discuss here will be:

  • pagination

  • caching

  • optimization of SQL queries

  • the activation of CORS

I will try to go as far as I can by trying to cover some common scenarios. I hope these scenarios will be useful for some of your projects.

If you start reading at this point, you’ll probably want the code to work, you can clone it like that:

$ git checkout tags/checkpoint_chapter09

Let’s now create a branch to start working:

$ git checkout -b chapter09

Pagination

A very common strategy to optimize an array of records from the database is to load just a few by paginating them and if you are familiar with this technique, you know that in Rails that’s really easy to achieve it whether if you are using will_paginate or kaminari.

Only tricky part in here is how we are supposed to handle the JSON output now to give enough information to the client on how the array is paginated. If you recall the first chapter I shared some resources on the practices I would be following in here. One of them was http://jsonapi.org/, which is a must-bookmark page.

If we read the format section, we will reach a sub-section called Top Level and in very few words they mention something about pagination:

"meta": meta-information about a resource, such as pagination.

It is not very descriptive, but we have a hint on what to look next about the pagination implementation, but don’t worry, that is exactly what we are going to do here.

Let’s start with the products list.

Products

We will start nice and easy by paginating the products list as we don’t have any kind of access restriction, which leads to easier testing.

First, we need to add the kaminari gem to our Gemfile:

$ bundle add kaminari

Now we can go to the index action on the products_controller and add the pagination methods as pointed on the documentation:

app/controllers/api/v1/products_controller.rb
class Api::V1::ProductsController < ApplicationController
  # ...
  def index
    @products = Product.page(params[:page])
                       .per(params[:per_page])
                       .search(params)

    render json: ProductSerializer.new(@products).serializable_hash.to_json
  end
  # ...
end

The only thing that changed is the query on the database to limit the result by 25 per page, which is the default. But we have not added any extra information to the JSON output.

We need to provide the pagination information on the meta tag in the following form:

{
  "data": [
    ...
  ],
  "links": {
    "first": "/api/v1/products?page=1",
    "last": "/api/v1/products?page=30",
    "prev": "/api/v1/products",
    "next": "/api/v1/products?page=2"
  }
}

Now we have the final structure for the meta tag we just need to output it on the JSON response. Let’s first add some specs:

test/controllers/api/v1/products_controller_test.rb
# ...
class Api::V1::ProductsControllerTest < ActionDispatch::IntegrationTest
  # ...
  test 'should show products' do
    get api_v1_products_url, as: :json
    assert_response :success

    json_response = JSON.parse(response.body, symbolize_names: true)
    assert_not_nil json_response.dig(:links, :first)
    assert_not_nil json_response.dig(:links, :last)
    assert_not_nil json_response.dig(:links, :prev)
    assert_not_nil json_response.dig(:links, :next)
  end
  # ...
end

The test we have just added should fail:

$ rake test
......................F

Failure:
Api::V1::ProductsControllerTest#test_should_show_products [test/controllers/api/v1/products_controller_test.rb:13]:
Expected nil to not be nil.

Let’s add the pagination information. We will make a part of it in a separate concerns in order to better decouple our code:

app/controllers/concerns/paginable.rb
# app/controllers/concerns/paginable.rb
module Paginable
  protected

  def current_page
    (params[:page] || 1).to_i
  end

  def per_page
    (params[:per_page] || 20).to_i
  end
end

And now we can use it in the controller.

app/controllers/api/v1/products_controller.rb
class Api::V1::ProductsController < ApplicationController
  include Paginable
  # ...

  def index
    @products = Product.page(current_page)
                       .per(per_page)
                       .search(params)

    options = {
      links: {
        first: api_v1_products_path(page: 1),
        last: api_v1_products_path(page: @products.total_pages),
        prev: api_v1_products_path(page: @products.prev_page),
        next: api_v1_products_path(page: @products.next_page),
      }
    }

    render json: ProductSerializer.new(@products, options).serializable_hash.to_json
  end
end

Now, if we check the specifications, they should all pass:

$ rake test
..........................................
42 runs, 65 assertions, 0 failures, 0 errors, 0 skips

Now we have made a superb optimization for the product list route. It is up to the customer to retrieve the page with the right per_page parameter for registrations.

Let’s make these changes and continue with the list of commands.

$ git add .
$ git commit -m "Adds pagination for the products index action to optimize response"

Orders list

Now it’s time to do exactly the same for the orders list endpoint, which should be really easy to implement. But first, let’s add some specs to the orders_controller_test.rb file:

test/controllers/api/v1/orders_controller_test.rb
# ...
class Api::V1::OrdersControllerTest < ActionDispatch::IntegrationTest
  # ...
  test 'should show orders' do
    get api_v1_orders_url, headers: { Authorization: JsonWebToken.encode(user_id: @order.user_id) }, as: :json
    assert_response :success

    json_response = JSON.parse(response.body, symbolize_names: true)
    assert_equal @order.user.orders.count, json_response[:data].count
    assert_not_nil json_response.dig(:links, :first)
    assert_not_nil json_response.dig(:links, :last)
    assert_not_nil json_response.dig(:links, :prev)
    assert_not_nil json_response.dig(:links, :next)
  end
  # ...
end

As you may already know, our tests are no longer passing:

$ rake test
......................................F

Failure:
Api::V1::OrdersControllerTest#test_should_show_orders [test/controllers/api/v1/orders_controller_test.rb:28]:
Expected nil to not be nil.

Let’s turn the red into green:

app/controllers/api/v1/orders_controller.rb
class Api::V1::OrdersController < ApplicationController
  include Paginable
  # ...

  def index
    @orders = current_user.orders
                          .page(current_page)
                          .per(per_page)

    options = {
      links: {
        first: api_v1_orders_path(page: 1),
        last: api_v1_orders_path(page: @orders.total_pages),
        prev: api_v1_orders_path(page: @orders.prev_page),
        next: api_v1_orders_path(page: @orders.next_page),
      }
    }

    render json: OrderSerializer.new(@orders, options).serializable_hash.to_json
  end
  # ...
end

Now all the tests should be nice and green:

$ rake test
..........................................
42 runs, 67 assertions, 0 failures, 0 errors, 0 skips

Let’s place and commit, because a refactor is coming:

$ git commit -am "Adds pagination for orders index action"

Refactoring pagination

If you have followed this tutorial or are an experienced Rails developer, you probably like to keep things DRY. You may have noticed that the code we just wrote is duplicated. I think it’s good to clean up the code a little once the functionality is implemented.

We will first clean up these tests that we duplicated in the file orders_controller_test.rb and products_controller_test.rb:

assert_not_nil json_response.dig(:links, :first)
assert_not_nil json_response.dig(:links, :last)
assert_not_nil json_response.dig(:links, :next)
assert_not_nil json_response.dig(:links, :prev)

To refactor it, we will move these assertions into the test_helper.rb file in a method we will use:

test/test_helper.rb
# ...
class ActiveSupport::TestCase
  # ...
  def assert_json_response_is_paginated json_response
    assert_not_nil json_response.dig(:links, :first)
    assert_not_nil json_response.dig(:links, :last)
    assert_not_nil json_response.dig(:links, :next)
    assert_not_nil json_response.dig(:links, :prev)
  end
end

This method can now be used to replace the four assertions in the orders_controller_test.rb and products_controller_test.rb files:

test/controllers/api/v1/orders_controller_test.rb
# ...
class Api::V1::OrdersControllerTest < ActionDispatch::IntegrationTest
  # ...
  test 'should show orders' do
    # ...
    assert_json_response_is_paginated json_response
  end
  # ...
end
test/controllers/api/v1/products_controller_test.rb
# ...
class Api::V1::ProductsControllerTest < ActionDispatch::IntegrationTest
  # ...
  test 'should show products' do
    # ...
    assert_json_response_is_paginated json_response
  end
  # ...
end

And both specs should be passing.

$ rake test
..........................................
42 runs, 71 assertions, 0 failures, 0 errors, 0 skips

Now we have done this simple factorization for testing, we can move on to implementing pagination for controllers and clean things up. If you remember the indexing action for both product and order controllers, they both have the same pagination format. Let’s move this logic into a method called get_links_serializer_options under the file paginable.rb, so we can access it on any controller that would need paging.

app/controllers/concerns/paginable.rb
module Paginable
  protected

  def get_links_serializer_options links_paths, collection
    {
      links: {
        first: send(links_paths, page: 1),
        last: send(links_paths, page: collection.total_pages),
        prev: send(links_paths, page: collection.prev_page),
        next: send(links_paths, page: collection.next_page),
      }
    }
  end
  # ...
end

And now we can substitute the pagination hash on both controllers for the method. Like so:

app/controllers/api/v1/orders_controller.rb
class Api::V1::OrdersController < ApplicationController
  include Paginable
  # ...

  def index
    @orders = current_user.orders
                          .page(current_page)
                          .per(per_page)

    options = get_links_serializer_options('api_v1_orders_path', @orders)

    render json: OrderSerializer.new(@orders, options).serializable_hash.to_json
  end
  # ...
end
app/controllers/api/v1/products_controller.rb
class Api::V1::ProductsController < ApplicationController
  include Paginable
  # ...

  def index
    @products = Product.page(current_page)
                       .per(per_page)
                       .search(params)

    options = get_links_serializer_options('api_v1_products_path', @products)

    render json: ProductSerializer.new(@products, options).serializable_hash.to_json
  end
  # ...
end

If you run the specs for each file, they should be all nice and green:

$ rake test
..........................................
42 runs, 71 assertions, 0 failures, 0 errors, 0 skips

This would be a good time to commit the changes and move on to the next section on caching.

$ git commit -am "Factorize pagination"

API Caching

There is currently an implementation to do caching with the gem jsonapi-serializer which is really easy to handle. Although in older versions of the gem, this implementation can change, it does the job.

If we request the product list, we will notice that the response time takes about 174 milliseconds using cURL:

$ curl -w 'Total: %{time_total}\n' -o /dev/null -s http://localhost:3000/api/v1/products
Total: 0,137088
Note
The -w option allows us to retrieve the time of the request, -o' redirects the response to a file, and `-s hides the cURL display

By adding only one line to the ProductSerializer class, we will see a significant improvement in response time!

app/serializers/order_serializer.rb
class OrderSerializer
  # ...
  cache_options store: Rails.cache, namespace: 'jsonapi-serializer', expires_in: 1.hour
end
app/serializers/product_serializer.rb
class ProductSerializer
  # ...
  cache_options store: Rails.cache, namespace: 'jsonapi-serializer', expires_in: 1.hour
end
app/serializers/user_serializer.rb
class UserSerializer
  # ...
  cache_options store: Rails.cache, namespace: 'jsonapi-serializer', expires_in: 1.hour
end

And that’s all! Let’s check for improvement:

$ curl -w 'Total: %{time_total}\n' -o /dev/null -s http://localhost:3000/api/v1/products
Total: 0,054786
$ curl -w 'Total: %{time_total}\n' -o /dev/null -s http://localhost:3000/api/v1/products
Total: 0,032341

So we went from 174 ms to 21 ms. The improvement is, therefore, enormous! Let’s commit our changes:

$ git commit -am "Adds caching for the serializers"

N+1 Queries

N+1* requests are a wound that can have a huge impact on the performance of an application. This phenomenon often occurs when using an ORM because it generates automatically SQL queries for us. This handy tool is double-edged because it can generate a large number of SQL queries.

Something to know about SQL queries is that it’s better to limit the number. In other words, a large request is often more efficient than a hundred small ones.

Here is an example where we want to recover all users who have already created a product. Open the Rails console with rails console and execute the following Ruby code:

Product.all.map { |product| product.user }

The interactive console of Rails shows us the SQL queries that are generated. See for yourself:

We see here that a large number of requests are generated:

  • Product.all = 1 request to recover the products

  • product.user = 1 request SELECT "users".* FROM "users" WHERE "users". "id" =? LIMIT 1 [[[["id", 1]]] per product recovered

Hence the name "N+1 request" since a request is made via a child link.

We can fix this simply by using includes. Includes will pre-load the child objects in a single request. It is very easy to use. If we repeat the previous example. Here is the result:

Product.includes(:user).all.map { |product| product.user }

The interactive console of Rails shows us the SQL queries that are generated. See for yourself:

Product Load (0.3ms)  SELECT "products".* FROM "products"
User Load (0.8ms)  SELECT "users".* FROM "users" WHERE "users"."id" IN (?, ?, ?)  [["id", 28], ["id", 29], ["id", 30]]

Rails make a second request that will retrieve all users at once.

Prevention of N + 1 requests

Imagine we want adding owners of the products for the path /products. We have already seen that with the fast_jsonapi library it is straightforward to do this:

app/controllers/api/v1/products_controller.rb
class Api::V1::ProductsController < ApplicationController
  # ...
  def index
    # ...
    options = get_links_serializer_options('api_v1_products_path', @products)
    options[:include] = [:user]

    render json: ProductSerializer.new(@products, options).serializable_hash.to_json
  end
  # ...
end

Now let’s make a request with cURL. I remind you we must obtain an authentication token before accessing the page.

$ curl -X POST --data "user[email]=ockymarvin@jacobi.co" --data "user[password]=locadex1234"  http://localhost:3000/api/v1/tokens
Note
"ockymarvin@jacobi.co" corresponds to a user created in my application with the seed. In your case, it will probably be different from mine since we used the Faker library.

With the help of the token obtained, we can now make a request to access the products

$ curl --header "Authorization=ey..." http://localhost:3000/api/v1/products

You will most likely see several requests in the Rails console running the webserver.

Started GET "/api/v1/products" for 127.0.0.1 at 2019-06-26 13:36:19 +0200
Processing by Api::V1::ProductsController#index as JSON
   (0.1ms)  SELECT COUNT(*) FROM "products"
   app/controllers/concerns/paginable.rb:9:in `get_links_serializer_options'
  Product Load (0.2ms)  SELECT "products".* FROM "products" LIMIT ? OFFSET ?  [["LIMIT", 20], ["OFFSET", 0]]
  ↳ app/controllers/api/v1/products_controller.rb:16:in `index'
  User Load (0.1ms)  SELECT "users".* FROM "users" WHERE "users"."id" = ? LIMIT ?  [["id", 36], ["LIMIT", 1]]
  ↳ app/controllers/api/v1/products_controller.rb:16:in `index'
   (0.5ms)  SELECT "products"."id" FROM "products" WHERE "products"."user_id" = ?  [["user_id", 36]]
   app/controllers/api/v1/products_controller.rb:16:in `index'
  CACHE User Load (0.0ms)  SELECT "users".* FROM "users" WHERE "users"."id" = ? LIMIT ?  [["id", 36], ["LIMIT", 1]]
  ↳ app/controllers/api/v1/products_controller.rb:16:in `index'
  CACHE User Load (0.0ms)  SELECT "users".* FROM "users" WHERE "users"."id" = ? LIMIT ?  [["id", 36], ["LIMIT", 1]]
  ↳ app/controllers/api/v1/products_controller.rb:16:in `index'
  CACHE User Load (0.0ms)  SELECT "users".* FROM "users" WHERE "users"."id" = ? LIMIT ?  [["id", 36], ["LIMIT", 1]]

It is, therefore, unfortunately very easy to create an N+1 query. Fortunately, a gem allows us to alert when this kind of situation occurs: Bullet. Bullet will notify us (by email, growl notification, Slack, console, etc…​) when it finds an N+1 request.

To install it, we add the gem to the GemFile

$ bundle add bullet --group development

And it is enough to update the configuration of our application for the development environment. In our case, we will only activate the rails_logger mode, which will be displayed:

config/environments/development.rb
Rails.application.configure do
  # ...
  config.after_initialize do
    Bullet.enable = true
    Bullet.rails_logger = true
  end
end

Restart the webserver and restart the last request with cURL:

$ curl --header "Authorization=ey..." http://localhost:3000/api/v1/products

And look at the Rails console. Bullet tells us that it has just detected an N+1 request.

GET /api/v1/products
USE eager loading detected
  Product => [:user]
  Add to your finder: :includes => [:user]

He even tells us how to correct it:

> Add to your search engine

includes ⇒ [: user]

So we correct our error in the controller:

app/controllers/api/v1/products_controller.rb
class Api::V1::ProductsController < ApplicationController
  # ...
  def index
    @products = Product.includes(:user)
                       .page(current_page)
                       .per(per_page)
                       .search(params)

    options = get_links_serializer_options('api_v1_products_path', @products)
    options[:include] = [:user]

    render json: ProductSerializer.new(@products, options).serializable_hash.to_json
  end
  # ...
end

There you go! It is now time to do our commit.

$ git commit -am "Add bullet to avoid N+1 query"

Activation of CORS

In this last section, I will discuss one last problem that you will probably encounter if you have to work with your API.

When you first request an external site (via an AJAX request for example), you will encounter an error of this kind:

Failed to load https://example.com/ No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin "https://anfo.pl" is therefore not allowed access. If an opaque response serves your needs, set the request’s mode to "no-cors" to fetch the resource with CORS disabled.

"But what does Access-Control-Allow-Origin mean?". The behavior you observe is the effect of the CORS implementation of browsers. Before the CORS standardization, there was no way to call an API terminal under another domain for security reasons. This has been (and still is to some extent) blocked by the same origin policy.

CORS is a mechanism that aims to allow requests made on your behalf, and at the same time block some requests made by dishonest scripts and is triggered when you make an HTTP request to:

  • a different field

  • a different sub-domain

  • a different port

  • a different protocol

We must manually enable this feature so that any client can make requests on our API.

Rails allow us to do this very easily. Take a look at the cors.rb file located in the initializers folder.

config/initializers/cors.rb
# ...

# Rails.application.config.middleware.insert_before 0, Rack::Cors do
#   allow do
#     origins 'example.com'
#
#     resource '*',
#       headers: :any,
#       methods: [:get, :post, :put, :patch, :delete, :options, :head]
#   end
# end

You see. It is enough to uncomment the code and modify it slightly to limit access to some actions or some HTTP verbs. In our case, this configuration is very convenient for us at the moment.

config/initializers/cors.rb
# ...

Rails.application.config.middleware.insert_before 0, Rack::Cors do
  allow do
    origins 'example.com'
    resource '*',
      headers: :any,
      methods: [:get, :post, :put, :patch, :delete, :options, :head]
  end
end

We must also install the gem rack-cors, which is commented in the Gemfile:

$ bundle add rack-cors

There you go! It is now time to make our last commit and merge our changes on the master branch.

$ git commit -am "Activate CORS"
$ git checkout master
$ git merge chapter09

Conclusion

If you get to that point, it means you’re done with the book. Good work! You have just become a great API Rails developer, that’s for sure.

So together we have built a solid and complete API. This one has all the qualities to dethrone Amazon, rest assured. Thank you for going through this great adventure with me, I hope you enjoyed the trip as much as I did.

I would like to remind you that this book’s source code is available in the format Asciidoctor on GitHub. So do not hesitate to fork the project if you want to improve it or correct a mistake that I missed.

If you like this book, don’t hesitate to let me know by email contact@rousseau-alexandre.fr. I am open to any criticism, good or bad, over a good beer :).