Elixir JSON RESTful API (without Phoenix)

How to build, from scratch, an Elixir JSON Rest API without using Phoenix (the database will be mongodb)

What we will do ?

We will see together how to build, from scratch, an Elixir JSON Rest API without using Phoenix.

Database

mongodb

Tasks to do

• init and configure the project.
• start mongodb driver as a worker (process) and attach it to a supervisor.
• add user CRUDs : find_all_users, user_by_email, user_by_username, add_user, delete_user, update_user_by_email and update_user_by_username.
• add needed endpoints.

This is the first step before using Phoenix in order to understand what Phoenix do behind the scene.

Project source code

helabenkhalfallah/elixir_json_restfull_api

Creating project

mix new elixir_json_restfull_api --sup

--sup will create an app suitable for use as an OTP application. The server will be supervised and restarted automatically in the event of a crash, while the server may crash the Erlang VM should not (at least not easily).

Configuring project

1. Add required code quality tools (mix.exs) :

{:credo, "~> 1.3", only: :dev, runtime: false},
{:dialyxir, "~> 1.0", only: :dev, runtime: false},
{:ex_doc, "~> 0.21.3", only: :dev, runtime: false},
{:inch_ex, github: "rrrene/inch_ex", only: [:dev, :test]},
{:excoveralls, "~> 0.12.3", only: :test}

2. Create alias (mix.exs) :

def aliases do
  [
    test_ci: [
      "test",
      "coveralls"
    ],
    code_review: [
      "dialyzer",
      "credo --strict"
    ],
    generate_docs: [
      "docs",
      "inch"
    ]
  ]
end

3. Change project configuration (mix.exs) :

def project do
  [
    app: :elixir_json_restfull_api,
    aliases: aliases(),
    version: "0.1.0",
    elixir: "~> 1.9",
    start_permanent: Mix.env() == :prod,
    deps: deps(),
    test_coverage: [tool: ExCoveralls],
    preferred_cli_env: [
      coveralls: :test,
      "coveralls.detail": :test,
      "coveralls.post": :test,
      "coveralls.html": :test
    ]
  ]
end

4. Mix commands :

Now we can use mix custom commands :

mix format
mix code_review # code review : credo & dialyxir
mix generate_docs # generate doc and doc coverage : ex_doc & inch_ex
mix test_ci # unit tests & coverage : EUnit & excoveralls

Cowboy like Node.js express server

1. Add dependencies

{:plug_cowboy, "~> 2.2"} # server
{:poison, "~> 4.0.1"} # json encode/decode

2. Modify mix.exs

From :

# Run "mix help compile.app" to learn about applications.
def application do
  [
    extra_applications: [:logger],
    mod: {ElixirJsonRestfullApi.Application, []}
  ]
end

To :

# Run "mix help compile.app" to learn about applications.
def application do
  [
    extra_applications: [:logger, :plug_cowboy],
    mod: {ElixirJsonRestfullApi.Application, []}
  ]
end

3. Modify application.ex

defmodule ElixirJsonRestfullApi.Application do
  # See https://hexdocs.pm/elixir/Application.html
  # for more information on OTP Applications
  @moduledoc false

  use Application

  def start(_type, _args) do
    import Supervisor.Spec

    children = [
      # List all child processes to be supervised

      # Start HTTP server
      Plug.Cowboy.child_spec(
        scheme: :http,
        plug: ElixirJsonRestfullApi.UserEndpoint,
        options: Application.get_env(:elixir_json_restfull_api, :endPoint)[:port]
      ),
    ]

    # See https://hexdocs.pm/elixir/Supervisor.html
    # for other strategies and supported options
    opts = [
      strategy: :one_for_one,
      name: ElixirJsonRestfullApi.Supervisor
    ]

    Supervisor.start_link(children, opts)
  end
end

4. Create config files

Config files let us handle application configuration according to deployment environment :

Config folder

Config.exs :

# This file is responsible for configuring your application
# and its dependencies with the aid of the Mix.Config module.
#
# This configuration file is loaded before any dependency and
# is restricted to this project.
use Mix.Config

# Import environment specific config. This must remain at the bottom
# of this file so it overrides the configuration defined above.
import_config "#{Mix.env()}.exs"

dev.exs :

use Mix.Config

# For development, we disable any cache and enable
# debugging and code reloading.

config :elixir_json_restfull_api, :endPoint, port: [port: 4000]

5. Create a UserEndpoint module

defmodule ElixirJsonRestfullApi.UserEndpoint do
  @moduledoc """
  User Model :
  ```
    {
      "username": "helabenkhalfallah",
      "password": "XXX",
      "lastName": "ben khalfallah",
      "firstName": "hela",
      "email": "helabenkhalfallah@hotmail.fr",
    }
  ```

  User endpoints :
  - /users : get all users (GET)
  - /user-by-email : find user by email (POST)
  - /user-by-user-name : find user by username (POST)
  - /add-user : add a new user (POST)
  - /delete-user : delete an existing user (POST)
  - /update-user-email : update an existing user by email (POST)
  - /update-update-user-name : update an existing user by username (POST)

  """

  @doc """
    Plug provides Plug.Router to dispatch incoming requests based on the path and method.
    When the router is called, it will invoke the :match plug, represented by a match/2function responsible
    for finding a matching route, and then forward it to the :dispatch plug which will execute the matched code.

    Mongo :
    https://hexdocs.pm/mongodb/Mongo.html#update_one/5

    Enum :
    https://hexdocs.pm/elixir/Enum.html#into/2

    Example :
    https://tomjoro.github.io/2017-02-09-ecto3-mongodb-phoenix/
  """
  use Plug.Router

  # This module is a Plug, that also implements it's own plug pipeline, below:

  # Using Plug.Logger for logging request information
  plug(Plug.Logger)

  # responsible for matching routes
  plug(:match)

  # Using Poison for JSON decoding
  # Note, order of plugs is important, by placing this _after_ the 'match' plug,
  # we will only parse the request AFTER there is a route match.
  plug(Plug.Parsers,
    parsers: [:json],
    pass: ["application/json"],
    json_decoder: Poison
  )

  # responsible for dispatching responses
  plug(:dispatch)

  # A simple route to test that the server is up
  # Note, all routes must return a connection as per the Plug spec.
  # Get all users
  get "/users" do
    conn
    |> put_resp_content_type("application/json")
    |> send_resp(200, Poison.encode!(%{response: "This a test message !"}))
  end


  # A catchall route, 'match' will match no matter the request method,
  # so a response is always returned, even if there is no route to match.
  match _ do
    send_resp(conn, 404, "Unknown request :( !")
  end
end

We are using the macros, get and post, from Plug.Router to generate our routes.match and dispatch are required in order for us to handle requests and dispatch responses.

match should be before we define our parser, this means we will not parse anything unless there is a route match.

mix deps.get
mix compile
iex -S mix run
recompile # each time we modify project we should run recompile

Running server — Postman

iex — call traces

Mongodb

1. Adding dependencies

{:mongodb_driver, "~> 0.6"},
{:poolboy, "~> 1.5"},

2. Change application.ex

defmodule ElixirJsonRestfullApi.Application do
  # See https://hexdocs.pm/elixir/Application.html
  # for more information on OTP Applications
  @moduledoc false

  use Application

  def start(_type, _args) do
    import Supervisor.Spec

    children = [
      # List all child processes to be supervised

      # Start HTTP server
      Plug.Cowboy.child_spec(
        scheme: :http,
        plug: ElixirJsonRestfullApi.UserEndpoint,
        options: Application.get_env(:elixir_json_restfull_api, :endPoint)[:port]
      ),

      # Start mongo
      worker(Mongo, [
        [
          name: :mongo,
          database: Application.get_env(:elixir_json_restfull_api, :db)[:database],
          pool_size: Application.get_env(:elixir_json_restfull_api, :db)[:pool_size]
        ]
      ])
    ]

    # See https://hexdocs.pm/elixir/Supervisor.html
    # for other strategies and supported options
    opts = [
      strategy: :one_for_one,
      name: ElixirJsonRestfullApi.Supervisor
    ]

    Supervisor.start_link(children, opts)
  end
end

3. Change config files (dev.exs for example)

use Mix.Config

# For development, we disable any cache and enable
# debugging and code reloading.

config :elixir_json_restfull_api, :endPoint, port: [port: 4000]

config :elixir_json_restfull_api, :db,
  database: "local", # db name
  pool_size: 3 # pool size

4. Create a UserReader module

defmodule ElixirJsonRestfullApi.UserReader do
  @moduledoc """
    User queries :
      - /users : get all users (GET)
      - /user-by-email : find user by email (POST)
      - /user-by-user-name : find user by username (POST)
  """

  @doc """
  Find all users from mongo db
  """
  @spec find_all_users() :: list
  def find_all_users do
    # Gets All Users from Mongo
    cursor = Mongo.find(:mongo, "User", %{})

    # Json encode result
    cursor
    |> Enum.to_list()
    |> handle_users_db_status()
  end

  @doc """
  Handle fetched database users status
  """
  @spec handle_users_db_status(list) :: list
  def handle_users_db_status(users) do
    if Enum.empty?(users) do
      []
    else
      users
    end
  end

  @doc """
  Find user by mail
  """
  @spec user_by_email(%{}) :: %{}
  def user_by_email(email) do
    Mongo.find_one(:mongo, "User", %{email: email})
  end

  @doc """
  Find user by username
  """
  @spec user_by_username(%{}) :: %{}
  def user_by_username(username) do
    Mongo.find_one(:mongo, "User", %{username: username})
  end
end

5. Create a UserWriter module

defmodule ElixirJsonRestfullApi.UserWriter do
  @moduledoc """
    User mutations :
      - /add-user : add a new user (POST)
      - /delete-user : delete an existing user (POST)
      - /update-user-email : update an existing user by email (POST)
      - /update-update-user-name : update an existing user by username (POST)
  """

  @doc """
  Add User
  """
  @spec add_user(%{}) :: any
  def add_user(user_to_add) do
    case Mongo.insert_one(:mongo, "User", user_to_add) do
      {:ok, user} -> {:ok, user}
      {:error, changeset} -> {:error, changeset}
    end
  end

  @doc """
  Delete User
  """
  @spec delete_user(%{}) :: any
  def delete_user(user_to_delete) do
    case Mongo.delete_one(:mongo, "User", user_to_delete) do
      {:ok, user} -> {:ok, user}
      {:error, changeset} -> {:error, changeset}
    end
  end

  @doc """
  Update user by email
  username & email are unique identifiers
  and should not been modified
  """
  @spec update_user_by_email(%{}) :: any
  def update_user_by_email(user) do
    case Mongo.update_one(
           :mongo,
           "User",
           %{"email" => user["email"]},
           %{"$set" => params_to_json(user)},
           return_document: :after
         ) do
      {:ok, user} -> {:ok, user}
      {:error, changeset} -> {:error, changeset}
    end
  end

  @doc """
  Update user by username
  username & email are unique identifiers
  and should not been modified
  """
  @spec update_user_by_username(%{}) :: any
  def update_user_by_username(user) do
    case Mongo.update_one(
           :mongo,
           "User",
           %{"username" => user["username"]},
           %{"$set" => params_to_json(user)},
           return_document: :after
         ) do
      {:ok, user} -> {:ok, user}
      {:error, changeset} -> {:error, changeset}
    end
  end

  @doc """
  Convert params object to mongo object
  """
  @spec params_to_json(%{}) :: any
  def params_to_json(params) do
    # we should keep same id
    # user can't modify id, username or email (unique identifiers)
    attributes =
      params
      |> Map.delete("id")
      |> Map.delete("username")
      |> Map.delete("email")

    reduced =
      Enum.into(
        attributes,
        %{},
        fn {key, value} ->
          {"#{key}", value}
        end
      )
  end
end

6. ServiceUtils module

defmodule ElixirJsonRestfullApi.ServiceUtils do
  @moduledoc """
  Services Utils
  """

  @doc """
  Extends BSON to encode Mongo DB Object Id binary type.
  Mongo object id are like : "_id": "5d658cd0de28b65b8db81606"
  """
  defimpl Poison.Encoder, for: BSON.ObjectId do
    def encode(id, options) do
      BSON.ObjectId.encode!(id)
      |> Poison.Encoder.encode(options)
    end
  end

  @doc """
  Format service success response
  """
  @spec endpoint_success(any) :: binary
  def endpoint_success(data) do
    Poison.encode!(%{
      "status" => 200,
      "data" => data
    })
  end

  @doc """
  Format service fail response
  """
  @spec endpoint_error(binary) :: binary
  def endpoint_error(error_type) do
    Poison.encode!(%{
      "status" => 200,
      "fail_reason" =>
        cond do
          error_type == 'empty' -> "Empty Data"
          error_type == 'not_found' -> "Not found"
          error_type == 'missing_email' -> "Missing email"
          error_type == 'missing_username' -> "Missing username"
          error_type == 'missing_prams' -> "Missing query params"
          true -> "An expected error was occurred"
        end
    })
  end
end

7. Modify UserEndPoint module

defmodule ElixirJsonRestfullApi.UserEndpoint do
  @moduledoc """
  User Model :
  ```
    {
      "username": "helabenkhalfallah",
      "password": "XXX",
      "lastName": "ben khalfallah",
      "firstName": "hela",
      "email": "helabenkhalfallah@hotmail.fr",
    }
  ```

  User endpoints :
  - /users : get all users (GET)
  - /user-by-email : find user by email (POST)
  - /user-by-user-name : find user by username (POST)
  - /add-user : add a new user (POST)
  - /delete-user : delete an existing user (POST)
  - /update-user-email : update an existing user by email (POST)
  - /update-update-user-name : update an existing user by username (POST)

  """

  alias ElixirJsonRestfullApi.ServiceUtils, as: ServiceUtils
  alias ElixirJsonRestfullApi.UserReader, as: UserReader
  alias ElixirJsonRestfullApi.UserWriter, as: UserWriter

  @doc """
    Plug provides Plug.Router to dispatch incoming requests based on the path and method.
    When the router is called, it will invoke the :match plug, represented by a match/2function responsible
    for finding a matching route, and then forward it to the :dispatch plug which will execute the matched code.

    Mongo :
    https://hexdocs.pm/mongodb/Mongo.html#update_one/5

    Enum :
    https://hexdocs.pm/elixir/Enum.html#into/2

    Type :
    https://hexdocs.pm/elixir/typespecs.html

    Example :
    https://tomjoro.github.io/2017-02-09-ecto3-mongodb-phoenix/
  """
  use Plug.Router

  # This module is a Plug, that also implements it's own plug pipeline, below:

  # Using Plug.Logger for logging request information
  plug(Plug.Logger)

  # responsible for matching routes
  plug(:match)

  # Using Poison for JSON decoding
  # Note, order of plugs is important, by placing this _after_ the 'match' plug,
  # we will only parse the request AFTER there is a route match.
  plug(Plug.Parsers,
    parsers: [:json],
    pass: ["application/json"],
    json_decoder: Poison
  )

  # responsible for dispatching responses
  plug(:dispatch)

  # A simple route to test that the server is up
  # Note, all routes must return a connection as per the Plug spec.
  # Get all users
  get "/users" do
    conn
    |> put_resp_content_type("application/json")
    |> send_resp(200, ServiceUtils.endpoint_success(UserReader.find_all_users()))
  end

  # Get user by email
  post "/user-by-email" do
    case conn.body_params do
      %{"email" => email} ->
        conn
        |> put_resp_content_type("application/json")
        |> send_resp(
          200,
          ServiceUtils.endpoint_success(UserReader.user_by_email(email))
        )

      _ ->
        conn
        |> put_resp_content_type("application/json")
        |> send_resp(200, ServiceUtils.endpoint_error("missing_email"))
    end
  end

  # Get user by user name
  post "/user-by-user-name" do
    case conn.body_params do
      %{"username" => username} ->
        conn
        |> put_resp_content_type("application/json")
        |> send_resp(
          200,
          ServiceUtils.endpoint_success(UserReader.user_by_username(username))
        )

      _ ->
        conn
        |> put_resp_content_type("application/json")
        |> send_resp(200, ServiceUtils.endpoint_error("missing_username"))
    end
  end

  # Add user
  post "/add-user" do
    {status, body} =
      case conn.body_params do
        %{"user" => user_to_add} ->
          case UserWriter.add_user(user_to_add) do
            {:ok, user} ->
              {
                200,
                ServiceUtils.endpoint_success(user)
              }

            {:error, _changeset} ->
              {
                200,
                ServiceUtils.endpoint_error("exception")
              }
          end

        _ ->
          {
            200,
            ServiceUtils.endpoint_error("missing_prams")
          }
      end

    send_resp(conn, status, body)
  end

  # Delete user
  post "/delete-user" do
    {status, body} =
      case conn.body_params do
        %{"user" => user_to_delete} ->
          case UserWriter.delete_user(user_to_delete) do
            {:ok, user} ->
              {
                200,
                ServiceUtils.endpoint_success(user)
              }

            {:error, _changeset} ->
              {
                200,
                ServiceUtils.endpoint_error("exception")
              }
          end

        _ ->
          {
            200,
            ServiceUtils.endpoint_error("missing_prams")
          }
      end

    send_resp(conn, status, body)
  end

  # Update User By Email
  post "/update-user-email" do
    {status, body} =
      case conn.body_params do
        %{"user" => user_to_update} ->
          case UserWriter.update_user_by_email(user_to_update) do
            {:ok, user} ->
              {
                200,
                ServiceUtils.endpoint_success(user)
              }

            {:error, _changeset} ->
              {
                200,
                ServiceUtils.endpoint_error("exception")
              }
          end

        _ ->
          {
            200,
            ServiceUtils.endpoint_error("missing_prams")
          }
      end

    send_resp(conn, status, body)
  end

  # Update User By User Name
  post "/update-user-name" do
    {status, body} =
      case conn.body_params do
        %{"user" => user_to_update} ->
          case UserWriter.update_user_by_username(user_to_update) do
            {:ok, user} ->
              {
                200,
                ServiceUtils.endpoint_success(user)
              }

            {:error, _changeset} ->
              {
                200,
                ServiceUtils.endpoint_error("exception")
              }
          end

        _ ->
          {
            200,
            ServiceUtils.endpoint_error("missing_prams")
          }
      end

    send_resp(conn, status, body)
  end

  # A catchall route, 'match' will match no matter the request method,
  # so a response is always returned, even if there is no route to match.
  match _ do
    send_resp(conn, 404, ServiceUtils.endpoint_error("exception"))
  end
end

8. run the application

mix deps.get
mix compile
iex -S mix run
recompile # each time we modify project we should run recompile

Project endpoints

User List

User By email

Delete an existing user (delete_count = 1)

Delete a non existing user (delete_count = 0)

That’s all, we have create our Elixir Web Server without using Phoenix and using our Data Writer and Reader.

9. Code review & generate documentation

mix format
mix code_review # code review : credo & dialyxir
mix generate_docs # generate doc and doc coverage : ex_doc & inch_ex

Restfull API — doc

UserReader — doc

Credo & Dialyzer code review

More details

Mongodb

Building a JSON Rest API using Elixir, Phoenix and PostgreSQL

Building a JSON Rest API for managing an Online Bookstore using Elixir, Phoenix and PostgreSQL

Thank you for reading my story.

You can find me at :

Twitter : https://twitter.com/b_k_hela

Github : https://github.com/helabenkhalfallah