Deploy a Clojure Pedestal API Server & React/ClojureScript Web Application to Docker

Introduction

In this post I’ll show how to build and deploy to docker a fully-functioning application comprising a secure Pedestal API web-server and a React front-end application written in ClojureScript which accesses the server.

For this example I’ll be using the Pedestal/React application I previously discussed in this blog post (with its code here for tag v1.0), and the code discussed in this post is available here.

Outline of the Steps

I’ll set up a working directory for the build, clone the target application into a sub-directory, compile the target, package it and its dependencies to java byte code, assemble them into a jar, create a docker image for the application, and deploy it as a docker service.

Background, Challenges & Tools

During this exercise, I’ll try to keep separate the application I’m building from the application doing the building. This isn’t strictly necessary but it will help illustrate a procedure generally applicable to any Clojure application.

One of the challenges of this approach is that I’ll need to deal with two deps.edn files - one for the build environment and one for the application being built. Each deps file contains relative paths (:paths and :extra-paths) relative to the root of the project directory to which it belongs.

If I am to avoid changing in any way the deps file for the project being built and yet still ensure that the built artifacts end up in the correct location within the build project’s directory structure I will need a way to inform the compiler about which paths to use, but relative to the build project’s directory and not as specified in the target’s deps.edn file.

As an example, let’s suppose that the build project is at ./clj-deploy-docker and the project being built will be cloned into ./clj-deploy-docker/target-app.

The deps.edn file in ./clj-deploy-docker/target-app will contain an entry for the alias :main as below

:aliases
 {:main
  {:paths ["src"]
   :extra-deps {ch.qos.logback/logback-classic {:mvn/version "1.2.3"}
                org.clojure/tools.logging {:mvn/version "0.4.1"}
                ring/ring-core {:mvn/version "1.8.0"}
                ring/ring-jetty-adapter {:mvn/version "1.8.0"}
                ring/ring-devel {:mvn/version "1.8.0"}
                io.pedestal/pedestal.service {:mvn/version "0.5.7"}
                io.pedestal/pedestal.route {:mvn/version "0.5.7"}
                io.pedestal/pedestal.jetty {:mvn/version "0.5.7"}
                buddy {:mvn/version "2.0.0"}
                hiccup {:mvn/version "1.0.5"}
                org.conscrypt/conscrypt-openjdk-uber {:mvn/version "2.2.1"}
                org.eclipse.jetty/jetty-alpn-conscrypt-server {:mvn/version "9.4.24.v20191120"}
                com.google.api-client/google-api-client {:mvn/version "1.30.6"}
                com.walmartlabs/dyn-edn 
                {:git/url "https://github.com/walmartlabs/dyn-edn.git" 
                 :sha "855a775959cf1bec531a303a323e6f05f7b260fb"}}
   :extra-paths ["resources" "common-src" ]}

In order to access and use this alias correctly from the build project’s directory (clj-deploy-docker) I will need to adjust (in some way) the paths so that the compiler is operating with the correct class path i.e. the class path of the target rather than the class path of the build. Therefore, I’ll need to let the compiler know (in some way) that the :paths and :extra-paths vectors should read

:paths ["target-app/src"]

and

:extra-paths ["target-app/resources" "target-app/common-src" ]

respectively.

On the other hand, the maven coordinates in the target’s deps.edn file are correct, so we can leave the :deps and :extra-deps values as they are found.

As for the “in some way”, I will be using the Badigeon library to achieve this. Many of Badigeon’s API’s take a :deps-map as input. This is an in-memory map whose structure is the same as a deps.edn file. This will allow me to read the deps file, make in-memory adjustments and feed it to to API to do the bundling and compiling with a classpath relative to any directory I choose (i.e. relative to ./clj-deploy-docker).

Create a Working Directory for the Project

Create a working directory for the project, and cd into it

$ mkdir clj-deploy-docker
$ cd clj-deploy-docker

Setting up the Application to be built

Now, I’ll clone the repository of the application I want to build into a directory target-app under my working directory.

$ git clone https://github.com/heykieran/clj-pedestal-google.git target-app

As mentioned above, for this exercise I will be using a library called Badigeon to organize and compile the sources. It leverages many of the tools & libraries already available in clojure.core and tools.deps; it’s very flexible and I find the API intuitive.

Creating the build runner

In my project’s working directory I create a deps.edn file.

$ touch deps.edn

and add the Badigeon dependency to the deps.edn file.

{:deps 
  {}
  :aliases
  {:build
  {:extra-paths ["build"]
   :extra-deps
   {badigeon/badigeon
    {:git/url "https://github.com/EwenG/badigeon.git"
     :sha "1edf7ae465db870ec0066f28226edb9b04873b70"
     :tag "0.0.11"}}}}}

Apart from the Clojure system and user dependencies this is the only dependency I’ll need in that file.

Also, for later use, I create a directory called build to contain the Clojure files to run the bundling, compilation and assembling processes.

$ mkdir build

Building the Front-End JS File

As outlined in my previous blog post, the following command will build the front-end production application’s js file from the ClojureScript sources for the application being dockerized.

$ cd target-app
$ clj -A:prod
$ cd ..

This will build the application’s front-end only and place the production js file (prod-main.js) in the target-app/target/public/cljs-out directory. This is the only change that will be made to the directories and files under target-app.

At a later stage I will move this file to its correct location under my project directory (./clj-deploy-docker) so that it can be included in the docker image.

Building the Back-End (JVM) Class Files

I’ll now cd into the build directory I created previously and create a package.clj file. This file will contain the -main method that ultimately performs the bundling, compilation and consolidation of the back-end Clojure files i.e. the JVM class files.

Some Background on Bundling, Compilation and Consolidation (Jar‘ing)

There are three distinct phases to assembling the JVM artifacts to include in the docker image and I will be using the Badigeon API to perform all three phases.

1. Bundling

   The bundling step creates a “bundle” at a specified file-system location of all the target project’s resources and dependencies, including any jar files that are needed. Note that because the Badigeon bundler does not merge in the system and user deps preferences, it will not automatically copy sources that are in src directory of your project, unless that directory is explicitly specified in the :paths or :extra-paths entries in the deps.edn file. During the bundling phase all the jar files required by your application, and all other resources on the classpath such as static html file, css files, user authored js files etc. will be copied to the specified target directory.

2. Compilation

   During the compilation step the compiled versions of your Clojure source files (as .class files) are generated and copied to a specified target directory.

3. Consolidation

   The final phase involves creating a jar file containing all the .class files needed by the application with an appropriate manifest file (META-INF/MANIFEST.MF) which has an entry indicating the application’s entry-point (a Main-Class entry), and an entry specifying the libraries to be used by the jar file (a Class-Path entry).

   Dependencies (found by Badigeon using the :deps and :extra-deps coordinates) will not be incorporated into this jar file. They will however be added to a ./lib directory as individual jar files and referenced by the Class-Path entry in the jar’s manifest file.

Once these three phases are complete, and the js file containing the front-end application is placed in its correct location, the application can be run using the java command line tool.

I’ll come to that presently, but first I’d like to take a slightly deeper look at the bundling, compilation and consolidation phases. The full details are available in the package.clj file from which the following code snippets have been extracted.

Notes on the code performing the three steps

First, I bundle

(bundle
     out-path
     {:deps-map translated-deps-map
      :aliases aliases
      :libs-path "lib"})

Given a deps-map and a vector of aliases ([:main]) this function will copy the projects’s resources needed to out-path, and also copy the jar files required to out-path/lib. Because the code I want to bundle is in the target-app directory, I’ll read the deps.edn file from its location under target-app and update the :path and :extra-paths entries so that they are now relative to the current working directory rather than target-app (see above).

Now the compilation phase runs:

(compile/compile
     'main.core
     {:compile-path
      classes-path
      :classpath
      (translate-path-to-absolute
       target-dir
       deps-map
       aliases)})

This compiles the main.core namespace, putting the .class files into the directory specified by the classes-path directory, using a classpath specified by the value of the :classpath entry. In my case, this is generated by reading the target-app/deps.edn file into deps-map and converting the relative components of :paths and :extra-paths vectors to absolute file-system locations.

Finally, the consolidation phase runs:

(spit manifest-path
          (jar/make-manifest
           'main.core
           {:Class-Path
            (str
             ". "
             (str/join
              " "
              (mapv
               #(str "lib/" (.getName %))
               (.listFiles (io/file "target/app/lib")))))}))
    
    (zip/zip
     classes-path
     (str (make-path out-path "app-runner") ".jar"))

This achieves two things:

1. It creates a manifest file in target/app/classes/META-INF, setting main.core as the entry point, and adds entries for all the jar files in the target/lib directory (which was created and populated during bundling) into the manifest file’s Class-Path header field. In order for the application to run there is an assumption that the final jar file and the lib directory will exist at the same level in the file system i.e. in the same directory.

2. It creates a app-runner.jar file from the contents of the target/app/classes directory. This jar file is the main application and will contain the manifest file just created with its Class-Path entry pointing to the non-application jar files it needs to run - i.e. those found in the lib directory.

In order to run all three steps, I can “execute” the package namespace passing the target-app directory name as an argument.

$ clj -A:build -m package "target-app"

This completes the Bundling, Compilation and Consolidation steps, and when it finishes I will have a directory structure, which with the addition of the front-end js file (which I compiled above) will constitute the complete application.

The result is a target folder containing an app-runner.jar file and any other supporting files needed to run the app. Many are extraneous; for instance all the classes files, now included in the jar file are also under this directory, as are the source code of the Clojure files.

I can copy the js file to its correct location using

$ mkdir -p target/app/public/cljs-out && \ 
  cp target-app/target/public/cljs-out/prod-main.js "$_"

Now, everything I need (and some I don’t) is available in the ./target/app/ directory.

Running the Application

Before running the compiled application I need to ensure that certain environment variables are defined and set correctly.

As discussed in my previous post, the application requires a number of environment variables to be set in order to configure itself correctly.

These are

# the https port number used by the Pedestal API server
ALLOC_SSL_PORT=8081 
# the password of Jetty's keystore 
ALLOC_KEYSTORE_PASSWORD=<password> 
# the http port number used by the Pedestal API server
ALLOC_PORT=8080 
# the file system location of the Jetty's keystore (as an absolute file path)
ALLOC_KEYSTORE_LOCATION=<location> 

I can cd into the built artifact’s directory (./target/app) and run the backend application directly from the jar file

$ cd target/app
$ java -jar app-runner.jar

or, because the classes still exist in a classes directory under the app directory

$ cd target/app
$ java -cp .:classes:lib/* main.core

The app will start, and when it’s fully initialized, I can navigate to https://localhost:8081/r/home to see it in action.

There is also a lot of unnecessary “residue” in the ./target/app directory, created during bundling, including directories containing clj and cljc files that are not actually needed to run the application (they will already have been compiled into the classes directory).

When I process the files for deployment to docker, these will be removed.

There remains then only the task of creating the docker image itself, which is outlined below.

Quick Review

Currently, we have in the target/app all the artifacts (with some extras) to run the application. Now we will rationalize those artifacts, removing all the unnecessary ones, leaving only those that are necessary for running our application and package what remains into a docker image, which we’ll place in the folder docker/deploy.

Create the Docker Image

The docker image I will use is very simple - a basic Debian stretch image with a Java8 SDK.

In my project directory I create a directory called docker and cd into it.

$ mkdir docker
$ cd docker 

and create a Dockerfile containing

FROM openjdk:8-stretch

COPY entrypoint.sh /sbin/entrypoint.sh
RUN chmod 755 /sbin/entrypoint.sh

EXPOSE 8081/tcp

COPY deploy /image

WORKDIR /image/app

ENV ALLOC_KEYSTORE_LOCATION=/image/local/jetty-keystore \
    ALLOC_KEYSTORE_PASSWORD=password \
    ALLOC_PORT=8080 \
    ALLOC_SSL_PORT=8081 

ENTRYPOINT ["/sbin/entrypoint.sh"]

The Dockerfile as defined will

• create an image from a base openjdk:8-stretch image,
• copy the file entrypoint.sh (which I haven’t created yet) from the docker/deploy folder to the image’s /sbin directory and set its mode to executable,
• enable network connectivity to port 8081 only (there will be no access to the unprotected http port 8080),
• copy the entire contents of the docker/deploy directory to the image’s /image directory,
• set the image’s working directory to /image/app,
• set the needed environment variables for the new image, and
• specify that the /sbin/entrypoint.sh script should be run when the container starts.

The deploy directory under the docker directory is the location where the application’s artifact will be assembled before their inclusion in the image when the COPY deploy /image command is run.

From my project’s folder (clj-deploy-docker) I run the following command to copy the entire app (including residue) to the docker/deploy folder (creating it if it doesn’t exist).

$ mkdir -p docker/deploy/app && cp -r target/app/* "$_"

Now, in order to remove the extraneous files I can run

$ 	find docker/deploy/app -maxdepth 1 -mindepth 1 -type d \( ! \( -name 'lib' -o -name 'public' \) \) -exec rm -rf {} \;

This deletes any sub-directory in the docker/deploy folder not named lib (which contain the jar files the application needs) or public (which contains all the non-JVM resources the application needs).

I now add the entrypoint.sh file to the docker folder. This script, which is run when the image is started, simply calls the application’s entry-point.

#!/bin/bash
# exit immediately if error
set -e

java -jar app-runner.jar

Finally, I need to ensure that the keystore used to encrypt the application’s https communication is available for the image build process, so I copy it from my local file system’s location to the /docker/deploy/local directory, from whence, during the image building process, it will be copied to the image’s /image/local folder.

# mkdir -p docker/deploy/local && cp <location-of-keystore> "$_"

Now, with everything cleaned-up and with the script and the keystore in place, I can create the application’s docker image, tagging it with the label testapp:dev.

$ docker build -t testapp:dev docker

and then run a container based on that image using

$ docker run --rm --name test 
    --env ALLOC_KEYSTORE_PASSWORD=<the-real-keystore-password> 
    -p:8081:8081 
    -it testapp:dev

I can then open my browser to https://localhost:8081/r/home in order to confirm it’s running correctly.

Note When I created the image I did not include the correct password for the keystore in the Dockerfile. Therefore, in order for the application to work correctly I’m required to pass the correct value by setting the env variable ALLOC_KEYSTORE_PASSWORD when I start the container. It will be used in lieu of the value embedded in the image.

Docker Secrets

Passing sensitive information using environment variables is satisfactory in many situations, but there is available a better approach: docker secrets.

Note Docker secrets are not available in stand-alone mode, the feature is only available in swarm mode.

Passing Configuration Values

For further details on the subject of passing configurations to a Clojure application you can refer to my blog post on the subject. The post also discusses more fully the mechanics of how the configuration is used by the application’s code.

Create a swarm

To create a local swarm for testing I can issue the following command

$ docker swarm init

Once the swarm has been initialized I can add a secret to the registry. Let’s suppose I want to protect the ALLOC_KEYSTORE_PASSWORD and avoid having to pass it to the image as an environment variable. I can simply create a docker secret to hold the value, protecting it from being stolen too easily. The following command will create a secret called ALLOC_KEYSTORE_PASSWORD, set its value to MYKEYSTOREPASSWORD and store it in the swarm’s registry.

$ printf "MYKEYSTOREPASSWORD" | docker secret create ALLOC_KEYSTORE_PASSWORD -

You can test that the secret was successfully created by issuing

$ docker secret ls

In order to use the secret, the container has to be started as a service within the swarm, and on the command line must be specified to what secrets the service has access. In order to start the container with access to the ALLOC_KEYSTORE_PASSWORD and linking swarm’s network to the host’s network I can issue the following command

$ docker service create --replicas 1 \
    --secret ALLOC_KEYSTORE_PASSWORD \ 
    --name testapp \
    --publish mode=host,published=8081,target=8081 \
    testapp:dev

This will start the service (named testapp) within the swarm, and the service will start serving the application similarly to when I used the docker run command above.

Once started the following command will return basic information about the service

$ docker service ls

and this information should look something like the following

ID                  NAME                MODE                REPLICAS            IMAGE               PORTS
kjzh1uc2ttng        testapp             replicated          1/1                 testapp:dev         

If I want to monitor the activity of the service I should monitor the logs of its associated container and, in order to do this I need to know the container’s ID.

I can issue the following command and note the value in the CONTAINER_ID column for the image testapp:dev and use it to interrogate the logs.

$ docker container ls

This will return something like the following:

CONTAINER ID        IMAGE               COMMAND                 CREATED             STATUS              PORTS                    NAMES
d5b269d508b4        testapp:dev         "/sbin/entrypoint.sh"   15 seconds ago      Up 14 seconds       0.0.0.0:8081->8081/tcp   testapp.1.tb4n70oe1t8tz3qdzo2aawmek

And I can view the logs of the running container using as much of the container’s ID as necessary to make it unique

$ docker logs d5b

This allows me to confirm that the application started correctly and is responding to requests.

As before, I can point my browser at https://localhost:8081/r/home and exercise the packaged application running as a docker service.

After some activity I can review the history of my interactions (and the server’s responses) by once again reviewing the logs:

$ docker logs d5

To shutdown the service, I run

$ docker service rm testapp

Review

There were quite a number of steps but I hope the detail was illuminative.

A later post will illustrate how to integrate the build process within the Clojure application directory structure rather than requiring that it be cloned into a separate working directory.

That post will also show how the build and deployment steps can be automated using a simple Makefile.