Passing a “Configuration” to a Clojure Application

Introduction

There are many ways to pass configuration values to a Clojure application. This piece will cover four of them:

1. Command Line Parameters
2. Environment Variables
3. Dynamic Environment Variables, and
4. Docker Secrets

The first two are briefly discussed, while greater time is spent on the final two. Of the four, the last is particularly useful to keep secure configuration values that ought to be kept so - passwords, private keys etc.

Using Command Line Parameters

If one starts Clojure from the command line using the -m option specifying a namespace, Clojure will execute the -main function from that namespace, passing any further arguments on the command line as parameters to -main.

For example, the following Clojure code

(ns main.core)

(defn -main [args]
  (println args))

which can be executed from the command line using

$ clj -m main.core "Hello World!"

will result in the string Hello World! being printed to the console.

Using Environment Variables

As an alternative to command line parameters, it’s often convenient to have your Clojure application read its parameters from the application’s execution environment i.e. environment variables or JVM system properties.

So running

$ export MYARGS="Hello World!"

at the command line, and changing the -main function to

(defn -main [& args]
  (println (System/getenv "MYARGS")))

you can now run the application using

$ clj -m main.core

and see the same result.

The value of the MYARGS environment variable is read from the environment and then printed to the console.

Unfortunately, as convenient as this is when executing the code, it can be a little inconvenient during development. If this is the only place you use the variable there’s little lost, but if the value is used in other areas of your application e.g. in other namespaces, any changes to its name or expected type will lead to an amount of error-prone “code surgery”.

Also, env variables are, by their nature, strings; so if you need the value as, for instance, an int you’ll need to perform the casting and error-checking at the time of initialization.

Using “Dynamic” Environment Variables

WalmartLabs have published a Clojure library on GitHub to address many of these issues. The library centralizes the reading of env variables, and also allows for existence-checking, the setting of default values, merging with JVM system properties, casting, type-checking, and composition.

The library makes it possible to define in a simple edn file the shape of your configuration data and have it parsed correctly from the environment (and other locations) into the structure you want.

As an example, if you have a file called config.edn somewhere on your classpath with

{:app-configuration
 {:myargs #dyn/prop MYARGS}}

and change the main/core.clj file to

(ns main.core
  (:require
   [clojure.edn :as edn]
   [clojure.java.io :as io]
   [com.walmartlabs.dyn-edn :refer [env-readers]]))

(def app-config
  (->> "config.edn"
     io/resource
     slurp
     (edn/read-string {:readers (env-readers)})))

(defn -main [& args]
  (println (get-in app-config [:app-configuration :myargs])))

and then run the application using

$ clj -m main.core

You’ll see the same result - but, the application’s configuration has been correctly (and automatically) parsed into a configuration structure and is available as a map named main.core/app-config that can be used throughout your application.

The use of the config.edn file also allows you to view the expected configuration parameters, or add to them, or change their default values in one central location - very convenient.

Using Docker Secrets - with Dynamic Environment Variables

An area where env variables are extensively used as configuration parameters is when an application is being run inside a docker container. By providing one or more -e options to the docker run command, it’s possible to establish the configuration environment for the application (if that’s where the application expects to find it).

Unfortunately, certain configuration parameters contain sensitive information, such as passwords or private keys and one can’t realistically embed those values in the application’s code. They may change frequently; they may need to differ from one container to another; and their very existence in the code represents a risk that they’ll “leak” into an SCM.

Of course, the use of environment variables is a good alternative to embedded code values, but represents a different, albeit smaller, set of risks. Anyone with access to the docker instance could recover the environment variables passed to a container during initialization.

In order to address this, Docker introduced the concept of secrets with docker swarm. Secrets allow sensitive information to be defined securely, and then selectively made available to containers which are running as docker services. It is only within the running container that the secret’s value is available as contents of files mounted from an in-memory filesystem, by default at /run/secrets/<secret_name>, where they can be accessed by the application.

In order to tie together environment variables with secrets, I’ve submitted a PR to the maintainer of the walmart-labs/dyn-edn library which, in addition to env variables and system properties, merges in docker secrets to the set of variable available to the library’s readers: #dyn/prop, #dyn/join, #dyn/long, #dyn/boolean, and #dyn/keyword.

Note The PR was accepted by the maintainer, but the library hasn’t yet made it to clojars. In order to use the secrets functionality you’ll need to add the following to your :deps entry in deps.edn. This will pull the appropriate version of the code.

com.walmartlabs/dyn-edn 
  {:git/url "https://github.com/walmartlabs/dyn-edn.git"
   :sha "855a775959cf1bec531a303a323e6f05f7b260fb"}

Our Example with Secrets

To use a docker secret in lieu of the MYARGS env variable used in previous examples all one needs to do is create a secret called MYARGS with the appropriate value

$ printf "Hello World!" | docker secret create MYARGS -

and, when starting the container as a docker service, authorize the service to use that secret

$ docker service create --replicas 1 --secret MYARGS --name <svcname> <image containing the app>

No change needs to be made to the config.edn file, or to the source code.