How to Do Production Schema Migrations with Datomic

This is a blog about the development of Yeller, The Exception Tracker with Answers

Read more about Yeller here

Yeller’s been using Datomic in production for quite a while now. One thing that’s super important to figure out with any database is how to do production schema migration. There are quite a lot of libraries out there, some stuff written up in training material (but not released) etc, so I thought I’d make a concrete recommendation:

  1. Use Conformity to do your schema migrations
  2. Load schema at application bootup

Use Conformity

The first tip is that you should use Conformity to manage installing your schema.

Conformity’s a very small library, but it gets you the most important thing for schema migrations in production:

schema changes should be applied once, and only once

Conformity ensures schema changes are applied once and only once. You have to name each schema change with a specific name (hint from active record here: use the datetime the migration was created at as it’s unique name), and then conformity ensures that it transacts that migration only once (it records “I have completed migration XXX in the database itself).

An Example

I keep datomic schemas in code rather than edn files. Here’s a sample schema:

(def users
  [{:db/ident :user/name
    :db/valueType :db.type/string
    :db/cardinality :db.cardinality/one}

   {:db/ident :user/email
    :db/valueType :db.type/string
    :db/cardinality :db.cardinality/one}

   {:db/ident :user/encrypted-password
    :db/valueType :db.type/string
    :db/cardinality :db.cardinality/one}])

Just a simple set of user attributes.

And here’s the migration code to run it:

(def schema
  {:yeller/migration-2014-04-25-22-25-32-UTC
   users})

(defn load-schema [connection]
  (conformity/ensure-conforms
   connection
   schema))

Nice and simple, and free of race conditions even if you run it multiple times etc.

Load Schema at Application Startup

Since database schema changes are idempotent with Conformity, you can apply them whenever you like. To save running multiple processes and complicating deploys, you can simply run them when the web application boots up.

Using conformity and loading schema at boot works very well in development as well as production. It protects you from a bunch of race conditions and from transacting the same migration twice. Yeller’s been using it in production since early 2014, and it’s been great.

This is a blog about the development of Yeller, the Exception Tracker with Answers.

Read more about Yeller here

Looking for more about running production applications, debugging, Clojure development and distributed systems? Subscribe to our newsletter: