Bolth: a faster, more humane runner for clojure.test
This is a blog about the development of Yeller, The Exception Tracker with Answers
Clojure’s built in clojure.test
test framework is good enough, most of the time. It’s small, very well known, and ships with the language. But it isn’t very human friendly to actually use, day to day. Test failures, especially for larger data structures get pretty hard to read. And the stacktraces it outputs on an exception are very gnarly. Also, folk using it on larger projects often end up writing a bunch of supporting code to do things like diagnose slower test runs, or exit with the right exit code in CI when the tests fail. Lastly, it doesn’t provide any support for running tests in parallel.
Bolth
So, today, I’m releasing bolth
, an alternative test runner for clojure.test
which solves all of the above problems:
- it uses the same data diffing method that
humane-test-ouput
uses to provide better test failures - it uses
io.aviso.exception
for pretty printing exception stacktraces. Aviso is a huge improvement over the standard jvm exception formatting - stacktraces are printed in the right order, and have colors and such. - it includes a load of formatting and reporting options (see the docstring for more on that)
Here’s what it looks like in action, on yeller’s full test suite:
Note:
- The full run is very fast. This is partly through the parallelization provided with bolth, partly through careful design. Fast feedback cycles are still not quite understood as the big deal that they are: with fast feedback, you can experiment on code with tests just as fast as you do in the repl, and you end up with real regression suites as well.
- With my current setup, test run output sticks around in the repl (on the right of the screen). This means you’re not rerunning tests just to look at the failure message (or worse: scrolling up with your mouse/tmux to read things).
- The whole repl pane is “cleared” (by printing a load of blank newlines) before a run (this is optional with bolth, but just an extra option to the runner). This stops the eyes getting distracted by previous test output.
Now let’s see what a failing run looks like:
And with an exception:
Another minor helper bolth
provides wraps clojure.tools.namespace.repl/refresh
to pretty print compilation exceptions. Here’s what that looks like:
Much nicer.
Showing slow tests
Bolth provides a lot of formatting and reporting options. By far and away my favorite though, is the reporting of slow tests it can do:
This just requires adding a single option :show-slow-tests true
to the runner. Typically test suite runtimes are completely dominated by a few slow tests. Yeller’s suite (before this option) took about 3 seconds, with 2.5 seconds of that time entirely contributed by 4 tests.
Fin
So, that’s it. The code’s up on github, the jar is on clojars. It should be relatively trivial to convert any existing clojure.test
workflow to bolth.
This is a blog about the development of Yeller, the Exception Tracker with Answers.