Hello, Clerk ๐Ÿ‘‹

Clerk enables a rich, local-first notebook experience using standard Clojure namespaces and Markdown files with Clojure code fences. You bring your own editor and workflow, your own interactive computing habits, and Clerk enhances all of that with literate programming and rich visualizations.

Inside clj files, comment blocks are interpreted as prose written in an extended dialect of Markdown. Clerk supports inline TeX, so we can insert the Eulerโ€“Lagrange equation quite easily:

ddtโˆ‚Lโˆ‚qห™โˆ’โˆ‚Lโˆ‚q=0.{\frac{d}{d t} \frac{โˆ‚ L}{โˆ‚ \dot{q}}}-\frac{โˆ‚ L}{โˆ‚ q}=0.

When Clerk interprets an md file, the relationship between code blocks and prose is reversed. Instead of the file being code by default with prose in comment blocks, it will be treated as Markdown by default with Clojure code in code fences. Clerk's code fences have a twist, though: they evaluate their contents.

There are loads of other goodies to share, most of which we'll see a bit farther down the page.

Operation

You can load, evaluate, and present a file with the clerk/show! function, but in most cases it's easier to start a file watcher with something like:

(clerk/serve! {:watch-paths ["notebooks" "src"]})

... which will automatically reload and re-eval any clj or md files that change, displaying the most recently changed one in your browser.

To make this performant enough to feel good, Clerk caches the computations it performs while evaluating each file. Likewise, to make sure it doesn't send too much data to the browser at once, Clerk paginates data structures within an interactive viewer.

Pagination

As an example, the infinite sequence returned by (range) will be loaded a little bit at a time as you click on the results. (Note the little underscore under the first paren, it lets you switch this sequence to a vertical rather than horizontal view).

(range)
(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 1000000+ more elided)

Opaque objects are printed as they would be in the Clojure REPL, like so:

(def notebooks
(clojure.java.io/file "notebooks"))
#object[java.io.File 0x6631b917 "
notebooks"
]

You can leave a form at the top-level like this to examine the result of evaluating it, though you'd probably use your live programming environment to do this most of the time.

(into #{} (map str) (file-seq notebooks))
#{"
notebooks"
"
notebooks/controls.clj"
"
notebooks/data_science.clj"
"
notebooks/dictionary.clj"
"
notebooks/elements.clj"
"
notebooks/git.clj"
"
notebooks/images.clj"
"
notebooks/index.md"
"
notebooks/introduction.clj"
"
notebooks/logo.clj"
"
notebooks/markdown.md"
"
notebooks/perceptron.clj"
"
notebooks/rule_30.clj"
"
notebooks/semantic.clj"
"
notebooks/sicmutils.clj"
"
notebooks/slideshow.md"
"
notebooks/src"
"
notebooks/src/demo"
"
notebooks/src/demo/lib.cljc"
"
notebooks/static_site_generation.md"
1 more elided}

Sometimes you don't want Clerk to cache a form. You can turn off caching for a form by placing a special piece of metadata before it, like this:

^:nextjournal.clerk/no-cache (shuffle (range 100))
[9 11 95 59 93 37 75 79 38 58 65 31 80 77 70 73 66 40 48 51 80 more elided]

Another useful technique is to put an instant marking the last time a form was run. This way you can update this result at any time by updating the instant.

(let [last-run #inst "2021-12-01T16:40:56.048896Z"]
(shuffle (range 100)))
[52 98 54 65 26 16 49 67 71 76 60 93 43 18 57 14 64 80 46 79 80 more elided]

Like other objects, UUIDs and insts are rendered as they would be in the REPL.

(take 10
(repeatedly (fn []
{:name (str
(rand-nth ["Oscar" "Karen" "Vlad" "Rebecca" "Conrad"]) " "
(rand-nth ["Miller" "Stasฤnyk" "Ronin" "Meyer" "Black"]))
:role (rand-nth [:admin :operator :manager :programmer :designer])
:id (java.util.UUID/randomUUID)
:created-at #inst "2021"})))
({:created-at #inst "
2021-01-01T00:00:00.000-00:00"
:id #uuid "
5e195633-bd0e-4e1c-990c-c92a47f0fe18"
:name "
Vlad Miller"
:role :manager}
{:created-at #inst "
2021-01-01T00:00:00.000-00:00"
:id #uuid "
272d8a3e-37f7-400d-8f3b-4f64617c916c"
:name "
Oscar Meyer"
:role :admin}
{:created-at #inst "
2021-01-01T00:00:00.000-00:00"
:id #uuid "
34fe5ede-ef29-4a66-9a78-a6c68b8d91ee"
:name "
Oscar Meyer"
:role :operator}
{:created-at #inst "
2021-01-01T00:00:00.000-00:00"
:id #uuid "
c250534c-6ac4-4cd7-b4bb-7bc80c2a5e45"
:name "
Oscar Stasฤnyk"
:role :programmer}
{:created-at #inst "
2021-01-01T00:00:00.000-00:00"
:id #uuid "
a22cfbc5-8001-43eb-989d-fd1a656a7bec"
:name "
Conrad Miller"
:role :designer}
{:created-at #inst "
2021-01-01T00:00:00.000-00:00"
:id #uuid "
7eefe6bc-49bd-40d6-9a2d-4b59661d588c"
:name "
Conrad Ronin"
:role :manager}
{:created-at #inst "
2021-01-01T00:00:00.000-00:00"
:id #uuid "
8c1eb34a-f436-4d32-9ba9-5ea1aa90f97c"
:name "
Vlad Meyer"
:role :admin}
{:created-at #inst "
2021-01-01T00:00:00.000-00:00"
:id #uuid "
52b4f53c-44dd-4083-a74d-84d88cc5a89e"
:name "
Rebecca Meyer"
:role :admin}
{:created-at #inst "
2021-01-01T00:00:00.000-00:00"
:id #uuid "
c8b2aa60-8d62-4c09-90a8-2d919f6bd5f3"
:name "
Rebecca Meyer"
:role :operator}
{:created-at #inst "
2021-01-01T00:00:00.000-00:00"
:id #uuid "
87e2d3bb-b4dc-4479-8de0-96f39fa86a03"
:name "
Karen Miller"
:role :designer})

Clerk also supports unicode, of course.

{:hello "๐Ÿ‘‹ world" :tacos (map (comp #(map (constantly '๐ŸŒฎ) %) range) (range 1 100))}
{:hello "
๐Ÿ‘‹ world"
:tacos ((๐ŸŒฎ) (๐ŸŒฎ ๐ŸŒฎ) (๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ) (๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ) (๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ) (๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ) (๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ) (๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ) (๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ) (๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ) (๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ) (๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ) (๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ) (๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ) (๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ) (๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ) (๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ) (๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ) (๐ŸŒฎ ๐ŸŒฎ ๐ŸŒฎ 16 more elided) (20 more elided) 79 more elided)}

๐Ÿ‘ Clerk Viewer API

In addition to these basic viewers for Clojure data structures, Clerk comes with a set of built-in viewers for many kinds of things, and a moldable viewer API that can be extended while you work.

๐Ÿงฉ Built-in Viewers

๐Ÿ”ข Data Tables

Clerk provides a built-in data table viewer that supports the three most common tabular data shapes out of the box: a sequence of maps, where each map's keys are column names; a seq of seq, which is just a grid of values with an optional header; a map of seqs, in with keys are column names and rows are the values for that column.

(clerk/table
(csv/read-csv "https://gist.githubusercontent.com/netj/8836201/raw/6f9306ad21398ea43cba4f7d537619d0e07d5ae3/iris.csv"))
:sepal.length
:sepal.width
:petal.length
:petal.width
:variety
5.13.51.4.2Setosa
4.931.4.2Setosa
4.73.21.3.2Setosa
4.63.11.5.2Setosa
53.61.4.2Setosa
5.43.91.7.4Setosa
4.63.41.4.3Setosa
53.41.5.2Setosa
4.42.91.4.2Setosa
4.93.11.5.1Setosa
5.43.71.5.2Setosa
4.83.41.6.2Setosa
4.831.4.1Setosa
4.331.1.1Setosa
5.841.2.2Setosa
5.74.41.5.4Setosa
5.43.91.3.4Setosa
5.13.51.4.3Setosa
5.73.81.7.3Setosa
5.13.81.5.3Setosa
130 more elided

๐Ÿ“Š Plotly

Clerk also has built-in support for Plotly's low-ceremony plotting:

(clerk/plotly {:data [{:z [[1 2 3] [3 2 1]] :type "surface"}]})
Loading...

๐Ÿ“ˆ Vega Lite

But Clerk also has Vega Lite for those who prefer that grammar.

(clerk/vl {:width 650
:height 400
:data {:url "https://vega.github.io/vega-datasets/data/us-10m.json"
:format {:type "topojson" :feature "counties"}}
:transform [{:lookup "id"
:from {:data {:url "https://vega.github.io/vega-datasets/data/unemployment.tsv"}
:key "id"
:fields ["rate"]}}]
:projection {:type "albersUsa"}
:mark "geoshape"
:encoding {:color {:field "rate"
:type "quantitative"}}})
Loading...

๐Ÿ“‘ Markdown

The same Markdown support Clerk uses for comment blocks is also available programmatically:

(clerk/md (clojure.string/join "\n" (map #(str %1 ". " %2) (range 1 4) ["Lambda" "Eval" "Apply"])))
  1. Lambda
  2. Eval
  3. Apply

๐Ÿค– Code

There's a code viewer uses that clojure-mode for syntax highlighting.

(clerk/code (macroexpand '(when test
expression-1
expression-2)))
(if test (do expression-1 expression-2))

๐Ÿงฎ TeX

As we've already seen, all comment blocks can contain TeX (we use KaTeX under the covers). In addition, you can call the TeX viewer programmatically. Here, for example, are Maxwell's equations in differential form:

(clerk/tex "
\\begin{alignedat}{2}
\\nabla\\cdot\\vec{E} = \\frac{\\rho}{\\varepsilon_0} & \\qquad \\text{Gauss' Law} \\\\
\\nabla\\cdot\\vec{B} = 0 & \\qquad \\text{Gauss' Law ($\\vec{B}$ Fields)} \\\\
\\nabla\\times\\vec{E} = -\\frac{\\partial \\vec{B}}{\\partial t} & \\qquad \\text{Faraday's Law} \\\\
\\nabla\\times\\vec{B} = \\mu_0\\vec{J}+\\mu_0\\varepsilon_0\\frac{\\partial\\vec{E}}{\\partial t} & \\qquad \\text{Ampere's Law}
\\end{alignedat}
")
โˆ‡โ‹…Eโƒ—=ฯฮต0Gaussโ€™ Lawโˆ‡โ‹…Bโƒ—=0Gaussโ€™ Law (Bโƒ— Fields)โˆ‡ร—Eโƒ—=โˆ’โˆ‚Bโƒ—โˆ‚tFaradayโ€™s Lawโˆ‡ร—Bโƒ—=ฮผ0Jโƒ—+ฮผ0ฮต0โˆ‚Eโƒ—โˆ‚tAmpereโ€™s Law \begin{alignedat}{2} \nabla\cdot\vec{E} = \frac{\rho}{\varepsilon_0} & \qquad \text{Gauss' Law} \\ \nabla\cdot\vec{B} = 0 & \qquad \text{Gauss' Law ($\vec{B}$ Fields)} \\ \nabla\times\vec{E} = -\frac{\partial \vec{B}}{\partial t} & \qquad \text{Faraday's Law} \\ \nabla\times\vec{B} = \mu_0\vec{J}+\mu_0\varepsilon_0\frac{\partial\vec{E}}{\partial t} & \qquad \text{Ampere's Law} \end{alignedat}

๐Ÿ•ธ Hiccup

The html viewer interprets hiccup when passed a vector. (This can be quite useful for building arbitrary layouts in your notebooks.)

(clerk/html [:table
[:tr [:td "โ—ค"] [:td "โ—ฅ"]]
[:tr [:td "โ—‰"] [:td "โ—‰"]]
[:tr [:td "โ—ฃ"] [:td "โ—ข"]]])
โ—คโ—ฅ
โ—‰โ—‰
โ—ฃโ—ข

Alternatively you can also just pass an HTML string, perhaps generated by your code:

(clerk/html "โ€œA brilliant solution to the wrong problem can be worse than no solution at all. Solve the correct problem.โ€<br/>โ€”<em>Donald Norman</em>")
โ€œA brilliant solution to the wrong problem can be worse than no solution at all. Solve the correct problem.โ€
โ€”Donald Norman

๐Ÿš€ Extensibility

In addition to these defaults, you can also attach a custom viewer to any form. Here we make our own little viewer to greet James Clerk Maxwell:

(clerk/with-viewer '(fn [name] [:div "Greetings to " [:strong name] "!"])
"James Clerk Maxwell")
Greetings to James Clerk Maxwell!

But we can do more interesting things, like using a predicate function to match numbers and turn them into headings, or converting string into paragraphs.

(clerk/with-viewers (clerk/add-viewers [{:pred number?
:render-fn '(fn [n] [(keyword (str "h" n)) (str "Heading " n)])}
{:pred string?
:render-fn '(fn [s] [:p s])}])
[1 "To begin at the beginning:"
2 "It is Spring, moonless night in the small town, starless and bible-black,"
3 "the cobblestreets silent and the hunched,"
4 "courters'-and- rabbits' wood limping invisible"
5 "down to the sloeblack, slow, black, crowblack, fishingboat-bobbing sea."])
[

Heading 1

To begin at the beginning:

Heading 2

It is Spring, moonless night in the small town, starless and bible-black,

Heading 3

the cobblestreets silent and the hunched,

Heading 4

courters'-and- rabbits' wood limping invisible

Heading 5

down to the sloeblack, slow, black, crowblack, fishingboat-bobbing sea.

]

Or you could use black and white squares to render numbers:

^::clerk/no-cache
(clerk/with-viewers (clerk/add-viewers [{:pred number?
:render-fn '(fn [n] [:div.inline-block {:style {:width 16 :height 16}
:class (if (pos? n) "bg-black" "bg-white border-solid border-2 border-black")}])}])
(take 10 (repeatedly #(rand-int 2))))
(
)

Or build your own colour parser and then use it to generate swatches:

(clerk/with-viewers (clerk/add-viewers
[{:pred #(and (string? %)
(re-matches
(re-pattern
(str "(?i)"
"(#(?:[0-9a-f]{2}){2,4}|(#[0-9a-f]{3})|"
"(rgb|hsl)a?\\((-?\\d+%?[,\\s]+){2,3}\\s*[\\d\\.]+%?\\))")) %))
:render-fn '(fn [color-str]
[:div.inline-block.rounded-sm.shadow
{:style {:width 16
:height 16
:border "1px solid rgba(0,0,0,.2)"
:background-color color-str}}])}])
["#571845"
"rgb(144,12,62)"
"rgba(199,0,57,1.0)"
"hsl(11,100%,60%)"
"hsla(46, 97%, 48%, 1.000)"])
[
]

Keep in mind when writing your own :render-fn that it will run entirely in the browser, and so will not have access to your local bindings on the JVM side. If you need to your viewer to pre-process what it sends to the browser, you can specify a :transform-fn that will be called before the data is sent over the wire.

๐Ÿž Customizing Data Fetching

Sometimes you might want to create a custom viewer that overrides Clerk's automatic paging behavior. In this example, we use a custom transform-fn that specifies a content-type to tell Clerk to serve arbitrary byte arrays as PNG images.

Notice that the image is conveyed out-of-band using the url-for function to get a URL from which to fetch the blob.

(clerk/add-viewers! [{:pred bytes?
:transform-fn (fn [{bytes :nextjournal/value}]
{:nextjournal/presented? true
:nextjournal/content-type "image/png"
:nextjournal/value bytes})
:render-fn '(fn [blob] [:img {:src (v/url-for blob)}])}])
[{:pred #object[clojure.core$bytes_QMARK_ 0x2406cb49 "
clojure.core$bytes_QMARK_@2406cb49"
]
:render-fn (fn [blob] [:img {:src (v/url-for blob)}]) :transform-fn #object[introduction$eval55980$fn__55982 0x79617715 "
introduction$eval55980$fn__55982@79617715"
]
}]
(.. (HttpClient/newHttpClient)
(send (.build (HttpRequest/newBuilder (URI. "https://upload.wikimedia.org/wikipedia/commons/5/57/James_Clerk_Maxwell.png")))
(HttpResponse$BodyHandlers/ofByteArray)) body)

This is just a taste of what's possible using Clerk. Take a look in the notebooks directory to see a collection of worked examples in different domains.

And don't forget to let us know how it goes!