ToC
Generated with Clerk from book.clj@70d0459

📓 Book of Clerk

⚖️ Rationale

Computational notebooks allow arguing from evidence by mixing prose with executable code. For a good overview of problems users encounter in traditional notebooks like Jupyter, see I don't like notebooks and What’s Wrong with Computational Notebooks? Pain Points, Needs, and Design Opportunities.

Specifically Clerk wants to address the following problems:

  • Less helpful than my editor
  • Notebook code being hard to reuse
  • Reproduction problems coming from out-of-order execution
  • Problems with archival and putting notebooks in source control

Clerk is a notebook library for Clojure that aims to address these problems by doing less, namely:

  • no editing environment, folks can keep using the editors they know and love
  • no new format: Clerk notebooks are regular Clojure namespaces (interspersed with markdown comments). This also means Clerk notebooks are meant to be stored in source control.
  • no out-of-order execution: Clerk notebooks always evaluate from top to bottom. Clerk builds a dependency graph of Clojure vars and only recomputes the needed changes to keep the feedback loop fast.
  • no external process: Clerk runs inside your Clojure process, giving Clerk access to all code on the classpath.

🚀 Getting Started

🤹 Clerk Demo

When you're not yet familiar with Clerk, we recommend cloning and playing with the nextjournal/clerk-demo repo.

git clone git@github.com:nextjournal/clerk-demo.git
cd clerk-demo

Then open dev/user.clj from the project in your favorite editor start a REPL into the project, see

🔌 In an Existing Project

To use Clerk in your project, add the following dependency to your deps.edn:

{:deps {io.github.nextjournal/clerk {:mvn/version "0.9.513"}}}

Require and start Clerk as part of your system start, e.g. in user.clj:

(require '[nextjournal.clerk :as clerk])

;; start Clerk's built-in webserver on the default port 7777, opening the browser when done
(clerk/serve! {:browse? true})

;; either call `clerk/show!` explicitly to show a given notebook.
(clerk/show! "notebooks/rule_30.clj")

You can then access Clerk at http://localhost:7777.

⏱ File Watcher

You can load, evaluate, and present a file with the clerk/show! function, but in most cases it's easier to start a file watcher with something like:

(clerk/serve! {:watch-paths ["notebooks" "src"]})

... which will automatically reload and re-eval any clj or md files that change, displaying the most recently changed one in your browser.

To make this performant enough to feel good, Clerk caches the computations it performs while evaluating each file. Likewise, to make sure it doesn't send too much data to the browser at once, Clerk paginates data structures within an interactive viewer.

🔪 Editor Integration

A recommended alternative to the file watcher is setting up a hotkey in your editor to save & clerk/show! the active file.

Emacs

In Emacs, add the following to your config:

(defun clerk-show ()
(interactive)
(when-let
((filename
(buffer-file-name)))
(save-buffer)
(cider-interactive-eval
(concat "(nextjournal.clerk/show! \"" filename "\")"))))

(define-key clojure-mode-map (kbd "<M-return>") 'clerk-show)

IntelliJ/Cursive

In IntelliJ/Cursive, you can set up REPL commands via:

  • going to Tools→REPL→Add New REPL Command, then
  • add the following command: (show! "~file-path");
  • make sure the command is executed in the nextjournal.clerk namespace;
  • lastly assign a shortcut of your choice via Settings→Keymap

Neovim + Conjure

With neovim + conjure one can use the following vimscript function to save the file and show it with Clerk:

function! ClerkShow()
exe "w"
exe "ConjureEval (nextjournal.clerk/show! \"" . expand("%:p") . "\")"
endfunction

nmap <silent> <localleader>cs :execute ClerkShow()<CR>

🔍 Viewers

Clerk comes with a number of useful built-in viewers e.g. for Clojure data, html & hiccup, tables, plots &c.

When showing large data structures, Clerk's default viewers will paginate the results.

🧩 Clojure Data

The default set of viewers are able to render Clojure data.

(def clojure-data
{:hello "world 👋"
:tacos (map #(repeat % '🌮) (range 1 30))
:zeta "The\npurpose\nof\nvisualization\nis\ninsight,\nnot\npictures."})
{:hello "
world 👋"
:tacos ((🌮) (🌮 🌮) (🌮 🌮 🌮) (🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 16 more elided) (20 more elided) 9 more elided) :zeta "
The↩︎purpose↩︎of↩︎visualization↩︎is↩︎insight,↩︎not↩︎pictures."
}

Viewers can handle lazy infinite sequences, partially loading data by default with the ability to load more data on request.

(range)
(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 1000000+ more elided)
(def fib (lazy-cat [0 1] (map + fib (rest fib))))
(0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181 73 more elided)

In addition, there's a number of built-in viewers that can be called explicity using functions.

🌐 Hiccup, HTML & SVG

The html viewer interprets hiccup when passed a vector.

(clerk/html [:div "As Clojurians we " [:em "really"] " enjoy hiccup"])
As Clojurians we really enjoy hiccup

Alternatively you can pass it an HTML string.

(clerk/html "Never <strong>forget</strong>.")
Never forget.

You can style elements, using Tailwind CSS.

(clerk/html [:button.bg-sky-500.hover:bg-sky-700.text-white.rounded-xl.px-2.py-1 "✨ Tailwind CSS"])

The html viewer is also able to display SVG, taking either a hiccup vector or a SVG string.

(clerk/html [:svg {:width 500 :height 100}
[:circle {:cx 25 :cy 50 :r 25 :fill "blue"}]
[:circle {:cx 100 :cy 75 :r 25 :fill "red"}]])

🔢 Tables

Clerk provides a built-in data table viewer that supports the three most common tabular data shapes out of the box: a sequence of maps, where each map's keys are column names; a seq of seq, which is just a grid of values with an optional header; a map of seqs, in with keys are column names and rows are the values for that column.

(clerk/table [[1 2]
[3 4]]) ;; seq of seqs
12
34
(clerk/table (clerk/use-headers [["odd numbers" "even numbers"]
[1 2]
[3 4]])) ;; seq of seqs with header
odd numbers
even numbers
12
34
(clerk/table [{"odd numbers" 1 "even numbers" 2}
{"odd numbers" 3 "even numbers" 4}]) ;; seq of maps
odd numbers
even numbers
12
34
(clerk/table {"odd numbers" [1 3]
"even numbers" [2 4]}) ;; map of seqs
odd numbers
even numbers
12
34

Internally the table viewer will normalize all of the above to a map with :rows and an optional :head key, also giving you control over the column order.

(clerk/table {:head ["odd numbers" "even numbers"]
:rows [[1 2] [3 4]]}) ;; map with `:rows` and optional `:head` keys
odd numbers
even numbers
12
34

🧮 TeX

As we've already seen, all comment blocks can contain TeX (we use KaTeX under the covers). In addition, you can call the TeX viewer programmatically. Here, for example, are Maxwell's equations in differential form:

(clerk/tex "
\\begin{alignedat}{2}
\\nabla\\cdot\\vec{E} = \\frac{\\rho}{\\varepsilon_0} & \\qquad \\text{Gauss' Law} \\\\
\\nabla\\cdot\\vec{B} = 0 & \\qquad \\text{Gauss' Law ($\\vec{B}$ Fields)} \\\\
\\nabla\\times\\vec{E} = -\\frac{\\partial \\vec{B}}{\\partial t} & \\qquad \\text{Faraday's Law} \\\\
\\nabla\\times\\vec{B} = \\mu_0\\vec{J}+\\mu_0\\varepsilon_0\\frac{\\partial\\vec{E}}{\\partial t} & \\qquad \\text{Ampere's Law}
\\end{alignedat}
")
E=ρε0Gauss’ LawB=0Gauss’ Law (B Fields)×E=BtFaraday’s Law×B=μ0J+μ0ε0EtAmpere’s Law \begin{alignedat}{2} \nabla\cdot\vec{E} = \frac{\rho}{\varepsilon_0} & \qquad \text{Gauss' Law} \\ \nabla\cdot\vec{B} = 0 & \qquad \text{Gauss' Law ($\vec{B}$ Fields)} \\ \nabla\times\vec{E} = -\frac{\partial \vec{B}}{\partial t} & \qquad \text{Faraday's Law} \\ \nabla\times\vec{B} = \mu_0\vec{J}+\mu_0\varepsilon_0\frac{\partial\vec{E}}{\partial t} & \qquad \text{Ampere's Law} \end{alignedat}

📊 Plotly

Clerk also has built-in support for Plotly's low-ceremony plotting. See Plotly's JavaScript docs for more examples and options.

(clerk/plotly {:data [{:z [[1 2 3] [3 2 1]] :type "surface"}]
:layout {:margin {:l 20 :r 0 :b 20 :t 20}}
:config {:displayModeBar false
:displayLogo false}})
Loading...

🗺 Vega Lite

But Clerk also has Vega Lite for those who prefer that grammar.

(clerk/vl {:width 650 :height 400 :data {:url "https://vega.github.io/vega-datasets/data/us-10m.json"
:format {:type "topojson" :feature "counties"}}
:transform [{:lookup "id" :from {:data {:url "https://vega.github.io/vega-datasets/data/unemployment.tsv"}
:key "id" :fields ["rate"]}}]
:projection {:type "albersUsa"} :mark "geoshape" :encoding {:color {:field "rate" :type "quantitative"}}
:embed/opts {:actions false}})
Loading...

You can provide a map of embed options to the vega viewer via the :embed/opts key.

🎼 Code

The code viewer uses clojure-mode for syntax highlighting.

(clerk/code (macroexpand '(when test
expression-1
expression-2)))
(if test (do expression-1 expression-2))
(clerk/code '(ns foo "A great ns" (:require [clojure.string :as str])))
(ns foo "A great ns" (:require [clojure.string :as str]))
(clerk/code "(defn my-fn\n \"This is a Doc String\"\n [args]\n 42)")
(defn my-fn
"This is a Doc String"
[args]
42)

🏞 Images

Clerk now has built-in support for the java.awt.image.BufferedImage class, which is the native image format of the JVM.

When combined with javax.imageio.ImageIO/read, one can easily load images in a variety of formats from a java.io.File, an java.io.InputStream, or any resource that a java.net.URL can address.

For example, we can fetch a photo of De zaaier, Vincent van Gogh's famous painting of a farmer sowing a field from Wiki Commons like this:

(ImageIO/read (URL. "https://upload.wikimedia.org/wikipedia/commons/thumb/3/31/The_Sower.jpg/1510px-The_Sower.jpg"))

We've put some effort into making the default image rendering pleasing. The viewer uses the dimensions and aspect ratio of each image to guess the best way to display it in classic DWIM fashion. For example, an image larger than 900px wide with an aspect ratio larger then two will be displayed full width:

(ImageIO/read (URL. "https://images.unsplash.com/photo-1532879311112-62b7188d28ce?ixlib=rb-1.2.1&ixid=MnwxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8"))

On the other hand, smaller images are centered and shown using their intrinsic dimensions:

(ImageIO/read (URL. "https://etc.usf.edu/clipart/36600/36667/thermos_36667_sm.gif"))

📒 Markdown

The same Markdown support Clerk uses for comment blocks is also available programmatically:

(clerk/md (clojure.string/join "\n" (map #(str "* Item " (inc %)) (range 3))))
  • Item 1
  • Item 2
  • Item 3

🔠 Grid Layouts

Layouts can be composed via rows and cols

Passing :width, :height or any other style attributes to ::clerk/opts will assign them on the row or col that contains your items. You can use this to size your containers accordingly.

(clerk/row image-1 image-2 image-3)
(clerk/col {::clerk/opts {:width 150}} image-1 image-2 image-3)

Laying out stuff is not limited to images. You can use it to lay out any Clerk viewer. E.g. combine it with HTML viewers to render nice captions:

(defn caption [text]
(clerk/html [:span.text-slate-500.text-xs.text-center.font-sans text]))
#object[nextjournal.clerk.book$caption 0x49afb902 "
nextjournal.clerk.book$caption@49afb902"
]
(clerk/row
(clerk/col image-1 (caption "Figure 1: Decorative A"))
(clerk/col image-2 (caption "Figure 2: Decorative B"))
(clerk/col image-3 (caption "Figure 3: Decorative C")))
Figure 1: Decorative A
Figure 2: Decorative B
Figure 3: Decorative C

Alternative notations

By default, row and col operate on & rest so you can pass any number of items to the functions. But the viewers are smart enough to accept any sequential list of items.

(v/row [image-1 image-2 image-3])

🍱 Composing Viewers

Viewers compose, so you can for example use the plotly viewer inside the grid viewers.

show code
#'nextjournal.clerk.book/contour-plot
(clerk/col (clerk/row donut-chart donut-chart donut-chart)
contour-plot)
Loading...
Loading...
Loading...
Loading...

🤹🏻 Applying Viewers

Metadata Notation

In the examples above, we've used convience helper functions like clerk/html or clerk/plotly to wrap values in a viewer. If you call this on the REPL, you'll notice a given value gets wrapped in a map under the :nextjournal/value key with the viewer being in the :nextjournal/viewer key.

You can also select a viewer using Clojure metadata in order to avoid Clerk interfering with the value.

^{::clerk/viewer clerk/table}
(def my-dataset
[{:temperature 41.0 :date (java.time.LocalDate/parse "2022-08-01")}
{:temperature 39.0 :date (java.time.LocalDate/parse "2022-08-01")}
{:temperature 34.0 :date (java.time.LocalDate/parse "2022-08-01")}
{:temperature 29.0 :date (java.time.LocalDate/parse "2022-08-01")}])
:temperature
:date
41#object[java.time.LocalDate 0x52fbc7a2 "
2022-08-01"
]
39#object[java.time.LocalDate 0x5a13a8c "
2022-08-01"
]
34#object[java.time.LocalDate 0x173e0562 "
2022-08-01"
]
29#object[java.time.LocalDate 0x236dd92e "
2022-08-01"
]

👁 Writing Viewers

Let's explore how Clerk viewers work and how you create your own to gain better insight into your problem at hand.

v/default-viewers
[{:pred #object[clojure.core$char_QMARK___5425 0x4106651f "
clojure.core$char_QMARK___5425@4106651f"
]
:render-fn (fn [c] (v/html [:span.cmt-string.inspected-value "
\"
c]))}
{:page-size 80 :pred #object[clojure.core$string_QMARK___5427 0x3113a37 "
clojure.core$string_QMARK___5427@3113a37"
]
:render-fn v/quoted-string-viewer}
{:pred #object[clojure.core$number_QMARK_ 0x5002fc11 "
clojure.core$number_QMARK_@5002fc11"
]
:render-fn v/number-viewer}
{:name :number-hex :render-fn (fn [num] (v/number-viewer (str "
0x"
(.toString (js/Number. num) 16))))}
{:pred #object[clojure.core$symbol_QMARK_ 0x1e11bc55 "
clojure.core$symbol_QMARK_@1e11bc55"
]
:render-fn (fn [x] (v/html [:span.cmt-keyword.inspected-value (str x)]))}
{:pred #object[clojure.core$keyword_QMARK_ 0x3c403b36 "
clojure.core$keyword_QMARK_@3c403b36"
]
:render-fn (fn [x] (v/html [:span.cmt-atom.inspected-value (str x)]))}
{:pred #object[clojure.core$nil_QMARK_ 0x9acee30 "
clojure.core$nil_QMARK_@9acee30"
]
:render-fn (fn [_] (v/html [:span.cmt-default.inspected-value "
nil"
]))}
{:pred #object[clojure.core$boolean_QMARK_ 0x133e019b "
clojure.core$boolean_QMARK_@133e019b"
]
:render-fn (fn [x] (v/html [:span.cmt-bool.inspected-value (str x)]))}
{:name :map-entry :page-size 2 :pred #object[clojure.core$map_entry_QMARK_ 0x7d28297b "
clojure.core$map_entry_QMARK_@7d28297b"
]
:render-fn (fn [xs opts] (v/html (into [:<>] (comp (v/inspect-children opts) (interpose "
"
))
xs)))}
{:pred #object[nextjournal.clerk.viewer$get_safe$fn__12031 0x70bd0c3 "
nextjournal.clerk.viewer$get_safe$fn__12031@70bd0c3"
]
:transform-fn #object[nextjournal.clerk.viewer$update_val$fn__12177 0x2fa7dc36 "
nextjournal.clerk.viewer$update_val$fn__12177@2fa7dc36"
]
}
{:name :read+inspect :render-fn (fn [x] (try (v/html [v/inspect (v/read-string x)]) (catch js/Error _e (v/unreadable-edn-viewer x))))} {:closing-paren "
]"
:opening-paren "
["
:page-size 20 :pred #object[clojure.core$vector_QMARK___5431 0x7cd91400 "
clojure.core$vector_QMARK___5431@7cd91400"
]
:render-fn v/coll-viewer}
{5 more elided} {5 more elided} {6 more elided} {3 more elided} {4 more elided} {3 more elided} {2 more elided} {2 more elided} 21 more elided]

These are the default viewers that come with Clerk.

(into #{} (map type) v/default-viewers)
#{clojure.lang.PersistentArrayMap}

Each viewer is a simple Clojure map.

(assoc (frequencies (mapcat keys v/default-viewers)) :total (count v/default-viewers))
{:closing-paren 4 :name 25 :opening-paren 4 :page-size 7 :pred 19 :render-fn 32 :total 41 :transform-fn 24}

We have a total of 41 viewers in the defaults. Let's start with a simple example and explain the different extensions points in the viewer api.

🎪 Presentation

Clerk's rendering happens in the browser. On the Clojure-side, a given document is presented. Presenting takes a value and transforms it such that Clerk can send it to the browser where it will be rendered.

show code

Let's start with one of the simplest examples. You can see that present takes our value 1 and transforms it into a map, with 1 under a :nextjournal/value key and the number viewer assigned under the :nextjournal/viewer key. We call this map a wrapped-value.

^{::clerk/viewer show-raw-value}
(v/present 1)
{:nextjournal/viewer {:render-fn {:form v/number-viewer}},
:nextjournal/value 1,
:path [],
:nextjournal/expanded-at {[] false}}

This data structure is is sent over Clerk's websocket to the browser, where it will be displayed using the :render-fn found in the :nextjournal/viewer key.

Now onto something slightly more complex, #{1 2 3}.

^{::clerk/viewer show-raw-value}
(v/present #{1 2 3})
{:nextjournal/viewer
{:render-fn {:form v/coll-viewer},
:opening-paren "#{",
:closing-paren ("}"),
:page-size 20},
:nextjournal/value
[{:nextjournal/viewer {:render-fn {:form v/number-viewer}},
:nextjournal/value 1,
:path [0]}
{:nextjournal/viewer {:render-fn {:form v/number-viewer}},
:nextjournal/value 2,
:path [1]}
{:nextjournal/viewer {:render-fn {:form v/number-viewer}},
:nextjournal/value 3,
:path [2]}],
:path [],
:nextjournal/expanded-at {[] false, [0] false, [1] false, [2] false}}

Here, we're giving it a set with 1, 2, 3 in it. In its generalized form, present is a function that does a depth-first traversal of a given tree, starting at the root node. It will select a viewer for this root node, and unless told otherwise, descend further down the tree to present its child nodes.

Compare this with the simple 1 example above! You should recognize the leaf values. Also note that the container is no longer a set, but it has been transformed into a vector. This transformation exists to support pagination of long unordered sequences like maps and sets and so we can efficiently access a value inside this tree using get-in.

You might ask yourself why we don't just send the unmodified value to the browser. For one, we could easily overload the browser with too much data. Secondly we will look at examples of being able to select viewers based on Clojure and Java types, which cannot be serialized and sent to the browser.

⚙️ Transform

When writing your own viewer, the first extention point you should reach for is :tranform-fn.

(v/with-viewer {:transform-fn #(clerk/html [:pre (pr-str %)])}
"Exploring the viewer api")
{:nextjournal/value "Exploring the viewer api", :nextjournal/viewers [{:pred #object[sicmutils.expression$literal_QMARK_ 0x510ba0cf "sicmutils.expression$literal_QMARK_@510ba0cf"], :transform-fn #object[clojure.core$comp$fn__5825 0x1911d197 "clojure.core$comp$fn__5825@1911d197"], :render-fn (fn [label->val] (v/html (reagent/with-let [!selected-label (reagent/atom (ffirst label->val))] [:<> (into [:div.flex.items-center.font-sans.text-xs.mb-3 [:span.text-slate-500.mr-2 "View-as:"]] (map (fn [label] [:button.px-3.py-1.font-medium.hover:bg-indigo-50.rounded-full.hover:text-indigo-600.transition {:class (if (= (clojure.core/deref !selected-label) label) "bg-indigo-100 text-indigo-600" "text-slate-500"), :on-click (fn* [] (reset! !selected-label label))} label])) (keys label->val)) [v/inspect-presented (get label->val (clojure.core/deref !selected-label))]])))} {:pred #object[clojure.core$char_QMARK___5425 0x4106651f "clojure.core$char_QMARK___5425@4106651f"], :render-fn (fn [c] (v/html [:span.cmt-string.inspected-value "\\" c]))} {:pred #object[clojure.core$string_QMARK___5427 0x3113a37 "clojure.core$string_QMARK___5427@3113a37"], :render-fn v/quoted-string-viewer, :page-size 80} {:pred #object[clojure.core$number_QMARK_ 0x5002fc11 "clojure.core$number_QMARK_@5002fc11"], :render-fn v/number-viewer} {:name :number-hex, :render-fn (fn [num] (v/number-viewer (str "0x" (.toString (js/Number. num) 16))))} {:pred #object[clojure.core$symbol_QMARK_ 0x1e11bc55 "clojure.core$symbol_QMARK_@1e11bc55"], :render-fn (fn [x] (v/html [:span.cmt-keyword.inspected-value (str x)]))} {:pred #object[clojure.core$keyword_QMARK_ 0x3c403b36 "clojure.core$keyword_QMARK_@3c403b36"], :render-fn (fn [x] (v/html [:span.cmt-atom.inspected-value (str x)]))} {:pred #object[clojure.core$nil_QMARK_ 0x9acee30 "clojure.core$nil_QMARK_@9acee30"], :render-fn (fn [_] (v/html [:span.cmt-default.inspected-value "nil"]))} {:pred #object[clojure.core$boolean_QMARK_ 0x133e019b "clojure.core$boolean_QMARK_@133e019b"], :render-fn (fn [x] (v/html [:span.cmt-bool.inspected-value (str x)]))} {:pred #object[clojure.core$map_entry_QMARK_ 0x7d28297b "clojure.core$map_entry_QMARK_@7d28297b"], :name :map-entry, :render-fn (fn [xs opts] (v/html (into [:<>] (comp (v/inspect-children opts) (interpose " ")) xs))), :page-size 2} {:pred #object[nextjournal.clerk.viewer$get_safe$fn__12031 0x70bd0c3 "nextjournal.clerk.viewer$get_safe$fn__12031@70bd0c3"], :transform-fn #object[nextjournal.clerk.viewer$update_val$fn__12177 0x2fa7dc36 "nextjournal.clerk.viewer$update_val$fn__12177@2fa7dc36"]} {:name :read+inspect, :render-fn (fn [x] (try (v/html [v/inspect (v/read-string x)]) (catch js/Error _e (v/unreadable-edn-viewer x))))} {:pred #object[clojure.core$vector_QMARK___5431 0x7cd91400 "clojure.core$vector_QMARK___5431@7cd91400"], :render-fn v/coll-viewer, :opening-paren "[", :closing-paren "]", :page-size 20} {:pred #object[clojure.core$set_QMARK_ 0x2b7e2212 "clojure.core$set_QMARK_@2b7e2212"], :render-fn v/coll-viewer, :opening-paren "#{", :closing-paren "}", :page-size 20} {:pred #object[clojure.core$sequential_QMARK_ 0x54398386 "clojure.core$sequential_QMARK_@54398386"], :render-fn v/coll-viewer, :opening-paren "(", :closing-paren ")", :page-size 20} {:pred #object[clojure.core$map_QMARK___5429 0xb0a1535 "clojure.core$map_QMARK___5429@b0a1535"], :name :map, :render-fn v/map-viewer, :opening-paren "{", :closing-paren "}", :page-size 10} {:pred #object[clojure.core$some_fn$sp1__8670 0x79d6b7d6 "clojure.core$some_fn$sp1__8670@79d6b7d6"], :transform-fn #object[clojure.core$comp$fn__5825 0xccdbaa6 "clojure.core$comp$fn__5825@ccdbaa6"], :render-fn (fn [x] (v/html [:span.inspected-value [:span.cmt-meta "#'" (str x)]]))} {:pred #object[nextjournal.clerk.viewer$fn__12219 0x45892a66 "nextjournal.clerk.viewer$fn__12219@45892a66"], :name :error, :render-fn v/throwable-viewer, :transform-fn #object[clojure.core$comp$fn__5825 0x55f1d5ff "clojure.core$comp$fn__5825@55f1d5ff"]} {:pred #object[nextjournal.clerk.viewer$fn__12222 0x1cdd67dc "nextjournal.clerk.viewer$fn__12222@1cdd67dc"], :transform-fn #object[nextjournal.clerk.viewer$fn__12225 0x621c72b7 "nextjournal.clerk.viewer$fn__12225@621c72b7"], :render-fn (fn [blob] (v/html [:figure.flex.flex-col.items-center.not-prose [:img {:src (v/url-for blob)}]]))} {:pred #object[nextjournal.clerk.viewer$fn__12230 0x47a56acf "nextjournal.clerk.viewer$fn__12230@47a56acf"], :transform-fn #object[nextjournal.clerk.viewer$update_val$fn__12177 0x75168a0a "nextjournal.clerk.viewer$update_val$fn__12177@75168a0a"]} {:pred #object[clojure.core$partial$fn__5857 0xb010dea "clojure.core$partial$fn__5857@b010dea"], :transform-fn #object[nextjournal.clerk.viewer$fn__12235 0x146f0684 "nextjournal.clerk.viewer$fn__12235@146f0684"]} {:pred #object[clojure.core$constantly$fn__5689 0x7d7dcf6 "clojure.core$constantly$fn__5689@7d7dcf6"], :transform-fn #object[nextjournal.clerk.viewer$update_val$fn__12177 0x4f6a67db "nextjournal.clerk.viewer$update_val$fn__12177@4f6a67db"]} {:name :elision, :render-fn v/elision-viewer, :transform-fn #object[nextjournal.clerk.viewer$mark_presented 0x4e623d4b "nextjournal.clerk.viewer$mark_presented@4e623d4b"]} {:name :latex, :render-fn v/katex-viewer, :transform-fn #object[nextjournal.clerk.viewer$mark_presented 0x4e623d4b "nextjournal.clerk.viewer$mark_presented@4e623d4b"]} {:name :mathjax, :render-fn v/mathjax-viewer, :transform-fn #object[nextjournal.clerk.viewer$mark_presented 0x4e623d4b "nextjournal.clerk.viewer$mark_presented@4e623d4b"]} {:name :html, :render-fn v/html, :transform-fn #object[clojure.core$comp$fn__5825 0xf65a857 "clojure.core$comp$fn__5825@f65a857"]} {:name :plotly, :render-fn v/plotly-viewer, :transform-fn #object[nextjournal.clerk.viewer$mark_presented 0x4e623d4b "nextjournal.clerk.viewer$mark_presented@4e623d4b"]} {:name :vega-lite, :render-fn v/vega-lite-viewer, :transform-fn #object[nextjournal.clerk.viewer$mark_presented 0x4e623d4b "nextjournal.clerk.viewer$mark_presented@4e623d4b"]} {:name :markdown, :transform-fn #object[nextjournal.clerk.viewer$fn__12241 0x63d2d419 "nextjournal.clerk.viewer$fn__12241@63d2d419"]} {:name :code, :render-fn v/code-viewer, :transform-fn #object[clojure.core$comp$fn__5825 0x7f1a2307 "clojure.core$comp$fn__5825@7f1a2307"]} {:name :code-folded, :render-fn v/foldable-code-viewer, :transform-fn #object[clojure.core$comp$fn__5825 0x20cdc1ca "clojure.core$comp$fn__5825@20cdc1ca"]} {:name :reagent, :render-fn v/reagent-viewer, :transform-fn #object[nextjournal.clerk.viewer$mark_presented 0x4e623d4b "nextjournal.clerk.viewer$mark_presented@4e623d4b"]} {:name :row, :render-fn (fn [items opts] (let [item-count (count items)] (v/html (into [:div {:class "md:flex md:flex-row md:gap-4 not-prose", :style opts}] (map (fn [item] [:div.flex.items-center.justify-center.flex-auto (v/inspect-presented opts item)])) items))))} {:name :col, :render-fn (fn [items opts] (v/html (into [:div {:class "md:flex md:flex-col md:gap-4 clerk-grid not-prose", :style opts}] (map (fn [item] [:div.flex.items-center.justify-center (v/inspect-presented opts item)])) items)))} {:name :table, :transform-fn #object[nextjournal.clerk.viewer$fn__12256 0x2c8f294 "nextjournal.clerk.viewer$fn__12256@2c8f294"]} {:name :table-error, :render-fn v/table-error, :page-size 1} {:name :clerk/code-block, :transform-fn #object[nextjournal.clerk.viewer$fn__12268 0x77cf9cc8 "nextjournal.clerk.viewer$fn__12268@77cf9cc8"]} {:name :clerk/result-block, :transform-fn #object[clojure.core$comp$fn__5825 0x59638b2d "clojure.core$comp$fn__5825@59638b2d"]} {:name :tagged-value, :render-fn (fn [{:keys [tag value space?]} opts] (v/html (v/tagged-value {:space? (:nextjournal/value space?)} (str "#" (:nextjournal/value tag)) [v/inspect-presented value]))), :transform-fn #object[nextjournal.clerk.viewer$mark_preserve_keys 0x17eec25a "nextjournal.clerk.viewer$mark_preserve_keys@17eec25a"]} {:name :clerk/result, :render-fn v/result-viewer, :transform-fn #object[nextjournal.clerk.viewer$mark_presented 0x4e623d4b "nextjournal.clerk.viewer$mark_presented@4e623d4b"]} {:name :clerk/notebook, :render-fn v/notebook-viewer, :transform-fn #object[nextjournal.clerk.viewer$fn__12276 0x52f7de61 "nextjournal.clerk.viewer$fn__12276@52f7de61"]} {:name :hide-result, :transform-fn #object[nextjournal.clerk.viewer$fn__12139 0x59d5312b "nextjournal.clerk.viewer$fn__12139@59d5312b"]}], :!budget #object[clojure.lang.Atom 0x6173f46a {:status :ready, :val 200}], :path [], :current-path []}

As you can see the argument to the :transform-fn isn't just the string we're passing it, but a wrapped-value. We will look at what this enables in a bit. But let's look at one of the simplest examples first.

A first simple example

(def greet-viewer
{:transform-fn (clerk/update-val #(clerk/html [:strong "Hello, " % " 👋"]))})
{:transform-fn #object[nextjournal.clerk.viewer$update_val$fn__12177 0x2a8c5790 "
nextjournal.clerk.viewer$update_val$fn__12177@2a8c5790"
]
}

For this simple greet-viewer we're only doing a simple value transformation. For this, clerk/update-val is a small helper function which takes a function f and returns a function to update only the value inside a wrapped-value, a shorthand for #(update % :nextjournal/val f)

(v/with-viewer greet-viewer
"James Clerk Maxwell")
Hello, James Clerk Maxwell 👋

The :transform-fn runs on the JVM, which means you can explore what it does at your REPL by calling v/present on such a value.

^{::clerk/viewer show-raw-value}
(v/present (v/with-viewer greet-viewer
"James Clerk Maxwell"))
{:nextjournal/viewer {:name :html, :render-fn {:form v/html}},
:nextjournal/value [:strong "Hello, " "James Clerk Maxwell" " 👋"],
:path [],
:nextjournal/expanded-at {[] false, nil false}}

Passing modified viewers down the tree

(defn add-child-viewers [viewer viewers]
(update viewer :transform-fn (fn [transform-fn-orig]
(fn [wrapped-value]
(update (transform-fn-orig wrapped-value) :nextjournal/viewers clerk/add-viewers viewers)))))
#object[nextjournal.clerk.book$add_child_viewers 0x3b74353a "
nextjournal.clerk.book$add_child_viewers@3b74353a"
]
v/table-viewer
{:name :table :transform-fn #object[nextjournal.clerk.viewer$fn__12256 0x2c8f294 "
nextjournal.clerk.viewer$fn__12256@2c8f294"
]
}
(def custom-table-viewer
(add-child-viewers v/table-viewer
[(assoc v/table-head-viewer :transform-fn (v/update-val (partial map (comp (partial str "Column: ") str/capitalize name))))
(assoc v/table-missing-viewer :render-fn '(fn [x] (v/html [:span.red "N/A"])))]))
{:name :table :transform-fn #object[nextjournal.clerk.book$add_child_viewers$fn__42284$fn__42285 0x45882adf "
nextjournal.clerk.book$add_child_viewers$fn__42284$fn__42285@45882adf"
]
}
(clerk/with-viewer custom-table-viewer
{:col/a [1 2 3 4] :col/b [1 2 3] :col/c [1 2 3]})
Column: A
Column: B
Column: C
111
222
333
4N/AN/A
(clerk/with-viewer custom-table-viewer
{:col/a [1 2 3 4] :col/b [1 2 3] :col/c [1 2 3]})
Column: A
Column: B
Column: C
111
222
333
4N/AN/A

🐢 Recursion

But this presentation and hence tranformation of nodes further down the tree isn't always what you want. For example, the plotly or vl viewers want to receive the child value unaltered in order to use it as a spec.

To stop Clerk's presentation from descending into child nodes, use clerk/mark-presented as a :transform-fn. Compare the result below in which [1 2 3] appears unaltered with what you see above.

^{::clerk/viewer show-raw-value}
(v/present (clerk/with-viewer {:transform-fn clerk/mark-presented
:render-fn '(fn [x] (v/html [:pre (pr-str x)]))}
[1 2 3]))
{:nextjournal/viewer
{:render-fn {:form (fn [x] (v/html [:pre (pr-str x)]))}},
:nextjournal/value [1 2 3],
:path [],
:nextjournal/expanded-at {[] false, nil false}}

Clerk's presentation will also transform maps into sequences in order to paginate large maps. When you're dealing with a map that you know is bounded and would like to preserve its keys, there's clerk/mark-preserve-keys. This will still transform (and paginate) the values of the map, but leave the keys unaltered.

^{::clerk/viewer show-raw-value}
(v/present (clerk/with-viewer {:transform-fn clerk/mark-preserve-keys}
{:hello 42}))
{:nextjournal/viewer
{:name :map,
:render-fn {:form v/map-viewer},
:opening-paren "{",
:closing-paren ("}"),
:page-size 10},
:nextjournal/value
{:hello
{:nextjournal/viewer {:render-fn {:form v/number-viewer}},
:nextjournal/value 42,
:path [:hello]}},
:path [],
:nextjournal/expanded-at {[] false}}

🔬 Render

As we've just seen, you can also do a lot with :transform-fn and using clerk/html on the JVM. When you want to run code in the browser where Clerk's viewers are rendered, reach for :render-fn. As an example, we'll write a multiviewer for a sicmutils literal expression that will compute two alternative representations and let the user switch between them in the browser.

We start with a simple function that takes a such an expression and turns it into a map with two representation, one TeX and the original form.

(defn transform-literal [expr]
{:TeX (-> expr sicm/->TeX clerk/tex)
:original (clerk/code (with-out-str (sicm/print-expression (sicm/freeze expr))))})
#object[nextjournal.clerk.book$transform_literal 0x68bc2ae8 "
nextjournal.clerk.book$transform_literal@68bc2ae8"
]

Our literal-viewer calls this transform-literal function and also calls clerk/mark-preserve-keys. This tells Clerk to leave the keys of the map as-is.

In our :render-fn, which is called in the browser we will recieve this map. Note that this is a quoted form, not a function. Clerk will send this form to the browser for evaluation. There it will create a reagent/atom that holds the selection state. Lastly, v/inspect-presented is a component that takes a wrapped-value that ran through v/present and show it.

(def literal-viewer
{:pred sicmutils.expression/literal?
:transform-fn (comp clerk/mark-preserve-keys
(clerk/update-val transform-literal))
:render-fn '(fn [label->val]
(v/html
(reagent/with-let [!selected-label (reagent/atom (ffirst label->val))]
[:<> (into
[:div.flex.items-center.font-sans.text-xs.mb-3
[:span.text-slate-500.mr-2 "View-as:"]]
(map (fn [label]
[:button.px-3.py-1.font-medium.hover:bg-indigo-50.rounded-full.hover:text-indigo-600.transition
{:class (if (= @!selected-label label) "bg-indigo-100 text-indigo-600" "text-slate-500")
:on-click #(reset! !selected-label label)}
label]))
(keys label->val))
[v/inspect-presented (get label->val @!selected-label)]])))})
{:pred #object[sicmutils.expression$literal_QMARK_ 0x510ba0cf "
sicmutils.expression$literal_QMARK_@510ba0cf"
]
:render-fn (fn [label->val] (v/html (reagent/with-let [!selected-label (reagent/atom (ffirst label->val))] [:<> (into [:div.flex.items-center.font-sans.text-xs.mb-3 [:span.text-slate-500.mr-2 "
View-as:"
]]
(map (fn [label] [:button.px-3.py-1.font-medium.hover:bg-indigo-50.rounded-full.hover:text-indigo-600.transition {:class (if (= (clojure.core/deref !selected-label) label) "
bg-indigo-100 text-indigo-600"
"
text-slate-500"
)
:on-click (fn* [] (reset! !selected-label label))}
label]))
(keys label->val))
[v/inspect-presented (get label->val (clojure.core/deref !selected-label))]])))
:transform-fn #object[clojure.core$comp$fn__5825 0x1911d197 "
clojure.core$comp$fn__5825@1911d197"
]
}

Now let's see if this works. Try switching to the original representation!

^{::clerk/viewer literal-viewer}
(sicm/+ (sicm/square (sicm/sin 'x))
(sicm/square (sicm/cos 'x)))
View-as:
sin2(x)+cos2(x){\sin}^{2}\left(x\right) + {\cos}^{2}\left(x\right)

🥇 Selection

Without a viewer specified, Clerk will go through the sequence of viewers and apply the :pred function in the viewer to find a matching one. Use v/viewer-for to select a viewer for a given value.

(def char?-viewer
(v/viewer-for v/default-viewers \A))
{:pred #object[clojure.core$char_QMARK___5425 0x4106651f "
clojure.core$char_QMARK___5425@4106651f"
]
:render-fn (fn [c] (v/html [:span.cmt-string.inspected-value "
\"
c]))}

If we select a specific viewer (here the v/html-viewer using clerk/html) this is the viewer we will get.

(def html-viewer
(v/viewer-for v/default-viewers (clerk/html [:h1 "foo"])))
{:name :html :render-fn v/html :transform-fn #object[clojure.core$comp$fn__5825 0xf65a857 "
clojure.core$comp$fn__5825@f65a857"
]
}

Instead of specifying a viewer for every value, we can also modify the viewers per namespace. Here, we add the literal-viewer from above to the whole namespace.

(clerk/add-viewers! [literal-viewer])
[{:pred #object[sicmutils.expression$literal_QMARK_ 0x510ba0cf "
sicmutils.expression$literal_QMARK_@510ba0cf"
]
:render-fn (fn [label->val] (v/html (reagent/with-let [!selected-label (reagent/atom (ffirst label->val))] [:<> (into [:div.flex.items-center.font-sans.text-xs.mb-3 [:span.text-slate-500.mr-2 "
View-as:"
]]
(map (fn [label] [:button.px-3.py-1.font-medium.hover:bg-indigo-50.rounded-full.hover:text-indigo-600.transition {:class (if (= (clojure.core/deref !selected-label) label) "
bg-indigo-100 text-indigo-600"
"
text-slate-500"
)
:on-click (fn* [] (reset! !selected-label label))}
label]))
(keys label->val))
[v/inspect-presented (get label->val (clojure.core/deref !selected-label))]])))
:transform-fn #object[clojure.core$comp$fn__5825 0x1911d197 "
clojure.core$comp$fn__5825@1911d197"
]
}]

As you can see we now get this viewer automatically, without needing to explicitly select it.

(sicm/+ (sicm/square (sicm/sin 'x))
(sicm/square (sicm/cos 'x)))
View-as:
sin2(x)+cos2(x){\sin}^{2}\left(x\right) + {\cos}^{2}\left(x\right)

🔓 Elisions

(def string?-viewer
(v/viewer-for v/default-viewers "Denn wir sind wie Baumstämme im Schnee."))
{:page-size 80 :pred #object[clojure.core$string_QMARK___5427 0x3113a37 "
clojure.core$string_QMARK___5427@3113a37"
]
:render-fn v/quoted-string-viewer}

Notice that for the string? viewer above, there's a :page-size of 80. This is the case for all collection viewers in Clerk and controls how many elements are displayed. So using the default string?-viewer above, we're showing the first 80 characters.

(def long-string
(str/join (into [] cat (repeat 10 "Denn wir sind wie Baumstämme im Schnee.\n"))))
"
Denn wir sind wie Baumstämme im Schnee.↩︎Denn wir sind wie Baumstämme im Schnee.320 more elided"

If we change the viewer and set a different :n in :page-size, we only see 10 characters.

(v/with-viewer (assoc string?-viewer :page-size 10)
long-string)
"
Denn wir s390 more elided"

Or, we can turn off eliding, by dissoc'ing :page-size alltogether.

(v/with-viewer (dissoc string?-viewer :page-size)
long-string)
"
Denn wir sind wie Baumstämme im Schnee.↩︎Denn wir sind wie Baumstämme im Schnee.↩︎Denn wir sind wie Baumstämme im Schnee.↩︎Denn wir sind wie Baumstämme im Schnee.↩︎Denn wir sind wie Baumstämme im Schnee.↩︎Denn wir sind wie Baumstämme im Schnee.↩︎Denn wir sind wie Baumstämme im Schnee.↩︎Denn wir sind wie Baumstämme im Schnee.↩︎Denn wir sind wie Baumstämme im Schnee.↩︎Denn wir sind wie Baumstämme im Schnee."

The operations above were changes to a single viewer. But we also have a function update-viewers to update a given viewers by applying a select-fn->update-fn map. Here, the predicate is the keyword :page-size and our update function is called for every viewer with :page-size and is dissoc'ing them.

(def without-pagination
{:page-size #(dissoc % :page-size)})
{:page-size #object[nextjournal.clerk.book$fn__42305 0x64e863f1 "
nextjournal.clerk.book$fn__42305@64e863f1"
]
}

Here's the updated-viewers:

(def viewers-without-lazy-loading
(v/update-viewers v/default-viewers without-pagination))
[{:pred #object[clojure.core$char_QMARK___5425 0x4106651f "
clojure.core$char_QMARK___5425@4106651f"
]
:render-fn (fn [c] (v/html [:span.cmt-string.inspected-value "
\"
c]))}
{:pred #object[clojure.core$string_QMARK___5427 0x3113a37 "
clojure.core$string_QMARK___5427@3113a37"
]
:render-fn v/quoted-string-viewer}
{:pred #object[clojure.core$number_QMARK_ 0x5002fc11 "
clojure.core$number_QMARK_@5002fc11"
]
:render-fn v/number-viewer}
{:name :number-hex :render-fn (fn [num] (v/number-viewer (str "
0x"
(.toString (js/Number. num) 16))))}
{:pred #object[clojure.core$symbol_QMARK_ 0x1e11bc55 "
clojure.core$symbol_QMARK_@1e11bc55"
]
:render-fn (fn [x] (v/html [:span.cmt-keyword.inspected-value (str x)]))}
{:pred #object[clojure.core$keyword_QMARK_ 0x3c403b36 "
clojure.core$keyword_QMARK_@3c403b36"
]
:render-fn (fn [x] (v/html [:span.cmt-atom.inspected-value (str x)]))}
{:pred #object[clojure.core$nil_QMARK_ 0x9acee30 "
clojure.core$nil_QMARK_@9acee30"
]
:render-fn (fn [_] (v/html [:span.cmt-default.inspected-value "
nil"
]))}
{:pred #object[clojure.core$boolean_QMARK_ 0x133e019b "
clojure.core$boolean_QMARK_@133e019b"
]
:render-fn (fn [x] (v/html [:span.cmt-bool.inspected-value (str x)]))}
{:name :map-entry :pred #object[clojure.core$map_entry_QMARK_ 0x7d28297b "
clojure.core$map_entry_QMARK_@7d28297b"
]
:render-fn (fn [xs opts] (v/html (into [:<>] (comp (v/inspect-children opts) (interpose "
"
))
xs)))}
{:pred #object[nextjournal.clerk.viewer$get_safe$fn__12031 0x70bd0c3 "
nextjournal.clerk.viewer$get_safe$fn__12031@70bd0c3"
]
:transform-fn #object[nextjournal.clerk.viewer$update_val$fn__12177 0x2fa7dc36 "
nextjournal.clerk.viewer$update_val$fn__12177@2fa7dc36"
]
}
{:name :read+inspect :render-fn (fn [x] (try (v/html [v/inspect (v/read-string x)]) (catch js/Error _e (v/unreadable-edn-viewer x))))} {:closing-paren "
]"
:opening-paren "
["
:pred #object[clojure.core$vector_QMARK___5431 0x7cd91400 "
clojure.core$vector_QMARK___5431@7cd91400"
]
:render-fn v/coll-viewer}
{:closing-paren "
}"
:opening-paren "
#{"
:pred #object[clojure.core$set_QMARK_ 0x2b7e2212 "
clojure.core$set_QMARK_@2b7e2212"
]
:render-fn v/coll-viewer}
{4 more elided} {5 more elided} {3 more elided} {4 more elided} {3 more elided} {2 more elided} {2 more elided} 21 more elided]

Now let's confirm these modified viewers don't have :page-size on them anymore.

(filter :page-size viewers-without-lazy-loading)
()

And compare it with the defaults:

(filter :page-size v/default-viewers)
({:page-size 80 :pred #object[clojure.core$string_QMARK___5427 0x3113a37 "
clojure.core$string_QMARK___5427@3113a37"
]
:render-fn v/quoted-string-viewer}
{:name :map-entry :page-size 2 :pred #object[clojure.core$map_entry_QMARK_ 0x7d28297b "
clojure.core$map_entry_QMARK_@7d28297b"
]
:render-fn (fn [xs opts] (v/html (into [:<>] (comp (v/inspect-children opts) (interpose "
"
))
xs)))}
{:closing-paren "
]"
:opening-paren "
["
:page-size 20 :pred #object[clojure.core$vector_QMARK___5431 0x7cd91400 "
clojure.core$vector_QMARK___5431@7cd91400"
]
:render-fn v/coll-viewer}
{:closing-paren "
}"
:opening-paren "
#{"
:page-size 20 :pred #object[clojure.core$set_QMARK_ 0x2b7e2212 "
clojure.core$set_QMARK_@2b7e2212"
]
:render-fn v/coll-viewer}
{:closing-paren "
)"
:opening-paren "
("
:page-size 20 :pred #object[clojure.core$sequential_QMARK_ 0x54398386 "
clojure.core$sequential_QMARK_@54398386"
]
:render-fn v/coll-viewer}
{:closing-paren "
}"
:name :map :opening-paren "
{"
:page-size 10 :pred #object[clojure.core$map_QMARK___5429 0xb0a1535 "
clojure.core$map_QMARK___5429@b0a1535"
]
:render-fn v/map-viewer}
{:name :table-error :page-size 1 :render-fn v/table-error})

Now let's display our clojure-data var from above using these modified viewers.

(clerk/with-viewers viewers-without-lazy-loading
clojure-data)
{:hello "
world 👋"
:tacos ((🌮) (🌮 🌮) (🌮 🌮 🌮) (🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮) (🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮 🌮)) :zeta "
The↩︎purpose↩︎of↩︎visualization↩︎is↩︎insight,↩︎not↩︎pictures."
}

👷 Loading Libraries

This is a custom viewer for Mermaid, a markdown-like syntax for creating diagrams from text. Note that this library isn't bundles with Clerk but we use a component based on d3-require to load it at runtime.

(def mermaid-viewer
{:transform-fn clerk/mark-presented
:render-fn '(fn [value]
(v/html
(when value
[v/with-d3-require {:package ["mermaid@8.14/dist/mermaid.js"]}
(fn [mermaid]
[:div {:ref (fn [el] (when el
(.render mermaid (str (gensym)) value #(set! (.-innerHTML el) %))))}])])))})
{:render-fn (fn [value] (v/html (when value [v/with-d3-require {:package ["
mermaid@8.14/dist/mermaid.js"
]}
(fn [mermaid] [:div {:ref (fn [el] (when el (.render mermaid (str (gensym)) value (fn* [%1] (set! (.-innerHTML el) %1)))))}])])))
:transform-fn #object[nextjournal.clerk$mark_presented 0x34c3d435 "
nextjournal.clerk$mark_presented@34c3d435"
]
}

We can then use the above viewer using with-viewer.

(clerk/with-viewer mermaid-viewer
"stateDiagram-v2
[*] --> Still
Still --> [*]

Still --> Moving
Moving --> Still
Moving --> Crash
Crash --> [*]")
Loading...

🙈 Controlling Visibility

Visibility for code and results can be controlled document-wide and per top-level form. By default, Clerk will always show code and results for a notebook.

You can use a map of the following shape to set the visibility of code and results individually:

{:nextjournal.clerk/visibility {:code :hide :result :show}}

The above example will hide the code and show only results.

Valid values are :show, :hide and :fold (only available for :code). Using {:code :fold} will hide the code cell initially but show an indicator to toggle its visibility:

show code
[8 4 6 17 19 2 18 9 23 11 14 24 21 7 3 5 13 12 15 22 5 more elided]

The visibility map can be used in the following ways:

Set document defaults via the ns form

(ns visibility
{:nextjournal.clerk/visibility {:code :fold}})

The above example will hide all code cells by default but show an indicator to toggle their visibility instead. In this case results will always show because that’s the default.

As metadata to control a single top-level form

^{::clerk/visibility {:code :hide}} (shuffle (range 25))

This will hide the code but only show the result:

[8 4 6 17 19 2 18 9 23 11 14 24 21 7 3 5 13 12 15 22 5 more elided]

Setting visibility as metadata will override the document-wide visibility settings for this one specific form.

As top-level form to change the document defaults

Independently of what defaults are set via your ns form, you can use a top-level map as a marker to override the visibility settings for any forms following it.

Example: Code is hidden by default but you want to show code for all top-level forms after a certain point:

(ns visibility
{:nextjournal.clerk/visibility {:code :hide}})

(+ 39 3) ;; code will be hidden
(range 25) ;; code will be hidden

{:nextjournal.clerk/visibility {:code :show}}

(range 500) ;; code will be visible
(rand-int 42) ;; code will be visible

This comes in quite handy for debugging too!

⚡️ Incremental Computation

🔖 Parsing

First, we parse a given Clojure file using rewrite-clj.

(def parsed
(parser/parse-file "book.clj"))
{:blocks [{:loc {:column 1 :line 2} :text "
^{:nextjournal.clerk/visibility {:code :hide}}↩︎(ns nextjournal.clerk.book↩︎ {:ne473 more elided"
:type :code}
{:loc {:column 1 :line 140} :text "
(def clojure-data↩︎ {:hello "world 👋"↩︎ :tacos (map #(repeat % '🌮) (range 1 378 more elided"
:type :code}
{:loc {:column 1 :line 147} :text "
(range)"
:type :code}
{:loc {:column 1 :line 149} :text "
(def fib (lazy-cat [0 1] (map + fib (rest fib))))"
:type :code}
{:loc {:column 1 :line 158} :text "
(clerk/html [:div "As Clojurians we " [:em "really"] " enjoy hiccup"])"
:type :code}
{:loc {:column 1 :line 161} :text "
(clerk/html "Never <strong>forget</strong>.")"
:type :code}
{:loc {:column 1 :line 164} :text "
(clerk/html [:button.bg-sky-500.hover:bg-sky-700.text-white.rounded-xl.px-2.py-119 more elided"
:type :code}
{:loc {:column 1 :line 167} :text "
(clerk/html [:svg {:width 500 :height 100}↩︎ [:circle {:cx 25 :cy 5081 more elided"
:type :code}
{:loc {:column 1 :line 179} :text "
(clerk/table [[1 2]↩︎ [3 4]]) ;; seq of seqs"
:type :code}
{:loc {:column 1 :line 182} :text "
(clerk/table (clerk/use-headers [["odd numbers" "even numbers"]↩︎ 91 more elided"
:type :code}
{:loc {:column 1 :line 186} :text "
(clerk/table [{"odd numbers" 1 "even numbers" 2}↩︎ {"odd numbers" 3 34 more elided"
:type :code}
{:loc {:column 1 :line 189} :text "
(clerk/table {"odd numbers" [1 3]↩︎ "even numbers" [2 4]}) ;; map of5 more elided"
:type :code}
{:loc {2 more elided} :text "
(clerk/table {:head ["odd numbers" "even numbers"]↩︎ :rows [[1 2] [352 more elided"
:type :code}
{3 more elided} {3 more elided} {3 more elided} {3 more elided} {3 more elided} {3 more elided} {3 more elided} 63 more elided]
:file "
book.clj"
}

🧐 Analysis

Then, each expression is analysed using tools.analyzer. A dependency graph, the analyzed form and the originating file is recorded.

(def analyzed
(ana/build-graph parsed))
{:->analysis-info {URL {} clojure.lang.AReference {:hash "
5dt5kPSaDo8cr8XWT3RvHd3WYHqg4t"
:jar "
/mnt/mvn-cache/org/clojure/clojure/1.10.3/clojure-1.10.3.jar"
}
clojure.lang.Compiler {:hash "
5dt5kPSaDo8cr8XWT3RvHd3WYHqg4t"
:jar "
/mnt/mvn-cache/org/clojure/clojure/1.10.3/clojure-1.10.3.jar"
}
clojure.lang.LazySeq {:hash "
5dt5kPSaDo8cr8XWT3RvHd3WYHqg4t"
:jar "
/mnt/mvn-cache/org/clojure/clojure/1.10.3/clojure-1.10.3.jar"
}
clojure.lang.LockingTransaction {:hash "
5dt5kPSaDo8cr8XWT3RvHd3WYHqg4t"
:jar "
/mnt/mvn-cache/org/clojure/clojure/1.10.3/clojure-1.10.3.jar"
}
clojure.lang.Namespace {:hash "
5dt5kPSaDo8cr8XWT3RvHd3WYHqg4t"
:jar "
/mnt/mvn-cache/org/clojure/clojure/1.10.3/clojure-1.10.3.jar"
}
clojure.lang.Numbers {:hash "
5dt5kPSaDo8cr8XWT3RvHd3WYHqg4t"
:jar "
/mnt/mvn-cache/org/clojure/clojure/1.10.3/clojure-1.10.3.jar"
}
clojure.lang.PersistentHashMap {:hash "
5dt5kPSaDo8cr8XWT3RvHd3WYHqg4t"
:jar "
/mnt/mvn-cache/org/clojure/clojure/1.10.3/clojure-1.10.3.jar"
}
clojure.lang.RT {:hash "
5dt5kPSaDo8cr8XWT3RvHd3WYHqg4t"
:jar "
/mnt/mvn-cache/org/clojure/clojure/1.10.3/clojure-1.10.3.jar"
}
clojure.lang.Symbol {:hash "
5dt5kPSaDo8cr8XWT3RvHd3WYHqg4t"
:jar "
/mnt/mvn-cache/org/clojure/clojure/1.10.3/clojure-1.10.3.jar"
}
159 more elided}
:blocks [{:file "
book.clj"
:form (ns nextjournal.clerk.book {:nextjournal.clerk/toc true} (:require [clojure.string :as str] [next.jdbc :as jdbc] [nextjournal.clerk :as clerk] [nextjournal.clerk.parser :as parser] [nextjournal.clerk.eval :as eval] [nextjournal.clerk.analyzer :as ana] [nextjournal.clerk.viewer :as v] [sicmutils.env :as sicm] [weavejester.dependency :as dep]) (:import (javax.imageio ImageIO) (java.net URL))) :freezable? true :id nextjournal.clerk.book/anon-expr-5ds7nTyPZorGYPMsut457N1V5BoaBq :loc {:column 1 :line 2} :text "
^{:nextjournal.clerk/visibility {:code :hide}}↩︎(ns nextjournal.clerk.book↩︎ {:ne473 more elided"
:type :code :visibility {:code :hide :result :hide}}
{:file "
book.clj"
:form (def clojure-data {:hello "
world 👋"
:tacos (map (fn* [1 more elided] 1 more elided) (3 more elided)) :zeta "
The↩︎purpose↩︎of↩︎visualization↩︎is↩︎insight,↩︎not↩︎pictures."
})
:freezable? true :id nextjournal.clerk.book/clojure-data :loc {2 more elided} :text "
(def clojure-data↩︎ {:hello "world 👋"↩︎ :tacos (map #(repeat % '🌮) (range 1 378 more elided"
:type :code :var nextjournal.clerk.book/clojure-data :vars #{1 more elided} :visibility {2 more elided}}
{8 more elided} {10 more elided} {8 more elided} {8 more elided} {8 more elided} {8 more elided} {8 more elided} {8 more elided} {8 more elided} {8 more elided} {8 more elided} {8 more elided} {8 more elided} {8 more elided} {8 more elided} {8 more elided} {8 more elided} {8 more elided} 63 more elided]
:file "
book.clj"
:graph {2 more elided} :ns #object[clojure.lang.Namespace 0x7e8b1274 "
nextjournal.clerk.book"
]
:ns? true :toc-visibility true}

This analysis is done recursively, descending into all dependency symbols.

(ana/find-location 'nextjournal.clerk.analyzer/analyze-file)
"
/root/.gitlibs/libs/io.github.nextjournal/clerk/2a1e5c573f3c71fa30ee1f5d6edcab0743 more elided"
(ana/find-location `dep/depend)
nil
(ana/find-location 'io.methvin.watcher.DirectoryChangeEvent)
"
/mnt/mvn-cache/io/methvin/directory-watcher/0.15.0/directory-watcher-0.15.0.jar"
(ana/find-location 'java.util.UUID)
nil
(let [{:keys [graph]} analyzed]
(dep/transitive-dependencies graph 'nextjournal.clerk.book/analyzed))
#{nextjournal.clerk.analyzer/build-graph nextjournal.clerk.book/parsed nextjournal.clerk.parser/parse-file}

🪣 Hashing

Then we can use this information to hash each expression.

(def hashes
(:->hash (ana/hash analyzed)))
{URL "
5dsgvkMmMLJgogxVktTjrTwjJ4afJa"
clojure.lang.AReference "
5dtVT9HVuLAa4BCJBWwz9KQ2b9u37n"
clojure.lang.Compiler "
5dtVT9HVuLAa4BCJBWwz9KQ2b9u37n"
clojure.lang.LazySeq "
5dtVT9HVuLAa4BCJBWwz9KQ2b9u37n"
clojure.lang.LockingTransaction "
5dtVT9HVuLAa4BCJBWwz9KQ2b9u37n"
clojure.lang.Namespace "
5dtVT9HVuLAa4BCJBWwz9KQ2b9u37n"
clojure.lang.Numbers "
5dtVT9HVuLAa4BCJBWwz9KQ2b9u37n"
clojure.lang.PersistentHashMap "
5dtVT9HVuLAa4BCJBWwz9KQ2b9u37n"
clojure.lang.RT "
5dtVT9HVuLAa4BCJBWwz9KQ2b9u37n"
clojure.lang.Symbol "
5dtVT9HVuLAa4BCJBWwz9KQ2b9u37n"
159 more elided}

🗃 Cached Evaluation

Clerk uses the hashes as filenames and only re-evaluates forms that haven't been seen before. The cache is using nippy.

(def rand-fifteen
(do (Thread/sleep 10)
(shuffle (range 15))))
[13 14 8 6 5 12 11 9 1 2 3 10 0 7 4]

We can look up the cache key using the var name in the hashes map.

(when-let [form-hash (get hashes 'nextjournal.clerk.book/rand-fifteen)]
(let [hash (slurp (eval/->cache-file (str "@" form-hash)))]
(eval/thaw-from-cas hash)))
[0 5 1 11 10 14 8 3 2 12 4 9 6 13 7]

As an escape hatch, you can tag a form or var with ::clerk/no-cache to always re-evaluate it. The following form will never be cached.

^::clerk/no-cache (shuffle (range 42))
[17 11 5 29 15 13 31 40 6 41 0 25 14 28 35 33 3 10 26 34 22 more elided]

For side effectful functions that should be cached, like a database query, you can add a value like this #inst to control when evaluation should happen.

(def query-results
(let [_run-at #_(java.util.Date.) #inst "2021-05-20T08:28:29.445-00:00"
ds (next.jdbc/get-datasource {:dbtype "sqlite" :dbname "chinook.db"})]
(with-open [conn (next.jdbc/get-connection ds)]
(clerk/table (next.jdbc/execute! conn ["SELECT AlbumId, Bytes, Name, TrackID, UnitPrice FROM tracks"])))))
:tracks/AlbumId
:tracks/Bytes
:tracks/Name
:tracks/TrackId
:tracks/UnitPrice
111170334For Those About To Rock (We Salute You)10.99
25510424Balls to the Wall20.99
33990994Fast As a Shark30.99
34331779Restless and Wild40.99
36290521Princess of the Dawn50.99
16713451Put The Finger On You60.99
17636561Let's Get It Up70.99
16852860Inject The Venom80.99
16599424Snowballed90.99
18611245Evil Walks100.99
16566314C.O.D.110.99
18596840Breaking The Rules120.99
16706347Night Of The Long Knives130.99
18817038Spellbound140.99
410847611Go Down150.99
47032162Dog Eat Dog160.99
412021261Let There Be Rock170.99
48776140Bad Boy Boogie180.99
410617116Problem Child190.99
412066294Overdose200.99
3483 more elided