lthms' avatar, a hand drawing looking person, wearing a headset, close to a window on a raining night

Hi, I’m Thomas Letan (lthms) (he/him).

You read something which caught your attention and you are interested in starting a discussion? Don’t hesitate to shoot me an email.

 Published on  Modified on

A Literate Toolchain To Build This Website

A literate program is a particular type of software program where code is not directly written in source files, but rather in a text document as code snippets. In essence, literate programming allows for writing in the same place both the software program and its technical documentation.

cleopatra is a “literate toolchain” I have implemented to build this website, and you are currently reading itThis sentence was true when this article was published, but things have changed since then. What you are reading is actually the rendered version of a Markdown document that was manually “translated” from the Org original document, named Interested readers can have a look at the original version here . Truth be told, said version is probably complete gibberish for anyone who isn’t me. For this reason, this version was actually heavily reworked… Because I have too much free time, probably. . That is, cleopatra is both the build system and an article of this website! To achieve this, cleopatra has been written as a collection of org files which can be either “tangled” using Babel  or “exported” as a HTML document. Tangling here refers to extract marked code blocks into files.

The page you are currently reading is cleopatra entry point. Its primary purpose is to define two Makefiles —makefile and— and the necessary emacs-lisp script to tangle this document.

On the one hand, makefile is the main entrypoint of cleopatra. It serves two purposes:

  1. It initiates a few global variables, and
  2. It provides a rule to tangle this document, that is to update itself and

On the other hand, is used to declare the various “generation processes” used to generate this website.

makefile and the emacs-lisp scripts are versioned, because they are necessary to bootstrap cleopatra; but since they are also defined in this document, it means cleopatra can update itself, in some sense. This is to be kept in mind when modifying this document to hastily.

Global Constants and Variables

First, makefile defines several global “constants”As far as I know, make does not support true constant values, It is assumed generation processes will not modify them. . In a nutshell,

For this website, these constants are defined as followsI will use a comment in the first line to recall to which file a given block code is expected to be tangled. .

# makefile:
ROOT := $(shell pwd)
CLEODIR := site/cleopatra

We then introduce two variables to list the output of the generation processes, with two purposes in mind: keeping the .gitignore up-to-date automatically, and providing rules to remove them.

# makefile:
ARTIFACTS := build.log

clean :
	@rm -rf ${ARTIFACTS}

cleanall : clean
	@rm -rf ${CONFIGURE}

Generation processes can declare new build outputs using the += assignement operators. Using another operator will likely provoke an undesirable result.

Tangling Org Documents

cleopatra is a literate program implemented with Org mode, an Emacs major editing mode. We provide the necessary bits to easily tangle Org documents.

The configuration of Babel is done using an emacs lisp script called tangle-org.el whose status is similar to Makefile. It is part of the bootstrap process, and therefore lives “outside” of cleopatra (it is not deleted with make clean for instance). However, it is overwritten when this file is tangled. If you try to modify it and find that cleopatra does not work properly, you should restore it.

;;; tangle-org.el:
(require 'org)
(cd (getenv "ROOT"))
(setq org-confirm-babel-evaluate nil)
(setq org-src-preserve-indentation t)
(add-to-list 'org-babel-default-header-args
             '(:mkdirp . "yes"))
 '((shell . t)))

We define variables that ensure that the $ROOT environment variable is set and tangle-org.el is loaded when using Emacs.

# makefile:
EMACSBIN := emacs
TANGLE := --batch \
          --load="${ROOT}/scripts/tangle-org.el" \
          2>> build.log

Finally, we introduce a canned recipe  to seamlessly tangle a given fileIt was the first time I had used canned recipes, and I don’t think I had the opportunity to re-use it ever again. .

# makefile:
define emacs-tangle =
echo "  tangle  $<"

Updating .gitignore Automatically

Assuming each generation process correctly defines its $ARTIFACTS and $CONFIGURE variables, we have all the information we need to update .gitignore automatically.

This is done by adding markers in .gitignore to define a region under the control of cleopatra, and writing a script to update said region after each build.

BEGIN_MARKER="# begin generated files"
END_MARKER="# begin generated files"

# remove the previous list of generated files to ignore
sed -i -e "/${BEGIN_MARKER}/,/${END_MARKER}/d" .gitignore
# remove trailing empty lines
sed -i -e :a -e '/^\n*$/{$d;N;};/\n$/ba' .gitignore

# output the list of files to ignore
echo "" >> .gitignore
echo ${BEGIN_MARKER} >> .gitignore
for f in $@; do
    echo "${f}" >> .gitignore
echo ${END_MARKER} >> .gitignore

The ignore rule of makefile is defined as follows.

# makefile:
ignore :
	@echo "  update  gitignore"
	@scripts/ \
	   ${ARTIFACTS} \


The core purpose of makefile remains to bootstrap the chain of generation processes. This chain is divided into three stages: prebuild, build, and postbuild.

This translates as follows in makefile.

# makefile:
default : postbuild ignore

init :
	@rm -f build.log

prebuild : init

build : prebuild

postbuild : build

.PHONY : init prebuild build postbuild ignore

A generation process in cleopatra is a Makefile which provides rules for these three stages, along with the utilities used by these rules. More precisely, a generation process proc is defined in The rules of for each stage are expected to be prefixed by proc-, e.g., proc-prebuild for the prebuild stage.

Eventually, the following dependencies are expected between within the chain of generation processes for every generation process.

prebuild : proc-prebuild
build : proc-build
postbuild : proc-postbuild

proc-build : proc-prebuild
proc-postbuild : proc build

cleopatra is a literate toolchain whose main purpose is to allow me to turn the scripts I wrote to generate my website into blogposts of said website. As such, it allows me to implement the generation processes using Org mode, which means that before being able to start generating HTML files, cleopatra has to tangle the generation processes.

To achieve this, cleopatra relies on a particular behavior of make regarding the include directive. If there exists a rule to generate a Makefile used as an operand of include, make will use this rule to update (if necessary) said Makefile before actually including it.

Therefore, the rules of the following form achieve our ambition of extensibility.

include ${PROC}.mk

prebuild : ${PROC}-prebuild
build : ${PROC}-build
postbuild : ${PROC}-postbuild

${PROC}-prebuild : ${PROC}.mk ${AUX}
${PROC}-build : ${PROC}-prebuild
${PROC}-postbuild : ${PROC}-build

${PROC}.mk ${AUX} &:\


.PHONY : ${PROC}-prebuild \
         ${PROC}-build \


We use &: is used in place of : to separate the target from its dependencies in the “tangle rule.”Yet another obscure Makefile trick I have never encountered again. This tells make that the recipe of this rule generates all these files.

Rather than writing these rules manually for each generation process we want to define, we rely on to noweb of Babel . We call extends the primitive to generate new generation processes.

We derive the rule to tangle using extends, using the following Org mode syntax.

#+BEGIN_SRC makefile :noweb yes
# makefile:
<<extends(IN="", PROC="bootstrap", AUX="scripts/")>>

which gives the following result.


prebuild : bootstrap-prebuild
build : bootstrap-build
postbuild : bootstrap-postbuild

bootstrap-prebuild : scripts/
bootstrap-build : bootstrap-prebuild
bootstrap-postbuild : bootstrap-build scripts/ &:\

CONFIGURE += scripts/

.PHONY : bootstrap-prebuild \
         bootstrap-build \

These are the last lines of makefile. The rest of the generation processes will be declared in

Generation Processes

In this section, we construct by enumerating the generation processes that are currently used to generate the website you are reading.

We recall that each generation process shall

  1. Define proc-prebuild, proc-build, and proc-postbuild
  2. Declare dependencies between stages of generation processes
  3. Declare build outputs (see ARTIFACTS and CONFIGURE)

Theming and Templating

The theme  generation process controls the general appearance of the website. More precisely, it introduces the main template used by soupault (main/templates.html), and the main SASS sheet used by this template.

If a generation process produces a set of styles within a specific SASS files, the current approach is

  1. To make this file a dependency of theme-build
  2. To modify style/main.sass in theme to import this file

Eventually, the second step will be automated, but, in the meantime, this customization is mandatory.

Configuring Soupault

The soupault  generation configures and run soupault, in order to generate a static website.

If a generation process proc produces files that will eventually be integrated to your website, its proc-build recipe needs to be executed before the soupault-build recipe. This can be enforced by making the dependency explicit to make, i.e.,

soupault-build : proc-build

Eventually, generation processes shall be allowed to produce specific soupault widgets to be integrated into soupault.conf.

Authoring Contents

The fact that cleopatra is a literate program which gradually generates itself was not intended: it is a consequence of my desire to be able to easily use whatever format I so desire for writing my contents, and Org documents in particular.

In the present website, contents can be written in the following format: