Benjamin Cable

Principal Engineer @ Lush

Golang Dorset Meetup Organiser


Self publishing with Julia

To inaugurate the writings section of this website, I will explain how I setup my own little CLI to help me create new posts, using Julia.

First I think I need to clarify why:

Every time I've tried to use a static site generator, I find myself questioning my need for one.

There are too many features I do not use, too many themes to consider, too many conventions. In the end, I like things to be dead simple, especially given my taste for rather spartan webpages. Hugo, for example, is fantastic, but I feel like I would be using 10% of its features, without really understanding them either.

I decided, then, that I would implement only the features I actually want, and that I would do this using Julia, since it's a language I've been interested in lately.

With that being said: I am aware that the implementation I came up with it primitive, somewhat dumb, and most likely doesn't do Julia any justice. It is, however, enough for the time being. I needed something to get me started, and this served as a sufficiently inoffensive exercise to get a taste for the language.

Starting small

The first thing I wanted was to setup somewhat of a CLI, that would contain usage information, and respond to command line flags, much like I would do it in Go or other languages.

First things first, I need an entry point, I setup Main.jl for this purpose:

#!/usr/bin/env julia

using Printf

include("Posts.jl")

const md = "--markdown"
const sitemap = "--sitemap"

const usage = """
        [cli] - Publishing Helper.

        commands:
        [$md]
                - Create a new post, using a Markdown file as input.
        
        [$sitemap]
                - Create a sitemap, reading the posts directory.
        """

"die - compact helper function for early exits"
die() = (println(usage);exit(2))

"dispatch - checks for a cli flag match, then call the appropriate method, or die"
function dispatch(s::String)
    if s == sitemap
        Posts.sitemap(ARGS)
    elseif s == md
        Posts.create_from_markdown(ARGS)
    else
        die()
    end
end

# entry point
if length(ARGS) > 0
    dispatch(ARGS[1])
else
    die()
end

There are not many interesting things about this file, so I shall be brief.

I define a usage notice in const usage = ... which is what I want to see printed if the program is ran without arguments.

One interesting thing to note about this however is that the preceding whitespace is ignored, and therefore not visible when printing.

as for die() = (...), it is simply an inlined, or compact function, it could have similarly been written using the long form syntax:

function die()
	# ...
end

However, you will notice that it is defined twice, making use of Julia's capacity for multiple dispatch.

Now, on to the more interesting part of this program: the Posts.jl module.

Creating new posts

First, I start by defining a Julia module:

module Posts

export create_from_markdown, sitemap

function create_from_markdown(arguments)
end

function sitemap(arguments)
end

end

This bare-bones definition is enough to setup the module and dispatch the functions. If you recall in the Main.jl file, I was sending ARGS as arguments to these two functions. ARGS is the special variable under which arguments are found when running the program. To avoid redefining/overriding it, I simply take it as arguments within those functions.

create_from_markdown

Here I want the following results:

When a new post is made, I would like it to be created in a folder structure which corresponds to the year/month/day at which it was made, e.g.,:

posts
└── 2018
    └── 12
        └── 16
            └── self_publishing_with_julia.html

I would also like the original markdown filename to be reused, but sanitised if needed (remove spaces, use underscores), and, finally, I would like this file to only be created once, no overwriting should happen. I want that action to be explicit, and not accidental.

This is the implementation I came up with:

using Dates
using Printf
using Markdown

export create_from_markdown, sitemap

const base = "posts"

function create_from_markdown(arguments)
	if length(arguments) > 1
		filename = join(arguments[2])
	end
	
	doc = Markdown.parse_file(filename)

	filename = replace(filename, ".md" => ".html")

	# convert the y, m, d Int64 tuple to strings
	y, m, d = [string(i) for i in yearmonthday(now())]
	# join the path
	path = joinpath(base, y, m, d)

	# create it if necessary
	mkpath(path)

	# touch the new file inside it
	
	file = joinpath(path, filename)
	
	if isfile(file)
		@printf("%s: already exists\n", filename)
		exit(1)
	end

	header = read("cli/header.html", String)
	body = html(doc)
	footer = read("cli/footer.html", String)

	document = header * body * footer

	write(file, document)
end

In total I've needed three modules from the standard library: Printf, Dates, and Markdown.

I've also defined a constant base for the posts folder, since I always want my files to go in there.

Perhaps one of the most interesting things in this file is the following list comprehension:

y, m, d = [string(i) for i in yearmonthday(now())]

The now() function exposed from the Dates module returns a DateTime object which prints like so: 2018-12-27T19:10:57.776.

Passing this object to yearmonthday results in a tuple of 3 Int64 numbers, on which I can cast to a string, using the list comprehension syntax, and multiple variables.

The rest of the implementation is very cookie-cutter, and writes my converted html file, along with added headers/footers for my specific use case in the right path.

sitemap

The last thing I wanted was a way to generate a sitemap. Creating new posts would be a hassle if I had to manually maintain a list of links.

In order to achieve this goal, I've come-up with this rather minimal implementation:

function sitemap(arguments)
	# ensure passed parameters are as-expected
	if length(arguments) < 2 || length(arguments) > 2
		println(stderr, "expected only one argument (directory to map)")
		exit(1)
	end
	dir = arguments[2]
	if !isdir(dir)
	    println(stderr, "expected a directory")
	    exit(1)
	end

	# define the sitemap base
	list = """<link rel="stylesheet" href="/css/main.css">
	<base target="_parent">
	<ul>"""
	
	close = "\n</ul>"

	posts = []
	for (root, dirs, files) in walkdir(dir)
		for file in files

		    # infer the file name
		    fname = begin
			    f = split(file, "_")
			    f = [uppercasefirst(x) for x in f]
			    f = join(f, " ")
			    f = replace(f, ".html" => "")
		    end

		    # create the html formatted link
		    list_item = @sprintf("\n\t<li><a href=\"%s\">%s</a></li>", joinpath(root, file), fname)

		    # push the items onto the posts array
		    push!(posts, list_item)
		end
	end

	# sort the items upside down
	#  -> latest post on top
	reverse!(posts)

	# append all items to the list
	for item in posts
	    list = list * item
	end

	# then close
	list = list * close

	write("map.html", list)
end

Nothing scary in here, I'm building a map.html file for later use as an iframe embed.

Note however the use of an interesting feature of Julia, compound expressions:

fname = begin
	f = split(file, "_")
	f = [uppercasefirst(x) for x in f]
	f = join(f, " ")
	f = replace(f, ".html" => "")
end

In this case, the value of fname will be the last value of the subexpression. This allows to cut down on a lot of cruft, while maintaining readability. I am not one for deeply embedded "clever" one-liners.

As a last thing, I sort my posts in reverse order, so the latest one appears on the top.

Once my map.html file is created, I embed it in my index.html file using the following tidbits:

The last little thing I did is to mask some of the invocations behind a Makefile:

THIS_FILE := $(lastword $(MAKEFILE_LIST))

post:
	./cli/Main.jl --markdown $(FILE)
	@$(MAKE) -f $(THIS_FILE) sitemap

sitemap:
	./cli/Main.jl --sitemap posts

The first line defining THIS_FILE is quite useful, and taken from a great response to a Stack Overflow thread.

With this setup FILE= ~/my_file.md make post is enough, and I am certain it is followed by make sitemap, saving me the trouble of doing that manually.


Home