Give Them The Boot

Presenter Notes

Give Them The Boot

by Josh Johnson, aka jjmojojjmojo

Presenter Notes

About Josh

  • Dislikes cleverness.
  • Hates magic, as it requires the sacrifice of unicorns.
  • Subscribes to the idea that explicit is better than implicit.
  • Compulsive question asker.

Presenter Notes

Background

I've had experience with many build tools.

  • Make
  • Ant
  • Buildout
  • Crushinator

Presenter Notes

All have their charm.

Make

General purpose task-definition and execution language.

Ant

XML-driven, java-centric build platform utilizing tasks.

Buildout

Python isolation + make-like task manager.

Crushinator

Personal project. Python framework to build code skeleton generators.

Presenter Notes

And They All Have Their Downsides

Make

Somewhat esoteric syntax.

Ant

XML. Classpath nightmares.

Buildout

INI file syntax is cumbersome.

Crushinator

Frameworks and boilerplate are just bad.

Presenter Notes

What About Leiningen?

More on that... later...

images/lein.jpg

Presenter Notes

Enter The Boot

Presenter Notes

What Is Boot?

Boot incorporates concepts from all of the aforementioned build tools, and is chiefly used to implement build pipelines, but there's fundamental difference:

images/boot-logo.png

Presenter Notes

Boot Is Not A Build Tool.

Presenter Notes

Boot Is A Toolbox Of Awesome

Boot provides a rich set of components and abstractions that we, as clojurists, can use to easily construct complex tooling, and other amazing things.

Presenter Notes

Boot is not (too) magical.

Nothing in boot is free, but...

Boot abstracts the difficult stuff, no unicorn sacrifice required.

Presenter Notes

Boot is Clojure!

Interacting with boot is like second nature:

  • No DSLs (well, one, but it's useful and optional)
  • No esoteric languages.
  • No boilerplate.
  • No hokey project concept.
  • Works from the REPL.

For a clojurist, it's just doing more of what we already do: CLOJURE.

images/clojure-and-boot.png

Presenter Notes

This sounds great, but what does it buy me?

Presenter Notes

The Boot Giveth Unto Us

Boot provides some absolutely killer features. Some are borrowed from or inspired by other tools, some solve specific problems. All are implemented in direct, sane ways.

  • Composible tasks.
  • Imutable filesystem isolation.
  • Imutable classpath isolation.
  • Clojure scripting.

All of these components and their ancillary code can be used by any clojure project, even outside the realm of tooling or a build pipeline.

Presenter Notes

Tasks

Presenter Notes

Tasks Are Code

Each task is a simple clojure function.

Easy to use macros are provided.

Simple, yet sophisticated command-line option DSL is parsed to provide a consistent user interface.

Presenter Notes

Immutability In Boot

In boot, nothing happens in real life until a task is complete. This means:

  • files that are generated by tasks are written to temporary directories.
  • a file set is copied as it is passed from task to task, maintaining autonomy and enforcing immutability.
  • the file set is committed to disk when the task is complete.
  • destination files and "live" state are not manipulated until the tasks say so - there's no need to "clean up" the destination files upon error.
  • the risk of inconsistent state is greatly minimized.

Presenter Notes

Tasks Are Composible

Tasks can form a pipeline. Nothing is shared except the file system abstraction. The environment is isolated from one task to the next.

Each task can implement middleware to manipulate the file set or do other things on behalf of the next, or previous, task.

As such, tasks can be primary actors, manipulators, or just affect the pipeline itself.

Presenter Notes

Tasks Take CLI Arguments

Boot provides a simple DSL that you can use in your task definition to take command line arguments.

Presenter Notes

Filesystem Abstraction

Presenter Notes

Nothing Is Shared

As touched on earlier, boot has a concept of immutable file sets.

This model allows strict isolation between tasks.

Presenter Notes

Classpath Isolation

Presenter Notes

Even The Runtime Isn't Shared!

Boot provides the concept of pods, which do some fancy footwork to provide a clean classpath within which arbitrary code can be executed.

This makes it possible to execute code with different versions of libraries loaded, without resorting to managing multiple JVMs.

Presenter Notes

Boot Scripting

Presenter Notes

The Killerest Feature Of Them All

Boot allows for writing clojure scripts.

The script declares its own dependencies, and boot manages downloading them.

Scripts can exist anywhere, be named anything, and require no other system setup beyond a JVM and boot itself.

Presenter Notes

Seriously, This Is Amazing

Boot scripting elevates clojure to the level of simplicity and utility of a score of non-jvm languages, such as Python, Ruby and Perl.

But Boot goes further:

  • Provides dependency management.
  • Intuitive CLI option handling.
  • Pod/Fileset access for extreme autonomy and idempotence.

Presenter Notes

I'm So, So Seriously, You Guys

Boot scripting means distributing applications in single-file, self-contained scripts.

  • Easily managed via source control.
  • Separated from library dependencies.
  • Encourages mixing and remixing of scripts.
  • Can replace shell scripting outright
  • You never have to write anything but clojure.
  • Java interop is at your fingertips.

Presenter Notes

I'm Sold! Enough talk, how do I get booting?

Presenter Notes

Wait, you said you were going to talk about Leiningen!

Presenter Notes

Why Another Tool?

Boot has its roots in clojurescript development, originating with the hoplon project.

Clojurescript, especially with hoplon in the mix, can lead to some very complex build pipelines.

Futzing with Lein plugins is messy and overly complex.

Hence, something simpler was needed to address specific problems.

And so boot was born.

Presenter Notes

Why Is Boot Better?

If I don't care about clojurescript, why should I care about boot?

The simple answer: boot is engineered to be better by design. Period.

How so, you may ask?

  • Boot is a suite of components, not a build tool.
  • Boot solves some very hard problems in straight forward ways.
  • Boot doesn't dictate you how to work.
  • Boot doesn't mess with your environment.
  • Boot doesn't require any boilerplate. No projects, no edn, nothing.

But most importantly:

  • Boot encourages you to solve your own problems.
  • Boot is (mostly) magic-free. Everything is on the table. Nothing "just happens".

Presenter Notes

That sounds mean!

But it's not! We all love Leiningen! It's what we know!

Here are the simple facts:

  • Leiningen is a de facto standard. There's no reason to settle for it if it doesn't meet your needs.
  • Leiningen filled a very essential and necessary void in the clojure community.
  • We wouldn't be where we are now, as a community, as a platform, without it.

But when you love something deeply enough, you are able to see its flaws.

Boot is simply a reaction to those flaws, not a condemnation of the tool or what it's done for us.

Presenter Notes

Seriously, enough talk, how do I get booting?

Presenter Notes

Installation

Installation is extra simple. Just download the latest boot executable, and put it somewhere where you can access it.

Linux/OS X

$ wget https://github.com/boot-clj/boot/releases/download/2.0.0/boot.sh
$ mv boot.sh boot && chmod a+x boot && sudo mv boot /usr/local/bin

Windows

C:\> wget https://github.com/boot-clj/boot/releases/download/2.0.0/boot.exe
C:\> move boot.exe C:\Windows\System32

Presenter Notes

Interacting With Boot

Now that we have the boot executable, we can ask it for help with the -h flag:

$ boot -h

Usage:   boot OPTS <task> TASK_OPTS <task> TASK_OPTS ...

OPTS:    -a --asset-paths PATH      Add PATH to set of asset directories.
         -b --boot-script           Print generated boot script for debugging.
         -B --no-boot-script        Ignore boot script in current directory.
         -C --no-colors             Remove ANSI escape codes from printed output.
         -d --dependencies ID:VER   Add dependency to project (eg. -d foo/bar:1.2.3).
         -e --set-env KEY=VAL       Add KEY => VAL to project env map.
         -h --help                  Print basic usage and help info.
         -P --no-profile            Skip loading of profile.boot script.
         -r --resource-paths PATH   Add PATH to set of resource directories.
         -q --quiet                 Suppress output from boot itself.
         -s --source-paths PATH     Add PATH to set of source directories.
         -t --target-path PATH      Set the target directory to PATH.
         -u --update                Update boot to latest release version.
         -v --verbose               More error info (-vv more verbose, etc.)
         -V --version               Print boot version info.

Presenter Notes

Interacting With Boot Tasks

The output on the previous slide is truncated for the sake of brevity - there are also entries for each task, and useful information about environment variables and configuration files that boot can utilize.

To get help with a specific task, you can pass the -h flag to it directly:

$ boot aot -h
Perform AOT compilation of Clojure namespaces.

Options:
  -h, --help          Print this help info.
  -a, --all           Compile all namespaces.
  -n, --namespace NS  Conj NS onto the set of namespaces to compile.

Presenter Notes

Helpful Hints

Ambiguous task options can be delineated with --

$ boot aot -n boo -n help -- pom jar

As stated before, boot tasks are composable. Each task specified becomes part of the pipeline:

$ boot -s "." show -f
.nrepl-history
build.boot
presentation.html
slides.rst
$ boot -s "." sift -v -i "presentation.html" show -f
.nrepl-history
build.boot
slides.rst

Presenter Notes

Helpful Hints

Some values are complex. Most are hinted at in the help output.

KEY:VAL indicates a map. The key and value are separated by a colon (:). Each additional use of that command-line parameter will conjoin the key and value.

KEY=VAL indicates a map as well, but the key will end up being a clojure keyword.

Most options that are plural can be supplied multiple times (e.g. --source-paths)

Presenter Notes

build.boot

Boot has a concept analogous to the Makefile in Make, except that it is also a place to set default values for command-line options.

Boot settings and task definitions are placed in build.boot.

Boot looks for this file in the current working directory.

All settings within can be provided via command-line options as well.

Presenter Notes

Boot Environment

Boot has the concept of an environment, which amounts to a singleton map of boot-specific settings.

The build.boot file creates a default namespace, named boot.user.

By default, most of boot.core is automatically imported into that namespace on your behalf.

The environment can be manipulated with the set-env! function in build.boot, or by various command-line arguments to the boot executable.

Presenter Notes

build.boot Example: Environment

Here we will declare a dependency, and run a repl.

In build.boot:

1
2
(set-env!
  :dependencies '[[me.raynes/fs "1.4.6"]])

In our shell

$ boot repl
Retrieving fs-1.4.6.jar from https://clojars.org/repo/
Retrieving xz-1.5.jar from https://repo1.maven.org/maven2/
Retrieving commons-compress-1.8.jar from https://repo1.maven.org/maven2/
boot.user=> (require '[me.raynes.fs :refer [list-dir name]])
boot.user=> (map #(name %1) (list-dir "."))
(".nrepl-history" ".nrepl-port" "build" "presentation" "slides")

Presenter Notes

build.boot Example: Environment

The previous build.boot example is equivalent to the following boot command-line:

$ boot -d "me.raynes/fs:1.4.6" repl

Presenter Notes

Example Tasks

Presenter Notes

The Simplest Task

Consider the following build.boot file:

(set-env!
  :dependencies '[[me.raynes/fs "1.4.6"]])

(ns boot.user
  (:require [me.raynes.fs :as fs]))

(deftask simple
  "Simple example task"
  []
  (prn (map #(fs/name %1) (fs/list-dir "."))))

Presenter Notes

The Simplest Task

If we now run boot -h, we will see it in the list of tasks:

$ boot -h
...
          simple                     Simple example task
...

We can also ask for help with our new task:

$ boot simple -h
Simple example task

Options:
  -h, --help  Print this help info.

Presenter Notes

The Simplest Task

This task doesn't interfere with anything in the pipeline. It doesn't produce or process files.

However, it can still be composed with other tasks:

$ boot -s "." simple show -f
("#slides" ".nrepl-history" "build" "build" "presentation" "slides")
.nrepl-history
build.boot
presentation.html
slides.rst

Presenter Notes

Task Option DSL

Boot provides a powerful DSL for processing command line arguments for tasks.

As part of the deftask macro, you specify the command-line arguments as specially formatted function arguments:

1
2
3
4
5
(deftask cli-example
  "This is the help text for this task"
  [f foo FOO str "The foo option."
   b bar int "The bar option"
   c compound KEY:VAL {kw str} "A compound option"])

Presenter Notes

Task Option DSL

Each column in the argument definition has a special purpose:

[f foo FOO str "The foo option."]
 ↑  ↑   ↑   ↑          ↑
 1  2   3   4          5
  1. is the short form of the argument -f
  2. is the long form and the name of the variable that will hold the value --foo
  3. indicates this option takes an argument, --foo FOO. The text here will be presented to the user in the help text.
  4. tells the DSL how to marshal the data - all information provided on the command line is a string, this indicates what clojure type the value will be converted to.
  5. is the help text for the specific option.

Presenter Notes

Task Option DSL

Returning to our original example:

1
2
3
4
5
(deftask cli-example
  "This is the help text for this task"
  [f foo FOO str "The foo option."
   b bar int "The bar option"
   c compound KEY:VAL {kw str} "A compound option"])

We see that this DSL defines three options:

--foo, which stores a string

--bar, which increments an integer

--comound, which constructs a map of strings, indexed by keywords.

Presenter Notes

Super Awesome Bonus

When writing your own boot scripts, or any CLI tool, you can utlize the task option DSL for any function, using the boot.cli/defclifn macro!

Presenter Notes

Taking CLI Options: Magic Vars

The deftask macro processes our argument DSL and gives us two variables: *args and *opts*.

*args* is a sequence of positional arguments (not used in task definitions)

*opts* is a map of options/flags.

(deftask cli-example
  "This is the help text for this task"
  [f foo FOO str "The foo option."
   b bar int "The bar option - incrementer"
   c compound KEY:VAL {kw str} "A compound option"]

   (prn *opts*)
   (prn *args*))

Presenter Notes

Taking CLI Options: What You Get

Running the example task now, we see in the help:

$ boot cli-example -h
This is the help text for this task

Options:
  -h, --help              Print this help info.
  -f, --foo FOO           Set the foo option to FOO.
  -b, --bar               Increase the bar option - incrementer
  -c, --compound KEY:VAL  Conj [KEY VAL] onto a compound option

And we can see what it looks like with a few options:

$ boot cli-example -f "hello" -bbbb -c hey:there -c hi:there -c ho:there
{:foo "hello", :bar 4, :compound {:ho "there", :hi "there", :hey "there"}}
[]

Presenter Notes

CLI FTW!

Boot's task option DSL provides many, many possibilities.

You can do some really amazing things with the boot task option DSL. It can save you a lot of time building a useful user interface.

For discussion of all of the different kinds of arguments, see Task Options DSL in the Boot Wiki.

Presenter Notes

Using The Fileset

Here is a task to uppercase all of the files in the fileset.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
(defn mv-uc
  "Does the heavy lifting for uc-filenames below"
  [fileset]
  (loop [files (:tree fileset) fs fileset]
  (if-let [[source fileobj] (first files)]
    (let [parts (string/split (str source) #"/")
          base (last parts) parent (butlast parts)
          dest (string/join "/" (concat parent [(string/upper-case base)]))]
       (recur
         (dissoc files source)
         (mv fs source dest)))
    fs)))

(deftask uc-filenames
  "Moves all of the files in fileset to upper-case versions"
  []
  (fn middleware [next-handler]
      (fn handler [fileset]
        (next-handler (mv-uc fileset)))))

Presenter Notes

Uppercase Fileset In Action

Before using our new task. Note that we have to specify a source directory. We'll use the git checkout of this repository (.):

$ boot -s . show -f
.git
├── HEAD
├── config
├── description
├── hooks

And with our task in the pipeline:

.git
├── CONFIG
├── DESCRIPTION
├── HEAD
├── INDEX
├── PACKED-REFS

Presenter Notes

Serve Static Files Within The Fileset

This task looks for any files, and serves them files over HTTP.

First, a simple ring application that will serve from a map of relative paths to absolute temporary paths:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
(defn mapper-app
  "Given a map of relative paths to temporary locations, serve
   the files within if they are requested"
  [mapping]
  (fn [request]
    (let [uri (subs (:uri request) 1)
          want (get mapping uri)]
      (if want
        (file-response (tmp-path want)
        (not-found "Not Found"))))))

Presenter Notes

Serve Static Files Within The Fileset

Next we'll build the task:

1
2
3
4
5
6
7
8
(deftask serve-source
  "Serve all files in the source tree"
  [p port PORT int "The port to listen on"]
  (fn middleware [next-handler]
    (fn handler [fileset]
      (jetty/run-jetty
        (mapper-app (:tree fileset))
        {:port (get *opts* :port 8080)}))))

Presenter Notes

Serve Static Files Within The Fileset

In order for this to work, we need to provide a source directory. Since we haven't specified one in our build.boot, we'll have to do so on the command line with the -s option:

$ bash -s . serve-source
2015-05-28 18:03:44.783:INFO:oejs.Server:jetty-7.6.13.v20130916
2015-05-28 18:03:44.806:INFO:oejs.AbstractConnector:Started SelectChannelConnector@0.0.0.0:8080

Now if we open up http://127.0.0.1:8080/presentation.html, we'll see this very presentation.

We'll also see something if we go to http://localhost:8080/.git/logs/HEAD

Presenter Notes

Chaining Can Help

Say we only want to serve .html files. We can use the sift task to filter the fileset:

$ boot -s . sift -i ".html$" serve-source

Now a request for http://localhost:8080/.git/logs/HEAD will return a 404.

Presenter Notes

Using Pods

Presenter Notes

Run A Ring Application With Pods

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
(deftask webserver-isolated []
"Run the web server in a pod"
(let [runner (pod/make-pod (get-env))]
  (pod/with-eval-in runner
    (def server false))
  (fn [next-task]
    (fn [fileset]
      (pod/with-eval-in runner
        (when server
          (do
            (println "Stopping server")
            (.stop server))))
      (pod/with-eval-in runner
        (require '[ring.adapter.jetty :as jetty]
                 '[my-code-here :refer [app]])
        (def server (jetty/run-jetty #'app {:port 8080 :join? false}))
        (println "Starting server")
        (.start server))))))

Presenter Notes

Working With Pipeline Modifiers

We can match our webserver task with the watch task to automatically reload the webserver whenever source files change:

$ boot watch webserver-isolated

Presenter Notes

Boot Scripts

Boot scripts are like any other shell script, except:

  • That they contain clojure code
  • They get a boot.user namespace by default
  • They have boot.core pre-loaded.
  • A -main function is defined, that is loaded when the script is run.
  • They have a boot environment that can declare dependencies.
  • You can specify source paths for code to load external to the script, and resources to add additional files to the classpath.

Presenter Notes

Boot Scripts: Minimal Example

This is the bare minimum boot script:

1
2
3
4
5
#!/usr/bin/env boot

(defn -main
  []
  (println "Hey There, Blimpy Boy"))

Presenter Notes

Boot Scripts: Minimal Example

By making this file executable, we can run it in the terminal:

$ chmod +x minimal-script
$ ./minimal-script
"Hey There, Blimpy Boy"

Presenter Notes

Scripting: What Next?

Essentially, the entire clojure world is at your fingertips with boot scripting!

Use it to Get Started With Clojure In < 10 Minutes.

Or do more Advanced Things, like distributing your scripts and ancillary data, building jars, running your own rudimentary maven repo, and post stuff to clojars.

Presenter Notes

Thanks

Thanks to the Boot project, Adzerk, Cognitect and all clojurists everywhere!

Presenter Notes

Image Credits

images/lein.jpg

"Winnie, the weiner dog, found the Twizzlers" http://imgur.com/H5pDMnu

images/boot-logo.png

"Boot Logo" http://boot-clj.com/

images/clojure-and-boot.png

"Boot And Clojire" http://boot-clj.com/

Presenter Notes