S6 Made Easy, with the S6 Overlay

This past December I wrote about s6, a small process supervisor that works very, very well in Docker containers. It provides all the basic functionalities you need to run multiple processes in a Docker container, and has minimal requirements – it’s compatible with Ubuntu, CentOS, and even Busybox.

One of the strengths of s6 is it’s very modular – it’s made up of many small, single-purpose components. There is a double-edged sword aspect to this: getting an s6-based system up and running requires a fair amount of work.

Laurent Bercot (the author of s6), Gorka Osa, and myself have since started a project to create a portable, easy-to-use supervision system, for containers, based around s6 – the s6-overlay project.

The Cool Demo

Before I get into details, let me show you a demo of the s6-overlay in action!

First, create a Dockerfile:

FROM ubuntu
ADD https://github.com/just-containers/s6-overlay/releases/download/v1.9.1.3/s6-overlay-amd64.tar.gz /tmp/
RUN tar xzf /tmp/s6-overlay-amd64.tar.gz -C /
RUN apt-get update &&
apt-get install -y nginx &&
echo "daemon off;" >> /etc/nginx/nginx.conf
ENTRYPOINT ["/init"]
CMD ["nginx"]

Build the image, start a container, and inspect the running processes:

$ docker build -t s6demo .
$ docker run --name s6demo -d -p 80:80 s6demo
$ docker top s6demo acxf -o pod,cmd
PID CMD
2095 _ s6-svscan
2134 | _ foreground
2139 | | _ foreground
2186 | | _ nginx
2187 | | _ nginx
2188 | | _ nginx
2189 | | _ nginx
2190 | | _ nginx
2135 | _ s6-supervise

You’re now running NGINX with a process supervisor! If NGINX dies for some reason, the container will stop, just as if you had run NGINX directly. But now if NGINX spawns a process (say one of the child processes fails), the s6-svscan program will clean up afterward.

The s6-overlay can take any provided CMD argument and run it under the supervision tree, so you can even run bash with this same image:

$ docker run --rm -ti s6demo /bin/bash
[fix-attrs.d] applying owners & permissions fixes...
[fix-attrs.d] 00-runscripts: applying...
[fix-attrs.d] 00-runscripts: exited 0.
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
root@032bf0dd24bf:/# ps acxf -o pid,cmd
PID CMD
1 s6-svscan
17 foreground
22 _ foreground
69 _ bash
80 _ ps
18 s6-supervise
root@032bf0dd24bf:/# exit
exit
/bin/bash exited 0
[cont-finish.d] executing container finish scripts...
[cont-finish.d] done.
[s6-finish] syncing disks.
[s6-finish] sending all processes the TERM signal.
[s6-finish] sending all processes the KILL signal and exiting.
$

Basically, an s6-overlay image is made by doing the following:

1. Download the latest release tarball
2. Extract the tarball to the root of your image
3. Set your image’s ENTRYPOINT to /init
4. Create service directories, or just set CMD to a program you want supervised.

Needed Dependencies

None! The overlay is distributed as a single tarball. All binaries are statically compiled (so they’ll run on any distro) and all scripts only reference binaries found in the tarball.

Getting It Into Your Image

There are two main ways to get the tarball into your image.

Method 1 – ADD and RUN

Use Docker’s ADD command to download the tarball, and a RUN command to extract it. This does add tar and gzip to the requirements for your base image.

ADD https://github.com/just-containers/s6-overlay/releases/download/v1.9.1.3/s6-overlay-amd64.tar.gz /tmp/
RUN tar xzf /tmp/s6-overlay-amd64.tar.gz -C /
ENTRYPOINT ["/init"]

Method 2 – Copy

Download and extract the tarball into some folder, then use Docker’s COPY command to add it to your image. This will allow you to build micro-images without tar or gzip.

$ curl -R -L -O https://github.com/just-containers/s6-overlay/releases/download/v1.9.1.3/s6-overlay-amd64.tar.gz
$ mkdir s6-overlay
$ tar xzf s6-overlay-amd64.tar.gz -C s6-overlay
$ cat Dockerfile
FROM ubuntu
COPY s6-overlay /
ENTRYPOINT ["/init"]

Using It

There are two methods for adding a service to s6-overlay:

Method 1 – Docker’s CMD

If your service doesn’t need any setup at runtime, you can just use Docker’s CMD directive. One of the goals of the s6-overlay is to still fit in with other Docker images – so when your CMD quits or crashes, the entire container comes down with it.

Method 2 – Create service directories

The s6-overlay has a folder at /etc/services.d for you to create service directories at. Each service should be a directory, and each directory should have a run and finish script. So, using NGINX as an example, you’d have the following tree:

etc
`-- services.d
`-- nginx
|-- finish
`-- run

The run and finish scripts can be written in any language you want – shell script, Laurent’s execline, perl, python – whatever you prefer.

Each service is started with a completely blank environment, and it’s up to you to setup the environment variables. When the container starts up, all the environment variables are saved to /var/run/s6/container_environment/. The s6-overlay has a script to re-import all of those environment variables – with-contenv. It loads the environment variables, then chainloads into another program – so I recommend just including it in your run script’s shebang (#!) line:

#!/usr/bin/with-contenv perl
use Data::Dumper;
print Dumper %ENV;

From there, the run script is pretty much up to you. The only requirement is that your program can’t fork, and you should exec your way into the program. So if you had NGINX setup with the default configuration, the following wouldn’t work:

#!/usr/bin/with-contenv sh
nginx

But the following would:

#!/usr/bin/with-contenv sh
exec nginx -g 'daemon off;'

exec is necessary to make sure nginx receives signals from s6.

finish scripts can be any symlinks to /bin/true if your service doesn’t need any clean-up – but make sure at least one service has a script that will stop the container.

Being a good Docker citizen

When you use service directories, you should pick a “key” service (or group of services), that should make the container stop. Most existing Docker images run a single process and exit when the process exits. Your image should be no different.

Going with my NGINX example, my finish script would be something like:

#!/bin/sh
s6-svscanctl -t /var/run/s6/services

This will instruct s6-svscan to bring down all services.

Extras

The s6-overlay has a few neat extra features, like running scripts before starting any services, and running scripts after ending all services:

Running scripts before bringing up services

This is a really handy feature – I have existing scripts where my run script is 100+ lines long, which really isn’t ideal – I have these big scripts running anytime a service restarts. The bulk of my run scripts could be run once, when the container starts up.

Just place scripts in /etc/cont-init.d, and they’ll be run in order (by filename) before starting any scripts. This is where you should be placing your setup scripts.

Running scripts after bringing down services

This is very similar to using /etc/cont-init.d – place scripts in /etc/cont-finish.d, and they’ll be run after bringing down all services, but before bringing down the container.

Using signals besides TERM

By default, s6 sends the TERM signal to a supervised process to bring it down – but some programs use different signals to shutdown. For example, consul uses INT to perform a graceful shutdown.

As of this writing, Laurent is working on a solution, but in the meantime you can do this:

#!/usr/bin/with-contenv sh

pid=$$

sigterm() {
kill -INT $pid
}

trap sigterm SIGTERM

consul &
pid=$!
wait

Further Reading

s6-overlay includes a ton of useful, simple tools from Laurent’s execline, s6, and s6-portable-utils packages. I highly suggest going through and reading their documentation, there’s a ton of great utilities you can use for bootstrapping your services.

Conclusion

I’m hoping you found this useful! I’ve been using s6 for sometime in my own images, but I’m planning on using s6-overlay as a sort of “best practices” approach to building images. If you have any questions, comments, or suggestions, please use the comments box below!

About John Regan

John Regan is a System Administrator for the Oklahoma Mesonet, a world-class network of environmental monitoring stations, a joint project of the University of Oklahoma and Oklahoma State University.

‹ Guide to Docker on OS X

Tutum Now Supports CentOS ›

Tagged with: s6
Posted in Tutorial