Docker tmpfs

Tmpfs in docker

For security reasons I run most of my containers as read-only. That works well as long as you don’t need storage for some temporary files, for instance PID files (for persistent data volumes are used).

As an example let’s have a look at this Dockerfile

FROM alpine:edge
RUN apk --update --no-cache add prosody
COPY prosody.cfg.lua /etc/prosody/prosody.cfg.lua
EXPOSE 5222 5269
USER prosody
CMD ["prosodyctl", "start"]

We build the image with docker build -t learning/tmpfs ..

Now when we run the image using the --read-only flag:

docker run --rm --read-only -it learning/tmpfs

We get the following error

mod_posix error Couldn't write pidfile at /var/run/prosody/prosody.pid; /var/run/prosody/prosody.pid: Read-only file system

and the container shuts down.

So what to do now? One solution would be to create a volume and mount it to the directory /var/run/ or even /var/run/prosody. However, you don’t really need to persistently save the files created in /var/run because they are simply recreated when the system restarts (often PID files are stored in /var/run). So docker added the --tmpfs flag. Let’s try:

docker run \
       --rm \
       --read-only \
       --tmpfs=/var/run/prosody \
       -it learning/tmpfs

We still get the same error. What is going on here? Well, the directory does now exist, but it is only writable for the root user. Prosody, however, runs under the prosody user. We need somehow to pass the user to the --tmpfs command. In the documentation we find

Mount a temporary filesystem (tmpfs) mount into a container, for example:

$ docker run -d --tmpfs /tmp:rw,size=787448k,mode=1777 my_image

This command mounts a tmpfs at /tmp within the container.  The supported mount
options are the same as the Linux default mount flags. If you do not specify
any options, the systems uses the following options:
rw,noexec,nosuid,nodev,size=65536k.

which does not help. In the documentation of mount we finally find

uid=value and gid=value
Set the owner and group of the  files  in  the  filesystem  (default: uid=gid=0).

So we need to find the UID of the prosody user. One way to do this is to look at /etc/passwd, where we find

prosody:x:100:101:Prosody XMPP Server:/var/lib/prosody:/sbin/nologin

Now that we know that our prosody user has UID=100 we simply can add it to the --tmpfs flag so that the /var/run/prosody directory is mounted under the prososdy user.

docker run \
       --rm \
       --read-only \
       --tmpfs=/var/run/prosody:uid=100 \
       -it learning/tmpfs

Now our container runs fine, except for the missing ssl/tls certificates, which is simply a prosody setup issue.

Now in addition to the uid=100 option, there are a few more of which the most common ones are

  • rw: that you can read and write to the mounted directory.
  • size: the size of the mounted directory. This is actually quite important, because if your directory fills up you might again end up not being able to write. In case of doubt, rather have a big tmpfs directory.
  • mode: the permissions of the directory