In 2018, I gave a talk at KubeCon on "Building Docker Images Without Docker". This post is an updated version of that talk – what's changed, what hasn't, and what's next.
Docker images are simply compressed tarballs with some metadata. The format is a little complicated to understand because it has many years of path-dependent technical debt baked into it. But this means you can construct them in any way you'd like – you don't need a Docker daemon, installation, or special environment. Package up the files, add the relevant metadata, and you'll be able to push, pull, and run them.
It's straightforward to build a Docker image with a Dockerfile (and Docker), but how would you build a Docker image without Docker?
Could you do it directly in code or with an existing build system?
What makes the Dockerfile format interesting is the existence of the RUN directive. This instruction takes a command argument and uses the Docker runtime to execute the command in a container before committing the result as a new layer. This workflow is powerful – it allows the DevOps way of scripting to translate to containers and makes it easy to add implicit resources (e.g., the files outputted by a package install).
At Google, I worked on tooling that allowed for "runtime-less" builds. These tools could be executed in unprivileged (i.e., outside of Kubernetes or Docker) environments. Some of the strategies:
- Building the image in a declarative build system (e.g., Bazel) – use a reproducible build system to build a tarball. No
RUNcommands, but (in theory) reproducible. I even built some "reproducible-ish" package installations from Ubuntu's package manager (a mainRUNuse case). - Skip the runtime and execute the command in the existing shell. If you're already in a sandboxed environment (i.e., in a build pod on Kubernetes), you don't need to isolate the commands again (another layer of Docker).
But for many, the RUN command is essential. And the tooling has gotten better over the years. In the 2018 talk, I alluded to some work being done on alternative frontends to the Dockerfile (that work turned into BuildKit later that year, which I gave another talk about, summed up here).
To use alternative frontends, you still need Moby (Docker), which now includes BuildKit. But you aren't wed to the Dockerfile format.
I described an early prototype of an alternative to the Dockerfile (mockerfile) in this blog post and GitHub repository. BuildKit's RPC endpoint means that you can build Docker images in a variety of new ways:
- Programmatically through libraries that build a DAG and send the DAG to the BuildKit server.
- Imperatively by directly sending the commands to the BuildKit server.
- A new configuration format that is compiled into a DAG