Build Docker images with BuildKit

2 minute read

Containers photo by CHUTTERSNAP

Let’s say it very clearly: If you are not using BuildKit to build your Docker images you are wasting your time!!!.

With BuildKit, the building process for Docker images can be reduced by ten times in some cases, because it turns that process out in something really incremental.

The release 18.09 of Docker [1] introduced a much-needed overhaul of the build architecture by integrating BuildKit [2], so if you are in a previous Docker release, I encourage you to upgrade as soon as possible, since the build process on those versions was a pain in the neck. No matter how many posts you read and followed to speed up the building process by caching Maven artifacts (in case of Java) or by splitting up your Dockerfile in thin tiers, those recipes just didn’t work, because Docker itself didn’t honor the caching of those tiers as they stated in its documentation; but that suddenly changed in Docker 18.09.

Apart from speeding the build process, BuildKit has a lot of additional features and it can be used as a standalone tool without Docker. The following are some of those features [2]:

  • Automatic garbage collection
  • Extendable frontend formats
  • Concurrent dependency resolution
  • Efficient instruction caching
  • Build cache import/export
  • Nested build job invocations
  • Distributable workers
  • Multiple output formats
  • Pluggable architecture
  • Execution without root privileges

It is worth mentioning that BuiltKit is only supported to build Linux containers [3].

Enable it for Docker

Let’s go to the point. How to activate it if you already have installed a Docker version higher than 18.09. The easiest way is setting the following environment variable:

$ DOCKER_BUILDKIT=1

But to enable a permanent solution, you should set the Docker daemon configuration in /etc/docker/daemon.json as follows:

{ "features": { "buildkit": true } }

After activating BuildKit you should notice a change in the output format (more colorful), but the most important aspect is speed, since caching is working once and for all:

$ docker build .

[+] Building 70.9s (34/59)
 => [runc 1/4] COPY hack/dockerfile/install/install.sh ./install.sh       14.0s
 => [frozen-images 3/4] RUN /download-frozen-image-v2.sh /build  buildpa  24.9s
 => [containerd 4/5] RUN PREFIX=/build/ ./install.sh containerd           37.1s
 => [tini 2/5] COPY hack/dockerfile/install/install.sh ./install.sh        4.9s
 => [vndr 2/4] COPY hack/dockerfile/install/vndr.installer ./              1.6s
 => [dockercli 2/4] COPY hack/dockerfile/install/dockercli.installer ./    5.9s
 => [proxy 2/4] COPY hack/dockerfile/install/proxy.installer ./           15.7s
 => [tomlv 2/4] COPY hack/dockerfile/install/tomlv.installer ./           12.4s
 => [gometalinter 2/4] COPY hack/dockerfile/install/gometalinter.install  25.5s
 => [vndr 3/4] RUN PREFIX=/build/ ./install.sh vndr                       33.2s
 => [tini 3/5] COPY hack/dockerfile/install/tini.installer ./              6.1s
 => [dockercli 3/4] RUN PREFIX=/build/ ./install.sh dockercli             18.0s
 => [runc 2/4] COPY hack/dockerfile/install/runc.installer ./              2.4s
 => [tini 4/5] RUN PREFIX=/build/ ./install.sh tini                       11.6s
 => [runc 3/4] RUN PREFIX=/build/ ./install.sh runc                       23.4s
 => [tomlv 3/4] RUN PREFIX=/build/ ./install.sh tomlv                      9.7s
 => [proxy 3/4] RUN PREFIX=/build/ ./install.sh proxy                     14.6s
 => [dev 2/23] RUN useradd --create-home --gid docker unprivilegeduser     5.1s
 => [gometalinter 3/4] RUN PREFIX=/build/ ./install.sh gometalinter        9.4s
 => [dev 3/23] RUN ln -sfv /go/src/github.com/docker/docker/.bashrc ~/.ba  4.3s
 => [dev 4/23] RUN echo source /usr/share/bash-completion/bash_completion  2.5s
 => [dev 5/23] RUN ln -s /usr/local/completion/bash/docker /etc/bash_comp  2.1s

Enable it for Docker Compose

To use BuildKit with Docker Compose [4] you need to install version 1.25.1 [5] or later.

To enable BuildKit just set the following environment variable:

$ COMPOSE_DOCKER_CLI_BUILD=1

Next steps

In this post we have seen how to activate BuildKit in your local environment, in a coming post I will write on how to take advantage of BuildKit in your CI pipelines, since it can be trickier when you do not have the rights to change the building environment.

You may also enjoy