Building multi-arch Docker images with Github Actions

Posted on 11 Jan 2023

Docker supports building images for multiple platforms, or architectures, such as linux/amd64 or linux/arm64. Doing so on your development machine is very easy and straightforward, thanks to buildx. And if you're building software with multi-arch support in mind (you should!), chances are you will end up having to release it to Docker Hub (or any private registry) at some point, in automated fashion.

Github Actions is a perfect tool for a job like this! We could set up Docker image building flow just like we would have done for running application tests or any CI/CD pipelines. As for actually building multi-arch Docker images, there are two options (that I know of): build all at once or in parallel.

The goal is to automate building and pushing of Docker images for multiple platforms to both Docker Hub and Github Container Registry when a new git tag (ie. v1.2.3) is pushed out to Github. Platforms selected for testing: linux/amd64, linux/arm64, linux/arm/v7 and linux/arm/v5.

Prerequisites

  • Docker Hub account. Make sure to add DOCKER_USERNAME and DOCKER_PASSWORD to actions secrets.
  • Github API token with write:packages scope, set as GH_TOKEN in secrets.
  • A working Dockerfile in your project.

Sequential build

name: docker

env:
  DOCKER_REPO: username/testapp
  GHCR_REPO: ghcr.io/username/testapp

on:
  push:
    tags:
      - "v*"

jobs:
  build-and-push:
    runs-on: ubuntu-latest
    timeout-minutes: 30

    steps:
      - name: Set vars
        id: vars
        run: |
          echo "version=${GITHUB_REF#refs/tags/v}" >> $GITHUB_OUTPUT

      # Prepare build environment
      - uses: actions/checkout@v3
      - uses: docker/setup-qemu-action@v2
      - uses: docker/setup-buildx-action@v2

      - name: Login to Docker Hub
        uses: docker/login-action@v2
        with:
          username: ${{ secrets.DOCKER_USERNAME }}
          password: ${{ secrets.DOCKER_PASSWORD }}

      - name: Login to Github Container Registry
        uses: docker/login-action@v2
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GH_TOKEN }}

      - name: Build and push images
        uses: docker/build-push-action@v3
        with:
          context: .
          push: true
          platforms: linux/amd64,linux/arm64,linux/arm/v7,linux/arm/v5
          tags: |
            ${{ env.DOCKER_REPO }}:${{ steps.vars.outputs.version }}
            ${{ env.GHCR_REPO }}:${{ steps.vars.outputs.version }}

The echo "version=${GITHUB_REF#refs/tags/v}" >> $GITHUB_OUTPUT line is not necessary if the tags you're pushing are in x.x.x format (unlike vx.x.x) since the convention for docker image tags is generally org/reponame:x.x.x. You still might need it to set other vars in step outputs later.

With docker/build-push-action@v3 we essentially building an image for all platforms in one go and then push it out to both container registries. For a few of my applications it roughly takes 20-25 minutes to complete, not to mention random networking issues and timeouts that could break the whole job and you'll have to start over. The workflow is simple, but slow. For most cases that's what you'd want to use.

In parallel

Building images for multiple platforms in parallel is taking advantage of matrix jobs. In our case we're splitting the primary build job into 4 smaller chunks that run independently, which allows us to retry them independently as well.

Main difference in the process is first we're pushing out arch-specific images as username/testapp:0.1.0-linux-amd64 (for git tag v0.1.0), and only after all images are built and pushed successfully, we're creating a unified image tag via docker manifest command, executed in a separate release job.

name: docker

on:
  push:
    tags:
      - "v*"

env:
  DOCKER_REPO: username/testapp
  GHCR_REPO: ghcr.io/username/testapp

jobs:
  build:
    runs-on: ubuntu-latest
    timeout-minutes: 30

    strategy:
      matrix:
        platform:
          - linux/amd64
          - linux/arm64
          - linux/arm/v5
          - linux/arm/v7

    steps:
      - name: Set vars
        id: vars
        run: |
          echo "version=${GITHUB_REF#refs/tags/v}" >> $GITHUB_OUTPUT
          echo "platform=$(echo -n ${{ matrix.platform }} | sed 's/\//-/g')" >> $GITHUB_OUTPUT

      - uses: actions/checkout@v3
      - uses: docker/setup-qemu-action@v2
      - uses: docker/setup-buildx-action@v2

      - name: Login to Docker Hub
        uses: docker/login-action@v2
        with:
          username: ${{ secrets.DOCKER_USERNAME }}
          password: ${{ secrets.DOCKER_PASSWORD }}

      - name: Login to Github Container Registry
        uses: docker/login-action@v2
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GH_TOKEN }}

      - name: Build docker images
        uses: docker/build-push-action@v3
        with:
          context: .
          push: true
          platforms: ${{ matrix.platform }}
          tags: |
            ${{ env.DOCKER_REPO }}:${{ steps.vars.outputs.version }}-${{ steps.vars.outputs.platform }}
            ${{ env.GHCR_REPO}}:${{ steps.vars.outputs.version }}-${{ steps.vars.outputs.platform }}

  release:
    runs-on: ubuntu-latest
    timeout-minutes: 10
    needs: build

    steps:
      - name: Set vars
        id: vars
        run: |
          echo "version=${GITHUB_REF#refs/tags/v}" >> $GITHUB_OUTPUT

      - name: Login to Docker Hub
        uses: docker/login-action@v2
        with:
          username: ${{ secrets.DOCKER_USERNAME }}
          password: ${{ secrets.DOCKER_PASSWORD }}

      - name: Login to Github Container Registry
        uses: docker/login-action@v2
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GH_TOKEN }}

      - name: Create Docker Hub manifest
        run: |
          docker manifest create $DOCKER_REPO:${{ steps.vars.outputs.version }} \
            $DOCKER_REPO:${{ steps.vars.outputs.version }}-linux-amd64 \
            $DOCKER_REPO:${{ steps.vars.outputs.version }}-linux-arm64 \
            $DOCKER_REPO:${{ steps.vars.outputs.version }}-linux-arm-v5 \
            $DOCKER_REPO:${{ steps.vars.outputs.version }}-linux-arm-v7

      - name: Create GHCR manifest
        run: |
          docker manifest create $GHCR_REPO:${{ steps.vars.outputs.version }} \
            $GHCR_REPO:${{ steps.vars.outputs.version }}-linux-amd64 \
            $GHCR_REPO:${{ steps.vars.outputs.version }}-linux-arm64 \
            $GHCR_REPO:${{ steps.vars.outputs.version }}-linux-arm-v5 \
            $GHCR_REPO:${{ steps.vars.outputs.version }}-linux-arm-v7

      - name: Push manifests
        run: |
          docker manifest push $DOCKER_REPO:${{ steps.vars.outputs.version }}
          docker manifest push $GHCR_REPO:${{ steps.vars.outputs.version }}

Oh, btw, pushing images to Github Registry is even simpler if you opt-in to use Github Actions permissions model. We can skip setting up the API token altogether with:

permissions:
  contents: read
  packages: write

With parallel builds we are able to cut the total job time more than in half, and maintain some flexibility: if a platform-specific image build fails for whatever reason, we could intervene and retry it (or do it automatically). Then release job will pick up again and finish off the remainder of workflow.

Before
After

Side effects

All this is great, but I would lie if I said there would be no gotchas. First, overall workflow became a bit more complex: our example is pretty basic but quite lengthy already, so adding in extra bits/steps/processes will make it much harder to grok.

Second, as a side effect of pushing intermediary images to the registry, we end up with tons of unwanted tags. Let's say our final multi-arch image is myapp:0.1.0, but you'll find a handful of platform-specific tags as well, like myapp:0.1.0-linux-amd64. Not a big deal if you don't mind, but cleaning those out will require extra work, (using a tool like regctl) and not all container registries may allow deleting tags via API, so keep that in mind.