Advanced Docker

1. Multi-Stage Builds Deep Dive

A multi-stage build uses multiple FROM statements in one Dockerfile. Each stage is an independent build environment; you copy only the artifacts you need into the final image. The result: smaller images, no compiler toolchains in production, and faster deploys.

┌─────────────────────────────────────────────────────────────────┐
│  Stage: builder          Stage: production                      │
│  ┌──────────────────┐    ┌──────────────────┐                  │
│  │ golang:1.22      │    │ distroless/base  │                  │
│  │ compile binary   │───▶│ COPY binary only │  final ~15 MB    │
│  │ (500 MB tools)   │    │ no shell, no gcc │                  │
│  └──────────────────┘    └──────────────────┘                  │
│  intermediate layers discarded  -  not in final image           │
└─────────────────────────────────────────────────────────────────┘

Go application (distroless)

# syntax=docker/dockerfile:1

FROM golang:1.22-alpine AS builder
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -ldflags="-s -w" -o /app/server ./cmd/server

FROM gcr.io/distroless/static-debian12:nonroot
COPY --from=builder /app/server /server
USER nonroot:nonroot
EXPOSE 8080
ENTRYPOINT ["/server"]

Node.js application

FROM node:20-alpine AS deps
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci --omit=dev

FROM node:20-alpine AS builder
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM node:20-alpine AS production
WORKDIR /app
ENV NODE_ENV=production
COPY --from=deps /app/node_modules ./node_modules
COPY --from=builder /app/dist ./dist
COPY package.json ./
USER node
EXPOSE 3000
CMD ["node", "dist/main.js"]

Named stages and cross-stage copies

FROM alpine AS base
RUN apk add --no-cache ca-certificates

FROM base AS tools
RUN apk add --no-cache curl jq

FROM base AS app
COPY --from=tools /usr/bin/curl /usr/bin/curl   # copy single binary
COPY --from=tools /usr/lib/libjq.so* /usr/lib/
COPY ./app /app
ENTRYPOINT ["/app/run.sh"]

Build a specific stage

# Build only the builder stage (for debugging compile issues)
docker build --target builder -t myapp:builder .

# Build production image (default: last stage)
docker build -t myapp:2.1.0 .

2. BuildKit Features

BuildKit is the modern Docker build engine (default since Docker 23.0). It provides parallel stage execution, advanced caching, secrets during build, SSH agent forwarding, and improved Dockerfile frontend features.

Enable BuildKit

# Enabled by default in recent Docker Desktop and Engine
export DOCKER_BUILDKIT=1

# Explicit builder instance (Buildx)
docker buildx create --name mybuilder --use
docker buildx inspect --bootstrap

Cache mounts (speed up package installs)

# syntax=docker/dockerfile:1

FROM node:20-alpine AS builder
WORKDIR /app
COPY package.json package-lock.json ./
RUN --mount=type=cache,target=/root/.npm \
    npm ci
COPY . .
RUN npm run build

FROM golang:1.22 AS builder
RUN --mount=type=cache,target=/go/pkg/mod \
    --mount=type=cache,target=/root/.cache/go-build \
    go build -o /app/server .

Build secrets (never in image layers)

# syntax=docker/dockerfile:1

FROM alpine
RUN --mount=type=secret,id=npm_token \
    NPM_TOKEN=$(cat /run/secrets/npm_token) && \
    echo "//registry.npmjs.org/:_authToken=${NPM_TOKEN}" > .npmrc && \
    npm ci && rm .npmrc

# Build with secret file
docker build --secret id=npm_token,src=./.npm_token .

SSH mount (private Git repos)

FROM alpine
RUN apk add --no-cache git openssh-client
RUN --mount=type=ssh \
    mkdir -p ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts && \
    git clone git@github.com:org/private-repo.git /src

# Build forwarding your SSH agent
docker build --ssh default .

Multi-platform builds

# Build and push amd64 + arm64 in one command
docker buildx build \
  --platform linux/amd64,linux/arm64 \
  -t myregistry/myapp:2.1.0 \
  --push .

# Inspect manifest
docker buildx imagetools inspect myregistry/myapp:2.1.0

3. Image Security & Scanning

Container security starts at the image: minimize attack surface, scan for CVEs, sign images, and enforce policies before deployment.

Image hardening checklist

• Use minimal base images: alpine, distroless, or chainguard - not full OS images.
• Run as non-root (USER directive); never run as UID 0 in production.
• Pin base image digests, not just tags: node@sha256:abc123...
• No secrets in Dockerfile ARG/ENV - use BuildKit secrets.
• Enable read_only root filesystem and drop all capabilities.
• Keep images updated; automate rebuilds on base image patches.

Scan images for vulnerabilities

# Docker Scout (built into Docker Desktop / CLI plugin)
docker scout quickview myapp:2.1.0
docker scout cves myapp:2.1.0
docker scout recommendations myapp:2.1.0

# Trivy (popular open-source scanner)
trivy image myapp:2.1.0
trivy image --severity HIGH,CRITICAL myregistry/myapp:2.1.0
trivy image --exit-code 1 --severity CRITICAL myapp:2.1.0  # fail CI on critical

# Grype
grype myapp:2.1.0

Image signing with Cosign

# Generate key pair (use KMS in production)
cosign generate-key-pair

# Sign image after push
cosign sign --key cosign.key myregistry/myapp:2.1.0

# Verify before deploy
cosign verify --key cosign.pub myregistry/myapp:2.1.0

# Sign with ephemeral keys in GitHub Actions (keyless)
cosign sign myregistry/myapp@${DIGEST}

Dockerfile security linting

# Hadolint  -  Dockerfile linter
hadolint Dockerfile

# Dockle  -  image best-practice checker
dockle myapp:2.1.0

4. Container Registries

A container registry stores and distributes images. Docker Hub is the default public registry; production teams use private registries (ECR, GCR/Artifact Registry, ACR, Harbor, GitLab Registry).

Tag, push, pull workflow

# Tag for registry (registry/namespace/image:tag)
docker tag myapp:2.1.0 myregistry.example.com/team/myapp:2.1.0
docker tag myapp:2.1.0 myregistry.example.com/team/myapp:latest

# Authenticate
docker login myregistry.example.com
# or: echo $TOKEN | docker login myregistry.example.com -u user --password-stdin

# Push
docker push myregistry.example.com/team/myapp:2.1.0
docker push myregistry.example.com/team/myapp:latest

# Pull on deployment host
docker pull myregistry.example.com/team/myapp:2.1.0

Registry comparison

AWS ECR example

AWS_ACCOUNT=123456789012
REGION=us-east-1
REPO=myapp

# Create repository
aws ecr create-repository --repository-name $REPO --region $REGION

# Login (token valid 12 hours)
aws ecr get-login-password --region $REGION | \
  docker login --username AWS --password-stdin \
  $AWS_ACCOUNT.dkr.ecr.$REGION.amazonaws.com

# Push
docker tag myapp:2.1.0 $AWS_ACCOUNT.dkr.ecr.$REGION.amazonaws.com/$REPO:2.1.0
docker push $AWS_ACCOUNT.dkr.ecr.$REGION.amazonaws.com/$REPO:2.1.0

Image retention and garbage collection

# List tags
aws ecr list-images --repository-name myapp

# Harbor, ECR, and GCR support lifecycle policies to prune untagged
# or old images automatically  -  configure in registry UI or IaC

# Local cleanup
docker image prune -a        # remove unused images
docker system df           # show disk usage

5. Docker in CI/CD

CI/CD pipelines build, scan, sign, and push images on every merge. The golden rule: build once, promote by digest - never rebuild the same release tag in staging and production.

GitHub Actions example

name: Build and Push

on:
  push:
    branches: [main]
    tags: ["v*"]

jobs:
  build:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write
    steps:
      - uses: actions/checkout@v4

      - uses: docker/setup-buildx-action@v3

      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - uses: docker/metadata-action@v5
        id: meta
        with:
          images: ghcr.io/${{ github.repository }}
          tags: |
            type=sha,prefix=
            type=semver,pattern={{version}}
            type=raw,value=latest,enable={{is_default_branch}}

      - uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=gha
          cache-to: type=gha,mode=max

      - name: Scan image
        run: |
          docker pull ghcr.io/${{ github.repository }}:latest
          trivy image --exit-code 1 --severity CRITICAL \
            ghcr.io/${{ github.repository }}:latest

GitLab CI example

build:
  image: docker:24
  services:
    - docker:24-dind
  variables:
    DOCKER_TLS_CERTDIR: "/certs"
    IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
  script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - docker build -t $IMAGE .
    - docker push $IMAGE
    - trivy image --exit-code 1 --severity CRITICAL $IMAGE

CI best practices

• Use docker buildx with GHA/GitLab cache backends for fast builds.
• Tag with git SHA (abc1234) and semver (v2.1.0); avoid mutable latest in deploy configs.
• Scan in CI; fail the pipeline on critical CVEs (with an allowlist process).
• Sign images and verify in deployment (Cosign, Notary v2).
• Never store registry passwords in the repo - use OIDC or short-lived tokens.

# Promote by digest (immutable)
DIGEST=$(docker buildx imagetools inspect myregistry/myapp:v2.1.0 --format '{{json .Manifest}}' | jq -r '.digest')
docker buildx imagetools create myregistry/myapp@$DIGEST --tag myregistry/myapp:prod-approved

6. Rootless Docker

Rootless mode runs the Docker daemon and containers as an unprivileged user. If a container escape occurs, the attacker gains user-level access, not root on the host. This significantly reduces blast radius on shared or multi-tenant hosts.

How it works

┌─────────────────────────────────────────────────────────────────┐
│  Rootful Docker (default)                                       │
│  dockerd runs as root  -  container escape = potential root     │
│                                                                 │
│  Rootless Docker                                                │
│  rootlesskit + slirp4netns/vpnkit                               │
│  dockerd runs as user  -  UID 0 in container maps to host user │
│  no privileged ports (<1024) without sysctl                     │
└─────────────────────────────────────────────────────────────────┘

Install rootless Docker (Linux)

# Prerequisites: uidmap, slirp4netns (or pasta in newer versions)
dockerd-rootless-setuptool.sh install

# Add to shell profile (printed by installer)
export PATH=/usr/bin:$PATH
export DOCKER_HOST=unix:///run/user/$(id -u)/docker.sock

# Verify
docker info | grep -i rootless
docker run hello-world

Limitations

User namespace remapping (rootful alternative)

# /etc/docker/daemon.json  -  map container root to subordinate UIDs
{
  "userns-remap": "default"
}

# Requires /etc/subuid and /etc/subgid entries for dockremap user
# Container UID 0 maps to unprivileged host UID (e.g. 100000)

7. Resource Limits & cgroups

cgroups (control groups) limit and account for CPU, memory, block I/O, and PIDs. Docker applies cgroup limits via CLI flags or Compose deploy.resources (Swarm) / docker run flags (standalone).

Memory limits

# Hard memory cap (OOM kill if exceeded)
docker run -d --name api --memory=512m --memory-swap=512m myapp:2.1.0

# Soft reservation (scheduling hint, not enforced hard)
docker run -d --memory-reservation=256m myapp:2.1.0

# OOM score adjustment (lower = less likely to be killed)
docker run -d --oom-score-adj=-500 myapp:2.1.0

CPU limits

# Limit to 1.5 CPUs
docker run -d --cpus=1.5 myapp:2.1.0

# Pin to specific cores
docker run -d --cpuset-cpus="0,1" myapp:2.1.0

# CPU shares (relative weight, default 1024)
docker run -d --cpu-shares=512 myapp:2.1.0

Compose resource limits

services:
  api:
    image: myapp/api:2.1.0
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 512M
        reservations:
          cpus: "0.25"
          memory: 128M

# Standalone compose (Docker Compose v2.10+)
services:
  api:
    mem_limit: 512m
    cpus: 1.0
    pids_limit: 100

Block I/O and PIDs

# Limit disk read/write bandwidth
docker run -d \
  --device-read-bps /dev/sda:10mb \
  --device-write-bps /dev/sda:5mb \
  myapp:2.1.0

# Prevent fork bombs
docker run -d --pids-limit=100 myapp:2.1.0

Monitor resource usage

docker stats                    # live CPU/mem/net/block for all containers
docker stats api --no-stream      # one-shot snapshot
docker inspect api --format='{{.HostConfig.Memory}}'
docker inspect api --format='{{.HostConfig.NanoCpus}}'

# cgroup v2 path (Linux)
cat /sys/fs/cgroup/system.slice/docker-$(docker inspect -f '{{.Id}}' api).scope/memory.current

8. Cheat Sheet

Full cheat sheet: Docker CLI Cheat Sheet