Image Engineering

2GB images with secrets baked in and :latest tags = amateur hour. This section covers how to build images that are small, secure, fast to build, and fast to deploy.

Multi-Stage Builds

The single most important Dockerfile technique. Separate build environment from runtime environment.

# Stage 1: Build (large, has compilers/tools)
FROM golang:1.22 AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o myapp .

# Stage 2: Runtime (tiny, only what's needed)
FROM gcr.io/distroless/static:nonroot
COPY --from=builder /app/myapp /
USER nonroot:nonroot
ENTRYPOINT ["/myapp"]

Result: Build stage has Go compiler (~800MB). Final image has only the binary (~15MB).

ELI5: Multi-stage builds are like building a house. Stage 1 is the construction site — cranes, scaffolding, cement mixers everywhere. Stage 2 is the finished house — you move in the furniture and tear down all the construction equipment. Your final image is the house, not the construction site.

Common mistake: Putting everything in one stage. Your production image ends up with gcc, make, dev headers, and test frameworks. Attack surface is huge. Build time dependencies should NEVER be in the final image.

Base Image Selection

Decision framework: Can you compile to a static binary? → scratch. Need a runtime (JVM, Node, Python)? → distroless. Need to debug in production (shell access)? → Alpine. Need specific system libraries? → slim. Only use full OS images during development.

Why distroless matters for security:

• No shell → attacker can’t get interactive access even with RCE
• No package manager → can’t install tools inside compromised container
• Minimal attack surface → fewer CVEs to patch

Common mistake: Using Alpine for Python/Node apps and then fighting musl vs glibc compatibility issues. Some Python C extensions don’t compile against musl. Use python:3.12-slim (Debian slim) instead if you hit issues.

Layer Caching Optimization

Docker builds images layer by layer. If a layer hasn’t changed, Docker reuses the cached version. Layer order determines cache effectiveness.

# BAD: source code change invalidates dependency install cache
COPY . .
RUN pip install -r requirements.txt

# GOOD: dependencies change rarely, source code changes often
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .

The rule: Order instructions from least-frequently-changed to most-frequently-changed.

FROM base           ← Almost never changes (long cache)
RUN apt-get install ← Changes when deps change
COPY package.json   ← Changes when deps change
RUN npm install     ← Cached unless package.json changed
COPY . .            ← Changes every commit (cache busted here)
RUN npm build       ← Always rebuilds

ELI5: Layer caching is like packing a suitcase in order. Put the heavy, rarely-changed items (shoes, toiletries) at the bottom. Put daily clothing (source code) on top. If you only need to change your shirt, you don’t have to unpack the entire suitcase — just the layers on top.

Cache Mounts (BuildKit)

# Cache pip downloads across builds
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install -r requirements.txt

# Cache apt packages
RUN --mount=type=cache,target=/var/cache/apt \
    apt-get update && apt-get install -y libpq-dev

Cache mounts persist between builds. Dependency reinstalls only download what’s new.

Reducing Image Size

Combine RUN Commands

# BAD: 3 layers, apt cache persists in layer 1
RUN apt-get update
RUN apt-get install -y curl
RUN rm -rf /var/lib/apt/lists/*

# GOOD: 1 layer, apt cache cleaned in same layer
RUN apt-get update && \
    apt-get install -y --no-install-recommends curl && \
    rm -rf /var/lib/apt/lists/*

Each RUN creates a layer. Deleting files in a later layer doesn’t reduce the overall image size — the file still exists in the earlier layer. Combine related operations into one RUN.

.dockerignore

# .dockerignore
.git
node_modules
*.md
Dockerfile
docker-compose.yml
.env
__pycache__
.pytest_cache
coverage/

Without .dockerignore, COPY . . sends everything (including .git, node_modules, etc.) to the build context. This slows builds and may leak secrets.

Size Comparison Example

BuildKit Features

BuildKit is Docker’s modern build engine (default since Docker 23.0). Key features:

Secret Mounts (Never Leak Secrets)

# Build with: docker build --secret id=npmrc,src=.npmrc .
RUN --mount=type=secret,id=npmrc,target=/root/.npmrc \
    npm install
# Secret never appears in any image layer

SSH Mounts (Private Git Repos)

# Build with: docker build --ssh default .
RUN --mount=type=ssh \
    git clone git@github.com:private/repo.git
# SSH key never appears in image

Multi-Architecture Images

Build images that work on both AMD64 and ARM64 (Apple Silicon, AWS Graviton).

# Create a multi-platform builder
docker buildx create --name multiarch --use

# Build and push for multiple platforms
docker buildx build --platform linux/amd64,linux/arm64 \
  -t myregistry/myapp:v1.0 --push .

Docker Hub stores a manifest list. When someone pulls the image, Docker automatically selects the right architecture.

Why this matters: AWS Graviton (ARM64) is 20-40% cheaper than x86. Apple Silicon Macs are ARM64. If your image is AMD64-only, you’re either running under emulation (slow) or locking out half your users.

Common mistake: Using COPY --from to copy binaries from other images in multi-arch builds without ensuring those images exist for all target platforms. Test on both architectures in CI.

Dockerfile Best Practices Summary

Key Takeaways for Interviews

1. “How do you optimize image size?” → Multi-stage builds, minimal base (distroless/Alpine), combine RUN layers, .dockerignore, –no-install-recommends.
2. “How do you handle secrets in Docker builds?” → BuildKit secret mounts (--mount=type=secret). Never COPY secrets, never use ARG/ENV for secrets, never delete in later layer.
3. “Explain layer caching” → Each instruction is a layer. Unchanged layers are cached. Order from least-changed (dependencies) to most-changed (source code).
4. “What’s distroless?” → Google’s minimal images with only the runtime. No shell, no package manager. Smallest attack surface possible while still running non-static binaries.
5. “Multi-arch builds?” → docker buildx with --platform. Produces manifest list. Clients auto-select correct architecture. Essential for ARM/Graviton cost savings.