Docker Compose Guide

The Docker Compose File That Works Locally but Explodes on Production

Every team hits the same wall: the Compose file that ran beautifully on a laptop in six months of development starts failing the moment it hits a fresh production host. The cause isn't Docker — it's that Compose defaults hide a thousand assumptions: image tags vs. digests, volume permissions, host networking, health checks, and the silent difference between `build` and `image`. Docker Compose is genuinely the easiest way to stand up a multi-service stack, and genuinely one of the easiest ways to build a reproducible disaster if you only learn the happy path. This guide walks the real flow — from a first working stack to one that survives restart, reboot, and a colleague's brand-new machine — with the pitfalls called out where they actually bite.

Your First Compose File: Structure Before Containers

A Compose file declares services as YAML keys, and the first decision is whether those services are pulled images or built locally. `image: postgres:16` pulls a public image; `build: .` reads a Dockerfile in the current directory. Mix them deliberately: use pinned, official images for infrastructure (database, cache, message broker) and `build` only for your own application code. Pinning with a digest (`postgres@sha256:...`) guarantees reproducibility, while a bare `postgres:latest` is a time bomb waiting for an upstream change to break your stack silently.

Every service needs a name, and that name is what other services use to reach it on the network. Refer to a service by its Compose service name in the connection string, not by `localhost` or an IP. That's the single most jarring shift for developers coming from raw Docker: Compose creates an internal network and DNS per service name automatically. Define the compose version implicitly (recent Compose ignores the `version` key anyway), and start with only the essential service keys — `image`/`build`, `ports`, `environment` — then add `volumes`, `depends_on`, and `healthcheck` as your stack grows.

Volumes: Where Your Data Lives and How It Survives a Rebuild

Containers are ephemeral by default; anything written to the container filesystem disappears when the container is recreated. That's exactly why databases and upload directories need volumes. A named volume (`volumes: - dbdata:/var/lib/postgresql`) is managed by Docker and survives container recreation, while a bind mount (`- ./data:/app/data`) maps a host directory into the container, which is good for development live-reload but a trap for production if the host path isn't stable. Choose the type by intent: named volumes for persistent application data, bind mounts for code you're iterating on or for injecting config.

The volume-permission problem is the one that burns people hardest. Many official images (notably Postgres and MySQL) run as a non-root user and fail to start if the mounted directory is owned by another UID, producing a cryptic "permission denied" in the entrypoint. The fix is to match the UID — either set `user` in the service, chown a bind-mount directory on the host, or use image-specific environment variables that set the data directory ownership. If a database container crashes immediately on first run, check the volume ownership before blaming anything else; it's the cause in a shockingly high share of fresh-setup failures.

Networking and Ports: The Defaults You Must Not Assume

Compose creates a per-project network and gives each service DNS by name. Expose ports to the host only where you need outside access — for a web server or API (`ports: - "8080:80"`), but not for a database that only your app should reach. Exposing Postgres on `5432` to the host in production is how databases get attacked. Intra-network communication uses the service name and the container's internal port, not the published port. This is a subtle point: inside the network, your app connects to `db:5432`, and whether you've published `5432` to the host is irrelevant to that path.

When you need two Compose projects to talk to each other, put them on a shared external network with `networks: - name: shared-net external: true`. This is common when you split an app and its dependency into separate stacks. Also know the difference between `ports` (host mapping, with a `host:container` shorthand and a collision-prone `random` option) and `expose` (network-only, not published). Over-publishing ports is a security and a collision headache in shared environments; publish the minimum your reachability requirements demand.

Environment Variables, Secrets, and the `.env` Trap

Compose reads environment variables in two ways that people conflate. The `.env` file, sitting beside the compose file, is used by Compose itself for interpolation — `$VERSION` in the YAML resolves against `.env`. The `environment:` block sets variables inside the container. Both matter, and confusing them causes the classic "my variable isn't visible in the container" bug. For values that differ between dev and prod, prefer interpolation plus a per-environment `.env`, and keep the secret material out of the compose file and its history.

For secrets, don't put passwords in `environment:` or in the compose YAML. Use the secrets mechanism or, more commonly in the real world, reference secret files mounted as volumes that you create and source at runtime. Git ignore your `.env` and any secret files — committing a database password to your repo is the default failure mode teams hit precisely once. And be explicit that `.env` is evaluated at `docker compose up` time, not on every container start: if you change the `.env`, you must recreate the containers for the change to take effect. Annoying, but it prevents the "I changed it but nothing happened" confusion that stalls so many debug sessions.

Controlling Startup Order With depends_on and Healthchecks

`depends_on` only guarantees start order, not readiness. A Compose service with `depends_on: - db` will start once the db container is started, but the db may not yet be accepting connections — a classic race that produces "connection refused" errors that mysteriously resolve on retry. The modern fix is a `healthcheck` on the dependency plus `depends_on: condition: service_healthy`. Define a healthcheck (official images usually provide one, e.g., `pg_isready` for Postgres or `redis-cli ping` for Redis) and gate your app on the dependency becoming healthy, not merely starting.

Add a healthcheck to every service you care about, because Compose and orchestration platforms use it for readiness, restarts, and load balancing. A healthcheck needs an interval, timeout, and retries tuned to your service: too aggressive and a slow start is killed; too lax and a hung service runs silently. The healthcheck status shows up in `docker compose ps` and in container inspection, and it turns "is it up?" from a guess into an observable fact. Combined with a restart policy (`restart: unless-stopped` is the sane default on a server), healthchecks are what make a Compose stack feel like a managed service rather than a fragile pile of containers.

Scaling, Resource Limits, and What Compose Handles vs. Hands Off

Compose can scale a service horizontally with `docker compose up --scale web=3`, but the service must be stateless and reachable through a load balancer for that to make sense. Out of the box there's no load balancer, so scaled web services each expose their own host port — you need a reverse proxy (NGINX, Traefik, Caddy) in front of them or a platform that adds orchestration. Compose also lets you set CPU and memory limits per service with `deploy.resources.limits` and memswap, which is essential on a shared host to stop one greedy container from starving the rest; otherwise a memory-leaking service can take down the whole box.

Know the ceiling: Compose is for single-host orchestration. When you need multi-node scheduling, auto-scaling, rolling deployments, and self-healing at cluster scale, that's Kubernetes' domain. Many teams graduate a working Compose stack into a Kubernetes manifest once a service outgrows a box — the mental model of services, networks, volumes, and healthchecks transfers cleanly, which is a big reason to get those Compose fundamentals right first. The Kubernetes basics course is the natural next step when you outgrow a single host. For a deeper map of both the container-native and the orchestrated path, the Docker 2026 guide ties the tooling trends together, while the step-by-step is covered in the 2026 beginners' guide.

A Comparison of Compose Alternatives for Running Multi-Service Stacks

Compose is the right default for a single host and a handful of services; each alternative above trades a little simplicity for a feature at a different layer. Choose based on host count and how much orchestration you need today, not on what you might need next year.

Profiles, Overrides, and Reusing One Compose Project Across Environments

A robust Compose setup separates configuration from environment. Use a base `docker-compose.yml` for services common to all environments, and environment-specific override files (`docker-compose.override.yml` for dev is auto-loaded; `docker-compose.prod.yml` you apply explicitly with `-f`). Overrides merge, so your dev file can add extra volumes for live-reload and your prod file can pin digests and disable debug services, all from one source of truth. This beats maintaining three near-duplicate compose files that drift out of sync.

Profiles go further: mark optional services (a metrics exporter, a test database, a dev mail catcher) with `profiles: - debug` and they only start when you pass `--profile debug`. That keeps your default `docker compose up` lean while making the full toolchain one flag away. As your stack grows past a few services, define tight service boundaries and document which profiles exist and what each provides; profile sprawl is the next thing to bite after the happy-path honeymoon. Good profiles and overrides are what keep a "local dev stack" from becoming an unmaintainable monolith of YAML.

Troubleshooting Compose: The Commands That Save You an Hour

When a stack misbehaves, run diagnostics in escalating order. `docker compose config` validates and prints the fully-resolved compose file with all interpolation and overrides applied — this exposes environment-variable mistakes instantly without touching a container. `docker compose ps` shows container states and health; `docker compose logs --tail=100 ` gives you the failing service's output. For a container that won't start, `docker compose exec sh` drops you inside to inspect, and `docker inspect ` shows the actual runtime config, health status, and exit code.

The most common gotchas have known signatures. "Port already in use" means a stale container still holds the host port — `docker compose down` clears it. A container exiting immediately usually means the entrypoint failed: read the first few lines of logs, and if it's a volume-permission error, fix the ownership. Intermittent "connection refused" on startup points at a missing `condition: service_healthy` in `depends_on`. And when nothing else makes sense, `docker compose down -v` (which deletes named volumes!) then `up` gives you a clean slate — but only use `-v` when you're sure you can lose the data. For the full journey from a first container to a production-worthy single-host deployment, the Docker DevOps guide covers the operational patterns — healthchecks, restart policies, and image hygiene — that turn a working stack into a reliable one, and its troubleshooting flow reuses exactly these commands.

From a Working Compose Stack to Maintainable Infrastructure

The difference between a stack that impresses and a stack that survives is documentation and discipline. Put a `README` beside the compose file that states what each service does, which profiles and overrides exist, the volume layout, and the exact command to bring it up in each environment. Pin your images, add healthchecks to every service, and set a restart policy. Keep secrets out of the repo, make volumes' ownership explicit, and publish only the ports you need. This is the unglamorous work, and it's what separates a stack a new teammate can bring up in ten minutes from one only its author can run.

Revisit your Compose file whenever you change how services depend on each other — adding a cache or a queue means new healthcheck and dependency chains, not just a new service block. And remember Compose's place in the stack: it's your single-host orchestration layer, and the concepts it teaches — services, networks, volumes, health, config via environment — are exactly the vocabulary you need whether you stay on Compose or move to Kubernetes. The teams that internalize these fundamentals are the same ones whose container stacks stop being a source of Friday outages and start being a boring, dependable part of the platform. The 2026 Docker landscape piece and the beginners' walkthrough are good companion reads for locking in that mental model before you scale it up.

For more, check out: .

For more, check out: .

My database container won't start with "Permission denied" — what's actually wrong?

Almost always it's volume ownership. The container's process runs as a non-root user, but the host directory or named volume is owned by another UID (commonly root). Fix by matching the image's expected UID: chown the bind-mount directory on the host, or set the image's data-dir environment variable (e.g., `PUID`/`PGID` for some images), or override the service `user`. Check `docker logs ` for the ownership clue before troubleshooting anything else.

What's the difference between `.env` and the `environment:` block in Compose?

`.env` is read by Compose itself for variable interpolation in the YAML (`$VERSION` resolves against it) and is never passed into the container. `environment:` sets variables that exist inside the container. If your app can't see a variable, it's in the wrong place. Git-ignore `.env` and never commit it; keep secrets out of both the YAML and container env where you can.

Why does my app fail with "connection refused" even though the database container is running?

Because `depends_on` only guarantees start order, not readiness — the database may still be initializing when your app connects. Add a `healthcheck` to the database (e.g., `pg_isready`) and use `depends_on: condition: service_healthy` on your app. That gates your app on the dependency becoming ready instead of merely started.

When should I use Compose over Kubernetes or vice versa?

Compose is for a single host, a handful of services, and rapid iteration — it's the right default for local dev and small deployments. Kubernetes is for multi-node scheduling, auto-scaling, rolling deployments, and self-healing at cluster scale. If you never expect to outgrow one box, Compose is enough; if you know you'll scale across nodes, invest in Kubernetes early. The service/network/volume/healthcheck concepts transfer directly.