Docker and Docker Compose

Docker is used to encapsulate applications and their running environments into containers to achieve environment isolation and consistent deployment.

Docker Compose is used to define and manage applications composed of multiple containers.

1. Core concepts

Dockerfile
    ↓ docker build
Image（mirror）
    ↓ docker run
Container（container）

One image can create multiple containers. Containers do not share their own writable file layers by default.

Stopping the container does not mean that the data disappears; but when the container is deleted, its writable layer is usually deleted together, so the database requires persistent storage.

2. Build the image

docker build -t sorivel-backend:1.0.0 .

Parameters:

For example:

docker build \
  -f Dockerfile.prod \
  -t sorivel-backend:1.2.0 \
  --build-arg APP_VERSION=1.2.0 \
  .

Build Context

docker build . will use the current directory as the build context.

COPY of Dockerfile can usually only read files that are not ignored in the build context.

For example:

COPY ./src /app/src

./src is relative to Build Context, not necessarily relative to the directory where Dockerfile is located.

.dockerignore

Very recommended configuration:

.git
node_modules
.venv
__pycache__
.env
.env.*
dist
*.log

Function:

• Reduce build context.
• Build faster.
• Reduce unnecessary cache invalidations.
• Reduce the risk of sensitive files accidentally entering the image.

.dockerignore is not equivalent to a secure key management tool, Secrets still needs to be properly managed.

3. Create and run containers

docker run -d \
  --name myapp \
  -p 8000:8000 \
  --env-file .env \
  myapp:1.0

port mapping

-p 8000:8000

Format:

Host Port : Container Port

If only local access to the host is allowed:

-p 127.0.0.1:8000:8000

It is more suitable for development services that do not require public access than directly binding all host network interfaces.

Override startup command

docker run --rm myapp:1.0 python manage.py shell

The command here will overwrite the CMD of the image; if the image is configured with ENTRYPOINT, the parameter combination behavior needs to be considered together.

4. Dockerfile common instructions

Dockerfile example

FROM python:3.12-slim

WORKDIR /app

ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1

COPY requirements.txt .

RUN pip install --no-cache-dir -r requirements.txt

COPY . .

EXPOSE 8000

CMD [
  "gunicorn",
  "config.asgi:application",
  "-k",
  "uvicorn_worker.UvicornWorker",
  "--bind",
  "0.0.0.0:8000"
]

Gunicorn, Uvicorn and corresponding Worker packages need to be installed in dependencies.

CMD and ENTRYPOINT

ENTRYPOINT ["python"]
CMD ["manage.py", "runserver"]

Combined default command:

python manage.py runserver

• ENTRYPOINT: Define the main program.
• CMD: Provides default commands or parameters.

Both can be overridden by run parameters, but in different ways.

It is generally recommended to use the JSON array form (Exec Form) in order to correctly handle process signals.

ARG and ENV

ARG APP_VERSION=dev
ENV APP_ENV=production

Difference:

• ARG: used during the build phase and not retained as a container environment variable by default.
• ENV: Can be used in both the build phase and the container run phase and remains in the image configuration.

Do not use ARG or ENV to save build keys.

5. Docker Image Layers and Build Cache

The Docker image is composed of multiple layers, and BuildKit can cache the results of the build step.

For example:

COPY requirements.txt .
RUN pip install -r requirements.txt

COPY . .

If only the Python business code is modified and the dependency files remain unchanged, Docker can usually reuse the build cache of previously installed dependencies.

And:

COPY . .
RUN pip install -r requirements.txt

Code changes may make caches that rely on installation steps more susceptible to invalidation.

Cache optimization principles

First copy the dependency description files with low change frequency, then install the dependencies, and finally copy the business code.

BuildKit Cache Mount

A build cache can be provided to the package manager:

RUN --mount=type=cache,target=/root/.cache/pip \
    pip install -r requirements.txt

This method allows the Pip download cache to be reused, which does not mean permanently writing the cache directory to the final image.

6. Multi-stage construction (Multi-stage Build)

Multi-stage builds are used to separate build dependencies from the final runtime environment.

Typical React + Vite:

FROM node:24-alpine AS builder

WORKDIR /app

RUN corepack enable

COPY package.json pnpm-lock.yaml ./
RUN pnpm install --frozen-lockfile

COPY . .
RUN pnpm build

FROM nginx:alpine

COPY --from=builder /app/dist /usr/share/nginx/html

Node Builder
    ↓
pnpm build
    ↓
dist/
    ↓ COPY --from
Nginx Image

The final image does not need to contain all build dependencies of the Node.js project.

COPY --chown

COPY --chown=appuser:appuser . /app

The owner can be set when copying files, suitable for containers run by users other than Root.

7. Volume and Bind Mount

Named Volume

Data volumes managed by Docker:

docker volume create postgres_data

Mount:

docker run \
  -v postgres_data:/var/lib/postgresql/data \
  postgres:17

The path here is for the PostgreSQL 17 official image.

Volume will not be automatically deleted due to normal deletion of the container, and is suitable for database persistence.

Bind Mount

Mount the host directory into the container:

docker run \
  -v ./src:/app/src \
  myapp:1.0

Host ./src
    ↕
Container /app/src

Suitable for local development and configuration file mounting.

difference

Note: In the production environment, do not arbitrarily change the entire source code directory Bind Mount to /app, otherwise the files written during image building may be overwritten.

8. Docker Network

Containers can communicate over the Docker network.

Docker Compose will create a network for the project by default, and containers in the same network can access each other through service names.

For example:

backend
  ↓ db:5432
PostgreSQL

backend
  ↓ redis:6379
Redis

Django database configuration usually uses:

DB_HOST=db
DB_PORT=5432

instead of:

DB_HOST=localhost

Because in a container, localhost usually refers to the container itself.

ports and expose

ports:
  - "8000:8000"

Publish to the host machine.

expose:
  - "8000"

Declare the container port but do not publish it to the host.

For service communication on the same Compose network, usually you can directly access the port actually listened by the container without writing expose.

9. Docker Compose

Compose defines multiple services in one YAML file.

For example:

services:
  backend:
    build:
      context: ./backend
      dockerfile: Dockerfile
      target: prod
      args:
        APP_VERSION: "1.2.0"
    image: sorivel-backend:1.2.0
    command: >
      gunicorn config.asgi:application
      -k uvicorn_worker.UvicornWorker
      --bind 0.0.0.0:8000
    env_file:
      - ./backend/.env
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_started
    restart: unless-stopped
    networks:
      - backend_network
      - frontend_network

  db:
    image: postgres:17
    environment:
      POSTGRES_DB: app
      POSTGRES_USER: app
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?Password required}
    volumes:
      - postgres_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app"]
      interval: 5s
      timeout: 5s
      retries: 5
      start_period: 20s
    networks:
      - backend_network

  redis:
    image: redis:7
    networks:
      - backend_network

volumes:
  postgres_data:

networks:
  backend_network:
  frontend_network:

Here is a structure example. The actual project also needs to configure Nginx, static resources, Redis persistence and key management as needed.

Important fields

depends_on

depends_on:
  db:
    condition: service_healthy

Indicates waiting for Health Check of DB to pass before starting Backend.

However, the database is not automatically guaranteed to be available throughout the entire application runtime. Django still needs to handle database connection failures and reconnections appropriately.

10. Environment Variables

Two mechanisms need to be distinguished here.

Compose interpolation variable

environment:
  APP_PORT: ${APP_PORT:-8000}

Compose parses variables from Shell, item .env, etc.

Among them:

• ${APP_PORT:-8000}: 8000 is used when not set or empty.
• ${VAR:?Required}: Not set or an error will be reported if it is empty.

env_file

env_file:
  - ./backend/.env

Inject variables from the file into the Backend container.

important difference

Compose .env
    ↓
Compose Configuration variable interpolation

services.backend.env_file
    ↓
Backend Container environment variables

The two are not the same thing.

When variables with the same name are specified in both environment and env_file, the explicit environment has higher priority.

You can check:

docker compose config

However, this command may output parsed sensitive information, and the complete results should not be submitted to the log or shared publicly.

11. Container life cycle

Created
   ↓
Running
   ↓
Stopped / Exited
   ↓
Removed

When the container's main process exits, the container usually exits as well.

For example:

CMD ["gunicorn", "..."]

Gunicorn is the main process; when it exits, the container ends.

Restart Policy

restart: unless-stopped

Indicates that after the container exits, Docker will try to restart according to the corresponding policy, unless it has been manually stopped.

It is not a guarantee that the container will automatically restart after Health Check failure.

PID 1 and signal

When a container stops, it usually sends a termination signal to the main process and then waits for a period of time.

Therefore it is recommended:

• Start the application using Exec Form.
• Make sure the app supports graceful exit.
• Don't let unnecessary Shell block the termination signal.
• Use init: true to handle child process recycling when necessary.

12. Docker Compose Common commands

Status and logs

docker compose ps

docker compose logs

docker compose logs -f backend

docker stats

Start and stop

# start
docker compose up -d

# Stop but keep container
docker compose stop

# Stop and delete the service container and network
docker compose down

# Delete simultaneously Compose Management related Volume
docker compose down -v

Named Volume where the production database resides should not be arbitrarily executed down -v.

Build and update

# Build only Backend mirror
docker compose build backend

# Don't use build cache
docker compose build --no-cache backend

# Build and start
docker compose up -d --build

# Update only Backend service
docker compose up -d --build backend

# Force container rebuild
docker compose up -d --force-recreate backend

If you modify Dockerfile or the source code inside the image, you should build a new image before starting it.

If you only modify the environment variables injected through env_file, you can re-create the container:

docker compose up -d --force-recreate backend

If you modify the Compose configuration:

docker compose up -d

Configuration changes are typically identified and corresponding services recreated as needed.

Enter the container

docker compose exec backend sh

docker compose exec backend python manage.py shell

docker compose exec backend python manage.py migrate

Not all container images contain Bash, especially Alpine or thin images, so sh is usually more versatile.

exec and run

docker compose exec backend python manage.py shell

Execute commands in an existing running container.

docker compose run --rm backend python manage.py shell

Create a new temporary service container to execute the command.

The latter is suitable for one-time management tasks, but its network, port, and lifecycle behavior are not exactly the same as exec.

13. Mirror repository (Registry)

Common mirror warehouses:

• Docker Hub。
• GitHub Container Registry（GHCR）。
• AWS ECR。

Example of image name:

ghcr.io/username/sorivel-backend:1.0.0

Build and push:

docker build \
  -t ghcr.io/username/sorivel-backend:1.0.0 \
  .

docker push ghcr.io/username/sorivel-backend:1.0.0

Pull:

docker pull ghcr.io/username/sorivel-backend:1.0.0

Multi-architecture build

docker buildx build \
  --platform linux/amd64,linux/arm64 \
  -t ghcr.io/username/sorivel-backend:1.0.0 \
  --push \
  .

Images corresponding to multiple CPU architectures can be built and pushed.

You need to use Buildx Builder that supports the target platform build, and perform cross-architecture builds through mechanisms such as QEMU when necessary.

14. Health Check

Dockerfile：

HEALTHCHECK --interval=30s --timeout=5s \
  CMD python -c \
  "import urllib.request; urllib.request.urlopen('http://localhost:8000/healthz')"

It can also be defined in Compose:

healthcheck:
  test:
    - CMD
    - python
    - -c
    - "import urllib.request; urllib.request.urlopen('http://localhost:8000/healthz')"
  interval: 30s
  timeout: 5s
  retries: 3

Health Check is used to determine whether the service can respond normally.

Common status:

starting
healthy
unhealthy

Note: The container is in Running status, which does not mean that the application must be Healthy.

15. Secrets and security

Do not write the password directly into Dockerfile or submit it to the Git repository.

In particular avoid:

ARG DATABASE_PASSWORD=secret
ENV DATABASE_PASSWORD=secret

If you need to access private resources during the build phase, you can use BuildKit Secret Mount.

For example:

RUN --mount=type=secret,id=pip_token \
    ...

For runtime keys, they can be injected using the Secret management capabilities of the deployment platform, or via configuration files protected with appropriate permissions.

Other suggestions:

• Try to run the application as a non-Root user.
• Internal service ports such as PostgreSQL and Redis are not exposed to the public network.
• Use an explicit image version instead of unconditionally relying on latest.
• Update the base image regularly.
• Establish an independent backup mechanism for PostgreSQL Volume.

16. Database operation and backup

Execute Migration:

docker compose exec backend python manage.py migrate

View Volume:

docker volume ls

PostgreSQL export

docker compose exec -T db \
  pg_dump -U app -d app > backup.sql

PostgreSQL import

docker compose exec -T db \
  psql -U app -d app < backup.sql

It is assumed here that the Compose service name is db, and the database and user are both app.

Before performing the import, you need to confirm the target database status and backup format. SQL Import does not equal a safe and complete database migration process.

Volume is not equal to Backup

Volume is responsible for persistence, but cannot prevent:

• Data deleted by mistake.
• The database logic is damaged.
• Disk failure.
• Error Migration.
• The host machine is lost.

Production databases still need to be backed up regularly and recovery procedures verified.

17. Cleanup command

# View image
docker images

# View container
docker ps -a

# View Volume
docker volume ls

# Delete specified image
docker rmi image_name

# Clean build cache no longer in use
docker builder prune

# Clean all unused build caches
docker builder prune -a

Complete reset of the development environment:

docker compose down -v --rmi all --remove-orphans
docker compose up -d --build

This operation is destructive, especially -v will delete the related Volume managed by Compose. Do not execute directly in an environment containing production data.

Summary

The core content of Docker can be divided into five parts:

Typical process of daily deployment:

Write Dockerfile
        ↓
docker compose build
        ↓
Docker Images
        ↓
docker compose up -d
        ↓
Running Containers
        ↓
Health Check / Logs
        ↓
Application

The most important practical principles:

1. The image is responsible for packaging the application, and the container is responsible for running it.
2. Database data must be persisted and backed up.
3. For production deployment, try to avoid directly mounting the entire source code directory.
4. Inter-container communication via Docker Network and service name.
5. Modifying the image content requires rebuilding and updating the container.
6. Do not write sensitive information to images or repositories.

---

Reference documentation

• Docker Overview
• Dockerfile Reference
• Docker Compose
• Volumes
• Networking
• Build Cache
• PostgreSQL Official Image