Fix: Docker Secrets Not Working — BuildKit --secret Not Mounting, Compose Secrets Undefined, or Secret Leaking into Image

The Problem

BuildKit secret mounts don’t appear inside the container during build:

# Dockerfile
RUN --mount=type=secret,id=npm_token \
    cat /run/secrets/npm_token  # Empty or file not found

docker build --secret id=npm_token,src=.npmrc .
# Error: secret file not found: /run/secrets/npm_token

Or Docker Compose secrets aren’t accessible in the container:

# docker-compose.yml
services:
  app:
    secrets:
      - db_password

secrets:
  db_password:
    file: ./secrets/db_password.txt

docker compose up
# cat /run/secrets/db_password  # Empty or permission denied

Or a secret was used during build and ended up in the image layers:

docker history myimage
# IMAGE          CREATED_AT     CREATED_BY
# abc123         2 min ago      RUN npm install  # Secret was ARG — leaked!

Why This Happens

Docker has two separate secret mechanisms that are often confused:

• Build-time secrets (BuildKit --mount=type=secret) — secrets mounted during docker build. They’re available only in the specific RUN instruction that mounts them, never baked into image layers. Requires BuildKit to be enabled.
• Runtime secrets (Docker Compose / Swarm secrets) — secrets mounted into the container filesystem at /run/secrets/<name> when the container runs. These are a runtime mechanism — they don’t exist during the build phase.
• ARG and ENV are not secrets — any value passed as ARG or ENV is stored in the image layer history and visible with docker history. Never use them for credentials.
• BuildKit not enabled — --mount=type=secret requires BuildKit. Without it, the --mount syntax is silently ignored or causes a build error.

Fix 1: Use BuildKit Secret Mounts for Build-Time Credentials

BuildKit secret mounts let you access credentials during build without baking them into the image:

# Enable BuildKit (required)
export DOCKER_BUILDKIT=1

# Or use the docker buildx command (always uses BuildKit)
docker buildx build .

# syntax=docker/dockerfile:1
FROM node:20-alpine

WORKDIR /app
COPY package*.json ./

# Mount the secret for this RUN only — never stored in layers
RUN --mount=type=secret,id=npm_token \
    NPM_TOKEN=$(cat /run/secrets/npm_token) \
    npm config set //registry.npmjs.org/:_authToken=$NPM_TOKEN && \
    npm ci && \
    npm config delete //registry.npmjs.org/:_authToken

COPY . .
RUN npm run build

# Pass the secret from a file
docker build --secret id=npm_token,src=$HOME/.npmrc .

# Or from an environment variable
docker build --secret id=npm_token,env=NPM_TOKEN .

# Or using buildx
docker buildx build --secret id=npm_token,src=$HOME/.npmrc .

Multiple secrets:

# syntax=docker/dockerfile:1
FROM python:3.12-slim

WORKDIR /app
COPY requirements.txt .

RUN --mount=type=secret,id=pip_config \
    pip install \
        --config-file /run/secrets/pip_config \
        -r requirements.txt

docker build \
    --secret id=pip_config,src=./pip.conf \
    --secret id=ca_cert,src=./ca.crt \
    .

Important: The # syntax=docker/dockerfile:1 comment at the top of your Dockerfile enables the BuildKit frontend. Without it, --mount=type=secret may not be recognized even when BuildKit is enabled.

Fix 2: Fix Docker Compose Runtime Secrets

Docker Compose secrets are injected at container runtime, not build time:

# docker-compose.yml
version: "3.8"

services:
  app:
    image: myapp:latest
    secrets:
      - db_password
      - api_key
    environment:
      # Reference secret path in app config, not the secret itself
      DB_PASSWORD_FILE: /run/secrets/db_password
    ports:
      - "3000:3000"

  db:
    image: postgres:16
    secrets:
      - db_password
    environment:
      POSTGRES_PASSWORD_FILE: /run/secrets/db_password
      POSTGRES_USER: myapp

secrets:
  db_password:
    file: ./secrets/db_password.txt  # Plain text file with the secret value

  api_key:
    environment: API_KEY  # Read from host environment variable

# Create the secrets directory (never commit this)
mkdir -p secrets
echo -n "mysecretpassword" > secrets/db_password.txt

# Add to .gitignore
echo "secrets/" >> .gitignore

docker compose up

Access secrets inside the container:

# Secrets are mounted at /run/secrets/<secret-name>
docker exec myapp cat /run/secrets/db_password

# In your application code

# Python — read secret from file
import os

def get_secret(name: str) -> str:
    secret_path = f"/run/secrets/{name}"
    if os.path.exists(secret_path):
        with open(secret_path) as f:
            return f.read().strip()
    # Fallback to environment variable for local dev
    return os.environ.get(name.upper(), "")

db_password = get_secret("db_password")

// Node.js — read secret from file
import { readFileSync, existsSync } from "fs";

function getSecret(name) {
  const secretPath = `/run/secrets/${name}`;
  if (existsSync(secretPath)) {
    return readFileSync(secretPath, "utf8").trim();
  }
  return process.env[name.toUpperCase()] ?? "";
}

const dbPassword = getSecret("db_password");

Docker Swarm secrets (production):

# Create a secret in Swarm
echo "mysecretpassword" | docker secret create db_password -

# Or from a file
docker secret create db_password ./secrets/db_password.txt

# List secrets (values are never shown)
docker secret ls

# Use in a service
docker service create \
    --name myapp \
    --secret db_password \
    myapp:latest

Fix 3: Verify Secrets Don’t Leak into Image Layers

After building, check that credentials aren’t stored in the image:

# Check image history for sensitive values
docker history --no-trunc myimage | grep -i "token\|password\|secret\|key"

# Inspect all layers
docker save myimage | tar -tv | grep -v "^d"

# Scan with dive (interactive layer inspector)
docker run --rm -it \
    -v /var/run/docker.sock:/var/run/docker.sock \
    wagoodman/dive myimage

Common mistakes that leak secrets:

# WRONG — ARG values are visible in docker history
ARG NPM_TOKEN
RUN npm config set //registry.npmjs.org/:_authToken=$NPM_TOKEN && npm ci

# WRONG — ENV persists in the image
ENV NPM_TOKEN=mytoken
RUN npm ci

# WRONG — copying a file with secrets into the image
COPY .npmrc .
RUN npm ci
# The .npmrc is now in the layer even if you RUN rm .npmrc later

# CORRECT — BuildKit secret mount (never stored)
RUN --mount=type=secret,id=npm_token \
    NPM_TOKEN=$(cat /run/secrets/npm_token) \
    npm config set //registry.npmjs.org/:_authToken=$NPM_TOKEN && \
    npm ci && \
    npm config delete //registry.npmjs.org/:_authToken

Multi-stage builds for isolation:

# syntax=docker/dockerfile:1

# Build stage — has access to secrets
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./

RUN --mount=type=secret,id=npm_token \
    NPM_TOKEN=$(cat /run/secrets/npm_token) \
    npm config set //registry.npmjs.org/:_authToken=$NPM_TOKEN && \
    npm ci

COPY . .
RUN npm run build

# Production stage — clean image, no secrets
FROM node:20-alpine AS production
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
# Secrets never touch this stage
EXPOSE 3000
CMD ["node", "dist/index.js"]

Fix 4: Integrate Secrets in CI/CD Pipelines

Never store secrets in Dockerfiles or docker-compose files committed to version control:

GitHub Actions:

# .github/workflows/build.yml
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Build with secret
        run: |
          docker buildx build \
            --secret id=npm_token,env=NPM_TOKEN \
            --tag myimage:latest \
            --push \
            .
        env:
          NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
          # NPM_TOKEN is set in GitHub repo Settings → Secrets

GitLab CI:

# .gitlab-ci.yml
build:
  stage: build
  script:
    - docker buildx build
        --secret id=npm_token,env=NPM_TOKEN
        --tag $CI_REGISTRY_IMAGE:latest
        --push
        .
  variables:
    DOCKER_BUILDKIT: "1"
  # NPM_TOKEN set in GitLab CI/CD Settings → Variables

Using Docker config for registry auth (not a build-time secret):

# Log in to a private registry — stores credentials in ~/.docker/config.json
docker login registry.example.com -u myuser -p mypassword

# Build and push — registry auth is handled by Docker daemon, not build secrets
docker buildx build --push --tag registry.example.com/myimage:latest .

Fix 5: Use External Secret Managers

For production, pull secrets from a secret manager rather than files or environment variables:

AWS Secrets Manager:

# syntax=docker/dockerfile:1
FROM python:3.12-slim

# Install AWS CLI or SDK at build time
RUN pip install boto3

COPY entrypoint.sh /entrypoint.sh
COPY app.py /app.py

# Fetch secrets at runtime in the entrypoint
ENTRYPOINT ["/entrypoint.sh"]

#!/bin/sh
# entrypoint.sh — fetch secrets at container start
DB_PASSWORD=$(aws secretsmanager get-secret-value \
    --secret-id prod/myapp/db_password \
    --query SecretString \
    --output text)
export DB_PASSWORD

exec python app.py

HashiCorp Vault with Docker:

# docker-compose.yml with Vault Agent sidecar
services:
  vault-agent:
    image: hashicorp/vault:latest
    command: agent -config=/vault/config/agent.hcl
    volumes:
      - ./vault-agent.hcl:/vault/config/agent.hcl
      - secrets-volume:/secrets

  app:
    image: myapp:latest
    volumes:
      - secrets-volume:/run/secrets:ro
    depends_on:
      - vault-agent

volumes:
  secrets-volume:

Kubernetes Secrets (for K8s deployments):

# k8s-secret.yaml — create secret
apiVersion: v1
kind: Secret
metadata:
  name: myapp-secrets
type: Opaque
data:
  db_password: bXlzZWNyZXRwYXNzd29yZA==  # base64 encoded
  api_key: c29tZWFwaWtleQ==

---
# deployment.yaml — mount as files or env vars
apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      containers:
        - name: myapp
          image: myapp:latest
          # Mount as files (more secure)
          volumeMounts:
            - name: secrets
              mountPath: /run/secrets
              readOnly: true
          # Or inject as environment variables
          env:
            - name: DB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: myapp-secrets
                  key: db_password
      volumes:
        - name: secrets
          secret:
            secretName: myapp-secrets

Fix 6: Handle .env Files and Local Development

For local development, use .env files with Docker Compose — but never commit them:

# .env — local development only, never commit this
DB_PASSWORD=localdevpassword
API_KEY=devkey123
NPM_TOKEN=mytoken

# docker-compose.yml — pass env vars to container
services:
  app:
    image: myapp:latest
    env_file:
      - .env  # Docker Compose reads this automatically
    # Or explicitly:
    environment:
      - DB_PASSWORD=${DB_PASSWORD}
      - API_KEY=${API_KEY}

# .gitignore — always exclude secret files
.env
.env.local
.env.production
secrets/
*.pem
*.key

Template for shared development setup:

# .env.example — commit this as a template
DB_PASSWORD=changeme
API_KEY=your_api_key_here
NPM_TOKEN=your_npm_token_here

# README instructions for new developers:
# cp .env.example .env
# Fill in real values in .env

Validate secrets are present before starting:

#!/bin/sh
# docker-entrypoint.sh — fail early if required secrets are missing

required_secrets="db_password api_key"

for secret in $required_secrets; do
    if [ ! -f "/run/secrets/$secret" ] && [ -z "$(eval echo \$$secret)" ]; then
        echo "ERROR: Required secret '$secret' is not set"
        echo "  Mount it as a Docker secret at /run/secrets/$secret"
        echo "  Or set the environment variable $secret"
        exit 1
    fi
done

exec "$@"

Still Not Working?

/run/secrets/ directory doesn’t exist in the container — this directory is created by Docker when secrets are mounted. If your container doesn’t use Docker Compose secrets or Swarm secrets, the directory won’t be there. For standalone containers (docker run), mount secrets manually with -v or use environment variables. The /run/secrets/ convention is a Docker feature, not a system default.

BuildKit secret mount path — default is /run/secrets/<id> — if your RUN command looks for the secret in a different location, it won’t find it. The default mount point is always /run/secrets/<id> where <id> is the --secret id=<id> value. Override with target=:

RUN --mount=type=secret,id=npm_token,target=/root/.npmrc \
    npm ci
# Secret is now at /root/.npmrc instead of /run/secrets/npm_token

docker compose up uses old secrets after changes — Docker Compose caches container state. If you changed the secret file, recreate the containers:

docker compose down
docker compose up --force-recreate

Permission denied on /run/secrets/<name> — secrets are owned by root with mode 0400 by default. If your process runs as a non-root user, either change the secret permissions in your Compose config or read it as root in the entrypoint and export as an environment variable:

secrets:
  db_password:
    file: ./secrets/db_password.txt
    # Docker Swarm supports uid/gid/mode settings:
    # mode: 0440
    # uid: "1000"

For related Docker issues, see Fix: Docker Build Failing and Fix: Docker Healthcheck Not Working.