Lesson 08 of 13 · Level 4 — Daemon & certificates
Configuring the daemon: daemon.json
daemon.json is where the whole Docker host is configured: log rotation, where data lives, registry mirrors, network ranges, proxies and more. Learn the settings that matter, how to validate a change, which ones apply with a reload, and how to recover when Docker won't start.
What daemon.json is
dockerd reads /etc/docker/daemon.json when it starts. It's the one place to set host-wide behaviour: how logs are rotated, where images and volumes are stored, which registries are mirrored or trusted, which IP ranges Docker may use, and more. It doesn't exist until you create it. Rootless Docker reads ~/.config/docker/daemon.json instead, and Docker Desktop has an editor for it under Settings → Docker Engine.
The house rules pinned on the fridge. Everyone who moves in (every new container) follows them. People already living there (existing containers) keep the rules they signed up with, until they move out and back in.
A sensible starting file
{
"log-driver": "json-file",
"log-opts": {
"max-size": "10m",
"max-file": "3"
},
"live-restore": true,
"default-address-pools": [
{ "base": "10.200.0.0/16", "size": 24 }
],
"bip": "10.199.0.1/24",
"registry-mirrors": ["https://registry.lab.local:5001"]
}
Plain JSON: no comments, no trailing commas, and values in log-opts must be strings ("3", not 3).
The settings that matter
| Setting | What it does | Why you'd set it |
|---|---|---|
log-driver, log-opts |
Default logging for new containers | Stop *-json.log files filling the disk (the default has no size limit) |
data-root |
Where images, containers and volumes live (default /var/lib/docker) |
Put Docker on a bigger or faster disk |
storage-driver |
How layers are stored (overlay2 is the modern default) |
Rarely: only when the default isn't supported |
registry-mirrors |
Mirrors tried first for Docker Hub pulls | Pull-through cache, rate limits, speed |
insecure-registries |
Registries that may skip TLS verification or use HTTP | Labs only. See the next lesson |
default-address-pools |
Subnets for new networks | Avoid clashes with VPNs, VPCs, office LANs |
bip |
IP and subnet of the default docker0 bridge |
Same reason |
dns, dns-search |
DNS servers given to containers | Hosts where the containers can't use the host's resolver |
live-restore |
Keep containers running while dockerd restarts | Upgrade Docker without downtime |
exec-opts: ["native.cgroupdriver=systemd"] |
cgroup driver | Match systemd hosts (the default on cgroup v2 systems) |
proxies |
HTTP(S) proxy for the daemon's own pulls (Engine 23.0+) | Corporate networks |
userns-remap |
Map container root to an unprivileged host UID range | Extra isolation on multi-user hosts |
debug, log-level |
Daemon's own verbosity | Troubleshooting |
Apply a change safely
$ sudo cp /etc/docker/daemon.json /etc/docker/daemon.json.bak
$ sudoedit /etc/docker/daemon.json
$ sudo dockerd --validate --config-file /etc/docker/daemon.json
configuration OK
$ sudo systemctl reload docker # if every changed key is reloadable
$ sudo systemctl restart docker # otherwise
$ docker info | grep -A3 -E 'Logging Driver|Registry Mirrors|Docker Root Dir'
- Reload (
SIGHUP) applies a subset of settings without touching containers. It includes, among others,debug,log-level,labels,live-restore,registry-mirrors,insecure-registriesand themax-concurrent-*download/upload limits. Docker's documentation fordockerdlists the reloadable options for your version. - Restart is needed for everything else (
data-root,bip,default-address-pools,log-driver,storage-driver...). Withoutlive-restore, a restart stops all containers; those with a restart policy come back up. - Log and default-network settings apply only to containers created afterwards. Recreate existing ones to pick them up.
Moving data-root
Changing data-root doesn't move anything: after the restart Docker starts with an empty store, and your images, containers and volumes seem to have vanished. Stop Docker, copy the old directory with rsync -aHAX /var/lib/docker/ /new/path/, then set data-root and start it again.
Proxies for the daemon
On a network where the internet is behind a proxy, docker pull is done by dockerd, not your shell, so your shell's HTTPS_PROXY doesn't help. Two ways:
{
"proxies": {
"http-proxy": "http://proxy.corp.local:3128",
"https-proxy": "http://proxy.corp.local:3128",
"no-proxy": "localhost,127.0.0.0/8,registry.lab.local,.corp.local"
}
}
or, on older engines, a systemd drop-in:
# /etc/systemd/system/docker.service.d/http-proxy.conf
[Service]
Environment="HTTPS_PROXY=http://proxy.corp.local:3128"
Environment="NO_PROXY=localhost,127.0.0.0/8,registry.lab.local"
followed by sudo systemctl daemon-reload && sudo systemctl restart docker. Put your internal registries in no-proxy, or pulls from them go to the proxy and fail. Proxy settings for containers and builds are separate: they go in the client's ~/.docker/config.json under proxies.
When Docker won't start
$ sudo systemctl restart docker
Job for docker.service failed because the control process exited with error code.
$ journalctl -u docker -n 20 --no-pager
... unable to configure the Docker daemon with file /etc/docker/daemon.json: invalid character '}' looking for beginning of object key string
The three usual causes:
- Invalid JSON, usually a trailing comma.
dockerd --validateorpython3 -m json.toolpoints at it. - Unknown or misspelt key (
"registry-mirror"without the s). The error names the key. - The same setting as a flag and in the file, typically
hosts, because the distribution's unit startsdockerd -H fd://. Remove it from one place. To managehostsin daemon.json, overrideExecStartwith a systemd drop-in that has no-Hflag.
Restore the backup to get the host running again first, then fix the change calmly.
Try it: log rotation that actually applies
- Run
docker run -d --name chatty alpine:3.20 sh -c 'while true; do date; done'and watchsudo du -h $(docker inspect -f '{{.LogPath}}' chatty)grow. - Add the
log-driver/log-optsblock to daemon.json, validate it and restart Docker. - The log keeps growing:
docker inspect -f '{{.HostConfig.LogConfig}}' chattystill shows the old settings. - Recreate the container. Its LogConfig now shows
max-size:10mand the file stops at 10 MB. - Break the JSON on purpose (add a trailing comma), try
dockerd --validate, then restore the backup.
Going deeper: Kubernetes nodes don't read daemon.json
Since Kubernetes 1.24 removed dockershim, Kubernetes nodes run containerd (or CRI-O) directly, and Docker's daemon.json has no effect on pods. The equivalents live in containerd's config.toml and per-registry hosts.toml files (mirrors, certificates), and log rotation is done by the kubelet (containerLogMaxSize, containerLogMaxFiles). Images built with Docker run unchanged; only the host configuration moves. The next lesson shows the containerd side of registry certificates.
Recap
/etc/docker/daemon.jsonconfigures the whole host. Plain JSON, strings inlog-opts.- Always set log rotation; consider
live-restore,default-address-pools/bip,registry-mirrors,data-root. - Validate before applying. Reload covers a subset of keys; the rest needs a restart. Log/network defaults need containers recreated.
- Won't start?
journalctl -u docker: bad JSON, unknown key, or a flag/file conflict (hosts).
This site is a public version of my personal engineering knowledge hub. It intentionally excludes confidential company information and internal operational details.