kogutdocs

Agent

Permissions and the Docker socket

Agent permissions, Docker socket access, and data sent to kogut.

The agent runs as an unprivileged user by default. Host metrics and systemd states need no extra permissions. Container access and incident logs need these groups:

PermissionAllowsWithout it
Docker socket access, usually group dockerContainer states, resources, and incident logsNo container data
Group systemd-journalIncident logs from systemd unitsNo unit logs

Matching a systemd service to its listening port needs read access to the process’s /proc/<pid>/fd, which for other users’ processes means root. Without it, the agent still finds running services with unit files in /etc/systemd/system.

Systemd protections

The install script’s unit uses:

  • DynamicUser=yes
  • SupplementaryGroups with the Docker and journal groups that exist on the host
  • NoNewPrivileges=yes
  • ProtectSystem=strict and ProtectHome=read-only
  • StateDirectory=kogut-agent for persistent state
  • PrivateTmp=yes

The unit grants CAP_NET_RAW, which the agent does not use. Ping checks use unprivileged ICMP sockets and need the agent’s group in net.ipv4.ping_group_range; see Ping checks.

To clear the extra groups and the capability:

sudo systemctl edit kogut-agent
[Service]
SupplementaryGroups=
AmbientCapabilities=

Docker socket access

The Docker socket grants control of the host, and a read-only mount does not restrict API requests. The agent sends only GET requests, but a compromised process with socket access could send writes. A socket proxy limits access to container and image reads:

docker run -d --name docker-proxy --restart unless-stopped \
	-e CONTAINERS=1 -e IMAGES=1 -e POST=0 \
	-v /var/run/docker.sock:/var/run/docker.sock:ro \
	-p 127.0.0.1:2375:2375 \
	tecnativa/docker-socket-proxy:latest

IMAGES=1 lets the agent read image digests for the rollback hint; without it, deploys record the image id instead.

With DOCKER_HOST=tcp://127.0.0.1:2375, the agent needs no socket access. To remove its Docker group, override with an empty SupplementaryGroups= followed by SupplementaryGroups=systemd-journal. For the container, see Docker.

When the agent cannot read a source

With every sample, the agent reports each source as ok, denied, absent (not on the host, or not where the agent looks), or timeout. When a source stops working, the agent logs one warning with the fix, and one line when it works again. The host page shows a line for each source that needs attention and keeps the last container and unit lists, with their time.

SourceProblemFix
containersdenied: permission denied on /var/run/docker.sockAdd the agent to the socket’s group. In Compose, group_add with $(stat -c %g /var/run/docker.sock). Or use a socket proxy with DOCKER_HOST.
containersabsent: /var/run/docker.sock is not a socketDocker creates a directory when a bind mount’s source is missing. Mount the real socket, or set DOCKER_HOST.
containersabsent: nothing listens on ...Start Docker or Podman, or point DOCKER_HOST at the running one.
units, journalabsent: systemctl not found, journalctl not foundNormal in the container image. For unit states and logs, install on the host.
journaldeniedAdd the agent to the group systemd-journal for unit logs in incidents.
disksabsent or denied: .../proc/self/mountinfo: ...The agent cannot read the mount table. In the container image, mount the host’s root at /host and set KOGUT_HOST_ROOT=/host, as in Docker.
diskstimeoutA filesystem does not answer, often a dead network mount. The agent tries again with each sample.
tempsabsent/sys/class/hwmon is missing. Virtual servers usually have no sensors.

A missing systemctl, journal, or sensor directory is normal on some hosts; the page does not show it and the agent logs it without a warning.

The agent inside the server

With KOGUT_LOCAL_AGENT=1, the agent runs inside kogut serve with the server’s user and permissions, so the server needs the groups above. For the install script’s unit, add both in an override:

sudo systemctl edit kogut
[Service]
Environment=KOGUT_LOCAL_AGENT=1
SupplementaryGroups=docker systemd-journal

The server container needs the host mounts, pid: host, host networking, and the socket group instead. These widen what a compromised server process could do on the host; to avoid that, run the agent on its own.

Agent data

The agent sends host facts, metrics, container and unit states, check results, and discovery results. Per-container and per-unit CPU and memory come from cgroups, without command lines or process lists. For an incident, the server can request logs: up to 200 lines and 64 KiB per container or unit, from 10 minutes before the first failure. Container logs match the service name or its Compose service or project label; unit logs come from the unit named after the service.

Logs are sent without redaction, so secrets in logs reach kogut. The agent sends no other logs, file contents, environment variables, or command lines.

The server can set intervals, assign checks, and request logs. It cannot run commands, restart services, or change host files. Apart from systemctl and journalctl, the agent runs only the command checks in a file on the host that it cannot change, and sends only their names, services, intervals, and results. See No code from the server.