Monitoring
Heartbeats
Detect missed or failed jobs with heartbeat pings.
A heartbeat catches a missed or failed cron job, backup, or worker run. The job calls a secret URL, and kogut marks the heartbeat down when the ping is late or reports failure.
Create the heartbeat
- Open Add check on Overview or a service’s page. Choose Heartbeat.
- Choose a period or a five-field cron schedule. Panel cron schedules use UTC.
- Set the grace period to cover the job’s runtime and any expected delay.
For a cron schedule in local time, set its timezone in kogut.toml:
[[heartbeat]]
name = "Nightly backup"
service = "postgres"
cron = "0 3 * * *"
timezone = "Europe/Warsaw"
grace = "1h"For a periodic worker, use every instead of cron. Syntax, defaults, and limits are in
the heartbeat reference. A new heartbeat stays
quiet until its first ping.
Copy the ping URL
- As an admin, click Ping URL in the heartbeat’s row on the service’s page.
- Click Create ping URL and copy the URL. kogut shows it only once.
- Store it as a secret in the job’s environment, for example as
KOGUT_PING_URL.
Set the server’s public URL first; otherwise the URL uses the address you opened the panel with. Creating another URL invalidates the old one. Anyone with the URL can report success or failure, but cannot read data.
Through the API, with a write token and the heartbeat’s public id from
GET /api/v1/services/<name>:
curl -fsS -X POST "https://kogut.acme.com/api/v1/checks/<id>/ping-url" \
-H "Authorization: Bearer $KOGUT_TOKEN"The response contains url and replaced, which is true when an earlier URL existed.
Keep an existing push URL
Jobs that call a push URL in the form /api/push/<token>?status=up can keep it. Set the
token on the heartbeat in kogut.toml and change only the host in the job’s URL:
[[heartbeat]]
name = "Nightly backup"
service = "postgres"
every = "1d"
ping_token = "${NIGHTLY_BACKUP_PUSH_TOKEN}"Both GET and POST work:
- No
status, an empty one, orstatus=upreports success. Any other value, or more than onestatus, reports failure. msgandpingare accepted and ignored.- An accepted ping returns
200with{"ok":true}; an unknown token returns404with{"ok":false,"msg":"..."}.
The same token also works at /ping/<token>, and a panel ping URL works at
/api/push/<token>. A heartbeat can have both a ping_token and a panel URL; creating a
panel URL replaces only the earlier panel URL. A ping_token that another heartbeat’s
panel URL uses is an error, and the file is not applied. Removing ping_token stops its
token at the next reload.
Report the job’s result
Call the URL after success. Add /start to report a start, /fail for failure, or an
exit code from /0 to /255, where only 0 is success. Both GET and POST work.
Accepted pings return 200 with OK, unknown tokens 404. Send pings to kogut’s own
address, not to the status page domain.
On a host with the kogut binary, kogut run sends
/start, runs the command, sends its exit code, and exits with it:
0 3 * * * KOGUT_PING_URL=https://kogut.acme.com/ping/btp_... kogut run -- pg_dump -Fc -f /var/backups/acme.dump acmeGive the URL in KOGUT_PING_URL, not --ping: other users can see command-line
arguments in ps.
Elsewhere, use curl, and ping only after the command succeeds:
0 3 * * * /usr/local/bin/backup.sh && curl -fsS -m 10 --retry 5 "$KOGUT_PING_URL"To catch a job that starts but never finishes, send /start before the command and its
exit code after:
#!/bin/sh
curl -fsS -m 10 --retry 5 "$KOGUT_PING_URL/start" >/dev/null
pg_dump -Fc acme > /var/backups/acme.dump
code=$?
curl -fsS -m 10 --retry 5 "$KOGUT_PING_URL/$code" >/dev/null
exit "$code"The start ping sets a finish deadline one grace period later. A failed or late heartbeat goes down without confirmation, and the next successful ping restores it.
Use a systemd timer
Wrap the service’s command with kogut run:
[Service]
Type=oneshot
EnvironmentFile=/etc/kogut/backup.env
ExecStart=/usr/local/bin/kogut run -- /usr/local/bin/backup.shWithout the kogut binary, use curl:
[Service]
Type=oneshot
EnvironmentFile=/etc/kogut/backup.env
ExecStartPre=-/bin/sh -c 'curl -fsS -m 10 --retry 5 "$KOGUT_PING_URL/start"'
ExecStart=/usr/local/bin/backup.sh
ExecStartPost=-/bin/sh -c 'curl -fsS -m 10 --retry 5 "$KOGUT_PING_URL"'
ExecStopPost=/bin/sh -c 'test "$SERVICE_RESULT" = success || curl -fsS -m 10 --retry 5 "$KOGUT_PING_URL/fail"'Put KOGUT_PING_URL in /etc/kogut/backup.env. The - prefix keeps a failed ping from
failing the job.
Use GitHub Actions
Store the URL as a repository secret and report the result even when earlier steps fail:
- name: Report to kogut
if: always()
run: curl -fsS -m 10 --retry 5 "${{ secrets.KOGUT_PING_URL }}/${{ job.status == 'success' && '0' || '1' }}"