kogutdocs

Alerts

Notification channels

Configure and test incident notifications.

Add a channel in Settings → Notifications, or add [[alert]] to kogut.toml (all keys). Overview warns when there are fewer than two channels. Use two providers, so one provider’s outage does not stop all notifications.

Add a channel in the panel

  1. As an admin, choose Add channel.
  2. Enter a name, choose the type, and fill in its connection details.
  3. Save, then choose Send test. A test sends at once, without retries, one at a time. A failure appears beside the button and marks the channel as failing.

File channels are read-only in the panel. They are named by their name, or else by type and position, such as ntfy and ntfy-2; do not use these names for panel channels. A file channel with a panel channel’s name and the same settings takes it over; see Move panel objects into the file.

In Edit, an empty secret field keeps the stored secret. Remove the stored access token and Remove the stored signing secret clear optional secrets; clearing the username removes an SMTP password. Delete channel also drops its queued messages.

Panel secrets, including Slack and Discord URLs, are readable in the database and backups, but the panel and API never return them. Only admins and write tokens see ntfy topic and webhook URLs; viewers and read tokens see names, types, and delivery states. To keep secrets out of the database, use ${ENV_NAME} in kogut.toml.

ntfy

Subscribe to a hard-to-guess ntfy topic of 1 to 64 letters, digits, -, or _, and add its URL:

kogut.toml
[[alert]]
type = "ntfy"
url = "https://ntfy.sh/acme-ops-7f3k2"

For an access-controlled server, add token = "${NTFY_TOKEN}", sent as a bearer token. Messages are plain text, high priority for problems and default priority for recovery.

Slack

Create an incoming webhook for a Slack channel and store its URL in the environment:

kogut.toml
[[alert]]
type = "slack"
url = "${SLACK_WEBHOOK_URL}"

Discord

Create a webhook under the channel’s Integrations → Webhooks and store its URL:

kogut.toml
[[alert]]
type = "discord"
url = "${DISCORD_WEBHOOK_URL}"

Messages use embeds, red for problems and green for recovery, without mentions.

Telegram

  1. Send /newbot to @BotFather and copy the bot token.

  2. Add the bot to a group or start a chat with it, then send a message.

  3. Open https://api.telegram.org/bot<token>/getUpdates and copy chat.id. Group IDs start with -.

  4. Add the channel:

    kogut.toml
    [[alert]]
    type = "telegram"
    bot_token = "${TELEGRAM_BOT_TOKEN}"
    chat_id = "-1001234567890"

Email

Use an existing SMTP server:

kogut.toml
[[alert]]
type = "email"
smtp = "smtp.acme.com:587"
username = "kogut@acme.com"
password = "${SMTP_PASSWORD}"
from = "kogut@acme.com"
to = ["ops@acme.com", "oncall@acme.com"]

Use plain ASCII addresses without display names. TLS and authentication options are in configuration. kogut checks TLS certificates against public authorities and sends passwords only after TLS starts. For a local relay, omit credentials and set tls = "none" and allow_private = true.

One plain-text message goes to all recipients. Recipients the SMTP server refuses, even temporarily, are not retried and appear only in the server log, not as a failing channel.

Webhooks

The endpoint receives JSON POST requests:

kogut.toml
[[alert]]
type = "webhook"
url = "https://hooks.acme.com/kogut"
secret = "${KOGUT_WEBHOOK_SECRET}"
FieldContent
title, bodyTitle and plain-text message
typefailure, success for recovery, or info for a test
kindincident_opened, incident_updated, incident_reminder, incident_resolved, or incident_test for incident messages and tests
idempotency_keyincident:<id>:<transition>:<channel>, or test:<time> for a test
incidentid as a string, url, title, state (open or resolved), opened_at in UTC epoch milliseconds, root service, and root host. ID is null for tests. URL is null without a public URL. Service and host may be null.
failingSignals with check, service, since in server-local HH:MM, and error. Empty after resolution.
summaryCause sentence and update or reminder lines
causesObjects with rule, score from 0 to 1, and text
affectedOther affected services and signals
sent_atWhen this attempt was sent, in UTC epoch milliseconds. A retry has a newer value.

Every request carries X-Kogut-Timestamp, the sent_at value. idempotency_key stays the same across retries, and a timeout can deliver twice, so ignore keys you have already processed.

Verify the signature

With secret set, each request carries X-Kogut-Signature: sha256=<hex>. It is the HMAC-SHA256, keyed with the secret, of the X-Kogut-Timestamp value, a ., and the raw request body: {timestamp}.{body}. Without secret, the signature header is left out.

Before parsing JSON:

  1. Recompute the signature over the timestamp header and the raw request bytes, and compare it in constant time.
  2. Reject the request if the timestamp is more than 5 minutes from your clock, so a captured request cannot be replayed later.
  3. Drop idempotency_key values you have already processed, which stops repeats within the 5 minutes.
verify.js
import { createHmac, timingSafeEqual } from "node:crypto";

const TOLERANCE_MS = 5 * 60 * 1000;

export function verify(rawBody, signatureHeader, timestampHeader, secret) {
	const timestamp = Number(timestampHeader);
	if (!Number.isInteger(timestamp)) return false;
	if (Math.abs(Date.now() - timestamp) > TOLERANCE_MS) return false;
	const expected =
		"sha256=" +
		createHmac("sha256", secret).update(`${timestampHeader}.`).update(rawBody).digest("hex");
	const a = Buffer.from(expected);
	const b = Buffer.from(signatureHeader ?? "");
	return a.length === b.length && timingSafeEqual(a, b);
}

Apprise

To reach other notification services, use an Apprise API server. Its notify endpoint reads kogut’s title, body, and type and ignores the rest.

  1. Run Apprise with persistent configuration and a loopback port. It has no sign-in:

    docker run -d --name apprise -p 127.0.0.1:8000:8000 -v apprise-config:/config caronc/apprise
  2. Save your Apprise URLs, separated by commas, under a key:

    curl -d 'urls=pover://USER_KEY@APP_TOKEN' http://127.0.0.1:8000/add/kogut
  3. Point a webhook at that key:

    kogut.toml
    [[alert]]
    type = "webhook"
    url = "http://127.0.0.1:8000/notify/kogut"
    allow_private = true
  4. Choose Send test to check delivery through Apprise.

If both run in Docker, put them on a shared network, omit -p, and use http://apprise:8000/notify/kogut; loopback in kogut’s container does not reach Apprise.

Delivery and failures

Messages persist in a database queue. Delivery is ordered per channel. HTTP sends use JSON POST requests with a 10-second timeout and no redirects. SMTP uses one connection per message, with 10 s to connect and 60 s for the conversation.

Connection failures, timeouts, HTTP 408, 429, and 5xx, and SMTP 4xx replies retry after 1, 5, 15, and 60 min, five attempts in total. Other failures are permanent. A failing channel shows the reason in Settings → Notifications, with secrets hidden, and is never disabled. An older message about an incident never arrives after a newer one.

Private, loopback, link-local, and other non-public destinations are refused. kogut checks the address on every request and connects to that address. To reach an internal server, set allow_private = true on an ntfy, webhook, or email channel; Slack, Discord, and Telegram do not accept it.