Notifications

When something happens that a person should know about — a server goes offline, a daemon key is revoked, access is granted — TurboPanel writes it to the bell of everyone concerned, and delivers it to any channel a rule routes it to: an email address, a chat room, a webhook. This chapter is the bell, Account → Notifications (/account/notifications), and what an operator can expect from delivery.

Private alpha — the catalogue is short on purpose

Only events that the platform actually raises are listed; today that is six. Each new emitter joins the catalogue in the same commit as the code that raises it, so what the rules matrix offers is always what can arrive.

The model

Term	Meaning
Event	One thing that happened, with a stable code (`server.offline`), a severity (`info`, `warning`, `critical`) and a scope: it belongs to one organization, or to the control plane as a whole.
Inbox	Your own rows behind the bell. A server event lands in the inbox of every member of that organization; the events that mirror the audit trail — deletes, key revokes, grant changes — reach only its owners and managers; a control plane event reaches every administratoristrator. No setup needed.
Channel	Somewhere outside the app a notification can be delivered. Yours (they follow your account across organizations), the organization's (managers edit them), or the control plane's — an administrator's receivers that hear every event on the control plane, managed through the admin API for now. Kinds: `email`, `webhook`, `slack`, `discord`, `telegram`.
Rule	What a channel receives: every event at or above a severity floor, or a chosen list of events. A channel with no rule receives nothing.
Delivery	One attempt ledger per event × channel: written before the send, retried on failure with backoff, abandoned after five attempts.

Three facts:

A channel's address is a credential. A webhook URL's path is the secret, and so is a bot token. They are stored sealed under the control plane's data-encryption key and never shown again — the screen shows the origin, or the last few characters.
Addresses must be https with no credentials in the URL. That is the whole rule: a LAN receiver (an Alertmanager next to the control plane, a 10.x address, a bare hostname) is allowed on every runtime. TurboPanel High Availability cannot reach a private address, so a delivery there simply fails and is recorded as such.
Delivering never blocks the thing being reported. A webhook that is down does not stop a deploy or a sweep; it leaves a failed delivery that the maintenance tick retries.

The events

Code	Severity	Scope	Inbox	When
`server.offline`	critical	organization	every member	The daemon stopped answering and the server was marked offline.
`fleet.mass_disconnect`	critical	instance	administrators	A whole sweep lost all its servers at once — usually the control plane's own network or broker.
`server.deleted`	info	organization	owners and managers	A server was deleted from the app.
`server.daemon_key_revoked`	warning	organization	owners and managers	A server's daemon key was revoked.
`access.grant_created`	info	organization	owners and managers	A permission grant was added.
`access.grant_revoked`	warning	organization	owners and managers	A permission grant was removed.

The bell

The bell in the header (the avatar badge on the phone apps) shows your unread count, polled once a minute. Open it for the newest rows: a coloured dot for severity, the title and the sentence, when it happened. Mark all read clears the badge; × dismisses a row; a row that names a server opens that server. Notification settings at the bottom opens the preferences screen.

Add a channel

Account → Notifications. Two panels: Your channels and Organization channels (the organization you last opened; only a manager may add or edit there).

Pick the kind — Email, Webhook, Slack, Discord or Telegram — and give the channel a Name.

Enter the address: an email address — your own for a personal channel, a member's account email for an organization one (other addresses wait for a verification flow that does not exist yet); an https URL that accepts a JSON POST; a Slack or Discord incoming-webhook URL; or, for Telegram, your bot token, a slash, then the chat id (123456:ABC…/987654321). For a webhook you may add a signing secret.

Choose the rules: Every event at a floor (Everything, Warnings and up, Critical only), or switch to Chosen events and tick them. Add channel.

Each channel is then a row with its rules (change them and Save rules), Pause / Resume, Remove, and the outcome of the last delivery in the past week.

What a receiver gets

Kind	Body
Webhook	JSON: `event`, `severity`, `title`, `body`, `text` (the one-line form), `organizationId`, `target` (`{ type, id }` or null), `context` (small non-secret facts), `at`. Headers `X-TurboPanel-Event: <code>` and, when the channel has a signing secret, `X-TurboPanel-Signature: sha256=<hex>` — HMAC-SHA256 of the raw body with the secret.
Slack	`{ "text": "…" }` — the title, the sentence, then `key=value` facts in parentheses. Mattermost and Rocket.Chat read the same field.
Discord	`{ "content": "…" }`, the same line.
Telegram	`sendMessage` to the chat id with the same line.
Email	Subject `[TurboPanel Critical] Server db-1 went offline`; the sentence, the facts as a list, and an Open in TurboPanel button when the event has a target.

Verify a webhook signature by computing sha256= + hex(HMAC-SHA256(secret, raw body)) and comparing it to the header with a constant-time comparison.

Delivery and retries

One attempt is made at the moment of the event, with a five-second budget per channel; the retry sweep leaves a fresh row alone for two minutes so a slow attempt is never sent twice. A failure — a non-2xx answer, a timeout, a refused address — is recorded with a short code (http_503, timeout, network, address_scheme_not_https) and retried by the maintenance tick after 1, 5, 25 and 125 minutes; after the fifth failure the delivery is abandoned. Email deliveries ride the control plane's mailer queue (self-hosted: RabbitMQ → the mailer service; High Availability: Mailgun) and are subject to its own retry. The channel row shows the last delivery's status and how many attempts it took.

Digest and quiet hours

An email channel can wait instead of sending every event the moment it happens. Both settings are per channel, both are off until you turn them on, and both appear on the channel's row.

Digest. Choose Hourly or Daily. Instead of one email per event you get one summary per window: events grouped by kind with a count, the first few of each listed with a link back to the app, and "and N more" for the rest. A summary lists at most eight kinds and five events per kind. An hourly window closes at the top of the hour; a daily window closes at 08:00.
Quiet hours. Choose a start and end (24-hour, for example 22:00 to 07:00; the window may cross midnight). Events that arrive inside it are held and go out as one summary when it ends. If you also have a digest, the held events go out with the digest window that closes after quiet hours end.

Some events never wait. An event that means something is down or something about security changed is sent at once, whatever the channel says: server.offline, fleet.mass_disconnect, server.daemon_key_revoked, access.grant_created and access.grant_revoked. Today that leaves server.deleted as the only routine event a digest or quiet hours can hold. The bell is never held either: it always shows an event the moment it happens.

Which clock. Quiet hours and digest windows are read in a time zone: your own profile zone for a personal channel (set from the channel row; it is yours, not the channel's), the organization's default time zone for an organization channel, and UTC for a control plane channel or when none is set. Clock changes are followed: a window of 22:00 to 07:00 still ends at 07:00 on the morning the clocks change.

Pause and resume still work. A paused channel receives nothing and keeps what it was holding; Resume sends the held events in the next summary. Turning a digest or quiet hours off sends whatever was held in the next maintenance tick. A summary that cannot be queued is retried on the following tick, not lost.

Digest and quiet hours are for email channels in this release. Webhook, Slack, Discord and Telegram channels still receive every event as it happens, because a receiver on the other end of a webhook usually wants each one.

Reference

Item	Value
Kinds	`email` · `webhook` · `slack` · `discord` · `telegram` (`push` is registered by the store apps, never typed)
Severities	`info` < `warning` < `critical`
Rule	`*` at a floor, or one row per event
Inbox fan-out	server events: every organization member; audit-mirroring events: owners and managers; control plane events: every administrator
Channel reach	organization channels, its members' own channels, and every control plane channel
Address rule	`https`, no credentials in the URL; any host, LAN included
Retry	1 → 5 → 25 → 125 minutes, five attempts, then abandoned
Digest	email channels only; `hourly` (top of the hour) or `daily` (08:00), in the owner's time zone; at most 8 kinds and 5 events per kind per summary
Quiet hours	email channels only; `HH:MM` start and end, may cross midnight, never equal; urgent events are never held
Delivery status `held`	waiting for quiet hours to end or a digest window to close; retries do not touch it
Signature	`X-TurboPanel-Signature: sha256=<hex HMAC-SHA256 of the raw body>`
Name	≤ 80 characters; address ≤ 2048; signing secret ≤ 256

Errors

Code	Status	Meaning
`address_rejected`	422	The URL failed the outbound gate; `reason` says which rule (`scheme_not_https`, `credentials_in_url`, `malformed`).
`address_invalid`, `address_required`	400	Not an email address / not a token-slash-chat-id pair / empty.
`address_not_a_member`	422	An email channel may only name your own address, or a member's account email for an organization channel.
`label_required`, `label_invalid`	400	The name is missing, too long, or has control characters.
`signing_secret_not_applicable`	400	Only a webhook channel signs its deliveries.
`signing_secret_invalid`	400	1–256 characters.
`rule_event_unknown`	400	A rule names an event that is not in the catalogue (`reason` carries it).
`rule_severity_invalid`, `rules_invalid`	400	The rule list is malformed.
`body_invalid`, `disabled_invalid`	400	The request body is not an object, or the pause flag is not true or false.
`digest_cadence_invalid`	400	The digest is not `hourly`, `daily` or null.
`quiet_hours_invalid`	400	Quiet hours need a start and an end as 24-hour `HH:MM`, and they cannot be the same time.
`time_zone_invalid`	400	Not a time zone from the list the app offers.
`time_zone_user_channels_only`	422	A time zone is set on your own profile, from one of your personal channels.
`timing_email_only`	422	Digest and quiet hours are for email channels.
`kind_invalid`, `scope_invalid`	400	Not one of the kinds above, or not `user` / `organization`.
`Forbidden`	403	Organization channels need `organization:manage`.
`Not found`	404	The channel or the inbox row is not yours.
`Encryption unavailable — no encryption key configured`	503	The control plane has no root secret, so a sealed address cannot be stored.

Servers — what server.offline and a revoked key mean.
Organizations, teams and access — the grants the access events describe.
Administering the control plane — the operator's control-plane-wide alert webhook.
Troubleshooting — configuring that webhook from the shell.

On this page