Upgrade and rollback

Every release is a GitHub Release with its packages as assets. The installer and the daemon both follow a channel — release (the latest promoted release), rc (the current release candidate) or canary (the newest green build of trunk, for a canary environment) — and resolve that channel's manifest.json straight from GitHub. A host keeps following the channel it was installed on, and one with no channel follows release. The control plane and the daemon are separate packages. A managed upgrade installs them in a fixed order. Re-running the installer still moves both on this host.

A run pins the manifest URL and the target commit it resolved when it started. A channel that moves while the rollout is in progress does not change the bytes a later batch installs.

Order

On a self-hosted host the order is fixed:

This host's daemon — the co-located daemon, the one enrolled on the control plane's own machine.
Control plane — the control plane package and the static UI on that same host.
Other servers — every other daemon, in batches.

The servers step does not start until both earlier steps are on the target build. A failed control-plane step fails the run and leaves that gate shut. A trunk channel has no control-plane package, so that step is skipped and the gate needs only the co-located daemon on target.

TurboPanel High Availability deploys the control plane itself. Admin → Updates shows that version as Managed by TurboPanel and rolls out daemons only. Per-server Update on the app stays off (updates_managed).

Mixed versions stay connected the whole time. The version floors do not change, and a daemon below its floor still holds its socket so the update can reach it. See Version compatibility.

First upgrade onto this release

Backup and automatic rollback need a co-located daemon that advertises managed-upgrade-v1. A daemon from before this release does not. Pre-flight names that and gives the recovery command. Take an operator backup of the control-plane database first — this transition cannot use the managed backup yet — then update only the daemon, pinned to one verified manifest. On a host that already runs this control plane, that command refreshes the co-located daemon in socket mode (daemon-colocated-refresh.yml). It does not run the remote-server installer (daemon-install.yml), so daemon.env keeps dialing the local instance socket, shared state and config ownership stay as they are, and the daemon unit still starts after the control plane. The control plane, UI, and database are not replaced. When the new daemon attaches and advertises managed-upgrade-v1, start the upgrade from Admin → Updates.

Terminal

curl -fsSL turbopanel.sh \
  | TURBOPANEL_DAEMON_ONLY=1 \
    TURBOPANEL_MANIFEST_URL=https://github.com/TurboPanel/turbopaneld/releases/download/<tag>/manifest.json \
    sh

Replace <tag> with the daemon release you are upgrading to, as listed on the daemon's GitHub Releases (for example v0.1.1). It must be a release that advertises managed-upgrade-v1; v0.1.0 does not. The pre-flight sheet fills in the right tag for you.

From the app

Admin → Updates (/admin/updates). An admin or superadmin account.

The status card reads TurboPanel is up to date, Update available, Updating…, or Needs attention.

Update TurboPanel opens Ready to update TurboPanel?. The sheet lists the pre-flight checks, notes that the control-plane database is backed up before the install, and shows the Recovery command with Copy command. Start update begins the run. Cancel closes the sheet.

Progress follows Co-located daemon, then Control plane, then the servers table. Each step moves through Preparing → Downloading → Writing new files (still running the old version) → Restarting onto the new version → Checking the new version is running → Done. The control plane restarts during its step, so the browser may show a gateway error; the page comes back as TurboPanel is updating and polls until the run settles. Retry on a server row dispatches that step again.

Automatic updates is off until you turn on Auto-update servers. Automatic runs start only when the target differs from what is installed, no run is already active, and — if you enable Maintenance window — only inside that UTC window. The default batch is 100% (Percent). Switch to Count to cap how many servers move together, then Apply.

A failed step is terminal for its batch: the next batch still starts. It does not hold the wave.

The control plane line is Managed by TurboPanel. There is no Update TurboPanel button and no Auto-update servers toggle — daemon rollouts run on their own, inside the maintenance window when one is set.

Connected daemons filters All, Updating, Needs attention, and Up to date. Batch size still applies (Percent or Count, Apply). Retry is on a row that needs it.

Only one run is active. A second start is refused while a run is pending or running, with 409 upgrade_run_active (Another update is already in progress). The control-plane step is not finished until the daemon has checked the new build, even if the new control plane already answers as the target version.

Batches and the maintenance window

Setting	Range	Default
Batch Percent	1–100	100
Batch Count	1–10000	—
Maintenance window	off, or a UTC start (minutes after midnight, 0–1439), a length (1–1440 minutes), and optional weekdays (Sunday = 0)	off; when on, empty weekdays means every day

Percent rounds up so a non-empty set of servers always moves at least one server. The next batch starts when every step in the current one is terminal (done, skipped, failed, rolled_back, or needs_attention). On TurboPanel High Availability a large batch drains across maintenance ticks so one tick does not enqueue the every server at once.

An offline server's step waits (waiting) and is dispatched when that daemon reconnects. A step with no progress for 15 minutes is retried, up to 3 dispatches, then needs_attention. A step that rolled back is retried once, then needs_attention.

Network failures, older builds and your apps' releases

Short network failures are retried. A DNS hiccup or a GitHub 5xx should not fail a whole update, so the fetches of the manifest and the downloads retry, in two places:

Fetch	Retried	Not retried
The installer, `run.sh` (curl)	A failed name lookup (curl exit 6) and a refused connection (exit 7): 4 attempts, waiting 2, 4 and 8 seconds. Other transient failures (HTTP 408, 429 and 5xx, timeouts): 3 attempts, 3 seconds apart. A download whose checksum does not match: up to 5 attempts, 3 seconds apart	4xx responses other than 408 and 429, and a signature failure
The daemon's update reads	HTTP 502, 503 and 504; 429, honoring `Retry-After` up to 10 seconds; resets, timeouts and DNS lookup failures: 4 attempts, waiting 2, 4 and 8 seconds	Every other response, including 4xx, and anything wrong with the body or its signature

When the attempts run out, the last error is shown unchanged. The installer's error message names the URL without its query string, because a signed download link can carry a token.

An older build is refused. The daemon and the control plane refuse a validly signed manifest older than the build they run. For a managed update this shows as a step with downgrade_refused when a server already runs something newer than the target; a server is never downgraded. To have the daemon on one host accept an older signed build deliberately, set TURBOPANEL_ALLOW_DOWNGRADE=1 in the daemon's environment (it lifts the daemon's check only, not the control plane's).

A restarted control plane is never installed over twice. If the control-plane step restarts and the daemon then reports nothing for TURBOPANEL_UPGRADE_VERIFY_TIMEOUT_MINUTES (20 by default), the step ends as needs_attention with verify_timeout and the run finishes, so the next Update TurboPanel can start. The step says whether the control plane is running the target build. Check the daemon log on that host, then Retry or cancel.

Updating the daemon does not make old releases roll-back-able

Rolling an app back to an earlier source release uses a record the daemon wrote on that server when it published the release. A release published before the daemon wrote records (a daemon that predates release records) has none, and an update does not create it: that release cannot be rolled back until you redeploy it. Releases published afterwards are unaffected. See Releases and rollback.

Pre-flight, backup, and `.prev`

Before anything is swapped, the host checks:

Code	Meaning
`preflight_disk`	Not enough free space, or the free-space check itself failed. A daemon install needs 512 MiB on `/opt/turbopanel`, 128 MiB on the state directory, and 256 MiB on the temp directory. A control-plane install needs 1 GiB on the install root and 1 GiB on the backup directory.
`preflight_manifest`	The pinned manifest could not be fetched, verified, or matched to `targetCommit`.
`preflight_trust`	Automatic-update TLS could not be established (public trust or the Platform CA). The daemon does not skip verification.
`preflight_backup`	`turbopanel-database` is not running, so the control-plane dump cannot be taken.
`preflight_in_progress`	Another install is already running on that host.

The control-plane step also requires the co-located daemon to advertise managed-upgrade-v1. Without it, pre-flight stops and names the installer command above.

The automatic backup is written before the control-plane package swap, at:

Text

<backup dir>/control-plane/<upgrade id>/

The backup directory defaults to /backup (TURBOPANEL_BACKUP_DIR moves it). Each attempt keeps a pg_dump -Fc of the control-plane database, a tarball of /etc/turbopanel, and meta.json (including the migration-history fingerprint). The newest three attempts are kept.

One previous generation stays beside the new files, by rename:

Package	Previous generation
Daemon	`/opt/turbopanel/bin/turbopaneld.prev`, `turbopaneld.js.prev` on hosts that run the JS runtime, `/opt/turbopanel/share/orchestration.prev`
Control plane	`/opt/turbopanel/bin/turbopanel.prev`, `/opt/turbopanel/lib/libduckdb.so.prev`, `/opt/turbopanel/share/ui.prev`

Automatic rollback

After the control plane restarts, the daemon polls GET /api/health on the control plane socket until version and revision.commit match the pinned manifest. It waits up to 10 minutes by default, because a small host can take several minutes to start the new build and apply migrations, and a rollback is worse than waiting. TURBOPANEL_UPDATE_HEALTH_TIMEOUT_SECONDS (30 to 3600, in the daemon's environment, /etc/turbopanel/daemon.env) changes the wait; an unset or out-of-range value keeps 10 minutes. A timeout or a mismatch restores the .prev generation. The database is restored from the dump only when the migration fingerprint changed. A healthy restore reports rolled_back with health_timeout, health_mismatch, or restart_failed. A restore that does not come back reports failed / recovery_required and includes the backup path plus the commands below.

A daemon self-update arms an update guard for about 10 minutes. If the new build crash-loops, the guard restores the daemon .prev files and restarts turbopaneld. Detail: Refresh a stuck daemon.

Recovery command

Run as root on the control-plane host. The pre-flight sheet copies the same command, with the upgrade id filled in:

Terminal

sudo -n /opt/turbopanel/share/orchestration/scripts/tp-orchestrate playbook -i localhost, -c local \
  -e turbopanel_upgrade_id=<upgrade id> \
  instance-rollback.yml

That restores the .prev builds and, when the migration fingerprint changed, the database dump under /backup/control-plane/<upgrade id>/. The helper is the installed absolute path, so it runs when the control plane is stopped and tp-orchestrate is not on PATH. To reinstall one pinned control-plane manifest instead:

Terminal

sudo -n /opt/turbopanel/share/orchestration/scripts/tp-orchestrate update-instance \
  --channel release \
  --manifest-url https://github.com/TurboPanel/turbopanel/releases/download/v0.1.0/manifest.json \
  --no-start

--manifest-url on that verb is the control-plane pin (TURBOPANEL_INSTANCE_MANIFEST_URL). It does not change the daemon pin (TURBOPANEL_MANIFEST_URL).

Pin a package by hand

The same pins work from the installer. Each package keeps its own pin; a later channel install that does not name a pin clears it.

Pin the control plane on its own host. TURBOPANEL_INSTANCE=1 marks the run as a control-plane install; without it, a run that names a pin is read as a daemon enrolment and stops asking for TURBOPANEL_LICENSE:

Terminal

curl -fsSL turbopanel.sh \
  | TURBOPANEL_INSTANCE=1 \
    TURBOPANEL_INSTANCE_MANIFEST_URL=https://github.com/TurboPanel/turbopanel/releases/download/v0.1.0/manifest.json sh

Pin a remote server's daemon. On a server enrolled with a self-hosted control plane, also pass TURBOPANEL_HOST with that control plane's URL, as in Refresh a stuck daemon; without it the server is enrolled with the TurboPanel High Availability control plane:

Terminal

curl -fsSL turbopanel.sh \
  | TURBOPANEL_LICENSE=<license> \
    TURBOPANEL_HOST=<control plane URL> \
    TURBOPANEL_MANIFEST_URL=https://github.com/TurboPanel/turbopaneld/releases/download/v0.1.0/manifest.json sh

Pins accept release manifests only. Each package's pin must be a manifest.json or manifest-<version>.json under its own repository's GitHub Releases (https://github.com/TurboPanel/<repo>/releases/download/<tag>/… or …/releases/latest/download/…); the daemon also accepts its CDN channel drop (https://dl.trbp.nl/channels/<channel>/manifest.json). The installer and the root helper refuse anything else, including a URL with .., a percent-encoded character, a port or a query string. A TURBOPANEL_*_MANIFEST_URL pin in daemon.env that does not match is ignored, as if unset.

The control plane checks migration history at every boot and refuses to serve a database migrated by a newer release than the binary (database was migrated by files this build does not ship). Roll the database back with the binary, or move the binary forward.

The daemon refuses a control-plane target below MIN_SUPPORTED_INSTANCE_VERSION (today 0.1.0). A target it cannot parse is allowed. Daemon self-update does not consult that floor.

History

Finished runs stay for 365 days, and at least the newest 50. Successful steps (done, skipped) are pruned after TURBOPANEL_UPGRADE_STEP_RETENTION_DAYS (default 14). Failed steps (failed, rolled_back, needs_attention) stay 90 days. The run row keeps its summary after the steps are gone. See Control plane configuration.

On this page