Skip to content

Backup and Restore

Back up the Flow Scheduler database schema and each node's configuration, verify the backup, and restore it after a database loss.

What a backup contains

A Flow Scheduler backup protects the Flow Scheduler's own state: the flowscheduler schema of the MetaMiner Database (MMDB) and the configuration files of the node that wrote it. It is not a system backup. Keep your normal database and server backups as well.

Each node backs itself up into its own folder under backup.dir (default backups in the server folder, C:\Program Files\Infolytik\MetaMiner\server\backups\). Each backup is a folder named fss-backup-<yyyyMMdd-HHmmss-SSS>-<node>, with -<label> added when you give one.

Path in the backup folder Contents
db\flowscheduler.dump pg_dump -n flowscheduler -Fc of the Flow Scheduler schema (PostgreSQL custom format): schedules, tasks, chains, run history, subscriptions, admins, allowed environments, API clients and cluster state
db\flowscheduler.dump.list The archive's table of contents, as plain text
config\ This node's config.json, config.ecf, config-overrides.properties and mmext-services.json, plus the key file config.ek when a restrictive folder ACL could be set
manifest.json Integrity and version record: PostgreSQL server and pg_dump versions, dump size, dumpSha256, approximate row counts per table, and whether the set is complete enough for a full restore

The backup folder also holds fss-backup-latest.txt and backup-history.jsonl.

Not included: the rest of the MMDB, the Tableau content the Flow Scheduler runs, logs, program files, and other nodes' configuration.

Keep backups off the database disk

When the backup folder is on the same disk as the database, the Backup page shows a warning. Set backup.dir to another volume or a UNC share, so one disk failure cannot take both.

Before you begin

  • To use the web interface for backups, you must be a Flow Scheduler admin.
  • Backup settings are per node. Set them on each node in MetaMiner Administrator (MMA) under Tools > FSS Configuration... > Properties. Keys and defaults: Configuration Reference.
  • The Flow Scheduler ships PostgreSQL 16 client tools. If the MMDB runs on a newer PostgreSQL version, install matching (or newer) client tools on each node first.

Steps

Back up a node now

  1. Sign in to the web interface through the address of the node you want to back up (not the load balancer) and open Admin > Settings > Backup. The backup runs on the node that serves the page, so in high availability repeat this on every node.
  2. Optional: type a label, for example before-upgrade.
  3. Click Back up this node. The status line shows "Last backup (this node)" with the scope and size, and the backup history table lists the attempt, including failed ones.

To start a backup from a script, use the Flow Scheduler Backup API.

Turn on scheduled backups

Scheduled backups are off by default (backup.enabled is false). On each node:

  1. In MMA, open Tools > FSS Configuration... > Properties.
  2. Set backup.enabled to true.
  3. Optional: change backup.cron (default Saturday 22:00), backup.dir and backup.retention (default 8 backups per node).
  4. Restart the Flow Scheduler (MetaMinerFSS service) on that node. Admin > Settings > Backup then shows "Schedule: enabled" with the next run time.

With the default backup.db.dump.scope of all-nodes, every node writes a complete backup. With leader-only, only the leader dumps the database and a standby writes a configuration-only backup.

If you plan to turn on run history retention, turn on scheduled backups first: purges run only while a recent full backup of the leader node exists. See Run History.

Use newer PostgreSQL client tools

A pg_dump client cannot back up a server newer than itself. The Flow Scheduler looks for client tools in this order:

  1. backup.pgtools.dir, if set (the tools root, such as C:\pg17, or its bin folder).
  2. A pgsql-win-<major> folder in the server folder, matching the database server's major version.
  3. A standard PostgreSQL installation in %ProgramFiles%\PostgreSQL\<major>\bin, for the server's major version up to three versions newer.
  4. The shipped pgsql-win-16.3.2 tools.

Set backup.pgtools.dir on each node when the tools are elsewhere, then restart the Flow Scheduler. The setting is read only at start.

Verify

A backup that finished successfully has already passed these checks: the dump is not empty, its table of contents is readable, and the expected Flow Scheduler tables are present. To check it yourself:

  1. List the archive contents and confirm the Flow Scheduler tables appear:

    "C:\Program Files\Infolytik\MetaMiner\server\pgsql-win-16.3.2\bin\pg_restore.exe" -l "<backup-folder>\db\flowscheduler.dump"

  2. Hash the dump and compare the result with dumpSha256 in manifest.json:

    certutil -hashfile "<backup-folder>\db\flowscheduler.dump" SHA256

  3. Compare the row counts you expect with tableCountsApprox in manifest.json.

Restore the full schema

Restore is a manual, deliberate step. It replaces the live Flow Scheduler schema, including run history and high-availability history. Never restore into a schema that a running Flow Scheduler is using.

Stop every node first

All nodes share one schema. Stop the Flow Scheduler on all nodes before you restore, so no node holds the leader lease or writes to the schema.

  1. On every node, stop the Flow Scheduler and wait until it reports STOPPED:

    sc.exe stop MetaMinerFSS
sc.exe query MetaMinerFSS

  2. Check versions. Read serverVersion and pgDumpVersion in the backup's manifest.json. Use a pg_restore whose major version is at least both the archive's pgDumpVersion and the target server's version, and do not restore into a PostgreSQL server older than the one the backup came from.

  3. Restore the schema with the pg_restore.exe chosen in step 2 (one command; replace the placeholders):

    Restore the Flow Scheduler schema
    pg_restore --dbname="host=<db-host> port=<db-port> dbname=<db-name> user=<db-user>" --clean --if-exists --no-owner --single-transaction --exit-on-error --schema=flowscheduler "<backup-folder>\db\flowscheduler.dump"

    --single-transaction and --exit-on-error make the restore all or nothing: if any step fails, the schema is left as it was. 4. If the node's configuration was lost too, copy the files from the backup's config\ folder back to server\config\, except config-overrides.properties, which belongs in the server folder. Adjust per-node values, such as the node id, if you restore onto a different host. 5. Start the Flow Scheduler on one node only:

    sc.exe start MetaMinerFSS

  4. Confirm that node is the leader on Admin > HA / Nodes (or ACTIVE on a single node).

  5. Start the Flow Scheduler on the remaining nodes.
  6. Check schedules, next run times and recent run history in the web interface.

Restore configuration only

To roll back a configuration change without touching run history, use the JSON configuration restore. It covers the configuration, such as schedules, tasks and chains.

Source Where
Manual export Admin > Settings > Config (JSON) > Export config (JSON)
Automatic snapshots The leader writes a JSON snapshot every hour to config-snapshots\ in the server folder (fss-config-latest.json plus up to 60 dated copies). Look on whichever node was leader.
  1. Open Admin > Settings > Config (JSON) and click Restore config....
  2. Choose the JSON file and run the preview (dry run). Read the list of changes.
  3. Apply the restore.

The restore is merge-only: it adds and updates items but does not delete items missing from the file. Run history is never touched.

Troubleshooting

Backups fail with a PostgreSQL version error

The database runs on a newer PostgreSQL version than the client tools. Install matching client tools, set backup.pgtools.dir on each node, and restart the Flow Scheduler.

The backup is marked config-only

backup.db.dump.scope is leader-only and the node is a standby. Set it to all-nodes (on that node) for complete local backups.

The backup is not marked full-DR

One cause: the key file config.ek was skipped because the backup folder's ACL could not be verified, so an encrypted configuration could not be decrypted from this backup. Use a local backup folder, or set backup.keyfile.allow.unverified.acl to true if the share is protected by other means.

Next steps