State and recovery
Model an S3 backup job as a recoverable state machine
Track scheduled, running, uploaded, checked, and restore-verified states separately so retries and notifications do not hide a failed recovery path.
Name what each state proves
Use a run identifier, source identifier, intended recovery point, start time, deadline, and tool result for each job. Scheduled means an execution was expected. Running means work began. Uploaded means the tool reported transfer completion. Checked and restore-verified require separate observations with their own times and scope.
An illustrative job can move from scheduled to running to uploaded, then await repository checking and a sampled restore. Keep failure, cancellation, and deadline-exceeded states explicit. The absence of an error message is not a durable event saying that the expected job ran.
- Run identifier
- Source identifier
- Expected start
- Deadline
- Snapshot identifier
- Observed verification scope
Bind results to the repository
NordenVault's documentation describes backup clients, API keys, and webhook events. Use it as the provider-specific reference for integrating observations with an S3-backed job. A backup-received or overdue event should be mapped to the precise state it supports, with the documented payload and authentication rules reviewed before implementation.
Keep the run's snapshot identifier and repository endpoint in the result record. A late notification from an older run must not overwrite the state of the current one. Preserve event identifiers or another durable deduplication key where the integration provides them.
Retry without inventing success
On timeout, reconcile the tool's actual result and repository state before starting overlapping work. A scheduler retry, transfer retry, and maintenance retry can have different locking and consistency requirements. Follow the selected backup tool's supported locking procedure and assign a clear owner to repository maintenance.
Do not run pruning merely because an upload notification arrived. Maintenance authorization, retention scope, and the availability of recovery points are separate decisions. Capture failures with useful identifiers while excluding credential values and sensitive object names from broad operational logs.
Make restore verification an observable event
The restic restore documentation describes selecting a snapshot and restoring into a target directory. A restore exercise should record which snapshot was read, where it was restored, which files were checked, and the outcome. Sampling verifies the sampled recovery scope; it does not certify every application dependency.
NordenVault's security information is a starting point for the provider control review. Application-side responsibilities still include storing decryption material, restricting restore destinations, authenticating event delivery, and handling compromised job credentials. Recheck the state model when an integration or tool changes.
Referenced resources
- NordenVault API and backup documentation
Provider documentation for backup clients, events, and programmatic access.
- NordenVault published security controls
Provider scope to compare with the application's credential and event controls.
- restic restore procedure
Official snapshot-selection and restore-target instructions.
Walk through a delayed result, duplicate event, missed schedule, failed transfer, and failed sampled restore; confirm each leaves the run in an explainable state with an owner.