GitHub user zidz created a discussion: RFC: Incremental RBD backup provider
(port to 4.23) — design + working prototype
## TL;DR
I built an **incremental RBD backup provider** for CloudStack (port to
**4.23.0.0**).
It uses Ceph RBD snapshots + `export-diff`/`import-diff` so an incremental
backup
is ~32x smaller than the full, and a restore replays the full→incremental chain.
Full + incremental + restore are verified end-to-end through the management API
against a real Ceph/KVM host. This is a **prototype for design feedback**, not a
finished PR — I'd like input on two open questions below.
## Motivation
The existing NAS backup provider copies whole disk images every time. On Ceph
RBD primary storage we can do far better: take an RBD snapshot per backup and
export only the diff since the previous snapshot. Backups become small and fast,
and a restore is `import` (base) + `import-diff` (chain).
## Design
- New plugin `plugins/backup/rbd` implementing `BackupProvider`.
- **Full** = `rbd snap create` + `protect` + `export` → `<disk>.<vol>.rbd`.
- **Incremental** = `export-diff --from-snap <parent>` →
`<disk>.<vol>.<ts>.rbdiff`.
- **Restore** = `import` base + `import-diff` each link, **truncated at the
selected backup** (restoring an earlier backup lands on that point-in-time).
- Chain metadata (chain id, position, parent snap) stored as backup details.
- Host-side engine: `scripts/vm/hypervisor/kvm/rbdbackup.sh` (called by the KVM
agent wrappers, mirroring the NAS path).
## What changed vs 4.23.0.0
- **New (additive):** `plugins/backup/rbd/**` (provider + tests + spring
module),
`scripts/vm/hypervisor/kvm/rbdbackup.sh`.
- **Existing code (minimal, null-guarded):**
- `core/.../backup/{Take,Delete,Restore}BackupCommand.java` — added RBD fields
(parent snap, base snap, rbd uri/snap, disk types). New `Command` fields
only;
rolling-upgrade safe (old agents ignore unknown fields).
- `kvm/.../Libvirt{Take,Delete,Restore}BackupCommandWrapper.java` — route
**pure-RBD** VMs to `rbdbackup.sh`; NAS path unchanged (guarded branch).
- `plugins/pom.xml` + `debian/rules` — build & package the new module.
## Enabling it today
`createBackupOffering` is currently **hard-gated to the KBOSS provider**, and
there's no `createBackupRepository` API, so the offering + repository must be
created via DB for now:
- set `backup.framework.enabled=true`, `backup.framework.provider.plugin=rbd`,
`rbd.backup.incremental.enabled=true`
- insert a `backup_repository` (NFS) and a `backup_offering` (provider=`rbd`),
linked by `backup_offering.external_id = backup_repository.uuid`
Full step-by-step is in `plugins/backup/rbd/README.md`.
## Verification (real Ceph/KVM host)
| Scenario | Result |
| --- | --- |
| Full backup | `.rbd` object, ~5 GB |
| Incremental (no disk writes) | `.rbdiff`, ~132 B |
| Incremental (512 MB written) | `.rbdiff`, ~512 MB |
| Restore #1 / #2 / #3 of a 3-backup chain | lands on the correct point-in-time
state each time |
| Restore-to-existing-VM | purges dest image, replays chain, data intact |
| `restore-as-template` | rebuilds chain into a new, freely-named image |
## Limitations / known issues
- **Restore is bound to the original VM** (framework limitation —
`restoreBackup`
resolves the VM from `backup.getVmId()`). For a new, custom-named VM use
`restore-as-template` → register as template → deploy.
- Offering/repository creation is manual (KBOSS-gated API) — see open question
1.
- The new module is always built/loaded (`plugins/pom.xml`, `debian/rules`), so
a
defect there has global blast radius — see open question 2.
## Open questions for the community
1. **Offering API:** should `createBackupOffering` be generalized beyond KBOSS
so
RBD (and other providers) can create offerings via API/UI, removing the
manual
DB step? Or is there a preferred existing mechanism I missed?
2. **Packaging:** is always-on module registration acceptable, or should the RBD
provider be opt-in (profile/flag) to limit build/startup coupling?
## Try it
Branch `feature/rbd-incremental-backup-4.23` (6 commits on top of `4.23.0.0`):
[compare
link](https://github.com/apache/cloudstack/compare/4.23.0.0...zidz:cloudstack:feature/rbd-incremental-backup-4.23?expand=1)
Docs: `plugins/backup/rbd/README.md`.
Feel free to get inspiration from, use full or parts of this implementation if
anyone have use for it in any way.
GitHub link: https://github.com/apache/cloudstack/discussions/14374
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
