abh1sar opened a new pull request, #14310: URL: https://github.com/apache/cloudstack/pull/14310
### Description This PR fixes how VM backup usage is recorded and billed. One commit per change: 1. **Backup usage billed at the last reported size for the whole period (#13070).** A `BACKUP.USAGE.METRIC` event overwrote the size of the active `usage_backup` row in place, so `BackupUsageParser` applied the last size to the entire aggregation window. A size change now closes the active row at the event date and opens a new row with the new size, so each size is billed only for the time it was in effect. An unchanged size is a no-op, rows that are already removed are no longer updated, and duplicate active rows for the same VM and offering are merged into one. 2. **Backups billed twice after re-assigning an offering.** Removing a backup offering while keeping the backups publishes no usage event, so the usage row stays active (correctly, as the backups still use storage). Re-assigning the same offering then added a second active row, and both were billed. The usage server now skips the new row when one is already active. 3. **`BACKUP.USAGE.METRIC` published on every backup sync.** The backup sync published the metric for every VM on every run (`backup.framework.sync.interval`, 5 minutes by default) whether or not it changed. The management server now remembers the last value published per zone, VM and offering, and only publishes changes. The cache is in memory: after a restart or on another management server each value is published once more, which the usage server ignores since the size is unchanged. Behaviour changes: - `usage_backup` gains a row per size change instead of one row per assignment, and billing records are split accordingly. - Far fewer `BACKUP.USAGE.METRIC` rows in `cloud.usage_event` and `cloud_usage.usage_event`: one per actual change instead of one per VM per sync. Existing rows are left in place. - On upgrade, the first sync publishes each VM's metric once, which also merges any duplicate active `usage_backup` rows created by the re-assign bug. Changes 2 and 3 depend on 1: with publish-on-change, duplicate rows are not merged until the size changes, so the re-assign guard is what keeps them from being created. Fixes: #13070 ### Types of changes - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] New feature (non-breaking change which adds functionality) - [x] Bug fix (non-breaking change which fixes an issue) - [ ] Enhancement (improves an existing feature and functionality) - [ ] Cleanup (Code refactoring and cleanup, that may add test cases) - [ ] Build/CI - [ ] Test (unit or integration test code) ### Feature/Enhancement Scale or Bug Severity #### Bug Severity - [ ] BLOCKER - [ ] Critical - [x] Major - [ ] Minor - [ ] Trivial ### How Has This Been Tested? Tested on 4.22 with NAS backup on an NFS repository. The usage job ran with a 15 minute aggregation range so each window could be checked. Before the fix: - The VM's backup size changed at 08:58, but every window from 00:00 was billed at the new size. - The VM had two active `usage_backup` rows from an earlier remove and re-assign of the offering, so every window had two billing records. - 98.8% of all usage events were `BACKUP.USAGE.METRIC` (73,669 events with 64 distinct values for 2 VMs). Usage after the fix (VM backup usage records per 15 minute window): | Scenario | Window | Period billed | Size | Duration | |---|---|---|---|---| | Duplicate rows merged at the first metric event | 09:26 to 09:41 | 09:26:00 to 09:38:04 | 18.81 GB and 22.19 GB (two rows) | 12.07 min each | | | | 09:38:04 to 09:40:59 | 22.19 GB (one row) | 2.93 min | | Backup taken mid-window | 09:41 to 09:56 | 09:41:00 to 09:43:04 | 22.19 GB | 2.07 min | | | | 09:43:04 to 09:55:59 | 25.56 GB | 12.93 min | | Backup taken, then deleted | 10:28 to 10:43 | 10:28:00 to 10:28:40 | 25.56 GB | 0.67 min | | | | 10:28:40 to 10:38:40 | 28.94 GB | 10.00 min | | | | 10:38:40 to 10:42:59 | 25.56 GB | 4.33 min | Before the fix, each of these windows would have been billed at a single size for the full 15 minutes (twice, while the duplicate row existed). Also: - Removing and re-assigning the offering with backups kept: the re-assign was skipped and one active row remained. - Over 25 minutes, the management server published 3 metric events: one after its restart, one when a backup was taken and one when it was deleted. Before, it published one every 5 minutes. The republish after the restart did not add a usage row. #### How did you try to break this feature and the system with this change? - Restarted the management server: each value was published once more and the usage server ignored it. - Re-assigned the offering while its backups were kept: no second row. - Created and deleted backups within one aggregation window: each change got its own row and billing record. - Metric for a VM without an active usage row: ignored, as before. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
