mikebridge opened a new pull request, #42863:
URL: https://github.com/apache/superset/pull/42863
### SUMMARY
Deduplicate consecutive scheduled retention audits that remain blocked for
the same entity while preserving the write-ahead audit invariant. The scheduled
purge still writes a provisional `pending` record before evaluating the
cascade, but removes only that current provisional record when its unambiguous,
strictly older scheduled predecessor is already `blocked`.
The change preserves completed audit history, retains evidence on ambiguous
ordering or persistence uncertainty, and leaves force-purge auditing
independent. It also adds suppression/fallback metrics and a composite
predecessor-lookup index.
Shortcut: SC-115343.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Not applicable; backend audit behavior only.
### TESTING INSTRUCTIONS
```bash
source ~/venv/superset/bin/activate
pytest -q \
tests/integration_tests/deletion_retention/audit_tests.py \
tests/integration_tests/deletion_retention/purge_tests.py \
tests/integration_tests/deletion_retention/force_purge_tests.py
```
Expected: 61 tests pass.
The migration was verified on an isolated SQLite metadata database with an
upgrade, downgrade to `c4a1b8e2d739`, and re-upgrade. `EXPLAIN QUERY PLAN`
selected `ix_purge_audit_log_retention_predecessor` for the predecessor lookup.
Changed-file pre-commit passes, including MyPy, Ruff, and Pylint.
### ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [x] Includes DB Migration (follow approval process in
[SIP-59](https://github.com/apache/superset/issues/13351))
- [x] Migration is atomic, supports rollback & is backwards-compatible
- [x] Confirm DB migration upgrade and downgrade tested
- [x] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
The migration adds one non-unique index to `purge_audit_log`. Runtime is
proportional to the existing audit-table row count. No application downtime is
expected, although the database may briefly block writes or hold a metadata
lock while creating the index; operators with unusually large audit tables
should schedule the migration accordingly. The downgrade removes only this
index.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
