EnxDev commented on code in PR #45029: URL: https://github.com/apache/superset/pull/45029#discussion_r4212260764
########## UPDATING.md: ########## @@ -1709,6 +1709,8 @@ Authorization reuses the resource's `can_read` permission and per-object `raise_ Entity version history (the `version_transaction` / `*_version` shadow tables that back version capture) is aged out by a nightly Celery beat task, `version_history.prune_old_versions` (`superset.tasks.version_history_retention`). +**Scheduled-run cap (behavior change):** `VERSION_HISTORY_PRUNE_MAX_TRANSACTIONS_PER_RUN` defaults to 1000 whole prunable transactions per invocation; associated shadow and change rows remain atomic. `0` or `None` restores unlimited runs. Invalid values skip the task before deletion. Capped results report `cap_reached`, `remaining_eligible`, and `remaining_count_complete`; the remainder is a lower bound from one candidate window after the live scan's stopping point when the probe does not cover the backlog, and `None` if the probe fails after committed deletion. Physical shadow-row counts remain separate. `VERSION_HISTORY_PRUNE_DRY_RUN=True` scans the entire prunable backlog without writes and reports the exact `eligible_backlog` and `estimated_capped_runs`; non-boolean values skip the scheduled task without deleting history. A large backlog now drains over successive scheduled runs rather than in one invocation. The cap is per task invocation, not a quota shared across concurrent w orkers. If the database commits a window but the client loses its acknowledgement, retries can prune additional windows beyond the cap (up to two extra windows per retried pass with the three-attempt policy); all pruned transactions still meet the retention policy. Review Comment: The last sentence here still says a lost commit ack lets retries prune up to two extra windows past the cap. That's no longer true: `_run_prune_pass` raises `_PruneCommitUncertainError` on an uncertain commit, `_run_pass_with_retry` doesn't retry it, and version-history.mdx already describes it that way. Operators read UPDATING.md first, so could we bring it in line? Something like: "If the database commits a window but the client loses its acknowledgement, the task stops without retrying that window and reports an error, so it never prunes past the cap." -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
