mikebridge commented on code in PR #45029:
URL: https://github.com/apache/superset/pull/45029#discussion_r4231368552
##########
superset/tasks/version_history_retention.py:
##########
@@ -513,20 +661,32 @@ def _prune_old_versions_impl(retention_days: int) ->
dict[str, Any]:
"pruned_child_shadows": 0,
"pruned_m2m_shadows": 0,
}
- total_retried = 0
- after_id = 0
+ total_retried: int = 0
+ after_id: int = 0
+ pass_stats: dict[str, Any]
+ retries: int
while True:
- pass_stats, retries = _run_pass_with_retry(cutoff, tables, after_id)
+ remaining_budget: int | None = (
+ max_per_run - totals["pruned_transactions"]
+ if max_per_run is not None
+ else None
+ )
+ pass_stats, retries = _run_pass_with_retry(
+ cutoff, tables, after_id, max_prune=remaining_budget
+ )
total_retried += retries
for key in totals:
totals[key] += pass_stats.get(key, 0)
+ if max_per_run is not None and totals["pruned_transactions"] >=
max_per_run:
Review Comment:
Agreed. Updated both module docstrings in 38f919541e to describe safe,
repeatable capped runs: each run removes at most the configured number of
transactions or root entities, and successive runs drain the remaining
prunable/purgeable backlog. The wording also accounts for blocked roots and
newly eligible items. I checked UPDATING.md and the retention docs; their
backlog descriptions already reflect successive runs. This changes
documentation only; the hooks pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
