jedcunningham commented on code in PR #46503:
URL: https://github.com/apache/airflow/pull/46503#discussion_r1970085002
##########
airflow/config_templates/config.yml:
##########
@@ -2686,6 +2686,37 @@ dag_processor:
type: integer
example: ~
default: "5"
+ stale_bundle_cleanup_interval:
+ description: |
+ On shared workers, bundle copies accumulate in local storage as tasks
run
+ and version of the bundle changes.
+ This setting represents the delta in seconds between checks for these
stale bundles.
+ Bundles which are older than `stale_bundle_cleanup_age_threshold` may
be removed. But
+ we always keep `stale_bundle_cleanup_min_versions` versions locally.
+ Set to negative to disable.
+ version_added: ~
+ type: integer
+ example: ~
+ default: "1800"
+ stale_bundle_cleanup_age_threshold:
+ description: |
+ Bundle versions used more recently than this threshold will not be
removed.
+ Recency of use is determined by when the task began running on the
worker,
+ that age is compared with this setting, given as time delta in seconds.
+ version_added: ~
+ type: integer
+ example: ~
+ default: "21600"
+ stale_bundle_cleanup_min_versions:
+ description: |
+ Minimum number of local bundle versions to retain on disk.
+ Local bundle versions older than `stale_bundle_cleanup_age_threshold`
will
+ only be deleted we have more than `stale_bundle_cleanup_age_threshold`
versions
Review Comment:
```suggestion
only be deleted we have more than
`stale_bundle_cleanup_min_versions` versions
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
