shreeharshshinde opened a new pull request, #61853:
URL: https://github.com/apache/airflow/pull/61853
### What does this PR support ?
Adds support for configuring Docker log cleanup retention in minutes via a
new environment variable:
```
AIRFLOW__LOG_RETENTION_MINUTES
```
When this variable is set, the log groomer uses `find -mmin` to delete
`.log` files older than the specified number of minutes. If it is not set, the
existing behavior based on `AIRFLOW__LOG_RETENTION_DAYS` remains unchanged.
---
### Why
Day-level retention granularity can be too coarse for large-scale
deployments generating high log volumes. This can result in excessive local log
file accumulation, which negatively impacts log exporters such as the
OpenTelemetry Collector FileLogReceiver that perform poorly when handling large
file counts.
This change enables more precise log cleanup scheduling without breaking
existing configurations.
---
### How
* Introduced `AIRFLOW__LOG_RETENTION_MINUTES`
* Added conditional logic in `scripts/docker/clean-logs.sh`:
* Use `-mmin` when minutes retention is provided
* Fall back to existing `-mtime` logic otherwise
* Preserved existing pruning, filtering, and empty directory cleanup behavior
---
### Backward Compatibility
No breaking changes:
* Existing `AIRFLOW__LOG_RETENTION_DAYS` continues to work
* Default behavior unchanged if new variable is unset
---
### Testing
* Verified behavior locally with test log files
* Confirmed deletion based on minute-level thresholds
* Confirmed fallback to day-based retention
---
Fixes #61814
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
