anu251989 opened a new issue, #28740:
URL: https://github.com/apache/airflow/issues/28740
### Apache Airflow version
Other Airflow 2 version (please specify below)
### What happened
In airflow 2.2.5, The worker and scheduler memory gradually increasing day
by day when no tasks are running. The log rotation also enabled. We have
enabled the celery inspect command with liveness check probe for workers to
resolve the communication issue between worker and airflow redis.
Liveness: exec [sh -c CONNECTION_CHECK_MAX_COUNT=0 exec /entrypoint python
-m celery --app airflow.executors.celery_executor.app inspect ping -d
celery@${HOSTNAME}] delay=30s timeout=60s period=30s #success=1 #failure=5
I have disabled the liveness check and didn't observe the memory leak. how
to enable the liveness checks to resolve disconnection between redis and
workers? why redis and workers communication not resuming automatically.
I observed memory leak issues there for scheduler pod also. but didn't find
the root cause.
### What you think should happen instead
why celery inspect command causing the memory leak in worker.
The redis and worker communication should resume automatically without any
intervention.
The scheduler memory leak should not happen. The scheduler utilization go up
while processing tasks and it has to comedown once done.
### How to reproduce
Install airflow 2.2.5 with Python 3.8 to replicate the issues.
for workers memory leak: enable the below command and observe the memory
utilization.
Liveness: exec [sh -c CONNECTION_CHECK_MAX_COUNT=0 exec /entrypoint python
-m celery --app airflow.executors.celery_executor.app inspect ping -d
celery@${HOSTNAME}] delay=30s timeout=60s period=30s #success=1 #failure=5
The scheduler no need of any changes. The memory gradually increasing day by
day.
### Operating System
PRETTY_NAME="Debian GNU/Linux 10 (buster)" NAME="Debian GNU/Linux"
VERSION_ID="10" VERSION="10 (buster)" VERSION_CODENAME=buster ID=debian
HOME_URL="https://www.debian.org/"; SUPPORT_URL="https://www.debian.org/support";
BUG_REPORT_URL="https://bugs.debian.org/";
### Versions of Apache Airflow Providers
airflow 2.2.5
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
NA
### Anything else
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
