praveer-rc opened a new issue, #57347:
URL: https://github.com/apache/airflow/issues/57347
### Apache Airflow version
Other Airflow 2/3 version (please specify below)
### If "Other Airflow 2/3 version" selected, which one?
3.0.2
### What happened?
So a little context about my architecture. I have installed airflow using
the chart version 1.18.0 on my k8s cluster. The airflow core pods (api,
triggerrer, schedular, etc.) are deployed in the airfow namespace. I'm using
the kubernetes executor so the worker pods are deployed in the tenant namespace
which has the secrets for the specific dag. Now I'm seeing two issues that
could be interconnected.
1. The worker pods that are deployed in the tenant namespace. After the task
run is complete, the pods remain in 'Completed' state and are never
automatically deleted.
2. (The major one) The scheduler is getting stuck with the message
"Parallelism limit reached" even though I've set it in thousands and it gets
stuck every few hours still with the same error message. This could be because
the pod isn't sending a completed signal back to the schedular.
### What you think should happen instead?
_No response_
### How to reproduce
Set - name: AIRFLOW__CORE__PARALLELISM
value: '128'
- name: AIRFLOW__CORE__DEFAULT_POOL_TASK_SLOT_COUNT
value: '256'
- name: AIRFLOW__CORE__MAX_ACTIVE_TASKS_PER_DAG
value: '32'
Send down tasks in another namespace using the kubernetes executor and check
after 32 tasks for the same pipeline, it starts logging "Parallelization limit
reached" and doesn't schedule any pods.
### Operating System
Kubernetes
### Versions of Apache Airflow Providers
_No response_
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
Chart version 1.18.0
### Anything else?
The problem occurs every few hours or minutes
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
