fuyi opened a new issue #8881:
URL: https://github.com/apache/airflow/issues/8881


   **Apache Airflow version**:
   
   1.10.8
   
   **Kubernetes version (if you are using kubernetes)** (use `kubectl version`):
   
   1.17.5
   
   **Environment**:
   
   - **Cloud provider or hardware configuration**:
   
   AWS
   
   - **OS** (e.g. from /etc/os-release):
   
   NAME="Ubuntu"
   VERSION="16.04.4 LTS (Xenial Xerus)"
   ID=ubuntu
   ID_LIKE=debian
   PRETTY_NAME="Ubuntu 16.04.4 LTS"
   VERSION_ID="16.04"
   HOME_URL="http://www.ubuntu.com/";
   SUPPORT_URL="http://help.ubuntu.com/";
   BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/";
   VERSION_CODENAME=xenial
   UBUNTU_CODENAME=xenial
   
   - **Kernel** (e.g. `uname -a`):
   
   Linux salt 4.4.0-1106-aws #117-Ubuntu SMP Wed Apr 8 09:52:02 UTC 2020 x86_64 
x86_64 x86_64 GNU/Linux
   
   - **Install tools**:
   - **Others**:
   
   **What happened**:
   
   In airflow.config file I have set the property **parallelism** to 50, which 
runs 50 tasks simultaneously. 
   
   After turning on KubernetesExecutor, I notice there are many tasks pending 
in the "queue" status, and there are only maximal 25 tasks running at the same 
time. 
   
   When I check airflow metric exported from statd, it shows that the metric 
"airflow_executor_running_tasks" hits its limit 50 already. see attachment.
   
   ![Screenshot 2020-05-14 at 11 09 
51](https://user-images.githubusercontent.com/295515/82089890-c0db3700-96f4-11ea-935a-a40b2611cabf.png)
   
   **What you expected to happen**:
   
   parallelism 50 can run 50 tasks simultaneously when Kubernetes Executor is 
enabled.
   
   **How to reproduce it**:
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[email protected]


Reply via email to