SolbiatiAlessandro opened a new issue #12368:
URL: https://github.com/apache/airflow/issues/12368


   **Description**
   
   Could we introduce a new counter in [Airflow 
Metrics](https://airflow.apache.org/docs/stable/metrics.html) to track **number 
of DAGs that are in 'running' state**?
   
   I am aware of the existing `dag_processing.processes` counter that is 
documented as `Number of currently running DAG parsing processes`. This counter 
track how many dags are being parsed, not how many are runnings.
   
   I am aware of the existing `executor.queued_tasks` and 
`executor.running_tasks`, but they are task-wise and not dag-wise.
   
   **Use case / motivation**
   
   Consider this example 
   ![Screenshot 2020-11-14 at 16 42 
40](https://user-images.githubusercontent.com/20618047/99152409-7a619780-2699-11eb-9680-bf74cb5adc67.png)
   
   I want to know how many DAGs are in the state of `running` to understand 
memory consumption issues and to understand how long is my "queue" of DAGs. To 
my understanding there is no metric currently that I can use to track that.
   
   I propose a counter `executor.running_dags` that would return the number of 
DAGs in the running state. 
   
   Does this make sense? Am I missing something? If folks I agree I would be 
happy to work on a PR :)
   
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[email protected]


Reply via email to