StefanKurek opened a new issue, #31132:
URL: https://github.com/apache/airflow/issues/31132
### Apache Airflow version
2.6.0
### What happened
When queuing up a number of DAGs on a basic Airflow installation (SQLite
with SequentialExecutor), a number of statd metrics were only reporting 0
values. This list included:
pool.queued_slots.<pool_name>
pool.running_slots.<pool_name>
executor.running_tasks
### What you think should happen instead
I would expect these metrics to show non 0 values at some point.
### How to reproduce
I have a fairly simple test DAG
```
from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from datetime import datetime, timedelta
default_args = {
'owner': 'airflow',
'depends_on_past': False,
'start_date': datetime(2023, 5, 2),
'retries': 1,
'retry_delay': timedelta(minutes=1),
'sla': timedelta(minutes=1),
}
dag = DAG(
'longtask_dag',
default_args=default_args,
description='Example DAG with a task that takes about 2 minutes to
execute',
schedule_interval='*/5 * * * *',
)
t1 = BashOperator(
task_id='long_running_task',
bash_command='sleep 120',
dag=dag
)
t2 = BashOperator(
task_id='short_running_task',
bash_command='echo "Short running task"',
dag=dag
)
t2 >> t1
```
I then queue up about 20 runs of this DAG in the webservice, and start to
monitor the statsd output from Airflow (after configuring it).
### Operating System
macOS Monterey v12.3.1
### Versions of Apache Airflow Providers
apache-airflow-providers-common-sql==1.4.0
apache-airflow-providers-ftp==3.3.1
apache-airflow-providers-http==4.3.0
apache-airflow-providers-imap==3.1.1
apache-airflow-providers-sqlite==3.3.2
### Deployment
Virtualenv installation
### Deployment details
I for the most part followed the part of the guide to install and run
airflow here:
https://www.redhat.com/en/blog/monitoring-apache-airflow-using-prometheus
### Anything else
This problem seems to be consistently reproducible.
There are also some related metrics that I am seeing similar issues with
that I have not made an issue for yet:
pool.open_slots.<pool_name>
executor.open_slots
These always seem to report the max number of slots (for either pool or
executor), even when there are tasks that are running.
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
