trau-sca opened a new issue, #56635:
URL: https://github.com/apache/airflow/issues/56635
### Apache Airflow version
3.1.0
### If "Other Airflow 2/3 version" selected, which one?
_No response_
### What happened?
Hi, we've been battling issues for the past few weeks on our api-server pod
being very slow to display the website information (up to a minute to load the
dags card screen). Sometimes the api server pod even restarts due to the
liveness probe. I don't see any errors when it happens either.
Quite a few requests look long, but the biggest one is
`dags?dag_runs_limit=14&limit=50&offset=0&exclude_stale=true&order_by=-dag_display_name`
taking 51s, but even `dags?is_favorite=true` takes 4 seconds.
When remoting into the dag-processor and doing a curl command for the first
one - it also takes 51s. I used the private endpoint assigned to the api-server
while I was in the dag-processor.
We are running on airflow 3.1.0 and the 1.18.0 helm chart in an Azure
Kubernetes cluster with an Azure PostgreSQL database.
No pods are going past their resource requests when this issue occurs, and
the resources for the database are very low in usage.
I have disabled sqlalchemy pooling, and use pgbouncer in airflow as well as
in the PostgreSQL database.
I have the following configuration for airflow:
```
pgbouncer:
enabled: true
maxClientConn: 500
metadataPoolSize: 50
resultBackendPoolSize: 10
extraIni: |
pool_mode = transaction
```
On PostgreSQL, I have pgbouncer's pool_mode set to session, and the
default_pool_size is 100.
We have 16 DAGs, and have a daily job that runs a db clean for anything
older than 90 days.
### What you think should happen instead?
_No response_
### How to reproduce
Clicking on DAGs card screen - I have 16 DAGs running with an Azure
PostgreSQL backend.
### Operating System
Azure Kubernetes
### Versions of Apache Airflow Providers
_No response_
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
Azure PostgreSQL 16.9, Azure Kubernetes v1.32.7
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
