ecodina opened a new issue, #61686:
URL: https://github.com/apache/airflow/issues/61686

   ### Apache Airflow version
   
   3.1.7
   
   ### If "Other Airflow 3 version" selected, which one?
   
   _No response_
   
   ### What happened?
   
   When using the keycloak auth manager I'm finding very slow response times 
(with SimpleAuthManager responses are immediate).
   
   For example, `GET 
ui/dashboard/historical_metrics_data?start_date=2026-02-08T19%3A41%3A38.997Z` 
takes around 25 seconds.
   
   When I deployed Airflow in our staging environment, I also experienced this 
problem. I solved it by increasing the available resources and implementing 
connection pooling in the auth manager. In there, I had around 20 dags.
   
   I am now trying to deploy our production instance, where we have 600 Dags. 
The instance is clean (no historical data), all Dags are still paused, and I'm 
the only user with access to it.
   
   I've tried different values for our connection pool, and increased the 
available resources to 8 CPUs, but it hasn't fixed the issue. Our worker nodes 
are capped at 8 CPUs, so increasing resources is not a possibility (and 8 CPUs 
should be more than enough for our use case: in Airflow 2 it consumed less than 
1 CPU with the same Dags using FAB provider).
   
   From my findings with our staging environment, the problem was verifying the 
SSL cert: that's what spiked CPU usage, and what connection pooling solved for 
us.
   
   I don't really know what to look at anymore (I've spent several days 
debugging the issue). Has this happened to anyone else? Could someone point me 
to a potential solution?
   
   ### What you think should happen instead?
   
   _No response_
   
   ### How to reproduce
   
   Use Airflow 3.1.7 with Keycloak Auth Manager in a container capped at 8 CPUs 
with 600 Dags
   
   ### Operating System
   
   Debian GNU/Linux 12 (bookworm)
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Docker-Compose
   
   ### Deployment details
   
   _No response_
   
   ### Anything else?
   
   @vincbeck @bugraoz93 I can work on this issue and open a PR, but I really 
don't know where to look at. Any feedback would be appreciated!
   
   ### Are you willing to submit PR?
   
   - [x] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to