amit-mittal commented on issue #53157: URL: https://github.com/apache/airflow/issues/53157#issuecomment-3059552943
I do agree with you that it shouldn't take that long and there is definitely something wrong with our setup. In pre-prod environments which we have currently upgrade to v3, we allocate less resources, but still it shouldn't have been this slow. Regarding the index, I linked you to the GitHub issue in my previous message where we clearly are not using the index as part of scheduler loop. I enabled `otel` tracing, but I couldn't find any traces for the critical request from WebServer (I do see traces for Scheduler). Is there a way to get more details about this? Sorry, I couldn't understand what you meant by "i section g what is going wrong - which api calls are taking long and looking at corresponding queries in the dn". Some details about our setup: - We are running MySQL 8.x with 16 GB memory and 10 GB InnoDB buffer pool size. Our CPU usage used to be at 10%, but now it hovers at 50%. - The instance handles around 200 DAGs, not too complicated. In database, we have last 2 years history. - We have set page limit in Airflow as 100 - AirflowWebServer has 3 GB memory and can go max up to 8 GB - AirflowScheduler has 4 GB memory and can go max up to 10 GB. We use `LocalExecutor`. - AirflowDAGProcessor has 2 GB memory and can go max up to 5 GB. Production instance has more resources, but currently the instance running on Airflow v3.0.2 has the above resources. If you have a recommendation (or need more details), we can increase the allocated resources and tweak the Airflow configurations. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
