xBis7 commented on PR #54103: URL: https://github.com/apache/airflow/pull/54103#issuecomment-3275796757
> Without the change it took an average of ~0.57s per scheduler iteration, but with the change it's up to 0.97s per iteration. My guess is that it has to do with the added computations at the query. It used to check the limits with python after the query but now it does the calculations up front for each row that it passes. That gave me a testing idea. I'm going to load 100 dags with at least 500 tasks each on the db and then re-capture the metrics to see how that affects the loop's performance. I'll be back with the numbers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
