GitHub user potiuk edited a comment on the discussion: Dagbag import is taking longer time than an older version, which makes some tasks get timeout exceed errors
That's at most an interesting discussion. The increase in import might come from longer import classes for Airflow internal classes for example. Airflow 2.9 has significantly more features and imports more classes, which make the import take longer. It might be triggered by some imports your dag does. Also there might be differences coming from what composer environment dependencies are. If you you want to get to the bottom of it - possibly the easiest way for you to see the difference (and for anyone being able to comment on this) is to get the two environment with your dags, exec to the environment, install `py-spy` on both and produce a flamegraph of simply running `python your_dag.py` - see https://github.com/benfred/py-spy?tab=readme-ov-file#record. Then it would be great if you publish both flamegraphs here or even perform analysis yourself and see where the differences came from. That would be really useful if you did that analysis, maybe thanks to that you can find some consistent issue that can be diagnosed and fixed - eiither in Airflow or composer environment. GitHub link: https://github.com/apache/airflow/discussions/44402#discussioncomment-11389224 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
