jose-workpath opened a new issue, #27483: URL: https://github.com/apache/airflow/issues/27483
### Apache Airflow version 2.4.2 ### What happened The web UI is very slow when loading the Graph view on DAGs that have a large number of expansions in the mapped tasks. The problem is very similar to the one described in #23786 (resolved), but for the Graph view instead of the grid view. It takes around 2-3 minutes to load DAGs that have ~1k expansions, with the default Airflow settings the web server worker will timeout. One can configure [web_server_worker_timeout](https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#web-server-worker-timeout) to increase the timeout wait time. ### What you think should happen instead The Web UI takes a reasonable amount of time to load the Graph view after the dag run is finished. ### How to reproduce Same way as in #23786, you can create a mapped task that spans a large number of expansions then when you run it the Graph view will take a very long amount of time to load and eventually time out. You can use this code to generate multiple dags with `2^x` expansions. After running the DAGs you should notice how slow it is when attempting to move to the Graph view of the DAGs with the largest number of expansions. ```python from datetime import datetime from airflow.models import DAG from airflow.operators.empty import EmptyOperator from airflow.operators.python import PythonOperator default_args = { 'owner': 'airflow', 'depends_on_past': False, 'email_on_failure': False, 'email_on_retry': False, } initial_scale = 7 max_scale = 12 scaling_factor = 2 for scale in range(initial_scale, max_scale + 1): dag_id = f"dynamic_task_mapping_{scale}" with DAG( dag_id=dag_id, default_args=default_args, catchup=False, schedule_interval=None, start_date=datetime(1970, 1, 1), render_template_as_native_obj=True, ) as dag: start = EmptyOperator(task_id="start") mapped = PythonOperator.partial( task_id="mapped", python_callable=lambda m: print(m), ).expand( op_args=[[x] for x in list(range(2**scale))] ) end = EmptyOperator(task_id="end") start >> mapped >> end globals()[dag_id] = dag ``` ### Operating System MacOS Version 12.6 (Apple M1) ### Versions of Apache Airflow Providers ``` apache-airflow-providers-amazon==4.0.0 apache-airflow-providers-common-sql==1.2.0 apache-airflow-providers-ftp==3.1.0 apache-airflow-providers-http==4.0.0 apache-airflow-providers-imap==3.0.0 apache-airflow-providers-sqlite==3.2.1 ``` ### Deployment Docker-Compose ### Deployment details _No response_ ### Anything else _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
