yuqian90 commented on issue #13066: URL: https://github.com/apache/airflow/issues/13066#issuecomment-745804077
I'd love to. But i'm busy with other things at the moment. Won't be able to get this at least till next month. If someone likes to take it please go for it. For the time being, @scauglog , did you consider simply adding a few layers of TaskGroup to nest the 300 tasks in each TaskGroup into a more levels? E.g. first level has 4 TaskGroups. Inside each TaskGroup, there are 10 sub-TaskGroups. In each sub-TaskGroup, there are 300 tasks. I guess it might help because the bottleneck is usually when Graph View tries to render all the nodes. As long as it doesn't have to render all 3000 tasks at once, it's usually not too bad. Let me know if that improves the speed. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
