KevinYang21 commented on issue #3873: [Airflow-2760] Decouple DAG parsing loop from scheduler loop URL: https://github.com/apache/incubator-airflow/pull/3873#issuecomment-429241610 @Fokko Thanks for reviewing. I actually created [a ticket](https://issues.apache.org/jira/browse/AIRFLOW-3194) for further refactoring in the code base to use with block as there're a lot such cases and I feel it is beyond the scope of this PR to change all of them( updated all use cases that I touched). Would you actually elaborate a bit more on the "weird stuff in the number of connections" please? Might be something we didn't catch. Also this reminds me about one thing: with the new logic, it is more likely that the scheduler will leave orphan processes in extreme cases, e.g. for some reason the agent process crashed(or got a SIGTERM) or cannot gracefully exit, the dag parsing processes will be left behind. Should we put a note somewhere about this? Actually we already have similar problem, e.g. scheduler process got SIGTERM. Do we just expect people to know this and clean up the left over processes?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
