dazza-codes edited a comment on issue #5788: [POC] multi-threading using asyncio URL: https://github.com/apache/airflow/pull/5788#issuecomment-569562651 Tagging this as related to - https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-28%3A+Add+AsyncExecutor+option - https://issues.apache.org/jira/browse/AIRFLOW-6395 👍 - I would like to see more done on this POC to create a new `AsyncExecutor` and apply it to async options for hooks, sensors and/or operators Once concern is that using `subprocess` might defeat the purpose of using coroutines for concurrency, if everything needs to be thrown across IPC anyway for this to work that way? That is, using subprocess incurs the overheads of multiprocessing, which should be avoided by async coroutines - right?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
