amoghrajesh commented on code in PR #76545: URL: https://github.com/apache/airflow/pull/76545#discussion_r4560034116
########## task-sdk/src/airflow/sdk/execution_time/comms.py: ########## Review Comment: Could async calls hand their work to one background worker owned by the decoder (ie: a `ThreadPoolExecutor(max_workers=1)`), instead of using a waiting line per event loop? Requests already go one at a time because of the lock, so a single worker costs nothing. It would remove the per loop waiting line and its memory workarounds (ie: `_LoopQueue`, `_LoopQueues`, the weak references). It would also remove the cancel marker, because a cancelled request that hasn't started never runs. Sync calls stay as they are. If the process can fork, the worker would need to be recreated in the child. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
