GitHub user tirkarthi added a comment to the discussion: Thread-Powered Local Executor
I am not sure about the complete use case but it seems like you are executing commands on remote hosts and waiting for long running commands which occupies the worker. Another option I could think of would be writing your own triggers using an asyncio compatible ssh library like asyncssh which will free up the workers and also could be more efficient with asyncio. https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/deferring.html https://github.com/ronf/asyncssh GitHub link: https://github.com/apache/airflow/discussions/57699#discussioncomment-14850814 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
