potiuk commented on issue #19192: URL: https://github.com/apache/airflow/issues/19192#issuecomment-1003697121
I think you are misusing Airlfow. Airlfow by definition can run multiple tasks even with single local executor, so you are likely misunderstanding how airflow operators are run and run it as a top-level code, and not as task? Can you please copy your DAG here? You are not supposed to run long running operations when the DAG file is parsed - parsing should be run rather quickly and from what it looks like you execute a long running process while parsing happens rather than when tasks are executed: https://airflow.apache.org/docs/apache-airflow/stable/best-practices.html#top-level-python-code -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
