potiuk commented on issue #26623: URL: https://github.com/apache/airflow/issues/26623#issuecomment-1258078992
OK. Thanks for the explanation. I think the reason why it has changed in 2.2.0 is this chage: https://github.com/apache/airflow/pull/16860 but I think the root cause is that chart / K8S executor configuration you use does not properly set DAGS_FOLDER variable. This change in 2.2.0 fixed the behaviour that has been accidentally broken in 2.2.0 - namely possibility of using different path in Scheduler and worker/K8S pod that executed it. This has been broken in 2.0.0 and it was fixed in 2.2.0, and it relied on the fact that DAGS_FOLDER variable is properly set in each of the components - and rather than passing the "absolute" path it passed `DAGS_FOLDER/<relative_path>` which uses this configuration to figure out where the DAGs are: https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#dags-folder So the root problem of yours is that you have missing or wrongly set `AIRFLOW__CORE__DAGS_FOLDER` in the POD that executes the task. And this is likely because you can have hard-coded `AIRFLOW__CORE__DAGS_FOLDER` pointing to `/opt/bitnami/airflow/` or similar settings. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
