jstaf opened a new pull request, #59123: URL: https://github.com/apache/airflow/pull/59123
In the official documentation, deploying Airflow with both git-sync (`dags.gitSync.enabled=true`) and persistence enabled (`dags.persistence.enabled=true`) is a supported deployment pattern: https://airflow.apache.org/docs/helm-chart/1.8.0/manage-dags-files.html#mounting-dags-using-git-sync-sidecar-with-persistence-enabled. On Airflow 2.x this works just fine, the scheduler pod runs a git-sync sidecar container which downloads DAGs onto a shared volume on behalf of the entire Airflow installation. On Airflow 3.x, there is an if-statement that turns off the git-sync sidecar in the scheduler pod since the dag-processor pod is now present https://github.com/apache/airflow/blob/main/chart/templates/scheduler/scheduler-deployment.yaml#L254-L256. However, when we look at the dag-processor pod, it incorrectly disables the git-sync sidecar when DAG persistence is enabled https://github.com/apache/airflow/blob/main/chart/templates/dag-processor/dag-processor-deployment.yaml#L198. This sadly means that git-sync doesn't run anywhere, because the dag-processor was supposed to be the one doing the syncing. This PR fixes the if statement so that the dag-processor _always_ runs the git-sync sidecar irrespective of if persistence is enabled or not. So with this change, Airflow 3.x actually works with both git-sync and DAG persistence enabled again. You can check this for yourself by doing `helm template airflow chart --set=dags.gitSync.enabled=true --set=dags.persistence.enabled=true | less` with and without this PR. (With this PR's change, there is a git-sync container in the rendered manifest, and without this PR the git-sync container is totally missing.) <!-- Please keep an empty line above the dashes. --> --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
