potiuk commented on PR #28822: URL: https://github.com/apache/airflow/pull/28822#issuecomment-1376884606
We had a lot of discussions about it with @jedcunningham in related issues and some time ago I wrote the article describing in detail why git-sync and networked persistence do not play along together (and do not serve any useful use case). https://medium.com/apache-airflow/shared-volumes-in-airflow-the-good-the-bad-and-the-ugly-22e9f681afca I think we've accumulated enough of the issues from our users that confirm that they REALLY did not want to use persistence and git-sync, and we had a lot of problems which we could attribute to lack of atomicity that is amplified by git-sync + network persistence. The last straw for me was this discussion https://apache-airflow.slack.com/archives/C027H098M1C/p1673333515991549. where the users wanted to use dags.persistence and dags.gitSync to combine development and production workflows in one (which would not work but the user was unaware of that as they did not know that how git-sync works under the hood it would overwrote the manually updated/copied dags). All-in-all - I think allowing this combo is actively harmful and we should help our users to make the right decision (and ourselves by not having similar discussions/getting issues created) by disabling the combo. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
