GitHub user nilskre added a comment to the discussion: Airflow development workflow and DAG Deployment from feature branches
Finally had the time to come back to this. Thanks a lot for the links and suggestions! It took me some time to understand the context and the idea/implementation of GitBundles. Finally realized that GitBundles are a great feature and are exactly what we need in our use case. Through GitBundles, we have been able to implement our own custom GitBundle. This custom `GitDagBundle` only contains changed DAGs from feature branches. The `dag_id` and `dag_display_name` are prefixed by the branch name to avoid collisions. Nevertheless from my own experience coming up with this solution was quite challenging. - In the beginning I was a little bit overwhelmed how everything works together and what are possible and clean ways to solve our problem. I think our setup with main/dev/feature branches the git-sync sidecar and the need to test DAGs on prod/dev prior to merging is a quite typical setup. For me a hands-on guide, explaining step by step how this could be achieved and which Airflow features are used for this, could have helped a lot. Would you see something like this as part of the Airflow docs? Or is this something which should be outside of the core Airflow docs? - Shouldn't the DAGBundle functionality be mentioned on [this docs page](https://airflow.apache.org/docs/helm-chart/1.7.0/manage-dags-files.html) as well? - The current [`GitDagBundle`](https://github.com/apache/airflow/blob/main/providers/git/src/airflow/providers/git/bundles/git.py) supports getting all DAGs from one repository. To make it much easier for others with the same use case: Maybe such a `FeatureBranchGitDagBundle` could also be part of the airflow git provider? Happy about your thoughts on this. @potiuk Thanks in advance! GitHub link: https://github.com/apache/airflow/discussions/54669#discussioncomment-14511848 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
