okayhooni opened a new pull request, #32013:
URL: https://github.com/apache/airflow/pull/32013
We currently deploy and operate 2 Airflow cluster (prod & beta) connected
with separate branch(like `prod`, and `beta`) via git-sync.
But, we had some difficulty when we needed to sync two separate
branches(`prod` and `beta`) periodically with proper branching strategy
- cherry-pick is too cumbersome to maintain repo.
- hard-reset is not recommended solution for GitOps
So, we decided to connect two Airflow deployments to same branch(like
`main`). But, if needed, we have to activate specific DAG on specific airflow
deployment only, based on the DAG tag!
I found `is_active` prop on `DagModel` ORM object. but I think it's not
efficient to set this prop on the `dag_policy`, due to metadb overhead per
parsing DAG.
How about introducing `AirflowClusterPolicySkipDag` exception, and use it
for skipping dag without Import error message on Web UI?
example)
```
def dag_policy(dag: DAG):
if "only_for_beta" in dag.tags:
raise AirflowClusterPolicySkipDag("`only_for_beta` tag is detected")
```
It's just making a back-door for another MR (I previously committed) below.
related: https://github.com/apache/airflow/pull/29056
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
