kpathak13 commented on issue #4727: [AIRFLOW-3902] Add new singleton_operator and unittest URL: https://github.com/apache/airflow/pull/4727#issuecomment-499530408 @BasPH, the solutions that you are describing, I feel, is just a workaround. Why would one want `TriggerDagRunOperator` and yet another `dag` if there is a simple solution to ensure one and only one processing pipeline is running within a `dag`? What about the visibility (I am sure this is an important aspect in data processing) if `TriggerDagRunOperator` does not trigger the other dag? With this operator, I can simply be certain that the critical data processing pipelines are not overwriting each other without any complicated dag(s) and also not consuming the resources (cpu, disk space) if they do not need to. In the event of an outage, I can be certain that with `catchup=True` dag configuration, any missed data processing pipeline can resume with 100% of confidence.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
