Hi everyone, We had some issues with the ShortCircuitOperator, so had to come up with an alternative implementation.
I've written up the details and the workaround in this blog posting: https://blog.diffractive.io/2018/08/07/replacement-shortcircuitoperator-for-airflow/ Seems that to fix this in airflow properly, it needs some design changes. I think the sanest solution would be to have the scheduler handle skip operations, rather than implement this inside an operator. That would mean either having the scheduler read xcom data, or elevating the status of the python member to a first class attribute on a task. Anyway, hoping this posting might at least save some users some headaches if they want the ShortCircuitOperator to behave better. Comments and feedback appreciated. Regards, Kevin
