xBis7 commented on PR #68082: URL: https://github.com/apache/airflow/pull/68082#issuecomment-4635976090
> Shouldn't this be an Asset Watcher? @vikramkoka No but it could be used together with an Asset Watcher. An Asset Watcher is essentially on the consumer side of events while this PR is on the producer side. The new listener produces a message to a Kafka topic for every event state change. We can consume the messages from the topic either using the existing consumer hook from the Kafka provider or using an Asset Watcher. For example, team A with Airflow installation A triggers dag1 and has enabled the listener which publishes events for every state change to a Kafka topic 1. dag_run.dag1.running 2. task1.running 3. task1.success 4. task2.running 5. task2.success 6. dag_run.dag1.success Team B with Airflow installation B, is monitoring the same kafka topic and has a deferred task that waits to run after it consumes the message `task1.success`. The message consumption can happen using an Asset Watcher. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
