jason810496 opened a new pull request, #64833:
URL: https://github.com/apache/airflow/pull/64833
## Why
Airflow 3 introduces event-driven DAGs with Assets, AssetWatchers, and
MessageQueueTrigger to enable reactive, message-driven pipelines. This PR adds
a comprehensive e2e
test suite to verify that Kafka-based event routing works correctly,
including proper handling of malformed messages and dead-letter queues.
## What
- Dag pattern: Implemented a producer-consumer pattern where producer sends
9 messages (8 valid orders + 1 malformed) to a Kafka topic, and consumer
triggers 9 times via `AssetWatcher`
- Test verification: Consumer Dag uses retries (3), short-circuit on
malformed messages, and routes failures to a dead-letter queue topic
- CI integration: Extended selective-checks to detect event-driven test file
changes and trigger the suite; added workflow inputs/outputs for test mode
selection
- Kafka integration: The `kafka.yaml` and `update_run.sh` are copied from
`breeze --integration kafka`
##### Was generative AI tooling used to co-author this PR?
- [x] Yes (please specify the tool below) Claude Code following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
