jason810496 opened a new pull request, #52015:
URL: https://github.com/apache/airflow/pull/52015
closes: #34213
## Why
Based on the discussion in the issue
- if we need to respect `commit_cadence="never", "end_of_batch",
"end_of_operator"`
- we should set `enable.auto.commit` option in Kafka Connection to `false`
- otherwise `enable.auto.commit` would be on by default, and the consumer
will auto commit the offset every 5 seconds.
## What
- Add validation for `commit_cadence` with `enable.auto.commit` option with
corresponding Kafka Connection
- Will log warning if the `commit_cadence` is set but `enable.auto.commit`
is not `false`
- Point out this behavior in the documentation as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
