[
https://issues.apache.org/jira/browse/KAFKA-10000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17567511#comment-17567511
]
Christopher L. Shannon commented on KAFKA-10000:
------------------------------------------------
There is a bug that I documented at KAFKA-14079 that is still relevant here
after the refactoring. The PR I created would need to be re-worked due to the
refactoring which I can do if requested or the fix could be rolled into another
PR here that exists if not merged, etc.
> Atomic commit of source connector records and offsets
> -----------------------------------------------------
>
> Key: KAFKA-10000
> URL: https://issues.apache.org/jira/browse/KAFKA-10000
> Project: Kafka
> Issue Type: New Feature
> Components: KafkaConnect
> Reporter: Chris Egerton
> Assignee: Chris Egerton
> Priority: Blocker
> Labels: needs-kip
> Fix For: 3.3.0
>
>
> It'd be nice to be able to configure source connectors such that their
> offsets are committed if and only if all records up to that point have been
> ack'd by the producer. This would go a long way towards EOS for source
> connectors.
>
> This differs from https://issues.apache.org/jira/browse/KAFKA-6079, which is
> marked as {{WONTFIX}} since it only concerns enabling the idempotent producer
> for source connectors and is not concerned with source connector offsets.
> This also differs from https://issues.apache.org/jira/browse/KAFKA-6080,
> which had a lot of discussion around allowing connector-defined transaction
> boundaries. The suggestion in this ticket is to only use source connector
> offset commits as the transaction boundaries for connectors; allowing
> connector-specified transaction boundaries can be addressed separately.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
