[
https://issues.apache.org/jira/browse/HUDI-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Mahindra updated HUDI-2671:
----------------------------------
Status: Resolved (was: Patch Available)
> Fix record offset handling in Kafka connect transaction participant
> -------------------------------------------------------------------
>
> Key: HUDI-2671
> URL: https://issues.apache.org/jira/browse/HUDI-2671
> Project: Apache Hudi
> Issue Type: Sub-task
> Reporter: Ethan Guo
> Assignee: Rajesh Mahindra
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 0.10.0
>
>
> When there are two sink workers running, there can be a case where one
> participant joins after the coordinator starts a first commit, which needs to
> be rolled back later since the other participant does not receive the
> START_COMMIT message for the transaction. In this case, later on in a new
> commit, `writeRecords()` can miss records because
> `ongoingTransactionInfo.getLastWrittenKafkaOffset()` is behind the record
> offsets in the buffer. This causes missing records in the target Hudi table.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
