Manoj Govindassamy created HUDI-3072:
----------------------------------------
Summary: AutoCommit misses to detect write conflicts during
concurrent transactions
Key: HUDI-3072
URL: https://issues.apache.org/jira/browse/HUDI-3072
Project: Apache Hudi
Issue Type: Task
Components: Writer Core
Reporter: Manoj Govindassamy
Assignee: Manoj Govindassamy
Fix For: 0.11.0
When auto commit is enabled, the table write operation triggers the commit of
the write result set. In this code path the transaction is started just in the
autoCommit block and the meta client here is not the latest one and doesn't
reflect the latest committed data set. The meta client it uses to detect the
conflicts does't show the potential commits from the concurrent transactions
and there by can miss out to detect the write conflicts during concurrent
transactions.
In the write and explicit commit code path, the table and meta client is built
again inside the transaction so as to get the latest refreshed view of the
timeline. It sees all the commits since the current inflight write started and
able to detect the write conflicts reliably.
Need to improve the autoCommit code path to detect the write conflicts
reliably.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
