[
https://issues.apache.org/jira/browse/HUDI-5465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan updated HUDI-5465:
--------------------------------------
Reviewers: sivabalan narayanan
> Fix compaction and rollback handling in MDT for multi-writer scenarios in DT
> ----------------------------------------------------------------------------
>
> Key: HUDI-5465
> URL: https://issues.apache.org/jira/browse/HUDI-5465
> Project: Apache Hudi
> Issue Type: Bug
> Components: metadata
> Reporter: sivabalan narayanan
> Assignee: Ethan Guo
> Priority: Blocker
> Fix For: 0.13.0
>
>
> lets say c50 is latest DC in MDT. c49 from DT comes through(multi-writer).
> triggers compaction in MDT(since ignoring c49 there are no other pending
> instants in DT). new base instant time is c50. and we add 49.deltacommit to
> MDT. and during the process we crash.
> rollback for 49 kicks in DT. When applying rollback of 49 to MDT, we detect
> 49 has already been compacted since last compacted time is 50 and {*}fail the
> rollback when we try to apply to MDT{*}.
>
> We need to fix this entire flow for rollbacks and compaction related
> multi-writer scenarios.
>
> Impact:
> writes to MDT might fail at some point and users have to disable MDT and make
> progress
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
