[
https://issues.apache.org/jira/browse/HUDI-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kudinkin updated HUDI-3342:
----------------------------------
Description:
While working on HUDI-3322, i've spotted following contraption:
When we are rolling back Delta Commits, we add corresponding
{{ROLLBACK_PREVIOUS_BLOCK}} Command Block at the back of the "queue". When we
restore, we issue a sequence of Rollbacks, which means that stack if such
Rollback Blocks could be of size > 1.
However, when reading that MOR table if the reader does not specify
`readBlocksLazily=true`, we'd be merging Blocks eagerly (when instants
increment) therefore essentially rendering such Rollback Blocks useless since
they can't "unmerge" previously merged records, resurrecting the data that was
supposed to be rolled back.
was:
While working on HUDI-3322, i've spotted following contraption:
When we are rolling back Delta Commits, we add corresponding
{{ROLLBACK_PREVIOUS_BLOCK}} Command Block at the back of the "queue".
However, when reading that MOR table if the reader does not specify
`readBlocksLazily=true`, we'd be merging Blocks eagerly therefore essentially
rendering such Rollback Blocks useless, resurrecting the data that was
previously rolled back.
> MOR Delta Block Rollbacks not applied if Lazy Block reading is disabled
> -----------------------------------------------------------------------
>
> Key: HUDI-3342
> URL: https://issues.apache.org/jira/browse/HUDI-3342
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Alexey Kudinkin
> Priority: Blocker
> Fix For: 0.11.0
>
>
> While working on HUDI-3322, i've spotted following contraption:
> When we are rolling back Delta Commits, we add corresponding
> {{ROLLBACK_PREVIOUS_BLOCK}} Command Block at the back of the "queue". When we
> restore, we issue a sequence of Rollbacks, which means that stack if such
> Rollback Blocks could be of size > 1.
> However, when reading that MOR table if the reader does not specify
> `readBlocksLazily=true`, we'd be merging Blocks eagerly (when instants
> increment) therefore essentially rendering such Rollback Blocks useless since
> they can't "unmerge" previously merged records, resurrecting the data that
> was supposed to be rolled back.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
