[
https://issues.apache.org/jira/browse/HUDI-2436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan closed HUDI-2436.
-------------------------------------
Resolution: Invalid
> rollback in cloud stores w/o append, wrt collecting failed log files to be
> deleted/logged
> -----------------------------------------------------------------------------------------
>
> Key: HUDI-2436
> URL: https://issues.apache.org/jira/browse/HUDI-2436
> Project: Apache Hudi
> Issue Type: Sub-task
> Components: Writer Core
> Reporter: sivabalan narayanan
> Assignee: sivabalan narayanan
> Priority: Major
> Fix For: 0.10.0
>
>
> I don't fully get
> [this|https://github.com/apache/hudi/pull/3651/files#r708820142] point. will
> follow up with Vinoth on the exact scenario.
> here is my understanding: of a scenario using cloud stores that does not
> support append.
> If there was crash during a commit, when listing log files to be logged, the
> last one which got crashed may not be part of the rollback plan. but thats
> should be fine. anyways, its not available via listing. and so I assume even
> during compaction those will not be available. we will proceed on with
> rollback by adding another log file. and this will get replayed to metadata
> table.
> If you are talking about the case, where a crash happens when rollback itself
> is being logged and crashed just before committing to metadata table.
> we should be ok here too. we will retry the rollback which will redo the
> action phase. and will add new log blocks (with same old logs that were part
> of failed writes, just that it may not be able to successfully delete). and
> this will get applied to metadata table. We just have to ensure when applying
> changes to metadata table, we consider all files from the plan and not just
> the ones that got successfully deleted.
>
> - with hdfs type of cloud stores, where appends are allowed, we just create
> new log blocks. and hence should not be an issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
