[
https://issues.apache.org/jira/browse/HUDI-5463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan updated HUDI-5463:
--------------------------------------
Description:
As of now, any rollback in DT is another DC in MDT. this may not scale for
record level index in MDT since we have to add 1000s of delete records and
finally have to resolve all valid and invalid records. So, its better to
rollback the commit in MDT as well instead of doing a DC.
Impact:
record level index is unusable w/o this change. While fixing other rollback
related tickets, do consider this as a possible option if this simplifies other
fixes.
was:
As of now, any rollback in DT is another DC in MDT. this may not scale for
record level index in MDT since we have to add 1000s of delete records and
finally have to resolve all valid and invalid records. So, its better to
rollback the commit in MDT as well instead of doing a DC.
> Apply rollback commits from data table as rollbacks in MDT instead of Delta
> commit
> ----------------------------------------------------------------------------------
>
> Key: HUDI-5463
> URL: https://issues.apache.org/jira/browse/HUDI-5463
> Project: Apache Hudi
> Issue Type: Bug
> Components: metadata
> Reporter: sivabalan narayanan
> Assignee: sivabalan narayanan
> Priority: Critical
> Fix For: 0.13.0
>
>
> As of now, any rollback in DT is another DC in MDT. this may not scale for
> record level index in MDT since we have to add 1000s of delete records and
> finally have to resolve all valid and invalid records. So, its better to
> rollback the commit in MDT as well instead of doing a DC.
>
> Impact:
> record level index is unusable w/o this change. While fixing other rollback
> related tickets, do consider this as a possible option if this simplifies
> other fixes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
