Ethan Guo created HUDI-3721:
-------------------------------
Summary: Metadata table blocks rollback and restore to savepoint
before bootstrapped/init commit
Key: HUDI-3721
URL: https://issues.apache.org/jira/browse/HUDI-3721
Project: Apache Hudi
Issue Type: Bug
Reporter: Ethan Guo
Assignee: Ethan Guo
Fix For: 0.11.0
Say the table has instants C1 to C4, and the data table does not have MDT
enabled. After C4, the writer enables MDT so MDT has DC4 as the first instant
with the same timestamp as C4. The rollback of any commit before C4 on data
table is going to fail now due to the following check. This is going to fail
restore to a savepoint before C4 as well if there is any. Yet the check itself
can be relaxed in such a case to allow the rollback to get through.
{code:java}
C1 C2 C3 C4
| metadata table init
// Case 2: The instant-to-rollback was never committed to Metadata Table.
This can happen if the instant-to-rollback
// was a failed commit (never completed) as only completed instants are
synced to Metadata Table.
// But the required Metadata Table instants should not have been archived
HoodieInstant syncedInstant = new HoodieInstant(false,
HoodieTimeline.DELTA_COMMIT_ACTION, instantToRollback);
if
(metadataTableTimeline.getCommitsTimeline().isBeforeTimelineStarts(syncedInstant.getTimestamp()))
{
throw new HoodieMetadataException(String.format("The instant %s
required to sync rollback of %s has been archived",
syncedInstant, instantToRollback));
}{code}
Prashant proposes to make the bootstrap commit of metadata table have a
specific suffix (just like we have 001 and 002 for compaction and clean). This
will make it trivial to detect such cases.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
