[
https://issues.apache.org/jira/browse/HUDI-5569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Raymond Xu reassigned HUDI-5569:
--------------------------------
Assignee: sivabalan narayanan (was: Jonathan Vexler)
> Files written by first commit/delta commit if it failed is detected as valid
> data files
> ---------------------------------------------------------------------------------------
>
> Key: HUDI-5569
> URL: https://issues.apache.org/jira/browse/HUDI-5569
> Project: Apache Hudi
> Issue Type: Bug
> Components: writer-core
> Reporter: sivabalan narayanan
> Assignee: sivabalan narayanan
> Priority: Critical
> Labels: pull-request-available
> Fix For: 0.13.0
>
>
> We have an method in HoodieFileGroup which detects whether a file group is
> committed or not. If timeline is such that,
> c1.inflight
> c2.complete
> c3.complete
>
> when we check for c1, it will return true.
> HoodieFileGroup.java
> {code:java}
> /**
> * A FileSlice is considered committed, if one of the following is true -
> There is a committed data file - There are
> * some log files, that are based off a commit or delta commit.
> */
> private boolean isFileSliceCommitted(FileSlice slice) {
> if (!compareTimestamps(slice.getBaseInstantTime(), LESSER_THAN_OR_EQUALS,
> lastInstant.get().getTimestamp())) {
> return false;
> }
> return timeline.containsOrBeforeTimelineStarts(slice.getBaseInstantTime());
> } {code}
> HoodieDefaultTimeline :
> {code:java}
> @Override
> public boolean containsOrBeforeTimelineStarts(String instant) {
> return getInstantsAsStream().anyMatch(s ->
> s.getTimestamp().equals(instant)) || isBeforeTimelineStarts(instant);
> } {code}
>
> This needs to be fixed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
