5herhom commented on code in PR #6254:
URL: https://github.com/apache/hudi/pull/6254#discussion_r939635098
##########
hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieCopyOnWriteTableInputFormat.java:
##########
@@ -261,6 +263,20 @@ private List<FileStatus> listStatusForSnapshotMode(JobConf
job,
return targetFiles;
}
+ protected boolean checkIfValidFileSlice(FileSlice fileSlice) {
+ Option<HoodieBaseFile> baseFileOpt = fileSlice.getBaseFile();
Review Comment:
> As per the naming convention, `isValidFileSlice` sounds better. Also, I
notice that the override of this method in `HoodieMergeOnReadTableInputFormat`
is logically same. So, why do we need to override?
Re: Also, I notice that the override of this method in
`HoodieMergeOnReadTableInputFormat` is logically same. So, why do we need to
override?
If do not override `checkIfValidFileSlice()`, the valid file slice in mor
which match the condition `!baseFileOpt.isPresent() &&
latestLogFileOpt.isPresent()` will be filtered.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
