danny0405 commented on code in PR #6254:
URL: https://github.com/apache/hudi/pull/6254#discussion_r939453566
##########
hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieCopyOnWriteTableInputFormat.java:
##########
@@ -189,11 +190,16 @@ protected List<FileStatus>
listStatusForIncrementalMode(JobConf job,
return HoodieInputFormatUtils.filterIncrementalFileStatus(jobContext,
tableMetaClient, timeline.get(), fileStatuses, commitsToCheck.get());
}
- protected FileStatus createFileStatusUnchecked(FileSlice fileSlice,
HiveHoodieTableFileIndex fileIndex, Option<HoodieVirtualKeyInfo>
virtualKeyInfoOpt) {
+ protected Option<? extends FileStatus> createFileStatusUnchecked(FileSlice
fileSlice, HiveHoodieTableFileIndex fileIndex, Option<HoodieVirtualKeyInfo>
virtualKeyInfoOpt) {
Option<HoodieBaseFile> baseFileOpt = fileSlice.getBaseFile();
+ Option<HoodieLogFile> latestLogFileOpt = fileSlice.getLatestLogFile();
if (baseFileOpt.isPresent()) {
return getFileStatusUnchecked(baseFileOpt.get());
+ } else if (latestLogFileOpt.isPresent()) {
+ // It happens when reading optimized query to mor.
+ LOG.info("File slice(" + fileSlice.getFileId() + ") has no base-file but
log-file. Skip the slice.");
+ return Option.empty();
Review Comment:
That's true, let's make this fix into release 0.12.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
