danny0405 commented on a change in pull request #3203:
URL: https://github.com/apache/hudi/pull/3203#discussion_r731567300
##########
File path:
hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieRealtimeInputFormatUtils.java
##########
@@ -161,6 +162,46 @@
return rtSplits.toArray(new InputSplit[0]);
}
+ // pick all incremental files and add them to rtSplits then filter out those
files.
+ private static Map<Path, List<FileSplit>> filterOutIncrementalSplits(
+ List<FileSplit> fileSplitList,
Review comment:
I still think there is no need to handle the incremental splits first,
can we just merge the handling into the line 139 ~ line 148, and logic for
`BaseFileWithLogsSplit` can be reused ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
