alexeykudinkin commented on code in PR #6254:
URL: https://github.com/apache/hudi/pull/6254#discussion_r939254528
##########
hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieCopyOnWriteTableInputFormat.java:
##########
@@ -267,9 +275,9 @@ private void validate(List<FileStatus> targetFiles,
List<FileStatus> legacyFileS
}
@Nonnull
- protected static FileStatus getFileStatusUnchecked(HoodieBaseFile baseFile) {
+ protected static Option<? extends FileStatus>
getFileStatusUnchecked(HoodieBaseFile baseFile) {
Review Comment:
This method should not be returning `Option`
##########
hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/realtime/HoodieMergeOnReadTableInputFormat.java:
##########
@@ -349,11 +350,11 @@ private static RealtimeFileStatus
createRealtimeFileStatusUnchecked(HoodieBaseFi
/**
* Creates {@link RealtimeFileStatus} for the file-slice where base file is
NOT present
*/
- private static RealtimeFileStatus
createRealtimeFileStatusUnchecked(HoodieLogFile latestLogFile,
-
Stream<HoodieLogFile> logFiles,
- String
basePath,
-
Option<HoodieInstant> latestCompletedInstantOpt,
-
Option<HoodieVirtualKeyInfo> virtualKeyInfoOpt) {
+ private static Option<? extends RealtimeFileStatus>
createRealtimeFileStatusUnchecked(HoodieLogFile latestLogFile,
Review Comment:
This method should not be changing either
##########
hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/realtime/HoodieMergeOnReadTableInputFormat.java:
##########
@@ -317,12 +317,13 @@ private static HoodieRealtimeBootstrapBaseFileSplit
createRealtimeBoostrapBaseFi
/**
* Creates {@link RealtimeFileStatus} for the file-slice where base file is
present
*/
- private static RealtimeFileStatus
createRealtimeFileStatusUnchecked(HoodieBaseFile baseFile,
-
Stream<HoodieLogFile> logFiles,
- String
basePath,
-
Option<HoodieInstant> latestCompletedInstantOpt,
-
Option<HoodieVirtualKeyInfo> virtualKeyInfoOpt) {
- FileStatus baseFileStatus = getFileStatusUnchecked(baseFile);
+ private static Option<? extends RealtimeFileStatus>
createRealtimeFileStatusUnchecked(HoodieBaseFile baseFile,
Review Comment:
Why is this method changing?
##########
hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieCopyOnWriteTableInputFormat.java:
##########
@@ -254,6 +260,8 @@ private List<FileStatus> listStatusForSnapshotMode(JobConf
job,
.stream()
.flatMap(Collection::stream)
.map(fileSlice -> createFileStatusUnchecked(fileSlice,
fileIndex, virtualKeyInfoOpt))
+ .filter(FileStatusOpt -> FileStatusOpt.isPresent())
Review Comment:
Instead of changing the API of the `createFileStatusUnchecked`, let's just
move up this filtering to be preceding its invocation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
