yihua commented on code in PR #13670:
URL: https://github.com/apache/hudi/pull/13670#discussion_r2250381531
##########
hudi-common/src/main/java/org/apache/hudi/common/table/read/InputSplit.java:
##########
@@ -81,4 +88,12 @@ public long getLength() {
public boolean isParquetBaseFile() {
return baseFileOption.map(baseFile ->
HoodieFileFormat.fromFileExtension(baseFile.getStoragePath().getFileExtension())
== HoodieFileFormat.PARQUET).orElse(false);
}
+
+ public boolean hasNoRecordsToMerge() {
+ return this.logFiles.isEmpty() && recordIterator.isEmpty();
+ }
+
+ public Option<Iterator<Pair<Serializable, BufferedRecord>>>
getRecordIterator() {
+ return this.recordIterator;
+ }
Review Comment:
In this way we can catch if `getLogFiles()` and `getRecordIterator()` are
called in the wrong place.
##########
hudi-common/src/main/java/org/apache/hudi/common/table/read/InputSplit.java:
##########
@@ -81,4 +88,12 @@ public long getLength() {
public boolean isParquetBaseFile() {
return baseFileOption.map(baseFile ->
HoodieFileFormat.fromFileExtension(baseFile.getStoragePath().getFileExtension())
== HoodieFileFormat.PARQUET).orElse(false);
}
+
+ public boolean hasNoRecordsToMerge() {
+ return this.logFiles.isEmpty() && recordIterator.isEmpty();
Review Comment:
```suggestion
return this.recordsToMerge.isLeft() &&
this.recordsToMerge.asLeft().isEmpty();
```
##########
hudi-common/src/main/java/org/apache/hudi/common/table/read/InputSplit.java:
##########
@@ -37,27 +41,30 @@
public class InputSplit {
private final Option<HoodieBaseFile> baseFileOption;
private final List<HoodieLogFile> logFiles;
+ private final Option<Iterator<Pair<Serializable, BufferedRecord>>>
recordIterator;
Review Comment:
Merge `logFiles` and `recordIterator` into `private final
Either<List<HoodieLogFile>, Iterator<Pair<Serializable, BufferedRecord>>>
recordsToMerge`.
##########
hudi-common/src/main/java/org/apache/hudi/common/table/read/InputSplit.java:
##########
@@ -81,4 +88,12 @@ public long getLength() {
public boolean isParquetBaseFile() {
return baseFileOption.map(baseFile ->
HoodieFileFormat.fromFileExtension(baseFile.getStoragePath().getFileExtension())
== HoodieFileFormat.PARQUET).orElse(false);
}
+
+ public boolean hasNoRecordsToMerge() {
+ return this.logFiles.isEmpty() && recordIterator.isEmpty();
+ }
+
+ public Option<Iterator<Pair<Serializable, BufferedRecord>>>
getRecordIterator() {
+ return this.recordIterator;
+ }
Review Comment:
```suggestion
public Iterator<Pair<Serializable, BufferedRecord>> getRecordIterator() {
ValidationUtils.checkArgument(recordsToMerge.isRight(), "...");
return this.recordsToMerge.asRight();
}
```
##########
hudi-common/src/main/java/org/apache/hudi/common/table/read/InputSplit.java:
##########
@@ -81,4 +88,12 @@ public long getLength() {
public boolean isParquetBaseFile() {
return baseFileOption.map(baseFile ->
HoodieFileFormat.fromFileExtension(baseFile.getStoragePath().getFileExtension())
== HoodieFileFormat.PARQUET).orElse(false);
}
+
+ public boolean hasNoRecordsToMerge() {
+ return this.logFiles.isEmpty() && recordIterator.isEmpty();
+ }
+
+ public Option<Iterator<Pair<Serializable, BufferedRecord>>>
getRecordIterator() {
+ return this.recordIterator;
+ }
Review Comment:
Do the same for `getLogFiles` to validate and return
`this.recordsToMerge.asLeft()`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
