cshuo commented on code in PR #13388:
URL: https://github.com/apache/hudi/pull/13388#discussion_r2131693035
##########
hudi-common/src/main/java/org/apache/hudi/io/storage/HoodieAvroFileReader.java:
##########
@@ -46,4 +49,29 @@ protected ClosableIterator<IndexedRecord>
getIndexedRecordIterator(Schema reader
}
public abstract ClosableIterator<IndexedRecord>
getIndexedRecordIterator(Schema readerSchema, Schema requestedSchema) throws
IOException;
+
+ public abstract ClosableIterator<IndexedRecord>
getIndexedRecordsByKeysIterator(List<String> keys,
Review Comment:
getIndexedRecordsIteratorByKeys?
##########
hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroReaderContext.java:
##########
@@ -94,25 +90,14 @@ public ClosableIterator<IndexedRecord>
getFileRecordIterator(
if (keyFilterOpt.isEmpty()) {
return reader.getIndexedRecordIterator(dataSchema, requiredSchema);
} else {
- // Currently predicate is only supported for HFile reader.
- if (!(reader instanceof HoodieAvroHFileReaderImplBase)) {
- return reader.getIndexedRecordIterator(dataSchema, requiredSchema);
+ if (reader.supportKeyPredicate() &&
!reader.extractKeys(keyFilterOpt).isEmpty()) {
+ List<String> keys = reader.extractKeys(keyFilterOpt);
+ return reader.getIndexedRecordsByKeysIterator(keys, requiredSchema);
+ } else if (reader.supportKeyPrefixPredicate() &&
!reader.extractKeyPrefixes(keyFilterOpt).isEmpty()) {
+ List<String> keyPrefixes = reader.extractKeyPrefixes(keyFilterOpt);
Review Comment:
`reader.extractKeyPrefixes` is called twice, the second one can be removed
to avoid additional cost?
##########
hudi-common/src/main/java/org/apache/hudi/io/storage/HoodieAvroFileReader.java:
##########
@@ -46,4 +49,29 @@ protected ClosableIterator<IndexedRecord>
getIndexedRecordIterator(Schema reader
}
public abstract ClosableIterator<IndexedRecord>
getIndexedRecordIterator(Schema readerSchema, Schema requestedSchema) throws
IOException;
+
+ public abstract ClosableIterator<IndexedRecord>
getIndexedRecordsByKeysIterator(List<String> keys,
+
Schema readerSchema)
+ throws IOException;
+
+ public abstract ClosableIterator<IndexedRecord>
getIndexedRecordsByKeyPrefixIterator(
Review Comment:
getIndexedRecordsIteratorByKeyPrefix?
##########
hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroReaderContext.java:
##########
@@ -94,25 +90,14 @@ public ClosableIterator<IndexedRecord>
getFileRecordIterator(
if (keyFilterOpt.isEmpty()) {
return reader.getIndexedRecordIterator(dataSchema, requiredSchema);
} else {
- // Currently predicate is only supported for HFile reader.
- if (!(reader instanceof HoodieAvroHFileReaderImplBase)) {
- return reader.getIndexedRecordIterator(dataSchema, requiredSchema);
+ if (reader.supportKeyPredicate() &&
!reader.extractKeys(keyFilterOpt).isEmpty()) {
Review Comment:
`reader.extractKeys` is called twice, the second one can be removed to avoid
additional cost?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
