the-other-tim-brown commented on code in PR #17763:
URL: https://github.com/apache/hudi/pull/17763#discussion_r2656632535
##########
hudi-hadoop-common/src/main/java/org/apache/hudi/common/util/AvroOrcUtils.java:
##########
@@ -356,15 +353,15 @@ public static void addToVector(TypeDescription type,
ColumnVector colVector, Sch
*
* @param unionVector The vector to store value.
* @param unionChildTypes All possible types for the value Object.
- * @param avroSchema Avro union schema for the value Object.
+ * @param schema Avro union schema for the value Object.
Review Comment:
nit: remove `Avro` from the comment here
##########
hudi-hadoop-common/src/main/java/org/apache/hudi/io/storage/hadoop/HoodieAvroParquetReader.java:
##########
@@ -108,27 +108,27 @@ public Set<Pair<String, Long>> filterRowKeys(Set<String>
candidateRowKeys) {
@Override
protected ClosableIterator<IndexedRecord>
getIndexedRecordIterator(HoodieSchema schema) throws IOException {
//TODO boundary for now to revisit in later pr to use HoodieSchema
- return getIndexedRecordIteratorInternal(schema.getAvroSchema(),
Collections.emptyMap());
+ return getIndexedRecordIteratorInternal(schema, Collections.emptyMap());
}
@Override
public ClosableIterator<IndexedRecord> getIndexedRecordIterator(HoodieSchema
readerSchema, HoodieSchema requestedSchema) throws IOException {
//TODO boundary for now to revisit in later pr to use HoodieSchema
- return getIndexedRecordIteratorInternal(requestedSchema.getAvroSchema(),
Collections.emptyMap());
+ return getIndexedRecordIteratorInternal(requestedSchema,
Collections.emptyMap());
}
@Override
public ClosableIterator<IndexedRecord> getIndexedRecordIterator(HoodieSchema
readerSchema, HoodieSchema requestedSchema, Map<String, String> renamedColumns)
throws IOException {
//TODO boundary for now to revisit in later pr to use HoodieSchema
- return getIndexedRecordIteratorInternal(requestedSchema.getAvroSchema(),
renamedColumns);
+ return getIndexedRecordIteratorInternal(requestedSchema, renamedColumns);
}
@Override
public HoodieSchema getSchema() {
if (fileSchema.isEmpty()) {
- fileSchema = Option.ofNullable(parquetUtils.readAvroSchema(storage,
path));
+ fileSchema = Option.ofNullable(parquetUtils.readHoodieSchema(storage,
path));
}
- return HoodieSchema.fromAvroSchema(fileSchema.get());
+ return fileSchema.get();
Review Comment:
Can you just use `orElseGet`?
##########
hudi-hadoop-common/src/main/java/org/apache/hudi/io/storage/hadoop/HoodieAvroParquetReader.java:
##########
@@ -107,28 +107,24 @@ public Set<Pair<String, Long>> filterRowKeys(Set<String>
candidateRowKeys) {
@Override
protected ClosableIterator<IndexedRecord>
getIndexedRecordIterator(HoodieSchema schema) throws IOException {
- //TODO boundary for now to revisit in later pr to use HoodieSchema
- return getIndexedRecordIteratorInternal(schema.getAvroSchema(),
Collections.emptyMap());
+ return getIndexedRecordIteratorInternal(schema, Collections.emptyMap());
}
@Override
public ClosableIterator<IndexedRecord> getIndexedRecordIterator(HoodieSchema
readerSchema, HoodieSchema requestedSchema) throws IOException {
//TODO boundary for now to revisit in later pr to use HoodieSchema
Review Comment:
Let's remove the TODO comments in this file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
