jonvex commented on code in PR #13654:
URL: https://github.com/apache/hudi/pull/13654#discussion_r2255095889
##########
hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HiveHoodieReaderContext.java:
##########
@@ -121,6 +128,10 @@ public ClosableIterator<ArrayWritable>
getFileRecordIterator(
private ClosableIterator<ArrayWritable> getFileRecordIterator(StoragePath
filePath, String[] hosts, long start, long length, Schema dataSchema,
Schema
requiredSchema, HoodieStorage storage) throws IOException {
+ boolean isParquet =
filePath.getFileExtension().equals(HoodieFileFormat.PARQUET.getFileExtension());
+ Schema avroFileSchema = isParquet ? HoodieIOFactory.getIOFactory(storage)
+ .getFileFormatUtils(filePath).readAvroSchema(storage, filePath) :
dataSchema;
+ Schema actualRequiredSchema = isParquet ?
AvroSchemaUtils.pruneDataSchema(avroFileSchema, requiredSchema,
Collections.emptySet()) : requiredSchema;
Review Comment:
it's actually that we don't want hfile. So I flipped it. Because mdt the
schema from the file is different and things fail if we try to use it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
