jonvex commented on code in PR #9743:
URL: https://github.com/apache/hudi/pull/9743#discussion_r1357354486
##########
hudi-common/src/main/java/org/apache/hudi/io/storage/HoodieAvroParquetReader.java:
##########
@@ -165,7 +165,10 @@ private ClosableIterator<IndexedRecord>
getIndexedRecordIteratorInternal(Schema
AvroReadSupport.setAvroReadSchema(conf, requestedSchema.get());
AvroReadSupport.setRequestedProjection(conf, requestedSchema.get());
}
- ParquetReader<IndexedRecord> reader = new
HoodieAvroParquetReaderBuilder<IndexedRecord>(path).withConf(conf).build();
+ ParquetReader<IndexedRecord> reader = new
HoodieAvroParquetReaderBuilder<IndexedRecord>(path).withConf(conf)
+ .set(ParquetInputFormat.STRICT_TYPE_CHECKING, "false")
+ .set(AvroSchemaConverter.ADD_LIST_ELEMENT_RECORDS, "false")
Review Comment:
tryOverrideDefaultConfigs pretty much hardcodes this. But
tryOverrideDefaultConfigs is wrong. It is not setting the right config. I
stepped through the code with the debugger and this is how you set it
correctly. This is no longer relevant for this pr since I am focused on row
writer enable so maybe I should revert this and make a jira?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
