voonhous commented on code in PR #14340:
URL: https://github.com/apache/hudi/pull/14340#discussion_r2597654596
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieDataBlock.java:
##########
@@ -362,24 +363,23 @@ protected abstract <T> ClosableIterator<HoodieRecord<T>>
deserializeRecords(
public abstract HoodieLogBlockType getBlockType();
- protected Option<Schema.Field> getKeyField(Schema schema) {
- return Option.ofNullable(schema.getField(keyFieldName));
- }
-
protected Option<String> getRecordKey(HoodieRecord record) {
- return Option.ofNullable(record.getRecordKey(readerSchema, keyFieldName));
+ return Option.ofNullable(record.getRecordKey(readerSchema.toAvroSchema(),
keyFieldName));
}
- protected Schema getSchemaFromHeader() {
+ protected HoodieSchema getSchemaFromHeader() {
String schemaStr = getLogBlockHeader().get(HeaderMetadataType.SCHEMA);
SCHEMA_MAP.computeIfAbsent(schemaStr,
(schemaString) -> {
try {
- return new Schema.Parser().parse(schemaStr);
- } catch (AvroTypeException e) {
+ return HoodieSchema.parse(schemaStr);
+ } catch (HoodieAvroSchemaException e) {
// Archived commits from earlier hudi versions fail the schema
check
- // So we retry in this one specific instance.
- return new
Schema.Parser().setValidateDefaults(false).parse(schemaStr);
+ // So we retry in this one specific instance with validation
disabled
Review Comment:
The exact test that was failing is (for my reference in the future):
```
org.apache.hudi.common.table.timeline.TestArchivedTimelineV1.shouldReadArchivedFileAndValidateContent
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
