cshuo commented on code in PR #14120:
URL: https://github.com/apache/hudi/pull/14120#discussion_r2446885563
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieAvroDataBlock.java:
##########
@@ -195,8 +198,8 @@ private RecordIterator(Schema readerSchema, Schema
writerSchema, byte[] content)
}
}
- public static RecordIterator getInstance(HoodieAvroDataBlock dataBlock,
byte[] content) throws IOException {
- return new RecordIterator(dataBlock.readerSchema,
dataBlock.getSchemaFromHeader(), content);
+ public static RecordIterator getInstance(HoodieAvroDataBlock dataBlock,
byte[] content, boolean skipLogicalTimestampEvolution) throws IOException {
Review Comment:
is the flag `skipLogicalTimestampEvolution` introduced only for the type
miss matched case for timestamp-millis written by HoodieStreamer? IIUC, in line
194:
> this.reader = new GenericDatumReader<>(writerSchema, writerSchema);
The logical type for timestamp field in `writerSchema` is timestamp-micros,
but the value read is in form of millis, should we just fix the value read in
such case to correct form, i.e., in form of micros?
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieAvroDataBlock.java:
##########
@@ -195,8 +198,8 @@ private RecordIterator(Schema readerSchema, Schema
writerSchema, byte[] content)
}
}
- public static RecordIterator getInstance(HoodieAvroDataBlock dataBlock,
byte[] content) throws IOException {
- return new RecordIterator(dataBlock.readerSchema,
dataBlock.getSchemaFromHeader(), content);
+ public static RecordIterator getInstance(HoodieAvroDataBlock dataBlock,
byte[] content, boolean skipLogicalTimestampEvolution) throws IOException {
Review Comment:
is the flag `skipLogicalTimestampEvolution` introduced only for the type
miss matched case for timestamp-millis written by HoodieStreamer? IIUC, in line
194:
> this.reader = new GenericDatumReader<>(writerSchema, writerSchema);
The logical type for timestamp field in `writerSchema` is timestamp-micros,
but the value read is in form of millis, should we just fix the value read in
such case to correct form, i.e., in form of micros?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
