kingeasternsun commented on a change in pull request #3987:
URL: https://github.com/apache/iceberg/pull/3987#discussion_r810451665
##########
File path:
flink/v1.13/flink/src/main/java/org/apache/iceberg/flink/data/FlinkParquetReaders.java
##########
@@ -321,6 +327,29 @@ public DecimalData read(DecimalData ignored) {
}
}
+ private static class TimestampInt96Reader extends
ParquetValueReaders.UnboxedReader<Long> {
+ private static final long UNIX_EPOCH_JULIAN = 2_440_588L;
+
+ TimestampInt96Reader(ColumnDescriptor desc) {
+ super(desc);
+ }
+
+ @Override
+ public Long read(Long ignored) {
+ return readLong();
+ }
+
+ @Override
+ public long readLong() {
+ final ByteBuffer byteBuffer =
column.nextBinary().toByteBuffer().order(ByteOrder.LITTLE_ENDIAN);
+ final long timeOfDayNanos = byteBuffer.getLong();
+ final int julianDay = byteBuffer.getInt();
+
+ return TimeUnit.DAYS.toMicros(julianDay - UNIX_EPOCH_JULIAN) +
+ TimeUnit.NANOSECONDS.toMicros(timeOfDayNanos);
Review comment:
> @kingeasternsun, there's no need to modify extra lines to fix
whitespace issues. That just causes more code churn and commit conflicts. We
just try to make sure formatting is correct in the lines that are already
changed.
>
> For Flink versions, we like to update one version at a time and port
changes across. That keeps reviews simpler and makes it so you don't have to
make the same changes in 3 separate projects when there is a review comment.
thanks， So I Just need to modify the flink version V1.14 and leave flink
v1.13 unchanged?
##########
File path:
flink/v1.13/flink/src/test/java/org/apache/iceberg/flink/data/TestFlinkParquetReader.java
##########
@@ -72,4 +95,86 @@ protected void writeAndValidate(Schema schema) throws
IOException {
writeAndValidate(RandomGenericData.generateDictionaryEncodableRecords(schema,
NUM_RECORDS, 21124), schema);
writeAndValidate(RandomGenericData.generateFallbackRecords(schema,
NUM_RECORDS, 21124, NUM_RECORDS / 20), schema);
}
+
+ protected List<RowData> rowDatasFromFile(InputFile inputFile, Schema schema)
throws IOException {
Review comment:
> Data is already plural, so there's no need to add the "s".
Thans , I'll fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
