voonhous commented on code in PR #17535:
URL: https://github.com/apache/hudi/pull/17535#discussion_r2620052284
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/io/storage/row/HoodieRowParquetWriteSupport.java:
##########
@@ -238,20 +239,32 @@ private ValueWriter makeWriter(Schema avroSchema,
DataType dataType) {
} else if (dataType == DataTypes.LongType || dataType instanceof
DayTimeIntervalType) {
return (row, ordinal) -> recordConsumer.addLong(row.getLong(ordinal));
} else if (dataType == DataTypes.TimestampType) {
- if (logicalType == null ||
logicalType.getName().equals(LogicalTypes.timestampMicros().getName())) {
- return (row, ordinal) -> recordConsumer.addLong((long)
timestampRebaseFunction.apply(row.getLong(ordinal)));
- } else if
(logicalType.getName().equals(LogicalTypes.timestampMillis().getName())) {
- return (row, ordinal) ->
recordConsumer.addLong(DateTimeUtils.microsToMillis((long)
timestampRebaseFunction.apply(row.getLong(ordinal))));
+ if (resolvedSchema != null && resolvedSchema.getType() ==
HoodieSchemaType.TIMESTAMP) {
+ HoodieSchema.Timestamp timestampSchema = (HoodieSchema.Timestamp)
resolvedSchema;
+ if (timestampSchema.getPrecision() == TimePrecision.MICROS) {
+ return (row, ordinal) -> recordConsumer.addLong((long)
timestampRebaseFunction.apply(row.getLong(ordinal)));
+ } else {
+ return (row, ordinal) ->
recordConsumer.addLong(DateTimeUtils.microsToMillis((long)
timestampRebaseFunction.apply(row.getLong(ordinal))));
+ }
} else {
- throw new UnsupportedOperationException("Unsupported Avro logical type
for TimestampType: " + logicalType);
+ // Default to micros precision when no timestamp schema is available
+ return (row, ordinal) -> recordConsumer.addLong((long)
timestampRebaseFunction.apply(row.getLong(ordinal)));
Review Comment:
Sure, it's I believe it's unlikely that we'll step into this clause, so,
Exception might be safer.
I was trying to capture the case of `logicalType == null`. I am not sure if
HoodieSchema will resolve an Avro schema to a case where logicalType == null.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
