yihua commented on code in PR #13711:
URL: https://github.com/apache/hudi/pull/13711#discussion_r2302215142
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/utils/SparkMetadataWriterUtils.java:
##########
@@ -178,12 +185,13 @@ public static ExpressionIndexComputationMetadata
getExpressionIndexRecordsUsingC
HoodieColumnRangeMetadata<Comparable> rangeMetadata =
HoodieColumnRangeMetadata.create(
relativeFilePath,
columnToIndex,
- minValue,
- maxValue,
+ (Comparable)
valueMetadata.standardizeJavaTypeAndPromote(minValue),
+ (Comparable)
valueMetadata.standardizeJavaTypeAndPromote(maxValue),
nullCount,
valueCount,
totalFileSize,
- totalUncompressedSize
+ totalUncompressedSize,
+ valueMetadata
Review Comment:
In this way, we can also incorporate `DecimalValueV1` and `DecimalValueV2`
implementation based on index layout, and possible introduce engine-specific
logic for backwards compatibility. So all the engine-specific implementation
details can be hided.
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/utils/SparkMetadataWriterUtils.java:
##########
@@ -178,12 +185,13 @@ public static ExpressionIndexComputationMetadata
getExpressionIndexRecordsUsingC
HoodieColumnRangeMetadata<Comparable> rangeMetadata =
HoodieColumnRangeMetadata.create(
relativeFilePath,
columnToIndex,
- minValue,
- maxValue,
+ (Comparable)
valueMetadata.standardizeJavaTypeAndPromote(minValue),
+ (Comparable)
valueMetadata.standardizeJavaTypeAndPromote(maxValue),
nullCount,
valueCount,
totalFileSize,
- totalUncompressedSize
+ totalUncompressedSize,
+ valueMetadata
Review Comment:
For better maintainability and readability, should we create a new POJO
`Value` for min and max values to be used instead of `Comparable`? If that
breaks backwards compatibility around write stat, we can create
`HoodieColumnRangeMetadataV2` for new logic. Then, the `Value` class can
handle all type promotion and value cast within each implementation, e.g.,
`LongValue`, `DecimalValue`, etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
