wombatu-kun commented on code in PR #12772:
URL: https://github.com/apache/hudi/pull/12772#discussion_r2113303215
##########
hudi-common/src/main/java/org/apache/hudi/common/engine/HoodieReaderContext.java:
##########
@@ -361,6 +361,17 @@ public Comparable convertValueToEngineType(Comparable
value) {
return value;
}
+ /**
+ * Compares two values.
+ * As [SPARK-46832] UTF8String doesn't support compareTo anymore, we need to
have engine-specific implementations.
+ * @param a first value that is compared to
+ * @param b second value
+ * @return a negative integer, zero, or a positive integer as first object
is less than, equal to, or greater than the second.
+ */
+ public int compareValues(Comparable a, Comparable b) {
+ return a.compareTo(b);
+ }
Review Comment:
I think, we can not get rid of this API because this class is in
`hudi-common` module, where we don't know neither `UTF8String` (Spark's inner
class) nor `HoodieUTF8String` (Hudi's own class potentially located in
`hudi-spark-client`), so we can not wrap `UTF8String` with `HoodieUTF8String`.
If this approach of wrapping `UTF8String` with `HoodieUTF8String` is not
applicable for all cases, I think it's better to not complicate more and not
mix it with SparkAdapter approach (already implemented and working, according
to tests). May be we can leave current implementation as is at least for the
first version of Spark4 support?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
