hwani3142 commented on code in PR #11548:
URL: https://github.com/apache/hudi/pull/11548#discussion_r1663798465
##########
hudi-common/src/main/java/org/apache/hudi/common/util/JsonUtils.java:
##########
@@ -44,6 +44,8 @@ public class JsonUtils {
MAPPER.setVisibility(PropertyAccessor.IS_GETTER,
JsonAutoDetect.Visibility.NONE);
MAPPER.setVisibility(PropertyAccessor.SETTER,
JsonAutoDetect.Visibility.NONE);
MAPPER.setVisibility(PropertyAccessor.CREATOR,
JsonAutoDetect.Visibility.NONE);
+
+ registerModules();
Review Comment:
@yihua
It was because of the use of JsonUtils in
[MetadataConversionUtils.convertCommitMetadata()](https://github.com/apache/hudi/blob/master/hudi-common/src/main/java/org/apache/hudi/common/table/timeline/MetadataConversionUtils.java#L257),
not on the adapter side.
I think JsonUtils needs it as a defense logic because there is no
application according to the spark version.
##########
hudi-common/src/main/java/org/apache/hudi/common/util/JsonUtils.java:
##########
@@ -60,7 +62,7 @@ public static String toString(Object value) {
}
public static void registerModules() {
- // NOTE: Registering [[JavaTimeModule]] is required for Jackson >= 2.11
(Spark >= 3.3)
+ // NOTE: Registering [[JavaTimeModule]] is required for Jackson >= 2.11
(Spark >= 3.2)
Review Comment:
@danny0405
The part that needed to be modified was just [adding the spark 3.2 jackson
version](https://github.com/apache/hudi/pull/11548/commits/0528626988810f9ab31105bb739c8e97d676a755),
but I corrected the related part by correcting the failed test.
This part appears to only cover spark version 3.3 or higher, so it was
changed to include version 3.2.
In fact, BaseSpark3Adapter use this, and only 3.2Adapter has implemented it.
If version 3.2 is scheduled to be deprecated, it may be something that can
be removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
