wombatu-kun commented on code in PR #19583:
URL: https://github.com/apache/hudi/pull/19583#discussion_r3765543027
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/cdc/InternalRowToJsonStringConverter.scala:
##########
@@ -90,6 +92,11 @@ class InternalRowToJsonStringConverter(schema: StructType) {
structMap.toMap
case _ => value // fallback
}
+ case dt if SparkAdapterSupport.sparkAdapter.isVariantType(dt) =>
+ // VariantVal.toString renders the variant as JSON; embed it as a
real JSON node so
+ // the image carries the variant's structure. Falling through to the
default would
+ // serialize the VariantVal bean, i.e. its raw value/metadata bytes
as base64.
+ mapper.readTree(value.toString)
Review Comment:
Correcting my own premise here: spark-variant guards the double and float
branches with isFinite and sends the non-finite arm through appendQuoted, so
4.0.2 and 4.1.1 both render a quoted "NaN" and readTree never sees a bare token
- the comments here and in the test overstate the trigger. Worth keeping the
fallback but rewording it, and noting that value.toString sits outside the try
while VariantVal.toString can itself throw MALFORMED_VARIANT, which
JsonProcessingException does not cover.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
