alexeykudinkin commented on code in PR #7821:
URL: https://github.com/apache/hudi/pull/7821#discussion_r1094049841
##########
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/payload/ExpressionPayload.scala:
##########
@@ -455,5 +456,50 @@ object ExpressionPayload {
field.schema, field.doc, field.defaultVal, field.order))
Schema.createRecord(a.getName, a.getDoc, a.getNamespace, a.isError,
mergedFields.asJava)
}
+
+
+ /**
+ * This object differs from Hudi's generic [[SerializationUtils]] in its
ability to serialize
+ * Spark's internal structures (various [[Expression]]s)
+ *
+ * For that purpose we re-use Spark's [[KryoSerializer]] instance sharing
configuration
+ * with enclosing [[SparkEnv]]. This is necessary to make sure that this
particular instance of Kryo
+ * user for serialization of Spark's internal structures (like
[[Expression]]s) is configured
+ * appropriately (class-loading, custom serializers, etc)
+ *
+ * TODO rebase on Spark's SerializerSupport
+ */
+ private[hudi] object Serializer {
+
Review Comment:
Checked Spark 3.1, 3.2 and 3.3, working fine
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
