alexeykudinkin commented on a change in pull request #5137:
URL: https://github.com/apache/hudi/pull/5137#discussion_r835784601
##########
File path:
hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionUtils.scala
##########
@@ -62,10 +62,10 @@ object AvroConversionUtils {
* @param rootCatalystType Catalyst [[StructType]] to be transformed into
* @return converter accepting Avro payload and transforming it into a
Catalyst one (in the form of [[InternalRow]])
*/
- def createAvroToInternalRowConverter(rootAvroType: Schema, rootCatalystType:
StructType): GenericRecord => Option[InternalRow] =
- record => sparkAdapter.createAvroDeserializer(rootAvroType,
rootCatalystType)
Review comment:
@xiarixiaoyao that's a very sneaky issue
Please leave a note in a comment explaining what was the issue and how
you've addressed it.
However, i'd suggest to just pull out SparkAdapter from the closure keeping
the API of this method intact: we don't want to push casting (to
InternalRow/GenericRecord) onto the users
```
def createAvroToInternalRowConverter(rootAvroType: Schema, rootCatalystType:
StructType): GenericRecord => Option[InternalRow] = {
val deserilizer = sparkAdapter.createAvroDeserializer(rootAvroType,
rootCatalystType)
record =>
deserializer.deserialize(record).map(_.asInstanceOf[InternalRow])
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
