xuanyuanking opened a new pull request #22878: [SPARK-25789][SQL] Support for Dataset of Avro URL: https://github.com/apache/spark/pull/22878 ## What changes were proposed in this pull request? Please credit to @bdrillard cause this mainly based on his previous work. This PR add support for Dataset of Avro records in an API that would allow the user to provide a class to an Encoder for Avro, analogous to the Bean encoder. - Add `ObjectCast` and `InitializeAvroObject`(analogous to `InitializeJavaBean`) expression. - Add an AvroEncoder for Datasets of Avro records to Spark. - Add type-inference utilities `AvroTypeInference` for Avro object and SQL DataType (analogous to `JavaTypeInference`). ## How was this patch tested? Add UT in AvroSuite.scala and manual test by modified SQLExample with external avro package.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
