HyukjinKwon commented on issue #24405: [SPARK-27506][SQL] Allow deserialization of Avro data using compatible schemas URL: https://github.com/apache/spark/pull/24405#issuecomment-526981210 Yes, please. What I am worried is that, 1. I would like to be very sure if this is something worth and regular users want. From what I read the comments and codes here, basically it allows to take a wider schema whereas currently it's strictly restricted to the avro schema converted from Spark's, am I correct? Can you show it with actual code examples (better if that's runnable)? I lately updated PR template (see https://github.com/apache/spark/commit/0ea8db9fd3d882140d8fa305dd69fc94db62cf8f) . It would be great if this is followed. 2. Can we merge those additional parameters into `options: java.util.Map[String, String]`? I don't think it's a good idea to keep adding parameters to the function while we have `options`. If Avro Datasource does not have equivalent options, we can add the options for specific for both functions. 3. Out of curiosity, how do we handle such case in Avro DataSource? Are we able to merge options if there are?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
