xkrogen commented on pull request #31490: URL: https://github.com/apache/spark/pull/31490#issuecomment-867228159
Just pushed up a new version which is pretty much a complete re-implementation on top of #32969. This still consolidates more of the field matching behavior between `AvroSerializer` and `AvroDeserializer`, as in the original PR, but it leverages the nature of `AvroSchemaHelper` to serve the results to the caller in what I think is a more clean way. Instead of passing back a `Seq[(Option, Option)]` which the caller then pattern-matches on to handle the various cases, the `AvroSchemaHelper` performs all of the matching up front and then provides three methods to access all of the fields matching various conditions (present in both schemas, only Avro, or only Catalyst). I think this makes the logic in the callers more simple and clear. I plan to also update benchmarks tomorrow to ensure that there is no regression on top of #32969. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
