vinothchandar commented on issue #915: [HUDI-268] Shade and relocate Avro dependency in hadoop-mr-bundle URL: https://github.com/apache/incubator-hudi/pull/915#issuecomment-534584283 @umehrot2 I am not sure if you want to spend a lot of time reproducing this. May be I can summarize the implications already. - HoodieRecordPayload abstraction has return types and parameters that take `GenericRecord` - Those payload implementations that live in hudi will be properly shaded e.g OverwriteWithLatestAvroPayload, since they are in the bundle - If an user implemented his/her own payload outside of Hudi and dropped the jar into Hive (like how we drop the mr-bundle), then for it to work avro needs to be shaded there as well. Otherwise the Hudi parent class will have `org.apache.hudi.org.apache.avro` while the user implementation will have `org.apache.avro` . So seems like we are choosing between custom payloads and logical type support. In the short term, I'd still vote for making either the shading controllable via a -D and get all the data types to work.. then look into how to untangle the shading aspects in a follow on.. By no means, I am trying to pick one over the other. just trying to first get spark 2.4 and all the data types working..
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
