eshu commented on issue #6692: URL: https://github.com/apache/hudi/issues/6692#issuecomment-1273877741
@xushiyan We create our own fat jar that including Hudi Spark 3.1 bundle, hudi-aws-bundle and some other libraries, such as connectors to Kafka, BigQuery, ES. Also there is a set of libraries that provided by Glue environment. But let me remind the situation again. 1. The error message is `org.apache.hudi.org.apache.avro.Schema$RecordSchema cannot be cast to org.apache.avro.Schema` - this is the answer what library is shading Avro. Do you see the prefix `org.apache.hudi` in the shaded class name? 2. The library works with Hudi 0.11.1. Unit tests and the staging and production environment are working without such issues. 3. Unit tests shows that Hudi 0.12.0 works on freshly created data. 4. The error happens in case when I try to use it on staging datasets that created with Hudi 0.11.1. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
