cdmikechen edited a comment on issue #770: remove com.databricks:spark-avro to build spark avro schema by itself URL: https://github.com/apache/incubator-hudi/pull/770#issuecomment-532919914 @vinothchandar @umehrot2 I've tried changing avro version to 1.8.2 in hudi pom.xml before. Spark 2.2 or 2.3 don't use avro 1.8.2 jars in hoodie jar, It will use avro 1.7.7 first and I will still encounter the same mistake(report missing logical type class and so on). Maybe you can try using below codes in shell to test avro 1.8.2 ``` --conf spark.driver.userClassPathFirst=true --conf spark.executor.userClassPathFirst=true ``` OR do some change like hive dependence in spark-bundle ``` <relocation> <pattern>org.apache.avro</pattern> <shadedPattern>com.apache.hudi.org.apache.avro</shadedPattern> </relocation> ``` In this way, we can be compatible with sparks 2.2, 2.3 and 2.4.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
