bvaradar commented on issue #625: fix(spark): fix shading for spark bundle to include all hive's libraries unshaded URL: https://github.com/apache/incubator-hudi/pull/625#issuecomment-481032184 @lyogev : Thanks a lot for reporting the issue. We needed to selectively include and shade a subset of hive dependencies for hoodie to work across hive versions with Spark (http://hudi.incubator.apache.org/quickstart.html#version-compatibility). Having said that, there is an issue causing hive dependencies to not get included in hoodie-spark bundle. Can you try this patch and see if this resolves the problem. https://gist.github.com/bvaradar/289134ac30f5103696918713e0d9bb1a As you expect, the plan is to use only one hudi bundle. You should include only hoodie-spark bundle if you are using spark-shell/spark-submit. If you are planning to use delta-streamer for ingesting data, you should only use hoodie-utilities bundle. This jar also has hive/spark dependencies included. Balaji.V
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
