vinothchandar commented on a change in pull request #1005:
[HUDI-91][HUDI-12]Migrate to spark 2.4.4, migrate to spark-avro library instead
of databricks-avro, add support for Decimal/Date types
URL: https://github.com/apache/incubator-hudi/pull/1005#discussion_r364995790
##########
File path: packaging/hudi-spark-bundle/pom.xml
##########
@@ -94,8 +94,6 @@
<include>org.apache.hive:hive-service-rpc</include>
<include>org.apache.hive:hive-metastore</include>
<include>org.apache.hive:hive-jdbc</include>
-
- <include>com.databricks:spark-avro_2.11</include>
Review comment:
I assume the tests will pass.. but I realize what you are saying.. the user
could be running on a higher spark version say and we would be bundling 2.4.4 .
Lets just open a JIRA to tackle this usability issue and keep it as -is now..
We can document the need for `--packages ...` when using spark-submit or
spark-shell clearly for now.. and move on.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
