bvaradar commented on issue #1005: [HUDI-91][HUDI-12]Migrate to spark 2.4.4, migrate to spark-avro library instead of databricks-avro, add support for Decimal/Date types URL: https://github.com/apache/incubator-hudi/pull/1005#issuecomment-554264414 @umehrot2 : Regarding your comment regarding moving to spark.hive version for hudi-spark-bundle and hudi-utilities-bundle, I am ok as long as hive sync works with hiveserver/HMS 2.x Regarding realtime queries, if we shade avro in hudi-hadoop-mr-bundle, we would need to follow same shading policy in user's jar when they use custom HoodieRecordPayload. AFAIK, there is no good solution here to resolve this dependency hell. If we go this approach, we would need to document this caveat and make it easy for users to perform this shading (with boiler-plate pom) @vinothchandar : Any suggestions ? @modi95 (cc @n3nash) : Please note that there will be similar issues with realtime queries and spark-2.4.4 when we eventually migrate RT tables to Spark 2.4.4
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
