[
https://issues.apache.org/jira/browse/HUDI-259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16952071#comment-16952071
]
Vinoth Chandar commented on HUDI-259:
-------------------------------------
Hi [~Pratyaksh] please use master branch for these changes.. Our first apache
release is imminent and there are tons of changes to pom since 0.4.7.
Can we just keep the scope of this ticket to just Hadoop version? By that I
mean, we may not actually bump the hadoop version on the pom, but
- do a build with `*-Dhadoop.version=3.1.0*`, fix compilation errors and make
code changes necessary (ultimately build should also pass with hadoop 2.x
version currently in pom)
- Take the build above and run it on the integration test environment and
ensure it passes.
Most of the cloud vendors still are on hadoop 2.x in a major way. we cannot
drop support for that.
On hive and spark
- Hive 3.x is a major issue since it has backwards incompatible changes (phew!)
There is a separate issue tracking that
- Spark 2.4 is what we are planning to move to. udit is already driving that.
Please let me know if this makes sense
> Hadoop 3 support for Hudi writing
> ---------------------------------
>
> Key: HUDI-259
> URL: https://issues.apache.org/jira/browse/HUDI-259
> Project: Apache Hudi (incubating)
> Issue Type: Improvement
> Components: Usability
> Reporter: Vinoth Chandar
> Priority: Major
>
> Sample issues
>
> [https://github.com/apache/incubator-hudi/issues/735]
> [https://github.com/apache/incubator-hudi/issues/877#issuecomment-528433568]
> [https://github.com/apache/incubator-hudi/issues/898]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
