[
https://issues.apache.org/jira/browse/BEAM-8569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16980532#comment-16980532
]
Tomo Suzuki commented on BEAM-8569:
-----------------------------------
I was trying to upgrade bigtable-client-core to the latest 1.12.1 version
([PR|https://github.com/apache/beam/pull/10144]) but it got conflict with
Hadoop version 2:
Conflict:
- Bigtable-client-core 1.12.1 needs Guava version >= 23.1 for
{{ImmutableList.builderWithExpectedSize}} method
([note|https://github.com/GoogleCloudPlatform/cloud-opensource-java/issues/1028#issuecomment-557633483])
- Hadoop-client 2.7.3 needs Guava version <= 20 for {{Objects.toStringHelper}}
method
([note|https://github.com/GoogleCloudPlatform/cloud-opensource-java/issues/1028#issuecomment-557697439])
I'm looking forward to this Hadoop upgrade!
> Support Hadoop 3 on Beam
> ------------------------
>
> Key: BEAM-8569
> URL: https://issues.apache.org/jira/browse/BEAM-8569
> Project: Beam
> Issue Type: Improvement
> Components: io-java-hadoop-file-system, io-java-hadoop-format,
> runner-spark
> Reporter: Ismaël Mejía
> Priority: Minor
>
> It seems that Hadoop 3 in production is finally happening. CDH supports it in
> their latest version and Spark 3 will include support for Hadoop 3 too.
> This is an uber ticket to cover the required changes to the codebase to
> ensure compliance with Hadoop 3.x
> Hadoop dependencies in Beam are mostly provided and APIs are until some point
> compatible, but we may require changes in the CI to test that new changes
> work both in Hadoop 2 and Hadoop 3 until we decide to remove support for
> Hadoop 3.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
