[
https://issues.apache.org/jira/browse/BEAM-375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349041#comment-15349041
]
Jean-Baptiste Onofré commented on BEAM-375:
-------------------------------------------
Generally speaking, the versions should be aligned using the
dependencyManagement in the parent pom.xml.
By HadoopIO, you mean HdfsIO (in sdks/java/io) ?
> HadoopIO and runners-spark conflict with hadoop.version
> -------------------------------------------------------
>
> Key: BEAM-375
> URL: https://issues.apache.org/jira/browse/BEAM-375
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-extensions
> Reporter: Pei He
> Assignee: Pei He
>
> HadoopIO currently uses 2.7.0 and runners-spark uses 2.2.0 for hadoop-client,
> hadoop-common.
> From [~amitsela]
> "Spark can be built against different hadoop versions, but the release in
> maven central is a 2.2.0 build (latest). ''
> For HadoopIO, I don't know why 2.7.0 is picked at the beginning. I can check
> if it will work with 2.2.0.
> I am creating this issue, since I think it there is a general question.
> In principle, HadoopIO and other sdks Sources should work with any runners.
> But, when one set of runners require version A, but the other set of runners
> require version B, we will need a general solution for it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
