[
https://issues.apache.org/jira/browse/BEAM-375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15968410#comment-15968410
]
liyuntian commented on BEAM-375:
--------------------------------
yes,I get state by "pipelineResult.getState().isTerminal()" ,But ,IT IS running
all the timeļ¼and the spark executors Disk Used is 0.0B, but when I change
hadoop version to 2.7.0,I can get the result correctly.so I want to know I
must use Hadoop2.7.0 in beam0.5?
> HadoopIO and runners-spark conflict with hadoop.version
> -------------------------------------------------------
>
> Key: BEAM-375
> URL: https://issues.apache.org/jira/browse/BEAM-375
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-extensions
> Reporter: Pei He
> Assignee: Pei He
>
> HadoopIO currently uses 2.7.0 and runners-spark uses 2.2.0 for hadoop-client,
> hadoop-common.
> From [~amitsela]
> "Spark can be built against different hadoop versions, but the release in
> maven central is a 2.2.0 build (latest). ''
> For HadoopIO, I don't know why 2.7.0 is picked at the beginning. I can check
> if it will work with 2.2.0.
> I am creating this issue, since I think it there is a general question.
> In principle, HadoopIO and other sdks Sources should work with any runners.
> But, when one set of runners require version A, but the other set of runners
> require version B, we will need a general solution for it.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
