[
https://issues.apache.org/jira/browse/SPARK-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14005578#comment-14005578
]
Tathagata Das edited comment on SPARK-1898 at 5/22/14 3:34 AM:
---------------------------------------------------------------
To get an idea of whether this affects currently available YARN distribution or
not, I ran Scala Spark Pi example and Spark shell against YARN 2.4 (which I
believe is one of the latest) using a HDP2.1 VM. Seemed to work fine. So this
probably works with existing released versions of YARN.
This change is a little scary, as its hard to know the implications of this
change with all existing versions of YARN. This change should be accepted after
a little bit of testing with various versions of YARN, so that we are sure that
we dont break compatibility. I am not sure it is a good idea to cut a new
release because of this change in Spark 1.0, since Spark 1.0 probably works
fine with currently released versions of YARN.
was (Author: tdas):
To get an idea of whether this affects currently available YARN distribution or
not, I ran Scala Spark Pi example and Spark shell against YARN 2.4 (which I
believe is one of the latest) using a HDP2.1 VM. Seemed to work fine. So this
probably works with existing released versions of YARN.
This change is a little scary, as its hard to know the implications of this
change with all existing versions of YARN. This change should be accepted after
a little bit of testing with various versions of YARN, so that we are sure that
we dont break compatibility. I am not sure it is a good idea to accept this
change in Spark 1.0, since Spark 1.0 probably works fine with currently
released versions of YARN.
> In deploy.yarn.Client, use YarnClient rather than YarnClientImpl
> ----------------------------------------------------------------
>
> Key: SPARK-1898
> URL: https://issues.apache.org/jira/browse/SPARK-1898
> Project: Spark
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Colin Patrick McCabe
>
> In {{deploy.yarn.Client}}, we should use {{YarnClient}} rather than
> {{YarnClientImpl}}. The latter is annotated as {{Private}} and {{Unstable}}
> in Hadoop, which means it could change in an incompatible way at any time.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
