[
https://issues.apache.org/jira/browse/YARN-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16576655#comment-16576655
]
Shane Kumpf edited comment on YARN-8623 at 8/10/18 6:07 PM:
------------------------------------------------------------
[~ccondit-target] - thanks for looking into this. I see what you mean about the
challenge with using that image. I think you are correct that the existing
apache/hadoop-runner image serves a different type of use case than we need
here.
IMO, our target should be an image capable of running MapReduce pi, as that's
the example we provide in the docs. If it also works for the Spark shell
example we provide in our docs, with the appropriate spark install/config, that
would be great, but I don't think it's a requirement to start.
Thinking about what we need to meet that goal, I think a majority of the users
we would be targeting with this guide will have all of Hadoop installed on the
nodes where these containers are running. Instead of trying to package the
latest version of Apache Hadoop as an image, I think our example would be
easier to maintain if we guide the user towards bind mounting the Hadoop
binaries and configuration from the NodeManager hosts. If we take that
approach, I believe the image should only need to include a JDK and set up
JAVA_HOME. We might even be able to use an existing openjdk image.
Assuming we can't leverage an existing image, one question I'm unsure about is
the process of creating an "official" image under the apache docker hub
namespace. [~elek] - can you share any insights around this process?
was (Author: [email protected]):
[~ccondit-target] - thanks for looking into this. I see what you mean about the
challenge with using that image. I think you are correct that the existing
apache/hadoop-runner image serves a different type of use case than we need
here.
IMO, our target should be an image capable of running MapReduce pi, as that's
the example we provide in the docs. If it also works for Spark shell example we
provide in our docs, with the appropriate spark install/config, that would be
great, but I don't think it's a requirement to start.
!/jira/images/icons/emoticons/smile.png!
Thinking about what we need to meet that goal, I think a majority of the users
we would be targeting with this guide will have all of Hadoop installed on the
nodes where these containers are running. Instead of trying to package the
latest version of Apache Hadoop as an image, I think our example would be
easier to maintain if we guide the user towards bind mounting the Hadoop
binaries and configuration from the NodeManager hosts. If we take that
approach, I believe the image should only need to include a JDK and set up
JAVA_HOME. We might even be able to use an existing openjdk image.
Assuming we can't leverage an existing image, one question I'm unsure about is
the process of creating an "official" image under the apache docker hub
namespace. [~elek] - can you share any insights around this process?
> Update Docker examples to use image which exists
> ------------------------------------------------
>
> Key: YARN-8623
> URL: https://issues.apache.org/jira/browse/YARN-8623
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Craig Condit
> Priority: Minor
> Labels: Docker
>
> The example Docker image given in the documentation
> (images/hadoop-docker:latest) does not exist. We could change
> images/hadoop-docker:latest to apache/hadoop-runner:latest, which does exist.
> We'd need to do a quick sanity test to see if the image works with YARN.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
