[
https://issues.apache.org/jira/browse/YARN-1151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13760146#comment-13760146
]
Steve Loughran commented on YARN-1151:
--------------------------------------
it gets more complex once you add in failure handling: how does the NM react to
the aux service process failing? How does the NM shut it down? where do the
logs go? Who does it run as?
We effectively do have a system for doing this, it is called YARN. What sounds
needed here is a way to tell *something* that an NM has started and then give
it the option of creating and deploying a container on it.
That something should, obviously, be a YARN app itself, since they are set up
to build up command lines, copy in JARs, handle failures, etc.
What we don't have is
# anything that starts a specific long-lived YARN AM service on cluster
startup.
# a way for an AM to list all the hosts and demand a container on every one,
irrespective of what is already there. (actually you could probably do it by
asking for 0 RAM and vhosts, but the min resource config options are designed
to stop users doing this). YARN-1160 covers that problem
> Ability to configure auxiliary services from HDFS-based JAR files
> -----------------------------------------------------------------
>
> Key: YARN-1151
> URL: https://issues.apache.org/jira/browse/YARN-1151
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
> Affects Versions: 2.1.0-beta
> Reporter: john lilley
> Priority: Minor
> Labels: auxiliary-service, yarn
>
> I would like to install an auxiliary service in Hadoop YARN without actually
> installing files/services on every node in the system. Discussions on the
> user@ list indicate that this is not easily done. The reason we want an
> auxiliary service is that our application has some persistent-data components
> that are not appropriate for HDFS. In fact, they are somewhat analogous to
> the mapper output of MapReduce's shuffle, which is what led me to
> auxiliary-services in the first place. It would be much easier if we could
> just place our service's JARs in HDFS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
