[
https://issues.apache.org/jira/browse/YARN-1564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated YARN-1564:
---------------------------------
Attachment: YARN-1564-001.patch
Here is a package containing the set of workflow yarn services we've been
using, and tests. They are described in the javadocs and package-info.java; the
key ones are:
# {{WorkflowCompositeService}}: stops when a child stops; propagates up
failures.
# {{WorkflowSequenceService}}: starts its children in sequence, stopping when
all have been run, or after the first failure.
# {{WorkflowForkedProcessService}} fork a potentially long-lived process;
matching the lifecycle of the YARN service to the process. This logs output,
caching a few recent lines, rather than buffers it -for that long-lived
execution.
# {{WorkflowCallbackService}} executes any {{Callable}} (in java 8, a lambda)
when started, stopping the service when it finishes, & propagating exceptions.
Lets you run anything in a workflow, or just add new YARN services without
subclassing.
I've been using these for about 9-12 months, building my YARN apps from them,
so I consider them fairly mature in terms of functionality.
Note also that the {{ServiceParent extends Service }} interface offering public
access to child services *is* effectively implemented by the YARN
{{CompositeService}} -all we need to do is explicitly implement the interface
and turn the two methods public.
{code}
void addService(Service service);
List<Service> getServices();
{code}
> add some basic workflow YARN services
> -------------------------------------
>
> Key: YARN-1564
> URL: https://issues.apache.org/jira/browse/YARN-1564
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: api
> Affects Versions: 2.4.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Minor
> Attachments: YARN-1564-001.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> I've been using some alternative composite services to help build workflows
> of process execution in a YARN AM.
> They and their tests could be moved in YARN for the use by others -this would
> make it easier to build aggregate services in an AM
--
This message was sent by Atlassian JIRA
(v6.2#6252)
