[
https://issues.apache.org/jira/browse/OOZIE-1770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15262965#comment-15262965
]
Robert Kanter commented on OOZIE-1770:
--------------------------------------
We'd use unmanaged AMs (basically, Yarn wouldn't take care of
running/restarting/etc the AMs) so that we can actually run them inside the
Oozie Server. In that setup, the only way for an AM to go down or be restarted
is if the Oozie Server was also restarted. In any case, Containers can
continue executing even if the AM isn't available, so that's not a problem. If
the AM is restarted, or if the RM restarts/fails over, all the AM needs to have
when things come back up is the AMRM token, so that the RM knows it's a legit
Container. We can store those in the Oozie database somewhere, given that the
AMs are running inside the Oozie Server. In summary, using the pool of
unmanaged AMs, we don't have to worry about any restart/fail over/etc behavior
because it already works. The only thing we'd have to worry about is the
Container restarting, which we already have a solution for because the MR
Launcher Job has a similar issue (we currently tag child jobs and then kill any
on Launcher startup); we can do the same here.
As for logs, the logs for all of the containers associated with an AM are
eventually aggregated together in HDFS (if log aggregation is enabled). In any
case, they are available via the Yarn Web UI and CLI at a per-container level,
so even though we'll have Oozie AMs with logs from multiple Actions that are
not necessarily from the same workflow, we should still be able to have a way
of looking up just that Action's logs. And because each AM is owned by a
specific user, there shouldn't be any security issues with that.
> Create Oozie Application Master for YARN
> ----------------------------------------
>
> Key: OOZIE-1770
> URL: https://issues.apache.org/jira/browse/OOZIE-1770
> Project: Oozie
> Issue Type: New Feature
> Reporter: Bowen Zhang
> Assignee: Bowen Zhang
> Attachments: OozieYarnAM.pdf, Prelim OYA Scoping Doc 001.pdf,
> oya-rm-screenshot.jpg, oya.patch
>
>
> After the first release of oozie on hadoop 2, it will be good if users can
> set execution engine in oozie conf, be it YARN AM or traditional MR. We can
> target this for post oozie 4.1 release.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
