[
https://issues.apache.org/jira/browse/OOZIE-1770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14635603#comment-14635603
]
Robert Kanter commented on OOZIE-1770:
--------------------------------------
Oozie currently requires all actions in a workflow to run as the same user, so
that's not an issue. As far as I know, we're not planning on changing this
anytime soon either.
With all actions run as the same user, the difference between options 2 and 3
is this: In option 2 and the current design, the Oozie server monitors all
running actions (it checks on them every 10 minutes unless it receives a
callback to check sooner). When an action finishes, the Oozie server does some
stuff (resolves EL Functions, determines which action to go to, handles forks,
decision nodes, etc) and then starts the next action in the workflow. In
option 3, that work would all be done by a separate entity (an AM), with the
Oozie server simply checking the status of the workflow only (though I imagine
Coordinators and Bundles would still live in the Oozie server).
Option 3 does have some advantages over option 2, though it would obviously
require a lot more re-working of Oozie's architecture than option 2. Moving
from a transactional DB to files in HDFS may be problematic. I'm not sure we
actually would need to do that. Other than Derby, the other databases let you
have multiple connections, so each of the AMs should be able to talk to the
database; though I don't know if this is too much pressure on the DB. And if
we need locks between the AMs and the Oozie server, we already have the
ZooKeeper-based locks. If we do option 3, I think the actions should still be
executed in separate containers, to maintain the stability of the Workflow AM
and to help scale out fork nodes.
That said, as far as I know, Oozie scales pretty well. Yahoo! can comment more
on this. And we already have Oozie HA, which lets you run multiple Oozie
servers. So I'm not sure we need option 3 over option 2.
It would be good to get some input from [~rohini], [~egashira], [~puru],
[~shwethags], [~qwertymaniac], [~bowenzhangusa], and even [~kamrul] and
[~tucu00].
> Create Oozie Application Master for YARN
> ----------------------------------------
>
> Key: OOZIE-1770
> URL: https://issues.apache.org/jira/browse/OOZIE-1770
> Project: Oozie
> Issue Type: New Feature
> Reporter: Bowen Zhang
> Assignee: Bowen Zhang
> Attachments: oya-rm-screenshot.jpg, oya.patch
>
>
> After the first release of oozie on hadoop 2, it will be good if users can
> set execution engine in oozie conf, be it YARN AM or traditional MR. We can
> target this for post oozie 4.1 release.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
