Hi Everyone,

This is regarding oozie on yarn am as supported in
https://issues.apache.org/jira/browse/OOZIE-1770.

Let's discuss with an example, A Java action in oozie wf which has been
configured with AM and mapper mem settings is running in the same
container. As far as the YARN application is concerned, I can see only 1
container where both AM and launcher runs (infact actual java code runs
here). Since this is being treated as an AM container, the AM quota under
the same queue is getting changed (used, available resources etc), not the
Non-AM quota. AM mem setting is 1 gb and mapper mem setting is 3 gb in this
particular example. Since 3gb is getting accounted for under AM quota, AM
resources quota is getting filled up easily which is affecting other jobs
to start and put those in pending state. In the earlier model, 1 gb under
AM quota and 3 gb under Non-AM quota has been accounted as expected and AM
resource quota is not getting filled up unnecessarily.

Yes, this feature saves overall resources consumption by running AM in the
same container, but accountability is causing a concern. I am looking for a
way to deal with this properly from the YARN queue configuration
perspective.

Please correct me if understanding is not correct.


Thanks,
Mani

Reply via email to