Danil Serdyuchenko created SAMZA-983:
----------------------------------------
Summary: A separate setting for AM vcpu count
Key: SAMZA-983
URL: https://issues.apache.org/jira/browse/SAMZA-983
Project: Samza
Issue Type: Improvement
Reporter: Danil Serdyuchenko
We hit an issue mentioned in: SAMZA-783 after we enabled hard cpu isolation on
our yarn cluster. From our analysis we could see a few compounding reasons for
the outage:
# A samza job with a large number of input topics
# A hard cpu isolation restricting AM container
# A large number of containers hitting the AM
>From some further investigation we noticed that cpu count for AM is hard
>coded:
>https://git-wip-us.apache.org/repos/asf?p=samza.git;a=blob;f=samza-yarn/src/main/scala/org/apache/samza/job/yarn/ClientHelper.scala;h=0998c43713bbde6b3edc7a76d1bc78403872b792#l95
It would be nice to have a separate configuration for this, so we can tune how
much CPU the AM needs independently, similar to `yarn.am.container.
memory.mb`. We want to move to hard CPU isolation on our yarn cluster, but this
would prevent us.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
