[
https://issues.apache.org/jira/browse/YARN-2889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16271444#comment-16271444
]
Arun Suresh commented on YARN-2889:
-----------------------------------
bq. I don't think you can expect user to respect this limit, at least not for
the first time. On the other hand, number of container requests is vary against
the size of the job, how you gonna define this limit? Therefore, it doesn't
seem to be practical to me.
Apologize - but am not sure I follow the argument.
So the limit currently is something we enforce at a per-AM-per-allocate call
level. If an AM asks for more O containers, the requests are queued in the RM
(in the application's {{OpportunisticContainerContext}}) and will be satisfied
in subsequent allocate calls.
bq. If NM queue is full, can we avoid assigning any O containers to that node?
That means when preparing top K least loaded nodes, we need to exclude nodes
whose queue is already full.
Agreed - that is a good idea. Something we have been planning on doing
actually. Feel free to raise a JIRA - will help review. To be honest, instead
of a total sort, a partial sorting of nodes which includes only nodes whose
queue length < x, where x is a small value to give a higher probably for the O
container to run - might be useful.
bq. Each queue size is limited, so I don't see why lots of O containers would
flood the system.
Hmmm.. so it is still possible for queues to be filled with O containers from a
single AM - thereby denying other AMs for getting O containers. YARN-7258
handles this somewhat, by spreading the O containers requested by an AM across
multiple allocate calls - so that other AMs get a chance.
bq. You can't say an AM is malicious if it requests only opportunistic
containers (too many). Unless this was the design, then you need to setup
correct user expectation with some document, and explain what is the correct
user case.
Understand - which is why we are interested in auto-limiting the number of O
containers allocated to an AM. Another option is to say - the AM does not
explicitly ask for O containers and the RM will allocate an O container based
on queue/cluster capacity or number of nodes with 0 queue length etc.
> Limit in the number of opportunistic container requests per AM
> --------------------------------------------------------------
>
> Key: YARN-2889
> URL: https://issues.apache.org/jira/browse/YARN-2889
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
> Reporter: Konstantinos Karanasos
> Assignee: Arun Suresh
>
> We introduce a way to limit the number of queueable requests that each AM can
> submit to the LocalRM.
> This way we can restrict the number of queueable containers handed out by the
> system, as well as throttle down misbehaving AMs (asking for too many
> queueable containers).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
