[jira] [Commented] (YARN-8009) YARN limit number of simultaneously running containers in the application level
[ https://issues.apache.org/jira/browse/YARN-8009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16391770#comment-16391770 ] Sachin Jose commented on YARN-8009: --- I've should've done some more details check in mapred-defailt.xml and tez-default.xml before raising this request. Thanks a lot [~eepayne] and [~miklos.szeg...@cloudera.com] > YARN limit number of simultaneously running containers in the application > level > --- > > Key: YARN-8009 > URL: https://issues.apache.org/jira/browse/YARN-8009 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn >Affects Versions: 3.0.0 >Reporter: Sachin Jose >Priority: Minor > Labels: features > > It would be really useful if the user can specify maximum containers can be > running simultaneously in the application level. Most of the long running > YARN application can be benefited out of it. At this moment, the only > available option to restrict resource over usage of long running is in the > YARN resource manager queue level. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8009) YARN limit number of simultaneously running containers in the application level
[
https://issues.apache.org/jira/browse/YARN-8009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16391504#comment-16391504
]
Eric Payne commented on YARN-8009:
--
[~sachinjose2...@gmail.com], this property is a per-framework setting, as
[~miklos.szeg...@cloudera.com] pointed out above for the distributed shell.
For MapReduce, the settings are {{mapreduce.job.running.map.limit}} and
{{mapreduce.job.running.reduce.limit}}.
For tez, it's {{tez.am.vertex.max-task-concurrency}}
> YARN limit number of simultaneously running containers in the application
> level
> ---
>
> Key: YARN-8009
> URL: https://issues.apache.org/jira/browse/YARN-8009
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: yarn
>Affects Versions: 3.0.0
>Reporter: Sachin Jose
>Priority: Minor
> Labels: features
>
> It would be really useful if the user can specify maximum containers can be
> running simultaneously in the application level. Most of the long running
> YARN application can be benefited out of it. At this moment, the only
> available option to restrict resource over usage of long running is in the
> YARN resource manager queue level.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8009) YARN limit number of simultaneously running containers in the application level
[ https://issues.apache.org/jira/browse/YARN-8009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16390116#comment-16390116 ] Miklos Szegedi commented on YARN-8009: -- Thank you for raising this [~sachinjose2...@gmail.com]. Normally the Application Master has the ability to specify the amount of containers. It can then provide an option to the user. See the the distributed shell example for details: https://github.com/apache/hadoop/blob/037d7834833df2d1e60f5015b60d42550b1ddce6/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java#L459 > YARN limit number of simultaneously running containers in the application > level > --- > > Key: YARN-8009 > URL: https://issues.apache.org/jira/browse/YARN-8009 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn >Affects Versions: 3.0.0 >Reporter: Sachin Jose >Priority: Minor > Labels: features > > It would be really useful if the user can specify maximum containers can be > running simultaneously in the application level. Most of the long running > YARN application can be benefited out of it. At this moment, the only > available option to restrict resource over usage of long running is in the > YARN resource manager queue level. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
