[jira] [Comment Edited] (YARN-10212) Create separate configuration for max global AM attempts
[
https://issues.apache.org/jira/browse/YARN-10212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17078598#comment-17078598
]
Jonathan Hung edited comment on YARN-10212 at 4/8/20, 6:56 PM:
---
Thanks [~BilwaST], in general looks good, some minor style issues:
* In TestResourceManager.java, can we change {{fail("Exception is expected
because the global max attempts" +}} to {{fail("Exception is expected because
AM max attempts" +}}
* In YarnConfiguration.java: {{* an application,if unset by user.}} -> can we
add a space after the comma
* In yarn-default.xml, for the comment for
yarn.resourcemanager.am.max-attempts:
*
{noformat}
The maximum number of application attempts. Each application
master can specify
its individual maximum number of application attempts via the API, but the
individual number cannot be more than the global upper bound.This value is
being set
only if global max attempts is unset. The default number is set to 2, to
{noformat}
can we change this to
*
{noformat}
The default maximum number of application attempts, if unset by
the user. Each application master can specify its individual maximum number of
application
attempts via the API, but the individual number cannot be more than the global
upper bound in
yarn.resourcemanager.am.global.max-attempts. The default number is set to 2,
to{noformat}
was (Author: jhung):
Thanks [~BilwaST], in general looks good, some minor style issues:
* In TestResourceManager.java, can we change {{fail("Exception is expected
because the global max attempts" +}} to {{fail("Exception is expected because
AM max attempts" +}}
* In YarnConfiguration.java: {{* an application,if unset by user.}} -> can we
add a space after the comma
* In yarn-default.xml, for the comment for
yarn.resourcemanager.am.max-attempts:
*
{noformat}
The maximum number of application attempts. Each application
master can specify
its individual maximum number of application attempts via the API, but the
individual number cannot be more than the global upper bound.This value is
being set
only if global max attempts is unset. The default number is set to 2, to
{noformat}
can we change this to
*
{noformat}
The default maximum number of application attempts, if unset by
the user. Each application master can specify its individual maximum number of
application attempts via the API, but the individual number cannot be more than
the global upper bound in yarn.resourcemanager.am.global.max-attempts. The
default number is set to 2, to{noformat}
> Create separate configuration for max global AM attempts
>
>
> Key: YARN-10212
> URL: https://issues.apache.org/jira/browse/YARN-10212
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Jonathan Hung
>Assignee: Bilwa S T
>Priority: Major
> Attachments: YARN-10212.001.patch, YARN-10212.002.patch,
> YARN-10212.003.patch
>
>
> Right now user's default max AM attempts is set to the same as global max AM
> attempts:
> {noformat}
> int globalMaxAppAttempts = conf.getInt(YarnConfiguration.RM_AM_MAX_ATTEMPTS,
> YarnConfiguration.DEFAULT_RM_AM_MAX_ATTEMPTS); {noformat}
> If we want to increase global max AM attempts, it will also increase the
> default. So we should create a separate global AM max attempts config to
> separate the two.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10212) Create separate configuration for max global AM attempts
[
https://issues.apache.org/jira/browse/YARN-10212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17074847#comment-17074847
]
Jonathan Hung edited comment on YARN-10212 at 4/3/20, 7:35 PM:
---
[~BilwaST] user's default max AM attempts is how many AM attempts they get if
they don't set individualMaxAppAttempts on client side.
I am proposing adding a new configuration like GLOBAL_RM_AM_MAX_ATTEMPTS, and
changing the code snippet above to something like:
{noformat}
int globalMaxAppAttempts =
conf.getInt(YarnConfiguration.GLOBAL_RM_AM_MAX_ATTEMPTS,
conf.getInt(YarnConfiguration.RM_AM_MAX_ATTEMPTS,
YarnConfiguration.DEFAULT_RM_AM_MAX_ATTEMPTS));{noformat}
If GLOBAL_RM_AM_MAX_ATTEMPTS is unset, it will fall back to current behavior.
But if GLOBAL_RM_AM_MAX_ATTEMPTS is set to something higher than
RM_AM_MAX_ATTEMPTS, then if user does not set individualMaxAppAttempts on
client side, their app's number of attempts will still use RM_AM_MAX_ATTEMPTS,
but user can set individualMaxAppAttempts to RM_AM_MAX_ATTEMPTS <
individualMaxAppAttempts <= GLOBAL_RM_AM_MAX_ATTEMPTS if they like.
was (Author: jhung):
[~BilwaST] user's default max AM attempts is how many AM attempts they get if
they don't set individualMaxAppAttempts on client side.
I am proposing adding a new configuration like GLOBAL_RM_AM_MAX_ATTEMPTS, and
changing the code snippet above to something like:
{noformat}
int globalMaxAppAttempts =
conf.getInt(YarnConfiguration.GLOBAL_RM_AM_MAX_ATTEMPTS,
conf.getInt(YarnConfiguration.RM_AM_MAX_ATTEMPTS,
YarnConfiguration.DEFAULT_RM_AM_MAX_ATTEMPTS));{noformat}
If GLOBAL_RM_AM_MAX_ATTEMPTS is unset, it will fall back to current behavior.
But if GLOBAL_RM_AM_MAX_ATTEMPTS is set to something higher (e.g. 4), then if
user does not set individualMaxAppAttempts on client side, their app's number
of attempts will still use RM_AM_MAX_ATTEMPTS, but user can set
individualMaxAppAttempts to RM_AM_MAX_ATTEMPTS < individualMaxAppAttempts <=
GLOBAL_RM_AM_MAX_ATTEMPTS if they like.
> Create separate configuration for max global AM attempts
>
>
> Key: YARN-10212
> URL: https://issues.apache.org/jira/browse/YARN-10212
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Jonathan Hung
>Assignee: Bilwa S T
>Priority: Major
>
> Right now user's default max AM attempts is set to the same as global max AM
> attempts:
> {noformat}
> int globalMaxAppAttempts = conf.getInt(YarnConfiguration.RM_AM_MAX_ATTEMPTS,
> YarnConfiguration.DEFAULT_RM_AM_MAX_ATTEMPTS); {noformat}
> If we want to increase global max AM attempts, it will also increase the
> default. So we should create a separate global AM max attempts config to
> separate the two.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10212) Create separate configuration for max global AM attempts
[
https://issues.apache.org/jira/browse/YARN-10212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17074847#comment-17074847
]
Jonathan Hung edited comment on YARN-10212 at 4/3/20, 7:34 PM:
---
[~BilwaST] user's default max AM attempts is how many AM attempts they get if
they don't set individualMaxAppAttempts on client side.
I am proposing adding a new configuration like GLOBAL_RM_AM_MAX_ATTEMPTS, and
changing the code snippet above to something like:
{noformat}
int globalMaxAppAttempts =
conf.getInt(YarnConfiguration.GLOBAL_RM_AM_MAX_ATTEMPTS,
conf.getInt(YarnConfiguration.RM_AM_MAX_ATTEMPTS,
YarnConfiguration.DEFAULT_RM_AM_MAX_ATTEMPTS));{noformat}
If GLOBAL_RM_AM_MAX_ATTEMPTS is unset, it will fall back to current behavior.
But if GLOBAL_RM_AM_MAX_ATTEMPTS is set to something higher (e.g. 4), then if
user does not set individualMaxAppAttempts on client side, their app's number
of attempts will still use RM_AM_MAX_ATTEMPTS, but user can set
individualMaxAppAttempts to RM_AM_MAX_ATTEMPTS < individualMaxAppAttempts <=
GLOBAL_RM_AM_MAX_ATTEMPTS if they like.
was (Author: jhung):
[~BilwaST] user's default max AM attempts is how many AM attempts they get if
they don't set individualMaxAppAttempts on client side.
I am proposing changing the code snippet above to something like:
{noformat}
int globalMaxAppAttempts =
conf.getInt(YarnConfiguration.GLOBAL_RM_AM_MAX_ATTEMPTS,
conf.getInt(YarnConfiguration.RM_AM_MAX_ATTEMPTS,
YarnConfiguration.DEFAULT_RM_AM_MAX_ATTEMPTS));{noformat}
If GLOBAL_RM_AM_MAX_ATTEMPTS is unset, it will fall back to current behavior.
But if GLOBAL_RM_AM_MAX_ATTEMPTS is set to something higher (e.g. 4), then if
user does not set individualMaxAppAttempts on client side, their app's number
of attempts will still use RM_AM_MAX_ATTEMPTS, but user can set
individualMaxAppAttempts to RM_AM_MAX_ATTEMPTS < individualMaxAppAttempts <=
GLOBAL_RM_AM_MAX_ATTEMPTS if they like.
> Create separate configuration for max global AM attempts
>
>
> Key: YARN-10212
> URL: https://issues.apache.org/jira/browse/YARN-10212
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Jonathan Hung
>Assignee: Bilwa S T
>Priority: Major
>
> Right now user's default max AM attempts is set to the same as global max AM
> attempts:
> {noformat}
> int globalMaxAppAttempts = conf.getInt(YarnConfiguration.RM_AM_MAX_ATTEMPTS,
> YarnConfiguration.DEFAULT_RM_AM_MAX_ATTEMPTS); {noformat}
> If we want to increase global max AM attempts, it will also increase the
> default. So we should create a separate global AM max attempts config to
> separate the two.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
