[jira] [Commented] (YARN-10208) Add metric in CapacityScheduler for evaluating the time difference between node heartbeats
[ https://issues.apache.org/jira/browse/YARN-10208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071476#comment-17071476 ] Bibin Chundatt commented on YARN-10208: --- +1 looks good to me. > Add metric in CapacityScheduler for evaluating the time difference between > node heartbeats > -- > > Key: YARN-10208 > URL: https://issues.apache.org/jira/browse/YARN-10208 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Pranjal Protim Borah >Assignee: Pranjal Protim Borah >Priority: Minor > Attachments: YARN-10208.001.patch, YARN-10208.002.patch, > YARN-10208.003.patch, YARN-10208.004.patch > > > Metric measuring average time interval between node heartbeats in capacity > scheduler on node update event. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10216) Utility to dynamically reload Configuration on the disk
[ https://issues.apache.org/jira/browse/YARN-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyrus Jackson updated YARN-10216: - Description: There should be a way to dynamically reload the configuration properties from the disk. The approach is to notify the individual classes that the Configuration is reloaded and make the changes on the fly. This is similar to how Hbase has done. was: There should be a way to dynamically reload the configuration properties from the disk. The approach is to notify the individual classes that the Configuration is reloaded. This is similar to how Hbase has done. > Utility to dynamically reload Configuration on the disk > --- > > Key: YARN-10216 > URL: https://issues.apache.org/jira/browse/YARN-10216 > Project: Hadoop YARN > Issue Type: New Feature >Reporter: Cyrus Jackson >Priority: Major > > There should be a way to dynamically reload the configuration properties from > the disk. The approach is to notify the individual classes that the > Configuration is reloaded and make the changes on the fly. This is similar to > how Hbase has done. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10216) Utility to dynamically reload Configuration on the disk
[ https://issues.apache.org/jira/browse/YARN-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyrus Jackson updated YARN-10216: - Description: There should be a way to dynamically reload the configuration properties from the disk. The approach is to notify the individual classes that the Configuration is reloaded. This is similar to how Hbase has done. was:Right > Utility to dynamically reload Configuration on the disk > --- > > Key: YARN-10216 > URL: https://issues.apache.org/jira/browse/YARN-10216 > Project: Hadoop YARN > Issue Type: New Feature >Reporter: Cyrus Jackson >Priority: Major > > There should be a way to dynamically reload the configuration properties from > the disk. The approach is to notify the individual classes that the > Configuration is reloaded. This is similar to how Hbase has done. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10212) Create separate configuration for max global AM attempts
[
https://issues.apache.org/jira/browse/YARN-10212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071465#comment-17071465
]
Bilwa S T commented on YARN-10212:
--
Hi [~jhung] , Yes i want to work on this.
> Create separate configuration for max global AM attempts
>
>
> Key: YARN-10212
> URL: https://issues.apache.org/jira/browse/YARN-10212
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Jonathan Hung
>Assignee: Bilwa S T
>Priority: Major
>
> Right now user's default max AM attempts is set to the same as global max AM
> attempts:
> {noformat}
> int globalMaxAppAttempts = conf.getInt(YarnConfiguration.RM_AM_MAX_ATTEMPTS,
> YarnConfiguration.DEFAULT_RM_AM_MAX_ATTEMPTS); {noformat}
> If we want to increase global max AM attempts, it will also increase the
> default. So we should create a separate global AM max attempts config to
> separate the two.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10216) Utility to dynamically reload Configuration on the disk
[ https://issues.apache.org/jira/browse/YARN-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyrus Jackson updated YARN-10216: - Summary: Utility to dynamically reload Configuration on the disk (was: Utility to reload Configuration on the disk) > Utility to dynamically reload Configuration on the disk > --- > > Key: YARN-10216 > URL: https://issues.apache.org/jira/browse/YARN-10216 > Project: Hadoop YARN > Issue Type: New Feature >Reporter: Cyrus Jackson >Priority: Major > > Right -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10216) Utility to reload Configuration on the disk
[ https://issues.apache.org/jira/browse/YARN-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyrus Jackson updated YARN-10216: - Description: Right > Utility to reload Configuration on the disk > --- > > Key: YARN-10216 > URL: https://issues.apache.org/jira/browse/YARN-10216 > Project: Hadoop YARN > Issue Type: New Feature >Reporter: Cyrus Jackson >Priority: Major > > Right -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10216) Utility to reload Configuration on the disk
Cyrus Jackson created YARN-10216: Summary: Utility to reload Configuration on the disk Key: YARN-10216 URL: https://issues.apache.org/jira/browse/YARN-10216 Project: Hadoop YARN Issue Type: New Feature Reporter: Cyrus Jackson -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10214) Separate TaskRunner for AM and NM tasks in SLS.
[
https://issues.apache.org/jira/browse/YARN-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071321#comment-17071321
]
Hadoop QA commented on YARN-10214:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
41s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 10s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 12s{color} | {color:orange} hadoop-tools/hadoop-sls: The patch generated 15
new + 8 unchanged - 1 fixed = 23 total (was 9) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 24s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 11m 56s{color}
| {color:red} hadoop-sls in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
33s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 67m 35s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.sls.appmaster.TestAMSimulator |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.8 Server=19.03.8 Image:yetus/hadoop:4454c6d14b7 |
| JIRA Issue | YARN-10214 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12998272/YARN-10214.001.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux acf7c339d286 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 80b877a |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/25786/artifact/out/diff-checkstyle-hadoop-tools_hadoop-sls.txt
|
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/25786/artifact/out/patch-unit-hadoop-tools_hadoop-sls.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25786/testReport/ |
| Max. process+thread count | 430 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-sls U:
[jira] [Updated] (YARN-10214) Separate TaskRunner for AM and NM tasks in SLS.
[ https://issues.apache.org/jira/browse/YARN-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyrus Jackson updated YARN-10214: - Attachment: (was: YARN-10214.001.patch) > Separate TaskRunner for AM and NM tasks in SLS. > > > Key: YARN-10214 > URL: https://issues.apache.org/jira/browse/YARN-10214 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler-load-simulator >Reporter: Cyrus Jackson >Assignee: Cyrus Jackson >Priority: Minor > Attachments: YARN-10214.001.patch > > > NM simulator and AM simulator uses the same TaskRunner Pool currently. > Defining separate pools for AM and NM will improve the run times. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10214) Separate TaskRunner for AM and NM tasks in SLS.
[ https://issues.apache.org/jira/browse/YARN-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyrus Jackson updated YARN-10214: - Attachment: YARN-10214.001.patch > Separate TaskRunner for AM and NM tasks in SLS. > > > Key: YARN-10214 > URL: https://issues.apache.org/jira/browse/YARN-10214 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler-load-simulator >Reporter: Cyrus Jackson >Assignee: Cyrus Jackson >Priority: Minor > Attachments: YARN-10214.001.patch > > > NM simulator and AM simulator uses the same TaskRunner Pool currently. > Defining separate pools for AM and NM will improve the run times. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10212) Create separate configuration for max global AM attempts
[
https://issues.apache.org/jira/browse/YARN-10212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071249#comment-17071249
]
Jonathan Hung commented on YARN-10212:
--
Hey [~BilwaST], do you plan to take on this task?
> Create separate configuration for max global AM attempts
>
>
> Key: YARN-10212
> URL: https://issues.apache.org/jira/browse/YARN-10212
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Jonathan Hung
>Assignee: Bilwa S T
>Priority: Major
>
> Right now user's default max AM attempts is set to the same as global max AM
> attempts:
> {noformat}
> int globalMaxAppAttempts = conf.getInt(YarnConfiguration.RM_AM_MAX_ATTEMPTS,
> YarnConfiguration.DEFAULT_RM_AM_MAX_ATTEMPTS); {noformat}
> If we want to increase global max AM attempts, it will also increase the
> default. So we should create a separate global AM max attempts config to
> separate the two.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10172) Default ApplicationPlacementType class should be configurable
[
https://issues.apache.org/jira/browse/YARN-10172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071043#comment-17071043
]
Adam Antal commented on YARN-10172:
---
Thanks for the patch [~cyrusjackson25]!
I have some minor nits here:
- Since it's not defined in the environment anymore, I think
{{APPLICATION_PLACEMENT_TYPE_CLASS}} or
{{APPLICATION_PLACEMENT_TYPE_CLASS_DEFAULT}} would be more appropriate there
(But I prefer the first since it's not a default, just a fallback in case the
env variable is not set. Also in {{YarnConfiguration}} class usually the
configuration's default are prefixed or suffixed with {{DEFAULT}}).
- Do you tried out the patch in a cluster? I see that there is a validation in
{{ApplicationPlacementAllocatorFactory}} that the class is available and it is
a subclass of a given type. I was wondering what could make debugging easier in
cases where the users try to find out how that class is actually instantiated.
INFO or DEBUG logging would be an overkill since it is a quite commonly created
class. I think I let it go, but if you can think of something, that would be
great.
> Default ApplicationPlacementType class should be configurable
> -
>
> Key: YARN-10172
> URL: https://issues.apache.org/jira/browse/YARN-10172
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Cyrus Jackson
>Assignee: Cyrus Jackson
>Priority: Minor
> Attachments: YARN-10172.001.patch, YARN-10172.002.patch
>
>
> This can be useful in scheduling apps based on the configured placement type
> class rather than resorting to LocalityAppPlacementAllocator
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10215) Endpoint for obtaining direct URL for the logs
[
https://issues.apache.org/jira/browse/YARN-10215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071030#comment-17071030
]
Hadoop QA commented on YARN-10215:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
18m 32s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
18s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 27s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 27s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
23s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
51s{color} | {color:green} hadoop-yarn-server-common in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
37s{color} | {color:green} hadoop-yarn-ui in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
47s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 91m 59s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs |
[jira] [Commented] (YARN-10208) Add metric in CapacityScheduler for evaluating the time difference between node heartbeats
[
https://issues.apache.org/jira/browse/YARN-10208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071012#comment-17071012
]
Hadoop QA commented on YARN-10208:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
37s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 27s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 29s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 1 new + 98 unchanged - 0 fixed = 99 total (was 98) {color}
|
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 27s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 89m 7s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}148m 6s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.8 Server=19.03.8 Image:yetus/hadoop:4454c6d14b7 |
| JIRA Issue | YARN-10208 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12998216/YARN-10208.004.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 57ea8265f57b 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 960c9eb |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/25784/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| unit |
[jira] [Commented] (YARN-10213) Using curator-leader-elector, rm will always be in standby state at some times.
[
https://issues.apache.org/jira/browse/YARN-10213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17070905#comment-17070905
]
Hadoop QA commented on YARN-10213:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
40s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 27s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 13s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 88m
35s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}147m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.8 Server=19.03.8 Image:yetus/hadoop:4454c6d14b7 |
| JIRA Issue | YARN-10213 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12998202/YARN-10213.002.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 8668d19715fb 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 960c9eb |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25783/testReport/ |
| Max. process+thread count | 924 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/25783/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Using curator-leader-elector, rm will always be
[jira] [Updated] (YARN-10208) Add metric in CapacityScheduler for evaluating the time difference between node heartbeats
[ https://issues.apache.org/jira/browse/YARN-10208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pranjal Protim Borah updated YARN-10208: Attachment: YARN-10208.004.patch > Add metric in CapacityScheduler for evaluating the time difference between > node heartbeats > -- > > Key: YARN-10208 > URL: https://issues.apache.org/jira/browse/YARN-10208 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Pranjal Protim Borah >Assignee: Pranjal Protim Borah >Priority: Minor > Attachments: YARN-10208.001.patch, YARN-10208.002.patch, > YARN-10208.003.patch, YARN-10208.004.patch > > > Metric measuring average time interval between node heartbeats in capacity > scheduler on node update event. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10215) Endpoint for obtaining direct URL for the logs
[ https://issues.apache.org/jira/browse/YARN-10215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Antal updated YARN-10215: -- Target Version/s: 3.3.0, 3.4.0 > Endpoint for obtaining direct URL for the logs > -- > > Key: YARN-10215 > URL: https://issues.apache.org/jira/browse/YARN-10215 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Affects Versions: 3.3.0 >Reporter: Adam Antal >Assignee: Andras Gyori >Priority: Major > > If CORS protected UIs are set up, there is an issue when the browser tries to > access the logs of a running container in the RM web UIv2. > Assuming ATS is not up, the browser follows the following call chain: > - Tries to access ATS, it fails, falls back to JHS > - From RM the browser received basic app info, we know that the application > is running > - From the JHS we got the list of containers and their log files. > - When we try to access a specific log file, the JHS redirects the request to > the NM's UI (on which node the container is running). This redirect is > performed by the browser automatically. In this setup the host is considered > as a protected information, thus the browser omits the "Origin" field from > the request when this redirect is done. The browser then denies access to the > NodeManager's web UI due to the CORS header set up for NM, but the Origin is > null in the redirect request. > - Finally, "Logs are unavailable" message is shown in the RM web UIv2 due to > the CORS violation. > We should fix this. As an approach we can expose another endpoints which only > returns the URL of the NodeManager what we should call directly from the UIv2 > in order to receive the log. This adds a bit of a complexity, but will enable > users to keep the CORS protected setup. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10199) Simplify UserGroupMappingPlacementRule#getPlacementForUser
[ https://issues.apache.org/jira/browse/YARN-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17070811#comment-17070811 ] Andras Gyori edited comment on YARN-10199 at 3/30/20, 9:14 AM: --- Thank you [~pbacsko] for the feedback. I have applied your recommendations as detailed below: # stored the frequently used values in a local variable # removed the redundant GROUP_MAPPING checks, and simplified the boolean expression # 4. Fixed As for now, I hold the same position as you. Further granulation of this logic is not justified yet, however, it will be easier to migrate to a class-based approach from this state. was (Author: gandras): Thank you [~pbacsko] for the feedback. I have applied you advises as detailed below: # stored the frequently used values in a local variable # Removed the redundant GROUP_MAPPING checks, and simplified the boolean expression # 4. Fixed As for now, I hold the same position as you. Further granulation of this logic is not justified yet, however, it will be easier to migrate to a class-based approach from this state. > Simplify UserGroupMappingPlacementRule#getPlacementForUser > -- > > Key: YARN-10199 > URL: https://issues.apache.org/jira/browse/YARN-10199 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler >Reporter: Andras Gyori >Assignee: Andras Gyori >Priority: Minor > Attachments: YARN-10199.001.patch, YARN-10199.002.patch, > YARN-10199.003.patch, YARN-10199.004.patch, YARN-10199.005.patch > > > The UserGroupMappingPlacementRule#getPlacementForUser method, which is mainly > responsible for queue naming, contains deeply nested branches. In order to > provide an extendable mapping logic, the branches could be flattened and > simplified. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10199) Simplify UserGroupMappingPlacementRule#getPlacementForUser
[ https://issues.apache.org/jira/browse/YARN-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17070811#comment-17070811 ] Andras Gyori commented on YARN-10199: - Thank you [~pbacsko] for the feedback. I have applied you advises as detailed below: # stored the frequently used values in a local variable # Removed the redundant GROUP_MAPPING checks, and simplified the boolean expression # 4. Fixed As for now, I hold the same position as you. Further granulation of this logic is not justified yet, however, it will be easier to migrate to a class-based approach from this state. > Simplify UserGroupMappingPlacementRule#getPlacementForUser > -- > > Key: YARN-10199 > URL: https://issues.apache.org/jira/browse/YARN-10199 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler >Reporter: Andras Gyori >Assignee: Andras Gyori >Priority: Minor > Attachments: YARN-10199.001.patch, YARN-10199.002.patch, > YARN-10199.003.patch, YARN-10199.004.patch, YARN-10199.005.patch > > > The UserGroupMappingPlacementRule#getPlacementForUser method, which is mainly > responsible for queue naming, contains deeply nested branches. In order to > provide an extendable mapping logic, the branches could be flattened and > simplified. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10213) Using curator-leader-elector, rm will always be in standby state at some times.
[ https://issues.apache.org/jira/browse/YARN-10213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17070810#comment-17070810 ] Sen Zhao commented on YARN-10213: - Patch002 add test. > Using curator-leader-elector, rm will always be in standby state at some > times. > --- > > Key: YARN-10213 > URL: https://issues.apache.org/jira/browse/YARN-10213 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.2.1, 3.1.3 >Reporter: Sen Zhao >Assignee: Sen Zhao >Priority: Major > Attachments: YARN-10213.001.patch, YARN-10213.002.patch > > > When we use cursor-leader-elector, > *CuratorBasedElectorService.initAndStartLeaderLatch()* > will initialize and start *leaderLatch* to elect the leader. > Now： > 1.ResourceManager will call > *CuratorBasedElectorService.initAndStartLeaderLatch()* in *serviceInit()*. > 2.ResourceManager will transition to standby in *serviceStart()*. > If *leaderLatch* succeeds in electing leader before rm transitions to > standby. ResourceManager will always be in standby state. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10213) Using curator-leader-elector, rm will always be in standby state at some times.
[ https://issues.apache.org/jira/browse/YARN-10213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sen Zhao updated YARN-10213: Attachment: YARN-10213.002.patch > Using curator-leader-elector, rm will always be in standby state at some > times. > --- > > Key: YARN-10213 > URL: https://issues.apache.org/jira/browse/YARN-10213 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.2.1, 3.1.3 >Reporter: Sen Zhao >Assignee: Sen Zhao >Priority: Major > Attachments: YARN-10213.001.patch, YARN-10213.002.patch > > > When we use cursor-leader-elector, > *CuratorBasedElectorService.initAndStartLeaderLatch()* > will initialize and start *leaderLatch* to elect the leader. > Now： > 1.ResourceManager will call > *CuratorBasedElectorService.initAndStartLeaderLatch()* in *serviceInit()*. > 2.ResourceManager will transition to standby in *serviceStart()*. > If *leaderLatch* succeeds in electing leader before rm transitions to > standby. ResourceManager will always be in standby state. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10215) Endpoint for obtaining direct URL for the logs
Adam Antal created YARN-10215: - Summary: Endpoint for obtaining direct URL for the logs Key: YARN-10215 URL: https://issues.apache.org/jira/browse/YARN-10215 Project: Hadoop YARN Issue Type: Sub-task Components: yarn Affects Versions: 3.3.0 Reporter: Adam Antal Assignee: Andras Gyori If CORS protected UIs are set up, there is an issue when the browser tries to access the logs of a running container in the RM web UIv2. Assuming ATS is not up, the browser follows the following call chain: - Tries to access ATS, it fails, falls back to JHS - From RM the browser received basic app info, we know that the application is running - From the JHS we got the list of containers and their log files. - When we try to access a specific log file, the JHS redirects the request to the NM's UI (on which node the container is running). This redirect is performed by the browser automatically. In this setup the host is considered as a protected information, thus the browser omits the "Origin" field from the request when this redirect is done. The browser then denies access to the NodeManager's web UI due to the CORS header set up for NM, but the Origin is null in the redirect request. - Finally, "Logs are unavailable" message is shown in the RM web UIv2 due to the CORS violation. We should fix this. As an approach we can expose another endpoints which only returns the URL of the NodeManager what we should call directly from the UIv2 in order to receive the log. This adds a bit of a complexity, but will enable users to keep the CORS protected setup. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10199) Simplify UserGroupMappingPlacementRule#getPlacementForUser
[ https://issues.apache.org/jira/browse/YARN-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Gyori updated YARN-10199: Attachment: YARN-10199.005.patch > Simplify UserGroupMappingPlacementRule#getPlacementForUser > -- > > Key: YARN-10199 > URL: https://issues.apache.org/jira/browse/YARN-10199 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler >Reporter: Andras Gyori >Assignee: Andras Gyori >Priority: Minor > Attachments: YARN-10199.001.patch, YARN-10199.002.patch, > YARN-10199.003.patch, YARN-10199.004.patch, YARN-10199.005.patch > > > The UserGroupMappingPlacementRule#getPlacementForUser method, which is mainly > responsible for queue naming, contains deeply nested branches. In order to > provide an extendable mapping logic, the branches could be flattened and > simplified. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10214) Separate TaskRunner for AM and NM tasks in SLS.
[ https://issues.apache.org/jira/browse/YARN-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyrus Jackson updated YARN-10214: - Description: NM simulator and AM simulator uses the same TaskRunner Pool currently. Defining separate pools for AM and NM will improve the run times. (was: NM simulator and AM simulator uses the same TaskRunner Pool currently. Defining separate pool for AM and NM will improve the run times.) > Separate TaskRunner for AM and NM tasks in SLS. > > > Key: YARN-10214 > URL: https://issues.apache.org/jira/browse/YARN-10214 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler-load-simulator >Reporter: Cyrus Jackson >Assignee: Cyrus Jackson >Priority: Minor > > NM simulator and AM simulator uses the same TaskRunner Pool currently. > Defining separate pools for AM and NM will improve the run times. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10214) Separate TaskRunner for AM and NM tasks in SLS.
[ https://issues.apache.org/jira/browse/YARN-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyrus Jackson reassigned YARN-10214: Assignee: Cyrus Jackson > Separate TaskRunner for AM and NM tasks in SLS. > > > Key: YARN-10214 > URL: https://issues.apache.org/jira/browse/YARN-10214 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler-load-simulator >Reporter: Cyrus Jackson >Assignee: Cyrus Jackson >Priority: Minor > > NM simulator and AM simulator uses the same TaskRunner Pool currently. > Defining separate pool for AM and NM will improve the run times. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10214) Separate TaskRunner for AM and NM tasks in SLS.
[ https://issues.apache.org/jira/browse/YARN-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyrus Jackson updated YARN-10214: - Priority: Minor (was: Major) > Separate TaskRunner for AM and NM tasks in SLS. > > > Key: YARN-10214 > URL: https://issues.apache.org/jira/browse/YARN-10214 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler-load-simulator >Reporter: Cyrus Jackson >Priority: Minor > > NM simulator and AM simulator uses the same TaskRunner Pool currently. > Defining separate pool for AM and NM -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10214) Separate TaskRunner for AM and NM tasks in SLS.
[ https://issues.apache.org/jira/browse/YARN-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyrus Jackson updated YARN-10214: - Description: NM simulator and AM simulator uses the same TaskRunner Pool currently. Defining separate pool for AM and NM > Separate TaskRunner for AM and NM tasks in SLS. > > > Key: YARN-10214 > URL: https://issues.apache.org/jira/browse/YARN-10214 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler-load-simulator >Reporter: Cyrus Jackson >Priority: Major > > NM simulator and AM simulator uses the same TaskRunner Pool currently. > Defining separate pool for AM and NM -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10214) Separate TaskRunner for AM and NM tasks in SLS.
[ https://issues.apache.org/jira/browse/YARN-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyrus Jackson updated YARN-10214: - Description: NM simulator and AM simulator uses the same TaskRunner Pool currently. Defining separate pool for AM and NM will improve the run times. (was: NM simulator and AM simulator uses the same TaskRunner Pool currently. Defining separate pool for AM and NM) > Separate TaskRunner for AM and NM tasks in SLS. > > > Key: YARN-10214 > URL: https://issues.apache.org/jira/browse/YARN-10214 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler-load-simulator >Reporter: Cyrus Jackson >Priority: Minor > > NM simulator and AM simulator uses the same TaskRunner Pool currently. > Defining separate pool for AM and NM will improve the run times. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10214) Separate TaskRunner for AM and NM tasks in SLS.
Cyrus Jackson created YARN-10214: Summary: Separate TaskRunner for AM and NM tasks in SLS. Key: YARN-10214 URL: https://issues.apache.org/jira/browse/YARN-10214 Project: Hadoop YARN Issue Type: Improvement Components: scheduler-load-simulator Reporter: Cyrus Jackson -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10208) Add metric in CapacityScheduler for evaluating the time difference between node heartbeats
[ https://issues.apache.org/jira/browse/YARN-10208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17070748#comment-17070748 ] Bibin Chundatt commented on YARN-10208: --- [~lapjarn] Minor nit: schedulerHeartBeatIntervalAverage variable and method name rename to schedulerNodeHBInterval > Add metric in CapacityScheduler for evaluating the time difference between > node heartbeats > -- > > Key: YARN-10208 > URL: https://issues.apache.org/jira/browse/YARN-10208 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Pranjal Protim Borah >Assignee: Pranjal Protim Borah >Priority: Minor > Attachments: YARN-10208.001.patch, YARN-10208.002.patch, > YARN-10208.003.patch > > > Metric measuring average time interval between node heartbeats in capacity > scheduler on node update event. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
