[jira] [Commented] (YARN-5825) ProportionalPreemptionalPolicy could use readLock over LeafQueue instead of synchronized block
[
https://issues.apache.org/jira/browse/YARN-5825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15820042#comment-15820042
]
Sunil G commented on YARN-5825:
---
I do not see this as an incompatible change. [~jianhe], could you please
confirm.
> ProportionalPreemptionalPolicy could use readLock over LeafQueue instead of
> synchronized block
> --
>
> Key: YARN-5825
> URL: https://issues.apache.org/jira/browse/YARN-5825
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Reporter: Sunil G
>Assignee: Sunil G
> Fix For: 2.9.0, 3.0.0-alpha2
>
> Attachments: YARN-5825.0001.patch, YARN-5825.0002.patch
>
>
> Currently in PCPP, {{synchronized (curQueue)}} is used in various places.
> Such instances could be replaced with a read lock. Thank you [~jianhe] for
> pointing out the same as comment
> [here|https://issues.apache.org/jira/browse/YARN-2009?focusedCommentId=15626578=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15626578]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5825) ProportionalPreemptionalPolicy could use readLock over LeafQueue instead of synchronized block
[
https://issues.apache.org/jira/browse/YARN-5825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15819695#comment-15819695
]
Andrew Wang commented on YARN-5825:
---
Doing a release notes sweep for 3.0.0-alpha2, noticed this JIRA. If it's
incompatible, should it have been committed to branch-2?
Also, if this is incompatible, could someone also add a release note detailing
the exposure? Thanks!
> ProportionalPreemptionalPolicy could use readLock over LeafQueue instead of
> synchronized block
> --
>
> Key: YARN-5825
> URL: https://issues.apache.org/jira/browse/YARN-5825
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Reporter: Sunil G
>Assignee: Sunil G
> Fix For: 2.9.0, 3.0.0-alpha2
>
> Attachments: YARN-5825.0001.patch, YARN-5825.0002.patch
>
>
> Currently in PCPP, {{synchronized (curQueue)}} is used in various places.
> Such instances could be replaced with a read lock. Thank you [~jianhe] for
> pointing out the same as comment
> [here|https://issues.apache.org/jira/browse/YARN-2009?focusedCommentId=15626578=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15626578]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5825) ProportionalPreemptionalPolicy could use readLock over LeafQueue instead of synchronized block
[
https://issues.apache.org/jira/browse/YARN-5825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15658534#comment-15658534
]
Hudson commented on YARN-5825:
--
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10827 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/10827/])
YARN-5825. ProportionalPreemptionalPolicy should use readLock over (jianhe: rev
fad9609d13e76e9e3a4e01c96f698bb60b03807e)
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/TestProportionalCapacityPreemptionPolicy.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/ProportionalCapacityPreemptionPolicy.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/FifoCandidatesSelector.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/ProportionalCapacityPreemptionPolicyMockFramework.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/IntraQueueCandidatesSelector.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueue.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java
> ProportionalPreemptionalPolicy could use readLock over LeafQueue instead of
> synchronized block
> --
>
> Key: YARN-5825
> URL: https://issues.apache.org/jira/browse/YARN-5825
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Reporter: Sunil G
>Assignee: Sunil G
> Fix For: 2.9.0, 3.0.0-alpha2
>
> Attachments: YARN-5825.0001.patch, YARN-5825.0002.patch
>
>
> Currently in PCPP, {{synchronized (curQueue)}} is used in various places.
> Such instances could be replaced with a read lock. Thank you [~jianhe] for
> pointing out the same as comment
> [here|https://issues.apache.org/jira/browse/YARN-2009?focusedCommentId=15626578=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15626578]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5825) ProportionalPreemptionalPolicy could use readLock over LeafQueue instead of synchronized block
[
https://issues.apache.org/jira/browse/YARN-5825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15658406#comment-15658406
]
Jian He commented on YARN-5825:
---
lgtm , committing
> ProportionalPreemptionalPolicy could use readLock over LeafQueue instead of
> synchronized block
> --
>
> Key: YARN-5825
> URL: https://issues.apache.org/jira/browse/YARN-5825
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Reporter: Sunil G
>Assignee: Sunil G
> Attachments: YARN-5825.0001.patch, YARN-5825.0002.patch
>
>
> Currently in PCPP, {{synchronized (curQueue)}} is used in various places.
> Such instances could be replaced with a read lock. Thank you [~jianhe] for
> pointing out the same as comment
> [here|https://issues.apache.org/jira/browse/YARN-2009?focusedCommentId=15626578=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15626578]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5825) ProportionalPreemptionalPolicy could use readLock over LeafQueue instead of synchronized block
[
https://issues.apache.org/jira/browse/YARN-5825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15657731#comment-15657731
]
Hadoop QA commented on YARN-5825:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 24s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 3 new + 557 unchanged - 0 fixed = 560 total (was 557)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 41m 2s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 57m 27s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.TestTokenClientRMService |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:e809691 |
| JIRA Issue | YARN-5825 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12838575/YARN-5825.0002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 280dae8b9979 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 503e73e |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/13877/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/13877/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/13877/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
[jira] [Commented] (YARN-5825) ProportionalPreemptionalPolicy could use readLock over LeafQueue instead of synchronized block
[
https://issues.apache.org/jira/browse/YARN-5825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655310#comment-15655310
]
Jian He commented on YARN-5825:
---
looks good overall,
looks like this method added is not used? can be removed
{code}
public ReentrantReadWriteLock.WriteLock getWriteLock() {
return writeLock;
}
{code}
> ProportionalPreemptionalPolicy could use readLock over LeafQueue instead of
> synchronized block
> --
>
> Key: YARN-5825
> URL: https://issues.apache.org/jira/browse/YARN-5825
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Reporter: Sunil G
>Assignee: Sunil G
> Attachments: YARN-5825.0001.patch
>
>
> Currently in PCPP, {{synchronized (curQueue)}} is used in various places.
> Such instances could be replaced with a read lock. Thank you [~jianhe] for
> pointing out the same as comment
> [here|https://issues.apache.org/jira/browse/YARN-2009?focusedCommentId=15626578=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15626578]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5825) ProportionalPreemptionalPolicy could use readLock over LeafQueue instead of synchronized block
[
https://issues.apache.org/jira/browse/YARN-5825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15654471#comment-15654471
]
Hadoop QA commented on YARN-5825:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 22s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 3 new + 114 unchanged - 0 fixed = 117 total (was 114)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 42m 20s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 60m 20s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.monitor.capacity.TestProportionalCapacityPreemptionPolicyIntraQueue
|
| |
hadoop.yarn.server.resourcemanager.monitor.capacity.TestProportionalCapacityPreemptionPolicy
|
| |
hadoop.yarn.server.resourcemanager.monitor.capacity.TestProportionalCapacityPreemptionPolicyForReservedContainers
|
| |
hadoop.yarn.server.resourcemanager.monitor.capacity.TestProportionalCapacityPreemptionPolicyForNodePartitions
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:e809691 |
| JIRA Issue | YARN-5825 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12838367/YARN-5825.0001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 4537a22cc49e 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / ca68f9c |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/13859/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| unit |
