[jira] [Commented] (YARN-2921) MockRM#waitForState methods can be too slow and flaky
[
https://issues.apache.org/jira/browse/YARN-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14541727#comment-14541727
]
Hadoop QA commented on YARN-2921:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 40s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 5 new or modified test files. |
| {color:green}+1{color} | javac | 7m 32s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 32s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 40s | There were no new checkstyle
issues. |
| {color:red}-1{color} | whitespace | 0m 0s | The patch has 1 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 34s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 39s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | yarn tests | 1m 57s | Tests passed in
hadoop-yarn-common. |
| {color:green}+1{color} | yarn tests | 49m 46s | Tests passed in
hadoop-yarn-server-resourcemanager. |
| | | 90m 20s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12732511/YARN-2921.008.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 92c38e4 |
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/7915/artifact/patchprocess/whitespace.txt
|
| hadoop-yarn-common test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7915/artifact/patchprocess/testrun_hadoop-yarn-common.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7915/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7915/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7915/console |
This message was automatically generated.
MockRM#waitForState methods can be too slow and flaky
-
Key: YARN-2921
URL: https://issues.apache.org/jira/browse/YARN-2921
Project: Hadoop YARN
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0, 2.7.0
Reporter: Karthik Kambatla
Assignee: Tsuyoshi Ozawa
Attachments: YARN-2921.001.patch, YARN-2921.002.patch,
YARN-2921.003.patch, YARN-2921.004.patch, YARN-2921.005.patch,
YARN-2921.006.patch, YARN-2921.007.patch, YARN-2921.008.patch,
YARN-2921.008.patch
MockRM#waitForState methods currently sleep for too long (2 seconds and 1
second). This leads to slow tests and sometimes failures if the
App/AppAttempt moves to another state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2921) MockRM#waitForState methods can be too slow and flaky
[ https://issues.apache.org/jira/browse/YARN-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14541734#comment-14541734 ] Tsuyoshi Ozawa commented on YARN-2921: -- [~leftnoteasy] all tests are green. Could you check v8 patch and my comments? https://issues.apache.org/jira/browse/YARN-2921?focusedCommentId=14539843page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14539843 MockRM#waitForState methods can be too slow and flaky - Key: YARN-2921 URL: https://issues.apache.org/jira/browse/YARN-2921 Project: Hadoop YARN Issue Type: Improvement Components: test Affects Versions: 2.6.0, 2.7.0 Reporter: Karthik Kambatla Assignee: Tsuyoshi Ozawa Attachments: YARN-2921.001.patch, YARN-2921.002.patch, YARN-2921.003.patch, YARN-2921.004.patch, YARN-2921.005.patch, YARN-2921.006.patch, YARN-2921.007.patch, YARN-2921.008.patch, YARN-2921.008.patch MockRM#waitForState methods currently sleep for too long (2 seconds and 1 second). This leads to slow tests and sometimes failures if the App/AppAttempt moves to another state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2921) MockRM#waitForState methods can be too slow and flaky
[ https://issues.apache.org/jira/browse/YARN-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542592#comment-14542592 ] Wangda Tan commented on YARN-2921: -- Thanks for explanation, [~ozawa]. Latest patch LGTM, checking it in.. MockRM#waitForState methods can be too slow and flaky - Key: YARN-2921 URL: https://issues.apache.org/jira/browse/YARN-2921 Project: Hadoop YARN Issue Type: Improvement Components: test Affects Versions: 2.6.0, 2.7.0 Reporter: Karthik Kambatla Assignee: Tsuyoshi Ozawa Attachments: YARN-2921.001.patch, YARN-2921.002.patch, YARN-2921.003.patch, YARN-2921.004.patch, YARN-2921.005.patch, YARN-2921.006.patch, YARN-2921.007.patch, YARN-2921.008.patch, YARN-2921.008.patch MockRM#waitForState methods currently sleep for too long (2 seconds and 1 second). This leads to slow tests and sometimes failures if the App/AppAttempt moves to another state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2921) MockRM#waitForState methods can be too slow and flaky
[
https://issues.apache.org/jira/browse/YARN-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539849#comment-14539849
]
Tsuyoshi Ozawa commented on YARN-2921:
--
{quote}
Fixed.
{quote}
Fixed on v8 patch.
MockRM#waitForState methods can be too slow and flaky
-
Key: YARN-2921
URL: https://issues.apache.org/jira/browse/YARN-2921
Project: Hadoop YARN
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0, 2.7.0
Reporter: Karthik Kambatla
Assignee: Tsuyoshi Ozawa
Attachments: YARN-2921.001.patch, YARN-2921.002.patch,
YARN-2921.003.patch, YARN-2921.004.patch, YARN-2921.005.patch,
YARN-2921.006.patch, YARN-2921.007.patch, YARN-2921.008.patch
MockRM#waitForState methods currently sleep for too long (2 seconds and 1
second). This leads to slow tests and sometimes failures if the
App/AppAttempt moves to another state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2921) MockRM#waitForState methods can be too slow and flaky
[
https://issues.apache.org/jira/browse/YARN-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540097#comment-14540097
]
Hadoop QA commented on YARN-2921:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 15m 4s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 5 new or modified test files. |
| {color:green}+1{color} | javac | 7m 43s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 53s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 58s | There were no new checkstyle
issues. |
| {color:red}-1{color} | whitespace | 0m 1s | The patch has 1 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 40s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | yarn tests | 1m 58s | Tests passed in
hadoop-yarn-common. |
| {color:red}-1{color} | yarn tests | 43m 48s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 84m 40s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests |
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices |
| | hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA |
| | hadoop.yarn.server.resourcemanager.TestRMRestart |
| | hadoop.yarn.server.resourcemanager.TestClientRMService |
| |
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokens |
| | hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12732234/YARN-2921.008.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 6d5da94 |
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/7885/artifact/patchprocess/whitespace.txt
|
| hadoop-yarn-common test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7885/artifact/patchprocess/testrun_hadoop-yarn-common.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7885/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7885/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7885/console |
This message was automatically generated.
MockRM#waitForState methods can be too slow and flaky
-
Key: YARN-2921
URL: https://issues.apache.org/jira/browse/YARN-2921
Project: Hadoop YARN
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0, 2.7.0
Reporter: Karthik Kambatla
Assignee: Tsuyoshi Ozawa
Attachments: YARN-2921.001.patch, YARN-2921.002.patch,
YARN-2921.003.patch, YARN-2921.004.patch, YARN-2921.005.patch,
YARN-2921.006.patch, YARN-2921.007.patch, YARN-2921.008.patch
MockRM#waitForState methods currently sleep for too long (2 seconds and 1
second). This leads to slow tests and sometimes failures if the
App/AppAttempt moves to another state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2921) MockRM#waitForState methods can be too slow and flaky
[ https://issues.apache.org/jira/browse/YARN-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540106#comment-14540106 ] Tsuyoshi Ozawa commented on YARN-2921: -- Hmm, all test failures say ZipException: invalid block type, and it looks not related to the patch. Kicking Jenkins again after resolving the issue. MockRM#waitForState methods can be too slow and flaky - Key: YARN-2921 URL: https://issues.apache.org/jira/browse/YARN-2921 Project: Hadoop YARN Issue Type: Improvement Components: test Affects Versions: 2.6.0, 2.7.0 Reporter: Karthik Kambatla Assignee: Tsuyoshi Ozawa Attachments: YARN-2921.001.patch, YARN-2921.002.patch, YARN-2921.003.patch, YARN-2921.004.patch, YARN-2921.005.patch, YARN-2921.006.patch, YARN-2921.007.patch, YARN-2921.008.patch MockRM#waitForState methods currently sleep for too long (2 seconds and 1 second). This leads to slow tests and sometimes failures if the App/AppAttempt moves to another state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2921) MockRM#waitForState methods can be too slow and flaky
[
https://issues.apache.org/jira/browse/YARN-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538956#comment-14538956
]
Wangda Tan commented on YARN-2921:
--
Hi [~ozawa],
Some comments:
- In MockAM.waitForState, I'm not very understand about the change: 1. why
minWaitMSec is needed? 2. Why fail the method if {{if (waitedMsecs =
timeoutMsecs)}} is true? I think it should check now-state against expected
state.
- In two MockRM.waitForState method, I think we should also check
app.getState() instead of time, correct?
- In TestRMRestart, you can use GenericTestUtils.waitFor instead.
MockRM#waitForState methods can be too slow and flaky
-
Key: YARN-2921
URL: https://issues.apache.org/jira/browse/YARN-2921
Project: Hadoop YARN
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0, 2.7.0
Reporter: Karthik Kambatla
Assignee: Tsuyoshi Ozawa
Attachments: YARN-2921.001.patch, YARN-2921.002.patch,
YARN-2921.003.patch, YARN-2921.004.patch, YARN-2921.005.patch,
YARN-2921.006.patch, YARN-2921.007.patch
MockRM#waitForState methods currently sleep for too long (2 seconds and 1
second). This leads to slow tests and sometimes failures if the
App/AppAttempt moves to another state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2921) MockRM#waitForState methods can be too slow and flaky
[
https://issues.apache.org/jira/browse/YARN-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537600#comment-14537600
]
Tsuyoshi Ozawa commented on YARN-2921:
--
[~leftnoteasy], could you take a look latest patch? This patch includes
following changes:
1. I found that Thread.isAlive() doesn't work correctly in AsyncDispatcher when
the state of thread is not referenced in busy loop. This problem looks a timing
bug, and the change of MockRM reproduced the problem. It would be bug of JIT
complication or something. but I don't track it deeply currently.
{code}
while (!drained eventHandlingThread.isAlive()) {
waitForDrained.wait(1000);
- LOG.info(Waiting for AsyncDispatcher to drain.);
+ LOG.info(Waiting for AsyncDispatcher to drain. Thread state is : +
+ eventHandlingThread.getState());
}
{code}
2. The failure of TestAMRestart is because of the change of period of polling.
I made the timeout of submitApps larger.
MockRM#waitForState methods can be too slow and flaky
-
Key: YARN-2921
URL: https://issues.apache.org/jira/browse/YARN-2921
Project: Hadoop YARN
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0, 2.7.0
Reporter: Karthik Kambatla
Assignee: Tsuyoshi Ozawa
Attachments: YARN-2921.001.patch, YARN-2921.002.patch,
YARN-2921.003.patch, YARN-2921.004.patch, YARN-2921.005.patch,
YARN-2921.006.patch, YARN-2921.007.patch
MockRM#waitForState methods currently sleep for too long (2 seconds and 1
second). This leads to slow tests and sometimes failures if the
App/AppAttempt moves to another state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2921) MockRM#waitForState methods can be too slow and flaky
[
https://issues.apache.org/jira/browse/YARN-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537581#comment-14537581
]
Hadoop QA commented on YARN-2921:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 44s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 5 new or modified test files. |
| {color:green}+1{color} | javac | 7m 36s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 36s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 38s | There were no new checkstyle
issues. |
| {color:red}-1{color} | whitespace | 0m 1s | The patch has 1 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 36s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 39s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | yarn tests | 1m 57s | Tests passed in
hadoop-yarn-common. |
| {color:green}+1{color} | yarn tests | 49m 48s | Tests passed in
hadoop-yarn-server-resourcemanager. |
| | | 90m 33s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12731846/YARN-2921.007.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 4536399 |
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/7859/artifact/patchprocess/whitespace.txt
|
| hadoop-yarn-common test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7859/artifact/patchprocess/testrun_hadoop-yarn-common.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7859/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7859/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7859/console |
This message was automatically generated.
MockRM#waitForState methods can be too slow and flaky
-
Key: YARN-2921
URL: https://issues.apache.org/jira/browse/YARN-2921
Project: Hadoop YARN
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0, 2.7.0
Reporter: Karthik Kambatla
Assignee: Tsuyoshi Ozawa
Attachments: YARN-2921.001.patch, YARN-2921.002.patch,
YARN-2921.003.patch, YARN-2921.004.patch, YARN-2921.005.patch,
YARN-2921.006.patch, YARN-2921.007.patch
MockRM#waitForState methods currently sleep for too long (2 seconds and 1
second). This leads to slow tests and sometimes failures if the
App/AppAttempt moves to another state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2921) MockRM#waitForState methods can be too slow and flaky
[
https://issues.apache.org/jira/browse/YARN-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537491#comment-14537491
]
Hadoop QA commented on YARN-2921:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 5m 19s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 5 new or modified test files. |
| {color:green}+1{color} | javac | 7m 31s | There were no new javac warning
messages. |
| {color:green}+1{color} | release audit | 0m 20s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 32s | There were no new checkstyle
issues. |
| {color:red}-1{color} | whitespace | 0m 1s | The patch has 1 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 39s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 13s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:red}-1{color} | yarn tests | 60m 3s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 78m 13s | |
\\
\\
|| Reason || Tests ||
| Timed out tests |
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
|
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12731836/YARN-2921.006.patch |
| Optional Tests | javac unit findbugs checkstyle |
| git revision | trunk / 4536399 |
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/7858/artifact/patchprocess/whitespace.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7858/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7858/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7858/console |
This message was automatically generated.
MockRM#waitForState methods can be too slow and flaky
-
Key: YARN-2921
URL: https://issues.apache.org/jira/browse/YARN-2921
Project: Hadoop YARN
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0, 2.7.0
Reporter: Karthik Kambatla
Assignee: Tsuyoshi Ozawa
Attachments: YARN-2921.001.patch, YARN-2921.002.patch,
YARN-2921.003.patch, YARN-2921.004.patch, YARN-2921.005.patch,
YARN-2921.006.patch
MockRM#waitForState methods currently sleep for too long (2 seconds and 1
second). This leads to slow tests and sometimes failures if the
App/AppAttempt moves to another state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2921) MockRM#waitForState methods can be too slow and flaky
[
https://issues.apache.org/jira/browse/YARN-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537314#comment-14537314
]
Hadoop QA commented on YARN-2921:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 5m 13s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 5 new or modified test files. |
| {color:green}+1{color} | javac | 7m 32s | There were no new javac warning
messages. |
| {color:green}+1{color} | release audit | 0m 20s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 44s | There were no new checkstyle
issues. |
| {color:red}-1{color} | whitespace | 0m 1s | The patch has 1 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 36s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 31s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 15s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:red}-1{color} | yarn tests | 49m 32s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 66m 48s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests |
hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart |
| | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueMappings |
| |
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs |
| | hadoop.yarn.server.resourcemanager.TestApplicationMasterService |
| | hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA |
| | hadoop.yarn.server.resourcemanager.TestResourceTrackerService |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerDynamicBehavior
|
| | hadoop.yarn.server.resourcemanager.TestApplicationACLs |
| | hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer |
| | hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler |
| | hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
|
| | hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps |
| | hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter |
| Timed out tests | org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart
|
| | org.apache.hadoop.yarn.server.resourcemanager.TestRM |
| |
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
|
| |
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs
|
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12731808/YARN-2921.005.patch |
| Optional Tests | javac unit findbugs checkstyle |
| git revision | trunk / 4536399 |
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/7855/artifact/patchprocess/whitespace.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7855/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7855/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7855/console |
This message was automatically generated.
MockRM#waitForState methods can be too slow and flaky
-
Key: YARN-2921
URL: https://issues.apache.org/jira/browse/YARN-2921
Project: Hadoop YARN
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0, 2.7.0
Reporter: Karthik Kambatla
Assignee: Tsuyoshi Ozawa
Attachments: YARN-2921.001.patch, YARN-2921.002.patch,
YARN-2921.003.patch, YARN-2921.004.patch, YARN-2921.005.patch
MockRM#waitForState methods currently sleep for too long (2 seconds and 1
second). This leads to slow tests and sometimes failures if the
App/AppAttempt moves to another state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2921) MockRM#waitForState methods can be too slow and flaky
[ https://issues.apache.org/jira/browse/YARN-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14527855#comment-14527855 ] Tsuyoshi Ozawa commented on YARN-2921: -- [~leftnoteasy] thank you for pinging me. Yes, it looks related. Let me survey MockRM#waitForState methods can be too slow and flaky - Key: YARN-2921 URL: https://issues.apache.org/jira/browse/YARN-2921 Project: Hadoop YARN Issue Type: Improvement Components: test Affects Versions: 2.6.0 Reporter: Karthik Kambatla Assignee: Tsuyoshi Ozawa Attachments: YARN-2921.001.patch, YARN-2921.002.patch, YARN-2921.003.patch, YARN-2921.004.patch MockRM#waitForState methods currently sleep for too long (2 seconds and 1 second). This leads to slow tests and sometimes failures if the App/AppAttempt moves to another state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2921) MockRM#waitForState methods can be too slow and flaky
[ https://issues.apache.org/jira/browse/YARN-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14527481#comment-14527481 ] Wangda Tan commented on YARN-2921: -- Patch LGTM generally, is failure related to your patch? [~ozawa]. MockRM#waitForState methods can be too slow and flaky - Key: YARN-2921 URL: https://issues.apache.org/jira/browse/YARN-2921 Project: Hadoop YARN Issue Type: Improvement Components: test Affects Versions: 2.6.0 Reporter: Karthik Kambatla Assignee: Tsuyoshi Ozawa Attachments: YARN-2921.001.patch, YARN-2921.002.patch, YARN-2921.003.patch, YARN-2921.004.patch MockRM#waitForState methods currently sleep for too long (2 seconds and 1 second). This leads to slow tests and sometimes failures if the App/AppAttempt moves to another state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2921) MockRM#waitForState methods can be too slow and flaky
[
https://issues.apache.org/jira/browse/YARN-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14524387#comment-14524387
]
Hadoop QA commented on YARN-2921:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 5m 11s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 5 new or modified test files. |
| {color:green}+1{color} | javac | 7m 28s | There were no new javac warning
messages. |
| {color:green}+1{color} | release audit | 0m 19s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 49s | There were no new checkstyle
issues. |
| {color:red}-1{color} | whitespace | 0m 1s | The patch has 1 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 31s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 13s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:red}-1{color} | yarn tests | 50m 51s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 68m 1s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests |
hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12696827/YARN-2921.004.patch |
| Optional Tests | javac unit findbugs checkstyle |
| git revision | trunk / d3d019c |
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/7586/artifact/patchprocess/whitespace.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7586/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7586/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7586/console |
This message was automatically generated.
MockRM#waitForState methods can be too slow and flaky
-
Key: YARN-2921
URL: https://issues.apache.org/jira/browse/YARN-2921
Project: Hadoop YARN
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0
Reporter: Karthik Kambatla
Assignee: Tsuyoshi Ozawa
Attachments: YARN-2921.001.patch, YARN-2921.002.patch,
YARN-2921.003.patch, YARN-2921.004.patch
MockRM#waitForState methods currently sleep for too long (2 seconds and 1
second). This leads to slow tests and sometimes failures if the
App/AppAttempt moves to another state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2921) MockRM#waitForState methods can be too slow and flaky
[ https://issues.apache.org/jira/browse/YARN-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14524134#comment-14524134 ] Wangda Tan commented on YARN-2921: -- Rekicked Jenkins. MockRM#waitForState methods can be too slow and flaky - Key: YARN-2921 URL: https://issues.apache.org/jira/browse/YARN-2921 Project: Hadoop YARN Issue Type: Improvement Components: test Affects Versions: 2.6.0 Reporter: Karthik Kambatla Assignee: Tsuyoshi Ozawa Attachments: YARN-2921.001.patch, YARN-2921.002.patch, YARN-2921.003.patch, YARN-2921.004.patch MockRM#waitForState methods currently sleep for too long (2 seconds and 1 second). This leads to slow tests and sometimes failures if the App/AppAttempt moves to another state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2921) MockRM#waitForState methods can be too slow and flaky
[
https://issues.apache.org/jira/browse/YARN-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14312543#comment-14312543
]
Tsuyoshi OZAWA commented on YARN-2921:
--
The test failure looks not related to my patch. [~kasha], could you take a
look? This patch deals with tests can get too slow problem by shortening
polling period. This value looks minimum value of sleep to success the tests,
though I haven't inspected the reasons yet.
{quote}
e.g. App moving to RUNNING state when we are waiting for it to get ACCEPTED.
{quote}
I also found that it is a bit challenging to do this automatically - maybe
Graph class is useful to know all transition, but I haven't done this not yet.
MockRM#waitForState methods can be too slow and flaky
-
Key: YARN-2921
URL: https://issues.apache.org/jira/browse/YARN-2921
Project: Hadoop YARN
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0
Reporter: Karthik Kambatla
Assignee: Tsuyoshi OZAWA
Attachments: YARN-2921.001.patch, YARN-2921.002.patch,
YARN-2921.003.patch, YARN-2921.004.patch
MockRM#waitForState methods currently sleep for too long (2 seconds and 1
second). This leads to slow tests and sometimes failures if the
App/AppAttempt moves to another state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2921) MockRM#waitForState methods can be too slow and flaky
[
https://issues.apache.org/jira/browse/YARN-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14307942#comment-14307942
]
Hadoop QA commented on YARN-2921:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12696827/YARN-2921.004.patch
against trunk revision c4980a2.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestAllocationFileLoaderService
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6523//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6523//console
This message is automatically generated.
MockRM#waitForState methods can be too slow and flaky
-
Key: YARN-2921
URL: https://issues.apache.org/jira/browse/YARN-2921
Project: Hadoop YARN
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0
Reporter: Karthik Kambatla
Assignee: Tsuyoshi OZAWA
Attachments: YARN-2921.001.patch, YARN-2921.002.patch,
YARN-2921.003.patch, YARN-2921.004.patch
MockRM#waitForState methods currently sleep for too long (2 seconds and 1
second). This leads to slow tests and sometimes failures if the
App/AppAttempt moves to another state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2921) MockRM#waitForState methods can be too slow and flaky
[
https://issues.apache.org/jira/browse/YARN-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14236872#comment-14236872
]
Hadoop QA commented on YARN-2921:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12685534/YARN-2921.002.patch
against trunk revision e227fb8.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels
org.apache.hadoop.yarn.server.resourcemanager.security.TestAMRMTokens
org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
org.apache.hadoop.yarn.server.resourcemanager.TestApplicationCleanup
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueMappings
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched
org.apache.hadoop.yarn.server.resourcemanager.TestRMHA
org.apache.hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokens
org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
org.apache.hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
org.apache.hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer
org.apache.hadoop.yarn.server.resourcemanager.TestMoveApplication
org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes
The following test timeouts occurred in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart
org.apache.hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart
org.apache.hadoop.yarn.server.resourcemanager.TestRM
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs
org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterService
org.apache.hadoop.yarn.server.resourcemanager.TestFifoScheduler
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6021//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6021//console
This message is automatically generated.
MockRM#waitForState methods can be too slow and flaky
-
Key: YARN-2921
URL: https://issues.apache.org/jira/browse/YARN-2921
Project: Hadoop YARN
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0
Reporter: Karthik Kambatla
Assignee: Tsuyoshi OZAWA
Attachments: YARN-2921.001.patch, YARN-2921.002.patch
MockRM#waitForState methods currently sleep for too long (2 seconds and 1
second). This leads to slow tests and sometimes failures if the
App/AppAttempt moves to another state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2921) MockRM#waitForState methods can be too slow and flaky
[
https://issues.apache.org/jira/browse/YARN-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14236913#comment-14236913
]
Hadoop QA commented on YARN-2921:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12685553/YARN-2921.003.patch
against trunk revision e227fb8.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart
org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
org.apache.hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
org.apache.hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6023//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6023//console
This message is automatically generated.
MockRM#waitForState methods can be too slow and flaky
-
Key: YARN-2921
URL: https://issues.apache.org/jira/browse/YARN-2921
Project: Hadoop YARN
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0
Reporter: Karthik Kambatla
Assignee: Tsuyoshi OZAWA
Attachments: YARN-2921.001.patch, YARN-2921.002.patch,
YARN-2921.003.patch
MockRM#waitForState methods currently sleep for too long (2 seconds and 1
second). This leads to slow tests and sometimes failures if the
App/AppAttempt moves to another state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2921) MockRM#waitForState methods can be too slow and flaky
[
https://issues.apache.org/jira/browse/YARN-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14234540#comment-14234540
]
Tsuyoshi OZAWA commented on YARN-2921:
--
Thanks for your suggestion, Karthik. I agree with that we should use
wait-by-loop approach in this case. I'll update it in a next patch.
{quote}
Other than the smaller sleep, we should also handle the case where the App or
AppAttempt enters the required state and then moves to a latter state. e.g. App
moving to RUNNING state when we are waiting for it to get ACCEPTED.
{quote}
I spent some time how I can implement it - how about using variadic as a
argument of waitFor()? This will work without changing code.
{code}
public void waitForState(RMAppAttemptState... finalStatesArray)
throws Exception {
EnumSetRMAppAttemptState finalStates
= EnumSet.copyOf(Arrays.asList(finalStatesArray));
while (!finalStates.contains(attempt.getAppAttemptState()) !isTimeout() {
}
{code}
MockRM#waitForState methods can be too slow and flaky
-
Key: YARN-2921
URL: https://issues.apache.org/jira/browse/YARN-2921
Project: Hadoop YARN
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0
Reporter: Karthik Kambatla
Assignee: Tsuyoshi OZAWA
Attachments: YARN-2921.001.patch
MockRM#waitForState methods currently sleep for too long (2 seconds and 1
second). This leads to slow tests and sometimes failures if the
App/AppAttempt moves to another state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2921) MockRM#waitForState methods can be too slow and flaky
[ https://issues.apache.org/jira/browse/YARN-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14233741#comment-14233741 ] Karthik Kambatla commented on YARN-2921: Are 2 and 3 required here? MockRM#waitForState methods can be too slow and flaky - Key: YARN-2921 URL: https://issues.apache.org/jira/browse/YARN-2921 Project: Hadoop YARN Issue Type: Improvement Components: test Affects Versions: 2.6.0 Reporter: Karthik Kambatla Assignee: Tsuyoshi OZAWA Attachments: YARN-2921.001.patch MockRM#waitForState methods currently sleep for too long (2 seconds and 1 second). This leads to slow tests and sometimes failures if the App/AppAttempt moves to another state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2921) MockRM#waitForState methods can be too slow and flaky
[
https://issues.apache.org/jira/browse/YARN-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14233745#comment-14233745
]
Tsuyoshi OZAWA commented on YARN-2921:
--
If we use the CountDownLatch, we can understand the timeout value more easily:
{code}
latch.await(40, TimeUnit.SECONDS))
{code}
If I shouldn't do this here, I'll revert it.
MockRM#waitForState methods can be too slow and flaky
-
Key: YARN-2921
URL: https://issues.apache.org/jira/browse/YARN-2921
Project: Hadoop YARN
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0
Reporter: Karthik Kambatla
Assignee: Tsuyoshi OZAWA
Attachments: YARN-2921.001.patch
MockRM#waitForState methods currently sleep for too long (2 seconds and 1
second). This leads to slow tests and sometimes failures if the
App/AppAttempt moves to another state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2921) MockRM#waitForState methods can be too slow and flaky
[ https://issues.apache.org/jira/browse/YARN-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14233748#comment-14233748 ] Tsuyoshi OZAWA commented on YARN-2921: -- About the MockAM#waitForState, should we fix it on this JIRA? MockRM#waitForState methods can be too slow and flaky - Key: YARN-2921 URL: https://issues.apache.org/jira/browse/YARN-2921 Project: Hadoop YARN Issue Type: Improvement Components: test Affects Versions: 2.6.0 Reporter: Karthik Kambatla Assignee: Tsuyoshi OZAWA Attachments: YARN-2921.001.patch MockRM#waitForState methods currently sleep for too long (2 seconds and 1 second). This leads to slow tests and sometimes failures if the App/AppAttempt moves to another state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2921) MockRM#waitForState methods can be too slow and flaky
[ https://issues.apache.org/jira/browse/YARN-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14233751#comment-14233751 ] Karthik Kambatla commented on YARN-2921: Using the latch would make sense if the threads have any other advantage; otherwise, I would leave both out and control time waited through number of iterations of the for loop. Fixing MockAM#waitFor here would be good too. We should make sure the cumulative wait time stays at least as long as what it is now to avoid any test failures. MockRM#waitForState methods can be too slow and flaky - Key: YARN-2921 URL: https://issues.apache.org/jira/browse/YARN-2921 Project: Hadoop YARN Issue Type: Improvement Components: test Affects Versions: 2.6.0 Reporter: Karthik Kambatla Assignee: Tsuyoshi OZAWA Attachments: YARN-2921.001.patch MockRM#waitForState methods currently sleep for too long (2 seconds and 1 second). This leads to slow tests and sometimes failures if the App/AppAttempt moves to another state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2921) MockRM#waitForState methods can be too slow and flaky
[ https://issues.apache.org/jira/browse/YARN-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14233758#comment-14233758 ] Karthik Kambatla commented on YARN-2921: Other than the smaller sleep, we should also handle the case where the App or AppAttempt enters the required state and then moves to a latter state. e.g. App moving to RUNNING state when we are waiting for it to get ACCEPTED. MockRM#waitForState methods can be too slow and flaky - Key: YARN-2921 URL: https://issues.apache.org/jira/browse/YARN-2921 Project: Hadoop YARN Issue Type: Improvement Components: test Affects Versions: 2.6.0 Reporter: Karthik Kambatla Assignee: Tsuyoshi OZAWA Attachments: YARN-2921.001.patch MockRM#waitForState methods currently sleep for too long (2 seconds and 1 second). This leads to slow tests and sometimes failures if the App/AppAttempt moves to another state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
