[jira] [Commented] (YARN-357) App submission should not be synchronized
[
https://issues.apache.org/jira/browse/YARN-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13573503#comment-13573503
]
Hudson commented on YARN-357:
-
Integrated in Hadoop-Mapreduce-trunk #1337 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1337/])
YARN-357. App submission should not be synchronized (daryn) (Revision
1443016)
Result = SUCCESS
daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1443016
Files :
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java
*
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java
App submission should not be synchronized
-
Key: YARN-357
URL: https://issues.apache.org/jira/browse/YARN-357
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 0.23.3, 3.0.0, 2.0.0-alpha
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Fix For: 3.0.0, 2.0.3-alpha, 0.23.7
Attachments: YARN-357.branch-23.patch, YARN-357.patch,
YARN-357.patch, YARN-357.txt
MAPREDUCE-2953 fixed a race condition with querying of app status by making
{{RMClientService#submitApplication}} synchronously invoke
{{RMAppManager#submitApplication}}. However, the {{synchronized}} keyword was
also added to {{RMAppManager#submitApplication}} with the comment:
bq. I made the submitApplication synchronized to keep it consistent with the
other routines in RMAppManager although I do not believe it needs it since
the rmapp datastructure is already a concurrentMap and I don't see anything
else that would be an issue.
It's been observed that app submission latency is being unnecessarily
impacted.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-357) App submission should not be synchronized
[
https://issues.apache.org/jira/browse/YARN-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13572545#comment-13572545
]
Hadoop QA commented on YARN-357:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12568245/YARN-357.branch-23.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/384//console
This message is automatically generated.
App submission should not be synchronized
-
Key: YARN-357
URL: https://issues.apache.org/jira/browse/YARN-357
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 0.23.3, 3.0.0, 2.0.0-alpha
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Attachments: YARN-357.branch-23.patch, YARN-357.patch,
YARN-357.patch, YARN-357.txt
MAPREDUCE-2953 fixed a race condition with querying of app status by making
{{RMClientService#submitApplication}} synchronously invoke
{{RMAppManager#submitApplication}}. However, the {{synchronized}} keyword was
also added to {{RMAppManager#submitApplication}} with the comment:
bq. I made the submitApplication synchronized to keep it consistent with the
other routines in RMAppManager although I do not believe it needs it since
the rmapp datastructure is already a concurrentMap and I don't see anything
else that would be an issue.
It's been observed that app submission latency is being unnecessarily
impacted.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-357) App submission should not be synchronized
[
https://issues.apache.org/jira/browse/YARN-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13571361#comment-13571361
]
Daryn Sharp commented on YARN-357:
--
Thanks for the review! I struggled quite a bit with writing these tests. I am
indeed expecting the barrier to cause the test to timeout. How do you suggest
I add an explicit error? Log the assert since there's not much else I can
(easily) do?
I couldn't get chaining to work with void return methods. The mock hangs on a
poll for repeated calls to a method with a doAnswer if there aren't more
doAnswers registered with it, but since I couldn't get the chaining to work
up-front, I added the chain after the method is hit the first time.
How would a custom event handler work better in this case? I think it'd just
mimic the current behavior? Could we maybe have the separate jira, for
removing other syncs, improve the tests in general? Or could you help me out
by updating the tests since I'm unclear what to do?
App submission should not be synchronized
-
Key: YARN-357
URL: https://issues.apache.org/jira/browse/YARN-357
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 0.23.3, 3.0.0, 2.0.0-alpha
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Attachments: YARN-357.patch, YARN-357.patch
MAPREDUCE-2953 fixed a race condition with querying of app status by making
{{RMClientService#submitApplication}} synchronously invoke
{{RMAppManager#submitApplication}}. However, the {{synchronized}} keyword was
also added to {{RMAppManager#submitApplication}} with the comment:
bq. I made the submitApplication synchronized to keep it consistent with the
other routines in RMAppManager although I do not believe it needs it since
the rmapp datastructure is already a concurrentMap and I don't see anything
else that would be an issue.
It's been observed that app submission latency is being unnecessarily
impacted.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-357) App submission should not be synchronized
[
https://issues.apache.org/jira/browse/YARN-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13571423#comment-13571423
]
Siddharth Seth commented on YARN-357:
-
bq. Thanks for the review! I struggled quite a bit with writing these tests. I
am indeed expecting the barrier to cause the test to timeout. How do you
suggest I add an explicit error? Log the assert since there's not much else I
can (easily) do?
A flag to track the error in the thread should work ?
I couldn't get chaining to work with void return methods. The mock hangs on a
poll for repeated calls to a method with a doAnswer if there aren't more
doAnswers registered with it, but since I couldn't get the chaining to work
up-front, I added the chain after the method is hit the first time.
bq. Chaining seemed to work ok, as long as the previous invocation had
completed. Even without chaining, the mock does not seem to handle multiple
parallel invocations. Struggled a bit with this as well - which is what makes
me a little wary of using mockito for this case
(http://code.google.com/p/mockito/wiki/FAQ).
bq. How would a custom event handler work better in this case? I think it'd
just mimic the current behavior? Could we maybe have the separate jira, for
removing other syncs, improve the tests in general? Or could you help me out by
updating the tests since I'm unclear what to do?
It'd behave the same, except would not use mockito for the EventHandler mock.
Sure, I'll try updating the unit test.
App submission should not be synchronized
-
Key: YARN-357
URL: https://issues.apache.org/jira/browse/YARN-357
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 0.23.3, 3.0.0, 2.0.0-alpha
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Attachments: YARN-357.patch, YARN-357.patch
MAPREDUCE-2953 fixed a race condition with querying of app status by making
{{RMClientService#submitApplication}} synchronously invoke
{{RMAppManager#submitApplication}}. However, the {{synchronized}} keyword was
also added to {{RMAppManager#submitApplication}} with the comment:
bq. I made the submitApplication synchronized to keep it consistent with the
other routines in RMAppManager although I do not believe it needs it since
the rmapp datastructure is already a concurrentMap and I don't see anything
else that would be an issue.
It's been observed that app submission latency is being unnecessarily
impacted.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-357) App submission should not be synchronized
[
https://issues.apache.org/jira/browse/YARN-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13565499#comment-13565499
]
Daryn Sharp commented on YARN-357:
--
Yes, the test failed on a non-patched trunk build.
App submission should not be synchronized
-
Key: YARN-357
URL: https://issues.apache.org/jira/browse/YARN-357
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 0.23.3, 3.0.0, 2.0.0-alpha
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Attachments: YARN-357.patch, YARN-357.patch
MAPREDUCE-2953 fixed a race condition with querying of app status by making
{{RMClientService#submitApplication}} synchronously invoke
{{RMAppManager#submitApplication}}. However, the {{synchronized}} keyword was
also added to {{RMAppManager#submitApplication}} with the comment:
bq. I made the submitApplication synchronized to keep it consistent with the
other routines in RMAppManager although I do not believe it needs it since
the rmapp datastructure is already a concurrentMap and I don't see anything
else that would be an issue.
It's been observed that app submission latency is being unnecessarily
impacted.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-357) App submission should not be synchronized
[
https://issues.apache.org/jira/browse/YARN-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13564688#comment-13564688
]
Jason Lowe commented on YARN-357:
-
TestRMWebServices failure is likely unrelated, see HADOOP-9246.
App submission should not be synchronized
-
Key: YARN-357
URL: https://issues.apache.org/jira/browse/YARN-357
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 0.23.3, 3.0.0, 2.0.0-alpha
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Attachments: YARN-357.patch, YARN-357.patch
MAPREDUCE-2953 fixed a race condition with querying of app status by making
{{RMClientService#submitApplication}} synchronously invoke
{{RMAppManager#submitApplication}}. However, the {{synchronized}} keyword was
also added to {{RMAppManager#submitApplication}} with the comment:
bq. I made the submitApplication synchronized to keep it consistent with the
other routines in RMAppManager although I do not believe it needs it since
the rmapp datastructure is already a concurrentMap and I don't see anything
else that would be an issue.
It's been observed that app submission latency is being unnecessarily
impacted.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-357) App submission should not be synchronized
[
https://issues.apache.org/jira/browse/YARN-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13561893#comment-13561893
]
Hadoop QA commented on YARN-357:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12566348/YARN-357.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 eclipse:eclipse{color}. The patch failed to build with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/364//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/364//console
This message is automatically generated.
App submission should not be synchronized
-
Key: YARN-357
URL: https://issues.apache.org/jira/browse/YARN-357
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 0.23.3, 3.0.0, 2.0.0-alpha
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Attachments: YARN-357.patch
MAPREDUCE-2953 fixed a race condition with querying of app status by making
{{RMClientService#submitApplication}} synchronously invoke
{{RMAppManager#submitApplication}}. However, the {{synchronized}} keyword was
also added to {{RMAppManager#submitApplication}} with the comment:
bq. I made the submitApplication synchronized to keep it consistent with the
other routines in RMAppManager although I do not believe it needs it since
the rmapp datastructure is already a concurrentMap and I don't see anything
else that would be an issue.
It's been observed that app submission latency is being unnecessarily
impacted.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
