[jira] [Comment Edited] (YARN-10447) TestLeafQueue: ActivitiesManager thread might interfere with ongoing stubbing
[
https://issues.apache.org/jira/browse/YARN-10447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203975#comment-17203975
]
Adam Antal edited comment on YARN-10447 at 9/29/20, 2:22 PM:
-
Thanks [~pbacsko] for the patch.
I think this patch does not work, because in {{CapacityScheduler.java}} you
have the following piece of code:
{code:java}
public void setActivitiesManagerEnabled(boolean enabled) {
this.activitiesManagerEnabled = true;
}
{code}
I think you meant {{this.activitiesManagerEnabled = enabled;}} there, is that
right?
On a sidenote, I can see a {{activitiesManager.start();}} guarded by the flag,
but there's no {{stop()}} call. Isn't that can cause memory leak?
was (Author: adam.antal):
Thanks [~pbacsko] for the patch.
I think this patch does not work, because in {{CapacityScheduler.java}} you
have the following piece of code:
{code:java}
public void setActivitiesManagerEnabled(boolean enabled) {
this.activitiesManagerEnabled = true;
}
{code}
I think you meant {{this.activitiesManagerEnabled = enabled; }} there, is that
right?
On a sidenote, I can see a {{activitiesManager.start();}} guarded by the flag,
but there's no {{stop()}} call. Isn't that can cause memory leak?
> TestLeafQueue: ActivitiesManager thread might interfere with ongoing stubbing
> -
>
> Key: YARN-10447
> URL: https://issues.apache.org/jira/browse/YARN-10447
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: YARN-10447-001.patch, YARN-10447-002.patch
>
>
> YARN-9784 fixed some concurrency related issues in {{TestLeafQueue}}, but not
> all of them. Occasionally it's still possible to receive an exception from
> Mockito and the two following stack traces can be observed in the console:
> {noformat}
>
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> Integer cannot be returned by isMultiNodePlacementEnabled()
> isMultiNodePlacementEnabled() should return boolean
> ***
> If you're unsure why you're getting above error read on.
> Due to the nature of the syntax above problem might occur because:
> 1. This exception *might* occur in wrongly written multi-threaded tests.
>Please refer to Mockito FAQ on limitations of concurrency testing.
> 2. A spy is stubbed using when(spy.foo()).then() syntax. It is safer to stub
> spies -
>- with doReturn|Throw() family of methods. More in javadocs for
> Mockito.spy() method.
> {noformat}
> or
> {noformat}
> 2020-09-22 14:44:52,584 INFO [main] capacity.TestUtils
> (TestUtils.java:getMockNode(227)) - node = 127.0.0.3 avail= vCores:1>
> 2020-09-22 14:44:52,585 INFO [main] capacity.TestUtils
> (TestUtils.java:getMockNode(227)) - node = 127.0.0.4 avail= vCores:1>
> Exception in thread "ActivitiesManager thread." java.lang.ClassCastException:
> java.lang.Integer cannot be cast to java.lang.Boolean
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler$$EnhancerByMockitoWithCGLIB$$272c72c5.isMultiNodePlacementEnabled()
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.activities.ActivitiesManager.dynamicallyUpdateAppActivitiesMaxQueueLengthIfNeeded(ActivitiesManager.java:266)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.activities.ActivitiesManager.access$500(ActivitiesManager.java:63)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.activities.ActivitiesManager$1.run(ActivitiesManager.java:347)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> It's probably best to disable ActivitiesManager thread entirely in this test
> class, there is no need for it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10447) TestLeafQueue: ActivitiesManager thread might interfere with ongoing stubbing
[
https://issues.apache.org/jira/browse/YARN-10447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203975#comment-17203975
]
Adam Antal edited comment on YARN-10447 at 9/29/20, 2:21 PM:
-
Thanks [~pbacsko] for the patch.
I think this patch does not work, because in {{CapacityScheduler.java}} you
have the following piece of code:
{code:java}
public void setActivitiesManagerEnabled(boolean enabled) {
this.activitiesManagerEnabled = true;
}
{code}
I think you meant {{ this.activitiesManagerEnabled = enabled; }} there, is
that right?
On a sidenote, I can see a {{activitiesManager.start();}} guarded by the flag,
but there's no {{stop()}} call. Isn't that can cause memory leak?
was (Author: adam.antal):
Thanks [~pbacsko] for the patch.
I think this patch does not work, because in {{CapacityScheduler.java}} you
have the following piece of code:
{code:java}
public void setActivitiesManagerEnabled(boolean enabled) {
this.activitiesManagerEnabled = true;
}
{code}
I think you meant {{ this.activitiesManagerEnabled = enabled;}} there, is that
right?
On a sidenote, I can see a {{activitiesManager.start();}} guarded by the flag,
but there's no {{stop()}} call. Isn't that can cause memory leak?
> TestLeafQueue: ActivitiesManager thread might interfere with ongoing stubbing
> -
>
> Key: YARN-10447
> URL: https://issues.apache.org/jira/browse/YARN-10447
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: YARN-10447-001.patch, YARN-10447-002.patch
>
>
> YARN-9784 fixed some concurrency related issues in {{TestLeafQueue}}, but not
> all of them. Occasionally it's still possible to receive an exception from
> Mockito and the two following stack traces can be observed in the console:
> {noformat}
>
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> Integer cannot be returned by isMultiNodePlacementEnabled()
> isMultiNodePlacementEnabled() should return boolean
> ***
> If you're unsure why you're getting above error read on.
> Due to the nature of the syntax above problem might occur because:
> 1. This exception *might* occur in wrongly written multi-threaded tests.
>Please refer to Mockito FAQ on limitations of concurrency testing.
> 2. A spy is stubbed using when(spy.foo()).then() syntax. It is safer to stub
> spies -
>- with doReturn|Throw() family of methods. More in javadocs for
> Mockito.spy() method.
> {noformat}
> or
> {noformat}
> 2020-09-22 14:44:52,584 INFO [main] capacity.TestUtils
> (TestUtils.java:getMockNode(227)) - node = 127.0.0.3 avail= vCores:1>
> 2020-09-22 14:44:52,585 INFO [main] capacity.TestUtils
> (TestUtils.java:getMockNode(227)) - node = 127.0.0.4 avail= vCores:1>
> Exception in thread "ActivitiesManager thread." java.lang.ClassCastException:
> java.lang.Integer cannot be cast to java.lang.Boolean
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler$$EnhancerByMockitoWithCGLIB$$272c72c5.isMultiNodePlacementEnabled()
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.activities.ActivitiesManager.dynamicallyUpdateAppActivitiesMaxQueueLengthIfNeeded(ActivitiesManager.java:266)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.activities.ActivitiesManager.access$500(ActivitiesManager.java:63)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.activities.ActivitiesManager$1.run(ActivitiesManager.java:347)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> It's probably best to disable ActivitiesManager thread entirely in this test
> class, there is no need for it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10447) TestLeafQueue: ActivitiesManager thread might interfere with ongoing stubbing
[
https://issues.apache.org/jira/browse/YARN-10447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203975#comment-17203975
]
Adam Antal edited comment on YARN-10447 at 9/29/20, 2:21 PM:
-
Thanks [~pbacsko] for the patch.
I think this patch does not work, because in {{CapacityScheduler.java}} you
have the following piece of code:
{code:java}
public void setActivitiesManagerEnabled(boolean enabled) {
this.activitiesManagerEnabled = true;
}
{code}
I think you meant {{this.activitiesManagerEnabled = enabled; }} there, is that
right?
On a sidenote, I can see a {{activitiesManager.start();}} guarded by the flag,
but there's no {{stop()}} call. Isn't that can cause memory leak?
was (Author: adam.antal):
Thanks [~pbacsko] for the patch.
I think this patch does not work, because in {{CapacityScheduler.java}} you
have the following piece of code:
{code:java}
public void setActivitiesManagerEnabled(boolean enabled) {
this.activitiesManagerEnabled = true;
}
{code}
I think you meant {{ this.activitiesManagerEnabled = enabled; }} there, is
that right?
On a sidenote, I can see a {{activitiesManager.start();}} guarded by the flag,
but there's no {{stop()}} call. Isn't that can cause memory leak?
> TestLeafQueue: ActivitiesManager thread might interfere with ongoing stubbing
> -
>
> Key: YARN-10447
> URL: https://issues.apache.org/jira/browse/YARN-10447
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: YARN-10447-001.patch, YARN-10447-002.patch
>
>
> YARN-9784 fixed some concurrency related issues in {{TestLeafQueue}}, but not
> all of them. Occasionally it's still possible to receive an exception from
> Mockito and the two following stack traces can be observed in the console:
> {noformat}
>
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> Integer cannot be returned by isMultiNodePlacementEnabled()
> isMultiNodePlacementEnabled() should return boolean
> ***
> If you're unsure why you're getting above error read on.
> Due to the nature of the syntax above problem might occur because:
> 1. This exception *might* occur in wrongly written multi-threaded tests.
>Please refer to Mockito FAQ on limitations of concurrency testing.
> 2. A spy is stubbed using when(spy.foo()).then() syntax. It is safer to stub
> spies -
>- with doReturn|Throw() family of methods. More in javadocs for
> Mockito.spy() method.
> {noformat}
> or
> {noformat}
> 2020-09-22 14:44:52,584 INFO [main] capacity.TestUtils
> (TestUtils.java:getMockNode(227)) - node = 127.0.0.3 avail= vCores:1>
> 2020-09-22 14:44:52,585 INFO [main] capacity.TestUtils
> (TestUtils.java:getMockNode(227)) - node = 127.0.0.4 avail= vCores:1>
> Exception in thread "ActivitiesManager thread." java.lang.ClassCastException:
> java.lang.Integer cannot be cast to java.lang.Boolean
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler$$EnhancerByMockitoWithCGLIB$$272c72c5.isMultiNodePlacementEnabled()
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.activities.ActivitiesManager.dynamicallyUpdateAppActivitiesMaxQueueLengthIfNeeded(ActivitiesManager.java:266)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.activities.ActivitiesManager.access$500(ActivitiesManager.java:63)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.activities.ActivitiesManager$1.run(ActivitiesManager.java:347)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> It's probably best to disable ActivitiesManager thread entirely in this test
> class, there is no need for it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
