[
https://issues.apache.org/jira/browse/MAPREDUCE-7139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleksandr Shevchenko updated MAPREDUCE-7139:
--------------------------------------------
Attachment: screenshot-1.png
> TestShuffleProvider#testShuffleProviders fails since a static common
> container launch context does't recreate
> -------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-7139
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7139
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Reporter: Oleksandr Shevchenko
> Assignee: Oleksandr Shevchenko
> Priority: Minor
> Attachments: MAPREDUCE-7139.001.patch, screenshot-1.png
>
>
> TestShuffleProvider#testShuffleProviders fails with the following error:
> {noformat}
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertNotNull(Assert.java:621)
> at
> org.apache.hadoop.mapreduce.v2.app.job.impl.TestShuffleProvider.testShuffleProviders(TestShuffleProvider.java:114)
> {noformat}
> This error occurs only when we run TestShuffleProvider#testShuffleProviders
> test after some test which creates TaskAttemptImpl (For example after tests
> in TestMapReduceChildJVM). TestShuffleProvider#testShuffleProviders creates
> container launch context, set additionals NM aux services and expects to get
> these services from container launch context after creating.
> But we have the following code in TaskAttemptImpl:
> {code}
> static ContainerLaunchContext createContainerLaunchContext(
> Map<ApplicationAccessType, String> applicationACLs,
> Configuration conf, Token<JobTokenIdentifier> jobToken, Task remoteTask,
> final org.apache.hadoop.mapred.JobID oldJobId,
> WrappedJvmID jvmID,
> TaskAttemptListener taskAttemptListener,
> Credentials credentials) {
> synchronized (commonContainerSpecLock) {
> if (commonContainerSpec == null) {
> commonContainerSpec = createCommonContainerLaunchContext(
> applicationACLs, conf, jobToken, oldJobId, credentials);
> }
> }
> {code}
> The property "commonContainerSpec" is static. As the result, we do not
> recreate common container launch within JVM. This is normal for AM JVM since
> all containers should be run with one context. But this leads to an error
> when we run unit tests in one JVM and expect to recreate container launch
> context.
> We can add a setter for "commonContainerSpec" with "VisibleForTesting"
> annotation and reset CLC in a test if we need it.
> Also, we can cleanup CLC after running tests in @After fixture which create
> CLC but we have too many tests which use TaskAttemptImpl and do not need
> cleanup.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
