[
https://issues.apache.org/jira/browse/MAPREDUCE-7139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleksandr Shevchenko updated MAPREDUCE-7139:
--------------------------------------------
Status: Open (was: Patch Available)
> TestShuffleProvider#testShuffleProviders and
> TestTaskAttemptContainerRequest#testAttemptContainerRequest fail since a
> static common container launch context does't recreate
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-7139
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7139
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Reporter: Oleksandr Shevchenko
> Assignee: Oleksandr Shevchenko
> Priority: Minor
> Attachments: MAPREDUCE-7139.001.patch, screenshot-1.png,
> screenshot-2.png
>
>
> TestShuffleProvider#testShuffleProviders fails with the following error:
> {noformat}
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertNotNull(Assert.java:621)
> at
> org.apache.hadoop.mapreduce.v2.app.job.impl.TestShuffleProvider.testShuffleProviders(TestShuffleProvider.java:114)
> {noformat}
> This error occurs only when we run TestShuffleProvider#testShuffleProviders
> test after some test which creates TaskAttemptImpl (For example after tests
> in TestMapReduceChildJVM). !screenshot-1.png!
> TestShuffleProvider#testShuffleProviders creates container launch context,
> set additionals NM aux services and expects to get these services from
> container launch context after creating.
> But we have the following code in TaskAttemptImpl:
> {code}
> static ContainerLaunchContext createContainerLaunchContext(
> Map<ApplicationAccessType, String> applicationACLs,
> Configuration conf, Token<JobTokenIdentifier> jobToken, Task remoteTask,
> final org.apache.hadoop.mapred.JobID oldJobId,
> WrappedJvmID jvmID,
> TaskAttemptListener taskAttemptListener,
> Credentials credentials) {
> synchronized (commonContainerSpecLock) {
> if (commonContainerSpec == null) {
> commonContainerSpec = createCommonContainerLaunchContext(
> applicationACLs, conf, jobToken, oldJobId, credentials);
> }
> }
> {code}
> The property "commonContainerSpec" is static. As the result, we do not
> recreate common container launch within JVM. This is normal for AM JVM since
> all containers should be run with one context. But this leads to an error
> when we run unit tests in one JVM and expect to recreate container launch
> context.
> We can add a setter for "commonContainerSpec" with "VisibleForTesting"
> annotation and reset CLC in a test if we need it.
> Also, we can cleanup CLC after running tests in @After fixture which create
> CLC but we have too many tests which use TaskAttemptImpl and do not need
> cleanup.
> The same issue occurre in
> TestTaskAttemptContainerRequest#testAttemptContainerRequest.
> !screenshot-2.png! Since we create CLC by
> TaskAttemptImpl.createContainerLaunchContext.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
