[
https://issues.apache.org/jira/browse/YARN-530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated YARN-530:
--------------------------------
Attachment: YARN-530-021.patch
This patch manages all the MR services properly -the change was trivial, once
tracked down. Rather than synchronize on the service itself for state changes,
we synchronize on a specific {{stateChangeLock}} -ensures that this doesn't
create any locks elsewhere.
When applied on its own, the YARN changes will successfully run all the
MapReduce tests except for two in {{TestStagingCleanup}} which are actually due
to {{ContainerLauncherRouter}} not checking {{containerLauncher}} for nullness
before stopping -and the shutdown hook calling it in INITED state, not STARTED
-i.e. is due to the stop() code not being robust enough to stop when not
started. That's fixed in the main MapReduce patch, because it does add the null
reference checks
{code}
testDeletionofStagingOnKill(org.apache.hadoop.mapreduce.v2.app.TestStagingCleanup)
Time elapsed: 57 sec <<< ERROR!
java.lang.NullPointerException
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter.stop(MRAppMaster.java:865)
at
org.apache.hadoop.yarn.service.ServiceOperations.stop(ServiceOperations.java:51)
at
org.apache.hadoop.yarn.service.ServiceOperations.stopQuietly(ServiceOperations.java:79)
at
org.apache.hadoop.yarn.service.CompositeService.stop(CompositeService.java:135)
at
org.apache.hadoop.yarn.service.CompositeService.serviceStop(CompositeService.java:108)
at
org.apache.hadoop.yarn.service.AbstractService.stop(AbstractService.java:216)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$MRAppMasterShutdownHook.run(MRAppMaster.java:1350)
at
org.apache.hadoop.mapreduce.v2.app.TestStagingCleanup.testDeletionofStagingOnKill(TestStagingCleanup.java:167)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at
org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
{code}
> Define Service model strictly, implement AbstractService for robust
> subclassing, migrate yarn-common services
> -------------------------------------------------------------------------------------------------------------
>
> Key: YARN-530
> URL: https://issues.apache.org/jira/browse/YARN-530
> Project: Hadoop YARN
> Issue Type: Sub-task
> Affects Versions: 2.0.4-alpha
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: YARN-117-019.patch, YARN-117changes.pdf,
> YARN-530-005.patch, YARN-530-008.patch, YARN-530-009.patch,
> YARN-530-010.patch, YARN-530-011.patch, YARN-530-012.patch,
> YARN-530-013.patch, YARN-530-014.patch, YARN-530-015.patch,
> YARN-530-016.patch, YARN-530-017.patch, YARN-530-018.patch,
> YARN-530-019.patch, YARN-530-020.patch, YARN-530-021.patch, YARN-530-2.patch,
> YARN-530-3.patch, YARN-530.4.patch, YARN-530.patch
>
>
> # Extend the YARN {{Service}} interface as discussed in YARN-117
> # Implement the changes in {{AbstractService}} and {{FilterService}}.
> # Migrate all services in yarn-common to the more robust service model, test.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
