In the output for a passing test, I saw: 2014-02-06 16:48:49,722 ERROR [Thread[Thread-71,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(557)) - InterruptedExcpetion recieved for ExpiredTokenRemover thread java.lang.InterruptedException: sleep interrupted
Meaning the above was not critical. branch-2.3 is receiving attention now. Discovering test failure there would be more helpful. Cheers On Thu, Feb 6, 2014 at 9:25 PM, Christopher Thomas < [email protected]> wrote: > I guess the ERROR lines in > > hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client- > jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2. > TestMRJobsWithHistoryService-output.txt > > led me to believe that the problem was with TestMRJobsWithHistoryService. > If that's not the case then what do these messages indicate? > > As I say I am a complete novice and the learning curve is very steep. > > > On 7 February 2014 14:47, Ted Yu <[email protected]> wrote: > >> I checked out source code from >> http://svn.apache.org/repos/asf/hadoop/common/branches/branch-2.3 and it >> builds. >> >> From TestMRJobsWithHistoryService.txt, the test passed. >> What led to this test being singled out among the 454 tests ? >> >> Thanks >> >> >> On Thu, Feb 6, 2014 at 7:26 PM, Christopher Thomas < >> [email protected]> wrote: >> >>> Yes well I tried 2.3, but I have found a number of problems building it. >>> I had to resort to manually applying patches that I found in the bug >>> tracking lists, which did not seem to have made it into all branches. So >>> for the moment I am sticking with 2.2.0 which is advertised as being >>> stable. >>> >>> I apologise for the confusion. >>> >>> Here is the contents >>> of >>> ./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService.txt, >>> though perhaps not that illuminating: >>> >>> >>> >>> ------------------------------------------------------------------------------- >>> Test set: org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService >>> >>> ------------------------------------------------------------------------------- >>> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 52.669 >>> sec >>> >>> >>> >>> >>> On 7 February 2014 14:12, Ted Yu <[email protected]> wrote: >>> >>>> The output was >>>> from >>>> hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService-output.txt >>>> >>>> Can you show us the contents >>>> of >>>> ./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService.txt >>>> ? >>>> >>>> BTW hadoop 2.3 release candidate is coming up. You may consider paying >>>> more attention to hadoop 2.3 >>>> >>>> Cheers >>>> >>>> >>>> On Thu, Feb 6, 2014 at 5:33 PM, Christopher Thomas < >>>> [email protected]> wrote: >>>> >>>>> I included the last part of >>>>> >>>>> hadoop-mapreduce-project/hadoop-mapreduce-client/ >>>>> hadoop-mapreduce-client-jobclient/target/surefire- >>>>> reports/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService. >>>>> txt >>>>> >>>>> in the second half of my initial posting, calling it "the output from >>>>> TestMRJobsWithHistoryService". >>>>> Sloppy terminology I know, sorry if I wasn't very clear. >>>>> >>>>> Regards >>>>> >>>>> Chris >>>>> >>>>> >>>>> On 7 February 2014 11:53, Ted Yu <[email protected]> wrote: >>>>> >>>>>> There isn't System.exit call in TestMRJobsWithHistoryService.java >>>>>> >>>>>> What >>>>>> did >>>>>> hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService.txt >>>>>> say ? >>>>>> >>>>>> Cheers >>>>>> >>>>>> >>>>>> On Thu, Feb 6, 2014 at 4:41 PM, Christopher Thomas < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I am a complete beginner to Hadoop, trying to build 2.2.0 from >>>>>>> source on a Macbook Pro running OS X Mavericks. >>>>>>> >>>>>>> I am following the 'instructions' at >>>>>>> http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SingleCluster.html >>>>>>> such >>>>>>> as they are. >>>>>>> >>>>>>> I get the following test failure: >>>>>>> >>>>>>> Forking command line: /bin/sh -c cd >>>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient >>>>>>> && >>>>>>> /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java >>>>>>> -Xmx1024m -XX:+HeapDumpOnOutOfMemoryError -jar >>>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire/surefirebooter1837947962445626736.jar >>>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire/surefire8537321961333093423tmp >>>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire/surefire_1668661782666182940486tmp >>>>>>> Running org.apache.hadoop.mapreduce.v2.TestMROldApiJobs >>>>>>> >>>>>>> Results : >>>>>>> >>>>>>> Tests run: 454, Failures: 0, Errors: 0, Skipped: 11 >>>>>>> >>>>>>> [INFO] >>>>>>> ------------------------------------------------------------------------ >>>>>>> [INFO] Reactor Summary: >>>>>>> [INFO] >>>>>>> [INFO] hadoop-mapreduce-client-jobclient ................. FAILURE >>>>>>> [1:18:07.440s] >>>>>>> [INFO] hadoop-mapreduce-client-hs-plugins ................ SKIPPED >>>>>>> [INFO] Apache Hadoop MapReduce Examples .................. SKIPPED >>>>>>> [INFO] hadoop-mapreduce .................................. SKIPPED >>>>>>> [INFO] >>>>>>> ------------------------------------------------------------------------ >>>>>>> [INFO] BUILD FAILURE >>>>>>> [INFO] >>>>>>> ------------------------------------------------------------------------ >>>>>>> [INFO] Total time: 1:18:07.828s >>>>>>> [INFO] Finished at: Thu Feb 06 23:01:45 EST 2014 >>>>>>> [INFO] Final Memory: 32M/94M >>>>>>> [INFO] >>>>>>> ------------------------------------------------------------------------ >>>>>>> [ERROR] Failed to execute goal >>>>>>> org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test >>>>>>> (default-test) >>>>>>> on project hadoop-mapreduce-client-jobclient: ExecutionException; nested >>>>>>> exception is java.util.concurrent.ExecutionException: >>>>>>> java.lang.RuntimeException: The forked VM terminated without saying >>>>>>> properly goodbye. VM crash or System.exit called ? >>>>>>> >>>>>>> >>>>>>> and the output from TestMRJobsWithHistoryService includes: >>>>>>> >>>>>>> >>>>>>> 2014-02-06 23:00:54,530 ERROR [Thread[Thread-252,5,main]] >>>>>>> delegation.AbstractDelegationTokenSecretManager >>>>>>> (AbstractDelegationTokenSecretManager.java:run(547)) - >>>>>>> InterruptedExcpetion >>>>>>> recieved for ExpiredTokenRemover thread java.lang.InterruptedException: >>>>>>> sleep interrupted >>>>>>> 2014-02-06 23:00:54,531 INFO [main] ipc.Server >>>>>>> (Server.java:stop(2437)) - Stopping server on 55131 >>>>>>> 2014-02-06 23:00:54,540 INFO [IPC Server listener on 55131] >>>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on >>>>>>> 55131 >>>>>>> 2014-02-06 23:00:54,540 INFO [IPC Server Responder] ipc.Server >>>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>>> 2014-02-06 23:00:54,544 INFO [main] mortbay.log >>>>>>> (Slf4jLog.java:info(67)) - Stopped >>>>>>> [email protected]:0 >>>>>>> 2014-02-06 23:00:54,647 INFO [main] hs.JobHistory >>>>>>> (JobHistory.java:serviceStop(136)) - Stopping JobHistory >>>>>>> 2014-02-06 23:00:54,647 INFO [main] hs.JobHistory >>>>>>> (JobHistory.java:serviceStop(138)) - Stopping History Cleaner/Move To >>>>>>> Done >>>>>>> 2014-02-06 23:00:54,656 INFO [main] mortbay.log >>>>>>> (Slf4jLog.java:info(67)) - Stopped >>>>>>> [email protected]:0 >>>>>>> 2014-02-06 23:00:54,759 INFO [main] >>>>>>> containermanager.ContainerManagerImpl >>>>>>> (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(328)) - >>>>>>> Applications still running : [application_1391688017341_0001] >>>>>>> 2014-02-06 23:00:54,762 INFO [main] >>>>>>> containermanager.ContainerManagerImpl >>>>>>> (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(336)) - >>>>>>> Waiting >>>>>>> for Applications to be Finished >>>>>>> 2014-02-06 23:00:54,762 INFO [AsyncDispatcher event handler] >>>>>>> application.Application (ApplicationImpl.java:handle(451)) - Application >>>>>>> application_1391688017341_0001 transitioned from RUNNING to >>>>>>> APPLICATION_RESOURCES_CLEANINGUP >>>>>>> 2014-02-06 23:00:54,763 INFO [DeletionService #3] >>>>>>> nodemanager.DefaultContainerExecutor >>>>>>> (DefaultContainerExecutor.java:deleteAsUser(368)) - Deleting absolute >>>>>>> path >>>>>>> : >>>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService-localDir-nm-0_0/usercache/root/appcache/application_1391688017341_0001 >>>>>>> 2014-02-06 23:00:54,763 INFO [DeletionService #0] >>>>>>> nodemanager.DefaultContainerExecutor >>>>>>> (DefaultContainerExecutor.java:deleteAsUser(368)) - Deleting absolute >>>>>>> path >>>>>>> : >>>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService-localDir-nm-0_1/usercache/root/appcache/application_1391688017341_0001 >>>>>>> 2014-02-06 23:00:54,765 INFO [AsyncDispatcher event handler] >>>>>>> containermanager.AuxServices (AuxServices.java:handle(175)) - Got event >>>>>>> APPLICATION_STOP for appId application_1391688017341_0001 >>>>>>> 2014-02-06 23:00:54,765 INFO [DeletionService #1] >>>>>>> nodemanager.DefaultContainerExecutor >>>>>>> (DefaultContainerExecutor.java:deleteAsUser(368)) - Deleting absolute >>>>>>> path >>>>>>> : >>>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService-localDir-nm-0_2/usercache/root/appcache/application_1391688017341_0001 >>>>>>> 2014-02-06 23:00:54,766 INFO [AsyncDispatcher event handler] >>>>>>> application.Application (ApplicationImpl.java:handle(451)) - Application >>>>>>> application_1391688017341_0001 transitioned from >>>>>>> APPLICATION_RESOURCES_CLEANINGUP to FINISHED >>>>>>> 2014-02-06 23:00:54,767 INFO [AsyncDispatcher event handler] >>>>>>> loghandler.NonAggregatingLogHandler >>>>>>> (NonAggregatingLogHandler.java:handle(118)) - Scheduling Log Deletion >>>>>>> for >>>>>>> application: application_1391688017341_0001, with delay of 10800 seconds >>>>>>> 2014-02-06 23:00:54,768 INFO [DeletionService #3] >>>>>>> nodemanager.DefaultContainerExecutor >>>>>>> (DefaultContainerExecutor.java:deleteAsUser(368)) - Deleting absolute >>>>>>> path >>>>>>> : >>>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService-localDir-nm-0_3/usercache/root/appcache/application_1391688017341_0001 >>>>>>> 2014-02-06 23:00:58,766 INFO [main] >>>>>>> containermanager.ContainerManagerImpl >>>>>>> (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(353)) - Done >>>>>>> waiting for Applications to be Finished. Still alive: >>>>>>> [application_1391688017341_0001] >>>>>>> 2014-02-06 23:00:58,768 INFO [main] ipc.Server >>>>>>> (Server.java:stop(2437)) - Stopping server on 55125 >>>>>>> 2014-02-06 23:00:58,771 INFO [IPC Server listener on 55125] >>>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on >>>>>>> 55125 >>>>>>> 2014-02-06 23:00:58,772 INFO [IPC Server Responder] ipc.Server >>>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>>> 2014-02-06 23:01:08,774 INFO [main] monitor.ContainersMonitorImpl >>>>>>> (ContainersMonitorImpl.java:isEnabled(159)) - ResourceCalculatorPlugin >>>>>>> is >>>>>>> unavailable on this system. >>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl >>>>>>> is disabled. >>>>>>> 2014-02-06 23:01:08,784 INFO [main] ipc.Server >>>>>>> (Server.java:stop(2437)) - Stopping server on 55126 >>>>>>> 2014-02-06 23:01:08,785 INFO [IPC Server listener on 55126] >>>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on >>>>>>> 55126 >>>>>>> 2014-02-06 23:01:08,785 INFO [IPC Server Responder] ipc.Server >>>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>>> 2014-02-06 23:01:08,785 INFO [Public Localizer] >>>>>>> localizer.ResourceLocalizationService >>>>>>> (ResourceLocalizationService.java:run(728)) - Public cache exiting >>>>>>> 2014-02-06 23:01:08,787 INFO [main] mortbay.log >>>>>>> (Slf4jLog.java:info(67)) - Stopped >>>>>>> [email protected]:0 >>>>>>> 2014-02-06 23:01:08,889 ERROR [Thread[Thread-17,5,main]] >>>>>>> delegation.AbstractDelegationTokenSecretManager >>>>>>> (AbstractDelegationTokenSecretManager.java:run(547)) - >>>>>>> InterruptedExcpetion >>>>>>> recieved for ExpiredTokenRemover thread java.lang.InterruptedException: >>>>>>> sleep interrupted >>>>>>> 2014-02-06 23:01:08,891 INFO [main] impl.MetricsSystemImpl >>>>>>> (MetricsSystemImpl.java:stop(200)) - Stopping JobHistoryServer metrics >>>>>>> system... >>>>>>> 2014-02-06 23:01:08,895 INFO [main] impl.MetricsSystemImpl >>>>>>> (MetricsSystemImpl.java:stop(206)) - JobHistoryServer metrics system >>>>>>> stopped. >>>>>>> 2014-02-06 23:01:08,895 INFO [main] impl.MetricsSystemImpl >>>>>>> (MetricsSystemImpl.java:shutdown(572)) - JobHistoryServer metrics system >>>>>>> shutdown complete. >>>>>>> 2014-02-06 23:01:08,896 WARN [ApplicationMaster Launcher] >>>>>>> amlauncher.ApplicationMasterLauncher >>>>>>> (ApplicationMasterLauncher.java:run(98)) - >>>>>>> org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher$LauncherThread >>>>>>> interrupted. Returning. >>>>>>> 2014-02-06 23:01:08,897 INFO [main] ipc.Server >>>>>>> (Server.java:stop(2437)) - Stopping server on 55122 >>>>>>> 2014-02-06 23:01:08,898 INFO [IPC Server listener on 55122] >>>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on >>>>>>> 55122 >>>>>>> 2014-02-06 23:01:08,898 INFO [main] ipc.Server >>>>>>> (Server.java:stop(2437)) - Stopping server on 55121 >>>>>>> 2014-02-06 23:01:08,898 INFO [IPC Server Responder] ipc.Server >>>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>>> 2014-02-06 23:01:08,902 INFO [IPC Server listener on 55121] >>>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on >>>>>>> 55121 >>>>>>> 2014-02-06 23:01:08,908 INFO [IPC Server Responder] ipc.Server >>>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>>> 2014-02-06 23:01:08,902 INFO [main] ipc.Server >>>>>>> (Server.java:stop(2437)) - Stopping server on 55120 >>>>>>> 2014-02-06 23:01:08,912 INFO [IPC Server listener on 55120] >>>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on >>>>>>> 55120 >>>>>>> 2014-02-06 23:01:08,912 INFO [IPC Server Responder] ipc.Server >>>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>>> 2014-02-06 23:01:08,912 INFO [main] ipc.Server >>>>>>> (Server.java:stop(2437)) - Stopping server on 55119 >>>>>>> 2014-02-06 23:01:08,917 INFO [IPC Server listener on 55119] >>>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on >>>>>>> 55119 >>>>>>> 2014-02-06 23:01:08,917 INFO [Ping Checker] >>>>>>> util.AbstractLivelinessMonitor >>>>>>> (AbstractLivelinessMonitor.java:run(127)) - >>>>>>> NMLivelinessMonitor thread interrupted >>>>>>> 2014-02-06 23:01:08,917 ERROR [ResourceManager Event Processor] >>>>>>> resourcemanager.ResourceManager (ResourceManager.java:run(435)) - >>>>>>> Returning, interrupted : java.lang.InterruptedException >>>>>>> 2014-02-06 23:01:08,917 INFO [IPC Server Responder] ipc.Server >>>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>>> 2014-02-06 23:01:08,918 INFO [Ping Checker] >>>>>>> util.AbstractLivelinessMonitor >>>>>>> (AbstractLivelinessMonitor.java:run(127)) - >>>>>>> AMLivelinessMonitor thread interrupted >>>>>>> 2014-02-06 23:01:08,918 INFO [Ping Checker] >>>>>>> util.AbstractLivelinessMonitor >>>>>>> (AbstractLivelinessMonitor.java:run(127)) - >>>>>>> org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer >>>>>>> thread interrupted >>>>>>> 2014-02-06 23:01:08,918 INFO [Ping Checker] >>>>>>> util.AbstractLivelinessMonitor >>>>>>> (AbstractLivelinessMonitor.java:run(127)) - >>>>>>> AMLivelinessMonitor thread interrupted >>>>>>> >>>>>>> Has anyone seen this error before when building hadoop, and does >>>>>>> anyone have any ideas? >>>>>>> >>>>>>> Regards >>>>>>> >>>>>>> Chris >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> >
