Thanks, I built 2.3 yesterday (checked out from from link suggested in earlier post of this thread) without problems apart from VM running out of memory which was fixed with
export MAVEN_OPTS=-Xmx2048m At least, I got a message saying successful build. Thanks for your help. On 8 February 2014 10:53, Ted Yu <[email protected]> wrote: > In the output for a passing test, I saw: > > 2014-02-06 16:48:49,722 ERROR [Thread[Thread-71,5,main]] > delegation.AbstractDelegationTokenSecretManager > (AbstractDelegationTokenSecretManager.java:run(557)) - > InterruptedExcpetion recieved for ExpiredTokenRemover thread > java.lang.InterruptedException: sleep interrupted > > Meaning the above was not critical. > > branch-2.3 is receiving attention now. > Discovering test failure there would be more helpful. > > Cheers > > > On Thu, Feb 6, 2014 at 9:25 PM, Christopher Thomas < > [email protected]> wrote: > >> I guess the ERROR lines in >> >> hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client- >> jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2. >> TestMRJobsWithHistoryService-output.txt >> >> led me to believe that the problem was with TestMRJobsWithHistoryService. >> If that's not the case then what do these messages indicate? >> >> As I say I am a complete novice and the learning curve is very steep. >> >> >> On 7 February 2014 14:47, Ted Yu <[email protected]> wrote: >> >>> I checked out source code from >>> http://svn.apache.org/repos/asf/hadoop/common/branches/branch-2.3 and >>> it builds. >>> >>> From TestMRJobsWithHistoryService.txt, the test passed. >>> What led to this test being singled out among the 454 tests ? >>> >>> Thanks >>> >>> >>> On Thu, Feb 6, 2014 at 7:26 PM, Christopher Thomas < >>> [email protected]> wrote: >>> >>>> Yes well I tried 2.3, but I have found a number of problems building >>>> it. I had to resort to manually applying patches that I found in the bug >>>> tracking lists, which did not seem to have made it into all branches. So >>>> for the moment I am sticking with 2.2.0 which is advertised as being >>>> stable. >>>> >>>> I apologise for the confusion. >>>> >>>> Here is the contents >>>> of >>>> ./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService.txt, >>>> though perhaps not that illuminating: >>>> >>>> >>>> >>>> ------------------------------------------------------------------------------- >>>> Test set: org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService >>>> >>>> ------------------------------------------------------------------------------- >>>> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 52.669 >>>> sec >>>> >>>> >>>> >>>> >>>> On 7 February 2014 14:12, Ted Yu <[email protected]> wrote: >>>> >>>>> The output was >>>>> from >>>>> hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService-output.txt >>>>> >>>>> Can you show us the contents >>>>> of >>>>> ./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService.txt >>>>> ? >>>>> >>>>> BTW hadoop 2.3 release candidate is coming up. You may consider paying >>>>> more attention to hadoop 2.3 >>>>> >>>>> Cheers >>>>> >>>>> >>>>> On Thu, Feb 6, 2014 at 5:33 PM, Christopher Thomas < >>>>> [email protected]> wrote: >>>>> >>>>>> I included the last part of >>>>>> >>>>>> hadoop-mapreduce-project/hadoop-mapreduce-client/ >>>>>> hadoop-mapreduce-client-jobclient/target/surefire- >>>>>> reports/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService. >>>>>> txt >>>>>> >>>>>> in the second half of my initial posting, calling it "the output >>>>>> from TestMRJobsWithHistoryService". Sloppy terminology I know, sorry >>>>>> if I wasn't very clear. >>>>>> >>>>>> Regards >>>>>> >>>>>> Chris >>>>>> >>>>>> >>>>>> On 7 February 2014 11:53, Ted Yu <[email protected]> wrote: >>>>>> >>>>>>> There isn't System.exit call in TestMRJobsWithHistoryService.java >>>>>>> >>>>>>> What >>>>>>> did >>>>>>> hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService.txt >>>>>>> say ? >>>>>>> >>>>>>> Cheers >>>>>>> >>>>>>> >>>>>>> On Thu, Feb 6, 2014 at 4:41 PM, Christopher Thomas < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I am a complete beginner to Hadoop, trying to build 2.2.0 from >>>>>>>> source on a Macbook Pro running OS X Mavericks. >>>>>>>> >>>>>>>> I am following the 'instructions' at >>>>>>>> http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SingleCluster.html >>>>>>>> such >>>>>>>> as they are. >>>>>>>> >>>>>>>> I get the following test failure: >>>>>>>> >>>>>>>> Forking command line: /bin/sh -c cd >>>>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient >>>>>>>> && >>>>>>>> /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java >>>>>>>> -Xmx1024m -XX:+HeapDumpOnOutOfMemoryError -jar >>>>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire/surefirebooter1837947962445626736.jar >>>>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire/surefire8537321961333093423tmp >>>>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire/surefire_1668661782666182940486tmp >>>>>>>> Running org.apache.hadoop.mapreduce.v2.TestMROldApiJobs >>>>>>>> >>>>>>>> Results : >>>>>>>> >>>>>>>> Tests run: 454, Failures: 0, Errors: 0, Skipped: 11 >>>>>>>> >>>>>>>> [INFO] >>>>>>>> ------------------------------------------------------------------------ >>>>>>>> [INFO] Reactor Summary: >>>>>>>> [INFO] >>>>>>>> [INFO] hadoop-mapreduce-client-jobclient ................. FAILURE >>>>>>>> [1:18:07.440s] >>>>>>>> [INFO] hadoop-mapreduce-client-hs-plugins ................ SKIPPED >>>>>>>> [INFO] Apache Hadoop MapReduce Examples .................. SKIPPED >>>>>>>> [INFO] hadoop-mapreduce .................................. SKIPPED >>>>>>>> [INFO] >>>>>>>> ------------------------------------------------------------------------ >>>>>>>> [INFO] BUILD FAILURE >>>>>>>> [INFO] >>>>>>>> ------------------------------------------------------------------------ >>>>>>>> [INFO] Total time: 1:18:07.828s >>>>>>>> [INFO] Finished at: Thu Feb 06 23:01:45 EST 2014 >>>>>>>> [INFO] Final Memory: 32M/94M >>>>>>>> [INFO] >>>>>>>> ------------------------------------------------------------------------ >>>>>>>> [ERROR] Failed to execute goal >>>>>>>> org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test >>>>>>>> (default-test) >>>>>>>> on project hadoop-mapreduce-client-jobclient: ExecutionException; >>>>>>>> nested >>>>>>>> exception is java.util.concurrent.ExecutionException: >>>>>>>> java.lang.RuntimeException: The forked VM terminated without saying >>>>>>>> properly goodbye. VM crash or System.exit called ? >>>>>>>> >>>>>>>> >>>>>>>> and the output from TestMRJobsWithHistoryService includes: >>>>>>>> >>>>>>>> >>>>>>>> 2014-02-06 23:00:54,530 ERROR [Thread[Thread-252,5,main]] >>>>>>>> delegation.AbstractDelegationTokenSecretManager >>>>>>>> (AbstractDelegationTokenSecretManager.java:run(547)) - >>>>>>>> InterruptedExcpetion >>>>>>>> recieved for ExpiredTokenRemover thread java.lang.InterruptedException: >>>>>>>> sleep interrupted >>>>>>>> 2014-02-06 23:00:54,531 INFO [main] ipc.Server >>>>>>>> (Server.java:stop(2437)) - Stopping server on 55131 >>>>>>>> 2014-02-06 23:00:54,540 INFO [IPC Server listener on 55131] >>>>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on >>>>>>>> 55131 >>>>>>>> 2014-02-06 23:00:54,540 INFO [IPC Server Responder] ipc.Server >>>>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>>>> 2014-02-06 23:00:54,544 INFO [main] mortbay.log >>>>>>>> (Slf4jLog.java:info(67)) - Stopped >>>>>>>> [email protected]:0 >>>>>>>> 2014-02-06 23:00:54,647 INFO [main] hs.JobHistory >>>>>>>> (JobHistory.java:serviceStop(136)) - Stopping JobHistory >>>>>>>> 2014-02-06 23:00:54,647 INFO [main] hs.JobHistory >>>>>>>> (JobHistory.java:serviceStop(138)) - Stopping History Cleaner/Move To >>>>>>>> Done >>>>>>>> 2014-02-06 23:00:54,656 INFO [main] mortbay.log >>>>>>>> (Slf4jLog.java:info(67)) - Stopped >>>>>>>> [email protected]:0 >>>>>>>> 2014-02-06 23:00:54,759 INFO [main] >>>>>>>> containermanager.ContainerManagerImpl >>>>>>>> (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(328)) - >>>>>>>> Applications still running : [application_1391688017341_0001] >>>>>>>> 2014-02-06 23:00:54,762 INFO [main] >>>>>>>> containermanager.ContainerManagerImpl >>>>>>>> (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(336)) - >>>>>>>> Waiting >>>>>>>> for Applications to be Finished >>>>>>>> 2014-02-06 23:00:54,762 INFO [AsyncDispatcher event handler] >>>>>>>> application.Application (ApplicationImpl.java:handle(451)) - >>>>>>>> Application >>>>>>>> application_1391688017341_0001 transitioned from RUNNING to >>>>>>>> APPLICATION_RESOURCES_CLEANINGUP >>>>>>>> 2014-02-06 23:00:54,763 INFO [DeletionService #3] >>>>>>>> nodemanager.DefaultContainerExecutor >>>>>>>> (DefaultContainerExecutor.java:deleteAsUser(368)) - Deleting absolute >>>>>>>> path >>>>>>>> : >>>>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService-localDir-nm-0_0/usercache/root/appcache/application_1391688017341_0001 >>>>>>>> 2014-02-06 23:00:54,763 INFO [DeletionService #0] >>>>>>>> nodemanager.DefaultContainerExecutor >>>>>>>> (DefaultContainerExecutor.java:deleteAsUser(368)) - Deleting absolute >>>>>>>> path >>>>>>>> : >>>>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService-localDir-nm-0_1/usercache/root/appcache/application_1391688017341_0001 >>>>>>>> 2014-02-06 23:00:54,765 INFO [AsyncDispatcher event handler] >>>>>>>> containermanager.AuxServices (AuxServices.java:handle(175)) - Got event >>>>>>>> APPLICATION_STOP for appId application_1391688017341_0001 >>>>>>>> 2014-02-06 23:00:54,765 INFO [DeletionService #1] >>>>>>>> nodemanager.DefaultContainerExecutor >>>>>>>> (DefaultContainerExecutor.java:deleteAsUser(368)) - Deleting absolute >>>>>>>> path >>>>>>>> : >>>>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService-localDir-nm-0_2/usercache/root/appcache/application_1391688017341_0001 >>>>>>>> 2014-02-06 23:00:54,766 INFO [AsyncDispatcher event handler] >>>>>>>> application.Application (ApplicationImpl.java:handle(451)) - >>>>>>>> Application >>>>>>>> application_1391688017341_0001 transitioned from >>>>>>>> APPLICATION_RESOURCES_CLEANINGUP to FINISHED >>>>>>>> 2014-02-06 23:00:54,767 INFO [AsyncDispatcher event handler] >>>>>>>> loghandler.NonAggregatingLogHandler >>>>>>>> (NonAggregatingLogHandler.java:handle(118)) - Scheduling Log Deletion >>>>>>>> for >>>>>>>> application: application_1391688017341_0001, with delay of 10800 >>>>>>>> seconds >>>>>>>> 2014-02-06 23:00:54,768 INFO [DeletionService #3] >>>>>>>> nodemanager.DefaultContainerExecutor >>>>>>>> (DefaultContainerExecutor.java:deleteAsUser(368)) - Deleting absolute >>>>>>>> path >>>>>>>> : >>>>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService-localDir-nm-0_3/usercache/root/appcache/application_1391688017341_0001 >>>>>>>> 2014-02-06 23:00:58,766 INFO [main] >>>>>>>> containermanager.ContainerManagerImpl >>>>>>>> (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(353)) - Done >>>>>>>> waiting for Applications to be Finished. Still alive: >>>>>>>> [application_1391688017341_0001] >>>>>>>> 2014-02-06 23:00:58,768 INFO [main] ipc.Server >>>>>>>> (Server.java:stop(2437)) - Stopping server on 55125 >>>>>>>> 2014-02-06 23:00:58,771 INFO [IPC Server listener on 55125] >>>>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on >>>>>>>> 55125 >>>>>>>> 2014-02-06 23:00:58,772 INFO [IPC Server Responder] ipc.Server >>>>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>>>> 2014-02-06 23:01:08,774 INFO [main] monitor.ContainersMonitorImpl >>>>>>>> (ContainersMonitorImpl.java:isEnabled(159)) - ResourceCalculatorPlugin >>>>>>>> is >>>>>>>> unavailable on this system. >>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl >>>>>>>> is disabled. >>>>>>>> 2014-02-06 23:01:08,784 INFO [main] ipc.Server >>>>>>>> (Server.java:stop(2437)) - Stopping server on 55126 >>>>>>>> 2014-02-06 23:01:08,785 INFO [IPC Server listener on 55126] >>>>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on >>>>>>>> 55126 >>>>>>>> 2014-02-06 23:01:08,785 INFO [IPC Server Responder] ipc.Server >>>>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>>>> 2014-02-06 23:01:08,785 INFO [Public Localizer] >>>>>>>> localizer.ResourceLocalizationService >>>>>>>> (ResourceLocalizationService.java:run(728)) - Public cache exiting >>>>>>>> 2014-02-06 23:01:08,787 INFO [main] mortbay.log >>>>>>>> (Slf4jLog.java:info(67)) - Stopped >>>>>>>> [email protected]:0 >>>>>>>> 2014-02-06 23:01:08,889 ERROR [Thread[Thread-17,5,main]] >>>>>>>> delegation.AbstractDelegationTokenSecretManager >>>>>>>> (AbstractDelegationTokenSecretManager.java:run(547)) - >>>>>>>> InterruptedExcpetion >>>>>>>> recieved for ExpiredTokenRemover thread java.lang.InterruptedException: >>>>>>>> sleep interrupted >>>>>>>> 2014-02-06 23:01:08,891 INFO [main] impl.MetricsSystemImpl >>>>>>>> (MetricsSystemImpl.java:stop(200)) - Stopping JobHistoryServer metrics >>>>>>>> system... >>>>>>>> 2014-02-06 23:01:08,895 INFO [main] impl.MetricsSystemImpl >>>>>>>> (MetricsSystemImpl.java:stop(206)) - JobHistoryServer metrics system >>>>>>>> stopped. >>>>>>>> 2014-02-06 23:01:08,895 INFO [main] impl.MetricsSystemImpl >>>>>>>> (MetricsSystemImpl.java:shutdown(572)) - JobHistoryServer metrics >>>>>>>> system >>>>>>>> shutdown complete. >>>>>>>> 2014-02-06 23:01:08,896 WARN [ApplicationMaster Launcher] >>>>>>>> amlauncher.ApplicationMasterLauncher >>>>>>>> (ApplicationMasterLauncher.java:run(98)) - >>>>>>>> org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher$LauncherThread >>>>>>>> interrupted. Returning. >>>>>>>> 2014-02-06 23:01:08,897 INFO [main] ipc.Server >>>>>>>> (Server.java:stop(2437)) - Stopping server on 55122 >>>>>>>> 2014-02-06 23:01:08,898 INFO [IPC Server listener on 55122] >>>>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on >>>>>>>> 55122 >>>>>>>> 2014-02-06 23:01:08,898 INFO [main] ipc.Server >>>>>>>> (Server.java:stop(2437)) - Stopping server on 55121 >>>>>>>> 2014-02-06 23:01:08,898 INFO [IPC Server Responder] ipc.Server >>>>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>>>> 2014-02-06 23:01:08,902 INFO [IPC Server listener on 55121] >>>>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on >>>>>>>> 55121 >>>>>>>> 2014-02-06 23:01:08,908 INFO [IPC Server Responder] ipc.Server >>>>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>>>> 2014-02-06 23:01:08,902 INFO [main] ipc.Server >>>>>>>> (Server.java:stop(2437)) - Stopping server on 55120 >>>>>>>> 2014-02-06 23:01:08,912 INFO [IPC Server listener on 55120] >>>>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on >>>>>>>> 55120 >>>>>>>> 2014-02-06 23:01:08,912 INFO [IPC Server Responder] ipc.Server >>>>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>>>> 2014-02-06 23:01:08,912 INFO [main] ipc.Server >>>>>>>> (Server.java:stop(2437)) - Stopping server on 55119 >>>>>>>> 2014-02-06 23:01:08,917 INFO [IPC Server listener on 55119] >>>>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on >>>>>>>> 55119 >>>>>>>> 2014-02-06 23:01:08,917 INFO [Ping Checker] >>>>>>>> util.AbstractLivelinessMonitor >>>>>>>> (AbstractLivelinessMonitor.java:run(127)) - >>>>>>>> NMLivelinessMonitor thread interrupted >>>>>>>> 2014-02-06 23:01:08,917 ERROR [ResourceManager Event Processor] >>>>>>>> resourcemanager.ResourceManager (ResourceManager.java:run(435)) - >>>>>>>> Returning, interrupted : java.lang.InterruptedException >>>>>>>> 2014-02-06 23:01:08,917 INFO [IPC Server Responder] ipc.Server >>>>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>>>> 2014-02-06 23:01:08,918 INFO [Ping Checker] >>>>>>>> util.AbstractLivelinessMonitor >>>>>>>> (AbstractLivelinessMonitor.java:run(127)) - >>>>>>>> AMLivelinessMonitor thread interrupted >>>>>>>> 2014-02-06 23:01:08,918 INFO [Ping Checker] >>>>>>>> util.AbstractLivelinessMonitor >>>>>>>> (AbstractLivelinessMonitor.java:run(127)) - >>>>>>>> org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer >>>>>>>> thread interrupted >>>>>>>> 2014-02-06 23:01:08,918 INFO [Ping Checker] >>>>>>>> util.AbstractLivelinessMonitor >>>>>>>> (AbstractLivelinessMonitor.java:run(127)) - >>>>>>>> AMLivelinessMonitor thread interrupted >>>>>>>> >>>>>>>> Has anyone seen this error before when building hadoop, and does >>>>>>>> anyone have any ideas? >>>>>>>> >>>>>>>> Regards >>>>>>>> >>>>>>>> Chris >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
