Ashish, I've separated unit tests into two categories. One is a unit test set that don't need mini yarn cluster, and you can do these unit tests though 'mvn test'. Another is a unit test set running on mini yarn cluster, and you can do them though 'mvn verify'.
If you execute 'mvn install', mvn internally includes 'verify phase'. So, you cannot pass 'mvn install' even though you passed 'mvn test'. I'm suspecting some bugs caused by TaskRunner. If I fix this bug, I'll let you know the resolution. Thanks, Hyunsik On Thu, Mar 28, 2013 at 9:16 PM, Ashish <[email protected]> wrote: > Thanks ! Which Integration test is this? > > > On Thu, Mar 28, 2013 at 5:44 PM, Hyunsik Choi <[email protected] > >wrote: > > > Thank you for your reporting. Actually, I haven't tested the integration > > test on Mac. I have just tried to do 'mvn verify' on Mac. The same > problem > > is happen. It is a bug. I've created the jira issue about this problem. > > > > https://issues.apache.org/jira/browse/TAJO-15 > > > > Thanks you, > > Hyunsik > > > > > > On Thu, Mar 28, 2013 at 8:39 PM, Ashish <[email protected]> wrote: > > > > > I am using Mac 10.8.3, and its happening on both JDK 6 and 7 > > > > > > java version "1.6.0_43" > > > Java(TM) SE Runtime Environment (build 1.6.0_43-b01-447-11M4203) > > > Java HotSpot(TM) 64-Bit Server VM (build 20.14-b01-447, mixed mode) > > > > > > java version "1.7.0_10" > > > Java(TM) SE Runtime Environment (build 1.7.0_10-b18) > > > Java HotSpot(TM) 64-Bit Server VM (build 23.6-b04, mixed mode) > > > > > > Adding a little more logs. Would be great if you tell me the starting > > point > > > for this. I shall try to hunt down more details > > > > > > 2013-03-28 16:42:39,039 INFO capacity.CapacityScheduler > > > (CapacityScheduler.java:completedContainer(776)) - Application > > > appattempt_1364469093530_0002_000001 released container > > > container_1364469093530_0002_01_000007 on node: host: a.b.c.d:60941 > > > #containers=0 available=4096 used=0 with event: FINISHED > > > 2013-03-28 16:42:39,235 INFO rmcontainer.RMContainerImpl > > > (RMContainerImpl.java:handle(220)) - > > container_1364469093530_0002_01_000008 > > > Container Transitioned from ALLOCATED to ACQUIRED > > > 2013-03-28 16:42:39,236 INFO rm.RMContainerAllocator > > > (RMContainerAllocator.java:makeRemoteRequest(172)) - Available > Resource: > > > <memory:6144, vCores:-1> > > > 2013-03-28 16:42:39,237 INFO rm.RMContainerAllocator > > > (RMContainerAllocator.java:makeRemoteRequest(173)) - Num of Allocated > > > Containers: 1 > > > 2013-03-28 16:42:39,237 INFO rm.RMContainerAllocator > > > (RMContainerAllocator.java:makeRemoteRequest(175)) - > > > ================================================================ > > > 2013-03-28 16:42:39,237 INFO rm.RMContainerAllocator > > > (RMContainerAllocator.java:makeRemoteRequest(177)) - > Container Id: > > > container_1364469093530_0002_01_000008 > > > 2013-03-28 16:42:39,237 INFO rm.RMContainerAllocator > > > (RMContainerAllocator.java:makeRemoteRequest(178)) - > Node Id: > > > a.b.c.d:60945 > > > 2013-03-28 16:42:39,237 INFO rm.RMContainerAllocator > > > (RMContainerAllocator.java:makeRemoteRequest(179)) - > Resource (Mem): > > 3072 > > > 2013-03-28 16:42:39,237 INFO rm.RMContainerAllocator > > > (RMContainerAllocator.java:makeRemoteRequest(180)) - > State : NEW > > > 2013-03-28 16:42:39,237 INFO rm.RMContainerAllocator > > > (RMContainerAllocator.java:makeRemoteRequest(181)) - > Priority: 92 > > > 2013-03-28 16:42:39,237 INFO rm.RMContainerAllocator > > > (RMContainerAllocator.java:makeRemoteRequest(183)) - > > > ================================================================ > > > 2013-03-28 16:42:39,238 INFO master.SubQuery > > > (SubQuery.java:transition(713)) - SubQuery > > > (sq_1364469093530_0002_000001_27) has 1 containers! > > > 2013-03-28 16:42:39,238 INFO master.TaskRunnerLauncherImpl > > > (TaskRunnerLauncherImpl.java:launch(393)) - Launching Container with > Id: > > > container_1364469093530_0002_01_000008 > > > 2013-03-28 16:42:39,239 INFO master.TaskRunnerLauncherImpl > > > (TaskRunnerLauncherImpl.java:createContainerLaunchContext(301)) - > > Completed > > > setting up taskrunner command ${JAVA_HOME}/bin/java -Xmx2000m > > > tajo.worker.TaskRunner a.b.c.d 58243 sq_1364469093530_0002_000001_27 > > > a.b.c.d:60945 container_1364469093530_0002_01_000008 1><LOG_DIR>/stdout > > > 2><LOG_DIR>/stderr > > > 2013-03-28 16:42:39,244 INFO containermanager.ContainerManagerImpl > > > (ContainerManagerImpl.java:startContainer(402)) - Start request for > > > container_1364469093530_0002_01_000008 by user xxxxxxx > > > 2013-03-28 16:42:39,245 INFO nodemanager.NMAuditLogger > > > (NMAuditLogger.java:logSuccess(89)) - USER=xxxxxxx IP=a.b.c.d > > > OPERATION=Start > > > Container Request TARGET=ContainerManageImpl RESULT=SUCCESS > > > APPID=application_1364469093530_0002 > > > CONTAINERID=container_1364469093530_0002_01_000008 > > > 2013-03-28 16:42:39,245 INFO application.Application > > > (ApplicationImpl.java:transition(255)) - Adding > > > container_1364469093530_0002_01_000008 to application > > > application_1364469093530_0002 > > > 2013-03-28 16:42:39,246 INFO container.Container > > > (ContainerImpl.java:handle(835)) - Container > > > container_1364469093530_0002_01_000008 transitioned from NEW to > > LOCALIZING > > > 2013-03-28 16:42:39,246 INFO master.TaskRunnerLauncherImpl > > > (TaskRunnerLauncherImpl.java:launch(424)) - PullServer port returned by > > > ContainerManager for container_1364469093530_0002_01_000008 : 60947 > > > 2013-03-28 16:42:39,246 INFO containermanager.AuxServices > > > (AuxServices.java:handle(160)) - Got event APPLICATION_INIT for appId > > > application_1364469093530_0002 > > > 2013-03-28 16:42:39,246 INFO containermanager.AuxServices > > > (AuxServices.java:handle(164)) - Got APPLICATION_INIT for service > > > tajo.pullserver > > > 2013-03-28 16:42:39,246 INFO master.Query (Query.java:handle(514)) - > > > Processing q_1364469093530_0002_000001 of type INIT_COMPLETED > > > 2013-03-28 16:42:39,246 INFO container.Container > > > (ContainerImpl.java:handle(835)) - Container > > > container_1364469093530_0002_01_000008 transitioned from LOCALIZING to > > > LOCALIZED > > > 2013-03-28 16:42:39,247 INFO util.RackResolver > > > (RackResolver.java:coreResolve(100)) - Resolved L-IDC77TDV7M-M.local to > > > /default-rack > > > 2013-03-28 16:42:39,339 INFO container.Container > > > (ContainerImpl.java:handle(835)) - Container > > > container_1364469093530_0002_01_000008 transitioned from LOCALIZED to > > > RUNNING > > > 2013-03-28 16:42:39,340 INFO monitor.ContainersMonitorImpl > > > (ContainersMonitorImpl.java:isEnabled(168)) - ResourceCalculatorPlugin > is > > > unavailable on this system. > > > > > > > > > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl > > > is disabled. > > > 2013-03-28 16:42:39,535 INFO nodemanager.DefaultContainerExecutor > > > (DefaultContainerExecutor.java:launchContainer(175)) - launchContainer: > > > [bash, > > > > > > > > > /Users/xxxxxxx/opensource/tajo/incubator-tajo/tajo-core/tajo-core-backend/target/tajo.TajoTestingCluster/tajo.TajoTestingCluster-localDir-nm-1_0/usercache/xxxxxxx/appcache/application_1364469093530_0002/container_1364469093530_0002_01_000008/default_container_executor.sh] > > > 2013-03-28 16:42:39,903 INFO nodemanager.NodeStatusUpdaterImpl > > > (NodeStatusUpdaterImpl.java:getNodeStatus(265)) - Sending out status > for > > > container: container_id {, app_attempt_id {, application_id {, id: 2, > > > cluster_timestamp: 1364469093530, }, attemptId: 1, }, id: 8, }, state: > > > C_RUNNING, diagnostics: "", exit_status: -1000, > > > 2013-03-28 16:42:39,904 INFO rmcontainer.RMContainerImpl > > > (RMContainerImpl.java:handle(220)) - > > container_1364469093530_0002_01_000008 > > > Container Transitioned from ACQUIRED to RUNNING > > > 2013-03-28 16:42:40,020 WARN nodemanager.DefaultContainerExecutor > > > (DefaultContainerExecutor.java:launchContainer(193)) - Exit code from > > task > > > is : 1 > > > 2013-03-28 16:42:40,021 INFO nodemanager.ContainerExecutor > > > (ContainerExecutor.java:logOutput(167)) - > > > 2013-03-28 16:42:40,021 WARN launcher.ContainerLaunch > > > (ContainerLaunch.java:call(274)) - Container exited with a non-zero > exit > > > code 1 > > > 2013-03-28 16:42:40,021 INFO container.Container > > > (ContainerImpl.java:handle(835)) - Container > > > container_1364469093530_0002_01_000008 transitioned from RUNNING to > > > EXITED_WITH_FAILURE > > > 2013-03-28 16:42:40,021 INFO launcher.ContainerLaunch > > > (ContainerLaunch.java:cleanupContainer(300)) - Cleaning up container > > > container_1364469093530_0002_01_000008 > > > 2013-03-28 16:42:40,040 INFO nodemanager.DefaultContainerExecutor > > > (DefaultContainerExecutor.java:deleteAsUser(273)) - Deleting absolute > > path > > > : > > > > > > > > > /Users/xxxxxxx/opensource/tajo/incubator-tajo/tajo-core/tajo-core-backend/target/tajo.TajoTestingCluster/tajo.TajoTestingCluster-localDir-nm-1_0/usercache/xxxxxxx/appcache/application_1364469093530_0002/container_1364469093530_0002_01_000008 > > > 2013-03-28 16:42:40,040 WARN nodemanager.NMAuditLogger > > > (NMAuditLogger.java:logFailure(150)) - USER=xxxxxxx OPERATION=Container > > > Finished - Failed TARGET=ContainerImpl RESULT=FAILURE > > DESCRIPTION=Container > > > failed with state: EXITED_WITH_FAILURE > > APPID=application_1364469093530_0002 > > > CONTAINERID=container_1364469093530_0002_01_000008 > > > 2013-03-28 16:42:40,041 INFO container.Container > > > (ContainerImpl.java:handle(835)) - Container > > > container_1364469093530_0002_01_000008 transitioned from > > > EXITED_WITH_FAILURE to DONE > > > 2013-03-28 16:42:40,041 INFO application.Application > > > (ApplicationImpl.java:transition(298)) - Removing > > > container_1364469093530_0002_01_000008 from application > > > application_1364469093530_0002 > > > 2013-03-28 16:42:40,041 INFO monitor.ContainersMonitorImpl > > > (ContainersMonitorImpl.java:isEnabled(168)) - ResourceCalculatorPlugin > is > > > unavailable on this system. > > > > > > > > > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl > > > is disabled. > > > 2013-03-28 16:42:40,241 INFO rm.RMContainerAllocator > > > (RMContainerAllocator.java:makeRemoteRequest(172)) - Available > Resource: > > > <memory:6144, vCores:-1> > > > 2013-03-28 16:42:40,241 INFO rm.RMContainerAllocator > > > (RMContainerAllocator.java:makeRemoteRequest(173)) - Num of Allocated > > > Containers: 0 > > > 2013-03-28 16:42:40,905 INFO nodemanager.NodeStatusUpdaterImpl > > > (NodeStatusUpdaterImpl.java:getNodeStatus(265)) - Sending out status > for > > > container: container_id {, app_attempt_id {, application_id {, id: 2, > > > cluster_timestamp: 1364469093530, }, attemptId: 1, }, id: 8, }, state: > > > C_COMPLETE, diagnostics: "\n", exit_status: 1, > > > 2013-03-28 16:42:40,905 INFO nodemanager.NodeStatusUpdaterImpl > > > (NodeStatusUpdaterImpl.java:getNodeStatus(271)) - Removed completed > > > container container_1364469093530_0002_01_000008 > > > 2013-03-28 16:42:40,906 INFO rmcontainer.RMContainerImpl > > > (RMContainerImpl.java:handle(220)) - > > container_1364469093530_0002_01_000008 > > > Container Transitioned from RUNNING to COMPLETED > > > 2013-03-28 16:42:40,906 INFO fica.FiCaSchedulerApp > > > (FiCaSchedulerApp.java:containerCompleted(219)) - Completed container: > > > container_1364469093530_0002_01_000008 in state: COMPLETED > event:FINISHED > > > 2013-03-28 16:42:40,906 INFO resourcemanager.RMAuditLogger > > > (RMAuditLogger.java:logSuccess(98)) - USER=xxxxxxx OPERATION=AM > Released > > > Container TARGET=SchedulerApp RESULT=SUCCESS > > > APPID=application_1364469093530_0002 > > > CONTAINERID=container_1364469093530_0002_01_000008 > > > 2013-03-28 16:42:40,906 INFO fica.FiCaSchedulerNode > > > (FiCaSchedulerNode.java:releaseContainer(150)) - Released container > > > container_1364469093530_0002_01_000008 of capacity <memory:3072, > > vCores:1> > > > on host a.b.c.d:60945, which currently has 0 containers, <memory:0, > > > vCores:0> used and <memory:4096, vCores:16> available, release > > > resources=true > > > 2013-03-28 16:42:40,906 INFO capacity.LeafQueue > > > (LeafQueue.java:releaseResource(1441)) - default used=<memory:0, > > vCores:0> > > > numContainers=0 user=xxxxxxx user-resources=<memory:0, vCores:0> > > > 2013-03-28 16:42:40,907 INFO capacity.LeafQueue > > > (LeafQueue.java:completedContainer(1385)) - completedContainer > > > container=Container: [ContainerId: > > container_1364469093530_0002_01_000008, > > > NodeId: a.b.c.d:60945, NodeHttpAddress: a.b.c.d:60948, Resource: > > > <memory:3072, vCores:1>, Priority: 92, State: NEW, Token: null, Status: > > > container_id {, app_attempt_id {, application_id {, id: 2, > > > cluster_timestamp: 1364469093530, }, attemptId: 1, }, id: 8, }, state: > > > C_COMPLETE, diagnostics: "\n", exit_status: 1, ] resource=<memory:3072, > > > vCores:1> queue=default: capacity=1.0, absoluteCapacity=1.0, > > > usedResources=<memory:0, vCores:0>usedCapacity=0.0, > > > absoluteUsedCapacity=0.0, numApps=1, numContainers=0 usedCapacity=0.0 > > > absoluteUsedCapacity=0.0 used=<memory:0, vCores:0> > cluster=<memory:12288, > > > vCores:48> > > > 2013-03-28 16:42:40,907 INFO capacity.ParentQueue > > > (ParentQueue.java:completedContainer(696)) - completedContainer > > queue=root > > > usedCapacity=0.0 absoluteUsedCapacity=0.0 used=<memory:0, vCores:0> > > > cluster=<memory:12288, vCores:48> > > > 2013-03-28 16:42:40,907 INFO capacity.CapacityScheduler > > > (CapacityScheduler.java:completedContainer(776)) - Application > > > appattempt_1364469093530_0002_000001 released container > > > container_1364469093530_0002_01_000008 on node: host: a.b.c.d:60945 > > > #containers=0 available=4096 used=0 with event: FINISHED > > > 2013-03-28 16:42:41,242 INFO rm.RMContainerAllocator > > > (RMContainerAllocator.java:makeRemoteRequest(172)) - Available > Resource: > > > <memory:6144, vCores:-1> > > > 2013-03-28 16:42:41,242 INFO rm.RMContainerAllocator > > > (RMContainerAllocator.java:makeRemoteRequest(173)) - Num of Allocated > > > Containers: 0 > > > 2013-03-28 16:42:42,245 INFO rm.RMContainerAllocator > > > (RMContainerAllocator.java:makeRemoteRequest(172)) - Available > Resource: > > > <memory:6144, vCores:-1> > > > 2013-03-28 16:42:42,246 INFO rm.RMContainerAllocator > > > (RMContainerAllocator.java:makeRemoteRequest(173)) - Num of Allocated > > > Containers: 0 > > > 2013-03-28 16:42:43,248 INFO rm.RMContainerAllocator > > > (RMContainerAllocator.java:makeRemoteRequest(172)) - Available > Resource: > > > <memory:6144, vCores:-1> > > > 2013-03-28 16:42:43,249 INFO rm.RMContainerAllocator > > > (RMContainerAllocator.java:makeRemoteRequest(173)) - Num of Allocated > > > Containers: 0 > > > 2013-03-28 16:42:44,251 INFO rm.RMContainerAllocator > > > (RMContainerAllocator.java:makeRemoteRequest(172)) - Available > Resource: > > > <memory:6144, vCores:-1> > > > 2013-03-28 16:42:44,252 INFO rm.RMContainerAllocator > > > (RMContainerAllocator.java:makeRemoteRequest(173)) - Num of Allocated > > > Containers: 0 > > > 2013-03-28 16:42:45,255 INFO rm.RMContainerAllocator > > > (RMContainerAllocator.java:makeRemoteRequest(172)) - Available > Resource: > > > <memory:6144, vCores:-1> > > > 2013-03-28 16:42:45,256 INFO rm.RMContainerAllocator > > > (RMContainerAllocator.java:makeRemoteRequest(173)) - Num of Allocated > > > Containers: 0 > > > 2013-03-28 16:42:46,259 INFO rm.RMContainerAllocator > > > (RMContainerAllocator.java:makeRemoteRequest(172)) - Available > Resource: > > > <memory:6144, vCores:-1> > > > 2013-03-28 16:42:46,260 INFO rm.RMContainerAllocator > > > (RMContainerAllocator.java:makeRemoteRequest(173)) - Num of Allocated > > > Containers: 0 > > > 2013-03-28 16:42:47,263 INFO rm.RMContainerAllocator > > > (RMContainerAllocator.java:makeRemoteRequest(172)) - Available > Resource: > > > <memory:6144, vCores:-1> > > > 2013-03-28 16:42:47,264 INFO rm.RMContainerAllocator > > > (RMContainerAllocator.java:makeRemoteRequest(173)) - Num of Allocated > > > Containers: 0 > > > 2013-03-28 16:42:48,267 INFO rm.RMContainerAllocator > > > (RMContainerAllocator.java:makeRemoteRequest(172)) - Available > Resource: > > > <memory:6144, vCores:-1> > > > > > > > > > > > > > > > On Thu, Mar 28, 2013 at 4:56 PM, Hyunsik Choi <[email protected] > > > >wrote: > > > > > > > Hi Ashish, > > > > > > > > The hanging unit tests are integration tests. You can skip the > > > integration > > > > test with this command 'mvn install -DskipTests'. > > > > > > > > However, I cannot figure out the cause of the hanging unit tests from > > > your > > > > log message. Could you let me know more information? > > > > > > > > 1) What are your linux and procps versions? Recently, I had > experienced > > > the > > > > same situation due to this bug ( > > > > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=688731). > > > > > > > > 2) What is the first exception of unit tests? > > > > > > > > Thanks, > > > > Hyunsik > > > > > > > > > > > > On Thu, Mar 28, 2013 at 8:07 PM, Ashish <[email protected]> > > wrote: > > > > > > > > > Build is working fine at my end on both JDK 6 and 7. > > > > > > > > > > mvn clean;mvn compile;mvn test run fine. > > > > > > > > > > Facing a strange issue while running "mvn clean install", the build > > > never > > > > > completes and get stuck in this loop > > > > > > > > > > 2013-03-28 16:35:48,581 INFO rm.RMContainerAllocator > > > > > (RMContainerAllocator.java:makeRemoteRequest(173)) - Num of > Allocated > > > > > Containers: 0 > > > > > 2013-03-28 16:35:49,584 INFO rm.RMContainerAllocator > > > > > (RMContainerAllocator.java:makeRemoteRequest(172)) - Available > > > Resource: > > > > > <memory:6144, vCores:-1> > > > > > 2013-03-28 16:35:49,584 INFO rm.RMContainerAllocator > > > > > (RMContainerAllocator.java:makeRemoteRequest(173)) - Num of > Allocated > > > > > Containers: 0 > > > > > 2013-03-28 16:35:50,588 INFO rm.RMContainerAllocator > > > > > (RMContainerAllocator.java:makeRemoteRequest(172)) - Available > > > Resource: > > > > > <memory:6144, vCores:-1> > > > > > 2013-03-28 16:35:50,588 INFO rm.RMContainerAllocator > > > > > (RMContainerAllocator.java:makeRemoteRequest(173)) - Num of > Allocated > > > > > Containers: 0 > > > > > 2013-03-28 16:35:51,590 INFO rm.RMContainerAllocator > > > > > (RMContainerAllocator.java:makeRemoteRequest(172)) - Available > > > Resource: > > > > > <memory:6144, vCores:-1> > > > > > 2013-03-28 16:35:51,590 INFO rm.RMContainerAllocator > > > > > (RMContainerAllocator.java:makeRemoteRequest(173)) - Num of > Allocated > > > > > Containers: 0 > > > > > > > > > > Had to kill the build after observing in this state for more than > 10 > > > min. > > > > > Any suggestions on what might be holding it? > > > > > > > > > > thanks > > > > > ashish > > > > > > > > > > > > > > > On Thu, Mar 28, 2013 at 7:52 AM, Hyunsik Choi < > > [email protected] > > > > > >wrote: > > > > > > > > > > > Thank you for letting me know. JDK 1.6 is right. I'll fix the jdk > > > > version > > > > > > of the getting started. > > > > > > > > > > > > - hyunsik > > > > > > > > > > > > > > > > > > On Thu, Mar 28, 2013 at 11:15 AM, Tanujit Ghosh < > > > > [email protected] > > > > > > >wrote: > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > The Tajo wiki is saying to set JAVA_HOME as > > > > > > > > > > > > > > export JAVA_HOME=/usr/lib/jvm/openjdk-1.7.x > > > > > > > > > > > > > > > > > > > > > But as per the Requirement i guess this should be jdk1.6. > > > > > > > > > > > > > > > > > > > > > Also compiling with jdk1.7 given a issue with ResultSetImpl > > class. > > > > > > > My environment is gone wrong at the moment, but will send out > the > > > > error > > > > > > > asap. > > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > Tanujit > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > thanks > > > > > ashish > > > > > > > > > > Blog: http://www.ashishpaliwal.com/blog > > > > > My Photo Galleries: http://www.pbase.com/ashishpaliwal > > > > > > > > > > > > > > > > > > > > > -- > > > thanks > > > ashish > > > > > > Blog: http://www.ashishpaliwal.com/blog > > > My Photo Galleries: http://www.pbase.com/ashishpaliwal > > > > > > > > > -- > thanks > ashish > > Blog: http://www.ashishpaliwal.com/blog > My Photo Galleries: http://www.pbase.com/ashishpaliwal >
