Hi, TestJobSubmission doesn't pass for me either. But it doesn't throw the error that you're seeing. Instead, it hangs at line 583 in TestJobSubmission.java. I am not sure yet what's happening though.
Thanks, Cheolsoo On Tue, Aug 21, 2012 at 12:24 AM, lulynn_2008 <[email protected]> wrote: > Hi Cheolsoo, > Please help to check whether TestJobSunbmission passed in your > environment. Thanks > > > Hi Cheolsoo, > TestHBaseStorage passed now with your fix. Thanks. > But TestJobSubmission still failed with the same previous error with > TestHBaseStorage. Please help to check. Thanks > > > > > > At 2012-08-18 14:26:32,"Cheolsoo Park" <[email protected]> wrote: > >Hi, > > > >I tried the same version of libraries as you're using and was able to get > >TestHBaseStorage passing in trunk. Here is what I did: > > > >1) Updated the version of hadoop, hbase, and zookeeper in > >ivy/libraries.properties: > > > >hadoop-core.version=1.0.3 > >> hadoop-test.version=1.0.3 > >> hbase.version=0.94.0 > >> zookeeper.version=3.4.3 > > > > > >2) Updated TestHBaseStorage.java since it doesn't compile with > hbase-0.94.0: > > > >HTable table = new HTable(tableName); => HTable table = new HTable(conf, > >> tableName); > > > > > >There are 6 places to fix. > > > >3) Added the following lines to ivy.xml: > > > ><dependency org="com.github.stephenc.high-scale-lib" name="high-scale-lib" > >> rev="${high-scale-lib.version}" conf="test->default"/> > >> <dependency org="com.google.protobuf" name="protobuf-java" > >> rev="${protobuf-java.version}" conf="test->default"/> > > > > > >I also added the following line to ivy/libraries.properties: > > > >high-scale-lib.version=1.1.1 > > > > > >In particular, I was able to reproduce your error without this line in > >ivy.xml: > > > ><dependency org="com.google.protobuf" name="protobuf-java" > >> rev="${protobuf-java.version}" conf="test->default"/> > > > > > >With all these changes, TestHBaseStorage passes for me. Please let me know > >if this works for you. > > > >Thanks, > >Cheolsoo > > > >On Thu, Aug 16, 2012 at 11:02 PM, lulynn_2008 <[email protected]> > wrote: > > > >> BTW, I am using hadoop-1.0.3, hbase-0.94.0 and zookeeper-3.4.3. > >> > >> > >> Hi Cheolsoo, > >> You are right. TestHBaseStorage also failed with the same error. > >> > >> I got following region server information, and seems hbase region has > been > >> started successfully. But hbase can not connect to zookeeper > >> > >> 12/08/13 06:34:12 INFO server.PrepRequestProcessor: Got user-level > >> KeeperException when processing sessionid:0x1391cf90aed0000 type:delete > >> cxid:0x24 zxid:0x10 txntype:-1 reqpath:n/a Error > >> Path:/hbase/root-region-server Error:KeeperErrorCode = NoNode for > >> /hbase/root-region-server > >> > >> 12/08/13 06:34:16 INFO regionserver.HRegion: creating HRegion pigtable_1 > >> HTD == {NAME => 'pigtable_1', FAMILIES => [{NAME => 'pig', > >> REPLICATION_SCOPE => '0', KEEP_DELETED_CELLS => 'false', COMPRESSION => > >> 'NONE', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true', MIN_VERSIONS => > >> '0', DATA_BLOCK_ENCODING => 'NONE', IN_MEMORY => 'false', BLOOMFILTER => > >> 'NONE', TTL => '2147483647', VERSIONS => '3', BLOCKSIZE => '65536'}]} > >> RootDir = hdfs://shihc015:54670/user/root/hbase Table name == pigtable_1 > >> > >> 12/08/13 06:34:43 INFO zookeeper.ClientCnxn: Opening socket connection > to > >> server shihc015/127.0.0.1:21818 > >> 12/08/13 06:34:43 WARN client.ZooKeeperSaslClient: SecurityException: > >> java.lang.SecurityException: Unable to locate a login configuration > >> occurred when trying to find JAAS configuration. > >> 12/08/13 06:34:43 INFO client.ZooKeeperSaslClient: Client will not > >> SASL-authenticate because the default JAAS configuration section > 'Client' > >> could not be found. If you are not using SASL, you may ignore this. On > the > >> other hand, if you expected SASL to work, please fix your JAAS > >> configuration. > >> 12/08/13 06:34:43 WARN zookeeper.ClientCnxn: Session 0x0 for server > null, > >> unexpected error, closing socket connection and attempting reconnect > >> java.net.ConnectException: Connection refused > >> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > >> at > >> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:610) > >> at > >> > org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286) > >> at > >> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035) > >> 12/08/13 06:34:43 WARN zookeeper.RecoverableZooKeeper: Possibly > transient > >> ZooKeeper exception: > >> org.apache.zookeeper.KeeperException$ConnectionLossException: > >> KeeperErrorCode = ConnectionLoss for /hbase/root-region-server > >> > >> > >> 12/08/16 22:33:05 ERROR zookeeper.ZooKeeperWatcher: hconnection Received > >> unexpected KeeperException, re-throwing exception > >> org.apache.zookeeper.KeeperException$ConnectionLossException: > >> KeeperErrorCode = ConnectionLoss for /hbase/root-region-server > >> at > >> org.apache.zookeeper.KeeperException.create(KeeperException.java:99) > >> at > >> org.apache.zookeeper.KeeperException.create(KeeperException.java:51) > >> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1131) > >> at > >> > org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getData(RecoverableZooKeeper.java:264) > >> at > >> > org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataInternal(ZKUtil.java:522) > >> at > >> > org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataAndWatch(ZKUtil.java:498) > >> at > >> > org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.getData(ZooKeeperNodeTracker.java:156) > >> at > >> > org.apache.hadoop.hbase.zookeeper.RootRegionTracker.getRootRegionLocation(RootRegionTracker.java:62) > >> at > >> > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:821) > >> at > >> > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801) > >> at > >> > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:933) > >> at > >> > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:832) > >> at > >> > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801) > >> at > >> > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:933) > >> at > >> > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:836) > >> at > >> > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801) > >> at > >> org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:234) > >> at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:174) > >> at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:133) > >> at > >> > org.apache.pig.backend.hadoop.hbase.HBaseStorage.setLocation(HBaseStorage.java:551) > >> at > >> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:385) > >> at > >> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:259) > >> at > >> > org.apache.pig.test.TestJobSubmission.testReducerNumEstimation(TestJobSubmission.java:492) > >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >> at > >> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60) > >> at > >> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37) > >> at java.lang.reflect.Method.invoke(Method.java:611) > >> at > >> > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) > >> at > >> > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) > >> at > >> > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) > >> at > >> > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) > >> at > >> > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > >> at > >> > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) > >> at > >> > org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79) > >> at > >> > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71) > >> > >> > >> > >> > >> > >> > >> At 2012-08-16 01:30:33,"Cheolsoo Park" <[email protected]> wrote: > >> >Hi, > >> > > >> >Caused by: org.apache.hadoop.hbase.client.NoServerForRegionException: > >> >> Unable to find region for passwd,,99999999999999 after 10 tries. > >> > > >> > > >> >It looks like the region server is not available. > >> >1) Does TestHBaseStorage pass? I guess that it fails with the same > error. > >> >2) In your log, do you see any errors related to the HBase region > server? > >> >For example, java.lang.IllegalStateException: Could not instantiate a > >> >region instance. > >> > > >> >If the region server failed to start, you will see this failure. > >> > > >> >Thanks, > >> >Cheolsoo > >> > > >> > > >> >On Wed, Aug 15, 2012 at 12:56 AM, lulynn_2008 <[email protected]> > >> wrote: > >> > > >> >> Hi All, > >> >> > >> >> In TestJobSubmission, testReducerNumEstimation failed with following > >> error > >> >> information. Please give a glance to check what the problem is. > Thanks. > >> >> > >> >> Error information: > >> >> Testcase: testReducerNumEstimation took 1,809.419 sec > >> >> Caused an ERROR > >> >> Internal error creating job configuration. > >> >> > >> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobCreationException: > >> >> ERROR 2017: Internal error creating job configuration. > >> >> at > >> >> > >> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:731) > >> >> at > >> >> > >> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:259) > >> >> at > >> >> > >> > org.apache.pig.test.TestJobSubmission.testReducerNumEstimation(TestJobSubmission.java:491) > >> >> Caused by: org.apache.hadoop.hbase.client.NoServerForRegionException: > >> >> Unable to find region for passwd,,99999999999999 after 10 tries. > >> >> at > >> >> > >> > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:926) > >> >> at > >> >> > >> > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:832) > >> >> at > >> >> > >> > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801) > >> >> at > >> >> > >> > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:933) > >> >> at > >> >> > >> > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:836) > >> >> at > >> >> > >> > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801) > >> >> at > >> >> org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:234) > >> >> at > org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:174) > >> >> at > org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:133) > >> >> at > >> >> > >> > org.apache.pig.backend.hadoop.hbase.HBaseStorage.setLocation(HBaseStorage.java:545) > >> >> at > >> >> > >> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:384) > >> >> > >> >> > >> > >> > >> > >> > > > >
