Hi Cheolsoo, Please help to check whether TestJobSunbmission passed in your environment. Thanks
Hi Cheolsoo, TestHBaseStorage passed now with your fix. Thanks. But TestJobSubmission still failed with the same previous error with TestHBaseStorage. Please help to check. Thanks At 2012-08-18 14:26:32,"Cheolsoo Park" <[email protected]> wrote: >Hi, > >I tried the same version of libraries as you're using and was able to get >TestHBaseStorage passing in trunk. Here is what I did: > >1) Updated the version of hadoop, hbase, and zookeeper in >ivy/libraries.properties: > >hadoop-core.version=1.0.3 >> hadoop-test.version=1.0.3 >> hbase.version=0.94.0 >> zookeeper.version=3.4.3 > > >2) Updated TestHBaseStorage.java since it doesn't compile with hbase-0.94.0: > >HTable table = new HTable(tableName); => HTable table = new HTable(conf, >> tableName); > > >There are 6 places to fix. > >3) Added the following lines to ivy.xml: > ><dependency org="com.github.stephenc.high-scale-lib" name="high-scale-lib" >> rev="${high-scale-lib.version}" conf="test->default"/> >> <dependency org="com.google.protobuf" name="protobuf-java" >> rev="${protobuf-java.version}" conf="test->default"/> > > >I also added the following line to ivy/libraries.properties: > >high-scale-lib.version=1.1.1 > > >In particular, I was able to reproduce your error without this line in >ivy.xml: > ><dependency org="com.google.protobuf" name="protobuf-java" >> rev="${protobuf-java.version}" conf="test->default"/> > > >With all these changes, TestHBaseStorage passes for me. Please let me know >if this works for you. > >Thanks, >Cheolsoo > >On Thu, Aug 16, 2012 at 11:02 PM, lulynn_2008 <[email protected]> wrote: > >> BTW, I am using hadoop-1.0.3, hbase-0.94.0 and zookeeper-3.4.3. >> >> >> Hi Cheolsoo, >> You are right. TestHBaseStorage also failed with the same error. >> >> I got following region server information, and seems hbase region has been >> started successfully. But hbase can not connect to zookeeper >> >> 12/08/13 06:34:12 INFO server.PrepRequestProcessor: Got user-level >> KeeperException when processing sessionid:0x1391cf90aed0000 type:delete >> cxid:0x24 zxid:0x10 txntype:-1 reqpath:n/a Error >> Path:/hbase/root-region-server Error:KeeperErrorCode = NoNode for >> /hbase/root-region-server >> >> 12/08/13 06:34:16 INFO regionserver.HRegion: creating HRegion pigtable_1 >> HTD == {NAME => 'pigtable_1', FAMILIES => [{NAME => 'pig', >> REPLICATION_SCOPE => '0', KEEP_DELETED_CELLS => 'false', COMPRESSION => >> 'NONE', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true', MIN_VERSIONS => >> '0', DATA_BLOCK_ENCODING => 'NONE', IN_MEMORY => 'false', BLOOMFILTER => >> 'NONE', TTL => '2147483647', VERSIONS => '3', BLOCKSIZE => '65536'}]} >> RootDir = hdfs://shihc015:54670/user/root/hbase Table name == pigtable_1 >> >> 12/08/13 06:34:43 INFO zookeeper.ClientCnxn: Opening socket connection to >> server shihc015/127.0.0.1:21818 >> 12/08/13 06:34:43 WARN client.ZooKeeperSaslClient: SecurityException: >> java.lang.SecurityException: Unable to locate a login configuration >> occurred when trying to find JAAS configuration. >> 12/08/13 06:34:43 INFO client.ZooKeeperSaslClient: Client will not >> SASL-authenticate because the default JAAS configuration section 'Client' >> could not be found. If you are not using SASL, you may ignore this. On the >> other hand, if you expected SASL to work, please fix your JAAS >> configuration. >> 12/08/13 06:34:43 WARN zookeeper.ClientCnxn: Session 0x0 for server null, >> unexpected error, closing socket connection and attempting reconnect >> java.net.ConnectException: Connection refused >> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) >> at >> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:610) >> at >> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286) >> at >> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035) >> 12/08/13 06:34:43 WARN zookeeper.RecoverableZooKeeper: Possibly transient >> ZooKeeper exception: >> org.apache.zookeeper.KeeperException$ConnectionLossException: >> KeeperErrorCode = ConnectionLoss for /hbase/root-region-server >> >> >> 12/08/16 22:33:05 ERROR zookeeper.ZooKeeperWatcher: hconnection Received >> unexpected KeeperException, re-throwing exception >> org.apache.zookeeper.KeeperException$ConnectionLossException: >> KeeperErrorCode = ConnectionLoss for /hbase/root-region-server >> at >> org.apache.zookeeper.KeeperException.create(KeeperException.java:99) >> at >> org.apache.zookeeper.KeeperException.create(KeeperException.java:51) >> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1131) >> at >> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getData(RecoverableZooKeeper.java:264) >> at >> org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataInternal(ZKUtil.java:522) >> at >> org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataAndWatch(ZKUtil.java:498) >> at >> org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.getData(ZooKeeperNodeTracker.java:156) >> at >> org.apache.hadoop.hbase.zookeeper.RootRegionTracker.getRootRegionLocation(RootRegionTracker.java:62) >> at >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:821) >> at >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801) >> at >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:933) >> at >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:832) >> at >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801) >> at >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:933) >> at >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:836) >> at >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801) >> at >> org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:234) >> at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:174) >> at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:133) >> at >> org.apache.pig.backend.hadoop.hbase.HBaseStorage.setLocation(HBaseStorage.java:551) >> at >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:385) >> at >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:259) >> at >> org.apache.pig.test.TestJobSubmission.testReducerNumEstimation(TestJobSubmission.java:492) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37) >> at java.lang.reflect.Method.invoke(Method.java:611) >> at >> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) >> at >> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) >> at >> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) >> at >> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) >> at >> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) >> at >> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) >> at >> org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79) >> at >> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71) >> >> >> >> >> >> >> At 2012-08-16 01:30:33,"Cheolsoo Park" <[email protected]> wrote: >> >Hi, >> > >> >Caused by: org.apache.hadoop.hbase.client.NoServerForRegionException: >> >> Unable to find region for passwd,,99999999999999 after 10 tries. >> > >> > >> >It looks like the region server is not available. >> >1) Does TestHBaseStorage pass? I guess that it fails with the same error. >> >2) In your log, do you see any errors related to the HBase region server? >> >For example, java.lang.IllegalStateException: Could not instantiate a >> >region instance. >> > >> >If the region server failed to start, you will see this failure. >> > >> >Thanks, >> >Cheolsoo >> > >> > >> >On Wed, Aug 15, 2012 at 12:56 AM, lulynn_2008 <[email protected]> >> wrote: >> > >> >> Hi All, >> >> >> >> In TestJobSubmission, testReducerNumEstimation failed with following >> error >> >> information. Please give a glance to check what the problem is. Thanks. >> >> >> >> Error information: >> >> Testcase: testReducerNumEstimation took 1,809.419 sec >> >> Caused an ERROR >> >> Internal error creating job configuration. >> >> >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobCreationException: >> >> ERROR 2017: Internal error creating job configuration. >> >> at >> >> >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:731) >> >> at >> >> >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:259) >> >> at >> >> >> org.apache.pig.test.TestJobSubmission.testReducerNumEstimation(TestJobSubmission.java:491) >> >> Caused by: org.apache.hadoop.hbase.client.NoServerForRegionException: >> >> Unable to find region for passwd,,99999999999999 after 10 tries. >> >> at >> >> >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:926) >> >> at >> >> >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:832) >> >> at >> >> >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801) >> >> at >> >> >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:933) >> >> at >> >> >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:836) >> >> at >> >> >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801) >> >> at >> >> org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:234) >> >> at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:174) >> >> at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:133) >> >> at >> >> >> org.apache.pig.backend.hadoop.hbase.HBaseStorage.setLocation(HBaseStorage.java:545) >> >> at >> >> >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:384) >> >> >> >> >> >> >> >>
