Hi Cheolsoo, TestHBaseStorage passed now with your fix. Thanks. But TestJobSubmission still failed with the same previous error with TestHBaseStorage. Please help to check. Thanks
At 2012-08-18 14:26:32,"Cheolsoo Park" <[email protected]> wrote: >Hi, > >I tried the same version of libraries as you're using and was able to get >TestHBaseStorage passing in trunk. Here is what I did: > >1) Updated the version of hadoop, hbase, and zookeeper in >ivy/libraries.properties: > >hadoop-core.version=1.0.3 >> hadoop-test.version=1.0.3 >> hbase.version=0.94.0 >> zookeeper.version=3.4.3 > > >2) Updated TestHBaseStorage.java since it doesn't compile with hbase-0.94.0: > >HTable table = new HTable(tableName); => HTable table = new HTable(conf, >> tableName); > > >There are 6 places to fix. > >3) Added the following lines to ivy.xml: > ><dependency org="com.github.stephenc.high-scale-lib" name="high-scale-lib" >> rev="${high-scale-lib.version}" conf="test->default"/> >> <dependency org="com.google.protobuf" name="protobuf-java" >> rev="${protobuf-java.version}" conf="test->default"/> > > >I also added the following line to ivy/libraries.properties: > >high-scale-lib.version=1.1.1 > > >In particular, I was able to reproduce your error without this line in >ivy.xml: > ><dependency org="com.google.protobuf" name="protobuf-java" >> rev="${protobuf-java.version}" conf="test->default"/> > > >With all these changes, TestHBaseStorage passes for me. Please let me know >if this works for you. > >Thanks, >Cheolsoo > >On Thu, Aug 16, 2012 at 11:02 PM, lulynn_2008 <[email protected]> wrote: > >> BTW, I am using hadoop-1.0.3, hbase-0.94.0 and zookeeper-3.4.3. >> >> >> Hi Cheolsoo, >> You are right. TestHBaseStorage also failed with the same error. >> >> I got following region server information, and seems hbase region has been >> started successfully. But hbase can not connect to zookeeper >> >> 12/08/13 06:34:12 INFO server.PrepRequestProcessor: Got user-level >> KeeperException when processing sessionid:0x1391cf90aed0000 type:delete >> cxid:0x24 zxid:0x10 txntype:-1 reqpath:n/a Error >> Path:/hbase/root-region-server Error:KeeperErrorCode = NoNode for >> /hbase/root-region-server >> >> 12/08/13 06:34:16 INFO regionserver.HRegion: creating HRegion pigtable_1 >> HTD == {NAME => 'pigtable_1', FAMILIES => [{NAME => 'pig', >> REPLICATION_SCOPE => '0', KEEP_DELETED_CELLS => 'false', COMPRESSION => >> 'NONE', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true', MIN_VERSIONS => >> '0', DATA_BLOCK_ENCODING => 'NONE', IN_MEMORY => 'false', BLOOMFILTER => >> 'NONE', TTL => '2147483647', VERSIONS => '3', BLOCKSIZE => '65536'}]} >> RootDir = hdfs://shihc015:54670/user/root/hbase Table name == pigtable_1 >> >> 12/08/13 06:34:43 INFO zookeeper.ClientCnxn: Opening socket connection to >> server shihc015/127.0.0.1:21818 >> 12/08/13 06:34:43 WARN client.ZooKeeperSaslClient: SecurityException: >> java.lang.SecurityException: Unable to locate a login configuration >> occurred when trying to find JAAS configuration. >> 12/08/13 06:34:43 INFO client.ZooKeeperSaslClient: Client will not >> SASL-authenticate because the default JAAS configuration section 'Client' >> could not be found. If you are not using SASL, you may ignore this. On the >> other hand, if you expected SASL to work, please fix your JAAS >> configuration. >> 12/08/13 06:34:43 WARN zookeeper.ClientCnxn: Session 0x0 for server null, >> unexpected error, closing socket connection and attempting reconnect >> java.net.ConnectException: Connection refused >> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) >> at >> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:610) >> at >> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286) >> at >> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035) >> 12/08/13 06:34:43 WARN zookeeper.RecoverableZooKeeper: Possibly transient >> ZooKeeper exception: >> org.apache.zookeeper.KeeperException$ConnectionLossException: >> KeeperErrorCode = ConnectionLoss for /hbase/root-region-server >> >> >> 12/08/16 22:33:05 ERROR zookeeper.ZooKeeperWatcher: hconnection Received >> unexpected KeeperException, re-throwing exception >> org.apache.zookeeper.KeeperException$ConnectionLossException: >> KeeperErrorCode = ConnectionLoss for /hbase/root-region-server >> at >> org.apache.zookeeper.KeeperException.create(KeeperException.java:99) >> at >> org.apache.zookeeper.KeeperException.create(KeeperException.java:51) >> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1131) >> at >> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getData(RecoverableZooKeeper.java:264) >> at >> org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataInternal(ZKUtil.java:522) >> at >> org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataAndWatch(ZKUtil.java:498) >> at >> org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.getData(ZooKeeperNodeTracker.java:156) >> at >> org.apache.hadoop.hbase.zookeeper.RootRegionTracker.getRootRegionLocation(RootRegionTracker.java:62) >> at >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:821) >> at >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801) >> at >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:933) >> at >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:832) >> at >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801) >> at >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:933) >> at >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:836) >> at >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801) >> at >> org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:234) >> at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:174) >> at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:133) >> at >> org.apache.pig.backend.hadoop.hbase.HBaseStorage.setLocation(HBaseStorage.java:551) >> at >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:385) >> at >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:259) >> at >> org.apache.pig.test.TestJobSubmission.testReducerNumEstimation(TestJobSubmission.java:492) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37) >> at java.lang.reflect.Method.invoke(Method.java:611) >> at >> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) >> at >> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) >> at >> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) >> at >> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) >> at >> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) >> at >> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) >> at >> org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79) >> at >> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71) >> >> >> >> >> >> >> At 2012-08-16 01:30:33,"Cheolsoo Park" <[email protected]> wrote: >> >Hi, >> > >> >Caused by: org.apache.hadoop.hbase.client.NoServerForRegionException: >> >> Unable to find region for passwd,,99999999999999 after 10 tries. >> > >> > >> >It looks like the region server is not available. >> >1) Does TestHBaseStorage pass? I guess that it fails with the same error. >> >2) In your log, do you see any errors related to the HBase region server? >> >For example, java.lang.IllegalStateException: Could not instantiate a >> >region instance. >> > >> >If the region server failed to start, you will see this failure. >> > >> >Thanks, >> >Cheolsoo >> > >> > >> >On Wed, Aug 15, 2012 at 12:56 AM, lulynn_2008 <[email protected]> >> wrote: >> > >> >> Hi All, >> >> >> >> In TestJobSubmission, testReducerNumEstimation failed with following >> error >> >> information. Please give a glance to check what the problem is. Thanks. >> >> >> >> Error information: >> >> Testcase: testReducerNumEstimation took 1,809.419 sec >> >> Caused an ERROR >> >> Internal error creating job configuration. >> >> >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobCreationException: >> >> ERROR 2017: Internal error creating job configuration. >> >> at >> >> >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:731) >> >> at >> >> >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:259) >> >> at >> >> >> org.apache.pig.test.TestJobSubmission.testReducerNumEstimation(TestJobSubmission.java:491) >> >> Caused by: org.apache.hadoop.hbase.client.NoServerForRegionException: >> >> Unable to find region for passwd,,99999999999999 after 10 tries. >> >> at >> >> >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:926) >> >> at >> >> >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:832) >> >> at >> >> >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801) >> >> at >> >> >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:933) >> >> at >> >> >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:836) >> >> at >> >> >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801) >> >> at >> >> org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:234) >> >> at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:174) >> >> at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:133) >> >> at >> >> >> org.apache.pig.backend.hadoop.hbase.HBaseStorage.setLocation(HBaseStorage.java:545) >> >> at >> >> >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:384) >> >> >> >> >> >> >> >>
