Hi
I'm trying to write a unit test for HBase coprocessor. However it seems
I'm doing something horribly wrong. The code I'm using to test my
coprocessor class is in the attachment.
As you can see, I'm using HBaseTestingUtility, and running a
mini-cluster with it. The error I keep getting is:
2012-04-12 13:00:39,924 [6,1334228432020] WARN RecoverableZooKeeper
:117 - Node /hbase/root-region-server already deleted, and this is
not a retry
2012-04-12 13:00:39,995 [6,1334228432020] INFO HBaseRPC
:240 - Server at localhost/127.0.0.1:45664 could not be reached
after 1 tries, giving up.
2012-04-12 13:00:39,995 [6,1334228432020] WARN AssignmentManager
:1493 - Failed assignment of -ROOT-,,0.70236052 to
localhost,45664,1334228432229, trying to assign elsewhere instead; retry=0
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting
up proxy interface org.apache.hadoop.hbase.ipc.HRegionInterface to
localhost/127.0.0.1:45664 after attempts=1
at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:242)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1235)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1222)
at
org.apache.hadoop.hbase.master.ServerManager.getServerConnection(ServerManager.java:496)
at
org.apache.hadoop.hbase.master.ServerManager.sendRegionOpen(ServerManager.java:429)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1453)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1200)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1175)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1170)
at
org.apache.hadoop.hbase.master.AssignmentManager.assignRoot(AssignmentManager.java:1918)
at
org.apache.hadoop.hbase.master.HMaster.assignRootAndMeta(HMaster.java:557)
at
org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:491)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:326)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
at
org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:656)
at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupConnection(HBaseClient.java:328)
at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:362)
at
org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1026)
at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:878)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
at $Proxy22.getProtocolVersion(Unknown Source)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:183)
at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303)
at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280)
at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332)
at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236)
... 14 more
2012-04-12 13:00:39,998 [6,1334228432020] WARN AssignmentManager
:1504 - Unable to find a viable location to assign region
-ROOT-,,0.70236052
2012-04-12 13:00:44,138 [.timeoutMonitor] INFO AssignmentManager
:2570 - Regions in transition timed out: -ROOT-,,0.70236052
state=OFFLINE, ts=1334228439998, server=null
2012-04-12 13:00:44,141 [.timeoutMonitor] INFO AssignmentManager
:2581 - Region has been OFFLINE for too long, reassigning
-ROOT-,,0.70236052 to a random server
2012-04-12 13:00:44,158 [pool-6-thread-1] INFO HBaseRPC
:240 - Server at localhost/127.0.0.1:45664 could not be reached
after 1 tries, giving up.
This may be related to me using the initHRegion() function - perhaps
that region cannot connect to the newly created HBase cluster?
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.hbase.*;
import org.apache.hadoop.hbase.client.Get;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.coprocessor.BaseRegionObserver;
import org.apache.hadoop.hbase.coprocessor.ObserverContext;
import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment;
import org.apache.hadoop.hbase.regionserver.HRegion;
import org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost;
import org.apache.hadoop.hbase.regionserver.wal.WALEdit;
import org.apache.hadoop.hbase.util.Bytes;
public class CoprocessorTest {
HRegion initHRegion (byte [] tableName, String callingMethod, Configuration conf, byte [] ... families) throws IOException {
HTableDescriptor htd = new HTableDescriptor(tableName);
for(byte [] family : families) {
htd.addFamily(new HColumnDescriptor(family));
}
HRegionInfo info = new HRegionInfo(htd.getName(), null, null, false);
Path path = new Path(DIR + callingMethod);
HRegion r = HRegion.createHRegion(info, path, conf, htd);
RegionCoprocessorHost host = new RegionCoprocessorHost(r, null, conf);
r.setCoprocessorHost(host);
return r;
}
@Test
public void testDataModification() throws Exception {
HBaseTestingUtility cluster = new HBaseTestingUtility();
cluster.startMiniCluster();
byte[] ROW = Bytes.toBytes("testRow");
byte[] TABLE = Bytes.toBytes(getClass().getName());
byte[] A = Bytes.toBytes("A");
byte[] STATS = Bytes.toBytes("stats");
byte[] CONTENT = Bytes.toBytes("content");
byte[][] FAMILIES = new byte[][] { A, STATS, CONTENT } ;
Configuration conf = HBaseConfiguration.create();
HRegion region = initHRegion(TABLE, getClass().getName(), conf, FAMILIES);
RegionCoprocessorHost h = region.getCoprocessorHost();
h.load(MyCoprocessor.class, Coprocessor.PRIORITY_HIGHEST, conf);
Put put = new Put(ROW);
put.add(A, A, A);
put.add(CONTENT, Bytes.toBytes("raw"), Bytes.toBytes("ABC");
int lockid = region.obtainRowLock(ROW);
region.put(put, lockid);
region.releaseRowLock(lockid);
}
}
