I am running test on the bulk load option on hbase I run the map and
everything runs fine for small data but I tryto load 3-500MB of data and I
get timed out and it tries to load the data several times
I found the problem just can not find how to fix it how to do you change the
negotiated timeout = 40000 to something else
I tried to do it in the job config before I load the data to a high number
to see if it helped
conf.set("zookeeper.session.timeout", "1210000");
but that does not seams to change the 40 sec timer below.
So the question is what option changes the 40 secs and where does it go in
the conf in zookeeper/hbase/ or hadoop conf file?
also can I change it just for this job or does it have to be changed cluster
wide?
12/05/06 23:54:23 INFO zookeeper.ZooKeeper: Initiating client connection,
connectString=node2:2181,node1:2181 sessionTimeout=180000
watcher=hconnection
12/05/06 23:54:23 INFO zookeeper.ClientCnxn: Opening socket connection to
server node1/10.0.0.71:2181
12/05/06 23:54:23 INFO zookeeper.ClientCnxn: Socket connection established
to node1/10.0.0.71:2181, initiating session
12/05/06 23:54:23 INFO zookeeper.ClientCnxn: Session establishment complete
on server node1/10.0.0.71:2181, sessionid = 0x37100d3f3000f3, negotiated
timeout = 40000
12/05/06 23:54:24 INFO zookeeper.ZooKeeper: Initiating client connection,
connectString=node2:2181,node1:2181 sessionTimeout=1210000
watcher=hconnection
12/05/06 23:54:24 INFO zookeeper.ClientCnxn: Opening socket connection to
server node2/10.0.0.72:2181
12/05/06 23:54:24 INFO zookeeper.ClientCnxn: Socket connection established
to node2/10.0.0.72:2181, initiating session
12/05/06 23:54:24 INFO zookeeper.ClientCnxn: Session establishment complete
on server node2/10.0.0.72:2181, sessionid = 0x137100d3f2600ee, negotiated
timeout = 40000
code used to load after the mr job is done
conf.set("zookeeper.session.timeout", "1210000");
conf.set("hbase.zookeeper.property.tickTime", "6000");
HTable table = new HTable("Repo");
LoadIncrementalHFiles lh = new LoadIncrementalHFiles(conf);
lh.doBulkLoad(new Path(otherArgs[1]), table);
