Beside the HBase side configuration, you need to add the following to your ZOOKEEPER/conf/zoo.cfg, and then restart ZooKeeper will allow you to change the 40 secs. maxSessionTimeout= 120000
Details can be found here. http://hadoop.apache.org/zookeeper/docs/current/zookeeperProgrammers.html#ch_zkSessions "The client sends a requested timeout, the server responds with the timeout that it can give the client. " The 40 secs timeout is because, ZK's tickTime is set to 2 secs by default, and a maximum timeout of 20 times the tickTime that ZK server will response. -Yifeng On May 7, 2012, at 2:07 PM, Billy Pearson wrote: > I am running test on the bulk load option on hbase I run the map and > everything runs fine for small data but I tryto load 3-500MB of data and I > get timed out and it tries to load the data several times > I found the problem just can not find how to fix it how to do you change the > negotiated timeout = 40000 to something else > > I tried to do it in the job config before I load the data to a high number to > see if it helped > conf.set("zookeeper.session.timeout", "1210000"); > but that does not seams to change the 40 sec timer below. > > So the question is what option changes the 40 secs and where does it go in > the conf in zookeeper/hbase/ or hadoop conf file? > also can I change it just for this job or does it have to be changed cluster > wide? > > 12/05/06 23:54:23 INFO zookeeper.ZooKeeper: Initiating client connection, > connectString=node2:2181,node1:2181 sessionTimeout=180000 watcher=hconnection > 12/05/06 23:54:23 INFO zookeeper.ClientCnxn: Opening socket connection to > server node1/10.0.0.71:2181 > 12/05/06 23:54:23 INFO zookeeper.ClientCnxn: Socket connection established to > node1/10.0.0.71:2181, initiating session > 12/05/06 23:54:23 INFO zookeeper.ClientCnxn: Session establishment complete > on server node1/10.0.0.71:2181, sessionid = 0x37100d3f3000f3, negotiated > timeout = 40000 > 12/05/06 23:54:24 INFO zookeeper.ZooKeeper: Initiating client connection, > connectString=node2:2181,node1:2181 sessionTimeout=1210000 watcher=hconnection > 12/05/06 23:54:24 INFO zookeeper.ClientCnxn: Opening socket connection to > server node2/10.0.0.72:2181 > 12/05/06 23:54:24 INFO zookeeper.ClientCnxn: Socket connection established to > node2/10.0.0.72:2181, initiating session > 12/05/06 23:54:24 INFO zookeeper.ClientCnxn: Session establishment complete > on server node2/10.0.0.72:2181, sessionid = 0x137100d3f2600ee, negotiated > timeout = 40000 > > > code used to load after the mr job is done > > conf.set("zookeeper.session.timeout", "1210000"); > conf.set("hbase.zookeeper.property.tickTime", "6000"); > HTable table = new HTable("Repo"); > LoadIncrementalHFiles lh = new LoadIncrementalHFiles(conf); > lh.doBulkLoad(new Path(otherArgs[1]), table); > > >
