What does your regionserver logs have to say for themselves?
On Tue, May 18, 2010 at 11:55 PM, Geoff Hendrey <[email protected]> wrote: > I am experiencing a problem in which Put operations transition from > working just fine, to blocking forever. I am doing Put from a reducer. I > have tried the following, but none of them prevents the Puts from > eventually blocking totally in all the reducers, until the task tracker > kills the task due to 10 minute timeout. > > 1) try using just one reducer (didn't help) > 2) try Put.setWriteToWall both true and false (didn't help) > 3) try autoflush true and false. When true, experiment with different > flush buffer sizes (didn't help) > > I'v been watching the HDFS namenode and datanode logs, and also the > HBase master and region servers. I am running a 3-node HDFS cluster > (20.2) sharing same 3 nodes with HBase 20.3. I see no problems in any > logs, except that the datanode logs eventually stop showing WRITE > operations (corresponding to the Put operations eventually coming to a > halt). The HBase shell remains snappy and I can do list and status > operations and scans without any issue from the shell. > > Anyone ever seen anything like this? > > -geoff > > > <blocked::http://www.decarta.com> > > > > >
