Sorry for the cross-post. Just wanted to make sure I addressed my question to the currently active mailing list. (see question below) -geoff
________________________________ From: Geoff Hendrey Sent: Tuesday, May 18, 2010 11:56 PM To: [email protected] Subject: Put slows down and eventually blocks I am experiencing a problem in which Put operations transition from working just fine, to blocking forever. I am doing Put from a reducer. I have tried the following, but none of them prevents the Puts from eventually blocking totally in all the reducers, until the task tracker kills the task due to 10 minute timeout. 1) try using just one reducer (didn't help) 2) try Put.setWriteToWall both true and false (didn't help) 3) try autoflush true and false. When true, experiment with different flush buffer sizes (didn't help) I'v been watching the HDFS namenode and datanode logs, and also the HBase master and region servers. I am running a 3-node HDFS cluster (20.2) sharing same 3 nodes with HBase 20.3. I see no problems in any logs, except that the datanode logs eventually stop showing WRITE operations (corresponding to the Put operations eventually coming to a halt). The HBase shell remains snappy and I can do list and status operations and scans without any issue from the shell. Anyone ever seen anything like this? -geoff <blocked::http://www.decarta.com>
