bq. HFileScanner for reader reader=hdfs:// 192.168.11.150:8020/hbase/vc2.in_link/6<http://192.168.11.150:8020/hbase/vc2.in_link/6b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75<http://192.168.11.150:8020/hbase/vc2.in_link/6b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75>
You can find, from master log, which region server hosted 6<http://192.168.11.150:8020/hbase/vc2.in_link/6b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75> b879cb43205cdae084a280c38fab34a<http://192.168.11.150:8020/hbase/vc2.in_link/6b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75> Then you can check the region server log on that server. On Mon, Apr 14, 2014 at 10:11 PM, Li Li <[email protected]> wrote: > where to find the server log? I mean there are many region servers. > should I find one by one? > > On Tue, Apr 15, 2014 at 12:31 PM, lars hofhansl <[email protected]> wrote: > > Thanks. > > Was there anything in the server logs at that time? > > The client does not report the full stack trace. > > > > I have not seen this one before. I assume HDFS was running at the time... > > > > > > -- Lars > > > > > > > > ________________________________ > > From: Li Li <[email protected]> > > To: [email protected]; lars hofhansl <[email protected]> > > Sent: Monday, April 14, 2014 9:09 PM > > Subject: Re: hbase exception: Could not reseek StoreFileScanner > > > > > > Version 0.94.11, r1513697, Wed Aug 14 04:54:46 UTC 2013 > > > > > > On Tue, Apr 15, 2014 at 12:03 PM, lars hofhansl <[email protected]> > wrote: > >> Hi Li, > >> > >> please always tell us which version of HBase/Hadoop you are using and > what it is that you were trying to do. > >> Thanks. > >> > >> -- Lars > >> > >> > >> > >> ________________________________ > >> From: Li Li <[email protected]> > >> To: [email protected] > >> Sent: Monday, April 14, 2014 5:32 PM > >> Subject: hbase exception: Could not reseek StoreFileScanner > >> > >> > >> Mon Apr 14 23:54:40 CST 2014, > >> org.apache.hadoop.hbase.client.HTable$9@14923f6b, java.io.IOException: > >> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner > >> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6 > >> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75, > >> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true] > >> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false] > >> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false] > >> [cacheCompressed=false], > >> > firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put, > >> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03 > >> > W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put, > >> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569, > >> > cur=\xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDC > >> F/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key > >> > \xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDCF/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0 > >> 3968 Mon Apr 14 23:55:50 CST 2014, > >> org.apache.hadoop.hbase.client.HTable$9@14923f6b, java.io.IOException: > >> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner > >> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6 > >> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75, > >> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true] > >> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false] > >> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false] > >> [cacheCompressed=false], > >> > firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put, > >> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03 > >> > W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put, > >> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569, > >> > cur=\xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDC > >> F/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key > >> > \xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDCF/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0 > >> 3969 > >> 3970 at > >> > org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:188) > >> 3971 at > >> org.apache.hadoop.hbase.client.HTable.checkAndPut(HTable.java:918) > >> ............... > >> > >> 3975 14-04-14 23:58:14,993 ERROR Thread-8 > >> com.founder.extractor.ExtractWorker Failed after attempts=14, > >> exceptions: > >> 3976 Mon Apr 14 23:51:52 CST 2014, > >> org.apache.hadoop.hbase.client.HTable$9@17b86244, java.io.IOException: > >> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner > >> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6 > >> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75, > >> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true] > >> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false] > >> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false] > >> [cacheCompressed=false], > >> > firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put, > >> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03 > >> > W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put, > >> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569, > >> > cur=\xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x1 > >> 6\x16\x0D&/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key > >> > \xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x16\x16\x0D&/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0 > >> 3977 Mon Apr 14 23:52:02 CST 2014, > >> org.apache.hadoop.hbase.client.HTable$9@17b86244, java.io.IOException: > >> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner > >> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6 > >> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75, > >> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true] > >> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false] > >> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false] > >> [cacheCompressed=false], > >> > firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put, > >> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03 > >> > W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put, > >> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569, > >> > cur=\xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x1 > >> 6\x16\x0D&/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key > >> > \xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x16\x16\x0D&/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0 >
