Thanks. Can you grep your master log for all lines including fc_test,751960d5f43b5e53e806a89afbbd0b819f30de7369db613413022d99a77b54b8,1256135307641 ?
Best regards, - Andy ________________________________ From: elsif <[email protected]> To: [email protected] Sent: Fri, October 23, 2009 12:45:23 PM Subject: Re: HBase Exceptions on version 0.20.1 Looks like the file was part of a compaction: 2009-10-21 11:58:38,434 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region fc_test,d46a332f60405431f0206d47143044676f1ff15239f24cb72c9743145d15f3e3,1256134371826 2009-10-21 11:58:38,442 DEBUG org.apache.hadoop.hbase.regionserver.Store: Skipped compaction of 1 file; compaction size of data: 173.6m; Skipped 3 files, size: 176812375 2009-10-21 11:58:38,451 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region fc_test,d46a332f60405431f0206d47143044676f1ff15239f24cb72c9743145d15f3e3,1256134371826 in 0sec 2009-10-21 11:58:38,451 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region fc_test,751960d5f43b5e53e806a89afbbd0b819f30de7369db613413022d99a77b54b8,1256135307641 2009-10-21 11:58:38,459 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of data: 170.2m; Skipped 1 file(s), size: 131805190 2009-10-21 11:58:38,459 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s) into /hbase/fc_test/compaction.dir/909519914, seqid=113712 2009-10-21 11:58:38,913 INFO org.apache.hadoop.hdfs.DFSClient: Could not obtain block blk_-6090437519946022724_469173 from any node: java.io.IOException: No live nodes contain current block I don't see anything in the master log relating to this: 2009-10-21 11:58:31,254 INFO org.apache.hadoop.hbase.master.BaseScanner: RegionManager.metaScanner scan of 1406 row(s) of meta region {server: 10.1.29.20:60020, regionname: .META.,,1, startKey: <>} complete 2009-10-21 11:58:31,254 INFO org.apache.hadoop.hbase.master.BaseScanner: All 1 . META. region(s) scanned 2009-10-21 11:59:26,548 INFO org.apache.hadoop.hbase.master.BaseScanner: RegionManager.metaScanner scanning meta region {server: 10.1.29.20:60020, regionname: .META.,,1, startKey: <>} Andrew Purtell wrote: > Please ignore my earlier response. That does look suspiciously like client > error. > > Can you correlate the block > /mnt/8d8aa987-017c-4f4c-822c-fe53f4163cee/current/subdir18/blk_-6090437519946022724 > to a file? And this file to a region? What is in the master log as relates > to this region around the time of the block errors? > > - Andy > >
