I have found this from maillist, maybe update to cdh3u3 or 0.90.5 will solve this problem, thanks all
It's supposed to but there are a few leaks such as: https://issues.apache.org/jira/browse/HBASE-4799 https://issues.apache.org/jira/browse/HBASE-4238 J-D On Mon, Nov 21, 2011 at 7:32 PM, 吕鹏 <[EMAIL PROTECTED]> wrote: > I set "hbase.hregion.max.filesize" to 9223372036854775807. When an average > region size on HDFS reaches 6G, i split all the regions in my hbase table > by using hbase java api HBaseAdmin.split. Before i do split, there are 1000 > regions in my table, and when the split is finished, there a 2000 region is > available as expected. But on the HDFS, there are 3000 region directories > in the /hbase/tablename/columnfamily/*. Can hbase delete the old regions > after region split automatically? > > *b2fb55650a67b28b791be4d6f813de8a *is one of the original regions before > split, and the hbase master log is shown as follows: > [hadoop@master3-hbase logs]$ grep b2fb55650a67b28b791be4d6f813de8a > hbase-hadoop-master-master3-hbase.sii.cn.log > 11/11/17 17:04:58 INFO master.ServerManager: Received REGION_SPLIT: > pageDB,4a7ef985,1320941062356.b2fb55650a67b28b791be4d6f813de8a.: > Daughters; pageDB,4a7ef985,1321520697601.aa5828bfb105e1109aa676e5361b1793., > pageDB,4a9fd9f59468cba2e7d210f41811e349,1321520697601.b0b4e65db3251fc48dc70c5484084a34. > from slave5-hbase.sii.cn,60020,1320980409810 > 11/11/17 23:45:30 INFO catalog.MetaEditor: Deleted daughter reference > pageDB,4a7ef985,1321520697601.aa5828bfb105e1109aa676e5361b1793., > qualifier=splitA, from parent pageDB,4a7ef985,1320941062356.* > b2fb55650a67b28b791be4d6f813de8a*. > 11/11/18 00:10:30 INFO catalog.MetaEditor: Deleted daughter reference > pageDB,4a9fd9f59468cba2e7d210f41811e349,1321520697601.b0b4e65db3251fc48dc70c5484084a34., > qualifier=splitB, from parent pageDB,4a7ef985,1320941062356.* > b2fb55650a67b28b791be4d6f813de8a*. > > I have load mydata into hbase. > After hbase calm down, I found that the data size is about 10 times the > orignal data size. > Finally I found the number of directory is /hbase/tableName is much more > than the region number > When I use hadoop fs -dus /hbase/tableName to see the size of the > directorys, I found several dir take up more than 10GB spaces and they > are not list in the table regions.In my opion, they are parent region > directories that forget to clean. > I'm try to do major_compact on this table now,but i don't know if this > can solve the problem. Anyone can help to clear those dirs? > >
