Everything Dave said is right, but I might also add that since edits are done in the Memstores and that forcing a major compaction _doesn't_ force flush the Memstores, the deletes won't be taken into account.
HBase also keeps all other sorts of data around, for example look at the .logs directory. That's where the write-ahead-logs are kept and they aren't cleaned that often (it's a matter of time, if you don't insert anything then one log will get cleaned per hour). J-D On Fri, Aug 26, 2011 at 9:12 AM, Buttler, David <[email protected]> wrote: > Major_compact is an asynchronous command. It may take some time for space to > be freed. If you are watching your data node logs you should be able to see > blocks being deleted. If you are on the local file system, you should be > table to periodically do a 'du -sh' command on the hbase table directory and > see the used space diminish. > > Dave > > -----Original Message----- > From: Joel Halbert [mailto:[email protected]] > Sent: Friday, August 26, 2011 2:37 AM > To: [email protected] > Subject: Re: delete does not free hdd space > > Hi, > > I experience exactly the same thing. > > After deleting many rows I do not see a decrease in size of data on > disk. > > When I run a major compact manually from the shell it outputs the same > as for Sasha: > > 0 row(s) in 0.9210 seconds > > I am running HBase standalone (without hadoop) on local disk. > > On Fri, 2011-08-26 at 10:42 +0300, sasha.maksimenko wrote: >> hi! >> hm >> I've deleted and after that run major_compact >> >> HBase Shell; enter 'help<RETURN>' for list of supported commands. >> Version: 0.20.6, r965666, Mon Jul 19 15:48:07 PDT 2010 >> hbase(main):001:0> major_compact 'task' >> 0 row(s) in 0.9210 seconds >> hbase(main):002:0> >> >> >> is it could be a problem that I use local file system - not HDFS? >> >> >> > If you run a major_compact from the shell that should purge out deleted >> > records. By default the major compaction happens on every table every 24 >> > hours. >> > >> > -chris >> > >> > On Aug 25, 2011, at 12:17 AM, sasha.maksimenko wrote: >> > >> >> hi! >> >> I need to free hdd space, so I want to delete some unimportant rows >> >> I'm using Thrift call >> >> deleteAllRow(TABLE, ROWID); >> >> to perform deletion of about 10 000 row (when I loaded them it was about >> >> 30Gb). When I perform this operation I checked thru hbase shell that rows >> >> are deleted, but used space of the disk is not changed >> >> I performed compact and major_compact operations (they returned 0 >> >> processed rows) but space is not changed too. >> >> >> >> So quesition what should I also do to indicate hbase physically remove >> >> this rows >> > > >
