How to delete rows in a FIFO manner

2010-08-06 Thread Thomas Downing
Hi, Continuing with testing HBase suitability in a high ingest rate environment, I've come up with a new stumbling block, likely due to my inexperience with HBase. We want to keep and purge records on a time basis: i.e, when a record is older than say, 24 hours, we want to purge it from the

Re: How to delete rows in a FIFO manner

2010-08-06 Thread Jean-Daniel Cryans
If the inserts are coming from more than 1 client, and your are trying to delete from only 1 client, then likely it won't work. You could try using a pool of deleters (multiple threads that delete rows) that you feed from the scanner. Or you could run a MapReduce that would parallelize that for

Re: How to delete rows in a FIFO manner

2010-08-06 Thread Venkatesh
Subject: Re: How to delete rows in a FIFO manner Thanks for the suggestions. The problem isn't generating the Delete objects, or the delete operation itself - both are fast enough. The problem is generating the list of row keys from which the Delete objects are created. For now

Re: How to delete rows in a FIFO manner

2010-08-06 Thread Thomas Downing
: Thomas Downingtdown...@proteus-technologies.com To: user@hbase.apache.orguser@hbase.apache.org Sent: Fri, Aug 6, 2010 11:39 am Subject: Re: How to delete rows in a FIFO manner Thanks for the suggestions. The problem isn't generating the Delete objects, or the delete operation itself - both