Let me clarify that a bit, after 10 days there will be about 2 - 3 millions records in that bucket, about 200,000 - 300,000 of those records will need to be pruned off nightly.
-Chad From: riak-users [mailto:[email protected]] On Behalf Of Chad Engler Sent: Thursday, February 07, 2013 10:53 AM To: [email protected] Subject: Nightly Prune We need records in our database to only live for a certain period of time (and we are using Bitcask). At first glance, expiry_secs seems to be exactly what we are looking for. The problem is, these records need to be pruned when they are older than 10 days *according to date information in the record*, not the time that it was inserted. Many times we will repost events and things from the day before or a week before into riak. So if a week old entry goes in, it should only live for 3 more days before being pruned. I've noticed that people are saying map reduces over large buckets are not recommended for things like this, so what is a good way to prune off old entries when they are over 10 days old (according to date data within the record)? Please also note we will have 2 - 3 million keys in a single bucket, which needs to be pruned along with 200,000 - 250,000 keys in another bucket that also need to be pruned. Thanks! Chad Engler Telephony Programmer PATLive 1.800.775.7790 x746 [email protected] <mailto:[email protected]> Hosted Communications | Friendly Service www.patlive.com <http://www.patlive.com/>
_______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
