I think that writes to Riak will always be faster than other operations. It has to do with the fact that writes are append only, whereas deletes, for example, must write a tombstone into all of the replicas.
I'm basically paraphrasing this message since I don't understand how this works 100%: http://lists.basho.com/pipermail/riak-users_lists.basho.com/2011-October/006048.html However I do think concurrency is your best bet - we needed a bulk delete job awhile back and the steps were: 1) generate a keylist, split into multiple files 2) spawn a process/thread for each file the key is finding the proper number of processes/files to keep your delete rate acceptable. -matt On Fri, Nov 25, 2011 at 1:06 PM, Keith Irwin <[email protected]> wrote: > I'm still interested in this problem (deletes lagging way behind inserts). Is > there anything about the technology of Riak itself that might explain this, > or is it instead something to do with the map/reduce phase in order to find > keys to delete? > > Andy indicated that the map part of what he was doing was really fast, but > iterating deletes were the problem. > > Perhaps spinning up a lot of concurrent deletes is the key? > > Keith > > On Nov 17, 2011, at 1:29 PM, Bartling, Aron wrote: > >> It's using a map job with key filters to list the keys to delete. This part >> of the process happens in a very reasonable amount of time. >> >> Thanks >> >> -----Original Message----- >> From: [email protected] >> [mailto:[email protected]] On Behalf Of Andy Skelton >> Sent: Thursday, November 17, 2011 11:24 AM >> To: [email protected] >> Subject: Re: Faster deletes? >> >> Aron Bartling wrote: >>> The deletion is based on the timestamp incorporated into the key >> >> Does this mean your deletion process is listing all of the keys in a bucket? >> >> _______________________________________________ >> riak-users mailing list >> [email protected] >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> _______________________________________________ >> riak-users mailing list >> [email protected] >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > _______________________________________________ > riak-users mailing list > [email protected] > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > _______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
