Dmitri, yes, I have AAE turned on and this discussion seems relevant to my case. As far as I have understood I have to wait for every tree to expire after changing expiry_secs setting (which should be less than a week). But according to 'riak-admin aee-status' entropy tree built time is less than 3 days for every partition while setting change had been made earlier. And only one node actually dropped its memory consumption by now. Am I missing something?
May be turning AEE off (at least for some time) will help me to reduce memory consumption since no trees with expired keys at all would be built? On 28 August 2013 19:12, Dmitri Zagidulin <[email protected]> wrote: > Alexander, quick question - do you have Active Anti-Entropy turned on? If > yes, check out this discussion: > http://riak-users.197444.n3.nabble.com/Active-Anti-Entropy-with-Bitcask-Key-Expiry-td4027688.html > (might shed more light on the matter). > > >> is there a cheap way to figure out the number of keys stored in a bucket? > No. You can do a streaming list keys, which is less expensive than listing > keys used to be, but I'm not sure that counts as cheap. > > >> how long does it take to purge old keys from the storage? > This depends on how often Bitcask merges are performed. The expired keys are > actually removed from disk when merging/compaction occurs (though until > then, they will return as not found to client requests). > > >> is it possible to see how many keys were expired for the last >> minute/hour/day? > I don't think so - Bitcask doesn't send any notification to Riak when it > expires the keys, and I don't think it keeps any records of its own. > > > > On Wed, Aug 28, 2013 at 4:56 AM, Alexander Ilyin <[email protected]> > wrote: >> >> Hi, >> >> we're using Riak 1.3.1 with a Bitcask storage engine on a 4 node cluster. >> Properties of the bucket used: >> {"props":{"allow_mult":false," >> >> basic_quorum":false,"big_vclock":50,"chash_keyfun":{"mod":"riak_core_util","fun":"chash_std_keyfun"},"dw":"quorum","last_write_wins":false,"linkfun":{"mod":"riak_kv_wm_link_walker","fun":"mapreduce_linkfun"},"n_val":2,"name":"c","notfound_ok":true,"old_vclock":86400,"postcommit":[],"pr":0,"precommit":[],"pw":0,"r":"one","rw":"quorum","small_vclock":50,"w":"one","young_vclock":20}} >> >> 3 nodes crashed today morning almost simultaneously, fourth node had >> crashed earlier this nignt. It's hard to figure out the reason of the crash >> looking into the logs, but I suppose this is due to insufficient RAM >> available on the servers (this maillist doesn't like big attachments but I >> can send logs if it will help). >> >> Actually, I'm dealing with Riak memory usage for some time already. Some >> time ago I figured out that memory usage is much more than it is promised in >> the documentation >> (http://riak-users.197444.n3.nabble.com/memory-consumption-td4028674.html). >> After that I tried to reduce number of keys stored, changing the expiry_secs >> setting from 2 months to 1 month. I expected the memory usage to drop in >> half because new keys are put into Riak at a approximately same rate for >> last two months and Bitcask stores all keys in memory. But that didn't >> happen. Memory usage have dropped a little but grew again in a few days. >> >> So the questions are: >> * is there a cheap way to figure out the number of keys stored in a >> bucket? >> * how long does it take to purge old keys from the storage? >> * is it possible to see how many keys were expired for the last >> minute/hour/day? >> * are there some other ways to reduce memory usage I'm missing except for >> adding new servers? >> >> Thanks in advance. >> >> _______________________________________________ >> riak-users mailing list >> [email protected] >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> > _______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
