Re: Using the 'fold' keys threshold

2014-06-05 Thread Alex Moore
Hi Venkat,

You can find those settings in our docs: 
http://docs.basho.com/riak/1.4.9/ops/advanced/backends/bitcask/#Configuring-Bitcask
 (search for “Fold Keys Threshold”).   

In Bitcask when we do range operations like “List Keys” or other operations 
that require us to fold over all the data, we take a snapshot of the “Keydir” 
to get a consistent read.  The Keydir is the hash table that holds the `key- 
latest object` mapping.  When we do this snapshot, we also start a delta of any 
changes since the snapshot.

We use the two “Fold Keys Threshold” options `max_fold_age` and `max_fold_puts` 
only when Bitcask is processing one fold operation, and gets a request for a 
second one.  These two options let the user choose whether to reuse the 
snapshot, or to block and get a new snapshot before starting the second fold.  
This lets you tradeoff between a potential performance boost and consistency.

By default we have Bitcask side toward consistency; it will get a new snapshot, 
by setting `max_fold_puts` to `0`.  If any new puts come in, we must grab a new 
snapshot before folding again. 

- Increasing `max_fold_puts` to `n` will let Bitcask reuse the snapshot 
if there are fewer than `n` changes in the delta.
- Increasing `max_fold_age` to `s` will let Bitcask reuse the snapshot 
if the snapshot is younger than `s` microseconds. 

Setting either of these to positive values can let folds ignore recent changes, 
so you can run into stale data. Because of that, we recommend that you don’t 
change them.
I hope this helps.

Thanks,
Alex


On May 13, 2014, at 3:02 PM, Venkatachalam Subramanian 
venkatsubb...@gmail.com wrote:

 Hi All,
 
 It was very helpful to get my first few questions about Riak/Bitcacsk 
 answered pretty quickly.
 
 I just have a another question on the same lines,
 
 I ran across the 'fold keys threshold' option in riak/bitcask.
 I could not find enough information about the 'fold keys' option to 
 understand it completely.
 
 Could someone tell me what 'fold keys' option is? what does it do? when could 
 we use it? 
 Does it help when you want to get the list of all keys available?
 
 I greatly appreciate your help.
 Thank You.
 
 -- 
 Regards,
 Venkat Subramanian
 
 
 ___
 riak-users mailing list
 riak-users@lists.basho.com
 http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


Using the 'fold' keys threshold

2014-05-13 Thread Venkatachalam Subramanian
Hi All,

It was very helpful to get my first few questions about Riak/Bitcacsk
answered pretty quickly.

I just have a another question on the same lines,

I ran across the 'fold keys threshold' option in riak/bitcask.
I could not find enough information about the 'fold keys' option to
understand it completely.

Could someone tell me what 'fold keys' option is? what does it do? when
could we use it?
Does it help when you want to get the list of all keys available?

I greatly appreciate your help.
Thank You.

-- 

*Regards, *Venkat Subramanian
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com