@John, I'm definitely looking forward to CRDT's but at the same time i'm looking into alternative approaches for achieving the same thing.
@Jason, your description is close to what i had in mind. Only real difference is merge would be on read. I did some testing and m/r seems to work by using an initial map phase calling `riak_object:get_values` There's also the addition of maximum number of siblings in riak-2.0[1] 2013/11/13 John Daily <[email protected]> > Jason, I don’t see any inherent problems, given reasonable management of > the situation as you describe. I’d have to chase the code path to see what > overhead you’re introducing to Riak’s processing, but if it’s working well > for you, then who am I to object? > > Perhaps someone who’s more familiar with the sibling management code could > chime in. > > -John > > > On Nov 12, 2013, at 5:10 PM, Jason Campbell <[email protected]> wrote: > > I am currently forcing siblings for time series data. The maximum bucket > sizes are very predictable due to the nature of the data. I originally used > the get/update/set cycle, but as I approach the end of the interval, > reading and writing 1MB+ objects at a high frequency kills network > bandwidth. So now, I append siblings, and I have a cron that merges the > previous siblings (a simple set union works for me, only entire objects are > ever deleted). > > I can see how it can be dangerous to insert siblings, bit if you have some > other method of knowing how much data is in one, I don't see size being an > issue. I have also considered using a counter to know how large an object > is without fetching it, which shouldn't be off by more than a few siblings > unless there is a network partition. > > So aside from size issues, which can be roughly predicted or worked > around, is there any reason to not create hundreds or thousands of siblings > and resolve them later? I realise sets could work well for my use case, but > they seem overkill for simple append operations when I don't need delete > functionality. Creating your own CRDTs are trivial if you never need to > delete. > > Thoughts are welcome, > Jason > *From: *John Daily > *Sent: *Wednesday, 13 November 2013 3:10 AM > *To: *Olav Frengstad > *Cc: *riak-users > *Subject: *Re: Forcing Siblings to Occur > > Forcing siblings other than for testing purposes is not typically a good > idea; as you indicate, the object size can easily become a problem as all > siblings will live inside the same Riak value. > > Your counter-example sounds a lot like a use case for server-side CRDTs; > data structures that allow the application to add values without retrieving > the server-side content first, and siblings are resolved by Riak. > > These will arrive with Riak 2.0; see > https://gist.github.com/russelldb/f92f44bdfb619e089a4d for an overview. > > -John > > On Nov 12, 2013, at 7:13 AM, Olav Frengstad <[email protected]> wrote: > > Do you consider forcing siblings a good idea? I would like to get some > input on possible use cases and pitfalls. > For instance i have considered to force siblings and then merge them on > read instead of fetching an object every time i want to update it > (especially with larger objects). > > It's not clear from the docs if there are any limitations, will the > maximum object size be the limitation:? > > A section of the docs[1] comees comes to mind: > > "Having an enormous object in your node can cause reads of that object to > crash the entire node. Other issues are increased cluster latency as the > object is replicated and out of memory errors." > > [1] > http://docs.basho.com/riak/latest/theory/concepts/Vector-Clocks/#Siblings > > 2013/11/9 Brian Roach <[email protected]> > >> On Fri, Nov 8, 2013 at 11:38 AM, Russell Brown <[email protected]> >> wrote: >> >> > If you’re using a well behaved client like the Riak-Java-Client, or any >> other that gets a vclock before doing a put, use whatever option stops that. >> >> for (int i = 0; i < numReplicasWanted; i++) { >> bucket.store("key", "value").withoutFetch().execute(); >> } >> >> :) >> >> - Roach >> >> _______________________________________________ >> riak-users mailing list >> [email protected] >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> > _______________________________________________ > riak-users mailing list > [email protected] > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > > _______________________________________________ > riak-users mailing list > [email protected] > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > -- Med Vennlig Hilsen Olav Frengstad Systemutvikler // FWT +47 920 42 090
_______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
