On Tue, Mar 11, 2014 at 12:34 PM, Matt Ingenthron <[email protected]>wrote:
> Hi Alk, > > Please see inline… > > From: Aliaksey Kandratsenka <[email protected]> > Reply-To: "[email protected]" <[email protected]> > Date: Tuesday, March 11, 2014 10:58 AM > To: "[email protected]" <[email protected]> > Subject: Re: request for comments: Project Left Ranger, range operations > in Couchbase > > > Very interesting. What's most unclear especially if you compare it to > couchdb's _all_docs is whether this will be limited to single vbucket or > not. > > > I've gone back and forth on this one. At the moment, I think the best > thing to do is for this to be able to target vbuckets or all vbuckets on a > given node. Unlike the _all_docs which has to do the merge-sort in one > place at the cluster, this does not define any collation and the client > would handle the results. This should be very efficient and scale well. > > > If it's limited to single vbucket then it's not as useful as _all_docs > (unless we also deploy long due client-side extension to control sharding > of docs into vbuckets). If it's across all vbuckets (like poor and "gonna > die" _all_docs implementation of couchbase is doing) then it's risking to > be too slow. > > > Hm. I don't agree. I mean, it's 1024 * log(n) if you don't control > sharding to buckets (which is not proposed), but it should not be slow > since it's going straight to couchstore via ep-engine. Yes, there's a big > burden at the client side. This may be in the client library or at the > application. > > So, in that way, it's different than what's happening for all_docs's > replacement. > For most practical purposes (not very big limit) it's going to be tons of times slower than index-based range query. And it's not going to be significantly faster than _all_docs today. Because with sufficiently big amount of data (and that's not even big "disk greater than memory") there will be one seek per vbucket. Compare that with single index that pre-aggregates results from all vbuckets and in most cases needs just single walk down btree (and usually 1 or few seeks) to fetch a whole bunch of doc ids. As I've tried to point our earlier multiple times we're dealing with fundamental issue here. If we have to visit all vbuckets just to get 10 ids greater than given key and do seek per vbucket that's a non-starter. Possible solutions could be: * switch to range sharding in K/V layer so that you have good chance of hitting just single partition (equivalent of vbucket). Pretty major change obviously * make ep-engine inherently aware of orderedness of keys. Both in ram and on disk. The later appears to require storing multiple vbuckets per file. Allowing for single by-id btree across many vbuckets and therefore solving seeks problem. Pretty major change as well * have fast specialized by-id secondary key index. That's AFAIK direction we're heading towards already. * go towards better better app-level control on vbucket sharding. Oracle's nosql product as well as Amazon's allow for that. In that case, client will be able to solve useful problems (e.g. "show me all email threads in _my_ inbox sorted by date") by doing range queries just in single partition. (as well as unlocking lots of other things people have been talking for a while, like transactions). Or some combination of above or maybe something I'm not aware of yet. But hopefully I've made it clear enough that "just exposing by-id btree" is not an option performance-wise. In any case MB I've pointed out above is going to be implemented soon (ep-engine part is done, hopefully efficiently; and ns_server part is in progress right now and it will be efficient). And we'll soon see if I'm wrong or not. -- You received this message because you are subscribed to the Google Groups "Couchbase" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
