Re: when to use another document and when not to?

[Sho Fukamachi]

On 21/07/2008, at 7:32 PM, Andrew Richards wrote:

1. Put all of the user's data data into the subscription document at
the time it's written. Then, when you get the subscription documents,
the necessary data will be right there.

Yes - and the advantage of this method is that you can also write "options" in there (such as the aforementioned "email on update" etc, since we all seem to be discussing hypothetical Twitter clones!). I agree data storage space is not really an issue we should care about much these days, especially with small things like user account info.

2. Or, just do a lot of GETs for all of the data you need. It actually
works. (Or even better, get them from something like memcached).

Hm, I would lean away from any n+1 situation like this. Memcached is fast but it's not unlimited .. it's much more elegant to be able to get it all at once if at all possible ..

While the first one does indeed introduce replicated data, it will
yield very nice performance benefits (this is in line with how CouchDB
works as a whole.) How often will users change their full names?
Enough so that you can't go back and rewrite their subscription
documents?

Agreed on all points.

The second one is faster than you think. How often will you need to
get the names of more than, say, 50 subscribers at a time? Does the
user viewing this data really need to be able to see all of this data
on the same page? Even with a lot of documents, pulling from CouchDB
is very fast. Memcached much more so.

There's still unavoidable HTTP overhead - I doubt executing a loop and pulling 100 (or worse, 1000) documents consecutively from any source that's not main memory on the local machine would be good enough for Amazon's 300ms rule of thumb (a good one, IMO). But it is an option.

3. Big joins like this are what make relational databases slow.

Oh, no doubt about it. I actually excised all joins from an RDBMS app about 6 months ago for this very reason, preferring to just pull the data in 2 or 3 stages. The difference is that I could do multi-key GETs (SELECTs).

4. CouchDB is not a replacement for relational databases.

thanks again.

Sho

Definitely, and none of this curbs my enthusiasm for it. All of these nuances can be designed around, possibly using some of the excellent suggestions you've given. Personally the ease of replication of CouchDB databases is such a draw that it renders this discussion practically moot for me anyway - I'm going to use it, JOINs or not, replication is that important to me. However - everyone's been bandying around "couchdb supports JOINs!" and I wanted to either find out what I was missing, or get it settled one way or the other : )

On Mon, Jul 21, 2008 at 12:29 AM, Sho Fukamachi <[EMAIL PROTECTED] > wrote:

On 21/07/2008, at 1:56 PM, Dean Landolt wrote:

Or, obviously, I would be delighted if someone could show me how I'm

completely wrong and it is actually possible to do this : )

You can. Complex keys. I put together a little test:

http://dev.deanlandolt.com:5984/test/_view/subscriptions/users

it so I could examine it locally : )

Firstly, I appreciate the effort you put into your reply. Great to be able to see your solution in action there. And I hope you don't mind I replicated

The map function just uses a two-level key...

function(doc) {
if (doc.type == 'user') {
 emit([doc.username,0], doc)
} else if (doc.type == 'subscription') {
 emit([doc.follower, doc.followee], doc)
}
}

But all that key is doing is sorting the results, right?

Read this for more details:
http://www.cmlenz.net/archives/2007/10/couchdb-joins

problem.

Believe me I've read that about 10 times. I still can't see how to solve the

But yes, you can do joins. You can query this view for just one user
simply:

http://dev.deanlandolt.com:5984/test/_view/subscriptions/users?startkey= [%22dlandolt%22]&endkey=[%22dlandolt%22,{}]

see it returns 3 rows.


your own.


you need is:

function(doc) {
   if (doc.type == 'user') {
     emit(doc.username, doc)
   } else if (doc.type == 'subscription') {
     emit(doc.followee, doc)
   }
}

And you get the exact same results, sans the sorting.


them up one by one. And same in reverse, if you wanted the full user

multi-key gets (see previous ML discussion).


"dlandolt". The user documents, *not* the subscription docs.






is annoying everyone here!

Sho

Again I think I am not making myself clear. If you look at that view, you The first row is the user whose name you have searched upon. In this case The next two rows are the *subscription* documents, which is not what I am talking about. It is easy to get the subscription documents and followee user document for any given followee username. If you abandon sorting all This is not the kind of join I meant - in fact this is not a join at all, as I understand them. A proper join would get you the follower *user* documents - not just the subscriptions. As it stands if you wanted, say, the full names of the follower users, you are then faced with an n+1 query to look documents of all followed users, starting with the follower's username. Making things worse is that CouchDB doesn't currently have the ability to do In other words, with a proper join, starting from the username 'katz', you could get back the *user* documents from both following users "sho" and This is a bit difficult to discuss without ambiguity (or resorting to SQL queries) so let me put it in terms of a use case "challenge" question: with that current DB, can you write a query that, starting with the username "katz", outputs the *names* (not usernames) of all users following him? *That* is a join query and that's what I can't see how to do in CouchDB. Many thanks for the discussion and sorry, again, for not explaining myself properly the first n times... my sincere apologies if my stubborn ignorance

Notice the {} in there -- from what I gather objects are at the bottom of the sort, so this query gets all data related to user dlandolt -- the

first

how

docs
that Damien recommended.

Hope that helps.

result (or any result with the second part of a key ending in 0 based on I wrote my view), and then everything following are the *subscription*

smime.p7s
Description: S/MIME cryptographic signature