----- Original Message ----
From: Sho Fukamachi <[EMAIL PROTECTED]>
To: [email protected]
Sent: Monday, July 21, 2008 5:56:51 AM
Subject: Re: when to use another document and when not to?
On 21/07/2008, at 7:32 PM, Andrew Richards wrote:
1. Put all of the user's data data into the subscription document at
the time it's written. Then, when you get the subscription documents,
the necessary data will be right there.
Yes - and the advantage of this method is that you can also write
"options" in there (such as the aforementioned "email on update" etc,
since we all seem to be discussing hypothetical Twitter clones!). I
agree data storage space is not really an issue we should care about
much these days, especially with small things like user account info.
2. Or, just do a lot of GETs for all of the data you need. It
actually
works. (Or even better, get them from something like memcached).
Hm, I would lean away from any n+1 situation like this. Memcached is
fast but it's not unlimited .. it's much more elegant to be able to
get it all at once if at all possible ..
While the first one does indeed introduce replicated data, it will
yield very nice performance benefits (this is in line with how
CouchDB
works as a whole.) How often will users change their full names?
Enough so that you can't go back and rewrite their subscription
documents?
Agreed on all points.
The second one is faster than you think. How often will you need to
get the names of more than, say, 50 subscribers at a time? Does the
user viewing this data really need to be able to see all of this data
on the same page? Even with a lot of documents, pulling from CouchDB
is very fast. Memcached much more so.
There's still unavoidable HTTP overhead - I doubt executing a loop and
pulling 100 (or worse, 1000) documents consecutively from any source
that's not main memory on the local machine would be good enough for
Amazon's 300ms rule of thumb (a good one, IMO). But it is an option.
3. Big joins like this are what make relational databases slow.
Oh, no doubt about it. I actually excised all joins from an RDBMS app
about 6 months ago for this very reason, preferring to just pull the
data in 2 or 3 stages. The difference is that I could do multi-key
GETs (SELECTs).
4. CouchDB is not a replacement for relational databases.
Definitely, and none of this curbs my enthusiasm for it. All of these
nuances can be designed around, possibly using some of the excellent
suggestions you've given. Personally the ease of replication of
CouchDB databases is such a draw that it renders this discussion
practically moot for me anyway - I'm going to use it, JOINs or not,
replication is that important to me.
However - everyone's been bandying around "couchdb supports JOINs!"
and I wanted to either find out what I was missing, or get it settled
one way or the other : )
thanks again.
Sho
On Mon, Jul 21, 2008 at 12:29 AM, Sho Fukamachi <[EMAIL PROTECTED]
wrote:
On 21/07/2008, at 1:56 PM, Dean Landolt wrote:
Or, obviously, I would be delighted if someone could show me how
I'm
completely wrong and it is actually possible to do this : )
You can. Complex keys. I put together a little test:
http://dev.deanlandolt.com:5984/test/_view/subscriptions/users
Firstly, I appreciate the effort you put into your reply. Great to
be able
to see your solution in action there. And I hope you don't mind I
replicated
it so I could examine it locally : )
The map function just uses a two-level key...
function(doc) {
if (doc.type == 'user') {
emit([doc.username,0], doc)
} else if (doc.type == 'subscription') {
emit([doc.follower, doc.followee], doc)
}
}
But all that key is doing is sorting the results, right?
Read this for more details:
http://www.cmlenz.net/archives/2007/10/couchdb-joins
Believe me I've read that about 10 times. I still can't see how to
solve the
problem.
But yes, you can do joins. You can query this view for just one
user
simply:
http://dev.deanlandolt.com:5984/test/_view/subscriptions/users?startkey=
[%22dlandolt%22]&endkey=[%22dlandolt%22,{}]
Again I think I am not making myself clear. If you look at that
view, you
see it returns 3 rows.
The first row is the user whose name you have searched upon. In
this case
your own.
The next two rows are the *subscription* documents, which is not
what I am
talking about. It is easy to get the subscription documents and
followee
user document for any given followee username. If you abandon
sorting all
you need is:
function(doc) {
if (doc.type == 'user') {
emit(doc.username, doc)
} else if (doc.type == 'subscription') {
emit(doc.followee, doc)
}
}
And you get the exact same results, sans the sorting.
This is not the kind of join I meant - in fact this is not a join
at all, as
I understand them. A proper join would get you the follower *user*
documents
- not just the subscriptions. As it stands if you wanted, say, the
full
names of the follower users, you are then faced with an n+1 query
to look
them up one by one. And same in reverse, if you wanted the full user
documents of all followed users, starting with the follower's
username.
Making things worse is that CouchDB doesn't currently have the
ability to do
multi-key gets (see previous ML discussion).
In other words, with a proper join, starting from the username
'katz', you
could get back the *user* documents from both following users "sho"
and
"dlandolt". The user documents, *not* the subscription docs.
This is a bit difficult to discuss without ambiguity (or resorting
to SQL
queries) so let me put it in terms of a use case "challenge"
question: with
that current DB, can you write a query that, starting with the
username
"katz", outputs the *names* (not usernames) of all users following
him?
*That* is a join query and that's what I can't see how to do in
CouchDB.
Many thanks for the discussion and sorry, again, for not explaining
myself
properly the first n times... my sincere apologies if my stubborn
ignorance
is annoying everyone here!
Sho
Notice the {} in there -- from what I gather objects are at the
bottom of
the sort, so this query gets all data related to user dlandolt --
the
first
result (or any result with the second part of a key ending in 0
based on
how
I wrote my view), and then everything following are the
*subscription*
docs
that Damien recommended.
Hope that helps.
