Robert,
Couchdb Lists work on top of views (and look great by the way), however
that brings me back to my initial post (causes an error on this mailing
list for some reason but you can find a copy here
http://mail-archives.apache.org/mod_mbox/couchdb-user/201205.mbox/%[email protected]%3E)
:-). Namely generating a view (well a design document with views in it) on
our data set takes between 6 (simple view) and 16 hours, takes up a lot of
disk space for what seems a small amount of data and burns a CPU at 100%
for the full time it runs i.e. no IO contention and can't use multiple
cores/cpus. So again am I doing something fundamentally wrong or is this
just the way Couch works and most people don't have a data set like ours so
it does not take that long to create views or does Big Couch solve the
issue (although it would seem 10 big couch nodes would still take an hour)
Looks like you work at Cloudant, so hopefully you might be able to provide
some answers based on real world experience?
Mike
-----Original Message-----
From: Robert Newson [mailto:[email protected]]
Sent: 24 May 2012 12:08
To: [email protected]
Subject: Re: Am I doing something fundamentally wrong?
Or use a list function;
http://wiki.apache.org/couchdb/Formatting_with_Show_and_List
You can use one with _all_docs and you can POST an array of ids too.
http://wiki.apache.org/couchdb/HTTP_view_API
Since 0.9 you can also issue POST requests to views where you can send
the following JSON structure in the body:
{"keys": ["key1", "key2", ...]}
B.
On 24 May 2012 11:58, Mike Kimber<[email protected]> wrote:
Looking at Show documentation and running a quick test I don't think
this helps as Show has to be referenced by a doc._id or view key. If these
aren't provided it returns null. This makes sense as its for generation of
a html, XML page/doc etc.
So I'd have to get a list of all doc ID's I want and then call the show
function for each and to get a filtered list I need a view.
Mike
-----Original Message-----
From: Mike Kimber [mailto:[email protected]]
Sent: 24 May 2012 10:47
To: [email protected]
Subject: RE: Am I doing something fundamentally wrong?
Aurélien,
Thanks for the response and apologies I didn't get a notification
(e-mail) of my original post (or the 2nd one) or your response. When I look
at my original post in Google Reader is has "An error occurred while
fetching this message, sorry !", so there must be something in the e-mail
that the mailing list system does not like.
In response to your original response " I'm a bit puzzled by the fact
that your map functions use the document ID". I do this because I load the
data into Luciddb and this allows me to join between tables. This is not my
end game this is just a compromise due to the time it takes to generate a
view and my need to play/discover with the data.
I will look at show to see if It helps, however it does not really
answer my original questions and it does not remove the more general issue
that view build takes a very long time, it only uses a single CPU and uses
a bucket load of space even with compression on (no idea why when it has a
lot less data than the original)
Thanks
Mike
-----Original Message-----
From: Aurélien Bénel [mailto:[email protected]]
Sent: 24 May 2012 07:40
To: [email protected]
Subject: Re: Am I doing something fundamentally wrong?
Hi Mike,
Didn't seem to get there first time so having another go
As I wrote in my earlier post, the use of 'map' functions in both of
your examples is overkill.
Use 'show' functions instead.They won't require an index to be built.
Regards,
Aurélien
