Databases (and views) need compacting even if you never update or delete a document. Try it, you might be surprised.
B. On 24 May 2012 15:19, Sean Copenhaver <[email protected]> wrote: > I believe multiple design documents will build views concurrently but one > design document is basically done sequentially by the change sequence... > not positive. > > So you could try splitting out your views into multiple design documents > and hit them to see if that helps spread out the CPU usage. I want to say a > lot of the CPU usage is the serialization process that is happening > communicating from CouchDB's core to the view engine process. > > Anyway with the list you specify any view and all_docs is a view with all > documents in a database. So if you know the ids you want to work with you > can doe a normal view query with a list function. > http://wiki.apache.org/couchdb/HTTP_Document_API#all_docs > > That's what Robert was trying to get at. > > On Thu, May 24, 2012 at 9:55 AM, Mike Kimber <[email protected]> wrote: > >> Robert, >> >> Couchdb Lists work on top of views (and look great by the way), however >> that brings me back to my initial post (causes an error on this mailing >> list for some reason but you can find a copy here >> http://mail-archives.apache.org/mod_mbox/couchdb-user/201205.mbox/%[email protected]%3E) >> :-). Namely generating a view (well a design document with views in it) on >> our data set takes between 6 (simple view) and 16 hours, takes up a lot of >> disk space for what seems a small amount of data and burns a CPU at 100% >> for the full time it runs i.e. no IO contention and can't use multiple >> cores/cpus. So again am I doing something fundamentally wrong or is this >> just the way Couch works and most people don't have a data set like ours so >> it does not take that long to create views or does Big Couch solve the >> issue (although it would seem 10 big couch nodes would still take an hour) >> >> Looks like you work at Cloudant, so hopefully you might be able to provide >> some answers based on real world experience? >> >> Mike >> >> >> >> -----Original Message----- >> From: Robert Newson [mailto:[email protected]] >> Sent: 24 May 2012 12:08 >> To: [email protected] >> Subject: Re: Am I doing something fundamentally wrong? >> >> Or use a list function; >> >> http://wiki.apache.org/couchdb/Formatting_with_Show_and_List >> >> You can use one with _all_docs and you can POST an array of ids too. >> >> http://wiki.apache.org/couchdb/HTTP_view_API >> >> > Since 0.9 you can also issue POST requests to views where you can send >> the following JSON structure in the body: >> > {"keys": ["key1", "key2", ...]} >> >> B. >> >> On 24 May 2012 11:58, Mike Kimber <[email protected]> wrote: >> > Looking at Show documentation and running a quick test I don't think >> this helps as Show has to be referenced by a doc._id or view key. If these >> aren't provided it returns null. This makes sense as its for generation of >> a html, XML page/doc etc. >> > >> > So I'd have to get a list of all doc ID's I want and then call the show >> function for each and to get a filtered list I need a view. >> > >> > Mike >> > >> > -----Original Message----- >> > From: Mike Kimber [mailto:[email protected]] >> > Sent: 24 May 2012 10:47 >> > To: [email protected] >> > Subject: RE: Am I doing something fundamentally wrong? >> > >> > Aurélien, >> > >> > Thanks for the response and apologies I didn't get a notification >> (e-mail) of my original post (or the 2nd one) or your response. When I look >> at my original post in Google Reader is has "An error occurred while >> fetching this message, sorry !", so there must be something in the e-mail >> that the mailing list system does not like. >> > >> > In response to your original response " I'm a bit puzzled by the fact >> that your map functions use the document ID". I do this because I load the >> data into Luciddb and this allows me to join between tables. This is not my >> end game this is just a compromise due to the time it takes to generate a >> view and my need to play/discover with the data. >> > >> > I will look at show to see if It helps, however it does not really >> answer my original questions and it does not remove the more general issue >> that view build takes a very long time, it only uses a single CPU and uses >> a bucket load of space even with compression on (no idea why when it has a >> lot less data than the original) >> > >> > Thanks >> > >> > Mike >> > >> > -----Original Message----- >> > From: Aurélien Bénel [mailto:[email protected]] >> > Sent: 24 May 2012 07:40 >> > To: [email protected] >> > Subject: Re: Am I doing something fundamentally wrong? >> > >> > Hi Mike, >> > >> >> Didn't seem to get there first time so having another go >> > >> > As I wrote in my earlier post, the use of 'map' functions in both of >> your examples is overkill. >> > Use 'show' functions instead.They won't require an index to be built. >> > >> > >> > Regards, >> > >> > Aurélien >> > > > > -- > “The limits of language are the limits of one's world. “ - Ludwig von > Wittgenstein > > "Water is fluid, soft and yielding. But water will wear away rock, which is > rigid and cannot yield. As a rule, whatever is fluid, soft and yielding > will overcome whatever is rigid and hard. This is another paradox: what is > soft is strong." - Lao-Tzu
