On Sun, Jun 14, 2009 at 2:03 PM, Marcus Persson Lindqvist<[email protected]> wrote: > Greetings all! > > In short - what factors are involved in memory consumtion for couchdb for a > large (x * 1000+) number of databases? Any hints welcome. > > > > I've recently starting to dig couchdb alot and are using it as primary > storage of a backend-type application to much success and relaxation. It > really saves a lot of pain not having to care much about the details of a > repository. > > Now, however, my application is growing in data and I'm looking for some > pointers of what to expect in terms of memory consumption (my primary > bottleneck). > > The data is highly segmentized - I'm using about 4 different "classes" of > documents from X different "sources" (X is currently 200 but might grow up > to 2000 or more), neither of which need to know about the others. Going > reduction of btrees and such, I figured I would use a separate database for > each, yielding 800 DBs at the moment. > > And kudos to couch for making it a breeze implementing, it was really nice > and smooth. > > But now I'm starting to see some memory consumtion growth and I'm looking > for pointers of how to think about this. What mechanisms actually cosumes > memory? What should one avoid? Is it better to use fewer databases for this > point of view. > > What would be a reasonable memory footprint and how does one caclulate on > it? Currently it consumed about 300MB. > > Each database is really just a pet store. I need to extract documents in > order. Thats it. I'm currently doing this with a simple view. (Are there any > "trivial" build-in way of getting documents i reversed insertion-order btw?) >
If you really just want to pull them out in the order you put them in, have a look at the _all_docs_by_seq view: http://127.0.0.1:5984/test_suite_db/_all_docs_by_seq?descending=true Do note that if you update a document after inserting it, it will move to a new position in the list. > And yeah, the load for most databases is really low, so insert/output > performance could be compromized for less memory consumtions. > > Any hints, tips or experiences? > > Marcus > -- Chris Anderson http://jchrisa.net http://couch.io
