Glenn, CouchDB will already create subdirectories for db's with /'s. Though its not as automatic it'll allow for clients to do exactly the structure you're thinking of.
HTH, Paul On Wed, Oct 14, 2009 at 7:48 PM, Glenn Rempe <[email protected]> wrote: > Millions op DB's? > > Wouldn't you run into filesystem limitations due to the fact that > CouchDB writes all of its DB's/indexes into a single dir? > > e.g For the ext3 filesystem "There is a limit of 31998 sub-directories > per one directory, stemming from its limit of 32000 links per inode." > > http://en.wikipedia.org/wiki/Ext3 > > And my limited knowledge of filesystem internals says that the more > files you have in a single dir the longer it will take to seek on > those files. > > "The ext2 inode specification allows for over 100 trillion files to > reside in a single directory, however because of the current > linked-list directoryimplementation, only about 10-15 thousand files > can realistically be stored in a single directory. This is why > systems such as Squid (http://www.squid-cache.org ) use cache > directories with many subdirectories - searching through tens of > thousands of files in one directory is sloooooooow." > > http://answers.google.com/answers/threadview/id/122241.html > > Of course this will vary by filesystem in absolute terms, but I think > the concept is the same for all current file systems. No? > > CouchDB might really be able to address this if it did something like > make subdirs under the couchdb data dir that were derived from > portions of a hash of the filename. Using such a 2 or 3 level deep > dir structure would indeed allow for a huge number of DB's. > > e.g. if the db name hashes to a123df4g34fd.couch > > Make dirs/files like: > > DATA_DIR/a1/12/3d/f4g34fd.couch # DB > DATA_DIR/a1/12/3d/.some_view_index_hidden_dir # view index > > No? > > On Tue, Oct 13, 2009 at 3:59 PM, Chris Anderson <[email protected]> wrote: >> On Tue, Oct 13, 2009 at 6:29 AM, Brian Karlak <[email protected]> wrote: >>> >>> One caveat, however: we have one (somewhat funky) usecase which creates a >>> large number of small databases. Could the existence of several thousand >>> small databases affect performance? >>> >> >> We definitely support the many-databases use case (eg: 1 per user, aka >> millions of databases). I think there is extra support for that in >> 0.9.1, and of course 0.10 has only improved from there. >> >> Chris >> >> >> -- >> Chris Anderson >> http://jchrisa.net >> http://couch.io >> > > > > -- > Glenn Rempe > > email : [email protected] > voice : (415) 894-5366 or (415)-89G-LENN > twitter : @grempe > contact info : http://www.rempe.us/contact.html > pgp : http://www.rempe.us/gnupg.txt >
