On Wed, Jul 15, 2015 at 6:30 PM, Sepherosa Ziehau <[email protected]> wrote:
> Maybe just use large file, and sub-index the chunks of a large file > and add open/read/write/lseek/close like APIs for users. You will > have more control than using database. > > On Wed, Jul 15, 2015 at 11:58 PM, Michael Neumann <[email protected]> > wrote: > > Hi, > > > > Lets say I want to store 100 million small files (each one about 1k in > size) > > in a HAMMER file system. > > Files are only written once, then kept unmodified and accessed randomly > > (older files will be access less often). > > It is basically a simple file based key/value store, but accessible by > > multiple processes. > > > > a) What is the overhead in size for HAMMER1? For HAMMER2 I expect each > file > > to take exactly 1k when the file > > is below 512 bytes. > > > > b) Can I store all files in one huge directory? Or is it better to fan > out > > the files into several sub-directories? > > > > c) What other issues I should expect to run into? For sure I should > enable > > swapcache :) > > > > I probably should use a "real" database like LMDB, but I like the > > versatility of files. > > > > Regards, > > > > Michael > > > > -- > Tomorrow Will Never Die > In reality , an operating system is one of the "best" data base management system . Question by Michael Neumann is very important with respect to this fact . I was thinking to use DragonFly BSD for such a task , but it seems that it is not useful on that issue , because assumption about that files should be large is not so much suitable for an operating system . Then XFS seems to be a good alternative in Fedora Server edition . Thank you very much . Mehmet Erol Sanliturk
