I have a couple of programs (not written by me, and not easily modifiable) that need to write quite a few (300,000-2,000,000) very small (about half are 228 byte files, the other half are around 7KB) files as quickly as possible to disk. In most cases, most of these files will be written to two directories (one for the little ones and one for the larger ones).
Right now, with the existing filesystem and its configuration (ext3, default Red Hat config), "as quickly as possible" is working out to be on the order of several days. This is tolerable (it is the way they have been running), but they consequently have to to do some rather silly workarounds to do simple things.
For example, if they want a directory listing, "ls" often fails. They have to "find . -name "afff7a1ba9-*" -print , which can take 7+ minutes in some cases). Or to remove a group of files ("find . -name "afff7a1ba9-*" -exec rm "{}" \;"), it takes 10+ minutes to delete 35 files.
Based on the specs I have read, I am leaning toward ReiserFS or XFS, but I would like the opinions of the list.
Opinions?
Thanks,
Adam Augustine____________________
BYU Unix Users Group http://uug.byu.edu/ ___________________________________________________________________
List Info: http://uug.byu.edu/cgi-bin/mailman/listinfo/uug-list
