On Wed, 25 Jul 2001 at 13:04, Orlando Andico wrote:
> I believe that even on mongo XFS is faster than ReiserFS on very large
> files. But that's a special case.
This special case, Orly, was the exact same case I used to test the Samba
and FTP transfers which you replied to and concluded that XFS is much
slower than ReiserFS. Hence my "Report: Qualifying XFS vs ReiserFS claims"
was specifically to rebut your statement that was based on my initial
tests.
For my original Samba optimization tests I used a large file because I
wanted to give both clients enough transfer time to give a fairly
acceptable average. This also allowed me to test the underlying filesystem
and hardware in the process, because the files were too large to fit in my
cache. I noticed, for example, that transfer start at a high of around 13
M/s, and then slowly decrease until they average at 4 to 5 M/s.
> Most UNIX systems have zillions of small files, and ReiserFS *is* much
> faster, particularly on metadata updates, than XFS on this case.
Depends for what.
o For a Squid cache, yes, of course. You not only have a lot of small
files, but you delete a significant number of them regularly, too. For
this reason it's hands down to ReiserFS, with awesome delete speeds
compared to any other Linux filesystem.
o For a mail spool, this depends on what MTA you use, but it will
probably be storing data in individual small files, as well. MTAs like
QMail have issues with ReiserFS, though, as they expect sync calls to
truly be synchronous. There is a fix, of course, but I just thought I'd
mention this.
o For storing users mail you do not have as massive an occurence of
deletes as you do in the above two scenarios. You have a significant
amount of read, create, and modify situations, though. The file sizes also
depend on what underlying format is used.
* Standard Unix mboxes store this one-file per mailbox. Now, these can't
be "zillions of small files" can they?
* Maildirs give each message its own file, and in PLUG at least these
messages are between 2kB to 10kB on the average, with a mean of maybe
3.5kB. Unfortunately we find no mongo test with this size. We can use the
1000 byte test, though. Take note that these tests are all on relatively
massive operations. The single process test, for example, tests 68352
files in 241 directories. Both single and dual process mongo tests give
ReiserFS the edge, though.
o For databases performance depends on what you're using.
* I have PostgreSQL 7.1.2 and I noticed that /var/lib/postgres/data/
contains directories with interesting contents. In particular there are a
significant number of numerically named files with sizes of 8192 bytes,
16384 bytes, and higher. I don't know how PostgreSQL accesses these files,
but for read and update operations I don't think ReiserFS will outperform
XFS significantly.
* I do not know how MySQL and Oracle store their files, but if I
remember correctly it has a bunch of large files that just keep growing.
If this is the case then what benefit will ReiserFS's capability to handle
"zillions of small files" do to you?
o I did a simple test to get the average file size of files in various
standard directories in my Linux installation using:
sum=0; n=0; for i in `ls -la | awk '{printf("%d\n",$5)}'`; do n=`expr $n +
1`; sum=`expr $sum + $i`; done; expr $sum / $n;
Here are the results (in bytes to be consistent with mongo's values):
/etc: 3840
/sbin: 36549
/bin: 30869
/usr/sbin: 51105
/usr/bin: 45186
/lib: 39009
/usr/lib: 81554
Take note that these values are even skewed, because directories (in XFS)
are reported with lower size values than the average files, but this
skewing is for the benefit of ReiserFS so I did not need to correct this
for the particular requirement of this discourse.
These fit your description of "most Unix systems", don't they? These do
not get deleted regularly, they do not get written regularly, they do not
get copied regularly. They are READ regularly, which is what executing
does, unless you do not have atime updates disabled.
Check the mongo benchmark results for files of size 10000 bytes (which
even skews the results to the benefit of ReiserFS and small files) and you
will see that XFS is already faster for reads by a factor of 0.94 for the
single process test, and by 0.88 for the multi process test. For larger
binaries or libraries, check out the 100000 byte tests. :)
o For filesystems storing files used by Windows clients, the file sizes
vary, but I have done some tests with a modified command of the above
using various subsets of my Samba shares. It is not common to delete these
files en masse. On the contrary they are read, updated or created. Copies
are also not done, and if ever, have to pass through the Samba daemon and
are therefore reads.
* We have cost estimates that are one-file-per-product Excel files. The
average filesize of the entire share of 1071 files was 243019 bytes (I
grepped out files without the extension ".xls" to exclude directories and
some possibly wrongly-located files or even temporary files). Again look
at the mongo results.
* We have various images stored in an image Samba share. These images
vary from product pictures to logos. A total of 3041 files (I grepped out
directories and files without a dot in them) had an average size of 540332
bytes. Again look at the mongo results.
In conclusion I do not think that ReiserFS can unanimously benefit "most
Unix systems", as my examples show that it is not always the case where a
filesystem will be populated by "zillions of small files". There are
situations where this is the case, and for those a filesystem good at
handling small files like ReiserFS is recommended.
Aside from the mongo.pl benchmark results in the Namesys website, another
benchmark (which unfortunately is not accepted by the ReiserFS developers
since they cannot reproduce the results) is available in
<http://aurora.zemris.fer.hr/filesystems/>.
--> Jijo
--
Federico Sevilla III :: [EMAIL PROTECTED]
Network Administrator :: The Leather Collection, Inc.
_
Philippine Linux Users Group. Web site and archives at http://plug.linux.org.ph
To leave: send "unsubscribe" in the body to [EMAIL PROTECTED]
To subscribe to the Linux Newbies' List: send "subscribe" in the body to
[EMAIL PROTECTED]
