On 07/30/1999 07:17 -0700, Roeland M.J. Meyer wrote:
>> I have seen neither FAQ entry nor document nor email message giving any
>> sort of heuristic to determine chunk size (even a note on what the unit
>> of measurement is, would be nice). If someone could explain it to me, I
>> would be happy to write it up.
End of included message
In my experience, this is largely dependent on what you intend
to do with the block device. Some things I usually use to determine
this:
1) What is the predominant I/O size for the subsystem?
For example, a database using the block device directly
will do I/O in some multiple of the pagesize. File
systems typically have a primary block size, but also
deal with fragments.
The chunk size should be no smaller than the typical I/O
size, because a single I/O would then be split up into
multiple accesses to multiple physical disks.
2) What are the largest I/Os that are submitted to the
subsystem?
Many subsystems, whether databases or file systems or
whatever, will do write gathering, so that a number
of consecutive blocks will be written at once.
Also, very large IOs are not usually done as a single
IO - a 1MB read will be split at some level into a
number of smaller (for example, 64K) reads.
You can take several approaches with this:
For RAID0, you can set your chunk size so that a
large write breaks down into a single 1-chunk write
on every individual drive. This gives maximal load
balancing, and predictable degradation of performance
as your IO load increases.
For RAID0, you can set your chunk size so that a
large write maps to a single 1-chunk write on one
drive (if it is properly aligned, anyway). This
may be more efficient if your ratio of large IOs
to small is low. Then, on average, most accesses
usually only hit one drive, but some will be split
into two.
For RAID0, you can set your chunk size so that a
large write maps to a single sub-1-chunk write
on one drive most of the time (i.e. chunk size
is > (2*largeIOsize)). This reduces the number
of large writes that span chunk boundaries, and
hence reduces the number of IOs that hit 2 disks.
For RAID5, you want to set your chunk size so that
a large write corresponds to (at least) one full
stripe write, since a full stripe write simply
writes a chunk to each drive, without having to
do a read/modify/write to update the parity. I
usually set up RAID5s so that a large write
corresponds to 2 full stripes, so that, no
matter how the write is aligned, there will
be at most 1 full stripe write, and 2 partial
stripe writes (best case would be 2 full stripe
writes). For example, using 4 drives in a RAID5
with large IOs = 64K, set the chunk size to about
11-12K - then 64K written = 1 33K (or 36K) full
stripe writes and 2 smaller writes.
Bottom line, though, it's best to set up several different
configurations and test them with real-life loads (not
necessarily in production, but use the same stuff to
benchmark as what you're going to use in production).
Hope that's useful....
tw
--
+------------------------------+--------------------------+
| Tim Walberg | Phone: 847-782-2472 |
| TERAbridge Technologies Corp | FAX: 847-623-1717 |
| 1375 Tri-State Parkway | [EMAIL PROTECTED] |
| Gurnee, IL 60031 | 800-SKY-TEL2 PIN 9353299 |
+------------------------------+--------------------------+
PGP signature
