Then what I get from this is that the fundimental unit of measure is
kilo-bytes (KB/1024 bytes)?
Further, that I will have to write up various cases? Okay, I'll do it.
Case for normal generic file system usage and a case for RDBMS usage.
These will include separate considerations. It'll take a while because I
have to interleave it with my WHOIS server project and my day-time job.
I might ask the list for certain empirical results, as I run into the
cases.
Yes, I think it is important, especially for ORDBMS users. I have both
Oracle8 and PostgreSQL. If MySQL users can identify themselves now I
will ask them for case studies later.
> -----Original Message-----
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED]]On Behalf Of Tim Walberg
> Sent: Friday, July 30, 1999 8:20 AM
> To: [EMAIL PROTECTED]
> Subject: Re: raid0 vs. raid5 read performance
>
>
> On 07/30/1999 07:17 -0700, Roeland M.J. Meyer wrote:
> >> I have seen neither FAQ entry nor document nor email
> message giving any
> >> sort of heuristic to determine chunk size (even a note
> on what the unit
> >> of measurement is, would be nice). If someone could
> explain it to me, I
> >> would be happy to write it up.
> End of included message
>
> In my experience, this is largely dependent on what you intend
> to do with the block device. Some things I usually use to determine
> this:
>
> 1) What is the predominant I/O size for the subsystem?
>
> For example, a database using the block device directly
> will do I/O in some multiple of the pagesize. File
> systems typically have a primary block size, but also
> deal with fragments.
>
> The chunk size should be no smaller than the typical I/O
> size, because a single I/O would then be split up into
> multiple accesses to multiple physical disks.
>
> 2) What are the largest I/Os that are submitted to the
> subsystem?
>
> Many subsystems, whether databases or file systems or
> whatever, will do write gathering, so that a number
> of consecutive blocks will be written at once.
>
> Also, very large IOs are not usually done as a single
> IO - a 1MB read will be split at some level into a
> number of smaller (for example, 64K) reads.
>
> You can take several approaches with this:
>
> For RAID0, you can set your chunk size so that a
> large write breaks down into a single 1-chunk write
> on every individual drive. This gives maximal load
> balancing, and predictable degradation of performance
> as your IO load increases.
>
> For RAID0, you can set your chunk size so that a
> large write maps to a single 1-chunk write on one
> drive (if it is properly aligned, anyway). This
> may be more efficient if your ratio of large IOs
> to small is low. Then, on average, most accesses
> usually only hit one drive, but some will be split
> into two.
>
> For RAID0, you can set your chunk size so that a
> large write maps to a single sub-1-chunk write
> on one drive most of the time (i.e. chunk size
> is > (2*largeIOsize)). This reduces the number
> of large writes that span chunk boundaries, and
> hence reduces the number of IOs that hit 2 disks.
>
> For RAID5, you want to set your chunk size so that
> a large write corresponds to (at least) one full
> stripe write, since a full stripe write simply
> writes a chunk to each drive, without having to
> do a read/modify/write to update the parity. I
> usually set up RAID5s so that a large write
> corresponds to 2 full stripes, so that, no
> matter how the write is aligned, there will
> be at most 1 full stripe write, and 2 partial
> stripe writes (best case would be 2 full stripe
> writes). For example, using 4 drives in a RAID5
> with large IOs = 64K, set the chunk size to about
> 11-12K - then 64K written = 1 33K (or 36K) full
> stripe writes and 2 smaller writes.
>
> Bottom line, though, it's best to set up several different
> configurations and test them with real-life loads (not
> necessarily in production, but use the same stuff to
> benchmark as what you're going to use in production).
>
> Hope that's useful....
>
>
>
> tw
>
>
> --
> +------------------------------+--------------------------+
> | Tim Walberg | Phone: 847-782-2472 |
> | TERAbridge Technologies Corp | FAX: 847-623-1717 |
> | 1375 Tri-State Parkway | [EMAIL PROTECTED] |
> | Gurnee, IL 60031 | 800-SKY-TEL2 PIN 9353299 |
> +------------------------------+--------------------------+
>
