Re: Stripe sizes with gstripe

2008-06-13 Thread Wojciech Puchar
Does gstripe read an entire stripe at a time?  If so, why do that instead 
of

just reading a few requested blocks?  If not, then is there any advantage
to large stripes?


Apparently it won't read anything larger than your stripe size which defaults 
to a miserable 4k.



depending from what's needed, but unless i need just huge linear transfer, 
i set stripe size to something huge, like 256MB.


then single read is rarely split on 2 disks, while multiple reads have 
good chances to touch different drives

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: Stripe sizes with gstripe

2008-06-13 Thread Kirk Strauser
On Thursday 12 June 2008, David Kelly wrote:

 Apparently it won't read anything larger than your stripe size which
 defaults to a miserable 4k.

Ugh.  It seems like there are a few possibilities here, and I'm not sure 
which is actually true.

Say you have two drives, striped.

1) Ideally, you could have a 512 byte stripe size.  A program tries to read 
4KB.  Then, gstripe would issue a single request to each drive to read 4 
blocks and interleaves the results.

2) Less ideally, you'd have a 128KB stripe size.  A program requests a 
single block, but gstripe reads the entire stripe to fulfill the request.  
Not so hot for random access.

3) Worst, maybe?  You have a 512 byte stripe.  A program reads 4KB.  gstripe 
reads 512B from da0, then 512B from da1, then 512B from da0, etc.

Actually, I guess you could also have a combination of #2 and #3, where 
small reads fetch an entire stripe while large reads are broken into lots 
of tiny ones.

So, back to gstripe.  Which of those is it most like?

 If there is a tuning knob that I have missed, would appreciate being
 told what.

Pass it along, would ya?  :-)

Oh, and don't forget to make your partition offsets
-- 
Kirk Strauser


signature.asc
Description: This is a digitally signed message part.


Re: Stripe sizes with gstripe

2008-06-13 Thread David Kelly
On Fri, Jun 13, 2008 at 09:08:48AM +0200, Wojciech Puchar wrote:
 Does gstripe read an entire stripe at a time?  If so, why do that instead 
 of
 just reading a few requested blocks?  If not, then is there any advantage
 to large stripes?
 
 Apparently it won't read anything larger than your stripe size which 
 defaults to a miserable 4k.
 
 
 depending from what's needed, but unless i need just huge linear transfer, 
 i set stripe size to something huge, like 256MB.
 
 then single read is rarely split on 2 disks, while multiple reads have 
 good chances to touch different drives

Come to think of it I didn't try setting the stripe size larger than the
ATA max transaction size of 128k.

Still, I don't understand what is going on when I use md5(1) on a
gigabyte file hosted on a gstripe partition with 128k stripes that
systat -v reports transactions are usually between 42k and 43k each?

On a non-striped filesystem the same operation runs 126k to 127k
transfers.

Transfer bandwidth seems to be limited by the number of transactions per
second more than the size of the transaction.

-- 
David Kelly N4HHE, [EMAIL PROTECTED]

Whom computers would destroy, they must first drive mad.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: Stripe sizes with gstripe

2008-06-13 Thread Kirk Strauser
On Friday 13 June 2008, David Kelly wrote:

 Still, I don't understand what is going on when I use md5(1) on a
 gigabyte file hosted on a gstripe partition with 128k stripes that
 systat -v reports transactions are usually between 42k and 43k each?

Even more unlikely, why are *my* numbers almost identical to yours?  Here's 
a snapshot of mine at this very second:

Disks   da0   da1   da2   da3   da4
KB/t  26.77 42.05 41.70 41.98 41.70

where da[1-4] are my gstripe providers with a 128KB stripe size.  I find it 
unlikely that our workloads are so similar that we'd coincidentally have 
almost the exact same values.
-- 
Kirk Strauser


signature.asc
Description: This is a digitally signed message part.


Re: Stripe sizes with gstripe

2008-06-13 Thread Kirk Strauser
On Thursday 12 June 2008, you wrote:

 If there is a tuning knob that I have missed, would appreciate being
 told what.

Dang it; hit send on accident.

Anyway, should the partition offsets on your gstripe volume be a multiple of 
the stripe size or of the filesystem's block size?
-- 
Kirk Strauser


signature.asc
Description: This is a digitally signed message part.


Re: Stripe sizes with gstripe

2008-06-13 Thread Wojciech Puchar

then single read is rarely split on 2 disks, while multiple reads have
good chances to touch different drives


Come to think of it I didn't try setting the stripe size larger than the
ATA max transaction size of 128k.


yes it is.

max transaction doesn't begin on boundary. may (usually will) start 
somewhere inbetween. making stripe size max transaction makes it 
inprobable.



BTW you mean ATA max transaction or FreeBSD default MAXBSIZE, which 
i change to 1MB

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: Stripe sizes with gstripe

2008-06-13 Thread Wojciech Puchar


Still, I don't understand what is going on when I use md5(1) on a
gigabyte file hosted on a gstripe partition with 128k stripes that
systat -v reports transactions are usually between 42k and 43k each?


because every single 128kB reads is CROSSING 128kB boundary?
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]


Stripe sizes with gstripe

2008-06-12 Thread Kirk Strauser
Does gstripe read an entire stripe at a time?  If so, why do that instead of 
just reading a few requested blocks?  If not, then is there any advantage 
to large stripes?
-- 
Kirk Strauser


signature.asc
Description: This is a digitally signed message part.


Re: Stripe sizes with gstripe

2008-06-12 Thread David Kelly


On Jun 12, 2008, at 3:21 PM, Kirk Strauser wrote:

Does gstripe read an entire stripe at a time?  If so, why do that  
instead of
just reading a few requested blocks?  If not, then is there any  
advantage

to large stripes?


Apparently it won't read anything larger than your stripe size which  
defaults to a miserable 4k.


Doesn't appear the driver collects multiple pending reads and merges  
them into one transaction. Don't know if ATA/SATA allows that or not,  
believe it was called scatter/gather in SCSI.


Built a gstripe volume a while back and played with various stripe  
sizes trying to maximize KB/t as shown in systat -v during massive  
file read/writes. A stripe size of 131072 (128k bytes) was the best I  
could get at about 43 KB/t. About 25 MB/sec each on two drives that  
are capable of 4x that on a Mac Pro (about 55 MB/sec single drive on  
inner tracks to 100 MB/sec on outer, reading with md5).


Anyway, throughput was terrible with the default 4k stripe.

If there is a tuning knob that I have missed, would appreciate being  
told what.


IIRC vinum's default stripe was about 87.5kB and systat -v reported  
transactions of that size.


--
David Kelly N4HHE, [EMAIL PROTECTED]

Whom computers would destroy, they must first drive mad.

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]