Re: howto boost write(2) performance?

2007-10-17 Thread Bill Davidsen
Michael Stiller wrote: Hi list, i'm developing an application (in C) which needs to write about 1Gbit/s (125Mb/s) to a disk array attached via U320 SCSI. It runs on Dual Core 2 Xeons @2Ghz utilizing kernel 2.6.22.7. People regularly report speeds higher than that on the RAID list, and I

Re: howto boost write(2) performance?

2007-10-17 Thread Bill Davidsen
Michael Stiller wrote: Hi list, i'm developing an application (in C) which needs to write about 1Gbit/s (125Mb/s) to a disk array attached via U320 SCSI. It runs on Dual Core 2 Xeons @2Ghz utilizing kernel 2.6.22.7. People regularly report speeds higher than that on the RAID list, and I

Re: howto boost write(2) performance?

2007-10-11 Thread Michael Stiller
On Tue, 2007-10-09 at 16:56 +0200, Andi Kleen wrote: > Michael Stiller <[EMAIL PROTECTED]> writes: > > > > The write(2) performance is not good enough, the writer threads take to > > much time, and i ask you for ideas, howto to boost the write > > performance. > > You could use an O_DIRECT

Re: howto boost write(2) performance?

2007-10-11 Thread Michael Stiller
On Tue, 2007-10-09 at 16:56 +0200, Andi Kleen wrote: Michael Stiller [EMAIL PROTECTED] writes: The write(2) performance is not good enough, the writer threads take to much time, and i ask you for ideas, howto to boost the write performance. You could use an O_DIRECT write if the data

Re: howto boost write(2) performance?

2007-10-10 Thread Michael Stiller
> > The kernel really cannot sustain 125MB/s? I assume the disk > array is capable? Yes, the array should be capable, LSI controller U320, SATA disks inside the array. > Where is the bottleneck? Does it keep all disks busy, or are > the CPUs overloaded? I'm not sure where the bottleneck is.

Re: howto boost write(2) performance?

2007-10-10 Thread Michael Stiller
The kernel really cannot sustain 125MB/s? I assume the disk array is capable? Yes, the array should be capable, LSI controller U320, SATA disks inside the array. Where is the bottleneck? Does it keep all disks busy, or are the CPUs overloaded? I'm not sure where the bottleneck is. CPU

Re: howto boost write(2) performance?

2007-10-09 Thread Nick Piggin
On Tuesday 09 October 2007 23:50, Michael Stiller wrote: > Hi list, > > i'm developing an application (in C) which needs to write about > 1Gbit/s (125Mb/s) to a disk array attached via U320 SCSI. > It runs on Dual Core 2 Xeons @2Ghz utilizing kernel 2.6.22.7. > > I buffer the data in (currently 4)

Re: howto boost write(2) performance?

2007-10-09 Thread Michael Tokarev
Boaz Harrosh wrote: [] > If your target is a SCSI target you can gain up to 15% by using > sg. Search on the net for the "sg utils" package. The source code > of sg_dd and others are a grate example of how to do it. > > Other wise O_DIRECT is your friend. Also look for asynchronous > I/O so you

Re: howto boost write(2) performance?

2007-10-09 Thread Boaz Harrosh
On Tue, Oct 09 2007 at 16:56 +0200, Andi Kleen <[EMAIL PROTECTED]> wrote: > Michael Stiller <[EMAIL PROTECTED]> writes: >> The write(2) performance is not good enough, the writer threads take to >> much time, and i ask you for ideas, howto to boost the write >> performance. > > You could use an

Re: howto boost write(2) performance?

2007-10-09 Thread Andi Kleen
Michael Stiller <[EMAIL PROTECTED]> writes: > > The write(2) performance is not good enough, the writer threads take to > much time, and i ask you for ideas, howto to boost the write > performance. You could use an O_DIRECT write if the data is suitably aligned and your IO sizes are big enough

Re: howto boost write(2) performance?

2007-10-09 Thread Gustavo Chain
El Tue, 09 Oct 2007 15:50:17 +0200 Michael Stiller <[EMAIL PROTECTED]> escribió: > Hi list, > > i'm developing an application (in C) which needs to write about > 1Gbit/s (125Mb/s) to a disk array attached via U320 SCSI. > It runs on Dual Core 2 Xeons @2Ghz utilizing kernel 2.6.22.7. > > I

howto boost write(2) performance?

2007-10-09 Thread Michael Stiller
Hi list, i'm developing an application (in C) which needs to write about 1Gbit/s (125Mb/s) to a disk array attached via U320 SCSI. It runs on Dual Core 2 Xeons @2Ghz utilizing kernel 2.6.22.7. I buffer the data in (currently 4) 400Mb buffers and use write(2) in a dedicated thread to write

howto boost write(2) performance?

2007-10-09 Thread Michael Stiller
Hi list, i'm developing an application (in C) which needs to write about 1Gbit/s (125Mb/s) to a disk array attached via U320 SCSI. It runs on Dual Core 2 Xeons @2Ghz utilizing kernel 2.6.22.7. I buffer the data in (currently 4) 400Mb buffers and use write(2) in a dedicated thread to write

Re: howto boost write(2) performance?

2007-10-09 Thread Gustavo Chain
El Tue, 09 Oct 2007 15:50:17 +0200 Michael Stiller [EMAIL PROTECTED] escribió: Hi list, i'm developing an application (in C) which needs to write about 1Gbit/s (125Mb/s) to a disk array attached via U320 SCSI. It runs on Dual Core 2 Xeons @2Ghz utilizing kernel 2.6.22.7. I buffer the

Re: howto boost write(2) performance?

2007-10-09 Thread Andi Kleen
Michael Stiller [EMAIL PROTECTED] writes: The write(2) performance is not good enough, the writer threads take to much time, and i ask you for ideas, howto to boost the write performance. You could use an O_DIRECT write if the data is suitably aligned and your IO sizes are big enough

Re: howto boost write(2) performance?

2007-10-09 Thread Boaz Harrosh
On Tue, Oct 09 2007 at 16:56 +0200, Andi Kleen [EMAIL PROTECTED] wrote: Michael Stiller [EMAIL PROTECTED] writes: The write(2) performance is not good enough, the writer threads take to much time, and i ask you for ideas, howto to boost the write performance. You could use an O_DIRECT

Re: howto boost write(2) performance?

2007-10-09 Thread Michael Tokarev
Boaz Harrosh wrote: [] If your target is a SCSI target you can gain up to 15% by using sg. Search on the net for the sg utils package. The source code of sg_dd and others are a grate example of how to do it. Other wise O_DIRECT is your friend. Also look for asynchronous I/O so you have a

Re: howto boost write(2) performance?

2007-10-09 Thread Nick Piggin
On Tuesday 09 October 2007 23:50, Michael Stiller wrote: Hi list, i'm developing an application (in C) which needs to write about 1Gbit/s (125Mb/s) to a disk array attached via U320 SCSI. It runs on Dual Core 2 Xeons @2Ghz utilizing kernel 2.6.22.7. I buffer the data in (currently 4) 400Mb