Hi Pasi,

Thank you very much for your help. I really appreciate it!

On Jan 5, 12:58 pm, Pasi Kärkkäinen <pa...@iki.fi> wrote:

> On Tue, Jan 05, 2010 at 02:05:03AM -0800, Jack Z wrote:
> > > Try using some benchmarking tool that can do multiple outstanding IOs..
> > > for example ltp disktest.
> > And I tried ltp disktest, too. But I'm not sure whether I used it
> > right because the result was a little surprising...
> > I did
> > disktest -w -S0:1k -B 1024 /dev/sdb
> > (/dev/sdb is the iSCSI device file, no partition or file system on it)
> > And the result was:
> > | 2010/01/05-02:58:26 | START | 27293 | v1.4.2 | /dev/sdb | Start
> > args: -w -S0:1024k -B 1024 -PA (-I b) (-N 8385867) (-K 4) (-c) (-p R)
> > (-L 1048577) (-D 0:100) (-t 0:2m) (-o 0)
> > | 2010/01/05-02:58:26 | INFO  | 27293 | v1.4.2 | /dev/sdb | Starting
> > pass
> > ^C| 2010/01/05-03:00:58 | STAT  | 27293 | v1.4.2 | /dev/sdb | Total
> > bytes written in 85578 transfers: 87631872
> > | 2010/01/05-03:00:58 | STAT  | 27293 | v1.4.2 | /dev/sdb | Total
> > write throughput: 701055.0B/s (0.67MB/s), IOPS 684.6/s.
> > | 2010/01/05-03:00:58 | STAT  | 27293 | v1.4.2 | /dev/sdb | Total
> > Write Time: 125 seconds (0d0h2m5s)
> > | 2010/01/05-03:00:58 | STAT  | 27293 | v1.4.2 | /dev/sdb | Total
> > overall runtime: 152 seconds (0d0h2m32s)
> > | 2010/01/05-03:00:58 | END   | 27293 | v1.4.2 | /dev/sdb | User
> > Interrupt: Test Done (Passed)
> > As you can see, the throughput was only 0.67MB/s and only 85578
> > written in 87631872 transfers...
> > I also tweaked the options with "-p l" and/or "-I bd" (change seek
> > pattern to linear and/or speficy IO type as block and direct IO) but
> > no improvement happened...
> Hmm.. so it does 684 IO operations per second (IOPS), and each IO was 1k
> in size, so it makes 684 kB/sec of throughput.
> 1000 milliseconds (1 second) divided by 684 IOPS is 1.46 milliseconds per IO..
> Are you sure you had 16ms of rtt?


Actually that was probably the output from 0.2 ms rtt instead of 16
ms... I'm sorry for the mistake. I tried again the same command on a
16ms RTT, and the IOPS was mostly around 150.

> Try to play and experiment with these options:
>
> -B 64k  (blocksize 64k, try also 4k)
> -I BD (block device, direct IO (O_DIRECT))
> -K 16 (16 threads, aka 16 outstanding IOs. -K 1 should be the same as dd)
>
> Examples:
>
> Sequential (linear) reads using blocksize 4k and 4 simultaneous threads, for 
> 60 seconds:
> disktest -B 4k -h 1 -I BD -K 4 -p l -P T -T 60 -r /dev/sdX
>
> Random writes:
>
> disktest -B 4k -h 1 -I BD -K 4 -p r -P T -T 60 -w /dev/sdX
>
> 30% random reads, 70% random writes:
> disktest -r -w -D30:70 -K2 -E32 -B 8k -T 60 -pR -Ibd -PA /dev/md4
>
> Hopefully that helps..


That did help! I tried the following combinations of -B -K and -p at
20 ms RTT and the other options were -h 30 -I BD -P T -S0:(1 GB size)

-B 4k/64k -K 4/64 -p l

When I put -p l there the performance went down
drastically...

-B 4k -K 4/64 -p r

The disk throughput was similar to the one I used in the previous
post
"disktest -w -S0:1k -B 1024 /dev/sdb " and it was much lower than dd
could get.

-B 64k -K 4 -p r

The disk throughput was higher than the last one but still not as
high
as dd could get.

-B 64k -K 64 -p r

The disk throughput was boosted to 8.06 MB/s and the IOPS was 129.0.
At the link layer, the traffic rate was 70.536 Mbps (the TCP baseline
was 96.202 Mbps). At the same time, dd ( bs=64K count=(1 GB size))
got
a throughput of 6.7 MB/s and the traffic rate on the link layer was
57.749 Mbps.

Although not much, it was still an improvement :) and it was the
first
improvement I have ever seen since I started my experiments! Thank
you
very much!

As for
> Oh, also make sure you have 'oflag=direct' for dd.

The result was surprisingly low again... Do you think the reason
might
be that I was running dd on a device file (/dev/sdb), which did not
have any partitions/file systems on it?

Thanks a lot!

Jack

-- 
You received this message because you are subscribed to the Google Groups 
"open-iscsi" group.
To post to this group, send email to open-is...@googlegroups.com.
To unsubscribe from this group, send email to 
open-iscsi+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/open-iscsi?hl=en.


Reply via email to