On Tue, Jan 05, 2010 at 02:05:03AM -0800, Jack Z wrote: > Hi Pasi, > > Thank you very much for your reply. > > > > I was testing the performance of open-iscsi initiator with IET target > > > over a 100Mbps Ethernet link with emulated rtt. What I did was to do > > > raw disk sequential write by > > > > > $ dd if=/dev/zero of=/dev/sdb bs=1024 count=1048576 > > > > Did you also try with bigger block sizes? 1k blocks are pretty small. > > > > try bs=1024k to see if it makes a difference. > > > I tried bs = 1024k and the throughput is improved, but not much... It > goes from 7.2MB/s to 8.0MB/s at a rtt of 16ms. And again, over 90% of > the TCP segments on the wire was only of 1448 bytes... >
Ok.. > > > dd will use only one outstanding IO, so you have wait for rtt > > milliseconds after every IO for the ack.. so that definitely slows you > > down a lot when rtt gets bigger. > > > > Try using some benchmarking tool that can do multiple outstanding IOs.. > > for example ltp disktest. > > > > And I tried ltp disktest, too. But I'm not sure whether I used it > right because the result was a little surprising... > > I did > > disktest -w -S0:1k -B 1024 /dev/sdb > > (/dev/sdb is the iSCSI device file, no partition or file system on it) > > And the result was: > > | 2010/01/05-02:58:26 | START | 27293 | v1.4.2 | /dev/sdb | Start > args: -w -S0:1024k -B 1024 -PA (-I b) (-N 8385867) (-K 4) (-c) (-p R) > (-L 1048577) (-D 0:100) (-t 0:2m) (-o 0) > | 2010/01/05-02:58:26 | INFO | 27293 | v1.4.2 | /dev/sdb | Starting > pass > ^C| 2010/01/05-03:00:58 | STAT | 27293 | v1.4.2 | /dev/sdb | Total > bytes written in 85578 transfers: 87631872 > | 2010/01/05-03:00:58 | STAT | 27293 | v1.4.2 | /dev/sdb | Total > write throughput: 701055.0B/s (0.67MB/s), IOPS 684.6/s. > | 2010/01/05-03:00:58 | STAT | 27293 | v1.4.2 | /dev/sdb | Total > Write Time: 125 seconds (0d0h2m5s) > | 2010/01/05-03:00:58 | STAT | 27293 | v1.4.2 | /dev/sdb | Total > overall runtime: 152 seconds (0d0h2m32s) > | 2010/01/05-03:00:58 | END | 27293 | v1.4.2 | /dev/sdb | User > Interrupt: Test Done (Passed) > > As you can see, the throughput was only 0.67MB/s and only 85578 > written in 87631872 transfers... > I also tweaked the options with "-p l" and/or "-I bd" (change seek > pattern to linear and/or speficy IO type as block and direct IO) but > no improvement happened... > Hmm.. so it does 684 IO operations per second (IOPS), and each IO was 1k in size, so it makes 684 kB/sec of throughput. 1000 milliseconds (1 second) divided by 684 IOPS is 1.46 milliseconds per IO.. Are you sure you had 16ms of rtt? > There must be something I've done wrong... Could you maybe help me out > here? > > Thanks a lot! > Try to play and experiment with these options: -B 64k (blocksize 64k, try also 4k) -I BD (block device, direct IO (O_DIRECT)) -K 16 (16 threads, aka 16 outstanding IOs. -K 1 should be the same as dd) Examples: Sequential (linear) reads using blocksize 4k and 4 simultaneous threads, for 60 seconds: disktest -B 4k -h 1 -I BD -K 4 -p l -P T -T 60 -r /dev/sdX Random writes: disktest -B 4k -h 1 -I BD -K 4 -p r -P T -T 60 -w /dev/sdX 30% random reads, 70% random writes: disktest -r -w -D30:70 -K2 -E32 -B 8k -T 60 -pR -Ibd -PA /dev/md4 Hopefully that helps.. -- Pasi
-- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-is...@googlegroups.com. To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.
