Re: iscsi problem for two nics
has anybody tested this with 10Gb cards? what is the performance gains? I have tested it lightly and found that there are some gains but without a 10Gb switch in the mix testing is very difficult. Paul Yao Wei wrote: > In short: > > > Case A > Initiator: IBM x3550 (two NICs), /dev/sda (local disk), 100-120 MB/s > (hdparm -t /dev/sda). > Target:Dell (only one NIC), /dev/sda (local disk), dev/sda (IET > disk),70-90 MB/s(hdparm -t /dev/sda) > > when login with iscsi, open-iscsi find new attached disk /dev/sdb, > remote reading speed is less than 10 MB (hdparm -t /dev/sdb) (*) > > > > Case B > Initiator: Dell (only one NIC),/dev/sda (local disk), 70-90 MB/s > (hdparm -t /dev/sda). > Target:IBM x3550 (two NICs), /dev/sda (local disk), dev/sda > (IET disk),100-120 MB/s(hdparm -t /dev/sda) > > when login with iscsi, open-iscsi also find new attached disk: /dev/ > sdb, remote reading speed is about 80 MB/s(hdparm -t /dev/sdb) (**) > > > > Case C > Initiator: ThinkPad x61 notebook (one NIC), /dev/sda (local disk), > 50-60 MB/s (hdparm -t /dev/sda). > Target:Dell (only one NIC), /dev/sda (local disk), > dev/sda (IET disk),70-90 MB/s(hdparm -t /dev/sda) > > when login with iscsi, open-iscsi find new attached disk /dev/sdb, > remote reading speed is about 40-50 MB (hdparm -t /dev/sdb) (***) > > > > Case D > Initiator: Dell (only one NIC), /dev/sda (local disk), > 70-90 MB/s (hdparm -t /dev/sda). > Target:ThinkPad x61 notebook (one NIC), /dev/sda (local disk), > dev/sda (IET disk),50-60 MB/s(hdparm -t /dev/sda) > > when login with iscsi, open-iscsi find new attached disk /dev/sdb, > remote reading speed is about 50 MB (hdparm -t /dev/sdb) (***) > > --- > > That is what confused me. > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: iscsi problem for two nics
In short: Case A Initiator: IBM x3550 (two NICs), /dev/sda (local disk), 100-120 MB/s (hdparm -t /dev/sda). Target:Dell (only one NIC), /dev/sda (local disk), dev/sda (IET disk),70-90 MB/s(hdparm -t /dev/sda) when login with iscsi, open-iscsi find new attached disk /dev/sdb, remote reading speed is less than 10 MB (hdparm -t /dev/sdb) (*) Case B Initiator: Dell (only one NIC),/dev/sda (local disk), 70-90 MB/s (hdparm -t /dev/sda). Target:IBM x3550 (two NICs), /dev/sda (local disk), dev/sda (IET disk),100-120 MB/s(hdparm -t /dev/sda) when login with iscsi, open-iscsi also find new attached disk: /dev/ sdb, remote reading speed is about 80 MB/s(hdparm -t /dev/sdb) (**) Case C Initiator: ThinkPad x61 notebook (one NIC), /dev/sda (local disk), 50-60 MB/s (hdparm -t /dev/sda). Target:Dell (only one NIC), /dev/sda (local disk), dev/sda (IET disk),70-90 MB/s(hdparm -t /dev/sda) when login with iscsi, open-iscsi find new attached disk /dev/sdb, remote reading speed is about 40-50 MB (hdparm -t /dev/sdb) (***) Case D Initiator: Dell (only one NIC), /dev/sda (local disk), 70-90 MB/s (hdparm -t /dev/sda). Target:ThinkPad x61 notebook (one NIC), /dev/sda (local disk), dev/sda (IET disk),50-60 MB/s(hdparm -t /dev/sda) when login with iscsi, open-iscsi find new attached disk /dev/sdb, remote reading speed is about 50 MB (hdparm -t /dev/sdb) (***) --- That is what confused me. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: iscsi problem for two nics
On 8 Sep 2009 at 10:31, mala...@us.ibm.com wrote: > > Yao Wei [njus...@gmail.com] wrote: > > > > The main purpose of my using iscsi is to implement remote disk > > copying. > > > > When I installed iscsi-target on Dell machine and using the whole disk > > to be the target: > > Lun 0 Path=/dev/sda,Type=fileio > > > > then on my ibm x3550 machine (with two NICs) , I first configure NIC1 > > and NIC2 with different MAC addresses in /etc/iscsi/ifaces/iface0 and > > iface1, and then I use iscsiadm -m discovery -t st -p ip -I iface1 -l > > to discovery and login the iscsi discovery. > > > > To test the write and read speed of remote disk, I first use hdparm - > > t /dev/sdb to get the speed of timing buffered disk reads: less than > > 10MB/sec. > > More, I directly using dd if=/dev/sdb of=/home/my.dd ibs=32k to copy > > the whole disk, it consume very much time, as slow as 10 MB/s. > > > > Later, I let IBM x3550 machine to be the iscsi-target, and Dell > > machine to be the initiator, and do the procedure as above, I found > > that the speed of hdparm command can obtain about 80-100 MB/s, and > > also the command of dd a 40GB hard disk can be finished within > > 15 minutes (about 50 MB/s). > > The speed here you got is more than twice of the other path, so I > imagine the bottleneck is not the network speed. It is probably your > disk speed on Dell that is limiting you resulting in poor 10MB/sec > performance. What does 'hdparm' say on the same disk from the Dell > system? I made a little experiment (on a live system and SAN, SLES10 SP2 on a Sun X4100 Opteron server, HP EVA 6000 with MPX100 iSCSI connectivity option): # hdparm -t /dev/sdz /dev/sdz: Timing buffered disk reads: 220 MB in 3.02 seconds = 72.95 MB/sec # hdparm -T /dev/sdz /dev/sdz: Timing cached reads: 3896 MB in 2.00 seconds = 1947.53 MB/sec # hdparm -t /dev/dm- dm-0 dm-10 dm-12 dm-14 dm-16 dm-2 dm-4 dm-6 dm-8 dm-1 dm-11 dm-13 dm-15 dm-17 dm-3 dm-5 dm-7 dm-9 # hdparm -t /dev/dm-16 /dev/dm-16: Timing buffered disk reads: 170 MB in 3.07 seconds = 55.39 MB/sec (dev/sdz is one direct path to the iSCSI box, while dm-16 is the multipath device that includes to sets of eight paths each) So it seems the multipath-device is slower than one single path. Regards, Ulrich --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: iscsi problem for two nics
On 09/07/2009 05:46 AM, Yao Wei wrote: > The main purpose of my using iscsi is to implement remote disk > copying. > > When I installed iscsi-target on Dell machine and using the whole disk > to be the target: > Lun 0 Path=/dev/sda,Type=fileio > > then on my ibm x3550 machine (with two NICs) , I first configure NIC1 > and NIC2 with different MAC addresses in /etc/iscsi/ifaces/iface0 and > iface1, and then I use iscsiadm -m discovery -t st -p ip -I iface1 -l > to discovery and login the iscsi discovery. > > To test the write and read speed of remote disk, I first use hdparm - > t /dev/sdb to get the speed of timing buffered disk reads: less than > 10MB/sec. > More, I directly using dd if=/dev/sdb of=/home/my.dd ibs=32k to copy > the whole disk, it consume very much time, as slow as 10 MB/s. > > Later, I let IBM x3550 machine to be the iscsi-target, and Dell > machine to be the initiator, and do the procedure as above, I found > that the speed of hdparm command can obtain about 80-100 MB/s, and > also the command of dd a 40GB hard disk can be finished within > 15 minutes (about 50 MB/s). > > I found that as long as I use other machine(only with one Nic), it > will not emerge the above circumstance. That is to say, whatever it be > an initiator or target, the remote read and write speed will not be > quite different. But, on machine with two NICs, it will perfor so > different. > I am just saying it might be the disk that is used here: Lun 0 Path=/dev/sda,Type=fileio If you set up the target on the x3550 and use Lun 0 Path=/dev/sda,Type=fileio then on the target box use hdparm to read /dev/sda what values do you get? Is it close to what you get when you read it from the initiator through iscsi remotely? I was suggesting that to make sure the disk being used for the iscsi-target/IET disks is the same speed you can take the disk out by doing this: Target iqn.2001-04.com.redhat:perftest Lun 0 Type=fileio,Type=nullio,Sectors=10485760 With this when you test both boxes at a more level playing field, and you can begin to narrow down what component is creating the throughut difference. Did I understand you right? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: iscsi problem for two nics
Yao Wei [njus...@gmail.com] wrote: > > The main purpose of my using iscsi is to implement remote disk > copying. > > When I installed iscsi-target on Dell machine and using the whole disk > to be the target: > Lun 0 Path=/dev/sda,Type=fileio > > then on my ibm x3550 machine (with two NICs) , I first configure NIC1 > and NIC2 with different MAC addresses in /etc/iscsi/ifaces/iface0 and > iface1, and then I use iscsiadm -m discovery -t st -p ip -I iface1 -l > to discovery and login the iscsi discovery. > > To test the write and read speed of remote disk, I first use hdparm - > t /dev/sdb to get the speed of timing buffered disk reads: less than > 10MB/sec. > More, I directly using dd if=/dev/sdb of=/home/my.dd ibs=32k to copy > the whole disk, it consume very much time, as slow as 10 MB/s. > > Later, I let IBM x3550 machine to be the iscsi-target, and Dell > machine to be the initiator, and do the procedure as above, I found > that the speed of hdparm command can obtain about 80-100 MB/s, and > also the command of dd a 40GB hard disk can be finished within > 15 minutes (about 50 MB/s). The speed here you got is more than twice of the other path, so I imagine the bottleneck is not the network speed. It is probably your disk speed on Dell that is limiting you resulting in poor 10MB/sec performance. What does 'hdparm' say on the same disk from the Dell system? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: iscsi problem for two nics
The main purpose of my using iscsi is to implement remote disk copying. When I installed iscsi-target on Dell machine and using the whole disk to be the target: Lun 0 Path=/dev/sda,Type=fileio then on my ibm x3550 machine (with two NICs) , I first configure NIC1 and NIC2 with different MAC addresses in /etc/iscsi/ifaces/iface0 and iface1, and then I use iscsiadm -m discovery -t st -p ip -I iface1 -l to discovery and login the iscsi discovery. To test the write and read speed of remote disk, I first use hdparm - t /dev/sdb to get the speed of timing buffered disk reads: less than 10MB/sec. More, I directly using dd if=/dev/sdb of=/home/my.dd ibs=32k to copy the whole disk, it consume very much time, as slow as 10 MB/s. Later, I let IBM x3550 machine to be the iscsi-target, and Dell machine to be the initiator, and do the procedure as above, I found that the speed of hdparm command can obtain about 80-100 MB/s, and also the command of dd a 40GB hard disk can be finished within 15 minutes (about 50 MB/s). I found that as long as I use other machine(only with one Nic), it will not emerge the above circumstance. That is to say, whatever it be an initiator or target, the remote read and write speed will not be quite different. But, on machine with two NICs, it will perfor so different. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: iscsi problem for two nics
On 09/04/2009 07:58 AM, Yao Wei wrote: > Dear sir: > > I have tested iscsi protocols on two machines using open-iscsi and > iscsi-target. On IBM X3550 machine which has two NICs (SUSE Linux Enterprise > 10.1) I simulate iscsi initiator , and on Dell Optiplex 755 machine (Fedora > 8 ) for iscsi target . The network environment is 1GbE. > > Unfortunately, when I discovery the target, and used hdparm to test the read > speed, I found the speed could only obtain 10MB/s. > > Later, I let the IBM X3550 to be the target, and let Dell 755 to be > initiator, the speed could obtain above 80MB/s. > > I can not know the reason. Looking forword to your replies. > What are you using for storage on the iscsi-target? For perf testing you probably want to use a null device (it does not actually do any disk IO so it is very fast): Target iqn.2001-04.com.redhat:perftest Lun 0 Type=fileio,Type=nullio,Sectors=10485760 How are you running hdparm? Have you tried other tools? What IO sizes are you using and how many IOs are you sending at a time? Have you tried different IO schedulers (echo noop > /sys/block/sdXYZ/queue/scheduler)? What IO throughput do you get when just running a nettool like netperf? If you run ethtool can you confirm the net layer did negotiate for 1 GbE and there was not a messup where we ended up with 10 Mb? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
iscsi problem for two nics
Dear sir: I have tested iscsi protocols on two machines using open-iscsi and iscsi-target. On IBM X3550 machine which has two NICs (SUSE Linux Enterprise 10.1) I simulate iscsi initiator , and on Dell Optiplex 755 machine (Fedora 8 ) for iscsi target . The network environment is 1GbE. Unfortunately, when I discovery the target, and used hdparm to test the read speed, I found the speed could only obtain 10MB/s. Later, I let the IBM X3550 to be the target, and let Dell 755 to be initiator, the speed could obtain above 80MB/s. I can not know the reason. Looking forword to your replies. -- Yao Wei, P.R. China Mobile: (+86) 138-1698-4972 Everything should be made as simple as possible, but not simpler. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
