Re: Open iSCSI Performance on IBM
Unfortunately not. I'm now trying to optimize some filesystem options to check if that increases performance... but from the iscsi part, I do not know what else to optimize. I could be reaching the physical limit of the system but I'm not sure since I do not know about any other performance numbers from any other systems. On Apr 13, 7:00 pm, jnantel wrote: > Have you made any headway with this issue? I'm having a write issue > that seems to share some similarities with yours. > > On Apr 13, 8:14 am, Gonçalo Borges wrote: > > > Hi... > > > > Is /apoio04/b1 a scsi/iscsi disk or is it LVM/DM/RAID on top of a > > > iscsi/scsi disk? > > > /apoio04/ is a RAID1 of two disks accessible via iscsi (in the > > following tests, I changed the mount point from /apoio04/ to /iscsi04- > > lun0/ but they are exactly the same). > > > > Could you set the IO scheduler to noop > > > echo noop > /sys/block/sdX/queue/scheduler and see if that makes a > > > difference. > > > I checked the definition and I have > > > [r...@core06 ~]# cat /sys/block/sdh/queue/scheduler > > noop anticipatory deadline [cfq] > > > Now I've changed to > > > [r...@core06 ~]# cat /sys/block/sdh/queue/scheduler > > [noop] anticipatory deadline cfq > > > and I've run the tests again. This is what I got: > > > [r...@core06 ~]# dd if=/dev/zero of=/iscsi04-lun0/b1 bs=64k > > count=125000 > > 125000+0 records in > > 125000+0 records out > > 819200 bytes (8.2 GB) copied, 470.332 seconds, 17.4 MB/s > > > [r...@core06 ~]# dd if=/dev/zero of=/iscsi04-lun0/b2 bs=128k > > count=62500 > > 62500+0 records in > > 62500+0 records out > > 819200 bytes (8.2 GB) copied, 470.973 seconds, 17.4 MB/s > > > Basically, the performance didn't increase :( > > > > And then also run > > > iscsiadm -m session -P 3 > > > [r...@core06 ~]# iscsiadm -m session -P 3 > > iSCSI Transport Class version 2.0-724 > > iscsiadm version 2.0-868 > > Target: iqn.1992-01.com.lsi:1535.600a0b80003ad11c490ade2d > > Current Portal: 10.131.2.14:3260,1 > > Persistent Portal: 10.131.2.14:3260,1 > > ** > > Interface: > > ** > > Iface Name: default > > Iface Transport: tcp > > Iface Initiatorname: iqn.1994-05.com.redhat:8c56e324f294 > > Iface IPaddress: 10.131.4.6 > > Iface HWaddress: default > > Iface Netdev: default > > SID: 37 > > iSCSI Connection State: LOGGED IN > > iSCSI Session State: Unknown > > Internal iscsid Session State: NO CHANGE > > > > Negotiated iSCSI params: > > > > HeaderDigest: None > > DataDigest: None > > MaxRecvDataSegmentLength: 131072 > > MaxXmitDataSegmentLength: 65536 > > FirstBurstLength: 8192 > > MaxBurstLength: 262144 > > ImmediateData: Yes > > InitialR2T: Yes > > MaxOutstandingR2T: 1 > > > > Attached SCSI devices: > > > > Host Number: 38 State: running > > scsi38 Channel 00 Id 0 Lun: 0 > > scsi38 Channel 00 Id 0 Lun: 1 > > scsi38 Channel 00 Id 0 Lun: 2 > > scsi38 Channel 00 Id 0 Lun: 3 > > scsi38 Channel 00 Id 0 Lun: 4 > > scsi38 Channel 00 Id 0 Lun: 5 > > scsi38 Channel 00 Id 0 Lun: 31 > > Current Portal: 10.131.2.13:3260,1 > > Persistent Portal: 10.131.2.13:3260,1 > > ** > > Interface: > > ** > > Iface Name: default > > Iface Transport: tcp > > Iface Initiatorname: iqn.1994-05.com.redhat:8c56e324f294 > > Iface IPaddress: 10.131.4.6 > > Iface HWaddress: default > > Iface Netdev: default > > SID: 38 > > iSCSI Connection State: LOGGED IN > > iSCSI Session State: Unknown > > Internal iscsid Session State: NO CHANGE > > > > Negotiated iSCSI params: > > > > HeaderDigest: None > > DataDigest: None > > MaxRecvDataSegmentLength: 131072 > > MaxXmitDataSegmentLength: 65536 > > FirstBurstLength: 8192 > > MaxBurstLength: 262144 > > ImmediateData: Yes > > InitialR2T: Yes > > MaxOutstandingR2T: 1 > > > > Attached SCSI devices: > > *
RE: Open iSCSI Performance on IBM
Disregard I just saw it was a dd option. Simone > -Original Message- > From: open-iscsi@googlegroups.com [mailto:open-is...@googlegroups.com] On > Behalf Of Simone Morellato > Sent: Monday, April 13, 2009 3:04 PM > To: open-iscsi@googlegroups.com > Subject: RE: Open iSCSI Performance on IBM > > > Hi Mike, > > Is bs=128K a linux, iscsi or IBM parameter? > > Thanks, > Simone > > > > -Original Message- > > From: open-iscsi@googlegroups.com [mailto:open-is...@googlegroups.com] > On > > Behalf Of Mike Christie > > Sent: Thursday, April 09, 2009 10:55 AM > > To: open-iscsi@googlegroups.com > > Subject: Re: Open iSCSI Performance on IBM > > > > > > Gonçalo Borges wrote: > > > Hi All... > > > > > > Sorry, the following could be a little bit off topic... > > > > > > Does any one has an idea of what is the expected performance for a IBM > > > DS 3300 system connected via open iSCSI? Using a RAID 1 with 2 disks, > > > I got the following numbers: > > > > > > > Is /apoio04/b1 a scsi/iscsi disk or is it LVM/DM/RAID on top of a > > iscsi/scsi disk? > > > > Could you set the IO scheduler to noop > > echo noop > /sys/block/sdX/queue/scheduler > > > > and see if that makes a difference. > > > > Also try > > > > bs=128k > > > > And then also run > > iscsiadm -m session -P 3 > > > > > > > > > Sequential Write: > > > > > > [r...@core12 ~]# dd if=/dev/zero of=/apoio04/b1 bs=64k count=125000 > > > 125000+0 records in > > > 125000+0 records out > > > 819200 bytes (8.2 GB) copied, 454.522 seconds, 18.0 MB/s > > > > > > Sequential Read: > > > [r...@core12 ~]# dd if=/apoio04/b1 of=/dev/null bs=64k count=125000 > > > 125000+0 records in > > > 125000+0 records out > > > 819200 bytes (8.2 GB) copied, 94.9401 seconds, 86.3 MB/s > > > > > > I restricted the RAM to be only 1GB, therefore there are no cache > > > effects in these numbers. Because the read stats are good, we exclude > > > network bottlenecks. Nevertheless, we were expecting more or less the > > > performance of a single disk (~50MB/s) for the write tests and we are > > > getting less than half. I do not know if this is really the physical > > > limit of the system or if there is a problem somewhere... > > > > > > I could not find any IBM official numbers, therefore, I though that > > > someone over here could give me any hint about the numbers they are > > > getting... > > > > > > Thanks in Advance > > > Cheers > > > Goncalo Borges > > > > > > > > > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
RE: Open iSCSI Performance on IBM
Hi Mike, Is bs=128K a linux, iscsi or IBM parameter? Thanks, Simone > -Original Message- > From: open-iscsi@googlegroups.com [mailto:open-is...@googlegroups.com] On > Behalf Of Mike Christie > Sent: Thursday, April 09, 2009 10:55 AM > To: open-iscsi@googlegroups.com > Subject: Re: Open iSCSI Performance on IBM > > > Gonçalo Borges wrote: > > Hi All... > > > > Sorry, the following could be a little bit off topic... > > > > Does any one has an idea of what is the expected performance for a IBM > > DS 3300 system connected via open iSCSI? Using a RAID 1 with 2 disks, > > I got the following numbers: > > > > Is /apoio04/b1 a scsi/iscsi disk or is it LVM/DM/RAID on top of a > iscsi/scsi disk? > > Could you set the IO scheduler to noop > echo noop > /sys/block/sdX/queue/scheduler > > and see if that makes a difference. > > Also try > > bs=128k > > And then also run > iscsiadm -m session -P 3 > > > > > Sequential Write: > > > > [r...@core12 ~]# dd if=/dev/zero of=/apoio04/b1 bs=64k count=125000 > > 125000+0 records in > > 125000+0 records out > > 819200 bytes (8.2 GB) copied, 454.522 seconds, 18.0 MB/s > > > > Sequential Read: > > [r...@core12 ~]# dd if=/apoio04/b1 of=/dev/null bs=64k count=125000 > > 125000+0 records in > > 125000+0 records out > > 819200 bytes (8.2 GB) copied, 94.9401 seconds, 86.3 MB/s > > > > I restricted the RAM to be only 1GB, therefore there are no cache > > effects in these numbers. Because the read stats are good, we exclude > > network bottlenecks. Nevertheless, we were expecting more or less the > > performance of a single disk (~50MB/s) for the write tests and we are > > getting less than half. I do not know if this is really the physical > > limit of the system or if there is a problem somewhere... > > > > I could not find any IBM official numbers, therefore, I though that > > someone over here could give me any hint about the numbers they are > > getting... > > > > Thanks in Advance > > Cheers > > Goncalo Borges > > > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Open iSCSI Performance on IBM
Have you made any headway with this issue? I'm having a write issue that seems to share some similarities with yours. On Apr 13, 8:14 am, Gonçalo Borges wrote: > Hi... > > > Is /apoio04/b1 a scsi/iscsi disk or is it LVM/DM/RAID on top of a > > iscsi/scsi disk? > > /apoio04/ is a RAID1 of two disks accessible via iscsi (in the > following tests, I changed the mount point from /apoio04/ to /iscsi04- > lun0/ but they are exactly the same). > > > > > Could you set the IO scheduler to noop > > echo noop > /sys/block/sdX/queue/scheduler and see if that makes a > > difference. > > I checked the definition and I have > > [r...@core06 ~]# cat /sys/block/sdh/queue/scheduler > noop anticipatory deadline [cfq] > > Now I've changed to > > [r...@core06 ~]# cat /sys/block/sdh/queue/scheduler > [noop] anticipatory deadline cfq > > and I've run the tests again. This is what I got: > > [r...@core06 ~]# dd if=/dev/zero of=/iscsi04-lun0/b1 bs=64k > count=125000 > 125000+0 records in > 125000+0 records out > 819200 bytes (8.2 GB) copied, 470.332 seconds, 17.4 MB/s > > [r...@core06 ~]# dd if=/dev/zero of=/iscsi04-lun0/b2 bs=128k > count=62500 > 62500+0 records in > 62500+0 records out > 819200 bytes (8.2 GB) copied, 470.973 seconds, 17.4 MB/s > > Basically, the performance didn't increase :( > > > And then also run > > iscsiadm -m session -P 3 > > [r...@core06 ~]# iscsiadm -m session -P 3 > iSCSI Transport Class version 2.0-724 > iscsiadm version 2.0-868 > Target: iqn.1992-01.com.lsi:1535.600a0b80003ad11c490ade2d > Current Portal: 10.131.2.14:3260,1 > Persistent Portal: 10.131.2.14:3260,1 > ** > Interface: > ** > Iface Name: default > Iface Transport: tcp > Iface Initiatorname: iqn.1994-05.com.redhat:8c56e324f294 > Iface IPaddress: 10.131.4.6 > Iface HWaddress: default > Iface Netdev: default > SID: 37 > iSCSI Connection State: LOGGED IN > iSCSI Session State: Unknown > Internal iscsid Session State: NO CHANGE > > Negotiated iSCSI params: > > HeaderDigest: None > DataDigest: None > MaxRecvDataSegmentLength: 131072 > MaxXmitDataSegmentLength: 65536 > FirstBurstLength: 8192 > MaxBurstLength: 262144 > ImmediateData: Yes > InitialR2T: Yes > MaxOutstandingR2T: 1 > > Attached SCSI devices: > > Host Number: 38 State: running > scsi38 Channel 00 Id 0 Lun: 0 > scsi38 Channel 00 Id 0 Lun: 1 > scsi38 Channel 00 Id 0 Lun: 2 > scsi38 Channel 00 Id 0 Lun: 3 > scsi38 Channel 00 Id 0 Lun: 4 > scsi38 Channel 00 Id 0 Lun: 5 > scsi38 Channel 00 Id 0 Lun: 31 > Current Portal: 10.131.2.13:3260,1 > Persistent Portal: 10.131.2.13:3260,1 > ** > Interface: > ** > Iface Name: default > Iface Transport: tcp > Iface Initiatorname: iqn.1994-05.com.redhat:8c56e324f294 > Iface IPaddress: 10.131.4.6 > Iface HWaddress: default > Iface Netdev: default > SID: 38 > iSCSI Connection State: LOGGED IN > iSCSI Session State: Unknown > Internal iscsid Session State: NO CHANGE > > Negotiated iSCSI params: > > HeaderDigest: None > DataDigest: None > MaxRecvDataSegmentLength: 131072 > MaxXmitDataSegmentLength: 65536 > FirstBurstLength: 8192 > MaxBurstLength: 262144 > ImmediateData: Yes > InitialR2T: Yes > MaxOutstandingR2T: 1 > > Attached SCSI devices: > > Host Number: 39 State: running > scsi39 Channel 00 Id 0 Lun: 0 > scsi39 Channel 00 Id 0 Lun: 1 > scsi39 Channel 00 Id 0 Lun: 2 > scsi39 Channel 00 Id 0 Lun: 3 > scsi39 Channel 00 Id 0 Lun: 4 > scsi39 Channel 00 Id 0 Lun: 5 > scsi39 Channel 00 Id 0 Lun: 31 > Current Portal: 10.131.2.16:3260,2 > Persistent Portal: 10.131.2.16:3260,2 > ** > Interface: >
Re: Open iSCSI Performance on IBM
Hi... > Is /apoio04/b1 a scsi/iscsi disk or is it LVM/DM/RAID on top of a > iscsi/scsi disk? /apoio04/ is a RAID1 of two disks accessible via iscsi (in the following tests, I changed the mount point from /apoio04/ to /iscsi04- lun0/ but they are exactly the same). > > Could you set the IO scheduler to noop > echo noop > /sys/block/sdX/queue/scheduler and see if that makes a difference. I checked the definition and I have [r...@core06 ~]# cat /sys/block/sdh/queue/scheduler noop anticipatory deadline [cfq] Now I've changed to [r...@core06 ~]# cat /sys/block/sdh/queue/scheduler [noop] anticipatory deadline cfq and I've run the tests again. This is what I got: [r...@core06 ~]# dd if=/dev/zero of=/iscsi04-lun0/b1 bs=64k count=125000 125000+0 records in 125000+0 records out 819200 bytes (8.2 GB) copied, 470.332 seconds, 17.4 MB/s [r...@core06 ~]# dd if=/dev/zero of=/iscsi04-lun0/b2 bs=128k count=62500 62500+0 records in 62500+0 records out 819200 bytes (8.2 GB) copied, 470.973 seconds, 17.4 MB/s Basically, the performance didn't increase :( > And then also run > iscsiadm -m session -P 3 > [r...@core06 ~]# iscsiadm -m session -P 3 iSCSI Transport Class version 2.0-724 iscsiadm version 2.0-868 Target: iqn.1992-01.com.lsi:1535.600a0b80003ad11c490ade2d Current Portal: 10.131.2.14:3260,1 Persistent Portal: 10.131.2.14:3260,1 ** Interface: ** Iface Name: default Iface Transport: tcp Iface Initiatorname: iqn.1994-05.com.redhat:8c56e324f294 Iface IPaddress: 10.131.4.6 Iface HWaddress: default Iface Netdev: default SID: 37 iSCSI Connection State: LOGGED IN iSCSI Session State: Unknown Internal iscsid Session State: NO CHANGE Negotiated iSCSI params: HeaderDigest: None DataDigest: None MaxRecvDataSegmentLength: 131072 MaxXmitDataSegmentLength: 65536 FirstBurstLength: 8192 MaxBurstLength: 262144 ImmediateData: Yes InitialR2T: Yes MaxOutstandingR2T: 1 Attached SCSI devices: Host Number: 38 State: running scsi38 Channel 00 Id 0 Lun: 0 scsi38 Channel 00 Id 0 Lun: 1 scsi38 Channel 00 Id 0 Lun: 2 scsi38 Channel 00 Id 0 Lun: 3 scsi38 Channel 00 Id 0 Lun: 4 scsi38 Channel 00 Id 0 Lun: 5 scsi38 Channel 00 Id 0 Lun: 31 Current Portal: 10.131.2.13:3260,1 Persistent Portal: 10.131.2.13:3260,1 ** Interface: ** Iface Name: default Iface Transport: tcp Iface Initiatorname: iqn.1994-05.com.redhat:8c56e324f294 Iface IPaddress: 10.131.4.6 Iface HWaddress: default Iface Netdev: default SID: 38 iSCSI Connection State: LOGGED IN iSCSI Session State: Unknown Internal iscsid Session State: NO CHANGE Negotiated iSCSI params: HeaderDigest: None DataDigest: None MaxRecvDataSegmentLength: 131072 MaxXmitDataSegmentLength: 65536 FirstBurstLength: 8192 MaxBurstLength: 262144 ImmediateData: Yes InitialR2T: Yes MaxOutstandingR2T: 1 Attached SCSI devices: Host Number: 39 State: running scsi39 Channel 00 Id 0 Lun: 0 scsi39 Channel 00 Id 0 Lun: 1 scsi39 Channel 00 Id 0 Lun: 2 scsi39 Channel 00 Id 0 Lun: 3 scsi39 Channel 00 Id 0 Lun: 4 scsi39 Channel 00 Id 0 Lun: 5 scsi39 Channel 00 Id 0 Lun: 31 Current Portal: 10.131.2.16:3260,2 Persistent Portal: 10.131.2.16:3260,2 ** Interface: ** Iface Name: default Iface Transport: tcp Iface Initiatorname: iqn.1994-05.com.redhat:8c56e324f294 Iface IPaddress: 10.131.4.6 Iface HWaddress: default Iface Netdev: default SID: 39 iSCSI Connection State: LOGGED IN iSCSI Session State:
Re: Open iSCSI Performance on IBM
Gonçalo Borges wrote: > Hi All... > > Sorry, the following could be a little bit off topic... > > Does any one has an idea of what is the expected performance for a IBM > DS 3300 system connected via open iSCSI? Using a RAID 1 with 2 disks, > I got the following numbers: > Is /apoio04/b1 a scsi/iscsi disk or is it LVM/DM/RAID on top of a iscsi/scsi disk? Could you set the IO scheduler to noop echo noop > /sys/block/sdX/queue/scheduler and see if that makes a difference. Also try bs=128k And then also run iscsiadm -m session -P 3 > Sequential Write: > > [r...@core12 ~]# dd if=/dev/zero of=/apoio04/b1 bs=64k count=125000 > 125000+0 records in > 125000+0 records out > 819200 bytes (8.2 GB) copied, 454.522 seconds, 18.0 MB/s > > Sequential Read: > [r...@core12 ~]# dd if=/apoio04/b1 of=/dev/null bs=64k count=125000 > 125000+0 records in > 125000+0 records out > 819200 bytes (8.2 GB) copied, 94.9401 seconds, 86.3 MB/s > > I restricted the RAM to be only 1GB, therefore there are no cache > effects in these numbers. Because the read stats are good, we exclude > network bottlenecks. Nevertheless, we were expecting more or less the > performance of a single disk (~50MB/s) for the write tests and we are > getting less than half. I do not know if this is really the physical > limit of the system or if there is a problem somewhere... > > I could not find any IBM official numbers, therefore, I though that > someone over here could give me any hint about the numbers they are > getting... > > Thanks in Advance > Cheers > Goncalo Borges > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Open iSCSI Performance on IBM
Basically, I want to have an idea of the performance other admins are getting... The hardware is not so important since I'm interested in the order of magnitude of the performance rates... On Apr 9, 10:22 am, Gonçalo Borges wrote: > > What is the write performance if you do RAID0 instead of RAID1? > > Basically 30MB/s, i.e., twice the value I get in a RAID1. > > > Also, you can use 'direct' I/O flag with "dd" command. > > I'll try using this one! > > Thanks and cheers > Goncalo --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Open iSCSI Performance on IBM
> What is the write performance if you do RAID0 instead of RAID1? Basically 30MB/s, i.e., twice the value I get in a RAID1. > Also, you can use 'direct' I/O flag with "dd" command. I'll try using this one! Thanks and cheers Goncalo --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Open iSCSI Performance on IBM
What is the write performance if you do RAID0 instead of RAID1? I don't have that hardware and I can't verify anything though. Also, you can use 'direct' I/O flag with "dd" command. Thanks, Malahal. Gon?alo Borges [borges.gonc...@gmail.com] wrote: > > Hi All... > > Sorry, the following could be a little bit off topic... > > Does any one has an idea of what is the expected performance for a IBM > DS 3300 system connected via open iSCSI? Using a RAID 1 with 2 disks, > I got the following numbers: > > Sequential Write: > > [r...@core12 ~]# dd if=/dev/zero of=/apoio04/b1 bs=64k count=125000 > 125000+0 records in > 125000+0 records out > 819200 bytes (8.2 GB) copied, 454.522 seconds, 18.0 MB/s > > Sequential Read: > [r...@core12 ~]# dd if=/apoio04/b1 of=/dev/null bs=64k count=125000 > 125000+0 records in > 125000+0 records out > 819200 bytes (8.2 GB) copied, 94.9401 seconds, 86.3 MB/s > > I restricted the RAM to be only 1GB, therefore there are no cache > effects in these numbers. Because the read stats are good, we exclude > network bottlenecks. Nevertheless, we were expecting more or less the > performance of a single disk (~50MB/s) for the write tests and we are > getting less than half. I do not know if this is really the physical > limit of the system or if there is a problem somewhere... > > I could not find any IBM official numbers, therefore, I though that > someone over here could give me any hint about the numbers they are > getting... > > Thanks in Advance > Cheers > Goncalo Borges > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "open-iscsi" group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
