Re: Open iSCSI Performance on IBM
Unfortunately not. I'm now trying to optimize some filesystem options to check if that increases performance... but from the iscsi part, I do not know what else to optimize. I could be reaching the physical limit of the system but I'm not sure since I do not know about any other performance numbers from any other systems. On Apr 13, 7:00 pm, jnantel nan...@hotmail.com wrote: Have you made any headway with this issue? I'm having a write issue that seems to share some similarities with yours. On Apr 13, 8:14 am, Gonçalo Borges borges.gonc...@gmail.com wrote: Hi... Is /apoio04/b1 a scsi/iscsi disk or is it LVM/DM/RAID on top of a iscsi/scsi disk? /apoio04/ is a RAID1 of two disks accessible via iscsi (in the following tests, I changed the mount point from /apoio04/ to /iscsi04- lun0/ but they are exactly the same). Could you set the IO scheduler to noop echo noop /sys/block/sdX/queue/scheduler and see if that makes a difference. I checked the definition and I have [r...@core06 ~]# cat /sys/block/sdh/queue/scheduler noop anticipatory deadline [cfq] Now I've changed to [r...@core06 ~]# cat /sys/block/sdh/queue/scheduler [noop] anticipatory deadline cfq and I've run the tests again. This is what I got: [r...@core06 ~]# dd if=/dev/zero of=/iscsi04-lun0/b1 bs=64k count=125000 125000+0 records in 125000+0 records out 819200 bytes (8.2 GB) copied, 470.332 seconds, 17.4 MB/s [r...@core06 ~]# dd if=/dev/zero of=/iscsi04-lun0/b2 bs=128k count=62500 62500+0 records in 62500+0 records out 819200 bytes (8.2 GB) copied, 470.973 seconds, 17.4 MB/s Basically, the performance didn't increase :( And then also run iscsiadm -m session -P 3 [r...@core06 ~]# iscsiadm -m session -P 3 iSCSI Transport Class version 2.0-724 iscsiadm version 2.0-868 Target: iqn.1992-01.com.lsi:1535.600a0b80003ad11c490ade2d Current Portal: 10.131.2.14:3260,1 Persistent Portal: 10.131.2.14:3260,1 ** Interface: ** Iface Name: default Iface Transport: tcp Iface Initiatorname: iqn.1994-05.com.redhat:8c56e324f294 Iface IPaddress: 10.131.4.6 Iface HWaddress: default Iface Netdev: default SID: 37 iSCSI Connection State: LOGGED IN iSCSI Session State: Unknown Internal iscsid Session State: NO CHANGE Negotiated iSCSI params: HeaderDigest: None DataDigest: None MaxRecvDataSegmentLength: 131072 MaxXmitDataSegmentLength: 65536 FirstBurstLength: 8192 MaxBurstLength: 262144 ImmediateData: Yes InitialR2T: Yes MaxOutstandingR2T: 1 Attached SCSI devices: Host Number: 38 State: running scsi38 Channel 00 Id 0 Lun: 0 scsi38 Channel 00 Id 0 Lun: 1 scsi38 Channel 00 Id 0 Lun: 2 scsi38 Channel 00 Id 0 Lun: 3 scsi38 Channel 00 Id 0 Lun: 4 scsi38 Channel 00 Id 0 Lun: 5 scsi38 Channel 00 Id 0 Lun: 31 Current Portal: 10.131.2.13:3260,1 Persistent Portal: 10.131.2.13:3260,1 ** Interface: ** Iface Name: default Iface Transport: tcp Iface Initiatorname: iqn.1994-05.com.redhat:8c56e324f294 Iface IPaddress: 10.131.4.6 Iface HWaddress: default Iface Netdev: default SID: 38 iSCSI Connection State: LOGGED IN iSCSI Session State: Unknown Internal iscsid Session State: NO CHANGE Negotiated iSCSI params: HeaderDigest: None DataDigest: None MaxRecvDataSegmentLength: 131072 MaxXmitDataSegmentLength: 65536 FirstBurstLength: 8192 MaxBurstLength: 262144 ImmediateData: Yes InitialR2T: Yes MaxOutstandingR2T: 1 Attached SCSI devices: Host Number: 39 State: running scsi39 Channel 00 Id 0 Lun: 0 scsi39 Channel 00 Id 0 Lun: 1
Re: Open iSCSI Performance on IBM
Hi... Is /apoio04/b1 a scsi/iscsi disk or is it LVM/DM/RAID on top of a iscsi/scsi disk? /apoio04/ is a RAID1 of two disks accessible via iscsi (in the following tests, I changed the mount point from /apoio04/ to /iscsi04- lun0/ but they are exactly the same). Could you set the IO scheduler to noop echo noop /sys/block/sdX/queue/scheduler and see if that makes a difference. I checked the definition and I have [r...@core06 ~]# cat /sys/block/sdh/queue/scheduler noop anticipatory deadline [cfq] Now I've changed to [r...@core06 ~]# cat /sys/block/sdh/queue/scheduler [noop] anticipatory deadline cfq and I've run the tests again. This is what I got: [r...@core06 ~]# dd if=/dev/zero of=/iscsi04-lun0/b1 bs=64k count=125000 125000+0 records in 125000+0 records out 819200 bytes (8.2 GB) copied, 470.332 seconds, 17.4 MB/s [r...@core06 ~]# dd if=/dev/zero of=/iscsi04-lun0/b2 bs=128k count=62500 62500+0 records in 62500+0 records out 819200 bytes (8.2 GB) copied, 470.973 seconds, 17.4 MB/s Basically, the performance didn't increase :( And then also run iscsiadm -m session -P 3 [r...@core06 ~]# iscsiadm -m session -P 3 iSCSI Transport Class version 2.0-724 iscsiadm version 2.0-868 Target: iqn.1992-01.com.lsi:1535.600a0b80003ad11c490ade2d Current Portal: 10.131.2.14:3260,1 Persistent Portal: 10.131.2.14:3260,1 ** Interface: ** Iface Name: default Iface Transport: tcp Iface Initiatorname: iqn.1994-05.com.redhat:8c56e324f294 Iface IPaddress: 10.131.4.6 Iface HWaddress: default Iface Netdev: default SID: 37 iSCSI Connection State: LOGGED IN iSCSI Session State: Unknown Internal iscsid Session State: NO CHANGE Negotiated iSCSI params: HeaderDigest: None DataDigest: None MaxRecvDataSegmentLength: 131072 MaxXmitDataSegmentLength: 65536 FirstBurstLength: 8192 MaxBurstLength: 262144 ImmediateData: Yes InitialR2T: Yes MaxOutstandingR2T: 1 Attached SCSI devices: Host Number: 38 State: running scsi38 Channel 00 Id 0 Lun: 0 scsi38 Channel 00 Id 0 Lun: 1 scsi38 Channel 00 Id 0 Lun: 2 scsi38 Channel 00 Id 0 Lun: 3 scsi38 Channel 00 Id 0 Lun: 4 scsi38 Channel 00 Id 0 Lun: 5 scsi38 Channel 00 Id 0 Lun: 31 Current Portal: 10.131.2.13:3260,1 Persistent Portal: 10.131.2.13:3260,1 ** Interface: ** Iface Name: default Iface Transport: tcp Iface Initiatorname: iqn.1994-05.com.redhat:8c56e324f294 Iface IPaddress: 10.131.4.6 Iface HWaddress: default Iface Netdev: default SID: 38 iSCSI Connection State: LOGGED IN iSCSI Session State: Unknown Internal iscsid Session State: NO CHANGE Negotiated iSCSI params: HeaderDigest: None DataDigest: None MaxRecvDataSegmentLength: 131072 MaxXmitDataSegmentLength: 65536 FirstBurstLength: 8192 MaxBurstLength: 262144 ImmediateData: Yes InitialR2T: Yes MaxOutstandingR2T: 1 Attached SCSI devices: Host Number: 39 State: running scsi39 Channel 00 Id 0 Lun: 0 scsi39 Channel 00 Id 0 Lun: 1 scsi39 Channel 00 Id 0 Lun: 2 scsi39 Channel 00 Id 0 Lun: 3 scsi39 Channel 00 Id 0 Lun: 4 scsi39 Channel 00 Id 0 Lun: 5 scsi39 Channel 00 Id 0 Lun: 31 Current Portal: 10.131.2.16:3260,2 Persistent Portal: 10.131.2.16:3260,2 ** Interface: ** Iface Name: default Iface Transport: tcp Iface Initiatorname: iqn.1994-05.com.redhat:8c56e324f294 Iface IPaddress: 10.131.4.6 Iface HWaddress: default Iface Netdev: default SID: 39 iSCSI Connection State: LOGGED IN iSCSI Session State: Unknown
Re: Open iSCSI Performance on IBM
Have you made any headway with this issue? I'm having a write issue that seems to share some similarities with yours. On Apr 13, 8:14 am, Gonçalo Borges borges.gonc...@gmail.com wrote: Hi... Is /apoio04/b1 a scsi/iscsi disk or is it LVM/DM/RAID on top of a iscsi/scsi disk? /apoio04/ is a RAID1 of two disks accessible via iscsi (in the following tests, I changed the mount point from /apoio04/ to /iscsi04- lun0/ but they are exactly the same). Could you set the IO scheduler to noop echo noop /sys/block/sdX/queue/scheduler and see if that makes a difference. I checked the definition and I have [r...@core06 ~]# cat /sys/block/sdh/queue/scheduler noop anticipatory deadline [cfq] Now I've changed to [r...@core06 ~]# cat /sys/block/sdh/queue/scheduler [noop] anticipatory deadline cfq and I've run the tests again. This is what I got: [r...@core06 ~]# dd if=/dev/zero of=/iscsi04-lun0/b1 bs=64k count=125000 125000+0 records in 125000+0 records out 819200 bytes (8.2 GB) copied, 470.332 seconds, 17.4 MB/s [r...@core06 ~]# dd if=/dev/zero of=/iscsi04-lun0/b2 bs=128k count=62500 62500+0 records in 62500+0 records out 819200 bytes (8.2 GB) copied, 470.973 seconds, 17.4 MB/s Basically, the performance didn't increase :( And then also run iscsiadm -m session -P 3 [r...@core06 ~]# iscsiadm -m session -P 3 iSCSI Transport Class version 2.0-724 iscsiadm version 2.0-868 Target: iqn.1992-01.com.lsi:1535.600a0b80003ad11c490ade2d Current Portal: 10.131.2.14:3260,1 Persistent Portal: 10.131.2.14:3260,1 ** Interface: ** Iface Name: default Iface Transport: tcp Iface Initiatorname: iqn.1994-05.com.redhat:8c56e324f294 Iface IPaddress: 10.131.4.6 Iface HWaddress: default Iface Netdev: default SID: 37 iSCSI Connection State: LOGGED IN iSCSI Session State: Unknown Internal iscsid Session State: NO CHANGE Negotiated iSCSI params: HeaderDigest: None DataDigest: None MaxRecvDataSegmentLength: 131072 MaxXmitDataSegmentLength: 65536 FirstBurstLength: 8192 MaxBurstLength: 262144 ImmediateData: Yes InitialR2T: Yes MaxOutstandingR2T: 1 Attached SCSI devices: Host Number: 38 State: running scsi38 Channel 00 Id 0 Lun: 0 scsi38 Channel 00 Id 0 Lun: 1 scsi38 Channel 00 Id 0 Lun: 2 scsi38 Channel 00 Id 0 Lun: 3 scsi38 Channel 00 Id 0 Lun: 4 scsi38 Channel 00 Id 0 Lun: 5 scsi38 Channel 00 Id 0 Lun: 31 Current Portal: 10.131.2.13:3260,1 Persistent Portal: 10.131.2.13:3260,1 ** Interface: ** Iface Name: default Iface Transport: tcp Iface Initiatorname: iqn.1994-05.com.redhat:8c56e324f294 Iface IPaddress: 10.131.4.6 Iface HWaddress: default Iface Netdev: default SID: 38 iSCSI Connection State: LOGGED IN iSCSI Session State: Unknown Internal iscsid Session State: NO CHANGE Negotiated iSCSI params: HeaderDigest: None DataDigest: None MaxRecvDataSegmentLength: 131072 MaxXmitDataSegmentLength: 65536 FirstBurstLength: 8192 MaxBurstLength: 262144 ImmediateData: Yes InitialR2T: Yes MaxOutstandingR2T: 1 Attached SCSI devices: Host Number: 39 State: running scsi39 Channel 00 Id 0 Lun: 0 scsi39 Channel 00 Id 0 Lun: 1 scsi39 Channel 00 Id 0 Lun: 2 scsi39 Channel 00 Id 0 Lun: 3 scsi39 Channel 00 Id 0 Lun: 4 scsi39 Channel 00 Id 0 Lun: 5 scsi39 Channel 00 Id 0 Lun: 31 Current Portal: 10.131.2.16:3260,2 Persistent Portal: 10.131.2.16:3260,2 ** Interface: ** Iface Name: default Iface Transport: tcp
RE: Open iSCSI Performance on IBM
Hi Mike, Is bs=128K a linux, iscsi or IBM parameter? Thanks, Simone -Original Message- From: open-iscsi@googlegroups.com [mailto:open-is...@googlegroups.com] On Behalf Of Mike Christie Sent: Thursday, April 09, 2009 10:55 AM To: open-iscsi@googlegroups.com Subject: Re: Open iSCSI Performance on IBM Gonçalo Borges wrote: Hi All... Sorry, the following could be a little bit off topic... Does any one has an idea of what is the expected performance for a IBM DS 3300 system connected via open iSCSI? Using a RAID 1 with 2 disks, I got the following numbers: Is /apoio04/b1 a scsi/iscsi disk or is it LVM/DM/RAID on top of a iscsi/scsi disk? Could you set the IO scheduler to noop echo noop /sys/block/sdX/queue/scheduler and see if that makes a difference. Also try bs=128k And then also run iscsiadm -m session -P 3 Sequential Write: [r...@core12 ~]# dd if=/dev/zero of=/apoio04/b1 bs=64k count=125000 125000+0 records in 125000+0 records out 819200 bytes (8.2 GB) copied, 454.522 seconds, 18.0 MB/s Sequential Read: [r...@core12 ~]# dd if=/apoio04/b1 of=/dev/null bs=64k count=125000 125000+0 records in 125000+0 records out 819200 bytes (8.2 GB) copied, 94.9401 seconds, 86.3 MB/s I restricted the RAM to be only 1GB, therefore there are no cache effects in these numbers. Because the read stats are good, we exclude network bottlenecks. Nevertheless, we were expecting more or less the performance of a single disk (~50MB/s) for the write tests and we are getting less than half. I do not know if this is really the physical limit of the system or if there is a problem somewhere... I could not find any IBM official numbers, therefore, I though that someone over here could give me any hint about the numbers they are getting... Thanks in Advance Cheers Goncalo Borges --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
RE: Open iSCSI Performance on IBM
Disregard I just saw it was a dd option. Simone -Original Message- From: open-iscsi@googlegroups.com [mailto:open-is...@googlegroups.com] On Behalf Of Simone Morellato Sent: Monday, April 13, 2009 3:04 PM To: open-iscsi@googlegroups.com Subject: RE: Open iSCSI Performance on IBM Hi Mike, Is bs=128K a linux, iscsi or IBM parameter? Thanks, Simone -Original Message- From: open-iscsi@googlegroups.com [mailto:open-is...@googlegroups.com] On Behalf Of Mike Christie Sent: Thursday, April 09, 2009 10:55 AM To: open-iscsi@googlegroups.com Subject: Re: Open iSCSI Performance on IBM Gonçalo Borges wrote: Hi All... Sorry, the following could be a little bit off topic... Does any one has an idea of what is the expected performance for a IBM DS 3300 system connected via open iSCSI? Using a RAID 1 with 2 disks, I got the following numbers: Is /apoio04/b1 a scsi/iscsi disk or is it LVM/DM/RAID on top of a iscsi/scsi disk? Could you set the IO scheduler to noop echo noop /sys/block/sdX/queue/scheduler and see if that makes a difference. Also try bs=128k And then also run iscsiadm -m session -P 3 Sequential Write: [r...@core12 ~]# dd if=/dev/zero of=/apoio04/b1 bs=64k count=125000 125000+0 records in 125000+0 records out 819200 bytes (8.2 GB) copied, 454.522 seconds, 18.0 MB/s Sequential Read: [r...@core12 ~]# dd if=/apoio04/b1 of=/dev/null bs=64k count=125000 125000+0 records in 125000+0 records out 819200 bytes (8.2 GB) copied, 94.9401 seconds, 86.3 MB/s I restricted the RAM to be only 1GB, therefore there are no cache effects in these numbers. Because the read stats are good, we exclude network bottlenecks. Nevertheless, we were expecting more or less the performance of a single disk (~50MB/s) for the write tests and we are getting less than half. I do not know if this is really the physical limit of the system or if there is a problem somewhere... I could not find any IBM official numbers, therefore, I though that someone over here could give me any hint about the numbers they are getting... Thanks in Advance Cheers Goncalo Borges --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Open iSCSI Performance on IBM
Hi All... Sorry, the following could be a little bit off topic... Does any one has an idea of what is the expected performance for a IBM DS 3300 system connected via open iSCSI? Using a RAID 1 with 2 disks, I got the following numbers: Sequential Write: [r...@core12 ~]# dd if=/dev/zero of=/apoio04/b1 bs=64k count=125000 125000+0 records in 125000+0 records out 819200 bytes (8.2 GB) copied, 454.522 seconds, 18.0 MB/s Sequential Read: [r...@core12 ~]# dd if=/apoio04/b1 of=/dev/null bs=64k count=125000 125000+0 records in 125000+0 records out 819200 bytes (8.2 GB) copied, 94.9401 seconds, 86.3 MB/s I restricted the RAM to be only 1GB, therefore there are no cache effects in these numbers. Because the read stats are good, we exclude network bottlenecks. Nevertheless, we were expecting more or less the performance of a single disk (~50MB/s) for the write tests and we are getting less than half. I do not know if this is really the physical limit of the system or if there is a problem somewhere... I could not find any IBM official numbers, therefore, I though that someone over here could give me any hint about the numbers they are getting... Thanks in Advance Cheers Goncalo Borges --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Open iSCSI Performance on IBM
What is the write performance if you do RAID0 instead of RAID1? Basically 30MB/s, i.e., twice the value I get in a RAID1. Also, you can use 'direct' I/O flag with dd command. I'll try using this one! Thanks and cheers Goncalo --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Open iSCSI Performance on IBM
Basically, I want to have an idea of the performance other admins are getting... The hardware is not so important since I'm interested in the order of magnitude of the performance rates... On Apr 9, 10:22 am, Gonçalo Borges borges.gonc...@gmail.com wrote: What is the write performance if you do RAID0 instead of RAID1? Basically 30MB/s, i.e., twice the value I get in a RAID1. Also, you can use 'direct' I/O flag with dd command. I'll try using this one! Thanks and cheers Goncalo --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Open iSCSI Performance on IBM
What is the write performance if you do RAID0 instead of RAID1? I don't have that hardware and I can't verify anything though. Also, you can use 'direct' I/O flag with dd command. Thanks, Malahal. Gon?alo Borges [borges.gonc...@gmail.com] wrote: Hi All... Sorry, the following could be a little bit off topic... Does any one has an idea of what is the expected performance for a IBM DS 3300 system connected via open iSCSI? Using a RAID 1 with 2 disks, I got the following numbers: Sequential Write: [r...@core12 ~]# dd if=/dev/zero of=/apoio04/b1 bs=64k count=125000 125000+0 records in 125000+0 records out 819200 bytes (8.2 GB) copied, 454.522 seconds, 18.0 MB/s Sequential Read: [r...@core12 ~]# dd if=/apoio04/b1 of=/dev/null bs=64k count=125000 125000+0 records in 125000+0 records out 819200 bytes (8.2 GB) copied, 94.9401 seconds, 86.3 MB/s I restricted the RAM to be only 1GB, therefore there are no cache effects in these numbers. Because the read stats are good, we exclude network bottlenecks. Nevertheless, we were expecting more or less the performance of a single disk (~50MB/s) for the write tests and we are getting less than half. I do not know if this is really the physical limit of the system or if there is a problem somewhere... I could not find any IBM official numbers, therefore, I though that someone over here could give me any hint about the numbers they are getting... Thanks in Advance Cheers Goncalo Borges --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Open iSCSI Performance on IBM
Gonçalo Borges wrote: Hi All... Sorry, the following could be a little bit off topic... Does any one has an idea of what is the expected performance for a IBM DS 3300 system connected via open iSCSI? Using a RAID 1 with 2 disks, I got the following numbers: Is /apoio04/b1 a scsi/iscsi disk or is it LVM/DM/RAID on top of a iscsi/scsi disk? Could you set the IO scheduler to noop echo noop /sys/block/sdX/queue/scheduler and see if that makes a difference. Also try bs=128k And then also run iscsiadm -m session -P 3 Sequential Write: [r...@core12 ~]# dd if=/dev/zero of=/apoio04/b1 bs=64k count=125000 125000+0 records in 125000+0 records out 819200 bytes (8.2 GB) copied, 454.522 seconds, 18.0 MB/s Sequential Read: [r...@core12 ~]# dd if=/apoio04/b1 of=/dev/null bs=64k count=125000 125000+0 records in 125000+0 records out 819200 bytes (8.2 GB) copied, 94.9401 seconds, 86.3 MB/s I restricted the RAM to be only 1GB, therefore there are no cache effects in these numbers. Because the read stats are good, we exclude network bottlenecks. Nevertheless, we were expecting more or less the performance of a single disk (~50MB/s) for the write tests and we are getting less than half. I do not know if this is really the physical limit of the system or if there is a problem somewhere... I could not find any IBM official numbers, therefore, I though that someone over here could give me any hint about the numbers they are getting... Thanks in Advance Cheers Goncalo Borges --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
