Re: iSCSI and FileSystem (ext2/ext3)
On Wed, Apr 15, 2009 at 8:24 PM, benoit plessis plessis.ben...@gmail.com wrote: I wanted to share some infos about a discovery we made using mysql over iSCSI. A few questions: - Was MySQL configured for direct I/O or for buffered I/O ? - Which mount options were used with the ext2 / ext3 filesystems ? Was noatime or relatime enabled ? Bart. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: iSCSI and FileSystem (ext2/ext3)
On Wed, Apr 15, 2009 at 9:19 PM, Pasi Kärkkäinen pa...@iki.fi wrote: noop is usually good for the initiator. cfq has a feature (or a bug?) that prevents achieving queue depths deeper than 1, and thus limits your bandwidth a lot when there are (or should be) many ios on the fly at the same time. Hello Pasi, Do you have any idea whether modifying the tunable CFQ parameters could resolve the bandwidth limiting behavior of CFQ ? $ ls /sys/block/sda/queue/iosched back_seek_max fifo_expire_async quantum slice_async_rq slice_sync back_seek_penalty fifo_expire_sync slice_async slice_idle Bart. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Q: iSCSI Session State: Unknown
Hi! I'm wondering: For iscsiadm -m session -P 3 (SLES10 SP2) I see output like this: iSCSI Connection State: LOGGED IN iSCSI Session State: Unknown Internal iscsid Session State: NO CHANGE What is a Session State of Unknown? I.e. hwat are the valid session states? Regards, Ulrich --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: iSCSI and FileSystem (ext2/ext3)
On Wed, Apr 15, 2009 at 9:19 PM, Pasi Kärkkäinen pa...@iki.fi wrote: noop is usually good for the initiator. cfq has a feature (or a bug?) that prevents achieving queue depths deeper than 1, and thus limits your bandwidth a lot when there are (or should be) many ios on the fly at the same time. Do you remember on which kernel version you observed the above behavior ? This might be a bug in the CFQ scheduler. I found the following in the 2.6.28 changelog: cfq-iosched: fix queue depth detection. See also http://www.eu.kernel.org/pub/linux/kernel/v2.6/ChangeLog-2.6.28 or http://lkml.org/lkml/2008/8/22/39. Bart. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: iSCSI and FileSystem (ext2/ext3)
On 15 Apr 2009 at 20:42, Bart Van Assche wrote: [...] Why are you using the noop scheduler on the initiator instead of deadline or CFQ ? The performance difference you observed is probably caused by something else than the filesystem. When running bonnie++ on a local filesystem, xfs gives better performance than ext2, and ext2 gives better performance than ext3. Considering that most remote iSCSI targets have intelligece of their own, the use of a no-op i/O scheduler seems justified. I think multiple I/O schedulers on both ends don't necessarily make things faster, unless there is a bunch of requests that can be compacted into one. Then throughput rises while response may decline. So it very much depends on what you really want to have. Regards, Ulrich --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: iSCSI and FileSystem (ext2/ext3)
On Thu, Apr 16, 2009 at 1:10 PM, Ulrich Windl ulrich.wi...@rz.uni-regensburg.de wrote: Considering that most remote iSCSI targets have intelligece of their own, the use of a no-op i/O scheduler seems justified. I think multiple I/O schedulers on both ends don't necessarily make things faster, unless there is a bunch of requests that can be compacted into one. Then throughput rises while response may decline. Is request merging implemented in the SCSI layer or in the I/O scheduler ? Bart. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: iSCSI and FileSystem (ext2/ext3)
On Thu, Apr 16, 2009 at 11:50:27AM +0200, Bart Van Assche wrote: On Wed, Apr 15, 2009 at 9:19 PM, Pasi Kärkkäinen pa...@iki.fi wrote: noop is usually good for the initiator. cfq has a feature (or a bug?) that prevents achieving queue depths deeper than 1, and thus limits your bandwidth a lot when there are (or should be) many ios on the fly at the same time. Do you remember on which kernel version you observed the above behavior ? This might be a bug in the CFQ scheduler. I found the following in the 2.6.28 changelog: cfq-iosched: fix queue depth detection. See also http://www.eu.kernel.org/pub/linux/kernel/v2.6/ChangeLog-2.6.28 or http://lkml.org/lkml/2008/8/22/39. Iirc it has been with RHEL5/CentOS5 2.6.18 based kernels.. Mike Christie has been writing about this aswell.. dunno about what kernels he has seen it with. Then again CFQ was designed for single disk workstations.. -- Pasi --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: iSCSI and FileSystem (ext2/ext3)
On 16 Apr 2009 at 13:19, Bart Van Assche wrote: On Thu, Apr 16, 2009 at 1:10 PM, Ulrich Windl ulrich.wi...@rz.uni-regensburg.de wrote: Considering that most remote iSCSI targets have intelligece of their own, the use of a no-op i/O scheduler seems justified. I think multiple I/O schedulers on both ends don't necessarily make things faster, unless there is a bunch of requests that can be compacted into one. Then throughput rises while response may decline. Is request merging implemented in the SCSI layer or in the I/O scheduler ? AFAIK, it's implemented in the I/O scheduler at least (depending on the actual type being used). I don't know about the SCSI layer. Regards, Ulrich --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: open-iscsi created only one disk but missing 2 more
What Target are you using? I beat my head on a issue when I first setup regarding access permissions. If you don't have the permissions right on the array, open-iscsi will let you see it but not log in. Another thing you may want to check, if you are sharing luns between systems, some arrays require you to explicitly allow that behavior. And last but not least, if you have a backend and a front end connection make sure you are connecting to the array with the right interface. Make sure one connection does not have a matching MTU *jumbo frames with the array it can cause strange behavior. On Apr 15, 10:27 pm, sundar mahadevan sundarmahadeva...@gmail.com wrote: I tried the same connecting another hard drive of 20 G with 3 logical volumes namely: asm 17G , ocr 924M and vote 760M. iscsiadm -m node -T iqn.2001-04.com.ubuntu:scsi.disk.vg1.ocr -p 192.168.20.22 -l iscsiadm -m node -T iqn.2001-04.com.ubuntu:scsi.disk.vg1.vote -p 192.168.20.22 -l iscsiadm -m node -T iqn.2001-04.com.ubuntu:scsi.disk.vg1.asm -p 192.168.20.22 -l Looks like it only created the first one: ocr Here is the log from /var/log/syslog Apr 15 21:10:17 sunny2 kernel: [ 1603.409561] Loading iSCSI transport class v2.0-870. Apr 15 21:10:17 sunny2 kernel: [ 1603.486634] iscsi: registered transport (tcp) Apr 15 21:10:18 sunny2 kernel: [ 1603.848719] iscsi: registered transport (iser) Apr 15 21:10:18 sunny2 iscsid: iSCSI logger with pid=5912 started! Apr 15 21:10:18 sunny2 kernel: [ 1604.408284] scsi2 : iSCSI Initiator over TCP/IP Apr 15 21:10:19 sunny2 iscsid: transport class version 2.0-870. iscsid version 2.0-865 Apr 15 21:10:19 sunny2 iscsid: iSCSI daemon with pid=5914 started! Apr 15 21:10:19 sunny2 iscsid: Could not verify connection 1:2. Dropping event. Apr 15 21:10:19 sunny2 iscsid: Could not verify connection 1:2. Dropping event. Apr 15 21:10:20 sunny2 kernel: [ 1606.064265] scsi3 : iSCSI Initiator over TCP/IP Apr 15 21:10:21 sunny2 iscsid: Could not verify connection 2:3. Dropping event. Apr 15 21:10:21 sunny2 iscsid: Could not verify connection 2:3. Dropping event. Apr 15 21:10:21 sunny2 kernel: [ 1607.592257] scsi4 : iSCSI Initiator over TCP/IP Apr 15 21:10:22 sunny2 iscsid: Could not verify connection 3:4. Dropping event. Apr 15 21:10:22 sunny2 iscsid: Could not verify connection 3:4. Dropping event. Apr 15 21:10:23 sunny2 kernel: [ 1609.120249] scsi5 : iSCSI Initiator over TCP/IP Apr 15 21:10:24 sunny2 iscsid: Could not verify connection 4:5. Dropping event. Apr 15 21:10:24 sunny2 iscsid: Could not verify connection 4:5. Dropping event. Apr 15 21:15:00 sunny2 kernel: [ 1886.664257] scsi6 : iSCSI Initiator over TCP/IP Apr 15 21:15:01 sunny2 iscsid: Could not verify connection 5:6. Dropping event. Apr 15 21:15:01 sunny2 kernel: [ 1886.946886] scsi 6:0:0:0: Direct-Access IET VIRTUAL-DISK 0 PQ: 0 ANSI: 4 Apr 15 21:15:01 sunny2 kernel: [ 1886.952267] sd 6:0:0:0: [sdb] 1892352 512-byte hardware sectors (969 MB) Apr 15 21:15:01 sunny2 kernel: [ 1886.956338] sd 6:0:0:0: [sdb] Write Protect is off Apr 15 21:15:01 sunny2 kernel: [ 1886.956365] sd 6:0:0:0: [sdb] Mode Sense: 77 00 00 08 Apr 15 21:15:01 sunny2 kernel: [ 1886.964101] sd 6:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA Apr 15 21:15:01 sunny2 kernel: [ 1886.976089] sd 6:0:0:0: [sdb] 1892352 512-byte hardware sectors (969 MB) Apr 15 21:15:01 sunny2 kernel: [ 1886.986197] sd 6:0:0:0: [sdb] Write Protect is off Apr 15 21:15:01 sunny2 kernel: [ 1886.986228] sd 6:0:0:0: [sdb] Mode Sense: 77 00 00 08 Apr 15 21:15:01 sunny2 kernel: [ 1887.82] sd 6:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA Apr 15 21:15:01 sunny2 kernel: [ 1887.002741] sdb: unknown partition table Apr 15 21:15:01 sunny2 kernel: [ 1887.042501] sd 6:0:0:0: [sdb] Attached SCSI disk Apr 15 21:15:01 sunny2 kernel: [ 1887.043773] sd 6:0:0:0: Attached scsi generic sg1 type 0 Apr 15 21:15:02 sunny2 iscsid: connection5:0 is operational now Apr 15 21:15:20 sunny2 iscsid: Could not get host for sid 5. Apr 15 21:15:20 sunny2 iscsid: could not get host_no for session 6. Apr 15 21:15:20 sunny2 iscsid: could not find session info for session5 Apr 15 21:15:20 sunny2 iscsid: session [iqn.2001-04.com.ubuntu:scsi.disk.vg1.vote,192.168.20.22,3260] already running. Apr 15 21:15:28 sunny2 iscsid: Nop-out timedout after 15 seconds on connection 5:0 state (3). Dropping session. Apr 15 21:15:28 sunny2 iscsid: connection5:0 is operational after recovery (1 attempts) Apr 15 21:15:33 sunny2 iscsid: Could not get host for sid 5. Apr 15 21:15:33 sunny2 iscsid: could not get host_no for session 6. Apr 15 21:15:33 sunny2 iscsid: could not find session info for session5 Apr 15 21:15:33 sunny2 iscsid: session [iqn.2001-04.com.ubuntu:scsi.disk.vg1.asm,192.168.20.22,3260] already running. Apr 15 21:15:54 sunny2 iscsid: Nop-out timedout after 15 seconds on
Re: open-iscsi created only one disk but missing 2 more
Hi, Please find my answers below: Appreciate you help. What Target are you using? both the systems are ubuntu 8.10 I beat my head on a issue when I first setup regarding access permissions. If you don't have the permissions right on the array, open-iscsi will let you see it but not log in. I 'm logged in as root and have executed all the commands. When you say array, fyi it is just a secondary hard drive conencted to the system. Another thing you may want to check, if you are sharing luns between systems, some arrays require you to explicitly allow that behavior. Is there a command for that? how do i check if i need to explicitly allow sharing? And last but not least, if you have a backend and a front end connection make sure you are connecting to the array with the right interface. Make sure one connection does not have a matching MTU *jumbo frames with the array it can cause strange behavior. I'm sorry i 'm unable to understand this(if you could elaborate on this pls: newbie). But to make things simple, my scenario is as follows: Home pcs(900Mhz/512MB RAM/P3): 2 systems with ubuntu 8.10 with both open-iscsi installed. On System 1 : i attached a second hard drive of size 20G. I installed lvm2 and I created 1 physical volume of size 20G, 1 volume group vg1 and then created 3 logical volumes /dev/vg1/asm 17G, /dev/vg1/ocr 960M and /dev/vg1/vote 750M. On System 2: executed commands: iscsiadm -m discovery -t st -p 192.168.20.22 The output was: 192.168.20.22:3260,1 iqn.2001-04.com.ubuntu:scsi.disk.vg.asm 192.168.20.22:3260,1 iqn.2001-04.com.ubuntu:scsi.disk.vg.ocr 192.168.20.22:3260,1 iqn.2001-04.com.ubuntu:scsi.disk.vg.vote iscsiadm -m node -T iqn.2001-04.com.ubuntu:scsi.disk.vg.asm -p 192.168.20.22 -l Output: Login session [iface: default, target: iqn.2001-04.com.ubuntu:scsi.disk.vg.asm, portal: 192.168.20.22,3260] iscsiadm -m node -T iqn.2001-04.com.ubuntu:scsi.disk.vg.vote -p 192.168.20.22 -l Output: Login session [iface: default, target: iqn.2001-04.com.ubuntu:scsi.disk.vg.vote, portal: 192.168.20.22,3260] iscsiadm -m node -T iqn.2001-04.com.ubuntu:scsi.disk.vg.ocr -p 192.168.20.22 -l Output: Login session [iface: default, target: iqn.2001-04.com.ubuntu:scsi.disk.vg.ocr, portal: 192.168.20.22,3260] I must really thank you for having replied me. I was just feeling bad that no one replied me on this issue. Thank you so much. On Apr 15, 10:27 pm, sundar mahadevan sundarmahadeva...@gmail.com wrote: I tried the same connecting another hard drive of 20 G with 3 logical volumes namely: asm 17G , ocr 924M and vote 760M. iscsiadm -m node -T iqn.2001-04.com.ubuntu:scsi.disk.vg1.ocr -p 192.168.20.22 -l iscsiadm -m node -T iqn.2001-04.com.ubuntu:scsi.disk.vg1.vote -p 192.168.20.22 -l iscsiadm -m node -T iqn.2001-04.com.ubuntu:scsi.disk.vg1.asm -p 192.168.20.22 -l Looks like it only created the first one: ocr Here is the log from /var/log/syslog Apr 15 21:10:17 sunny2 kernel: [ 1603.409561] Loading iSCSI transport class v2.0-870. Apr 15 21:10:17 sunny2 kernel: [ 1603.486634] iscsi: registered transport (tcp) Apr 15 21:10:18 sunny2 kernel: [ 1603.848719] iscsi: registered transport (iser) Apr 15 21:10:18 sunny2 iscsid: iSCSI logger with pid=5912 started! Apr 15 21:10:18 sunny2 kernel: [ 1604.408284] scsi2 : iSCSI Initiator over TCP/IP Apr 15 21:10:19 sunny2 iscsid: transport class version 2.0-870. iscsid version 2.0-865 Apr 15 21:10:19 sunny2 iscsid: iSCSI daemon with pid=5914 started! Apr 15 21:10:19 sunny2 iscsid: Could not verify connection 1:2. Dropping event. Apr 15 21:10:19 sunny2 iscsid: Could not verify connection 1:2. Dropping event. Apr 15 21:10:20 sunny2 kernel: [ 1606.064265] scsi3 : iSCSI Initiator over TCP/IP Apr 15 21:10:21 sunny2 iscsid: Could not verify connection 2:3. Dropping event. Apr 15 21:10:21 sunny2 iscsid: Could not verify connection 2:3. Dropping event. Apr 15 21:10:21 sunny2 kernel: [ 1607.592257] scsi4 : iSCSI Initiator over TCP/IP Apr 15 21:10:22 sunny2 iscsid: Could not verify connection 3:4. Dropping event. Apr 15 21:10:22 sunny2 iscsid: Could not verify connection 3:4. Dropping event. Apr 15 21:10:23 sunny2 kernel: [ 1609.120249] scsi5 : iSCSI Initiator over TCP/IP Apr 15 21:10:24 sunny2 iscsid: Could not verify connection 4:5. Dropping event. Apr 15 21:10:24 sunny2 iscsid: Could not verify connection 4:5. Dropping event. Apr 15 21:15:00 sunny2 kernel: [ 1886.664257] scsi6 : iSCSI Initiator over TCP/IP Apr 15 21:15:01 sunny2 iscsid: Could not verify connection 5:6. Dropping event. Apr 15 21:15:01 sunny2 kernel: [ 1886.946886] scsi 6:0:0:0: Direct-Access IET VIRTUAL-DISK 0 PQ: 0 ANSI: 4 Apr 15 21:15:01 sunny2 kernel: [ 1886.952267] sd 6:0:0:0: [sdb] 1892352 512-byte hardware sectors (969 MB) Apr 15 21:15:01 sunny2 kernel: [ 1886.956338] sd 6:0:0:0: [sdb] Write Protect is off Apr 15 21:15:01 sunny2 kernel: [ 1886.956365] sd 6:0:0:0: [sdb] Mode Sense: 77 00 00 08 Apr 15
Re: open-iscsi created only one disk but missing 2 more
sundar mahadevan wrote: I tried the same connecting another hard drive of 20 G with 3 logical volumes namely: asm 17G , ocr 924M and vote 760M. iscsiadm -m node -T iqn.2001-04.com.ubuntu:scsi.disk.vg1.ocr -p 192.168.20.22 -l iscsiadm -m node -T iqn.2001-04.com.ubuntu:scsi.disk.vg1.vote -p 192.168.20.22 -l iscsiadm -m node -T iqn.2001-04.com.ubuntu:scsi.disk.vg1.asm -p 192.168.20.22 -l Looks like it only created the first one: ocr Here is the log from /var/log/syslog Apr 15 21:10:17 sunny2 kernel: [ 1603.409561] Loading iSCSI transport class v2.0-870. Apr 15 21:10:17 sunny2 kernel: [ 1603.486634] iscsi: registered transport (tcp) Apr 15 21:10:18 sunny2 kernel: [ 1603.848719] iscsi: registered transport (iser) Apr 15 21:10:18 sunny2 iscsid: iSCSI logger with pid=5912 started! Apr 15 21:10:18 sunny2 kernel: [ 1604.408284] scsi2 : iSCSI Initiator over TCP/IP Apr 15 21:10:19 sunny2 iscsid: transport class version 2.0-870. iscsid version 2.0-865 Apr 15 21:10:19 sunny2 iscsid: iSCSI daemon with pid=5914 started! Apr 15 21:10:19 sunny2 iscsid: Could not verify connection 1:2. Dropping event. Apr 15 21:10:19 sunny2 iscsid: Could not verify connection 1:2. Dropping event. Are you building the open-iscsi tools or are they part of the distro you are using? Are you also using qla4xxx or just using iscsi_tcp? If you are building your own tools make sure if you are using a 64 bit kernel then the tools are also compiled as 64 bits. Make sure that you only have one set of tools installed. Do a whereis iscsid and whereis iscsiadm. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Q: iSCSI Session State: Unknown
Ulrich Windl wrote: Hi! I'm wondering: For iscsiadm -m session -P 3 (SLES10 SP2) I see output like this: iSCSI Connection State: LOGGED IN iSCSI Session State: Unknown Internal iscsid Session State: NO CHANGE What is a Session State of Unknown? I.e. hwat are the valid session states? Older kernel. The kernel just does not export the session state in: /sys/class/iscsi_session/sessionX/state The valid state would be LOGGED_IN. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: iSCSI and FileSystem (ext2/ext3)
Bart Van Assche wrote: On Thu, Apr 16, 2009 at 1:10 PM, Ulrich Windl ulrich.wi...@rz.uni-regensburg.de wrote: Considering that most remote iSCSI targets have intelligece of their own, the use of a no-op i/O scheduler seems justified. I think multiple I/O schedulers on both ends don't necessarily make things faster, unless there is a bunch of requests that can be compacted into one. Then throughput rises while response may decline. Is request merging implemented in the SCSI layer or in the I/O scheduler ? I/O scheduler. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: open-iscsi created only one disk but missing 2 more
Thanks for your reply. But i might sound a little stupid to ask this: A very basic question: The hard disks i 'm using is ATA. I believe that open-iscsi works on scsi devices. Please enlighten me on this. On Thu, Apr 16, 2009 at 10:08 AM, Mike Christie micha...@cs.wisc.edu wrote: sundar mahadevan wrote: I tried the same connecting another hard drive of 20 G with 3 logical volumes namely: asm 17G , ocr 924M and vote 760M. iscsiadm -m node -T iqn.2001-04.com.ubuntu:scsi.disk.vg1.ocr -p 192.168.20.22 -l iscsiadm -m node -T iqn.2001-04.com.ubuntu:scsi.disk.vg1.vote -p 192.168.20.22 -l iscsiadm -m node -T iqn.2001-04.com.ubuntu:scsi.disk.vg1.asm -p 192.168.20.22 -l Looks like it only created the first one: ocr Here is the log from /var/log/syslog Apr 15 21:10:17 sunny2 kernel: [ 1603.409561] Loading iSCSI transport class v2.0-870. Apr 15 21:10:17 sunny2 kernel: [ 1603.486634] iscsi: registered transport (tcp) Apr 15 21:10:18 sunny2 kernel: [ 1603.848719] iscsi: registered transport (iser) Apr 15 21:10:18 sunny2 iscsid: iSCSI logger with pid=5912 started! Apr 15 21:10:18 sunny2 kernel: [ 1604.408284] scsi2 : iSCSI Initiator over TCP/IP Apr 15 21:10:19 sunny2 iscsid: transport class version 2.0-870. iscsid version 2.0-865 Apr 15 21:10:19 sunny2 iscsid: iSCSI daemon with pid=5914 started! Apr 15 21:10:19 sunny2 iscsid: Could not verify connection 1:2. Dropping event. Apr 15 21:10:19 sunny2 iscsid: Could not verify connection 1:2. Dropping event. Are you building the open-iscsi tools or are they part of the distro you are using? Are you also using qla4xxx or just using iscsi_tcp? If you are building your own tools make sure if you are using a 64 bit kernel then the tools are also compiled as 64 bits. Make sure that you only have one set of tools installed. Do a whereis iscsid and whereis iscsiadm. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Time of Log-In
Ulrich Windl wrote: Hello, while thinking about an udev/multipath timing problem with device discovery, I wondered how difficult it would be to record and the report time of session establihment (i.e. log-in). iscsiadm -m session -P 3 does not show that. Would that tinme be related to SID? No. I can add it though. I am still busy with work stuff trying to finish adding bnx2i and making sure cxgb3i is ok. When that stuff gets finished I will work on all your and eveyrone else's requests more. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: open-iscsi created only one disk but missing 2 more
Are you building the open-iscsi tools or are they part of the distro you are using? I just used apt-get install open-iscsi on both systems. Are you also using qla4xxx or just using iscsi_tcp? I dont think i use qla4xxx. I think i use iscsi_tcp. If you are building your own tools make sure if you are using a 64 bit kernel then the tools are also compiled as 64 bits. I'm on a 32 bit system. In that case what is a 64 bit of use to me. Make sure that you only have one set of tools installed. Do a whereis iscsid and whereis iscsiadm. whereis iscsid iscsid: /sbin/iscsid /usr/share/man/man8/iscsid.8.gz whereis iscsiadm iscsiadm: /sbin/iscsiadm /usr/share/man/man8/iscsiadm.8.gz You are using IET right? If so it does not matter what disks you use. IET can handle it. No i dont use IET. I use open-iscsi on both systems. I tried the same connecting another hard drive of 20 G with 3 logical volumes namely: asm 17G , ocr 924M and vote 760M. iscsiadm -m node -T iqn.2001-04.com.ubuntu:scsi.disk.vg1.ocr -p 192.168.20.22 -l iscsiadm -m node -T iqn.2001-04.com.ubuntu:scsi.disk.vg1.vote -p 192.168.20.22 -l iscsiadm -m node -T iqn.2001-04.com.ubuntu:scsi.disk.vg1.asm -p 192.168.20.22 -l Looks like it only created the first one: ocr Here is the log from /var/log/syslog Apr 15 21:10:17 sunny2 kernel: [ 1603.409561] Loading iSCSI transport class v2.0-870. Apr 15 21:10:17 sunny2 kernel: [ 1603.486634] iscsi: registered transport (tcp) Apr 15 21:10:18 sunny2 kernel: [ 1603.848719] iscsi: registered transport (iser) Apr 15 21:10:18 sunny2 iscsid: iSCSI logger with pid=5912 started! Apr 15 21:10:18 sunny2 kernel: [ 1604.408284] scsi2 : iSCSI Initiator over TCP/IP Apr 15 21:10:19 sunny2 iscsid: transport class version 2.0-870. iscsid version 2.0-865 Apr 15 21:10:19 sunny2 iscsid: iSCSI daemon with pid=5914 started! Apr 15 21:10:19 sunny2 iscsid: Could not verify connection 1:2. Dropping event. Apr 15 21:10:19 sunny2 iscsid: Could not verify connection 1:2. Dropping event. Are you building the open-iscsi tools or are they part of the distro you are using? Are you also using qla4xxx or just using iscsi_tcp? If you are building your own tools make sure if you are using a 64 bit kernel then the tools are also compiled as 64 bits. Make sure that you only have one set of tools installed. Do a whereis iscsid and whereis iscsiadm. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: open-iscsi created only one disk but missing 2 more
sundar mahadevan wrote: Are you building the open-iscsi tools or are they part of the distro you are using? I just used apt-get install open-iscsi on both systems. What kernel are you using (uname -a)? Are you also using qla4xxx or just using iscsi_tcp? I dont think i use qla4xxx. I think i use iscsi_tcp. If you are building your own tools make sure if you are using a 64 bit kernel then the tools are also compiled as 64 bits. I'm on a 32 bit system. In that case what is a 64 bit of use to me. Make sure that you only have one set of tools installed. Do a whereis iscsid and whereis iscsiadm. whereis iscsid iscsid: /sbin/iscsid /usr/share/man/man8/iscsid.8.gz whereis iscsiadm iscsiadm: /sbin/iscsiadm /usr/share/man/man8/iscsiadm.8.gz You are using IET right? If so it does not matter what disks you use. IET can handle it. No i dont use IET. I use open-iscsi on both systems. IET is your target. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: open-iscsi created only one disk but missing 2 more
kernel: 2.6.27-11 IET is your target. To my understanding IET is iscsi enterprise target which is similar to open-iscsi in implementing iscsi targets. open-iscsi and IET are different organisations implementing iscsi. Am i right? On Thu, Apr 16, 2009 at 10:46 AM, Mike Christie micha...@cs.wisc.edu wrote: sundar mahadevan wrote: Are you building the open-iscsi tools or are they part of the distro you are using? I just used apt-get install open-iscsi on both systems. What kernel are you using (uname -a)? Are you also using qla4xxx or just using iscsi_tcp? I dont think i use qla4xxx. I think i use iscsi_tcp. If you are building your own tools make sure if you are using a 64 bit kernel then the tools are also compiled as 64 bits. I'm on a 32 bit system. In that case what is a 64 bit of use to me. Make sure that you only have one set of tools installed. Do a whereis iscsid and whereis iscsiadm. whereis iscsid iscsid: /sbin/iscsid /usr/share/man/man8/iscsid.8.gz whereis iscsiadm iscsiadm: /sbin/iscsiadm /usr/share/man/man8/iscsiadm.8.gz You are using IET right? If so it does not matter what disks you use. IET can handle it. No i dont use IET. I use open-iscsi on both systems. IET is your target. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: iSCSI and FileSystem (ext2/ext3)
On Thu, Apr 16, 2009 at 2:07 PM, Pasi Kärkkäinen pa...@iki.fi wrote: Iirc it has been with RHEL5/CentOS5 2.6.18 based kernels.. Mike Christie has been writing about this aswell.. dunno about what kernels he has seen it with. Then again CFQ was designed for single disk workstations.. I have seen the above statement about CFQ only once in the past, and that was in this post: http://www.mail-archive.com/cen...@centos.org/msg04648.html. But if CFQ really was designed for single disk workstations, I do not understand why it has been chosen as the default in Red Hat Enterprise Linux 4. There must have been a good reason behind that choice. Bart. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: open-iscsi created only one disk but missing 2 more
Could someone enlighten me on this question please: Do i have to install iscsitarget on system 1 and access it with open-iscsi(iscsi initiator) from system 2 or can i have open-iscsi installed on both system 1 and system 2 and get it working? On Thu, Apr 16, 2009 at 10:51 AM, sundar mahadevan sundarmahadeva...@gmail.com wrote: kernel: 2.6.27-11 IET is your target. To my understanding IET is iscsi enterprise target which is similar to open-iscsi in implementing iscsi targets. open-iscsi and IET are different organisations implementing iscsi. Am i right? On Thu, Apr 16, 2009 at 10:46 AM, Mike Christie micha...@cs.wisc.edu wrote: sundar mahadevan wrote: Are you building the open-iscsi tools or are they part of the distro you are using? I just used apt-get install open-iscsi on both systems. What kernel are you using (uname -a)? Are you also using qla4xxx or just using iscsi_tcp? I dont think i use qla4xxx. I think i use iscsi_tcp. If you are building your own tools make sure if you are using a 64 bit kernel then the tools are also compiled as 64 bits. I'm on a 32 bit system. In that case what is a 64 bit of use to me. Make sure that you only have one set of tools installed. Do a whereis iscsid and whereis iscsiadm. whereis iscsid iscsid: /sbin/iscsid /usr/share/man/man8/iscsid.8.gz whereis iscsiadm iscsiadm: /sbin/iscsiadm /usr/share/man/man8/iscsiadm.8.gz You are using IET right? If so it does not matter what disks you use. IET can handle it. No i dont use IET. I use open-iscsi on both systems. IET is your target. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: [PATCH] bind offloaded connection to port
On Tue, 2009-04-14 at 19:19 -0700, Mike Christie wrote: Hey offload guys, If we are using a offload card, then iface_set_param will match the iface info to a scsi_host and pass that info down to setup the net settings of the port (currently we just set the ip address). When we create the tcp/ip connection by calling ep_connect, we currently just go by the routing table info. I think there are two problems with this. 1. Some drivers do not have access to a routing table. Some drivers like qla4xxx do not even know about other ports. 2. If you have two initiator ports on the same subnet, the user may have set things up so that session1 was supposed to be run through port1. and session2 was supposed to be run through port2. It looks like we could end with both sessions going through one of the ports. Also how do you edit the routing table for the offload cards? You cannot use normal net tools like route can you? 3. If we set up hostA in the iface_set_param step, but then the routing info leads us to hostB, we are stuck. I did the attached patches to fix this. Basically we just pass down the scsi host we want to go through. Well, ok I began to fix this :) For qla4xxx or serverengines I think this will work fine. For bnx2i and cxgb3i, I am not sure. See the TODO and note in cxgb3i in kern-ep-connect-through-host.patch. bnx2i guys, you guys do somehting similar so will this work? In ep_connect can I control which host/port to use? this will work for bnx2i The patches were made against my iscsi tress. The kernel one was made over the iscsi brandh and that was just updated so you might want to reclone. The userspace one was made over the open-iscsi git tree head. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Multipath + iscsi + SLES10 SP2 / REDHAT 5.3 / Oracle Linux 5 update 3
FINAL RESULTS *
First of all I'd thank Mike Christie for all his help. Mike I'll
tapping your brain again for some read performance help.
This for the benefit of anyone using the Dell Equallogic PS5000XV
PS5000E with SLES10 SP2 / Redhat 5.3 / Centos 5.3 / Oracle Linux +
Multipath ( MPIO ) and open-iscsi ( iscsi ). Sorry about weird
formatting, making sure this is going get hit for people that were in
my predicament.
As from this thread my issue was amazingly slow performance with
sequential writes with my multipath, around 35 meg/s, configuration
when measured with IOMETER. First things first... THROW OUT IOMETER
FOR LINUX , it has problems with queue depth. With that said, with
default iscsi and multipath setup we saw between 60-80meg/sec
performance with multipath. In essence it was slower than single
interface in certain block sizes. When I got done my write performance
was pushing 180-190meg/sec with blocks as small as 4k ( sequential
write test using dt).
Here are my tweaks:
After making any multipath changes do multipath -F then multipath
otherwise your changes won't take effect.
/etc/multipath.conf
device {
vendor EQLOGIC
product 100E-00
path_grouping_policy multibus
getuid_callout /sbin/scsi_id -g -u -s /block/%n
features 1 queue_if_no_path--- important
path_checker readsector0
failback immediate
path_selector round-robin 0
rr_min_io 512 important, only works with large queue
depth and cms in iscsi.conf
rr_weight priorities
}
/etc/iscsi/iscsi.conf ( restarting iscsi seems to apply the configs
fine)
# To control how many commands the session will queue set
# node.session.cmds_max to an integer between 2 and 2048 that is also
# a power of 2. The default is 128.
node.session.cmds_max = 1024
# To control the device's queue depth set node.session.queue_depth
# to a value between 1 and 128. The default is 32.
node.session.queue_depth = 128
Other changes I've made are basic gigabit network tuning for large
transfers and turning off some congestion functions, some scheduler
changes (noop is amazing for sub 4k blocks but awful for 4meg chunks
or higher). I've turned off TSO on the network cards, apparently it's
not supported with jumbo frames and actually slows down performance.
dc1stgdb14:~ # ethtool -k eth7
Offload parameters for eth7:
rx-checksumming: off
tx-checksumming: off
scatter-gather: off
tcp segmentation offload: off
dc1stgdb14:~ # ethtool -k eth10
Offload parameters for eth10:
rx-checksumming: off
tx-checksumming: off
scatter-gather: off
tcp segmentation offload: off
dc1stgdb14:~ #
On Apr 13, 4:36 pm, jnantel nan...@hotmail.com wrote:
I am having a major issue with multipath + iscsi writeperformance
with anything random or any sequential write with data sizes smaller
than 4meg (128k 64k 32k 16k 8k). With 32k block size, I am able to
get a maximum throughput of 33meg/s write. Myperformancegets cut by
a third with each smaller size, with 4k blocks giving me a whopping
4meg/s combined throughput. Now bumping the data size up to 32meg
gets me 160meg/sec throughput, and 64 gives me 190meg/s and finally to
top it out 128meg gives me 210megabytes/sec. My question is what
factors would limit myperformancein the 4-128k range?
Some basics about myperformancelab:
2 identical 1 gigabit paths (2 dual port intel pro 1000 MTs) in
separate pcie slots.
Hardware:
2 x Dell R900 6 quad core, 128gig ram, 2 x Dual port Intel Pro MT
Cisco 3750s with 32gigabit stackwise interconnect
2 x Dell Equallogic PS5000XV arrays
1 x Dell Equallogic PS5000E arrays
Operating systems
SLES 10 SP2 , RHEL5 Update 3, Oracle Linux 5 update 3
/etc/mutipath.conf
defaults {
udev_dir /dev
polling_interval 10
selector round-robin 0
path_grouping_policy multibus
getuid_callout /sbin/scsi_id -g -u -s /block/%n
prio_callout /bin/true
path_checker readsector0
features 1 queue_if_no_path
rr_min_io 10
max_fds 8192
# rr_weight priorities
failback immediate
# no_path_retry fail
# user_friendly_names yes
/etc/iscsi/iscsi.conf (non default values)
node.session.timeo.replacement_timeout = 15
node.conn[0].timeo.noop_out_interval = 5
node.conn[0].timeo.noop_out_timeout = 30
node.session.cmds_max = 128
node.session.queue_depth = 32
node.session.iscsi.FirstBurstLength = 262144
node.session.iscsi.MaxBurstLength = 16776192
node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144
node.conn[0].iscsi.MaxXmitDataSegmentLength = 262144
discovery.sendtargets.iscsi.MaxRecvDataSegmentLength = 65536
Scheduler:
cat /sys/block/sdb/queue/scheduler
[noop] anticipatory deadline cfq
cat /sys/block/sdc/queue/scheduler
[noop] anticipatory deadline
