Re: Multipath + iscsi + SLES10 SP2 / REDHAT 5.3 / Oracle Linux 5 update 3
On Mon, Apr 13, 2009 at 10:33 PM, Mike Christie micha...@cs.wisc.edu wrote: I think linux is just not so good with smaller IO sizes like 4K. I do not see good performance with Fibre Channel or iscsi. Most people run a filesystem on top of a block device imported via open-iscsi. It is well known that a filesystem performs I/O to the underlying block device using block sizes between 4 KB and 64 KB, with a significant fraction being 4 KB I/O's. If there was a performance problem in Linux with regard to small block sizes, filesystem performance in Linux would suffer. I have not yet seen statistics that show that Linux' filesystem performance is worse than for other operating systems. But I have already seen measurements that show the contrary. Bart. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Tuning iSCSI between Linux and NetAPP
Dmitry Yusupov wrote: Hi Frank, Hi Dimitri May I also suggest an alternative for iSCSI target? We had a very good experience with open-iscsi and NexentaStor [1]. At the beginning we had problems with single-threaded traffic latency but the ultimate fix was on Linux side to ensure that io scheduler is set to noop: # echo noop /sys/block/sd*/queue/scheduler Once set, we saturated physical networking connection with small block sizes in single-threaded scenario. OK I'm gonna try this NexentaStor also is very good candidate for CIFS workgroups/AD environments with the whole SMB stack implemented in the kernel, which boosts performance over the top. And as far as iSCSI target - I would recommend to use COMSTAR, which is ZFS integrated in Nexenta [2]. [1] http://www.nexenta.com/products [2] http://blogs.nexenta.org/blog/2009/03/03/nexenta-iscsi-with-comstarzfs-integration/ It might good products but I still have Netapp filers :-) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Tuning iSCSI between Linux and NetAPP
By default the filer has these iscsi (Ontap 7.3.1) values set iscsi.enable on iscsi.isns.rev 22 iscsi.max_connections_per_session use_system_default iscsi.max_error_recovery_level use_system_default jnantel wrote: I use a netapp filer...where are these values set? Host or Array? iscsi.iswt.max_ios_per_session 64 iscsi.max_connections_per_session 16 iscsi.max_ios_per_session64 On Apr 14, 6:40 am, benoit plessis plessis.ben...@gmail.com wrote: First i would ask why the hell ? The netapp filer is a very good CIFS/SMB share server. Using it as an iSCSI target -- which is not is primary function (netapp filer are more NAS than SAN) -- will only create limitations (unable to resize volume on the fly, unable to use wafl attributes to store windows security acl, ...) with no visible gain ... Also your server seem very overkill to me, i must hope it won't have to be just a samba=iscsi interface ... For iSCSI and netapp in general, first make sure that you have at least 10% of free space inside the volume, and 10% of free space inside the aggregate or else perf could suffer and more important you won't be able to launch the reallocate process (defrag). The following is the recommended netapp/iscsi optimisations, however open-iscsi doesn't support multiple connections per session now (iirc), so the best way to have parallel access is to use multipath iscsi.iswt.max_ios_per_session 64 iscsi.max_connections_per_session 16 iscsi.max_ios_per_session64 2009/4/14 Frank Bonnet f.bon...@esiee.fr Hello I'm setting up a Samba server that will use iSCSI to access some shares on a NetAPP filer ( FAS 2050 ) I would like to know if some of you has already build such configuration and if there are some tricks to optimize it. The Linux server is a HP Proliant quad CPU and runs Debian Lenny, it has 16 Gb of RAM. Thanks a lot. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: iSCSI and FileSystem (ext2/ext3)
On Wed, Apr 15, 2009 at 8:24 PM, benoit plessis plessis.ben...@gmail.com wrote: I wanted to share some infos about a discovery we made using mysql over iSCSI. We have a bunch of replicated mysql server, initially all using ext3, due to perfs problems we tried comparing persf in ext3 vs ext2, and we found the following: server using ext3 normal iops 100 normal bw 25/30Mbps peak iops 1000 peak bw 45/52Mbps server using ext2 normal iops 40 normal bw 4/5 Mbps peak iops 50 peak bw 7/8 Mbps All servers using the noop scheduler. The ext3 FS wasn't even using journalised datas, only the standard metadata configuration, but the impact on resource usage is quite impressive So the question is, what do you use as FS over iSCSI ? Why are you using the noop scheduler on the initiator instead of deadline or CFQ ? The performance difference you observed is probably caused by something else than the filesystem. When running bonnie++ on a local filesystem, xfs gives better performance than ext2, and ext2 gives better performance than ext3. Bart. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: iSCSI and FileSystem (ext2/ext3)
On Wed, Apr 15, 2009 at 08:42:33PM +0200, Bart Van Assche wrote: On Wed, Apr 15, 2009 at 8:24 PM, benoit plessis plessis.ben...@gmail.com wrote: I wanted to share some infos about a discovery we made using mysql over iSCSI. We have a bunch of replicated mysql server, initially all using ext3, due to perfs problems we tried comparing persf in ext3 vs ext2, and we found the following: server using ext3 normal iops 100 normal bw 25/30Mbps peak iops 1000 peak bw 45/52Mbps server using ext2 normal iops 40 normal bw 4/5 Mbps peak iops 50 peak bw 7/8 Mbps All servers using the noop scheduler. The ext3 FS wasn't even using journalised datas, only the standard metadata configuration, but the impact on resource usage is quite impressive So the question is, what do you use as FS over iSCSI ? Why are you using the noop scheduler on the initiator instead of deadline or CFQ ? noop is usually good for the initiator. cfq has a feature (or a bug?) that prevents achieving queue depths deeper than 1, and thus limits your bandwidth a lot when there are (or should be) many ios on the fly at the same time. -- Pasi --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: open-iscsi created only one disk but missing 2 more
Could someone please help me with this issue. On Wed, Apr 15, 2009 at 11:41 AM, sundar mahadevan sundarmahadeva...@gmail.com wrote: Hi, I'm a newbie trying to setup open-iscsi on ubuntu 8.10(kernel 2.6.27-11). I created 3 logical volumes(asm : 37G, ocr: 2G and vote 1G) under a volume group vg1 comprising of 3 physical volumes (totalling 40G) Here are my commands from client side: sudo iscsiadm -m discovery -t st -p 192.168.20.22 lists: 192.168.20.22:3260,1 iqn.2001-04.com.ezhome:scsi.disk.vg1.asm 192.168.20.22:3260,1 iqn.2001-04.com.ezhome:scsi.disk.vg1.ocr 192.168.20.22:3260,1 iqn.2001-04.com.ezhome:scsi.disk.vg1.vote sudo /etc/init.d/open-iscsi restart sudo iscsiadm -m node -T iqn.2001-04.com.ezhome:scsi.disk.vg1.asm -p 192.168.20.22 -o update -n node.conn[0].startup -v automatic sudo iscsiadm -m node -T iqn.2001-04.com.ezhome:scsi.disk.vg1.ocr -p 192.168.20.22 -o update -n node.conn[0].startup -v automatic sudo iscsiadm -m node -T iqn.2001-04.com.ezhome:scsi.disk.vg1.vote -p 192.168.20.22 -o update -n node.conn[0].startup -v automatic sudo /etc/init.d/open-iscsi restart fdisk -l and dmesg 1) list a new drive /dev/sdb of size 39.7 GB but there should be 2 more drives and the size of /dev/sdb is surprising because there are no drives of size 39.7G 2) What am i missing here for other 2 drives to be visible. 3) Changes made to /etc/ietd.conf and then /etc/init.d/open-iscsi restart does not get reflected. I have to reboot the system to notice the changes. Any ideas? 4) am i supposed to use /etc/init.d/iscsitarget stop/restart to stop and restart iscsitargets? Thanks in advance. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: open-iscsi created only one disk but missing 2 more
I tried the same connecting another hard drive of 20 G with 3 logical volumes namely: asm 17G , ocr 924M and vote 760M. iscsiadm -m node -T iqn.2001-04.com.ubuntu:scsi.disk.vg1.ocr -p 192.168.20.22 -l iscsiadm -m node -T iqn.2001-04.com.ubuntu:scsi.disk.vg1.vote -p 192.168.20.22 -l iscsiadm -m node -T iqn.2001-04.com.ubuntu:scsi.disk.vg1.asm -p 192.168.20.22 -l Looks like it only created the first one: ocr Here is the log from /var/log/syslog Apr 15 21:10:17 sunny2 kernel: [ 1603.409561] Loading iSCSI transport class v2.0-870. Apr 15 21:10:17 sunny2 kernel: [ 1603.486634] iscsi: registered transport (tcp) Apr 15 21:10:18 sunny2 kernel: [ 1603.848719] iscsi: registered transport (iser) Apr 15 21:10:18 sunny2 iscsid: iSCSI logger with pid=5912 started! Apr 15 21:10:18 sunny2 kernel: [ 1604.408284] scsi2 : iSCSI Initiator over TCP/IP Apr 15 21:10:19 sunny2 iscsid: transport class version 2.0-870. iscsid version 2.0-865 Apr 15 21:10:19 sunny2 iscsid: iSCSI daemon with pid=5914 started! Apr 15 21:10:19 sunny2 iscsid: Could not verify connection 1:2. Dropping event. Apr 15 21:10:19 sunny2 iscsid: Could not verify connection 1:2. Dropping event. Apr 15 21:10:20 sunny2 kernel: [ 1606.064265] scsi3 : iSCSI Initiator over TCP/IP Apr 15 21:10:21 sunny2 iscsid: Could not verify connection 2:3. Dropping event. Apr 15 21:10:21 sunny2 iscsid: Could not verify connection 2:3. Dropping event. Apr 15 21:10:21 sunny2 kernel: [ 1607.592257] scsi4 : iSCSI Initiator over TCP/IP Apr 15 21:10:22 sunny2 iscsid: Could not verify connection 3:4. Dropping event. Apr 15 21:10:22 sunny2 iscsid: Could not verify connection 3:4. Dropping event. Apr 15 21:10:23 sunny2 kernel: [ 1609.120249] scsi5 : iSCSI Initiator over TCP/IP Apr 15 21:10:24 sunny2 iscsid: Could not verify connection 4:5. Dropping event. Apr 15 21:10:24 sunny2 iscsid: Could not verify connection 4:5. Dropping event. Apr 15 21:15:00 sunny2 kernel: [ 1886.664257] scsi6 : iSCSI Initiator over TCP/IP Apr 15 21:15:01 sunny2 iscsid: Could not verify connection 5:6. Dropping event. Apr 15 21:15:01 sunny2 kernel: [ 1886.946886] scsi 6:0:0:0: Direct-Access IET VIRTUAL-DISK 0PQ: 0 ANSI: 4 Apr 15 21:15:01 sunny2 kernel: [ 1886.952267] sd 6:0:0:0: [sdb] 1892352 512-byte hardware sectors (969 MB) Apr 15 21:15:01 sunny2 kernel: [ 1886.956338] sd 6:0:0:0: [sdb] Write Protect is off Apr 15 21:15:01 sunny2 kernel: [ 1886.956365] sd 6:0:0:0: [sdb] Mode Sense: 77 00 00 08 Apr 15 21:15:01 sunny2 kernel: [ 1886.964101] sd 6:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA Apr 15 21:15:01 sunny2 kernel: [ 1886.976089] sd 6:0:0:0: [sdb] 1892352 512-byte hardware sectors (969 MB) Apr 15 21:15:01 sunny2 kernel: [ 1886.986197] sd 6:0:0:0: [sdb] Write Protect is off Apr 15 21:15:01 sunny2 kernel: [ 1886.986228] sd 6:0:0:0: [sdb] Mode Sense: 77 00 00 08 Apr 15 21:15:01 sunny2 kernel: [ 1887.82] sd 6:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA Apr 15 21:15:01 sunny2 kernel: [ 1887.002741] sdb: unknown partition table Apr 15 21:15:01 sunny2 kernel: [ 1887.042501] sd 6:0:0:0: [sdb] Attached SCSI disk Apr 15 21:15:01 sunny2 kernel: [ 1887.043773] sd 6:0:0:0: Attached scsi generic sg1 type 0 Apr 15 21:15:02 sunny2 iscsid: connection5:0 is operational now Apr 15 21:15:20 sunny2 iscsid: Could not get host for sid 5. Apr 15 21:15:20 sunny2 iscsid: could not get host_no for session 6. Apr 15 21:15:20 sunny2 iscsid: could not find session info for session5 Apr 15 21:15:20 sunny2 iscsid: session [iqn.2001-04.com.ubuntu:scsi.disk.vg1.vote,192.168.20.22,3260] already running. Apr 15 21:15:28 sunny2 iscsid: Nop-out timedout after 15 seconds on connection 5:0 state (3). Dropping session. Apr 15 21:15:28 sunny2 iscsid: connection5:0 is operational after recovery (1 attempts) Apr 15 21:15:33 sunny2 iscsid: Could not get host for sid 5. Apr 15 21:15:33 sunny2 iscsid: could not get host_no for session 6. Apr 15 21:15:33 sunny2 iscsid: could not find session info for session5 Apr 15 21:15:33 sunny2 iscsid: session [iqn.2001-04.com.ubuntu:scsi.disk.vg1.asm,192.168.20.22,3260] already running. Apr 15 21:15:54 sunny2 iscsid: Nop-out timedout after 15 seconds on connection 5:0 state (3). Dropping session. Apr 15 21:15:54 sunny2 iscsid: connection5:0 is operational after recovery (1 attempts) Could someone help me please. I have been battling to figure this out for a while. It really is frustrating that nothing works. Thanks in advance. On Wed, Apr 15, 2009 at 3:58 PM, sundar mahadevan sundarmahadeva...@gmail.com wrote: Could someone please help me with this issue. On Wed, Apr 15, 2009 at 11:41 AM, sundar mahadevan sundarmahadeva...@gmail.com wrote: Hi, I'm a newbie trying to setup open-iscsi on ubuntu 8.10(kernel 2.6.27-11). I created 3 logical volumes(asm : 37G, ocr: 2G and vote 1G) under a volume group vg1 comprising of 3 physical volumes (totalling 40G) Here are my commands from client side: sudo
