Hi Joseph, we also have Dell equallogics in our shop.  I can tell you
A LOT about them and how to setup with multipath/iscsi..etc.  Mike
Christie is an incredible resource for sure, and he helped me a great
deal with my setup.  Be thankful you aren't using SLES10 SP2.

Performance, you should be able to hit 200-210 meg/s write performance
with RAID10 and around 150-160 with RAID6. Read performance for both
usually pegged at 215-220meg/sec sustained. This is of course with
sequential data.

couple of goofy things I did notice in multipath.conf:

1. features "1 queue_if_no_path"   and no_path_retry conflict
2. rr_min_io has the largest effect on performance in multipath.  Play
with different options (8/16/32/64) depending on your data sizes.  You
can reconfigure on the fly using "multipathd -k"  and the
"reconfigure" command. I am currently using 16 as I have lots of small
data chunks and I want lots of io in flight.

node.session.cmds_max = 1024
node.session.queue_depth = 64 (96 works good here as well if you have
small data sizes)

node.conn[0].timeo.noop_out_interval = 15
node.conn[0].timeo.noop_out_timeout = 45

The above are my settings, Mike's probably work better.

Couple of performance inhibiters I found...Make sure you have flow
control set on the switch.  TSO/SG/TX and RX offloading work by
assembly packets into 64k chunks prior to sending...This is death to

If you have a beefy CPU turn them off like this:

ethtool -K eth4 rx off
ethtool -K eth4 tso off
ethtool -K eth4 tx off
ethtool -K eth4 sg off

Tune your TCP stack for gigabit performance for sure.

On Jul 22, 12:01 pm, "Hoot, Joseph" <joe.h...@itec.suny.edu> wrote:
> Mike or others,
> Do you or others think that my "ping/CHAP" issue that I'm having -- aka
> sessions resetting (logging out) could be related to the more recent
> "ping timeout" and maxcmds patch that has been worked on and discussed?
> I am attempting to get my test systems updated to test this, but I
> figured that I'd toss it out there to see if they might be related.
> Thanks,
> Joe
> -----Original Message-----
> From: open-iscsi@googlegroups.com [mailto:open-is...@googlegroups.com]
> On Behalf Of Mike Christie
> Sent: Tuesday, July 14, 2009 6:01 PM
> To: open-iscsi@googlegroups.com
> Subject: Re: iscsiadm -m iface + routing
> On 07/13/2009 09:20 AM, Hoot, Joseph wrote:
> > Mike,
> > Just as an FYI (in case you were most curious about this issue) I've
> > narrowed this issue down to something with CHAP.  On my EqualLogic, if
> I
> > disable CHAP, I can't reproduce this issue.
> > So I did the following.  I after upgrading to the latest OEL 5.3
> release
> > of the iscsi-initiator, I could still reproduce the problem.
> Therefore,
> > I did the following:
> > 1) Setup another test environment using the same hardware (physical
> > different hardware, but all same firmware, models, etc..)
> > 2) presented a new test volume from EqualLogic
> > 3) ran the ping test (ping -Ieth2  ping -Ieth3
> >
> > 4) I couldn't reproduce the issue.
> > 5) I checked what the difference were-- CHAP the only difference.
> > 6) So I turned on CHAP authentication to the volume.
> > 7) rm -rf /var/lib/iscsi/nodes/* /var/lib/iscsi/send_targets/*
> > 8) rediscovered targets (after modifying /etc/iscsi/iscsid.conf with
> > CHAP information)
> > node.session.auth.authmethod = CHAP
> > node.session.auth.username = mychapuserhere
> > node.session.auth.password = mypasshere
> > 9) ran the same ping test and was able to get iscsi sessions to fail
> > within 2 minutes.
> > 10) I wanted to prove that CHAP was the issue. So I logout out of all
> > iscsi sessions.
> > 11) I disabled CHAP on the EqualLogic
> > 12) rediscovery targets and re-logged in to the sessions (without CHAP
> > authentication)
> > 13) ran the ping tests and couldn't break it after 30 minutes.
> > 14) added CHAP again and was able to break the sessions within 2
> > minutes.
> > So definitely something odd with CHAP (my guess, either in open-iscsi
> > code or EqualLogic code).  I've asked Roger Lopez, from Dell, to
> attempt
> > to reproduce this in his lab.  He has EqualLogic and Oracle VM
> Servers.
> > Oracle developers that I'm working with don't currently have access to
> > an EqualLogic, but they are attempting to reproduce this with their
> > iSCSI equipment as well.  I'm going to setup port mirroring on our
> > switch and run tcpdumps to see what I can get.
> This is strange because CHAP does not come into play in the normal IO
> path. When we login we will do authentication with CHAP, but after that
> succeeds it nevers comes up when doing IO. It would only come up again
> when the session/connection is dropped/disconnected and we relogin. For
> the relogin we will do the CHAP authentication again.
> Maybe some memory error where chap values are overwriting some other
> memory.
> There was one recent fix by Dell, where when using CHAP they could
> segfault iscsid. Here is the updated RPM that I am working on for RHEL
> 5.4 that has the fix:
> http://people.redhat.com/mchristi/iscsi/rhel5.4/iscsi-initiator-utils/
You received this message because you are subscribed to the Google Groups 
"open-iscsi" group.
To post to this group, send email to open-iscsi@googlegroups.com
To unsubscribe from this group, send email to 
For more options, visit this group at http://groups.google.com/group/open-iscsi

Reply via email to