Port discovery in the latest linux distributions and NPIV make it really easy to reach the maximum number of open ports. In this example, I have 2 devices on the same channel with NPIV. 63 target ports are discovered for each device number. Linux keeps these target ports open, so 126 open ports are used up for this channel.
root@r16-4e11:chccwdev --online 0.0.52c0-0.0.52c1 Setting device 0.0.52c0 online Done Setting device 0.0.52c1 online Done root@r16-4e11:ls 0.0.52c0 0.0.52c1 0.0.52c0: 0x20140080e5183bf0 0x5005076304010739 0x500507630e86022c 0xc05076fff7003f11 0xc05076ffff002131 online 0x20250080e5183bf0 0x5005076304014739 0x50060e8005430e5e 0xc05076fff7003f21 0xc05076ffff002321 peer_d_id 0x50000972c00b4118 0x5005076304080634 0xc05076fff7003c01 0xc05076fff7003f31 0xc05076ffff002331 peer_wwnn 0x50000972c00b419c 0x5005076304100634 0xc05076fff7003c11 0xc05076fffc001221 availability peer_wwpn 0x5001738027920141 0x5005076304100739 0xc05076fff7003c21 0xc05076fffc001231 card_version port_remove 0x5001738027920151 0x5005076304104739 0xc05076fff7003c31 0xc05076fffc001301 cmb_enable port_rescan 0x5001738027920161 0x5005076304110739 0xc05076fff7003d01 0xc05076fffc001311 cutype power 0x5001738027920171 0x5005076304114739 0xc05076fff7003d11 0xc05076fffc001421 devtype status 0x5001738027920181 0x500507630b004007 0xc05076fff7003d21 0xc05076fffd003641 driver subsystem 0x5001738027920191 0x500507630b008007 0xc05076fff7003d31 0xc05076fffd003651 failed uevent 0x50050763004edd01 0x500507630b00c007 0xc05076fff7003e01 0xc05076fffd003681 hardware_version 0x50050763004edd02 0x500507630b0b4007 0xc05076fff7003e11 0xc05076fffd003691 host0 0x5005076304000634 0x500507630b0b8007 0xc05076fff7003e21 0xc05076fffd0036c1 in_recovery 0x5005076304000739 0x500507630b0bc007 0xc05076fff7003e31 0xc05076fffd0036d1 lic_version 0x5005076304004739 0x500507630e84022c 0xc05076fff7003f01 0xc05076ffff002121 modalias 0.0.52c1: 0x20140080e5183bf0 0x5005076304010739 0x500507630e86022c 0xc05076fff7003f11 0xc05076ffff002131 online 0x20250080e5183bf0 0x5005076304014739 0x50060e8005430e5e 0xc05076fff7003f21 0xc05076ffff002321 peer_d_id 0x50000972c00b4118 0x5005076304080634 0xc05076fff7003c01 0xc05076fff7003f31 0xc05076ffff002331 peer_wwnn 0x50000972c00b419c 0x5005076304100634 0xc05076fff7003c11 0xc05076fffc001221 availability peer_wwpn 0x5001738027920141 0x5005076304100739 0xc05076fff7003c21 0xc05076fffc001231 card_version port_remove 0x5001738027920151 0x5005076304104739 0xc05076fff7003c31 0xc05076fffc001301 cmb_enable port_rescan 0x5001738027920161 0x5005076304110739 0xc05076fff7003d01 0xc05076fffc001311 cutype power 0x5001738027920171 0x5005076304114739 0xc05076fff7003d11 0xc05076fffc001421 devtype status 0x5001738027920181 0x500507630b004007 0xc05076fff7003d21 0xc05076fffd003641 driver subsystem 0x5001738027920191 0x500507630b008007 0xc05076fff7003d31 0xc05076fffd003651 failed uevent 0x50050763004edd01 0x500507630b00c007 0xc05076fff7003e01 0xc05076fffd003681 hardware_version 0x50050763004edd02 0x500507630b0b4007 0xc05076fff7003e11 0xc05076fffd003691 host1 0x5005076304000634 0x500507630b0b8007 0xc05076fff7003e21 0xc05076fffd0036c1 in_recovery 0x5005076304000739 0x500507630b0bc007 0xc05076fff7003e31 0xc05076fffd0036d1 lic_version 0x5005076304004739 0x500507630e84022c 0xc05076fff7003f01 0xc05076ffff002121 modalias This was the situation that Lu was running into when he was using lots of device numbers. When he hit the limit, there was a bug in the firmware that we had fixed in G40938.007. Zoning and/or using fewer device numbers on each channel can help alleviate this. Regards, Ray Higgs System z FCP Firmware Development Bld. 706, B42 2455 South Road Poughkeepsie, NY 12601 (845) 435-8666, T/L 295-8666 [email protected] Linux on 390 Port <[email protected]> wrote on 10/09/2011 09:16:42 AM: > From: Rogério Soares <[email protected]> > To: [email protected] > Date: 10/09/2011 09:21 AM > Subject: Re: zfcp disk error > Sent by: Linux on 390 Port <[email protected]> > > Great numbers !!! > > Here, we limit to 64 sub-channels for each physical channel... each guest > use 2 sub-channels from different cards.. works well :) > > > > On Sat, Oct 8, 2011 at 10:05 PM, Lu GL Gao <[email protected]> wrote: > > > We had finished multipathing work in our environment. Our problem we > > perviously had is caused by using too much FCP sub-channels of every one > > physical channel.I summarize our problem resolution below: > > (1)The number of FCP sub-channels(for every FCP physical channel) used > > should <= 32. > > The total number of connections for every FCP physical channel should <= > > 128. (note: a connection is subchannel->hba_port->LUN) > > (2)After prolem fixed, we had update MCF for Ficon Express card from > > G40938.006 to G40938.007, based on Raymond's advice. > > (3)Sometimes, if you add too much zfcp devices through YAST, it will be > > occasionally happened that wwpn list of luns can not fetched by clicking > > "get LUN" button. > > I don't know why it happens. But we can bypass it by defining few zfcp a > > time. > > > > Finally, thanks everybody here to share your experience for this problem! > > > > ---------------------------------------------------------------------- > > For LINUX-390 subscribe / signoff / archive access instructions, > > send email to [email protected] with the message: INFO LINUX-390 or > > visit > > http://www.marist.edu/htbin/wlvindex?LINUX-390 > > ---------------------------------------------------------------------- > > For more information on Linux on System z, visit > > http://wiki.linuxvm.org/ > > > > ---------------------------------------------------------------------- > For LINUX-390 subscribe / signoff / archive access instructions, > send email to [email protected] with the message: INFO LINUX-390 or visit > http://www.marist.edu/htbin/wlvindex?LINUX-390 > ---------------------------------------------------------------------- > For more information on Linux on System z, visit > http://wiki.linuxvm.org/ > ---------------------------------------------------------------------- For LINUX-390 subscribe / signoff / archive access instructions, send email to [email protected] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 ---------------------------------------------------------------------- For more information on Linux on System z, visit http://wiki.linuxvm.org/
