RE: Number of devices that SCSI can support
The capi bug is fixed in RHEL 4.6. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Ellison, Bob Sent: Thursday, January 10, 2008 9:19 AM To: Vinay Venkataraghavan; Andrew Vasquez Cc: James Bottomley; linux-scsi@vger.kernel.org; Matthew Wilcox Subject: RE: Number of devices that SCSI can support I ran into this under RHEL4 and it turned out to be udev and the capi20 declaration. I commented the declaration out in 50-udev.rules (as well as the capi/%n) and started seeing my sd node appear (i.e. device 68:0). -- Bob -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Vinay Venkataraghavan Sent: Wednesday, January 09, 2008 5:31 PM To: Andrew Vasquez Cc: James Bottomley; linux-scsi@vger.kernel.org; Matthew Wilcox Subject: Re: Number of devices that SCSI can support Thank you all for responding. While I agree that the scsi stack in linux should be capable of supporting many targets, I still see this problem all the time. It seemed to consistently happen on /dev/sdbm which corresponds to the 65th lun or target. My SAN configuration consists of: 4 targets with 16 luns each = 4 * 16 = 64 total devices. But I ran another test by changing my targets information and saw this issue for /dev/sdbm As mentioned on this thread, I am attaching some more information. Looks like SCSI inquiry is going through, and the inquiry succeeds. As you can see from the attached message below the /sys/class/scsi_device file also gets created: at /sys/class/scsi_device/1\:0\:3\:15/device/ and below this we find all the related entries such as: block/ dumpqueue_depth state delete generic/rescan timeout detach_statemodel rev type device_blocked power/ scsi_level vendor So from this information it looks like the scsi mid layer had detected that particular lun. But the /dev/sdbm device has not been created. It was also mentioned that it could be a udev problem. The other curious issue is that if take the major number and the minor number from this entry and manually create the device file as such: mknod /dev/sdbm 68 0 it works after I rescan all the targets and luns and I am able to perform I/O on this lun. Any other ideas. Thanks, Vinay - Original Message From: Andrew Vasquez <[EMAIL PROTECTED]> To: Vinay Venkataraghavan <[EMAIL PROTECTED]> Cc: James Bottomley <[EMAIL PROTECTED]>; linux-scsi@vger.kernel.org; Matthew Wilcox <[EMAIL PROTECTED]> Sent: Wednesday, January 9, 2008 7:49:33 AM Subject: Re: Number of devices that SCSI can support On Wed, 09 Jan 2008, Matthew Wilcox wrote: > On Wed, Jan 09, 2008 at 09:05:52AM -0600, James Bottomley wrote: > > On Tue, 2008-01-08 at 19:22 -0700, Matthew Wilcox wrote: > > > On Tue, Jan 08, 2008 at 04:55:46PM -0800, Vinay Venkataraghavan wrote: > > > > Is there a limit on the number of devices that SCSI supports. In other words, I have a QLogic HBA card, and I am connecting to a SAN which has 64 targets. > > > > > > I've personally had over five hundred LUNs. You shouldn't be hitting a > > > limit here. > > > > I believe the largest test that's been run was the old OSDL CGL > > workgroup ... they went up to 4096. > > > > However, LUN support depends on the driver and HBA parameters as well > > (some choose to have arbitrary limits). > > I was using a qlogic HBA for my tests, so I don't think this is the > problem -- although the original poster claims to have 64 targets, and I > had only one target with 128 luns (attached 4 times). > > -- > Intel are signing my paycheques ... these opinions are still mine > "Bill, look, we understand that you're interested in selling us this > operating system, but compare it to ours. We can't possibly take such > a retrograde step." Not sure what's going on as well, perhaps some logs could help... But the inbox qla2xxx driver in RHEL4 set's an HBA's scsi_host->max_id count to 512 (also verified with several test rings), so there shouldn't be a problem handling 64 distinct targets (FC ports). - To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Never miss a thing. Make Yahoo your home page. http://www.yahoo.com/r/hs - To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html - To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a mes
RE: Number of devices that SCSI can support
I ran into this under RHEL4 and it turned out to be udev and the capi20 declaration. I commented the declaration out in 50-udev.rules (as well as the capi/%n) and started seeing my sd node appear (i.e. device 68:0). -- Bob -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Vinay Venkataraghavan Sent: Wednesday, January 09, 2008 5:31 PM To: Andrew Vasquez Cc: James Bottomley; linux-scsi@vger.kernel.org; Matthew Wilcox Subject: Re: Number of devices that SCSI can support Thank you all for responding. While I agree that the scsi stack in linux should be capable of supporting many targets, I still see this problem all the time. It seemed to consistently happen on /dev/sdbm which corresponds to the 65th lun or target. My SAN configuration consists of: 4 targets with 16 luns each = 4 * 16 = 64 total devices. But I ran another test by changing my targets information and saw this issue for /dev/sdbm As mentioned on this thread, I am attaching some more information. Looks like SCSI inquiry is going through, and the inquiry succeeds. As you can see from the attached message below the /sys/class/scsi_device file also gets created: at /sys/class/scsi_device/1\:0\:3\:15/device/ and below this we find all the related entries such as: block/ dumpqueue_depth state delete generic/rescan timeout detach_statemodel rev type device_blocked power/ scsi_level vendor So from this information it looks like the scsi mid layer had detected that particular lun. But the /dev/sdbm device has not been created. It was also mentioned that it could be a udev problem. The other curious issue is that if take the major number and the minor number from this entry and manually create the device file as such: mknod /dev/sdbm 68 0 it works after I rescan all the targets and luns and I am able to perform I/O on this lun. Any other ideas. Thanks, Vinay - Original Message From: Andrew Vasquez <[EMAIL PROTECTED]> To: Vinay Venkataraghavan <[EMAIL PROTECTED]> Cc: James Bottomley <[EMAIL PROTECTED]>; linux-scsi@vger.kernel.org; Matthew Wilcox <[EMAIL PROTECTED]> Sent: Wednesday, January 9, 2008 7:49:33 AM Subject: Re: Number of devices that SCSI can support On Wed, 09 Jan 2008, Matthew Wilcox wrote: > On Wed, Jan 09, 2008 at 09:05:52AM -0600, James Bottomley wrote: > > On Tue, 2008-01-08 at 19:22 -0700, Matthew Wilcox wrote: > > > On Tue, Jan 08, 2008 at 04:55:46PM -0800, Vinay Venkataraghavan wrote: > > > > Is there a limit on the number of devices that SCSI supports. In other words, I have a QLogic HBA card, and I am connecting to a SAN which has 64 targets. > > > > > > I've personally had over five hundred LUNs. You shouldn't be hitting a > > > limit here. > > > > I believe the largest test that's been run was the old OSDL CGL > > workgroup ... they went up to 4096. > > > > However, LUN support depends on the driver and HBA parameters as well > > (some choose to have arbitrary limits). > > I was using a qlogic HBA for my tests, so I don't think this is the > problem -- although the original poster claims to have 64 targets, and I > had only one target with 128 luns (attached 4 times). > > -- > Intel are signing my paycheques ... these opinions are still mine > "Bill, look, we understand that you're interested in selling us this > operating system, but compare it to ours. We can't possibly take such > a retrograde step." Not sure what's going on as well, perhaps some logs could help... But the inbox qla2xxx driver in RHEL4 set's an HBA's scsi_host->max_id count to 512 (also verified with several test rings), so there shouldn't be a problem handling 64 distinct targets (FC ports). - To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Never miss a thing. Make Yahoo your home page. http://www.yahoo.com/r/hs - To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html - To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Number of devices that SCSI can support
Thank you all for responding. While I agree that the scsi stack in linux should be capable of supporting many targets, I still see this problem all the time. It seemed to consistently happen on /dev/sdbm which corresponds to the 65th lun or target. My SAN configuration consists of: 4 targets with 16 luns each = 4 * 16 = 64 total devices. But I ran another test by changing my targets information and saw this issue for /dev/sdbm As mentioned on this thread, I am attaching some more information. Looks like SCSI inquiry is going through, and the inquiry succeeds. As you can see from the attached message below the /sys/class/scsi_device file also gets created: at /sys/class/scsi_device/1\:0\:3\:15/device/ and below this we find all the related entries such as: block/ dumpqueue_depth state delete generic/rescan timeout detach_statemodel rev type device_blocked power/ scsi_level vendor So from this information it looks like the scsi mid layer had detected that particular lun. But the /dev/sdbm device has not been created. It was also mentioned that it could be a udev problem. The other curious issue is that if take the major number and the minor number from this entry and manually create the device file as such: mknod /dev/sdbm 68 0 it works after I rescan all the targets and luns and I am able to perform I/O on this lun. Any other ideas. Thanks, Vinay - Original Message From: Andrew Vasquez <[EMAIL PROTECTED]> To: Vinay Venkataraghavan <[EMAIL PROTECTED]> Cc: James Bottomley <[EMAIL PROTECTED]>; linux-scsi@vger.kernel.org; Matthew Wilcox <[EMAIL PROTECTED]> Sent: Wednesday, January 9, 2008 7:49:33 AM Subject: Re: Number of devices that SCSI can support On Wed, 09 Jan 2008, Matthew Wilcox wrote: > On Wed, Jan 09, 2008 at 09:05:52AM -0600, James Bottomley wrote: > > On Tue, 2008-01-08 at 19:22 -0700, Matthew Wilcox wrote: > > > On Tue, Jan 08, 2008 at 04:55:46PM -0800, Vinay Venkataraghavan wrote: > > > > Is there a limit on the number of devices that SCSI supports. In other words, I have a QLogic HBA card, and I am connecting to a SAN which has 64 targets. > > > > > > I've personally had over five hundred LUNs. You shouldn't be hitting a > > > limit here. > > > > I believe the largest test that's been run was the old OSDL CGL > > workgroup ... they went up to 4096. > > > > However, LUN support depends on the driver and HBA parameters as well > > (some choose to have arbitrary limits). > > I was using a qlogic HBA for my tests, so I don't think this is the > problem -- although the original poster claims to have 64 targets, and I > had only one target with 128 luns (attached 4 times). > > -- > Intel are signing my paycheques ... these opinions are still mine > "Bill, look, we understand that you're interested in selling us this > operating system, but compare it to ours. We can't possibly take such > a retrograde step." Not sure what's going on as well, perhaps some logs could help... But the inbox qla2xxx driver in RHEL4 set's an HBA's scsi_host->max_id count to 512 (also verified with several test rings), so there shouldn't be a problem handling 64 distinct targets (FC ports). - To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Never miss a thing. Make Yahoo your home page. http://www.yahoo.com/r/hs - To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Number of devices that SCSI can support
On Wed, 09 Jan 2008, Matthew Wilcox wrote: > On Wed, Jan 09, 2008 at 09:05:52AM -0600, James Bottomley wrote: > > On Tue, 2008-01-08 at 19:22 -0700, Matthew Wilcox wrote: > > > On Tue, Jan 08, 2008 at 04:55:46PM -0800, Vinay Venkataraghavan wrote: > > > > Is there a limit on the number of devices that SCSI supports. In other > > > > words, I have a QLogic HBA card, and I am connecting to a SAN which has > > > > 64 targets. > > > > > > I've personally had over five hundred LUNs. You shouldn't be hitting a > > > limit here. > > > > I believe the largest test that's been run was the old OSDL CGL > > workgroup ... they went up to 4096. > > > > However, LUN support depends on the driver and HBA parameters as well > > (some choose to have arbitrary limits). > > I was using a qlogic HBA for my tests, so I don't think this is the > problem -- although the original poster claims to have 64 targets, and I > had only one target with 128 luns (attached 4 times). > > -- > Intel are signing my paycheques ... these opinions are still mine > "Bill, look, we understand that you're interested in selling us this > operating system, but compare it to ours. We can't possibly take such > a retrograde step." Not sure what's going on as well, perhaps some logs could help... But the inbox qla2xxx driver in RHEL4 set's an HBA's scsi_host->max_id count to 512 (also verified with several test rings), so there shouldn't be a problem handling 64 distinct targets (FC ports). - To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Number of devices that SCSI can support
On Wed, Jan 09, 2008 at 09:05:52AM -0600, James Bottomley wrote: > On Tue, 2008-01-08 at 19:22 -0700, Matthew Wilcox wrote: > > On Tue, Jan 08, 2008 at 04:55:46PM -0800, Vinay Venkataraghavan wrote: > > > Is there a limit on the number of devices that SCSI supports. In other > > > words, I have a QLogic HBA card, and I am connecting to a SAN which has > > > 64 targets. > > > > I've personally had over five hundred LUNs. You shouldn't be hitting a > > limit here. > > I believe the largest test that's been run was the old OSDL CGL > workgroup ... they went up to 4096. > > However, LUN support depends on the driver and HBA parameters as well > (some choose to have arbitrary limits). I was using a qlogic HBA for my tests, so I don't think this is the problem -- although the original poster claims to have 64 targets, and I had only one target with 128 luns (attached 4 times). -- Intel are signing my paycheques ... these opinions are still mine "Bill, look, we understand that you're interested in selling us this operating system, but compare it to ours. We can't possibly take such a retrograde step." - To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Number of devices that SCSI can support
On Tue, 2008-01-08 at 19:22 -0700, Matthew Wilcox wrote: > On Tue, Jan 08, 2008 at 04:55:46PM -0800, Vinay Venkataraghavan wrote: > > Is there a limit on the number of devices that SCSI supports. In other > > words, I have a QLogic HBA card, and I am connecting to a SAN which has 64 > > targets. > > I've personally had over five hundred LUNs. You shouldn't be hitting a > limit here. I believe the largest test that's been run was the old OSDL CGL workgroup ... they went up to 4096. However, LUN support depends on the driver and HBA parameters as well (some choose to have arbitrary limits). So, firstly, if the inquiry strings appear (as in you see a scsiX:X:X:64 and above in dmesg) then I'd look at udev issues. If the inquiry strings don't appear, it's probably a device or driver programmed limit. James - To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: Number of devices that SCSI can support
You will want to check the BIOS settings on the Qlogic HBA, if you are running into a problem. Also, there may be limitations that are inherent to the storage to which you are connecting. Regards, Frank -Original Message- From: [EMAIL PROTECTED] on behalf of Matthew Wilcox Sent: Tue 1/8/2008 9:22 PM To: Vinay Venkataraghavan Cc: linux-scsi@vger.kernel.org Subject: Re: Number of devices that SCSI can support On Tue, Jan 08, 2008 at 04:55:46PM -0800, Vinay Venkataraghavan wrote: > Is there a limit on the number of devices that SCSI supports. In other words, > I have a QLogic HBA card, and I am connecting to a SAN which has 64 targets. I've personally had over five hundred LUNs. You shouldn't be hitting a limit here. > I am running 2.6.9-42. That sounds like a Red Hat Enterprise kernel. Perhaps you should contact them for support? -- Intel are signing my paycheques ... these opinions are still mine "Bill, look, we understand that you're interested in selling us this operating system, but compare it to ours. We can't possibly take such a retrograde step." - To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html - To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Number of devices that SCSI can support
On Tue, Jan 08, 2008 at 04:55:46PM -0800, Vinay Venkataraghavan wrote: > Is there a limit on the number of devices that SCSI supports. In other words, > I have a QLogic HBA card, and I am connecting to a SAN which has 64 targets. I've personally had over five hundred LUNs. You shouldn't be hitting a limit here. > I am running 2.6.9-42. That sounds like a Red Hat Enterprise kernel. Perhaps you should contact them for support? -- Intel are signing my paycheques ... these opinions are still mine "Bill, look, we understand that you're interested in selling us this operating system, but compare it to ours. We can't possibly take such a retrograde step." - To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Number of devices that SCSI can support
Hello everybody, Is there a limit on the number of devices that SCSI supports. In other words, I have a QLogic HBA card, and I am connecting to a SAN which has 64 targets. I also have one local SCSI disk. So the total number of targets that I need to see is 65. So now what happens is that all but ONE disc/target gets correctly mapped and a device file is created except for one. This happens to be /dev/sdbm which maps to /dev/sg64 Looks like the SCSI midlayer also sees the target but the only thing that does not seem to happen is that the device file /dev/sdbm is not getting created. I am attaching below the output: If you go all the way down, the last entry which is a valid disc gets mapped correctly but a /dev/sdbm device file does not get created. Any thoughts as to why this is happening. Is there a limitation on SCSI ability to handle only 64 discs? I would really appreciate any help. I am running 2.6.9-42. I might be wrong, but I think i tried this same configuration on RHEL5 and I think it works. I will double check though. Was there a patch that was applied to fix this issue. Thanks in advance. Vinay [EMAIL PROTECTED] ~]# sg_map -x /dev/sg0 0 0 0 0 0 /dev/sda /dev/sg1 2 0 0 0 0 /dev/sdb /dev/sg2 2 0 1 0 0 /dev/sdc /dev/sg3 2 0 2 0 0 /dev/sdd /dev/sg4 2 0 3 0 0 /dev/sde /dev/sg5 2 0 4 0 0 /dev/sdf /dev/sg6 2 0 5 0 0 /dev/sdg /dev/sg7 2 0 6 0 0 /dev/sdh /dev/sg8 2 0 7 0 0 /dev/sdi /dev/sg9 2 0 8 0 0 /dev/sdj /dev/sg10 2 0 9 0 0 /dev/sdk /dev/sg11 2 0 10 0 0 /dev/sdl /dev/sg12 2 0 11 0 0 /dev/sdm /dev/sg13 2 0 12 0 0 /dev/sdn /dev/sg14 2 0 13 0 0 /dev/sdo /dev/sg15 2 0 14 0 0 /dev/sdp /dev/sg16 2 0 15 0 0 /dev/sdq /dev/sg17 2 0 16 0 0 /dev/sdr /dev/sg18 2 0 17 0 0 /dev/sds /dev/sg19 2 0 18 0 0 /dev/sdt /dev/sg20 2 0 19 0 0 /dev/sdu /dev/sg21 2 0 20 0 0 /dev/sdv /dev/sg22 2 0 21 0 0 /dev/sdw /dev/sg23 2 0 22 0 0 /dev/sdx /dev/sg24 2 0 23 0 0 /dev/sdy /dev/sg25 2 0 24 0 0 /dev/sdz /dev/sg26 2 0 25 0 0 /dev/sdaa /dev/sg27 2 0 26 0 0 /dev/sdab /dev/sg28 2 0 27 0 0 /dev/sdac /dev/sg29 2 0 28 0 0 /dev/sdad /dev/sg30 2 0 29 0 0 /dev/sdae /dev/sg31 2 0 30 0 0 /dev/sdaf /dev/sg32 2 0 31 0 0 /dev/sdag /dev/sg33 2 0 32 0 0 /dev/sdah /dev/sg34 2 0 33 0 0 /dev/sdai /dev/sg35 2 0 34 0 0 /dev/sdaj /dev/sg36 2 0 35 0 0 /dev/sdak /dev/sg37 2 0 36 0 0 /dev/sdal /dev/sg38 2 0 37 0 0 /dev/sdam /dev/sg39 2 0 38 0 0 /dev/sdan /dev/sg40 2 0 39 0 0 /dev/sdao /dev/sg41 2 0 40 0 0 /dev/sdap /dev/sg42 2 0 41 0 0 /dev/sdaq /dev/sg43 2 0 42 0 0 /dev/sdar /dev/sg44 2 0 43 0 0 /dev/sdas /dev/sg45 2 0 44 0 0 /dev/sdat /dev/sg46 2 0 45 0 0 /dev/sdau /dev/sg47 2 0 46 0 0 /dev/sdav /dev/sg48 2 0 47 0 0 /dev/sdaw /dev/sg49 2 0 48 0 0 /dev/sdax /dev/sg50 2 0 49 0 0 /dev/sday /dev/sg51 2 0 50 0 0 /dev/sdaz /dev/sg52 2 0 51 0 0 /dev/sdba /dev/sg53 2 0 52 0 0 /dev/sdbb /dev/sg54 2 0 53 0 0 /dev/sdbc /dev/sg55 2 0 54 0 0 /dev/sdbd /dev/sg56 2 0 55 0 0 /dev/sdbe /dev/sg57 2 0 56 0 0 /dev/sdbf /dev/sg58 2 0 57 0 0 /dev/sdbg /dev/sg59 2 0 58 0 0 /dev/sdbh /dev/sg60 2 0 59 0 0 /dev/sdbi /dev/sg61 2 0 60 0 0 /dev/sdbj /dev/sg62 2 0 61 0 0 /dev/sdbk /dev/sg63 2 0 62 0 0 /dev/sdbl /dev/sg64 2 0 63 0 0 Never miss a thing. Make Yahoo your home page. http://www.yahoo.com/r/hs - To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
