Re: [PATCH 0/4] scsi: 64-bit LUN support
On 04/08/2013 05:37 PM, Tomas Henzl wrote: On 04/05/2013 05:24 PM, James Smart wrote: On 4/4/2013 6:17 AM, Hannes Reinecke wrote: On 03/31/2013 07:44 PM, Tomas Henzl wrote: What we can do is to decode the LUN and compare it to max_lun provided by the driver, I think that sg_luns is able to do that, so what is needed is just to follow the SAM. I have seen reports of problem on three different drivers connected to various external storage, all of them having the same basic reason - the driver sets a max_lun and then LUN comes encoded with a newer addressing method and something like this is shown 'kernel: scsi: host 2 channel 0 id 2 lun16643 has a LUN larger than allowed by the host adapter' Decoding the real LUN value would fix this problem, by decoding is only meant the use in scsi_report_lun_scan. The LUN would be stored exactly the same way as it is now. I know we can patch the certain drivers too, but when max_lun were what the name says - max LU number, it would fix my problem very easy. Errm. No. Decoding LUNs is _evil_. It has only a relevance on the target, and even then it might choose to ignore it. So we cannot try to out-guess the target here. OK, I can see the problems with decoding the LUN one of them is the need to again encode the LUN to address format + number. I'm not sure if the hw would work if another address mode were used. When we understand the LUN as a complex structure then it makes no sense to compare to max_lun as a number - http://lxr.linux.no/#linux+v3.8.6/drivers/scsi/scsi_scan.c#L1471 Oh, but it does. See below. The error you're reporting is that lpfc is setting max_luns to '255', which of course is less than 16643. Increasing max_luns on lpfc to '0x' will fix your problem; nothing to do with 64-bit LUNs ... I think I haven't mentioned lpfc, but it doesn't matter. Fixing this in individual drivers by increasing the max_lun is not easy, because the firmware could have some reasons for the max lun (some tables, ..., fact is I have no idea how this is implemented in the hw). If the fix for this were just to set max_lun to 0x in every driver it means that we could remove the max_lun and the test completely. A kernel option like 'ignore_max_lun' would help, but I somehow dislike it, what do you think? Well, I've thought about this, too. You are right in the sense that 'max_lun' actually has a double meaning. First it's being used as the upper limit for sequential scan, where is has a strictly sequential meaning. So any internal LUN structure doesn't come into play here are we're just 'counting'. Secondly it's being used as a simple validation for any LUN numbers reported via REPORT LUNS. Here it the max_lun value actually refers to the amount of _bits_ in a LUN number the HBA can transfer. Again, the internal LUN structure doesn't come into play here; this is purely a hardware limitation on the HBA. Whether a LUN is valid or not is none of our concern; if the target accepts the LUN is has to be valid. If it doesn't then we don't care whether the LUN structure is valid or not; there is no device to be had anyway. However, after consulting SAM it is true that a plain 'max_lun' is incorrect for any LUN value higher than 255. LUN values higher than 255 should be represented with the 'flat space addressing' model, ie bit 6 should be set. Sadly, the various SAM revisions differ on how LUNs lower than 255 should be treated; they might or might not have set the flat space addressing model. So yeah, I guess we should be handling the HBA restriction different from the max_lun setting. I see to cook up a patch. Cheers, Hannes -- Dr. Hannes Reinecke zSeries Storage h...@suse.de +49 911 74053 688 SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: J. Hawn, J. Guild, F. Imendörffer, HRB 16746 (AG Nürnberg) -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 0/4] scsi: 64-bit LUN support
-Original Message- From: Hannes Reinecke [mailto:h...@suse.de] Sent: Tuesday, 09 April, 2013 2:38 AM To: Tomas Henzl Cc: james.sm...@emulex.com; Chad Dupuis; linux-scsi@vger.kernel.org; James Bottomley; Jeremy Linton; Elliott, Robert (Server Storage); Bart Van Assche; Bud Brown Subject: Re: [PATCH 0/4] scsi: 64-bit LUN support On 04/08/2013 05:37 PM, Tomas Henzl wrote: On 04/05/2013 05:24 PM, James Smart wrote: On 4/4/2013 6:17 AM, Hannes Reinecke wrote: On 03/31/2013 07:44 PM, Tomas Henzl wrote: You are right in the sense that 'max_lun' actually has a double meaning. First it's being used as the upper limit for sequential scan, where is has a strictly sequential meaning. So any internal LUN structure doesn't come into play here are we're just 'counting'. Secondly it's being used as a simple validation for any LUN numbers reported via REPORT LUNS. Here it the max_lun value actually refers to the amount of _bits_ in a LUN number the HBA can transfer. Again, the internal LUN structure doesn't come into play here; this is purely a hardware limitation on the HBA. Whether a LUN is valid or not is none of our concern; if the target accepts the LUN is has to be valid. If it doesn't then we don't care whether the LUN structure is valid or not; there is no device to be had anyway. However, after consulting SAM it is true that a plain 'max_lun' is incorrect for any LUN value higher than 255. LUN values higher than 255 should be represented with the 'flat space addressing' model, ie bit 6 should be set. Sadly, the various SAM revisions differ on how LUNs lower than 255 should be treated; they might or might not have set the flat space addressing model. If you treat the LUN as an 8-byte single big-endian value, then every LUN using an address method other than the peripheral device addressing method (00b) is greater than 255; in fact, they're all greater than 4,611,686,018,427,387,904 (0x4000___). One LUN value is not represented by different addressing models. For example, these are four different LUNs, not multiple ways to express the same LUN 0: a) LUN 0x___ address method 00b target or lun = _b b) LUN 0x4000___ address method 01b flat space lun = 00___b c) LUN 0xD200___ address method 11b length 01b extended address method 0x2 extended flat space lun = 0x00_ d) LUN 0xE200___ address method 11b length 10b extended address method 0x2 long extended flat space lun = 0x00_ Sequential scanning should be an exception for very old devices that don't support REPORT LUNS; a small maximum scan value would be safer than scanning to max_luns, which could be huge. Three plausible lower scan limits: a) address method 00b with TARGET OR LUN set to 0..7: the number of LUNs supported by SCSI-1, which had a 3-bit LUN field in the CDBs themselves (long since obsolete) b) address method 00b with TARGET OR LUN set to 0..255: the number of LUNs supported by devices that only support address method 00b; the 8-bit TARGET OR LUN field portion of the first two bytes of the LUN. c) first two bytes set to 0..65535: the possible LUNs supported by devices with protocols with 2-byte LUN fields. I think scanning this many would take too long, though, and might trigger bugs in devices that don't decode the LUN correctly. -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/4] scsi: 64-bit LUN support
On 04/09/2013 04:27 PM, Elliott, Robert (Server Storage) wrote: -Original Message- From: Hannes Reinecke [mailto:h...@suse.de] Sent: Tuesday, 09 April, 2013 2:38 AM To: Tomas Henzl Cc: james.sm...@emulex.com; Chad Dupuis; linux-scsi@vger.kernel.org; James Bottomley; Jeremy Linton; Elliott, Robert (Server Storage); Bart Van Assche; Bud Brown Subject: Re: [PATCH 0/4] scsi: 64-bit LUN support On 04/08/2013 05:37 PM, Tomas Henzl wrote: On 04/05/2013 05:24 PM, James Smart wrote: On 4/4/2013 6:17 AM, Hannes Reinecke wrote: On 03/31/2013 07:44 PM, Tomas Henzl wrote: You are right in the sense that 'max_lun' actually has a double meaning. First it's being used as the upper limit for sequential scan, where is has a strictly sequential meaning. So any internal LUN structure doesn't come into play here are we're just 'counting'. Secondly it's being used as a simple validation for any LUN numbers reported via REPORT LUNS. Here it the max_lun value actually refers to the amount of _bits_ in a LUN number the HBA can transfer. Again, the internal LUN structure doesn't come into play here; this is purely a hardware limitation on the HBA. Whether a LUN is valid or not is none of our concern; if the target accepts the LUN is has to be valid. If it doesn't then we don't care whether the LUN structure is valid or not; there is no device to be had anyway. However, after consulting SAM it is true that a plain 'max_lun' is incorrect for any LUN value higher than 255. LUN values higher than 255 should be represented with the 'flat space addressing' model, ie bit 6 should be set. Sadly, the various SAM revisions differ on how LUNs lower than 255 should be treated; they might or might not have set the flat space addressing model. If you treat the LUN as an 8-byte single big-endian value, then every LUN using an address method other than the peripheral device addressing method (00b) is greater than 255; in fact, they're all greater than 4,611,686,018,427,387,904 (0x4000___). Yes. I know. One LUN value is not represented by different addressing models. For example, these are four different LUNs, not multiple ways to express the same LUN 0: a) LUN 0x___ address method 00b target or lun = _b b) LUN 0x4000___ address method 01b flat space lun = 00___b c) LUN 0xD200___ address method 11b length 01b extended address method 0x2 extended flat space lun = 0x00_ d) LUN 0xE200___ address method 11b length 10b extended address method 0x2 long extended flat space lun = 0x00_ Yes. Sequential scanning should be an exception for very old devices that don't support REPORT LUNS; a small maximum scan value would be safer than scanning to max_luns, which could be huge. Or, to be more precise, for modern devices claiming SCSI-2 or SCSI-3 conformance. Some storage arrays have the habit of doing so. Three plausible lower scan limits: a) address method 00b with TARGET OR LUN set to 0..7: the number of LUNs supported by SCSI-1, which had a 3-bit LUN field in the CDBs themselves (long since obsolete) b) address method 00b with TARGET OR LUN set to 0..255: the number of LUNs supported by devices that only support address method 00b; the 8-bit TARGET OR LUN field portion of the first two bytes of the LUN. c) first two bytes set to 0..65535: the possible LUNs supported by devices with protocols with 2-byte LUN fields. I think scanning this many would take too long, though, and might trigger bugs in devices that don't decode the LUN correctly. My current thinking is to restrict sequential LUN scan to 255 LUNs (ie address method 00b) and adding a BLIST flag if someone wants/needs to scan more than that. I thought of decoding the address method for LUNs higher than 255, but then we might run into the differences between the various SAM versions: - SAM / SPC (ie SCSI-3) does _not_ define an internal LUN structure, so in theory any LUN is valid - SAM-2 / SPC-3 states that peripheral addressing should be used for LUNs less than 255, and flat space addressing for LUNs between 256 and 16383 - SAM-3 / SPC-4 and beyound states that peripheral _or_ flat space addressing can be used for LUNs less then 255, and flat space addressing should be used for LUNs between 256 and 16383 So any decoding we do would fail for SCSI-3 devices. And the amount of devices supporting more than 255 LUNs and _NOT_ supporting REPORT_LUN should be relatively small. Hence I think decoding LUNs is not worth the trouble and the BLIST approach should work just nicely here. Cheers, Hannes -- Dr. Hannes Reinecke zSeries Storage h...@suse.de +49 911 74053 688 SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: J. Hawn, J. Guild, F. Imendörffer, HRB
Re: [PATCH 0/4] scsi: 64-bit LUN support
On 04/05/2013 05:24 PM, James Smart wrote: On 4/4/2013 6:17 AM, Hannes Reinecke wrote: On 03/31/2013 07:44 PM, Tomas Henzl wrote: What we can do is to decode the LUN and compare it to max_lun provided by the driver, I think that sg_luns is able to do that, so what is needed is just to follow the SAM. I have seen reports of problem on three different drivers connected to various external storage, all of them having the same basic reason - the driver sets a max_lun and then LUN comes encoded with a newer addressing method and something like this is shown 'kernel: scsi: host 2 channel 0 id 2 lun16643 has a LUN larger than allowed by the host adapter' Decoding the real LUN value would fix this problem, by decoding is only meant the use in scsi_report_lun_scan. The LUN would be stored exactly the same way as it is now. I know we can patch the certain drivers too, but when max_lun were what the name says - max LU number, it would fix my problem very easy. Errm. No. Decoding LUNs is _evil_. It has only a relevance on the target, and even then it might choose to ignore it. So we cannot try to out-guess the target here. OK, I can see the problems with decoding the LUN one of them is the need to again encode the LUN to address format + number. I'm not sure if the hw would work if another address mode were used. When we understand the LUN as a complex structure then it makes no sense to compare to max_lun as a number - http://lxr.linux.no/#linux+v3.8.6/drivers/scsi/scsi_scan.c#L1471 The error you're reporting is that lpfc is setting max_luns to '255', which of course is less than 16643. Increasing max_luns on lpfc to '0x' will fix your problem; nothing to do with 64-bit LUNs ... I think I haven't mentioned lpfc, but it doesn't matter. Fixing this in individual drivers by increasing the max_lun is not easy, because the firmware could have some reasons for the max lun (some tables, ..., fact is I have no idea how this is implemented in the hw). If the fix for this were just to set max_lun to 0x in every driver it means that we could remove the max_lun and the test completely. A kernel option like 'ignore_max_lun' would help, but I somehow dislike it, what do you think? The reason lpfc set max_luns to 255 is due to the midlayer using max_luns as a (SCSI-2 device) max sequential scan loop top value, not necessarily as a max lun # as what's now in the report luns scan loop. When we were attached to jbods (loop, etc) - we saw 2 problems: our scan time dramatically increased (several minutes based on a 16k max_lun value); and as the jbod only decoded 8 bits - it happened to respond successfully to any lun value where the lower 8-bits were 0, meaning lots of midlayer ghost devices were created when in reality there was only 1 lun present.Changing the max_luns value is fine as long as you know what's attached. -- james s -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/4] scsi: 64-bit LUN support
On 4/4/2013 6:17 AM, Hannes Reinecke wrote: On 03/31/2013 07:44 PM, Tomas Henzl wrote: What we can do is to decode the LUN and compare it to max_lun provided by the driver, I think that sg_luns is able to do that, so what is needed is just to follow the SAM. I have seen reports of problem on three different drivers connected to various external storage, all of them having the same basic reason - the driver sets a max_lun and then LUN comes encoded with a newer addressing method and something like this is shown 'kernel: scsi: host 2 channel 0 id 2 lun16643 has a LUN larger than allowed by the host adapter' Decoding the real LUN value would fix this problem, by decoding is only meant the use in scsi_report_lun_scan. The LUN would be stored exactly the same way as it is now. I know we can patch the certain drivers too, but when max_lun were what the name says - max LU number, it would fix my problem very easy. Errm. No. Decoding LUNs is _evil_. It has only a relevance on the target, and even then it might choose to ignore it. So we cannot try to out-guess the target here. The error you're reporting is that lpfc is setting max_luns to '255', which of course is less than 16643. Increasing max_luns on lpfc to '0x' will fix your problem; nothing to do with 64-bit LUNs ... The reason lpfc set max_luns to 255 is due to the midlayer using max_luns as a (SCSI-2 device) max sequential scan loop top value, not necessarily as a max lun # as what's now in the report luns scan loop. When we were attached to jbods (loop, etc) - we saw 2 problems: our scan time dramatically increased (several minutes based on a 16k max_lun value); and as the jbod only decoded 8 bits - it happened to respond successfully to any lun value where the lower 8-bits were 0, meaning lots of midlayer ghost devices were created when in reality there was only 1 lun present.Changing the max_luns value is fine as long as you know what's attached. -- james s -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/4] scsi: 64-bit LUN support
On 03/31/2013 07:44 PM, Tomas Henzl wrote: On 03/30/2013 05:53 PM, Hannes Reinecke wrote: On 03/29/2013 05:32 PM, Tomas Henzl wrote: [ .. ] in scsi_report_lun_scan is max_lun compared with the result of scsilun_to_int, but in that value is also stored the address method. This means, that we compare the max_lun to a LUN 'handle' which doesn't seem to make much sense. This makes that test dependent on which address method is used and not only to the LUN number which is I think expected. The solution is to have a new function 'scsilun_to_num', (I can send a patch) or let the individual drivers set the max_lun to -1 and test for the allowed LUNs in the driver. You sure this is necessary? This is not directly related to your 64bit patch, I just 'used' this thread to discuss another issue - sorry. Yeah, and thereby decreasing the chance of getting my patchset in :-( I would like to avoid having to parse the LUN number for validity as we cannot guarantee this check has any meaning for the target. The only authoritative check can be made by the target. What we can do is to decode the LUN and compare it to max_lun provided by the driver, I think that sg_luns is able to do that, so what is needed is just to follow the SAM. I have seen reports of problem on three different drivers connected to various external storage, all of them having the same basic reason - the driver sets a max_lun and then LUN comes encoded with a newer addressing method and something like this is shown 'kernel: scsi: host 2 channel 0 id 2 lun16643 has a LUN larger than allowed by the host adapter' Decoding the real LUN value would fix this problem, by decoding is only meant the use in scsi_report_lun_scan. The LUN would be stored exactly the same way as it is now. I know we can patch the certain drivers too, but when max_lun were what the name says - max LU number, it would fix my problem very easy. Errm. No. Decoding LUNs is _evil_. It has only a relevance on the target, and even then it might choose to ignore it. So we cannot try to out-guess the target here. The error you're reporting is that lpfc is setting max_luns to '255', which of course is less than 16643. Increasing max_luns on lpfc to '0x' will fix your problem; nothing to do with 64-bit LUNs ... In the 64-bit context the max_luns should rather be interpreted as a 'max bits' setting, ie the number of _bits_ per LUN number the HBA is able to transport. But renaming 'max_luns' to 'max_bits' is a bit pointless, as it would break backwards compability for no real gain. So with my patchset we have a two-step LUN validation: - max_luns controls the max LUN number (read: max number of bits per LUN) which can be transported to the target This means in fact only 32 or 64 bit - so a single bit flag is enough? No, it is not. We have SPI narrow with max_luns = 8, SPI wide with max_luns = 16, some (older) FC HBAs supporting only 8 or 16 bits LUNs, some RAID arrays supporting 32-bit LUNs etc. Cheers, Hannes -- Dr. Hannes Reinecke zSeries Storage h...@suse.de +49 911 74053 688 SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: J. Hawn, J. Guild, F. Imendörffer, HRB 16746 (AG Nürnberg) -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/4] scsi: 64-bit LUN support
On 03/30/2013 05:53 PM, Hannes Reinecke wrote: On 03/29/2013 05:32 PM, Tomas Henzl wrote: On 03/27/2013 08:37 AM, Hannes Reinecke wrote: On 03/26/2013 07:00 PM, Chad Dupuis wrote: On Tue, 19 Feb 2013, Hannes Reinecke wrote: This patchset updates the SCSI midlayer to use 64-bit LUNs internally. It eliminates the need to limit the number of LUNs artificially to avoid aliasing issues; the SCSI midlayer can now accept any LUN presented to it. The LLDD specific settings for 'max_lun' have been left untouched; it should be raised to '~0' if the HBA supports 64-bit LUNs internally. However, it is up to the driver maintainer to raise that limit. Hannes Reinecke (4): scsi_scan: Fixup scsilun_to_int() scsi: use 64-bit LUNs scsi: use 64-bit value for 'max_luns' scsi: Remove CONFIG_SCSI_MULTI_LUN Hannes, As we've reviewed these patches internally, the one question that keeps coming up is how do we handle hardware that cannot handle a 64-bit LUN address? For example, some of our older 2G/bps hardware can only handle a 16-bit LUN address. Currently we convert the u32 value to u16. Do we do the same for the 64-bit conversion? Can a way be devised to opt-out of receiving a 64-bit address in the first place (IIRC this was an option in the v1 patch set)? Yes, you can. The idea here is to let 'max_luns' control this behaviour; 'max_luns' is the highest LUN number the host can support. So for 16-bit LUN you would set max_luns to '0x', and for 32-bit LUN addresses you would be setting max_luns to '0x'. Hi all, in scsi_report_lun_scan is max_lun compared with the result of scsilun_to_int, but in that value is also stored the address method. This means, that we compare the max_lun to a LUN 'handle' which doesn't seem to make much sense. This makes that test dependent on which address method is used and not only to the LUN number which is I think expected. The solution is to have a new function 'scsilun_to_num', (I can send a patch) or let the individual drivers set the max_lun to -1 and test for the allowed LUNs in the driver. You sure this is necessary? This is not directly related to your 64bit patch, I just 'used' this thread to discuss another issue - sorry. I would like to avoid having to parse the LUN number for validity as we cannot guarantee this check has any meaning for the target. The only authoritative check can be made by the target. What we can do is to decode the LUN and compare it to max_lun provided by the driver, I think that sg_luns is able to do that, so what is needed is just to follow the SAM. I have seen reports of problem on three different drivers connected to various external storage, all of them having the same basic reason - the driver sets a max_lun and then LUN comes encoded with a newer addressing method and something like this is shown 'kernel: scsi: host 2 channel 0 id 2 lun16643 has a LUN larger than allowed by the host adapter' Decoding the real LUN value would fix this problem, by decoding is only meant the use in scsi_report_lun_scan. The LUN would be stored exactly the same way as it is now. I know we can patch the certain drivers too, but when max_lun were what the name says - max LU number, it would fix my problem very easy. In the 64-bit context the max_luns should rather be interpreted as a 'max bits' setting, ie the number of _bits_ per LUN number the HBA is able to transport. But renaming 'max_luns' to 'max_bits' is a bit pointless, as it would break backwards compability for no real gain. So with my patchset we have a two-step LUN validation: - max_luns controls the max LUN number (read: max number of bits per LUN) which can be transported to the target This means in fact only 32 or 64 bit - so a single bit flag is enough? - The target validates the transported LUN number. Hence I don't think we would need an additional function here. But yes, we need to update scsi_debug as this should validate the incoming LUN number. As should the target mode drivers. But this can be left for later once the 64-bit changes are in. Cheers, Hannes -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/4] scsi: 64-bit LUN support
On 03/29/2013 05:32 PM, Tomas Henzl wrote: On 03/27/2013 08:37 AM, Hannes Reinecke wrote: On 03/26/2013 07:00 PM, Chad Dupuis wrote: On Tue, 19 Feb 2013, Hannes Reinecke wrote: This patchset updates the SCSI midlayer to use 64-bit LUNs internally. It eliminates the need to limit the number of LUNs artificially to avoid aliasing issues; the SCSI midlayer can now accept any LUN presented to it. The LLDD specific settings for 'max_lun' have been left untouched; it should be raised to '~0' if the HBA supports 64-bit LUNs internally. However, it is up to the driver maintainer to raise that limit. Hannes Reinecke (4): scsi_scan: Fixup scsilun_to_int() scsi: use 64-bit LUNs scsi: use 64-bit value for 'max_luns' scsi: Remove CONFIG_SCSI_MULTI_LUN Hannes, As we've reviewed these patches internally, the one question that keeps coming up is how do we handle hardware that cannot handle a 64-bit LUN address? For example, some of our older 2G/bps hardware can only handle a 16-bit LUN address. Currently we convert the u32 value to u16. Do we do the same for the 64-bit conversion? Can a way be devised to opt-out of receiving a 64-bit address in the first place (IIRC this was an option in the v1 patch set)? Yes, you can. The idea here is to let 'max_luns' control this behaviour; 'max_luns' is the highest LUN number the host can support. So for 16-bit LUN you would set max_luns to '0x', and for 32-bit LUN addresses you would be setting max_luns to '0x'. Hi all, in scsi_report_lun_scan is max_lun compared with the result of scsilun_to_int, but in that value is also stored the address method. This means, that we compare the max_lun to a LUN 'handle' which doesn't seem to make much sense. This makes that test dependent on which address method is used and not only to the LUN number which is I think expected. The solution is to have a new function 'scsilun_to_num', (I can send a patch) or let the individual drivers set the max_lun to -1 and test for the allowed LUNs in the driver. You sure this is necessary? I would like to avoid having to parse the LUN number for validity as we cannot guarantee this check has any meaning for the target. The only authoritative check can be made by the target. In the 64-bit context the max_luns should rather be interpreted as a 'max bits' setting, ie the number of _bits_ per LUN number the HBA is able to transport. But renaming 'max_luns' to 'max_bits' is a bit pointless, as it would break backwards compability for no real gain. So with my patchset we have a two-step LUN validation: - max_luns controls the max LUN number (read: max number of bits per LUN) which can be transported to the target - The target validates the transported LUN number. Hence I don't think we would need an additional function here. But yes, we need to update scsi_debug as this should validate the incoming LUN number. As should the target mode drivers. But this can be left for later once the 64-bit changes are in. Cheers, Hannes -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/4] scsi: 64-bit LUN support
On 03/27/2013 08:37 AM, Hannes Reinecke wrote: On 03/26/2013 07:00 PM, Chad Dupuis wrote: On Tue, 19 Feb 2013, Hannes Reinecke wrote: This patchset updates the SCSI midlayer to use 64-bit LUNs internally. It eliminates the need to limit the number of LUNs artificially to avoid aliasing issues; the SCSI midlayer can now accept any LUN presented to it. The LLDD specific settings for 'max_lun' have been left untouched; it should be raised to '~0' if the HBA supports 64-bit LUNs internally. However, it is up to the driver maintainer to raise that limit. Hannes Reinecke (4): scsi_scan: Fixup scsilun_to_int() scsi: use 64-bit LUNs scsi: use 64-bit value for 'max_luns' scsi: Remove CONFIG_SCSI_MULTI_LUN Hannes, As we've reviewed these patches internally, the one question that keeps coming up is how do we handle hardware that cannot handle a 64-bit LUN address? For example, some of our older 2G/bps hardware can only handle a 16-bit LUN address. Currently we convert the u32 value to u16. Do we do the same for the 64-bit conversion? Can a way be devised to opt-out of receiving a 64-bit address in the first place (IIRC this was an option in the v1 patch set)? Yes, you can. The idea here is to let 'max_luns' control this behaviour; 'max_luns' is the highest LUN number the host can support. So for 16-bit LUN you would set max_luns to '0x', and for 32-bit LUN addresses you would be setting max_luns to '0x'. Hi all, in scsi_report_lun_scan is max_lun compared with the result of scsilun_to_int, but in that value is also stored the address method. This means, that we compare the max_lun to a LUN 'handle' which doesn't seem to make much sense. This makes that test dependent on which address method is used and not only to the LUN number which is I think expected. The solution is to have a new function 'scsilun_to_num', (I can send a patch) or let the individual drivers set the max_lun to -1 and test for the allowed LUNs in the driver. Thanks, Tomas However, since you mention it, maybe I should add a 'scsilun_to_u32' conversion routine, as this is requested in several places. Cheers, Hannes -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/4] scsi: 64-bit LUN support
On 03/26/2013 07:00 PM, Chad Dupuis wrote: On Tue, 19 Feb 2013, Hannes Reinecke wrote: This patchset updates the SCSI midlayer to use 64-bit LUNs internally. It eliminates the need to limit the number of LUNs artificially to avoid aliasing issues; the SCSI midlayer can now accept any LUN presented to it. The LLDD specific settings for 'max_lun' have been left untouched; it should be raised to '~0' if the HBA supports 64-bit LUNs internally. However, it is up to the driver maintainer to raise that limit. Hannes Reinecke (4): scsi_scan: Fixup scsilun_to_int() scsi: use 64-bit LUNs scsi: use 64-bit value for 'max_luns' scsi: Remove CONFIG_SCSI_MULTI_LUN Hannes, As we've reviewed these patches internally, the one question that keeps coming up is how do we handle hardware that cannot handle a 64-bit LUN address? For example, some of our older 2G/bps hardware can only handle a 16-bit LUN address. Currently we convert the u32 value to u16. Do we do the same for the 64-bit conversion? Can a way be devised to opt-out of receiving a 64-bit address in the first place (IIRC this was an option in the v1 patch set)? Yes, you can. The idea here is to let 'max_luns' control this behaviour; 'max_luns' is the highest LUN number the host can support. So for 16-bit LUN you would set max_luns to '0x', and for 32-bit LUN addresses you would be setting max_luns to '0x'. However, since you mention it, maybe I should add a 'scsilun_to_u32' conversion routine, as this is requested in several places. Cheers, Hannes -- Dr. Hannes Reinecke zSeries Storage h...@suse.de +49 911 74053 688 SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: J. Hawn, J. Guild, F. Imendörffer, HRB 16746 (AG Nürnberg) -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/4] scsi: 64-bit LUN support
On Wed, 27 Mar 2013, Hannes Reinecke wrote: On 03/26/2013 07:00 PM, Chad Dupuis wrote: On Tue, 19 Feb 2013, Hannes Reinecke wrote: This patchset updates the SCSI midlayer to use 64-bit LUNs internally. It eliminates the need to limit the number of LUNs artificially to avoid aliasing issues; the SCSI midlayer can now accept any LUN presented to it. The LLDD specific settings for 'max_lun' have been left untouched; it should be raised to '~0' if the HBA supports 64-bit LUNs internally. However, it is up to the driver maintainer to raise that limit. Hannes Reinecke (4): scsi_scan: Fixup scsilun_to_int() scsi: use 64-bit LUNs scsi: use 64-bit value for 'max_luns' scsi: Remove CONFIG_SCSI_MULTI_LUN Hannes, As we've reviewed these patches internally, the one question that keeps coming up is how do we handle hardware that cannot handle a 64-bit LUN address? For example, some of our older 2G/bps hardware can only handle a 16-bit LUN address. Currently we convert the u32 value to u16. Do we do the same for the 64-bit conversion? Can a way be devised to opt-out of receiving a 64-bit address in the first place (IIRC this was an option in the v1 patch set)? Yes, you can. The idea here is to let 'max_luns' control this behaviour; 'max_luns' is the highest LUN number the host can support. So for 16-bit LUN you would set max_luns to '0x', and for 32-bit LUN addresses you would be setting max_luns to '0x'. However, since you mention it, maybe I should add a 'scsilun_to_u32' conversion routine, as this is requested in several places. Cheers, Hannes Hannes, thanks for the clarification. This message and any attached documents contain information from QLogic Corporation or its wholly-owned subsidiaries that may be confidential. If you are not the intended recipient, you may not read, copy, distribute, or use this information. If you have received this transmission in error, please notify the sender immediately by reply e-mail and then delete this message. -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/4] scsi: 64-bit LUN support
On Tue, 19 Feb 2013, Hannes Reinecke wrote: This patchset updates the SCSI midlayer to use 64-bit LUNs internally. It eliminates the need to limit the number of LUNs artificially to avoid aliasing issues; the SCSI midlayer can now accept any LUN presented to it. The LLDD specific settings for 'max_lun' have been left untouched; it should be raised to '~0' if the HBA supports 64-bit LUNs internally. However, it is up to the driver maintainer to raise that limit. Hannes Reinecke (4): scsi_scan: Fixup scsilun_to_int() scsi: use 64-bit LUNs scsi: use 64-bit value for 'max_luns' scsi: Remove CONFIG_SCSI_MULTI_LUN Hannes, As we've reviewed these patches internally, the one question that keeps coming up is how do we handle hardware that cannot handle a 64-bit LUN address? For example, some of our older 2G/bps hardware can only handle a 16-bit LUN address. Currently we convert the u32 value to u16. Do we do the same for the 64-bit conversion? Can a way be devised to opt-out of receiving a 64-bit address in the first place (IIRC this was an option in the v1 patch set)? Thanks, Chad This message and any attached documents contain information from QLogic Corporation or its wholly-owned subsidiaries that may be confidential. If you are not the intended recipient, you may not read, copy, distribute, or use this information. If you have received this transmission in error, please notify the sender immediately by reply e-mail and then delete this message. -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/4] scsi: 64-bit LUN support
On 13-03-26 02:00 PM, Chad Dupuis wrote: On Tue, 19 Feb 2013, Hannes Reinecke wrote: This patchset updates the SCSI midlayer to use 64-bit LUNs internally. It eliminates the need to limit the number of LUNs artificially to avoid aliasing issues; the SCSI midlayer can now accept any LUN presented to it. The LLDD specific settings for 'max_lun' have been left untouched; it should be raised to '~0' if the HBA supports 64-bit LUNs internally. However, it is up to the driver maintainer to raise that limit. Hannes Reinecke (4): scsi_scan: Fixup scsilun_to_int() scsi: use 64-bit LUNs scsi: use 64-bit value for 'max_luns' scsi: Remove CONFIG_SCSI_MULTI_LUN Hannes, As we've reviewed these patches internally, the one question that keeps coming up is how do we handle hardware that cannot handle a 64-bit LUN address? For example, some of our older 2G/bps hardware can only handle a 16-bit LUN address. Currently we convert the u32 value to u16. Do we do the same for the 64-bit conversion? Can a way be devised to opt-out of receiving a 64-bit address in the first place (IIRC this was an option in the v1 patch set)? Chad, Perhaps I'm missing something. Given a t10_LUN and linux32_LUN and the proposed linux64_LUN then the traditional 16 bit LUN value (flat space addressing ?) would be either: (t10_LUN[0] 8) | t10_LUN[1] or linux32_LUN 0x or linux64_LUN 0x Doug Gilbert -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/4] scsi: 64-bit LUN support
On 13-02-21 11:32 AM, James Bottomley wrote:
On Thu, 2013-02-21 at 16:15 +, Elliott, Robert (Server Storage)
wrote:
Regarding changes like this:
- printk(MYIOC_s_NOTE_FMT [%d:%d:%d:%d]
+ printk(MYIOC_s_NOTE_FMT [%d:%d:%d:%llu]
FCP_ResponseInfo=%08xh\n, ioc-name,
sc-device-host-host_no, sc-device-channel,
sc-device-id, sc-device-lun,
It might be preferable to print the LUN values in hex rather than
decimal, particularly if they are large values. SAM-5 includes some
guidance for displaying LUNs, shown below.
We can't really change from decimal to hex without causing confusion and
possibly breaking ABIs. All the existing SCSI references look like
h:c:t:l and all expect l to be a simple decimal. It's not just in the
logs, we have active use of this form in all the /sys/class/scsi_*/
directories and some tools may parse this value.
One important goal is to match the format, if any, that the user must
use in a configuration file or command line argument, so
cutting-and-pasting the LUN value works. So, the answer might differ
for prints from different drivers. If a driver expects decimal input
values, then print decimal.
SAM-5 excerpt:
4.7.2 Logical unit representation format
[...]
We're a bit bound by kernel convention here as well. To retain
compatibility with SPI and flat addressing schemes, we really need to
show the 8 and 16 bit flat addresses as simple decimal numerics.
However, we *might* be free to move to a more hierarchical scheme with
the multi-level luns, since I don't think there's to many people who've
got arrays that output them (yet).
For those of you that don't remember, back when the Linux
OS was born, a SCSI READ(6) command looked like this:
==
Bit| 7| 6| 5| 4| 3| 2| 1| 0|
Byte |||||||||
==
0 | Operation Code (08h)|
-|---|
1 | Logical Unit Number | (MSB) |
-|-- ---|
2 | Logical Block Address |
-|--- ---|
3 | (LSB) |
-|---|
4 | Transfer Length |
-|---|
5 | Control |
==
Notice that 3 bit LUN field tucked into byte 1 of the cdb.
There may be still remnants of code in the Linux SCSI
subsystem to support that. Since then LUNs have grown to 16,
32 and now 64 bits. Will they keep expanding?
My point is that a T10 sanctioned LUN representation
can be a hard act for an OS to follow.
Fixing Linux's broken 32 bit word-flipped LUN
should be done as a matter of urgency, IMO. Here I am
referring to Hannes Reinecke patch:
[PATCH 1/4] scsi_scan: Fixup scsilun_to_int()
Extending Linux's 32 bit word-flipped LUN to a 64 bit
word-flipped LUN seems like the least disruptive change
for the Linux user space. Existing small LUN formats
would continue to yield the same sysfs and bsg
file/directory names. And my lsscsi utility would
continue to work.
lsscsi will still need some work to support 64 bit
word-flipped LUNs. The simple one is to expand the LUN
decoding (currently expecting a 32 bit signed integer)
to accept uint64_t . Also I would like to add a
--lunhex option ('-x' as the more obvious short option
letters are already used) to show the LUN in hex along
the lines of what Robert Elliott (and SAM-5) propose.
As an example showing the REPORT LUNS well known LUNS
as a non-trivial LUN:
# lsscsi
[0:0:0:0]diskATA INTEL SSDSC2CW12 400i /dev/sda
[6:0:0:1]diskLinuxscsi_debug 0004 /dev/sdb
[6:0:0:49409]wlunLinuxscsi_debug 0004 -
# lsscsi -x
[0:0:0:0x] diskATA INTEL SSDSC2CW12 400i /dev/sda
[6:0:0:0x0001] diskLinuxscsi_debug 0004 /dev/sdb
[6:0:0:0xc101] wlunLinuxscsi_debug 0004 -
# lsscsi -xx
[0:0:0:0x] diskATA INTEL SSDSC2CW12 400i /dev/sda
[6:0:0:0x0001] diskLinuxscsi_debug 0004 /dev/sdb
[6:0:0:0xc101] wlunLinuxscsi_debug 0004 -
Also, given an
Re: [PATCH 0/4] scsi: 64-bit LUN support
On 02/21/2013 05:15 PM, Elliott, Robert (Server Storage) wrote: Regarding changes like this: - printk(MYIOC_s_NOTE_FMT [%d:%d:%d:%d] + printk(MYIOC_s_NOTE_FMT [%d:%d:%d:%llu] FCP_ResponseInfo=%08xh\n, ioc-name, sc-device-host-host_no, sc-device-channel, sc-device-id, sc-device-lun, It might be preferable to print the LUN values in hex rather than decimal, particularly if they are large values. SAM-5 includes some guidance for displaying LUNs, shown below. One important goal is to match the format, if any, that the user must use in a configuration file or command line argument, so cutting-and-pasting the LUN value works. So, the answer might differ for prints from different drivers. If a driver expects decimal input values, then print decimal. Yes. And no. The patchset is meant to be as a drop-in replacement, ie with _no_ user visible changes. So I retained the original output. Also, the above number is identical to the sysfs representation, where the user will find a device like /sys/class/scsi_device/X:Y:Z:L with each X, Y, Z, and L being a decimal number. _If_ we were to change that we would have to do it consistently for all instances, plus the sysfs layout. But I guess we need to have a discussion about this, as I think not everybody agrees with that assessment. Would be a nice topic for LSF, though. Cheers, Hannes -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 0/4] scsi: 64-bit LUN support
Regarding changes like this: - printk(MYIOC_s_NOTE_FMT [%d:%d:%d:%d] + printk(MYIOC_s_NOTE_FMT [%d:%d:%d:%llu] FCP_ResponseInfo=%08xh\n, ioc-name, sc-device-host-host_no, sc-device-channel, sc-device-id, sc-device-lun, It might be preferable to print the LUN values in hex rather than decimal, particularly if they are large values. SAM-5 includes some guidance for displaying LUNs, shown below. One important goal is to match the format, if any, that the user must use in a configuration file or command line argument, so cutting-and-pasting the LUN value works. So, the answer might differ for prints from different drivers. If a driver expects decimal input values, then print decimal. SAM-5 excerpt: 4.7.2 Logical unit representation format When an application client displays or otherwise makes a 64-bit LUN value visible, the application client should display it in hexadecimal format with byte 0 first (i.e., on the left) and byte 7 last (i.e., on the right), regardless of the internal representation of the LUN value (e.g., a single level LUN with an ADDRESS METHOD field set to 01b (i.e., flat space addressing) and a FLAT SPACE LUN field set to 0001h should be displayed as 40 01 00 00 00 00 00 00h, not 00 00 00 00 00 00 01 40h). A separator (e.g., space, dash, or colon) may be included between each byte, each two bytes (e.g., 4001---h), or each four bytes (e.g., 4001 h). [The trailing h is just the T10 documentation convention... a 0x prefix is fine too] [The next three paragraph allow stripping off unnecessary trailing zeros:] When displaying a single level LUN structure using the peripheral device addressing method (see table 11) or a single level LUN structure using the flat space addressing method (see table 12), an application client may display the value as a single 2-byte value representing only the first level LUN (e.g., 40 01h). A separator (e.g., space, dash, or colon) may be included between each byte. When displaying a single level LUN structure using the extended flat space addressing method (see table 13), an application client may display the value as a single 4-byte value representing only the first level LUN (e.g., D2 00 00 01h). A separator (e.g., space, dash, or colon) may be included between each byte, or between each two bytes (e.g., D200 0001h). When displaying a single level LUN structure using the long extended flat space addressing method (see table 14), an application client may display the value as a single 6-byte value representing only the first level LUN (e.g., E2 00 00 01 00 01h). A separator (e.g. space, dash, or colon) may be included between each byte, or between each two bytes (e.g., E200 0001 0001h). When displaying a 16-bit LUN value, an application client should display the value as a single 2-byte value (e.g., 40 01h). A separator (e.g., space, dash, or colon) may be included between each byte. -Original Message- From: Hannes Reinecke [mailto:h...@suse.de] Sent: Tuesday, 19 February, 2013 2:18 AM To: linux-scsi@vger.kernel.org Cc: James Bottomley; Jeremy Linton; Elliott, Robert (Server Storage); Bart Van Assche; Hannes Reinecke Subject: [PATCH 0/4] scsi: 64-bit LUN support This patchset updates the SCSI midlayer to use 64-bit LUNs internally. It eliminates the need to limit the number of LUNs artificially to avoid aliasing issues; the SCSI midlayer can now accept any LUN presented to it. The LLDD specific settings for 'max_lun' have been left untouched; it should be raised to '~0' if the HBA supports 64-bit LUNs internally. However, it is up to the driver maintainer to raise that limit. -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/4] scsi: 64-bit LUN support
On Thu, 2013-02-21 at 16:15 +, Elliott, Robert (Server Storage) wrote: Regarding changes like this: - printk(MYIOC_s_NOTE_FMT [%d:%d:%d:%d] + printk(MYIOC_s_NOTE_FMT [%d:%d:%d:%llu] FCP_ResponseInfo=%08xh\n, ioc-name, sc-device-host-host_no, sc-device-channel, sc-device-id, sc-device-lun, It might be preferable to print the LUN values in hex rather than decimal, particularly if they are large values. SAM-5 includes some guidance for displaying LUNs, shown below. We can't really change from decimal to hex without causing confusion and possibly breaking ABIs. All the existing SCSI references look like h:c:t:l and all expect l to be a simple decimal. It's not just in the logs, we have active use of this form in all the /sys/class/scsi_*/ directories and some tools may parse this value. One important goal is to match the format, if any, that the user must use in a configuration file or command line argument, so cutting-and-pasting the LUN value works. So, the answer might differ for prints from different drivers. If a driver expects decimal input values, then print decimal. SAM-5 excerpt: 4.7.2 Logical unit representation format [...] We're a bit bound by kernel convention here as well. To retain compatibility with SPI and flat addressing schemes, we really need to show the 8 and 16 bit flat addresses as simple decimal numerics. However, we *might* be free to move to a more hierarchical scheme with the multi-level luns, since I don't think there's to many people who've got arrays that output them (yet). James -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
