I am sponsoring the following fasttrack for myself. Micro/patch
binding is requested. The timer is set to expire on Nov. 12 2008.
-Chris
1. Introduction
1.1. Project/Component Working Name:
SCSI_HBA_ADDR_COMPLEX SCSA data structure linkage
1.2. Name of Document Author/Supplier:
Author: Chris Horne
1.3. Date of This Document:
Thu Oct 23 2008
4. Technical Description
4.1 Background:
Initial Sun Common SCSI Architecture (SCSA) data structures were
designed to support copper scsi addressing, where both the target
and lun numbers were small integers. In this model, an HBA driver
was expected to maintain per-scsi_device(9S) HBA private data inside
an array embedded in the HBA's initiator port soft state. This
array, indexed by the scsi_address(9S) a_target and a_lun(9S)
fields, was dimensioned by the maximum target and lun numbers
supported by the HBA driver.
The limitation of this approach was exposed by transports such as
fiber-channel, and overcoming the limitation was complicated by the
fact that a scsi_address(9S) is embedded at the base of two
well-know SCSA data structures: both the scsi_device(9S) structure
and the scsi_pkt(9S) structure.
To solve this problem, interfaces were needed to allow an HBA driver
to maintain and access per-scsi_device(9S) HBA private data, given
only the scsi_address(9S) information of an allocated scsi_pkt(9S)
(as established by a 'pkt->pkt_address = *ap;' structure assignment
off the scsi_init_pkt(9F)/tran_init_pkt(9E) implementation).
Given the SCSA environment and development constraints at the time,
'cloning' the scsi_hba_tran(9S) structure was an effective data
structure implementation approach (an expedient kludge that
minimized breakage, but wasted space).
Unfortunately, the 'clone' data structure implementation details
ended up being leaked as interfaces:
o scsi_hba_attach(9E) SCSI_HBA_TRAN_CLONE flag.
o scsi_hba_tran(9S) 'tran_sd' field, pointing to
scsi_device(9S).
o scsi_hba_tran(9S) 'tran_tgt_private' field, used to implement
per-scsi_device(9S) HBA private data (inside the 'cloned'
scsi_hba_tran(9S) structure).
The first ARC references to SCSI_HBA_TRAN_CLONE is in [1], from
1993. The oldest CR is 1156862 "scsi_hba: if cloning, passes wrong
transport structure to hba in tran_tgt_free", from the same
timeframe. There is no ARC record introducing SCSI_HBA_TRAN_CLONE
as an interface, public or private: records go back to 1991. It
seems to have just leaked into the DDI based on need.
4.2 Proposal:
This case proposes solving the same basic problem with interfaces
that minimize exposure of implementation artifacts. The proposed
interfaces are consolidation private, but future promotion is
expected (with future deprecation of the "clone" 'interfaces').
The new proposed interfaces are:
o SCSI_HBA_ADDR_COMPLEX:
A scsi_hba_attach_setup(9E) caller can indicate that an
initiator port requires a 'complex' target/lun address space
representation the new SCSI_HBA_ADDR_COMPLEX flag. The
properties associated with a scsi_device(9S) target/lun
unit-address are available to the HBA at tran_tgt_init(9E)
time. The HBA driver can translate unit-address property values
into an appropriate private form, and store that information in
its per-scsi_device(9S) HBA private data.
o scsi_device_hba_private_set(9F),
scsi_device_hba_private_set(9F):
A SCSI_HBA_ADDR_COMPLEX HBA can maintain/obtain
per-scsi_device(9S) HBA private data using
scsi_device_hba_private_set(9F) and
scsi_device_hba_private_get(9F).
o scsi_address_device(9F):
A SCSI_HBA_ADDR_COMPLEX HBA can call scsi_address_device(9F) to
obtain a pointer to the scsi_device(9S) given a pointer to a
scsi_address(9S).
A SCSI_HBA_ADDR_COMPLEX HBA is expected to establish
per-scsi_device(9S) HBA private data in its tran_tgt_init(9E)
implementation, and free that HBA private data in its
tran_tgt_fini(9E) implementation.
In its tran_start(9E) implementation, SCSI_HBA_ADDR_COMPLEX HBA can
obtain its scsi_device(9S) pointer from the scsi_address(9S) by
calling scsi_address_device(9F). Once the HBA driver obtains the
scsi_device(9S) pointer, the per-scsi_device(9S) HBA private data
can be obtained using scsi_device_hba_private_get(9F).
At this point in time, implementing the proposed interfaces no
longer requires 'cloning': the CRs that required 'cloning' [3] have
now been resolved. The new implementation will be placing the SCSI
parallel interconnect (SPI) scsi_address(9S) fields (a_target,
a_lun) in a union, and in the SCSI_HBA_ADDR_COMPLEX case the union
will be a pointer to the scsi_device(9S) structure, with
per-scsi_device(9S) HBA private data maintained in the
scsi_device(9S) itself (sd_hba_private), instead of in a cloned
scsi_hba_tran(9S) structure. Pictures (.gif) of the old and new
data structure linkage are provided in the materials directory.
While it would be possible to have the new (9F) interfaces function
correctly for the SCSI_HBA_TRAN_CLONE case too, that is not
planned.
Having a single scsi_hba_tran(9S) structure per SCSI initiator_port
will be important for futute SCSA work, where the SCSA framework
will need to be enhanced to maintain new per-target_port
information.
4.3 Proposed Interfaces
------------------------------------------------------------------
Interface Name Comm.Level Comments
------------------------------------------------------------------
SCSI_HBA_ADDR_COMPLEX Private scsi_hba_attach_setup(9F)
flag indicating HBA
transport has 'complex'
address space.
SCSI_HBA_ADDR_SPI Private default: HBA expects
scsi_address(9S) in
SCSI Parallel Interconnect
form (a_target, a_lun).
SCSI_HBA_TRAN_CLONE no-change use discouraged.
scsi_address_device(9F) Private Given a pointer to a
scsi_address(9S),
return pointer to
scsi_device(9S).
scsi_device_hba_private_get(9F) Private per-scsi_device(9S)
scsi_device_hba_private_set(9F) HBA private data
interfaces.
Prototypes for new (9F) functions: common/sys/scsi/conf/device.h:
struct scsi_device *scsi_address_device(struct scsi_address *sa);
void scsi_device_hba_private_set(struct scsi_device *sd, void *data);
void *scsi_device_hba_private_get(struct scsi_device *sd);
4.4 Man Pages
See Appendix A
4.4 Release Binding
Micro/patch binding is requested.
4.5 References
[1] PSARC/1993/259 SCSI HBA interface [Jerry Gilliam]
http://sac.sfbay/PSARC/1993/259
http://www.opensolaris.org/os/community/arc/caselog/PSARC/1993/259
[2] PSARC/1996/113 SCSA SCSI-3 enhancements [Sudershan Goyal]
http://sac.sfbay/PSARC/1996/113
http://www.opensolaris.org/os/community/arc/caselog/PSARC/1996/113
[3] CRs associated with scsi_address/scsi_device/scsi_pkt cleanup
6269673 scsi target drivers should not reference scsi_address(4s)...
6657256 SCSA should detect scsi_pkt allocation violations
6695222 ata has dependency on scsi_device(9S) size
6755515 preparation for scsi_address being unionized
http://monaco.sfbay.sun.com/detail.jsf?cr=6269673,6657256,6695222,6755515
Appendix A:
A.1: Changes to scsi_hba_attach_setup(9F)
=========================================
:r!diff -U5 -w scsi_hba_attach_setup.9f.orig scsi_hba_attach_setup.9f
--- scsi_hba_attach_setup.9f.orig Tue Oct 21 19:12:37 2008
+++ scsi_hba_attach_setup.9f Tue Oct 28 08:57:27 2008
@@ -25,12 +25,14 @@
hba_lim A pointer to a ddi_dma_lim(9S) structure.
hba_tran A pointer to a scsi_hba_tran(9S) structure.
- hba_flags Flag modifiers. The only defined flag value
- is SCSI_HBA_TRAN_CLONE.
+ hba_flags Flag modifiers. The only defined flag values are
+ SCSI_HBA_ADDR_SPI, SCSI_HBA_ADDR_COMPLEX, and
+ SCSI_HBA_TRAN_CLONE. SCSI_HBA_TRAN_CLONE is
+ obsolete, use SCSI_HBA_ADDR_COMPLEX instead.
hba_options Optional features provided by the HBA driver
for future extensions; must be NULL.
hba_dma_attr A pointer to a ddi_dma_attr(9S) structure.
@@ -55,10 +57,24 @@
scsi_hba_attach() and scsi_hba_attach_setup() use the
dev_bus_ops field in the dev_ops(9S) structure. The HBA
driver should initialize this field to NULL before calling
scsi_hba_attach() or scsi_hba_attach_setup().
+ A scsi_address(9S) unit-address form for a scsi_device(9S) is
+ chosen based on whether SCSI_HBA_ADDR_SPI or SCSI_HBA_ADDR_COMPLEX
+ is requested in hba_flags. Only one flag can be specified; if
+ neither flag is specified SCSI_HBA_ADDR_SPI is assumed. If
+ SCSI_HBA_ADDR_SPI is selected then the a_target and a_lun fields
+ of a scsi_address structure are established by the SCSA library
+ code prior to tran_tgt_init(9E). If SCSI_HBA_ADDR_COMPLEX is
+ selected then the host adapter driver is expected to establish an
+ opaque private representation of device unit-address, based
+ on property values, during tran_tgt_init(9E). See scsi_address(9S)
+ for more detail.
+
+ The SCSI_HBA_TRAN_CLONE flag is also supported, but use
+ of SCSI_HBA_ADDR_COMPLEX is prefered.
If SCSI_HBA_TRAN_CLONE is requested in hba_flags, the
hba_tran structure will be cloned once for each target
attached to the HBA. The cloning of the structure will occur
before the tran_tgt_init(9E) entry point is called to ini-
tialize a target. At all subsequent HBA entry points,
@@ -189,5 +205,9 @@
get device driver after scsi_hba_detach() is called.
The scsi_hba_attach() function is obsolete and will be dis-
continued in a future release. This function is replaced by
scsi_hba_attach_setup().
+
+ The use of SCSI_HBA_TRAN_CLONE is obsolete and will be
+ discontinued in a future release, use SCSI_HBA_ADDR_COMPLEX
+ instead.
=========================================
A.2: Changes to scsi_address(9S)
:r!diff -U5 -w scsi_address.9s.orig scsi_address.9s
--- scsi_address.9s.orig Tue Oct 21 19:12:37 2008
+++ scsi_address.9s Mon Oct 27 20:34:36 2008
@@ -8,45 +8,107 @@
INTERFACE LEVEL
Solaris architecture specific (Solaris DDI)
DESCRIPTION
- A scsi_address structure defines the addressing components
- for a SCSI target device. The address of the target device
- is separated into two components: target number and logical
- unit number. The two addressing components are used to
- uniquely identify any type of SCSI device; however, most
- devices can be addressed with the target component of the
- address.
-
- In the case where only the target component is used to
- address the device, the logical unit should be set to 0. If
- the SCSI target device supports logical units, then the HBA
- must interpret the logical units field of the data struc-
- ture.
-
- The pkt_address member of a scsi_pkt(9S) is initialized by
- scsi_init_pkt(9F).
+ A scsi_address structure stores the host routing and device
+ unit-address information necessary to reference a specific SCSI
+ target device logical unit function.
+
+ Host routing information is used by the system to identify the
+ HBA and initiator port.
+
+ Device unit-address information is SCSA's representation of the
+ "@unit-address" portion of a SCSI target driver scsi_device node
+ in the the /devices tree. Separate portions of the device
+ unit-address information define the target port address, the
+ logical unit address on the target, and (optionally) a
+ subfunction implemented by a logical unit. Device scsi_device
+ unit-address information is used exclusively by the host adapter
+ driver (with the legacy exception of target drivers communicating
+ with SCSI Parallel Interconnect (SPI) SCSI-1 devices that embed
+ SCSI logical unit addressing in the CDB).
+
+ Two forms of scsi_device unit-address representation are
+ supported within the scsi_address structure: a host adapter
+ driver chooses the form by its hba_flags argument to
+ scsi_hba_attach_setup(9F): SCSI_HBA_ADDR_SPI or
+ SCSI_HBA_ADDR_COMPLEX. For SCSI interconnects that have a complex
+ scsi_device unit-address space, like fibre-channel,
+ SCSI_HBA_ADDR_COMPLEX should be selected. To support legacy host
+ adapter drivers that don't explicitly choose a form,
+ SCSI_HBA_ADDR_SPI is assumed.
+
+ For host adapter drivers that choose SCSI_HBA_ADDR_SPI, two
+ scsi_device unit-address portions, target number (a_target) and
+ logical unit number (a_lun), are directly represented within the
+ scsi_address structure.
+
+ For host adapter drivers that choose SCSI_HBA_ADDR_COMPLEX, the
+ tran_tgt_init(9E) implementation is expected to store a
+ per-scsi_device(9S) HBA private data pointer, based on its
+ interpretation of the scsi(9P) device unit-address properties,
+ using scsi_device_hba_private_set(9F). This approach allows SCSI
+ interconnects to have their own private scsi_device unit-address
+ representation, which is particularly important for the target
+ portion of the unit-address, which is transport-specific. During
+ tran_start(9E), an HBA driver can locate the scsi_device(9S)
+ structure using scsi_address_device(9F), and obtain the
+ per-scsi_device(9S) HBA private data pointer using
+ scsi_device_hba_private_get(9F). The HBA private data should be
+ freed by tran_tgt_free(9E).
+
+ The pkt_address member of a scsi_pkt(9S) is initialized after
+ scsi_init_pkt(9F) as a result of the host adapter driver
+ performing a structure assignment in its tran_init_pkt(9E)
+ implementation.
STRUCTURE MEMBERS
- scsi_hba_tran_t *a_hba_tran; /* Transport vectors for the SCSI bus */
- ushort_t a_target; /* SCSI target id */
- uchar_t a_lun; /* SCSI logical unit */
-
- a_hba_tran is a pointer to the controlling HBA's transport
- vector structure. The SCSA interface uses this field to pass
- any transport requests from the SCSI target device drivers
- to the HBA driver.
-
- a_target is the target component of the SCSI address.
-
- a_lun is the logical unit component of the SCSI address. The
- logical unit is used to further distinguish a SCSI target
- device that supports multiple logical units from one that
- does not. The makecom(9F) family of functions use the a_lun
- field to set the logical unit field in the SCSI CDB, for
- compatibility with SCSI-1.
+
+ scsi_hba_tran_t *a_hba_tran; /* Transport vector */
+ ushort_t a_target; /* SPI: SCSI target id */
+ uchar_t a_lun; /* SPI: SCSI logical unit */
+ scsi_device *a_sd; /* COMPLEX: pointer to scsi_device */
+
+ a_hba_tran Pointer to the controlling host adapter driver
+ transport vector structure. The SCSA interface
+ uses this field to pass any transport requests
+ from the SCSI target device drivers to the host
+ adapter driver.
+
+ For SCSI_HBA_ADDR_COMPLEX:
+
+ a_sd A pointer to the scsi_device(9S) structure.
+ This field is set by SCSA library code. An HBA
+ driver should not access this field directly -
+ use scsi_address_device(9F) instead.
+
+ For SCSI_HBA_ADDR_SPI:
+
+ a_target target component of the device unit-address,
+ limited to the range 0-65535).
+
+ a_lun logical unit component of the device
+ unit-address, limited to the range 0-255. The
+ logical unit is used to further distinguish a
+ SCSI target device that supports multiple
+ logical units from one that does not. In the
+ case where only the target component is defined
+ in the device unit-address, the logical unit is
+ assumed to be 0. If the SCSI target device
+ supports logical units, then the host adapter
+ driver must interpret the logical units field
+ of the data structure. A scsi target driver
+ that supports SCSI-1 devices also needs to
+ access a_lun to embed logical unit addressing
+ in the scsi CDBs it constructs.
SEE ALSO
- makecom(9F), scsi_init_pkt(9F), scsi_hba_tran(9S),
- scsi_pkt(9S)
+ makecom(9F), scsi_address_device(9F), scsi_init_pkt(9F),
scsi_hba_tran(9S),
+ scsi_pkt(9S), tran_get_bus_addr(9E), tran_tgt_init(9E),
Writing Device Drivers
+
+ NOTES: A host adapter driver with complex addressing requirements
+ should use SCSI_HBA_ADDR_COMPLEX and
+ scsi_device_hba_private_get(9F) instead of the older, more
+ wasteful, SCSI_HBA_TRAN_CLONE scsi_hba_tran(9S)
+ tran_tgt_private approach.
A.3: Changes to scsi_device(9S)
=========================================
:r!diff -U5 -w scsi_device.9s.orig scsi_device.9s
--- scsi_device.9s.orig Tue Oct 21 19:12:37 2008
+++ scsi_device.9s Tue Oct 28 08:55:51 2008
@@ -8,39 +8,52 @@
INTERFACE LEVEL
Solaris DDI specific (Solaris DDI).
DESCRIPTION
- The scsi_device structure stores common information about
- each SCSI logical unit, including pointers to areas that
- contain both generic and device specific information. There
- is one scsi_device structure for each logical unit attached
- to the system. The host adapter driver initializes part of
- this structure prior to probe(9E) and destroys this struc-
- ture after a probe failure or successful detach(9E).
+ The scsi_device structure stores common information about a
+ target device logical unit function, including pointers to areas
+ that contain both generic and device specific information. There
+ is one scsi_device structure for each device unit-address attached
+ to the system, and one SCSI target driver instance (e.g. sd(7D),
+ st(7D), sgen(7D)) that provides access to the function. The device
+ unit-address is represented by the scsi_address(9S) structure
+ embedded in the scsi_device structure.
+
+ The scsi_device structures is allocated and freed by the SCSA
+ library code. The host adapter driver can initialize part of
+ this structure in tran_tgt_init(9E) prior to probe(9E) and
+ destroy its initialization in tran_tgt_free(9E) on probe failure or
+ after successful detach(9E).
STRUCTURE MEMBERS
struct scsi_address sd_address; /* Routing information */
dev_info_t *sd_dev; /* Cross-reference */
/* to our dev_info_t */
kmutex_t sd_mutex; /* Mutex for this device */
struct scsi_inquiry *sd_inq; /* scsi_inquiry data structure */
struct scsi_extended_sense *sd_sense; /* Optional request */
/* sense buffer ptr */
- caddr_t sd_private; /* Target drivers private data */
+ void *sd_private; /* Target drivers private data */
+ void *sd_hba_private; /* HBA drivers private data */
-
- sd_address contains the routing information that the target
- driver normally copies into a scsi_pkt(9S) structure using
- the collection of makecom(9F) functions. The SCSA library
- routines use this information to determine which host
- adapter, SCSI bus, and target/logical unit number (lun) a
- command is intended for. This structure is initialized by
- the host adapter driver.
+ sd_address contains the host routing and scsi_device unit-address
+ information necessary to reference a specific SCSI target device
+ logical unit function.
+
+ A SCSI_HBA_ADDR_COMPLEX host adapter, as defined by
+ scsi_hba_attach_setup(9F) flags, can use
+ scsi_device_hba_private_set(9F) during tran_tgt_init(9E) to save a
+ pointer to per-scsi_device HBA private data. , and free that
+ allocation in its tran_tgt_free(9E) implementation. The HBA
+ driver can access per-scsi_device HBA private data during
+ tran_start(9E) by using scsi_address_device(9F) and
+ scsi_device_hba_private_get(9F). Any established per-scsi_device
+ HBA private data should be freed during tran_tgt_fini(9E).
sd_dev is a pointer to the corresponding dev_info structure.
- This pointer is initialized by the host adapter driver.
+ This pointer is initialized by the SCSA library code.
sd_mutex is a mutual exclusion lock for this device. It is
used to serialize access to a device. The host adapter
driver initializes this mutex. See mutex(9F).
@@ -51,14 +64,21 @@
sd_sense is initially NULL (zero). If the target driver
wants to use this field for storing REQUEST SENSE data, it
should allocate an scsi_extended_sense(9S) buffer and set
this field to the address of this buffer.
- sd_private is reserved for the use of target drivers and
- should generally be used to point to target specific data
- structures.
+ sd_private is reserved for the use of scsi_device target
+ drivers and should generally be used to point to target
+ specific data structures.
+
+ sd_hba_private is reserved for the use by the SCSA library code
+ and its implementation of scsi_device_hba_private_get(9F) and
+ scsi_hba_device_private_set(9F).
SEE ALSO
detach(9E), probe(9E), makecom(9F), mutex(9F),
- scsi_probe(9F), scsi_extended_sense(9S), scsi_pkt(9S)
+ scsi_address(9S),
+ scsi_probe(9F), scsi_extended_sense(9S), scsi_pkt(9S),
+ scsi_hba_attach_setup(9F),
+ scsi_device_hba_private_get(9F)
Writing Device Drivers
A.4: New to scsi_device_hba_private_get.9f
=========================================
:r scsi_device_hba_private_get.9f
NOTE: used ddi_get_driver_private(9F) as a template...
Kernel Functions for Drivers scsi_device_hba_private_get(9F)
NAME
scsi_device_hba_private_get, scsi_device_hba_private_set - get or
set the address of the HBA drivers per-scsi_device(9S) private
area
SYNOPSIS
#include <sys/conf.h>
#include <sys/ddi.h>
#include <sys/sunddi.h>
void scsi_device_hba_private_set(struct scsi_device *sd, void *data);
void *scsi_device_hba_private_get(struct scsi_device *sd);
INTERFACE LEVEL
Solaris DDI specific (Solaris DDI).
PARAMETERS
scsi_device_hba_private_get()
sd Pointer to scsi_device(9S) structure to get data from.
scsi_device_hba_private_set()
sd Pointer to device information structure to set.
data Data area address to set.
DESCRIPTION
The scsi_device_hba_private_get() function returns the address of
the scsi_device's HBA private data area from the scsi_device(9S)
structure pointed to by sd.
The scsi_device_hba_private_set() function sets the address of the
scsi_device's HBA private data area for the scsi_device(9S)
structure pointed to by sd with the value of data.
RETURN VALUES
The scsi_device_hba_private_get() function returns the contents of
sd_hba_private. If scsi_device_hba_private_set() has not been
previously called with sd, an unpredictable value is returned.
CONTEXT
These functions can be called from user, interrupt, or kernel
context.
SEE ALSO
scsi_device(9S)
A.5: New to scsi_address_device.9f
=========================================
:r scsi_address_device.9f
Kernel Functions for Drivers scsi_address_device(9F)
NAME
scsi_address_device - return scsi_device(9S) pointer given
scsi_address(9S) pointer.
SYNOPSIS
#include <sys/scsi/scsi.h>
struct scsi_device *scsi_address_device(struct scsi_address *ap);
INTERFACE LEVEL
Solaris DDI specific (Solaris DDI).
PARAMETERS
ap Pointer to a scsi_address(9S) structure.
DESCRIPTION
The scsi_address_device() routine returns a pointer to a
scsi_device(9S) structure associated with the specified
scsi_address(9S) structure.
This function can only be used by an host adaptor driver that
calls scsi_hba_attach_setup(9F) with SCSI_HBA_ADDR_COMPLEX.
The ability to determine the scsi_device(9S) associated with a
scsi_address(9S) allows, for example, a host adaptor driver's
tran_start(9E) implementation, which only has access to a
scsi_address(9S) pointer, to gain access to per-scsi_device(9S)
host adaptor driver private data. Given the scsi_device(9S)
structure pointer returned by scsi_address_device(9F), the
per-scsi_device(9S) host adaptor private data can be obtained by
calling scsi_hba_device_private_get(9F). The host adaptor driver
establishes its per-scsi_device(9S) host adaptor driver private
data pointer during tran_tgt_init(9E) by calling
scsi_hba_device_private_set(9F).
RETURN VALUE
Returns a pointer to a scsi_device(9S) structure associated with
the specified scsi_address(9S), or NULL.
CONTEXT
This routine can be called from any context.
EXAMPLES
Example 1
SEE ALSO
scsi_hba_attach_setup(9F), scsi_address(9S), scsi_device(9S),
scsi_device_hba_private_get(9F), scsi_hba_device_private_set(9F),
tran_tgt_init(9E).
ANSI Small Computer System Interface-2 (SCSI-2)
6. Resources and Schedule
6.4. Steering Committee requested information
6.4.1. Consolidation C-team Name:
ON
6.5. ARC review type: FastTrack
6.6. ARC Exposure: open
