I have a fairly large set of changes to the sa(4) driver and mt(1) driver
that I'm planning to commit in the near future.

A description of the changes is here and below in this message.

If you have tape hardware and the inclination, I'd appreciate testing and

Rough draft commit message:


The patches against FreeBSD/head as of SVN revision 278706:


And (untested) patches against FreeBSD stable/10 as of SVN revision 278721.


The intent is to get the tape infrastructure more up to date, so we can
support LTFS and more modern tape drives:


I have ported IBM's LTFS Single Drive Edition to FreeBSD.  The port depends
on the patches linked above.  It isn't fully cleaned up and ready for
redistribution.  If you're interested, though, let me know and I'll tell
you when it is ready to go out.  You need an IBM LTO-5, LTO-6, TS1140 or
TS1150 tape drive.  HP drives aren't supported by IBM's LTFS, and older
drives don't have the necessary features to support LTFS.

The commit message below outlines most of the changes.

A few comments:

1. I'm planning to commit the XPT_DEV_ADVINFO changes separately.

2. The XML output is similar to what GEOM and CTL do.  It would be nice to
   figure out how to put a standard schema on it so that standard tools
   could read it.  I don't know how feasible that is, since I haven't
   time to dig into it.  If anyone has suggestions on whether that is
   feasible or advisable, I'd appreciate feedback.

3. I have tested with a reasonable amount of tape hardware (see below for a
   list), but more testing and feedback would be good.

4. Standard 'mt status' output looks like this:

# mt -f /dev/nsa3 status  -v
Drive: sa3: <IBM ULTRIUM-HH6 E4J1> Serial Number: 101500520A
Mode      Density              Blocksize      bpi      Compression
Current:  0x5a:LTO-6           variable       384607   enabled (0xff)
Current Driver State: at rest.
Partition:   0      Calc File Number:   0     Calc Record Number: 0
Residual:    0  Reported File Number:   0 Reported Record Number: 0
Flags: BOP

5. 'mt status -v' looks like this:

# mt -f /dev/nsa3 status  -v
Drive: sa3: <IBM ULTRIUM-HH6 E4J1> Serial Number: 101500520A
Mode      Density              Blocksize      bpi      Compression
Current:  0x5a:LTO-6           variable       384607   enabled (0xff)
Current Driver State: at rest.
Partition:   0      Calc File Number:   0     Calc Record Number: 0
Residual:    0  Reported File Number:   0 Reported Record Number: 0
Flags: BOP
Tape I/O parameters:
  Maximum I/O size allowed by driver and controller (maxio): 1081344 bytes
  Maximum I/O size reported by controller (cpi_maxio): 5197824 bytes
  Maximum block size supported by tape drive and media (max_blk): 8388608 bytes
  Minimum block size supported by tape drive and media (min_blk): 1 bytes
  Block granularity supported by tape drive and media (blk_gran): 0 bytes
  Maximum possible I/O size (max_effective_iosize): 1081344 bytes

6. Existing applications should work without changes.  If not, please let
   me know.  Hopefully they will move over time to the new interfaces.

7. There are lots of additional features that could be added later.
   Append-only support, encryption, more log pages, etc.

8. I have SCSI READ ATTRIBUTE changes for camcontrol(8) that will go in
   separately.  These changes allow displaying the contents of the MAM
   (Medium Auxiliary Memory) chips on LTO, TS and other modern tape drives.
   These are good, and a future possible direction is adding attributes 
   to the status XML from the sa(4) driver.

Significant upgrades to sa(4) and mt(1).

The primary focus of these changes is to modernize FreeBSD's
tape infrastructure so that we can take advantage of some of the
features of modern tape drives and allow support for LTFS.

Significant changes and new features include:

 o sa(4) driver status and parameter information is now exported via an
   XML structure.  This will allow for changes and improvements later
   on that will not break userland applications.  The old MTIOCGET
   status ioctl remains, so applications using the existing interface
   will not break.

 o 'mt status' now reports drive-reported tape position information
   as well as the previously available calculated tape position
   information.  These numbers will be different at times, because
   the drive-reported block numbers are relative to BOP (Beginning
   of Partition), but the block numbers calculated previously via
   sa(4) (and still provided) are relative to the last filemark.
   Both numbers are now provided.  'mt status' now also shows the
   drive INQUIRY information, serial number and any position flags
   (BOP, EOT, etc.) provided with the tape position information.
   'mt status -v' adds information on the maximum possible I/O size,
   and the underlying values used to calculate it.

 o The extra sa(4) /dev entries (/dev/saN.[0-3]) have been removed.

   The extra devices were originally added as place holders for
   density-specific device nodes.  Some OSes (NetBSD, NetApp's OnTap
   and Solaris) have had device nodes that, when you write to them,
   will automatically select a given density for particular tape drives.
   This is a convenient way of switching densities, but it was never
   implemented in FreeBSD.  Only the device nodes were there, and that
   sometimes confused users.
   For modern tape devices, the density is generally not selectable
   (e.g. with LTO) or defaults to the highest availble density when
   the tape is rewritten from BOT (e.g. TS11X0).  So, for most users,
   density selection won't be necessary.  If they do need to select
   the density, it is easy enough to use 'mt density' to change it.

 o Protection information is now supported.  This is either a
   Reed-Solomon CRC or CRC32 that is included at the end of each block
   read and written.  On write, the tape drive verifies the CRC, and
   on read, the tape drive provides a CRC for the userland application
   to verify.

 o New, extensible tape driver parameter get/set interface.

 o Density reporting information.  For drives that support it,
   'mt getdensity' will show detailed information on what formats the
   tape drive supports, and what formats the tape drive supports.

 o Some mt(1) functionality moved into a new mt(3) library so that
   external applications can reuse the code.

 o The new mt(3) library includes helper routines to aid in parsing
   the XML output of the sa(4) driver, and build a tree of driver

 o Support for the MTLOAD (load a tape in the drive) and MTWEOFI
   (write filemark immediate) ioctls needed by IBM's LTFS

 o Improve device departure behavior for the sa(4) driver.  The previous
   implementation led to hangs when the device was open.

 o This has been tested on the following types of drives:
        IBM TS1150
        IBM TS1140
        IBM LTO-6
        IBM LTO-5
        HP LTO-2
        Seagate DDS-4
        Quantum DLT-4000
        Exabyte 8505
        Sony DDS-2

        Add libmt.

        New mt(3) library that contains functions moved from mt(1) and
        new functions needed to interact with the updated sa(4) driver.

        This includes XML parser helper functions that application writers
        can use when writing code to query tape parameters.

        Add -lmt to CRUNCH_LIBS.

        Add a new flag value for the XPT_DEV_ADVINFO CCB, CDAI_FLAG_NONE.

        Make sure the flags for the XPT_DEV_ADVINFO CCB are set correctly.
        This prevents unintended attempts to set advanced information
        values when XPT_DEV_ADVINFO CCBs are not pre-zeroed.

        Clarify this man page a bit, and since it contains what is
        essentially the mtio.h header file, add new ioctls and structure
        definitions from mtio.h.

        Update BUGS and maintainer section.

        Add SCSI SECURITY PROTOCOL IN/OUT CDB definitions and CDB building

        Many tape driver changes, largely outlined above.

        Increase the sa(4) driver read/write timeout from 4 to 32
        minutes.  This is based on the recommended values for IBM LTO
        5/6 drives.  This may also avoid timeouts for other tape
        hardware that can take a long time to do retries and error
        recovery.  Longer term, a better way to handle this is to ask
        the drive for recommended timeout values using the REPORT
        SUPPORTED OPCODES command.  Modern IBM and Oracle tape drives
        at least support that command, and it would allow for more
        accurate timeout values.

        Add XML status generation.  This is done with a series of
        macros to eliminate as much duplicate code as possible.  The
        new XML-based status values are reported through the new
        MTIOCEXTGET ioctl.

        Add XML driver parameter reporting, using the new MTIOCPARAMGET

        Add a new driver parameter setting interface, using the new

        Add a new MTIOCRBLIM ioctl to get block limits information.

        Add CCB/CDB building routines scsi_locate_16, scsi_locate_10,
        and scsi_read_position_10().
        scsi_locate_10 implements the LOCATE command, as does the
        existing scsi_set_position() command.  It just supports
        additional arguments and features.  If/when we figure out a
        good way to provide backward compatibility for older
        applications using the old function API, we can just revamp
        scsi_set_position().  The same goes for
        scsi_read_position_10() and the existing scsi_read_position()
        Revamp sasetpos() to take the new mtlocate structure as an
        argument.  It now will use either scsi_locate_10() or
        scsi_locate_16(), depending upon the arguments the user
        supplies.  As before, once we change position we don't have a
        clear idea of what the current logical position of the tape
        drive is.

        For tape drives that support long form position data, we
        read the current position and store that for later reporting
        after changing the position.  This should help applications
        like Bacula speed tape access under FreeBSD once they are
        modified to support the new ioctls.
        Add a new quirk, SA_QUIRK_NO_LONG_POS, that is set for all
        drives that report SCSI-2 or older, as well as drives that
        report an Illegal Request type error for READ POSITION with
        the long format.  So we should automatically detect drives
        that don't support the long form and stop asking for it after
        an initial try.

        Add a partition number to the sa(4) softc.

        Improve device departure handling. The previous implementation
        led to hangs when the device was open.

        If an application had the sa(4) driver open, and attempted to
        close it after it went away, the cam_periph_release() call in
        saclose() would cause the periph to get destroyed because that
        was the last reference to it.  Because destroy_dev() was
        called from the sa(4) driver's cleanup routine (sacleanup()),
        and would block waiting for the close to happen, a deadlock
        would result.

        So instead of calling destroy_dev() from the cleanup routine,
        call destroy_dev_sched_cb() from saoninvalidate() and wait for
        the callback.
        Acquire a reference for devfs in saregister(), and release it
        in the new sadevgonecb() routine when all devfs devices for     
        the particular sa(4) driver instance are gone.
        Add a new function, sasetupdev(), to centralize setting
        per-instance devfs device parameters instead of repeating the
        code in saregister().

        Add an open count to the softc, so we know how many
        peripheral driver references are a result of open
        Add the D_TRACKCLOSE flag to the cdevsw flags so
        that we get a 1:1 mapping of open to close calls
        instead of a N:1 mapping.
        This should be a no-op for everything except the
        control device, since we don't allow more than one
        open on non-control devices.
        However, since we do allow multiple opens on the
        control device, the combination of the open count
        and the D_TRACKCLOSE flag should result in an
        accurate peripheral driver reference count, and an
        accurate open count.
        The accurate open count allows us to release all
        peripheral driver references that are the result
        of open contexts once we get the callback from devfs.

        Add a number of new mt(4) ioctls and the requisite data
        structures.  None of the existing interfaces been removed
        or changed.

        This includes definitions for the following new ioctls:

        MTIOCRBLIM      /* get block limits */
        MTIOCEXTLOCATE  /* seek to position */
        MTIOCEXTGET     /* get tape status */
        MTIOCPARAMGET   /* get tape params */
        MTIOCPARAMSET   /* set tape params */
        MTIOCSETLIST    /* set N params */

        mt(1) now depends on libmt, libsbuf and libbsdxml.

        Document new mt(1) features and subcommands.

        Implement support for mt(1) subcommands that need to
        use getopt(3) for their arguments.
        Implement a new 'mt status' command to replace the old
        'mt status' command.  The old status command has been
        renamed 'ostatus'.

        The new status function uses the MTIOCEXTGET ioctl, and
        therefore parses the XML data to determine drive status.
        The -x argument to 'mt status' allows the user to dump out
        the raw XML reported by the kernel.

        The new status display is mostly the same as the old status
        display, except that it doesn't print the redundant density
        mode information, and it does print the current partition
        number and position flags.

        Add a new command, 'mt locate', that will supersede the
        old 'mt setspos' and 'mt sethpos' commands.  'mt locate'
        implements all of the functionality of the MTIOCEXTLOCATE
        ioctl, and allows the user to change the logical position
        of the tape drive in a number of ways.  (Partition,
        block number, file number, set mark number, end of data.)
        The immediate bit and the explicit address bits are
        implemented, but not documented in the man page.

        Add a new 'mt weofi' command to use the new MTWEOFI ioctl.
        This allows the user to ask the drive to write a filemark
        without waiting around for the operation to complete.

        Add a new 'mt getdensity' command that gets the XML-based
        tape drive density report from the sa(4) driver and displays
        it.  This uses the SCSI REPORT DENSITY SUPPORT command
        to get comprehensive information from the tape drive about
        what formats it is able to read and write.

        Add a new 'mt protect' command that allows getting and setting
        tape drive protection information.  The protection information
        is a CRC tacked on to the end of every read/write from and to
        the tape drive.

Sponsored by:   Spectra Logic
MFC after:      1 month


Kenneth Merry
freebsd-current@freebsd.org mailing list
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Reply via email to