On Sat, Oct 24, 2009 at 4:49 AM, Adam Cheal <ach...@pnimedia.com> wrote:
> The iostat I posted previously was from a system we had already tuned the > zfs:zfs_vdev_max_pending depth down to 10 (as visible by the max of about 10 > in actv per disk). > > I reset this value in /etc/system to 7, rebooted, and started a scrub. > iostat output showed busier disks (%b is higher, which seemed odd) but a cap > of about 7 queue items per disk, proving the tuning was effective. iostat at > a high-water mark during the test looked like this: > > ...and sure enough about 20 minutes into it I get this (bus reset?): > > scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,6...@4 > /pci1000,3...@0/s...@34,0 (sd49): > incomplete read- retrying > scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,6...@4 > /pci1000,3...@0/s...@21,0 (sd30): > incomplete read- retrying > scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,6...@4 > /pci1000,3...@0/s...@1e,0 (sd27): > incomplete read- retrying > scsi: [ID 365881 kern.info] /p...@0,0/pci8086,6...@4/pci1000,3...@0 (mpt0): > Rev. 8 LSI, Inc. 1068E found. > scsi: [ID 365881 kern.info] /p...@0,0/pci8086,6...@4/pci1000,3...@0 (mpt0): > mpt0 supports power management. > scsi: [ID 365881 kern.info] /p...@0,0/pci8086,6...@4/pci1000,3...@0 (mpt0): > mpt0: IOC Operational. > > During the "bus reset", iostat output looked like this: > > > During our previous testing, we had tried even setting this max_pending > value down to 1, but we still hit the problem (albeit it took a little > longer to hit it) and I couldn't find anything else I could set to throttle > IO to the disk, hence the frustration. > > If you hadn't seen this output, would you say that 7 was a "reasonable" > value for that max_pending queue for our architecture and should give the > LSI controller in this situation enough breathing room to operate? If so, I > *should* be able to scrub the disks successfully (ZFS isn't to blame) and > therefore have to point the finger at the > mpt-driver/LSI-firmware/disk-firmware instead. > -- > > A little bit of searching google says: http://downloadmirror.intel.com/17968/eng/ESRT2_IR_readme.txt
_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
