On Thu, Feb 25, 2010 at 02:55:21PM -0800, [email protected] wrote: > On Thu, Feb 25, 2010 at 02:27:25PM -0800, Danek Duvall wrote: > > Shawn Walker wrote: > > > > > On 02/25/10 12:09 PM, Kristin Amundsen-Cubanski wrote: > > > > > > >ioctl(4, ZFS_IOC_OBJSET_STATS, 0x08043060) = 0 > > > >ioctl(4, ZFS_IOC_OBJSET_STATS, 0x08043060) Err#12 ENOMEM > > > >ioctl(4, ZFS_IOC_OBJSET_STATS, 0x08043060) = 0 > > > >ioctl(4, ZFS_IOC_SNAPSHOT_LIST_NEXT, 0x08043480) = 0 > > > >ioctl(4, ZFS_IOC_OBJSET_STATS, 0x08041FE0) = 0 > > > > > > ENOMEM suggests a memory problem in my naive evaluation, but perhaps > > > another pkg team member has another idea. > > > > This really needs to be asked on zfs-discuss. > > Perhaps, but even asking on zfs-discuss may not narrow the problem down > enough to solve it. This could be a problem with beadm not properly > handling an error from a zfs command. Or, it could be a problem in > libzfs, or even something else. > > The limited information we have shows this as a weird interaction > between the device that's invoking the ioctl(), and the data that's been > received. > > In particular, the ZFS_IOC_OBJSET_STATS ioctl is implemented by > zfs_ioc_objset_stats(). That code calls put_nvlist(), which can return > ENOMEM if the zfs_cmd_t's nvlist doesn't contain enough space for the > result from the dmu_objset_stats() call. The call to put_nvlist() > copies the data out of the kernel into a user buffer. These sizes must > match, or the kernel will overwrite arbitrary user data with the result > of the stat call. To make matters interesting, each block device > defines the maximum size of its nvlist. The Comstar sbd hard codes its > max size to 1024 in sbd_zvolget(). The zvol code itself picks 8000 in > devzvol_handle_ioctl(). > > To debug this properly, we need to figure what user code is calling this > ioctl and where the error handling has gone wrong. It would also be > interesting to know if iSCSI/Comstar is in use on this installation, as > the small buffer size may be exacerbating this problem.
In fact, this failure may not be fatal in any of these cases. Libzfs, pyzfs, sbd, and devzovl all use a starting size for the destination buffer. If the ioctl returns ENOMEM, it also sets the zfs_cmd_t to contain the size of the buffer needed to accomodate the desired nvlist. The code that calls these routines detects this error case, allocates a larger buffer, and tries again. This is why there's a successful call to ZFS_IOC_OBJSET_STATS after the ENOMEM. It looks like whatever is wrong, it's not caused by the ENOMEM from the ZFS_IOC_OBJSET_STATS call. -j _______________________________________________ pkg-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
