Re: [OpenIndiana-discuss] ZVOL (et al) /device node access rights
To follow-up on the described problems (since I am not the only one seeing them), I posted these issues in the illumos bugtracker: * https://www.illumos.org/issues/3283 ZFS RFE: correctly remember device node ownership and ACLs for ZVOLs * https://www.illumos.org/issues/3284 devfs BUG: ACLs on device node can become applied to wrong devices; UID/GID not retained //Jim Klimov 2012-10-14 17:08, Jim Klimov wrote: While updating the Wiki page on virtualization, Edward Ned Harvey wrote of, and brought to my attention, this peculiar situation: A VirtualBox VM can use delegated zvols as dsk or rdsk devices on the host, just like it can use delegated raw disks or partitions, likely iSCSI volumes and other block devices. According to Edward, block devices yield better performance than VDI files for VM disks. A VM can be executed by an unprivileged user, and thus the device node needs to be RW accessible to that non-root user (whom and why to trust - that's the admin's problem, OS should not limit that). So, the problem detected with ZVOLs (and I expect it can have a wider range on other devices) is that the ownership of the device node for a zvol is forgotten upon reboot or other pool reimport. That is, the node used by a VM should be chown'ed upon every VM startup. That's inconvenient, so to say. I played more with this and found that I can also set ACLs with /bin/chmod on device nodes, and that is even remembered across reboots, however with /dev/zvol/*dsk/pool/vol being a dynamically assigned symlink like /devices/pseudo/zfs@0:4(,raw) there is a problem: the symlink and device node is created when I look at it (i.e. upon first ls or another access to the /dev/zvol/... object), and the device node occupies the first available number. The /devices filesystem seems to remember ACL entries (but not ownerships) across reboots only in conjunction with its object names, so upon each reboot (reimport) of the pool, the same device node name can get assigned to different zvols. This is not only useless in terms of stably providing access to certain devices for certain users, but also harmful as after a reboot an unexpected user (among those earlier trusted) can gain access to incorrect devices (and might even enforce that somehow, by being first to access the device at the correct moment) and cause DoS or intentional illicit access to other users' data. So here is the picture as is. I am not sure what exactly to ask, so I guess it's a call for opinions on how the situation can be improved, in terms of remembering correct ownerships and ACLs for those devices (not nodes) that the rights were set for, in order to both increase usability and security of non-root device access. In the particular case of ZVOL devices, I guess attributes can be added to the ZVOLs that would hold the POSIX and ACL access rights and owner:group info (do people agree that is a worthy RFE?). For non-zfs devices like local disk or iscsi or USB - I am not sure if the problem exists the same way (not tested) or how it can be addressed if it exists (some config file for devfs?) Thanks, //Jim Klimov ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] ZVOL (et al) /device node access rights
I can only answer on the first question: Apart from inconveniences discussed above it is stable. In my case I'm using FC based volumes (COMSTAR), directly as raw LUN's and as zvol's. Best Regards Andrej On Tue, Oct 16, 2012 at 3:11 AM, Heinrich van Riel heinrich.vanr...@gmail.com wrote: I have been looking at changing from file to zvol, not for performance reasons. A few questions on using ZVOL 1. Is it as stable as using a file for disk? 2. Is it possible that snapshots sizes could be smaller vs file? (This is my main reason, snapshot replication.) 3. Can this be used with KVM? I am thinking about dropping vbx ose once I have all amd systems replaced. Thanks On Mon, Oct 15, 2012 at 11:49 AM, Andrej Javoršek dr...@ntf.uni-lj.si wrote: Hello, I'm running VB guests as normal (unprivileged) user and I have impression that ownership (I'm using chown on zvol's) is not always lost during reboot. The other part of my comment was joke on my behalf, since restarts of host OS are rare and often I forget to check if guests came up properly. But I completely agree with you that the behaviour is not favorable. Regards Andrej On Mon, Oct 15, 2012 at 4:39 PM, Jim Klimov jimkli...@cos.ru wrote: I am not sure I understood your comment?.. 2012-10-15 11:14, Andrej Javoršek wrote: Hello, I have impression that it is not always necessary to chown raw zvol's. It seems necessary when we need to allow a non-root user to use the zvol directly, such as a backing store for his VM's virtual disk. It happens occasionally on some zvols (and only when I initiate reboot and forget about it) :) What happens? The need to chown? Sorry for misunderstandings if any, //Jim Regards Andrej On Sun, Oct 14, 2012 at 3:08 PM, Jim Klimov j...@cos.ru wrote: While updating the Wiki page on virtualization, Edward Ned Harvey wrote of, and brought to my attention, this peculiar situation: A VirtualBox VM can use delegated zvols as dsk or rdsk devices on the host, just like it can use delegated raw disks or partitions, likely iSCSI volumes and other block devices. According to Edward, block devices yield better performance than VDI files for VM disks. A VM can be executed by an unprivileged user, and thus the device node needs to be RW accessible to that non-root user (whom and why to trust - that's the admin's problem, OS should not limit that). So, the problem detected with ZVOLs (and I expect it can have a wider range on other devices) is that the ownership of the device node for a zvol is forgotten upon reboot or other pool reimport. That is, the node used by a VM should be chown'ed upon every VM startup. That's inconvenient, so to say. I played more with this and found that I can also set ACLs with /bin/chmod on device nodes, and that is even remembered across reboots, however with /dev/zvol/*dsk/pool/vol being a dynamically assigned symlink like /devices/pseudo/zfs@0:4(,raw) there is a problem: the symlink and device node is created when I look at it (i.e. upon first ls or another access to the /dev/zvol/... object), and the device node occupies the first available number. The /devices filesystem seems to remember ACL entries (but not ownerships) across reboots only in conjunction with its object names, so upon each reboot (reimport) of the pool, the same device node name can get assigned to different zvols. This is not only useless in terms of stably providing access to certain devices for certain users, but also harmful as after a reboot an unexpected user (among those earlier trusted) can gain access to incorrect devices (and might even enforce that somehow, by being first to access the device at the correct moment) and cause DoS or intentional illicit access to other users' data. So here is the picture as is. I am not sure what exactly to ask, so I guess it's a call for opinions on how the situation can be improved, in terms of remembering correct ownerships and ACLs for those devices (not nodes) that the rights were set for, in order to both increase usability and security of non-root device access. In the particular case of ZVOL devices, I guess attributes can be added to the ZVOLs that would hold the POSIX and ACL access rights and owner:group info (do people agree that is a worthy RFE?). For non-zfs devices like local disk or iscsi or USB - I am not sure if the problem exists the same way (not tested) or how it can be addressed if it exists (some config file for devfs?) Thanks, //Jim Klimov ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss
Re: [OpenIndiana-discuss] ZVOL (et al) /device node access rights
Hello, I have impression that it is not always necessary to chown raw zvol's. It happens occasionally on some zvols (and only when I initiate reboot and forget about it) :) Regards Andrej On Sun, Oct 14, 2012 at 3:08 PM, Jim Klimov j...@cos.ru wrote: While updating the Wiki page on virtualization, Edward Ned Harvey wrote of, and brought to my attention, this peculiar situation: A VirtualBox VM can use delegated zvols as dsk or rdsk devices on the host, just like it can use delegated raw disks or partitions, likely iSCSI volumes and other block devices. According to Edward, block devices yield better performance than VDI files for VM disks. A VM can be executed by an unprivileged user, and thus the device node needs to be RW accessible to that non-root user (whom and why to trust - that's the admin's problem, OS should not limit that). So, the problem detected with ZVOLs (and I expect it can have a wider range on other devices) is that the ownership of the device node for a zvol is forgotten upon reboot or other pool reimport. That is, the node used by a VM should be chown'ed upon every VM startup. That's inconvenient, so to say. I played more with this and found that I can also set ACLs with /bin/chmod on device nodes, and that is even remembered across reboots, however with /dev/zvol/*dsk/pool/vol being a dynamically assigned symlink like /devices/pseudo/zfs@0:4(,raw) there is a problem: the symlink and device node is created when I look at it (i.e. upon first ls or another access to the /dev/zvol/... object), and the device node occupies the first available number. The /devices filesystem seems to remember ACL entries (but not ownerships) across reboots only in conjunction with its object names, so upon each reboot (reimport) of the pool, the same device node name can get assigned to different zvols. This is not only useless in terms of stably providing access to certain devices for certain users, but also harmful as after a reboot an unexpected user (among those earlier trusted) can gain access to incorrect devices (and might even enforce that somehow, by being first to access the device at the correct moment) and cause DoS or intentional illicit access to other users' data. So here is the picture as is. I am not sure what exactly to ask, so I guess it's a call for opinions on how the situation can be improved, in terms of remembering correct ownerships and ACLs for those devices (not nodes) that the rights were set for, in order to both increase usability and security of non-root device access. In the particular case of ZVOL devices, I guess attributes can be added to the ZVOLs that would hold the POSIX and ACL access rights and owner:group info (do people agree that is a worthy RFE?). For non-zfs devices like local disk or iscsi or USB - I am not sure if the problem exists the same way (not tested) or how it can be addressed if it exists (some config file for devfs?) Thanks, //Jim Klimov __**_ OpenIndiana-discuss mailing list OpenIndiana-discuss@**openindiana.orgOpenIndiana-discuss@openindiana.org http://openindiana.org/**mailman/listinfo/openindiana-**discusshttp://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] ZVOL (et al) /device node access rights
Hello, I'm running VB guests as normal (unprivileged) user and I have impression that ownership (I'm using chown on zvol's) is not always lost during reboot. The other part of my comment was joke on my behalf, since restarts of host OS are rare and often I forget to check if guests came up properly. But I completely agree with you that the behaviour is not favorable. Regards Andrej On Mon, Oct 15, 2012 at 4:39 PM, Jim Klimov jimkli...@cos.ru wrote: I am not sure I understood your comment?.. 2012-10-15 11:14, Andrej Javoršek wrote: Hello, I have impression that it is not always necessary to chown raw zvol's. It seems necessary when we need to allow a non-root user to use the zvol directly, such as a backing store for his VM's virtual disk. It happens occasionally on some zvols (and only when I initiate reboot and forget about it) :) What happens? The need to chown? Sorry for misunderstandings if any, //Jim Regards Andrej On Sun, Oct 14, 2012 at 3:08 PM, Jim Klimov j...@cos.ru wrote: While updating the Wiki page on virtualization, Edward Ned Harvey wrote of, and brought to my attention, this peculiar situation: A VirtualBox VM can use delegated zvols as dsk or rdsk devices on the host, just like it can use delegated raw disks or partitions, likely iSCSI volumes and other block devices. According to Edward, block devices yield better performance than VDI files for VM disks. A VM can be executed by an unprivileged user, and thus the device node needs to be RW accessible to that non-root user (whom and why to trust - that's the admin's problem, OS should not limit that). So, the problem detected with ZVOLs (and I expect it can have a wider range on other devices) is that the ownership of the device node for a zvol is forgotten upon reboot or other pool reimport. That is, the node used by a VM should be chown'ed upon every VM startup. That's inconvenient, so to say. I played more with this and found that I can also set ACLs with /bin/chmod on device nodes, and that is even remembered across reboots, however with /dev/zvol/*dsk/pool/vol being a dynamically assigned symlink like /devices/pseudo/zfs@0:4(,raw) there is a problem: the symlink and device node is created when I look at it (i.e. upon first ls or another access to the /dev/zvol/... object), and the device node occupies the first available number. The /devices filesystem seems to remember ACL entries (but not ownerships) across reboots only in conjunction with its object names, so upon each reboot (reimport) of the pool, the same device node name can get assigned to different zvols. This is not only useless in terms of stably providing access to certain devices for certain users, but also harmful as after a reboot an unexpected user (among those earlier trusted) can gain access to incorrect devices (and might even enforce that somehow, by being first to access the device at the correct moment) and cause DoS or intentional illicit access to other users' data. So here is the picture as is. I am not sure what exactly to ask, so I guess it's a call for opinions on how the situation can be improved, in terms of remembering correct ownerships and ACLs for those devices (not nodes) that the rights were set for, in order to both increase usability and security of non-root device access. In the particular case of ZVOL devices, I guess attributes can be added to the ZVOLs that would hold the POSIX and ACL access rights and owner:group info (do people agree that is a worthy RFE?). For non-zfs devices like local disk or iscsi or USB - I am not sure if the problem exists the same way (not tested) or how it can be addressed if it exists (some config file for devfs?) Thanks, //Jim Klimov ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
