Re: [systemd-devel] remote-fs ordering, iSCSI and _netdev
On Tuesday 28 October 2014 at 06:41:32, Andrei Borzenkov wrote: В Mon, 27 Oct 2014 14:10:47 -0700 Chris Leech cle...@redhat.com пишет: At boot fstab-generator is picking up on the _netdev option in fstab, and the generated mount units are ordered against remote-fs properly. If I leave a filesystem mounted at shutdown, it will be unmounted before the iSCSI session is destroyed or the network is shut down and everything works as expected. But there are two cases that are problematic, adding entries to fstab at runtime and manually mounting without adding to fstab (while still using the _netdev option, some hint is needed). The first case actually ends up being the second, with the possible work-around of always remembering to run a daemon-reload after editing fstab to run fstab-generator again. Even known network filesystems still have a problem. If network filesystem is mounted on boot, it pulls in network-online.target which (hopefully) serves as synchronization point on shutdown. If there is no network filesystem to mount at boot, network-online.target is not started. If you mount NFS manually later there is nothing to wait for on shutdown so network could be teared down before filesystem is unmounted. Isn't this (unmount before teardown) achieved by After=network.target? That target is passive, so it is pulled in by a provider, and all should work even in case of manually mounted filesystems. Am I wrong somewhere? -- Ivan Shapovalov / intelfx / signature.asc Description: This is a digitally signed message part. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] remote-fs ordering, iSCSI and _netdev
В Sun, 09 Nov 2014 20:15:56 +0300 Ivan Shapovalov intelfx...@gmail.com пишет: On Tuesday 28 October 2014 at 06:41:32, Andrei Borzenkov wrote: В Mon, 27 Oct 2014 14:10:47 -0700 Chris Leech cle...@redhat.com пишет: At boot fstab-generator is picking up on the _netdev option in fstab, and the generated mount units are ordered against remote-fs properly. If I leave a filesystem mounted at shutdown, it will be unmounted before the iSCSI session is destroyed or the network is shut down and everything works as expected. But there are two cases that are problematic, adding entries to fstab at runtime and manually mounting without adding to fstab (while still using the _netdev option, some hint is needed). The first case actually ends up being the second, with the possible work-around of always remembering to run a daemon-reload after editing fstab to run fstab-generator again. Even known network filesystems still have a problem. If network filesystem is mounted on boot, it pulls in network-online.target which (hopefully) serves as synchronization point on shutdown. If there is no network filesystem to mount at boot, network-online.target is not started. If you mount NFS manually later there is nothing to wait for on shutdown so network could be teared down before filesystem is unmounted. Isn't this (unmount before teardown) achieved by After=network.target? That target is passive, so it is pulled in by a provider, and all should work even in case of manually mounted filesystems. Am I wrong somewhere? Actually no. Now when I revisited the problem, it was misconfiguration on openSUSE side and misunderstanding on mine :) signature.asc Description: PGP signature ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] remote-fs ordering, iSCSI and _netdev
On Thu, Nov 06, 2014 at 09:10:51PM -0800, Chris Leech wrote: Not sure, maybe it's possible to detect this by scsi info in /sys. I took a look at what lsscsi is doing to guess at transport type. iSCSI is kind of ugly, FCoE is really ugly, and for both of those there exists a variety of drivers with varying levels of dependence on the networking layer. It would be nice to work on getting some of the more common works working automatically, but _netdev seems like a nice failsafe. BTW, in lsblk(8) we have TRANS column (lsblk -S, function get_transport()) to report scsi transport type and it uses /sys, but I have no clue how reliable it's. Karel -- Karel Zak k...@redhat.com http://karelzak.blogspot.com ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] remote-fs ordering, iSCSI and _netdev
On Fri, Oct 31, 2014 at 09:32:57AM +0100, Karel Zak wrote: It would be really better to have within systemd a generic function is_net_blkdev() than rely on external fragile configuration. I have doubts that anyone uses -o _netdev on command line when manually mounts filesystem. The one thing I do like about _netdev, is that if it works is a simple workaround that can be documented for when more complicated detection schemes fail. Rather than trying to keep a list of all known network filesystem and block protocols up to date. Not sure, maybe it's possible to detect this by scsi info in /sys. I took a look at what lsscsi is doing to guess at transport type. iSCSI is kind of ugly, FCoE is really ugly, and for both of those there exists a variety of drivers with varying levels of dependence on the networking layer. It would be nice to work on getting some of the more common works working automatically, but _netdev seems like a nice failsafe. I'm sending an RFC patchset with a start at addressing this. - Chris ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] remote-fs ordering, iSCSI and _netdev
On Thu, Oct 30, 2014 at 03:04:59PM -0700, Chris Leech wrote: On Thu, Oct 30, 2014 at 12:10:16PM +0100, Karel Zak wrote: On Tue, Oct 28, 2014 at 02:29:35AM +0100, Lennart Poettering wrote: On Mon, 27.10.14 14:10, Chris Leech (cle...@redhat.com) wrote: So for any mounts to remote block devices (unlike remote file system protocols which are detected by the fs name), unless there is an fstab entry at the time fstab-generator is run they get treated like local fs mounts and connectivity to the storage target may be disrupted before unmounting (possibly resulting in file system errors). I'm currently at a loss for how to handle this, other than to claim that if filesystems are going to be left mounted they should be added to fstab and a daemon-reload is required. IIRC mount nowadays stores the full mount option string, including all the userspace-only options in /run. We could either read those directly from there in systemd, or we could make systemd make use of libmount to get that information. _netdev is information about device rather than about filesystem. Would be possible to have this info (this is iSCSI) in udev db? Yes, the _netdev option is ugly. For iSCSI specifically, we'd have to trace the block device back to the scsi_host, then match that up with an iscsi_host from the transport class. Or come up with some change to make that process easier. And it would need to work for dm/md device over the actually scsi device. It would be really better to have within systemd a generic function is_net_blkdev() than rely on external fragile configuration. I have doubts that anyone uses -o _netdev on command line when manually mounts filesystem. Not sure, maybe it's possible to detect this by scsi info in /sys. Karel -- Karel Zak k...@redhat.com http://karelzak.blogspot.com ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] remote-fs ordering, iSCSI and _netdev
On Tue, Oct 28, 2014 at 02:29:35AM +0100, Lennart Poettering wrote: On Mon, 27.10.14 14:10, Chris Leech (cle...@redhat.com) wrote: So for any mounts to remote block devices (unlike remote file system protocols which are detected by the fs name), unless there is an fstab entry at the time fstab-generator is run they get treated like local fs mounts and connectivity to the storage target may be disrupted before unmounting (possibly resulting in file system errors). I'm currently at a loss for how to handle this, other than to claim that if filesystems are going to be left mounted they should be added to fstab and a daemon-reload is required. IIRC mount nowadays stores the full mount option string, including all the userspace-only options in /run. We could either read those directly from there in systemd, or we could make systemd make use of libmount to get that information. _netdev is information about device rather than about filesystem. Would be possible to have this info (this is iSCSI) in udev db? You know, all userpsace mount options suck, it's always fragile to maintain mount options in userspace (due to namespaces, ...) Karel, what are the details there? Would it be OK if we read the files in /run directly to augment whatever we got from /proc/self/mountinfo? I'd like to keep /run/mount/utab as private libmount file. It would be better to use mnt_table_parse_mtab() libmount function to get parsed mountinfo + userspace mount options. IMHO you can implement it as optional feature #ifdef HAVE_LIBMOUNT to optionally use mnt_table_parse_mtab() from libmount rather than directly parse /proc/self/mountinfo. Karel -- Karel Zak k...@redhat.com http://karelzak.blogspot.com ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] remote-fs ordering, iSCSI and _netdev
On Thu, Oct 30, 2014 at 12:10:16PM +0100, Karel Zak wrote: On Tue, Oct 28, 2014 at 02:29:35AM +0100, Lennart Poettering wrote: On Mon, 27.10.14 14:10, Chris Leech (cle...@redhat.com) wrote: So for any mounts to remote block devices (unlike remote file system protocols which are detected by the fs name), unless there is an fstab entry at the time fstab-generator is run they get treated like local fs mounts and connectivity to the storage target may be disrupted before unmounting (possibly resulting in file system errors). I'm currently at a loss for how to handle this, other than to claim that if filesystems are going to be left mounted they should be added to fstab and a daemon-reload is required. IIRC mount nowadays stores the full mount option string, including all the userspace-only options in /run. We could either read those directly from there in systemd, or we could make systemd make use of libmount to get that information. _netdev is information about device rather than about filesystem. Would be possible to have this info (this is iSCSI) in udev db? Yes, the _netdev option is ugly. For iSCSI specifically, we'd have to trace the block device back to the scsi_host, then match that up with an iscsi_host from the transport class. Or come up with some change to make that process easier. And it would need to work for dm/md device over the actually scsi device. You know, all userpsace mount options suck, it's always fragile to maintain mount options in userspace (due to namespaces, ...) Karel, what are the details there? Would it be OK if we read the files in /run directly to augment whatever we got from /proc/self/mountinfo? I'd like to keep /run/mount/utab as private libmount file. It would be better to use mnt_table_parse_mtab() libmount function to get parsed mountinfo + userspace mount options. IMHO you can implement it as optional feature #ifdef HAVE_LIBMOUNT to optionally use mnt_table_parse_mtab() from libmount rather than directly parse /proc/self/mountinfo. Thanks for the direction, I've worked up a patch that does this and using libmount here was pretty easy. Unfortunatly with systemd triggering off of the change event from /proc/self/mountinfo, it ends up racing with the mount command to access /run/mount/utab and the _netdev option I'm looking for isn't consistantly picked up. I'll see what else I can come up with. - Chris ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] remote-fs ordering, iSCSI and _netdev
On Tue, Oct 28, 2014 at 06:41:32AM +0300, Andrei Borzenkov wrote: В Mon, 27 Oct 2014 14:10:47 -0700 Chris Leech cle...@redhat.com пишет: But there are two cases that are problematic, adding entries to fstab at runtime and manually mounting without adding to fstab (while still using the _netdev option, some hint is needed). The first case actually ends up being the second, with the possible work-around of always remembering to run a daemon-reload after editing fstab to run fstab-generator again. Even known network filesystems still have a problem. If network filesystem is mounted on boot, it pulls in network-online.target which (hopefully) serves as synchronization point on shutdown. If there is no network filesystem to mount at boot, network-online.target is not started. If you mount NFS manually later there is nothing to wait for on shutdown so network could be teared down before filesystem is unmounted. Hmm, I hadn't noticed that with iSCSI because a service gets started to connect to the target so the dependencies can be taken care of there. Should the remote mount unit be generating a Wants dependency along with the Before/After to ensure the synchronization point targets are active? Or would that not work for some reason? - Chris ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] remote-fs ordering, iSCSI and _netdev
On Tue, Oct 28, 2014 at 01:57:06AM +0100, Zbigniew Jędrzejewski-Szmek wrote: On Mon, Oct 27, 2014 at 02:10:47PM -0700, Chris Leech wrote: ... If there's no matching mount unit from fstab-generator, one gets created dynamically when the fs is mounted by monitoring /proc/self/mountinfo. Actually, it is more correct to say that a unit *always* get created based on /proc/self/mountinfo. If there was a unit previously, it is replaced by the new one, but inherits the dependencies. In effect it leads to the behaviour you described. Thanks for making that clear. So for any mounts to remote block devices (unlike remote file system protocols which are detected by the fs name), unless there is an fstab entry at the time fstab-generator is run they get treated like local fs mounts and connectivity to the storage target may be disrupted before unmounting (possibly resulting in file system errors). Yes, that seems right. It seems reasonable to change the code which generates units based on /p/s/mounintinfo to behave as if _netdev option was specified, for the known network filesystem types. That's in place and (I'm haven't been testing it but I think) working. The problem is with network block devices where the fstype is the on-disk filesystem. - Chris ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] remote-fs ordering, iSCSI and _netdev
On Thu, Oct 30, 2014 at 03:09:25PM -0700, Chris Leech wrote: On Tue, Oct 28, 2014 at 01:57:06AM +0100, Zbigniew Jędrzejewski-Szmek wrote: On Mon, Oct 27, 2014 at 02:10:47PM -0700, Chris Leech wrote: ... If there's no matching mount unit from fstab-generator, one gets created dynamically when the fs is mounted by monitoring /proc/self/mountinfo. Actually, it is more correct to say that a unit *always* get created based on /proc/self/mountinfo. If there was a unit previously, it is replaced by the new one, but inherits the dependencies. In effect it leads to the behaviour you described. Thanks for making that clear. So for any mounts to remote block devices (unlike remote file system protocols which are detected by the fs name), unless there is an fstab entry at the time fstab-generator is run they get treated like local fs mounts and connectivity to the storage target may be disrupted before unmounting (possibly resulting in file system errors). Yes, that seems right. It seems reasonable to change the code which generates units based on /p/s/mounintinfo to behave as if _netdev option was specified, for the known network filesystem types. That's in place and (I'm haven't been testing it but I think) working. The problem is with network block devices where the fstype is the on-disk filesystem. Sorry, I don't know too much about iscsi. Is it easy to tell that the device requires network to function? Some udev tag or property that could be easily queried? Zbyszek ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] remote-fs ordering, iSCSI and _netdev
В Fri, 31 Oct 2014 01:53:26 +0100 Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl пишет: On Thu, Oct 30, 2014 at 03:09:25PM -0700, Chris Leech wrote: On Tue, Oct 28, 2014 at 01:57:06AM +0100, Zbigniew Jędrzejewski-Szmek wrote: On Mon, Oct 27, 2014 at 02:10:47PM -0700, Chris Leech wrote: ... If there's no matching mount unit from fstab-generator, one gets created dynamically when the fs is mounted by monitoring /proc/self/mountinfo. Actually, it is more correct to say that a unit *always* get created based on /proc/self/mountinfo. If there was a unit previously, it is replaced by the new one, but inherits the dependencies. In effect it leads to the behaviour you described. Thanks for making that clear. So for any mounts to remote block devices (unlike remote file system protocols which are detected by the fs name), unless there is an fstab entry at the time fstab-generator is run they get treated like local fs mounts and connectivity to the storage target may be disrupted before unmounting (possibly resulting in file system errors). Yes, that seems right. It seems reasonable to change the code which generates units based on /p/s/mounintinfo to behave as if _netdev option was specified, for the known network filesystem types. That's in place and (I'm haven't been testing it but I think) working. The problem is with network block devices where the fstype is the on-disk filesystem. Sorry, I don't know too much about iscsi. Is it easy to tell that the device requires network to function? Some udev tag or property that could be easily queried? It does not help in case of layered configuration (LVM on MD on iSCSI). one would need to go all the way to find out if some of devices involved may depend on network. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] remote-fs ordering, iSCSI and _netdev
В Thu, 30 Oct 2014 15:15:03 -0700 Chris Leech cle...@redhat.com пишет: On Tue, Oct 28, 2014 at 06:41:32AM +0300, Andrei Borzenkov wrote: В Mon, 27 Oct 2014 14:10:47 -0700 Chris Leech cle...@redhat.com пишет: But there are two cases that are problematic, adding entries to fstab at runtime and manually mounting without adding to fstab (while still using the _netdev option, some hint is needed). The first case actually ends up being the second, with the possible work-around of always remembering to run a daemon-reload after editing fstab to run fstab-generator again. Even known network filesystems still have a problem. If network filesystem is mounted on boot, it pulls in network-online.target which (hopefully) serves as synchronization point on shutdown. If there is no network filesystem to mount at boot, network-online.target is not started. If you mount NFS manually later there is nothing to wait for on shutdown so network could be teared down before filesystem is unmounted. Hmm, I hadn't noticed that with iSCSI because a service gets started to connect to the target so the dependencies can be taken care of there. Should the remote mount unit be generating a Wants dependency along with the Before/After to ensure the synchronization point targets are active? Or would that not work for some reason? Wants works only when unit is started but it does not happen here - you manually do mount server:/path /mnt and while unit gets created afterwards, there is no unit startup phase in systemd sense so Wants does not kick in. May be it could be extended similar to what happens with device - device cannot be started as well, but systemd fakes startup when device appears. We could do the same for mount units newly appeared. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] remote-fs ordering, iSCSI and _netdev
On Mon, Oct 27, 2014 at 02:10:47PM -0700, Chris Leech wrote: Hi, I was hoping someone could help me make sure I'm not overlooking something with trying to manage mounts on iSCSI disks. I have an iscsi.service which starts and stops sessions to iSCSI targets. It's set with Before=remote-fs-pre.target and Wants=remote-fs-pre.target to ensure that remote fs ordering is enabled. Unfortunately mount points only get configured as remote if there is a record in /etc/fstab at the time fstab-generator is run. At boot fstab-generator is picking up on the _netdev option in fstab, and the generated mount units are ordered against remote-fs properly. If I leave a filesystem mounted at shutdown, it will be unmounted before the iSCSI session is destroyed or the network is shut down and everything works as expected. But there are two cases that are problematic, adding entries to fstab at runtime and manually mounting without adding to fstab (while still using the _netdev option, some hint is needed). The first case actually ends up being the second, with the possible work-around of always remembering to run a daemon-reload after editing fstab to run fstab-generator again. If there's no matching mount unit from fstab-generator, one gets created dynamically when the fs is mounted by monitoring /proc/self/mountinfo. Actually, it is more correct to say that a unit *always* get created based on /proc/self/mountinfo. If there was a unit previously, it is replaced by the new one, but inherits the dependencies. In effect it leads to the behaviour you described. So for any mounts to remote block devices (unlike remote file system protocols which are detected by the fs name), unless there is an fstab entry at the time fstab-generator is run they get treated like local fs mounts and connectivity to the storage target may be disrupted before unmounting (possibly resulting in file system errors). Yes, that seems right. It seems reasonable to change the code which generates units based on /p/s/mounintinfo to behave as if _netdev option was specified, for the known network filesystem types. I'm currently at a loss for how to handle this, other than to claim that if filesystems are going to be left mounted they should be added to fstab and a daemon-reload is required. That is always an option, but I think that in this case a simple patch will be nicer. Zbyszek ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] remote-fs ordering, iSCSI and _netdev
On Mon, 27.10.14 14:10, Chris Leech (cle...@redhat.com) wrote: So for any mounts to remote block devices (unlike remote file system protocols which are detected by the fs name), unless there is an fstab entry at the time fstab-generator is run they get treated like local fs mounts and connectivity to the storage target may be disrupted before unmounting (possibly resulting in file system errors). I'm currently at a loss for how to handle this, other than to claim that if filesystems are going to be left mounted they should be added to fstab and a daemon-reload is required. IIRC mount nowadays stores the full mount option string, including all the userspace-only options in /run. We could either read those directly from there in systemd, or we could make systemd make use of libmount to get that information. Karel, what are the details there? Would it be OK if we read the files in /run directly to augment whatever we got from /proc/self/mountinfo? Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
