On Fri, Mar 21, 2014 at 4:05 PM, Wouter Verhelst <wou...@debian.org> wrote: > > First, let me explain how NBD works. > > The client side of the NBD protocol is implemented partially in > user space, and partially in kernel space. The user space part handles > connecting and the initial protocol negotiation; but once that has been > done, nbd-client calls the NBD_DO_IT ioctl() on an open /dev/nbdX file, > which hands the socket file descriptor to the kernel and which does not > return until the device is disconnected (with "nbd-client -d", or > because the link to the server died). As such, the nbd-client process > needs to continue running while the device is connected. > > In addition, nbd-client needs to fork() and open() the /dev/nbdX device > to support partitioned NBD devices (due to a deadlock issue, that can't > be done from the initial NBD_DO_IT ioctl handling, so it is done in the > first open() instead). >
But the parent nbd-client remains around, right? > For supporting root-on-NBD in conjunction with systemd, I've already > added a -systemd-mark option to nbd-client so it will make argv[0][0] > read as '@' (I think that method is slightly ugly, but that's a > discussion for another time). In Debian, I've already supported > root-on-NBD for quite a while with an initramfs script and some code in > the init script of nbd-client which adds the PID for the root NBD device > to the list of PIDs that shouldn't be killed; I understand that dracut > (and hence Fedora as well) have similar support (though I'm not sure how > well it all works). > > For non-root NBD devices, however, the situation would be slightly more > complex. This is supported in Debian by the package's init script; > AFAIK, though, no other distribution has support for that in its init > scripts (or upstart/systemd configuration, yada yada). > > Currently, in Debian, the situation is that there is a configuration > file, /etc/nbd-client, which is sourced in the init script, and which > contains bash arrays with configuration. The init script then loops over > those bash arrays and runs the appropriate nbd-client command to connect > the device. Any actual mounting (etc) of the device, then, is left to > other init scripts. It expects that filesystems on NBD devices have the > "_netdev" option in its fstab entry listed, so that it will be mounted > by the "mountnfs" rather than "mountall" init script. > > As can be expected, this took several iterations to get right in all > corner cases. It seems to be working fine now, however. > > When converting this to systemd unit files, from skimming over the > documentation, I guess I'll need something along the lines of the > following: > > - I will need to create dev-nbd@.device unit files. These unit files would > connect the device when needed. How do you determine when it is needed? >From your description I'd suggest as first step generator that mostly does the same as already done by iitscript - create generator that reads /etc/nbd-client and creates service (not device) file for every NBD device. It could be a link to common template, or separate unit - it does not really matter. - this service will simply start nbd-client as you describe above. Those services will be of Type=simple; and they probably should be After=network.target and definitely Before=remote-fs-pre.target for _netdev to work. > - It may be a good idea to move the configuration from a sourced shell > script snipped to "something else". I do want to retain some backwards > compatibility, but it's okay if that's just a program interpreting the > shell script snippet and outputting something more modern. > Generator allows you to retain existing configuration. And at the very beginning generator can be just a shell script (especially if it does not need to do more than just "ln -s"). But I suspect you will need at least pass information about NBD server and I presume it may be different for each device, right? > I do foresee some problems, though, and I'd like to see if these are > indeed problems or whether I just need to read more documentation. I > haven't found an easy answer in the documentation that I've read, but > then maybe I haven't been looking very well. > > NBD device nodes are a bit special in that due to the way NBD devices > are connected, the device must exist at all times, even before it is > connected; I suspect (though have not actually tried) that systemd will > only try to "start" a .device unit file if the device node itself is not > there yet. For NBD, the difference between a connected device and a > not-connected one can be spotted in the apparent size of the block > device (the BLKGETSIZE64 ioctl will return 0 for a not-connected device) > and in the presence (or lack thereof) of a file /sys/block/nbdX/pid (if > it exists, it contains the PID of the nbd-client process handling the > connection; if it does not, the device is not connected), not by the > presence (or lack thereof) of the device node itself. > > This is not the case for partitions of NBD devices, however; these will > only show up after the first open(), as explained above. As such, I > might need two templates: one which connects the NBD device (for a > /dev/nbdX device), and one for the partition (/dev/nbdXpY) which simply > depends on the regular NBD device. However, if I understand correctly, > it would not seem to be possible to create an nbdX template that does > not also match nbdXpY. > nbdX is unrelated to nbdXpY. If those devices are created by kernel, you do not really need to do anything. _______________________________________________ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
