Bug#942808: ITP: dropbear-rescue -- A set of initramfs scripts to add and run dropbear when the system boots in rescue mode
On 2019-10-25 7:50 p.m., Guilhem Moulin wrote: Control: retitle -1 race condition: init-bottom script doesn't abort/cleanup configure_networking() Control: tag -1 pending On Fri, 25 Oct 2019 at 02:26:39 +0200, Guilhem Moulin wrote: Ah right, I understand the problem now. Whether configure_networking() is run (at premount stage) in the background or not depends on the boot method. On local (non-NFS) mounts it's done in the background, and should be interrupted at bottom stage. However if no other script is waiting for interactive user input the bottom script might run before dropbear had a chance to run yielding a race condition at bottom stage. This is a bug. Implemented a fix at https://salsa.debian.org/debian/dropbear/commit/1ab168b9 , could you please confirm that it solves the race for you? It appears that ipconfig doesn't react to SIGTERM, so I've not been able to properly abort configure_networking(). Instead, the init-bottom script now waits (for up to 1min) until dropbear is started before bringing the network down. Of course it's a bit of a waste as it needlessly delays the boot process (it's no longer possible to log in at that stage anyway), but at least when execution is handed over to init(1) it's with a clean network stack and not with a running ipconfig process. A quick note: I use this code snippet to show the user how to continue the boot process in case he logins remotely. https://github.com/lukavia/dropbear-rescue/blob/8b84e98c8e7c892250906bb167d3828046619ade/src/hooks/dropbear-rescue#L75 This is in the initramfs build hook file: echo "echo " >> "$home/.profile" echo 'echo To continue booting execute: kill -1$(ps -o pid,args | grep "sh -i" | grep -v grep | cut -d" " -f-3)' >> "$home/.profile" I'm sure it could be written better, but I think it would be nice addition. Best Regards
Bug#942808: ITP: dropbear-rescue -- A set of initramfs scripts to add and run dropbear when the system boots in rescue mode
On 2019-10-25 7:50 p.m., Guilhem Moulin wrote: Control: retitle -1 race condition: init-bottom script doesn't abort/cleanup configure_networking() Control: tag -1 pending On Fri, 25 Oct 2019 at 02:26:39 +0200, Guilhem Moulin wrote: Ah right, I understand the problem now. Whether configure_networking() is run (at premount stage) in the background or not depends on the boot method. On local (non-NFS) mounts it's done in the background, and should be interrupted at bottom stage. However if no other script is waiting for interactive user input the bottom script might run before dropbear had a chance to run yielding a race condition at bottom stage. This is a bug. Implemented a fix at https://salsa.debian.org/debian/dropbear/commit/1ab168b9 , could you please confirm that it solves the race for you? It appears that ipconfig doesn't react to SIGTERM, so I've not been able to properly abort configure_networking(). Instead, the init-bottom script now waits (for up to 1min) until dropbear is started before bringing the network down. Of course it's a bit of a waste as it needlessly delays the boot process (it's no longer possible to log in at that stage anyway), but at least when execution is handed over to init(1) it's with a clean network stack and not with a running ipconfig process. Now that #943459 resolves the race condition a separate package makes no sense. If someone wants ssh in initramfs it would only be needed to install dropbear-initramfs. Running only on panic would be a minor optimization to avoid running dropbear and configuring network when where is no crypttab. This could actually be achieved by simply moving the init-premount/dropbear to (the undocumented yet ;) ) panic folder. So I would consider the this case closed. Thank you very much
Bug#942808: ITP: dropbear-rescue -- A set of initramfs scripts to add and run dropbear when the system boots in rescue mode
Control: retitle -1 ITP: dropbear-rescue -- A set of initramfs scripts to add and run dropbear when the system boots in rescue mode Control: tag -1 - pending On Sat, 26 Oct 2019 at 01:50:02 +0200, Guilhem Moulin wrote: > Control: retitle -1 race condition: init-bottom script doesn't abort/cleanup > configure_networking() > Control: tag -1 pending Oops sorry, meant to reply to the clone (#943459) not the original :-/ -- Guilhem. signature.asc Description: PGP signature
Bug#942808: ITP: dropbear-rescue -- A set of initramfs scripts to add and run dropbear when the system boots in rescue mode
Control: clone -1 -2 Control: reassign -2 dropbear-initramfs 2019.78-2 Control: retitle -2 race condition: init-bottom doesn't abort/cleanup run_networking() Control: severity -2 normal On Thu, 24 Oct 2019 at 18:48:12 -0400, Anton Avramov wrote: > However I've ran into a problem where if there is no panic and > initramfs ipconfig command uses dhcp it would actually finish after > the boot process is complete. And if the system itself uses static ip > it gets overwritten by the dhcp. Ah right, I understand the problem now. Whether configure_networking() is run (at premount stage) in the background or not depends on the boot method. On local (non-NFS) mounts it's done in the background, and should be interrupted at bottom stage. However if no other script is waiting for interactive user input the bottom script might run before dropbear had a chance to run yielding a race condition at bottom stage. This is a bug. > So the starting script in premount for dropbear should take care not > to start if there is no /etc/crypttab and only start in case of panic. dropbear-initramfs has nothing to do with /etc/crypttab, and it's not because there are encrypted volumes to unlock that the boot process is race-free: for instance these volumes might be unlocked in an unattended fashion with a cheap PBKDF (or just plain dm-crypt). Furthermore right now the way the encrypted volumes are unlocked at initramfs stage, incl. the location of the crypttab(5), is considered as an internal detail of the cryptsetup-initramfs package :-P That said I maintain that package too, so I could help documenting the necessary interface if needs be :-D > Considering my comment above should there be an option to activate > dropbear in case of panic but no crypttab or there should be a config > option to include and start dropbear in initramfs in case crypttab > doesn't exists? Once the race is eliminated from the init-bottom script (bug #-2), run_dropbear() will unconditionally start, but be properly terminated/cleaned before starting the init(1) binary. That's your second alternative; it's the simplest solution (either way there is a bug that needs fixing) and it doesn't require further documentation from initramfs-tools(7). I also think it'd be reasonable to have an option to run the premount script at panic stage instead. Cheers, -- Guilhem. signature.asc Description: PGP signature
Bug#942808: ITP: dropbear-rescue -- A set of initramfs scripts to add and run dropbear when the system boots in rescue mode
On 2019-10-24 4:25 p.m., Guilhem Moulin wrote: On Wed, 23 Oct 2019 at 16:04:31 -0400, Anton Avramov wrote: On Mon., Oct. 21, 2019, 21:08 Guilhem Moulin, wrote: Given the scope of this package, I strongly believe it'd make more sense to merge it with src:dropbear rather than shipping a separate source package. I agree. However I don't know how I can contribute to that. It's never too late :-) `apt showsrc dropbear-initramfs` gives a link to the repository (hosted at salsa in the ‘Debian’ group). Bugs, including feature requests, are to be filed against the Debian BTS like for any other package: https://bugs.debian.org/dropbear-initramfs . Thank you very much! I'll do that then. Following dropbear-initramfs package You seem to suggest that your code takes inspiration from dropbear-initramfs, but your copyright information doesn't reflect that. Sorry about that! I've tried to change that, but I'm not absolutely certain I've done it right. AFAICT only src/* and etc/* are taken from src:dropbear's debian/initramfs/* (and modified by you), the rest is your own work. Also AFAICT upstream isn't a copyright holder for these scripts. Regarding debian/control, your package shouldn't break or replace dropbear <2015.68-1. This was the case the case for drobear-initramfs after the package split, but it doesn't apply here. And AFAIK there is no need to add a lower-bound on dropbear-bin's version. that will install dropbear and would have the appropriate initramfs scripts to start it in case the system enters rescue mode. Why couldn't that be done in dropbear-initramfs instead? Right now the script is run at premount stage, but I guess we could have an option to run it at another stage instead. The only argument I can think of is to give the option to have separate access for panic and crypt unlock. […] And in principle I can give someone the right to unlock the fs, but not login in case of panic where in that case and admin with higher access should step in. One can already separate access levels with a single authorized_keys(5) file containing multiple keys and different restriction options. dropbear-initramfs spawns the sshd at init-premount stage, ie after udev has been started and modules loaded. AFAIK it's unreliable to start it earlier, because there might not be any network stack yet. So if initramfs-tools panics earlier you'll not be able to log in remotely will you? Yes, but in case it panics later it would be nice to be able to login :) (Anyway, again ‘panic’ is not a documented initramfs-tools(7) subdirectory.) I'll try to find some time and do something about that also. And if it panics later, for instance due to a missing block device, then one should be able to log-in using dropbear-initramfs alone. Initially I've tried to just trick dropbear-initramfs by creating empty /etc/crypttab and then creating only a panic script to start if it wasn't started. However I've ran into a problem where if there is no panic and initramfs ipconfig command uses dhcp it would actually finish after the boot process is complete. And if the system itself uses static ip it gets overwritten by the dhcp. So the starting script in premount for dropbear should take care not to start if there is no /etc/crypttab and only start in case of panic. By compatible I mean they can run together without interfering given they use different port. Since there are only 2 scripts in initramfs I'm not sure how would they suppose to share the code if we assume they are different packages Using Depends: plus maybe some refactoring on the drobpear-initramfs side would be a way, but IMHO no separate binary package (let alone a source package) is needed :-P dropbear-initramfs's purpose is to expose the dropbear sshd at initramfs stage, starting as early as possible and for as long as possible. It can be useful to unlock encrypted volumes, but it's not its only use-case. OK. Lets go that path. Considering my comment above should there be an option to activate dropbear in case of panic but no crypttab or there should be a config option to include and start dropbear in initramfs in case crypttab doesn't exists? Best Regards
Bug#942808: ITP: dropbear-rescue -- A set of initramfs scripts to add and run dropbear when the system boots in rescue mode
On Wed, 23 Oct 2019 at 16:04:31 -0400, Anton Avramov wrote: > On Mon., Oct. 21, 2019, 21:08 Guilhem Moulin, wrote: >> Given the scope of this package, I strongly believe it'd make more sense >> to merge it with src:dropbear rather than shipping a separate source >> package. > > I agree. However I don't know how I can contribute to that. It's never too late :-) `apt showsrc dropbear-initramfs` gives a link to the repository (hosted at salsa in the ‘Debian’ group). Bugs, including feature requests, are to be filed against the Debian BTS like for any other package: https://bugs.debian.org/dropbear-initramfs . >>> Following dropbear-initramfs package >> You seem to suggest that your code takes inspiration from >> dropbear-initramfs, but your copyright information doesn't reflect >> that. > > Sorry about that! I've tried to change that, but I'm not absolutely > certain I've done it right. AFAICT only src/* and etc/* are taken from src:dropbear's debian/initramfs/* (and modified by you), the rest is your own work. Also AFAICT upstream isn't a copyright holder for these scripts. Regarding debian/control, your package shouldn't break or replace dropbear <2015.68-1. This was the case the case for drobear-initramfs after the package split, but it doesn't apply here. And AFAIK there is no need to add a lower-bound on dropbear-bin's version. >>> that will install dropbear and would have the appropriate initramfs >>> scripts to start it in case the system enters rescue mode. >> >> Why couldn't that be done in dropbear-initramfs instead? Right now the >> script is run at premount stage, but I guess we could have an option to >> run it at another stage instead. > > The only argument I can think of is to give the option to have separate > access for panic and crypt unlock. > […] > And in principle I can give someone the right to unlock the fs, but > not login in case of panic where in that case and admin with higher > access should step in. One can already separate access levels with a single authorized_keys(5) file containing multiple keys and different restriction options. dropbear-initramfs spawns the sshd at init-premount stage, ie after udev has been started and modules loaded. AFAIK it's unreliable to start it earlier, because there might not be any network stack yet. So if initramfs-tools panics earlier you'll not be able to log in remotely will you? (Anyway, again ‘panic’ is not a documented initramfs-tools(7) subdirectory.) And if it panics later, for instance due to a missing block device, then one should be able to log-in using dropbear-initramfs alone. > By compatible I mean they can run together without interfering given they > use different port. Since there are only 2 scripts in initramfs I'm not > sure how would they suppose to share the code if we assume they are > different packages Using Depends: plus maybe some refactoring on the drobpear-initramfs side would be a way, but IMHO no separate binary package (let alone a source package) is needed :-P dropbear-initramfs's purpose is to expose the dropbear sshd at initramfs stage, starting as early as possible and for as long as possible. It can be useful to unlock encrypted volumes, but it's not its only use-case. -- Guilhem. signature.asc Description: PGP signature
Bug#942808: ITP: dropbear-rescue -- A set of initramfs scripts to add and run dropbear when the system boots in rescue mode
Hi Guilhem, Thank you for replying to my request. On Mon., Oct. 21, 2019, 21:08 Guilhem Moulin, wrote: > Hi Anton, > > [dropbear package maintainer here.] > > On Mon, 21 Oct 2019 at 16:07:11 -0400, Anton Avramov wrote: > > For a long time now I've maintained servers remotely. One problem that > > I've faced is what when there is problem booting I lose ability to login > > remotely and help the person on premises. Further more that person can > > be not technically capable or comfortable with writing console > > commands. There are instances where there is not even a keyboard or > > monitor attached. > > Given the scope of this package, I strongly believe it'd make more sense > to merge it with src:dropbear rather than shipping a separate source > package. > I agree. However I don't know how I can contribute to that. > > Following dropbear-initramfs package > > You seem to suggest that your code takes inspiration from > dropbear-initramfs, > but your copyright information doesn't reflect that. > Sorry about that! I've tried to change that, but I'm not absolutely certain I've done it right. > > that will install dropbear and would have the appropriate initramfs > > scripts to start it in case the system enters rescue mode. > > Why couldn't that be done in dropbear-initramfs instead? Right now the > script is run at premount stage, but I guess we could have an option to > run it at another stage instead. The only argument I can think of is to give the option to have separate access for panic and crypt unlock. For example in my scenario I have crypt unlock on different port with different hostkeys. And in principle I can give someone the right to unlock the fs, but not login in case of panic where in that case and admin with higher access should step in. However the first step would be to > document the ‘panic’ stage in initramfs-tools(7) (the “boot scripts” > section of the manual doesn't mention it AFAICT). > > > I've tried to keep the package compatible with dropbear-initramfs > > package. > > Depending what you mean by “compatible”, please note that the only > public interface of that package is the configuration file. Relying on > something else would be another argument in favor of integrating this > into src:dropbear. Also I didn't review the code but at first glance I > see similarities, please see §4.13 of the Debian Policy. > By compatible I mean they can run together without interfering given they use different port. Since there are only 2 scripts in initramfs I'm not sure how would they suppose to share the code if we assume they are different packages > Cheers, > -- > Guilhem. >
Bug#942808: ITP: dropbear-rescue -- A set of initramfs scripts to add and run dropbear when the system boots in rescue mode
Hi Anton, [dropbear package maintainer here.] On Mon, 21 Oct 2019 at 16:07:11 -0400, Anton Avramov wrote: > For a long time now I've maintained servers remotely. One problem that > I've faced is what when there is problem booting I lose ability to login > remotely and help the person on premises. Further more that person can > be not technically capable or comfortable with writing console > commands. There are instances where there is not even a keyboard or > monitor attached. Given the scope of this package, I strongly believe it'd make more sense to merge it with src:dropbear rather than shipping a separate source package. > Following dropbear-initramfs package You seem to suggest that your code takes inspiration from dropbear-initramfs, but your copyright information doesn't reflect that. > that will install dropbear and would have the appropriate initramfs > scripts to start it in case the system enters rescue mode. Why couldn't that be done in dropbear-initramfs instead? Right now the script is run at premount stage, but I guess we could have an option to run it at another stage instead. However the first step would be to document the ‘panic’ stage in initramfs-tools(7) (the “boot scripts” section of the manual doesn't mention it AFAICT). > I've tried to keep the package compatible with dropbear-initramfs > package. Depending what you mean by “compatible”, please note that the only public interface of that package is the configuration file. Relying on something else would be another argument in favor of integrating this into src:dropbear. Also I didn't review the code but at first glance I see similarities, please see §4.13 of the Debian Policy. Cheers, -- Guilhem. signature.asc Description: PGP signature
Bug#942808: ITP: dropbear-rescue -- A set of initramfs scripts to add and run dropbear when the system boots in rescue mode
Package: wnpp Severity: wishlist Owner: Anton Avramov * Package name: dropbear-rescue Version : 0.1 Upstream Author : Anton Avramov * URL : https://github.com/lukavia/dropbear-rescue * License : GPL2 Programming Lang: sh Description : A set of initramfs scripts to add and run dropbear when the system boots in rescue mode For a long time now I've maintained servers remotely. One problem that I've faced is what when there is problem booting I lose ability to login remotely and help the person on premises. Further more that person can be not technically capable or comfortable with writing console commands. There are instances where there is not even a keyboard or monitor attached. Following dropbear-initramfs package I've decided to make a package that will install dropbear and would have the appropriate initramfs scripts to start it in case the system enters rescue mode. The hooks will try to preserve the hostkeys so security is preserved. In addition they will not allow password authentication. I've tried to keep the package compatible with dropbear-initramfs package. Since this is my first package any suggestions or contributions are welcome. I'm also looking for a sponsor for this package so others can benefit from it if it gets included in official Debian repositories.
