Hi!
On Monday 17 June 2019 16:56:49 Axel Beckert wrote:
> Control: tag -1 - moreinfo + patch
>
> Hi Pali,
>
> thanks for the quick response! Let's try to figure out the difference
> between our configurations to find the culrprit.
>
> Pali Rohár wrote:
> > > Pali Rohár wrote:
> > > > severity 919188 critical
> > >
> > > Why? You neither explained that it breaks the whole system nor that it
> > > causes other, unrelated packages to break nor that it causes serious
> > > data-loss.
> > >
> > > Please check https://www.debian.org/Bugs/Developer#severities and
> > > explain why you think that the severity you've chosen fits this bug
> > > report.
> >
> > Hi! The reason is that it breaks boot process.
>
> Just to make this clear:
>
> It actually doesn't finish to boot and other services which would
> start after sslh are not starting at all, correct?
Currently seems that everything was started. But I do not know if it is
because of parallelism or it is intended due to to forking of startpar.
Or if there is race condition... Such bugs which affects boot process,
where is involved parallelism or forking are annoying; plus they needs
full reboot of machine, etc...
> > startpar is still running and sslh stdout output is going to tty1
> > where /dev/console points.
>
> The latter is definitely annoying, but I wouldn't consider that part
> "breaking the boot process".
>
> JFTR: For me, the log messages of sslh go to either /var/log/user.log
> (if started via inetd) or /var/log/auth.log (if started as standalone
> daemon via init script).
>
> > > Addtionally I can't reproduce this bug no matter which variant I'm
> > > trying. And I'm running sslh under sysvinit on Debian Testing since
> > > 2012 (in standalone and single-thread mode):
> >
> > Strange, it happens for me on all Debian servers where sslh is
> > installed and in use. Also exactly same problem is on small Raspberry PI
> > installation. And proposed fix fixes it...
>
> Ah right, you've added a patch. Forgot to tag the bug report as
> "patch". Done now.
>
> (I also checked that I don't have a similar patch like you proposed in
> place. That would have been embarrassing. But I haven't. :-)
>
> > Some maybe configuration of something other takes effect on it too?
>
> I suspect so.
>
> > For me it is 100% reproducible, no race condition or randomness.
>
> Same here, just the other way round. ;-)
>
> > > So please also show your sslh entry in /etc/inetd.conf and your
> > > /etc/default/sslh.
> [...]
> > $ cat /etc/default/sslh
> > # Default options for sslh initscript
> > # sourced by /etc/init.d/sslh
> >
> > # Disabled by default, to force yourself
> > # to read the configuration:
> > # - /usr/share/doc/sslh/README.Debian (quick start)
> > # - /usr/share/doc/sslh/README, at "Configuration" section
> > # - sslh(8) via "man sslh" for more configuration details.
> > # Once configuration ready, you *must* set RUN to yes here
> > # and try to start sslh (standalone mode only)
> >
> > RUN=yes
> >
> > # binary to use: forked (sslh) or single-thread (sslh-select) version
> > # systemd users: don't forget to modify /lib/systemd/system/sslh.service
> > DAEMON=/usr/sbin/sslh
> >
> > DAEMON_OPTS="--user sslh --transparent --listen MYDOMAIN:443 --ssh
> > localhost:22 --http localhost:80 --ssl localhost:443 --pidfile
> > /var/run/sslh/sslh.pid"
>
> Here's my working configuration for comparison:
>
> -------------------------------------------------------------------------------
> $ cat /etc/default/sslh
> # Default options for sslh initscript
> # sourced by /etc/init.d/sslh
>
> # Disabled by default, to force yourself
> # to read the configuration:
> # - /usr/share/doc/sslh/README.Debian (quick start)
> # - /usr/share/doc/sslh/README, at "Configuration" section
> # - sslh(8) via "man sslh" for more configuration details.
> # Once configuration ready, you *must* set RUN to yes here
> # and try to start sslh (standalone mode only)
>
> RUN=yes
>
> # binary to use: forked (sslh) or single-thread (sslh-select) version
> DAEMON=/usr/sbin/sslh-select
>
> DAEMON_OPTS="--user sslh --listen 0.0.0.0:443 --ssh 127.0.0.1:22 --ssl
> 127.0.0.1:442 --pidfile /var/run/sslh/sslh.pid"
> -------------------------------------------------------------------------------
>
> I already tested that /usr/sbin/sslh-select (single-thread) vs
> /usr/sbin/sslh (forking) doesn't make a difference. So let's focus on
> the options. Here's a wdiff of our two lines:
>
> DAEMON_OPTS="--user sslh {+--transparent+} --listen [-0.0.0.0:443-]
> {+MYDOMAIN:443+} --ssh [-127.0.0.1:22-] {+localhost:22 --http localhost:80+}
> --ssl [-127.0.0.1:442-] {+localhost:443+} --pidfile /var/run/sslh/sslh.pid"
>
> So the differences are
>
> * you're using --transparent,
Yes, this is one of the key features of sslh.
> * that your listening only one one IP while I listen on all IPs
> (accordingly my Apache listens on a different port instead of just
> localhost),
>
> * that you use localhost (could be IPv6, too) and I only use 127.0.0.1
> (i.e. only IPv4),
Yes, IPv6 is in use. Both MYDOMAIN and localhost involves IPv6 sockets.
And I have working forwarding of both IPv4 and IPv6 connections.
> * that you also have --http (didn't know about that and quite some
> other option until now, seems as ssl;h has evolved a lot since I
> initially configured it).
>
> I dont' think that the different IPs and ports should make difference.
>
> So the most obvious difference is --transparent:
>
> Just adding --transparent to config (without all the iptables stuff
> necessary for it to work), started up fine (both, on reboot as well as
> with "service sslh restart), but it indeed lsof shows these:
>
> sslh-sele 4326 sslh 0u CHR 136,2 0t0 5 /dev/pts/2
> sslh-sele 4326 sslh 1u CHR 136,2 0t0 5 /dev/pts/2
> sslh-sele 4326 sslh 2u CHR 136,2 0t0 5 /dev/pts/2
This is exactly what I'm observing.
So problematic option which cause this is --transparent? Interesting,
I have not thought that such option may change behavior of usage
stdin/stdout/stderr.
> Then again, I get those open ttys also if I just restart without
> --transparent.
So you was able to reproduce it also without --transparent? Just only
once?
> So that's what you're seeing, only that it doesn't hang
> for me at boot time. Strange.
>
> So maybe these open terminals are a red herring (or its own, less
> severe issue) and not related to startpar and the init script not
> finishing?
I think it is a problem. startpar does not finish its job because child
process still holds terminal.
> I also added --http to my setup, but no change either.
>
> So you could try your setup once without --transparent and without
> your patch? (Not sure if the iptables stuff needs to be disabled for
> that, too.)
>
> If even that doesn't make a change for you, you could try to replace
> all hostnames with IP addresses, but then again, I can't imagine that
> DNS lookups could cause such a lockup.
Ok, I will try both change and let you know.
--
Pali Rohár
pali.ro...@gmail.com
