Re: Network services fail on startup
I had to restart my system, after having set the interfaces to auto, and everything worked fine. Generalizing freely, my problem is solved. It's a great relief: checking that everything was OK after every reboot was a drag. Ross On Tue, Jul 14, 2020 at 2:57 PM Ross Boylan wrote: > Thanks to everyone for their help. Since I am using allow-hotplug, I'll > change that and see if it's enough to cure the problem. > > Then I can look into the new filter tools. > Ross > > On Tue, Jul 14, 2020 at 4:32 AM Greg Wooledge wrote: > >> On Mon, Jul 13, 2020 at 06:41:39PM -0700, Ross Boylan wrote: >> > I am having intermittent problems on startup in which network services >> do >> > not start properly, generally with messages suggesting the network >> > interface they need is not available. If I stop and start them after, >> they >> > will run. >> >> The number one cause of this is having the interface marked as >> "allow-hotplug" instead of "auto" in the interfaces(5) file. >> >> Edit /etc/network/interfaces and see if your interface is defined in >> this file at all. (If it's not, then it's being defined some *other* >> way, either by Network Manager, or by systemd, or something else). >> >> If you see your interface marked with "allow-hotplug name", change it >> to "auto name". >> >> The installer thinks every system is some silly mobile/laptop thing, >> so it defaults all ethernet interfaces to "allow-hotplug", even if >> the interface is soldered onto the motherboard and is absolutely >> not "hot-pluggable". For most desktop or server systems, this will >> be the wrong choice. And it causes *exactly* the symptom you're >> describing here. >> >>
Re: Network services fail on startup
Thanks to everyone for their help. Since I am using allow-hotplug, I'll change that and see if it's enough to cure the problem. Then I can look into the new filter tools. Ross On Tue, Jul 14, 2020 at 4:32 AM Greg Wooledge wrote: > On Mon, Jul 13, 2020 at 06:41:39PM -0700, Ross Boylan wrote: > > I am having intermittent problems on startup in which network services do > > not start properly, generally with messages suggesting the network > > interface they need is not available. If I stop and start them after, > they > > will run. > > The number one cause of this is having the interface marked as > "allow-hotplug" instead of "auto" in the interfaces(5) file. > > Edit /etc/network/interfaces and see if your interface is defined in > this file at all. (If it's not, then it's being defined some *other* > way, either by Network Manager, or by systemd, or something else). > > If you see your interface marked with "allow-hotplug name", change it > to "auto name". > > The installer thinks every system is some silly mobile/laptop thing, > so it defaults all ethernet interfaces to "allow-hotplug", even if > the interface is soldered onto the motherboard and is absolutely > not "hot-pluggable". For most desktop or server systems, this will > be the wrong choice. And it causes *exactly* the symptom you're > describing here. > >
Re: Network services fail on startup
Andrei POPESCU wrote: > On Ma, 14 iul 20, 07:11:39, Dan Ritter wrote: > > The way to handle a giant blocklist efficiently is ipset, which manipulates > > large groups of IPs that will be matched for a particular rule. > > Disclaimer: I'm not an expert on either iptables or nftables, this is > just based on some documentation I read. > > As far as I understand, while iptables (in buster) is indeed a frontend > to nftables, nftables has new features that are not usable with iptables > syntax. > https://wiki.nftables.org/wiki-nftables/index.php/Moving_from_iptables_to_nftables > In particular regarding ipset, this page suggests manual translation is > necessary: > > https://wiki.nftables.org/wiki-nftables/index.php/Moving_from_ipset_to_nftables You are correct in all regards, but you are also not taking the next, necessary step. A firewall which is currently using iptables can be rewritten to use iptables and ipset; or it can be rewritten to use nftables with ipset. In either case, ipset is the correct tool, it just changes syntax in between versions, so to avoid duplicating effort, one might prefer to make one conversion rather than two. -dsr-
Re: Network services fail on startup
On Ma, 14 iul 20, 07:11:39, Dan Ritter wrote: > Andrei POPESCU wrote: > > On Lu, 13 iul 20, 18:41:39, Ross Boylan wrote: > > > > > > The interface has a pre-up script that has over 1,000 iptables add lines > > > for blacklists, and I suspect this is slowing things down enough to cause > > > trouble. I was not having problems when the script was shorter. > > > > P.S. as far as I understand nftables should handle these much better > > than iptables. May or may not help with your actual problem. > > iptables is currently a frontend to nftables. > > The way to handle a giant blocklist efficiently is ipset, which manipulates > large groups of IPs that will be matched for a particular rule. Disclaimer: I'm not an expert on either iptables or nftables, this is just based on some documentation I read. As far as I understand, while iptables (in buster) is indeed a frontend to nftables, nftables has new features that are not usable with iptables syntax. https://wiki.nftables.org/wiki-nftables/index.php/Moving_from_iptables_to_nftables In particular regarding ipset, this page suggests manual translation is necessary: https://wiki.nftables.org/wiki-nftables/index.php/Moving_from_ipset_to_nftables Kind regards, Andrei -- http://wiki.debian.org/FAQsFromDebianUser signature.asc Description: PGP signature
Re: Network services fail on startup
On Mon, Jul 13, 2020 at 06:41:39PM -0700, Ross Boylan wrote: > I am having intermittent problems on startup in which network services do > not start properly, generally with messages suggesting the network > interface they need is not available. If I stop and start them after, they > will run. The number one cause of this is having the interface marked as "allow-hotplug" instead of "auto" in the interfaces(5) file. Edit /etc/network/interfaces and see if your interface is defined in this file at all. (If it's not, then it's being defined some *other* way, either by Network Manager, or by systemd, or something else). If you see your interface marked with "allow-hotplug name", change it to "auto name". The installer thinks every system is some silly mobile/laptop thing, so it defaults all ethernet interfaces to "allow-hotplug", even if the interface is soldered onto the motherboard and is absolutely not "hot-pluggable". For most desktop or server systems, this will be the wrong choice. And it causes *exactly* the symptom you're describing here.
Re: Network services fail on startup
Andrei POPESCU wrote: > On Lu, 13 iul 20, 18:41:39, Ross Boylan wrote: > > > > The interface has a pre-up script that has over 1,000 iptables add lines > > for blacklists, and I suspect this is slowing things down enough to cause > > trouble. I was not having problems when the script was shorter. > > P.S. as far as I understand nftables should handle these much better > than iptables. May or may not help with your actual problem. iptables is currently a frontend to nftables. The way to handle a giant blocklist efficiently is ipset, which manipulates large groups of IPs that will be matched for a particular rule. -dsr-
Re: Network services fail on startup
Ross Boylan wrote: > tftpd-hpa is the service that fails most consistently. Fail also on my system without systemd. Everytime I reboot the server, I have to restart manually. I found out for the server it is better for now to not use systemd. I have GRUB_CMDLINE_LINUX_DEFAULT="quiet init=/lib/sysvinit/init" in /etc/default/grub Except tftpd-hpa there are no other issues - all network services start properly. I have most of the services listed also running: tftpd-hpa nmbd hddtemp exim4.service smbd.service inetd.service isc-dhcp-server.service as said only tftpd-hpa fails. I think it could be that it is broken somewhere. What if you remove it and try reboot? At least this theory will be eliminated.
Re: Network services fail on startup
On Lu, 13 iul 20, 18:41:39, Ross Boylan wrote: > > The interface has a pre-up script that has over 1,000 iptables add lines > for blacklists, and I suspect this is slowing things down enough to cause > trouble. I was not having problems when the script was shorter. P.S. as far as I understand nftables should handle these much better than iptables. May or may not help with your actual problem. Kind regards, Andrei -- http://wiki.debian.org/FAQsFromDebianUser signature.asc Description: PGP signature
Re: Network services fail on startup
On Lu, 13 iul 20, 18:41:39, Ross Boylan wrote: > I am having intermittent problems on startup in which network services do > not start properly, generally with messages suggesting the network > interface they need is not available. If I stop and start them after, they > will run. I suspect this is a result of the interaction of systemd, > ifupdown, and a pre-up script that takes a while to execute. But if I look > at things now it seems the dependencies in place should suffice. Of > course, this is after I restarted it, and so perhaps the dependency is > added dynamically, too late to do any good. Units should always be generated the same way and I doubt restarting regenerates them. Things you could try: - move the pre-up script to a separate unit that ifupdown depends on (After= + Wants= or even Requires=) - switch to systemd-networkd See also the description of network-online.target in systemd.special(7) and the linked document "Running Services After the Network is up". Kind regards, Andrei -- http://wiki.debian.org/FAQsFromDebianUser signature.asc Description: PGP signature
Network services fail on startup
I am having intermittent problems on startup in which network services do not start properly, generally with messages suggesting the network interface they need is not available. If I stop and start them after, they will run. I suspect this is a result of the interaction of systemd, ifupdown, and a pre-up script that takes a while to execute. But if I look at things now it seems the dependencies in place should suffice. Of course, this is after I restarted it, and so perhaps the dependency is added dynamically, too late to do any good. I think what I need is for network-online.target not to be ready until after the interfaces ifup controls are up, but I'm not sure how to do it. Also not sure it's the problem, particularly since some information indicates it already has such a dependency. Details: My main interface is controlled by /etc/network/interfaces that includes allow-hotplug /enp* iface enp5s0 inet static # etc The interface has a pre-up script that has over 1,000 iptables add lines for blacklists, and I suspect this is slowing things down enough to cause trouble. I was not having problems when the script was shorter. This is a problem other have encountered: https://unix.stackexchange.com/questions/209832/debian-systemd-network-online-target-not-working https://unix.stackexchange.com/questions/257888/systemd-wait-for-network-interface-to-be-up-before-running-service Various solutions are suggested, with a leading one being a dependency on network-online.target. But all my network services already have that dependency. For example, # systemctl cat tftpd-hpa # /run/systemd/generator.late/tftpd-hpa.service # Automatically generated by systemd-sysv-generator [Unit] Documentation=man:systemd-sysv-generator(8) SourcePath=/etc/init.d/tftpd-hpa Description=LSB: HPA's tftp server Before=multi-user.target Before=multi-user.target Before=multi-user.target Before=graphical.target After=remote-fs.target After=network-online.target Wants=network-online.target tftpd-hpa is the service that fails most consistently. # systemctl list-dependencies tftpd-hpa Mon 13 Jul 2020 02:42:22 PM PDT tftpd-hpa.service ● ├─system.slice ● ├─network-online.target ● │ ├─networking.service ● │ └─NetworkManager-wait-online.service ● └─sysinit.target etc. Notably absent from network-online.target's dependencies is anything related to ifup. However, ifup@enp5s0.service appears as an After dependency of network-online.target when viewed another way: # systemctl show network-online.target Id=network-online.target Names=network-online.target Wants=NetworkManager-wait-online.service networking.service WantedBy=tftpd-hpa.service nmbd.service hddtemp.service docker.service mythtv-backend.service exim4.service hylafax.service smbd.service inetd.service isc-dhcp-server.service Conflicts=shutdown.target Before=hylafax.service isc-dhcp-server.service smbd.service exim4.service apt-daily.service rc-local.service apt-daily-upgrade.service mythtv-backend.service apcupsd.service tftpd-hpa.service docker.service inetd.service hddtemp.service shutdown.target nmbd.service After=ifup@enp5s0.service network.target networking.service NetworkManager-wait-online.service Documentation=man:systemd.special(7) https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget Any suggestions? Ideally I would not be tied to the interface name, so that if I plug the network cable into another port things will still work. Thanks. Ross Boylan
