[Touch-packages] [Bug 1881972] Re: systemd-networkd crashes with invalid pointer
This bug was fixed in the package systemd - 237-3ubuntu10.42
---
systemd (237-3ubuntu10.42) bionic; urgency=medium
[ Dan Streetman ]
*
d/p/lp1860926/0001-networkd-Allow-to-retain-configs-even-if-carrier-is-.patch,
d/p/lp1860926/0002-network-Change-IgnoreCarrierLoss-default-to-value-of.patch,
d/p/lp1860926/0003-network-always-drop-configs-when-corresponding-netwo.patch:
- Add IgnoreCarrierLoss and default to value of ConfigureWithoutCarrier
(LP: #1860926)
https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=9a12a31a62f1a50cd3a67a164ee34c546809815e
https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=3cc3870fde47982a4dda53f820e18065e5488e7e
* d/e/rules-ubuntu/40-vm-hotadd.rules:
- Hotadd only offline memory and CPUs
(LP: #1876018)
https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=ba305d7ad00e80bc1a03f93e6986eef7cbbb18fc
* d/p/lp1881972-network-strdup-iif-and-oif-when-creating-RoutingPoli.patch:
- Avoid double-free by strdup'ing iif/oif strings for new policy rules
(LP: #1881972)
https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=874056f0d429aaa2cc872c3b35ec33cd3b740483
* d/p/lp1886197-seccomp-more-comprehensive-protection-against-libsec.patch
- Fix FTBFS on arm64 due to libseccomp changes (LP: #1886197)
https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=c284a72ca2e3d87bfe1c20afb2fcfb379cda544f
*
d/p/lp1832754/0001-umount-Try-unmounting-even-if-remounting-read-only-f.patch,
d/p/lp1832754/0002-umount-Don-t-bother-remounting-api-and-ro-filesystem.patch:
- Try unmounting even if ro-remount fails, and don't bother remounting
api/ro fs
(LP: #1832754)
https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=a518baa673aeaaf42000a3a01b7e03347652b216
[ Alex Murray, Jamie Strandboge ]
* d/p/lp1886115-pid1-fix-free-of-uninitialized-pointer-in-unit_fail_.patch:
- Fix free of uninitialized pointer (LP: #1886115)
https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=491c76fd0f2fba0007a9b54d63a50f21add643c8
-- Dan Streetman Wed, 08 Jul 2020 14:59:14
-0400
** Changed in: systemd (Ubuntu Bionic)
Status: Fix Committed => Fix Released
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1881972
Title:
systemd-networkd crashes with invalid pointer
Status in systemd package in Ubuntu:
Fix Released
Status in systemd source package in Bionic:
Fix Released
Bug description:
[impact]
systemd-networkd double-free causes crash under some circumstances,
such as adding/removing ip rules
[test case]
Use networkd-dispatcher events to add and remove IP rules. The example
scripts below are contrived (and by themselves likely to break access
to a machine) but would be adequate to trigger the bug. Put scripts
like these in place, reboot or run "netplan apply", and then leave the
machine running for a few DHCP renewal cycles.
=== /etc/networkd-dispatcher/configured.d/test.sh ===
#!/bin/bash
/sbin/ip rule add iif lo lookup 99
/sbin/ip rule add to 10.0.0.0/8 iif lo lookup main
=== END ===
=== /etc/networkd-dispatcher/configuring.d/test.sh ===
#!/bin/bash
# Tear down existing ip rules so they aren't duplicated
OLDIFS="${IFS}"
IFS="
"
for rule in `ip rule show|grep "iif lo" | cut -d: -f2-`; do
IFS="${OLDIFS}"
ip rule delete ${rule}
done
IFS="${OLDIFS}"
=== END ===
[regression potential]
this strdup's strings during addition of routing policy rules, so any
regression would likely occur when adding/modifying/removing ip rules,
possibly including networkd segfault or failure to add/remove/modify
ip rules.
[scope]
this is needed for bionic.
this is fixed by upstream commit
eeab051b28ba6e1b4a56d369d4c6bf7cfa71947c which is included starting in
v240, so this is already included in Focal and later.
I did not research what original commit introduced the problem, but
the reporter indicates this did not happen for Xenial so it's unlikely
this is a problem in Xenial or earlier.
[original description]
This is a serious regression with systemd-networkd that I ran in to
while setting up a NAT router in AWS. The AWS AMI ubuntu/images/hvm-
ssd/ubuntu-bionic-18.04-amd64-server-20200131 with
systemd-237-3ubuntu10.33 does NOT have the problem, but the next most
recent AWS AMI ubuntu/images/hvm-ssd/ubuntu-
bionic-18.04-amd64-server-20200311 with systemd-including
237-3ubuntu10.39 does.
Also, a system booted from the (good) 20200131 AMI starts showing the
problem after updating only systemd (to 237-3ubuntu10.41) and its
direct dependencies (e.g. 'apt-get install systemd'). So I'm fairly
confident that a change to the
[Touch-packages] [Bug 1881972] Re: systemd-networkd crashes with invalid pointer
the reproducer from description causes the coredump rather easily on
bionic:
ubuntu@lp1881972-b:~$ dpkg -l systemd|grep systemd
ii systemd237-3ubuntu10.41 amd64system and service manager
from journal:
Jul 30 17:49:08 lp1881972-b systemd-networkd[705]: munmap_chunk(): invalid
pointer
Jul 30 17:49:08 lp1881972-b systemd[1]: systemd-networkd.service: Main process
exited, code=dumped, status=6/ABRT
Jul 30 17:49:08 lp1881972-b systemd[1]: systemd-networkd.service: Failed with
result 'core-dump'.
after upgrading:
ubuntu@lp1881972-b:~$ dpkg -l systemd|grep systemd
ii systemd237-3ubuntu10.42 amd64system and service manager
many lease renewals:
ubuntu@lp1881972-b:~$ journalctl -b | grep Configured
Jul 30 17:58:39 lp1881972-b systemd[1]: Starting Wait for Network to be
Configured...
Jul 30 17:58:40 lp1881972-b systemd-networkd[712]: ens3: Configured
Jul 30 17:58:40 lp1881972-b systemd[1]: Started Wait for Network to be
Configured.
Jul 30 17:59:36 lp1881972-b systemd-networkd[712]: ens3: Configured
Jul 30 18:00:29 lp1881972-b systemd-networkd[712]: ens3: Configured
Jul 30 18:01:24 lp1881972-b systemd-networkd[712]: ens3: Configured
Jul 30 18:02:16 lp1881972-b systemd-networkd[712]: ens3: Configured
Jul 30 18:03:09 lp1881972-b systemd-networkd[712]: ens3: Configured
Jul 30 18:04:03 lp1881972-b systemd-networkd[712]: ens3: Configured
Jul 30 18:04:57 lp1881972-b systemd-networkd[712]: ens3: Configured
no core dumps:
ubuntu@lp1881972-b:~$ journalctl -b | grep core-dump
ubuntu@lp1881972-b:~$
** Tags removed: verification-needed verification-needed-bionic
** Tags added: verification-done verification-done-bionic
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1881972
Title:
systemd-networkd crashes with invalid pointer
Status in systemd package in Ubuntu:
Fix Released
Status in systemd source package in Bionic:
Fix Committed
Bug description:
[impact]
systemd-networkd double-free causes crash under some circumstances,
such as adding/removing ip rules
[test case]
Use networkd-dispatcher events to add and remove IP rules. The example
scripts below are contrived (and by themselves likely to break access
to a machine) but would be adequate to trigger the bug. Put scripts
like these in place, reboot or run "netplan apply", and then leave the
machine running for a few DHCP renewal cycles.
=== /etc/networkd-dispatcher/configured.d/test.sh ===
#!/bin/bash
/sbin/ip rule add iif lo lookup 99
/sbin/ip rule add to 10.0.0.0/8 iif lo lookup main
=== END ===
=== /etc/networkd-dispatcher/configuring.d/test.sh ===
#!/bin/bash
# Tear down existing ip rules so they aren't duplicated
OLDIFS="${IFS}"
IFS="
"
for rule in `ip rule show|grep "iif lo" | cut -d: -f2-`; do
IFS="${OLDIFS}"
ip rule delete ${rule}
done
IFS="${OLDIFS}"
=== END ===
[regression potential]
this strdup's strings during addition of routing policy rules, so any
regression would likely occur when adding/modifying/removing ip rules,
possibly including networkd segfault or failure to add/remove/modify
ip rules.
[scope]
this is needed for bionic.
this is fixed by upstream commit
eeab051b28ba6e1b4a56d369d4c6bf7cfa71947c which is included starting in
v240, so this is already included in Focal and later.
I did not research what original commit introduced the problem, but
the reporter indicates this did not happen for Xenial so it's unlikely
this is a problem in Xenial or earlier.
[original description]
This is a serious regression with systemd-networkd that I ran in to
while setting up a NAT router in AWS. The AWS AMI ubuntu/images/hvm-
ssd/ubuntu-bionic-18.04-amd64-server-20200131 with
systemd-237-3ubuntu10.33 does NOT have the problem, but the next most
recent AWS AMI ubuntu/images/hvm-ssd/ubuntu-
bionic-18.04-amd64-server-20200311 with systemd-including
237-3ubuntu10.39 does.
Also, a system booted from the (good) 20200131 AMI starts showing the
problem after updating only systemd (to 237-3ubuntu10.41) and its
direct dependencies (e.g. 'apt-get install systemd'). So I'm fairly
confident that a change to the systemd package between
237-3ubuntu10.33 and 237-3ubuntu10.39 introduced the problem and it is
still present.
On the NAT router I use three interfaces and have separate routing
tables for admin and forwarded traffic. Things come up fine initially
but every 30-60 minutes (DHCP lease renewal time?) one or more
interfaces is reconfigured and most of the time systemd-networkd will
crash and need to be restarted. Eventually the system becomes
unreachable when the default crash loop backoff logic prevents the
network service from being restarted at all. The log excerpt attached
illustrates the crash loop.
Also including the netplan and networkd config files
[Touch-packages] [Bug 1881972] Re: systemd-networkd crashes with invalid pointer
Hello John, or anyone else affected,
Accepted systemd into bionic-proposed. The package will build now and be
available at
https://launchpad.net/ubuntu/+source/systemd/237-3ubuntu10.42 in a few
hours, and then in the -proposed repository.
Please help us by testing this new package. See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
to enable and use -proposed. Your feedback will aid us getting this
update out to other Ubuntu users.
If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, what testing has been
performed on the package and change the tag from verification-needed-
bionic to verification-done-bionic. If it does not fix the bug for you,
please add a comment stating that, and change the tag to verification-
failed-bionic. In either case, without details of your testing we will
not be able to proceed.
Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in
advance for helping!
N.B. The updated package will be released to -updates after the bug(s)
fixed by this package have been verified and the package has been in
-proposed for a minimum of 7 days.
** Changed in: systemd (Ubuntu Bionic)
Status: In Progress => Fix Committed
** Tags added: verification-needed verification-needed-bionic
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1881972
Title:
systemd-networkd crashes with invalid pointer
Status in systemd package in Ubuntu:
Fix Released
Status in systemd source package in Bionic:
Fix Committed
Bug description:
[impact]
systemd-networkd double-free causes crash under some circumstances,
such as adding/removing ip rules
[test case]
Use networkd-dispatcher events to add and remove IP rules. The example
scripts below are contrived (and by themselves likely to break access
to a machine) but would be adequate to trigger the bug. Put scripts
like these in place, reboot or run "netplan apply", and then leave the
machine running for a few DHCP renewal cycles.
=== /etc/networkd-dispatcher/configured.d/test.sh ===
#!/bin/bash
/sbin/ip rule add iif lo lookup 99
/sbin/ip rule add to 10.0.0.0/8 iif lo lookup main
=== END ===
=== /etc/networkd-dispatcher/configuring.d/test.sh ===
#!/bin/bash
# Tear down existing ip rules so they aren't duplicated
OLDIFS="${IFS}"
IFS="
"
for rule in `ip rule show|grep "iif lo" | cut -d: -f2-`; do
IFS="${OLDIFS}"
ip rule delete ${rule}
done
IFS="${OLDIFS}"
=== END ===
[regression potential]
this strdup's strings during addition of routing policy rules, so any
regression would likely occur when adding/modifying/removing ip rules,
possibly including networkd segfault or failure to add/remove/modify
ip rules.
[scope]
this is needed for bionic.
this is fixed by upstream commit
eeab051b28ba6e1b4a56d369d4c6bf7cfa71947c which is included starting in
v240, so this is already included in Focal and later.
I did not research what original commit introduced the problem, but
the reporter indicates this did not happen for Xenial so it's unlikely
this is a problem in Xenial or earlier.
[original description]
This is a serious regression with systemd-networkd that I ran in to
while setting up a NAT router in AWS. The AWS AMI ubuntu/images/hvm-
ssd/ubuntu-bionic-18.04-amd64-server-20200131 with
systemd-237-3ubuntu10.33 does NOT have the problem, but the next most
recent AWS AMI ubuntu/images/hvm-ssd/ubuntu-
bionic-18.04-amd64-server-20200311 with systemd-including
237-3ubuntu10.39 does.
Also, a system booted from the (good) 20200131 AMI starts showing the
problem after updating only systemd (to 237-3ubuntu10.41) and its
direct dependencies (e.g. 'apt-get install systemd'). So I'm fairly
confident that a change to the systemd package between
237-3ubuntu10.33 and 237-3ubuntu10.39 introduced the problem and it is
still present.
On the NAT router I use three interfaces and have separate routing
tables for admin and forwarded traffic. Things come up fine initially
but every 30-60 minutes (DHCP lease renewal time?) one or more
interfaces is reconfigured and most of the time systemd-networkd will
crash and need to be restarted. Eventually the system becomes
unreachable when the default crash loop backoff logic prevents the
network service from being restarted at all. The log excerpt attached
illustrates the crash loop.
Also including the netplan and networkd config files below.
# grep . /etc/netplan/*
/etc/netplan/50-cloud-init.yaml:# This file is generated from information
provided by the datasource. Changes
/etc/netplan/50-cloud-init.yaml:# to it will not persist across an instance
reboot. To disable cloud-init's
[Touch-packages] [Bug 1881972] Re: systemd-networkd crashes with invalid pointer
** Changed in: systemd (Ubuntu Bionic)
Status: Incomplete => In Progress
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1881972
Title:
systemd-networkd crashes with invalid pointer
Status in systemd package in Ubuntu:
Fix Released
Status in systemd source package in Bionic:
In Progress
Bug description:
[impact]
systemd-networkd double-free causes crash under some circumstances,
such as adding/removing ip rules
[test case]
Use networkd-dispatcher events to add and remove IP rules. The example
scripts below are contrived (and by themselves likely to break access
to a machine) but would be adequate to trigger the bug. Put scripts
like these in place, reboot or run "netplan apply", and then leave the
machine running for a few DHCP renewal cycles.
=== /etc/networkd-dispatcher/configured.d/test.sh ===
#!/bin/bash
/sbin/ip rule add iif lo lookup 99
/sbin/ip rule add to 10.0.0.0/8 iif lo lookup main
=== END ===
=== /etc/networkd-dispatcher/configuring.d/test.sh ===
#!/bin/bash
# Tear down existing ip rules so they aren't duplicated
OLDIFS="${IFS}"
IFS="
"
for rule in `ip rule show|grep "iif lo" | cut -d: -f2-`; do
IFS="${OLDIFS}"
ip rule delete ${rule}
done
IFS="${OLDIFS}"
=== END ===
[regression potential]
this strdup's strings during addition of routing policy rules, so any
regression would likely occur when adding/modifying/removing ip rules,
possibly including networkd segfault or failure to add/remove/modify
ip rules.
[scope]
this is needed for bionic.
this is fixed by upstream commit
eeab051b28ba6e1b4a56d369d4c6bf7cfa71947c which is included starting in
v240, so this is already included in Focal and later.
I did not research what original commit introduced the problem, but
the reporter indicates this did not happen for Xenial so it's unlikely
this is a problem in Xenial or earlier.
[original description]
This is a serious regression with systemd-networkd that I ran in to
while setting up a NAT router in AWS. The AWS AMI ubuntu/images/hvm-
ssd/ubuntu-bionic-18.04-amd64-server-20200131 with
systemd-237-3ubuntu10.33 does NOT have the problem, but the next most
recent AWS AMI ubuntu/images/hvm-ssd/ubuntu-
bionic-18.04-amd64-server-20200311 with systemd-including
237-3ubuntu10.39 does.
Also, a system booted from the (good) 20200131 AMI starts showing the
problem after updating only systemd (to 237-3ubuntu10.41) and its
direct dependencies (e.g. 'apt-get install systemd'). So I'm fairly
confident that a change to the systemd package between
237-3ubuntu10.33 and 237-3ubuntu10.39 introduced the problem and it is
still present.
On the NAT router I use three interfaces and have separate routing
tables for admin and forwarded traffic. Things come up fine initially
but every 30-60 minutes (DHCP lease renewal time?) one or more
interfaces is reconfigured and most of the time systemd-networkd will
crash and need to be restarted. Eventually the system becomes
unreachable when the default crash loop backoff logic prevents the
network service from being restarted at all. The log excerpt attached
illustrates the crash loop.
Also including the netplan and networkd config files below.
# grep . /etc/netplan/*
/etc/netplan/50-cloud-init.yaml:# This file is generated from information
provided by the datasource. Changes
/etc/netplan/50-cloud-init.yaml:# to it will not persist across an instance
reboot. To disable cloud-init's
/etc/netplan/50-cloud-init.yaml:# network configuration capabilities, write a
file
/etc/netplan/50-cloud-init.yaml:#
/etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
/etc/netplan/50-cloud-init.yaml:# network: {config: disabled}
/etc/netplan/50-cloud-init.yaml:network:
/etc/netplan/50-cloud-init.yaml:version: 2
/etc/netplan/50-cloud-init.yaml:ethernets:
/etc/netplan/50-cloud-init.yaml:ens5:
/etc/netplan/50-cloud-init.yaml:dhcp4: true
/etc/netplan/50-cloud-init.yaml:match:
/etc/netplan/50-cloud-init.yaml:macaddress: xx:xx:xx:xx:xx:xx
/etc/netplan/50-cloud-init.yaml:set-name: ens5
/etc/netplan/99_config.yaml:network:
/etc/netplan/99_config.yaml: version: 2
/etc/netplan/99_config.yaml: renderer: networkd
/etc/netplan/99_config.yaml: ethernets:
/etc/netplan/99_config.yaml:ens6:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: yy:yy:yy:yy:yy:yy
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-routes: false
/etc/netplan/99_config.yaml:ens7:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: zz:zz:zz:zz:zz:zz
[Touch-packages] [Bug 1881972] Re: systemd-networkd crashes with invalid pointer
** Description changed:
[impact]
systemd-networkd double-free causes crash under some circumstances, such
as adding/removing ip rules
[test case]
- see original description
+ Use networkd-dispatcher events to add and remove IP rules. The example
+ scripts below are contrived (and by themselves likely to break access to
+ a machine) but would be adequate to trigger the bug. Put scripts like
+ these in place, reboot or run "netplan apply", and then leave the
+ machine running for a few DHCP renewal cycles.
+
+ === /etc/networkd-dispatcher/configured.d/test.sh ===
+ #!/bin/bash
+
+ /sbin/ip rule add iif lo lookup 99
+ /sbin/ip rule add to 10.0.0.0/8 iif lo lookup main
+ === END ===
+ === /etc/networkd-dispatcher/configuring.d/test.sh ===
+ #!/bin/bash
+
+ # Tear down existing ip rules so they aren't duplicated
+ OLDIFS="${IFS}"
+ IFS="
+ "
+ for rule in `ip rule show|grep "iif lo" | cut -d: -f2-`; do
+ IFS="${OLDIFS}"
+ ip rule delete ${rule}
+ done
+ IFS="${OLDIFS}"
+ === END ===
[regression potential]
this strdup's strings during addition of routing policy rules, so any
regression would likely occur when adding/modifying/removing ip rules,
possibly including networkd segfault or failure to add/remove/modify ip
rules.
[scope]
this is needed for bionic.
this is fixed by upstream commit
eeab051b28ba6e1b4a56d369d4c6bf7cfa71947c which is included starting in
v240, so this is already included in Focal and later.
I did not research what original commit introduced the problem, but the
reporter indicates this did not happen for Xenial so it's unlikely this
is a problem in Xenial or earlier.
[original description]
This is a serious regression with systemd-networkd that I ran in to
while setting up a NAT router in AWS. The AWS AMI ubuntu/images/hvm-ssd
/ubuntu-bionic-18.04-amd64-server-20200131 with systemd-237-3ubuntu10.33
does NOT have the problem, but the next most recent AWS AMI
ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-20200311 with
systemd-including 237-3ubuntu10.39 does.
Also, a system booted from the (good) 20200131 AMI starts showing the
problem after updating only systemd (to 237-3ubuntu10.41) and its direct
dependencies (e.g. 'apt-get install systemd'). So I'm fairly confident
that a change to the systemd package between 237-3ubuntu10.33 and
237-3ubuntu10.39 introduced the problem and it is still present.
On the NAT router I use three interfaces and have separate routing
tables for admin and forwarded traffic. Things come up fine initially
but every 30-60 minutes (DHCP lease renewal time?) one or more
interfaces is reconfigured and most of the time systemd-networkd will
crash and need to be restarted. Eventually the system becomes
unreachable when the default crash loop backoff logic prevents the
network service from being restarted at all. The log excerpt attached
illustrates the crash loop.
Also including the netplan and networkd config files below.
# grep . /etc/netplan/*
/etc/netplan/50-cloud-init.yaml:# This file is generated from information
provided by the datasource. Changes
/etc/netplan/50-cloud-init.yaml:# to it will not persist across an instance
reboot. To disable cloud-init's
/etc/netplan/50-cloud-init.yaml:# network configuration capabilities, write a
file
/etc/netplan/50-cloud-init.yaml:#
/etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
/etc/netplan/50-cloud-init.yaml:# network: {config: disabled}
/etc/netplan/50-cloud-init.yaml:network:
/etc/netplan/50-cloud-init.yaml:version: 2
/etc/netplan/50-cloud-init.yaml:ethernets:
/etc/netplan/50-cloud-init.yaml:ens5:
/etc/netplan/50-cloud-init.yaml:dhcp4: true
/etc/netplan/50-cloud-init.yaml:match:
/etc/netplan/50-cloud-init.yaml:macaddress: xx:xx:xx:xx:xx:xx
/etc/netplan/50-cloud-init.yaml:set-name: ens5
/etc/netplan/99_config.yaml:network:
/etc/netplan/99_config.yaml: version: 2
/etc/netplan/99_config.yaml: renderer: networkd
/etc/netplan/99_config.yaml: ethernets:
/etc/netplan/99_config.yaml:ens6:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: yy:yy:yy:yy:yy:yy
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-routes: false
/etc/netplan/99_config.yaml:ens7:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: zz:zz:zz:zz:zz:zz
/etc/netplan/99_config.yaml: mtu: 1500
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-mtu: false
/etc/netplan/99_config.yaml:use-routes: false
# grep . /etc/networkd-dispatcher/*/*
/etc/networkd-dispatcher/configured.d/nat:#!/bin/bash
[Touch-packages] [Bug 1881972] Re: systemd-networkd crashes with invalid pointer
@r-2ohn-d please do write up a test case for @vorlon, thanks
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1881972
Title:
systemd-networkd crashes with invalid pointer
Status in systemd package in Ubuntu:
Fix Released
Status in systemd source package in Bionic:
Incomplete
Bug description:
[impact]
systemd-networkd double-free causes crash under some circumstances,
such as adding/removing ip rules
[test case]
see original description
[regression potential]
this strdup's strings during addition of routing policy rules, so any
regression would likely occur when adding/modifying/removing ip rules,
possibly including networkd segfault or failure to add/remove/modify
ip rules.
[scope]
this is needed for bionic.
this is fixed by upstream commit
eeab051b28ba6e1b4a56d369d4c6bf7cfa71947c which is included starting in
v240, so this is already included in Focal and later.
I did not research what original commit introduced the problem, but
the reporter indicates this did not happen for Xenial so it's unlikely
this is a problem in Xenial or earlier.
[original description]
This is a serious regression with systemd-networkd that I ran in to
while setting up a NAT router in AWS. The AWS AMI ubuntu/images/hvm-
ssd/ubuntu-bionic-18.04-amd64-server-20200131 with
systemd-237-3ubuntu10.33 does NOT have the problem, but the next most
recent AWS AMI ubuntu/images/hvm-ssd/ubuntu-
bionic-18.04-amd64-server-20200311 with systemd-including
237-3ubuntu10.39 does.
Also, a system booted from the (good) 20200131 AMI starts showing the
problem after updating only systemd (to 237-3ubuntu10.41) and its
direct dependencies (e.g. 'apt-get install systemd'). So I'm fairly
confident that a change to the systemd package between
237-3ubuntu10.33 and 237-3ubuntu10.39 introduced the problem and it is
still present.
On the NAT router I use three interfaces and have separate routing
tables for admin and forwarded traffic. Things come up fine initially
but every 30-60 minutes (DHCP lease renewal time?) one or more
interfaces is reconfigured and most of the time systemd-networkd will
crash and need to be restarted. Eventually the system becomes
unreachable when the default crash loop backoff logic prevents the
network service from being restarted at all. The log excerpt attached
illustrates the crash loop.
Also including the netplan and networkd config files below.
# grep . /etc/netplan/*
/etc/netplan/50-cloud-init.yaml:# This file is generated from information
provided by the datasource. Changes
/etc/netplan/50-cloud-init.yaml:# to it will not persist across an instance
reboot. To disable cloud-init's
/etc/netplan/50-cloud-init.yaml:# network configuration capabilities, write a
file
/etc/netplan/50-cloud-init.yaml:#
/etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
/etc/netplan/50-cloud-init.yaml:# network: {config: disabled}
/etc/netplan/50-cloud-init.yaml:network:
/etc/netplan/50-cloud-init.yaml:version: 2
/etc/netplan/50-cloud-init.yaml:ethernets:
/etc/netplan/50-cloud-init.yaml:ens5:
/etc/netplan/50-cloud-init.yaml:dhcp4: true
/etc/netplan/50-cloud-init.yaml:match:
/etc/netplan/50-cloud-init.yaml:macaddress: xx:xx:xx:xx:xx:xx
/etc/netplan/50-cloud-init.yaml:set-name: ens5
/etc/netplan/99_config.yaml:network:
/etc/netplan/99_config.yaml: version: 2
/etc/netplan/99_config.yaml: renderer: networkd
/etc/netplan/99_config.yaml: ethernets:
/etc/netplan/99_config.yaml:ens6:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: yy:yy:yy:yy:yy:yy
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-routes: false
/etc/netplan/99_config.yaml:ens7:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: zz:zz:zz:zz:zz:zz
/etc/netplan/99_config.yaml: mtu: 1500
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-mtu: false
/etc/netplan/99_config.yaml:use-routes: false
# grep . /etc/networkd-dispatcher/*/*
/etc/networkd-dispatcher/configured.d/nat:#!/bin/bash
/etc/networkd-dispatcher/configured.d/nat:# Do additional configuration for
the inside and outside interfaces
/etc/networkd-dispatcher/configured.d/nat:# route table used for
forwarded/routed/natted traffic
/etc/networkd-dispatcher/configured.d/nat:FWD_TABLE=99
/etc/networkd-dispatcher/configured.d/nat:if [ "${IFACE}" = "ens6" ]; then
/etc/networkd-dispatcher/configured.d/nat: # delete link-local route for
inside in default table
[Touch-packages] [Bug 1881972] Re: systemd-networkd crashes with invalid pointer
The scripts for configured.d and configuring.d to add and remove IP
rules (included above) are likely the culprit. @ddstreet would you like
me to write that up more compactly?
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1881972
Title:
systemd-networkd crashes with invalid pointer
Status in systemd package in Ubuntu:
Fix Released
Status in systemd source package in Bionic:
Incomplete
Bug description:
[impact]
systemd-networkd double-free causes crash under some circumstances,
such as adding/removing ip rules
[test case]
see original description
[regression potential]
this strdup's strings during addition of routing policy rules, so any
regression would likely occur when adding/modifying/removing ip rules,
possibly including networkd segfault or failure to add/remove/modify
ip rules.
[scope]
this is needed for bionic.
this is fixed by upstream commit
eeab051b28ba6e1b4a56d369d4c6bf7cfa71947c which is included starting in
v240, so this is already included in Focal and later.
I did not research what original commit introduced the problem, but
the reporter indicates this did not happen for Xenial so it's unlikely
this is a problem in Xenial or earlier.
[original description]
This is a serious regression with systemd-networkd that I ran in to
while setting up a NAT router in AWS. The AWS AMI ubuntu/images/hvm-
ssd/ubuntu-bionic-18.04-amd64-server-20200131 with
systemd-237-3ubuntu10.33 does NOT have the problem, but the next most
recent AWS AMI ubuntu/images/hvm-ssd/ubuntu-
bionic-18.04-amd64-server-20200311 with systemd-including
237-3ubuntu10.39 does.
Also, a system booted from the (good) 20200131 AMI starts showing the
problem after updating only systemd (to 237-3ubuntu10.41) and its
direct dependencies (e.g. 'apt-get install systemd'). So I'm fairly
confident that a change to the systemd package between
237-3ubuntu10.33 and 237-3ubuntu10.39 introduced the problem and it is
still present.
On the NAT router I use three interfaces and have separate routing
tables for admin and forwarded traffic. Things come up fine initially
but every 30-60 minutes (DHCP lease renewal time?) one or more
interfaces is reconfigured and most of the time systemd-networkd will
crash and need to be restarted. Eventually the system becomes
unreachable when the default crash loop backoff logic prevents the
network service from being restarted at all. The log excerpt attached
illustrates the crash loop.
Also including the netplan and networkd config files below.
# grep . /etc/netplan/*
/etc/netplan/50-cloud-init.yaml:# This file is generated from information
provided by the datasource. Changes
/etc/netplan/50-cloud-init.yaml:# to it will not persist across an instance
reboot. To disable cloud-init's
/etc/netplan/50-cloud-init.yaml:# network configuration capabilities, write a
file
/etc/netplan/50-cloud-init.yaml:#
/etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
/etc/netplan/50-cloud-init.yaml:# network: {config: disabled}
/etc/netplan/50-cloud-init.yaml:network:
/etc/netplan/50-cloud-init.yaml:version: 2
/etc/netplan/50-cloud-init.yaml:ethernets:
/etc/netplan/50-cloud-init.yaml:ens5:
/etc/netplan/50-cloud-init.yaml:dhcp4: true
/etc/netplan/50-cloud-init.yaml:match:
/etc/netplan/50-cloud-init.yaml:macaddress: xx:xx:xx:xx:xx:xx
/etc/netplan/50-cloud-init.yaml:set-name: ens5
/etc/netplan/99_config.yaml:network:
/etc/netplan/99_config.yaml: version: 2
/etc/netplan/99_config.yaml: renderer: networkd
/etc/netplan/99_config.yaml: ethernets:
/etc/netplan/99_config.yaml:ens6:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: yy:yy:yy:yy:yy:yy
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-routes: false
/etc/netplan/99_config.yaml:ens7:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: zz:zz:zz:zz:zz:zz
/etc/netplan/99_config.yaml: mtu: 1500
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-mtu: false
/etc/netplan/99_config.yaml:use-routes: false
# grep . /etc/networkd-dispatcher/*/*
/etc/networkd-dispatcher/configured.d/nat:#!/bin/bash
/etc/networkd-dispatcher/configured.d/nat:# Do additional configuration for
the inside and outside interfaces
/etc/networkd-dispatcher/configured.d/nat:# route table used for
forwarded/routed/natted traffic
/etc/networkd-dispatcher/configured.d/nat:FWD_TABLE=99
/etc/networkd-dispatcher/configured.d/nat:if [ "${IFACE}" = "ens6" ]; then
[Touch-packages] [Bug 1881972] Re: systemd-networkd crashes with invalid pointer
"see original description" is inadequate as a test case. I am reading
the original description and it does not describe a step-by-step
reproducer.
** Changed in: systemd (Ubuntu Bionic)
Status: In Progress => Incomplete
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1881972
Title:
systemd-networkd crashes with invalid pointer
Status in systemd package in Ubuntu:
Fix Released
Status in systemd source package in Bionic:
Incomplete
Bug description:
[impact]
systemd-networkd double-free causes crash under some circumstances,
such as adding/removing ip rules
[test case]
see original description
[regression potential]
this strdup's strings during addition of routing policy rules, so any
regression would likely occur when adding/modifying/removing ip rules,
possibly including networkd segfault or failure to add/remove/modify
ip rules.
[scope]
this is needed for bionic.
this is fixed by upstream commit
eeab051b28ba6e1b4a56d369d4c6bf7cfa71947c which is included starting in
v240, so this is already included in Focal and later.
I did not research what original commit introduced the problem, but
the reporter indicates this did not happen for Xenial so it's unlikely
this is a problem in Xenial or earlier.
[original description]
This is a serious regression with systemd-networkd that I ran in to
while setting up a NAT router in AWS. The AWS AMI ubuntu/images/hvm-
ssd/ubuntu-bionic-18.04-amd64-server-20200131 with
systemd-237-3ubuntu10.33 does NOT have the problem, but the next most
recent AWS AMI ubuntu/images/hvm-ssd/ubuntu-
bionic-18.04-amd64-server-20200311 with systemd-including
237-3ubuntu10.39 does.
Also, a system booted from the (good) 20200131 AMI starts showing the
problem after updating only systemd (to 237-3ubuntu10.41) and its
direct dependencies (e.g. 'apt-get install systemd'). So I'm fairly
confident that a change to the systemd package between
237-3ubuntu10.33 and 237-3ubuntu10.39 introduced the problem and it is
still present.
On the NAT router I use three interfaces and have separate routing
tables for admin and forwarded traffic. Things come up fine initially
but every 30-60 minutes (DHCP lease renewal time?) one or more
interfaces is reconfigured and most of the time systemd-networkd will
crash and need to be restarted. Eventually the system becomes
unreachable when the default crash loop backoff logic prevents the
network service from being restarted at all. The log excerpt attached
illustrates the crash loop.
Also including the netplan and networkd config files below.
# grep . /etc/netplan/*
/etc/netplan/50-cloud-init.yaml:# This file is generated from information
provided by the datasource. Changes
/etc/netplan/50-cloud-init.yaml:# to it will not persist across an instance
reboot. To disable cloud-init's
/etc/netplan/50-cloud-init.yaml:# network configuration capabilities, write a
file
/etc/netplan/50-cloud-init.yaml:#
/etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
/etc/netplan/50-cloud-init.yaml:# network: {config: disabled}
/etc/netplan/50-cloud-init.yaml:network:
/etc/netplan/50-cloud-init.yaml:version: 2
/etc/netplan/50-cloud-init.yaml:ethernets:
/etc/netplan/50-cloud-init.yaml:ens5:
/etc/netplan/50-cloud-init.yaml:dhcp4: true
/etc/netplan/50-cloud-init.yaml:match:
/etc/netplan/50-cloud-init.yaml:macaddress: xx:xx:xx:xx:xx:xx
/etc/netplan/50-cloud-init.yaml:set-name: ens5
/etc/netplan/99_config.yaml:network:
/etc/netplan/99_config.yaml: version: 2
/etc/netplan/99_config.yaml: renderer: networkd
/etc/netplan/99_config.yaml: ethernets:
/etc/netplan/99_config.yaml:ens6:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: yy:yy:yy:yy:yy:yy
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-routes: false
/etc/netplan/99_config.yaml:ens7:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: zz:zz:zz:zz:zz:zz
/etc/netplan/99_config.yaml: mtu: 1500
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-mtu: false
/etc/netplan/99_config.yaml:use-routes: false
# grep . /etc/networkd-dispatcher/*/*
/etc/networkd-dispatcher/configured.d/nat:#!/bin/bash
/etc/networkd-dispatcher/configured.d/nat:# Do additional configuration for
the inside and outside interfaces
/etc/networkd-dispatcher/configured.d/nat:# route table used for
forwarded/routed/natted traffic
/etc/networkd-dispatcher/configured.d/nat:FWD_TABLE=99
[Touch-packages] [Bug 1881972] Re: systemd-networkd crashes with invalid pointer
No crashes on my test machine for 12 days. Push it!
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1881972
Title:
systemd-networkd crashes with invalid pointer
Status in systemd package in Ubuntu:
Fix Released
Status in systemd source package in Bionic:
In Progress
Bug description:
[impact]
systemd-networkd double-free causes crash under some circumstances,
such as adding/removing ip rules
[test case]
see original description
[regression potential]
this strdup's strings during addition of routing policy rules, so any
regression would likely occur when adding/modifying/removing ip rules,
possibly including networkd segfault or failure to add/remove/modify
ip rules.
[scope]
this is needed for bionic.
this is fixed by upstream commit
eeab051b28ba6e1b4a56d369d4c6bf7cfa71947c which is included starting in
v240, so this is already included in Focal and later.
I did not research what original commit introduced the problem, but
the reporter indicates this did not happen for Xenial so it's unlikely
this is a problem in Xenial or earlier.
[original description]
This is a serious regression with systemd-networkd that I ran in to
while setting up a NAT router in AWS. The AWS AMI ubuntu/images/hvm-
ssd/ubuntu-bionic-18.04-amd64-server-20200131 with
systemd-237-3ubuntu10.33 does NOT have the problem, but the next most
recent AWS AMI ubuntu/images/hvm-ssd/ubuntu-
bionic-18.04-amd64-server-20200311 with systemd-including
237-3ubuntu10.39 does.
Also, a system booted from the (good) 20200131 AMI starts showing the
problem after updating only systemd (to 237-3ubuntu10.41) and its
direct dependencies (e.g. 'apt-get install systemd'). So I'm fairly
confident that a change to the systemd package between
237-3ubuntu10.33 and 237-3ubuntu10.39 introduced the problem and it is
still present.
On the NAT router I use three interfaces and have separate routing
tables for admin and forwarded traffic. Things come up fine initially
but every 30-60 minutes (DHCP lease renewal time?) one or more
interfaces is reconfigured and most of the time systemd-networkd will
crash and need to be restarted. Eventually the system becomes
unreachable when the default crash loop backoff logic prevents the
network service from being restarted at all. The log excerpt attached
illustrates the crash loop.
Also including the netplan and networkd config files below.
# grep . /etc/netplan/*
/etc/netplan/50-cloud-init.yaml:# This file is generated from information
provided by the datasource. Changes
/etc/netplan/50-cloud-init.yaml:# to it will not persist across an instance
reboot. To disable cloud-init's
/etc/netplan/50-cloud-init.yaml:# network configuration capabilities, write a
file
/etc/netplan/50-cloud-init.yaml:#
/etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
/etc/netplan/50-cloud-init.yaml:# network: {config: disabled}
/etc/netplan/50-cloud-init.yaml:network:
/etc/netplan/50-cloud-init.yaml:version: 2
/etc/netplan/50-cloud-init.yaml:ethernets:
/etc/netplan/50-cloud-init.yaml:ens5:
/etc/netplan/50-cloud-init.yaml:dhcp4: true
/etc/netplan/50-cloud-init.yaml:match:
/etc/netplan/50-cloud-init.yaml:macaddress: xx:xx:xx:xx:xx:xx
/etc/netplan/50-cloud-init.yaml:set-name: ens5
/etc/netplan/99_config.yaml:network:
/etc/netplan/99_config.yaml: version: 2
/etc/netplan/99_config.yaml: renderer: networkd
/etc/netplan/99_config.yaml: ethernets:
/etc/netplan/99_config.yaml:ens6:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: yy:yy:yy:yy:yy:yy
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-routes: false
/etc/netplan/99_config.yaml:ens7:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: zz:zz:zz:zz:zz:zz
/etc/netplan/99_config.yaml: mtu: 1500
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-mtu: false
/etc/netplan/99_config.yaml:use-routes: false
# grep . /etc/networkd-dispatcher/*/*
/etc/networkd-dispatcher/configured.d/nat:#!/bin/bash
/etc/networkd-dispatcher/configured.d/nat:# Do additional configuration for
the inside and outside interfaces
/etc/networkd-dispatcher/configured.d/nat:# route table used for
forwarded/routed/natted traffic
/etc/networkd-dispatcher/configured.d/nat:FWD_TABLE=99
/etc/networkd-dispatcher/configured.d/nat:if [ "${IFACE}" = "ens6" ]; then
/etc/networkd-dispatcher/configured.d/nat: # delete link-local route for
inside in default table
[Touch-packages] [Bug 1881972] Re: systemd-networkd crashes with invalid pointer
So far so good running the latest package for 10 hours. I'll let it run
another day or two but previously I would have seen the issue by now.
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1881972
Title:
systemd-networkd crashes with invalid pointer
Status in systemd package in Ubuntu:
Fix Released
Status in systemd source package in Bionic:
In Progress
Bug description:
[impact]
systemd-networkd double-free causes crash under some circumstances,
such as adding/removing ip rules
[test case]
see original description
[regression potential]
this strdup's strings during addition of routing policy rules, so any
regression would likely occur when adding/modifying/removing ip rules,
possibly including networkd segfault or failure to add/remove/modify
ip rules.
[scope]
this is needed for bionic.
this is fixed by upstream commit
eeab051b28ba6e1b4a56d369d4c6bf7cfa71947c which is included starting in
v240, so this is already included in Focal and later.
I did not research what original commit introduced the problem, but
the reporter indicates this did not happen for Xenial so it's unlikely
this is a problem in Xenial or earlier.
[original description]
This is a serious regression with systemd-networkd that I ran in to
while setting up a NAT router in AWS. The AWS AMI ubuntu/images/hvm-
ssd/ubuntu-bionic-18.04-amd64-server-20200131 with
systemd-237-3ubuntu10.33 does NOT have the problem, but the next most
recent AWS AMI ubuntu/images/hvm-ssd/ubuntu-
bionic-18.04-amd64-server-20200311 with systemd-including
237-3ubuntu10.39 does.
Also, a system booted from the (good) 20200131 AMI starts showing the
problem after updating only systemd (to 237-3ubuntu10.41) and its
direct dependencies (e.g. 'apt-get install systemd'). So I'm fairly
confident that a change to the systemd package between
237-3ubuntu10.33 and 237-3ubuntu10.39 introduced the problem and it is
still present.
On the NAT router I use three interfaces and have separate routing
tables for admin and forwarded traffic. Things come up fine initially
but every 30-60 minutes (DHCP lease renewal time?) one or more
interfaces is reconfigured and most of the time systemd-networkd will
crash and need to be restarted. Eventually the system becomes
unreachable when the default crash loop backoff logic prevents the
network service from being restarted at all. The log excerpt attached
illustrates the crash loop.
Also including the netplan and networkd config files below.
# grep . /etc/netplan/*
/etc/netplan/50-cloud-init.yaml:# This file is generated from information
provided by the datasource. Changes
/etc/netplan/50-cloud-init.yaml:# to it will not persist across an instance
reboot. To disable cloud-init's
/etc/netplan/50-cloud-init.yaml:# network configuration capabilities, write a
file
/etc/netplan/50-cloud-init.yaml:#
/etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
/etc/netplan/50-cloud-init.yaml:# network: {config: disabled}
/etc/netplan/50-cloud-init.yaml:network:
/etc/netplan/50-cloud-init.yaml:version: 2
/etc/netplan/50-cloud-init.yaml:ethernets:
/etc/netplan/50-cloud-init.yaml:ens5:
/etc/netplan/50-cloud-init.yaml:dhcp4: true
/etc/netplan/50-cloud-init.yaml:match:
/etc/netplan/50-cloud-init.yaml:macaddress: xx:xx:xx:xx:xx:xx
/etc/netplan/50-cloud-init.yaml:set-name: ens5
/etc/netplan/99_config.yaml:network:
/etc/netplan/99_config.yaml: version: 2
/etc/netplan/99_config.yaml: renderer: networkd
/etc/netplan/99_config.yaml: ethernets:
/etc/netplan/99_config.yaml:ens6:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: yy:yy:yy:yy:yy:yy
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-routes: false
/etc/netplan/99_config.yaml:ens7:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: zz:zz:zz:zz:zz:zz
/etc/netplan/99_config.yaml: mtu: 1500
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-mtu: false
/etc/netplan/99_config.yaml:use-routes: false
# grep . /etc/networkd-dispatcher/*/*
/etc/networkd-dispatcher/configured.d/nat:#!/bin/bash
/etc/networkd-dispatcher/configured.d/nat:# Do additional configuration for
the inside and outside interfaces
/etc/networkd-dispatcher/configured.d/nat:# route table used for
forwarded/routed/natted traffic
/etc/networkd-dispatcher/configured.d/nat:FWD_TABLE=99
/etc/networkd-dispatcher/configured.d/nat:if [ "${IFACE}" = "ens6" ]; then
[Touch-packages] [Bug 1881972] Re: systemd-networkd crashes with invalid pointer
I was not able to reproduce the original issue on
237-3ubuntu10.42~202007071725~ubuntu18.04.1 after letting it run for 12+
hours. I have now installed the newer
237-3ubuntu10.42~202007081907~ubuntu18.04.1 from the same PPA. I no
longer see a SEGV when the service first starts at boot, thanks! I will
let it run a few hours again to confirm that the original issue has been
addressed.
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1881972
Title:
systemd-networkd crashes with invalid pointer
Status in systemd package in Ubuntu:
Fix Released
Status in systemd source package in Bionic:
In Progress
Bug description:
[impact]
systemd-networkd double-free causes crash under some circumstances,
such as adding/removing ip rules
[test case]
see original description
[regression potential]
this strdup's strings during addition of routing policy rules, so any
regression would likely occur when adding/modifying/removing ip rules,
possibly including networkd segfault or failure to add/remove/modify
ip rules.
[scope]
this is needed for bionic.
this is fixed by upstream commit
eeab051b28ba6e1b4a56d369d4c6bf7cfa71947c which is included starting in
v240, so this is already included in Focal and later.
I did not research what original commit introduced the problem, but
the reporter indicates this did not happen for Xenial so it's unlikely
this is a problem in Xenial or earlier.
[original description]
This is a serious regression with systemd-networkd that I ran in to
while setting up a NAT router in AWS. The AWS AMI ubuntu/images/hvm-
ssd/ubuntu-bionic-18.04-amd64-server-20200131 with
systemd-237-3ubuntu10.33 does NOT have the problem, but the next most
recent AWS AMI ubuntu/images/hvm-ssd/ubuntu-
bionic-18.04-amd64-server-20200311 with systemd-including
237-3ubuntu10.39 does.
Also, a system booted from the (good) 20200131 AMI starts showing the
problem after updating only systemd (to 237-3ubuntu10.41) and its
direct dependencies (e.g. 'apt-get install systemd'). So I'm fairly
confident that a change to the systemd package between
237-3ubuntu10.33 and 237-3ubuntu10.39 introduced the problem and it is
still present.
On the NAT router I use three interfaces and have separate routing
tables for admin and forwarded traffic. Things come up fine initially
but every 30-60 minutes (DHCP lease renewal time?) one or more
interfaces is reconfigured and most of the time systemd-networkd will
crash and need to be restarted. Eventually the system becomes
unreachable when the default crash loop backoff logic prevents the
network service from being restarted at all. The log excerpt attached
illustrates the crash loop.
Also including the netplan and networkd config files below.
# grep . /etc/netplan/*
/etc/netplan/50-cloud-init.yaml:# This file is generated from information
provided by the datasource. Changes
/etc/netplan/50-cloud-init.yaml:# to it will not persist across an instance
reboot. To disable cloud-init's
/etc/netplan/50-cloud-init.yaml:# network configuration capabilities, write a
file
/etc/netplan/50-cloud-init.yaml:#
/etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
/etc/netplan/50-cloud-init.yaml:# network: {config: disabled}
/etc/netplan/50-cloud-init.yaml:network:
/etc/netplan/50-cloud-init.yaml:version: 2
/etc/netplan/50-cloud-init.yaml:ethernets:
/etc/netplan/50-cloud-init.yaml:ens5:
/etc/netplan/50-cloud-init.yaml:dhcp4: true
/etc/netplan/50-cloud-init.yaml:match:
/etc/netplan/50-cloud-init.yaml:macaddress: xx:xx:xx:xx:xx:xx
/etc/netplan/50-cloud-init.yaml:set-name: ens5
/etc/netplan/99_config.yaml:network:
/etc/netplan/99_config.yaml: version: 2
/etc/netplan/99_config.yaml: renderer: networkd
/etc/netplan/99_config.yaml: ethernets:
/etc/netplan/99_config.yaml:ens6:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: yy:yy:yy:yy:yy:yy
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-routes: false
/etc/netplan/99_config.yaml:ens7:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: zz:zz:zz:zz:zz:zz
/etc/netplan/99_config.yaml: mtu: 1500
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-mtu: false
/etc/netplan/99_config.yaml:use-routes: false
# grep . /etc/networkd-dispatcher/*/*
/etc/networkd-dispatcher/configured.d/nat:#!/bin/bash
/etc/networkd-dispatcher/configured.d/nat:# Do additional configuration for
the inside and outside interfaces
[Touch-packages] [Bug 1881972] Re: systemd-networkd crashes with invalid pointer
thanks! i'd missed a patch in backporting for another bug.
a new version is building now in the same ppa, could you test with it
once it's built? Specifically, version
237-3ubuntu10.42~202007081907~ubuntu18.04.1 (or later)
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1881972
Title:
systemd-networkd crashes with invalid pointer
Status in systemd package in Ubuntu:
Fix Released
Status in systemd source package in Bionic:
In Progress
Bug description:
[impact]
systemd-networkd double-free causes crash under some circumstances,
such as adding/removing ip rules
[test case]
see original description
[regression potential]
this strdup's strings during addition of routing policy rules, so any
regression would likely occur when adding/modifying/removing ip rules,
possibly including networkd segfault or failure to add/remove/modify
ip rules.
[scope]
this is needed for bionic.
this is fixed by upstream commit
eeab051b28ba6e1b4a56d369d4c6bf7cfa71947c which is included starting in
v240, so this is already included in Focal and later.
I did not research what original commit introduced the problem, but
the reporter indicates this did not happen for Xenial so it's unlikely
this is a problem in Xenial or earlier.
[original description]
This is a serious regression with systemd-networkd that I ran in to
while setting up a NAT router in AWS. The AWS AMI ubuntu/images/hvm-
ssd/ubuntu-bionic-18.04-amd64-server-20200131 with
systemd-237-3ubuntu10.33 does NOT have the problem, but the next most
recent AWS AMI ubuntu/images/hvm-ssd/ubuntu-
bionic-18.04-amd64-server-20200311 with systemd-including
237-3ubuntu10.39 does.
Also, a system booted from the (good) 20200131 AMI starts showing the
problem after updating only systemd (to 237-3ubuntu10.41) and its
direct dependencies (e.g. 'apt-get install systemd'). So I'm fairly
confident that a change to the systemd package between
237-3ubuntu10.33 and 237-3ubuntu10.39 introduced the problem and it is
still present.
On the NAT router I use three interfaces and have separate routing
tables for admin and forwarded traffic. Things come up fine initially
but every 30-60 minutes (DHCP lease renewal time?) one or more
interfaces is reconfigured and most of the time systemd-networkd will
crash and need to be restarted. Eventually the system becomes
unreachable when the default crash loop backoff logic prevents the
network service from being restarted at all. The log excerpt attached
illustrates the crash loop.
Also including the netplan and networkd config files below.
# grep . /etc/netplan/*
/etc/netplan/50-cloud-init.yaml:# This file is generated from information
provided by the datasource. Changes
/etc/netplan/50-cloud-init.yaml:# to it will not persist across an instance
reboot. To disable cloud-init's
/etc/netplan/50-cloud-init.yaml:# network configuration capabilities, write a
file
/etc/netplan/50-cloud-init.yaml:#
/etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
/etc/netplan/50-cloud-init.yaml:# network: {config: disabled}
/etc/netplan/50-cloud-init.yaml:network:
/etc/netplan/50-cloud-init.yaml:version: 2
/etc/netplan/50-cloud-init.yaml:ethernets:
/etc/netplan/50-cloud-init.yaml:ens5:
/etc/netplan/50-cloud-init.yaml:dhcp4: true
/etc/netplan/50-cloud-init.yaml:match:
/etc/netplan/50-cloud-init.yaml:macaddress: xx:xx:xx:xx:xx:xx
/etc/netplan/50-cloud-init.yaml:set-name: ens5
/etc/netplan/99_config.yaml:network:
/etc/netplan/99_config.yaml: version: 2
/etc/netplan/99_config.yaml: renderer: networkd
/etc/netplan/99_config.yaml: ethernets:
/etc/netplan/99_config.yaml:ens6:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: yy:yy:yy:yy:yy:yy
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-routes: false
/etc/netplan/99_config.yaml:ens7:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: zz:zz:zz:zz:zz:zz
/etc/netplan/99_config.yaml: mtu: 1500
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-mtu: false
/etc/netplan/99_config.yaml:use-routes: false
# grep . /etc/networkd-dispatcher/*/*
/etc/networkd-dispatcher/configured.d/nat:#!/bin/bash
/etc/networkd-dispatcher/configured.d/nat:# Do additional configuration for
the inside and outside interfaces
/etc/networkd-dispatcher/configured.d/nat:# route table used for
forwarded/routed/natted traffic
/etc/networkd-dispatcher/configured.d/nat:FWD_TABLE=99
[Touch-packages] [Bug 1881972] Re: systemd-networkd crashes with invalid pointer
Here's one of the new coredumps I'm getting at boot now. Note that I don't have debugging symbols installed for the PPA version of systemd. # coredumpctl gdb 714 PID: 714 (systemd-network) UID: 100 (systemd-network) GID: 102 (systemd-network) Signal: 11 (SEGV) Timestamp: Wed 2020-07-08 17:17:01 UTC (8min ago) Command Line: /lib/systemd/systemd-networkd Executable: /lib/systemd/systemd-networkd Control Group: /system.slice/systemd-networkd.service Unit: systemd-networkd.service Slice: system.slice Boot ID: df33bbaec4134b45aaabe8b3fca7dade Machine ID: ec267b3475883f9edb99f554607bb456 Hostname: ip-10-0-4-251 Storage: /var/lib/systemd/coredump/core.systemd-network.100.df33bbaec4134b45aaabe8b3fca7dade.714.159422862100.lz4 Message: Process 714 (systemd-network) of user 100 dumped core. Stack trace of thread 714: #0 0x5627a7f3425c n/a (systemd-networkd) #1 0x5627a7fb5760 n/a (systemd-networkd) #2 0x5627a7f26526 sd_netlink_process (systemd-networkd) #3 0x5627a7f267c3 n/a (systemd-networkd) #4 0x5627a7f2b6be n/a (systemd-networkd) #5 0x5627a7f2b93a sd_event_dispatch (systemd-networkd) #6 0x5627a7f2bac9 sd_event_run (systemd-networkd) #7 0x5627a7f2bd0b sd_event_loop (systemd-networkd) #8 0x5627a7eff3d6 n/a (systemd-networkd) #9 0x7f6d90500b97 __libc_start_main (libc.so.6) #10 0x5627a7effaba n/a (systemd-networkd) ** Attachment added: "core.systemd-network.100.df33bbaec4134b45aaabe8b3fca7dade.714.159422862100.lz4" https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1881972/+attachment/5390826/+files/core.systemd-network.100.df33bbaec4134b45aaabe8b3fca7dade.714.159422862100.lz4 -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1881972 Title: systemd-networkd crashes with invalid pointer Status in systemd package in Ubuntu: Fix Released Status in systemd source package in Bionic: In Progress Bug description: [impact] systemd-networkd double-free causes crash under some circumstances, such as adding/removing ip rules [test case] see original description [regression potential] this strdup's strings during addition of routing policy rules, so any regression would likely occur when adding/modifying/removing ip rules, possibly including networkd segfault or failure to add/remove/modify ip rules. [scope] this is needed for bionic. this is fixed by upstream commit eeab051b28ba6e1b4a56d369d4c6bf7cfa71947c which is included starting in v240, so this is already included in Focal and later. I did not research what original commit introduced the problem, but the reporter indicates this did not happen for Xenial so it's unlikely this is a problem in Xenial or earlier. [original description] This is a serious regression with systemd-networkd that I ran in to while setting up a NAT router in AWS. The AWS AMI ubuntu/images/hvm- ssd/ubuntu-bionic-18.04-amd64-server-20200131 with systemd-237-3ubuntu10.33 does NOT have the problem, but the next most recent AWS AMI ubuntu/images/hvm-ssd/ubuntu- bionic-18.04-amd64-server-20200311 with systemd-including 237-3ubuntu10.39 does. Also, a system booted from the (good) 20200131 AMI starts showing the problem after updating only systemd (to 237-3ubuntu10.41) and its direct dependencies (e.g. 'apt-get install systemd'). So I'm fairly confident that a change to the systemd package between 237-3ubuntu10.33 and 237-3ubuntu10.39 introduced the problem and it is still present. On the NAT router I use three interfaces and have separate routing tables for admin and forwarded traffic. Things come up fine initially but every 30-60 minutes (DHCP lease renewal time?) one or more interfaces is reconfigured and most of the time systemd-networkd will crash and need to be restarted. Eventually the system becomes unreachable when the default crash loop backoff logic prevents the network service from being restarted at all. The log excerpt attached illustrates the crash loop. Also including the netplan and networkd config files below. # grep . /etc/netplan/* /etc/netplan/50-cloud-init.yaml:# This file is generated from information provided by the datasource. Changes /etc/netplan/50-cloud-init.yaml:# to it will not persist across an instance reboot. To disable cloud-init's /etc/netplan/50-cloud-init.yaml:# network configuration capabilities, write a file /etc/netplan/50-cloud-init.yaml:# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
[Touch-packages] [Bug 1881972] Re: systemd-networkd crashes with invalid pointer
I added the ppa and did a dist-upgrade then rebooted. systemd-networkd
now consistently crashes once at boot (looks like a different crash
though). But then everything appears to work after networkd restarts
once. I will let it run and see if the invalid pointer crash happens.
# journalctl -l -u systemd-networkd -b 0
-- Logs begin at Wed 2020-06-03 15:00:29 UTC, end at Wed 2020-07-08 17:20:23
UTC. --
Jul 08 17:17:01 ip-10-0-4-251 systemd[1]: Starting Network Service...
Jul 08 17:17:01 ip-10-0-4-251 systemd-networkd[714]: Enumeration completed
Jul 08 17:17:01 ip-10-0-4-251 systemd[1]: Started Network Service.
Jul 08 17:17:01 ip-10-0-4-251 systemd-networkd[714]: ens5: Link UP
Jul 08 17:17:01 ip-10-0-4-251 systemd-networkd[714]: ens5: Gained carrier
Jul 08 17:17:01 ip-10-0-4-251 systemd-networkd[714]: ens5: Link DOWN
Jul 08 17:17:01 ip-10-0-4-251 systemd-networkd[714]: ens5: Lost carrier
Jul 08 17:17:01 ip-10-0-4-251 systemd[1]: systemd-networkd.service: Main
process exited, code=dumped, status=11/SEGV
Jul 08 17:17:01 ip-10-0-4-251 systemd[1]: systemd-networkd.service: Failed with
result 'core-dump'.
Jul 08 17:17:01 ip-10-0-4-251 systemd[1]: systemd-networkd.service: Service has
no hold-off time, scheduling restart.
Jul 08 17:17:01 ip-10-0-4-251 systemd[1]: systemd-networkd.service: Scheduled
restart job, restart counter is at 1.
Jul 08 17:17:01 ip-10-0-4-251 systemd[1]: Stopped Network Service.
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1881972
Title:
systemd-networkd crashes with invalid pointer
Status in systemd package in Ubuntu:
Fix Released
Status in systemd source package in Bionic:
In Progress
Bug description:
[impact]
systemd-networkd double-free causes crash under some circumstances,
such as adding/removing ip rules
[test case]
see original description
[regression potential]
this strdup's strings during addition of routing policy rules, so any
regression would likely occur when adding/modifying/removing ip rules,
possibly including networkd segfault or failure to add/remove/modify
ip rules.
[scope]
this is needed for bionic.
this is fixed by upstream commit
eeab051b28ba6e1b4a56d369d4c6bf7cfa71947c which is included starting in
v240, so this is already included in Focal and later.
I did not research what original commit introduced the problem, but
the reporter indicates this did not happen for Xenial so it's unlikely
this is a problem in Xenial or earlier.
[original description]
This is a serious regression with systemd-networkd that I ran in to
while setting up a NAT router in AWS. The AWS AMI ubuntu/images/hvm-
ssd/ubuntu-bionic-18.04-amd64-server-20200131 with
systemd-237-3ubuntu10.33 does NOT have the problem, but the next most
recent AWS AMI ubuntu/images/hvm-ssd/ubuntu-
bionic-18.04-amd64-server-20200311 with systemd-including
237-3ubuntu10.39 does.
Also, a system booted from the (good) 20200131 AMI starts showing the
problem after updating only systemd (to 237-3ubuntu10.41) and its
direct dependencies (e.g. 'apt-get install systemd'). So I'm fairly
confident that a change to the systemd package between
237-3ubuntu10.33 and 237-3ubuntu10.39 introduced the problem and it is
still present.
On the NAT router I use three interfaces and have separate routing
tables for admin and forwarded traffic. Things come up fine initially
but every 30-60 minutes (DHCP lease renewal time?) one or more
interfaces is reconfigured and most of the time systemd-networkd will
crash and need to be restarted. Eventually the system becomes
unreachable when the default crash loop backoff logic prevents the
network service from being restarted at all. The log excerpt attached
illustrates the crash loop.
Also including the netplan and networkd config files below.
# grep . /etc/netplan/*
/etc/netplan/50-cloud-init.yaml:# This file is generated from information
provided by the datasource. Changes
/etc/netplan/50-cloud-init.yaml:# to it will not persist across an instance
reboot. To disable cloud-init's
/etc/netplan/50-cloud-init.yaml:# network configuration capabilities, write a
file
/etc/netplan/50-cloud-init.yaml:#
/etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
/etc/netplan/50-cloud-init.yaml:# network: {config: disabled}
/etc/netplan/50-cloud-init.yaml:network:
/etc/netplan/50-cloud-init.yaml:version: 2
/etc/netplan/50-cloud-init.yaml:ethernets:
/etc/netplan/50-cloud-init.yaml:ens5:
/etc/netplan/50-cloud-init.yaml:dhcp4: true
/etc/netplan/50-cloud-init.yaml:match:
/etc/netplan/50-cloud-init.yaml:macaddress: xx:xx:xx:xx:xx:xx
/etc/netplan/50-cloud-init.yaml:set-name: ens5
/etc/netplan/99_config.yaml:network:
/etc/netplan/99_config.yaml: version: 2
[Touch-packages] [Bug 1881972] Re: systemd-networkd crashes with invalid pointer
@r-2ohn-d can you test with systemd from this ppa:
https://launchpad.net/~ddstreet/+archive/ubuntu/systemd
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1881972
Title:
systemd-networkd crashes with invalid pointer
Status in systemd package in Ubuntu:
Fix Released
Status in systemd source package in Bionic:
In Progress
Bug description:
[impact]
systemd-networkd double-free causes crash under some circumstances,
such as adding/removing ip rules
[test case]
see original description
[regression potential]
this strdup's strings during addition of routing policy rules, so any
regression would likely occur when adding/modifying/removing ip rules,
possibly including networkd segfault or failure to add/remove/modify
ip rules.
[scope]
this is needed for bionic.
this is fixed by upstream commit
eeab051b28ba6e1b4a56d369d4c6bf7cfa71947c which is included starting in
v240, so this is already included in Focal and later.
I did not research what original commit introduced the problem, but
the reporter indicates this did not happen for Xenial so it's unlikely
this is a problem in Xenial or earlier.
[original description]
This is a serious regression with systemd-networkd that I ran in to
while setting up a NAT router in AWS. The AWS AMI ubuntu/images/hvm-
ssd/ubuntu-bionic-18.04-amd64-server-20200131 with
systemd-237-3ubuntu10.33 does NOT have the problem, but the next most
recent AWS AMI ubuntu/images/hvm-ssd/ubuntu-
bionic-18.04-amd64-server-20200311 with systemd-including
237-3ubuntu10.39 does.
Also, a system booted from the (good) 20200131 AMI starts showing the
problem after updating only systemd (to 237-3ubuntu10.41) and its
direct dependencies (e.g. 'apt-get install systemd'). So I'm fairly
confident that a change to the systemd package between
237-3ubuntu10.33 and 237-3ubuntu10.39 introduced the problem and it is
still present.
On the NAT router I use three interfaces and have separate routing
tables for admin and forwarded traffic. Things come up fine initially
but every 30-60 minutes (DHCP lease renewal time?) one or more
interfaces is reconfigured and most of the time systemd-networkd will
crash and need to be restarted. Eventually the system becomes
unreachable when the default crash loop backoff logic prevents the
network service from being restarted at all. The log excerpt attached
illustrates the crash loop.
Also including the netplan and networkd config files below.
# grep . /etc/netplan/*
/etc/netplan/50-cloud-init.yaml:# This file is generated from information
provided by the datasource. Changes
/etc/netplan/50-cloud-init.yaml:# to it will not persist across an instance
reboot. To disable cloud-init's
/etc/netplan/50-cloud-init.yaml:# network configuration capabilities, write a
file
/etc/netplan/50-cloud-init.yaml:#
/etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
/etc/netplan/50-cloud-init.yaml:# network: {config: disabled}
/etc/netplan/50-cloud-init.yaml:network:
/etc/netplan/50-cloud-init.yaml:version: 2
/etc/netplan/50-cloud-init.yaml:ethernets:
/etc/netplan/50-cloud-init.yaml:ens5:
/etc/netplan/50-cloud-init.yaml:dhcp4: true
/etc/netplan/50-cloud-init.yaml:match:
/etc/netplan/50-cloud-init.yaml:macaddress: xx:xx:xx:xx:xx:xx
/etc/netplan/50-cloud-init.yaml:set-name: ens5
/etc/netplan/99_config.yaml:network:
/etc/netplan/99_config.yaml: version: 2
/etc/netplan/99_config.yaml: renderer: networkd
/etc/netplan/99_config.yaml: ethernets:
/etc/netplan/99_config.yaml:ens6:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: yy:yy:yy:yy:yy:yy
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-routes: false
/etc/netplan/99_config.yaml:ens7:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: zz:zz:zz:zz:zz:zz
/etc/netplan/99_config.yaml: mtu: 1500
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-mtu: false
/etc/netplan/99_config.yaml:use-routes: false
# grep . /etc/networkd-dispatcher/*/*
/etc/networkd-dispatcher/configured.d/nat:#!/bin/bash
/etc/networkd-dispatcher/configured.d/nat:# Do additional configuration for
the inside and outside interfaces
/etc/networkd-dispatcher/configured.d/nat:# route table used for
forwarded/routed/natted traffic
/etc/networkd-dispatcher/configured.d/nat:FWD_TABLE=99
/etc/networkd-dispatcher/configured.d/nat:if [ "${IFACE}" = "ens6" ]; then
/etc/networkd-dispatcher/configured.d/nat: # delete link-local
[Touch-packages] [Bug 1881972] Re: systemd-networkd crashes with invalid pointer
** Description changed:
+ [impact]
+
+ systemd-networkd double-free causes crash under some circumstances, such
+ as adding/removing ip rules
+
+ [test case]
+
+ see original description
+
+ [regression potential]
+
+ this strdup's strings during addition of routing policy rules, so any
+ regression would likely occur when adding/modifying/removing ip rules,
+ possibly including networkd segfault or failure to add/remove/modify ip
+ rules.
+
+ [scope]
+
+ this is needed for bionic.
+
+ this is fixed by upstream commit
+ eeab051b28ba6e1b4a56d369d4c6bf7cfa71947c which is included starting in
+ v240, so this is already included in Focal and later.
+
+ I did not research what original commit introduced the problem, but the
+ reporter indicates this did not happen for Xenial so it's unlikely this
+ is a problem in Xenial or earlier.
+
+ [original description]
+
This is a serious regression with systemd-networkd that I ran in to
while setting up a NAT router in AWS. The AWS AMI ubuntu/images/hvm-ssd
/ubuntu-bionic-18.04-amd64-server-20200131 with systemd-237-3ubuntu10.33
does NOT have the problem, but the next most recent AWS AMI
ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-20200311 with
systemd-including 237-3ubuntu10.39 does.
Also, a system booted from the (good) 20200131 AMI starts showing the
problem after updating only systemd (to 237-3ubuntu10.41) and its direct
dependencies (e.g. 'apt-get install systemd'). So I'm fairly confident
that a change to the systemd package between 237-3ubuntu10.33 and
237-3ubuntu10.39 introduced the problem and it is still present.
On the NAT router I use three interfaces and have separate routing
tables for admin and forwarded traffic. Things come up fine initially
but every 30-60 minutes (DHCP lease renewal time?) one or more
interfaces is reconfigured and most of the time systemd-networkd will
crash and need to be restarted. Eventually the system becomes
unreachable when the default crash loop backoff logic prevents the
network service from being restarted at all. The log excerpt attached
illustrates the crash loop.
Also including the netplan and networkd config files below.
# grep . /etc/netplan/*
/etc/netplan/50-cloud-init.yaml:# This file is generated from information
provided by the datasource. Changes
/etc/netplan/50-cloud-init.yaml:# to it will not persist across an instance
reboot. To disable cloud-init's
/etc/netplan/50-cloud-init.yaml:# network configuration capabilities, write a
file
/etc/netplan/50-cloud-init.yaml:#
/etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
/etc/netplan/50-cloud-init.yaml:# network: {config: disabled}
/etc/netplan/50-cloud-init.yaml:network:
/etc/netplan/50-cloud-init.yaml:version: 2
/etc/netplan/50-cloud-init.yaml:ethernets:
/etc/netplan/50-cloud-init.yaml:ens5:
/etc/netplan/50-cloud-init.yaml:dhcp4: true
/etc/netplan/50-cloud-init.yaml:match:
/etc/netplan/50-cloud-init.yaml:macaddress: xx:xx:xx:xx:xx:xx
/etc/netplan/50-cloud-init.yaml:set-name: ens5
/etc/netplan/99_config.yaml:network:
/etc/netplan/99_config.yaml: version: 2
/etc/netplan/99_config.yaml: renderer: networkd
/etc/netplan/99_config.yaml: ethernets:
/etc/netplan/99_config.yaml:ens6:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: yy:yy:yy:yy:yy:yy
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-routes: false
/etc/netplan/99_config.yaml:ens7:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: zz:zz:zz:zz:zz:zz
/etc/netplan/99_config.yaml: mtu: 1500
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-mtu: false
/etc/netplan/99_config.yaml:use-routes: false
# grep . /etc/networkd-dispatcher/*/*
/etc/networkd-dispatcher/configured.d/nat:#!/bin/bash
/etc/networkd-dispatcher/configured.d/nat:# Do additional configuration for
the inside and outside interfaces
/etc/networkd-dispatcher/configured.d/nat:# route table used for
forwarded/routed/natted traffic
/etc/networkd-dispatcher/configured.d/nat:FWD_TABLE=99
/etc/networkd-dispatcher/configured.d/nat:if [ "${IFACE}" = "ens6" ]; then
/etc/networkd-dispatcher/configured.d/nat: # delete link-local route for
inside in default table
/etc/networkd-dispatcher/configured.d/nat: /sbin/ip route delete 10.0.3.0/24
2>/dev/null || true
/etc/networkd-dispatcher/configured.d/nat: # add link-local route for inside
in table 99
/etc/networkd-dispatcher/configured.d/nat: /sbin/ip route replace
10.0.3.0/24 dev ens6 scope link src 10.0.3.171 table ${FWD_TABLE}
/etc/networkd-dispatcher/configured.d/nat: # add routes
[Touch-packages] [Bug 1881972] Re: systemd-networkd crashes with invalid pointer
I think this is fixed by upstream commit
eeab051b28ba6e1b4a56d369d4c6bf7cfa71947c
** Also affects: systemd (Ubuntu Bionic)
Importance: Undecided
Status: New
** Changed in: systemd (Ubuntu)
Status: New => Fix Released
** Changed in: systemd (Ubuntu Bionic)
Assignee: (unassigned) => Dan Streetman (ddstreet)
** Changed in: systemd (Ubuntu Bionic)
Importance: Undecided => Medium
** Changed in: systemd (Ubuntu Bionic)
Status: New => In Progress
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1881972
Title:
systemd-networkd crashes with invalid pointer
Status in systemd package in Ubuntu:
Fix Released
Status in systemd source package in Bionic:
In Progress
Bug description:
This is a serious regression with systemd-networkd that I ran in to
while setting up a NAT router in AWS. The AWS AMI ubuntu/images/hvm-
ssd/ubuntu-bionic-18.04-amd64-server-20200131 with
systemd-237-3ubuntu10.33 does NOT have the problem, but the next most
recent AWS AMI ubuntu/images/hvm-ssd/ubuntu-
bionic-18.04-amd64-server-20200311 with systemd-including
237-3ubuntu10.39 does.
Also, a system booted from the (good) 20200131 AMI starts showing the
problem after updating only systemd (to 237-3ubuntu10.41) and its
direct dependencies (e.g. 'apt-get install systemd'). So I'm fairly
confident that a change to the systemd package between
237-3ubuntu10.33 and 237-3ubuntu10.39 introduced the problem and it is
still present.
On the NAT router I use three interfaces and have separate routing
tables for admin and forwarded traffic. Things come up fine initially
but every 30-60 minutes (DHCP lease renewal time?) one or more
interfaces is reconfigured and most of the time systemd-networkd will
crash and need to be restarted. Eventually the system becomes
unreachable when the default crash loop backoff logic prevents the
network service from being restarted at all. The log excerpt attached
illustrates the crash loop.
Also including the netplan and networkd config files below.
# grep . /etc/netplan/*
/etc/netplan/50-cloud-init.yaml:# This file is generated from information
provided by the datasource. Changes
/etc/netplan/50-cloud-init.yaml:# to it will not persist across an instance
reboot. To disable cloud-init's
/etc/netplan/50-cloud-init.yaml:# network configuration capabilities, write a
file
/etc/netplan/50-cloud-init.yaml:#
/etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
/etc/netplan/50-cloud-init.yaml:# network: {config: disabled}
/etc/netplan/50-cloud-init.yaml:network:
/etc/netplan/50-cloud-init.yaml:version: 2
/etc/netplan/50-cloud-init.yaml:ethernets:
/etc/netplan/50-cloud-init.yaml:ens5:
/etc/netplan/50-cloud-init.yaml:dhcp4: true
/etc/netplan/50-cloud-init.yaml:match:
/etc/netplan/50-cloud-init.yaml:macaddress: xx:xx:xx:xx:xx:xx
/etc/netplan/50-cloud-init.yaml:set-name: ens5
/etc/netplan/99_config.yaml:network:
/etc/netplan/99_config.yaml: version: 2
/etc/netplan/99_config.yaml: renderer: networkd
/etc/netplan/99_config.yaml: ethernets:
/etc/netplan/99_config.yaml:ens6:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: yy:yy:yy:yy:yy:yy
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-routes: false
/etc/netplan/99_config.yaml:ens7:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: zz:zz:zz:zz:zz:zz
/etc/netplan/99_config.yaml: mtu: 1500
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-mtu: false
/etc/netplan/99_config.yaml:use-routes: false
# grep . /etc/networkd-dispatcher/*/*
/etc/networkd-dispatcher/configured.d/nat:#!/bin/bash
/etc/networkd-dispatcher/configured.d/nat:# Do additional configuration for
the inside and outside interfaces
/etc/networkd-dispatcher/configured.d/nat:# route table used for
forwarded/routed/natted traffic
/etc/networkd-dispatcher/configured.d/nat:FWD_TABLE=99
/etc/networkd-dispatcher/configured.d/nat:if [ "${IFACE}" = "ens6" ]; then
/etc/networkd-dispatcher/configured.d/nat: # delete link-local route for
inside in default table
/etc/networkd-dispatcher/configured.d/nat: /sbin/ip route delete 10.0.3.0/24
2>/dev/null || true
/etc/networkd-dispatcher/configured.d/nat: # add link-local route for inside
in table 99
/etc/networkd-dispatcher/configured.d/nat: /sbin/ip route replace
10.0.3.0/24 dev ens6 scope link src 10.0.3.171 table ${FWD_TABLE}
/etc/networkd-dispatcher/configured.d/nat: # add routes to VPC cidrs via
inside gateway in
[Touch-packages] [Bug 1881972] Re: systemd-networkd crashes with invalid pointer
I've gathered several core dumps now but the stacktraces are identical.
The logged reason for dumping core is "free(): invalid pointer".
I wonder if there is a race condition with whatever networkd itself is
doing when it reconfigures the interface (which it seems to do more
aggressively for DHCP renewals than it used to) and what my scripts in
/etc/networkd-dispatcher/configured.d and /etc/networkd-
dispatcher/configuring.d are doing. Assuming the
"routing_policy_rule_free" function is equivalent to "ip rule delete
..." there could be a conflict with my "configuring.d" script in
particular. The "configured.d" script adds some extra routing policy
rules and the "configuring.d" script deletes them so they aren't
duplicated every time the network is configured.
** Changed in: systemd (Ubuntu)
Status: Incomplete => New
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1881972
Title:
systemd-networkd crashes with invalid pointer
Status in systemd package in Ubuntu:
New
Bug description:
This is a serious regression with systemd-networkd that I ran in to
while setting up a NAT router in AWS. The AWS AMI ubuntu/images/hvm-
ssd/ubuntu-bionic-18.04-amd64-server-20200131 with
systemd-237-3ubuntu10.33 does NOT have the problem, but the next most
recent AWS AMI ubuntu/images/hvm-ssd/ubuntu-
bionic-18.04-amd64-server-20200311 with systemd-including
237-3ubuntu10.39 does.
Also, a system booted from the (good) 20200131 AMI starts showing the
problem after updating only systemd (to 237-3ubuntu10.41) and its
direct dependencies (e.g. 'apt-get install systemd'). So I'm fairly
confident that a change to the systemd package between
237-3ubuntu10.33 and 237-3ubuntu10.39 introduced the problem and it is
still present.
On the NAT router I use three interfaces and have separate routing
tables for admin and forwarded traffic. Things come up fine initially
but every 30-60 minutes (DHCP lease renewal time?) one or more
interfaces is reconfigured and most of the time systemd-networkd will
crash and need to be restarted. Eventually the system becomes
unreachable when the default crash loop backoff logic prevents the
network service from being restarted at all. The log excerpt attached
illustrates the crash loop.
Also including the netplan and networkd config files below.
# grep . /etc/netplan/*
/etc/netplan/50-cloud-init.yaml:# This file is generated from information
provided by the datasource. Changes
/etc/netplan/50-cloud-init.yaml:# to it will not persist across an instance
reboot. To disable cloud-init's
/etc/netplan/50-cloud-init.yaml:# network configuration capabilities, write a
file
/etc/netplan/50-cloud-init.yaml:#
/etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
/etc/netplan/50-cloud-init.yaml:# network: {config: disabled}
/etc/netplan/50-cloud-init.yaml:network:
/etc/netplan/50-cloud-init.yaml:version: 2
/etc/netplan/50-cloud-init.yaml:ethernets:
/etc/netplan/50-cloud-init.yaml:ens5:
/etc/netplan/50-cloud-init.yaml:dhcp4: true
/etc/netplan/50-cloud-init.yaml:match:
/etc/netplan/50-cloud-init.yaml:macaddress: xx:xx:xx:xx:xx:xx
/etc/netplan/50-cloud-init.yaml:set-name: ens5
/etc/netplan/99_config.yaml:network:
/etc/netplan/99_config.yaml: version: 2
/etc/netplan/99_config.yaml: renderer: networkd
/etc/netplan/99_config.yaml: ethernets:
/etc/netplan/99_config.yaml:ens6:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: yy:yy:yy:yy:yy:yy
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-routes: false
/etc/netplan/99_config.yaml:ens7:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: zz:zz:zz:zz:zz:zz
/etc/netplan/99_config.yaml: mtu: 1500
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-mtu: false
/etc/netplan/99_config.yaml:use-routes: false
# grep . /etc/networkd-dispatcher/*/*
/etc/networkd-dispatcher/configured.d/nat:#!/bin/bash
/etc/networkd-dispatcher/configured.d/nat:# Do additional configuration for
the inside and outside interfaces
/etc/networkd-dispatcher/configured.d/nat:# route table used for
forwarded/routed/natted traffic
/etc/networkd-dispatcher/configured.d/nat:FWD_TABLE=99
/etc/networkd-dispatcher/configured.d/nat:if [ "${IFACE}" = "ens6" ]; then
/etc/networkd-dispatcher/configured.d/nat: # delete link-local route for
inside in default table
/etc/networkd-dispatcher/configured.d/nat: /sbin/ip route delete 10.0.3.0/24
2>/dev/null || true
[Touch-packages] [Bug 1881972] Re: systemd-networkd crashes with invalid pointer
# coredumpctl gdb 28819
PID: 28819 (systemd-network)
UID: 100 (systemd-network)
GID: 102 (systemd-network)
Signal: 6 (ABRT)
Timestamp: Tue 2020-06-16 19:36:22 UTC (16min ago)
Command Line: /lib/systemd/systemd-networkd
Executable: /lib/systemd/systemd-networkd
Control Group: /system.slice/systemd-networkd.service
Unit: systemd-networkd.service
Slice: system.slice
Boot ID: 578e8b2c2e1a43afbd27211be1a4f531
Machine ID: ec267b3475883f9edb99f554607bb456
Hostname: ip-10-0-4-251
Storage:
/var/lib/systemd/coredump/core.systemd-network.100.578e8b2c2e1a43afbd27211be1a4f531.28819.159233618200.lz4
Message: Process 28819 (systemd-network) of user 100 dumped core.
Stack trace of thread 28819:
#0 0x7f740d023e97 raise (libc.so.6)
#1 0x7f740d025801 abort (libc.so.6)
#2 0x7f740d06e897 n/a (libc.so.6)
#3 0x7f740d07590a n/a (libc.so.6)
#4 0x7f740d07ce1c cfree (libc.so.6)
#5 0x55fa5c16276b routing_policy_rule_free
(systemd-networkd)
#6 0x55fa5c1f69e2 manager_rtnl_process_rule
(systemd-networkd)
#7 0x55fa5c1731d6 process_match (systemd-networkd)
#8 0x55fa5c173413 io_callback (systemd-networkd)
#9 0x55fa5c178350 source_dispatch (systemd-networkd)
#10 0x55fa5c1785ea sd_event_dispatch (systemd-networkd)
#11 0x55fa5c178779 sd_event_run (systemd-networkd)
#12 0x55fa5c1789bb sd_event_loop (systemd-networkd)
#13 0x55fa5c1413a6 main (systemd-networkd)
#14 0x7f740d006b97 __libc_start_main (libc.so.6)
#15 0x55fa5c141a8a _start (systemd-networkd)
** Attachment added: "core dump file"
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1881972/+attachment/5384495/+files/core.systemd-network.100.578e8b2c2e1a43afbd27211be1a4f531.28819.159233618200.lz4
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1881972
Title:
systemd-networkd crashes with invalid pointer
Status in systemd package in Ubuntu:
Incomplete
Bug description:
This is a serious regression with systemd-networkd that I ran in to
while setting up a NAT router in AWS. The AWS AMI ubuntu/images/hvm-
ssd/ubuntu-bionic-18.04-amd64-server-20200131 with
systemd-237-3ubuntu10.33 does NOT have the problem, but the next most
recent AWS AMI ubuntu/images/hvm-ssd/ubuntu-
bionic-18.04-amd64-server-20200311 with systemd-including
237-3ubuntu10.39 does.
Also, a system booted from the (good) 20200131 AMI starts showing the
problem after updating only systemd (to 237-3ubuntu10.41) and its
direct dependencies (e.g. 'apt-get install systemd'). So I'm fairly
confident that a change to the systemd package between
237-3ubuntu10.33 and 237-3ubuntu10.39 introduced the problem and it is
still present.
On the NAT router I use three interfaces and have separate routing
tables for admin and forwarded traffic. Things come up fine initially
but every 30-60 minutes (DHCP lease renewal time?) one or more
interfaces is reconfigured and most of the time systemd-networkd will
crash and need to be restarted. Eventually the system becomes
unreachable when the default crash loop backoff logic prevents the
network service from being restarted at all. The log excerpt attached
illustrates the crash loop.
Also including the netplan and networkd config files below.
# grep . /etc/netplan/*
/etc/netplan/50-cloud-init.yaml:# This file is generated from information
provided by the datasource. Changes
/etc/netplan/50-cloud-init.yaml:# to it will not persist across an instance
reboot. To disable cloud-init's
/etc/netplan/50-cloud-init.yaml:# network configuration capabilities, write a
file
/etc/netplan/50-cloud-init.yaml:#
/etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
/etc/netplan/50-cloud-init.yaml:# network: {config: disabled}
/etc/netplan/50-cloud-init.yaml:network:
/etc/netplan/50-cloud-init.yaml:version: 2
/etc/netplan/50-cloud-init.yaml:ethernets:
/etc/netplan/50-cloud-init.yaml:ens5:
/etc/netplan/50-cloud-init.yaml:dhcp4: true
/etc/netplan/50-cloud-init.yaml:match:
/etc/netplan/50-cloud-init.yaml:macaddress: xx:xx:xx:xx:xx:xx
/etc/netplan/50-cloud-init.yaml:set-name: ens5
/etc/netplan/99_config.yaml:network:
/etc/netplan/99_config.yaml: version: 2
/etc/netplan/99_config.yaml: renderer: networkd
/etc/netplan/99_config.yaml: ethernets:
/etc/netplan/99_config.yaml:ens6:
/etc/netplan/99_config.yaml: match:
[Touch-packages] [Bug 1881972] Re: systemd-networkd crashes with invalid pointer
can you attach one of the coredumps? That will significantly help with
finding where exactly the problem is.
** Changed in: systemd (Ubuntu)
Status: New => Incomplete
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1881972
Title:
systemd-networkd crashes with invalid pointer
Status in systemd package in Ubuntu:
Incomplete
Bug description:
This is a serious regression with systemd-networkd that I ran in to
while setting up a NAT router in AWS. The AWS AMI ubuntu/images/hvm-
ssd/ubuntu-bionic-18.04-amd64-server-20200131 with
systemd-237-3ubuntu10.33 does NOT have the problem, but the next most
recent AWS AMI ubuntu/images/hvm-ssd/ubuntu-
bionic-18.04-amd64-server-20200311 with systemd-including
237-3ubuntu10.39 does.
Also, a system booted from the (good) 20200131 AMI starts showing the
problem after updating only systemd (to 237-3ubuntu10.41) and its
direct dependencies (e.g. 'apt-get install systemd'). So I'm fairly
confident that a change to the systemd package between
237-3ubuntu10.33 and 237-3ubuntu10.39 introduced the problem and it is
still present.
On the NAT router I use three interfaces and have separate routing
tables for admin and forwarded traffic. Things come up fine initially
but every 30-60 minutes (DHCP lease renewal time?) one or more
interfaces is reconfigured and most of the time systemd-networkd will
crash and need to be restarted. Eventually the system becomes
unreachable when the default crash loop backoff logic prevents the
network service from being restarted at all. The log excerpt attached
illustrates the crash loop.
Also including the netplan and networkd config files below.
# grep . /etc/netplan/*
/etc/netplan/50-cloud-init.yaml:# This file is generated from information
provided by the datasource. Changes
/etc/netplan/50-cloud-init.yaml:# to it will not persist across an instance
reboot. To disable cloud-init's
/etc/netplan/50-cloud-init.yaml:# network configuration capabilities, write a
file
/etc/netplan/50-cloud-init.yaml:#
/etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
/etc/netplan/50-cloud-init.yaml:# network: {config: disabled}
/etc/netplan/50-cloud-init.yaml:network:
/etc/netplan/50-cloud-init.yaml:version: 2
/etc/netplan/50-cloud-init.yaml:ethernets:
/etc/netplan/50-cloud-init.yaml:ens5:
/etc/netplan/50-cloud-init.yaml:dhcp4: true
/etc/netplan/50-cloud-init.yaml:match:
/etc/netplan/50-cloud-init.yaml:macaddress: xx:xx:xx:xx:xx:xx
/etc/netplan/50-cloud-init.yaml:set-name: ens5
/etc/netplan/99_config.yaml:network:
/etc/netplan/99_config.yaml: version: 2
/etc/netplan/99_config.yaml: renderer: networkd
/etc/netplan/99_config.yaml: ethernets:
/etc/netplan/99_config.yaml:ens6:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: yy:yy:yy:yy:yy:yy
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-routes: false
/etc/netplan/99_config.yaml:ens7:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: zz:zz:zz:zz:zz:zz
/etc/netplan/99_config.yaml: mtu: 1500
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-mtu: false
/etc/netplan/99_config.yaml:use-routes: false
# grep . /etc/networkd-dispatcher/*/*
/etc/networkd-dispatcher/configured.d/nat:#!/bin/bash
/etc/networkd-dispatcher/configured.d/nat:# Do additional configuration for
the inside and outside interfaces
/etc/networkd-dispatcher/configured.d/nat:# route table used for
forwarded/routed/natted traffic
/etc/networkd-dispatcher/configured.d/nat:FWD_TABLE=99
/etc/networkd-dispatcher/configured.d/nat:if [ "${IFACE}" = "ens6" ]; then
/etc/networkd-dispatcher/configured.d/nat: # delete link-local route for
inside in default table
/etc/networkd-dispatcher/configured.d/nat: /sbin/ip route delete 10.0.3.0/24
2>/dev/null || true
/etc/networkd-dispatcher/configured.d/nat: # add link-local route for inside
in table 99
/etc/networkd-dispatcher/configured.d/nat: /sbin/ip route replace
10.0.3.0/24 dev ens6 scope link src 10.0.3.171 table ${FWD_TABLE}
/etc/networkd-dispatcher/configured.d/nat: # add routes to VPC cidrs via
inside gateway in table 99
/etc/networkd-dispatcher/configured.d/nat: /sbin/ip route replace
10.0.0.0/16 via 10.0.3.1 table ${FWD_TABLE}
/etc/networkd-dispatcher/configured.d/nat: # add rules to use table 99
/etc/networkd-dispatcher/configured.d/nat: /sbin/ip rule add iif ens6
lookup ${FWD_TABLE}
/etc/networkd-dispatcher/configured.d/nat: /sbin/ip rule add oif ens6
[Touch-packages] [Bug 1881972] Re: systemd-networkd crashes with invalid pointer
The system I pulled the networkd log from had not yet become unreachable
but here's a snippet from one that did:
May 27 07:38:18 ip-10-0-4-228 systemd[1]: systemd-networkd.service: Failed with
result 'core-dump'.
May 27 07:38:18 ip-10-0-4-228 systemd[1]: Failed to start Network Service.
May 27 07:38:18 ip-10-0-4-228 systemd[1]: Dependency failed for Wait for
Network to be Configured.
May 27 07:38:18 ip-10-0-4-228 systemd[1]: systemd-networkd-wait-online.service:
Job systemd-networkd-wait-online.service/start failed with result 'dependency'
May 27 07:38:18 ip-10-0-4-228 systemd[1]: systemd-networkd.socket: Failed with
result 'service-start-limit-hit'.
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1881972
Title:
systemd-networkd crashes with invalid pointer
Status in systemd package in Ubuntu:
New
Bug description:
This is a serious regression with systemd-networkd that I ran in to
while setting up a NAT router in AWS. The AWS AMI ubuntu/images/hvm-
ssd/ubuntu-bionic-18.04-amd64-server-20200131 with
systemd-237-3ubuntu10.33 does NOT have the problem, but the next most
recent AWS AMI ubuntu/images/hvm-ssd/ubuntu-
bionic-18.04-amd64-server-20200311 with systemd-including
237-3ubuntu10.39 does.
Also, a system booted from the (good) 20200131 AMI starts showing the
problem after updating only systemd (to 237-3ubuntu10.41) and its
direct dependencies (e.g. 'apt-get install systemd'). So I'm fairly
confident that a change to the systemd package between
237-3ubuntu10.33 and 237-3ubuntu10.39 introduced the problem and it is
still present.
On the NAT router I use three interfaces and have separate routing
tables for admin and forwarded traffic. Things come up fine initially
but every 30-60 minutes (DHCP lease renewal time?) one or more
interfaces is reconfigured and most of the time systemd-networkd will
crash and need to be restarted. Eventually the system becomes
unreachable when the default crash loop backoff logic prevents the
network service from being restarted at all. The log excerpt attached
illustrates the crash loop.
Also including the netplan and networkd config files below.
# grep . /etc/netplan/*
/etc/netplan/50-cloud-init.yaml:# This file is generated from information
provided by the datasource. Changes
/etc/netplan/50-cloud-init.yaml:# to it will not persist across an instance
reboot. To disable cloud-init's
/etc/netplan/50-cloud-init.yaml:# network configuration capabilities, write a
file
/etc/netplan/50-cloud-init.yaml:#
/etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
/etc/netplan/50-cloud-init.yaml:# network: {config: disabled}
/etc/netplan/50-cloud-init.yaml:network:
/etc/netplan/50-cloud-init.yaml:version: 2
/etc/netplan/50-cloud-init.yaml:ethernets:
/etc/netplan/50-cloud-init.yaml:ens5:
/etc/netplan/50-cloud-init.yaml:dhcp4: true
/etc/netplan/50-cloud-init.yaml:match:
/etc/netplan/50-cloud-init.yaml:macaddress: xx:xx:xx:xx:xx:xx
/etc/netplan/50-cloud-init.yaml:set-name: ens5
/etc/netplan/99_config.yaml:network:
/etc/netplan/99_config.yaml: version: 2
/etc/netplan/99_config.yaml: renderer: networkd
/etc/netplan/99_config.yaml: ethernets:
/etc/netplan/99_config.yaml:ens6:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: yy:yy:yy:yy:yy:yy
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-routes: false
/etc/netplan/99_config.yaml:ens7:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: zz:zz:zz:zz:zz:zz
/etc/netplan/99_config.yaml: mtu: 1500
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-mtu: false
/etc/netplan/99_config.yaml:use-routes: false
# grep . /etc/networkd-dispatcher/*/*
/etc/networkd-dispatcher/configured.d/nat:#!/bin/bash
/etc/networkd-dispatcher/configured.d/nat:# Do additional configuration for
the inside and outside interfaces
/etc/networkd-dispatcher/configured.d/nat:# route table used for
forwarded/routed/natted traffic
/etc/networkd-dispatcher/configured.d/nat:FWD_TABLE=99
/etc/networkd-dispatcher/configured.d/nat:if [ "${IFACE}" = "ens6" ]; then
/etc/networkd-dispatcher/configured.d/nat: # delete link-local route for
inside in default table
/etc/networkd-dispatcher/configured.d/nat: /sbin/ip route delete 10.0.3.0/24
2>/dev/null || true
/etc/networkd-dispatcher/configured.d/nat: # add link-local route for inside
in table 99
/etc/networkd-dispatcher/configured.d/nat: /sbin/ip route replace
10.0.3.0/24 dev ens6 scope link src 10.0.3.171
[Touch-packages] [Bug 1881972] Re: systemd-networkd crashes with invalid pointer
Also of note is that on systems with systemd-237-3ubuntu10.33 or older I
don't see the "ens5: Configured" log messages at all after initial
configuration, even though I'm sure DHCP renewals are happening.
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1881972
Title:
systemd-networkd crashes with invalid pointer
Status in systemd package in Ubuntu:
New
Bug description:
This is a serious regression with systemd-networkd that I ran in to
while setting up a NAT router in AWS. The AWS AMI ubuntu/images/hvm-
ssd/ubuntu-bionic-18.04-amd64-server-20200131 with
systemd-237-3ubuntu10.33 does NOT have the problem, but the next most
recent AWS AMI ubuntu/images/hvm-ssd/ubuntu-
bionic-18.04-amd64-server-20200311 with systemd-including
237-3ubuntu10.39 does.
Also, a system booted from the (good) 20200131 AMI starts showing the
problem after updating only systemd (to 237-3ubuntu10.41) and its
direct dependencies (e.g. 'apt-get install systemd'). So I'm fairly
confident that a change to the systemd package between
237-3ubuntu10.33 and 237-3ubuntu10.39 introduced the problem and it is
still present.
On the NAT router I use three interfaces and have separate routing
tables for admin and forwarded traffic. Things come up fine initially
but every 30-60 minutes (DHCP lease renewal time?) one or more
interfaces is reconfigured and most of the time systemd-networkd will
crash and need to be restarted. Eventually the system becomes
unreachable when the default crash loop backoff logic prevents the
network service from being restarted at all. The log excerpt attached
illustrates the crash loop.
Also including the netplan and networkd config files below.
# grep . /etc/netplan/*
/etc/netplan/50-cloud-init.yaml:# This file is generated from information
provided by the datasource. Changes
/etc/netplan/50-cloud-init.yaml:# to it will not persist across an instance
reboot. To disable cloud-init's
/etc/netplan/50-cloud-init.yaml:# network configuration capabilities, write a
file
/etc/netplan/50-cloud-init.yaml:#
/etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
/etc/netplan/50-cloud-init.yaml:# network: {config: disabled}
/etc/netplan/50-cloud-init.yaml:network:
/etc/netplan/50-cloud-init.yaml:version: 2
/etc/netplan/50-cloud-init.yaml:ethernets:
/etc/netplan/50-cloud-init.yaml:ens5:
/etc/netplan/50-cloud-init.yaml:dhcp4: true
/etc/netplan/50-cloud-init.yaml:match:
/etc/netplan/50-cloud-init.yaml:macaddress: xx:xx:xx:xx:xx:xx
/etc/netplan/50-cloud-init.yaml:set-name: ens5
/etc/netplan/99_config.yaml:network:
/etc/netplan/99_config.yaml: version: 2
/etc/netplan/99_config.yaml: renderer: networkd
/etc/netplan/99_config.yaml: ethernets:
/etc/netplan/99_config.yaml:ens6:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: yy:yy:yy:yy:yy:yy
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-routes: false
/etc/netplan/99_config.yaml:ens7:
/etc/netplan/99_config.yaml: match:
/etc/netplan/99_config.yaml:macaddress: zz:zz:zz:zz:zz:zz
/etc/netplan/99_config.yaml: mtu: 1500
/etc/netplan/99_config.yaml: dhcp4: true
/etc/netplan/99_config.yaml: dhcp4-overrides:
/etc/netplan/99_config.yaml:use-mtu: false
/etc/netplan/99_config.yaml:use-routes: false
# grep . /etc/networkd-dispatcher/*/*
/etc/networkd-dispatcher/configured.d/nat:#!/bin/bash
/etc/networkd-dispatcher/configured.d/nat:# Do additional configuration for
the inside and outside interfaces
/etc/networkd-dispatcher/configured.d/nat:# route table used for
forwarded/routed/natted traffic
/etc/networkd-dispatcher/configured.d/nat:FWD_TABLE=99
/etc/networkd-dispatcher/configured.d/nat:if [ "${IFACE}" = "ens6" ]; then
/etc/networkd-dispatcher/configured.d/nat: # delete link-local route for
inside in default table
/etc/networkd-dispatcher/configured.d/nat: /sbin/ip route delete 10.0.3.0/24
2>/dev/null || true
/etc/networkd-dispatcher/configured.d/nat: # add link-local route for inside
in table 99
/etc/networkd-dispatcher/configured.d/nat: /sbin/ip route replace
10.0.3.0/24 dev ens6 scope link src 10.0.3.171 table ${FWD_TABLE}
/etc/networkd-dispatcher/configured.d/nat: # add routes to VPC cidrs via
inside gateway in table 99
/etc/networkd-dispatcher/configured.d/nat: /sbin/ip route replace
10.0.0.0/16 via 10.0.3.1 table ${FWD_TABLE}
/etc/networkd-dispatcher/configured.d/nat: # add rules to use table 99
/etc/networkd-dispatcher/configured.d/nat: /sbin/ip rule add iif ens6
lookup ${FWD_TABLE}
/etc/networkd-dispatcher/configured.d/nat: /sbin/ip
