[Kernel-packages] [Bug 1793753] Re: kernel panic - null pointer dereference on ipset operations
I've tested the proposed kernel under the previous test environment -
after two hours the host was still up, where as it would have previously
crashed within 30 minutes or so. As such, this appears to prevent the
panic.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1793753
Title:
kernel panic - null pointer dereference on ipset operations
Status in linux package in Ubuntu:
Fix Released
Status in linux source package in Xenial:
Fix Released
Bug description:
== SRU Justification ==
A regression was introduced in Xenial, even prior to v4.4 Final. I did
not test prior to this kernel once I found the bug was fixed in
mainline. The bug reporter experienced crashes on machines running
iptables using ipsets. He could get a trace from the console on one of
them which is attached to the bug report.
On these machines, some ipset commands are automatically run to update the
sets, and/or to dump them (ipset restore, swap, delete ... / ipset save).
I was able to reproduce this bug as was cking. This bug was found to be
fixed by mainline commits 596cf3fe5854 and e5173418ac59.
== Fixes ==
596cf3fe5854 ("netfilter: ipset: fix race condition in ipset save, swap and
delete")
e5173418ac59 ("netfilter: ipset: Fix race between dump and swap")
== Regression Potential ==
Low. This fixes a regression and is limited to netfilter.
== Test Case ==
A test kernel was built with these patches and tested by myself and cking.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1793753/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1793753] Re: kernel panic - null pointer dereference on ipset operations
I've been able to reproduce this panic by running Ubuntu Xenial under qemu, with a script that effectively does ipset restore/swap/destroy in a loop, while also running ipset save in a separate loop. ** Attachment added: "ipset-panic.txt" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1793753/+attachment/5191487/+files/ipset-panic.txt -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1793753 Title: kernel panic - null pointer dereference on ipset operations Status in linux package in Ubuntu: Confirmed Bug description: Hi, We've experienced crashes on machines running iptables using ipsets. We could get a trace from the console on one of them (attached file kernel-trace.txt). On these machines, some ipset commands are automatically run to update the sets, and/or to dump them (ipset restore, swap, delete ... / ipset save). We strongly suspect the panic is happening due to a race when ipset updates happen at the same time as a dump. These machines are running xenial. Before the crash, they were on 4.4.0-116-generic #140-Ubuntu, but then rebooted into 4.4.0-135-generic #161-Ubuntu. I have an ipset save running in loops on one of these machines to try and reproduce quicker. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1793753/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1673303] Re: [Xenial] net: better skb->sender_cpu and skb->napi_id cohabitation
The firewalls deployed with the patched kernel have now been running stably in production for more than 24 hours. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1673303 Title: [Xenial] net: better skb->sender_cpu and skb->napi_id cohabitation Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: Fix Released Bug description: == Xenial SRU == We've twice now tried to roll out new firewalls and twice had to revert back when the new firewalls almost immediately hung after cutover. At first we thought it was hardware issues, but after we reproduced it on 4 different firewalls, we realised it was more likely to be a problem with the Xenial kernel. We think we're running into something similar to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1579943 And Joel thinks the following patch might fix it: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=52bd2d62ce6758d811edcbd2256eb9ea7f6a56cb Unfortunately, even when we mimic live production traffic on the new firewalls with port mirroring, we only have a ~20% success rate at reproducing the kernel hang and I'm keen not to have any more failed migration attempts (and the corresponding downtime for many many services). == Fix == See http://kernel.ubuntu.com/~ogasawara/lp1579943/ == Testing == We've just successfully migrated four firewalls that are running with the patched kernel. Previously two of them would have survived for less than 2 minutes, both have now been running in production for over an hour. I'll provide another update tomorrow, however at this stage I'd suggest that it makes sense to get this into an SRU. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1673303/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
