[Bug 1886668] Re: linux 4.15.0-109-generic network DoS regression vs -108
This bug was fixed in the package linux - 5.4.0-42.46 --- linux (5.4.0-42.46) focal; urgency=medium * focal/linux: 5.4.0-42.46 -proposed tracker (LP: #1887069) * linux 4.15.0-109-generic network DoS regression vs -108 (LP: #1886668) - SAUCE: Revert "netprio_cgroup: Fix unlimited memory leak of v2 cgroups" linux (5.4.0-41.45) focal; urgency=medium * focal/linux: 5.4.0-41.45 -proposed tracker (LP: #1885855) * Packaging resync (LP: #1786013) - update dkms package versions * CVE-2019-19642 - kernel/relay.c: handle alloc_percpu returning NULL in relay_open * CVE-2019-16089 - SAUCE: nbd_genl_status: null check for nla_nest_start * CVE-2020-11935 - aufs: do not call i_readcount_inc() * ip_defrag.sh in net from ubuntu_kernel_selftests failed with 5.0 / 5.3 / 5.4 kernel (LP: #1826848) - selftests: net: ip_defrag: ignore EPERM * Update lockdown patches (LP: #1884159) - SAUCE: acpi: disallow loading configfs acpi tables when locked down * seccomp_bpf fails on powerpc (LP: #1885757) - SAUCE: selftests/seccomp: fix ptrace tests on powerpc * Introduce the new NVIDIA 418-server and 440-server series, and update the current NVIDIA drivers (LP: #1881137) - [packaging] add signed modules for the 418-server and the 440-server flavours -- Khalid Elmously Thu, 09 Jul 2020 19:50:26 -0400 ** Changed in: linux (Ubuntu Groovy) Status: In Progress => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1886668 Title: linux 4.15.0-109-generic network DoS regression vs -108 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886668/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1886668] Re: linux 4.15.0-109-generic network DoS regression vs -108
This bug was fixed in the package linux - 5.3.0-64.58
---
linux (5.3.0-64.58) eoan; urgency=medium
* eoan/linux: 5.3.0-64.58 -proposed tracker (LP: #1887088)
* linux 4.15.0-109-generic network DoS regression vs -108 (LP: #1886668)
- SAUCE: Revert "netprio_cgroup: Fix unlimited memory leak of v2 cgroups"
linux (5.3.0-63.57) eoan; urgency=medium
* eoan/linux: 5.3.0-63.57 -proposed tracker (LP: #1885495)
* seccomp_bpf fails on powerpc (LP: #1885757)
- SAUCE: selftests/seccomp: fix ptrace tests on powerpc
* The thread level parallelism would be a bottleneck when searching for the
shared pmd by using hugetlbfs (LP: #1882039)
- hugetlbfs: take read_lock on i_mmap for PMD sharing
* Eoan update: upstream stable patchset 2020-06-30 (LP: #1885775)
- ipv6: fix IPV6_ADDRFORM operation logic
- net_failover: fixed rollback in net_failover_open()
- bridge: Avoid infinite loop when suppressing NS messages with invalid
options
- vxlan: Avoid infinite loop when suppressing NS messages with invalid
options
- tun: correct header offsets in napi frags mode
- Input: mms114 - fix handling of mms345l
- ARM: 8977/1: ptrace: Fix mask for thumb breakpoint hook
- sched/fair: Don't NUMA balance for kthreads
- Input: synaptics - add a second working PNP_ID for Lenovo T470s
- drivers/net/ibmvnic: Update VNIC protocol version reporting
- powerpc/xive: Clear the page tables for the ESB IO mapping
- ath9k_htc: Silence undersized packet warnings
- RDMA/uverbs: Make the event_queue fds return POLLERR when disassociated
- x86/cpu/amd: Make erratum #1054 a legacy erratum
- perf probe: Accept the instance number of kretprobe event
- mm: add kvfree_sensitive() for freeing sensitive data objects
- aio: fix async fsync creds
- x86_64: Fix jiffies ODR violation
- x86/PCI: Mark Intel C620 MROMs as having non-compliant BARs
- x86/speculation: Prevent rogue cross-process SSBD shutdown
- x86/reboot/quirks: Add MacBook6,1 reboot quirk
- efi/efivars: Add missing kobject_put() in sysfs entry creation error path
- ALSA: es1688: Add the missed snd_card_free()
- ALSA: hda/realtek - add a pintbl quirk for several Lenovo machines
- ALSA: usb-audio: Fix inconsistent card PM state after resume
- ALSA: usb-audio: Add vendor, product and profile name for HP Thunderbolt
Dock
- ACPI: sysfs: Fix reference count leak in acpi_sysfs_add_hotplug_profile()
- ACPI: CPPC: Fix reference count leak in acpi_cppc_processor_probe()
- ACPI: GED: add support for _Exx / _Lxx handler methods
- ACPI: PM: Avoid using power resources if there are none for D0
- nilfs2: fix null pointer dereference at nilfs_segctor_do_construct()
- spi: dw: Fix controller unregister order
- spi: bcm2835aux: Fix controller unregister order
- spi: bcm-qspi: when tx/rx buffer is NULL set to 0
- PM: runtime: clk: Fix clk_pm_runtime_get() error path
- crypto: cavium/nitrox - Fix 'nitrox_get_first_device()' when ndevlist is
fully iterated
- ALSA: pcm: disallow linking stream to itself
- x86/{mce,mm}: Unmap the entire page if the whole page is affected and
poisoned
- KVM: x86: Fix APIC page invalidation race
- KVM: x86/mmu: Consolidate "is MMIO SPTE" code
- KVM: x86: only do L1TF workaround on affected processors
- x86/speculation: Avoid force-disabling IBPB based on STIBP and enhanced
IBRS.
- x86/speculation: PR_SPEC_FORCE_DISABLE enforcement for indirect branches.
- spi: Fix controller unregister order
- spi: pxa2xx: Fix controller unregister order
- spi: bcm2835: Fix controller unregister order
- spi: pxa2xx: Fix runtime PM ref imbalance on probe error
- crypto: virtio: Fix use-after-free in
virtio_crypto_skcipher_finalize_req()
- crypto: virtio: Fix src/dst scatterlist calculation in
__virtio_crypto_skcipher_do_req()
- crypto: virtio: Fix dest length calculation in
__virtio_crypto_skcipher_do_req()
- selftests/net: in rxtimestamp getopt_long needs terminating null entry
- ovl: initialize error in ovl_copy_xattr
- proc: Use new_inode not new_inode_pseudo
- video: fbdev: w100fb: Fix a potential double free.
- KVM: nSVM: fix condition for filtering async PF
- KVM: nSVM: leave ASID aside in copy_vmcb_control_area
- KVM: nVMX: Consult only the "basic" exit reason when routing nested exit
- KVM: MIPS: Define KVM_ENTRYHI_ASID to cpu_asid_mask(&boot_cpu_data)
- KVM: MIPS: Fix VPN2_MASK definition for variable cpu_vmbits
- KVM: arm64: Make vcpu_cp1x() work on Big Endian hosts
- scsi: megaraid_sas: TM command refire leads to controller firmware crash
- ath9k: Fix use-after-free Read in ath9k_wmi_ctrl_rx
- ath9k: Fix use-after-free Write in ath9k_htc_rx_msg
- ath9x: Fix stack-out-of-bounds Write in ath9k_hif_usb_rx_cb
- ath9k: Fix general protection fault in ath9k_hif_usb_
[Bug 1886668] Re: linux 4.15.0-109-generic network DoS regression vs -108
I was also affected in bionic (linux-image-5.3.0-62-generic), but (since today?) there is an update available: linux-image-5.4.0-42-generic (5.4.0-42.46~18.04.1) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1886668 Title: linux 4.15.0-109-generic network DoS regression vs -108 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886668/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1886668] Re: linux 4.15.0-109-generic network DoS regression vs -108
I think this is needed in linux-generic-hwe-18.04. We get similar crashes with 5.3.0.62 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1886668 Title: linux 4.15.0-109-generic network DoS regression vs -108 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886668/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1886668] Re: linux 4.15.0-109-generic network DoS regression vs -108
This bug was fixed in the package linux - 5.4.0-42.46 --- linux (5.4.0-42.46) focal; urgency=medium * focal/linux: 5.4.0-42.46 -proposed tracker (LP: #1887069) * linux 4.15.0-109-generic network DoS regression vs -108 (LP: #1886668) - SAUCE: Revert "netprio_cgroup: Fix unlimited memory leak of v2 cgroups" linux (5.4.0-41.45) focal; urgency=medium * focal/linux: 5.4.0-41.45 -proposed tracker (LP: #1885855) * Packaging resync (LP: #1786013) - update dkms package versions * CVE-2019-19642 - kernel/relay.c: handle alloc_percpu returning NULL in relay_open * CVE-2019-16089 - SAUCE: nbd_genl_status: null check for nla_nest_start * CVE-2020-11935 - aufs: do not call i_readcount_inc() * ip_defrag.sh in net from ubuntu_kernel_selftests failed with 5.0 / 5.3 / 5.4 kernel (LP: #1826848) - selftests: net: ip_defrag: ignore EPERM * Update lockdown patches (LP: #1884159) - SAUCE: acpi: disallow loading configfs acpi tables when locked down * seccomp_bpf fails on powerpc (LP: #1885757) - SAUCE: selftests/seccomp: fix ptrace tests on powerpc * Introduce the new NVIDIA 418-server and 440-server series, and update the current NVIDIA drivers (LP: #1881137) - [packaging] add signed modules for the 418-server and the 440-server flavours -- Khalid Elmously Thu, 09 Jul 2020 19:50:26 -0400 ** Changed in: linux (Ubuntu Focal) Status: Fix Committed => Fix Released ** CVE added: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2019-16089 ** CVE added: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2019-19642 ** CVE added: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2020-11935 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1886668 Title: linux 4.15.0-109-generic network DoS regression vs -108 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886668/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1886668] Re: linux 4.15.0-109-generic network DoS regression vs -108
This bug was fixed in the package linux - 4.15.0-111.112 --- linux (4.15.0-111.112) bionic; urgency=medium * bionic/linux: 4.15.0-111.112 -proposed tracker (LP: #1886999) * Bionic update: upstream stable patchset 2020-05-07 (LP: #1877461) - SAUCE: mlxsw: Add missmerged ERR_PTR hunk * linux 4.15.0-109-generic network DoS regression vs -108 (LP: #1886668) - SAUCE: Revert "netprio_cgroup: Fix unlimited memory leak of v2 cgroups" -- Khalid Elmously Thu, 09 Jul 2020 16:03:14 -0400 ** Changed in: linux (Ubuntu Bionic) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1886668 Title: linux 4.15.0-109-generic network DoS regression vs -108 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886668/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1886668] Re: linux 4.15.0-109-generic network DoS regression vs -108
Concur. Previously in bionic 4.15.0-109 I would get this within 5 or 6 hours *max* (often much less) with 4.15.0-111 I'm at about 24 hours so far and no sign. Looks good. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1886668 Title: linux 4.15.0-109-generic network DoS regression vs -108 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886668/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1886668] Re: linux 4.15.0-109-generic network DoS regression vs -108
I confirm that I can't reproduce the bug with the reproducer from comment #7 with bionic/linux 4.15.0-111. ** Tags removed: verification-needed-bionic ** Tags added: verification-done-bionic -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1886668 Title: linux 4.15.0-109-generic network DoS regression vs -108 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886668/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1886668] Re: linux 4.15.0-109-generic network DoS regression vs -108
This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed- eoan' to 'verification-done-eoan'. If the problem still exists, change the tag 'verification-needed-eoan' to 'verification-failed-eoan'. If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you! ** Tags added: verification-needed-eoan -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1886668 Title: linux 4.15.0-109-generic network DoS regression vs -108 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886668/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1886668] Re: linux 4.15.0-109-generic network DoS regression vs -108
This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed- focal' to 'verification-done-focal'. If the problem still exists, change the tag 'verification-needed-focal' to 'verification-failed-focal'. If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you! ** Tags added: verification-needed-focal -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1886668 Title: linux 4.15.0-109-generic network DoS regression vs -108 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886668/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1886668] Re: linux 4.15.0-109-generic network DoS regression vs -108
This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed- bionic' to 'verification-done-bionic'. If the problem still exists, change the tag 'verification-needed-bionic' to 'verification-failed- bionic'. If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you! ** Tags added: verification-needed-bionic -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1886668 Title: linux 4.15.0-109-generic network DoS regression vs -108 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886668/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1886668] Re: linux 4.15.0-109-generic network DoS regression vs -108
** Changed in: linux (Ubuntu Eoan) Status: In Progress => Fix Committed ** Changed in: linux (Ubuntu Focal) Status: In Progress => Fix Committed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1886668 Title: linux 4.15.0-109-generic network DoS regression vs -108 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886668/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1886668] Re: linux 4.15.0-109-generic network DoS regression vs -108
https://launchpad.net/~cascardo/+archive/ubuntu/ppa/+sourcepub/11419106 /+listing-archive-extra So, this package on my ppa is built for bionic, but should work on other series too. It has a service that will call a wrapper that will start the reproducer and reboot. The reason for the reboot is because once we add a task to net_prio cgroup, it will disable cgroup bpf and we can't call the reproducer again. And the reproducer, though it can cause the refcount to go below 0 every time, it won't always cause the exact crash from this bug. Once you want to disable the reproducer, you should add to the kernel cmdline the parameter "systemd.mask=cgroup-bpf-net-prio-crash.service". Then, you need to remove the package and can get your system back. You may be running some service that will add a task to net_prio or net_cls cgroup, thus preventing the reproducer to run at all (but not stop it from rebooting your system over and over again). lxd comes to mind here. You may check that it's the case (before installing the reproducer) by looking at dmesg and searching for: cgroup: cgroup: disabling cgroup2 socket matching due to net_prio or net_cls activation The following WARN is the demonstration that the refcount underflow has happened (though not the crash): [ 12.581125] [ cut here ] [ 12.585021] percpu ref (cgroup_bpf_release_fn) <= 0 (-357) after switching to atomic [ 12.585092] WARNING: CPU: 2 PID: 665 at lib/percpu-refcount.c:160 percpu_ref_switch_to_atomic_rcu+0x12e/0x140 The crash will cause a panic and likely prevent the system from rebooting, showing you have reproduced the issue. If you never see the WARN, the bug has been mitigated, though it can still happen if we modify the reproducer slightly to also change net_cls.classid. Cascardo. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1886668 Title: linux 4.15.0-109-generic network DoS regression vs -108 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886668/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1886668] Re: linux 4.15.0-109-generic network DoS regression vs -108
** Tags added: patch -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1886668 Title: linux 4.15.0-109-generic network DoS regression vs -108 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886668/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1886668] Re: linux 4.15.0-109-generic network DoS regression vs -108
** Also affects: linux (Ubuntu Groovy) Importance: Undecided Assignee: Thadeu Lima de Souza Cascardo (cascardo) Status: Invalid ** Also affects: linux (Ubuntu Focal) Importance: Undecided Status: New ** Also affects: linux (Ubuntu Eoan) Importance: Undecided Status: New ** Changed in: linux (Ubuntu Groovy) Status: Invalid => In Progress ** Changed in: linux (Ubuntu Focal) Status: New => In Progress ** Changed in: linux (Ubuntu Eoan) Status: New => In Progress -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1886668 Title: linux 4.15.0-109-generic network DoS regression vs -108 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886668/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1886668] Re: linux 4.15.0-109-generic network DoS regression vs -108
** Changed in: linux (Ubuntu Bionic) Status: In Progress => Fix Committed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1886668 Title: linux 4.15.0-109-generic network DoS regression vs -108 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886668/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1886668] Re: linux 4.15.0-109-generic network DoS regression vs -108
Test kernels at https://people.canonical.com/~cascardo/lp1886668/. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1886668 Title: linux 4.15.0-109-generic network DoS regression vs -108 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886668/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1886668] Re: linux 4.15.0-109-generic network DoS regression vs -108
** Description changed: [Impact] On systems using cgroups and sockets extensively, like docker, kubernetes, lxd, libvirt, a crash might happen when using linux 4.15.0-109-generic. [Fix] Revert the patch that disables sk_alloc cgroup refcounting when tasks are added to net_prio cgroup. [Test case] - Test that such environments where the issue is reproduced survive some hours of uptime. See attached test case that reproduces a different but possibly related issue. + Test that such environments where the issue is reproduced survive some hours of uptime. A different bug was reproduced with a work-in-progress code and was not reproduced with the culprit reverted. [Regression potential] The reverted commit fix a memory leak on similar scenarios. But a leak is better than a crash. Two other bugs have been opened to track a real fix for this issue and the leak. - -- - Reported from a user: Several of our infrastructure VMs recently started crashing (oops attached), after they upgraded to -109. -108 appears to be stable. Analysing the crash, it appears to be a wild pointer access in a BPF filter, which makes this (probably) a network-traffic triggered crash. [ 696.396831] general protection fault: [#1] SMP PTI [ 696.396843] Modules linked in: iscsi_target_mod target_core_mod ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack br_netfilter bridge nfsv3 cmac arc4 md4 rpcsec_gss_krb5 nfsv4 nls_utf8 cifs nfs aufs ccm fscache binfmt_misc overlay xfs libcrc32c intel_rapl crct10dif_pclmul crc32_pclmul ghash_clmulni_intel ppdev pcbc aesni_intel aes_x86_64 crypto_simd glue_helper cryptd input_leds joydev intel_rapl_perf serio_raw parport_pc parport mac_hid sch_fq_codel nfsd 8021q auth_rpcgss garp nfs_acl mrp lockd stp llc grace xenfs sunrpc xen_privcmd ip_tables x_tables autofs4 hid_generic usbhid hid psmouse i2c_piix4 pata_acpi floppy [ 696.396966] CPU: 6 PID: 0 Comm: swapper/6 Not tainted 4.15.0-109-generic #110-Ubuntu [ 696.396979] Hardware name: Xen HVM domU, BIOS 4.7.6-1.26 12/03/2018 [ 696.396993] RIP: 0010:__cgroup_bpf_run_filter_skb+0xbb/0x1e0 [ 696.397005] RSP: 0018:893fdcb83a70 EFLAGS: 00010292 [ 696.397015] RAX: 6d69546e6f697469 RBX: RCX: 0014 [ 696.397028] RDX: RSI: 893fd036 RDI: 893fb5154800 [ 696.397041] RBP: 893fdcb83ad0 R08: 0001 R09: [ 696.397058] R10: R11: 0003 R12: 0014 [ 696.397075] R13: 893fb5154800 R14: 0020 R15: 893fc6ba4d00 [ 696.397091] FS: () GS:893fdcb8() knlGS: [ 696.397107] CS: 0010 DS: ES: CR0: 80050033 [ 696.397119] CR2: 00c0001b4000 CR3: 0006dce0a004 CR4: 003606e0 [ 696.397135] DR0: DR1: DR2: [ 696.397152] DR3: DR6: fffe0ff0 DR7: 0400 [ 696.397169] Call Trace: [ 696.397175] [ 696.397183] sk_filter_trim_cap+0xd0/0x1b0 [ 696.397191] tcp_v4_rcv+0x8b7/0xa80 [ 696.397199] ip_local_deliver_finish+0x66/0x210 [ 696.397208] ip_local_deliver+0x7e/0xe0 [ 696.397215] ? ip_rcv_finish+0x430/0x430 [ 696.397223] ip_rcv_finish+0x129/0x430 [ 696.397230] ip_rcv+0x296/0x360 [ 696.397238] ? inet_del_offload+0x40/0x40 [ 696.397249] __netif_receive_skb_core+0x432/0xb80 [ 696.397261] ? skb_send_sock+0x50/0x50 [ 696.397271] ? tcp4_gro_receive+0x137/0x1a0 [ 696.397280] __netif_receive_skb+0x18/0x60 [ 696.397290] ? __netif_receive_skb+0x18/0x60 [ 696.397300] netif_receive_skb_internal+0x45/0xe0 [ 696.397309] napi_gro_receive+0xc5/0xf0 [ 696.397317] xennet_poll+0x9ca/0xbc0 [ 696.397325] net_rx_action+0x140/0x3a0 [ 696.397334] __do_softirq+0xe4/0x2d4 [ 696.397344] irq_exit+0xc5/0xd0 [ 696.397352] xen_evtchn_do_upcall+0x30/0x50 [ 696.397361] xen_hvm_callback_vector+0x90/0xa0 [ 696.397371] [ 696.397378] RIP: 0010:native_safe_halt+0x12/0x20 [ 696.397390] RSP: 0018:94c4862cbe80 EFLAGS: 0246 ORIG_RAX: ff0c [ 696.397405] RAX: 8efc1800 RBX: 0006 RCX: [ 696.397419] RDX: RSI: RDI: [ 696.397435] RBP: 94c4862cbe80 R08: 0002 R09: 0001 [ 696.397449] R10: 0010 R11: 0397 R12: 0006 [ 696.397462] R13: R14: R15: [ 696.397479] ? __sched_text_end+0x1/0x1 [ 696.397489] default_idle+0x20/0x100 [ 696.397499] arch_cpu_idle+0x15/0x20
[Bug 1886668] Re: linux 4.15.0-109-generic network DoS regression vs -108
Bugs are LP#1886860 and LP#1886859. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1886668 Title: linux 4.15.0-109-generic network DoS regression vs -108 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886668/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1886668] Re: linux 4.15.0-109-generic network DoS regression vs -108
** Description changed: + [Impact] + On systems using cgroups and sockets extensively, like docker, kubernetes, lxd, libvirt, a crash might happen when using linux 4.15.0-109-generic. + + [Fix] + Revert the patch that disables sk_alloc cgroup refcounting when tasks are added to net_prio cgroup. + + [Test case] + Test that such environments where the issue is reproduced survive some hours of uptime. See attached test case that reproduces a different but possibly related issue. + + [Regression potential] + The reverted commit fix a memory leak on similar scenarios. But a leak is better than a crash. Two other bugs have been opened to track a real fix for this issue and the leak. + + + -- + + Reported from a user: - Several of our infrastructure VMs recently started crashing (oops - attached), after they upgraded to -109. -108 appears to be stable. - - Analysing the crash, it appears to be a wild pointer access in a BPF - filter, which makes this (probably) a network-traffic triggered crash. + Several of our infrastructure VMs recently started crashing (oops + attached), after they upgraded to -109. -108 appears to be stable. + + Analysing the crash, it appears to be a wild pointer access in a BPF + filter, which makes this (probably) a network-traffic triggered crash. [ 696.396831] general protection fault: [#1] SMP PTI [ 696.396843] Modules linked in: iscsi_target_mod target_core_mod ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack br_netfilter bridge nfsv3 cmac arc4 md4 rpcsec_gss_krb5 nfsv4 nls_utf8 cifs nfs aufs ccm fscache binfmt_misc overlay xfs libcrc32c intel_rapl crct10dif_pclmul crc32_pclmul ghash_clmulni_intel ppdev pcbc aesni_intel aes_x86_64 crypto_simd glue_helper cryptd input_leds joydev intel_rapl_perf serio_raw parport_pc parport mac_hid sch_fq_codel nfsd 8021q auth_rpcgss garp nfs_acl mrp lockd stp llc grace xenfs sunrpc xen_privcmd ip_tables x_tables autofs4 hid_generic usbhid hid psmouse i2c_piix4 pata_acpi floppy [ 696.396966] CPU: 6 PID: 0 Comm: swapper/6 Not tainted 4.15.0-109-generic #110-Ubuntu [ 696.396979] Hardware name: Xen HVM domU, BIOS 4.7.6-1.26 12/03/2018 [ 696.396993] RIP: 0010:__cgroup_bpf_run_filter_skb+0xbb/0x1e0 [ 696.397005] RSP: 0018:893fdcb83a70 EFLAGS: 00010292 [ 696.397015] RAX: 6d69546e6f697469 RBX: RCX: 0014 [ 696.397028] RDX: RSI: 893fd036 RDI: 893fb5154800 [ 696.397041] RBP: 893fdcb83ad0 R08: 0001 R09: [ 696.397058] R10: R11: 0003 R12: 0014 [ 696.397075] R13: 893fb5154800 R14: 0020 R15: 893fc6ba4d00 [ 696.397091] FS: () GS:893fdcb8() knlGS: [ 696.397107] CS: 0010 DS: ES: CR0: 80050033 [ 696.397119] CR2: 00c0001b4000 CR3: 0006dce0a004 CR4: 003606e0 [ 696.397135] DR0: DR1: DR2: [ 696.397152] DR3: DR6: fffe0ff0 DR7: 0400 [ 696.397169] Call Trace: [ 696.397175] [ 696.397183] sk_filter_trim_cap+0xd0/0x1b0 [ 696.397191] tcp_v4_rcv+0x8b7/0xa80 [ 696.397199] ip_local_deliver_finish+0x66/0x210 [ 696.397208] ip_local_deliver+0x7e/0xe0 [ 696.397215] ? ip_rcv_finish+0x430/0x430 [ 696.397223] ip_rcv_finish+0x129/0x430 [ 696.397230] ip_rcv+0x296/0x360 [ 696.397238] ? inet_del_offload+0x40/0x40 [ 696.397249] __netif_receive_skb_core+0x432/0xb80 [ 696.397261] ? skb_send_sock+0x50/0x50 [ 696.397271] ? tcp4_gro_receive+0x137/0x1a0 [ 696.397280] __netif_receive_skb+0x18/0x60 [ 696.397290] ? __netif_receive_skb+0x18/0x60 [ 696.397300] netif_receive_skb_internal+0x45/0xe0 [ 696.397309] napi_gro_receive+0xc5/0xf0 [ 696.397317] xennet_poll+0x9ca/0xbc0 [ 696.397325] net_rx_action+0x140/0x3a0 [ 696.397334] __do_softirq+0xe4/0x2d4 [ 696.397344] irq_exit+0xc5/0xd0 [ 696.397352] xen_evtchn_do_upcall+0x30/0x50 [ 696.397361] xen_hvm_callback_vector+0x90/0xa0 [ 696.397371] [ 696.397378] RIP: 0010:native_safe_halt+0x12/0x20 [ 696.397390] RSP: 0018:94c4862cbe80 EFLAGS: 0246 ORIG_RAX: ff0c [ 696.397405] RAX: 8e
[Bug 1886668] Re: linux 4.15.0-109-generic network DoS regression vs -108
This is caused by net_cls and net_prio cgroups disabling cgroup BPF and causing it to stop refcounting when allocating new sockets. Releasing those sockets will cause the refcount to go negative, leading to the potential use-after-free. Though this revert won't prevent the issue from happening as it could still theoretically be caused by setting net_cls.classid or net_prio.ifpriomap, this will prevent it from happening on default system configurations. A combination of systemd use of cgroup BPF and extensive cgroup use including net_prio will cause this. Reports usually involve using lxd, libvirt, docker or kubernetes and some systemd service with IPAddressDeny or IPAddressAllow. And though this patch has been introduced to avoid some potential memory leaks, the cure is worse than the disease. We will need to revisit both issues later on and reapply this patch when we have a real fix for the crash. Cascardo. ** Patch added: "0001-UBUNTU-SAUCE-Revert-netprio_cgroup-Fix-unlimited-mem.patch" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886668/+attachment/5390827/+files/0001-UBUNTU-SAUCE-Revert-netprio_cgroup-Fix-unlimited-mem.patch ** Also affects: linux (Ubuntu Bionic) Importance: Undecided Status: New ** Changed in: linux (Ubuntu) Status: Incomplete => Invalid ** Changed in: linux (Ubuntu Bionic) Status: New => In Progress ** Changed in: linux (Ubuntu Bionic) Assignee: (unassigned) => Thadeu Lima de Souza Cascardo (cascardo) ** Changed in: linux (Ubuntu Bionic) Importance: Undecided => Critical -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1886668 Title: linux 4.15.0-109-generic network DoS regression vs -108 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886668/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1886668] Re: linux 4.15.0-109-generic network DoS regression vs -108
** Information type changed from Private Security to Public Security -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1886668 Title: linux 4.15.0-109-generic network DoS regression vs -108 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886668/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
