[Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
This bug was fixed in the package linux - 4.4.0-161.189 --- linux (4.4.0-161.189) xenial; urgency=medium * xenial/linux: 4.4.0-161.189 -proposed tracker (LP: #1841544) * flock not mediated by 'k' (LP: 1658219) - Revert "UBUNTU: SAUCE: apparmor: flock mediation is not being, enforced on cache check" * Packaging resync (LP: #1786013) - [Packaging] resync getabis linux (4.4.0-160.188) xenial; urgency=medium * xenial/linux: 4.4.0-160.188 -proposed tracker (LP: #1840021) * Packaging resync (LP: #1786013) - [Packaging] update helper scripts * EeePC 1005px laptop backlight is off after system boot up (LP: #1837117) - platform/x86: asus-wmi: Only Tell EC the OS will handle display hotkeys from asus_nb_wmi * CVE-2019-10638 - [Config] CONFIG_TEST_HASH=n - siphash: add cryptographically secure PRF - inet: switch IP ID generator to siphash * Stacked onexec transitions fail when under NO NEW PRIVS restrictions (LP: #1839037) - SAUCE: apparmor: fix nnp subset check failure, when stacking * AppArmor onexec transition causes WARN kernel stack trace (LP: #1838627) - SAUCE: apparmor: fix audit failures when performing profile transitions * flock not mediated by 'k' (LP: 1658219) // Ubuntu 16.04: read access incorrectly implies 'm' rule (LP: 1838090) - SAUCE: apparmor: flock mediation is not being, enforced on cache check * bcache: bch_allocator_thread(): hung task timeout (LP: #1784665) // Tight timeout for bcache removal causes spurious failures (LP: #1796292) - SAUCE: bcache: fix deadlock in bcache_allocator * bcache: bch_allocator_thread(): hung task timeout (LP: #1784665) - bcache: improve bcache_reboot() - bcache: add journal statistic - bcache: fix high CPU occupancy during journal - bcache: fix incorrect sysfs output value of strip size - bcache: fix error return value in memory shrink - bcache: fix using of loop variable in memory shrink - bcache: Fix indentation - bcache: Add __printf annotation to __bch_check_keys() - bcache: Annotate switch fall-through - bcache: Fix kernel-doc warnings - bcache: Remove an unused variable - bcache: Suppress more warnings about set-but-not-used variables - bcache: Reduce the number of sparse complaints about lock imbalances - bcache: Move couple of functions to sysfs.c * CVE-2019-3900 - vhost: introduce vhost_vq_avail_empty() - vhost_net: tx batching - vhost_net: do not stall on zerocopy depletion - vhost-net: set packet weight of tx polling to 2 * vq size - vhost_net: use packet weight for rx handler, too - vhost_net: introduce vhost_exceeds_weight() - vhost: introduce vhost_exceeds_weight() - vhost_net: fix possible infinite loop - vhost: scsi: add weight support * Xenial: ZFS deadlock in shrinker path with xattrs (LP: #1839521) - SAUCE: (noup) Update zfs to 0.6.5.6-0ubuntu28 * CVE-2019-13648 - powerpc/tm: Fix oops on sigreturn on systems without TM * CVE-2018-20856 - block: blk_init_allocated_queue() set q->fq as NULL in the fail case * CVE-2019-14283 - floppy: fix out-of-bounds read in copy_buffer * CVE-2019-14284 - floppy: fix div-by-zero in setup_format_params * Xenial update: 4.4.186 upstream stable release (LP: #1838467) - Input: elantech - enable middle button support on 2 ThinkPads - samples, bpf: fix to change the buffer size for read() - mac80211: mesh: fix RCU warning - dt-bindings: can: mcp251x: add mcp25625 support - can: mcp251x: add support for mcp25625 - Input: imx_keypad - make sure keyboard can always wake up system - ARM: davinci: da850-evm: call regulator_has_full_constraints() - ARM: davinci: da8xx: specify dma_coherent_mask for lcdc - md: fix for divide error in status_resync - bnx2x: Check if transceiver implements DDM before access - udf: Fix incorrect final NOT_ALLOCATED (hole) extent length - x86/ptrace: Fix possible spectre-v1 in ptrace_get_debugreg() - x86/tls: Fix possible spectre-v1 in do_get_thread_area() - mwifiex: Abort at too short BSS descriptor element - fscrypt: don't set policy for a dead directory - mwifiex: Don't abort on small, spec-compliant vendor IEs - USB: serial: ftdi_sio: add ID for isodebug v1 - USB: serial: option: add support for GosunCn ME3630 RNDIS mode - usb: gadget: ether: Fix race between gether_disconnect and rx_submit - usb: renesas_usbhs: add a workaround for a race condition of workqueue - staging: comedi: dt282x: fix a null pointer deref on interrupt - staging: comedi: amplc_pci230: fix null pointer deref on interrupt - carl9170: fix misuse of device driver API - VMCI: Fix integer overflow in VMCI handle arrays - MIPS: Remove superfluous check for __linux__ - e1000e: start network tx queue only when link is up - perf/core: Fix perf_sample_regs_user() mm check -
[Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
This bug was fixed in the package linux - 5.0.0-27.28 --- linux (5.0.0-27.28) disco; urgency=medium * disco/linux: 5.0.0-27.28 -proposed tracker (LP: #1840816) * [Potential Regression] System crashes when running ftrace test in ubuntu_kernel_selftests (LP: #1840750) - x86/kprobes: Set instruction page as executable linux (5.0.0-26.27) disco; urgency=medium * disco/linux: 5.0.0-26.27 -proposed tracker (LP: #1839972) * Packaging resync (LP: #1786013) - [Packaging] update helper scripts * alsa/hdmi: add icelake hdmi audio support for a Dell machine (LP: #1836916) - ALSA: hda: hdmi - add Icelake support - ALSA: hda/hdmi - Remove duplicated define - ALSA: hda/hdmi - Fix i915 reverse port/pin mapping * input/mouse: alps trackpoint-only device doesn't work (LP: #1836752) - Input: alps - don't handle ALPS cs19 trackpoint-only device - Input: alps - fix a mismatch between a condition check and its comment * [18.04 FEAT] Enhanced hardware support (LP: #1836857) - s390: report new CPU capabilities - s390: add alignment hints to vector load and store * System does not auto detect disconnection of external monitor (LP: #1835001) - drm/i915: Add support for retrying hotplug - drm/i915: Enable hotplug retry * [18.04 FEAT] Enhanced CPU-MF hardware counters - kernel part (LP: #1836860) - s390/cpum_cf: Add support for CPU-MF SVN 6 - s390/cpumf: Add extended counter set definitions for model 8561 and 8562 * EeePC 1005px laptop backlight is off after system boot up (LP: #1837117) - platform/x86: asus-wmi: Only Tell EC the OS will handle display hotkeys from asus_nb_wmi * br_netfilter: namespace sysctl operations (LP: #1836910) - netfilter: bridge: port sysctls to use brnf_net - netfilter: bridge: namespace bridge netfilter sysctls - netfilter: bridge: prevent UAF in brnf_exit_net() * ideapad_laptop disables WiFi/BT radios on Lenovo Y540 (LP: #1837136) - platform/x86: ideapad-laptop: Remove no_hw_rfkill_list * shiftfs: allow overlayfs (LP: #1838677) - SAUCE: shiftfs: enable overlayfs on shiftfs * bcache: bch_allocator_thread(): hung task timeout (LP: #1784665) - bcache: never writeback a discard operation - bcache: improve bcache_reboot() - SAUCE: bcache: fix deadlock in bcache_allocator * Regressions in CMA allocation rework (LP: #1839395) - dma-contiguous: do not overwrite align in dma_alloc_contiguous() - dma-contiguous: page-align the size in dma_free_contiguous() * CVE-2019-3900 - vhost: introduce vhost_exceeds_weight() - vhost_net: fix possible infinite loop - vhost: vsock: add weight support - vhost: scsi: add weight support * Disco update: 5.0.21 upstream stable release (LP: #1837518) - bonding/802.3ad: fix slave link initialization transition states - cxgb4: offload VLAN flows regardless of VLAN ethtype - inet: switch IP ID generator to siphash - ipv4/igmp: fix another memory leak in igmpv3_del_delrec() - ipv4/igmp: fix build error if !CONFIG_IP_MULTICAST - ipv6: Consider sk_bound_dev_if when binding a raw socket to an address - ipv6: Fix redirect with VRF - llc: fix skb leak in llc_build_and_send_ui_pkt() - mlxsw: spectrum_acl: Avoid warning after identical rules insertion - net: dsa: mv88e6xxx: fix handling of upper half of STATS_TYPE_PORT - net: fec: fix the clk mismatch in failed_reset path - net-gro: fix use-after-free read in napi_gro_frags() - net: mvneta: Fix err code path of probe - net: mvpp2: fix bad MVPP2_TXQ_SCHED_TOKEN_CNTR_REG queue value - net: phy: marvell10g: report if the PHY fails to boot firmware - net: sched: don't use tc_action->order during action dump - net: stmmac: fix reset gpio free missing - r8169: fix MAC address being lost in PCI D3 - usbnet: fix kernel crash after disconnect - net/mlx5: Avoid double free in fs init error unwinding path - tipc: Avoid copying bytes beyond the supplied data - net/mlx5: Allocate root ns memory using kzalloc to match kfree - net/mlx5e: Disable rxhash when CQE compress is enabled - net: stmmac: fix ethtool flow control not able to get/set - net: stmmac: dma channel control register need to be init first - bnxt_en: Fix aggregation buffer leak under OOM condition. - bnxt_en: Fix possible BUG() condition when calling pci_disable_msix(). - bnxt_en: Reduce memory usage when running in kdump kernel. - net/tls: fix state removal with feature flags off - net/tls: don't ignore netdev notifications if no TLS features - cxgb4: Revert "cxgb4: Remove SGE_HOST_PAGE_SIZE dependency on page size" - net: correct zerocopy refcnt with udp MSG_MORE - crypto: vmx - ghash: do nosimd fallback manually - xen/pciback: Don't disable PCI_COMMAND on PCI device reset. - Revert "tipc: fix modprobe tipc failed after switch order of device registration" -
[Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
@jason-hobbs, I don't see any kernel error in the log attached to comment #53, so it seems to be a different issue, maybe a performance regression that is causing curtin to timeout. I would keep this bug closed / fixed and open a new one. If you can easily reproduce it, could you open a new bug and collect some info on the box after the problem happens? (in particular uname -a, dmesg, ps axuw). Thanks! -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1784665 Title: bcache: bch_allocator_thread(): hung task timeout Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Committed Status in linux source package in Bionic: Fix Committed Status in linux source package in Disco: Fix Committed Status in linux source package in Eoan: Fix Released Bug description: [Impact] bcache_allocator() can call the following: bch_allocator_thread() -> bch_prio_write() -> bch_bucket_alloc() -> wait on >set->bucket_wait But the wake up event on bucket_wait is supposed to come from bch_allocator_thread() itself causing a deadlock. [Test Case] This is a simple script that can easily trigger the deadlock condition: https://launchpadlibrarian.net/381282009/bcache-basic-repro.sh A better test case has been also provided in bug 1796292 (duplicate of this bug): https://bugs.launchpad.net/curtin/+bug/1796292/+attachment/5280353/+files/curtin-nvme.sh [Fix] Fix by making the call to bch_prio_write() non-blocking, so that bch_allocator_thread() never waits on itself. Moreover, make sure to wake up the garbage collector thread when bch_prio_write() is failing to allocate buckets to increase the chance of freeing up more buckets. In addition to that it would be safe to also import other upstream bcache fixes (all clean cherry picks): 7e865eba00a3df2dc8c4746173a8ca1c1c7f042e bcache: fix potential deadlock in cached_def_free() 80265d8dfd77792e133793cef44a21323aac2908 bcache: acquire bch_register_lock later in cached_dev_free() ce4c3e19e5201424357a0c82176633b32a98d2ec bcache: Replace bch_read_string_list() by __sysfs_match_string() ecb37ce9baac653cc09e2b631393dde3df82979f bcache: Move couple of functions to sysfs.c 04cbc21137bfa4d7b8771a5b14f3d6c9b2aee671 bcache: Move couple of string arrays to sysfs.c 5f2b18ec8e1643410a2369f06888951cdedea0bf bcache: Fix a compiler warning in bcache_device_init() 20d3a518713e394efa5a899c84574b4b79ec5098 bcache: Reduce the number of sparse complaints about lock imbalances 42361469ae84c851e40cb1f94c8c9a14cdd94039 bcache: Suppress more warnings about set-but-not-used variables f0d3814090ac77de94c42b7124c37ece23629197 bcache: Remove an unused variable 47344e330eabc1515cbe6061eb337100a3ab6d37 bcache: Fix kernel-doc warnings 9dfbdec7b7fea1ff1b7b5d5d12980dbc7dca46c7 bcache: Annotate switch fall-through 4a4e443835a43a79113cc237c472c0d268eb1e1c bcache: Add __printf annotation to __bch_check_keys() fd01991d5c20098c5c1ffc4dca6c821cc60a2f74 bcache: Fix indentation ca71df31661a0518ed58a1a59cf1993962153ebb bcache: fix using of loop variable in memory shrink f3641c3abd1da978ee969b0203b71b86ec1bfa93 bcache: fix error return value in memory shrink 688892b3bc05e25da94866e32210e5f503f16f69 bcache: fix incorrect sysfs output value of strip size 09a44ca2114737e0932257619c16a2b50c7807f1 bcache: use pr_info() to inform duplicated CACHE_SET_IO_DISABLE set c4dc2497d50d9c6fb16aa0d07b6a14f3b2adb1e0 bcache: fix high CPU occupancy during journal a728eacbbdd229d1d903e46261c57d5206f87a4a bcache: add journal statistic 616486ab52ab7f9739b066d958bdd20e65aefd74 bcache: fix writeback target calc on large devices 1f0ffa67349c56ea54c03ccfd1e073c990e7411e bcache: only set BCACHE_DEV_WB_RUNNING when cached device attached eb8cbb6df38f6e5124a3d5f1f8a3dbf519537c60 bcache: improve bcache_reboot() 9951379b0ca88c95876ad9778b9099e19a95d566 bcache: never writeback a discard operation [Regression Potential] The upstream fixes are all clean cherry picks from stable (most of them are small cleanups), so regression potential is minimal. The only special patch is "UBUNTU: SAUCE: bcache: fix deadlock in bcache_allocator()" that is addressing the main deadlock bug (that seems to be a mainline bug - not fixed yet). We should spend more time trying to reproduce this deadlock with a mainline kernel and post the patch to the LKML for review / feedback. However, considering that this patch seems to fix/prevent the specific deadlock problem reported in this bug (tested on the affected platform) it can be considered safe to apply it. [Original Bug Report] $ cat /proc/version_signature Ubuntu 4.15.0-29.31-generic 4.15.18 $ lsb_release -rd Description: Ubuntu Cosmic Cuttlefish (development branch) Release: 18.10 $
[Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
Marking verification done for Xenial based on comment #57. ** Tags removed: verification-needed-xenial ** Tags added: verification-done-xenial -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1784665 Title: bcache: bch_allocator_thread(): hung task timeout Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Committed Status in linux source package in Bionic: Fix Committed Status in linux source package in Disco: Fix Committed Status in linux source package in Eoan: Fix Released Bug description: [Impact] bcache_allocator() can call the following: bch_allocator_thread() -> bch_prio_write() -> bch_bucket_alloc() -> wait on >set->bucket_wait But the wake up event on bucket_wait is supposed to come from bch_allocator_thread() itself causing a deadlock. [Test Case] This is a simple script that can easily trigger the deadlock condition: https://launchpadlibrarian.net/381282009/bcache-basic-repro.sh A better test case has been also provided in bug 1796292 (duplicate of this bug): https://bugs.launchpad.net/curtin/+bug/1796292/+attachment/5280353/+files/curtin-nvme.sh [Fix] Fix by making the call to bch_prio_write() non-blocking, so that bch_allocator_thread() never waits on itself. Moreover, make sure to wake up the garbage collector thread when bch_prio_write() is failing to allocate buckets to increase the chance of freeing up more buckets. In addition to that it would be safe to also import other upstream bcache fixes (all clean cherry picks): 7e865eba00a3df2dc8c4746173a8ca1c1c7f042e bcache: fix potential deadlock in cached_def_free() 80265d8dfd77792e133793cef44a21323aac2908 bcache: acquire bch_register_lock later in cached_dev_free() ce4c3e19e5201424357a0c82176633b32a98d2ec bcache: Replace bch_read_string_list() by __sysfs_match_string() ecb37ce9baac653cc09e2b631393dde3df82979f bcache: Move couple of functions to sysfs.c 04cbc21137bfa4d7b8771a5b14f3d6c9b2aee671 bcache: Move couple of string arrays to sysfs.c 5f2b18ec8e1643410a2369f06888951cdedea0bf bcache: Fix a compiler warning in bcache_device_init() 20d3a518713e394efa5a899c84574b4b79ec5098 bcache: Reduce the number of sparse complaints about lock imbalances 42361469ae84c851e40cb1f94c8c9a14cdd94039 bcache: Suppress more warnings about set-but-not-used variables f0d3814090ac77de94c42b7124c37ece23629197 bcache: Remove an unused variable 47344e330eabc1515cbe6061eb337100a3ab6d37 bcache: Fix kernel-doc warnings 9dfbdec7b7fea1ff1b7b5d5d12980dbc7dca46c7 bcache: Annotate switch fall-through 4a4e443835a43a79113cc237c472c0d268eb1e1c bcache: Add __printf annotation to __bch_check_keys() fd01991d5c20098c5c1ffc4dca6c821cc60a2f74 bcache: Fix indentation ca71df31661a0518ed58a1a59cf1993962153ebb bcache: fix using of loop variable in memory shrink f3641c3abd1da978ee969b0203b71b86ec1bfa93 bcache: fix error return value in memory shrink 688892b3bc05e25da94866e32210e5f503f16f69 bcache: fix incorrect sysfs output value of strip size 09a44ca2114737e0932257619c16a2b50c7807f1 bcache: use pr_info() to inform duplicated CACHE_SET_IO_DISABLE set c4dc2497d50d9c6fb16aa0d07b6a14f3b2adb1e0 bcache: fix high CPU occupancy during journal a728eacbbdd229d1d903e46261c57d5206f87a4a bcache: add journal statistic 616486ab52ab7f9739b066d958bdd20e65aefd74 bcache: fix writeback target calc on large devices 1f0ffa67349c56ea54c03ccfd1e073c990e7411e bcache: only set BCACHE_DEV_WB_RUNNING when cached device attached eb8cbb6df38f6e5124a3d5f1f8a3dbf519537c60 bcache: improve bcache_reboot() 9951379b0ca88c95876ad9778b9099e19a95d566 bcache: never writeback a discard operation [Regression Potential] The upstream fixes are all clean cherry picks from stable (most of them are small cleanups), so regression potential is minimal. The only special patch is "UBUNTU: SAUCE: bcache: fix deadlock in bcache_allocator()" that is addressing the main deadlock bug (that seems to be a mainline bug - not fixed yet). We should spend more time trying to reproduce this deadlock with a mainline kernel and post the patch to the LKML for review / feedback. However, considering that this patch seems to fix/prevent the specific deadlock problem reported in this bug (tested on the affected platform) it can be considered safe to apply it. [Original Bug Report] $ cat /proc/version_signature Ubuntu 4.15.0-29.31-generic 4.15.18 $ lsb_release -rd Description: Ubuntu Cosmic Cuttlefish (development branch) Release: 18.10 $ apt-cache policy linux-image-`uname -r` linux-image-4.15.0-29-generic: Installed: 4.15.0-29.31 Candidate: 4.15.0-29.31 Version table: *** 4.15.0-29.31 500 500 http://archive.ubuntu.com/ubuntu cosmic/main amd64 Packages 100
[Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
This bug was fixed in the package linux - 5.2.0-13.14 --- linux (5.2.0-13.14) eoan; urgency=medium * eoan/linux: 5.2.0-13.14 -proposed tracker (LP: #1840261) * NULL pointer dereference when Inserting the VIMC module (LP: #1840028) - media: vimc: fix component match compare * Miscellaneous upstream changes - selftests/bpf: remove bpf_util.h from BPF C progs linux (5.2.0-12.13) eoan; urgency=medium * eoan/linux: 5.2.0-12.13 -proposed tracker (LP: #1840184) * Eoan update: v5.2.8 upstream stable release (LP: #1840178) - scsi: fcoe: Embed fc_rport_priv in fcoe_rport structure - libnvdimm/bus: Prepare the nd_ioctl() path to be re-entrant - libnvdimm/bus: Fix wait_nvdimm_bus_probe_idle() ABBA deadlock - ALSA: usb-audio: Sanity checks for each pipe and EP types - ALSA: usb-audio: Fix gpf in snd_usb_pipe_sanity_check - HID: wacom: fix bit shift for Cintiq Companion 2 - HID: Add quirk for HP X1200 PIXART OEM mouse - atm: iphase: Fix Spectre v1 vulnerability - bnx2x: Disable multi-cos feature. - drivers/net/ethernet/marvell/mvmdio.c: Fix non OF case - ife: error out when nla attributes are empty - ip6_gre: reload ipv6h in prepare_ip6gre_xmit_ipv6 - ip6_tunnel: fix possible use-after-free on xmit - ipip: validate header length in ipip_tunnel_xmit - mlxsw: spectrum: Fix error path in mlxsw_sp_module_init() - mvpp2: fix panic on module removal - mvpp2: refactor MTU change code - net: bridge: delete local fdb on device init failure - net: bridge: mcast: don't delete permanent entries when fast leave is enabled - net: bridge: move default pvid init/deinit to NETDEV_REGISTER/UNREGISTER - net: fix ifindex collision during namespace removal - net/mlx5e: always initialize frag->last_in_page - net/mlx5: Use reversed order when unregister devices - net: phy: fixed_phy: print gpio error only if gpio node is present - net: phylink: don't start and stop SGMII PHYs in SFP modules twice - net: phylink: Fix flow control for fixed-link - net: phy: mscc: initialize stats array - net: qualcomm: rmnet: Fix incorrect UL checksum offload logic - net: sched: Fix a possible null-pointer dereference in dequeue_func() - net sched: update vlan action for batched events operations - net: sched: use temporary variable for actions indexes - net/smc: do not schedule tx_work in SMC_CLOSED state - net: stmmac: Use netif_tx_napi_add() for TX polling function - NFC: nfcmrvl: fix gpio-handling regression - ocelot: Cancel delayed work before wq destruction - tipc: compat: allow tipc commands without arguments - tipc: fix unitilized skb list crash - tun: mark small packets as owned by the tap sock - net/mlx5: Fix modify_cq_in alignment - net/mlx5e: Prevent encap flow counter update async to user query - r8169: don't use MSI before RTL8168d - bpf: fix XDP vlan selftests test_xdp_vlan.sh - selftests/bpf: add wrapper scripts for test_xdp_vlan.sh - selftests/bpf: reduce time to execute test_xdp_vlan.sh - net: fix bpf_xdp_adjust_head regression for generic-XDP - hv_sock: Fix hang when a connection is closed - net: phy: fix race in genphy_update_link - net/smc: avoid fallback in case of non-blocking connect - rocker: fix memory leaks of fib_work on two error return paths - mlxsw: spectrum_buffers: Further reduce pool size on Spectrum-2 - net/mlx5: Add missing RDMA_RX capabilities - net/mlx5e: Fix matching of speed to PRM link modes - compat_ioctl: pppoe: fix PPPOEIOCSFWD handling - drm/i915/vbt: Fix VBT parsing for the PSR section - Revert "mac80211: set NETIF_F_LLTX when using intermediate tx queues" - spi: bcm2835: Fix 3-wire mode if DMA is enabled - Linux 5.2.8 * Miscellaneous Ubuntu changes - SAUCE: selftests/bpf: do not include Kbuild.include in makefile - update dkms package versions linux (5.2.0-11.12) eoan; urgency=medium * eoan/linux: 5.2.0-11.12 -proposed tracker (LP: #1839646) * Packaging resync (LP: #1786013) - [Packaging] resync getabis - [Packaging] update helper scripts * Eoan update: v5.2.7 upstream stable release (LP: #1839588) - Revert "UBUNTU: SAUCE: Revert "loop: Don't change loop device under exclusive opener"" - ARM: riscpc: fix DMA - ARM: dts: rockchip: Make rk3288-veyron-minnie run at hs200 - ARM: dts: rockchip: Make rk3288-veyron-mickey's emmc work again - clk: meson: mpll: properly handle spread spectrum - ARM: dts: rockchip: Mark that the rk3288 timer might stop in suspend - ftrace: Enable trampoline when rec count returns back to one - arm64: dts: qcom: qcs404-evb: fix l3 min voltage - soc: qcom: rpmpd: fixup rpmpd set performance state - arm64: dts: marvell: mcbin: enlarge PCI memory window - soc: imx: soc-imx8: Correct return value of error handle - dmaengine: tegra-apb: Error out if
[Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
** Changed in: linux (Ubuntu Bionic) Status: New => Fix Committed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1784665 Title: bcache: bch_allocator_thread(): hung task timeout Status in linux package in Ubuntu: Fix Committed Status in linux source package in Xenial: Fix Committed Status in linux source package in Bionic: Fix Committed Status in linux source package in Disco: Fix Committed Status in linux source package in Eoan: Fix Committed Bug description: [Impact] bcache_allocator() can call the following: bch_allocator_thread() -> bch_prio_write() -> bch_bucket_alloc() -> wait on >set->bucket_wait But the wake up event on bucket_wait is supposed to come from bch_allocator_thread() itself causing a deadlock. [Test Case] This is a simple script that can easily trigger the deadlock condition: https://launchpadlibrarian.net/381282009/bcache-basic-repro.sh A better test case has been also provided in bug 1796292 (duplicate of this bug): https://bugs.launchpad.net/curtin/+bug/1796292/+attachment/5280353/+files/curtin-nvme.sh [Fix] Fix by making the call to bch_prio_write() non-blocking, so that bch_allocator_thread() never waits on itself. Moreover, make sure to wake up the garbage collector thread when bch_prio_write() is failing to allocate buckets to increase the chance of freeing up more buckets. In addition to that it would be safe to also import other upstream bcache fixes (all clean cherry picks): 7e865eba00a3df2dc8c4746173a8ca1c1c7f042e bcache: fix potential deadlock in cached_def_free() 80265d8dfd77792e133793cef44a21323aac2908 bcache: acquire bch_register_lock later in cached_dev_free() ce4c3e19e5201424357a0c82176633b32a98d2ec bcache: Replace bch_read_string_list() by __sysfs_match_string() ecb37ce9baac653cc09e2b631393dde3df82979f bcache: Move couple of functions to sysfs.c 04cbc21137bfa4d7b8771a5b14f3d6c9b2aee671 bcache: Move couple of string arrays to sysfs.c 5f2b18ec8e1643410a2369f06888951cdedea0bf bcache: Fix a compiler warning in bcache_device_init() 20d3a518713e394efa5a899c84574b4b79ec5098 bcache: Reduce the number of sparse complaints about lock imbalances 42361469ae84c851e40cb1f94c8c9a14cdd94039 bcache: Suppress more warnings about set-but-not-used variables f0d3814090ac77de94c42b7124c37ece23629197 bcache: Remove an unused variable 47344e330eabc1515cbe6061eb337100a3ab6d37 bcache: Fix kernel-doc warnings 9dfbdec7b7fea1ff1b7b5d5d12980dbc7dca46c7 bcache: Annotate switch fall-through 4a4e443835a43a79113cc237c472c0d268eb1e1c bcache: Add __printf annotation to __bch_check_keys() fd01991d5c20098c5c1ffc4dca6c821cc60a2f74 bcache: Fix indentation ca71df31661a0518ed58a1a59cf1993962153ebb bcache: fix using of loop variable in memory shrink f3641c3abd1da978ee969b0203b71b86ec1bfa93 bcache: fix error return value in memory shrink 688892b3bc05e25da94866e32210e5f503f16f69 bcache: fix incorrect sysfs output value of strip size 09a44ca2114737e0932257619c16a2b50c7807f1 bcache: use pr_info() to inform duplicated CACHE_SET_IO_DISABLE set c4dc2497d50d9c6fb16aa0d07b6a14f3b2adb1e0 bcache: fix high CPU occupancy during journal a728eacbbdd229d1d903e46261c57d5206f87a4a bcache: add journal statistic 616486ab52ab7f9739b066d958bdd20e65aefd74 bcache: fix writeback target calc on large devices 1f0ffa67349c56ea54c03ccfd1e073c990e7411e bcache: only set BCACHE_DEV_WB_RUNNING when cached device attached eb8cbb6df38f6e5124a3d5f1f8a3dbf519537c60 bcache: improve bcache_reboot() 9951379b0ca88c95876ad9778b9099e19a95d566 bcache: never writeback a discard operation [Regression Potential] The upstream fixes are all clean cherry picks from stable (most of them are small cleanups), so regression potential is minimal. The only special patch is "UBUNTU: SAUCE: bcache: fix deadlock in bcache_allocator()" that is addressing the main deadlock bug (that seems to be a mainline bug - not fixed yet). We should spend more time trying to reproduce this deadlock with a mainline kernel and post the patch to the LKML for review / feedback. However, considering that this patch seems to fix/prevent the specific deadlock problem reported in this bug (tested on the affected platform) it can be considered safe to apply it. [Original Bug Report] $ cat /proc/version_signature Ubuntu 4.15.0-29.31-generic 4.15.18 $ lsb_release -rd Description: Ubuntu Cosmic Cuttlefish (development branch) Release: 18.10 $ apt-cache policy linux-image-`uname -r` linux-image-4.15.0-29-generic: Installed: 4.15.0-29.31 Candidate: 4.15.0-29.31 Version table: *** 4.15.0-29.31 500 500 http://archive.ubuntu.com/ubuntu cosmic/main amd64 Packages 100 /var/lib/dpkg/status 3) mkfs.ext4 /dev/bcache0 returns successful creating an
[Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
Overnight testing of the revised deployment configuration has no errors, 200 runs completed. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1784665 Title: bcache: bch_allocator_thread(): hung task timeout Status in linux package in Ubuntu: Fix Committed Status in linux source package in Xenial: Fix Committed Status in linux source package in Bionic: New Status in linux source package in Disco: Fix Committed Status in linux source package in Eoan: Fix Committed Bug description: [Impact] bcache_allocator() can call the following: bch_allocator_thread() -> bch_prio_write() -> bch_bucket_alloc() -> wait on >set->bucket_wait But the wake up event on bucket_wait is supposed to come from bch_allocator_thread() itself causing a deadlock. [Test Case] This is a simple script that can easily trigger the deadlock condition: https://launchpadlibrarian.net/381282009/bcache-basic-repro.sh A better test case has been also provided in bug 1796292 (duplicate of this bug): https://bugs.launchpad.net/curtin/+bug/1796292/+attachment/5280353/+files/curtin-nvme.sh [Fix] Fix by making the call to bch_prio_write() non-blocking, so that bch_allocator_thread() never waits on itself. Moreover, make sure to wake up the garbage collector thread when bch_prio_write() is failing to allocate buckets to increase the chance of freeing up more buckets. In addition to that it would be safe to also import other upstream bcache fixes (all clean cherry picks): 7e865eba00a3df2dc8c4746173a8ca1c1c7f042e bcache: fix potential deadlock in cached_def_free() 80265d8dfd77792e133793cef44a21323aac2908 bcache: acquire bch_register_lock later in cached_dev_free() ce4c3e19e5201424357a0c82176633b32a98d2ec bcache: Replace bch_read_string_list() by __sysfs_match_string() ecb37ce9baac653cc09e2b631393dde3df82979f bcache: Move couple of functions to sysfs.c 04cbc21137bfa4d7b8771a5b14f3d6c9b2aee671 bcache: Move couple of string arrays to sysfs.c 5f2b18ec8e1643410a2369f06888951cdedea0bf bcache: Fix a compiler warning in bcache_device_init() 20d3a518713e394efa5a899c84574b4b79ec5098 bcache: Reduce the number of sparse complaints about lock imbalances 42361469ae84c851e40cb1f94c8c9a14cdd94039 bcache: Suppress more warnings about set-but-not-used variables f0d3814090ac77de94c42b7124c37ece23629197 bcache: Remove an unused variable 47344e330eabc1515cbe6061eb337100a3ab6d37 bcache: Fix kernel-doc warnings 9dfbdec7b7fea1ff1b7b5d5d12980dbc7dca46c7 bcache: Annotate switch fall-through 4a4e443835a43a79113cc237c472c0d268eb1e1c bcache: Add __printf annotation to __bch_check_keys() fd01991d5c20098c5c1ffc4dca6c821cc60a2f74 bcache: Fix indentation ca71df31661a0518ed58a1a59cf1993962153ebb bcache: fix using of loop variable in memory shrink f3641c3abd1da978ee969b0203b71b86ec1bfa93 bcache: fix error return value in memory shrink 688892b3bc05e25da94866e32210e5f503f16f69 bcache: fix incorrect sysfs output value of strip size 09a44ca2114737e0932257619c16a2b50c7807f1 bcache: use pr_info() to inform duplicated CACHE_SET_IO_DISABLE set c4dc2497d50d9c6fb16aa0d07b6a14f3b2adb1e0 bcache: fix high CPU occupancy during journal a728eacbbdd229d1d903e46261c57d5206f87a4a bcache: add journal statistic 616486ab52ab7f9739b066d958bdd20e65aefd74 bcache: fix writeback target calc on large devices 1f0ffa67349c56ea54c03ccfd1e073c990e7411e bcache: only set BCACHE_DEV_WB_RUNNING when cached device attached eb8cbb6df38f6e5124a3d5f1f8a3dbf519537c60 bcache: improve bcache_reboot() 9951379b0ca88c95876ad9778b9099e19a95d566 bcache: never writeback a discard operation [Regression Potential] The upstream fixes are all clean cherry picks from stable (most of them are small cleanups), so regression potential is minimal. The only special patch is "UBUNTU: SAUCE: bcache: fix deadlock in bcache_allocator()" that is addressing the main deadlock bug (that seems to be a mainline bug - not fixed yet). We should spend more time trying to reproduce this deadlock with a mainline kernel and post the patch to the LKML for review / feedback. However, considering that this patch seems to fix/prevent the specific deadlock problem reported in this bug (tested on the affected platform) it can be considered safe to apply it. [Original Bug Report] $ cat /proc/version_signature Ubuntu 4.15.0-29.31-generic 4.15.18 $ lsb_release -rd Description: Ubuntu Cosmic Cuttlefish (development branch) Release: 18.10 $ apt-cache policy linux-image-`uname -r` linux-image-4.15.0-29-generic: Installed: 4.15.0-29.31 Candidate: 4.15.0-29.31 Version table: *** 4.15.0-29.31 500 500 http://archive.ubuntu.com/ubuntu cosmic/main amd64 Packages 100 /var/lib/dpkg/status 3) mkfs.ext4 /dev/bcache0 returns successful
Re: [Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
@ Ryan we do not test Xenial or Disco On Thu, Aug 22, 2019 at 7:41 PM Ryan Harper <1784...@bugs.launchpad.net> wrote: > Finally, I did verify xenial proposed with our original test. I had > over 100 installs with no issue. > > @Jason > > Have you had any runs on Xenial or Disco? (or do you not test those)? > > -- > You received this bug notification because you are a member of Canonical > Field Critical, which is subscribed to a duplicate bug report (1796292). > https://bugs.launchpad.net/bugs/1784665 > > Title: > bcache: bch_allocator_thread(): hung task timeout > > Status in linux package in Ubuntu: > Fix Committed > Status in linux source package in Xenial: > Fix Committed > Status in linux source package in Bionic: > New > Status in linux source package in Disco: > Fix Committed > Status in linux source package in Eoan: > Fix Committed > > Bug description: > [Impact] > > bcache_allocator() can call the following: > >bch_allocator_thread() > -> bch_prio_write() >-> bch_bucket_alloc() > -> wait on >set->bucket_wait > > But the wake up event on bucket_wait is supposed to come from > bch_allocator_thread() itself causing a deadlock. > > [Test Case] > > This is a simple script that can easily trigger the deadlock condition: > https://launchpadlibrarian.net/381282009/bcache-basic-repro.sh > > A better test case has been also provided in bug 1796292 (duplicate of > this bug): > > https://bugs.launchpad.net/curtin/+bug/1796292/+attachment/5280353/+files/curtin-nvme.sh > > [Fix] > > Fix by making the call to bch_prio_write() non-blocking, so that > bch_allocator_thread() never waits on itself. Moreover, make sure to > wake up the garbage collector thread when bch_prio_write() is failing > to allocate buckets to increase the chance of freeing up more buckets. > > In addition to that it would be safe to also import other upstream > bcache fixes (all clean cherry picks): > > 7e865eba00a3df2dc8c4746173a8ca1c1c7f042e bcache: fix potential deadlock > in cached_def_free() > 80265d8dfd77792e133793cef44a21323aac2908 bcache: acquire > bch_register_lock later in cached_dev_free() > ce4c3e19e5201424357a0c82176633b32a98d2ec bcache: Replace > bch_read_string_list() by __sysfs_match_string() > ecb37ce9baac653cc09e2b631393dde3df82979f bcache: Move couple of > functions to sysfs.c > 04cbc21137bfa4d7b8771a5b14f3d6c9b2aee671 bcache: Move couple of string > arrays to sysfs.c > 5f2b18ec8e1643410a2369f06888951cdedea0bf bcache: Fix a compiler warning > in bcache_device_init() > 20d3a518713e394efa5a899c84574b4b79ec5098 bcache: Reduce the number of > sparse complaints about lock imbalances > 42361469ae84c851e40cb1f94c8c9a14cdd94039 bcache: Suppress more warnings > about set-but-not-used variables > f0d3814090ac77de94c42b7124c37ece23629197 bcache: Remove an unused > variable > 47344e330eabc1515cbe6061eb337100a3ab6d37 bcache: Fix kernel-doc warnings > 9dfbdec7b7fea1ff1b7b5d5d12980dbc7dca46c7 bcache: Annotate switch > fall-through > 4a4e443835a43a79113cc237c472c0d268eb1e1c bcache: Add __printf annotation > to __bch_check_keys() > fd01991d5c20098c5c1ffc4dca6c821cc60a2f74 bcache: Fix indentation > ca71df31661a0518ed58a1a59cf1993962153ebb bcache: fix using of loop > variable in memory shrink > f3641c3abd1da978ee969b0203b71b86ec1bfa93 bcache: fix error return value > in memory shrink > 688892b3bc05e25da94866e32210e5f503f16f69 bcache: fix incorrect sysfs > output value of strip size > 09a44ca2114737e0932257619c16a2b50c7807f1 bcache: use pr_info() to inform > duplicated CACHE_SET_IO_DISABLE set > c4dc2497d50d9c6fb16aa0d07b6a14f3b2adb1e0 bcache: fix high CPU occupancy > during journal > a728eacbbdd229d1d903e46261c57d5206f87a4a bcache: add journal statistic > 616486ab52ab7f9739b066d958bdd20e65aefd74 bcache: fix writeback target > calc on large devices > 1f0ffa67349c56ea54c03ccfd1e073c990e7411e bcache: only set > BCACHE_DEV_WB_RUNNING when cached device attached > eb8cbb6df38f6e5124a3d5f1f8a3dbf519537c60 bcache: improve bcache_reboot() > 9951379b0ca88c95876ad9778b9099e19a95d566 bcache: never writeback a > discard operation > > [Regression Potential] > > The upstream fixes are all clean cherry picks from stable (most of > them are small cleanups), so regression potential is minimal. > > The only special patch is "UBUNTU: SAUCE: bcache: fix deadlock in > bcache_allocator()" that is addressing the main deadlock bug (that > seems to be a mainline bug - not fixed yet). We should spend more time > trying to reproduce this deadlock with a mainline kernel and post the > patch to the LKML for review / feedback. > > However, considering that this patch seems to fix/prevent the specific > deadlock problem reported in this bug (tested on the affected > platform) it can be considered safe to apply it. > > [Original Bug Report] > > $ cat /proc/version_signature > Ubuntu 4.15.0-29.31-generic 4.15.18 > > $ lsb_release
[Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
Finally, I did verify xenial proposed with our original test. I had over 100 installs with no issue. @Jason Have you had any runs on Xenial or Disco? (or do you not test those)? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1784665 Title: bcache: bch_allocator_thread(): hung task timeout Status in linux package in Ubuntu: Fix Committed Status in linux source package in Xenial: Fix Committed Status in linux source package in Bionic: New Status in linux source package in Disco: Fix Committed Status in linux source package in Eoan: Fix Committed Bug description: [Impact] bcache_allocator() can call the following: bch_allocator_thread() -> bch_prio_write() -> bch_bucket_alloc() -> wait on >set->bucket_wait But the wake up event on bucket_wait is supposed to come from bch_allocator_thread() itself causing a deadlock. [Test Case] This is a simple script that can easily trigger the deadlock condition: https://launchpadlibrarian.net/381282009/bcache-basic-repro.sh A better test case has been also provided in bug 1796292 (duplicate of this bug): https://bugs.launchpad.net/curtin/+bug/1796292/+attachment/5280353/+files/curtin-nvme.sh [Fix] Fix by making the call to bch_prio_write() non-blocking, so that bch_allocator_thread() never waits on itself. Moreover, make sure to wake up the garbage collector thread when bch_prio_write() is failing to allocate buckets to increase the chance of freeing up more buckets. In addition to that it would be safe to also import other upstream bcache fixes (all clean cherry picks): 7e865eba00a3df2dc8c4746173a8ca1c1c7f042e bcache: fix potential deadlock in cached_def_free() 80265d8dfd77792e133793cef44a21323aac2908 bcache: acquire bch_register_lock later in cached_dev_free() ce4c3e19e5201424357a0c82176633b32a98d2ec bcache: Replace bch_read_string_list() by __sysfs_match_string() ecb37ce9baac653cc09e2b631393dde3df82979f bcache: Move couple of functions to sysfs.c 04cbc21137bfa4d7b8771a5b14f3d6c9b2aee671 bcache: Move couple of string arrays to sysfs.c 5f2b18ec8e1643410a2369f06888951cdedea0bf bcache: Fix a compiler warning in bcache_device_init() 20d3a518713e394efa5a899c84574b4b79ec5098 bcache: Reduce the number of sparse complaints about lock imbalances 42361469ae84c851e40cb1f94c8c9a14cdd94039 bcache: Suppress more warnings about set-but-not-used variables f0d3814090ac77de94c42b7124c37ece23629197 bcache: Remove an unused variable 47344e330eabc1515cbe6061eb337100a3ab6d37 bcache: Fix kernel-doc warnings 9dfbdec7b7fea1ff1b7b5d5d12980dbc7dca46c7 bcache: Annotate switch fall-through 4a4e443835a43a79113cc237c472c0d268eb1e1c bcache: Add __printf annotation to __bch_check_keys() fd01991d5c20098c5c1ffc4dca6c821cc60a2f74 bcache: Fix indentation ca71df31661a0518ed58a1a59cf1993962153ebb bcache: fix using of loop variable in memory shrink f3641c3abd1da978ee969b0203b71b86ec1bfa93 bcache: fix error return value in memory shrink 688892b3bc05e25da94866e32210e5f503f16f69 bcache: fix incorrect sysfs output value of strip size 09a44ca2114737e0932257619c16a2b50c7807f1 bcache: use pr_info() to inform duplicated CACHE_SET_IO_DISABLE set c4dc2497d50d9c6fb16aa0d07b6a14f3b2adb1e0 bcache: fix high CPU occupancy during journal a728eacbbdd229d1d903e46261c57d5206f87a4a bcache: add journal statistic 616486ab52ab7f9739b066d958bdd20e65aefd74 bcache: fix writeback target calc on large devices 1f0ffa67349c56ea54c03ccfd1e073c990e7411e bcache: only set BCACHE_DEV_WB_RUNNING when cached device attached eb8cbb6df38f6e5124a3d5f1f8a3dbf519537c60 bcache: improve bcache_reboot() 9951379b0ca88c95876ad9778b9099e19a95d566 bcache: never writeback a discard operation [Regression Potential] The upstream fixes are all clean cherry picks from stable (most of them are small cleanups), so regression potential is minimal. The only special patch is "UBUNTU: SAUCE: bcache: fix deadlock in bcache_allocator()" that is addressing the main deadlock bug (that seems to be a mainline bug - not fixed yet). We should spend more time trying to reproduce this deadlock with a mainline kernel and post the patch to the LKML for review / feedback. However, considering that this patch seems to fix/prevent the specific deadlock problem reported in this bug (tested on the affected platform) it can be considered safe to apply it. [Original Bug Report] $ cat /proc/version_signature Ubuntu 4.15.0-29.31-generic 4.15.18 $ lsb_release -rd Description: Ubuntu Cosmic Cuttlefish (development branch) Release: 18.10 $ apt-cache policy linux-image-`uname -r` linux-image-4.15.0-29-generic: Installed: 4.15.0-29.31 Candidate: 4.15.0-29.31 Version table: *** 4.15.0-29.31 500 500 http://archive.ubuntu.com/ubuntu cosmic/main amd64
[Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
Also, I had some confusion earlier about what kernel I was testing. apt-cache policy shows the package version 4.15.0.59.61, however that's the meta package, the actual kernel is the .66 one. # dpkg --list | grep linux-image ii linux-image-4.15.0-59-generic 4.15.0-59.66 amd64Signed kernel image generic ii linux-image-virtual 4.15.0.59.61 amd64Virtual Linux kernel image # cat /proc/version Linux version 4.15.0-59-generic (buildd@lgw01-amd64-035) (gcc version 7.4.0 (Ubuntu 7.4.0-1ubuntu1~18.04.1)) #66-Ubuntu SMP Wed Aug 14 10:56:44 UTC 2019 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1784665 Title: bcache: bch_allocator_thread(): hung task timeout Status in linux package in Ubuntu: Fix Committed Status in linux source package in Xenial: Fix Committed Status in linux source package in Bionic: New Status in linux source package in Disco: Fix Committed Status in linux source package in Eoan: Fix Committed Bug description: [Impact] bcache_allocator() can call the following: bch_allocator_thread() -> bch_prio_write() -> bch_bucket_alloc() -> wait on >set->bucket_wait But the wake up event on bucket_wait is supposed to come from bch_allocator_thread() itself causing a deadlock. [Test Case] This is a simple script that can easily trigger the deadlock condition: https://launchpadlibrarian.net/381282009/bcache-basic-repro.sh A better test case has been also provided in bug 1796292 (duplicate of this bug): https://bugs.launchpad.net/curtin/+bug/1796292/+attachment/5280353/+files/curtin-nvme.sh [Fix] Fix by making the call to bch_prio_write() non-blocking, so that bch_allocator_thread() never waits on itself. Moreover, make sure to wake up the garbage collector thread when bch_prio_write() is failing to allocate buckets to increase the chance of freeing up more buckets. In addition to that it would be safe to also import other upstream bcache fixes (all clean cherry picks): 7e865eba00a3df2dc8c4746173a8ca1c1c7f042e bcache: fix potential deadlock in cached_def_free() 80265d8dfd77792e133793cef44a21323aac2908 bcache: acquire bch_register_lock later in cached_dev_free() ce4c3e19e5201424357a0c82176633b32a98d2ec bcache: Replace bch_read_string_list() by __sysfs_match_string() ecb37ce9baac653cc09e2b631393dde3df82979f bcache: Move couple of functions to sysfs.c 04cbc21137bfa4d7b8771a5b14f3d6c9b2aee671 bcache: Move couple of string arrays to sysfs.c 5f2b18ec8e1643410a2369f06888951cdedea0bf bcache: Fix a compiler warning in bcache_device_init() 20d3a518713e394efa5a899c84574b4b79ec5098 bcache: Reduce the number of sparse complaints about lock imbalances 42361469ae84c851e40cb1f94c8c9a14cdd94039 bcache: Suppress more warnings about set-but-not-used variables f0d3814090ac77de94c42b7124c37ece23629197 bcache: Remove an unused variable 47344e330eabc1515cbe6061eb337100a3ab6d37 bcache: Fix kernel-doc warnings 9dfbdec7b7fea1ff1b7b5d5d12980dbc7dca46c7 bcache: Annotate switch fall-through 4a4e443835a43a79113cc237c472c0d268eb1e1c bcache: Add __printf annotation to __bch_check_keys() fd01991d5c20098c5c1ffc4dca6c821cc60a2f74 bcache: Fix indentation ca71df31661a0518ed58a1a59cf1993962153ebb bcache: fix using of loop variable in memory shrink f3641c3abd1da978ee969b0203b71b86ec1bfa93 bcache: fix error return value in memory shrink 688892b3bc05e25da94866e32210e5f503f16f69 bcache: fix incorrect sysfs output value of strip size 09a44ca2114737e0932257619c16a2b50c7807f1 bcache: use pr_info() to inform duplicated CACHE_SET_IO_DISABLE set c4dc2497d50d9c6fb16aa0d07b6a14f3b2adb1e0 bcache: fix high CPU occupancy during journal a728eacbbdd229d1d903e46261c57d5206f87a4a bcache: add journal statistic 616486ab52ab7f9739b066d958bdd20e65aefd74 bcache: fix writeback target calc on large devices 1f0ffa67349c56ea54c03ccfd1e073c990e7411e bcache: only set BCACHE_DEV_WB_RUNNING when cached device attached eb8cbb6df38f6e5124a3d5f1f8a3dbf519537c60 bcache: improve bcache_reboot() 9951379b0ca88c95876ad9778b9099e19a95d566 bcache: never writeback a discard operation [Regression Potential] The upstream fixes are all clean cherry picks from stable (most of them are small cleanups), so regression potential is minimal. The only special patch is "UBUNTU: SAUCE: bcache: fix deadlock in bcache_allocator()" that is addressing the main deadlock bug (that seems to be a mainline bug - not fixed yet). We should spend more time trying to reproduce this deadlock with a mainline kernel and post the patch to the LKML for review / feedback. However, considering that this patch seems to fix/prevent the specific deadlock problem reported in this bug (tested on the affected platform) it can be considered
[Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
I've adjusted my bionic testing with the simpler configuration. I cannot reproduce the failure so far. I'll leave this running over night. I suspect there's something else going on baremetal that we can't reproduce in a VM. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1784665 Title: bcache: bch_allocator_thread(): hung task timeout Status in linux package in Ubuntu: Fix Committed Status in linux source package in Xenial: Fix Committed Status in linux source package in Bionic: New Status in linux source package in Disco: Fix Committed Status in linux source package in Eoan: Fix Committed Bug description: [Impact] bcache_allocator() can call the following: bch_allocator_thread() -> bch_prio_write() -> bch_bucket_alloc() -> wait on >set->bucket_wait But the wake up event on bucket_wait is supposed to come from bch_allocator_thread() itself causing a deadlock. [Test Case] This is a simple script that can easily trigger the deadlock condition: https://launchpadlibrarian.net/381282009/bcache-basic-repro.sh A better test case has been also provided in bug 1796292 (duplicate of this bug): https://bugs.launchpad.net/curtin/+bug/1796292/+attachment/5280353/+files/curtin-nvme.sh [Fix] Fix by making the call to bch_prio_write() non-blocking, so that bch_allocator_thread() never waits on itself. Moreover, make sure to wake up the garbage collector thread when bch_prio_write() is failing to allocate buckets to increase the chance of freeing up more buckets. In addition to that it would be safe to also import other upstream bcache fixes (all clean cherry picks): 7e865eba00a3df2dc8c4746173a8ca1c1c7f042e bcache: fix potential deadlock in cached_def_free() 80265d8dfd77792e133793cef44a21323aac2908 bcache: acquire bch_register_lock later in cached_dev_free() ce4c3e19e5201424357a0c82176633b32a98d2ec bcache: Replace bch_read_string_list() by __sysfs_match_string() ecb37ce9baac653cc09e2b631393dde3df82979f bcache: Move couple of functions to sysfs.c 04cbc21137bfa4d7b8771a5b14f3d6c9b2aee671 bcache: Move couple of string arrays to sysfs.c 5f2b18ec8e1643410a2369f06888951cdedea0bf bcache: Fix a compiler warning in bcache_device_init() 20d3a518713e394efa5a899c84574b4b79ec5098 bcache: Reduce the number of sparse complaints about lock imbalances 42361469ae84c851e40cb1f94c8c9a14cdd94039 bcache: Suppress more warnings about set-but-not-used variables f0d3814090ac77de94c42b7124c37ece23629197 bcache: Remove an unused variable 47344e330eabc1515cbe6061eb337100a3ab6d37 bcache: Fix kernel-doc warnings 9dfbdec7b7fea1ff1b7b5d5d12980dbc7dca46c7 bcache: Annotate switch fall-through 4a4e443835a43a79113cc237c472c0d268eb1e1c bcache: Add __printf annotation to __bch_check_keys() fd01991d5c20098c5c1ffc4dca6c821cc60a2f74 bcache: Fix indentation ca71df31661a0518ed58a1a59cf1993962153ebb bcache: fix using of loop variable in memory shrink f3641c3abd1da978ee969b0203b71b86ec1bfa93 bcache: fix error return value in memory shrink 688892b3bc05e25da94866e32210e5f503f16f69 bcache: fix incorrect sysfs output value of strip size 09a44ca2114737e0932257619c16a2b50c7807f1 bcache: use pr_info() to inform duplicated CACHE_SET_IO_DISABLE set c4dc2497d50d9c6fb16aa0d07b6a14f3b2adb1e0 bcache: fix high CPU occupancy during journal a728eacbbdd229d1d903e46261c57d5206f87a4a bcache: add journal statistic 616486ab52ab7f9739b066d958bdd20e65aefd74 bcache: fix writeback target calc on large devices 1f0ffa67349c56ea54c03ccfd1e073c990e7411e bcache: only set BCACHE_DEV_WB_RUNNING when cached device attached eb8cbb6df38f6e5124a3d5f1f8a3dbf519537c60 bcache: improve bcache_reboot() 9951379b0ca88c95876ad9778b9099e19a95d566 bcache: never writeback a discard operation [Regression Potential] The upstream fixes are all clean cherry picks from stable (most of them are small cleanups), so regression potential is minimal. The only special patch is "UBUNTU: SAUCE: bcache: fix deadlock in bcache_allocator()" that is addressing the main deadlock bug (that seems to be a mainline bug - not fixed yet). We should spend more time trying to reproduce this deadlock with a mainline kernel and post the patch to the LKML for review / feedback. However, considering that this patch seems to fix/prevent the specific deadlock problem reported in this bug (tested on the affected platform) it can be considered safe to apply it. [Original Bug Report] $ cat /proc/version_signature Ubuntu 4.15.0-29.31-generic 4.15.18 $ lsb_release -rd Description: Ubuntu Cosmic Cuttlefish (development branch) Release: 18.10 $ apt-cache policy linux-image-`uname -r` linux-image-4.15.0-29-generic: Installed: 4.15.0-29.31 Candidate: 4.15.0-29.31 Version table: *** 4.15.0-29.31 500 500
[Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
** Changed in: linux (Ubuntu Bionic) Status: Fix Committed => New -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1784665 Title: bcache: bch_allocator_thread(): hung task timeout Status in linux package in Ubuntu: Fix Committed Status in linux source package in Xenial: Fix Committed Status in linux source package in Bionic: New Status in linux source package in Disco: Fix Committed Status in linux source package in Eoan: Fix Committed Bug description: [Impact] bcache_allocator() can call the following: bch_allocator_thread() -> bch_prio_write() -> bch_bucket_alloc() -> wait on >set->bucket_wait But the wake up event on bucket_wait is supposed to come from bch_allocator_thread() itself causing a deadlock. [Test Case] This is a simple script that can easily trigger the deadlock condition: https://launchpadlibrarian.net/381282009/bcache-basic-repro.sh A better test case has been also provided in bug 1796292 (duplicate of this bug): https://bugs.launchpad.net/curtin/+bug/1796292/+attachment/5280353/+files/curtin-nvme.sh [Fix] Fix by making the call to bch_prio_write() non-blocking, so that bch_allocator_thread() never waits on itself. Moreover, make sure to wake up the garbage collector thread when bch_prio_write() is failing to allocate buckets to increase the chance of freeing up more buckets. In addition to that it would be safe to also import other upstream bcache fixes (all clean cherry picks): 7e865eba00a3df2dc8c4746173a8ca1c1c7f042e bcache: fix potential deadlock in cached_def_free() 80265d8dfd77792e133793cef44a21323aac2908 bcache: acquire bch_register_lock later in cached_dev_free() ce4c3e19e5201424357a0c82176633b32a98d2ec bcache: Replace bch_read_string_list() by __sysfs_match_string() ecb37ce9baac653cc09e2b631393dde3df82979f bcache: Move couple of functions to sysfs.c 04cbc21137bfa4d7b8771a5b14f3d6c9b2aee671 bcache: Move couple of string arrays to sysfs.c 5f2b18ec8e1643410a2369f06888951cdedea0bf bcache: Fix a compiler warning in bcache_device_init() 20d3a518713e394efa5a899c84574b4b79ec5098 bcache: Reduce the number of sparse complaints about lock imbalances 42361469ae84c851e40cb1f94c8c9a14cdd94039 bcache: Suppress more warnings about set-but-not-used variables f0d3814090ac77de94c42b7124c37ece23629197 bcache: Remove an unused variable 47344e330eabc1515cbe6061eb337100a3ab6d37 bcache: Fix kernel-doc warnings 9dfbdec7b7fea1ff1b7b5d5d12980dbc7dca46c7 bcache: Annotate switch fall-through 4a4e443835a43a79113cc237c472c0d268eb1e1c bcache: Add __printf annotation to __bch_check_keys() fd01991d5c20098c5c1ffc4dca6c821cc60a2f74 bcache: Fix indentation ca71df31661a0518ed58a1a59cf1993962153ebb bcache: fix using of loop variable in memory shrink f3641c3abd1da978ee969b0203b71b86ec1bfa93 bcache: fix error return value in memory shrink 688892b3bc05e25da94866e32210e5f503f16f69 bcache: fix incorrect sysfs output value of strip size 09a44ca2114737e0932257619c16a2b50c7807f1 bcache: use pr_info() to inform duplicated CACHE_SET_IO_DISABLE set c4dc2497d50d9c6fb16aa0d07b6a14f3b2adb1e0 bcache: fix high CPU occupancy during journal a728eacbbdd229d1d903e46261c57d5206f87a4a bcache: add journal statistic 616486ab52ab7f9739b066d958bdd20e65aefd74 bcache: fix writeback target calc on large devices 1f0ffa67349c56ea54c03ccfd1e073c990e7411e bcache: only set BCACHE_DEV_WB_RUNNING when cached device attached eb8cbb6df38f6e5124a3d5f1f8a3dbf519537c60 bcache: improve bcache_reboot() 9951379b0ca88c95876ad9778b9099e19a95d566 bcache: never writeback a discard operation [Regression Potential] The upstream fixes are all clean cherry picks from stable (most of them are small cleanups), so regression potential is minimal. The only special patch is "UBUNTU: SAUCE: bcache: fix deadlock in bcache_allocator()" that is addressing the main deadlock bug (that seems to be a mainline bug - not fixed yet). We should spend more time trying to reproduce this deadlock with a mainline kernel and post the patch to the LKML for review / feedback. However, considering that this patch seems to fix/prevent the specific deadlock problem reported in this bug (tested on the affected platform) it can be considered safe to apply it. [Original Bug Report] $ cat /proc/version_signature Ubuntu 4.15.0-29.31-generic 4.15.18 $ lsb_release -rd Description: Ubuntu Cosmic Cuttlefish (development branch) Release: 18.10 $ apt-cache policy linux-image-`uname -r` linux-image-4.15.0-29-generic: Installed: 4.15.0-29.31 Candidate: 4.15.0-29.31 Version table: *** 4.15.0-29.31 500 500 http://archive.ubuntu.com/ubuntu cosmic/main amd64 Packages 100 /var/lib/dpkg/status 3) mkfs.ext4 /dev/bcache0 returns successful creating an ext4
[Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
** Attachment added: "spinda.maas-curtin_config.txt" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1784665/+attachment/5284072/+files/spinda.maas-curtin_config.txt -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1784665 Title: bcache: bch_allocator_thread(): hung task timeout Status in linux package in Ubuntu: Fix Committed Status in linux source package in Xenial: Fix Committed Status in linux source package in Bionic: Fix Committed Status in linux source package in Disco: Fix Committed Status in linux source package in Eoan: Fix Committed Bug description: [Impact] bcache_allocator() can call the following: bch_allocator_thread() -> bch_prio_write() -> bch_bucket_alloc() -> wait on >set->bucket_wait But the wake up event on bucket_wait is supposed to come from bch_allocator_thread() itself causing a deadlock. [Test Case] This is a simple script that can easily trigger the deadlock condition: https://launchpadlibrarian.net/381282009/bcache-basic-repro.sh A better test case has been also provided in bug 1796292 (duplicate of this bug): https://bugs.launchpad.net/curtin/+bug/1796292/+attachment/5280353/+files/curtin-nvme.sh [Fix] Fix by making the call to bch_prio_write() non-blocking, so that bch_allocator_thread() never waits on itself. Moreover, make sure to wake up the garbage collector thread when bch_prio_write() is failing to allocate buckets to increase the chance of freeing up more buckets. In addition to that it would be safe to also import other upstream bcache fixes (all clean cherry picks): 7e865eba00a3df2dc8c4746173a8ca1c1c7f042e bcache: fix potential deadlock in cached_def_free() 80265d8dfd77792e133793cef44a21323aac2908 bcache: acquire bch_register_lock later in cached_dev_free() ce4c3e19e5201424357a0c82176633b32a98d2ec bcache: Replace bch_read_string_list() by __sysfs_match_string() ecb37ce9baac653cc09e2b631393dde3df82979f bcache: Move couple of functions to sysfs.c 04cbc21137bfa4d7b8771a5b14f3d6c9b2aee671 bcache: Move couple of string arrays to sysfs.c 5f2b18ec8e1643410a2369f06888951cdedea0bf bcache: Fix a compiler warning in bcache_device_init() 20d3a518713e394efa5a899c84574b4b79ec5098 bcache: Reduce the number of sparse complaints about lock imbalances 42361469ae84c851e40cb1f94c8c9a14cdd94039 bcache: Suppress more warnings about set-but-not-used variables f0d3814090ac77de94c42b7124c37ece23629197 bcache: Remove an unused variable 47344e330eabc1515cbe6061eb337100a3ab6d37 bcache: Fix kernel-doc warnings 9dfbdec7b7fea1ff1b7b5d5d12980dbc7dca46c7 bcache: Annotate switch fall-through 4a4e443835a43a79113cc237c472c0d268eb1e1c bcache: Add __printf annotation to __bch_check_keys() fd01991d5c20098c5c1ffc4dca6c821cc60a2f74 bcache: Fix indentation ca71df31661a0518ed58a1a59cf1993962153ebb bcache: fix using of loop variable in memory shrink f3641c3abd1da978ee969b0203b71b86ec1bfa93 bcache: fix error return value in memory shrink 688892b3bc05e25da94866e32210e5f503f16f69 bcache: fix incorrect sysfs output value of strip size 09a44ca2114737e0932257619c16a2b50c7807f1 bcache: use pr_info() to inform duplicated CACHE_SET_IO_DISABLE set c4dc2497d50d9c6fb16aa0d07b6a14f3b2adb1e0 bcache: fix high CPU occupancy during journal a728eacbbdd229d1d903e46261c57d5206f87a4a bcache: add journal statistic 616486ab52ab7f9739b066d958bdd20e65aefd74 bcache: fix writeback target calc on large devices 1f0ffa67349c56ea54c03ccfd1e073c990e7411e bcache: only set BCACHE_DEV_WB_RUNNING when cached device attached eb8cbb6df38f6e5124a3d5f1f8a3dbf519537c60 bcache: improve bcache_reboot() 9951379b0ca88c95876ad9778b9099e19a95d566 bcache: never writeback a discard operation [Regression Potential] The upstream fixes are all clean cherry picks from stable (most of them are small cleanups), so regression potential is minimal. The only special patch is "UBUNTU: SAUCE: bcache: fix deadlock in bcache_allocator()" that is addressing the main deadlock bug (that seems to be a mainline bug - not fixed yet). We should spend more time trying to reproduce this deadlock with a mainline kernel and post the patch to the LKML for review / feedback. However, considering that this patch seems to fix/prevent the specific deadlock problem reported in this bug (tested on the affected platform) it can be considered safe to apply it. [Original Bug Report] $ cat /proc/version_signature Ubuntu 4.15.0-29.31-generic 4.15.18 $ lsb_release -rd Description: Ubuntu Cosmic Cuttlefish (development branch) Release: 18.10 $ apt-cache policy linux-image-`uname -r` linux-image-4.15.0-29-generic: Installed: 4.15.0-29.31 Candidate: 4.15.0-29.31 Version table: *** 4.15.0-29.31 500 500 http://archive.ubuntu.com/ubuntu cosmic/main amd64
[Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
We're still seeing a bcache timeout failure during curtin install 2019-08-22T10:16:40+00:00 spinda cloud-init[1604]: finish: cmd-install/stage-partitioning/builtin/cmd-block-meta/clear-holders: FAIL: removing previous storage devices 2019-08-22T10:16:40+00:00 spinda cloud-init[1604]: TIMED BLOCK_META: 1203.679 I attached the rsyslog from a unit that failed. Linux version 4.15.0-59-generic (buildd@lgw01-amd64-035) (gcc version 7.4.0 (Ubuntu 7.4.0-1ubuntu1~18.04.1)) #66-Ubuntu SMP Wed Aug 14 10:56:44 UTC 2019 (Ubuntu 4.15.0-59.66-generic 4.15.18) ** Attachment added: "messages" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1784665/+attachment/5284071/+files/messages ** Tags removed: verification-done-bionic ** Tags added: verification-failed-bionic -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1784665 Title: bcache: bch_allocator_thread(): hung task timeout Status in linux package in Ubuntu: Fix Committed Status in linux source package in Xenial: Fix Committed Status in linux source package in Bionic: Fix Committed Status in linux source package in Disco: Fix Committed Status in linux source package in Eoan: Fix Committed Bug description: [Impact] bcache_allocator() can call the following: bch_allocator_thread() -> bch_prio_write() -> bch_bucket_alloc() -> wait on >set->bucket_wait But the wake up event on bucket_wait is supposed to come from bch_allocator_thread() itself causing a deadlock. [Test Case] This is a simple script that can easily trigger the deadlock condition: https://launchpadlibrarian.net/381282009/bcache-basic-repro.sh A better test case has been also provided in bug 1796292 (duplicate of this bug): https://bugs.launchpad.net/curtin/+bug/1796292/+attachment/5280353/+files/curtin-nvme.sh [Fix] Fix by making the call to bch_prio_write() non-blocking, so that bch_allocator_thread() never waits on itself. Moreover, make sure to wake up the garbage collector thread when bch_prio_write() is failing to allocate buckets to increase the chance of freeing up more buckets. In addition to that it would be safe to also import other upstream bcache fixes (all clean cherry picks): 7e865eba00a3df2dc8c4746173a8ca1c1c7f042e bcache: fix potential deadlock in cached_def_free() 80265d8dfd77792e133793cef44a21323aac2908 bcache: acquire bch_register_lock later in cached_dev_free() ce4c3e19e5201424357a0c82176633b32a98d2ec bcache: Replace bch_read_string_list() by __sysfs_match_string() ecb37ce9baac653cc09e2b631393dde3df82979f bcache: Move couple of functions to sysfs.c 04cbc21137bfa4d7b8771a5b14f3d6c9b2aee671 bcache: Move couple of string arrays to sysfs.c 5f2b18ec8e1643410a2369f06888951cdedea0bf bcache: Fix a compiler warning in bcache_device_init() 20d3a518713e394efa5a899c84574b4b79ec5098 bcache: Reduce the number of sparse complaints about lock imbalances 42361469ae84c851e40cb1f94c8c9a14cdd94039 bcache: Suppress more warnings about set-but-not-used variables f0d3814090ac77de94c42b7124c37ece23629197 bcache: Remove an unused variable 47344e330eabc1515cbe6061eb337100a3ab6d37 bcache: Fix kernel-doc warnings 9dfbdec7b7fea1ff1b7b5d5d12980dbc7dca46c7 bcache: Annotate switch fall-through 4a4e443835a43a79113cc237c472c0d268eb1e1c bcache: Add __printf annotation to __bch_check_keys() fd01991d5c20098c5c1ffc4dca6c821cc60a2f74 bcache: Fix indentation ca71df31661a0518ed58a1a59cf1993962153ebb bcache: fix using of loop variable in memory shrink f3641c3abd1da978ee969b0203b71b86ec1bfa93 bcache: fix error return value in memory shrink 688892b3bc05e25da94866e32210e5f503f16f69 bcache: fix incorrect sysfs output value of strip size 09a44ca2114737e0932257619c16a2b50c7807f1 bcache: use pr_info() to inform duplicated CACHE_SET_IO_DISABLE set c4dc2497d50d9c6fb16aa0d07b6a14f3b2adb1e0 bcache: fix high CPU occupancy during journal a728eacbbdd229d1d903e46261c57d5206f87a4a bcache: add journal statistic 616486ab52ab7f9739b066d958bdd20e65aefd74 bcache: fix writeback target calc on large devices 1f0ffa67349c56ea54c03ccfd1e073c990e7411e bcache: only set BCACHE_DEV_WB_RUNNING when cached device attached eb8cbb6df38f6e5124a3d5f1f8a3dbf519537c60 bcache: improve bcache_reboot() 9951379b0ca88c95876ad9778b9099e19a95d566 bcache: never writeback a discard operation [Regression Potential] The upstream fixes are all clean cherry picks from stable (most of them are small cleanups), so regression potential is minimal. The only special patch is "UBUNTU: SAUCE: bcache: fix deadlock in bcache_allocator()" that is addressing the main deadlock bug (that seems to be a mainline bug - not fixed yet). We should spend more time trying to reproduce this deadlock with a mainline kernel and post the patch to the LKML for review / feedback. However,
[Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
I've verified that disco-proposed linux-virtual kernel succeeds our test-case (curtin-nvme). I've 50 installs with no issue. root@ubuntu:~# apt-cache policy linux-virtual linux-virtual: Installed: 5.0.0.26.27 Candidate: 5.0.0.26.27 Version table: *** 5.0.0.26.27 500 500 http://archive.ubuntu.com/ubuntu disco-proposed/main amd64 Packages 100 /var/lib/dpkg/status 5.0.0.25.26 500 500 http://archive.ubuntu.com/ubuntu disco-updates/main amd64 Packages 500 http://security.ubuntu.com/ubuntu disco-security/main amd64 Packages 5.0.0.13.14 500 500 http://archive.ubuntu.com/ubuntu disco/main amd64 Packages root@ubuntu:~# uname -r 5.0.0-26-generic root@ubuntu:~# lsb_release -rd Description:Ubuntu 19.04 Release:19.04 ** Tags removed: verification-needed-disco ** Tags added: verification-done-disco -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1784665 Title: bcache: bch_allocator_thread(): hung task timeout Status in linux package in Ubuntu: Fix Committed Status in linux source package in Xenial: Fix Committed Status in linux source package in Bionic: Fix Committed Status in linux source package in Disco: Fix Committed Status in linux source package in Eoan: Fix Committed Bug description: [Impact] bcache_allocator() can call the following: bch_allocator_thread() -> bch_prio_write() -> bch_bucket_alloc() -> wait on >set->bucket_wait But the wake up event on bucket_wait is supposed to come from bch_allocator_thread() itself causing a deadlock. [Test Case] This is a simple script that can easily trigger the deadlock condition: https://launchpadlibrarian.net/381282009/bcache-basic-repro.sh A better test case has been also provided in bug 1796292 (duplicate of this bug): https://bugs.launchpad.net/curtin/+bug/1796292/+attachment/5280353/+files/curtin-nvme.sh [Fix] Fix by making the call to bch_prio_write() non-blocking, so that bch_allocator_thread() never waits on itself. Moreover, make sure to wake up the garbage collector thread when bch_prio_write() is failing to allocate buckets to increase the chance of freeing up more buckets. In addition to that it would be safe to also import other upstream bcache fixes (all clean cherry picks): 7e865eba00a3df2dc8c4746173a8ca1c1c7f042e bcache: fix potential deadlock in cached_def_free() 80265d8dfd77792e133793cef44a21323aac2908 bcache: acquire bch_register_lock later in cached_dev_free() ce4c3e19e5201424357a0c82176633b32a98d2ec bcache: Replace bch_read_string_list() by __sysfs_match_string() ecb37ce9baac653cc09e2b631393dde3df82979f bcache: Move couple of functions to sysfs.c 04cbc21137bfa4d7b8771a5b14f3d6c9b2aee671 bcache: Move couple of string arrays to sysfs.c 5f2b18ec8e1643410a2369f06888951cdedea0bf bcache: Fix a compiler warning in bcache_device_init() 20d3a518713e394efa5a899c84574b4b79ec5098 bcache: Reduce the number of sparse complaints about lock imbalances 42361469ae84c851e40cb1f94c8c9a14cdd94039 bcache: Suppress more warnings about set-but-not-used variables f0d3814090ac77de94c42b7124c37ece23629197 bcache: Remove an unused variable 47344e330eabc1515cbe6061eb337100a3ab6d37 bcache: Fix kernel-doc warnings 9dfbdec7b7fea1ff1b7b5d5d12980dbc7dca46c7 bcache: Annotate switch fall-through 4a4e443835a43a79113cc237c472c0d268eb1e1c bcache: Add __printf annotation to __bch_check_keys() fd01991d5c20098c5c1ffc4dca6c821cc60a2f74 bcache: Fix indentation ca71df31661a0518ed58a1a59cf1993962153ebb bcache: fix using of loop variable in memory shrink f3641c3abd1da978ee969b0203b71b86ec1bfa93 bcache: fix error return value in memory shrink 688892b3bc05e25da94866e32210e5f503f16f69 bcache: fix incorrect sysfs output value of strip size 09a44ca2114737e0932257619c16a2b50c7807f1 bcache: use pr_info() to inform duplicated CACHE_SET_IO_DISABLE set c4dc2497d50d9c6fb16aa0d07b6a14f3b2adb1e0 bcache: fix high CPU occupancy during journal a728eacbbdd229d1d903e46261c57d5206f87a4a bcache: add journal statistic 616486ab52ab7f9739b066d958bdd20e65aefd74 bcache: fix writeback target calc on large devices 1f0ffa67349c56ea54c03ccfd1e073c990e7411e bcache: only set BCACHE_DEV_WB_RUNNING when cached device attached eb8cbb6df38f6e5124a3d5f1f8a3dbf519537c60 bcache: improve bcache_reboot() 9951379b0ca88c95876ad9778b9099e19a95d566 bcache: never writeback a discard operation [Regression Potential] The upstream fixes are all clean cherry picks from stable (most of them are small cleanups), so regression potential is minimal. The only special patch is "UBUNTU: SAUCE: bcache: fix deadlock in bcache_allocator()" that is addressing the main deadlock bug (that seems to be a mainline bug - not fixed yet). We should spend more time trying to reproduce this deadlock with a mainline
[Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
I've also verified Solutions QA tests succeed with the proposed Bionic kernel. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1784665 Title: bcache: bch_allocator_thread(): hung task timeout Status in linux package in Ubuntu: Fix Committed Status in linux source package in Xenial: Fix Committed Status in linux source package in Bionic: Fix Committed Status in linux source package in Disco: Fix Committed Status in linux source package in Eoan: Fix Committed Bug description: [Impact] bcache_allocator() can call the following: bch_allocator_thread() -> bch_prio_write() -> bch_bucket_alloc() -> wait on >set->bucket_wait But the wake up event on bucket_wait is supposed to come from bch_allocator_thread() itself causing a deadlock. [Test Case] This is a simple script that can easily trigger the deadlock condition: https://launchpadlibrarian.net/381282009/bcache-basic-repro.sh A better test case has been also provided in bug 1796292 (duplicate of this bug): https://bugs.launchpad.net/curtin/+bug/1796292/+attachment/5280353/+files/curtin-nvme.sh [Fix] Fix by making the call to bch_prio_write() non-blocking, so that bch_allocator_thread() never waits on itself. Moreover, make sure to wake up the garbage collector thread when bch_prio_write() is failing to allocate buckets to increase the chance of freeing up more buckets. In addition to that it would be safe to also import other upstream bcache fixes (all clean cherry picks): 7e865eba00a3df2dc8c4746173a8ca1c1c7f042e bcache: fix potential deadlock in cached_def_free() 80265d8dfd77792e133793cef44a21323aac2908 bcache: acquire bch_register_lock later in cached_dev_free() ce4c3e19e5201424357a0c82176633b32a98d2ec bcache: Replace bch_read_string_list() by __sysfs_match_string() ecb37ce9baac653cc09e2b631393dde3df82979f bcache: Move couple of functions to sysfs.c 04cbc21137bfa4d7b8771a5b14f3d6c9b2aee671 bcache: Move couple of string arrays to sysfs.c 5f2b18ec8e1643410a2369f06888951cdedea0bf bcache: Fix a compiler warning in bcache_device_init() 20d3a518713e394efa5a899c84574b4b79ec5098 bcache: Reduce the number of sparse complaints about lock imbalances 42361469ae84c851e40cb1f94c8c9a14cdd94039 bcache: Suppress more warnings about set-but-not-used variables f0d3814090ac77de94c42b7124c37ece23629197 bcache: Remove an unused variable 47344e330eabc1515cbe6061eb337100a3ab6d37 bcache: Fix kernel-doc warnings 9dfbdec7b7fea1ff1b7b5d5d12980dbc7dca46c7 bcache: Annotate switch fall-through 4a4e443835a43a79113cc237c472c0d268eb1e1c bcache: Add __printf annotation to __bch_check_keys() fd01991d5c20098c5c1ffc4dca6c821cc60a2f74 bcache: Fix indentation ca71df31661a0518ed58a1a59cf1993962153ebb bcache: fix using of loop variable in memory shrink f3641c3abd1da978ee969b0203b71b86ec1bfa93 bcache: fix error return value in memory shrink 688892b3bc05e25da94866e32210e5f503f16f69 bcache: fix incorrect sysfs output value of strip size 09a44ca2114737e0932257619c16a2b50c7807f1 bcache: use pr_info() to inform duplicated CACHE_SET_IO_DISABLE set c4dc2497d50d9c6fb16aa0d07b6a14f3b2adb1e0 bcache: fix high CPU occupancy during journal a728eacbbdd229d1d903e46261c57d5206f87a4a bcache: add journal statistic 616486ab52ab7f9739b066d958bdd20e65aefd74 bcache: fix writeback target calc on large devices 1f0ffa67349c56ea54c03ccfd1e073c990e7411e bcache: only set BCACHE_DEV_WB_RUNNING when cached device attached eb8cbb6df38f6e5124a3d5f1f8a3dbf519537c60 bcache: improve bcache_reboot() 9951379b0ca88c95876ad9778b9099e19a95d566 bcache: never writeback a discard operation [Regression Potential] The upstream fixes are all clean cherry picks from stable (most of them are small cleanups), so regression potential is minimal. The only special patch is "UBUNTU: SAUCE: bcache: fix deadlock in bcache_allocator()" that is addressing the main deadlock bug (that seems to be a mainline bug - not fixed yet). We should spend more time trying to reproduce this deadlock with a mainline kernel and post the patch to the LKML for review / feedback. However, considering that this patch seems to fix/prevent the specific deadlock problem reported in this bug (tested on the affected platform) it can be considered safe to apply it. [Original Bug Report] $ cat /proc/version_signature Ubuntu 4.15.0-29.31-generic 4.15.18 $ lsb_release -rd Description: Ubuntu Cosmic Cuttlefish (development branch) Release: 18.10 $ apt-cache policy linux-image-`uname -r` linux-image-4.15.0-29-generic: Installed: 4.15.0-29.31 Candidate: 4.15.0-29.31 Version table: *** 4.15.0-29.31 500 500 http://archive.ubuntu.com/ubuntu cosmic/main amd64 Packages 100 /var/lib/dpkg/status 3) mkfs.ext4 /dev/bcache0 returns successful
[Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
I've verified that bionic-proposed linux-virtual kernel succeeds our test-case (curtin-nvme). I've 50 installs with no issue. ubuntu@ubuntu:~$ apt-cache policy linux-virtual linux-virtual: Installed: (none) Candidate: 4.15.0.59.61 Version table: 4.15.0.59.61 500 500 http://archive.ubuntu.com/ubuntu bionic-proposed/main amd64 Packages 4.15.0.58.60 500 500 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages 500 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages 4.15.0.20.23 500 500 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages ubuntu@ubuntu:~$ lsb_release -rd Description:Ubuntu 18.04.2 LTS Release:18.04 ubuntu@ubuntu:~$ uname -r 4.15.0-59-generic ** Tags removed: verification-needed-bionic ** Tags added: verification-done-bionic -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1784665 Title: bcache: bch_allocator_thread(): hung task timeout Status in linux package in Ubuntu: Fix Committed Status in linux source package in Xenial: Fix Committed Status in linux source package in Bionic: Fix Committed Status in linux source package in Disco: Fix Committed Status in linux source package in Eoan: Fix Committed Bug description: [Impact] bcache_allocator() can call the following: bch_allocator_thread() -> bch_prio_write() -> bch_bucket_alloc() -> wait on >set->bucket_wait But the wake up event on bucket_wait is supposed to come from bch_allocator_thread() itself causing a deadlock. [Test Case] This is a simple script that can easily trigger the deadlock condition: https://launchpadlibrarian.net/381282009/bcache-basic-repro.sh A better test case has been also provided in bug 1796292 (duplicate of this bug): https://bugs.launchpad.net/curtin/+bug/1796292/+attachment/5280353/+files/curtin-nvme.sh [Fix] Fix by making the call to bch_prio_write() non-blocking, so that bch_allocator_thread() never waits on itself. Moreover, make sure to wake up the garbage collector thread when bch_prio_write() is failing to allocate buckets to increase the chance of freeing up more buckets. In addition to that it would be safe to also import other upstream bcache fixes (all clean cherry picks): 7e865eba00a3df2dc8c4746173a8ca1c1c7f042e bcache: fix potential deadlock in cached_def_free() 80265d8dfd77792e133793cef44a21323aac2908 bcache: acquire bch_register_lock later in cached_dev_free() ce4c3e19e5201424357a0c82176633b32a98d2ec bcache: Replace bch_read_string_list() by __sysfs_match_string() ecb37ce9baac653cc09e2b631393dde3df82979f bcache: Move couple of functions to sysfs.c 04cbc21137bfa4d7b8771a5b14f3d6c9b2aee671 bcache: Move couple of string arrays to sysfs.c 5f2b18ec8e1643410a2369f06888951cdedea0bf bcache: Fix a compiler warning in bcache_device_init() 20d3a518713e394efa5a899c84574b4b79ec5098 bcache: Reduce the number of sparse complaints about lock imbalances 42361469ae84c851e40cb1f94c8c9a14cdd94039 bcache: Suppress more warnings about set-but-not-used variables f0d3814090ac77de94c42b7124c37ece23629197 bcache: Remove an unused variable 47344e330eabc1515cbe6061eb337100a3ab6d37 bcache: Fix kernel-doc warnings 9dfbdec7b7fea1ff1b7b5d5d12980dbc7dca46c7 bcache: Annotate switch fall-through 4a4e443835a43a79113cc237c472c0d268eb1e1c bcache: Add __printf annotation to __bch_check_keys() fd01991d5c20098c5c1ffc4dca6c821cc60a2f74 bcache: Fix indentation ca71df31661a0518ed58a1a59cf1993962153ebb bcache: fix using of loop variable in memory shrink f3641c3abd1da978ee969b0203b71b86ec1bfa93 bcache: fix error return value in memory shrink 688892b3bc05e25da94866e32210e5f503f16f69 bcache: fix incorrect sysfs output value of strip size 09a44ca2114737e0932257619c16a2b50c7807f1 bcache: use pr_info() to inform duplicated CACHE_SET_IO_DISABLE set c4dc2497d50d9c6fb16aa0d07b6a14f3b2adb1e0 bcache: fix high CPU occupancy during journal a728eacbbdd229d1d903e46261c57d5206f87a4a bcache: add journal statistic 616486ab52ab7f9739b066d958bdd20e65aefd74 bcache: fix writeback target calc on large devices 1f0ffa67349c56ea54c03ccfd1e073c990e7411e bcache: only set BCACHE_DEV_WB_RUNNING when cached device attached eb8cbb6df38f6e5124a3d5f1f8a3dbf519537c60 bcache: improve bcache_reboot() 9951379b0ca88c95876ad9778b9099e19a95d566 bcache: never writeback a discard operation [Regression Potential] The upstream fixes are all clean cherry picks from stable (most of them are small cleanups), so regression potential is minimal. The only special patch is "UBUNTU: SAUCE: bcache: fix deadlock in bcache_allocator()" that is addressing the main deadlock bug (that seems to be a mainline bug - not fixed yet). We should spend more time trying to reproduce this deadlock with a mainline kernel and post
[Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed- xenial' to 'verification-done-xenial'. If the problem still exists, change the tag 'verification-needed-xenial' to 'verification-failed- xenial'. If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you! ** Tags added: verification-needed-xenial -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1784665 Title: bcache: bch_allocator_thread(): hung task timeout Status in linux package in Ubuntu: Fix Committed Status in linux source package in Xenial: Fix Committed Status in linux source package in Bionic: Fix Committed Status in linux source package in Disco: Fix Committed Status in linux source package in Eoan: Fix Committed Bug description: [Impact] bcache_allocator() can call the following: bch_allocator_thread() -> bch_prio_write() -> bch_bucket_alloc() -> wait on >set->bucket_wait But the wake up event on bucket_wait is supposed to come from bch_allocator_thread() itself causing a deadlock. [Test Case] This is a simple script that can easily trigger the deadlock condition: https://launchpadlibrarian.net/381282009/bcache-basic-repro.sh A better test case has been also provided in bug 1796292 (duplicate of this bug): https://bugs.launchpad.net/curtin/+bug/1796292/+attachment/5280353/+files/curtin-nvme.sh [Fix] Fix by making the call to bch_prio_write() non-blocking, so that bch_allocator_thread() never waits on itself. Moreover, make sure to wake up the garbage collector thread when bch_prio_write() is failing to allocate buckets to increase the chance of freeing up more buckets. In addition to that it would be safe to also import other upstream bcache fixes (all clean cherry picks): 7e865eba00a3df2dc8c4746173a8ca1c1c7f042e bcache: fix potential deadlock in cached_def_free() 80265d8dfd77792e133793cef44a21323aac2908 bcache: acquire bch_register_lock later in cached_dev_free() ce4c3e19e5201424357a0c82176633b32a98d2ec bcache: Replace bch_read_string_list() by __sysfs_match_string() ecb37ce9baac653cc09e2b631393dde3df82979f bcache: Move couple of functions to sysfs.c 04cbc21137bfa4d7b8771a5b14f3d6c9b2aee671 bcache: Move couple of string arrays to sysfs.c 5f2b18ec8e1643410a2369f06888951cdedea0bf bcache: Fix a compiler warning in bcache_device_init() 20d3a518713e394efa5a899c84574b4b79ec5098 bcache: Reduce the number of sparse complaints about lock imbalances 42361469ae84c851e40cb1f94c8c9a14cdd94039 bcache: Suppress more warnings about set-but-not-used variables f0d3814090ac77de94c42b7124c37ece23629197 bcache: Remove an unused variable 47344e330eabc1515cbe6061eb337100a3ab6d37 bcache: Fix kernel-doc warnings 9dfbdec7b7fea1ff1b7b5d5d12980dbc7dca46c7 bcache: Annotate switch fall-through 4a4e443835a43a79113cc237c472c0d268eb1e1c bcache: Add __printf annotation to __bch_check_keys() fd01991d5c20098c5c1ffc4dca6c821cc60a2f74 bcache: Fix indentation ca71df31661a0518ed58a1a59cf1993962153ebb bcache: fix using of loop variable in memory shrink f3641c3abd1da978ee969b0203b71b86ec1bfa93 bcache: fix error return value in memory shrink 688892b3bc05e25da94866e32210e5f503f16f69 bcache: fix incorrect sysfs output value of strip size 09a44ca2114737e0932257619c16a2b50c7807f1 bcache: use pr_info() to inform duplicated CACHE_SET_IO_DISABLE set c4dc2497d50d9c6fb16aa0d07b6a14f3b2adb1e0 bcache: fix high CPU occupancy during journal a728eacbbdd229d1d903e46261c57d5206f87a4a bcache: add journal statistic 616486ab52ab7f9739b066d958bdd20e65aefd74 bcache: fix writeback target calc on large devices 1f0ffa67349c56ea54c03ccfd1e073c990e7411e bcache: only set BCACHE_DEV_WB_RUNNING when cached device attached eb8cbb6df38f6e5124a3d5f1f8a3dbf519537c60 bcache: improve bcache_reboot() 9951379b0ca88c95876ad9778b9099e19a95d566 bcache: never writeback a discard operation [Regression Potential] The upstream fixes are all clean cherry picks from stable (most of them are small cleanups), so regression potential is minimal. The only special patch is "UBUNTU: SAUCE: bcache: fix deadlock in bcache_allocator()" that is addressing the main deadlock bug (that seems to be a mainline bug - not fixed yet). We should spend more time trying to reproduce this deadlock with a mainline kernel and post the patch to the LKML for review / feedback. However, considering that this patch seems to fix/prevent the specific deadlock problem reported in this bug (tested on the affected platform) it can be
[Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed- bionic' to 'verification-done-bionic'. If the problem still exists, change the tag 'verification-needed-bionic' to 'verification-failed- bionic'. If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you! ** Tags added: verification-needed-bionic -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1784665 Title: bcache: bch_allocator_thread(): hung task timeout Status in linux package in Ubuntu: Fix Committed Status in linux source package in Xenial: Fix Committed Status in linux source package in Bionic: Fix Committed Status in linux source package in Disco: Fix Committed Status in linux source package in Eoan: Fix Committed Bug description: [Impact] bcache_allocator() can call the following: bch_allocator_thread() -> bch_prio_write() -> bch_bucket_alloc() -> wait on >set->bucket_wait But the wake up event on bucket_wait is supposed to come from bch_allocator_thread() itself causing a deadlock. [Test Case] This is a simple script that can easily trigger the deadlock condition: https://launchpadlibrarian.net/381282009/bcache-basic-repro.sh A better test case has been also provided in bug 1796292 (duplicate of this bug): https://bugs.launchpad.net/curtin/+bug/1796292/+attachment/5280353/+files/curtin-nvme.sh [Fix] Fix by making the call to bch_prio_write() non-blocking, so that bch_allocator_thread() never waits on itself. Moreover, make sure to wake up the garbage collector thread when bch_prio_write() is failing to allocate buckets to increase the chance of freeing up more buckets. In addition to that it would be safe to also import other upstream bcache fixes (all clean cherry picks): 7e865eba00a3df2dc8c4746173a8ca1c1c7f042e bcache: fix potential deadlock in cached_def_free() 80265d8dfd77792e133793cef44a21323aac2908 bcache: acquire bch_register_lock later in cached_dev_free() ce4c3e19e5201424357a0c82176633b32a98d2ec bcache: Replace bch_read_string_list() by __sysfs_match_string() ecb37ce9baac653cc09e2b631393dde3df82979f bcache: Move couple of functions to sysfs.c 04cbc21137bfa4d7b8771a5b14f3d6c9b2aee671 bcache: Move couple of string arrays to sysfs.c 5f2b18ec8e1643410a2369f06888951cdedea0bf bcache: Fix a compiler warning in bcache_device_init() 20d3a518713e394efa5a899c84574b4b79ec5098 bcache: Reduce the number of sparse complaints about lock imbalances 42361469ae84c851e40cb1f94c8c9a14cdd94039 bcache: Suppress more warnings about set-but-not-used variables f0d3814090ac77de94c42b7124c37ece23629197 bcache: Remove an unused variable 47344e330eabc1515cbe6061eb337100a3ab6d37 bcache: Fix kernel-doc warnings 9dfbdec7b7fea1ff1b7b5d5d12980dbc7dca46c7 bcache: Annotate switch fall-through 4a4e443835a43a79113cc237c472c0d268eb1e1c bcache: Add __printf annotation to __bch_check_keys() fd01991d5c20098c5c1ffc4dca6c821cc60a2f74 bcache: Fix indentation ca71df31661a0518ed58a1a59cf1993962153ebb bcache: fix using of loop variable in memory shrink f3641c3abd1da978ee969b0203b71b86ec1bfa93 bcache: fix error return value in memory shrink 688892b3bc05e25da94866e32210e5f503f16f69 bcache: fix incorrect sysfs output value of strip size 09a44ca2114737e0932257619c16a2b50c7807f1 bcache: use pr_info() to inform duplicated CACHE_SET_IO_DISABLE set c4dc2497d50d9c6fb16aa0d07b6a14f3b2adb1e0 bcache: fix high CPU occupancy during journal a728eacbbdd229d1d903e46261c57d5206f87a4a bcache: add journal statistic 616486ab52ab7f9739b066d958bdd20e65aefd74 bcache: fix writeback target calc on large devices 1f0ffa67349c56ea54c03ccfd1e073c990e7411e bcache: only set BCACHE_DEV_WB_RUNNING when cached device attached eb8cbb6df38f6e5124a3d5f1f8a3dbf519537c60 bcache: improve bcache_reboot() 9951379b0ca88c95876ad9778b9099e19a95d566 bcache: never writeback a discard operation [Regression Potential] The upstream fixes are all clean cherry picks from stable (most of them are small cleanups), so regression potential is minimal. The only special patch is "UBUNTU: SAUCE: bcache: fix deadlock in bcache_allocator()" that is addressing the main deadlock bug (that seems to be a mainline bug - not fixed yet). We should spend more time trying to reproduce this deadlock with a mainline kernel and post the patch to the LKML for review / feedback. However, considering that this patch seems to fix/prevent the specific deadlock problem reported in this bug (tested on the affected platform) it can be
[Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed- disco' to 'verification-done-disco'. If the problem still exists, change the tag 'verification-needed-disco' to 'verification-failed-disco'. If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you! ** Tags added: verification-needed-disco -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1784665 Title: bcache: bch_allocator_thread(): hung task timeout Status in linux package in Ubuntu: Fix Committed Status in linux source package in Xenial: Fix Committed Status in linux source package in Bionic: Fix Committed Status in linux source package in Disco: Fix Committed Status in linux source package in Eoan: Fix Committed Bug description: [Impact] bcache_allocator() can call the following: bch_allocator_thread() -> bch_prio_write() -> bch_bucket_alloc() -> wait on >set->bucket_wait But the wake up event on bucket_wait is supposed to come from bch_allocator_thread() itself causing a deadlock. [Test Case] This is a simple script that can easily trigger the deadlock condition: https://launchpadlibrarian.net/381282009/bcache-basic-repro.sh A better test case has been also provided in bug 1796292 (duplicate of this bug): https://bugs.launchpad.net/curtin/+bug/1796292/+attachment/5280353/+files/curtin-nvme.sh [Fix] Fix by making the call to bch_prio_write() non-blocking, so that bch_allocator_thread() never waits on itself. Moreover, make sure to wake up the garbage collector thread when bch_prio_write() is failing to allocate buckets to increase the chance of freeing up more buckets. In addition to that it would be safe to also import other upstream bcache fixes (all clean cherry picks): 7e865eba00a3df2dc8c4746173a8ca1c1c7f042e bcache: fix potential deadlock in cached_def_free() 80265d8dfd77792e133793cef44a21323aac2908 bcache: acquire bch_register_lock later in cached_dev_free() ce4c3e19e5201424357a0c82176633b32a98d2ec bcache: Replace bch_read_string_list() by __sysfs_match_string() ecb37ce9baac653cc09e2b631393dde3df82979f bcache: Move couple of functions to sysfs.c 04cbc21137bfa4d7b8771a5b14f3d6c9b2aee671 bcache: Move couple of string arrays to sysfs.c 5f2b18ec8e1643410a2369f06888951cdedea0bf bcache: Fix a compiler warning in bcache_device_init() 20d3a518713e394efa5a899c84574b4b79ec5098 bcache: Reduce the number of sparse complaints about lock imbalances 42361469ae84c851e40cb1f94c8c9a14cdd94039 bcache: Suppress more warnings about set-but-not-used variables f0d3814090ac77de94c42b7124c37ece23629197 bcache: Remove an unused variable 47344e330eabc1515cbe6061eb337100a3ab6d37 bcache: Fix kernel-doc warnings 9dfbdec7b7fea1ff1b7b5d5d12980dbc7dca46c7 bcache: Annotate switch fall-through 4a4e443835a43a79113cc237c472c0d268eb1e1c bcache: Add __printf annotation to __bch_check_keys() fd01991d5c20098c5c1ffc4dca6c821cc60a2f74 bcache: Fix indentation ca71df31661a0518ed58a1a59cf1993962153ebb bcache: fix using of loop variable in memory shrink f3641c3abd1da978ee969b0203b71b86ec1bfa93 bcache: fix error return value in memory shrink 688892b3bc05e25da94866e32210e5f503f16f69 bcache: fix incorrect sysfs output value of strip size 09a44ca2114737e0932257619c16a2b50c7807f1 bcache: use pr_info() to inform duplicated CACHE_SET_IO_DISABLE set c4dc2497d50d9c6fb16aa0d07b6a14f3b2adb1e0 bcache: fix high CPU occupancy during journal a728eacbbdd229d1d903e46261c57d5206f87a4a bcache: add journal statistic 616486ab52ab7f9739b066d958bdd20e65aefd74 bcache: fix writeback target calc on large devices 1f0ffa67349c56ea54c03ccfd1e073c990e7411e bcache: only set BCACHE_DEV_WB_RUNNING when cached device attached eb8cbb6df38f6e5124a3d5f1f8a3dbf519537c60 bcache: improve bcache_reboot() 9951379b0ca88c95876ad9778b9099e19a95d566 bcache: never writeback a discard operation [Regression Potential] The upstream fixes are all clean cherry picks from stable (most of them are small cleanups), so regression potential is minimal. The only special patch is "UBUNTU: SAUCE: bcache: fix deadlock in bcache_allocator()" that is addressing the main deadlock bug (that seems to be a mainline bug - not fixed yet). We should spend more time trying to reproduce this deadlock with a mainline kernel and post the patch to the LKML for review / feedback. However, considering that this patch seems to fix/prevent the specific deadlock problem reported in this bug (tested on the affected platform) it can be
[Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
** Changed in: linux (Ubuntu Bionic) Status: Confirmed => Fix Committed ** Changed in: linux (Ubuntu Disco) Status: Confirmed => Fix Committed ** Changed in: linux (Ubuntu Xenial) Status: Confirmed => Fix Committed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1784665 Title: bcache: bch_allocator_thread(): hung task timeout Status in linux package in Ubuntu: Fix Committed Status in linux source package in Xenial: Fix Committed Status in linux source package in Bionic: Fix Committed Status in linux source package in Disco: Fix Committed Status in linux source package in Eoan: Fix Committed Bug description: [Impact] bcache_allocator() can call the following: bch_allocator_thread() -> bch_prio_write() -> bch_bucket_alloc() -> wait on >set->bucket_wait But the wake up event on bucket_wait is supposed to come from bch_allocator_thread() itself causing a deadlock. [Test Case] This is a simple script that can easily trigger the deadlock condition: https://launchpadlibrarian.net/381282009/bcache-basic-repro.sh A better test case has been also provided in bug 1796292 (duplicate of this bug): https://bugs.launchpad.net/curtin/+bug/1796292/+attachment/5280353/+files/curtin-nvme.sh [Fix] Fix by making the call to bch_prio_write() non-blocking, so that bch_allocator_thread() never waits on itself. Moreover, make sure to wake up the garbage collector thread when bch_prio_write() is failing to allocate buckets to increase the chance of freeing up more buckets. In addition to that it would be safe to also import other upstream bcache fixes (all clean cherry picks): 7e865eba00a3df2dc8c4746173a8ca1c1c7f042e bcache: fix potential deadlock in cached_def_free() 80265d8dfd77792e133793cef44a21323aac2908 bcache: acquire bch_register_lock later in cached_dev_free() ce4c3e19e5201424357a0c82176633b32a98d2ec bcache: Replace bch_read_string_list() by __sysfs_match_string() ecb37ce9baac653cc09e2b631393dde3df82979f bcache: Move couple of functions to sysfs.c 04cbc21137bfa4d7b8771a5b14f3d6c9b2aee671 bcache: Move couple of string arrays to sysfs.c 5f2b18ec8e1643410a2369f06888951cdedea0bf bcache: Fix a compiler warning in bcache_device_init() 20d3a518713e394efa5a899c84574b4b79ec5098 bcache: Reduce the number of sparse complaints about lock imbalances 42361469ae84c851e40cb1f94c8c9a14cdd94039 bcache: Suppress more warnings about set-but-not-used variables f0d3814090ac77de94c42b7124c37ece23629197 bcache: Remove an unused variable 47344e330eabc1515cbe6061eb337100a3ab6d37 bcache: Fix kernel-doc warnings 9dfbdec7b7fea1ff1b7b5d5d12980dbc7dca46c7 bcache: Annotate switch fall-through 4a4e443835a43a79113cc237c472c0d268eb1e1c bcache: Add __printf annotation to __bch_check_keys() fd01991d5c20098c5c1ffc4dca6c821cc60a2f74 bcache: Fix indentation ca71df31661a0518ed58a1a59cf1993962153ebb bcache: fix using of loop variable in memory shrink f3641c3abd1da978ee969b0203b71b86ec1bfa93 bcache: fix error return value in memory shrink 688892b3bc05e25da94866e32210e5f503f16f69 bcache: fix incorrect sysfs output value of strip size 09a44ca2114737e0932257619c16a2b50c7807f1 bcache: use pr_info() to inform duplicated CACHE_SET_IO_DISABLE set c4dc2497d50d9c6fb16aa0d07b6a14f3b2adb1e0 bcache: fix high CPU occupancy during journal a728eacbbdd229d1d903e46261c57d5206f87a4a bcache: add journal statistic 616486ab52ab7f9739b066d958bdd20e65aefd74 bcache: fix writeback target calc on large devices 1f0ffa67349c56ea54c03ccfd1e073c990e7411e bcache: only set BCACHE_DEV_WB_RUNNING when cached device attached eb8cbb6df38f6e5124a3d5f1f8a3dbf519537c60 bcache: improve bcache_reboot() 9951379b0ca88c95876ad9778b9099e19a95d566 bcache: never writeback a discard operation [Regression Potential] The upstream fixes are all clean cherry picks from stable (most of them are small cleanups), so regression potential is minimal. The only special patch is "UBUNTU: SAUCE: bcache: fix deadlock in bcache_allocator()" that is addressing the main deadlock bug (that seems to be a mainline bug - not fixed yet). We should spend more time trying to reproduce this deadlock with a mainline kernel and post the patch to the LKML for review / feedback. However, considering that this patch seems to fix/prevent the specific deadlock problem reported in this bug (tested on the affected platform) it can be considered safe to apply it. [Original Bug Report] $ cat /proc/version_signature Ubuntu 4.15.0-29.31-generic 4.15.18 $ lsb_release -rd Description: Ubuntu Cosmic Cuttlefish (development branch) Release: 18.10 $ apt-cache policy linux-image-`uname -r` linux-image-4.15.0-29-generic: Installed: 4.15.0-29.31 Candidate: 4.15.0-29.31 Version table: *** 4.15.0-29.31 500
[Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
** Changed in: linux (Ubuntu Eoan) Status: Confirmed => Fix Committed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1784665 Title: bcache: bch_allocator_thread(): hung task timeout Status in linux package in Ubuntu: Fix Committed Status in linux source package in Xenial: Confirmed Status in linux source package in Bionic: Confirmed Status in linux source package in Disco: Confirmed Status in linux source package in Eoan: Fix Committed Bug description: [Impact] bcache_allocator() can call the following: bch_allocator_thread() -> bch_prio_write() -> bch_bucket_alloc() -> wait on >set->bucket_wait But the wake up event on bucket_wait is supposed to come from bch_allocator_thread() itself causing a deadlock. [Test Case] This is a simple script that can easily trigger the deadlock condition: https://launchpadlibrarian.net/381282009/bcache-basic-repro.sh A better test case has been also provided in bug 1796292 (duplicate of this bug): https://bugs.launchpad.net/curtin/+bug/1796292/+attachment/5280353/+files/curtin-nvme.sh [Fix] Fix by making the call to bch_prio_write() non-blocking, so that bch_allocator_thread() never waits on itself. Moreover, make sure to wake up the garbage collector thread when bch_prio_write() is failing to allocate buckets to increase the chance of freeing up more buckets. In addition to that it would be safe to also import other upstream bcache fixes (all clean cherry picks): 7e865eba00a3df2dc8c4746173a8ca1c1c7f042e bcache: fix potential deadlock in cached_def_free() 80265d8dfd77792e133793cef44a21323aac2908 bcache: acquire bch_register_lock later in cached_dev_free() ce4c3e19e5201424357a0c82176633b32a98d2ec bcache: Replace bch_read_string_list() by __sysfs_match_string() ecb37ce9baac653cc09e2b631393dde3df82979f bcache: Move couple of functions to sysfs.c 04cbc21137bfa4d7b8771a5b14f3d6c9b2aee671 bcache: Move couple of string arrays to sysfs.c 5f2b18ec8e1643410a2369f06888951cdedea0bf bcache: Fix a compiler warning in bcache_device_init() 20d3a518713e394efa5a899c84574b4b79ec5098 bcache: Reduce the number of sparse complaints about lock imbalances 42361469ae84c851e40cb1f94c8c9a14cdd94039 bcache: Suppress more warnings about set-but-not-used variables f0d3814090ac77de94c42b7124c37ece23629197 bcache: Remove an unused variable 47344e330eabc1515cbe6061eb337100a3ab6d37 bcache: Fix kernel-doc warnings 9dfbdec7b7fea1ff1b7b5d5d12980dbc7dca46c7 bcache: Annotate switch fall-through 4a4e443835a43a79113cc237c472c0d268eb1e1c bcache: Add __printf annotation to __bch_check_keys() fd01991d5c20098c5c1ffc4dca6c821cc60a2f74 bcache: Fix indentation ca71df31661a0518ed58a1a59cf1993962153ebb bcache: fix using of loop variable in memory shrink f3641c3abd1da978ee969b0203b71b86ec1bfa93 bcache: fix error return value in memory shrink 688892b3bc05e25da94866e32210e5f503f16f69 bcache: fix incorrect sysfs output value of strip size 09a44ca2114737e0932257619c16a2b50c7807f1 bcache: use pr_info() to inform duplicated CACHE_SET_IO_DISABLE set c4dc2497d50d9c6fb16aa0d07b6a14f3b2adb1e0 bcache: fix high CPU occupancy during journal a728eacbbdd229d1d903e46261c57d5206f87a4a bcache: add journal statistic 616486ab52ab7f9739b066d958bdd20e65aefd74 bcache: fix writeback target calc on large devices 1f0ffa67349c56ea54c03ccfd1e073c990e7411e bcache: only set BCACHE_DEV_WB_RUNNING when cached device attached eb8cbb6df38f6e5124a3d5f1f8a3dbf519537c60 bcache: improve bcache_reboot() 9951379b0ca88c95876ad9778b9099e19a95d566 bcache: never writeback a discard operation [Regression Potential] The upstream fixes are all clean cherry picks from stable (most of them are small cleanups), so regression potential is minimal. The only special patch is "UBUNTU: SAUCE: bcache: fix deadlock in bcache_allocator()" that is addressing the main deadlock bug (that seems to be a mainline bug - not fixed yet). We should spend more time trying to reproduce this deadlock with a mainline kernel and post the patch to the LKML for review / feedback. However, considering that this patch seems to fix/prevent the specific deadlock problem reported in this bug (tested on the affected platform) it can be considered safe to apply it. [Original Bug Report] $ cat /proc/version_signature Ubuntu 4.15.0-29.31-generic 4.15.18 $ lsb_release -rd Description: Ubuntu Cosmic Cuttlefish (development branch) Release: 18.10 $ apt-cache policy linux-image-`uname -r` linux-image-4.15.0-29-generic: Installed: 4.15.0-29.31 Candidate: 4.15.0-29.31 Version table: *** 4.15.0-29.31 500 500 http://archive.ubuntu.com/ubuntu cosmic/main amd64 Packages 100 /var/lib/dpkg/status 3) mkfs.ext4 /dev/bcache0 returns successful creating an ext4
[Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
** Also affects: linux (Ubuntu Xenial) Importance: Undecided Status: New ** Changed in: linux (Ubuntu Xenial) Status: New => Confirmed ** Changed in: linux (Ubuntu Xenial) Importance: Undecided => High ** Changed in: linux (Ubuntu Xenial) Assignee: (unassigned) => Andrea Righi (arighi) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1784665 Title: bcache: bch_allocator_thread(): hung task timeout Status in linux package in Ubuntu: Confirmed Status in linux source package in Xenial: Confirmed Status in linux source package in Bionic: Confirmed Status in linux source package in Disco: Confirmed Status in linux source package in Eoan: Confirmed Bug description: [Impact] bcache_allocator() can call the following: bch_allocator_thread() -> bch_prio_write() -> bch_bucket_alloc() -> wait on >set->bucket_wait But the wake up event on bucket_wait is supposed to come from bch_allocator_thread() itself causing a deadlock. [Test Case] This is a simple script that can easily trigger the deadlock condition: https://launchpadlibrarian.net/381282009/bcache-basic-repro.sh A better test case has been also provided in bug 1796292 (duplicate of this bug): https://bugs.launchpad.net/curtin/+bug/1796292/+attachment/5280353/+files/curtin-nvme.sh [Fix] Fix by making the call to bch_prio_write() non-blocking, so that bch_allocator_thread() never waits on itself. Moreover, make sure to wake up the garbage collector thread when bch_prio_write() is failing to allocate buckets to increase the chance of freeing up more buckets. In addition to that it would be safe to also import other upstream bcache fixes (all clean cherry picks): 7e865eba00a3df2dc8c4746173a8ca1c1c7f042e bcache: fix potential deadlock in cached_def_free() 80265d8dfd77792e133793cef44a21323aac2908 bcache: acquire bch_register_lock later in cached_dev_free() ce4c3e19e5201424357a0c82176633b32a98d2ec bcache: Replace bch_read_string_list() by __sysfs_match_string() ecb37ce9baac653cc09e2b631393dde3df82979f bcache: Move couple of functions to sysfs.c 04cbc21137bfa4d7b8771a5b14f3d6c9b2aee671 bcache: Move couple of string arrays to sysfs.c 5f2b18ec8e1643410a2369f06888951cdedea0bf bcache: Fix a compiler warning in bcache_device_init() 20d3a518713e394efa5a899c84574b4b79ec5098 bcache: Reduce the number of sparse complaints about lock imbalances 42361469ae84c851e40cb1f94c8c9a14cdd94039 bcache: Suppress more warnings about set-but-not-used variables f0d3814090ac77de94c42b7124c37ece23629197 bcache: Remove an unused variable 47344e330eabc1515cbe6061eb337100a3ab6d37 bcache: Fix kernel-doc warnings 9dfbdec7b7fea1ff1b7b5d5d12980dbc7dca46c7 bcache: Annotate switch fall-through 4a4e443835a43a79113cc237c472c0d268eb1e1c bcache: Add __printf annotation to __bch_check_keys() fd01991d5c20098c5c1ffc4dca6c821cc60a2f74 bcache: Fix indentation ca71df31661a0518ed58a1a59cf1993962153ebb bcache: fix using of loop variable in memory shrink f3641c3abd1da978ee969b0203b71b86ec1bfa93 bcache: fix error return value in memory shrink 688892b3bc05e25da94866e32210e5f503f16f69 bcache: fix incorrect sysfs output value of strip size 09a44ca2114737e0932257619c16a2b50c7807f1 bcache: use pr_info() to inform duplicated CACHE_SET_IO_DISABLE set c4dc2497d50d9c6fb16aa0d07b6a14f3b2adb1e0 bcache: fix high CPU occupancy during journal a728eacbbdd229d1d903e46261c57d5206f87a4a bcache: add journal statistic 616486ab52ab7f9739b066d958bdd20e65aefd74 bcache: fix writeback target calc on large devices 1f0ffa67349c56ea54c03ccfd1e073c990e7411e bcache: only set BCACHE_DEV_WB_RUNNING when cached device attached eb8cbb6df38f6e5124a3d5f1f8a3dbf519537c60 bcache: improve bcache_reboot() 9951379b0ca88c95876ad9778b9099e19a95d566 bcache: never writeback a discard operation [Regression Potential] The upstream fixes are all clean cherry picks from stable (most of them are small cleanups), so regression potential is minimal. The only special patch is "UBUNTU: SAUCE: bcache: fix deadlock in bcache_allocator()" that is addressing the main deadlock bug (that seems to be a mainline bug - not fixed yet). We should spend more time trying to reproduce this deadlock with a mainline kernel and post the patch to the LKML for review / feedback. However, considering that this patch seems to fix/prevent the specific deadlock problem reported in this bug (tested on the affected platform) it can be considered safe to apply it. [Original Bug Report] $ cat /proc/version_signature Ubuntu 4.15.0-29.31-generic 4.15.18 $ lsb_release -rd Description: Ubuntu Cosmic Cuttlefish (development branch) Release: 18.10 $ apt-cache policy linux-image-`uname -r` linux-image-4.15.0-29-generic: Installed: 4.15.0-29.31 Candidate:
[Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
** Changed in: linux (Ubuntu Disco) Status: New => Confirmed ** Changed in: linux (Ubuntu Disco) Importance: Undecided => High -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1784665 Title: bcache: bch_allocator_thread(): hung task timeout Status in linux package in Ubuntu: Confirmed Status in linux source package in Bionic: Confirmed Status in linux source package in Disco: Confirmed Status in linux source package in Eoan: Confirmed Bug description: [Impact] bcache_allocator() can call the following: bch_allocator_thread() -> bch_prio_write() -> bch_bucket_alloc() -> wait on >set->bucket_wait But the wake up event on bucket_wait is supposed to come from bch_allocator_thread() itself causing a deadlock. [Test Case] This is a simple script that can easily trigger the deadlock condition: https://launchpadlibrarian.net/381282009/bcache-basic-repro.sh A better test case has been also provided in bug 1796292 (duplicate of this bug): https://bugs.launchpad.net/curtin/+bug/1796292/+attachment/5280353/+files/curtin-nvme.sh [Fix] Fix by making the call to bch_prio_write() non-blocking, so that bch_allocator_thread() never waits on itself. Moreover, make sure to wake up the garbage collector thread when bch_prio_write() is failing to allocate buckets to increase the chance of freeing up more buckets. In addition to that it would be safe to also import other upstream bcache fixes (all clean cherry picks): 7e865eba00a3df2dc8c4746173a8ca1c1c7f042e bcache: fix potential deadlock in cached_def_free() 80265d8dfd77792e133793cef44a21323aac2908 bcache: acquire bch_register_lock later in cached_dev_free() ce4c3e19e5201424357a0c82176633b32a98d2ec bcache: Replace bch_read_string_list() by __sysfs_match_string() ecb37ce9baac653cc09e2b631393dde3df82979f bcache: Move couple of functions to sysfs.c 04cbc21137bfa4d7b8771a5b14f3d6c9b2aee671 bcache: Move couple of string arrays to sysfs.c 5f2b18ec8e1643410a2369f06888951cdedea0bf bcache: Fix a compiler warning in bcache_device_init() 20d3a518713e394efa5a899c84574b4b79ec5098 bcache: Reduce the number of sparse complaints about lock imbalances 42361469ae84c851e40cb1f94c8c9a14cdd94039 bcache: Suppress more warnings about set-but-not-used variables f0d3814090ac77de94c42b7124c37ece23629197 bcache: Remove an unused variable 47344e330eabc1515cbe6061eb337100a3ab6d37 bcache: Fix kernel-doc warnings 9dfbdec7b7fea1ff1b7b5d5d12980dbc7dca46c7 bcache: Annotate switch fall-through 4a4e443835a43a79113cc237c472c0d268eb1e1c bcache: Add __printf annotation to __bch_check_keys() fd01991d5c20098c5c1ffc4dca6c821cc60a2f74 bcache: Fix indentation ca71df31661a0518ed58a1a59cf1993962153ebb bcache: fix using of loop variable in memory shrink f3641c3abd1da978ee969b0203b71b86ec1bfa93 bcache: fix error return value in memory shrink 688892b3bc05e25da94866e32210e5f503f16f69 bcache: fix incorrect sysfs output value of strip size 09a44ca2114737e0932257619c16a2b50c7807f1 bcache: use pr_info() to inform duplicated CACHE_SET_IO_DISABLE set c4dc2497d50d9c6fb16aa0d07b6a14f3b2adb1e0 bcache: fix high CPU occupancy during journal a728eacbbdd229d1d903e46261c57d5206f87a4a bcache: add journal statistic 616486ab52ab7f9739b066d958bdd20e65aefd74 bcache: fix writeback target calc on large devices 1f0ffa67349c56ea54c03ccfd1e073c990e7411e bcache: only set BCACHE_DEV_WB_RUNNING when cached device attached eb8cbb6df38f6e5124a3d5f1f8a3dbf519537c60 bcache: improve bcache_reboot() 9951379b0ca88c95876ad9778b9099e19a95d566 bcache: never writeback a discard operation [Regression Potential] The upstream fixes are all clean cherry picks from stable (most of them are small cleanups), so regression potential is minimal. The only special patch is "UBUNTU: SAUCE: bcache: fix deadlock in bcache_allocator()" that is addressing the main deadlock bug (that seems to be a mainline bug - not fixed yet). We should spend more time trying to reproduce this deadlock with a mainline kernel and post the patch to the LKML for review / feedback. However, considering that this patch seems to fix/prevent the specific deadlock problem reported in this bug (tested on the affected platform) it can be considered safe to apply it. [Original Bug Report] $ cat /proc/version_signature Ubuntu 4.15.0-29.31-generic 4.15.18 $ lsb_release -rd Description: Ubuntu Cosmic Cuttlefish (development branch) Release: 18.10 $ apt-cache policy linux-image-`uname -r` linux-image-4.15.0-29-generic: Installed: 4.15.0-29.31 Candidate: 4.15.0-29.31 Version table: *** 4.15.0-29.31 500 500 http://archive.ubuntu.com/ubuntu cosmic/main amd64 Packages 100 /var/lib/dpkg/status 3) mkfs.ext4 /dev/bcache0 returns successful creating an ext4
[Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
** No longer affects: linux (Ubuntu Cosmic) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1784665 Title: bcache: bch_allocator_thread(): hung task timeout Status in linux package in Ubuntu: Confirmed Status in linux source package in Bionic: Confirmed Status in linux source package in Disco: New Status in linux source package in Eoan: Confirmed Bug description: [Impact] bcache_allocator() can call the following: bch_allocator_thread() -> bch_prio_write() -> bch_bucket_alloc() -> wait on >set->bucket_wait But the wake up event on bucket_wait is supposed to come from bch_allocator_thread() itself causing a deadlock. [Test Case] This is a simple script that can easily trigger the deadlock condition: https://launchpadlibrarian.net/381282009/bcache-basic-repro.sh A better test case has been also provided in bug 1796292 (duplicate of this bug): https://bugs.launchpad.net/curtin/+bug/1796292/+attachment/5280353/+files/curtin-nvme.sh [Fix] Fix by making the call to bch_prio_write() non-blocking, so that bch_allocator_thread() never waits on itself. Moreover, make sure to wake up the garbage collector thread when bch_prio_write() is failing to allocate buckets to increase the chance of freeing up more buckets. In addition to that it would be safe to also import other upstream bcache fixes (all clean cherry picks): 7e865eba00a3df2dc8c4746173a8ca1c1c7f042e bcache: fix potential deadlock in cached_def_free() 80265d8dfd77792e133793cef44a21323aac2908 bcache: acquire bch_register_lock later in cached_dev_free() ce4c3e19e5201424357a0c82176633b32a98d2ec bcache: Replace bch_read_string_list() by __sysfs_match_string() ecb37ce9baac653cc09e2b631393dde3df82979f bcache: Move couple of functions to sysfs.c 04cbc21137bfa4d7b8771a5b14f3d6c9b2aee671 bcache: Move couple of string arrays to sysfs.c 5f2b18ec8e1643410a2369f06888951cdedea0bf bcache: Fix a compiler warning in bcache_device_init() 20d3a518713e394efa5a899c84574b4b79ec5098 bcache: Reduce the number of sparse complaints about lock imbalances 42361469ae84c851e40cb1f94c8c9a14cdd94039 bcache: Suppress more warnings about set-but-not-used variables f0d3814090ac77de94c42b7124c37ece23629197 bcache: Remove an unused variable 47344e330eabc1515cbe6061eb337100a3ab6d37 bcache: Fix kernel-doc warnings 9dfbdec7b7fea1ff1b7b5d5d12980dbc7dca46c7 bcache: Annotate switch fall-through 4a4e443835a43a79113cc237c472c0d268eb1e1c bcache: Add __printf annotation to __bch_check_keys() fd01991d5c20098c5c1ffc4dca6c821cc60a2f74 bcache: Fix indentation ca71df31661a0518ed58a1a59cf1993962153ebb bcache: fix using of loop variable in memory shrink f3641c3abd1da978ee969b0203b71b86ec1bfa93 bcache: fix error return value in memory shrink 688892b3bc05e25da94866e32210e5f503f16f69 bcache: fix incorrect sysfs output value of strip size 09a44ca2114737e0932257619c16a2b50c7807f1 bcache: use pr_info() to inform duplicated CACHE_SET_IO_DISABLE set c4dc2497d50d9c6fb16aa0d07b6a14f3b2adb1e0 bcache: fix high CPU occupancy during journal a728eacbbdd229d1d903e46261c57d5206f87a4a bcache: add journal statistic 616486ab52ab7f9739b066d958bdd20e65aefd74 bcache: fix writeback target calc on large devices 1f0ffa67349c56ea54c03ccfd1e073c990e7411e bcache: only set BCACHE_DEV_WB_RUNNING when cached device attached eb8cbb6df38f6e5124a3d5f1f8a3dbf519537c60 bcache: improve bcache_reboot() 9951379b0ca88c95876ad9778b9099e19a95d566 bcache: never writeback a discard operation [Regression Potential] The upstream fixes are all clean cherry picks from stable (most of them are small cleanups), so regression potential is minimal. The only special patch is "UBUNTU: SAUCE: bcache: fix deadlock in bcache_allocator()" that is addressing the main deadlock bug (that seems to be a mainline bug - not fixed yet). We should spend more time trying to reproduce this deadlock with a mainline kernel and post the patch to the LKML for review / feedback. However, considering that this patch seems to fix/prevent the specific deadlock problem reported in this bug (tested on the affected platform) it can be considered safe to apply it. [Original Bug Report] $ cat /proc/version_signature Ubuntu 4.15.0-29.31-generic 4.15.18 $ lsb_release -rd Description: Ubuntu Cosmic Cuttlefish (development branch) Release: 18.10 $ apt-cache policy linux-image-`uname -r` linux-image-4.15.0-29-generic: Installed: 4.15.0-29.31 Candidate: 4.15.0-29.31 Version table: *** 4.15.0-29.31 500 500 http://archive.ubuntu.com/ubuntu cosmic/main amd64 Packages 100 /var/lib/dpkg/status 3) mkfs.ext4 /dev/bcache0 returns successful creating an ext4 filesystem on top of a bcache device 4) mkfs.ext4 doesn't return and kernel prints hung process
[Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
** Also affects: linux (Ubuntu Eoan) Importance: High Assignee: Andrea Righi (arighi) Status: Confirmed ** Also affects: linux (Ubuntu Disco) Importance: Undecided Status: New ** Changed in: linux (Ubuntu Disco) Assignee: (unassigned) => Andrea Righi (arighi) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1784665 Title: bcache: bch_allocator_thread(): hung task timeout Status in linux package in Ubuntu: Confirmed Status in linux source package in Bionic: Confirmed Status in linux source package in Cosmic: Confirmed Status in linux source package in Disco: New Status in linux source package in Eoan: Confirmed Bug description: [Impact] bcache_allocator() can call the following: bch_allocator_thread() -> bch_prio_write() -> bch_bucket_alloc() -> wait on >set->bucket_wait But the wake up event on bucket_wait is supposed to come from bch_allocator_thread() itself causing a deadlock. [Test Case] This is a simple script that can easily trigger the deadlock condition: https://launchpadlibrarian.net/381282009/bcache-basic-repro.sh A better test case has been also provided in bug 1796292 (duplicate of this bug): https://bugs.launchpad.net/curtin/+bug/1796292/+attachment/5280353/+files/curtin-nvme.sh [Fix] Fix by making the call to bch_prio_write() non-blocking, so that bch_allocator_thread() never waits on itself. Moreover, make sure to wake up the garbage collector thread when bch_prio_write() is failing to allocate buckets to increase the chance of freeing up more buckets. In addition to that it would be safe to also import other upstream bcache fixes (all clean cherry picks): 7e865eba00a3df2dc8c4746173a8ca1c1c7f042e bcache: fix potential deadlock in cached_def_free() 80265d8dfd77792e133793cef44a21323aac2908 bcache: acquire bch_register_lock later in cached_dev_free() ce4c3e19e5201424357a0c82176633b32a98d2ec bcache: Replace bch_read_string_list() by __sysfs_match_string() ecb37ce9baac653cc09e2b631393dde3df82979f bcache: Move couple of functions to sysfs.c 04cbc21137bfa4d7b8771a5b14f3d6c9b2aee671 bcache: Move couple of string arrays to sysfs.c 5f2b18ec8e1643410a2369f06888951cdedea0bf bcache: Fix a compiler warning in bcache_device_init() 20d3a518713e394efa5a899c84574b4b79ec5098 bcache: Reduce the number of sparse complaints about lock imbalances 42361469ae84c851e40cb1f94c8c9a14cdd94039 bcache: Suppress more warnings about set-but-not-used variables f0d3814090ac77de94c42b7124c37ece23629197 bcache: Remove an unused variable 47344e330eabc1515cbe6061eb337100a3ab6d37 bcache: Fix kernel-doc warnings 9dfbdec7b7fea1ff1b7b5d5d12980dbc7dca46c7 bcache: Annotate switch fall-through 4a4e443835a43a79113cc237c472c0d268eb1e1c bcache: Add __printf annotation to __bch_check_keys() fd01991d5c20098c5c1ffc4dca6c821cc60a2f74 bcache: Fix indentation ca71df31661a0518ed58a1a59cf1993962153ebb bcache: fix using of loop variable in memory shrink f3641c3abd1da978ee969b0203b71b86ec1bfa93 bcache: fix error return value in memory shrink 688892b3bc05e25da94866e32210e5f503f16f69 bcache: fix incorrect sysfs output value of strip size 09a44ca2114737e0932257619c16a2b50c7807f1 bcache: use pr_info() to inform duplicated CACHE_SET_IO_DISABLE set c4dc2497d50d9c6fb16aa0d07b6a14f3b2adb1e0 bcache: fix high CPU occupancy during journal a728eacbbdd229d1d903e46261c57d5206f87a4a bcache: add journal statistic 616486ab52ab7f9739b066d958bdd20e65aefd74 bcache: fix writeback target calc on large devices 1f0ffa67349c56ea54c03ccfd1e073c990e7411e bcache: only set BCACHE_DEV_WB_RUNNING when cached device attached eb8cbb6df38f6e5124a3d5f1f8a3dbf519537c60 bcache: improve bcache_reboot() 9951379b0ca88c95876ad9778b9099e19a95d566 bcache: never writeback a discard operation [Regression Potential] The upstream fixes are all clean cherry picks from stable (most of them are small cleanups), so regression potential is minimal. The only special patch is "UBUNTU: SAUCE: bcache: fix deadlock in bcache_allocator()" that is addressing the main deadlock bug (that seems to be a mainline bug - not fixed yet). We should spend more time trying to reproduce this deadlock with a mainline kernel and post the patch to the LKML for review / feedback. However, considering that this patch seems to fix/prevent the specific deadlock problem reported in this bug (tested on the affected platform) it can be considered safe to apply it. [Original Bug Report] $ cat /proc/version_signature Ubuntu 4.15.0-29.31-generic 4.15.18 $ lsb_release -rd Description: Ubuntu Cosmic Cuttlefish (development branch) Release: 18.10 $ apt-cache policy linux-image-`uname -r` linux-image-4.15.0-29-generic: Installed: 4.15.0-29.31 Candidate: 4.15.0-29.31 Version table:
[Kernel-packages] [Bug 1784665] Re: bcache: bch_allocator_thread(): hung task timeout
SRU pull request sent to the kernel-team mailing list: https://lists.ubuntu.com/archives/kernel-team/2019-August/102784.html -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1784665 Title: bcache: bch_allocator_thread(): hung task timeout Status in linux package in Ubuntu: Confirmed Status in linux source package in Bionic: Confirmed Status in linux source package in Cosmic: Confirmed Bug description: [Impact] bcache_allocator() can call the following: bch_allocator_thread() -> bch_prio_write() -> bch_bucket_alloc() -> wait on >set->bucket_wait But the wake up event on bucket_wait is supposed to come from bch_allocator_thread() itself causing a deadlock. [Test Case] This is a simple script that can easily trigger the deadlock condition: https://launchpadlibrarian.net/381282009/bcache-basic-repro.sh A better test case has been also provided in bug 1796292 (duplicate of this bug): https://bugs.launchpad.net/curtin/+bug/1796292/+attachment/5280353/+files/curtin-nvme.sh [Fix] Fix by making the call to bch_prio_write() non-blocking, so that bch_allocator_thread() never waits on itself. Moreover, make sure to wake up the garbage collector thread when bch_prio_write() is failing to allocate buckets to increase the chance of freeing up more buckets. In addition to that it would be safe to also import other upstream bcache fixes (all clean cherry picks): 7e865eba00a3df2dc8c4746173a8ca1c1c7f042e bcache: fix potential deadlock in cached_def_free() 80265d8dfd77792e133793cef44a21323aac2908 bcache: acquire bch_register_lock later in cached_dev_free() ce4c3e19e5201424357a0c82176633b32a98d2ec bcache: Replace bch_read_string_list() by __sysfs_match_string() ecb37ce9baac653cc09e2b631393dde3df82979f bcache: Move couple of functions to sysfs.c 04cbc21137bfa4d7b8771a5b14f3d6c9b2aee671 bcache: Move couple of string arrays to sysfs.c 5f2b18ec8e1643410a2369f06888951cdedea0bf bcache: Fix a compiler warning in bcache_device_init() 20d3a518713e394efa5a899c84574b4b79ec5098 bcache: Reduce the number of sparse complaints about lock imbalances 42361469ae84c851e40cb1f94c8c9a14cdd94039 bcache: Suppress more warnings about set-but-not-used variables f0d3814090ac77de94c42b7124c37ece23629197 bcache: Remove an unused variable 47344e330eabc1515cbe6061eb337100a3ab6d37 bcache: Fix kernel-doc warnings 9dfbdec7b7fea1ff1b7b5d5d12980dbc7dca46c7 bcache: Annotate switch fall-through 4a4e443835a43a79113cc237c472c0d268eb1e1c bcache: Add __printf annotation to __bch_check_keys() fd01991d5c20098c5c1ffc4dca6c821cc60a2f74 bcache: Fix indentation ca71df31661a0518ed58a1a59cf1993962153ebb bcache: fix using of loop variable in memory shrink f3641c3abd1da978ee969b0203b71b86ec1bfa93 bcache: fix error return value in memory shrink 688892b3bc05e25da94866e32210e5f503f16f69 bcache: fix incorrect sysfs output value of strip size 09a44ca2114737e0932257619c16a2b50c7807f1 bcache: use pr_info() to inform duplicated CACHE_SET_IO_DISABLE set c4dc2497d50d9c6fb16aa0d07b6a14f3b2adb1e0 bcache: fix high CPU occupancy during journal a728eacbbdd229d1d903e46261c57d5206f87a4a bcache: add journal statistic 616486ab52ab7f9739b066d958bdd20e65aefd74 bcache: fix writeback target calc on large devices 1f0ffa67349c56ea54c03ccfd1e073c990e7411e bcache: only set BCACHE_DEV_WB_RUNNING when cached device attached eb8cbb6df38f6e5124a3d5f1f8a3dbf519537c60 bcache: improve bcache_reboot() 9951379b0ca88c95876ad9778b9099e19a95d566 bcache: never writeback a discard operation [Regression Potential] The upstream fixes are all clean cherry picks from stable (most of them are small cleanups), so regression potential is minimal. The only special patch is "UBUNTU: SAUCE: bcache: fix deadlock in bcache_allocator()" that is addressing the main deadlock bug (that seems to be a mainline bug - not fixed yet). We should spend more time trying to reproduce this deadlock with a mainline kernel and post the patch to the LKML for review / feedback. However, considering that this patch seems to fix/prevent the specific deadlock problem reported in this bug (tested on the affected platform) it can be considered safe to apply it. [Original Bug Report] $ cat /proc/version_signature Ubuntu 4.15.0-29.31-generic 4.15.18 $ lsb_release -rd Description: Ubuntu Cosmic Cuttlefish (development branch) Release: 18.10 $ apt-cache policy linux-image-`uname -r` linux-image-4.15.0-29-generic: Installed: 4.15.0-29.31 Candidate: 4.15.0-29.31 Version table: *** 4.15.0-29.31 500 500 http://archive.ubuntu.com/ubuntu cosmic/main amd64 Packages 100 /var/lib/dpkg/status 3) mkfs.ext4 /dev/bcache0 returns successful creating an ext4 filesystem on top of a bcache device 4) mkfs.ext4 doesn't
