Also marking Ubuntu / Groovy as Fix Released, as Groovy/devel has the
5.8 kernel already, which ships the fix.
** Changed in: linux (Ubuntu)
Status: Fix Committed => Fix Released
** Changed in: linux (Ubuntu Groovy)
Status: Won't Fix => Fix Released
--
You received this bug
This bug was fixed in the package linux - 4.15.0-115.116
---
linux (4.15.0-115.116) bionic; urgency=medium
* bionic/linux: 4.15.0-115.116 -proposed tracker (LP: #1893055)
* [Potential Regression] dscr_inherit_exec_test from powerpc in
ubuntu_kernel_selftests failed on B/E/F
This bug was fixed in the package linux - 5.4.0-45.49
---
linux (5.4.0-45.49) focal; urgency=medium
* focal/linux: 5.4.0-45.49 -proposed tracker (LP: #1893050)
* [Potential Regression] dscr_inherit_exec_test from powerpc in
ubuntu_kernel_selftests failed on B/E/F (LP:
The Eoan Ermine has reached end of life, so this bug will not be fixed
for that release
** Changed in: linux (Ubuntu Eoan)
Status: Fix Committed => Won't Fix
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
Verification done on "eoan" (5.3/linux-hwe on Bionic)
$ uname -rv
5.3.0-65-generic #59-Ubuntu SMP Tue Jul 28 07:27:41 UTC 2020
$ sudo make-bcache --bdev $DEV --block 8k
...
[ 103.766185] bcache: bcache_device_init() bcache0: sb/logical block size
(8192) greater than page size (4096) falling
This bug was fixed in the package linux - 4.4.0-187.217
---
linux (4.4.0-187.217) xenial; urgency=medium
* xenial/linux: 4.4.0-187.217 -proposed tracker (LP: #1888274)
* Regression in kernel 4.15.0-91 causes kernel panic with Bcache
(LP: #1867916)
- bcache: check and
Verification done for Focal.
$ uname -rv
5.4.0-43-generic #47-Ubuntu SMP Sat Aug 8 06:34:35 UTC 2020
$ sudo make-bcache --bdev $DEV --block 8k
...
[ 71.251993] bcache: bcache_device_init() bcache0: sb/logical block size
(8192) greater than page size (4096) falling back to device logical block
Verification done for Bionic.
$ uname -rv
4.15.0-113-generic #114-Ubuntu SMP Sun Aug 9 07:27:58 UTC 2020
$ sudo make-bcache --bdev $DEV --block 8k
...
[ 18.467465] bcache: bcache_device_init() bcache0: sb/logical block size
(8192) greater than page size (4096) falling back to device logical
This bug is awaiting verification that the kernel in -proposed solves
the problem. Please test the kernel and update this bug with the
results. If the problem is solved, change the tag 'verification-needed-
focal' to 'verification-done-focal'. If the problem still exists, change
the tag
** Changed in: linux (Ubuntu Bionic)
Status: In Progress => Fix Committed
** Changed in: linux (Ubuntu Focal)
Status: In Progress => Fix Committed
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
This bug is awaiting verification that the kernel in -proposed solves
the problem. Please test the kernel and update this bug with the
results. If the problem is solved, change the tag 'verification-needed-
bionic' to 'verification-done-bionic'. If the problem still exists,
change the tag
Verification done on xenial-proposed.
The kernel in -proposed logs the block size change.
The kernel in -updates fails.
xenial-proposed:
---
$ uname -rv
4.4.0-187-generic #217-Ubuntu SMP Tue Jul 21 04:18:15 UTC 2020
$ apt-cache madison linux-image-4.4.0-187-generic
Hi Sebastian, don't worry, we'll use the synthetic reproducer for the
verification steps.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1867916
Title:
Regression in kernel 4.15.0-91 causes kernel
We don’t have any systems with Xenial any longer, so I can’t test there.
In fact, just this week we decommissioned the Bionic-based system where
I originally discovered this bug, so even there, I can’t test it any
longer.
--
You received this bug notification because you are a member of Ubuntu
This bug is awaiting verification that the kernel in -proposed solves
the problem. Please test the kernel and update this bug with the
results. If the problem is solved, change the tag 'verification-needed-
xenial' to 'verification-done-xenial'. If the problem still exists,
change the tag
** Changed in: linux (Ubuntu Eoan)
Status: In Progress => Fix Committed
** Changed in: linux (Ubuntu Xenial)
Status: In Progress => Fix Committed
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
With the regression fix tested, I've gone ahead and fixed the core
problem on my system, switching the backing device to 4k. Documented
below and worked for me, but PLEASE don't take it at face value, since
part of it is literally destroying the bcache header area of the backing
disk.
#
Works on focal, thank you
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1867916
Title:
Regression in kernel 4.15.0-91 causes kernel panic with Bcache
To manage notifications about this bug go to:
Hey Sebastian! Great; thanks for testing!
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1867916
Title:
Regression in kernel 4.15.0-91 causes kernel panic with Bcache
To manage notifications about
Thank you, Mauricio! I can confirm that your test kernel for Bionic
(4.15.0-110.111+lp1867916.1) fixes the problem for me.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1867916
Title:
Regression in
Test kernels with the fix are available in:
https://people.canonical.com/~mfo/lp1867916/
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1867916
Title:
Regression in kernel 4.15.0-91 causes kernel
[X/B/D/E/F][PATCH 0/1] bcache: fix oops for block size > page size
https://lists.ubuntu.com/archives/kernel-team/2020-July/111846.html
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1867916
Title:
Focal / Testing
=
modified
$ uname -rv
5.4.0-41-generic #45+lp1867916.1 SMP Mon Jul 6 16:41:46 -03 2020
$ sudo make-bcache --bdev $DEV --block 8k
[ 29.593270] bcache: bcache_device_init() bcache0: sb/logical block size
(8192) greater than page size (4096) falling back to device
Xenial / Testing
==
modified
$ uname -rv
4.4.0-186-generic #216+lp1867916.1 SMP Mon Jul 6 18:45:47 -03 2020
$ sudo make-bcache --bdev $DEV --block 8k
[ 60.860259] bcache: bcache_device_init() bcache0: sb/logical block size
(8192) greater than page size (4096) falling back to
Test-case
=
echo 9 | sudo tee /proc/sys/kernel/printk # all messages on console
IMG="$HOME/disk.img"
rm -f $IMG
truncate --size 1G $IMG
DEV="$(sudo losetup --find --show $IMG)"
sudo modprobe bcache # just in case
sudo make-bcache --bdev $DEV --block 8k
** Also affects: linux (Ubuntu
Bionic / Testing
==
modified
$ uname -rv
4.15.0-110-generic #111+lp1867916.1 SMP Mon Jul 6 19:09:14 -03 2020
$ sudo make-bcache --bdev $DEV --block 8k
[ 22.066760] bcache: bcache_device_init() bcache0: sb/logical block size
(8192) greater than page size (4096) falling back to
Disco / Testing
=
* Using the linux-hwe-5.0 from "Disco" (EOL) on Bionic for the 5.0
kernel.
modified
$ uname -rv
5.0.0-57-generic #61~18.04.1+lp1867916.1 SMP Mon Jul 6 19:27:05 -03 2020
$ sudo make-bcache --bdev $DEV --block 8k
[ 109.818171] bcache: bcache_device_init() bcache0:
Eoan / Testing
modified
$ uname -rv
5.3.0-63-generic #57+lp1867916.1 SMP Mon Jul 6 18:33:27 -03 2020
$ sudo make-bcache --bdev $DEV --block 8k
[ 29.620685] bcache: bcache_device_init() bcache0: sb/logical block size
(8192) greater than page size (4096) falling back to device
For documentation purposes,
The bcache-specific patch has been sent today.
Waiting on review before providing test kernels.
[PATCH] bcache: check and adjust logical block size for backing devices
https://www.spinics.net/lists/linux-bcache/msg08411.html
cheers,
Mauricio
--
You received this
Hi Sebastian, Ryan,
Thanks for checking and clarifying; it's very helpful.
Sebastian,
Yes, I think that makes sense for the physical block size (but not an expert :)
The problem with bcache is it takes it for logical block size as well,
which turns into that error -- as it cannot be greater
On 6/2/20 10:19 AM, Mauricio Faria de Oliveira wrote:
> Do you remember specifying a block size of 512 kB for 'make-bcache' when
> creating it?
>
> e.g., make-bcache --bdev|-B /dev/mdN --block|-w 512k # or similarly.
My set was created by hand, and it's entirely possible I specified
"--block
Hi Mauricio,
according to my Bash history, I used the following two commands to
create the Bcache device:
make-bcache -B -b 524288 -w 524288 -o 8192 /dev/vg0/vg2-backend
make-bcache -C -b 4194304 -w 4096 -o 8192 /dev/vg1/vg2-cache
I think the reason why I used this block size was that the MD
Sebastian, thanks for the information!
Sebastian, Ryan,
Do you remember specifying a block size of 512 kB for 'make-bcache' when
creating it?
e.g., make-bcache --bdev|-B /dev/mdN --block|-w 512k # or similarly.
Thanks,
Mauricio
--
You received this bug notification because you are a member
Hi Mauricio,
I have attached the requested information.
Best regards,
Sebastian
** Attachment added: "lp1867916-queue_block_size.seb"
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1867916/+attachment/5379837/+files/lp1867916-queue_block_size.seb
--
You received this bug
Hi Ryan and Sebastian,
Thanks for testing! Glad it helps.
This is the patch [1] to address the issue on a general level,
but it had a suggestion to be done at the driver level, which
may or may not interfere with the patch being applied -- both
approaches are not mutually exclusive.
I'm working
Launchpad has imported 1 comments from the remote bug at
https://bugzilla.kernel.org/show_bug.cgi?id=207811.
If you reply to an imported comment from within Launchpad, your comment
will be sent to the remote bug automatically. Read more about
Launchpad's inter-bugtracker facilities at
Hi Mauricio,
I can confirm that your patched kernel fixes the problem for me on
Bionic (kernel 4.15).
Thanks for your help!
-Sebastian
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1867916
Title:
Works on focal!
Linux nibbler 5.4.0-34-generic #38+lp1867916b1 SMP Sun May 31 21:41:06
-03 2020 x86_64 x86_64 x86_64 GNU/Linux
Thanks! I'm curious to see the patch; I tried root causing it myself
and suspected it had to do with something like an overflow in an
unanticipated block size, but
Hi Ryan and Sebastian,
Could you please verify whether these test kernels resolve the problem?
There's 5.4 for Focal/20.04 and 4.15 for Bionic/18.04.
https://people.canonical.com/~mfo/lp1867916/focal/
https://people.canonical.com/~mfo/lp1867916/bionic/
Thank you,
Mauricio
--
You received this
Sure, done.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1867916
Title:
Regression in kernel 4.15.0-91 causes kernel panic with Bcache
To manage notifications about this bug go to:
** Attachment added: "lp1867916-lsblk"
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1867916/+attachment/5378817/+files/lp1867916-lsblk
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1867916
Ryan,
Thanks! That matches the expected output.
There's a bit more that I got curious about,
which is quicker to get with lsblk than in kdump:
$ lsblk --ascii > lp1867916-lsblk 2>&1
cheers,
Mauricio
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is
Done, please see attached. (The sd* devices do tend to move around; the
bcache backing device is currently at sdb1.)
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1867916
Title:
Regression in
** Attachment added: "lp1867916-bcache-super-show.md"
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1867916/+attachment/5378590/+files/lp1867916-bcache-super-show.md
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
** Attachment added: "lp1867916-bcache-super-show.sd"
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1867916/+attachment/5378589/+files/lp1867916-bcache-super-show.sd
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
** Attachment added: "lp1867916-queue_block_size"
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1867916/+attachment/5378588/+files/lp1867916-queue_block_size
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
Hi Ryan,
Thanks. I've found the problem; working on a fix next week.
Could you please provide/upload the output of these commands?
I can get those from the crashdump, but need to double check.
$ sudo grep ^ /sys/block/*/queue/*_block_size >
lp1867916-queue_block_size 2>&1
$ sudo
Here's a crashdump from the working kernel, with everything enabled:
https://www.finnie.org/stuff/lp1867916-crashdump-20200528.tar.xz
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1867916
Title:
The bcache device is raid+bcache, with luks below it (and then lvm), so
it's not directly mountable. Do you just want the stack set up and
functional like normal?
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
Ryan,
Good progress with your crashdump; thanks!
I could root cause the source of the BUG() / NULL pointer address 0x8,
and its relation to the offending commit you identified.
Could you please provide another crashdump, with the _working_ kernel
4.15.0-88,
with the bcache device _mounted_?
Hi Ryan,
Sorry, I couldn't look at this earlier, but can put some cycles this week.
I'm looking at the stack trace, crashdump, and the offending patch.
Thanks again for your work on this.
cheers,
Mauricio
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is
Mike: Sorry, that is not related. I'd suggest filing a new bug.
Mauricio: Any update on this? This also affects the 5.4 line, so after
upgrading to focal I needed to remain on 4.15.0-88.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to
For what it is worth, I have a kernel issue that appears after upgrading from
4.15.0-88 to 4.15.0-91 and it still is present in 4.15.0-96.
For my system the kernel does not panic, however it eventually starts erroring
and then mounts /tmp as ro.
If I reboot in maintenance mode fsck the disk,
Hi Ryan, that's really great findings, thanks.
I'm out on holiday, and will check it next week.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1867916
Title:
Regression in kernel 4.15.0-91 causes
I've bisected the problem down to commit
c35a4a858d0616e7817026d88f377c7201ad449a ("block: fix an integer
overflow in logical block size", upstream
ad6bf88a6c19a39fb3b0045d78ea880325dfcf15).
I don't know what the exact problem is with the commit, but seems to be
in the area of fs/block_dev.c
** Tags added: seg
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1867916
Title:
Regression in kernel 4.15.0-91 causes kernel panic with Bcache
To manage notifications about this bug go to:
BTW, for future searchers, I've uploaded dmesg.202004081903 separately,
and pasted the crash here:
[ 194.36] bcache: bch_journal_replay() journal replay done, 3 keys in 6
entries, seq 23285862
[ 194.444622] bcache: register_cache() registered cache device sdb1
[ 194.448381] bcache:
Hi Ryan,
Thanks a bunch for the crashdump; that helps.
I can confirm there's a valid and matching stack trace in the dmesg
file.
I'll take a look.
cheers,
Mauricio
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
Here you go: https://www.finnie.org/stuff/lp1867916-crashdump.tar.xz
(138MB)
Some notes on the process:
- Also blacklisted it87 (DKMS) so the running kernel wasn't "tainted"
- Also disabled the relevant crypttab entry for this group
- 768M produced "crashkernel reservation failed - No suitable
Ryan,
Part 1)
--
First, please try to reproduce the problem later, not so early in boot,
by disabling the bcache module on the kernel boot parameters, and then
loading it after the system has booted successfully.
(This should be possible as you mentioned the boot disk isn't involved.)
1)
Hi Sebastian and Ryan,
I've created the setup that each of you described on a VM,
but unfortunately it wasn't able to reproduce the problem.
I double checked the output of lsblk with Sebastian's and
it's the same (also the bcache sysfs configuration/status),
and with Ryan's description, and both
I can confirm going from 4.15.0-88 to 4.15.0-91 on my bcache system
panics in the same way. Here's my layout:
-> sd{c,d,f,g,h}: each 4TB gpt, sd{c,d,f,g,h}1: each type linux_raid_member
--> md0: raid6, sd{c,d,f,g,h}1
--> sda: 512GB gpt, sda1: type bcache
---> bcache0: md0 + sda1
>
Hi Sebastian,
Thanks for collecting the data and describing the block device topology,
that's very helpful.
I'll try to reproduce the problem and analyze it.
cheers,
Mauricio
** Changed in: linux (Ubuntu)
Assignee: (unassigned) => Mauricio Faria de Oliveira (mfo)
** Changed in: linux
Hi Mauricio,
thanks for your support. I have attached the requested information to
the bug. I had to pseudomize some of the logical volume names because
they are considered sensitive, but I did so in a consistent way so the
structure should still be clear.
So that you don’t have to look through
** Attachment added: "block device hierarchy"
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1867916/+attachment/5347771/+files/lp1867916.1.out
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
Hi Sebastian,
Sure, that's certainly understandable.
For starters, the topology of the block/bcache devices would do it:
(set -x; lsblk; sudo dmsetup table; grep -r ^ /sys/block/bcache*/bcache)
> lp1867916.1.out 2>&1
Then please upload the output file (or copy/paste, if file access is
though.)
Hi Mauricio,
there is the problem that this tarball contains an awful lot of
sensitive data, some of that personal data that is protected under the
GDPR.
Maybe you could tell me which specific parts / sections you are
interested in, so that I can anonymize and upload them.
Thanks,
Sebastian
--
Hi Sebastian,
Thanks for reporting this bug.
It indeed seems related to bcache per the stack trace in the screenshot.
Could you please collect and upload a sosreport of the failing system
(w/ any kernel version.)
$ sudo sosreport --batch --case-id lp1867916
This should generate a
68 matches
Mail list logo
