[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[LouieGosselin]

I'd like to follow up because the issue seems to have cleared up for us
after installing linux 5.0.1 about 40 days ago. It's hard to say whether
everyone is experiencing the same bugs, but give 5.x a shot and let us
know how it goes!


Just to recap. Every week or so we were seeing R/O file systems with the follow 
errors, which required reboot & fsck.

EXT4-fs error (device vda2): ext4_mb_generate_buddy:757: group 144, block 
bitmap and bg descriptor inconsistent: 23914 vs 23913 free clusters
Aborting journal on device vda2-8.
EXT4-fs (vda2): Remounting filesystem read-only

We never experienced any corruption on the host itself, only under KVM
guests.

Host DELL Poweredge 2950III
Several KVM Guests: linux OS, distro doesn't make any difference, all 
randomly vulnerable during periods of high disk activity.

Not sure it matters, but in our case we were using LVM2 volumes on the
host and kvm media was configured as follows
"media=disk,if=virtio,cache=none,aio=native,format=raw".

We initially thought just one guest was affected, but over time we saw
it happen with many distros and kernels. It wasn't until we had an
extended period of downtime that we decided to reinstall the host with a
5.x kernel. None of the guests experienced any issues since, fingers
crossed.


At this point, it's hard to recommend Ubuntu 19.04 given that it's only a few 
months away from EOL, however the 5.x kernel seems promising whereas the Ubuntu 
18.04LTS runs an older kernel that is still known to exhibit the corruption. 
For LTS I'd look into running it under a custom setup with a newer kernel.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Brad Figg]

** Tags added: cscc

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Bug Watch Updater]

Launchpad has imported 7 comments from the remote bug at
https://bugzilla.kernel.org/show_bug.cgi?id=104571.

If you reply to an imported comment from within Launchpad, your comment
will be sent to the remote bug automatically. Read more about
Launchpad's inter-bugtracker facilities at
https://help.launchpad.net/InterBugTracking.


On 2015-09-15T08:10:58+00:00 linux-ext4 wrote:

This bug report is about ext4 metadata corruption on large (>=10TB) ext4 
volumes.
This was also reported in 2014 [ 
http://marc.info/?l=linux-ext4=139878494527370=2 ]

I'm getting sporadic FS errors like this one:
(More of these i've pasted at https://8n1.org/10745/cc34)

|  EXT4-fs error (device vdb): ext4_mb_generate_buddy:757:
|   group 79842, block bitmap and bg descriptor inconsistent: 10073 vs 10071
|   free clusters
| Aborting journal on device vdb-8.

An e2fsck run then shows:
| Pass 5: Checking group summary information
| Block bitmap differences:  +(2616281446--2616281447)
| Free blocks count wrong (170942497, counted=129906218).
| Free inodes count wrong (670863012, counted=670860975).

I've patched my kernel with WARN_ON(1); inserted in tactical places and caught
one such situation:

| EXT4-fs (vdb): pa 880016544888: logic 982168, phys. 2469410748, len 104
| EXT4-fs error (device vdb): ext4_mb_release_inode_pa:3773: group 75360, free 
38, pa_free 36
| Aborting journal on device vdb-8.
| EXT4-fs (vdb): Remounting filesystem read-only
| [ cut here ]
| WARNING: CPU: 1 PID: 1706 at fs/ext4/mballoc.c:3774 
ext4_mb_release_inode_pa.isra.27+0x1cb/0x2c0()
| Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter xt_tcpudp 
ip6_tables
| nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter 
ip_tables
| x_tables cirrus ttm drm_kms_helper drm kvm_intel kvm ppdev syscopyarea 
sysfillrect
| 8250_fintek serio_raw i2c_piix4 sysimgblt pvpanic parport_pc mac_hid nfsd 
auth_rpcgss nfs_acl
| lockd grace sunrpc lp parport autofs4 psmouse floppy pata_acpi
| CPU: 1 PID: 1706 Comm: deluged Not tainted 3.19.8-ckt4 #1
| Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
|  81ab4fef 8800da1bb978 817c3760 0007
|   8800da1bb9b8 8107696a 8800da1bb9a8
|  0026 3825 3824 880016544888
| Call Trace:
|  [] dump_stack+0x45/0x57
|  [] warn_slowpath_common+0x8a/0xc0
|  [] warn_slowpath_null+0x1a/0x20
|  [] ext4_mb_release_inode_pa.isra.27+0x1cb/0x2c0
|  [] ? ext4_read_block_bitmap_nowait+0x26f/0x5f0
|  [] ext4_discard_preallocations+0x30a/0x490
|  [] ext4_da_update_reserve_space+0x178/0x1b0
|  [] ext4_ext_map_blocks+0xcd9/0xe50
|  [] ext4_map_blocks+0x129/0x570
|  [] ? ext4_writepages+0x35d/0xca0
|  [] ? __ext4_journal_start_sb+0x69/0xe0
|  [] ext4_writepages+0x582/0xca0
|  [] do_writepages+0x1e/0x30
|  [] __filemap_fdatawrite_range+0x59/0x60
|  [] filemap_write_and_wait+0x2c/0x60
|  [] do_vfs_ioctl+0x3fd/0x4e0
|  [] SyS_ioctl+0x81/0xa0
|  [] system_call_fastpath+0x16/0x1b
| ---[ end trace c7de4d0d78cb95b6 ]---
| EXT4-fs error (device vdb) in ext4_writepages:2412: IO failure
| EXT4-fs (vdb): ext4_writepages: jbd2_start: 9223372036854775751 pages, ino 
84149503; err -30

After this, the system started logging a lot of this same message:
| EXT4-fs error (device vdb): ext4_find_extent:900: inode #84149503: comm 
deluged: 
|pblk 225181822 bad header/extent: invalid magic - magic 53fd, entries 
37907,
|max 27407(0), depth 50401(0)

Ran e2fsck and got:
| Pass 5: Checking group summary information
| Block bitmap differences:   +(1431556444--1431556445) +(2469410748-2469410749)
| Free blocks count wrong (134030133, counted=57970467).
| Free inodes count wrong (670746893, counted=670746452).

Which is usually the same output for fsck in these situations.

This server is a QEMU KVM virtual machine running on Intel x64 hardware.

Reply at:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/comments/34


On 2015-09-28T08:04:48+00:00 linux-ext4 wrote:

Hit by this bug again:

| EXT4-fs error (device vdb): ext4_mb_generate_buddy:757: group 76916,
|block bitmap and bg descriptor inconsistent: 959 vs 957 free
|clusters
| Aborting journal on device vdb-8.
| EXT4-fs (vdb): Remounting filesystem read-only
| [ cut here ]
| WARNING: CPU: 0 PID: 5000 at fs/ext4/mballoc.c:758
|ext4_mb_generate_buddy+0x1fa/0x340()
| Modules linked in: btrfs xor raid6_pq ufs qnx4 hfsplus hfs minix ntfs
|msdos jfs xfs libcrc32c xt_tcpudp nf_conntrack_ipv6 nf_defrag_ipv6
|ip6table_filter ip6_tables nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack
|nf_conntrack iptable_filter ip_tables x_tables cirrus ttm drm_kms_helper
|drm kvm_intel kvm ppdev syscopyarea sysfillrect serio_raw pvpanic
|sysimgblt 8250_fintek 

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Dominik Psenner]

Unfortunately the symptom still persists on bionic (Ubuntu 18.04.1 LTS)
with kernel 4.15.0-43-generic. This issue may be a symptom that's caused
by lice and fleas. I updated
https://bugzilla.kernel.org/show_bug.cgi?id=104571 by adding a comment
with additional information.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Simon Déziel]

Chenyuchai, yes,
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?h=7dec5603b6b8dc4c3e1c65d318bd2a5a8c62a424
is the fix. It was integrated in the Trusty kernel 3.13.0-87.133

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[chenyuchai]

Hi，all:
  I need your confirmation.Is the Fix: 
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?h=7dec5603b6b8dc4c3e1c65d318bd2a5a8c62a424
 to solve this issue?If not,pls show the patch,thank you very much!

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Joseph Salisbury]

** Tags removed: verification-needed-trusty
** Tags added: verification-done-trusty

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Bug Watch Updater]

** Changed in: linux (Debian)
   Status: Confirmed => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Launchpad Bug Tracker]

This bug was fixed in the package linux-lts-utopic -
3.16.0-73.95~14.04.1

---
linux-lts-utopic (3.16.0-73.95~14.04.1) trusty; urgency=low

  [ Kamal Mostafa ]

  * CVE-2016-1583 (LP: #1588871)
- ecryptfs: fix handling of directory opening
- SAUCE: proc: prevent stacking filesystems on top
- SAUCE: ecryptfs: forbid opening files without mmap handler

 -- Andy Whitcroft   Thu, 09 Jun 2016 08:46:24 +0100

** Changed in: linux-lts-utopic (Ubuntu Trusty)
   Status: Fix Committed => Fix Released

** CVE added: http://www.cve.mitre.org/cgi-
bin/cvename.cgi?name=2016-1583

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Launchpad Bug Tracker]

This bug was fixed in the package linux - 3.13.0-87.133

---
linux (3.13.0-87.133) trusty; urgency=low

  [ Kamal Mostafa ]

  * Release Tracking Bug
- LP: #1585315

  [ Upstream Kernel Changes ]

  * Revert "usb: hub: do not clear BOS field during reset device"
- LP: #1582864

linux (3.13.0-87.132) trusty; urgency=low

  [ Kamal Mostafa ]

  * Release Tracking Bug
- LP: #1582398

  [ Kamal Mostafa ]

  * [Config] Drop ozwpan from the ABI

  [ Luis Henriques ]

  * [Config] CONFIG_USB_WPAN_HCD=n
- LP: #1463740
- CVE-2015-4004

  [ Prarit Bhargava ]

  * SAUCE: (no-up) ACPICA: Dispatcher: Update thread ID for recursive
method calls
- LP: #1577898

  [ Upstream Kernel Changes ]

  * usbnet: cleanup after bind() in probe()
- LP: #1567191
- CVE-2016-3951
  * KVM: x86: bit-ops emulation ignores offset on 64-bit
- LP: #1423672
  * USB: usbip: fix potential out-of-bounds write
- LP: #1572666
- CVE-2016-3955
  * x86/mm/32: Enable full randomization on i386 and X86_32
- LP: #1568523
- CVE-2016-3672
  * Input: gtco - fix crash on detecting device without endpoints
- LP: #1575706
- CVE-2016-2187
  * atl2: Disable unimplemented scatter/gather feature
- LP: #1561403
- CVE-2016-2117
  * ALSA: usb-audio: Skip volume controls triggers hangup on Dell USB Dock
- LP: #1577905
  * fs/pnode.c: treat zero mnt_group_id-s as unequal
- LP: #1572316
  * propogate_mnt: Handle the first propogated copy being a slave
- LP: #1572316
  * drm: Balance error path for GEM handle allocation
- LP: #1579610
  * x86/mm: Add barriers and document switch_mm()-vs-flush synchronization
- LP: #1538429
- CVE-2016-2069
  * x86/mm: Improve switch_mm() barrier comments
- LP: #1538429
- CVE-2016-2069
  * net: fix infoleak in llc
- LP: #1578496
- CVE-2016-4485
  * net: fix infoleak in rtnetlink
- LP: #1578497
- CVE-2016-4486

 -- Kamal Mostafa   Tue, 24 May 2016 11:04:30 -0700

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Launchpad Bug Tracker]

This bug was fixed in the package linux - 3.13.0-87.133

---
linux (3.13.0-87.133) trusty; urgency=low

  [ Kamal Mostafa ]

  * Release Tracking Bug
- LP: #1585315

  [ Upstream Kernel Changes ]

  * Revert "usb: hub: do not clear BOS field during reset device"
- LP: #1582864

linux (3.13.0-87.132) trusty; urgency=low

  [ Kamal Mostafa ]

  * Release Tracking Bug
- LP: #1582398

  [ Kamal Mostafa ]

  * [Config] Drop ozwpan from the ABI

  [ Luis Henriques ]

  * [Config] CONFIG_USB_WPAN_HCD=n
- LP: #1463740
- CVE-2015-4004

  [ Prarit Bhargava ]

  * SAUCE: (no-up) ACPICA: Dispatcher: Update thread ID for recursive
method calls
- LP: #1577898

  [ Upstream Kernel Changes ]

  * usbnet: cleanup after bind() in probe()
- LP: #1567191
- CVE-2016-3951
  * KVM: x86: bit-ops emulation ignores offset on 64-bit
- LP: #1423672
  * USB: usbip: fix potential out-of-bounds write
- LP: #1572666
- CVE-2016-3955
  * x86/mm/32: Enable full randomization on i386 and X86_32
- LP: #1568523
- CVE-2016-3672
  * Input: gtco - fix crash on detecting device without endpoints
- LP: #1575706
- CVE-2016-2187
  * atl2: Disable unimplemented scatter/gather feature
- LP: #1561403
- CVE-2016-2117
  * ALSA: usb-audio: Skip volume controls triggers hangup on Dell USB Dock
- LP: #1577905
  * fs/pnode.c: treat zero mnt_group_id-s as unequal
- LP: #1572316
  * propogate_mnt: Handle the first propogated copy being a slave
- LP: #1572316
  * drm: Balance error path for GEM handle allocation
- LP: #1579610
  * x86/mm: Add barriers and document switch_mm()-vs-flush synchronization
- LP: #1538429
- CVE-2016-2069
  * x86/mm: Improve switch_mm() barrier comments
- LP: #1538429
- CVE-2016-2069
  * net: fix infoleak in llc
- LP: #1578496
- CVE-2016-4485
  * net: fix infoleak in rtnetlink
- LP: #1578497
- CVE-2016-4486

 -- Kamal Mostafa   Tue, 24 May 2016 11:04:30 -0700

** Changed in: linux (Ubuntu Trusty)
   Status: Fix Committed => Fix Released

** CVE added: http://www.cve.mitre.org/cgi-
bin/cvename.cgi?name=2015-4004

** CVE added: http://www.cve.mitre.org/cgi-
bin/cvename.cgi?name=2016-2069

** CVE added: http://www.cve.mitre.org/cgi-
bin/cvename.cgi?name=2016-2117

** CVE added: http://www.cve.mitre.org/cgi-
bin/cvename.cgi?name=2016-2187

** CVE added: http://www.cve.mitre.org/cgi-
bin/cvename.cgi?name=2016-3672

** CVE added: http://www.cve.mitre.org/cgi-
bin/cvename.cgi?name=2016-3951

** CVE added: http://www.cve.mitre.org/cgi-
bin/cvename.cgi?name=2016-3955

** CVE added: http://www.cve.mitre.org/cgi-
bin/cvename.cgi?name=2016-4485

** CVE added: http://www.cve.mitre.org/cgi-
bin/cvename.cgi?name=2016-4486

** Changed in: linux (Ubuntu Trusty)
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Simon Déziel]

Louie, https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=818502#22
provides a simple way to reproduce. If you could give it a try that
would be appreciated.

FYI, when my hypervisor was running Trusty (3.13), the problem was
reproducible on fresh VMs with brand new ext4 FSes, so hopefully that
will be easy to reproduce for you.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[LouieGosselin]

I have a "PE 2950III Intel(R) Xeon(R) CPU X5460 @ 3.16GHz" server here
and I've been trying to test this out. I'm using an "rsync" copy of an
original server exhibiting the problem. So far though I've been unable
to reproduce the original error at all.

It would seem that using the exact same OS/kernel/binaries, the error
doesn't happen on a fresh filesystem, I guess there must have been
something about the filesystem image itself that triggered the fault. So
my dilemma is that I don't know how to reproduce this fault on a fresh
install. So while I can test this update, I'm not sure how valid the
test will be on an installation that isn't faulting.

Does anyone have a suggestion or have an idea about how to reproduce the
conditions?

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Kamal Mostafa]

Anyone affected by this issue:  We're looking for a positive
verification that the Trusty kernel version currently available in
-proposed (3.13.0-87.132) fixes this problem.  If you can provide that
confirmation, please do!

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Kamal Mostafa]

This bug is awaiting verification that the kernel in -proposed solves
the problem. Please test the kernel and update this bug with the
results. If the problem is solved, change the tag 'verification-needed-
trusty' to 'verification-done-trusty'.

If verification is not done by 5 working days from today, this fix will
be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how
to enable and use -proposed. Thank you!


** Tags added: verification-needed-trusty

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Simon Déziel]

I recently reinstalled the affected host to run Xenial so I can no
longer test the proposed fix for the 3.13 kernel.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Tim Gardner]

** Changed in: linux (Ubuntu Trusty)
   Status: Confirmed => Fix Committed

** Changed in: linux-lts-utopic (Ubuntu Trusty)
   Status: New => Fix Committed

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Kamal Mostafa]

Additional positive test result notes:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=818502#27
https://bugzilla.kernel.org/show_bug.cgi?id=102731#c43

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Chris J Arges]

** Description changed:

+  SRU Justification:
+ 
+ Impact: Users of VMs running 3.13/3.16 and ext4 can experience data 
corruption in the guest.
+ Fix: 
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?h=7dec5603b6b8dc4c3e1c65d318bd2a5a8c62a424
+ Testcase: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=818502#22
+ 
+ --
+ 
  I noticed that one of my VM had this "dmesg -T" output:
  
  [Tue Feb 17 09:53:27 2015] systemd-udevd[5433]: starting version 204
  [Thu Feb 19 06:25:08 2015] EXT4-fs error (device vda1): 
ext4_mb_generate_buddy:756: group 5, block bitmap and bg descriptor 
inconsistent: 16446 vs 16445 free clusters
  [Thu Feb 19 06:25:09 2015] Aborting journal on device vda1-8.
  [Thu Feb 19 06:25:09 2015] EXT4-fs (vda1): Remounting filesystem read-only
  [Thu Feb 19 06:25:09 2015] [ cut here ]
  [Thu Feb 19 06:25:09 2015] WARNING: CPU: 0 PID: 9946 at 
/build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 
__ext4_handle_dirty_metadata+0x1a2/0x1c0()
  [Thu Feb 19 06:25:09 2015] Modules linked in: nf_conntrack_ipv6 
nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 
nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables 
x_tables serio_raw psmouse floppy
  [Thu Feb 19 06:25:09 2015] CPU: 0 PID: 9946 Comm: logrotate Not tainted 
3.13.0-45-generic #74-Ubuntu
  [Thu Feb 19 06:25:09 2015] Hardware name: QEMU Standard PC (i440FX + PIIX, 
1996), BIOS Bochs 01/01/2011
  [Thu Feb 19 06:25:09 2015]  0009 880003a11aa0 
81720eb6 
  [Thu Feb 19 06:25:09 2015]  880003a11ad8 810677cd 
88c41340 
  [Thu Feb 19 06:25:09 2015]  88000a58e000 81835280 
1302 880003a11ae8
  [Thu Feb 19 06:25:09 2015] Call Trace:
  [Thu Feb 19 06:25:09 2015]  [] dump_stack+0x45/0x56
  [Thu Feb 19 06:25:09 2015]  [] 
warn_slowpath_common+0x7d/0xa0
  [Thu Feb 19 06:25:09 2015]  [] warn_slowpath_null+0x1a/0x20
  [Thu Feb 19 06:25:09 2015]  [] 
__ext4_handle_dirty_metadata+0x1a2/0x1c0
  [Thu Feb 19 06:25:09 2015]  [] ? ext4_dirty_inode+0x2a/0x60
  [Thu Feb 19 06:25:09 2015]  [] ext4_free_blocks+0x646/0xbf0
  [Thu Feb 19 06:25:09 2015]  [] ? wake_up_bit+0x25/0x30
  [Thu Feb 19 06:25:09 2015]  [] ext4_ext_rm_leaf+0x505/0x8f0
  [Thu Feb 19 06:25:09 2015]  [] ? 
__ext4_ext_check+0x197/0x370
  [Thu Feb 19 06:25:09 2015]  [] ? 
ext4_ext_remove_space+0xc0/0x7e0
  [Thu Feb 19 06:25:09 2015]  [] 
ext4_ext_remove_space+0x31c/0x7e0
  [Thu Feb 19 06:25:09 2015]  [] ext4_ext_truncate+0xb0/0xe0
  [Thu Feb 19 06:25:09 2015]  [] ext4_truncate+0x379/0x3c0
  [Thu Feb 19 06:25:09 2015]  [] ext4_evict_inode+0x408/0x4d0
  [Thu Feb 19 06:25:09 2015]  [] evict+0xb0/0x1b0
  [Thu Feb 19 06:25:09 2015]  [] iput+0xf5/0x180
  [Thu Feb 19 06:25:09 2015]  [] __dentry_kill+0x1a8/0x200
  [Thu Feb 19 06:25:09 2015]  [] dput+0xa5/0x180
  [Thu Feb 19 06:25:09 2015]  [] __fput+0x176/0x260
  [Thu Feb 19 06:25:09 2015]  [] fput+0xe/0x10
  [Thu Feb 19 06:25:09 2015]  [] task_work_run+0xa7/0xe0
  [Thu Feb 19 06:25:09 2015]  [] do_notify_resume+0x97/0xb0
  [Thu Feb 19 06:25:09 2015]  [] int_signal+0x12/0x17
  [Thu Feb 19 06:25:09 2015] ---[ end trace ebff9843d81b5c41 ]---
  [Thu Feb 19 06:25:09 2015] BUG: unable to handle kernel NULL pointer 
dereference at 0028
  [Thu Feb 19 06:25:09 2015] IP: [] 
__ext4_error_inode+0x31/0x160
- [Thu Feb 19 06:25:09 2015] PGD 167067 PUD 161067 PMD 0 
- [Thu Feb 19 06:25:09 2015] Oops:  [#1] SMP 
+ [Thu Feb 19 06:25:09 2015] PGD 167067 PUD 161067 PMD 0
+ [Thu Feb 19 06:25:09 2015] Oops:  [#1] SMP
  [Thu Feb 19 06:25:09 2015] Modules linked in: nf_conntrack_ipv6 
nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 
nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables 
x_tables serio_raw psmouse floppy
  [Thu Feb 19 06:25:09 2015] CPU: 0 PID: 9946 Comm: logrotate Tainted: G
W 3.13.0-45-generic #74-Ubuntu
  [Thu Feb 19 06:25:09 2015] Hardware name: QEMU Standard PC (i440FX + PIIX, 
1996), BIOS Bochs 01/01/2011
  [Thu Feb 19 06:25:09 2015] task: 880009ac4800 ti: 880003a1 
task.ti: 880003a1
  [Thu Feb 19 06:25:09 2015] RIP: 0010:[]  
[] __ext4_error_inode+0x31/0x160
  [Thu Feb 19 06:25:09 2015] RSP: :880003a11a58  EFLAGS: 00010292
  [Thu Feb 19 06:25:09 2015] RAX:  RBX:  RCX: 
0086
  [Thu Feb 19 06:25:09 2015] RDX: 1302 RSI: 81a6e81f RDI: 

  [Thu Feb 19 06:25:09 2015] RBP: 880003a11ae8 R08: 81a78568 R09: 
0005
  [Thu Feb 19 06:25:09 2015] R10: ffe2 R11: 880003a117ce R12: 
0086
  [Thu Feb 19 06:25:09 2015] R13: 81835280 R14: 1302 R15: 
81a78568
  [Thu Feb 19 06:25:09 2015] FS:  7f74eaca4840() 
GS:88000b80() knlGS:
  [Thu Feb 19 06:25:09 2015] CS:  0010 DS:  

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Chris J Arges]

** Changed in: linux (Ubuntu Trusty)
 Assignee: (unassigned) => Chris J Arges (arges)

** Changed in: linux-lts-utopic (Ubuntu Trusty)
 Assignee: (unassigned) => Chris J Arges (arges)

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Simon Déziel]

Thanks Chris. So far, no regression with the Trusty kernel:

$ uname -a
Linux xeon 3.13.0-86-generic #130~lp1423672v201604200743 SMP Wed Apr 20 
12:44:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Since the issue only happens rarely, more people testing it would be
welcome.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Chris J Arges]

Leo,
Also uploaded a lts-utopic build here:
http://people.canonical.com/~arges/lp1423672/

It will all debs with '3.16' in the name.

** Also affects: linux-lts-utopic (Ubuntu)
   Importance: Undecided
   Status: New

** Changed in: linux-lts-utopic (Ubuntu)
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Leonardo Borda]

Hi Chris,

This is also seen on kernel 3.16.0-51-generic. Could you get us a kernel
build test for 3.16 as well ?

Thank you
Leo

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Chris J Arges]

Here's a test build for trusty 3.13 with
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?h=7dec5603b6b8dc4c3e1c65d318bd2a5a8c62a424
applied:

http://people.canonical.com/~arges/lp1423672/

Can someone verify this does fix the issue so this can be SRU'ed into
3.13?

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[LouieGosselin]

Good work.

We also have PE2950III systems running "Intel(R) Xeon(R) CPU X5460  @
3.16GHz".

If this is indeed the fix, I'm confused why it would only affect certain cpus?
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?h=7dec5603b6b8dc4c3e1c65d318bd2a5a8c62a424

I'll have to come up with a plan to replace debian's stable/jessie
kernel with an unmanaged one on the host. I'm not keen on doing that as
the DRAC units on these are not very reliable...

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Emmanuel Lacour]

Here we have this problem on  two servers runnning "Intel(R) Xeon(R) CPU
X5460  @ 3.16GHz" (and debian 7.0 backport kernel 3.16 on host, 4.x on
VM). We doesn't seems to have this problem on others nodes runnning:

Intel(R) Xeon(R) CPU   E5345  @ 2.33GHz
Intel(R) Xeon(R) CPU5160  @ 3.00GHz
Intel(R) Xeon(R) CPU   E5320  @ 1.86GHz
Intel(R) Xeon(R) CPU   X3470  @ 2.93GHz

We are going to try an upgrade to see if it solves the problem. If yes,
it's a pretty good news!

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Simon Déziel]

Thanks Václav, your conclusion about older Intel CPU seems to match my
setup since this only happens on a  Xeon E3110 (which is in fact a re-
branded Core2 Duo E8400).

Thanks for bisecting this and figure the fix was:

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?h=7dec5603b6b8dc4c3e1c65d318bd2a5a8c62a424

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Simon Déziel]

** Changed in: linux (Ubuntu)
   Status: Incomplete => Confirmed

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Václav Ovsík]

This bug is probably what I describe in
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=818502

Bug in Debian 3.16 kernel (kernel from current stable Jessie) used as
KVM hypervisor on older Intel CPU.


** Bug watch added: Debian Bug tracker #818502
   http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=818502

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[nunyaz info]

I am able to reproduce this bug every single time I suspend and then
resume one of my laptops. Which is running xubun tu 15.10 with
4.2.0-16-generic kernel on a lenovo R400. I suspect possible a hardware
problem as this only happens on 1 out of 3 R400 that I have. I'll report
back if I remember.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Udo Giacomozzi]

FYI, the same errors happened to me on real hardware (a Raspberry Pi 2
B), based on a custom Debian Jessie image.

The image uses a ext2 filesystem created on a x86 Boot2docker host
(Kernel "Linux fbd0c1340061 4.1.12-boot2docker #1 SMP Tue Nov 3 06:03:36
UTC 2015 x86_64 GNU/Linux").

While remounting the filesystem as r-w on the Raspberry and writing to
it I got:

[53406.370524] EXT4-fs (mmcblk0p3): mounting ext2 file system using the ext4 
subsystem
[53406.575883] EXT4-fs (mmcblk0p3): mounted filesystem without journal. Opts: 
(null)
[53416.394833] EXT4-fs error (device mmcblk0p3): ext4_mb_generate_buddy:757: 
group 1, block bitmap and bg descriptor inconsistent: 8953 vs 8990 free clusters
[53435.245967] EXT4-fs error (device mmcblk0p3): ext4_lookup:1417: inode #2: 
comm rsync: deleted inode referenced: 46849
[53481.805006] EXT4-fs error (device mmcblk0p3): ext4_lookup:1417: inode #2: 
comm ls: deleted inode referenced: 46849

The Raspi isn't running the most current Kernel yet (it's "Linux
intermodul 3.19.3-v7 #1 SMP PREEMPT Mon Nov 30 08:37:00 UTC 2015 armv7l
GNU/Linux"), but perhaps this helps analyzing this bug as it's not a
VM...

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[LouieGosselin]

Oops...the above kvm command line is correct but it did not crash with -m 1000, 
that's what production is using now.
It was crashing consistently with  -m 512 about a minute into the synthetic FS 
load.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[LouieGosselin]

chris,

Here is what you asked for, sorry for not getting it earlier.

I don't use virsh. This is how I started KVM to trigger the problem
interactively (curses interface):

kvm -drive
file=/dev/raid/shared,media=disk,if=none,cache=none,aio=native,format=raw,id=hd0
-device virtio-blk-pci,drive=hd0 -smp 2 -m 1000 -netdev
tap,ifname=vm_shared,script=no,downscript=no,id=eth0 -device virtio-net-
pci,netdev=eth0,mac=52:54:00:12:34:58 -name shared -runas shared -curses


fdisk -l
Disk /dev/vda: 100 GiB, 107374182400 bytes, 209715200 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: AE4BDF3E-0B83-4C17-B104-A5139722F263

Device   Start   End   Sectors  Size Type
/dev/vda1 2048   3905535   3903488  1.9G Linux swap
/dev/vda2  3905536 209713151 205807616 98.1G Linux filesystem


It hasn't happened in this particular VM since upping the RAM so the VM doesn't 
swap.

My intention was to reproduce on non-production hardware, and then try
different kernels, rule out LVM, virtio, etc.  But I'm in the middle of
a new assignment, I probably won't have time to do this myself before
December.


** Attachment added: "/boot/config-3.16.0-4-amd64 for the VM in question"
   
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+attachment/4518472/+files/config.txt

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Bug Watch Updater]

** Changed in: linux (Debian)
   Status: Unknown => Confirmed

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Jan Wagner]

** Bug watch added: Debian Bug tracker #772848
   http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=772848

** Changed in: linux (Debian)
   Importance: Undecided => Unknown

** Changed in: linux (Debian)
   Status: New => Unknown

** Changed in: linux (Debian)
 Remote watch: None => Debian Bug tracker #772848

** Also affects: linux-kernel-headers via
   http://bugzilla.kernel.org/show_bug.cgi?id=104571
   Importance: Unknown
   Status: Unknown

** Project changed: linux-kernel-headers => linux

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Jan Wagner]

Looks like there are some more LKBT issues for this one:

https://bugzilla.kernel.org/show_bug.cgi?id=102731
https://bugzilla.kernel.org/show_bug.cgi?id=89621

** Bug watch added: Linux Kernel Bug Tracker #102731
   http://bugzilla.kernel.org/show_bug.cgi?id=102731

** Bug watch added: Linux Kernel Bug Tracker #89621
   http://bugzilla.kernel.org/show_bug.cgi?id=89621

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[BlueT - Matthew Lien - 練喆明]

I'm running LoCo mirror for my country and this affects our file server for 
over a week already.
All our official file services got offline now.
We're also providing free VM/Hosting for OpenSource projects, I can't image how 
the disaster will become.

Any suggestion?


All VMs are VMware guest.

root@ncnu-ftp:/ubuntu/mirror# LANG=C free -m
 total   used   free sharedbuffers cached
Mem:  7984   1453   6531  0 66352
-/+ buffers/cache:   1033   6950
Swap: 4091  0   4091


root@ncnu-ftp:/ubuntu/mirror# LANG=C touch /lala
touch: cannot touch '/lala': Read-only file system

root@ncnu-ftp:/ubuntu/mirror# LANG=C mount -o remount,rw /
mount: cannot remount block device /dev/mapper/ub--ftp-root read-write, is 
write-protected

root@ncnu-ftp:/ubuntu/mirror# dumpe2fs /dev/mapper/ub--ftp-root /dev/|grep -i 
Filesystem\ state
dumpe2fs 1.42.9 (4-Feb-2014)
Filesystem state: clean with errors

root@ncnu-ftp:/ubuntu/mirror# LC_ALL=C dmesg -T |grep error
[Thu Nov  5 00:35:46 2015] EXT4-fs (dm-0): re-mounted. Opts: errors=remount-ro
[Thu Nov  5 00:35:55 2015] EXT4-fs error (device dm-0): 
ext4_mb_generate_buddy:756: group 371, block bitmap and bg descriptor 
inconsistent: 25478 vs 25477 free clusters
[Thu Nov  5 00:35:55 2015] IP: [] 
__ext4_error_inode+0x31/0x160
[Thu Nov  5 00:35:55 2015] RIP: 0010:[]  [] 
__ext4_error_inode+0x31/0x160
[Thu Nov  5 00:35:55 2015] RIP  [] 
__ext4_error_inode+0x31/0x160
[Thu Nov  5 00:35:55 2015] EXT4-fs error (device dm-0): ext4_remount:4816: 
Abort forced by user
[Thu Nov  5 00:40:39 2015] EXT4-fs (dm-0): error count since last fsck: 15
[Thu Nov  5 00:40:39 2015] EXT4-fs (dm-0): initial error at time 1445904785: 
ext4_reserve_inode_write:4928
[Thu Nov  5 00:40:39 2015] EXT4-fs (dm-0): last error at time 1446654956: 
ext4_remount:4816
[Thu Nov  5 01:18:38 2015] EXT4-fs error (device dm-0): ext4_remount:4816: 
Abort forced by user
[Thu Nov  5 01:18:59 2015] EXT4-fs error (device dm-0): ext4_remount:4816: 
Abort forced by user
[Thu Nov  5 01:28:59 2015] EXT4-fs error (device dm-0): ext4_remount:4816: 
Abort forced by user
[Thu Nov  5 01:58:08 2015] EXT4-fs error (device dm-0): ext4_remount:4816: 
Abort forced by user
[Thu Nov  5 02:20:41 2015] EXT4-fs error (device dm-0): ext4_remount:4816: 
Abort forced by user

root@ncnu-ftp:/ubuntu/mirror# uname -a
Linux ncnu-ftp 3.13.0-63-generic #103-Ubuntu SMP Fri Aug 14 21:42:59 UTC 2015 
x86_64 x86_64 x86_64 GNU/Linux

root@ncnu-ftp:/ubuntu/mirror# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:Ubuntu 14.04.3 LTS
Release:14.04
Codename:   trusty

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Re: [Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Jan Wagner]

Am 29.10.15 um 03:29 schrieb LouieGosselin:
> I'm posting again to add that I conducted some more tests and ext3 does
> not encounter corruption under the same conditions. I hope this
> information is helpful to others, if anyone needs more information let
> me know and I'll see what I can do. I'll probably switch my own VMs to
> ext3 so I don't have to worry about these FS crashes.

I created a new LVM Volume and a new ext3 FS. I copied the whole FS from
ext4 to ext3 and running now the VM on ext3.

Today the ext4_mb_generate_buddy bug did hit me agian. With ext3 FS and
running the VM with "cache=none" setting.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Chris J Arges]

Louie,

Can you post some additional information that may help in debugging this issue.
1) Can you post the output of 'virsh dumpxml ' of the affected VM?
2) Can you post the /boot/config-`uname -r` of the affected VM's kernel?
3) What type of partitioning layout does your VM have?

It seems that it has been reproducible in 3.13 to 3.16 in this report,
and the upstream report shows 3.19.

I tried this in a local VM with 32GB disk, 2 cpu, 512MB memory, swapfile
and could not trivially reproduce with the filebench test case you
mentioned above.

--chris

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[LouieGosselin]

It's happened again. I've spent several hours on this and I've been able
to recreate the failure under some synthetic conditions with a
sacrificial VM.

The filebench defaults do not cause an ext4 crash for me, but the
following do:

load workloads/fileserver
set $dir=/tmp/
set $nfiles=20
set $meandirwidth=3
run 120

The ext4 error never happens in the filebench'es init phase, only 50s or
so into the 50 threaded run phase. Less extreme settings won't produce a
consistent crash.

Reducing the amount of free memory makes the errors much more likely.

This is before running filebench:
 total   used   free sharedbuffers cached
Mem:  482M99M   382M   300K27M20M
-/+ buffers/cache:52M   429M
Swap: 1.9G94M   1.8G

This is while running filebench one second before the crash:
 total   used   free sharedbuffers cached
Mem:  482M   476M   5.6M   284K27M18M
-/+ buffers/cache:   430M51M
Swap: 1.9G   253M   1.6G
2769.63

The error is reproducible in cloned VMs.

Moving swap to another disk changes nothing.

As far as I can tell, the error never happens with ext4 filesystems
other than the root FS where executables are running from.

I've tried bonnie, stress-ng, and simple scripts, I have not been able
to get these to crash ext4.

The sacrificial VM has not crashed after add an extra 500MB to it.

Although production was never under such heavy loads, I've added 500MB
to the production VM to see if it helps anyways.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[LouieGosselin]

I'm posting again to add that I conducted some more tests and ext3 does
not encounter corruption under the same conditions. I hope this
information is helpful to others, if anyone needs more information let
me know and I'll see what I can do. I'll probably switch my own VMs to
ext3 so I don't have to worry about these FS crashes.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Re: [Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Jan Wagner]

On 10/13/2015 06:00  PW, Simon Déziel wrote:
> On 10/13/2015 11:50 AM, Jan Wagner wrote:
>> I'm trying now '"cache=none"' for this VM and see how that turns out.
> 
> FYI, all my VMs use cache=none and have small HDDs (lv of 2-4G) yet the
> regularly trip on this bug.

looks like this didn't fixed the problem for me.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[LouieGosselin]

It happened here again.

8/24 ext4 corruption
9/14 ext4 corruption
9/29 update/reboot
10/16 ext4 corruption

This time the corruption was severe. 1743 files from multiple directories got 
moved into lost+found. 
It took me almost 2 hours this morning to verify & fix everything. Fortunately 
every time this has happened, all the files were dated prior to the daily 
backup and "diff -qr ..." shows exactly what was lost.

As far as I can tell this is not a memory issue and the ext4 FS is using
10G out of 100G.

Every time the corruption has been in /var/mail.  However the VM is
mostly used for mail so it may not be significant. The /var/mail branch
itself is 4.3G

3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1+deb8u4 (2015-09-19) x86_64
GNU/Linux

I'm holding back the kernels on other VM's so this is the only VM with
the problem.


Is anyone able to reproduce this on demand?

I really need to do something because this is causing downtime during normal 
business hours. I'll probably try one of the following:
1. Rebuild the FS from scratch and see if ext4 corruption continues.
2. Use ext3 or something else.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Jan Wagner]

I got also hit with the problem on Debian Jessie VM [virtio] (KVM host
is Jessie as well).

The following bug might be related:
https://bugzilla.kernel.org/show_bug.cgi?id=104571

I'm trying now '"cache=none"' for this VM and see how that turns out.


** Bug watch added: Linux Kernel Bug Tracker #104571
   http://bugzilla.kernel.org/show_bug.cgi?id=104571

** Attachment added: "Screenshot_service_2015-10-13_17:25:18.png"
   
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+attachment/4493620/+files/Screenshot_service_2015-10-13_17%3A25%3A18.png

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Re: [Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Simon Déziel]

On 10/13/2015 11:50 AM, Jan Wagner wrote:
> I got also hit with the problem on Debian Jessie VM [virtio] (KVM host
> is Jessie as well).
> 
> The following bug might be related:
> https://bugzilla.kernel.org/show_bug.cgi?id=104571
> 
> I'm trying now '"cache=none"' for this VM and see how that turns out.

FYI, all my VMs use cache=none and have small HDDs (lv of 2-4G) yet the
regularly trip on this bug.

Simon

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Emmanuel Lacour]

Adding memory did not helps, but it seems to have fixed the issue by
just removing the fstab option errors=remount-ro on the mountpoints
having problems. It's usually only set on root fs and we have no problem
on root fs, but by mistake this option was added to other mountpoints
and triggered the bug.

No more ro-remounts since one week and no ext4 messages in logs.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[David Brown]

I'm getting this same issue every day

3.2.0-4-amd64 #1 SMP Debian 3.2.68-1+deb7u4 x86_64 GNU/Linux

Guest is running under proxmox 3.4 virtio

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Emmanuel Lacour]

** Also affects: linux (Debian)
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Emmanuel Lacour]

Same problem here on 2 VMs, appeared 4 times in one week.

VMs are running Debian Jessie, on Debian wheezy hypervisors running ceph
giant, libvirt 1.2.9-9~bpo70+1, qemu 1:2.1+dfsg-12~bpo70+1.

FS is ext4, using virtio and rbd with writeback.

After first crash we upgraded kernel to 4.1.6-1~bpo8+1. But it crashed
again next day :(

Seems to be related to memory usage at least for last crash.

Attached the trace.

We added more memory to see if that helps to avoid triggering the bug.

** Attachment added: "kern.log with backtrace"
   
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+attachment/4466843/+files/kern.log

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[davidak]

We have the same error on a 14.04 VM running on a XenServer 6.2
(paravirtualized).

EXT4-fs error (device dm-1): ext4_mb_generate_buddy:756: group 36, block
bitmap and bg descriptor inconsistent: 30504 vs 30149 free clusters

Linux xxx 3.13.0-63-generic #103-Ubuntu SMP Fri Aug 14 21:42:59 UTC 2015
x86_64 x86_64 x86_64 GNU/Linux

i created a new VM and restored a backup and run the daily cronjobs to
reproduce it but it didn't happen again.

run-parts -v --exit-on-error /etc/cron.daily

output of dmesg -T

...
[Mon Sep 14 14:47:00 2015] random: lvm urandom read with 64 bits of entropy 
available
[Mon Sep 14 14:47:00 2015] bio: create slab  at 1
[Mon Sep 14 14:47:00 2015] EXT4-fs (dm-1): INFO: recovery required on readonly 
filesystem
[Mon Sep 14 14:47:00 2015] EXT4-fs (dm-1): write access will be enabled during 
recovery
[Mon Sep 14 14:47:01 2015] random: nonblocking pool is initialized
[Mon Sep 14 14:47:01 2015] EXT4-fs (dm-1): orphan cleanup on readonly fs
[Mon Sep 14 14:47:01 2015] EXT4-fs (dm-1): 16 orphan inodes deleted
[Mon Sep 14 14:47:01 2015] EXT4-fs (dm-1): recovery complete
[Mon Sep 14 14:47:01 2015] EXT4-fs (dm-1): mounted filesystem with ordered data 
mode. Opts: (null)
...

that seams to be ok. i also tried with the cron.weekly...

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[maxnuv]

Same error, on a single VM, virtio driver, ext4:

EXT4-fs (vda1): pa 880010c3daf8: logic 512, pyhs. 6750208, len 512
EXT4-fs error (device vda1): ext4_mb_release_inode_pa:3773: group 206, free 
143, pa_free 142
Aborting journal on device vda1-8
Disk quota active

No memory error (checked)
No disk error (checked too)

At system reset, filesystem check and correction, quota data corrupted,
quotacheck take long time to complete.

The error is strictly correlated to filesystem activity, when raise,
error is more frequent.

ubuntu 14.04 all updated, kernel 3.16.0-49-generic

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[LouieGosselin]

I'm on Debian, but it's happening to me as well.
KVM with virtual disks backed by LVM volumes on the host.

Both the VM and the host are running
Linux version 3.16.0-4-amd64 (debian-ker...@lists.debian.org) (gcc version 
4.8.4 (Debian 4.8.4-1) ) #1 SMP Debian 3.16.7-ckt11-1+deb8u2 (2015-07-17)


First occurred on August 24th. Manual fsck required, lots of files converted to 
lost-inodes.
[1992712.418275] EXT4-fs error (device vda2): ext4_mb_generate_buddy:757: group 
96, block bitmap and bg descriptor inconsistent: 24017 vs 24015 free clusters
[1992712.513438] Aborting journal on device vda2-8.
[1992712.514007] EXT4-fs (vda2): Remounting filesystem read-only
[1992712.514205] EXT4-fs error (device vda2) in ext4_evict_inode:243: Journal 
has aborted

Happened again today September 14th in the same VM.
[1489393.753098] EXT4-fs error (device vda2): ext4_mb_generate_buddy:757: group 
144, block bitmap and bg descriptor inconsistent: 23914 vs 23913 free clusters
[1489393.803865] Aborting journal on device vda2-8.
[1489393.804439] EXT4-fs (vda2): Remounting filesystem read-only

This is the first syslog activity since I rebooted in August, no block
IO errors on the guest or host.

Manual fsck required, files were lost, but everything is running again.

It has not happened in other VMs running older kernels. It also has not
happened on the host, however there's very little file system activity
on the host. The fact that it hasn't happened on other VMs leads me to
believe the bug is inside the guest rather than with KVM - perhaps ext4
or the virtual disk driver.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[maxnuv]

Same error, virtual machine 14.04 running on kvm, server 14.04.

No hardware issue, the entire server was replaced after the second stop.

No hard disk issue, the server is on raid hardware disk.

The server is running more than one virtual server, only one affected.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Peter]

I'm having this exact issue with Debian Jessie. Virtual machines running
with LVM storage unpredictably failing after cron.daily.

[680406.357587] EXT4-fs error (device vda1): ext4_mb_generate_buddy:757: group 
49, block bitmap and bg descriptor inconsistent: 18356 vs 18355 free clusters
[680406.361696] Aborting journal on device vda1-8.
[680406.453757] EXT4-fs error (device vda1): ext4_journal_check_start:56:
[680406.453999] EXT4-fs (vda1): Remounting filesystem read-only
[680406.457551] Detected aborted journal

[766809.056039] EXT4-fs (vda1): error count since last fsck: 2
[766809.056069] EXT4-fs (vda1): initial error at time 1441383430: 
ext4_mb_generate_buddy:757
[766809.056082] EXT4-fs (vda1): last error at time 1441383430: 
ext4_journal_check_start:56
[853316.576033] EXT4-fs (vda1): error count since last fsck: 2
[853316.576052] EXT4-fs (vda1): initial error at time 1441383430: 
ext4_mb_generate_buddy:757
[853316.576066] EXT4-fs (vda1): last error at time 1441383430: 
ext4_journal_check_start:56

root@collect1 ~ # uname -a
Linux collect1 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1+deb8u3 (2015-08-04) 
x86_64 GNU/Linux

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[gschoenberger]

Seems we are also confronted with this error in one of our VMWare guests:
[45780.021968] EXT4-fs error (device dm-9): ext4_mb_generate_buddy:756: group 
1411, block bitmap and bg descriptor inconsistent: 32757 vs 32768 free clusters
[45780.022029] Aborting journal on device dm-9-8.
[45780.055657] EXT4-fs (dm-9): Remounting filesystem read-only
[45780.055733] EXT4-fs error (device dm-9) in ext4_orphan_add:2609: Journal has 
aborted

It happened four times the last three weeks, pretty annoying. We have tried to 
harden ext4 in some way by using:
* ext4 rw,relatime,nodelalloc,errors=remount-ro,commit=2
as mount options. We have also increased hung_task_timeout_secs as we have 
found some hints in a RedHat KB article.

Any ideas on how to fix that ext4 error in the guest?

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y

[Simon Déziel]

And yet again:

root@rproxy:~# dmesg -T | tail
[Thu Jun 11 15:28:56 2015] type=1400 audit(1434050938.364:24): 
apparmor=STATUS operation=profile_replace profile=unconfined name=nc 
pid=658 comm=apparmor_parser
[Thu Jun 11 15:28:59 2015] init: plymouth-upstart-bridge main process ended, 
respawning
[Thu Jun 11 15:29:02 2015] init: console-font main process (757) terminated 
with status 71
[Thu Jun 11 15:29:03 2015] init: plymouth-splash main process (815) terminated 
with status 1
[Thu Jun 11 18:18:40 2015] random: nonblocking pool is initialized
[Mon Jun 15 16:53:35 2015] EXT4-fs (vda1): pa 8800061e1000: logic 3293, 
phys. 469944, len 424
[Mon Jun 15 16:53:35 2015] EXT4-fs error (device vda1): 
ext4_mb_release_inode_pa:3753: group 14, free 13, pa_free 11
[Mon Jun 15 16:53:35 2015] Aborting journal on device vda1-8.
[Mon Jun 15 16:53:36 2015] EXT4-fs (vda1): Remounting filesystem read-only
[Mon Jun 15 16:53:36 2015] EXT4-fs error (device vda1): 
ext4_journal_check_start:56: Detected aborted journal

root@rproxy:~# uname -a
Linux rproxy.dmz.sdeziel.info 3.13.0-54-generic #91-Ubuntu SMP Tue May 26 
19:15:08 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1423672

Title:
  ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor
  inconsistent: X vs Y

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs