Public bug reported:
BugLink: https://bugs.launchpad.net/bugs/1896154
[Impact]
Since 929be17a9b49 ("btrfs: Switch btrfs_trim_free_extents to
find_first_clear_extent_bit") which landed in 5.3, btrfs wont trim a
range that has already been trimmed, and will instead go looking for a
range where the CHUNK_TRIMMED and CHUNK_ALLOCATED bits aren't set.
If a device had been shrunk, the CHUNK_TRIMMED and CHUNK_ALLOCATED bits
are never cleared, which means that btrfs could go looking for a range
to trim which is beyond the new device size. This leads to an underflow
in a length calculation for the range to trim, and we will end up
trimming past the device's boundary.
This has an unfortunate side effect of mangling and filling the root
disk with garbage data, and it will not stop until the root disk is
totally filled, and makes the instance unusable.
[Fix]
The issue was fixed in the following commit, in 5.9-rc1:
commit c57dd1f2f6a7cd1bb61802344f59ccdc5278c983
Author: Qu Wenruo <w...@suse.com>
Date: Fri Jul 31 19:29:11 2020 +0800
Subject: btrfs: trim: fix underflow in trim length to prevent access beyond
device boundary
Link:
https://github.com/torvalds/linux/commit/c57dd1f2f6a7cd1bb61802344f59ccdc5278c983
The fix clears the CHUNK_TRIMMED and CHUNK_ALLOCATED bits when a device
is being shrunk, and performs some additional checks to ensure we do not
trim past the device size boundary.
The fix was backported to 5.7.17 and 5.8.3 upstream stable, but it seems
5.4 was skipped.
The patch required a minor backport to 5.4, with the CHUNK_STATE_MASK
#define moving files back to fs/btrfs/extent_io.h, as the file had been
renamed in later kernels.
[Testcase]
The easiest way to reproduce is to use a cloud instance that supplies a
real NVMe drive, that supports TRIM and block discards.
Warning, this will fill the root disk with garbage data, ONLY run on a
throwaway instance!
Run the following commands:
$ dev=/dev/nvme0n1
$ mnt=/mnt
$ mkfs.btrfs -f $dev -b 10G
$ mount $dev $mnt
$ fstrim $mnt
$ btrfs filesystem resize 1:-1G $mnt
$ fstrim $mnt
The last command will appear to hang, while the root filesystem will
begin filling with garbage data. Once the root filesystem fills, you
will see the following error:
fstrim: /mnt: FITRIM ioctl failed: Input/output error
/dev/sda1 29G 29G 0 100% /
A test kernel is available from the following PPA:
https://launchpad.net/~mruffell/+archive/ubuntu/sf293389-test
If you install the test kernel, then the final fstrim command completes
successfully in a short amount of time.
[Regression Potential]
If a regression were to occur, it could affect users who are attempting
to shrink or resize their btrfs volume. Most users already understand
that changing the size of a volume is a risky operation, and would have
a backup.
If a regression occurs, then there is potential for data loss when users
resize or shrink their btrfs volumes. Standard volume creation would not
be affected.
The patches have been backported to upstream stable, and are trusted by
the community.
** Affects: linux (Ubuntu)
Importance: Undecided
Status: Fix Released
** Affects: linux (Ubuntu Focal)
Importance: Medium
Assignee: Matthew Ruffell (mruffell)
Status: In Progress
** Tags: sts
** Also affects: linux (Ubuntu Focal)
Importance: Undecided
Status: New
** Changed in: linux (Ubuntu)
Status: New => Fix Released
** Changed in: linux (Ubuntu Focal)
Status: New => In Progress
** Changed in: linux (Ubuntu Focal)
Importance: Undecided => Medium
** Changed in: linux (Ubuntu Focal)
Assignee: (unassigned) => Matthew Ruffell (mruffell)
** Description changed:
- BugLink: https://bugs.launchpad.net/bugs/
+ BugLink: https://bugs.launchpad.net/bugs/1896154
[Impact]
Since 929be17a9b49 ("btrfs: Switch btrfs_trim_free_extents to
find_first_clear_extent_bit") which landed in 5.3, btrfs wont trim a
range that has already been trimmed, and will instead go looking for a
range where the CHUNK_TRIMMED and CHUNK_ALLOCATED bits aren't set.
If a device had been shrunk, the CHUNK_TRIMMED and CHUNK_ALLOCATED bits
are never cleared, which means that btrfs could go looking for a range
to trim which is beyond the new device size. This leads to an underflow
in a length calculation for the range to trim, and we will end up
trimming past the device's boundary.
This has an unfortunate side effect of mangling and filling the root
disk with garbage data, and it will not stop until the root disk is
totally filled, and makes the instance unusable.
[Fix]
The issue was fixed in the following commit, in 5.9-rc1:
commit c57dd1f2f6a7cd1bb61802344f59ccdc5278c983
Author: Qu Wenruo <w...@suse.com>
Date: Fri Jul 31 19:29:11 2020 +0800
Subject: btrfs: trim: fix underflow in trim length to prevent access beyond
device boundary
Link:
https://github.com/torvalds/linux/commit/c57dd1f2f6a7cd1bb61802344f59ccdc5278c983
The fix clears the CHUNK_TRIMMED and CHUNK_ALLOCATED bits when a device
is being shrunk, and performs some additional checks to ensure we do not
trim past the device size boundary.
The fix was backported to 5.7.17 and 5.8.3 upstream stable, but it seems
5.4 was skipped.
The patch required a minor backport to 5.4, with the CHUNK_STATE_MASK
#define moving files back to fs/btrfs/extent_io.h, as the file had been
renamed in later kernels.
[Testcase]
The easiest way to reproduce is to use a cloud instance that supplies a
real NVMe drive, that supports TRIM and block discards.
Warning, this will fill the root disk with garbage data, ONLY run on a
throwaway instance!
Run the following commands:
$ dev=/dev/nvme0n1
$ mnt=/mnt
$ mkfs.btrfs -f $dev -b 10G
$ mount $dev $mnt
$ fstrim $mnt
$ btrfs filesystem resize 1:-1G $mnt
- $ fstrim $mnt
+ $ fstrim $mnt
The last command will appear to hang, while the root filesystem will
begin filling with garbage data. Once the root filesystem fills, you
will see the following error:
- fstrim: /mnt: FITRIM ioctl failed: Input/output error
- /dev/sda1 29G 29G 0 100% /
+ fstrim: /mnt: FITRIM ioctl failed: Input/output error
+ /dev/sda1 29G 29G 0 100% /
A test kernel is available from the following PPA:
https://launchpad.net/~mruffell/+archive/ubuntu/sf293389-test
If you install the test kernel, then the final fstrim command completes
successfully in a short amount of time.
[Regression Potential]
If a regression were to occur, it could affect users who are attempting
to shrink or resize their btrfs volume. Most users already understand
that changing the size of a volume is a risky operation, and would have
a backup.
If a regression occurs, then there is potential for data loss when users
resize or shrink their btrfs volumes. Standard volume creation would not
be affected.
The patches have been backported to upstream stable, and are trusted by
the community.
** Tags added: sts
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1896154
Title:
btrfs: trimming a btrfs device which has been shrunk previously fails
and fills root disk with garbage data
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1896154/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
