On Tue, Feb 16, 2021 at 11:42 PM Qu Wenruo <quwenruo.bt...@gmx.com> wrote: > > > > On 2021/2/16 下午10:45, Filipe Manana wrote: > > On Wed, Jun 24, 2020 at 10:00 PM Qu Wenruo <w...@suse.com> wrote: > >> > >> [BUG] > >> The following script could lead to corrupted btrfs fs after > >> btrfs-convert: > >> > >> fallocate -l 1G test.img > >> mkfs.ext4 test.img > >> mount test.img $mnt > >> fallocate -l 200m $mnt/file1 > >> fallocate -l 200m $mnt/file2 > >> fallocate -l 200m $mnt/file3 > >> fallocate -l 200m $mnt/file4 > >> fallocate -l 205m $mnt/file1 > >> fallocate -l 205m $mnt/file2 > >> fallocate -l 205m $mnt/file3 > >> fallocate -l 205m $mnt/file4 > >> umount $mnt > >> btrfs-convert test.img > >> > >> The result btrfs will have a device extent beyond its boundary: > >> pening filesystem to check... > >> Checking filesystem on test.img > >> UUID: bbcd7399-fd5b-41a7-81ae-d48bc6935e43 > >> [1/7] checking root items > >> [2/7] checking extents > >> ERROR: dev extent devid 1 physical offset 993198080 len 85786624 is > >> beyond device boundary 1073741824 > >> ERROR: errors found in extent allocation tree or chunk allocation > >> [3/7] checking free space cache > >> [4/7] checking fs roots > >> [5/7] checking only csums items (without verifying data) > >> [6/7] checking root refs > >> [7/7] checking quota groups skipped (not enabled on this FS) > >> found 913960960 bytes used, error(s) found > >> total csum bytes: 891500 > >> total tree bytes: 1064960 > >> total fs tree bytes: 49152 > >> total extent tree bytes: 16384 > >> btree space waste bytes: 144885 > >> file data blocks allocated: 2129063936 > >> referenced 1772728320 > >> > >> [CAUSE] > >> Btrfs-convert first collect all used blocks in the original fs, then > >> slightly enlarge the used blocks range as new btrfs data chunks. > >> > >> However the enlarge part has a problem, that it doesn't take the device > >> boundary into consideration. > >> > >> Thus it caused device extents and data chunks to go beyond device > >> boundary. > >> > >> [FIX] > >> Just to extra check before inserting data chunks into > >> btrfs_convert_context::data_chunk. > >> > >> Reported-by: Jiachen YANG <farsee...@gmail.com> > >> Signed-off-by: Qu Wenruo <w...@suse.com> > > > > So, having upgraded a test box from btrfs-progs v5.6.1 to v5.10.1, I > > now have btrfs/136 (fstests) failing: > > > > $ ./check btrfs/136 > > FSTYP -- btrfs > > PLATFORM -- Linux/x86_64 debian8 5.11.0-rc7-btrfs-next-81 #1 SMP > > PREEMPT Tue Feb 16 12:29:07 WET 2021 > > MKFS_OPTIONS -- /dev/sdc > > MOUNT_OPTIONS -- /dev/sdc /home/fdmanana/btrfs-tests/scratch_1 > > > > btrfs/136 7s ... [failed, exit status 1]- output mismatch (see > > /home/fdmanana/git/hub/xfstests/results//btrfs/136.out.bad) > > --- tests/btrfs/136.out 2020-06-10 19:29:03.818519162 +0100 > > +++ /home/fdmanana/git/hub/xfstests/results//btrfs/136.out.bad > > 2021-02-16 14:31:30.669559188 +0000 > > @@ -1,2 +1,3 @@ > > QA output created by 136 > > -Silence is golden > > +btrfs-convert failed > > +(see /home/fdmanana/git/hub/xfstests/results//btrfs/136.full for > > details) > > ... > > (Run 'diff -u /home/fdmanana/git/hub/xfstests/tests/btrfs/136.out > > /home/fdmanana/git/hub/xfstests/results//btrfs/136.out.bad' to see > > the entire diff) > > Ran: btrfs/136 > > Failures: btrfs/136 > > Failed 1 of 1 tests > > > > A bisect pointed to this patch. > > Did you get this failure on your test box as well? > > Nope. > > Just tested with btrfs-progs v5.10.1, it passes: > $ which btrfs > /usr/bin/btrfs > $ btrfs --version > btrfs-progs v5.10.1 > $ sudo ./check btrfs/136 > FSTYP -- btrfs > PLATFORM -- Linux/x86_64 btrfs-desktop-vm 5.11.0-rc4-custom+ #4 > SMP PREEMPT Mon Jan 25 18:35:22 CST 2021 > MKFS_OPTIONS -- /dev/mapper/test-scratch1 > MOUNT_OPTIONS -- /dev/mapper/test-scratch1 /mnt/scratch > > btrfs/136 6s ... 10s > Ran: btrfs/136 > Passed all 1 tests > > Would you mind to provide the 136.full to help debugging the failure?
Sure, here it is (also at https://pastebin.com/XhEX2dju): root 10:06:39 /home/fdmanana/git/hub/xfstests (master)> cat /home/fdmanana/git/hub/xfstests/results//btrfs/136.full mke2fs 1.45.6 (20-Mar-2020) Discarding device blocks: done Creating filesystem with 5242880 4k blocks and 1310720 inodes Filesystem UUID: 9519afc8-8ea0-42da-8ac5-3b4c20469dd1 Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000 Allocating group tables: done Writing inode tables: done Creating journal (32768 blocks): done Writing superblocks and filesystem accounting information: done Run fsstress -p 20 -n 100 -d /home/fdmanana/btrfs-tests/scratch_1/ext3_ext4_data/ext3 tune2fs 1.45.6 (20-Mar-2020) e2fsck 1.45.6 (20-Mar-2020) Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 3A: Optimizing directories Pass 4: Checking reference counts Pass 5: Checking group summary information /dev/sdc: ***** FILE SYSTEM WAS MODIFIED ***** /dev/sdc: 235/1310720 files (22.1% non-contiguous), 129757/5242880 blocks Run fsstress -p 20 -n 100 -d /home/fdmanana/btrfs-tests/scratch_1/ext3_ext4_data/ext4 ERROR: missing data block for bytenr 1048576 ERROR: failed to create ext2_saved/image: -2 WARNING: an error occurred during conversion, filesystem is partially created but not finalized and not mountable create btrfs filesystem: blocksize: 4096 nodesize: 16384 features: extref, skinny-metadata (default) checksum: crc32c creating ext2 image file btrfs-convert failed root 10:14:18 /home/fdmanana/git/hub/xfstests (master)> Thanks > > Thanks, > Qu > > > > Thanks. > > > >> --- > >> convert/main.c | 2 ++ > >> 1 file changed, 2 insertions(+) > >> > >> diff --git a/convert/main.c b/convert/main.c > >> index c86ddd988c63..7709e9a6c085 100644 > >> --- a/convert/main.c > >> +++ b/convert/main.c > >> @@ -669,6 +669,8 @@ static int calculate_available_space(struct > >> btrfs_convert_context *cctx) > >> cur_off = cache->start; > >> cur_len = max(cache->start + cache->size - cur_off, > >> min_stripe_size); > >> + /* data chunks should never exceed device boundary */ > >> + cur_len = min(cctx->total_bytes - cur_off, cur_len); > >> ret = add_merge_cache_extent(data_chunks, cur_off, > >> cur_len); > >> if (ret < 0) > >> goto out; > >> -- > >> 2.27.0 > >> > > > > -- Filipe David Manana, “Whether you think you can, or you think you can't — you're right.”
