Bug#685726: linux-image-3.2.0-3-amd64: return error when trying to format image file (mkfs -t ext4 file.img)
On Wed, 2012-12-26 at 01:27 +0400, Nikita Gubenko wrote: Guys, any update on this bug? Can reproduce on 3.2.0-4-amd64 with vhd image. We're still waiting for someone to report results after applying the patches from http://bugs.debian.org/685726#42. Ben. -- Ben Hutchings It is easier to change the specification to fit the program than vice versa. signature.asc Description: This is a digitally signed message part
Processed: Re: Bug#685726: linux-image-3.2.0-3-amd64: return error when trying to format image file (mkfs -t ext4 file.img)
Processing commands for cont...@bugs.debian.org: # Ben Hutchings wrote: # # Please can you test the patches from the previous email? tags 685726 + upstream patch moreinfo Bug #685726 [src:linux] linux-image-3.2.0-3-amd64: return error when trying to format image file (mkfs -t ext4 file.img) Added tag(s) upstream, moreinfo, and patch. End of message, stopping processing here. Please contact me if you need assistance. -- 685726: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=685726 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/handler.s.c.134839082125224.transcr...@bugs.debian.org
Bug#685726: linux-image-3.2.0-3-amd64: return error when trying to format image file (mkfs -t ext4 file.img)
On Wed, 2012-08-29 at 14:11 -0400, Theodore Ts'o wrote: On Tue, Aug 28, 2012 at 04:09:05PM -0700, Ben Hutchings wrote: As discussed, Linux 3.2.y needs a backport of the fixes for this, which I think are: 968dee7 ext4: fix hole punch failure when depth is greater than 0 89a4e48 ext4: fix kernel BUG on large-scale rm -rf commands I have a backport of these patches rebased onto 3.2.28. They pass the full set of ext4 regression tests. However, for some reason I wasn't able to reproduce the problem on 3.2.28, even though I thought: mke2fs -t ext4 /dev/vdd mount -t ext4 /dev/vdd /vdd dd if=/dev/zero of=/vdd/test.img bs=1M count=200 mke2fs -t ext4 -F /vdd/test.img should have been a reliable reproduction with the problem. I'm pretty sure it worked to trigger the problem before, but for some reason it's not working for me. Anyway, here are the patches. If someone could confirm wheterh this resolve your problem, I would really appreciate it. Thanks!! Please can you test the patches from the previous email? There are instructions for building a patched kernel package at http://kernel-handbook.alioth.debian.org/ch-common-tasks.html#s-common-official. Ben. -- Ben Hutchings Usenet is essentially a HUGE group of people passing notes in class. - Rachel Kadel, `A Quick Guide to Newsgroup Etiquette' signature.asc Description: This is a digitally signed message part
Bug#685726: linux-image-3.2.0-3-amd64: return error when trying to format image file (mkfs -t ext4 file.img)
On Tue, Aug 28, 2012 at 04:09:05PM -0700, Ben Hutchings wrote: As discussed, Linux 3.2.y needs a backport of the fixes for this, which I think are: 968dee7 ext4: fix hole punch failure when depth is greater than 0 89a4e48 ext4: fix kernel BUG on large-scale rm -rf commands I have a backport of these patches rebased onto 3.2.28. They pass the full set of ext4 regression tests. However, for some reason I wasn't able to reproduce the problem on 3.2.28, even though I thought: mke2fs -t ext4 /dev/vdd mount -t ext4 /dev/vdd /vdd dd if=/dev/zero of=/vdd/test.img bs=1M count=200 mke2fs -t ext4 -F /vdd/test.img should have been a reliable reproduction with the problem. I'm pretty sure it worked to trigger the problem before, but for some reason it's not working for me. Anyway, here are the patches. If someone could confirm wheterh this resolve your problem, I would really appreciate it. Thanks!! - Ted From 4ec615eb5c0dc86d941520e40dfd9afdf8ccef87 Mon Sep 17 00:00:00 2001 From: Lukas Czerner lczer...@redhat.com Date: Mon, 19 Mar 2012 23:03:19 -0400 Subject: [PATCH 1/3] ext4: rewrite punch hole to use ext4_ext_remove_space() Commit 5f95d21fb6f2aaa52830e5b7fb405f6c71d3ab85 upstream. This commit rewrites ext4 punch hole implementation to use ext4_ext_remove_space() instead of its home gown way of doing this via ext4_ext_map_blocks(). There are several reasons for changing this. Firstly it is quite non obvious that punching hole needs to ext4_ext_map_blocks() to punch a hole, especially given that this function should map blocks, not unmap it. It also required a lot of new code in ext4_ext_map_blocks(). Secondly the design of it is not very effective. The reason is that we are trying to punch out blocks in ext4_ext_punch_hole() in opposite direction than in ext4_ext_rm_leaf() which causes the ext4_ext_rm_leaf() to iterate through the whole tree from the end to the start to find the requested extent for every extent we are going to punch out. And finally the current implementation does not use the existing code, but bring a lot of new code, which is IMO unnecessary since there already is some infrastructure we can use. Specifically ext4_ext_remove_space(). This commit changes ext4_ext_remove_space() to accept 'end' parameter so we can not only truncate to the end of file, but also remove the space in the middle of the file (punch a hole). Moreover, because the last block to punch out, might be in the middle of the extent, we have to split the extent at 'end + 1' so ext4_ext_rm_leaf() can easily either remove the whole fist part of split extent, or change its size. ext4_ext_remove_space() is then used to actually remove the space (extents) from within the hole, instead of ext4_ext_map_blocks(). Note that this also fix the issue with punch hole, where we would forget to remove empty index blocks from the extent tree, resulting in double free block error and file system corruption. This is simply because we now use different code path, where this problem does not exist. This has been tested with fsx running for several days and xfstests, plus xfstest #251 with '-o discard' run on the loop image (which converts discard requestes into punch hole to the backing file). All of it on 1K and 4K file system block size. Signed-off-by: Lukas Czerner lczer...@redhat.com Signed-off-by: Theodore Ts'o ty...@mit.edu --- fs/ext4/extents.c | 170 -- 1 file changed, 88 insertions(+), 82 deletions(-) diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c index 54f2bdc..43ccd03 100644 --- a/fs/ext4/extents.c +++ b/fs/ext4/extents.c @@ -45,6 +45,14 @@ #include trace/events/ext4.h +/* + * used by extent splitting. + */ +#define EXT4_EXT_MAY_ZEROOUT 0x1 /* safe to zeroout if split fails \ + due to ENOSPC */ +#define EXT4_EXT_MARK_UNINIT1 0x2 /* mark first half uninitialized */ +#define EXT4_EXT_MARK_UNINIT2 0x4 /* mark second half uninitialized */ + static int ext4_split_extent(handle_t *handle, struct inode *inode, struct ext4_ext_path *path, @@ -52,6 +60,13 @@ static int ext4_split_extent(handle_t *handle, int split_flag, int flags); +static int ext4_split_extent_at(handle_t *handle, + struct inode *inode, + struct ext4_ext_path *path, + ext4_lblk_t split, + int split_flag, + int flags); + static int ext4_ext_truncate_extend_restart(handle_t *handle, struct inode *inode, int needed) @@ -2307,7 +2322,7 @@ ext4_ext_rm_leaf(handle_t *handle, struct inode *inode, struct ext4_extent *ex; /* the header must be checked already in ext4_ext_remove_space() */ - ext_debug(truncate since %u in leaf\n, start); + ext_debug(truncate since %u in leaf to %u\n, start, end); if (!path[depth].p_hdr) path[depth].p_hdr = ext_block_hdr(path[depth].p_bh); eh = path[depth].p_hdr; @@ -2342,7 +2357,7 @@
Bug#685726: linux-image-3.2.0-3-amd64: return error when trying to format image file (mkfs -t ext4 file.img)
On Sun, 2012-08-26 at 00:56 +0400, hr...@infotech.am wrote: On Fri, 2012-08-24 at 03:30 +0400, Hrayr Grigoryan wrote: Package: src:linux Version: 3.2.23-1 Severity: critical Tags: lfs Justification: causes serious data loss Dear Maintainer, When I'm trying to format file image with the command mkfs -t ext4 file.img, it returns the following errors [ 142.328065] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 1022, depth: 2 pblock 0 [ 142.328387] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 1023, depth: 2 pblock 0 [ 142.328699] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 9254, depth: 2 pblock 0 [ 142.329018] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 1057, depth: 2 pblock 0 [...] Have you tested this on any other kernel versions (earlier or later)? Yes, I have tested with the earlier versions, all earlier versions of 3.x.x kernels have this bug. I talked to the ext4 maintainer about this. He recognised the problem and said you can work around this by adding the option '-E nodiscard' when running mke2fs. This is supposed to be fixed in Linux 3.5.3 (not yet available in Debian). We will try to get it fixed in Linux 3.2.y soon. Ben. -- Ben Hutchings It is a miracle that curiosity survives formal education. - Albert Einstein signature.asc Description: This is a digitally signed message part
Bug#685726: linux-image-3.2.0-3-amd64: return error when trying to format image file (mkfs -t ext4 file.img)
As discussed, Linux 3.2.y needs a backport of the fixes for this, which I think are: 968dee7 ext4: fix hole punch failure when depth is greater than 0 89a4e48 ext4: fix kernel BUG on large-scale rm -rf commands Ben. On Sun, 2012-08-26 at 00:56 +0400, hr...@infotech.am wrote: On Fri, 2012-08-24 at 03:30 +0400, Hrayr Grigoryan wrote: Package: src:linux Version: 3.2.23-1 Severity: critical Tags: lfs Justification: causes serious data loss Dear Maintainer, When I'm trying to format file image with the command mkfs -t ext4 file.img, it returns the following errors [ 142.328065] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 1022, depth: 2 pblock 0 [ 142.328387] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 1023, depth: 2 pblock 0 [ 142.328699] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 9254, depth: 2 pblock 0 [ 142.329018] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 1057, depth: 2 pblock 0 So you have an ext4 filesystem in file.img, on top of an ext4 filesystem on /dev/sdb1? yes, all are ext4, no other file systems I have on that machine. The problem visible only for big images f.x. 160Gb and more. I did the tests on different hardwares, but result the same. [...] How is test.img created, before you run mkfs? Is it a sparse file or does it have all data blocks allocated? No it is not sparse file, test.img was created with the following command dd if=/dev/zero of=test.img bs=1M count=160k Have you tested this on any other kernel versions (earlier or later)? Yes, I have tested with the earlier versions, all earlier versions of 3.x.x kernels have this bug. Hrayr. Ben. -- Ben Hutchings Experience is what causes a person to make new mistakes instead of old ones. -- Ben Hutchings Quantity is no substitute for quality, but it's the only one we've got. signature.asc Description: This is a digitally signed message part
Bug#685726: linux-image-3.2.0-3-amd64: return error when trying to format image file (mkfs -t ext4 file.img)
On Fri, 2012-08-24 at 03:30 +0400, Hrayr Grigoryan wrote: Package: src:linux Version: 3.2.23-1 Severity: critical Tags: lfs Justification: causes serious data loss Dear Maintainer, When I'm trying to format file image with the command mkfs -t ext4 file.img, it returns the following errors [ 142.328065] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 1022, depth: 2 pblock 0 [ 142.328387] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 1023, depth: 2 pblock 0 [ 142.328699] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 9254, depth: 2 pblock 0 [ 142.329018] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 1057, depth: 2 pblock 0 So you have an ext4 filesystem in file.img, on top of an ext4 filesystem on /dev/sdb1? yes, all are ext4, no other file systems I have on that machine. The problem visible only for big images f.x. 160Gb and more. I did the tests on different hardwares, but result the same. [...] How is test.img created, before you run mkfs? Is it a sparse file or does it have all data blocks allocated? No it is not sparse file, test.img was created with the following command dd if=/dev/zero of=test.img bs=1M count=160k Have you tested this on any other kernel versions (earlier or later)? Yes, I have tested with the earlier versions, all earlier versions of 3.x.x kernels have this bug. Hrayr. Ben. -- Ben Hutchings Experience is what causes a person to make new mistakes instead of old ones. -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/2dc67ec745eaea6e198106c238aff5dd.squir...@mail.infotech.am
Bug#685726: linux-image-3.2.0-3-amd64: return error when trying to format image file (mkfs -t ext4 file.img)
On Fri, 2012-08-24 at 03:30 +0400, Hrayr Grigoryan wrote: Package: src:linux Version: 3.2.23-1 Severity: critical Tags: lfs Justification: causes serious data loss Dear Maintainer, When I'm trying to format file image with the command mkfs -t ext4 file.img, it returns the following errors [ 142.328065] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 1022, depth: 2 pblock 0 [ 142.328387] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 1023, depth: 2 pblock 0 [ 142.328699] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 9254, depth: 2 pblock 0 [ 142.329018] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 1057, depth: 2 pblock 0 So you have an ext4 filesystem in file.img, on top of an ext4 filesystem on /dev/sdb1? The problem visible only for big images f.x. 160Gb and more. I did the tests on different hardwares, but result the same. [...] How is test.img created, before you run mkfs? Is it a sparse file or does it have all data blocks allocated? Have you tested this on any other kernel versions (earlier or later)? If not, can you try Linux 3.5, which is available in the experimental suite? Ben. -- Ben Hutchings Experience is what causes a person to make new mistakes instead of old ones. signature.asc Description: This is a digitally signed message part
Bug#685726: linux-image-3.2.0-3-amd64: return error when trying to format image file (mkfs -t ext4 file.img)
Package: src:linux Version: 3.2.23-1 Severity: critical Tags: lfs Justification: causes serious data loss Dear Maintainer, When I'm trying to format file image with the command mkfs -t ext4 file.img, it returns the following errors [ 142.328065] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 1022, depth: 2 pblock 0 [ 142.328387] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 1023, depth: 2 pblock 0 [ 142.328699] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 9254, depth: 2 pblock 0 [ 142.329018] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 1057, depth: 2 pblock 0 The problem visible only for big images f.x. 160Gb and more. I did the tests on different hardwares, but result the same. -- Package-specific info: ** Version: Linux version 3.2.0-3-amd64 (Debian 3.2.23-1) (debian-kernel@lists.debian.org) (gcc version 4.6.3 (Debian 4.6.3-8) ) #1 SMP Mon Jul 23 02:45:17 UTC 2012 ** Command line: BOOT_IMAGE=/vmlinuz-3.2.0-3-amd64 root=UUID=7d0da86e-f774-4008-87fb-1a120de8c5bf ro quiet ** Tainted: O (4096) * Out-of-tree module has been loaded. ** Kernel log: [ 142.293445] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 928, depth: 2 pblock 0 [ 142.293765] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 929, depth: 2 pblock 0 [ 142.294084] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 930, depth: 2 pblock 0 [ 142.294417] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 931, depth: 2 pblock 0 [ 142.294731] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 932, depth: 2 pblock 0 [ 142.295053] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 933, depth: 2 pblock 0 [ 142.295375] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 934, depth: 2 pblock 0 [ 142.295686] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 935, depth: 2 pblock 0 [ 142.296010] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 936, depth: 2 pblock 0 [ 142.296333] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 937, depth: 2 pblock 0 [ 142.296648] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 938, depth: 2 pblock 0 [ 142.297005] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 939, depth: 2 pblock 0 [ 142.297302] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 940, depth: 2 pblock 0 [ 142.297619] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 941, depth: 2 pblock 0 [ 142.297943] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 942, depth: 2 pblock 0 [ 142.298268] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 943, depth: 2 pblock 0 [ 142.298591] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 944, depth: 2 pblock 0 [ 142.298899] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 945, depth: 2 pblock 0 [ 142.299221] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 946, depth: 2 pblock 0 [ 142.299539] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 947, depth: 2 pblock 0 [ 142.299861] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 948, depth: 2 pblock 0 [ 142.300164] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 949, depth: 2 pblock 0 [ 142.300486] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 950, depth: 2 pblock 0 [ 142.300798] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad extent address lblock: 951, depth: 2 pblock 0 [ 142.301110] EXT4-fs error (device sdb1): ext4_ext_map_blocks:3769: inode #12: comm mkfs.ext4: bad
