Re: inline extents

On Wed, Sep 19, 2018 at 09:18:16PM -0600, Chris Murphy wrote: > > ext4 has inline data, too, so there's every chance grub will corrupt > > ext4 filesystems with tit's wonderful new feature. I'm not sure if > > the ext4 metadata cksums cover the entire inode and inline data, but > > if they do it's

Re: inline extents

Adding fsdevel@, linux-ext4, and btrfs@ (which has a separate subject on this same issue) On Wed, Sep 19, 2018 at 7:45 PM, Dave Chinner wrote: >On Wed, Sep 19, 2018 at 10:23:38AM -0600, Chris Murphy wrote: >> Fedora 29 has a new feature to test if boot+startup fails, so the >> bootloader can

Re: [PATCH v2] Btrfs: fix physical offset reported by fiemap for inline extents

On 20.06.2018 12:02, fdman...@kernel.org wrote: > From: Filipe Manana > > So fix this by ensuring the physical offset is always set to 0 when we > are processing an extent with a special block_start value. > > Fixes: 9d311e11fc1f ("Btrfs: fiemap: pass correct bytenr when fm_extent_count >

[PATCH v2] Btrfs: fix physical offset reported by fiemap for inline extents

From: Filipe Manana Commit 9d311e11fc1f ("Btrfs: fiemap: pass correct bytenr when fm_extent_count is zero") introduced a regression where we no longer report 0 as the physical offset for inline extents (and other extents with a special block_start value). This is because it a

Re: [PATCH] Btrfs: fix physical offset reported by fiemap for inline extents

onger >> report 0 as the physical offset for inline extents. This is because it >> always sets the variable used to report the physical offset ("disko") >> as em->block_start plus some offset, and em->block_start has the value >> 18446744073709551614 ((u64) -2)

Re: [PATCH] Btrfs: fix physical offset reported by fiemap for inline extents

fdman...@kernel.org 於 2018-06-19 19:31 寫到: From: Filipe Manana Commit 9d311e11fc1f ("Btrfs: fiemap: pass correct bytenr when fm_extent_count is zero") introduced a regression where we no longer report 0 as the physical offset for inline extents. This is because it always sets the var

Re: [PATCH] Btrfs: fix physical offset reported by fiemap for inline extents

On Tue, Jun 19, 2018 at 12:31:42PM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > So fix this by ensuring the physical offset is always set to 0 when we > are processing an inline extent. > > Fixes: 9d311e11fc1f ("Btrfs: fiemap: pass correct bytenr when fm_extent_count > is zero") >

[PATCH] Btrfs: fix physical offset reported by fiemap for inline extents

From: Filipe Manana Commit 9d311e11fc1f ("Btrfs: fiemap: pass correct bytenr when fm_extent_count is zero") introduced a regression where we no longer report 0 as the physical offset for inline extents. This is because it always sets the variable used to report the physical offs

Re: [PATCH] btrfs: add missing memset while reading compressed inline extents

x...@waya.furryterror.org> wrote: >>>>>> >>>>>> >>>>>> From: Zygo Blaxell <ce3g8...@umail.furryterror.org> >>>>>> >>>>>> This is a story about 4 distinct (and very old) btrfs bugs. >>>

Re: [PATCH] btrfs: add missing memset while reading compressed inline extents

<zblax...@waya.furryterror.org> wrote: From: Zygo Blaxell <ce3g8...@umail.furryterror.org> This is a story about 4 distinct (and very old) btrfs bugs. Commit c8b978188c ("Btrfs: Add zlib compression support") added three data corruption bugs for inline extents (bugs #1-3). Com

Re: [PATCH] btrfs: add missing memset while reading compressed inline extents

ell >>> <zblax...@waya.furryterror.org> wrote: >>>> >>>> From: Zygo Blaxell <ce3g8...@umail.furryterror.org> >>>> >>>> This is a story about 4 distinct (and very old) btrfs bugs. >>>> >>>> Commit c8b978188c ("

Re: [v4] btrfs: add missing memset while reading compressed inline extents

On 10/Mar/2017 16:45, Zygo Blaxell wrote: > This is a story about 4 distinct (and very old) btrfs bugs. > <...> Tested-by: Ismael Luceno I encountered the issue in the wild; it caused frequent segfaults at ld.so after some hours of operation, as well as integrity check

Re: [PATCH] btrfs: add missing memset while reading compressed inline extents

and very old) btrfs bugs. Commit c8b978188c ("Btrfs: Add zlib compression support") added three data corruption bugs for inline extents (bugs #1-3). Commit 93c82d5750 ("Btrfs: zero page past end of inline file items") fixed bug #1: uncompressed inline extents followed by a hol

[PATCH v4] btrfs: add missing memset while reading compressed inline extents

This is a story about 4 distinct (and very old) btrfs bugs. Commit c8b978188c ("Btrfs: Add zlib compression support") added three data corruption bugs for inline extents (bugs #1-3). Commit 93c82d5750 ("Btrfs: zero page past end of inline file items") fixed bug #1: uncompr

Re: [PATCH v3] btrfs: add missing memset while reading compressed inline extents

the math reflect any possible > >>>>pg_offset? I do agree it shouldn't be happening, but its easy to correct > >>>>for and the WARN is likely to get lost. > >>> > >>>I'm not sure how to do that. It looks like I'd have to pass pg_offset > >&

Re: [PATCH v3] btrfs: add missing memset while reading compressed inline extents

ine extents followed by more data, there are a few wacky corners allowing inline extents at the end of the file. Lets not mix that change in with this one though. For now, just get the memset right and we can pass pg_offset down in a later patch. Are you saying "fix the memset in the patch&quo

Re: [PATCH v3] btrfs: add missing memset while reading compressed inline extents

ret = btrfs_decompress(compress_type, tmp, page, > > extent_offset, inline_size, max_size, pg_offset); > > > >and in the compression functions get pg_offset from the argument list > >instead of hardcoding zero. > > Yeah, it's a good point. Both zlib and l

Re: [PATCH v3] btrfs: add missing memset while reading compressed inline extents

rom the argument list instead of hardcoding zero. Yeah, it's a good point. Both zlib and lzo are assuming a zero pg_offset right now, but just like there are wacky corners allowing inline extents followed by more data, there are a few wacky corners allowing inline extents at the end of the f

Re: [PATCH v3] btrfs: add missing memset while reading compressed inline extents

On Thu, Mar 09, 2017 at 10:39:49AM -0500, Chris Mason wrote: > > > On 03/08/2017 09:12 PM, Zygo Blaxell wrote: > >This is a story about 4 distinct (and very old) btrfs bugs. > > > > Really great write up. > > [ ... ] > > > > >diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c > >index

Re: [PATCH v3] btrfs: add missing memset while reading compressed inline extents

On 03/08/2017 09:12 PM, Zygo Blaxell wrote: This is a story about 4 distinct (and very old) btrfs bugs. Really great write up. [ ... ] diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 25ac2cf..4d41a31 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -6805,6 +6805,12 @@

Re: [PATCH] btrfs: add missing memset while reading compressed inline extents

very old) btrfs bugs. > > > > Commit c8b978188c ("Btrfs: Add zlib compression support") added > > three data corruption bugs for inline extents (bugs #1-3). > > > > Commit 93c82d5750 ("Btrfs: zero page past end of inline file items") > > fixed bug #1:

[PATCH v3] btrfs: add missing memset while reading compressed inline extents

This is a story about 4 distinct (and very old) btrfs bugs. Commit c8b978188c ("Btrfs: Add zlib compression support") added three data corruption bugs for inline extents (bugs #1-3). Commit 93c82d5750 ("Btrfs: zero page past end of inline file items") fixed bug #1: uncompr

Re: [PATCH] btrfs: add missing memset while reading compressed inline extents

;) added > three data corruption bugs for inline extents (bugs #1-3). > > Commit 93c82d5750 ("Btrfs: zero page past end of inline file items") > fixed bug #1: uncompressed inline extents followed by a hole and more > extents could get non-zero data in the hole as they

[PATCH] btrfs: add missing memset while reading compressed inline extents

From: Zygo Blaxell <ce3g8...@umail.furryterror.org> This is a story about 4 distinct (and very old) btrfs bugs. Commit c8b978188c ("Btrfs: Add zlib compression support") added three data corruption bugs for inline extents (bugs #1-3). Commit 93c82d5750 ("Btrfs: zero page pa

Re: [PATCH] btrfs: fix hole read corruption for compressed inline extents

Hi Zygo, On Tue, Dec 13, 2016 at 01:40:13AM -0500, Zygo Blaxell wrote: > On Sun, Dec 11, 2016 at 10:56:59PM +0100, Xin Zhou wrote: > >Hi Zygo, > >  > >Since the corruption happens after I/O and checksum, > >could it be possible to add some bug catcher code in debug build, to help

Re: [PATCH] Btrfs: fix file corruption after cloning inline extents

; cloning function as well) we end up allowing copy an inline extent from >> >> > the source file into a non-zero offset of the destination file. This is >> >> > something not expected and that the btrfs code is not prepared to deal >> >> > with - all in

Re: [PATCH] Btrfs: fix file corruption after cloning inline extents

file into a non-zero offset of the destination file. This is > >> > something not expected and that the btrfs code is not prepared to deal > >> > with - all inline extents must be at a file offset equals to 0. > >> > > >> > >> Somehow I failed

Re: [PATCH] Btrfs: fix file corruption after cloning inline extents

> Using the clone ioctl (or extent_same ioctl, which calls the same extent >> > cloning function as well) we end up allowing copy an inline extent from >> > the source file into a non-zero offset of the destination file. This is >> > something not expected and that

Re: [PATCH] Btrfs: fix file corruption after cloning inline extents

oning function as well) we end up allowing copy an inline extent from > > the source file into a non-zero offset of the destination file. This is > > something not expected and that the btrfs code is not prepared to deal > > with - all inline extents must be at a file offset equals to 0

Re: [PATCH] Btrfs: fix file corruption after cloning inline extents

file into a non-zero offset of the destination file. This is > something not expected and that the btrfs code is not prepared to deal > with - all inline extents must be at a file offset equals to 0. > Somehow I failed to reproduce the BUG_ON with this case. > For example, the

Re: [PATCH] btrfs: fix hole read corruption for compressed inline extents

_extent will return uninitialized kernel memory > instead of zero bytes in the hole. > > Commit 93c82d5750 ("Btrfs: zero page past end of inline file items") > fills the hole by memset to zero after *uncompressed* inline extents. > > This patch provides the missing memse

Re: [PATCH] btrfs: fix hole read corruption for compressed inline extents

tion can only occur if the first extent (the inline extent at offset 0) is compressed (encoded). Uncompressed inline extents (like the one above) will not be corrupted due to the fix in commit 93c82d5750. If commit 93c82d5750 is reverted, you can get corruption on uncompressed files too.

Re: [PATCH] btrfs: fix hole read corruption for compressed inline extents

rryterror.org> To: "Roman Mamedov" <r...@romanrm.net>, "Filipe Manana" <fdman...@gmail.com> Cc: linux-btrfs@vger.kernel.org Subject: Re: [PATCH] btrfs: fix hole read corruption for compressed inline extents Ping? I know at least two people have read this patch, but it h

Re: [PATCH] btrfs: fix hole read corruption for compressed inline extents

> + char *map = kmap(page); > > + memset(map + max_size, 0, PAGE_SIZE - max_size); > > + kunmap(page); > > + } > > kfree(tmp); > > return ret; > > } > > Wasn't this already posted as: > > btrfs: fix silent

Re: [PATCH] btrfs: fix hole read corruption for compressed inline extents

; > > + } > > kfree(tmp); > > return ret; > > } > > Wasn't this already posted as: > > btrfs: fix silent data corruption while reading compressed inline extents > https://patchwork.kernel.org/patch/9371971/ > > but you don't indicate t

Re: [PATCH] btrfs: fix hole read corruption for compressed inline extents

x_size < PAGE_SIZE) { > + char *map = kmap(page); > + memset(map + max_size, 0, PAGE_SIZE - max_size); > + kunmap(page); > + } > kfree(tmp); > return ret; > } Wasn't this already posted as: btrfs: fix si

[PATCH] btrfs: fix hole read corruption for compressed inline extents

o page past end of inline file items") fills the hole by memset to zero after *uncompressed* inline extents. This patch provides the missing memset for holes after *compressed* inline extents. The offending holes appear in the wild and will appear during routine data integrity audits (e.g. compa

Re: [PATCH] btrfs: fix silent data corruption while reading compressed inline extents

east as old as 3.5.7 (the oldest kernel I can conveniently > > test), and possibly much older. > > > > The code may not be correct if the extent is larger than a page, so add > > a WARN_ON for that case. > > > > To reproduce the bug, run this on a 3072M kvm VM: >

Re: [PATCH] btrfs: fix silent data corruption while reading compressed inline extents

ur favorite block device here > blk=/dev/vdc > > # Create test filesystem and mount point > mkdir -p /try > mkfs.btrfs -dsingle -mdup -O ^extref,^skinny-metadata,^no-holes -f > "$blk" || exit 1 > mount -ocompress-force,flushon

[PATCH] btrfs: fix silent data corruption while reading compressed inline extents

|| exit 1 mount -ocompress-force,flushoncommit,max_inline=8192,noatime "$blk" /try || exit 1 # Create a few inline extents in larger files. # Multiple processes seem to be necessary. y=/usr; for x in $(seq 10 19); do rsync -axHSWI "$y/." "$x"

Re: mixed inline, non-inline extents leading to EIO when reading small files

ib,flushoncommit,space_cache,skip_balance,commit=300 > > Am I missing anything? I've got this queued up to send out. We hit this with holes instead of extents, and without compression, but its a similar problem: Btrfs: deal with duplciates during extent_map insertion in btrfs_get_exten

mixed inline, non-inline extents leading to EIO when reading small files

I frequently see these in /etc/lvm/backup/*. Something that LVM does when it writes these files triggers the problem. This problem occurs in kernels 3.18..4.4.11 (i.e. all the kernels I've tested). btrfs-debug-tree finds this: item 26 key (2702988 INODE_ITEM 0) itemoff 12632 itemsize

[PATCH V9 09/12] Btrfs: Limit inline extents to root->sectorsize

cow_file_range_inline() limits the size of an inline extent to PAGE_CACHE_SIZE. This breaks in subpagesize-blocksize scenarios. Fix this by comparing against root->sectorsize. Reviewed-by: Josef Bacik Signed-off-by: Chandan Rajendra ---

[PATCH V8 09/13] Btrfs: Limit inline extents to root->sectorsize

cow_file_range_inline() limits the size of an inline extent to PAGE_CACHE_SIZE. This breaks in subpagesize-blocksize scenarios. Fix this by comparing against root->sectorsize. Reviewed-by: Josef Bacik Signed-off-by: Chandan Rajendra ---

[PATCH V7 09/13] Btrfs: Limit inline extents to root->sectorsize

cow_file_range_inline() limits the size of an inline extent to PAGE_CACHE_SIZE. This breaks in subpagesize-blocksize scenarios. Fix this by comparing against root->sectorsize. Reviewed-by: Josef Bacik Signed-off-by: Chandan Rajendra ---

[PATCH V6 09/13] Btrfs: Limit inline extents to root->sectorsize

cow_file_range_inline() limits the size of an inline extent to PAGE_CACHE_SIZE. This breaks in subpagesize-blocksize scenarios. Fix this by comparing against root->sectorsize. Reviewed-by: Josef Bacik Signed-off-by: Chandan Rajendra ---

[PATCH] Btrfs: fix file corruption and data loss after cloning inline extents

ing_inline_extents "-O no-holes" "-o compress" status=0 exit Signed-off-by: Filipe Manana <fdman...@suse.com> --- fs/btrfs/ioctl.c | 192 ++- 1 file changed, 149 insertions(+), 43 deletions(-) diff --git a/fs/btr

[PATCH 1/2] fstests: btrfs test for cloning of inline extents

From: Filipe Manana <fdman...@suse.com> Test several cases of cloning inline extents that used to lead to file corruption or data loss. These file corruption and data loss cases are fixed by the linux kernel patch titled: "Btrfs: fix file corruption and data loss after cloning inl

[PATCH V5 09/13] Btrfs: Limit inline extents to root->sectorsize

cow_file_range_inline() limits the size of an inline extent to PAGE_CACHE_SIZE. This breaks in subpagesize-blocksize scenarios. Fix this by comparing against root->sectorsize. Signed-off-by: Chandan Rajendra --- fs/btrfs/inode.c | 2 +- 1 file changed, 1

[RFC PATCH V4 09/13] Btrfs: Limit inline extents to root->sectorsize

cow_file_range_inline() limits the size of an inline extent to PAGE_CACHE_SIZE. This breaks in subpagesize-blocksize scenarios. Fix this by comparing against root->sectorsize. Signed-off-by: Chandan Rajendra --- fs/btrfs/inode.c | 2 +- 1 file changed, 1

[PATCH V3 09/11] Btrfs: Limit inline extents to root-sectorsize

cow_file_range_inline() limits the size of an inline extent to PAGE_CACHE_SIZE. This breaks in subpagesize-blocksize scenarios. Fix this by comparing against root-sectorsize. Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com --- fs/btrfs/inode.c | 2 +- 1 file changed, 1 insertion(+), 1

[PATCH V2 09/11] Btrfs: Limit inline extents to root-sectorsize

cow_file_range_inline() limits the size of an inline extent to PAGE_CACHE_SIZE. This breaks in subpagesize-blocksize scenarios. Fix this by comparing against root-sectorsize. Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com --- fs/btrfs/inode.c | 2 +- 1 file changed, 1 insertion(+), 1

[PATCH 09/11] Btrfs: Limit inline extents to root-sectorsize

cow_file_range_inline() limits the size of an inline extent to PAGE_CACHE_SIZE. This breaks in subpagesize-blocksize scenarios. Fix this by comparing against root-sectorsize. Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com --- fs/btrfs/inode.c | 2 +- 1 file changed, 1 insertion(+), 1

[PATCH] Btrfs: fix file corruption after cloning inline extents

code is not prepared to deal with - all inline extents must be at a file offset equals to 0. For example, the following excerpt of a test case for fstests triggers a crash/BUG_ON() on a write operation after an inline extent is cloned into a non-zero offset: _scratch_mkfs $seqres.full 21

[RFC PATCH V11 20/21] Btrfs: subpagesize-blockssize: Limit inline extents to root-sectorsize.

cow_file_range_inline() limits the size of an inline extent to PAGE_CACHE_SIZE. This breaks in subpagesize-blocksize scenarios. Fix this by comparing against root-sectorsize. Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com --- fs/btrfs/inode.c | 2 +- 1 file changed, 1 insertion(+), 1

[PATCH] Btrfs: delete inline extents when we find them during logging

Apparently when we do inline extents we allow the data to overlap the last chunk of the btrfs_file_extent_item, which means that we can possibly have a btrfs_file_extent_item that isn't actually as large as a btrfs_file_extent_item. This messes with us when we try to overwrite the extent when

[PATCH] Btrfs: fix decompressing of snappy-compressed inline extents

The first four bytes is the length of all data chunks, and the first four bytes of each chunk is the length of compressed chunk data, even when there's only one chunk, which is the case for inline extents. Signed-off-by: Li Zefan l...@cn.fujitsu.com --- fs/btrfs/snappy.c |4 1 files

clone ioctl bug with inline extents

Hi all, I'm hitting a problem cloning inline extents that I haven't had much success tracking down. It's simple enough to reproduce: echo src echo 2 dst clone_range src 0 29 dst 0 cmp src dst # fails! dst is size 29 but contains 2\n\0\0\0\0... where

Re: clone ioctl bug with inline extents

On Tue, 9 Aug 2011, Sage Weil wrote: Hi all, I'm hitting a problem cloning inline extents that I haven't had much success tracking down. It's simple enough to reproduce: echo src echo 2 dst clone_range src 0 29 dst 0 cmp src dst # fails! dst

[PATCH] Btrfs: release reservations properly when doing inline extents

We were only releasing our metadata reservations when doing inline extents, which isn't correct since we don't need our data reservation either. So call btrfs_delalloc_release_space() instead of btrfs_delalloc_release_metadata(). This would have been caught earlier but we don't check