Re: Support demand on Btrfs crashed fs.

2021-03-31 Thread Thierry Testeur
WDC Red with Firmware Version: 80.00A80 Exact, 8 of thems on my array, and have forgotten to disable write cache :/ And all that you decribed is my case. I understand that it could be a very hard situation for expect an massive data recovery, even if the datas are still here, the fs structure is

Re: Any ideas what this warnings are about?

2021-03-31 Thread Markus Schaaf
Am 31.03.21 um 03:58 schrieb Zygo Blaxell: [ cut here ] WARNING: CPU: 1 PID: 314 at fs/fs-writeback.c:2472 __writeback_inodes_sb_nr+0xb8/0xd0 Modules linked in: iTCO_wdt wireguard curve25519_x86_64 libchacha20poly1305 intel_pmc_bxt iTCO_vendor_support chacha_x86_64 poly

[PATCH] btrfs: fix exhaustion of the system chunk array due to concurrent allocations

2021-03-31 Thread fdmanana
From: Filipe Manana When we are running out of space for updating the chunk tree, that is, when we are low on available space in the system space info, if we have many task concurrently allocating block groups, via fallocate for example, many of them can end up all allocating new system chunks wh

[PATCH] btrfs: improve btree readahead for full send operations

2021-03-31 Thread fdmanana
From: Filipe Manana Currently a full send operation uses the standard btree readahead when iterating over the subvolume/snapshot btree, which despite bringing good performance benefits, it could be improved in a few aspects for use cases such as full send operations, which are guaranteed to visit

Re: [PATCH] btrfs: make reflinks respect O_SYNC O_DSYNC and S_SYNC flags

2021-03-31 Thread Filipe Manana
On Mon, Mar 29, 2021 at 7:49 PM David Sterba wrote: > > On Tue, Mar 23, 2021 at 06:39:49PM +, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > If we reflink to or from a file opened with O_SYNC/O_DSYNC or to/from a > > file that has the S_SYNC attribute set, we totally ignore that a

Re: Support demand on Btrfs crashed fs.

2021-03-31 Thread Lukas Straub
On Wed, 31 Mar 2021 02:17:48 +0200 Thierry Testeur wrote: > Hello, > > if anyone can help me with the problem above? > Have tried a Photorec (even if i know the chance are really poor), and > have got some non-sens files, lkie pdf of 2Gb, most of them are > unusable, except smal size file,

Re: [GIT PULL][PATCH v9 0/3] Update to zstd-1.4.10

2021-03-31 Thread Oleksandr Natalenko
Hello. On Tue, Mar 30, 2021 at 03:51:09PM -0700, Nick Terrell wrote: > From: Nick Terrell > > Please pull from > > g...@github.com:terrelln/linux.git tags/v9-zstd-1.4.10 > > to get these changes. Alternatively the patchset is included. > > This patchset upgrades the zstd library to the late

Re[4]: Filesystem sometimes Hangs

2021-03-31 Thread Hendrik Friedel
Hello Chris, thanks again for your reply. 5.10.0-0.bpo.3-amd64 It's probably OK. I'm not sure what upstream stable version this translates into, but current stable are 5.10.27 and 5.11.11. There have been multiple btrfs bug fixes since 5.10.0 was released. I missed in your first email thi

Re: Re[4]: Filesystem sometimes Hangs

2021-03-31 Thread Chris Murphy
On Wed, Mar 31, 2021 at 8:03 AM Hendrik Friedel wrote: > >>[Mo Mär 29 09:29:21 2021] BTRFS info (device sdc2): turning on sync discard > > > >Remove the discard mount option for this file system and see if that > >fixes the problem. Run it for a week or two, or until you're certain > >the proble

Re: Any ideas what this warnings are about?

2021-03-31 Thread Chris Murphy
On Wed, Mar 31, 2021 at 4:33 AM Markus Schaaf wrote: > > Am 31.03.21 um 03:58 schrieb Zygo Blaxell: > > >> [ cut here ] > >> WARNING: CPU: 1 PID: 314 at fs/fs-writeback.c:2472 > >> __writeback_inodes_sb_nr+0xb8/0xd0 > >> Modules linked in: iTCO_wdt wireguard curve25519_x86

Re: Support demand on Btrfs crashed fs.

2021-03-31 Thread Thierry Testeur
Yep, compression enabled (original fstab before having tried restore options): compress=lzo Best regards, Thierry Le mer. 31 mars 2021 à 14:23, Lukas Straub a écrit : > > On Wed, 31 Mar 2021 02:17:48 +0200 > Thierry Testeur wrote: > > > Hello, > > > > if anyone can help me with the problem abov

Re: [PATCH v3 00/13] btrfs: support read-write for subpage metadata

2021-03-31 Thread Qu Wenruo
On 2021/3/30 上午2:53, David Sterba wrote: On Thu, Mar 25, 2021 at 03:14:32PM +0800, Qu Wenruo wrote: v3: - Rename the sysfs to supported_sectorsizes - Rebased to latest misc-next branch This removes 2 cleanup patches. - Add new overview comment for subpage metadata V3 is now in for-next

Re: [PATCH v3 05/10] fsdax: Replace mmap entry in case of CoW

2021-03-31 Thread Ritesh Harjani
On 21/03/19 09:52AM, Shiyang Ruan wrote: > We replace the existing entry to the newly allocated one in case of CoW. > Also, we mark the entry as PAGECACHE_TAG_TOWRITE so writeback marks this > entry as writeprotected. This helps us snapshots so new write > pagefaults after snapshots trigger a CoW.

Re: [PATCH v3 06/10] fsdax: Add dax_iomap_cow_copy() for dax_iomap_zero

2021-03-31 Thread Ritesh Harjani
On 21/03/19 09:52AM, Shiyang Ruan wrote: > Punch hole on a reflinked file needs dax_copy_edge() too. Otherwise, > data in not aligned area will be not correct. So, add the srcmap to > dax_iomap_zero() and replace memset() as dax_copy_edge(). > > Signed-off-by: Shiyang Ruan > --- > fs/dax.c

[PATCH v9 2/9] fs: add O_ALLOW_ENCODED open flag

2021-03-31 Thread Omar Sandoval
From: Omar Sandoval The upcoming RWF_ENCODED operation introduces some security concerns: 1. Compressed writes will pass arbitrary data to decompression algorithms in the kernel. 2. Compressed reads can leak truncated/hole punched data. Therefore, we need to require privilege for RWF_ENCODED

[PATCH v9 0/9] fs: interface for directly reading/writing compressed data

2021-03-31 Thread Omar Sandoval
From: Omar Sandoval This series adds an API for reading compressed data on a filesystem without decompressing it as well as support for writing compressed data directly to the filesystem. I have test cases (including fsstress support) and example programs which I'll send up once the dust settles

[PATCH v9 5/9] btrfs: add ram_bytes and offset to btrfs_ordered_extent

2021-03-31 Thread Omar Sandoval
From: Omar Sandoval Currently, we only create ordered extents when ram_bytes == num_bytes and offset == 0. However, RWF_ENCODED writes may create extents which only refer to a subset of the full unencoded extent, so we need to plumb these fields through the ordered extent infrastructure and pass

[PATCH v9 3/9] fs: add RWF_ENCODED for reading/writing compressed data

2021-03-31 Thread Omar Sandoval
From: Omar Sandoval Btrfs supports transparent compression: data written by the user can be compressed when written to disk and decompressed when read back. However, we'd like to add an interface to write pre-compressed data directly to the filesystem, and the matching interface to read compresse

[PATCH v9 1/9] iov_iter: add copy_struct_from_iter()

2021-03-31 Thread Omar Sandoval
From: Omar Sandoval This is essentially copy_struct_from_user() but for an iov_iter. Suggested-by: Aleksa Sarai Reviewed-by: Josef Bacik Signed-off-by: Omar Sandoval --- include/linux/uio.h | 1 + lib/iov_iter.c | 91 + 2 files changed, 92 in

[PATCH v9 4/9] btrfs: don't advance offset for compressed bios in btrfs_csum_one_bio()

2021-03-31 Thread Omar Sandoval
From: Omar Sandoval btrfs_csum_one_bio() loops over each filesystem block in the bio while keeping a cursor of its current logical position in the file in order to look up the ordered extent to add the checksums to. However, this doesn't make much sense for compressed extents, as a sector on disk

[PATCH v9 6/9] btrfs: support different disk extent size for delalloc

2021-03-31 Thread Omar Sandoval
From: Omar Sandoval Currently, we always reserve the same extent size in the file and extent size on disk for delalloc because the former is the worst case for the latter. For RWF_ENCODED writes, we know the exact size of the extent on disk, which may be less than or greater than (for bookends) t

[PATCH v9 8/9] btrfs: implement RWF_ENCODED reads

2021-03-31 Thread Omar Sandoval
From: Omar Sandoval There are 4 main cases: 1. Inline extents: we copy the data straight out of the extent buffer. 2. Hole/preallocated extents: we fill in zeroes. 3. Regular, uncompressed extents: we read the sectors we need directly from disk. 4. Regular, compressed extents: we read the ent

[PATCH v9 9/9] btrfs: implement RWF_ENCODED writes

2021-03-31 Thread Omar Sandoval
From: Omar Sandoval The implementation resembles direct I/O: we have to flush any ordered extents, invalidate the page cache, and do the io tree/delalloc/extent map/ordered extent dance. From there, we can reuse the compression code with a minor modification to distinguish the write from writebac

[PATCH v9 7/9] btrfs: optionally extend i_size in cow_file_range_inline()

2021-03-31 Thread Omar Sandoval
From: Omar Sandoval Currently, an inline extent is always created after i_size is extended from btrfs_dirty_pages(). However, for encoded writes, we only want to update i_size after we successfully created the inline extent. Add an update_i_size parameter to cow_file_range_inline() and insert_inl

Re: [PATCH v2] vfs: fix fsconfig(2) LSM mount option handling for btrfs

2021-03-31 Thread Christoph Hellwig
On Mon, Mar 29, 2021 at 11:00:39AM +0200, Ondrej Mosnacek wrote: > After taking a closer look, it seems this won't actually work... The > problem is that since btrfs still uses the legacy mount API, it has no > way to get to fs_context in btrfs_mount() and thus both of your > suggestions aren't rea