[PULL] volume and uuid_mutex cleanups

Hi David, Here I have put together a set of volume related patches which were sent to the ML as independent patches earlier. These have been reviewed and tested. Please pull. g...@github.com:asj/btrfs-devel.git misc-next-for-kdave - [Anand:2] 6049bd5e9694 btrfs: add helper

Re: [PATCH v3 1/2] btrfs: make fs_devices to be a local variable

On 07/11/2018 09:22 AM, Gu Jinxiang wrote: fs_devices is always passed to btrfs_scan_one_device which overrides it. And in the call stack below fs_devices is passed to btrfs_scan_one_device from btrfs_mount_root. And in btrfs_mount_root the output fs_devices of this call stack is not used.

[DOC] BTRFS Volume operations, Device Lists and Locks all in one page

BTRFS Volume operations, Device Lists and Locks all in one page: Devices are managed in two contexts, the scan context and the mounted context. In scan context the threads originate from the btrfs_control ioctl and in the mounted context the threads originates from the mount point ioctl.

About hung task on generic/041

Hi, When I run generic/041 with v4.18-rc3 (turn on kasan and hung task detection), btrfs-transaction kthread will trigger the hung task timeout (stall at wait_event in btrfs_commit_transaction). At the same time, you can see that xfs_io -c fsync will occupy 100% of the CPU. I am not sure whether

Re: [PATCH v3 2/2] btrfs: get fs_devices pointer form btrfs_scan_one_device

On 07/11/2018 09:22 AM, Gu Jinxiang wrote: Instead of pointer to btrfs_fs_devices as an arg in btrfs_scan_one_device, better to make it as a return value. Yep this was in the list to fix. However I didn't like the idea to return the btrfs_fs_devices pointer, instead return the btrfs_device

Re: btrfs check lowmem, take 2

On Tue, Jul 10, 2018 at 12:09 PM, Marc MERLIN wrote: > Thanks to Su and Qu, I was able to get my filesystem to a point that > it's mountable. > I then deleted loads of snapshots and I'm down to 26. > > IT now looks like this: > gargamel:~# btrfs fi show /mnt/mnt > Label: 'dshelf2' uuid:

Re: btrfs check mode normal still hard crash-hanging systems

On Wed, Jul 11, 2018 at 11:09:56AM -0600, Chris Murphy wrote: > On Tue, Jul 10, 2018 at 12:09 PM, Marc MERLIN wrote: > > Thanks to Su and Qu, I was able to get my filesystem to a point that > > it's mountable. > > I then deleted loads of snapshots and I'm down to 26. > > > > IT now looks like

Re: Corrupted FS with "open_ctree failed" and "failed to recover balance: -5"

On Wed, Jul 11, 2018 at 9:37 AM, Udo Waechter wrote: > Hello everyone, > > I have a corrupted filesystem which I can't seem to recover. > > The machine is: > Debian Linux, kernel 4.9 and btrfs-progs v4.13.3 > > I have a HDD RAID5 with LVM and the volume in question is a LVM volume. > On top of

Corrupted FS with "open_ctree failed" and "failed to recover balance: -5"

Hello everyone, I have a corrupted filesystem which I can't seem to recover. The machine is: Debian Linux, kernel 4.9 and btrfs-progs v4.13.3 I have a HDD RAID5 with LVM and the volume in question is a LVM volume. On top of that I had a RAID1 SSD cache with lvm-cache. Yesterday both! SSDs died

[PATCH v2] btrfs: use customized batch size for total_bytes_pinned

In commit b150a4f10d878 ("Btrfs: use a percpu to keep track of possibly pinned bytes") we use total_bytes_pinned to track how many bytes we are going to free in this transaction. When we are close to ENOSPC, we check it and know if we can make the allocation by commit the current transaction. For

Why original mode doesn't use swap? (Original: Re: btrfs check lowmem, take 2)

On 2018年07月12日 01:09, Chris Murphy wrote: > On Tue, Jul 10, 2018 at 12:09 PM, Marc MERLIN wrote: >> Thanks to Su and Qu, I was able to get my filesystem to a point that >> it's mountable. >> I then deleted loads of snapshots and I'm down to 26. >> >> IT now looks like this: >> gargamel:~#

Re: [DOC] BTRFS Volume operations, Device Lists and Locks all in one page

On 2018年07月11日 15:50, Anand Jain wrote: > > > BTRFS Volume operations, Device Lists and Locks all in one page: > > Devices are managed in two contexts, the scan context and the mounted > context. In scan context the threads originate from the btrfs_control > ioctl and in the mounted context

[PATCH v14.8 09/14] btrfs: dedupe: Implement btrfs_dedupe_calc_hash interface

From: Wang Xiaoguang Unlike in-memory or on-disk dedupe method, only SHA256 hash method is supported yet, so implement btrfs_dedupe_calc_hash() interface using SHA256. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang Reviewed-by: Josef Bacik Signed-off-by: Lu Fengqi ---

[PATCH v14.8 03/14] btrfs: dedupe: Introduce dedupe framework and its header

From: Wang Xiaoguang Introduce the header for btrfs in-band(write time) de-duplication framework and needed header. The new de-duplication framework is going to support 2 different dedupe methods and 1 dedupe hash. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang Signed-off-by: Lu

[PATCH v14.8 08/14] btrfs: dedupe: Introduce function to search for an existing hash

From: Wang Xiaoguang Introduce static function inmem_search() to handle the job for in-memory hash tree. The trick is, we must ensure the delayed ref head is not being run at the time we search the for the hash. With inmem_search(), we can implement the btrfs_dedupe_search() interface.

[PATCH v14.8 13/14] btrfs: relocation: Enhance error handling to avoid BUG_ON

From: Qu Wenruo Since the introduction of btrfs dedupe tree, it's possible that balance can race with dedupe disabling. When this happens, dedupe_enabled will make btrfs_get_fs_root() return PTR_ERR(-ENOENT). But due to a bug in error handling branch, when this happens backref_cache->nr_nodes

[PATCH v14.8 04/14] btrfs: dedupe: Introduce function to initialize dedupe info

From: Wang Xiaoguang Add generic function to initialize dedupe info. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang Reviewed-by: Josef Bacik Signed-off-by: Lu Fengqi --- fs/btrfs/Makefile | 2 +- fs/btrfs/dedupe.c | 174 +

[PATCH v14.8 12/14] btrfs: dedupe: Add ioctl for inband deduplication

From: Wang Xiaoguang Add ioctl interface for inband deduplication, which includes: 1) enable 2) disable 3) status And a pseudo RO compat flag, to imply that btrfs now supports inband dedup. However we don't add any ondisk format change, it's just a pseudo RO compat flag. All these ioctl

[PATCH v14.8 00/14] Btrfs In-band De-duplication

This patchset can be fetched from github: https://github.com/littleroad/linux.git dedupe_latest This is just a normal rebase update. Now the new base is v4.18-rc4 Normal test cases from auto group exposes no regression, and ib-dedupe group can pass without problem. xfstests ib-dedupe group can

[PATCH v14.8 02/14] btrfs: Introduce COMPRESS reserve type to fix false enospc for compression

From: Wang Xiaoguang When testing btrfs compression, sometimes we got ENOSPC error, though fs still has much free space, xfstests generic/171, generic/172, generic/173, generic/174, generic/175 can reveal this bug in my test environment when compression is enabled. After some debugging work, we

[PATCH v14.8 10/14] btrfs: ordered-extent: Add support for dedupe

From: Wang Xiaoguang Add ordered-extent support for dedupe. Note, current ordered-extent support only supports non-compressed source extent. Support for compressed source extent will be added later. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang Reviewed-by: Josef Bacik ---

[PATCH v14.8 01/14] btrfs: introduce type based delalloc metadata reserve

From: Wang Xiaoguang Introduce type based metadata reserve parameter for delalloc space reservation/freeing function. The problem we are going to solve is, btrfs use different max extent size for different mount options. For compression, the max extent size is 128K, while for non-compress

[PATCH v14.8 06/14] btrfs: dedupe: Introduce function to remove hash from in-memory tree

From: Wang Xiaoguang Introduce static function inmem_del() to remove hash from in-memory dedupe tree. And implement btrfs_dedupe_del() and btrfs_dedup_disable() interfaces. Also for btrfs_dedupe_disable(), add new functions to wait existing writer and block incoming writers to eliminate all

[PATCH v14.8 11/14] btrfs: dedupe: Inband in-memory only de-duplication implement

From: Qu Wenruo Core implement for inband de-duplication. It reuses the async_cow_start() facility to do the calculate dedupe hash. And use dedupe hash to do inband de-duplication at extent level. The workflow is as below: 1) Run delalloc range for an inode 2) Calculate hash for the delalloc

[PATCH v14.8 14/14] btrfs: dedupe: Introduce new reconfigure ioctl

From: Qu Wenruo Introduce new reconfigure ioctl and new FORCE flag for in-band dedupe ioctls. Now dedupe enable and reconfigure ioctl are stateful. | Current state | Ioctl| Next state | | Disabled

[PATCH v14.8 07/14] btrfs: delayed-ref: Add support for increasing data ref under spinlock

From: Qu Wenruo For in-band dedupe, btrfs needs to increase data ref with delayed_ref locked, so add a new function btrfs_add_delayed_data_ref_lock() to increase extent ref with delayed_refs already locked. Export init_delayed_ref_head and init_delayed_ref_common for inband dedupe.

[PATCH v10.3 4/5] btrfs-progs: dedupe: Add status subcommand

From: Qu Wenruo Add status subcommand for dedupe command group. Signed-off-by: Qu Wenruo --- Documentation/btrfs-dedupe-inband.asciidoc | 3 + btrfs-completion | 2 +- cmds-dedupe-ib.c | 81 ++ 3 files changed, 85

[PATCH v10.3 3/5] btrfs-progs: dedupe: Add disable support for inband dedupelication

From: Qu Wenruo Add disable subcommand for dedupe command group. Signed-off-by: Qu Wenruo Signed-off-by: Lu Fengqi --- Documentation/btrfs-dedupe-inband.asciidoc | 5 +++ btrfs-completion | 2 +- cmds-dedupe-ib.c | 42

[PATCH v10.3 5/5] btrfs-progs: dedupe: introduce reconfigure subcommand

From: Qu Wenruo Introduce reconfigure subcommand to co-operate with new kernel ioctl modification. Signed-off-by: Qu Wenruo --- Documentation/btrfs-dedupe-inband.asciidoc | 7 ++ cmds-dedupe-ib.c | 75 +- 2 files changed, 66 insertions(+), 16

[PATCH v10.3 1/5] btrfs-progs: Basic framework for dedupe-inband command group

From: Qu Wenruo Add basic ioctl header and command group framework for later use. Alone with basic man page doc. Signed-off-by: Qu Wenruo Signed-off-by: Lu Fengqi --- Documentation/Makefile.in | 1 + Documentation/btrfs-dedupe-inband.asciidoc | 40 ++

[PATCH v10.3 2/5] btrfs-progs: dedupe: Add enable command for dedupe command group

From: Qu Wenruo Add enable subcommand for dedupe commmand group. Signed-off-by: Qu Wenruo Signed-off-by: Lu Fengqi --- Documentation/btrfs-dedupe-inband.asciidoc | 114 +- btrfs-completion | 6 +- cmds-dedupe-ib.c | 241

[PATCH v10.3 0/5] In-band de-duplication for btrfs-progs

Patchset can be fetched from github: https://github.com/littleroad/btrfs-progs.git dedupe_latest Inband dedupe(in-memory backend only) ioctl support for btrfs-progs. v7 changes: Update ctree.h to follow kernel structure change Update print-tree to follow kernel structure change V8 changes:

Re: Transaction aborted (error -28) btrfs_run_delayed_refs*0x163/0x190

On 07/10/2018 09:38 AM, Martin Raiber wrote: > This is probably a known issue. See > https://www.spinics.net/lists/linux-btrfs/msg75647.html > You could apply the patch in this thread and mount with enospc_debug to > confirm it is the same issue. > -- > To unsubscribe from this list: send the