Re: nvme+btrfs+compression sensibility and benchmark

On 18.04.2018 18:10, Brendan Hide wrote: > Hi, all > > I'm looking for some advice re compression with NVME. Compression helps > performance with a minor CPU hit - but is it still worth it with the far > higher throughputs offered by newer PCI and NVME-type SSDs? > > I've ordered a PCIe-to-M.2

Re: [PATCH v2] btrfs: add comment about BTRFS_FS_EXCL_OP

On Wed, Apr 18, 2018 at 02:59:25PM +0800, Anand Jain wrote: > Adds comments about BTRFS_FS_EXCL_OP to existing comments > about the device locks. > > Signed-off-by: Anand Jain Reviewed-by: David Sterba -- To unsubscribe from this list: send the line

nvme+btrfs+compression sensibility and benchmark

Hi, all I'm looking for some advice re compression with NVME. Compression helps performance with a minor CPU hit - but is it still worth it with the far higher throughputs offered by newer PCI and NVME-type SSDs? I've ordered a PCIe-to-M.2 adapter along with a 1TB 960 Evo drive for my home

Re: Kernel unaligned access at ... btrfs_real_readdir+0x51c/0x718 [btrfs]

On Tue, Apr 17, 2018 at 12:18:45PM +0200, René Rebe wrote: > On 16 Apr 2018, at 21:15, David Sterba wrote: > > On Mon, Apr 16, 2018 at 09:55:45PM +0200, René Rebe wrote: > >> On 04/16/2018 06:48 PM, David Sterba wrote: > >>> The warnings are valid, there's unaligned access

[PATCH] btrfs: fix unaligned access in readdir

The last update to readdir introduced a temporary buffer to store the emitted readdir data, but as there are file names of variable length, there's a lot of unaligned access. This was observed on a sparc64 machine: Kernel unaligned access at TPC[102f3080] btrfs_real_readdir+0x51c/0x718

Re: [PATCH] btrfs: optimize free space tree bitmap conversion

On Tue, Apr 17, 2018 at 04:19:04PM -0700, Howard McLauchlan wrote: > Presently, convert_free_space_to_extents() does a linear scan of the > bitmap. We can speed this up with find_next_{bit,zero_bit}_le(). > > This patch replaces the linear scan with find_next_{bit,zero_bit}_le(). > Testing shows

Re: nvme+btrfs+compression sensibility and benchmark

On 2018-04-18 11:10, Brendan Hide wrote: Hi, all I'm looking for some advice re compression with NVME. Compression helps performance with a minor CPU hit - but is it still worth it with the far higher throughputs offered by newer PCI and NVME-type SSDs? I've ordered a PCIe-to-M.2 adapter

Re: Filesystem full with unallocated space?

On 17.04.2018 19:08, Matthew Lai wrote: > Hello! > > I am getting ENOSPC on my root filesystem with plenty of unallocated > space according to "fi usage". Any idea why that may be? This is a root > partition for Debian Stable. Not doing anything unusual that I'm aware > of. No snapshots. > >

Re: [PATCH] btrfs: fix unaligned access in readdir

On Wed, Apr 18, 2018 at 6:17 AM, David Sterba wrote: > The last update to readdir introduced a temporary buffer to store the > emitted readdir data, but as there are file names of variable length, > there's a lot of unaligned access. > > This was observed on a sparc64 machine: >

Re: [PATCH v3] btrfs: Fix race condition between delayed refs and blockgroup removal

On Wed, Apr 18, 2018 at 09:41:54AM +0300, Nikolay Borisov wrote: > When the delayed refs for a head are all run, eventually > cleanup_ref_head is called which (in case of deletion) obtains a > reference for the relevant btrfs_space_info struct by querying the bg > for the range. This is

[PATCH 1/2] btrfs: remove le_bitmap_clear()

le_bitmap_clear() was implemented as a parallel to le_bitmap_set() but is actually not being used. This patch removes it. Signed-off-by: Howard McLauchlan --- fs/btrfs/extent_io.c | 20 fs/btrfs/extent_io.h | 1 - 2 files changed, 21 deletions(-) diff

[PATCH 2/2] btrfs: optimize free space tree bitmap conversion

Presently, convert_free_space_to_extents() does a linear scan of the bitmap. We can speed this up with find_next_{bit,zero_bit}_le(). This patch replaces the linear scan with find_next_{bit,zero_bit}_le(). Testing shows a 20-33% decrease in execution time for convert_free_space_to_extents().

Re: nvme+btrfs+compression sensibility and benchmark

On Wed, Apr 18, 2018 at 06:14:07PM +0300, Nikolay Borisov wrote: > > > On 18.04.2018 18:10, Brendan Hide wrote: > > Hi, all > > > > I'm looking for some advice re compression with NVME. Compression helps > > performance with a minor CPU hit - but is it still worth it with the far > > higher

Re: [PATCH] btrfs: optimize free space tree bitmap conversion

On 04/18/2018 08:50 AM, David Sterba wrote: > On Tue, Apr 17, 2018 at 04:19:04PM -0700, Howard McLauchlan wrote: >> Presently, convert_free_space_to_extents() does a linear scan of the >> bitmap. We can speed this up with find_next_{bit,zero_bit}_le(). >> >> This patch replaces the linear scan

Re: nvme+btrfs+compression sensibility and benchmark

On Wed, Apr 18, 2018 at 10:38 AM, Austin S. Hemmelgarn wrote: > For reference, the zstd compression in BTRFS uses level 3 by default (as > does zlib compression IIRC), though I'm not sure about lzop (I think it > uses the lowest compression setting). > The user space

Re: [PATCH v5 0/3] Allow rmdir(2) to delete a subvolume

On Wed, Apr 18, 2018 at 11:32:57AM +0900, Misono Tomohiro wrote: > cangelog: > v4 -> v5 ... Merge 2nd and 3rd patches and update commit log >No code change in total > v3 -> v4 ... Reorganize patches and update commit log >No code change in total > v2 -> v3 ...

Re: nvme+btrfs+compression sensibility and benchmark

Thank you, all Though the info is useful, there's not a clear consensus on what I should expect. For interest's sake, I'll post benchmarks from the device itself when it arrives. I'm expecting at least that I'll be blown away :) On 04/18/2018 09:23 PM, Chris Murphy wrote: On Wed, Apr 18,

Re: nvme+btrfs+compression sensibility and benchmark

On 18.04.2018 21:28, David Sterba wrote: > On Wed, Apr 18, 2018 at 06:14:07PM +0300, Nikolay Borisov wrote: >> >> >> On 18.04.2018 18:10, Brendan Hide wrote: >>> Hi, all >>> >>> I'm looking for some advice re compression with NVME. Compression helps >>> performance with a minor CPU hit - but is

Re: [PATCH] btrfs: delayed-inode: Remove wrong qgroup meta reservation calls

On Wed, Apr 18, 2018 at 09:08:10AM +0800, Qu Wenruo wrote: > >>dst_rsv = _info->delayed_block_rsv; > >> > >>num_bytes = btrfs_calc_trans_metadata_size(fs_info, 1); > >> + > >> + /* > >> + * Here we migrate space rsv from transaction rsv, since have > >> + * already reserved space

Re: [PATCH v2] btrfs: Rewrite retry logic in do_chunk_alloc

On 18.04.2018 10:27, Nikolay Borisov wrote: > do_chunk_alloc implements logic to detect whether there is currently > pending chunk allocation (by means of space_info->chunk_alloc being > set) and if so it loops around to the 'again' label. Additionally, > based on the state of the space_info

Re: Hard link not persisted on fsync

On Wed, Apr 18, 2018 at 1:02 AM, Jayashree Mohan wrote: > Hi, > > A gentle reminder on the crash consistency bug that we found on btrfs: Why do you call it a consistency bug? The filesystem does not stay in inconsistent state. The link count stays 1 and the dentry used

[PATCH] btrfs: update uuid_mutex and device_list_mutex comments

Make the uuid_mutex and device_list_mutex comments inline with the changes. Signed-off-by: Anand Jain --- Hi David, Can you kindly add this patch to the set: 'Review uuid_mutex usage' Thanks, Anand fs/btrfs/volumes.c | 21 + 1 file changed, 9

[PATCH v3] btrfs-progs: dump-tree: add degraded option

From: Anand Jain btrfs inspect dump-tree picks the disk with the largest generation to read the root tree by scanning for the required devices by default. But in 2 or more disks RAID1/5/6 you may need to know what's in the disks individually, so this option --noscan

[PATCH v3] btrfs: Fix race condition between delayed refs and blockgroup removal

When the delayed refs for a head are all run, eventually cleanup_ref_head is called which (in case of deletion) obtains a reference for the relevant btrfs_space_info struct by querying the bg for the range. This is problematic because when the last extent of a bg is deleted a race window emerges

[PATCH v2] btrfs: add comment about BTRFS_FS_EXCL_OP

Adds comments about BTRFS_FS_EXCL_OP to existing comments about the device locks. Signed-off-by: Anand Jain --- v1->v2: add more clarity to the comments added. fs/btrfs/volumes.c | 32 1 file changed, 32 insertions(+) diff --git

[PATCH] btrfs-progs: restore: fix mistake on overwrite_ok() if relative path is given

From: Ting-Chang Hou fstatat will return -1 with errno EBADF if path_name is relative path. This caused an error of the return value of overwrite_ok(). When restoring the subvolume to destination with relative path, it will overwrite the existing file rather than skip it.

Re: [PATCH] btrfs: add chattr support for send/receive

On Wed, Apr 18, 2018 at 12:39 AM, Howard McLauchlan wrote: > Presently btrfs send/receive does not propagate inode attribute flags; > all chattr operations are effectively discarded upon transmission. > > This patch adds kernel support for inode attribute flags. Userspace >

Re: [PATCH] btrfs: add chattr support for send/receive

On Tue, Apr 17, 2018 at 04:39:35PM -0700, Howard McLauchlan wrote: > Presently btrfs send/receive does not propagate inode attribute flags; > all chattr operations are effectively discarded upon transmission. > > This patch adds kernel support for inode attribute flags. Userspace > support can be

[PATCH v2] btrfs: Rewrite retry logic in do_chunk_alloc

do_chunk_alloc implements logic to detect whether there is currently pending chunk allocation (by means of space_info->chunk_alloc being set) and if so it loops around to the 'again' label. Additionally, based on the state of the space_info (e.g. whether it's full or not) and the return value of

Re: [PATCH v3] btrfs: Fix race condition between delayed refs and blockgroup removal

On 18.04.2018 23:47, David Sterba wrote: > On Wed, Apr 18, 2018 at 09:41:54AM +0300, Nikolay Borisov wrote: >> When the delayed refs for a head are all run, eventually >> cleanup_ref_head is called which (in case of deletion) obtains a >> reference for the relevant btrfs_space_info struct by

Re: [PATCH 2/2] btrfs: optimize free space tree bitmap conversion

On 04/18/2018 03:26 PM, David Sterba wrote: > On Wed, Apr 18, 2018 at 02:30:59PM -0700, Howard McLauchlan wrote: >> Presently, convert_free_space_to_extents() does a linear scan of the >> bitmap. We can speed this up with find_next_{bit,zero_bit}_le(). >> >> This patch replaces the linear scan

Re: d_instantiate() and unlock_new_inode() order in btrfs_mkdir()

On Wed, Apr 18, 2018 at 05:00:29PM -0700, Eric Biggers wrote: > Hi Chris and other btrfs folks, > > btrfs_mkdir() calls d_instantiate() before unlock_new_inode(), which is wrong > because it exposes the inode to lookups before it's been fully initialized. Huh? It *is* fully initialized by that

Re: d_instantiate() and unlock_new_inode() order in btrfs_mkdir()

On Thu, Apr 19, 2018 at 01:06:13AM +0100, Al Viro wrote: > On Wed, Apr 18, 2018 at 05:00:29PM -0700, Eric Biggers wrote: > > Hi Chris and other btrfs folks, > > > > btrfs_mkdir() calls d_instantiate() before unlock_new_inode(), which is > > wrong > > because it exposes the inode to lookups

Re: [PATCH v3] btrfs: Do super block verification before writing it to disk

On Tue, Apr 17, 2018 at 09:47:19AM +0800, Qu Wenruo wrote: > @@ -2680,7 +2681,7 @@ int open_ctree(struct super_block *sb, > > memcpy(fs_info->fsid, fs_info->super_copy->fsid, BTRFS_FSID_SIZE); > > - ret = btrfs_check_super_valid(fs_info); > + ret =

Re: [PATCH 1/2] btrfs: remove le_bitmap_clear()

On Wed, Apr 18, 2018 at 02:30:58PM -0700, Howard McLauchlan wrote: > le_bitmap_clear() was implemented as a parallel to le_bitmap_set() but > is actually not being used. This patch removes it. > > Signed-off-by: Howard McLauchlan Reviewed-by: David Sterba

Re: [PATCH v3] btrfs: Do super block verification before writing it to disk

On 2018年04月19日 06:04, David Sterba wrote: > On Tue, Apr 17, 2018 at 09:47:19AM +0800, Qu Wenruo wrote: >> @@ -2680,7 +2681,7 @@ int open_ctree(struct super_block *sb, >> >> memcpy(fs_info->fsid, fs_info->super_copy->fsid, BTRFS_FSID_SIZE); >> >> -ret =

d_instantiate() and unlock_new_inode() order in btrfs_mkdir()

Hi Chris and other btrfs folks, btrfs_mkdir() calls d_instantiate() before unlock_new_inode(), which is wrong because it exposes the inode to lookups before it's been fully initialized. Most filesystems get it right, but f2fs and btrfs don't. I sent a f2fs patch

Re: [PATCH 2/2] btrfs: optimize free space tree bitmap conversion

On Wed, Apr 18, 2018 at 02:30:59PM -0700, Howard McLauchlan wrote: > Presently, convert_free_space_to_extents() does a linear scan of the > bitmap. We can speed this up with find_next_{bit,zero_bit}_le(). > > This patch replaces the linear scan with find_next_{bit,zero_bit}_le(). > Testing shows

Re: d_instantiate() and unlock_new_inode() order in btrfs_mkdir()

On Thu, Apr 19, 2018 at 01:15:59AM +0100, Al Viro wrote: > On Thu, Apr 19, 2018 at 01:06:13AM +0100, Al Viro wrote: > > On Wed, Apr 18, 2018 at 05:00:29PM -0700, Eric Biggers wrote: > > > Hi Chris and other btrfs folks, > > > > > > btrfs_mkdir() calls d_instantiate() before unlock_new_inode(),

[PATCH v2 3/3] btrfs: remove le_test_bit()

With commit b18253ec57c0 ("btrfs: optimize free space tree bitmap conversion"), there are no more callers to le_test_bit(). This patch removes le_test_bit(). Signed-off-by: Howard McLauchlan --- fs/btrfs/extent_io.h | 5 - 1 file changed, 5 deletions(-) diff --git

[PATCH v2 2/3] btrfs: optimize free space tree bitmap conversion

Presently, convert_free_space_to_extents() does a linear scan of the bitmap. We can speed this up with find_next_{bit,zero_bit}_le(). This patch replaces the linear scan with find_next_{bit,zero_bit}_le(). Testing shows a 20-33% decrease in execution time for convert_free_space_to_extents().

[PATCH v2 1/3] btrfs: clean up le_bitmap_{set, clear}()

le_bitmap_set() is only used by free-space-tree, so move it there and make it static. le_bitmap_clear() is not used, so remove it. Signed-off-by: Howard McLauchlan --- V1->V2: Also move le_bitmap_set() based on 4.17-rc1 fs/btrfs/extent_io.c | 40