Re: [PATCH] Improve error stats message

2018-03-07 Thread Nikolay Borisov
On 7.03.2018 19:37, Diego wrote: > A typical notification of filesystem errors looks like this: > > BTRFS error (device sda2): bdev /dev/sda2 errs: wr 0, rd 1, flush 0, corrupt > 0, gen 0 > > The device name is being printed twice. Also, these abbreviatures > feel unnecesary. Make the

[PATCH v2] btrfs-progs: free-space-cache: Enhance free space cache free space check

2018-03-07 Thread Qu Wenruo
When we found free space difference between free space cache and block group item, we just discard this free space cache. Normally such difference is caused by btrfs_reserve_extent() called by delalloc which is out of a transaction. And since all btrfs_release_extent() is called with a

Re: How to change/fix 'Received UUID'

2018-03-07 Thread Andrei Borzenkov
08.03.2018 09:06, Marc MERLIN пишет: > On Tue, Mar 06, 2018 at 12:02:47PM -0800, Marc MERLIN wrote: >>> https://github.com/knorrie/python-btrfs/commit/1ace623f95300ecf581b1182780fd6432a46b24d >> >> Well, I had never heard about it until now, thank you. >> >> I'll see if I can make it work when I

[PATCH] btrfs-progs: free-space-cache: Enhance free space cache free space check

2018-03-07 Thread Qu Wenruo
When we found free space difference between free space cache and block group item, we just discard this free space cache. Normally such difference is caused by btrfs_reserve_extent() called by delalloc which is out of a transaction. And since all btrfs_release_extent() is called with a

Re: Inconsistence between sender and receiver

2018-03-07 Thread Andrei Borzenkov
07.03.2018 21:49, Liu Bo пишет: > Hi, > > In the following steps[1], if on receiver side has got > changed via 'btrfs property set', then after doing incremental > updates, receiver gets a different snapshot from what sender has sent. > > The reason behind it is that there is no change about

Re: [PATCH v2] fstests: btrfs/146: make sure hit all stripes in the case of compression

2018-03-07 Thread Eryu Guan
On Thu, Mar 08, 2018 at 01:56:45PM +0800, Lu Fengqi wrote: > In the case of compression, each 128K input data chunk will be compressed > to 4K (because of the characters written are duplicate). Therefore we have > to write (128K * 16) to make sure every stripe can be hit. > > Signed-off-by: Lu

Re: How to change/fix 'Received UUID'

2018-03-07 Thread Marc MERLIN
On Tue, Mar 06, 2018 at 12:02:47PM -0800, Marc MERLIN wrote: > > https://github.com/knorrie/python-btrfs/commit/1ace623f95300ecf581b1182780fd6432a46b24d > > Well, I had never heard about it until now, thank you. > > I'll see if I can make it work when I get a bit of time. Sorry, I missed the

[PATCH v2] fstests: btrfs/146: make sure hit all stripes in the case of compression

2018-03-07 Thread Lu Fengqi
In the case of compression, each 128K input data chunk will be compressed to 4K (because of the characters written are duplicate). Therefore we have to write (128K * 16) to make sure every stripe can be hit. Signed-off-by: Lu Fengqi --- V2: Modify the regular

Re: [PATCH 10/20] btrfs-progs: help: convert ints used as bools to bool

2018-03-07 Thread Qu Wenruo
On 2018年03月08日 10:40, je...@suse.com wrote: > From: Jeff Mahoney > > We use an int for 'full', 'all', and 'err' when we really mean a boolean. > > Signed-off-by: Jeff Mahoney Reviewed-by: Qu Wenruo Thanks, Qu > --- > btrfs.c | 14

Re: [PATCH 08/20] btrfs-progs: qgroups: introduce btrfs_qgroup_query

2018-03-07 Thread Qu Wenruo
On 2018年03月08日 10:40, je...@suse.com wrote: > From: Jeff Mahoney > > The only mechanism we have in the progs for searching qgroups is to load > all of them and filter the results. This works for qgroup show but > to add quota information to 'btrfs subvoluem show' it's pretty

Re: [PATCH 07/20] btrfs-progs: qgroups: introduce and use info and limit structures

2018-03-07 Thread Qu Wenruo
On 2018年03月08日 10:40, je...@suse.com wrote: > From: Jeff Mahoney > > We use structures to pass the info and limit from the kernel as items > but store the individual values separately in btrfs_qgroup. We already > have a btrfs_qgroup_limit structure that's used for setting the

Re: [PATCH 06/20] btrfs-progs: qgroups: add pathname to show output

2018-03-07 Thread Qu Wenruo
On 2018年03月08日 10:40, je...@suse.com wrote: > From: Jeff Mahoney > > The btrfs qgroup show command currently only exports qgroup IDs, > forcing the user to resolve which subvolume each corresponds to. > > This patch adds pathname resolution to qgroup show so that when > the -P

[PATCH 2/3] net: Remove accidental VLAs from proc buffers

2018-03-07 Thread Kees Cook
In the quest to remove all stack VLAs from the kernel[1], this refactors the stack array size calculation to avoid using max(), which makes the compiler think the size isn't fixed. [1] https://lkml.org/lkml/2018/3/7/621 Signed-off-by: Kees Cook --- net/ipv4/proc.c | 10

[PATCH 3/3] btrfs: tree-checker: Avoid accidental stack VLA

2018-03-07 Thread Kees Cook
In the quest to remove all stack VLAs from the kernel[1], this refactors the stack array size calculation to avoid using max(), which makes the compiler think the size isn't fixed. [1] https://lkml.org/lkml/2018/3/7/621 Signed-off-by: Kees Cook ---

[PATCH v2 1/3] vsprintf: Remove accidental VLA usage

2018-03-07 Thread Kees Cook
In the quest to remove all stack VLAs from the kernel[1], this introduces a new "simple max" macro, and changes the "sym" array size calculation to use it. The value is actually a fixed size, but since the max() macro uses some extensive tricks for safety, it ends up looking like a variable size

[PATCH 0/3] Remove accidental VLA usage

2018-03-07 Thread Kees Cook
This series adds SIMPLE_MAX() to be used in places where a stack array is actually fixed, but the compiler still warns about VLA usage due to confusion caused by the safety checks in the max() macro. I'm sending these via -mm since that's where I've introduced SIMPLE_MAX(), and they should all

[PATCH 06/20] btrfs-progs: qgroups: add pathname to show output

2018-03-07 Thread jeffm
From: Jeff Mahoney The btrfs qgroup show command currently only exports qgroup IDs, forcing the user to resolve which subvolume each corresponds to. This patch adds pathname resolution to qgroup show so that when the -P option is used, the last column contains the pathname of

[PATCH 02/20] btrfs-progs: qgroups: fix misleading index check

2018-03-07 Thread jeffm
From: Jeff Mahoney In print_single_qgroup_table we check the loop index against BTRFS_QGROUP_CHILD, but what we really mean is "last column." Since we have an enum value to indicate the last value, use that instead of assuming that BTRFS_QGROUP_CHILD is always last.

[PATCH 05/20] btrfs-progs: btrfs-list: add btrfs_cleanup_root_info

2018-03-07 Thread jeffm
From: Jeff Mahoney Currently we can pass back root_info structures to callers but have to free the strings manually. This adds a helper to do it and uses it in cmd_subvol_show. Signed-off-by: Jeff Mahoney --- btrfs-list.c | 18 +++---

[PATCH 12/20] btrfs-progs: filesystem balance: split out special handling

2018-03-07 Thread jeffm
From: Jeff Mahoney In preparation to use cmd_struct as the command entry point, we need to split out the 'filesystem balance' handling to not call cmd_balance directly. The reason is that the flags that indicate a command is hidden are a part of cmd_struct and so we can use a

[PATCH 11/20] btrfs-progs: reorder placement of help declarations for send/receive

2018-03-07 Thread jeffm
From: Jeff Mahoney The usage definitions for send and receive follow the command definitions, which use them. This works because we declare them in commands.h. When we move to using cmd_struct as the entry point, these declarations will be removed, breaking the commands. Since

[PATCH 03/20] btrfs-progs: constify pathnames passed as arguments

2018-03-07 Thread jeffm
From: Jeff Mahoney It's unlikely we're going to modify a pathname argument, so codify that and use const. Reviewed-by: Qu Wenruo Signed-off-by: Jeff Mahoney --- chunk-recover.c | 4 ++-- cmds-device.c | 2 +- cmds-fi-usage.c | 6 +++---

[PATCH 07/20] btrfs-progs: qgroups: introduce and use info and limit structures

2018-03-07 Thread jeffm
From: Jeff Mahoney We use structures to pass the info and limit from the kernel as items but store the individual values separately in btrfs_qgroup. We already have a btrfs_qgroup_limit structure that's used for setting the limit. This patch introduces a btrfs_qgroup_info

[PATCH 01/20] btrfs-progs: quota: Add -W option to rescan to wait without starting rescan

2018-03-07 Thread jeffm
From: Jeff Mahoney This patch adds a new -W option to wait for a rescan without starting a new operation. This is useful for things like xfstests where we want do to do a "btrfs quota enable" and not continue until the subsequent rescan has finished. In addition to documenting

[PATCH 09/20] btrfs-progs: subvolume: add quota info to btrfs sub show

2018-03-07 Thread jeffm
From: Jeff Mahoney This patch reports on the first-level qgroup, if any, associated with a particular subvolume. It displays the usage and limit, subject to the usual unit parameters. Signed-off-by: Jeff Mahoney --- cmds-subvolume.c | 50

[PATCH 04/20] btrfs-progs: btrfs-list: add rb_entry helpers for root_info

2018-03-07 Thread jeffm
From: Jeff Mahoney We use rb_entry all over the place for the root_info pointers. Add a helper to make the code more readable. Signed-off-by: Jeff Mahoney --- btrfs-list.c | 30 -- 1 file changed, 20 insertions(+), 10 deletions(-)

[PATCH 08/20] btrfs-progs: qgroups: introduce btrfs_qgroup_query

2018-03-07 Thread jeffm
From: Jeff Mahoney The only mechanism we have in the progs for searching qgroups is to load all of them and filter the results. This works for qgroup show but to add quota information to 'btrfs subvoluem show' it's pretty wasteful. This patch splits out setting up the search

[PATCH 17/20] btrfs-progs: add support for output formats

2018-03-07 Thread jeffm
From: Jeff Mahoney This adds a global --format option to request extended output formats from each command. Most of it is plumbing a new cmd_context structure that's established at the beginning of argument parsing into the command callbacks. That structure currently only

[PATCH 19/20] btrfs-progs: qgroups: add json output for usage command

2018-03-07 Thread jeffm
From: Jeff Mahoney One of the common requests I receive is for 'df' like facilities for subvolume usage. Really, the request is for monitoring tools to be able to understand when subvolumes may be approaching quota in the same manner traditional file systems approach ENOSPC.

[PATCH 20/20] btrfs-progs: handle command groups directly for common case

2018-03-07 Thread jeffm
From: Jeff Mahoney Most command groups just pass their own command group to handle_command_group. We can remove the explicit definitions of command group callbacks by passing the cmd_struct to handle_command_group and allowing it to resolve the group from it. Signed-off-by:

[PATCH 13/20] btrfs-progs: use cmd_struct as command entry point

2018-03-07 Thread jeffm
From: Jeff Mahoney Rather than having global command usage and callbacks used to create cmd_structs in the command array, establish the cmd_struct structures separately and use those. The next commit in the series passes the cmd_struct to the command callbacks such that we can

[PATCH 14/20] btrfs-progs: pass cmd_struct to command callback function

2018-03-07 Thread jeffm
From: Jeff Mahoney This patch passes the cmd_struct to the command callback function. This has several purposes: It allows the command callback to identify which command was used to call it. It also gives us direct access to the usage associated with that command.

[PATCH 16/20] btrfs-progs: pass cmd_struct to usage()

2018-03-07 Thread jeffm
From: Jeff Mahoney Now that every call site has a cmd_struct, we can just pass the cmd_struct to usage to print the usager information. This allows us to interpret the format flags we'll add later in this series to inform the user of which output formats any given command

[PATCH 18/20] btrfs-progs: add generic support for json output

2018-03-07 Thread jeffm
From: Jeff Mahoney This patch adds support for JSON and JSON-compat output. The latter is intended to be compatible with Javascript's integers being represented as 64-bit floats, with only 53 bits usable for the integer component. Compat mode output will post 64-bit integers as

[PATCH 15/20] btrfs-progs: pass cmd_struct to clean_args_no_options{,_relaxed}

2018-03-07 Thread jeffm
From: Jeff Mahoney Now that we have a cmd_struct everywhere, we can pass it to clean_args_no_options and have it resolve the usage string from it there. This is necessary for it to pass the cmd_struct to usage() in the next patch. Signed-off-by: Jeff Mahoney

[PATCH v2 00/20] btrfs-progs: qgroups usability

2018-03-07 Thread jeffm
From: Jeff Mahoney Thanks to Qu Wenruo, Nikolay Borisov, and Tomohiro Misono for taking the time to review my previous patchset. I've incorporated your suggestions into this version. Obviously this one is quite a bit longer than the first version. After I posted it, Dave and I

[PATCH 10/20] btrfs-progs: help: convert ints used as bools to bool

2018-03-07 Thread jeffm
From: Jeff Mahoney We use an int for 'full', 'all', and 'err' when we really mean a boolean. Signed-off-by: Jeff Mahoney --- btrfs.c | 14 +++--- help.c | 25 + help.h | 4 ++-- 3 files changed, 22 insertions(+), 21

Re: [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota

2018-03-07 Thread Qu Wenruo
Ping again. Since David is planning to merge qgroup patchset, this feature would greatly improve test coverage. Thanks, Qu On 2018年01月11日 14:04, Qu Wenruo wrote: > Ping? > > Or do I need to rebase the patchset? > > Thanks, > Qu > > On 2017年11月07日 16:42, Qu Wenruo wrote: >> Can be fetched

[PATCH] fstests: test regression of -EEXIST on creating new file after log replay

2018-03-07 Thread Liu Bo
The regression is introduced to btrfs in linux v4.4 and it refuses to create new files after log replay by returning -EEXIST. Although the problem is on btrfs only, there is no btrfs stuff in terms of test, so this makes it generic. The kernel fix is Btrfs: fix unexpected -EEXIST when creating

Re: [PATCH] btrfs: qgroups, properly handle no reservations

2018-03-07 Thread Qu Wenruo
On 2018年03月08日 00:02, David Sterba wrote: > On Thu, Feb 22, 2018 at 10:05:36AM +0800, Qu Wenruo wrote: >> >> >> On 2018年02月22日 09:50, Jeff Mahoney wrote: >>> On 2/21/18 8:36 PM, Qu Wenruo wrote: On 2018年02月22日 04:19, je...@suse.com wrote: > From: Jeff Mahoney

Re: [PATCH 3/8] btrfs-progs: constify pathnames passed as arguments

2018-03-07 Thread Jeff Mahoney
On 3/7/18 3:17 AM, Nikolay Borisov wrote: > > > On 2.03.2018 20:46, je...@suse.com wrote: >> From: Jeff Mahoney >> >> It's unlikely we're going to modify a pathname argument, so codify that >> and use const. >> >> Signed-off-by: Jeff Mahoney >> --- >>

Re: [PATCH] Improve error stats message

2018-03-07 Thread Hugo Mills
On Wed, Mar 07, 2018 at 08:02:51PM +0100, Diego wrote: > El miércoles, 7 de marzo de 2018 19:24:53 (CET) Hugo Mills escribió: > >On multi-device filesystems, the two are not necessarily the same. > > Ouch. FWIW, I was moved to do this because I saw this conversation on > IRC which made me

Re: [PATCH 6/8] btrfs-progs: qgroups: introduce btrfs_qgroup_query

2018-03-07 Thread Jeff Mahoney
On 3/7/18 3:02 AM, Misono, Tomohiro wrote: > On 2018/03/03 3:47, je...@suse.com wrote: >> From: Jeff Mahoney >> >> The only mechanism we have in the progs for searching qgroups is to load >> all of them and filter the results. This works for qgroup show but >> to add quota

Re: [PATCH 7/8] btrfs-progs: subvolume: add quota info to btrfs sub show

2018-03-07 Thread Jeff Mahoney
On 3/7/18 1:09 AM, Qu Wenruo wrote: > > > On 2018年03月03日 02:47, je...@suse.com wrote: >> From: Jeff Mahoney >> >> This patch reports on the first-level qgroup, if any, associated with >> a particular subvolume. It displays the usage and limit, subject >> to the usual unit

[PATCH v2] Btrfs: scrub: batch rebuild for raid56

2018-03-07 Thread Liu Bo
In case of raid56, writes and rebuilds always take BTRFS_STRIPE_LEN(64K) as unit, however, scrub_extent() sets blocksize as unit, so rebuild process may be triggered on every block on a same stripe. A typical example would be that when we're replacing a disappeared disk, all reads on the disks

Inconsistence between sender and receiver

2018-03-07 Thread Liu Bo
Hi, In the following steps[1], if on receiver side has got changed via 'btrfs property set', then after doing incremental updates, receiver gets a different snapshot from what sender has sent. The reason behind it is that there is no change about file 'foo' in the send stream, such that

Re: [PATCH 6/8] btrfs-progs: qgroups: introduce btrfs_qgroup_query

2018-03-07 Thread Jeff Mahoney
On 3/7/18 12:58 AM, Qu Wenruo wrote: > > > On 2018年03月03日 02:47, je...@suse.com wrote: >> diff --git a/qgroup.c b/qgroup.c >> index b1be3311..2d0a6947 100644 >> --- a/qgroup.c >> +++ b/qgroup.c >> @@ -1267,6 +1249,66 @@ static int __qgroups_search(int fd, struct >> qgroup_lookup *qgroup_lookup)

Re: [PATCH] Improve error stats message

2018-03-07 Thread Diego
El miércoles, 7 de marzo de 2018 19:24:53 (CET) Hugo Mills escribió: >On multi-device filesystems, the two are not necessarily the same. Ouch. FWIW, I was moved to do this because I saw this conversation on IRC which made me think that people aren't understanding what the message means:

Re: [PATCH 1/2] btrfs: Add unprivileged subvolume search ioctl

2018-03-07 Thread Goffredo Baroncelli
On 03/07/2018 01:40 AM, Misono, Tomohiro wrote: > On 2018/03/07 5:29, Goffredo Baroncelli wrote: >> On 03/06/2018 09:30 AM, Misono, Tomohiro wrote: >>> Add new unprivileged ioctl (BTRFS_IOC_GET_SUBVOL_INFO) which searches >>> and returns only subvolume related item

Re: [PATCH] Improve error stats message

2018-03-07 Thread Hugo Mills
On Wed, Mar 07, 2018 at 06:37:29PM +0100, Diego wrote: > A typical notification of filesystem errors looks like this: > > BTRFS error (device sda2): bdev /dev/sda2 errs: wr 0, rd 1, flush 0, corrupt > 0, gen 0 > > The device name is being printed twice. For good reason -- the first part

[PATCH] Improve error stats message

2018-03-07 Thread Diego
A typical notification of filesystem errors looks like this: BTRFS error (device sda2): bdev /dev/sda2 errs: wr 0, rd 1, flush 0, corrupt 0, gen 0 The device name is being printed twice. Also, these abbreviatures feel unnecesary. Make the message look like this instead: BTRFS error (device

Re: [PATCH 1/2] btrfs: Add unprivileged subvolume search ioctl

2018-03-07 Thread David Sterba
On Wed, Mar 07, 2018 at 09:40:18AM +0900, Misono, Tomohiro wrote: > On 2018/03/07 5:29, Goffredo Baroncelli wrote: > > On 03/06/2018 09:30 AM, Misono, Tomohiro wrote: > >> Add new unprivileged ioctl (BTRFS_IOC_GET_SUBVOL_INFO) which searches > >> and returns only subvolume related item

Re: [PATCH 4/8] btrfs-progs: qgroups: add pathname to show output

2018-03-07 Thread Jeff Mahoney
On 3/7/18 12:45 AM, Qu Wenruo wrote: > > > On 2018年03月03日 02:47, je...@suse.com wrote: >> diff --git a/cmds-qgroup.c b/cmds-qgroup.c >> index 48686436..94cd0fd3 100644 >> --- a/cmds-qgroup.c >> +++ b/cmds-qgroup.c >> @@ -280,8 +280,10 @@ static const char * const cmd_qgroup_show_usage[] = { >>

Re: [PATCH] btrfs: qgroup: Fix root item corruption when multiple same source snapshiots are created with quota enabled

2018-03-07 Thread David Sterba
On Fri, Feb 02, 2018 at 11:45:46AM +, Filipe Manana wrote: > On Tue, Dec 19, 2017 at 7:44 AM, Qu Wenruo wrote: > > When multiple pending snapshots referring the same source subvolume are > > executed, enabled quota will cause root item corruption, where root > > items are using

Re: [PATCH v2] Btrfs: fix unexpected cow in run_delalloc_nocow

2018-03-07 Thread David Sterba
On Wed, Jan 31, 2018 at 05:09:13PM -0700, Liu Bo wrote: > Fstests generic/475 provides a way to fail metadata reads while > checking if checksum exists for the inode inside run_delalloc_nocow(), > and csum_exist_in_range() interprets error (-EIO) as inode having > checksum and makes its caller

Re: [PATCH v3] btrfs: fix bare unsigned declarations

2018-03-07 Thread David Sterba
On Mon, Feb 26, 2018 at 04:46:05PM +0800, Anand Jain wrote: > We have btrfs_fs_info::data_chunk_allocations and > btrfs_fs_info::metadata_ratio declared as unsigned which would > be unsinged int and kernel style prefers unsigned int over bare > unsigned. So this patch changes them to u32. > >

Re: [PATCH] btrfs: Relax memory barrier in btrfs_tree_unlock

2018-03-07 Thread David Sterba
On Wed, Feb 14, 2018 at 02:37:26PM +0200, Nikolay Borisov wrote: > When performing an unlock on an extent buffer we'd like to order the > decrement of extent_buffer::blocking_writers with waking up any > waiters. In such situations it's sufficient to use smp_mb__after_atomic > rather than the

Re: [PATCH] btrfs: qgroups, properly handle no reservations

2018-03-07 Thread David Sterba
On Thu, Feb 22, 2018 at 10:05:36AM +0800, Qu Wenruo wrote: > > > On 2018年02月22日 09:50, Jeff Mahoney wrote: > > On 2/21/18 8:36 PM, Qu Wenruo wrote: > >> > >> > >> On 2018年02月22日 04:19, je...@suse.com wrote: > >>> From: Jeff Mahoney > >>> > >>> There are several places where we

Re: [PATCH] btrfs: Add nossd_spread mount option

2018-03-07 Thread David Sterba
On Wed, Feb 21, 2018 at 03:31:40PM -0800, Howard McLauchlan wrote: > Btrfs has two mount options for SSD optimizations: ssd and ssd_spread. > Presently there is an option to disable all SSD optimizations, but there > isn't an option to disable just ssd_spread. > > This patch adds a mount option

Re: [PATCH 8/8] btrfs-progs: qgroups: export qgroups usage information as JSON

2018-03-07 Thread Jeff Mahoney
On 3/7/18 1:34 AM, Qu Wenruo wrote: > > > On 2018年03月03日 02:47, je...@suse.com wrote: >> diff --git a/configure.ac b/configure.ac >> index 56d17c3a..6aec672a 100644 >> --- a/configure.ac >> +++ b/configure.ac >> @@ -197,6 +197,12 @@ PKG_STATIC(UUID_LIBS_STATIC, [uuid]) >>

[RFC PATCH] btrfs: Fix memory ordering of unlocked dio reads vs truncate

2018-03-07 Thread Nikolay Borisov
Signed-off-by: Nikolay Borisov --- Hello, Sending it as an RFC for the time being to see how people are going to react and also I'd like some feedback on the mb semantics. For this purposed I've CC'ed some memory ordering people :) fs/btrfs/btrfs_inode.h | 17

Re: [PATCH 1/5] btrfs: Parse options after node/sector size initialized

2018-03-07 Thread David Sterba
On Fri, Mar 02, 2018 at 01:22:50PM +0800, Qu Wenruo wrote: > This provides the basis for later max_inline enhancement, which needs to > access fs_info->nodesize. I've checked if this patch can be applied independently, but no, see the comment below. > Signed-off-by: Qu Wenruo >

Re: [PATCH] Btrfs: scrub: batch rebuild for raid56

2018-03-07 Thread Nikolay Borisov
On 7.03.2018 16:43, David Sterba wrote: > On Tue, Mar 06, 2018 at 11:22:21AM -0700, Liu Bo wrote: >> On Tue, Mar 06, 2018 at 11:47:47AM +0100, David Sterba wrote: >>> On Fri, Mar 02, 2018 at 04:10:37PM -0700, Liu Bo wrote: In case of raid56, writes and rebuilds always take

Re: [PATCH v2] btrfs: drop nonvaring variable, instead define it

2018-03-07 Thread David Sterba
On Wed, Mar 07, 2018 at 05:29:18PM +0800, Anand Jain wrote: > btrfs_defrag_leaves() declares min_trans = 0; as variable, but > doesn't vary it, so define it. > > Signed-off-by: Anand Jain Reviewed-by: David Sterba > --- > v2->v1: Use

Re: [PATCH] Btrfs: scrub: batch rebuild for raid56

2018-03-07 Thread David Sterba
On Tue, Mar 06, 2018 at 11:22:21AM -0700, Liu Bo wrote: > On Tue, Mar 06, 2018 at 11:47:47AM +0100, David Sterba wrote: > > On Fri, Mar 02, 2018 at 04:10:37PM -0700, Liu Bo wrote: > > > In case of raid56, writes and rebuilds always take BTRFS_STRIPE_LEN(64K) > > > as unit, however, scrub_extent()

Re: [PATCH v8 55/63] btrfs: Convert page cache to XArray

2018-03-07 Thread David Sterba
On Tue, Mar 06, 2018 at 11:24:05AM -0800, Matthew Wilcox wrote: > From: Matthew Wilcox > > Signed-off-by: Matthew Wilcox Acked-by: David Sterba -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the

Re: [PATCH v8 06/63] btrfs: Use filemap_range_has_page()

2018-03-07 Thread David Sterba
On Tue, Mar 06, 2018 at 11:23:16AM -0800, Matthew Wilcox wrote: > From: Matthew Wilcox > > The current implementation of btrfs_page_exists_in_range() gives the > wrong answer if the workingset code has stored a shadow entry in the > page cache. The

Re: [PATCH] fstests: btrfs/004: increase the buffer size of logical-resolve to the maximum value 64K

2018-03-07 Thread Filipe Manana
On Wed, Mar 7, 2018 at 7:07 AM, Eryu Guan wrote: > On Tue, Mar 06, 2018 at 03:02:31PM +0800, Lu Fengqi wrote: >> Because of commit e76e13ce8c0b ("fsstress: implement the >> clonerange/deduperange ioctls"), dedupe makes the number of references to >> the same extent item

Re: [PATCH 2/2] Btrfs: fix fiemap extent SHARED flag error with range clone.

2018-03-07 Thread Qu Wenruo
On 2018年03月07日 20:17, Nikolay Borisov wrote: > > > On 7.03.2018 14:14, Qu Wenruo wrote: >> >> > > > SHARED flag is determined after extent map merge, so here we can't rely on em here. >>> >>> Shouldn't extent maps correspond to 1:1 disk-state. I.e. they are just >>> the

Re: [PATCH 2/2] Btrfs: fix fiemap extent SHARED flag error with range clone.

2018-03-07 Thread Nikolay Borisov
On 7.03.2018 14:14, Qu Wenruo wrote: > > >>> >>> SHARED flag is determined after extent map merge, so here we can't rely >>> on em here. >> >> Shouldn't extent maps correspond to 1:1 disk-state. I.e. they are just >> the memory cache of the extent state. So if we merge them, shouldn't we

Re: [PATCH 2/2] Btrfs: fix fiemap extent SHARED flag error with range clone.

2018-03-07 Thread Qu Wenruo
On 2018年03月07日 19:27, Nikolay Borisov wrote: > > > On 7.03.2018 13:18, Qu Wenruo wrote: >> >> >> On 2018年03月07日 19:01, robbieko wrote: >>> Qu Wenruo 於 2018-03-07 18:42 寫到: On 2018年03月07日 18:33, Qu Wenruo wrote: > > > On 2018年03月07日 16:20, robbieko wrote: >> From: Robbie

Re: [PATCH 2/2] Btrfs: fix fiemap extent SHARED flag error with range clone.

2018-03-07 Thread Nikolay Borisov
On 7.03.2018 13:18, Qu Wenruo wrote: > > > On 2018年03月07日 19:01, robbieko wrote: >> Qu Wenruo 於 2018-03-07 18:42 寫到: >>> On 2018年03月07日 18:33, Qu Wenruo wrote: On 2018年03月07日 16:20, robbieko wrote: > From: Robbie Ko > > [BUG] > Range

Re: [PATCH 2/2] Btrfs: fix fiemap extent SHARED flag error with range clone.

2018-03-07 Thread Qu Wenruo
On 2018年03月07日 19:01, robbieko wrote: > Qu Wenruo 於 2018-03-07 18:42 寫到: >> On 2018年03月07日 18:33, Qu Wenruo wrote: >>> >>> >>> On 2018年03月07日 16:20, robbieko wrote: From: Robbie Ko [BUG] Range clone can cause fiemap to return error result. Like:

Re: [PATCH 1/2] Btrfs: fiemap: pass correct bytenr when fm_extent_count is zero

2018-03-07 Thread Nikolay Borisov
On 7.03.2018 12:27, robbieko wrote: > Nikolay Borisov 於 2018-03-07 18:19 寫到: >> On  7.03.2018 10:20, robbieko wrote: >>> From: Robbie Ko >>> >>>  # mount /dev/vdb5 /mnt/btrfs >>>  # dd if=/dev/zero bs=16K count=4 oflag=dsync of=/mnt/btrfs/file >>>  # xfs_io -c "fiemap

Re: [PATCH 2/2] Btrfs: fix fiemap extent SHARED flag error with range clone.

2018-03-07 Thread robbieko
Qu Wenruo 於 2018-03-07 18:42 寫到: On 2018年03月07日 18:33, Qu Wenruo wrote: On 2018年03月07日 16:20, robbieko wrote: From: Robbie Ko [BUG] Range clone can cause fiemap to return error result. Like: # mount /dev/vdb5 /mnt/btrfs # dd if=/dev/zero bs=16K count=2 oflag=dsync

Re: [PATCH 2/2] Btrfs: fix fiemap extent SHARED flag error with range clone.

2018-03-07 Thread Qu Wenruo
On 2018年03月07日 18:33, Qu Wenruo wrote: > > > On 2018年03月07日 16:20, robbieko wrote: >> From: Robbie Ko >> >> [BUG] >> Range clone can cause fiemap to return error result. >> Like: >> # mount /dev/vdb5 /mnt/btrfs >> # dd if=/dev/zero bs=16K count=2 oflag=dsync

Re: [PATCH 2/2] Btrfs: fix fiemap extent SHARED flag error with range clone.

2018-03-07 Thread Qu Wenruo
On 2018年03月07日 16:20, robbieko wrote: > From: Robbie Ko > > [BUG] > Range clone can cause fiemap to return error result. > Like: > # mount /dev/vdb5 /mnt/btrfs > # dd if=/dev/zero bs=16K count=2 oflag=dsync of=/mnt/btrfs/file > # xfs_io -c "fiemap -v" /mnt/btrfs/file

Re: [PATCH 1/2] Btrfs: fiemap: pass correct bytenr when fm_extent_count is zero

2018-03-07 Thread robbieko
Nikolay Borisov 於 2018-03-07 18:19 寫到: On 7.03.2018 10:20, robbieko wrote: From: Robbie Ko # mount /dev/vdb5 /mnt/btrfs # dd if=/dev/zero bs=16K count=4 oflag=dsync of=/mnt/btrfs/file # xfs_io -c "fiemap -v" /mnt/btrfs/file /mnt/btrfs/file: EXT: FILE-OFFSET

Re: [PATCH 1/2] Btrfs: fiemap: pass correct bytenr when fm_extent_count is zero

2018-03-07 Thread Nikolay Borisov
On 7.03.2018 10:20, robbieko wrote: > From: Robbie Ko > > # mount /dev/vdb5 /mnt/btrfs > # dd if=/dev/zero bs=16K count=4 oflag=dsync of=/mnt/btrfs/file > # xfs_io -c "fiemap -v" /mnt/btrfs/file > /mnt/btrfs/file: > EXT: FILE-OFFSET BLOCK-RANGE TOTAL

Re: [PATCH 0/2] btrfs fiemap related BUG fix.

2018-03-07 Thread robbieko
Qu Wenruo 於 2018-03-07 17:27 寫到: On 2018年03月07日 16:20, robbieko wrote: From: Robbie Ko This patchset intends to fix btrfs fiemap related bug. The fiemap has the following problems: 1) Wrong extent count when fm_extent_count is zero. 2) SHARED bit is not correct I

Re: [PATCH 0/2] btrfs fiemap related BUG fix.

2018-03-07 Thread Qu Wenruo
On 2018年03月07日 16:20, robbieko wrote: > From: Robbie Ko > > This patchset intends to fix btrfs fiemap related bug. > > The fiemap has the following problems: > > 1) Wrong extent count when fm_extent_count is zero. > > > 2) SHARED bit is not correct > I have two

[PATCH v2] btrfs: drop nonvaring variable, instead define it

2018-03-07 Thread Anand Jain
btrfs_defrag_leaves() declares min_trans = 0; as variable, but doesn't vary it, so define it. Signed-off-by: Anand Jain --- v2->v1: Use BTRFS_OLDEST_GENERATION at more places where needed. fs/btrfs/ctree.h | 2 ++ fs/btrfs/ioctl.c | 2 +-

Re: [PATCH 5/8] btrfs-progs: qgroups: introduce and use info and limit structures

2018-03-07 Thread Nikolay Borisov
On 2.03.2018 20:47, je...@suse.com wrote: > From: Jeff Mahoney > > We use structures to pass the info and limit from the kernel as items > but store the individual values separately in btrfs_qgroup. We already > have a btrfs_qgroup_limit structure that's used for setting the

[PATCH] fstests: btrfs/146: make sure hit all stripes in the case of compression

2018-03-07 Thread Lu Fengqi
In the case of compression, each 128K input data chunk will be compressed to 4K (because of the characters written are duplicate). Therefore we have to write (128K * 16) to make sure every stripe can be hit. Signed-off-by: Lu Fengqi --- tests/btrfs/146 | 11 +--

[PATCH 0/2] btrfs fiemap related BUG fix.

2018-03-07 Thread robbieko
From: Robbie Ko This patchset intends to fix btrfs fiemap related bug. The fiemap has the following problems: 1) Wrong extent count when fm_extent_count is zero. 2) SHARED bit is not correct I have two ideas, but I do not know which one is the best. Like: # dd

[PATCH 1/2] Btrfs: fiemap: pass correct bytenr when fm_extent_count is zero

2018-03-07 Thread robbieko
From: Robbie Ko # mount /dev/vdb5 /mnt/btrfs # dd if=/dev/zero bs=16K count=4 oflag=dsync of=/mnt/btrfs/file # xfs_io -c "fiemap -v" /mnt/btrfs/file /mnt/btrfs/file: EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS 0: [0..127]:25088..25215 128 0x1

[PATCH 2/2] Btrfs: fix fiemap extent SHARED flag error with range clone.

2018-03-07 Thread robbieko
From: Robbie Ko [BUG] Range clone can cause fiemap to return error result. Like: # mount /dev/vdb5 /mnt/btrfs # dd if=/dev/zero bs=16K count=2 oflag=dsync of=/mnt/btrfs/file # xfs_io -c "fiemap -v" /mnt/btrfs/file /mnt/btrfs/file: EXT: FILE-OFFSET BLOCK-RANGE

Re: [PATCH 3/8] btrfs-progs: constify pathnames passed as arguments

2018-03-07 Thread Nikolay Borisov
On 2.03.2018 20:46, je...@suse.com wrote: > From: Jeff Mahoney > > It's unlikely we're going to modify a pathname argument, so codify that > and use const. > > Signed-off-by: Jeff Mahoney > --- > chunk-recover.c | 4 ++-- > cmds-device.c | 2 +- >

Re: [PATCH 2/8] btrfs-progs: qgroups: fix misleading index check

2018-03-07 Thread Nikolay Borisov
On 2.03.2018 20:46, je...@suse.com wrote: > From: Jeff Mahoney > > In print_single_qgroup_table we check the loop index against > BTRFS_QGROUP_CHILD, but what we really mean is "last column." Since > we have an enum value to indicate the last value, use that instead > of

Re: [PATCH 6/8] btrfs-progs: qgroups: introduce btrfs_qgroup_query

2018-03-07 Thread Misono, Tomohiro
On 2018/03/03 3:47, je...@suse.com wrote: > From: Jeff Mahoney > > The only mechanism we have in the progs for searching qgroups is to load > all of them and filter the results. This works for qgroup show but > to add quota information to 'btrfs subvoluem show' it's pretty

Re: [PATCH 4.4 1/2] btrfs: Don't clear SGID when inheriting ACLs

2018-03-07 Thread Nikolay Borisov
On 7.03.2018 09:57, Nikolay Borisov wrote: > From: Jan Kara > > When new directory 'DIR1' is created in a directory 'DIR0' with SGID bit > set, DIR1 is expected to have SGID bit set (and owning group equal to > the owning group of 'DIR0'). However when 'DIR0' also has some