RE: [PATCH v4 1/3] btrfs: Add unprivileged ioctl which returns subvolume information

2018-05-15 Thread Gu, Jinxiang
Hi, > -Original Message- > From: linux-btrfs-ow...@vger.kernel.org > [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Tomohiro Misono > Sent: Friday, May 11, 2018 3:26 PM > To: linux-btrfs@vger.kernel.org > Subject: [PATCH v4 1/3] btrfs: Add unprivileged ioctl which returns

Re: [PATCH v4 1/3] btrfs: Add unprivileged ioctl which returns subvolume information

2018-05-15 Thread Misono Tomohiro
On 2018/05/15 15:31, Gu, Jinxiang/顾 金香 wrote: > Hi, > >> -Original Message- >> From: linux-btrfs-ow...@vger.kernel.org >> [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Tomohiro Misono >> Sent: Friday, May 11, 2018 3:26 PM >> To: linux-btrfs@vger.kernel.org >> Subject: [PATCH v4

[PATCH 2/2] btrfs: sysfs: Add entry which shows rmdir(2) can work for subvolume

2018-05-15 Thread Misono Tomohiro
Deletion of a subvolume by rmdir(2) has become allowed by the 'commit cd2decf640b1 ("btrfs: Allow rmdir(2) to delete an empty subvolume")'. It is a kind of new feature and this commits add new sysfs entry /sys/fs/btrfs/features/rmdir_subvol to indicate the feature. Since the behavior is

[PATCH 1/2] btrfs: inode: Don't compress if NODATASUM or NODATACOW set

2018-05-15 Thread Qu Wenruo
As btrfs(5) specified: Note If nodatacow or nodatasum are enabled, compression is disabled. If NODATASUM or NODATACOW set, we should not compress the extent. Normally NODATACOW is detected properly in run_delalloc_range() so compression won't happen for NODATACOW. However for

[PATCH 0/2] btrfs: Enhance btrfs handling compression and

2018-05-15 Thread Qu Wenruo
James Harvey reported one corruption where lzo compressed extent without data csum is causing "decompress failed" kernel message, and then serious random kernel memory corruption. For the "decrompress failed" kernel message, it's indeed corrupted compressed data. However we can still harden btrfs

[PATCH 2/2] btrfs: lzo: Avoid decompressing obviously corrupted data

2018-05-15 Thread Qu Wenruo
Unlike zlib decompression, lzo decompression doesn't need any initialization, thus we can't detect early corruption from initialization. However for lzo compressed extent, its first 4bytes records the real unaligned compressed data size. We could use this as a clue, since any compressed extent

Re: [PATCH v2 1/3] btrfs-progs: check: check symlinks with append/immutable flags

2018-05-15 Thread Qu Wenruo
On 2018年05月15日 09:33, Su Yue wrote: > Define new macro I_ERR_ODD_INODE_FLAGS to represents odd inode flags. > > Symlinks should never have append/immutable flags. > While processing inodes, if found a symlink with append/immutable > flags, mark the inode record with I_ERR_ODD_INODE_FLAGS. > >

[PATCH] btrfs: property: Set incompat flag of lzo/zstd compression

2018-05-15 Thread Misono Tomohiro
Incompat flag of lzo/zstd compression should be set at: 1. mount time (-o compress/compress-force) 2. when defrag is done 3. when property is set Currently 3. is missing and this commit adds this. Signed-off-by: Tomohiro Misono --- fs/btrfs/props.c | 12

RE: [PATCH v4 1/3] btrfs: Add unprivileged ioctl which returns subvolume information

2018-05-15 Thread Gu, Jinxiang
Hi, add a missed a comment. > -Original Message- > From: Misono Tomohiro [mailto:misono.tomoh...@jp.fujitsu.com] > Sent: Tuesday, May 15, 2018 3:04 PM > To: Gu, Jinxiang/顾 金香 ; linux-btrfs@vger.kernel.org > Subject: Re: [PATCH v4 1/3] btrfs: Add unprivileged ioctl

Re: [PATCH] btrfs: property: Set incompat flag of lzo/zstd compression

2018-05-15 Thread Su Yue
On 05/15/2018 03:51 PM, Misono Tomohiro wrote: > Incompat flag of lzo/zstd compression should be set at: > 1. mount time (-o compress/compress-force) > 2. when defrag is done > 3. when property is set > > Currently 3. is missing and this commit adds this. > If I don't misunderstand,

Re: [PATCH 0/2] btrfs: sysfs: Add entry which shows rmdir(2) can work for subvolume

2018-05-15 Thread Nikolay Borisov
On 15.05.2018 10:30, Misono Tomohiro wrote: > [based on current misc-next] > > This adds new sysfs entry > /sys/fs/btrfs/features/rmdir_subvol > to indicate that the kernel can delete a subvolume by rmdir(2), > which is allowed by: https://www.spinics.net/lists/linux-btrfs/msg76938.html > >

Re: [PATCH 0/2] btrfs: sysfs: Add entry which shows rmdir(2) can work for subvolume

2018-05-15 Thread Misono Tomohiro
On 2018/05/15 17:03, Nikolay Borisov wrote: > > > On 15.05.2018 10:30, Misono Tomohiro wrote: >> [based on current misc-next] >> >> This adds new sysfs entry >> /sys/fs/btrfs/features/rmdir_subvol >> to indicate that the kernel can delete a subvolume by rmdir(2), >> which is allowed by:

[PATCH 0/2] btrfs: sysfs: Add entry which shows rmdir(2) can work for subvolume

2018-05-15 Thread Misono Tomohiro
[based on current misc-next] This adds new sysfs entry /sys/fs/btrfs/features/rmdir_subvol to indicate that the kernel can delete a subvolume by rmdir(2), which is allowed by: https://www.spinics.net/lists/linux-btrfs/msg76938.html The first patch is a cleanup and the second one is a main

[PATCH 1/2] btrfs: sysfs: Use enum/define value intead of magic number

2018-05-15 Thread Misono Tomohiro
Signed-off-by: Tomohiro Misono --- fs/btrfs/sysfs.c | 11 ++- fs/btrfs/sysfs.h | 2 +- 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c index fa6c8c88b250..217d401fe8ae 100644 --- a/fs/btrfs/sysfs.c +++

Re: [PATCH v2 2/3] btrfs-progs: lowmem: check symlinks with append/immutable flags

2018-05-15 Thread Qu Wenruo
On 2018年05月15日 09:33, Su Yue wrote: > Define new error bit INODE_FLAGS_ERROR to represents invalid inode > flags error. > > Symlinks should never have append/immutable flags set. > While checking inodes, if found a symlink with append/immutable > flags, report and record the inode flags error.

RE: [PATCH v4 2/3] btrfs: Add unprivileged ioctl which returns subvolume's ROOT_REF

2018-05-15 Thread Gu, Jinxiang
> -Original Message- > From: linux-btrfs-ow...@vger.kernel.org > [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Tomohiro Misono > Sent: Friday, May 11, 2018 3:26 PM > To: linux-btrfs@vger.kernel.org > Subject: [PATCH v4 2/3] btrfs: Add unprivileged ioctl which returns >

Re: [PATCH 2/2] btrfs: lzo: Avoid decompressing obviously corrupted data

2018-05-15 Thread Nikolay Borisov
On 15.05.2018 10:36, Qu Wenruo wrote: > Unlike zlib decompression, lzo decompression doesn't need any > initialization, thus we can't detect early corruption from > initialization. > > However for lzo compressed extent, its first 4bytes records the real > unaligned compressed data size. > We

Re: Clarification needed about libbtrfs & libbtrfsutil

2018-05-15 Thread Dimitri John Ledkov
On 14 May 2018 at 21:22, Omar Sandoval wrote: > On Mon, May 14, 2018 at 09:40:19AM +0100, Dimitri John Ledkov wrote: >> Are both of these meant to be public libraries, installed on the user >> systems, and available in .so variant as well for 3rd party >> development and

Re: [PATCH] btrfs: property: Set incompat flag of lzo/zstd compression

2018-05-15 Thread David Sterba
On Tue, May 15, 2018 at 04:51:26PM +0900, Misono Tomohiro wrote: > Incompat flag of lzo/zstd compression should be set at: > 1. mount time (-o compress/compress-force) > 2. when defrag is done > 3. when property is set > > Currently 3. is missing and this commit adds this. That was missed

Re: [PATCH 2/2] btrfs: sysfs: Add entry which shows rmdir(2) can work for subvolume

2018-05-15 Thread David Sterba
On Tue, May 15, 2018 at 04:33:12PM +0900, Misono Tomohiro wrote: > Deletion of a subvolume by rmdir(2) has become allowed by the > 'commit cd2decf640b1 ("btrfs: Allow rmdir(2) to delete an empty > subvolume")'. > > It is a kind of new feature and this commits add new sysfs entry >

Re: verify key failure

2018-05-15 Thread Liu Bo
On Tue, May 15, 2018 at 9:29 AM, Qu Wenruo wrote: > > > On 2018年05月14日 22:35, Liu Bo wrote: >> Hi, >> >> I got another warning of verify_level_key by running btrfs/124 in a loop, >> I'm testing against 4.17-rc3. >> >> Not sure if it's false positive. >> >> [101414.336691]

Re: [PATCH 1/2] btrfs: sysfs: Use enum/define value intead of magic number

2018-05-15 Thread David Sterba
On Tue, May 15, 2018 at 04:31:47PM +0900, Misono Tomohiro wrote: > Signed-off-by: Tomohiro Misono Reviewed-by: David Sterba -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to

Re: 4.15.6 crash: BUG at fs/btrfs/ctree.c:1862

2018-05-15 Thread Marc MERLIN
On Tue, May 15, 2018 at 09:36:11AM +0100, Filipe Manana wrote: > We got a fix for this recently: https://patchwork.kernel.org/patch/10396523/ Thanks very much for the notice, sorry that I missed it. Marc -- "A mouse is a device used to point at the xterm you want to type in" - A.S.R. Microsoft

[PATCH V3] test online label ioctl

2018-05-15 Thread Eric Sandeen
This tests the online label ioctl that btrfs has, which has been recently proposed for XFS. To run, it requires an updated xfs_io with the label command and a filesystem that supports it A slight change here to _require_xfs_io_command as well, so that tests which simply fail with "Inappropriate

Re: verify key failure

2018-05-15 Thread Liu Bo
On Tue, May 15, 2018 at 12:10 AM, Chris Mason wrote: > > > On 14 May 2018, at 10:35, Liu Bo wrote: > >> Hi, >> >> I got another warning of verify_level_key by running btrfs/124 in a loop, >> I'm testing against 4.17-rc3. >> >> Not sure if it's false positive. > > > How long does this

[PATCH] btrfs-progs: remove BTRFS_LIST_LAYOUT_RAW

2018-05-15 Thread Gu Jinxiang
Since commit 9005b603d723 ("btrfs-progs: use libbtrfsutil for subvol show"), BTRFS_LIST_LAYOUT_RAW has no usage. So, remove it. Signed-off-by: Gu Jinxiang --- btrfs-list.c | 20 btrfs-list.h | 3 +-- 2 files changed, 1 insertion(+), 22 deletions(-)

Re: [PATCH 1/2] btrfs: inode: Don't compress if NODATASUM or NODATACOW set

2018-05-15 Thread Nikolay Borisov
On 15.05.2018 11:48, Qu Wenruo wrote: static inline int inode_need_compress(struct inode *inode, u64 start, u64 end) { struct btrfs_fs_info *fs_info

Re: [PATCH 1/2] btrfs: inode: Don't compress if NODATASUM or NODATACOW set

2018-05-15 Thread Qu Wenruo
On 2018年05月15日 18:36, Nikolay Borisov wrote: > > > On 15.05.2018 11:48, Qu Wenruo wrote: > > > > > static inline int inode_need_compress(struct inode *inode, u64 start, u64 > end) > { >

Re: [PATCH] Btrfs: add parent_transid parameter to veirfy_level_key

2018-05-15 Thread David Sterba
On Wed, May 16, 2018 at 01:37:37AM +0800, Liu Bo wrote: > @parent_transid could tell whether the eb's generation has been verified > by the caller. Can you please write why you add this change? Like is it an enhancement or debugging help or someting like that. -- To unsubscribe from this list:

Re: [PATCH 0/15] Review uuid_mutex usage

2018-05-15 Thread David Sterba
On Thu, Apr 12, 2018 at 10:29:23AM +0800, Anand Jain wrote: > uuid_mutex lock is not a per-fs lock but a global lock. The main aim of > this patch-set is to critically review the usage of this lock, and delete > the unnecessary once. By doing this we improve the concurrency of > device operations

[PATCH] Btrfs: fix the corruption by reading stale btree blocks

2018-05-15 Thread Liu Bo
If a btree block, aka. extent buffer, is not available in the extent buffer cache, it'll be read out from the disk instead, i.e. btrfs_search_slot() read_block_for_search() # hold parent and its lock, go to read child btrfs_release_path() read_tree_block() # read child Unfortunately,

Re: [PATCH 10/15] btrfs: drop uuid_mutex in btrfs_free_extra_devids()

2018-05-15 Thread David Sterba
On Thu, Apr 12, 2018 at 10:29:33AM +0800, Anand Jain wrote: > btrfs_free_extra_devids() frees the orphan fsid::devid but its search is > limited to btrfs_fs_devices::devices, so we dont need uuid_mutex. >From that it's not clear why there's no locking at all now: > @@ -897,7 +897,6 @@ void

[PATCH 3/6] Btrfs: move get root of btrfs_search_slot to a helper

2018-05-15 Thread Liu Bo
It's good to have a helper instead of having all get-root details open-coded. The new helper locks (if necessary) and sets root node of the path. There is no functional change in this commit. Signed-off-by: Liu Bo --- fs/btrfs/ctree.c | 112

[PATCH 1/6] Btrfs: remove superfluous free_extent_buffer

2018-05-15 Thread Liu Bo
@tmp must be NULL at this point, free_extent_buffer is not needed at all. Signed-off-by: Liu Bo --- fs/btrfs/ctree.c | 1 - 1 file changed, 1 deletion(-) diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c index b3f6f300e492..9fa3d77c98d4 100644 --- a/fs/btrfs/ctree.c

[PATCH 4/6] Btrfs: remove unused check of skip_locking

2018-05-15 Thread Liu Bo
The check is superfluous since all of callers who set search_for_commit also have skip_locking set. Signed-off-by: Liu Bo --- fs/btrfs/ctree.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c index 399839df7a8f..cf34eca41d4e

[PATCH 5/6] Btrfs: grab write lock directly if write_lock_level is the max level

2018-05-15 Thread Liu Bo
In case of (cow && (p->keep_locks || p->lowest_level)), write_lock_level is the max level, and we should grab write lock of root node from the very beginning. Signed-off-by: Liu Bo --- fs/btrfs/ctree.c | 29 - 1 file changed, 16

[PATCH 0/6] btrfs_search_slot cleanups

2018-05-15 Thread Liu Bo
These're the cleanups I made for btrfs_search_slot() and its callees. Liu Bo (6): Btrfs: remove superfluous free_extent_buffer Btrfs: use more straightforward extent_buffer_uptodate Btrfs: move get root of btrfs_search_slot to a helper Btrfs: remove unused check of skip_locking Btrfs:

[PATCH 2/6] Btrfs: use more straightforward extent_buffer_uptodate

2018-05-15 Thread Liu Bo
In read_block_for_search(), it's straightforward to use extent_buffer_uptodate() instead since 0 is passed as parent transid to btrfs_buffer_uptodate(), which means the check for parent transid is not needed. Signed-off-by: Liu Bo --- fs/btrfs/ctree.c | 2 +- 1 file

[PATCH 6/6] Btrfs: remove always true check in unlock_up

2018-05-15 Thread Liu Bo
@path->lock[i] is always true at this point. Signed-off-by: Liu Bo --- fs/btrfs/ctree.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c index f7c3f581f647..16d28a4ec54f 100644 --- a/fs/btrfs/ctree.c +++

Re: [PATCH 12/15] btrfs: drop uuid_mutex in close_fs_devices()

2018-05-15 Thread David Sterba
On Thu, Apr 12, 2018 at 10:29:35AM +0800, Anand Jain wrote: > close_fs_devices() closes devices of a given fsid, and it is limited > to all the devices of a fsid, so we don't have to hold the global > uuid_mutex, instead we need the device_list_mutex as the device state is > being changed. > >

Re: [PATCH 0/6] btrfs_search_slot cleanups

2018-05-15 Thread David Sterba
On Wed, May 16, 2018 at 01:52:02AM +0800, Liu Bo wrote: > These're the cleanups I made for btrfs_search_slot() and its callees. All the patches have very terse changelog and I have no idea why you think the code change is correct so I can compare my review against that. It's touching code around

Re: [PATCH] btrfs: inode: Don't compress if NODATASUM or NODATACOW set

2018-05-15 Thread David Sterba
On Mon, May 14, 2018 at 11:46:09PM +0300, Timofey Titovets wrote: > > > @@ -396,6 +396,14 @@ static inline int inode_need_compress(struct inode > *inode, u64 start, u64 end) > > > { > > > struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb); > > > > > > + /* > > > + * Btrfs

[PATCH] Btrfs: add parent_transid parameter to veirfy_level_key

2018-05-15 Thread Liu Bo
@parent_transid could tell whether the eb's generation has been verified by the caller. Signed-off-by: Liu Bo --- fs/btrfs/disk-io.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index

Re: [PATCH] Btrfs: fix the corruption by reading stale btree blocks

2018-05-15 Thread Filipe Manana
On Tue, May 15, 2018 at 6:37 PM, Liu Bo wrote: > If a btree block, aka. extent buffer, is not available in the extent > buffer cache, it'll be read out from the disk instead, i.e. > > btrfs_search_slot() > read_block_for_search() # hold parent and its lock, go to read

Re: [PATCH] btrfs: property: Set incompat flag of lzo/zstd compression

2018-05-15 Thread Anand Jain
On 05/15/2018 03:51 PM, Misono Tomohiro wrote: Incompat flag of lzo/zstd compression should be set at: 1. mount time (-o compress/compress-force) 2. when defrag is done 3. when property is set Currently 3. is missing and this commit adds this. Signed-off-by: Tomohiro Misono

Re: [PATCH 1/2] btrfs: inode: Don't compress if NODATASUM or NODATACOW set

2018-05-15 Thread Nikolay Borisov
On 15.05.2018 10:36, Qu Wenruo wrote: > As btrfs(5) specified: > > Note > If nodatacow or nodatasum are enabled, compression is disabled. > > If NODATASUM or NODATACOW set, we should not compress the extent. > > Normally NODATACOW is detected properly in run_delalloc_range() so >

Re: [PATCH] btrfs: property: Set incompat flag of lzo/zstd compression

2018-05-15 Thread Su Yue
On 05/15/2018 04:05 PM, Su Yue wrote: > > > On 05/15/2018 03:51 PM, Misono Tomohiro wrote: >> Incompat flag of lzo/zstd compression should be set at: >> 1. mount time (-o compress/compress-force) >> 2. when defrag is done >> 3. when property is set >> >> Currently 3. is missing and this

Re: [PATCH 1/2] btrfs: inode: Don't compress if NODATASUM or NODATACOW set

2018-05-15 Thread Qu Wenruo
On 2018年05月15日 16:21, Nikolay Borisov wrote: > > > On 15.05.2018 10:36, Qu Wenruo wrote: >> As btrfs(5) specified: >> >> Note >> If nodatacow or nodatasum are enabled, compression is disabled. >> >> If NODATASUM or NODATACOW set, we should not compress the extent. >> >> Normally

Re: [PATCH 2/2] btrfs: lzo: Avoid decompressing obviously corrupted data

2018-05-15 Thread Qu Wenruo
On 2018年05月15日 16:05, Nikolay Borisov wrote: > > > On 15.05.2018 10:36, Qu Wenruo wrote: >> Unlike zlib decompression, lzo decompression doesn't need any >> initialization, thus we can't detect early corruption from >> initialization. >> >> However for lzo compressed extent, its first 4bytes

Re: [PATCH 2/2] btrfs: lzo: Avoid decompressing obviously corrupted data

2018-05-15 Thread Nikolay Borisov
On 15.05.2018 11:32, Qu Wenruo wrote: > > > On 2018年05月15日 16:05, Nikolay Borisov wrote: >> >> >> On 15.05.2018 10:36, Qu Wenruo wrote: >>> Unlike zlib decompression, lzo decompression doesn't need any >>> initialization, thus we can't detect early corruption from >>> initialization. >>> >>>

Re: [PATCH 1/2] btrfs: inode: Don't compress if NODATASUM or NODATACOW set

2018-05-15 Thread Nikolay Borisov
On 15.05.2018 11:30, Qu Wenruo wrote: > > > On 2018年05月15日 16:21, Nikolay Borisov wrote: >> >> >> On 15.05.2018 10:36, Qu Wenruo wrote: >>> As btrfs(5) specified: >>> >>> Note >>> If nodatacow or nodatasum are enabled, compression is disabled. >>> >>> If NODATASUM or NODATACOW set, we

Re: 4.15.6 crash: BUG at fs/btrfs/ctree.c:1862

2018-05-15 Thread Filipe Manana
On Tue, May 15, 2018 at 12:10 AM, Marc MERLIN wrote: > static noinline struct extent_buffer * > read_node_slot(struct btrfs_fs_info *fs_info, struct extent_buffer *parent, >int slot) > { > int level = btrfs_header_level(parent); > struct

Re: [PATCH] btrfs: property: Set incompat flag of lzo/zstd compression

2018-05-15 Thread Duncan
Su Yue posted on Tue, 15 May 2018 16:05:01 +0800 as excerpted: > > On 05/15/2018 03:51 PM, Misono Tomohiro wrote: >> Incompat flag of lzo/zstd compression should be set at: >> 1. mount time (-o compress/compress-force) >> 2. when defrag is done 3. when property is set >> >> Currently 3. is

Re: [PATCH] btrfs: property: Set incompat flag of lzo/zstd compression

2018-05-15 Thread Su Yue
On 05/15/2018 04:35 PM, Duncan wrote: > Su Yue posted on Tue, 15 May 2018 16:05:01 +0800 as excerpted: > > >> >> On 05/15/2018 03:51 PM, Misono Tomohiro wrote: >>> Incompat flag of lzo/zstd compression should be set at: >>> 1. mount time (-o compress/compress-force) >>> 2. when defrag is

Re: [PATCH 1/2] btrfs: inode: Don't compress if NODATASUM or NODATACOW set

2018-05-15 Thread Qu Wenruo
On 2018年05月15日 16:35, Nikolay Borisov wrote: > > > On 15.05.2018 11:30, Qu Wenruo wrote: >> >> >> On 2018年05月15日 16:21, Nikolay Borisov wrote: >>> >>> >>> On 15.05.2018 10:36, Qu Wenruo wrote: As btrfs(5) specified: Note If nodatacow or nodatasum are enabled,

[PATCH v14.6 11/14] btrfs: dedupe: Inband in-memory only de-duplication implement

2018-05-15 Thread Lu Fengqi
From: Qu Wenruo Core implement for inband de-duplication. It reuses the async_cow_start() facility to do the calculate dedupe hash. And use dedupe hash to do inband de-duplication at extent level. The workflow is as below: 1) Run delalloc range for an inode 2) Calculate

[PATCH v14.6 06/14] btrfs: dedupe: Introduce function to remove hash from in-memory tree

2018-05-15 Thread Lu Fengqi
From: Wang Xiaoguang Introduce static function inmem_del() to remove hash from in-memory dedupe tree. And implement btrfs_dedupe_del() and btrfs_dedup_disable() interfaces. Also for btrfs_dedupe_disable(), add new functions to wait existing writer and block incoming

[PATCH v14.6 01/14] btrfs: introduce type based delalloc metadata reserve

2018-05-15 Thread Lu Fengqi
From: Wang Xiaoguang Introduce type based metadata reserve parameter for delalloc space reservation/freeing function. The problem we are going to solve is, btrfs use different max extent size for different mount options. For compression, the max extent size is 128K,

[PATCH v14.6 00/14] Btrfs In-band De-duplication

2018-05-15 Thread Lu Fengqi
This patchset can be fetched from github: https://github.com/littleroad/linux.git dedupe_latest This is just a normal rebase update. Now the new base is v4.17-rc5 Normal test cases from auto group exposes no regression, and ib-dedupe group can pass without problem. Changelog: v2: Totally

[PATCH v14.6 05/14] btrfs: dedupe: Introduce function to add hash into in-memory tree

2018-05-15 Thread Lu Fengqi
From: Wang Xiaoguang Introduce static function inmem_add() to add hash into in-memory tree. And now we can implement the btrfs_dedupe_add() interface. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang

[PATCH v14.6 13/14] btrfs: relocation: Enhance error handling to avoid BUG_ON

2018-05-15 Thread Lu Fengqi
From: Qu Wenruo Since the introduction of btrfs dedupe tree, it's possible that balance can race with dedupe disabling. When this happens, dedupe_enabled will make btrfs_get_fs_root() return PTR_ERR(-ENOENT). But due to a bug in error handling branch, when this happens

[PATCH v14.6 02/14] btrfs: Introduce COMPRESS reserve type to fix false enospc for compression

2018-05-15 Thread Lu Fengqi
From: Wang Xiaoguang When testing btrfs compression, sometimes we got ENOSPC error, though fs still has much free space, xfstests generic/171, generic/172, generic/173, generic/174, generic/175 can reveal this bug in my test environment when compression is enabled.

[PATCH v14.6 08/14] btrfs: dedupe: Introduce function to search for an existing hash

2018-05-15 Thread Lu Fengqi
From: Wang Xiaoguang Introduce static function inmem_search() to handle the job for in-memory hash tree. The trick is, we must ensure the delayed ref head is not being run at the time we search the for the hash. With inmem_search(), we can implement the

[PATCH v10.2 4/5] btrfs-progs: dedupe: Add status subcommand

2018-05-15 Thread Lu Fengqi
From: Qu Wenruo Add status subcommand for dedupe command group. Signed-off-by: Qu Wenruo --- Documentation/btrfs-dedupe-inband.asciidoc | 3 + btrfs-completion | 2 +- cmds-dedupe-ib.c | 81

[PATCH v10.2 5/5] btrfs-progs: dedupe: introduce reconfigure subcommand

2018-05-15 Thread Lu Fengqi
From: Qu Wenruo Introduce reconfigure subcommand to co-operate with new kernel ioctl modification. Signed-off-by: Qu Wenruo --- Documentation/btrfs-dedupe-inband.asciidoc | 7 ++ cmds-dedupe-ib.c | 75

[PATCH v10.2 3/5] btrfs-progs: dedupe: Add disable support for inband dedupelication

2018-05-15 Thread Lu Fengqi
From: Qu Wenruo Add disable subcommand for dedupe command group. Signed-off-by: Qu Wenruo Signed-off-by: Lu Fengqi --- Documentation/btrfs-dedupe-inband.asciidoc | 5 +++ btrfs-completion |

[PATCH v14.6 04/14] btrfs: dedupe: Introduce function to initialize dedupe info

2018-05-15 Thread Lu Fengqi
From: Wang Xiaoguang Add generic function to initialize dedupe info. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang Reviewed-by: Josef Bacik Signed-off-by: Lu Fengqi

[PATCH v14.6 12/14] btrfs: dedupe: Add ioctl for inband deduplication

2018-05-15 Thread Lu Fengqi
From: Wang Xiaoguang Add ioctl interface for inband deduplication, which includes: 1) enable 2) disable 3) status And a pseudo RO compat flag, to imply that btrfs now supports inband dedup. However we don't add any ondisk format change, it's just a pseudo RO compat

[PATCH v10.2 0/5] In-band de-duplication for btrfs-progs

2018-05-15 Thread Lu Fengqi
Patchset can be fetched from github: https://github.com/littleroad/btrfs-progs.git dedupe_latest Inband dedupe(in-memory backend only) ioctl support for btrfs-progs. v7 changes: Update ctree.h to follow kernel structure change Update print-tree to follow kernel structure change V8 changes:

[PATCH v10.2 2/5] btrfs-progs: dedupe: Add enable command for dedupe command group

2018-05-15 Thread Lu Fengqi
From: Qu Wenruo Add enable subcommand for dedupe commmand group. Signed-off-by: Qu Wenruo Signed-off-by: Lu Fengqi --- Documentation/btrfs-dedupe-inband.asciidoc | 114 +- btrfs-completion

[PATCH v14.6 10/14] btrfs: ordered-extent: Add support for dedupe

2018-05-15 Thread Lu Fengqi
From: Wang Xiaoguang Add ordered-extent support for dedupe. Note, current ordered-extent support only supports non-compressed source extent. Support for compressed source extent will be added later. Signed-off-by: Qu Wenruo Signed-off-by:

[PATCH v14.6 07/14] btrfs: delayed-ref: Add support for increasing data ref under spinlock

2018-05-15 Thread Lu Fengqi
From: Qu Wenruo For in-band dedupe, btrfs needs to increase data ref with delayed_ref locked, so add a new function btrfs_add_delayed_data_ref_lock() to increase extent ref with delayed_refs already locked. Signed-off-by: Qu Wenruo Reviewed-by:

[PATCH v14.6 03/14] btrfs: dedupe: Introduce dedupe framework and its header

2018-05-15 Thread Lu Fengqi
From: Wang Xiaoguang Introduce the header for btrfs in-band(write time) de-duplication framework and needed header. The new de-duplication framework is going to support 2 different dedupe methods and 1 dedupe hash. Signed-off-by: Qu Wenruo

[PATCH v10.2 1/5] btrfs-progs: Basic framework for dedupe-inband command group

2018-05-15 Thread Lu Fengqi
From: Qu Wenruo Add basic ioctl header and command group framework for later use. Alone with basic man page doc. Signed-off-by: Qu Wenruo Signed-off-by: Lu Fengqi --- Documentation/Makefile.in | 1 +

[PATCH v14.6 14/14] btrfs: dedupe: Introduce new reconfigure ioctl

2018-05-15 Thread Lu Fengqi
From: Qu Wenruo Introduce new reconfigure ioctl and new FORCE flag for in-band dedupe ioctls. Now dedupe enable and reconfigure ioctl are stateful. | Current state | Ioctl| Next state |

[PATCH v14.6 09/14] btrfs: dedupe: Implement btrfs_dedupe_calc_hash interface

2018-05-15 Thread Lu Fengqi
From: Wang Xiaoguang Unlike in-memory or on-disk dedupe method, only SHA256 hash method is supported yet, so implement btrfs_dedupe_calc_hash() interface using SHA256. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang

Re: [PATCH 2/2] btrfs: add balance args info during start and resume

2018-05-15 Thread Anand Jain
All comments accepted in v2 in ml, except for the one below.   clear_bit(BTRFS_FS_BALANCE_RUNNING, _info->flags);   if (bargs) { @@ -3947,10 +4096,8 @@ static int balance_kthread(void *data)   int ret = 0;   mutex_lock(_info->balance_mutex); -    if (fs_info->balance_ctl) { -  

[PATCH v2 1/3] btrfs: balance: prefix kernel logs

2018-05-15 Thread Anand Jain
Kernel logs are very important for the forensic investigations of the issues in general make it easy to use it. This patch adds 'balance:' prefix so that it can be easily searched. Signed-off-by: Anand Jain --- v1-v2: Change log update. fs/btrfs/volumes.c | 34

[PATCH v2 3/3] btrfs: balance: add kernel log for end or paused

2018-05-15 Thread Anand Jain
Add a kernel log when the balance ends, either for cancel or completed or if it is paused. --- v1->v2: Moved from 2/3 to 3/3 fs/btrfs/volumes.c | 7 +++ 1 file changed, 7 insertions(+) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index ce68c4f42f94..a4e243a29f5c 100644 ---

[PATCH v2 0/3] btrfs: balance: improve kernel logs

2018-05-15 Thread Anand Jain
Kernel logs are very important for the forensic investigations of the issues, these patchs make balance logs easy to review. Anand Jain (3): btrfs: balance: prefix kernel logs btrfs: balance: add args info during start and resume btrfs: balance: add kernel log for end or paused

[PATCH] Btrfs: implement unlocked buffered write

2018-05-15 Thread robbieko
From: Robbie Ko This idea is from direct io. By this patch, we can make the buffered write parallel, and improve the performance and latency. But because we can not update isize without i_mutex, the unlocked buffered write just can be done in front of the EOF. We needn't

Re: [PATCH] btrfs: test ENOSPC caused by many orphan items

2018-05-15 Thread Omar Sandoval
On Wed, May 16, 2018 at 09:48:58AM +0800, Eryu Guan wrote: > On Wed, May 09, 2018 at 11:21:55PM -0700, Omar Sandoval wrote: > > From: Omar Sandoval > > > > Btrfs has a bug where we can prematurely ENOSPC if we have lots of > > orphaned files, i.e., deleted files which are still

Re: [PATCH] btrfs: test ENOSPC caused by many orphan items

2018-05-15 Thread Eryu Guan
On Tue, May 15, 2018 at 07:14:02PM -0700, Omar Sandoval wrote: > On Wed, May 16, 2018 at 09:48:58AM +0800, Eryu Guan wrote: > > On Wed, May 09, 2018 at 11:21:55PM -0700, Omar Sandoval wrote: > > > From: Omar Sandoval > > > > > > Btrfs has a bug where we can prematurely ENOSPC if

Re: [PATCH V3] test online label ioctl

2018-05-15 Thread Dave Chinner
On Tue, May 15, 2018 at 10:22:37AM -0500, Eric Sandeen wrote: > This tests the online label ioctl that btrfs has, which has been > recently proposed for XFS. > > To run, it requires an updated xfs_io with the label command and a > filesystem that supports it > > A slight change here to

Re: [PATCH] Btrfs: fix the corruption by reading stale btree blocks

2018-05-15 Thread Qu Wenruo
On 2018年05月16日 01:37, Liu Bo wrote: > If a btree block, aka. extent buffer, is not available in the extent > buffer cache, it'll be read out from the disk instead, i.e. > > btrfs_search_slot() > read_block_for_search() # hold parent and its lock, go to read child > btrfs_release_path() >

Re: [PATCH] Btrfs: add parent_transid parameter to veirfy_level_key

2018-05-15 Thread Qu Wenruo
On 2018年05月16日 01:37, Liu Bo wrote: > @parent_transid could tell whether the eb's generation has been verified > by the caller. > > Signed-off-by: Liu Bo Looks pretty useful to debug the btrfs/124 bug you just fixed. But a small nitpick inlined below: > --- >

Re: [PATCH] btrfs: test ENOSPC caused by many orphan items

2018-05-15 Thread Eryu Guan
On Wed, May 09, 2018 at 11:21:55PM -0700, Omar Sandoval wrote: > From: Omar Sandoval > > Btrfs has a bug where we can prematurely ENOSPC if we have lots of > orphaned files, i.e., deleted files which are still open. Add a test > which repeatedly creates and deletes a file while

Re: [PATCH] btrfs: update uuid_mutex and device_list_mutex comments

2018-05-15 Thread Anand Jain
On 04/24/2018 11:48 PM, David Sterba wrote: On Wed, Apr 18, 2018 at 05:56:31PM +0800, Anand Jain wrote: @@ -155,29 +155,26 @@ static int __btrfs_map_block(struct btrfs_fs_info *fs_info, * * uuid_mutex (global lock) * - * protects the fs_uuids list that

[PATCH v2] btrfs: update uuid_mutex and device_list_mutex comments

2018-05-15 Thread Anand Jain
Make the uuid_mutex and device_list_mutex comments inline with the changes. Signed-off-by: Anand Jain --- v1->v2: Fix typo. fs_devs -> fs_uuids fs/btrfs/volumes.c | 21 + 1 file changed, 9 insertions(+), 12 deletions(-) diff --git

Re: [PATCH v4 1/3] btrfs: Add unprivileged ioctl which returns subvolume information

2018-05-15 Thread Misono Tomohiro
On 2018/05/15 16:57, Gu, Jinxiang/顾 金香 wrote: > Hi, add a missed a comment. > >> -Original Message- >> From: Misono Tomohiro [mailto:misono.tomoh...@jp.fujitsu.com] >> Sent: Tuesday, May 15, 2018 3:04 PM >> To: Gu, Jinxiang/顾 金香 ; linux-btrfs@vger.kernel.org >>

[PATCH 14/14] mm: turn on vm_fault_t type checking

2018-05-15 Thread Christoph Hellwig
Switch vm_fault_t to point to an unsigned int with __bіtwise annotations. This both catches any old ->fault or ->page_mkwrite instance with plain compiler type checking, as well as finding more intricate problems with sparse. Signed-off-by: Christoph Hellwig ---

[PATCH 13/14] mm: move arch specific VM_FAULT_* flags to mm.h

2018-05-15 Thread Christoph Hellwig
Various architectures define their own internal flags. Not sure a public header like mm.h is a good place, but keeping them inside the arch code with possible conflicts also seems like a bad idea. Maybe we just need to stop overloading the value instead. Signed-off-by: Christoph Hellwig

[PATCH 08/14] ocfs2: separate errno from VM_FAULT_* values

2018-05-15 Thread Christoph Hellwig
Signed-off-by: Christoph Hellwig --- fs/ocfs2/mmap.c | 36 +++- 1 file changed, 19 insertions(+), 17 deletions(-) diff --git a/fs/ocfs2/mmap.c b/fs/ocfs2/mmap.c index fb9a20e3d608..e75c1fc5333e 100644 --- a/fs/ocfs2/mmap.c +++ b/fs/ocfs2/mmap.c @@

[PATCH v5 2/3] btrfs: Add unprivileged ioctl which returns subvolume's ROOT_REF

2018-05-15 Thread Tomohiro Misono
Add unprivileged ioctl BTRFS_IOC_GET_SUBVOL_ROOTREF which returns ROOT_REF information of the subvolume containing this inode except the subvolume name (this is because to prevent potential name leak). The subvolume name will be gained by user version of ino_lookup ioctl

[PATCH v5 1/3] btrfs: Add unprivileged ioctl which returns subvolume information

2018-05-15 Thread Tomohiro Misono
Add new unprivileged ioctl BTRFS_IOC_GET_SUBVOL_INFO which returns the information of subvolume containing this inode. (i.e. returns the information in ROOT_ITEM and ROOT_BACKREF.) Signed-off-by: Tomohiro Misono --- v4 -> v5 - Update error handling of

[PATCH v5 0/3] btrfs: Add three new unprivileged ioctls to allow normal users to call "sub list/show" etc.

2018-05-15 Thread Tomohiro Misono
[based on current misc-next] changelog: v4 -> v5 - Update error handling of 1st/2nd patch. See each log for details - Fix misspelling v3 -> v4 - call btrfs_next_leaf() after btrfs_search_slot() when the slot position exceeds the number of items - rebased to current misc-next v2 -> v3

[PATCH v5 3/3] btrfs: Add unprivileged version of ino_lookup ioctl

2018-05-15 Thread Tomohiro Misono
Add unprivileged version of ino_lookup ioctl BTRFS_IOC_INO_LOOKUP_USER to allow normal users to call "btrfs subvololume list/show" etc. in combination with BTRFS_IOC_GET_SUBVOL_INFO/BTRFS_IOC_GET_SUBVOL_ROOTREF. This can be used like BTRFS_IOC_INO_LOOKUP but the argument is different. This is

[PATCH 11/14] ttm: separate errno from VM_FAULT_* values

2018-05-15 Thread Christoph Hellwig
Signed-off-by: Christoph Hellwig --- drivers/gpu/drm/ttm/ttm_bo_vm.c | 42 + 1 file changed, 22 insertions(+), 20 deletions(-) diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c index 8eba95b3c737..255e7801f62c 100644 ---

[PATCH 12/14] lustre: separate errno from VM_FAULT_* values

2018-05-15 Thread Christoph Hellwig
Signed-off-by: Christoph Hellwig --- .../staging/lustre/lustre/llite/llite_mmap.c | 37 +++ .../lustre/lustre/llite/vvp_internal.h| 2 +- 2 files changed, 14 insertions(+), 25 deletions(-) diff --git a/drivers/staging/lustre/lustre/llite/llite_mmap.c

[PATCH 05/14] ceph: untangle ceph_filemap_fault

2018-05-15 Thread Christoph Hellwig
Streamline the code to have a somewhat natural flow, and separate the errno values from the VM_FAULT_* values. Signed-off-by: Christoph Hellwig --- fs/ceph/addr.c | 100 + 1 file changed, 51 insertions(+), 49 deletions(-) diff --git

  1   2   >