Re: [PATCH v2] btrfs-progs: ctree: Add extra level check for read_node_slot()

On 2018年03月20日 13:16, Qu Wenruo wrote: > > > On 2018年03月19日 20:36, David Sterba wrote: >> On Thu, Feb 08, 2018 at 08:59:40AM +0800, Qu Wenruo wrote: >>> Strangely, we have level check in btrfs_print_tree() while we don't have >>> the same check in read_node_slot(). >>> >>> That's to say, for

Re: [PATCH] btrfs-progs: inspect-dump-tree: add '-f|--follow' options to print all children tree blocks for '-b'

On 2018年03月20日 02:44, David Sterba wrote: > On Wed, Mar 14, 2018 at 09:05:38AM +0800, Qu Wenruo wrote: >> When debuging with "btrfs inspect dump-tree", it's not that handy if we >> want to iterate all child tree blocks starting from a specified block. >> >> -b can only print a single block,

Re: [PATCH v2] btrfs-progs: ctree: Add extra level check for read_node_slot()

On 2018年03月19日 20:36, David Sterba wrote: > On Thu, Feb 08, 2018 at 08:59:40AM +0800, Qu Wenruo wrote: >> Strangely, we have level check in btrfs_print_tree() while we don't have >> the same check in read_node_slot(). >> >> That's to say, for the following corruption, btrfs_search_slot() or >>

Re: [PATCH v5 0/2] Remove false-positive VLAs when using max()

On Tue, Mar 20, 2018 at 7:29 AM, Linus Torvalds wrote: > On Mon, Mar 19, 2018 at 2:43 AM, David Laight wrote: >> >> Is it necessary to have the full checks for old versions of gcc? >> >> Even -Wvla could be predicated on very recent gcc -

Re: [RFC PATCH v3 5/7] btrfs-progs: sub list: Allow normal user to call "subvolume list/show"

On 2018/03/20 2:09, Goffredo Baroncelli wrote: > On 03/19/2018 08:32 AM, Misono, Tomohiro wrote [snip] >> static void print_subvolume_column(struct root_info *subv, >> enum btrfs_list_column_enum column) >> { >> @@ -1492,19 +1800,33 @@ next: >> static int

Re: [PATCH v5 0/2] Remove false-positive VLAs when using max()

On Mon, Mar 19, 2018 at 2:43 AM, David Laight wrote: > > Is it necessary to have the full checks for old versions of gcc? > > Even -Wvla could be predicated on very recent gcc - since we aren't > worried about whether gcc decides to generate a vla, but whether > the

Re: [PATCH] btrfs: Validate child tree block's level and first key

Hi Qu, Thank you for the patch! Yet something to improve: [auto build test ERROR on v4.16-rc4] [also build test ERROR on next-20180319] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url: https://github.com/0day-ci/linux/commits/Qu-Wenruo

Re: [PATCH] btrfs: fix lockdep splat in btrfs_alloc_subvolume_writers

On 3/19/18 2:08 PM, David Sterba wrote: > On Mon, Mar 19, 2018 at 01:52:05PM -0400, Jeff Mahoney wrote: >> On 3/16/18 4:12 PM, David Sterba wrote: >>> On Fri, Mar 16, 2018 at 02:36:27PM -0400, je...@suse.com wrote: From: Jeff Mahoney While running btrfs/011, I hit

Re: [PATCH] btrfs: fix lockdep splat in btrfs_alloc_subvolume_writers

On Mon, Mar 19, 2018 at 01:52:05PM -0400, Jeff Mahoney wrote: > On 3/16/18 4:12 PM, David Sterba wrote: > > On Fri, Mar 16, 2018 at 02:36:27PM -0400, je...@suse.com wrote: > >> From: Jeff Mahoney > >> > >> While running btrfs/011, I hit the following lockdep splat. > >> > >> This

Re: [PATCH] btrfs-progs: print-tree: Do correct offset output for ROOT_ITEM

On Mon, Mar 19, 2018 at 06:28:10PM +0800, Qu Wenruo wrote: > Commit Fixes: 8c36786c8198 ("btrfs-progs: print-tree: Print offset as tree > objectid for ROOT_ITEM") > changes how we translate offset of ROOT_ITEM. > > However the fact is, even for ROOT_ITEM, we have different meaning of > offset. >

Re: [PATCH 4.14 024/110] btrfs: use proper endianness accessors for super_copy

On Sat, Mar 17, 2018 at 06:27:15PM +0100, Christoph Biedl wrote: > Greg Kroah-Hartman wrote... > > > On Thu, Mar 15, 2018 at 07:55:42PM +0100, Christoph Biedl wrote: > > > > > commit 3c181c12c431fe33b669410d663beb9cceefcd1b upstream. > > > > On big-endian systems, this change intruduces severe

Re: [PATCH 1/1] btrfs-progs: docs: annual typo, clarity, & grammar review & fixups

On Thu, Mar 15, 2018 at 08:49:26PM -0400, Nicholas D Steeves wrote: > Hi Qu, > > So sorry for the incredibly delayed reply [it got lost in my drafts > folder], I sincerely appreciate the time you took to respond. There > is a lot in your responses that I suspect would benefit readers of the >

Re: [PATCH] btrfs-progs: inspect-dump-tree: add '-f|--follow' options to print all children tree blocks for '-b'

On Wed, Mar 14, 2018 at 09:05:38AM +0800, Qu Wenruo wrote: > When debuging with "btrfs inspect dump-tree", it's not that handy if we > want to iterate all child tree blocks starting from a specified block. > > -b can only print a single block, while without -b "btrfs inspect dump-tree" > will

Re: [PATCH] btrfs-progs: print-tree: Enhance warning on tree block level mismatch and error handling

On Thu, Mar 15, 2018 at 12:48:15PM +0800, Qu Wenruo wrote: > This patch enhance the tree block level mismatch by the following > methods: > > 1) Merge same warning branches into one >We had two branches showing the same message, and their condition >is also the same. Merge them > > 2)

Re: Crashes running btrfs scrub

On Sun, Mar 18, 2018 at 3:52 PM, waxhead wrote: > Liu Bo wrote: >> >> On Sat, Mar 17, 2018 at 5:26 PM, Liu Bo wrote: >>> >>> On Fri, Mar 16, 2018 at 2:46 PM, Mike Stevens >>> wrote: > > Could you please paste the

Re: [PATCH] btrfs: fix lockdep splat in btrfs_alloc_subvolume_writers

On 3/16/18 4:12 PM, David Sterba wrote: > On Fri, Mar 16, 2018 at 02:36:27PM -0400, je...@suse.com wrote: >> From: Jeff Mahoney >> >> While running btrfs/011, I hit the following lockdep splat. >> >> This is the important bit: >>pcpu_alloc+0x1ac/0x5e0 >>

Re: [RFC PATCH v3 5/7] btrfs-progs: sub list: Allow normal user to call "subvolume list/show"

On 03/19/2018 08:32 AM, Misono, Tomohiro wrote: > Allow normal user to call "subvolume list/show" by using 3 new > unprivileged ioctls (BTRFS_IOC_GET_SUBVOL_INFO, > BTRFS_IOC_GET_SUBVOL_ROOTREF and BTRFS_IOC_INO_LOOKUP_USER). > > Note that for root, "subvolume list" returns all the subvolume in

Re: [PATCH] btrfs-progs: mkfs: add uuid and otime to ROOT_ITEM of FS_TREE

On Mon, Mar 19, 2018 at 02:26:38PM +0100, Christoph Anton Mitterer wrote: > On Mon, 2018-03-19 at 14:02 +0100, David Sterba wrote: > > We can do that by a special purpose tool. > > No average user will ever run (even know) about that... > > Could you perhaps either do it automatically in fsck

Re: [PATCH v2 7/7] btrfs-progs: test/mkfs: Add test case for rootdir inline extent size

On Tue, Mar 13, 2018 at 09:56:16AM +0800, Qu Wenruo wrote: > Add a test case for mkfs --rootdir, using files with different file > sizes to check if invalid large inline extent could exist. > > Signed-off-by: Qu Wenruo > --- > tests/mkfs-tests/014-rootdir-inline-extent/test.sh |

Re: [PATCH] btrfs-progs: mkfs: add uuid and otime to ROOT_ITEM of FS_TREE

On Mon, 2018-03-19 at 14:02 +0100, David Sterba wrote: > We can do that by a special purpose tool. No average user will ever run (even know) about that... Could you perhaps either do it automatically in fsck (which is IMO als a bad idea as fsck should be read-only per default)... or at least add

Re: [PATCH v2 1/7] btrfs-progs: convert/ext2: Fix inline extent creation check

On Tue, Mar 13, 2018 at 09:56:10AM +0800, Qu Wenruo wrote: > Signed-off-by: Qu Wenruo Please squash patches 1 and 2 together and write a changelog, somethiing like is in the cover letter. This is a bugfix, we need to document what was broken and how it's fixed. -- To unsubscribe

Re: [PATCH] btrfs-progs: mkfs: add uuid and otime to ROOT_ITEM of FS_TREE

On Mon, Mar 19, 2018 at 02:02:23PM +0100, David Sterba wrote: > On Mon, Mar 19, 2018 at 08:20:10AM +, Hugo Mills wrote: > > On Mon, Mar 19, 2018 at 05:16:42PM +0900, Misono, Tomohiro wrote: > > > Currently, the top-level subvolume lacks the UUID. As a result, both > > > non-snapshot subvolume

Re: [PATCH] btrfs-progs: mkfs: add uuid and otime to ROOT_ITEM of FS_TREE

On Mon, Mar 19, 2018 at 08:20:10AM +, Hugo Mills wrote: > On Mon, Mar 19, 2018 at 05:16:42PM +0900, Misono, Tomohiro wrote: > > Currently, the top-level subvolume lacks the UUID. As a result, both > > non-snapshot subvolume and snapshot of top-level subvolume do not have > > Parent UUID and

Re: [PATCH] btrfs-progs: convert/ext2: Remove check for ext2_ext_attr_entry->e_value_block

On Wed, Mar 14, 2018 at 08:56:57AM +0800, Qu Wenruo wrote: > In latest e2fsprogs (1.44.0) definition of ext2_ext_attr_entry has > removed member e_value_block, as currently ext* doesn't support it set > anyway. > > So remove such check so that we can pass compile. > > Signed-off-by: Qu Wenruo

Re: [PATCH 2/2 v3] Btrfs-progs: add fsck test for filesystem with shared prealloc extents

On Wed, Mar 14, 2018 at 08:11:18PM +, fdman...@kernel.org wrote: > From: Filipe Manana > > Verify that a filesystem check operation (fsck) does not report the > following scenario as an error: > > An extent is shared between two inodes, as a result of clone/reflink >

Re: [PATCH 1/2 v3] Btrfs-progs: check, fix false error reports for shared prealloc extents

On Thu, Mar 15, 2018 at 02:35:36PM +0200, Nikolay Borisov wrote: > On 14.03.2018 22:11, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > Under some cases the filesystem checker reports an error when it finds > > checksum items for an extent that is referenced by an

Re: [PATCH v2] btrfs-progs: ctree: Add extra level check for read_node_slot()

On Thu, Feb 08, 2018 at 08:59:40AM +0800, Qu Wenruo wrote: > Strangely, we have level check in btrfs_print_tree() while we don't have > the same check in read_node_slot(). > > That's to say, for the following corruption, btrfs_search_slot() or > btrfs_next_leaf() can return invalid leaf: > >

Re: [PATCH] btrfs-progs: print-tree: Do correct offset output for ROOT_ITEM

On 19.03.2018 12:28, Qu Wenruo wrote: > Commit Fixes: 8c36786c8198 ("btrfs-progs: print-tree: Print offset as tree > objectid for ROOT_ITEM") > changes how we translate offset of ROOT_ITEM. > > However the fact is, even for ROOT_ITEM, we have different meaning of > offset. > > For tree reloc

Re: High CPU usage of RT threads...

Thank you for the analysis, Nikolay. Will try to upgrade the kernel and check if the issue reproduces. Regards, Shyam On Mon, Mar 19, 2018 at 5:21 PM, Nikolay Borisov wrote: > > > On 19.03.2018 13:48, Shyam Prasad N wrote: >> On Mon, Mar 19, 2018 at 4:37 PM, Nikolay Borisov

Re: High CPU usage of RT threads...

On 19.03.2018 13:48, Shyam Prasad N wrote: > On Mon, Mar 19, 2018 at 4:37 PM, Nikolay Borisov wrote: >> >> >> On 19.03.2018 13:02, Shyam Prasad N wrote: >>> Hi, >>> >>> Attaching the sysrq-trigger output. >> >> Has this been obtained while the machine experienced a period of

Re: High CPU usage of RT threads...

On Mon, Mar 19, 2018 at 4:37 PM, Nikolay Borisov wrote: > > > On 19.03.2018 13:02, Shyam Prasad N wrote: >> Hi, >> >> Attaching the sysrq-trigger output. > > Has this been obtained while the machine experienced a period of a lot > of blocked threads? Because the output shows a

Re: [PATCH] btrfs-progs: print-tree: Do correct offset output for ROOT_ITEM

On 2018年03月19日 19:15, Nikolay Borisov wrote: > > > On 19.03.2018 12:54, Nikolay Borisov wrote: >> >> >> On 19.03.2018 12:50, Qu Wenruo wrote: >>> What about this value? >>> 18446744073709551607 >>> >>> It's data reloc tree objectid. >>> And it's pretty long since it's -9ULL. >>> >>> At least

Re: [PATCH] btrfs-progs: print-tree: Do correct offset output for ROOT_ITEM

On 19.03.2018 12:54, Nikolay Borisov wrote: > > > On 19.03.2018 12:50, Qu Wenruo wrote: >> What about this value? >> 18446744073709551607 >> >> It's data reloc tree objectid. >> And it's pretty long since it's -9ULL. >> >> At least this makes 2 objectid more readable. > > Fair enough. >

Re: High CPU usage of RT threads...

Hi, Attaching the sysrq-trigger output. Regards, Shyam On Mon, Mar 19, 2018 at 12:45 PM, Nikolay Borisov wrote: > > > On 19.03.2018 09:13, Shyam Prasad N wrote: >> Hi Nikolay, >> >> Thanks for your reply on this. >> >> Checked the stack trace for many of the stuck threads.

Re: [PATCH] btrfs-progs: print-tree: Do correct offset output for ROOT_ITEM

On 19.03.2018 12:50, Qu Wenruo wrote: > What about this value? > 18446744073709551607 > > It's data reloc tree objectid. > And it's pretty long since it's -9ULL. > > At least this makes 2 objectid more readable. Fair enough. -- To unsubscribe from this list: send the line "unsubscribe

Re: [PATCH] btrfs-progs: print-tree: Do correct offset output for ROOT_ITEM

On 2018年03月19日 18:46, Nikolay Borisov wrote: > > > On 19.03.2018 12:28, Qu Wenruo wrote: >> Commit Fixes: 8c36786c8198 ("btrfs-progs: print-tree: Print offset as tree >> objectid for ROOT_ITEM") >> changes how we translate offset of ROOT_ITEM. >> >> However the fact is, even for ROOT_ITEM, we

Re: [PATCH] btrfs-progs: print-tree: Do correct offset output for ROOT_ITEM

On 19.03.2018 12:28, Qu Wenruo wrote: > Commit Fixes: 8c36786c8198 ("btrfs-progs: print-tree: Print offset as tree > objectid for ROOT_ITEM") > changes how we translate offset of ROOT_ITEM. > > However the fact is, even for ROOT_ITEM, we have different meaning of > offset. > > For tree reloc

Re: [PATCH v5 2/2] Remove false-positive VLAs when using max()

On 03/16/2018 07:25 AM, Kees Cook wrote: > As part of removing VLAs from the kernel[1], we want to build with -Wvla, > but it is overly pessimistic and only accepts constant expressions for > stack array sizes, instead of also constant values. The max() macro > triggers the warning, so this

[PATCH] btrfs-progs: print-tree: Do correct offset output for ROOT_ITEM

Commit Fixes: 8c36786c8198 ("btrfs-progs: print-tree: Print offset as tree objectid for ROOT_ITEM") changes how we translate offset of ROOT_ITEM. However the fact is, even for ROOT_ITEM, we have different meaning of offset. For tree reloc tree, it's indeed subvolume id. But for other trees,

RE: [PATCH v5 0/2] Remove false-positive VLAs when using max()

From: linus...@gmail.com [mailto:linus...@gmail.com] On Behalf Of Linus Torvalds > Sent: 18 March 2018 23:36 ... > > Yeah, and since we're in the situation that *new* gcc versions work > for us anyway, and we only have issues with older gcc's (that sadly > people still use), even if there was a

[PATCH] btrfs: Validate child tree block's level and first key

We have several reports about node pointer points to incorrect child tree blocks, which could have even wrong owner and level but still with valid generation and checksum. Although btrfs check could handle it and print error message like: leaf parent key incorrect 60670574592 Kernel doesn't have

Re: Kernel warning - not sure if this is important

On 19.03.2018 10:21, Peter Chant wrote: > I got this kernel warning overnight. Possibly during or after a dedup > using duperemove. I'm not sure if that is relevent. Seems to relate to > fs/btrfs/backref.c line 1266. > > Don't know if it is important. Thought I'd post it just in case. > >

Kernel warning - not sure if this is important

I got this kernel warning overnight. Possibly during or after a dedup using duperemove. I'm not sure if that is relevent. Seems to relate to fs/btrfs/backref.c line 1266. Don't know if it is important. Thought I'd post it just in case. I'm afraid this is a screen-shot in the old fashioned

Re: [PATCH] btrfs-progs: mkfs: add uuid and otime to ROOT_ITEM of FS_TREE

On Mon, Mar 19, 2018 at 05:16:42PM +0900, Misono, Tomohiro wrote: > Currently, the top-level subvolume lacks the UUID. As a result, both > non-snapshot subvolume and snapshot of top-level subvolume do not have > Parent UUID and cannot be distinguisued. Therefore "fi show" of > top-level lists all

[PATCH] btrfs-progs: mkfs: add uuid and otime to ROOT_ITEM of FS_TREE

Currently, the top-level subvolume lacks the UUID. As a result, both non-snapshot subvolume and snapshot of top-level subvolume do not have Parent UUID and cannot be distinguisued. Therefore "fi show" of top-level lists all the subvolumes which lacks the UUID in "Snapshot(s)" filed. Also, it

[RFC PATCH v3 7/7] btrfs-porgs: test: Add cli-test/009 to check subvolume list for both root and normal user

Signed-off-by: Tomohiro Misono --- tests/cli-tests/009-subvolume-list/test.sh | 136 + 1 file changed, 136 insertions(+) create mode 100755 tests/cli-tests/009-subvolume-list/test.sh diff --git

[RFC PATCH v3 5/7] btrfs-progs: sub list: Allow normal user to call "subvolume list/show"

Allow normal user to call "subvolume list/show" by using 3 new unprivileged ioctls (BTRFS_IOC_GET_SUBVOL_INFO, BTRFS_IOC_GET_SUBVOL_ROOTREF and BTRFS_IOC_INO_LOOKUP_USER). Note that for root, "subvolume list" returns all the subvolume in the filesystem by default, but for normal user, it returns

[RFC PATCH v3 6/7] btrfs-progs: test: Add helper function to check if test user exists

Test user 'progs-test' will be used to test the behavior of normal user. In order to pass this check, add the user by "useradd -M progs-test". Note that progs-test should not have root privileges. Signed-off-by: Tomohiro Misono --- tests/common | 10 ++

[RFC PATCH v3 4/7] btrfs-progs: fallback to open without O_NOATIME flag in find_mount_root()

O_NOATIME flag requires effective UID of process matches file's owner or has CAP_FOWNER capabilities. Fallback to open without O_NOATIME flag so that normal user can also call find_mount_root(). This is a preparation work to allow normal user to call "subvolume show". Signed-off-by: Tomohiro

[RFC PATCH v3 3/7] btrfs-progs: sub list: Pass specified path down to btrfs_list_subvols()

This is a preparetion work to allow normal user to call "subvolume list/show". Signed-off-by: Tomohiro Misono --- btrfs-list.c | 16 +--- btrfs-list.h | 7 --- cmds-subvolume.c | 6 +++--- utils.c | 10 +- 4 files

[RFC PATCH v3 1/7] btrfs-progs: sub list: Call rb_free_nodes() in error path

After btrfs_list_subvols() is called, root_lookup may hold some allocated memory area even if the function fails. Therefore rb_free_nodes() should be called. Signed-off-by: Tomohiro Misono --- btrfs-list.c | 16 1 file changed, 12 insertions(+),

[RFC PATCH v3 2/7] btrfs-progs: ioctl: Add 3 definitions of new unprivileged ioctl

Add 3 definitions of new unprivileged ioctl (BTRFS_IOC_GET_SUBVOL_INFO, BTRFS_IOC_GET_SUBVOL_ROOTREF and BTRFS_IOC_INO_LOOKUP_USER). They will be used to implement user version of "btrfs subvolume list" etc. Signed-off-by: Tomohiro Misono --- ioctl.h | 86

[RFC PATCH v3 0/7] btrfs-progs: Allow normal user to call "subvolume list/show"

changelog: v2 -> v3 - use get_euid() to check the caller's privilege (and remove 3rd patch) - improve error handling v1 -> v2 - add independent error handling patch (1st patch) - reimplement according to ioctl change - various cleanup === This RFC implements user version of "subvolume

[PATCH v3 3/3] btrfs: Add unprivileged version of ino_lookup ioctl

Add unprivileged version of ino_lookup ioctl BTRFS_IOC_INO_LOOKUP_USER to allow normal users to call "btrfs subvololume list/show" etc. in combination with BTRFS_IOC_GET_SUBVOL_INFO/BTRFS_IOC_GET_SUBVOL_ROOTREF. This can be used like BTRFS_IOC_INO_LOOKUP but the argument is different. This is

[PATCH v3 1/3] btrfs: Add unprivileged ioctl which returns subvolume information

Add new unprivileged ioctl BTRFS_IOC_GET_SUBVOL_INFO which returns the information of subvolume containing this inode. (i.e. returns the information in ROOT_ITEM and ROOT_BACKREF.) Signed-off-by: Tomohiro Misono --- fs/btrfs/ioctl.c | 118

[PATCH v3 2/3] btrfs: Add unprivileged ioctl which returns subvolume's ROOT_REF

Add unprivileged ioctl BTRFS_IOC_GET_SUBVOL_ROOTREF which returns ROOT_REF information of the subvolume containing this inode. The min id of root ref's subvolume to be searched is specified by min_id in struct btrfs_ioctl_get_subvol_rootref_args. If there are more root refs than

[PATCH v3 0/3] btrfs: Add three new unprivileged ioctls to allow normal users to call "sub list/show" etc.

changelog: v2-> v3 - fix kbuild test bot warning v1 -> v2 - completely reimplement 1st/2nd ioctl to have user friendly api - various cleanup, remove unnecessary goto === This adds three new unprivileged ioctls: 1st patch: ioctl which returns subvolume information of ROOT_ITEM and

Re: High CPU usage of RT threads...

On 19.03.2018 09:13, Shyam Prasad N wrote: > Hi Nikolay, > > Thanks for your reply on this. > > Checked the stack trace for many of the stuck threads. Looks like all > of them are stuck in this loop... > [] exit_to_usermode_loop+0x72/0xd0 > [] prepare_exit_to_usermode+0x26/0x30 > []

Re: High CPU usage of RT threads...

Hi Nikolay, Thanks for your reply on this. Checked the stack trace for many of the stuck threads. Looks like all of them are stuck in this loop... [] exit_to_usermode_loop+0x72/0xd0 [] prepare_exit_to_usermode+0x26/0x30 [] retint_user+0x8/0x10 [] 0x Seems like it is stuck in a

Re: [RFC PATCH v2 6/8] btrfs-progs: sub list: Allow normal user to call "subvolume list/show"

On 2018/03/17 22:23, Goffredo Baroncelli wrote: > On 03/15/2018 09:15 AM, Misono, Tomohiro wrote: >> Allow normal user to call "subvolume list/show" by using 3 new >> unprivileged ioctls (BTRFS_IOC_GET_SUBVOL_INFO, >> BTRFS_IOC_GET_SUBVOL_ROOTREF and BTRFS_IOC_INO_LOOKUP_USER). >> >> Note that for

Re: [RFC PATCH v2 3/8] btrfs-progs: sub list: Add helper function which checks the permission for tree search ioctl

On 2018/03/17 22:23, Goffredo Baroncelli wrote: > On 03/15/2018 09:13 AM, Misono, Tomohiro wrote: >> This is a preparetion work to allow normal user to call >> "subvolume list/show". >> >> Signed-off-by: Tomohiro Misono >> --- >> btrfs-list.c | 33