Re: [PATCH V2] test online label ioctl

2018-05-14 Thread Dave Chinner
On Mon, May 14, 2018 at 06:26:07PM -0500, Eric Sandeen wrote: > On 5/14/18 6:11 PM, Dave Chinner wrote: > > On Mon, May 14, 2018 at 12:09:16PM -0500, Eric Sandeen wrote: > >> This tests the online label ioctl that btrfs has, which has been > >> recently proposed for XFS. > >> > >> To run, it

Re: [PATCH] btrfs: inode: Don't compress if NODATASUM or NODATACOW set

2018-05-14 Thread Qu Wenruo
On 2018年05月15日 00:52, David Sterba wrote: > On Mon, May 14, 2018 at 03:02:10PM +0800, Qu Wenruo wrote: >> As btrfs(5) specified: >> >> Note >> If nodatacow or nodatasum are enabled, compression is disabled. >> >> If NODATASUM or NODATACOW set, we should not compress the extent. >> >>

Re: verify key failure

2018-05-14 Thread Qu Wenruo
On 2018年05月14日 22:35, Liu Bo wrote: > Hi, > > I got another warning of verify_level_key by running btrfs/124 in a loop, I'm > testing against 4.17-rc3. > > Not sure if it's false positive. > > [101414.336691] WARNING: CPU: 3 PID: 30194 at fs/btrfs/disk-io.c:455 >

[PATCH v2 1/3] btrfs-progs: check: check symlinks with append/immutable flags

2018-05-14 Thread Su Yue
Define new macro I_ERR_ODD_INODE_FLAGS to represents odd inode flags. Symlinks should never have append/immutable flags. While processing inodes, if found a symlink with append/immutable flags, mark the inode record with I_ERR_ODD_INODE_FLAGS. This is for original mode. Signed-off-by: Su Yue

[PATCH v2 0/3] btrfs-progs: check: verify symlinks with append/immutable flags

2018-05-14 Thread Su Yue
This patchset can be fetch from my github: https://github.com/Damenly/btrfs-progs/commits/odd_inode_flags It's based on devel. symlinks should never have append/immutable attributes. This patchset enables btrfs check to verify such corruption. PATCH[1] is for original mode. PATCH[2] is for

[PATCH v2 2/3] btrfs-progs: lowmem: check symlinks with append/immutable flags

2018-05-14 Thread Su Yue
Define new error bit INODE_FLAGS_ERROR to represents invalid inode flags error. Symlinks should never have append/immutable flags set. While checking inodes, if found a symlink with append/immutable flags, report and record the inode flags error. This is for lowmem mode. Signed-off-by: Su Yue

[PATCH v2 3/3] btrfs-progs: fsck-tests: add test case to check symlinks with bad flags

2018-05-14 Thread Su Yue
There are two bad symlinks in the test case. One is with immutable attribute. Another one is with append attribute. Signed-off-by: Su Yue --- .../034-bad-inode-flags/default_case.img | Bin 0 -> 4096 bytes tests/fsck-tests/034-bad-inode-flags/test.sh | 15

Re: [PATCH] btrfs: inode: Don't compress if NODATASUM or NODATACOW set

2018-05-14 Thread james harvey
Don't know if this will help. I just learned about pstore, and see in there a dmesg that's interesting. The serial port kernel errors started this time with "BUG: unable to handle kernel paging request". The pstore dmesg has everything from there until the end of the first trace. But, the

4.15.6 crash: BUG at fs/btrfs/ctree.c:1862

2018-05-14 Thread Marc MERLIN
static noinline struct extent_buffer * read_node_slot(struct btrfs_fs_info *fs_info, struct extent_buffer *parent, int slot) { int level = btrfs_header_level(parent); struct extent_buffer *eb; if (slot < 0 || slot >= btrfs_header_nritems(parent))

Re: [PATCH V2] test online label ioctl

2018-05-14 Thread Eric Sandeen
On 5/14/18 6:11 PM, Dave Chinner wrote: > On Mon, May 14, 2018 at 12:09:16PM -0500, Eric Sandeen wrote: >> This tests the online label ioctl that btrfs has, which has been >> recently proposed for XFS. >> >> To run, it requires an updated xfs_io with the label command and a >> filesystem that

Re: [PATCH V2] test online label ioctl

2018-05-14 Thread Dave Chinner
On Mon, May 14, 2018 at 12:09:16PM -0500, Eric Sandeen wrote: > This tests the online label ioctl that btrfs has, which has been > recently proposed for XFS. > > To run, it requires an updated xfs_io with the label command and a > filesystem that supports it > > A slight change here to

Re: [PATCH] btrfs: inode: Don't compress if NODATASUM or NODATACOW set

2018-05-14 Thread james harvey
On Mon, May 14, 2018 at 12:52 PM, David Sterba wrote: > On Mon, May 14, 2018 at 03:02:10PM +0800, Qu Wenruo wrote: >> As btrfs(5) specified: >> >> Note >> If nodatacow or nodatasum are enabled, compression is disabled. >> >> If NODATASUM or NODATACOW set, we should

Re: [PATCH] btrfs: inode: Don't compress if NODATASUM or NODATACOW set

2018-05-14 Thread james harvey
On Mon, May 14, 2018 at 6:35 AM, Qu Wenruo wrote: > And if possible, please don't just remove those offending files (yet). > Your binary dump would help a lot locating the root case. Absolutely. This is on a 50G LVM root volume, so I've been able to leave the original

Re: [PATCH] btrfs: inode: Don't compress if NODATASUM or NODATACOW set

2018-05-14 Thread Timofey Titovets
пн, 14 мая 2018 г. в 20:32, David Sterba : > On Mon, May 14, 2018 at 03:02:10PM +0800, Qu Wenruo wrote: > > As btrfs(5) specified: > > > > Note > > If nodatacow or nodatasum are enabled, compression is disabled. > > > > If NODATASUM or NODATACOW set, we should not

Re: Clarification needed about libbtrfs & libbtrfsutil

2018-05-14 Thread Omar Sandoval
On Mon, May 14, 2018 at 09:40:19AM +0100, Dimitri John Ledkov wrote: > Are both of these meant to be public libraries, installed on the user > systems, and available in .so variant as well for 3rd party > development and public dynamic linking? > > Or are these private internal libraries, which

[PATCH V2] test online label ioctl

2018-05-14 Thread Eric Sandeen
This tests the online label ioctl that btrfs has, which has been recently proposed for XFS. To run, it requires an updated xfs_io with the label command and a filesystem that supports it A slight change here to _require_xfs_io_command as well, so that tests which simply fail with "Inappropriate

Re: [PATCH] test online label ioctl

2018-05-14 Thread Eric Sandeen
This tests the online label ioctl that btrfs has, which has been recently proposed for XFS. To run, it requires an updated xfs_io with the label command and a filesystem that supports it A slight change here to _require_xfs_io_command as well, so that tests which simply fail with "Inappropriate

Re: [PATCH] btrfs: inode: Don't compress if NODATASUM or NODATACOW set

2018-05-14 Thread David Sterba
On Mon, May 14, 2018 at 03:02:10PM +0800, Qu Wenruo wrote: > As btrfs(5) specified: > > Note > If nodatacow or nodatasum are enabled, compression is disabled. > > If NODATASUM or NODATACOW set, we should not compress the extent. > > And in fact, we have bug report about corrupted

Re: [PATCH] btrfs: inode: Don't compress if NODATASUM or NODATACOW set

2018-05-14 Thread David Sterba
On Mon, May 14, 2018 at 12:46:31PM +0300, Nikolay Borisov wrote: > [Adding David to CC] > > On 14.05.2018 12:39, Roman Mamedov wrote: > > On Mon, 14 May 2018 11:36:26 +0300 > > Nikolay Borisov wrote: > > > >> So what made you have these expectation, is it codified somewhere >

Re: [PATCH 1/3] fs: add initial bh_result->b_private value to __blockdev_direct_IO()

2018-05-14 Thread David Sterba
On Fri, May 11, 2018 at 01:30:01PM -0700, Omar Sandoval wrote: > On Fri, May 11, 2018 at 09:05:38PM +0100, Al Viro wrote: > > On Thu, May 10, 2018 at 11:30:10PM -0700, Omar Sandoval wrote: > > > do_blockdev_direct_IO(struct kiocb *iocb, struct inode *inode, > > > struct

Re: verify key failure

2018-05-14 Thread Chris Mason
On 14 May 2018, at 10:35, Liu Bo wrote: Hi, I got another warning of verify_level_key by running btrfs/124 in a loop, I'm testing against 4.17-rc3. Not sure if it's false positive. How long does this take to trigger? -chris -- To unsubscribe from this list: send the line "unsubscribe

Re: [PATCH v3 3/3] btrfs: replace waitqueue_actvie with cond_wake_up

2018-05-14 Thread Nikolay Borisov
On 14.05.2018 15:23, David Sterba wrote: > Use the wrappers and reduce the amount of low-level details about the > waitqueue management. > > Signed-off-by: David Sterba Reviewed-by: Nikolay Borisov > --- > fs/btrfs/compression.c | 7 +-- >

Re: [PATCH v4 11/12] Btrfs: renumber BTRFS_INODE_ runtime flags

2018-05-14 Thread David Sterba
On Fri, May 11, 2018 at 01:13:39PM -0700, Omar Sandoval wrote: > From: Omar Sandoval > > We got rid of BTRFS_INODE_HAS_ORPHAN_ITEM and > BTRFS_INODE_ORPHAN_META_RESERVED, so we can renumber the flags to make > them consecutive again. > > Signed-off-by: Omar Sandoval

Re: [PATCH 2/2] vfs: dedupe should return EPERM if permission is not granted

2018-05-14 Thread David Sterba
On Fri, May 11, 2018 at 12:26:51PM -0700, Mark Fasheh wrote: > Right now we return EINVAL if a process does not have permission to dedupe a > file. This was an oversight on my part. EPERM gives a true description of > the nature of our error, and EINVAL is already used for the case that the >

Re: [PATCH] Btrfs: fix duplicate extents after fsync of file with prealloc extents

2018-05-14 Thread David Sterba
On Wed, May 09, 2018 at 04:01:46PM +0100, fdman...@kernel.org wrote: > From: Filipe Manana ... > Fixes: 471d557afed1 ("Btrfs: fix loss of prealloc extents past i_size after > fsync log replay") > Signed-off-by: Filipe Manana Thanks for the excellent and

Re: [PATCH v3 2/3] btrfs: add barriers to btrfs_sync_log before log_commit_wait wakeups

2018-05-14 Thread Nikolay Borisov
On 14.05.2018 15:23, David Sterba wrote: > Currently the code assumes that there's an implied barrier by the > sequence of code preceding the wakeup, namely the mutex unlock. > > As Nikolay pointed out: > > I think this is wrong (not your code) but the original assumption that > the RELEASE

verify key failure

2018-05-14 Thread Liu Bo
Hi, I got another warning of verify_level_key by running btrfs/124 in a loop, I'm testing against 4.17-rc3. Not sure if it's false positive. [101414.336691] WARNING: CPU: 3 PID: 30194 at fs/btrfs/disk-io.c:455 btree_read_extent_buffer_pages+0x183/0x220 [btrfs] [101414.340372] Modules linked

Re: [PATCH] Btrfs: fix xattr loss after power failure

2018-05-14 Thread David Sterba
On Fri, May 11, 2018 at 04:42:42PM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > If a file has xattrs, we fsync it, to ensure we clear the flags > BTRFS_INODE_NEEDS_FULL_SYNC and BTRFS_INODE_COPY_EVERYTHING from its > inode, the current transaction commits and

Re: [PATCH v3] btrfs: incremental send, fix BUG when invalid memory access

2018-05-14 Thread David Sterba
On Mon, May 14, 2018 at 12:39:36PM +0100, Filipe Manana wrote: > On Mon, May 14, 2018 at 3:51 AM, robbieko wrote: > > Signed-off-by: Robbie Ko > Reviewed-by: Filipe Manana > > Looks good, I would only change the subject to

Re: [PATCH v2 0/2] btrfs: qgroup rescan fixes part 1

2018-05-14 Thread David Sterba
On Mon, May 14, 2018 at 09:38:11AM +0800, Qu Wenruo wrote: > This patchset is mainly focused on fixing qgroup rescan corruption. > > Since the whole btrfs qgroup is based on the modification between 2 > transactions, it only has correct qgroup delta. > While if the rescan can't provide a correct

Hang/deadlock in 4.14.40 running rsync

2018-05-14 Thread E V
Running an rsync to copy data onto a btrfs filesystem used for backups. It's appearing to deadlock/hang. The rsync process stops doing any IO, and no other IO is ongoing against the filesystem. The rsync is in state D in ps and is unkillable even with kill -9. The /proc//stack for the rsync: []

[PATCH v3 3/3] btrfs: replace waitqueue_actvie with cond_wake_up

2018-05-14 Thread David Sterba
Use the wrappers and reduce the amount of low-level details about the waitqueue management. Signed-off-by: David Sterba --- fs/btrfs/compression.c | 7 +-- fs/btrfs/delayed-inode.c | 9 +++-- fs/btrfs/dev-replace.c | 10 -- fs/btrfs/extent-tree.c | 7

[PATCH v3 2/3] btrfs: add barriers to btrfs_sync_log before log_commit_wait wakeups

2018-05-14 Thread David Sterba
Currently the code assumes that there's an implied barrier by the sequence of code preceding the wakeup, namely the mutex unlock. As Nikolay pointed out: I think this is wrong (not your code) but the original assumption that the RELEASE semantics provided by mutex_unlock is sufficient. According

[PATCH v3 0/3] Cleanup waitqueue_active and barriers

2018-05-14 Thread David Sterba
Reduce number of standalone barriers before waitqueue_active calls. Changes v3: * fix wrong use of the _nomb variant in tree-log.c:btrfs_sync_log, * update comments to be more specific about the waitqueue_active and barrier Changes v2: * add 2 barriers to btrfs_sync_log and do not assume

[PATCH v3 1/3] btrfs: introduce conditional wakeup helpers

2018-05-14 Thread David Sterba
Add convenience wrappers for the waitqueue management that involves memory barriers to prevent deadlocks. The helpers will let us remove barriers and the necessary comments in several places. Reviewed-by: Nikolay Borisov Signed-off-by: David Sterba ---

Re: [PATCH 1/3] btrfs-progs: check: check symlinks with append/immutable flags

2018-05-14 Thread Su Yue
On 2018/5/14 7:18 PM, Nikolay Borisov wrote: > > > On 14.05.2018 13:29, Su Yue wrote: >> Define new macro I_ERR_ODD_INODE_FLAGS to represents odd inode flags. >> >> Symlinks should never have append/immutable flags. >> While processing inodes, if found a symlink with append/immutable >> flags,

Re: [PATCH 2/3] btrfs-progs: lowmem: check symlinks with append/immutable flags

2018-05-14 Thread Su Yue
Indeed better. Will do it in V2. Thanks, Su On Mon, May 14, 2018 at 7:19 PM Nikolay Borisov wrote: > On 14.05.2018 13:29, Su Yue wrote: > > Symlinks should never have append/immutable flags. > > While checking inodes, if found a symlink with append/immutable > > flags,

Re: [PATCH 1/3] btrfs-progs: check: check symlinks with append/immutable flags

2018-05-14 Thread Su Yue
On 2018/5/14 7:22 PM, Qu Wenruo wrote: > > > On 2018年05月14日 18:29, Su Yue wrote: >> Define new macro I_ERR_ODD_INODE_FLAGS to represents odd inode flags. >> >> Symlinks should never have append/immutable flags. >> While processing inodes, if found a symlink with append/immutable >> flags, mark

Re: Btrfs installation advices

2018-05-14 Thread Austin S. Hemmelgarn
On 2018-05-12 21:58, faurepi...@gmail.com wrote: Thanks you two very much for your answers. So if I sum up correctly, I could: 1- use Self-Encrypting Drive (SED), since my drive is a Samsung NVMe 960 EVO, which is supposed to support SED according to

Re: [PATCH v3] btrfs: incremental send, fix BUG when invalid memory access

2018-05-14 Thread Filipe Manana
On Mon, May 14, 2018 at 3:51 AM, robbieko wrote: > From: Robbie Ko > > [BUG] > btrfs incremental send BUG happens when creating a snapshot of snapshot > that is being used by send. > > [REASON] > The problem can happen if while we are doing a send

Re: [PATCH 2/3] btrfs-progs: lowmem: check symlinks with append/immutable flags

2018-05-14 Thread Qu Wenruo
On 2018年05月14日 18:29, Su Yue wrote: > Symlinks should never have append/immutable flags. > While checking inodes, if found a symlink with append/immutable > flags, report and record the fatal error. > > This is for lowmem mode. > > Signed-off-by: Su Yue > --- >

Re: [PATCH 1/3] btrfs-progs: check: check symlinks with append/immutable flags

2018-05-14 Thread Qu Wenruo
On 2018年05月14日 18:29, Su Yue wrote: > Define new macro I_ERR_ODD_INODE_FLAGS to represents odd inode flags. > > Symlinks should never have append/immutable flags. > While processing inodes, if found a symlink with append/immutable > flags, mark the inode record with I_ERR_ODD_INODE_FLAGS. > >

Re: [PATCH 1/3] btrfs-progs: check: check symlinks with append/immutable flags

2018-05-14 Thread Nikolay Borisov
On 14.05.2018 13:29, Su Yue wrote: > Define new macro I_ERR_ODD_INODE_FLAGS to represents odd inode flags. > > Symlinks should never have append/immutable flags. > While processing inodes, if found a symlink with append/immutable > flags, mark the inode record with I_ERR_ODD_INODE_FLAGS. > >

Re: [PATCH 2/3] btrfs-progs: lowmem: check symlinks with append/immutable flags

2018-05-14 Thread Nikolay Borisov
On 14.05.2018 13:29, Su Yue wrote: > Symlinks should never have append/immutable flags. > While checking inodes, if found a symlink with append/immutable > flags, report and record the fatal error. > > This is for lowmem mode. > > Signed-off-by: Su Yue > --- >

[PATCH 4/9] btrfs-progs: btrfs-corrupt-block: Change -I flag parameter format

2018-05-14 Thread Nikolay Borisov
Presently, if we want to corrupt a particular item we need to call corrupt block like so: btrfs-corrupt-block -I -K -r "numeric rootid" This is problematic because the -K option not only sets the key to the item that the -I option should corrupt but it also signals that we

[PATCH 3/9] btrfs-progs: btrfs-corrupt-block: Factor out key parsing function

2018-05-14 Thread Nikolay Borisov
Currently passing a key with -K handling is open coded. I intend on changing the interface a bit to make the program more usable. To aid in this factor out common code which parses a triplet of the "u64,u8,u64" format, corresponding to a btrfs key. No functional changes. Signed-off-by: Nikolay

[PATCH 5/9] btrfs-progs: btrfs-corrupt-block: Convert -K flag argument handling to common function

2018-05-14 Thread Nikolay Borisov
There is now a common function used to parse btrfs keys triplets so use that one. No functional changes. Signed-off-by: Nikolay Borisov --- btrfs-corrupt-block.c | 16 +--- 1 file changed, 5 insertions(+), 11 deletions(-) diff --git a/btrfs-corrupt-block.c

[PATCH 0/9] Overhaul btrfs-corrupt-block

2018-05-14 Thread Nikolay Borisov
btrfs-corrupt-block is a very useful tool albeit very neglected. This series aims to give it much needed attention. There is a mix of code-improvements and bug fixes. Code improvement mainly consists of factoring our duplicated code (Patch 1,3,6) and improving the interface of some options

[PATCH 6/9] btrfs-progs: btrfs-corrupt-block: Factor out common "-r" handling code

2018-05-14 Thread Nikolay Borisov
Since more and more of the "corrupt XXX" options are going to support combination with -r option, let's extract the common code needed for this. No functional changes. Signed-off-by: Nikolay Borisov --- btrfs-corrupt-block.c | 17 +++-- 1 file changed, 7

[PATCH 8/9] btrfs-progs: btrfs-corrupt-block: Change format of -d option

2018-05-14 Thread Nikolay Borisov
Currently if we want to delete an item we need to invoke corrupt block like so: btrfs-corrupt-block [-r ] -K -d Instead, this patch converts the format to: btrfs-corrupt-block [-r ] -d Signed-off-by: Nikolay Borisov ---

[PATCH 2/9] btrfs-progs: btrfs-corrupt-block: Correctly handle -r when passing -I

2018-05-14 Thread Nikolay Borisov
The documentation for the -I option (corrupt an item) states: An item to corrupt (must also specify the field to corrupt and a root+key for the item) The code on the other hand doesn't check whether -r is in fact passed, and even if it is it's not handled at all. This means presently -I is

[PATCH 9/9] btrfs-progs: btrfs-corrupt-block: Fix -D option

2018-05-14 Thread Nikolay Borisov
Currently the -D option is essentially defunct since it's the root, where we are going to corrupt a dir item is always set to the tree root. Fix this by passing the root from the "-r" option. Aditionally convert the interface for this option to the new format. So if one wants to corrupt a dir item

[PATCH 1/9] btrfs-progs: btrfs-corrupt-block: Factor out specific-root code

2018-05-14 Thread Nikolay Borisov
Some options operate on a specific root so let's extract the code which deals with this. No functional change. Signed-off-by: Nikolay Borisov --- btrfs-corrupt-block.c | 37 +++-- 1 file changed, 23 insertions(+), 14 deletions(-) diff --git

Re: "decompress failed" in 1-2 files always causes kernel oops, check/scrub pass

2018-05-14 Thread Qu Wenruo
On 2018年05月14日 18:29, james harvey wrote: > On Mon, May 14, 2018 at 2:36 AM, Qu Wenruo wrote: >> OK, I could reproduce it now. >> >> Just mount with -o nodatasum, then create a file. >> Remount with compress-force=lzo, then write something. >> >> So at least btrfs should

Re: [RFC][PATCH] btrfs: take the last remnants of ->d_fsdata use out

2018-05-14 Thread David Sterba
On Sun, May 13, 2018 at 07:03:18PM +0100, Al Viro wrote: > [spotted while going through ->d_fsdata handling around d_splice_alias(); > don't really care which tree that goes through] > > The only thing even looking at ->d_fsdata in there (since 2012) > had been kfree(dentry->d_fsdata) in

Re: [PATCH] btrfs: inode: Don't compress if NODATASUM or NODATACOW set

2018-05-14 Thread Qu Wenruo
On 2018年05月14日 17:30, james harvey wrote: > On Mon, May 14, 2018 at 4:20 AM, Roman Mamedov wrote: >> On Mon, 14 May 2018 11:10:34 +0300 >> Nikolay Borisov wrote: >> >>> But if we have mounted the fs with FORCE_COMPRESS shouldn't we disregard >>> the inode

Re: "decompress failed" in 1-2 files always causes kernel oops, check/scrub pass

2018-05-14 Thread james harvey
On Mon, May 14, 2018 at 2:36 AM, Qu Wenruo wrote: > OK, I could reproduce it now. > > Just mount with -o nodatasum, then create a file. > Remount with compress-force=lzo, then write something. > > So at least btrfs should disallow such thing. > > Thanks, > Qu Would the

[PATCH 3/3] btrfs-progs: fsck-tests: add test case to check symlinks with odd flags

2018-05-14 Thread Su Yue
There are two bad symlinks in the test case. One is with immutable attribute. Another one is with append attribute. Signed-off-by: Su Yue --- .../034-odd-inode-flags/default_case.img | Bin 0 -> 4096 bytes tests/fsck-tests/034-odd-inode-flags/test.sh | 15

[PATCH 1/3] btrfs-progs: check: check symlinks with append/immutable flags

2018-05-14 Thread Su Yue
Define new macro I_ERR_ODD_INODE_FLAGS to represents odd inode flags. Symlinks should never have append/immutable flags. While processing inodes, if found a symlink with append/immutable flags, mark the inode record with I_ERR_ODD_INODE_FLAGS. This is for original mode. Signed-off-by: Su Yue

[PATCH 0/3] btrfs-progs: check: verify symlinks with append/immutable flags

2018-05-14 Thread Su Yue
This patchset can be fetch from my github: https://github.com/Damenly/btrfs-progs/commits/odd_inode_flags symlinks should never have append/immutable attributes. This patchset enables btrfs check to verify such corruption. PATCH[1] is for original mode. PATCH[2] is for original mode. PATCH[3]

[PATCH 2/3] btrfs-progs: lowmem: check symlinks with append/immutable flags

2018-05-14 Thread Su Yue
Symlinks should never have append/immutable flags. While checking inodes, if found a symlink with append/immutable flags, report and record the fatal error. This is for lowmem mode. Signed-off-by: Su Yue --- check/mode-lowmem.c | 10 ++ 1 file changed, 10

Re: [PATCH] btrfs: inode: Don't compress if NODATASUM or NODATACOW set

2018-05-14 Thread Nikolay Borisov
[Adding David to CC] On 14.05.2018 12:39, Roman Mamedov wrote: > On Mon, 14 May 2018 11:36:26 +0300 > Nikolay Borisov wrote: > >> So what made you have these expectation, is it codified somewhere >> (docs/man pages etc)? I'm fine with that semantics IF this is what >> people

Re: [PATCH] btrfs: inode: Don't compress if NODATASUM or NODATACOW set

2018-05-14 Thread Roman Mamedov
On Mon, 14 May 2018 11:36:26 +0300 Nikolay Borisov wrote: > So what made you have these expectation, is it codified somewhere > (docs/man pages etc)? I'm fine with that semantics IF this is what > people expect. "Compression ...does not work for NOCOW files":

Re: [PATCH] btrfs: inode: Don't compress if NODATASUM or NODATACOW set

2018-05-14 Thread james harvey
On Mon, May 14, 2018 at 4:36 AM, Nikolay Borisov wrote: > On 14.05.2018 11:20, Roman Mamedov wrote: >> On Mon, 14 May 2018 11:10:34 +0300 >> Nikolay Borisov wrote: >> >>> But if we have mounted the fs with FORCE_COMPRESS shouldn't we disregard >>> the inode

Re: [PATCH] btrfs: inode: Don't compress if NODATASUM or NODATACOW set

2018-05-14 Thread james harvey
On Mon, May 14, 2018 at 4:20 AM, Roman Mamedov wrote: > On Mon, 14 May 2018 11:10:34 +0300 > Nikolay Borisov wrote: > >> But if we have mounted the fs with FORCE_COMPRESS shouldn't we disregard >> the inode flags, presumably the admin knows what he is doing?

4.13 call trace in btrfs_update_device

2018-05-14 Thread Lukas Tribus
Hello, seeing calltraces like this on a Ubuntu 4.13 kernel, not sure what happens here and what the impact is: [13729.871604] [ cut here ] [13729.871647] WARNING: CPU: 1 PID: 3048 at /build/linux-hwe-rDkE7z/linux-hwe-4.13.0/fs/btrfs/ctree.h:1559

Clarification needed about libbtrfs & libbtrfsutil

2018-05-14 Thread Dimitri John Ledkov
Are both of these meant to be public libraries, installed on the user systems, and available in .so variant as well for 3rd party development and public dynamic linking? Or are these private internal libraries, which are installed as public runtime only, simply to share code between the utils,

Re: [PATCH] btrfs: inode: Don't compress if NODATASUM or NODATACOW set

2018-05-14 Thread Nikolay Borisov
On 14.05.2018 11:20, Roman Mamedov wrote: > On Mon, 14 May 2018 11:10:34 +0300 > Nikolay Borisov wrote: > >> But if we have mounted the fs with FORCE_COMPRESS shouldn't we disregard >> the inode flags, presumably the admin knows what he is doing? > > Please don't.

Re: [PATCH] btrfs: inode: Don't compress if NODATASUM or NODATACOW set

2018-05-14 Thread Roman Mamedov
On Mon, 14 May 2018 11:10:34 +0300 Nikolay Borisov wrote: > But if we have mounted the fs with FORCE_COMPRESS shouldn't we disregard > the inode flags, presumably the admin knows what he is doing? Please don't. Personally I always assumed chattr +C would prevent both CoW and

Re: [PATCH] btrfs: inode: Don't compress if NODATASUM or NODATACOW set

2018-05-14 Thread Qu Wenruo
On 2018年05月14日 16:10, Nikolay Borisov wrote: > > > On 14.05.2018 10:02, Qu Wenruo wrote: >> As btrfs(5) specified: >> >> Note >> If nodatacow or nodatasum are enabled, compression is disabled. >> >> If NODATASUM or NODATACOW set, we should not compress the extent. >> >> And in fact,

Re: [PATCH] btrfs: inode: Don't compress if NODATASUM or NODATACOW set

2018-05-14 Thread Nikolay Borisov
On 14.05.2018 10:02, Qu Wenruo wrote: > As btrfs(5) specified: > > Note > If nodatacow or nodatasum are enabled, compression is disabled. > > If NODATASUM or NODATACOW set, we should not compress the extent. > > And in fact, we have bug report about corrupted compressed extent >

[PATCH 3/3] btrfs-progs: fsck-tests: Add test case for detecting compressed extent without csum

2018-05-14 Thread Qu Wenruo
Signed-off-by: Qu Wenruo --- .../compressed_extent_without_csum.raw.xz | Bin 0 -> 21996 bytes .../032-compressed-nodatasum/test.sh | 24 ++ 2 files changed, 24 insertions(+) create mode 100644

[PATCH 0/3] btrfs-progs: Detect compressed extent without csum

2018-05-14 Thread Qu Wenruo
Patches can be fetch from github: https://github.com/adam900710/btrfs-progs/tree/compress_nodatasum It's based on v4.16 stable branch. James Harvey from mail list reports a strange kernel panic, whichs show obviously kernel memory corruption, while after btrfs decompression failure. It turns

Re: [PATCH 1/3] btrfs-progs: check/lowmem: Add checks for compressed extent without csum

2018-05-14 Thread Su Yue
On 05/14/2018 03:03 PM, Qu Wenruo wrote: > There is one report of compressed extent happens in btrfs, but has no > csum and then leads to possible decompress error screwing up kernel > memory. > > Although it's a kernel bug, and won't cause problem until compressed > data get corrupted, let's

Re: [PATCH 2/3] btrfs-progs: check/original: Add checks for compressed extent without csum

2018-05-14 Thread Su Yue
On 05/14/2018 03:03 PM, Qu Wenruo wrote: > There is one report of compressed extent happens in btrfs, but has no > csum and then leads to possible decompress error screwing up kernel > memory. > > Although it's a kernel bug, and won't cause problem until compressed > data get corrupted, let's

[PATCH 1/3] btrfs-progs: check/lowmem: Add checks for compressed extent without csum

2018-05-14 Thread Qu Wenruo
There is one report of compressed extent happens in btrfs, but has no csum and then leads to possible decompress error screwing up kernel memory. Although it's a kernel bug, and won't cause problem until compressed data get corrupted, let's catch such problem in advance. This patch will catch

[PATCH 3/3] btrfs-progs: fsck-tests: Add test case for detecting compressed extent without csum

2018-05-14 Thread Qu Wenruo
Signed-off-by: Qu Wenruo --- .../compressed_extent_without_csum.raw.xz | Bin 0 -> 21996 bytes .../032-compressed-nodatasum/test.sh | 24 ++ 2 files changed, 24 insertions(+) create mode 100644

[PATCH 0/3] btrfs-progs: Detect compressed extent without csum

2018-05-14 Thread Qu Wenruo
Patches can be fetch from github: https://github.com/adam900710/btrfs-progs/tree/compress_nodatasum It's based on v4.16 stable branch. James Harvey from mail list reports a strange kernel panic, whichs show obviously kernel memory corruption, while after btrfs decompression failure. It turns

[PATCH 2/3] btrfs-progs: check/original: Add checks for compressed extent without csum

2018-05-14 Thread Qu Wenruo
There is one report of compressed extent happens in btrfs, but has no csum and then leads to possible decompress error screwing up kernel memory. Although it's a kernel bug, and won't cause problem until compressed data get corrupted, let's catch such problem in advance. This patch will catch

[PATCH] btrfs: inode: Don't compress if NODATASUM or NODATACOW set

2018-05-14 Thread Qu Wenruo
As btrfs(5) specified: Note If nodatacow or nodatasum are enabled, compression is disabled. If NODATASUM or NODATACOW set, we should not compress the extent. And in fact, we have bug report about corrupted compressed extent leading to memory corruption in mail list. Although

[PATCH 2/3] btrfs-progs: check/original: Add checks for compressed extent without csum

2018-05-14 Thread Qu Wenruo
There is one report of compressed extent happens in btrfs, but has no csum and then leads to possible decompress error screwing up kernel memory. Although it's a kernel bug, and won't cause problem until compressed data get corrupted, let's catch such problem in advance. This patch will catch

[PATCH 1/3] btrfs-progs: check/lowmem: Add checks for compressed extent without csum

2018-05-14 Thread Qu Wenruo
There is one report of compressed extent happens in btrfs, but has no csum and then leads to possible decompress error screwing up kernel memory. Although it's a kernel bug, and won't cause problem until compressed data get corrupted, let's catch such problem in advance. This patch will catch

Re: "decompress failed" in 1-2 files always causes kernel oops, check/scrub pass

2018-05-14 Thread Qu Wenruo
On 2018年05月14日 13:30, Qu Wenruo wrote: > > > On 2018年05月14日 12:41, james harvey wrote: >> On Sun, May 13, 2018 at 10:08 PM, Qu Wenruo wrote: >>> On 2018年05月12日 13:08, james harvey wrote: Hardware is fine. Passes memtest86+ in SMP mode. Works fine on all