Re: [PATCH v2 rev log added] fstests: btrfs verify hardening agaist duplicate fsid

2018-10-26 Thread Nikolay Borisov
t;> +. ./common/filter >>> + >>> +# remove previous $seqres.full before test >>> +rm -f $seqres.full >>> + >>> +# real QA test starts here >>> + >>> +# Modify as appropriate. >>> +_supported_fs btrfs >>> +_supp

Re: [PATCH v2 rev log added] fstests: btrfs verify hardening agaist duplicate fsid

2018-10-26 Thread Anand Jain
variable or use one of the SCRATCH_MNT/TEST_MNT global variables. Also I checked with Eric re. the use of $mnt in tests and his conclusion is : " looks like a bug" No its not. As few lines above, I have assigned it as.. mnt=$TEST_DIR/$seq.mnt +mkdir $mnt +_mkfs_dev $dev_foo +_mount

Re: [PATCH v2 rev log added] fstests: btrfs verify hardening agaist duplicate fsid

2018-10-26 Thread Nikolay Borisov
t the test. Either use sane names - primary_dev/secondary dev or device_1 and device_2. > + > +echo dev_foo=$dev_foo >> $seqres.full > +echo dev_bar=$dev_bar >> $seqres.full > +echo | tee -a $seqres.full > + > +rm -rf $mnt > /dev/null 2>&1 So what is $mnt? I ca

Re: Kernel crash related to LZO compression

2018-10-26 Thread Dmitry Katsubo
On 2018-10-25 20:49, Chris Murphy wrote: I would say the first step no matter what if you're using an older kernel, is to boot a current Fedora or Arch live or install media, mount the Btrfs and try to read the problem files and see if the problem still happens. I can't even being to estimate

Re: [PATCH] btrfs-progs: add cli to forget one or all scanned devices

2018-10-26 Thread Nikolay Borisov
On 26.10.18 г. 17:27 ч., Anand Jain wrote: > This patch adds cli > btrfs device forget [dev] > to remove the given device structure in the kernel if the device > is unmounted. If no argument is given it shall remove all stale > (device which are not mounted) from the kernel. > >

Re: [PATCH] btrfs: introduce feature to forget a btrfs device

2018-10-26 Thread Nikolay Borisov
On 26.10.18 г. 17:27 ч., Anand Jain wrote: > Support for a new command 'btrfs dev forget [dev]' is proposed here > to undo the effects of 'btrfs dev scan [dev]'. For this purpose > this patch proposes to use ioctl #5 as it was empty. > IOW(BTRFS_IOCTL_MAGIC, 5, ..) > This patch adds new

Re: [PATCH] btrfs-progs: add cli to forget one or all scanned devices

2018-10-26 Thread Anand Jain
On 10/26/2018 08:21 PM, Nikolay Borisov wrote: On 24.10.2018 07:31, Anand Jain wrote: This patch adds cli btrfs device forget [dev] to remove the given device structure in the kernel if the device is unmounted. If no argument is given it shall remove all stale (device which are not

Re: [PATCH] btrfs: Fix error handling in btrfs_cleanup_ordered_extents

2018-10-26 Thread kbuild test robot
Hi Nikolay, Thank you for the patch! Yet something to improve: [auto build test ERROR on v4.19] [also build test ERROR on next-20181019] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url:

Re: [PATCH 5/5] btrfs: Replace BUG_ON with ASSERT in find_lock_delalloc_range

2018-10-26 Thread Qu Wenruo
On 2018/10/26 下午7:43, Nikolay Borisov wrote: > lock_delalloc_pages should only return 2 values - 0 in case of success > and -EAGAIN if the range of pages to be locked should be shrunk due to > some of gone. Manual inspections confirms that this is > indeed the case since __process_pages_contig

Re: [PATCH 4/5] btrfs: Sink find_lock_delalloc_range's 'max_bytes' argument

2018-10-26 Thread Qu Wenruo
On 2018/10/26 下午7:43, Nikolay Borisov wrote: > All callers of this function pass BTRFS_MAX_EXTENT_SIZE (128M) so let's > reduce the argument count and make that a local variable. No functional > changes. > > Signed-off-by: Nikolay Borisov Reviewed-by: Qu Wenruo Thanks, Qu > --- >

Re: [PATCH 3/5] btrfs: Remove superfluous check form btrfs_remove_chunk

2018-10-26 Thread Qu Wenruo
On 2018/10/26 下午7:43, Nikolay Borisov wrote: > It's unnecessary to check map->stripes[i].dev for NULL given its value > is already set and dereferenced above the the check. No functional changes. > > Signed-off-by: Nikolay Borisov Reviewed-by: Qu Wenruo Thanks, Qu > --- >

Re: [PATCH 1/5] btrfs: Ensure at least 1g is free for balance

2018-10-26 Thread Hans van Kranenburg
On 10/26/18 2:16 PM, Nikolay Borisov wrote: > > (Adding Chris to CC since he is the original author of the code) > > On 26.10.2018 15:09, Hans van Kranenburg wrote: >> On 10/26/18 1:43 PM, Nikolay Borisov wrote: >>> The first part of balance operation is to shrink every constituting >>> device

Re: [PATCH 2/5] btrfs: Refactor btrfs_can_relocate

2018-10-26 Thread Qu Wenruo
a bit masked by the > fact that the 0 is re-used from the return value of find_free_dev_extent. > Instead, now just increment dev_nr if we find a suitable extent and explicitly > set can_reloc to true if enough devices with unallocated space are present. > No functional changes. >

Re: [PATCH 1/5] btrfs: Ensure at least 1g is free for balance

2018-10-26 Thread Qu Wenruo
On 2018/10/26 下午8:08, Nikolay Borisov wrote: > > > On 26.10.2018 15:04, Qu Wenruo wrote: >> >> >> On 2018/10/26 下午7:43, Nikolay Borisov wrote: >>> The first part of balance operation is to shrink every constituting >>> device to ensure there is free space for chunk allocation. However, the

Re: [PATCH] btrfs: Fix error handling in btrfs_cleanup_ordered_extents

2018-10-26 Thread kbuild test robot
Hi Nikolay, Thank you for the patch! Yet something to improve: [auto build test ERROR on v4.19] [also build test ERROR on next-20181019] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url:

Re: [PATCH] btrfs-progs: add cli to forget one or all scanned devices

2018-10-26 Thread Nikolay Borisov
On 24.10.2018 07:31, Anand Jain wrote: > This patch adds cli > btrfs device forget [dev] > to remove the given device structure in the kernel if the device > is unmounted. If no argument is given it shall remove all stale > (device which are not mounted) from the kernel. > > Signed-off-by:

Re: [PATCH 1/5] btrfs: Ensure at least 1g is free for balance

2018-10-26 Thread Nikolay Borisov
(Adding Chris to CC since he is the original author of the code) On 26.10.2018 15:09, Hans van Kranenburg wrote: > On 10/26/18 1:43 PM, Nikolay Borisov wrote: >> The first part of balance operation is to shrink every constituting >> device to ensure there is free space for chunk allocation. >

Re: [PATCH 1/5] btrfs: Ensure at least 1g is free for balance

2018-10-26 Thread Hans van Kranenburg
On 10/26/18 1:43 PM, Nikolay Borisov wrote: > The first part of balance operation is to shrink every constituting > device to ensure there is free space for chunk allocation. A very useful thing to be able to do if there's no unallocated raw disk space left, is use balance to rewrite some

Re: [PATCH 1/5] btrfs: Ensure at least 1g is free for balance

2018-10-26 Thread Nikolay Borisov
On 26.10.2018 15:04, Qu Wenruo wrote: > > > On 2018/10/26 下午7:43, Nikolay Borisov wrote: >> The first part of balance operation is to shrink every constituting >> device to ensure there is free space for chunk allocation. However, the code >> has been buggy ever since its introduction since

Re: [PATCH 1/5] btrfs: Ensure at least 1g is free for balance

2018-10-26 Thread Qu Wenruo
On 2018/10/26 下午7:43, Nikolay Borisov wrote: > The first part of balance operation is to shrink every constituting > device to ensure there is free space for chunk allocation. However, the code > has been buggy ever since its introduction since calculating the space to > shrink > the device by

Re: [PATCH v2] btrfs: Fix error handling in btrfs_cleanup_ordered_extents

2018-10-26 Thread Nikolay Borisov
On 26.10.2018 14:53, Qu Wenruo wrote: > > > On 2018/10/26 下午7:41, Nikolay Borisov wrote: >> Running btrfs/124 in a loop hung up on me sporadically with the >> following call trace: >> btrfs D0 5760 5324 0x >> Call Trace: >> ? __schedule+0x243/0x800 >>

Re: [PATCH v2] btrfs: Fix error handling in btrfs_cleanup_ordered_extents

2018-10-26 Thread Qu Wenruo
On 2018/10/26 下午7:41, Nikolay Borisov wrote: > Running btrfs/124 in a loop hung up on me sporadically with the > following call trace: > btrfs D0 5760 5324 0x > Call Trace: >? __schedule+0x243/0x800 >schedule+0x33/0x90 >

Re: [PATCH] btrfs: Fix error handling in btrfs_cleanup_ordered_extents

2018-10-26 Thread Holger Hoffstätte
On 10/26/18 13:13, Nikolay Borisov wrote: + if (page_start >= offset && page_end <= (offset + bytes - 1) { fs/btrfs/inode.c: In function 'btrfs_cleanup_ordered_extents': fs/btrfs/inode.c:140:62: error: expected ')' before '{' token if (page_start >= offset && page_end <= (offset +

Re: [PATCH] btrfs-progs: fix gcc8 default build warning caused by '-Wformat-truncation'

2018-10-25 Thread Qu Wenruo
On 2018/10/26 上午9:58, Su Yanjun wrote: > When using gcc8 compiles utils.c, it complains as below: > > utils.c:852:45: warning: '%s' directive output may be truncated writing > up to 4095 bytes into a region of size 4084 [-Wformat-truncation=] >snprintf(path, sizeof(path), "/dev/mapper/%s",

Re: Kernel crash related to LZO compression

2018-10-25 Thread Qu Wenruo
On 2018/10/25 下午11:56, Dmitry Katsubo wrote: > Dear btrfs community, > > My excuses for the dumps for rather old kernel (4.9.25), nevertheless I > wonder > about your opinion about the below reported kernel crashes. > > As I could understand the situation (correct me if I am wrong), it

Re: [PATCH 2/2] btrfs-progs: Deprecate unused super block member log_root_transid

2018-10-25 Thread Nikolay Borisov
On 25.10.2018 14:18, David Sterba wrote: > On Wed, Oct 24, 2018 at 03:48:07PM +0100, Nikolay Borisov wrote: >> >> >> On 24.10.18 г. 15:30 ч., David Sterba wrote: >>> On Fri, Oct 12, 2018 at 05:57:29PM +0800, Qu Wenruo wrote: + printf("log_root_transid (deprecated)\t%llu\n",

Re: Kernel crash related to LZO compression

2018-10-25 Thread Chris Murphy
On Thu, Oct 25, 2018 at 9:56 AM, Dmitry Katsubo wrote: > Dear btrfs community, > > My excuses for the dumps for rather old kernel (4.9.25), nevertheless I > wonder > about your opinion about the below reported kernel crashes. > > As I could understand the situation (correct me if I am wrong), it

Re: Failover for unattached USB device

2018-10-25 Thread Chris Murphy
On Thu, Oct 25, 2018 at 3:47 AM, Dmitry Katsubo wrote: > > > BTRFS error (device sdf): bdev /dev/sdh errs: wr 0, rd 1867, flush 0, > corrupt 0, gen 0 > BTRFS error (device sdf): bdev /dev/sdg errs: wr 0, rd 1867, flush 0, > corrupt 0, gen 0 > > Attempts lasted for 29 minutes. Yep, and it floods

Re: [PATCH 0/3] fix pinned underflow in generic/475

2018-10-25 Thread Josef Bacik
On Wed, Oct 24, 2018 at 08:24:00PM +0800, Lu Fengqi wrote: > When running generic/475, pinned underflow may occur. This patch will > fix this problem, but there are still other warnings need to addressed in > this case. > > Patch 1-2 introduce a macro and wrappers to help detect underflow > Patch

Re: [PATCH 0/3] btrfs-progs: Minor qgroup subcommand usage update

2018-10-25 Thread David Sterba
On Mon, Aug 06, 2018 at 02:00:05PM +0800, Qu Wenruo wrote: > It turns out even commit e5a6610c943b ("btrfs-progs: qgroup assign: add > option to schedule rescan") introduced the ability to auto rescan, it's > less known option and even some developer find it hard to understand. > (Well, the whole

Re: [PATCH 0/2] btrfstune fs_info cleanups

2018-10-25 Thread David Sterba
On Mon, Aug 20, 2018 at 04:28:03PM +0300, Nikolay Borisov wrote: > While inspecting btrfstune source code I came across 2 functions which > needlessly > take fs_info argument. This small series rectifies this. No functional > changes. > > Nikolay Borisov (2): > btrfstune: Remove fs_info arg

Re: [PATCH v2 0/6] btrfs-progs: print-tree: Minor enhancement along with --bfs/--dfs options

2018-10-25 Thread David Sterba
On Fri, Sep 28, 2018 at 10:04:40AM +0800, Qu Wenruo wrote: > This patchset can be fetched from github: > https://github.com/adam900710/btrfs-progs/tree/dump_tree_enhance > > The main point of this patchset is to allow "btrfs ins dump-tree" to > print tree blocks in breadth-first order when level

Re: [PATCH] btrfs-progs: image: Warn about log tree generation mismatch when restore

2018-10-25 Thread David Sterba
On Thu, Oct 04, 2018 at 02:02:13PM +0800, Qu Wenruo wrote: > At restore time, btrfs-image will try to fixup dev extents, this will > trigger a transaction. > > It's normally OK to start a new transaction, but for image with log > tree, a new transaction will bump up generation, and log tree

Re: [PATCH 2/2] btrfs-progs: Deprecate unused super block member log_root_transid

2018-10-25 Thread David Sterba
On Wed, Oct 24, 2018 at 03:48:07PM +0100, Nikolay Borisov wrote: > > > On 24.10.18 г. 15:30 ч., David Sterba wrote: > > On Fri, Oct 12, 2018 at 05:57:29PM +0800, Qu Wenruo wrote: > >> + printf("log_root_transid (deprecated)\t%llu\n", > >> +

Re: Failover for unattached USB device

2018-10-25 Thread Dmitry Katsubo
On 2018-10-24 20:05, Chris Murphy wrote: I think about the best we can expect in the short term is that Btrfs goes read-only before the file system becomes corrupted in a way it can't recover with a normal mount. And I'm not certain it is in this state of development right now for all cases. And

Re: [PATCH v4] Btrfs: fix deadlock on tree root leaf when finding free extent

2018-10-24 Thread Josef Bacik
On Wed, Oct 24, 2018 at 01:48:40PM +0100, Filipe Manana wrote: > On Wed, Oct 24, 2018 at 1:40 PM Josef Bacik wrote: > > > > On Wed, Oct 24, 2018 at 12:53:59PM +0100, Filipe Manana wrote: > > > On Wed, Oct 24, 2018 at 12:37 PM Josef Bacik wrote: > > > > > > > > On Wed, Oct 24, 2018 at 10:13:03AM

Re: [PATCH] btrfs-progs: super-recover: fix double free fs_devices memory

2018-10-24 Thread David Sterba
On Fri, Oct 12, 2018 at 04:25:23PM +0800, robbieko wrote: > From: Robbie Ko > > super-recover collects btrfs devices information using existed > functions scan_one_devices(). > > Problem is fs_devices is freed twice. One in __open_ctree_fd() when > error happens and the other in

Re: Failover for unattached USB device

2018-10-24 Thread Chris Murphy
On Wed, Oct 24, 2018 at 9:03 AM, Dmitry Katsubo wrote: > On 2018-10-17 00:14, Dmitry Katsubo wrote: >> >> As a workaround I can monitor dmesg output but: >> >> 1. It would be nice if I could tell btrfs that I would like to mount >> read-only >> after a certain error rate per minute is reached. >>

Re: [PATCH 0/3] Misc refactoring of check_file_extent

2018-10-24 Thread David Sterba
On Thu, Sep 13, 2018 at 03:05:04PM +0300, Nikolay Borisov wrote: > While looking at check_file_extent I thought that the code might be a bit > cleaner than it actually is and cleaner as well. The first patch factors out > the code dealing with inline extents into a separate function aptly named

Re: [PATCH] btrfs-progs: fix compile warning when using gcc8 to compile btrfs-progs

2018-10-24 Thread David Sterba
On Fri, Oct 12, 2018 at 03:41:53PM +0800, Su Yanjun wrote: > When using gcc8 to compile btrfs-progs, it complains as below: > > ctree.c: In function 'btrfs_search_slot_for_read': > ctree.c:1249:45: warning: passing argument 3 of 'btrfs_search_slot' > discards 'const' qualifier from

Re: [PATCH] btrfs-progs: fix compile warning when using gcc8 to compile btrfs-progs

2018-10-24 Thread David Sterba
On Fri, Oct 12, 2018 at 09:36:37AM +0800, Su Yanjun wrote: > When using gcc8 compiles utils.c, it complains as below: > > utils.c:852:45: warning: '%s' directive output may be truncated writing > up to 4095 bytes into a region of size 4084 [-Wformat-truncation=] >snprintf(path, sizeof(path),

Re: [PATCH v2] btrfs-progs: change filename limit to 255 when creating subvolume

2018-10-24 Thread David Sterba
On Tue, Sep 18, 2018 at 09:17:54AM +0800, Su Yanjun wrote: > Modify the file name length limit to meet the Linux naming convention. > In addition, the file name length is always bigger than 0, no need to > compare with 0 again. > > Changelog: > v2: > Fix the same problem in creating snapshot

Re: [PATCH] btrfs-progs: tests: Add the testcase for subvolume name length limit test

2018-10-24 Thread David Sterba
On Tue, Sep 18, 2018 at 04:44:48PM +0800, Su Yanjun wrote: > Total of three conditions are tested. One for short name, one with > name length 255, the last one with more than 255. > > This case should pass after commit > 'btrfs-progs: change filename limit to 255 when creating subvolume'. > >

Re: [PATCH 0/2] Enhanced error messages for btrfs-convert

2018-10-24 Thread David Sterba
On Fri, Sep 14, 2018 at 03:25:03PM +0800, Qu Wenruo wrote: > This patchset can be fetched from github: > https://github.com/adam900710/btrfs-progs/tree/convert_error_messages > As usual, it's based on latest stable tag (v4.17.1). > > There is one error report of btrfs-convert, the error message

Re: Failover for unattached USB device

2018-10-24 Thread Dmitry Katsubo
On 2018-10-17 00:14, Dmitry Katsubo wrote: As a workaround I can monitor dmesg output but: 1. It would be nice if I could tell btrfs that I would like to mount read-only after a certain error rate per minute is reached. 2. It would be nice if btrfs could detect that both drives are not

Re: [PATCH 2/2] btrfs-progs: Deprecate unused super block member log_root_transid

2018-10-24 Thread Nikolay Borisov
On 24.10.18 г. 15:30 ч., David Sterba wrote: > On Fri, Oct 12, 2018 at 05:57:29PM +0800, Qu Wenruo wrote: >> +printf("log_root_transid (deprecated)\t%llu\n", >> + le64_to_cpu(sb->__unused_log_root_transid)); > > This should be entirely removed. It

Re: [PATCH 2/2] btrfs-progs: Deprecate unused super block member log_root_transid

2018-10-24 Thread David Sterba
On Fri, Oct 12, 2018 at 05:57:29PM +0800, Qu Wenruo wrote: > +printf("log_root_transid (deprecated)\t%llu\n", > + le64_to_cpu(sb->__unused_log_root_transid)); > >>> > >>> This should be entirely removed. > >> > >> It looks OK to me. > >> Just like the old leafsize.

Re: [RFC][PATCH v2 10/10] btrfs: use common file type conversion

2018-10-24 Thread Phillip Potter
On Wed, Oct 24, 2018 at 12:11:28PM +0200, David Sterba wrote: > On Tue, Oct 23, 2018 at 10:17:28PM +0100, Phillip Potter wrote: > > Deduplicate the btrfs file type conversion implementation. > > The per-filesystem changelogs need a brief explanation why this is done, > like "Filesystems that use

Re: [PATCH v4] Btrfs: fix deadlock on tree root leaf when finding free extent

2018-10-24 Thread Filipe Manana
On Wed, Oct 24, 2018 at 1:40 PM Josef Bacik wrote: > > On Wed, Oct 24, 2018 at 12:53:59PM +0100, Filipe Manana wrote: > > On Wed, Oct 24, 2018 at 12:37 PM Josef Bacik wrote: > > > > > > On Wed, Oct 24, 2018 at 10:13:03AM +0100, fdman...@kernel.org wrote: > > > > From: Filipe Manana > > > > > >

Re: [PATCH v4] Btrfs: fix deadlock on tree root leaf when finding free extent

2018-10-24 Thread Josef Bacik
On Wed, Oct 24, 2018 at 12:53:59PM +0100, Filipe Manana wrote: > On Wed, Oct 24, 2018 at 12:37 PM Josef Bacik wrote: > > > > On Wed, Oct 24, 2018 at 10:13:03AM +0100, fdman...@kernel.org wrote: > > > From: Filipe Manana > > > > > > When we are writing out a free space cache, during the

Re: [PATCH v4] Btrfs: fix deadlock on tree root leaf when finding free extent

2018-10-24 Thread Filipe Manana
On Wed, Oct 24, 2018 at 12:37 PM Josef Bacik wrote: > > On Wed, Oct 24, 2018 at 10:13:03AM +0100, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > When we are writing out a free space cache, during the transaction commit > > phase, we can end up in a deadlock which results in a stack

Re: [PATCH v4] Btrfs: fix deadlock on tree root leaf when finding free extent

2018-10-24 Thread Josef Bacik
On Wed, Oct 24, 2018 at 10:13:03AM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > When we are writing out a free space cache, during the transaction commit > phase, we can end up in a deadlock which results in a stack trace like the > following: > > schedule+0x28/0x80 >

Re: [PATCH v3] Btrfs: fix deadlock on tree root leaf when finding free extent

2018-10-24 Thread Filipe Manana
On Wed, Oct 24, 2018 at 5:08 AM Josef Bacik wrote: > > On Mon, Oct 22, 2018 at 11:05:08PM +0100, Filipe Manana wrote: > > On Mon, Oct 22, 2018 at 8:18 PM Josef Bacik wrote: > > > > > > On Mon, Oct 22, 2018 at 08:10:37PM +0100, fdman...@kernel.org wrote: > > > > From: Filipe Manana > > > > > > >

Re: Have 15GB missing in btrfs filesystem.

2018-10-23 Thread Qu Wenruo
On 2018/10/24 上午8:36, Marc MERLIN wrote: > Normally when btrfs fi show will show lost space because > your trees aren't balanced. Not always the case. Btrfs uses two level allocation: 1) Chunk 2 types, data, metadata, and system. And that's "used" space of "btrfs fi show", or "Device

Re: Have 15GB missing in btrfs filesystem.

2018-10-23 Thread Andrei Borzenkov
24.10.2018 3:36, Marc MERLIN пишет: > Normally when btrfs fi show will show lost space because > your trees aren't balanced. > Balance usually reclaims that space, or most of it. > In this case, not so much. > > kernel 4.17.6: > > saruman:/mnt/btrfs_pool1# btrfs fi show . > Label: 'btrfs_pool1'

Re: [PATCH] btrfs: introduce feature to forget a btrfs device

2018-10-23 Thread Anand Jain
On 10/12/2018 04:48 PM, Nikolay Borisov wrote: On 12.10.2018 07:06, Anand Jain wrote: Support for a new command 'btrfs dev forget [dev]' is proposed here to undo the effects of 'btrfs dev scan [dev]'. For this purpose this patch proposes to use ioctl #5 as it was empty.

Re: [PATCH] btrfs-progs: add cli to forget one or all scanned devices

2018-10-23 Thread Anand Jain
On 10/12/2018 05:03 PM, Nikolay Borisov wrote: On 12.10.2018 07:06, Anand Jain wrote: This patch adds cli btrfs device forget [dev] to remove the given device structure in the kernel if the device is unmounted. If no argument is given it shall remove all stale (device which are not

Re: [PATCH v3] Btrfs: fix deadlock on tree root leaf when finding free extent

2018-10-23 Thread Josef Bacik
On Mon, Oct 22, 2018 at 11:05:08PM +0100, Filipe Manana wrote: > On Mon, Oct 22, 2018 at 8:18 PM Josef Bacik wrote: > > > > On Mon, Oct 22, 2018 at 08:10:37PM +0100, fdman...@kernel.org wrote: > > > From: Filipe Manana > > > > > > When we are writing out a free space cache, during the

Re: [PATCH 01/13] btrfs-progs: lowmem: add argument path to punch_extent_hole()

2018-10-23 Thread Su Yue
On 10/23/18 6:04 PM, Qu Wenruo wrote: On 2018/10/23 下午5:41, Su Yue wrote: Since repair will do CoW, the outer path may be invalid, add an argument path to punch_extent_hole(). When punch_extent_hole() returns, path will still point to the item before calling punch_extent_hole();

Re: [PATCH 03/13] btrfs-progs: lowmem: fix false alert if extent item has been repaired

2018-10-23 Thread Su Yue
On 10/23/18 6:30 PM, Qu Wenruo wrote: On 2018/10/23 下午5:41, Su Yue wrote: Previously, @err are assigned immediately after check but before repair. repair_extent_item()'s return value also confuses the caller. If error has been repaired and returns 0, check_extent_item() will try to

Re: [PATCH 11/13] btrfs-progs: check: Delete file extent item with unaligned extent backref

2018-10-23 Thread Qu Wenruo
On 2018/10/23 下午5:41, Su Yue wrote: > From: Su Yanjun > > In original mode, if some file extent item has unaligned extent backref, > fixup_extent_refs can't repair it. This patch will check extent alignment > then delete file extent with unaligned extent backref. This looks a little strange

Re: [PATCH 10/13] btrfs-progs: check: fix bug in find_possible_backrefs

2018-10-23 Thread Qu Wenruo
On 2018/10/23 下午5:41, Su Yue wrote: > From: Su Yanjun > > It may cost more time to search all extent data of correspond files but > should not influence total speed too much cause that only corrupted > extent items are participated in. Sorry, I didn't really get the point of the modification

Re: [PATCH 09/13] btrfs-progs: Revert "btrfs-progs: Record orphan data extent ref to corresponding root."

2018-10-23 Thread Qu Wenruo
On 2018/10/23 下午5:41, Su Yue wrote: > From: Su Yanjun > > The reason for revert is that according to the existing situation, the > probability of problem in the extent tree is higher than in the fs Tree. > So this feature should be removed. > The same problem as previous patch. We need an

Re: [PATCH 08/13] btrfs-progs: Revert "btrfs-progs: Add repair and report function for orphan file extent."

2018-10-23 Thread Qu Wenruo
On 2018/10/23 下午5:41, Su Yue wrote: > From: Su Yanjun > > The reason for revert is that according to the existing situation, the > probability of problem in the extent tree is higher than in the fs tree. In fact we should find a place to address the priority when we check/repair fs trees.

Re: [PATCH 03/13] btrfs-progs: lowmem: fix false alert if extent item has been repaired

2018-10-23 Thread Qu Wenruo
On 2018/10/24 上午9:27, Su Yue wrote: > > > On 10/23/18 6:30 PM, Qu Wenruo wrote: >> >> >> On 2018/10/23 下午5:41, Su Yue wrote: >>> Previously, @err are assigned immediately after check but before >>> repair. >>> repair_extent_item()'s return value also confuses the caller. If >>> error has been

Re: [PATCH 07/13] btrfs-progs: lowmem: delete unaligned bytes extent data under repair

2018-10-23 Thread Qu Wenruo
On 2018/10/23 下午5:41, Su Yue wrote: > If found a extent data item has unaligned part, lowmem repair > just deletes it. Fair enough solution. Much better than the unpredictable original mode solution. > > Signed-off-by: Su Yue Reviewed-by: Qu Wenruo Thanks, Qu > --- >

Re: [PATCH 06/13] btrfs-progs: lowmem: rename delete_extent_tree_item() to delete_item()

2018-10-23 Thread Qu Wenruo
On 2018/10/23 下午5:41, Su Yue wrote: > The function can delete items in trees besides extent tree. > Rename and move it for further use. > > Signed-off-by: Su Yue Reviewed-by: Qu Wenruo > --- > check/mode-lowmem.c | 99 +++-- > 1 file changed, 51

Re: [PATCH 05/13] btrfs-progs: lowmem: check unaligned disk_bytenr for extent_data

2018-10-23 Thread Qu Wenruo
On 2018/10/23 下午5:41, Su Yue wrote: > Add support to check unaligned disk_bytenr for extent_data. > > Signed-off-by: Su Yue Reviewed-by: Qu Wenruo Thanks, Qu > --- > check/mode-lowmem.c | 9 - > 1 file changed, 8 insertions(+), 1 deletion(-) > > diff --git a/check/mode-lowmem.c

Re: [PATCH 04/13] btrfs-progs: lowmem: fix false alert about the existence of gaps in the check_file_extent

2018-10-23 Thread Qu Wenruo
On 2018/10/23 下午5:41, Su Yue wrote: > From: Lu Fengqi > > The 'end' parameter of check_file_extent tracks the ending offset of the > last checked extent. This is used to detect gaps between adjacent extents. > > Currently such gaps are wrongly detected since for regular extents only > the

Re: Conversion to btrfs raid1 profile on added ext device renders some systems unable to boot into converted rootfs

2018-10-23 Thread Tony Prokott
The trouble is yet unresolved, symptoms are as they were, but I've diagnosed a step further. Maybe you can help me advance the diagnosis or better pose my question among debian experts, related to adjusting the building of initrd. On Thu, 18 Oct 2018 00:08:08 -0700 Qu Wenruo wrote

Re: [PATCH 00/10] Freespace tree repair support v2

2018-10-23 Thread David Sterba
patch 5 to include the bitmap conversion code, now userspace > handles the case of fragmented fst. > > 3. Re-ordered patches 6 and 7 so that we first hook into the extent > alloc/dealloc sequence (patch 6) and finally enable support for fst in patch > 7. > > 4. Updated

Re: [PATCH 05/10] btrfs-progs: Pull free space tree related code from kernel

2018-10-23 Thread David Sterba
On Thu, Oct 04, 2018 at 11:26:36AM -0700, Omar Sandoval wrote: > On Mon, Oct 01, 2018 at 05:46:16PM +0300, Nikolay Borisov wrote: > > To help implement free space tree checker in user space some kernel > > function are necessary, namely iterating/deleting/adding freespace > > items, some internal

Re: [PATCH 03/13] btrfs-progs: lowmem: fix false alert if extent item has been repaired

2018-10-23 Thread Qu Wenruo
On 2018/10/23 下午5:41, Su Yue wrote: > Previously, @err are assigned immediately after check but before > repair. > repair_extent_item()'s return value also confuses the caller. If > error has been repaired and returns 0, check_extent_item() will try > to continue check the nonexistent and cause

Re: [PATCH 02/13] btrfs-progs: lowmem: move nbytes check before isize check

2018-10-23 Thread Qu Wenruo
On 2018/10/23 下午5:41, Su Yue wrote: > For files, lowmem repair will try to check nbytes and isize, > but isize check depends nbytes. > > Once bytes has been repaired, then isize should be checked and > repaired. > So move nbytes check before isize check. Also set nbytes to > extent_size once

Re: [PATCH 01/13] btrfs-progs: lowmem: add argument path to punch_extent_hole()

2018-10-23 Thread Qu Wenruo
On 2018/10/23 下午5:41, Su Yue wrote: > Since repair will do CoW, the outer path may be invalid, > add an argument path to punch_extent_hole(). > When punch_extent_hole() returns, path will still point to the item > before calling punch_extent_hole(); > > Signed-off-by: Su Yue Overall it looks

Re: [PATCH 00/13] btrfs-progs: fixes of file extent in original and lowmem check

2018-10-23 Thread Qu Wenruo
On 2018/10/23 下午5:41, Su Yue wrote: > This patchset can be fetched from my repo: > https://github.com/Damenly/btrfs-progs/tree/file_extent_fixes > which is based on kdave/devel whose HEAD is: > commit 4f20c27ab33aab3efffe13cdae1b8837c821d0d7 (kdave/devel) > Author: Nikolay Borisov > Date:

Re: Btrfs resize seems to deadlock

2018-10-22 Thread Liu Bo
On Sat, Oct 20, 2018 at 1:34 PM Filipe Manana wrote: > > On Sat, Oct 20, 2018 at 9:27 PM Liu Bo wrote: > > > > On Fri, Oct 19, 2018 at 7:09 PM Andrew Nelson > > wrote: > > > > > > I am having an issue with btrfs resize in Fedora 28. I am attempting > > > to enlarge my Btrfs partition. Every

Re: Filesystem corruption?

2018-10-22 Thread Qu Wenruo
On 2018/10/23 上午4:02, Gervais, Francois wrote: > Hi, > > I think I lost power on my btrfs disk and it looks like it is now in an > unfunctional state. What does the word "unfunctional" mean? Unable to mount? Or what else? > > Any idea how I could debug that issue? > > Here is what I have:

Re: [PATCH v3] Btrfs: fix deadlock on tree root leaf when finding free extent

2018-10-22 Thread Filipe Manana
On Mon, Oct 22, 2018 at 8:18 PM Josef Bacik wrote: > > On Mon, Oct 22, 2018 at 08:10:37PM +0100, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > When we are writing out a free space cache, during the transaction commit > > phase, we can end up in a deadlock which results in a stack

Re: [PATCH v9 0/6] Btrfs: implement swap file support

2018-10-22 Thread Omar Sandoval
On Fri, Oct 19, 2018 at 05:43:18PM +0200, David Sterba wrote: > On Thu, Sep 27, 2018 at 11:17:32AM -0700, Omar Sandoval wrote: > > From: Omar Sandoval > > This series implements swap file support for Btrfs. > > > > Changes from v8 [1]: > > > > - Fixed a bug in btrfs_swap_activate() which would

Re: [PATCH v3] Btrfs: fix deadlock on tree root leaf when finding free extent

2018-10-22 Thread Josef Bacik
On Mon, Oct 22, 2018 at 08:10:37PM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > When we are writing out a free space cache, during the transaction commit > phase, we can end up in a deadlock which results in a stack trace like the > following: > > schedule+0x28/0x80 >

Re: [PATCH v2] Btrfs: fix deadlock on tree root leaf when finding free extent

2018-10-22 Thread Filipe Manana
On Mon, Oct 22, 2018 at 7:56 PM Josef Bacik wrote: > > On Mon, Oct 22, 2018 at 07:48:30PM +0100, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > When we are writing out a free space cache, during the transaction commit > > phase, we can end up in a deadlock which results in a stack

Re: [PATCH v2] Btrfs: fix deadlock on tree root leaf when finding free extent

2018-10-22 Thread Josef Bacik
On Mon, Oct 22, 2018 at 07:48:30PM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > When we are writing out a free space cache, during the transaction commit > phase, we can end up in a deadlock which results in a stack trace like the > following: > > schedule+0x28/0x80 >

Re: [PATCH] Btrfs: fix deadlock on tree root leaf when finding free extent

2018-10-22 Thread Filipe Manana
On Mon, Oct 22, 2018 at 7:07 PM Josef Bacik wrote: > > On Mon, Oct 22, 2018 at 10:09:46AM +0100, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > When we are writing out a free space cache, during the transaction commit > > phase, we can end up in a deadlock which results in a stack

Re: [PATCH] Btrfs: fix use-after-free when dumping free space

2018-10-22 Thread David Sterba
On Mon, Oct 22, 2018 at 10:43:06AM +0100, fdman...@kernel.org wrote: > From: Filipe Manana ... > Reported-by: Nikolay Borisov > Signed-off-by: Filipe Manana Added to misc-next, thanks.

Re: [PATCH] Btrfs: fix deadlock on tree root leaf when finding free extent

2018-10-22 Thread Josef Bacik
On Mon, Oct 22, 2018 at 10:09:46AM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > When we are writing out a free space cache, during the transaction commit > phase, we can end up in a deadlock which results in a stack trace like the > following: > > schedule+0x28/0x80 >

Re: [PATCH] Btrfs: fix use-after-free when dumping free space

2018-10-22 Thread Josef Bacik
On Mon, Oct 22, 2018 at 10:43:06AM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > We were iterating a block group's free space cache rbtree without locking > first the lock that protects it (the free_space_ctl->free_space_offset > rbtree is protected by the free_space_ctl->tree_lock

Re: [PATCH] Btrfs: fix deadlock on tree root leaf when finding free extent

2018-10-22 Thread Filipe Manana
On Mon, Oct 22, 2018 at 10:10 AM wrote: > > From: Filipe Manana > > When we are writing out a free space cache, during the transaction commit > phase, we can end up in a deadlock which results in a stack trace like the > following: > > schedule+0x28/0x80 > btrfs_tree_read_lock+0x8e/0x120

Re: [PATCH] Btrfs: fix deadlock on tree root leaf when finding free extent

2018-10-22 Thread Filipe Manana
On Mon, Oct 22, 2018 at 10:10 AM wrote: > > From: Filipe Manana > > When we are writing out a free space cache, during the transaction commit > phase, we can end up in a deadlock which results in a stack trace like the > following: > > schedule+0x28/0x80 > btrfs_tree_read_lock+0x8e/0x120

Re: Btrfs resize seems to deadlock

2018-10-22 Thread Filipe Manana
On Mon, Oct 22, 2018 at 10:06 AM Andrew Nelson wrote: > > OK, an update: After unmouting and running btrfs check, the drive > reverted to reporting the old size. Not sure if this was due to > unmounting / mounting or doing btrfs check. Btrfs check should have > been running in readonly mode. It

Re: Btrfs resize seems to deadlock

2018-10-22 Thread Andrew Nelson
OK, an update: After unmouting and running btrfs check, the drive reverted to reporting the old size. Not sure if this was due to unmounting / mounting or doing btrfs check. Btrfs check should have been running in readonly mode. Since it looked like something was wrong with the resize process, I

Re: How to recover from btrfs scrub errors? (uncorrectable errors, checksum error at logical)

2018-10-22 Thread Qu Wenruo
On 2018/10/22 下午2:29, Otto Kekäläinen wrote: > I never got a reply to this thread, I replied to you but got no rely: https://lore.kernel.org/linux-btrfs/eba5de6f-535a-0f5d-e415-9cd622d71...@gmx.com/ And your steps are just what I suggested. Thanks, Qu > but I am not replying to myself in >

Re: How to recover from btrfs scrub errors? (uncorrectable errors, checksum error at logical)

2018-10-22 Thread Otto Kekäläinen
I never got a reply to this thread, but I am not replying to myself in case somebody has the same issue and is reading the archive: The problem went away after: - deleted all snapshots as they seemed to slow down btrfs I/O so much that simple commands like rm and rsync were unusable - replaced

Re: Untar on empty partition returns ENOSPACE

2018-10-21 Thread Jean-Denis Girard
Le 20/10/2018 à 10:39, Cerem Cem ASLAN a écrit : > You don't even need dd_rescue or something for RaspberryPi creation, > you may use > https://github.com/aktos-io/dcs-tools#cloning-a-target-into-a-new-target That's interesting, but a bit overkill for my needs : I'll stay with my simple script

Re: [PATCH] btrfs/154: test for device dynamic rescan

2018-10-21 Thread Anand Jain
On 10/21/2018 05:31 PM, Filipe Manana wrote: On Sun, Oct 21, 2018 at 10:20 AM Nikolay Borisov wrote: On 21.10.2018 10:16, Filipe Manana wrote: On Mon, Nov 13, 2017 at 2:26 AM Anand Jain wrote: Make sure missing device is included in the alloc list when it is scanned on a mounted FS.

Re: [PATCH v2 rev log added] fstests: btrfs verify hardening agaist duplicate fsid

2018-10-21 Thread Eryu Guan
On Tue, Oct 09, 2018 at 02:28:21AM +0800, Anand Jain wrote: > We have a known bug in btrfs, that we let the device path be changed > after the device has been mounted. So using this loop hole the new > copied device would appears as if its mounted immediately after its > been copied. So this test

Re: [PATCH] btrfs/154: test for device dynamic rescan

2018-10-21 Thread Filipe Manana
On Sun, Oct 21, 2018 at 10:20 AM Nikolay Borisov wrote: > > > > On 21.10.2018 10:16, Filipe Manana wrote: > > On Mon, Nov 13, 2017 at 2:26 AM Anand Jain wrote: > >> > >> Make sure missing device is included in the alloc list when it is > >> scanned on a mounted FS. > >> > >> This test case needs

Re: [PATCH] btrfs/154: test for device dynamic rescan

2018-10-21 Thread Nikolay Borisov
On 21.10.2018 10:16, Filipe Manana wrote: > On Mon, Nov 13, 2017 at 2:26 AM Anand Jain wrote: >> >> Make sure missing device is included in the alloc list when it is >> scanned on a mounted FS. >> >> This test case needs btrfs kernel patch which is in the ML >> [PATCH] btrfs: handle

Re: [PATCH] btrfs/154: test for device dynamic rescan

2018-10-21 Thread Filipe Manana
On Mon, Nov 13, 2017 at 2:26 AM Anand Jain wrote: > > Make sure missing device is included in the alloc list when it is > scanned on a mounted FS. > > This test case needs btrfs kernel patch which is in the ML > [PATCH] btrfs: handle dynamically reappearing missing device > Without the kernel

Re: Btrfs resize seems to deadlock

2018-10-21 Thread Filipe Manana
On Sun, Oct 21, 2018 at 6:05 AM Andrew Nelson wrote: > > Also, is the drive in a safe state to use? Is there anything I should > run on the drive to check consistency? It should be in a safe state. You can verify it running "btrfs check /dev/" (it's a readonly operation). If you are able to

<    4   5   6   7   8   9   10   11   12   13   >