[PATCH 1/6] btrfs-progs: Exit gracefully if we hit ENOSPC when allocating tree block

2018-08-02 Thread Qu Wenruo
When running test fuzz/003, we could hit the following BUG_ON: -- == RUN MAYFAIL /home/adam/btrfs/btrfs-progs/btrfs check --init-csum-tree /home/adam/btrfs/btrfs-progs/tests//fuzz-tests/images/bko-155621-bad-block-group-offset.raw.restored Unable to find block group for 0 Unable to find

[PATCH 0/6] btrfs-progs: Variant fixes for fuzz-tests

2018-08-02 Thread Qu Wenruo
This can be fetched from github: https://github.com/adam900710/btrfs-progs/tree/fixes_for_fuzz_test The base HEAD is: commit d7a1b84756157d544a9ddc399ef48c6132eaafcf (david/devel) Author: Qu Wenruo Date: Thu Jul 5 15:37:31 2018 +0800 btrfs-progs: check/original: Don't overwrite return

[PATCH 2/6] btrfs-progs: Exit gracefully when failed to repair root dir item

2018-08-02 Thread Qu Wenruo
Another BUG_ON() during fuzz/003: -- == RUN MAYFAIL /home/adam/btrfs/btrfs-progs/btrfs check --init-csum-tree /home/adam/btrfs/btrfs-progs/tests//fuzz-tests/images/bko-161821.raw.restored [1/7] checking root items Fixed 0 roots. [2/7] checking extents parent transid verify failed on

[PATCH 3/6] btrfs-progs: Don't report dirty leaked eb using BUG_ON

2018-08-02 Thread Qu Wenruo
Another BUG_ON() during fuzz/003: -- == RUN MAYFAIL /home/adam/btrfs/btrfs-progs/btrfs check --init-csum-tree /home/adam/btrfs/btrfs-progs/tests//fuzz-tests/images/bko-161821.raw.restored [1/7] checking root items Fixed 0 roots. [2/7] checking extents parent transid verify failed on

[PATCH 4/6] btrfs-progs: Fix infinite loop when failed to repair bad key order

2018-08-02 Thread Qu Wenruo
[BUG] An infinite loop can be triggered during fuzz/003: -- == RUN MAYFAIL /home/adam/btrfs/btrfs-progs/btrfs check --repair /home/adam/btrfs/btrfs-progs/tests//fuzz-tests/images/bko-199833-reloc-recovery-crash.raw.restored [1/7] checking root items Fixed 0 roots. [2/7] checking extents

[PATCH 6/6] btrfs-progs: rescue-super: Don't double free fs_devices

2018-08-02 Thread Qu Wenruo
[BUG] During fuzz/007 we hit the following error: -- == RUN MAYFAIL /home/adam/btrfs/btrfs-progs/btrfs rescue super-recover -y -v /home/adam/btrfs/btrfs-progs/tests//fuzz-tests/images/bko-200409.raw.restored.scratch ERROR: tree_root block unaligned: 33554431 ERROR: superblock checksum

[PATCH 5/6] btrfs-progs: Exit gracefull when we failed to alloc dev extent

2018-08-02 Thread Qu Wenruo
Another BUG_ON() during fuzz/003: -- == RUN MAYFAIL /home/adam/btrfs/btrfs-progs/btrfs check --repair /home/adam/btrfs/btrfs-progs/tests//fuzz-tests/images/bko-199833-reloc-recovery-crash.raw.restored [1/7] checking root items Fixed 0 roots. [2/7] checking extents ctree.c:1650:

Re: btrfs check --repair questions

2018-08-02 Thread Qu Wenruo
On 2018年08月03日 12:55, Nathan Dehnel wrote: > Question 1: The command has been running for a couple days on a 10TB > array with no visible change in output. Is it actually fixing anything? > > enabling repair mode > Checking filesystem on > /dev/disk/by-uuid/e1ee5980-c54b-4b6e-82e2-3dbdcee1dd24

btrfs check --repair questions

2018-08-02 Thread Nathan Dehnel
Question 1: The command has been running for a couple days on a 10TB array with no visible change in output. Is it actually fixing anything? enabling repair mode Checking filesystem on /dev/disk/by-uuid/e1ee5980-c54b-4b6e-82e2-3dbdcee1dd24 UUID: e1ee5980-c54b-4b6e-82e2-3dbdcee1dd24 checking

Re: [PATCH] btrfs: qgroup: Remove qgroup item along with subvolume deletion

2018-08-02 Thread Misono Tomohiro
On 2018/08/03 13:23, Lu Fengqi wrote: > On Fri, Aug 03, 2018 at 12:17:26PM +0800, Qu Wenruo wrote: >> >> >> On 2018年08月03日 12:08, Misono Tomohiro wrote: >>> When qgroup is on, subvolume deletion does not remove qgroup items >>> of the subvolume (qgroup info, limits, relation) from quota tree and

Re: [PATCH] btrfs: qgroup: Remove qgroup item along with subvolume deletion

2018-08-02 Thread Lu Fengqi
On Fri, Aug 03, 2018 at 12:17:26PM +0800, Qu Wenruo wrote: > > >On 2018年08月03日 12:08, Misono Tomohiro wrote: >> When qgroup is on, subvolume deletion does not remove qgroup items >> of the subvolume (qgroup info, limits, relation) from quota tree and >> they need to get removed manually by "btrfs

Re: [PATCH] btrfs: qgroup: Remove qgroup item along with subvolume deletion

2018-08-02 Thread Qu Wenruo
On 2018年08月03日 12:08, Misono Tomohiro wrote: > When qgroup is on, subvolume deletion does not remove qgroup items > of the subvolume (qgroup info, limits, relation) from quota tree and > they need to get removed manually by "btrfs qgroup destroy". > > Since level 0 qgroup cannot be

[PATCH] btrfs: qgroup: Remove qgroup item along with subvolume deletion

2018-08-02 Thread Misono Tomohiro
When qgroup is on, subvolume deletion does not remove qgroup items of the subvolume (qgroup info, limits, relation) from quota tree and they need to get removed manually by "btrfs qgroup destroy". Since level 0 qgroup cannot be used/inherited by any other subvolume, let's remove them

Re: [PATCH 2/4] nfs: check for NULL vfsmount in nfs_getattr

2018-08-02 Thread Mark Fasheh
Hi Al, On Thu, Aug 02, 2018 at 01:43:48AM +0100, Al Viro wrote: > On Tue, Jul 31, 2018 at 10:51:57PM +, Mark Fasheh wrote: > > Hi Al, > > > > On Tue, Jul 31, 2018 at 11:16:05PM +0100, Al Viro wrote: > > > On Tue, Jul 31, 2018 at 02:10:43PM -0700, Mark Fasheh wrote: > > > > ->getattr from

Re: [PATCH v2 0/6] btrfs: Enhanced validation check for fuzzed images

2018-08-02 Thread Qu Wenruo
On 2018年08月03日 00:40, David Sterba wrote: > On Wed, Aug 01, 2018 at 10:37:15AM +0800, Qu Wenruo wrote: >> The branch can be fetched from the following git repo: >> https://github.com/adam900710/linux/tree/tree_checker_enhance >> >> It's based on v4.18-rc1, with 3 patches already merged into

Re: [PATCH 0/4] Some random fuzz test fixes

2018-08-02 Thread David Sterba
On Thu, Jul 05, 2018 at 03:37:27PM +0800, Qu Wenruo wrote: > Can be fetched from github: > https://github.com/adam900710/btrfs-progs/tree/fuzzed_enhance > > Some random bugs exposed by the ill-fated fuzz-test (003 never seems to > pass). > > It will be a big work to make fuzz/003 to pass, but at

Re: [PATCH 1/4] btrfs-progs: check: Remove the ability to rebuild root overwritting existing tree blocks

2018-08-02 Thread David Sterba
On Fri, Jul 06, 2018 at 01:53:24AM +, Gu, Jinxiang wrote: > > >> Subject: [PATCH 1/4] btrfs-progs: check: Remove the ability to rebuild > > >> root overwritting existing tree blocks > > >> > > >> We have function btrfs_fsck_reinit_root() to reinit csum or extent tree. > > > > > > Nit: > > >

Re: [PATCH v2 0/2] btrfs-progs: build improvements

2018-08-02 Thread David Sterba
On Thu, Jul 26, 2018 at 01:34:37PM -0700, Omar Sandoval wrote: > From: Omar Sandoval > > Hi, Dave, > > Here's v2 of "btrfs-progs: make all programs and libraries optional", > this time much less overkill. Now, it's just --disable-programs, > --disable-shared, and --disable-static. Based on your

Re: [PATCH] btrfs: revert fs_devices state on error of btrfs_init_new_device()

2018-08-02 Thread David Sterba
On Fri, Jul 27, 2018 at 09:04:55AM +0900, Naohiro Aota wrote: > When btrfs hits error after modifying fs_devices in > btrfs_init_new_device() (such as btrfs_add_dev_item() returns error), it > leaves everything as is, but frees allocated btrfs_device. As a result, > fs_devices->devices and

Re: [PATCH v2 0/6] btrfs: Enhanced validation check for fuzzed images

2018-08-02 Thread David Sterba
On Wed, Aug 01, 2018 at 10:37:15AM +0800, Qu Wenruo wrote: > The branch can be fetched from the following git repo: > https://github.com/adam900710/linux/tree/tree_checker_enhance > > It's based on v4.18-rc1, with 3 patches already merged into misc-next. > > This patchset introduced the

Re: [LKP] [lkp-robot] [mm] 9092c71bb7: blogbench.write_score -12.3% regression

2018-08-02 Thread Josef Bacik
On Thu, Aug 02, 2018 at 01:55:23PM +0800, Huang, Ying wrote: > "Huang, Ying" writes: > > > Hi, Chris, > > > > Chris Mason writes: > > > >> On 19 Jun 2018, at 23:51, Huang, Ying wrote: > > "Huang, Ying" writes: > > > >> Hi, Josef, > >> > >> Do you have time to take a look at

Re: Unmountable root partition

2018-08-02 Thread Cerem Cem ASLAN
FYI: According to ddrescue, there were read errors in +45% of the partition. There were no chances to save anything from the disk. 2018-08-01 15:07 GMT+03:00 Cerem Cem ASLAN : > Yes, command output is as is, because I just copied and pasted into the > mail. When I omit the `-t btrfs` part,

Re: [PATCH] Btrfs: send, fix incorrect file layout after hole punching beyond eof

2018-08-02 Thread David Sterba
On Mon, Jul 30, 2018 at 12:39:58PM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > When doing an incremental send, if we have a file in the parent snapshot > that has prealloc extents beyond EOF and in the send snapshot it got a > hole punch that partially covers the prealloc

Re: [PATCH v2] Btrfs: fix mount failure after fsync due to hard link recreation

2018-08-02 Thread David Sterba
On Fri, Jul 20, 2018 at 10:59:06AM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > If we end up with logging an inode reference item which has the same name > but different index from the one we have persisted, we end up failing when > replaying the log with an errno value of

Re: [PATCH] btrfs: Use wrapper macro for rcu string to remove duplicate code

2018-08-02 Thread David Sterba
On Thu, Aug 02, 2018 at 04:19:07PM +0900, Misono Tomohiro wrote: > Cleanup patch and no functional changes. > > Signed-off-by: Misono Tomohiro Nice. Added to misc-next, thanks. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to

Re: [PATCH] btrfs: dereferencing error pointer in btrfs_quota_enable()

2018-08-02 Thread David Sterba
On Thu, Aug 02, 2018 at 11:44:29AM +0300, Dan Carpenter wrote: > The error handling in "goto out;" expects that if "trans" is non-NULL > that means it's valid. Unfortunately it could also be an error pointer. > > Fixes: c9a6fe84fe43 ("btrfs: qgroups: Move transaction management inside >

Re: BTRFS and databases

2018-08-02 Thread Martin Raiber
On 02.08.2018 14:27 Austin S. Hemmelgarn wrote: > On 2018-08-02 06:56, Qu Wenruo wrote: >> >> On 2018年08月02日 18:45, Andrei Borzenkov wrote: >>> >>> Отправлено с iPhone >>> 2 авг. 2018 г., в 10:02, Qu Wenruo написал(а): > On 2018年08月01日 11:45, MegaBrutal wrote: > Hi all,

Re: [PATCH v3 4/4] btrfs: add helper btrfs_num_devices() to deduce num_devices

2018-08-02 Thread Anand Jain
On 08/02/2018 08:21 PM, David Sterba wrote: On Thu, Aug 02, 2018 at 01:11:40PM +0300, Nikolay Borisov wrote: On 2.08.2018 13:09, Anand Jain wrote: When the replace is running the fs_devices::num_devices also includes the replace device, however in some operations like device delete and

BTRFS and databases

2018-08-02 Thread Remi Gauvin
On 2018-08-02 03:07 AM, Qu Wenruo wrote: > For data, since we have cow (along with csum), it should be no problem > to recover. > > And since datacow is used, transaction on each device should be atomic, > thus we should be able to handle one-time device out-of-sync case. > (For multiple

Re: BTRFS and databases

2018-08-02 Thread Austin S. Hemmelgarn
On 2018-08-02 06:56, Qu Wenruo wrote: On 2018年08月02日 18:45, Andrei Borzenkov wrote: Отправлено с iPhone 2 авг. 2018 г., в 10:02, Qu Wenruo написал(а): On 2018年08月01日 11:45, MegaBrutal wrote: Hi all, I know it's a decade-old question, but I'd like to hear your thoughts of today. By

Re: [PATCH v3 4/4] btrfs: add helper btrfs_num_devices() to deduce num_devices

2018-08-02 Thread David Sterba
On Thu, Aug 02, 2018 at 01:11:40PM +0300, Nikolay Borisov wrote: > > > On 2.08.2018 13:09, Anand Jain wrote: > > When the replace is running the fs_devices::num_devices also includes > > the replace device, however in some operations like device delete and > > balance it needs the actual

Re: [PATCH 11/15] btrfs-progs: Add delayed refs infrastructure

2018-08-02 Thread David Sterba
On Mon, Jul 30, 2018 at 05:34:54PM +0900, Misono Tomohiro wrote: > to correctly count num_entries. > (I noticed that num_entries went to negative value when I'm running gdb) > > However, num_entries is actually not used in progs at all (it is used for > throttling in kernel), so maybe we can just

Re: BTRFS and databases

2018-08-02 Thread Qu Wenruo
On 2018年08月02日 18:45, Andrei Borzenkov wrote: > > > Отправлено с iPhone > >> 2 авг. 2018 г., в 10:02, Qu Wenruo написал(а): >> >> >> >>> On 2018年08月01日 11:45, MegaBrutal wrote: >>> Hi all, >>> >>> I know it's a decade-old question, but I'd like to hear your thoughts >>> of today. By now, I

Re: BTRFS and databases

2018-08-02 Thread Qu Wenruo
On 2018年08月02日 18:35, Andrei Borzenkov wrote: > > > Отправлено с iPhone > >> 2 авг. 2018 г., в 12:16, Martin Steigerwald написал(а): >> >> Hugo Mills - 01.08.18, 10:56: On Wed, Aug 01, 2018 at 05:45:15AM +0200, MegaBrutal wrote: I know it's a decade-old question, but I'd like to

Re: BTRFS and databases

2018-08-02 Thread Andrei Borzenkov
Отправлено с iPhone > 2 авг. 2018 г., в 10:02, Qu Wenruo написал(а): > > > >> On 2018年08月01日 11:45, MegaBrutal wrote: >> Hi all, >> >> I know it's a decade-old question, but I'd like to hear your thoughts >> of today. By now, I became a heavy BTRFS user. Almost everywhere I use >> BTRFS,

Re: BTRFS and databases

2018-08-02 Thread Martin Steigerwald
Andrei Borzenkov - 02.08.18, 12:35: > Отправлено с iPhone > > > 2 авг. 2018 г., в 12:16, Martin Steigerwald > > написал(а):> > > Hugo Mills - 01.08.18, 10:56: > >>> On Wed, Aug 01, 2018 at 05:45:15AM +0200, MegaBrutal wrote: > >>> I know it's a decade-old question, but I'd like to hear your >

Re: BTRFS and databases

2018-08-02 Thread Andrei Borzenkov
Отправлено с iPhone > 2 авг. 2018 г., в 12:16, Martin Steigerwald написал(а): > > Hugo Mills - 01.08.18, 10:56: >>> On Wed, Aug 01, 2018 at 05:45:15AM +0200, MegaBrutal wrote: >>> I know it's a decade-old question, but I'd like to hear your >>> thoughts >>> of today. By now, I became a heavy

Re: [PATCH] btrfs: dereferencing error pointer in btrfs_quota_enable()

2018-08-02 Thread Nikolay Borisov
On 2.08.2018 11:44, Dan Carpenter wrote: > The error handling in "goto out;" expects that if "trans" is non-NULL > that means it's valid. Unfortunately it could also be an error pointer. > > Fixes: c9a6fe84fe43 ("btrfs: qgroups: Move transaction management inside >

Re: BTRFS and databases

2018-08-02 Thread ein
On 08/02/2018 11:16 AM, Martin Steigerwald wrote: > However I wonder: Is this it? Is there nothing that can be improved in > BTRFS to handle database and VM files in a better way, without altering > any default settings? Poor performance is not the biggest BTRFS problem, it's known for silent

Re: [PATCH v3 4/4] btrfs: add helper btrfs_num_devices() to deduce num_devices

2018-08-02 Thread Nikolay Borisov
On 2.08.2018 13:09, Anand Jain wrote: > When the replace is running the fs_devices::num_devices also includes > the replace device, however in some operations like device delete and > balance it needs the actual num_devices without the repalce devices, so > now the function btrfs_num_devices()

Re: [PATCH v2 4/4] btrfs: add helper btrfs_num_devices() to deduce num_devices

2018-08-02 Thread Anand Jain
On 08/01/2018 10:41 PM, David Sterba wrote: On Thu, Jul 26, 2018 at 02:53:34PM +0800, Anand Jain wrote: When the replace is running the fs_devices::num_devices also includes the replace device, however in some operations like device delete and balance it needs the actual num_devices without

[PATCH v3 4/4] btrfs: add helper btrfs_num_devices() to deduce num_devices

2018-08-02 Thread Anand Jain
When the replace is running the fs_devices::num_devices also includes the replace device, however in some operations like device delete and balance it needs the actual num_devices without the repalce devices, so now the function btrfs_num_devices() just provides that. And here is a scenario how

Re: [PATCH v2 2/4] btrfs: fix race between free_stale_devices and close_fs_devices

2018-08-02 Thread Anand Jain
On 08/01/2018 10:29 PM, David Sterba wrote: On Thu, Jul 26, 2018 at 02:53:32PM +0800, Anand Jain wrote: From: Anand Jain %fs_devices can be free-ed by btrfs_free_stale_devices() when the close_fs_devices() drops fs_devices::opened to zero, but close_fs_devices tries to access the

Re: BTRFS and databases

2018-08-02 Thread Martin Steigerwald
Hugo Mills - 01.08.18, 10:56: > On Wed, Aug 01, 2018 at 05:45:15AM +0200, MegaBrutal wrote: > > I know it's a decade-old question, but I'd like to hear your > > thoughts > > of today. By now, I became a heavy BTRFS user. Almost everywhere I > > use BTRFS, except in situations when it is obvious

Re: [PATCH] btrfs: Use wrapper macro for rcu string to remove duplicate code

2018-08-02 Thread Qu Wenruo
On 2018年08月02日 15:19, Misono Tomohiro wrote: > Cleanup patch and no functional changes. > > Signed-off-by: Misono Tomohiro Looks good to me. Reviewed-by: Qu Wenruo Thanks, Qu > --- > fs/btrfs/ioctl.c | 6 ++ > fs/btrfs/scrub.c | 8 ++-- > fs/btrfs/super.c | 9 +++-- >

[PATCH] btrfs: dereferencing error pointer in btrfs_quota_enable()

2018-08-02 Thread Dan Carpenter
The error handling in "goto out;" expects that if "trans" is non-NULL that means it's valid. Unfortunately it could also be an error pointer. Fixes: c9a6fe84fe43 ("btrfs: qgroups: Move transaction management inside btrfs_quota_enable/disable") Signed-off-by: Dan Carpenter diff --git

Re: [PATCH v2] btrfs: locking: Allow btrfs_tree_lock() to return error to avoid deadlock

2018-08-02 Thread Qu Wenruo
On 2018年08月01日 21:45, Qu Wenruo wrote: > > > On 2018年08月01日 21:27, David Sterba wrote: >> On Tue, Jul 31, 2018 at 02:52:23PM +0800, Qu Wenruo wrote: >>> [BUG] >>> For certains crafted image, whose csum root leaf has missing backref, if >>> we try to trigger write with data csum, it could cause

[PATCH] btrfs: Use wrapper macro for rcu string to remove duplicate code

2018-08-02 Thread Misono Tomohiro
Cleanup patch and no functional changes. Signed-off-by: Misono Tomohiro --- fs/btrfs/ioctl.c | 6 ++ fs/btrfs/scrub.c | 8 ++-- fs/btrfs/super.c | 9 +++-- fs/btrfs/volumes.c | 21 ++--- 4 files changed, 13 insertions(+), 31 deletions(-) diff --git

Re: BTRFS and databases

2018-08-02 Thread Qu Wenruo
On 2018年08月01日 22:33, Remi Gauvin wrote: > On 2018-07-31 11:45 PM, MegaBrutal wrote: > >> I know that with nodatacow, I take away most of the benefits of BTRFS >> (those are actually hurting database performance – the exact CoW >> nature that is elsewhere a blessing, with databases it's a

Re: BTRFS and databases

2018-08-02 Thread Qu Wenruo
On 2018年08月01日 11:45, MegaBrutal wrote: > Hi all, > > I know it's a decade-old question, but I'd like to hear your thoughts > of today. By now, I became a heavy BTRFS user. Almost everywhere I use > BTRFS, except in situations when it is obvious there is no benefit > (e.g. /var/log, /boot). At