[PATCH] Btrfs: bail out gracefully rather than BUG_ON

2017-10-30 Thread Liu Bo
If a file's DIR_ITEM key is invalid (due to memory errors) and gets written to disk, a future lookup_path can end up with kernel panic due to BUG_ON(). This gets rid of the BUG_ON(), meanwhile output the corrupted key and return ENOENT if it's invalid. Signed-off-by: Liu Bo --- The di

[PATCH] Btrfs: kill threshold for submit_workers

2017-10-30 Thread Liu Bo
lloc_workqueue() can use NO_THRESHOLD for 'submit_workers' and the 'max_active' will be min(num_devices, thread_pool_size). [1]: raid1 example, primary bio /\ bio1 bio2 | | dev1 dev2 | | endio1 endio2 \/ endio Signed-off-

Re: brtfs BUG and erroneous IO

2017-10-26 Thread Liu Bo
On Thu, Oct 26, 2017 at 09:01:33PM +0200, Guillaume Bouchard wrote: > Hi. > > I have a btrfs filesystem on linux 4.13.9 mounted with this options: > > /dev/sda3 on / type btrfs (rw,relatime,ssd,space_cache,subvolid=5,subvol=/) > /dev/sda3 on /nix/store type btrfs > (ro,relatime,ssd,space_cache,su

Re: What is the purpose of EXTENT_PAGE_MAPPED

2017-10-26 Thread Liu Bo
On Tue, Oct 24, 2017 at 05:47:11AM -0500, Goldwyn Rodrigues wrote: > > EXTENT_PAGE_MAPPED gets set in set_page_extent_mapped(), but I don't see > it being cross checked anytime. What is the purpose of setting it? Please check commit d1310b2e0cd98eb1348553e69b73827b436dca7b, it was used to differe

Re: Kernel Oops / btrfs problems

2017-10-26 Thread Liu Bo
On Wed, Oct 25, 2017 at 02:05:48PM +0200, andreas.bt...@diezwickers.de wrote: > Hi, > > I've had problems with a btrfs filesystem on a usb disk. I made a > successfull backup of all data and created the filesystem from scratch. > I'm not able to restore all backuped data because of a kernel oops.

[PATCH v2] Btrfs: add write_flags for compression bio

2017-10-23 Thread Liu Bo
differentiated from each other, because from the POV of block layer, all bios need to be recognized by these flags in order to do some management, e.g. throttlling. This passes writeback_control to compression write path so that it can send bios with proper flags to block layer. Signed-off-by: Liu

[PATCH v3] Btrfs: free btrfs_device in place

2017-10-23 Thread Liu Bo
It's pointless to defer it to a kthread helper as we're not under a special context. For reference, commit 1f78160ce1b1 ("Btrfs: using rcu lock in the reader side of devices list") introduced RCU freeing for device structures. Signed-off-by: Liu Bo Reviewed-by: Anand Jain

[PATCH] fstests: btrfs/143: make test case more reliable

2017-10-23 Thread Liu Bo
s it to do 'fadvise -d' to firstly access all metadata it needs to locate the file and then only drops the test file's page cache. Also this changes it to read the file only if pid%2 == 1. Reported-by: Nikolay Borisov Signed-off-by: Liu Bo --- tests/btrfs/143 | 20 ++-

Re: [PATCH] Btrfs: avoid deadlock in btrfs_async_run_delayed_root

2017-10-20 Thread Liu Bo
Hi, Please ignore this patch, I misunderstood the lock order. thanks, -liubo On Fri, Oct 20, 2017 at 05:40:12PM -0600, Liu Bo wrote: > Here %path is set with ->leave_spinning=1, thus it'll hold the rwlock > after btrfs_search_slot(), however, when processing de

[PATCH] Btrfs: remove redundant btrfs_balance_delayed_items

2017-10-20 Thread Liu Bo
In functions like btrfs_create(), we run both btrfs_balance_delayed_items() and btrfs_btree_balance_dirty() after the operation, but btrfs_btree_balance_dirty() is surely going to run btrfs_balance_delayed_items(). This keeps only btrfs_btree_balance_dirty(). Signed-off-by: Liu Bo --- fs/btrfs

[PATCH] Btrfs: avoid deadlock in btrfs_async_run_delayed_root

2017-10-20 Thread Liu Bo
x is necessary to avoid race between async works, this remove the ->leave_spining setting. Signed-off-by: Liu Bo --- fs/btrfs/delayed-inode.c | 1 - 1 file changed, 1 deletion(-) diff --git a/fs/btrfs/delayed-inode.c b/fs/btrfs/delayed-inode.c index 19e4ad2..a857a2f 100644 --- a/fs/btrfs/dela

Re: [PATCH v2] Fstest: btrfs/151: test if device delete ends up with losing raid profile

2017-10-16 Thread Liu Bo
On Sat, Oct 14, 2017 at 01:33:17PM +0800, Eryu Guan wrote: > On Fri, Oct 13, 2017 at 01:40:05PM -0600, Liu Bo wrote: > > Currently running 'btrfs device delete' can end up with losing data > > raid profile (if any), this test is to reproduce the problem. > > > &

[PATCH v2 RESEND] Fstest: btrfs/151: test if device delete ends up with losing raid profile

2017-10-16 Thread Liu Bo
Currently running 'btrfs device delete' can end up with losing data raid profile (if any), this test is to reproduce the problem. The fix is "Btrfs: avoid losing data raid profile when deleting a device" Signed-off-by: Liu Bo --- v2: - Add _require_btrfs_dev_del_by

Re: [PATCH] Fstest: btrfs/151: test if device delete ends up with losing raid profile

2017-10-16 Thread Liu Bo
On Mon, Oct 16, 2017 at 09:57:11AM +0300, Nikolay Borisov wrote: > > > On 13.10.2017 21:08, Liu Bo wrote: > > On Thu, Oct 12, 2017 at 03:06:57PM +0800, Eryu Guan wrote: > >> On Mon, Oct 09, 2017 at 11:39:21AM -0600, Liu Bo wrote: > >>> Currently running &

Re: [PATCH] Btrfs: add write_flags for compression bio

2017-10-16 Thread Liu Bo
On Mon, Oct 16, 2017 at 03:33:10PM +0200, David Sterba wrote: > On Fri, Oct 13, 2017 at 03:10:35PM -0600, Liu Bo wrote: > > Compression code path has only flaged bios with REQ_OP_WRITE no matter > > where the bios come from. This breaks the rule that sync writes and > > wri

Re: [PATCH v2] Btrfs: avoid losing data raid profile when deleting a device

2017-10-16 Thread Liu Bo
On Mon, Oct 16, 2017 at 12:22:44PM +0800, Anand Jain wrote: > > > On 10/14/2017 04:51 AM, Liu Bo wrote: > >On Wed, Oct 11, 2017 at 10:38:50AM +0300, Nikolay Borisov wrote: > >> > >> > >>On 10.10.2017 20:53, Liu Bo wrote: > >>>We've

[PATCH] Btrfs: add write_flags for compression bio

2017-10-13 Thread Liu Bo
flags to block layer. Signed-off-by: Liu Bo --- fs/btrfs/compression.c | 7 --- fs/btrfs/compression.h | 3 ++- fs/btrfs/extent_io.c | 2 +- fs/btrfs/extent_io.h | 3 ++- fs/btrfs/inode.c | 15 +++ 5 files changed, 20 insertions(+), 10 deletions(-) diff --git a/fs/btrfs

Re: [PATCH v2] Btrfs: avoid losing data raid profile when deleting a device

2017-10-13 Thread Liu Bo
On Wed, Oct 11, 2017 at 10:38:50AM +0300, Nikolay Borisov wrote: > > > On 10.10.2017 20:53, Liu Bo wrote: > > We've avoided data losing raid profile when doing balance, but it > > turns out that deleting a device could also result in the same > > problem &g

[PATCH v2] Fstest: btrfs/151: test if device delete ends up with losing raid profile

2017-10-13 Thread Liu Bo
Currently running 'btrfs device delete' can end up with losing data raid profile (if any), this test is to reproduce the problem. The fix is "Btrfs: avoid losing data raid profile when deleting a device" Signed-off-by: Liu Bo --- test

Re: [PATCH v8 1/2] btrfs: introduce device dynamic state transition to failed

2017-10-13 Thread Liu Bo
On Tue, Oct 03, 2017 at 11:59:19PM +0800, Anand Jain wrote: > From: Anand Jain > > This patch provides helper functions to force a device to failed, > and we need it for the following reasons, > 1) a. It can be reported that device has failed when it does and >b. Close the device when it goes

Re: [PATCH v8 2/2] btrfs: check device for critical errors and mark failed

2017-10-13 Thread Liu Bo
On Sun, Oct 08, 2017 at 10:23:58PM +0800, Anand Jain wrote: > > > On 10/07/2017 07:56 AM, Liu Bo wrote: > > On Thu, Oct 05, 2017 at 09:56:59PM +0800, Anand Jain wrote: > > > > > > > > > On 10/05/2017 04:11 AM, Liu Bo wrote: > > > > On T

Re: [PATCH] Fstest: btrfs/151: test if device delete ends up with losing raid profile

2017-10-13 Thread Liu Bo
On Thu, Oct 12, 2017 at 03:06:57PM +0800, Eryu Guan wrote: > On Mon, Oct 09, 2017 at 11:39:21AM -0600, Liu Bo wrote: > > Currently running 'btrfs device delete' can end up with losing data raid > > profile (if any), this test is to reproduce the problem. > > > &

Re: [PATCH] btrfs: fix false EIO for missing device

2017-10-13 Thread Liu Bo
io->bi_status = BLK_STS_IOERR; > + if (atomic_read(&bbio->error) > bbio->max_errors) > + bio->bi_status = BLK_STS_IOERR; > + else > + bio->bi_status = 0; Thanks for the fix, I'd prefer BLK_S

[PATCH v2] Btrfs: free btrfs_device in place

2017-10-11 Thread Liu Bo
s like mkfs and md are unable to access the device immediately after we do umount. Signed-off-by: Liu Bo Reviewed-by: Anand Jain --- v2: Clarify the lifetime of device and device->bdev respectively and clear the concern about raising the 'device is in use' problem. fs/btrfs/vol

[PATCH v2] Btrfs-progs: do not add stale device into fs_devices

2017-10-11 Thread Liu Bo
, this could confuse users as people may monitor btrfs by running that cli. This does the similar thing to what kernel side has done. Signed-off-by: Liu Bo Reviewed-by: Anand Jain --- v2: remove a wrong parameter. volumes.c | 15 ++- 1 file changed, 14 insertions(+), 1 delet

Re: [PATCH] Btrfs: free btrfs_device in place

2017-10-11 Thread Liu Bo
On Wed, Oct 11, 2017 at 07:54:04PM +0200, David Sterba wrote: > On Tue, Oct 10, 2017 at 03:51:03PM -0600, Liu Bo wrote: > > It's pointless to defer it to a kthread helper as we're not under any > > special context. > > I agree the doubly deferred freeing is point

Re: [PATCH] Btrfs: remove rcu_barrier in btrfs_close_devices

2017-10-11 Thread Liu Bo
On Wed, Oct 11, 2017 at 03:41:23PM +0800, Anand Jain wrote: > > > On 10/11/2017 02:11 PM, Anand Jain wrote: > > > > > > On 10/11/2017 05:51 AM, Liu Bo wrote: > > > It was introduced because btrfs used to do blkdev_put in a deferred > > > work,

Re: [PATCH] Btrfs-progs: do not add stale device into fs_devices

2017-10-11 Thread Liu Bo
On Wed, Oct 11, 2017 at 12:33:15PM +0300, Nikolay Borisov wrote: > > > On 11.10.2017 03:28, Liu Bo wrote: > > If one of btrfs's devices was pulled out and we've replaced it with a > > new one, then they have the same uuid. > > > > If that device get

Re: [PATCH 3/3] Btrfs: remove nr_async_submits and async_submit_draining

2017-10-11 Thread Liu Bo
On Wed, Oct 11, 2017 at 07:20:32PM +0200, David Sterba wrote: > On Wed, Sep 27, 2017 at 02:31:17PM +0200, David Sterba wrote: > > On Thu, Sep 07, 2017 at 11:22:22AM -0600, Liu Bo wrote: > > > Now that we have the combo of flushing twice, which can make sure IO > > > ha

[PATCH] Btrfs-progs: do not add stale device into fs_devices

2017-10-10 Thread Liu Bo
, this could confuse users as people may monitor btrfs by running that cli. This does the similar thing to what kernel side has done. Signed-off-by: Liu Bo --- volumes.c | 15 ++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/volumes.c b/volumes.c index 2f3943d..

Re: [PATCH v2 1/4] btrfs: add_missing_dev() should return the actual error

2017-10-10 Thread Liu Bo
fs_info *fs_info, > } > > device = add_missing_dev(fs_devices, devid, dev_uuid); > - if (!device) > - return -ENOMEM; > + if (IS_ERR(device)) > + return PTR_ERR(device); Could you please

[PATCH] Btrfs: remove rcu_barrier in btrfs_close_devices

2017-10-10 Thread Liu Bo
It was introduced because btrfs used to do blkdev_put in a deferred work, now that btrfs has put blkdev in place, this rcu_barrier can be removed. Signed-off-by: Liu Bo --- fs/btrfs/volumes.c | 6 -- 1 file changed, 6 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index

[PATCH] Btrfs: free btrfs_device in place

2017-10-10 Thread Liu Bo
It's pointless to defer it to a kthread helper as we're not under any special context. Signed-off-by: Liu Bo --- fs/btrfs/volumes.c | 14 ++ 1 file changed, 2 insertions(+), 12 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index d983cea..4a72c45 10064

[PATCH v2] Btrfs: avoid losing data raid profile when deleting a device

2017-10-10 Thread Liu Bo
l the time so their raid profile is persistent. Reported-by: James Alandt Signed-off-by: Liu Bo --- v2: - return the correct error. - move helper ahead of __btrfs_balance(). fs/btrfs/volumes.c | 84 ++ 1 file changed, 65 insertions(+

Re: [PATCH v2 5/5] btrfs: ensure that metadata and flush are issued from the root cgroup

2017-10-10 Thread Liu Bo
If this isn't desirable, > please feel free to drop the section. > > v2: Fixed missing @bh in submit_bh_blkcg_css() call. > Looks good. Reviewed-by: Liu Bo -liubo > Signed-off-by: Tejun Heo > Cc: Chris Mason > Cc: Josef Bacik > --- > fs/btrfs/check-integ

Re: [PATCH] Btrfs: avoid losing data raid profile when deleting a device

2017-10-10 Thread Liu Bo
On Tue, Oct 10, 2017 at 09:57:46AM +0300, Nikolay Borisov wrote: > > > On 9.10.2017 21:01, Liu Bo wrote: > > We've avoided data losing raid profile when doing balance, but it > > turns out that deleting a device could also result in the same > > problem. &g

[PATCH] Btrfs: avoid losing data raid profile when deleting a device

2017-10-09 Thread Liu Bo
l the time so their raid profile is persistent. Reported-by: James Alandt Signed-off-by: Liu Bo --- fs/btrfs/volumes.c | 87 ++ 1 file changed, 68 insertions(+), 19 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index 4a

[PATCH] Fstest: btrfs/151: test if device delete ends up with losing raid profile

2017-10-09 Thread Liu Bo
Currently running 'btrfs device delete' can end up with losing data raid profile (if any), this test is to reproduce the problem. The fix is "Btrfs: avoid losing data raid profile when deleting a device" Signed-off-by: Liu Bo --- test

Re: [PATCH 3/3] btrfs: get rid of sector_t and use u64 offset in submit_extent_page

2017-10-06 Thread Liu Bo
0, 0, 0, false); > @@ -3748,7 +3749,7 @@ static noinline_for_stack int write_one_eb(struct > extent_buffer *eb, > clear_page_dirty_for_io(p); > set_page_writeback(p); > ret = submit_extent_page(REQ_OP_WRITE | write_flags, tree, wbc

Re: [PATCH 1/3] btrfs: scrub: get rid of sector_t

2017-10-06 Thread Liu Bo
rcu_str_deref(dev->name), > - (unsigned long long)swarn.sector, > + swarn.physical, > ref_level ? "node" : "leaf", > ret <

Re: [PATCH 2/3] btrfs: rename page offset parameter in submit_extent_page

2017-10-06 Thread Liu Bo
io, page, page_size, offset); > + bio_add_page(bio, page, page_size, pg_offset); > bio->bi_end_io = end_io_func; > bio->bi_private = tree; > bio->bi_write_hint = page->mapping->host->i_write_hint; Reviewed-by: Liu Bo -liubo -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: "BTRFS error (device vda1): couldn't get super buffer head for bytenr x"

2017-10-06 Thread Liu Bo
On Fri, Oct 06, 2017 at 12:25:17PM +0200, Nick Gilmour wrote: > Hi all, > > I have converted .vdi disk (BTRFS) into a .img disk, resized it from > 500GB to 150GB and then copied into a ZFS Volume. I've imported the VM > into VMM and it started normally but an upgrade failed. I've rebooted > and g

Re: [PATCH v8 2/2] btrfs: check device for critical errors and mark failed

2017-10-06 Thread Liu Bo
On Thu, Oct 05, 2017 at 09:56:59PM +0800, Anand Jain wrote: > > > On 10/05/2017 04:11 AM, Liu Bo wrote: > > On Tue, Oct 03, 2017 at 11:59:20PM +0800, Anand Jain wrote: > > > From: Anand Jain > > > > > > Write and flush errors are critical errors, upo

Re: [PATCH v8 2/2] btrfs: check device for critical errors and mark failed

2017-10-06 Thread Liu Bo
On Thu, Oct 05, 2017 at 07:07:44AM -0400, Austin S. Hemmelgarn wrote: > On 2017-10-04 16:11, Liu Bo wrote: > > On Tue, Oct 03, 2017 at 11:59:20PM +0800, Anand Jain wrote: > > > From: Anand Jain > > > > > > Write and flush errors are critical errors, upon whic

Re: [GIT PULL] Btrfs fixes for 4.14-rc4

2017-10-06 Thread Liu Bo
On Fri, Oct 06, 2017 at 11:25:12PM +0100, Tomasz Kłoczko wrote: > On rc3 is possible to observe warning about possible circular locking > dependency which I've reported on btrfs list few days ago: > Thanks for the report, neither this nor the one you reported on rc2 looks like a deadlock to me. -

Re: [PATCH] btrfs: avoid overflow when sector_t is 32 bit

2017-10-05 Thread Liu Bo
On Wed, Oct 04, 2017 at 08:23:05PM +0200, David Sterba wrote: > On Wed, Oct 04, 2017 at 11:13:51AM -0600, Liu Bo wrote: > > On Wed, Oct 04, 2017 at 04:22:28PM +0200, David Sterba wrote: > > > On Tue, Oct 03, 2017 at 07:31:10PM +0200, Goffredo Baroncelli wrote: > > >

Re: [PATCH v8 2/2] btrfs: check device for critical errors and mark failed

2017-10-04 Thread Liu Bo
On Tue, Oct 03, 2017 at 11:59:20PM +0800, Anand Jain wrote: > From: Anand Jain > > Write and flush errors are critical errors, upon which the device fd > must be closed and marked as failed. > Can we defer the job of closing device to umount? We can go mark the device failed and skip it while d

Re: [PATCH] btrfs: avoid overflow when sector_t is 32 bit

2017-10-04 Thread Liu Bo
On Wed, Oct 04, 2017 at 04:22:28PM +0200, David Sterba wrote: > On Tue, Oct 03, 2017 at 07:31:10PM +0200, Goffredo Baroncelli wrote: > > From: Goffredo Baroncelli > > > > Jean-Denis Girard noticed commit c821e7f3 "pass bytes to > > btrfs_bio_alloc" (https://patchwork.kernel.org/patch/9763081/) in

Re: [PATCH 1/3] Btrfs: remove nr_async_bios

2017-09-29 Thread Liu Bo
On Wed, Sep 27, 2017 at 01:30:13PM +0200, David Sterba wrote: > On Thu, Sep 07, 2017 at 11:22:20AM -0600, Liu Bo wrote: > > This was intended to congest higher layers to not send bios, but as > > > > 1) the congested bit has been taken by writeback > > Can you p

[PATCH v3] fstests: btrfs/150 regression test for reading compressed data

2017-09-27 Thread Liu Bo
We had a bug in btrfs compression code which could end up with a kernel panic. This is adding a regression test for the bug and I've also sent a kernel patch to fix the bug. The patch is "Btrfs: fix kernel oops while reading compressed data". Signed-off-by: Liu Bo --- v3: - Ena

Re: [PATCH v2] fstests: btrfs/150 regression test for reading compressed data

2017-09-27 Thread Liu Bo
On Wed, Sep 27, 2017 at 05:46:44PM +0800, Eryu Guan wrote: > On Tue, Sep 26, 2017 at 05:18:51PM -0700, Liu Bo wrote: > > On Tue, Sep 26, 2017 at 04:37:52PM -0700, Liu Bo wrote: > > > On Tue, Sep 26, 2017 at 05:02:36PM +0800, Eryu Guan wrote: > > > > On Fri, Sep 22, 20

Re: [PATCH v2] fstests: btrfs/150 regression test for reading compressed data

2017-09-26 Thread Liu Bo
On Tue, Sep 26, 2017 at 04:37:52PM -0700, Liu Bo wrote: > On Tue, Sep 26, 2017 at 05:02:36PM +0800, Eryu Guan wrote: > > On Fri, Sep 22, 2017 at 05:21:27PM -0600, Liu Bo wrote: > > > We had a bug in btrfs compression code which could end up with a > > > kernel panic. &g

Re: [PATCH v2] fstests: btrfs/150 regression test for reading compressed data

2017-09-26 Thread Liu Bo
On Tue, Sep 26, 2017 at 05:02:36PM +0800, Eryu Guan wrote: > On Fri, Sep 22, 2017 at 05:21:27PM -0600, Liu Bo wrote: > > We had a bug in btrfs compression code which could end up with a > > kernel panic. > > > > This is adding a regression test for the bug and I've

Re: dmesg: csum "varying number" expected csum 0x0 mirror 1 (with trace/oops)

2017-09-26 Thread Liu Bo
On Wed, Sep 27, 2017 at 12:21:55AM +0200, Kai Krakow wrote: > Hello! > > I came across noting some kernel messages which seem to be related to > btrfs, should I worry? > > I'm currently running scrub on the device now. > > inode-resolve points to an unimportant, easily recoverable file: > > $ s

Re: [PATCH] Btrfs: use btrfs_op instead of bio_op in __btrfs_map_block

2017-09-22 Thread Liu Bo
On Sat, Sep 23, 2017 at 09:49:38AM +0900, Satoru Takeuchi wrote: > At Tue, 19 Sep 2017 17:50:09 -0600, > Liu Bo wrote: > > > > This seems to be a leftover of commit cf8cddd38bab ("btrfs: don't > > abuse REQ_OP_* flags for btrfs_map_block"). > > >

Re: [PATCH] Btrfs: use self-explaining variable

2017-09-22 Thread Liu Bo
On Sat, Sep 23, 2017 at 08:46:55AM +0800, Qu Wenruo wrote: > > > On 2017年09月23日 07:36, Liu Bo wrote: > > This uses a bool 'do_backup' to help understand this piece of code. > > > > Signed-off-by: Liu Bo > > --- > > This is based on a

[PATCH] Btrfs: use self-explaining variable

2017-09-22 Thread Liu Bo
This uses a bool 'do_backup' to help understand this piece of code. Signed-off-by: Liu Bo --- This is based on a patch "Btrfs: do not backup tree roots when fsync". fs/btrfs/disk-io.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/disk-io.c

[PATCH v2] fstests: btrfs/150 regression test for reading compressed data

2017-09-22 Thread Liu Bo
We had a bug in btrfs compression code which could end up with a kernel panic. This is adding a regression test for the bug and I've also sent a kernel patch to fix the bug. The patch is "Btrfs: fix kernel oops while reading compressed data". Signed-off-by: Liu Bo --- v2:

Re: [PATCH] fstests: btrfs/150 regression test for reading compressed data

2017-09-22 Thread Liu Bo
On Thu, Sep 21, 2017 at 02:39:52PM +1000, Dave Chinner wrote: > On Wed, Sep 20, 2017 at 05:52:43PM -0600, Liu Bo wrote: > > We had a bug in btrfs compression code which could end up with a > > kernel panic. > > > > This is adding a regression test for the bug and I'

Re: [PATCH] fstests: btrfs/150 regression test for reading compressed data

2017-09-22 Thread Liu Bo
On Thu, Sep 21, 2017 at 03:03:45PM +0800, Lu Fengqi wrote: > On Wed, Sep 20, 2017 at 05:52:43PM -0600, Liu Bo wrote: > >We had a bug in btrfs compression code which could end up with a > >kernel panic. > > > >This is adding a regression test for the bug and I've al

[PATCH] Btrfs: fix memory leak in raid56

2017-09-22 Thread Liu Bo
The local bio_list may have pending bios when doing cleanup, it can end up with memory leak if they don't get free'd. Signed-off-by: Liu Bo --- fs/btrfs/raid56.c | 22 -- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/raid56.c b/fs/btrf

Re: using fio to test btrfs compression

2017-09-21 Thread Liu Bo
On Mon, Sep 18, 2017 at 01:06:45PM +0530, shally verma wrote: > Hi > > I wanted to test btrfs compression using fio command but somehow > during fio writes, I don't see code taking route of compression blocks > where as If I do a copy to btrfs compression enabled mount point then > I can easily se

[PATCH] fstests: btrfs/150 regression test for reading compressed data

2017-09-20 Thread Liu Bo
We had a bug in btrfs compression code which could end up with a kernel panic. This is adding a regression test for the bug and I've also sent a kernel patch to fix the bug. The patch is "Btrfs: fix kernel oops while reading compressed data". Signed-off-by: Liu Bo ---

[PATCH 2/2] Btrfs: skip checksum when reading compressed data if some IO have failed

2017-09-20 Thread Liu Bo
at in order to return error quickly to the upper layer. Please note that we need to do this after recording the failed mirror index so that read-repair in the upper layer's endio can work properly. Signed-off-by: Liu Bo --- fs/btrfs/compression.c | 9 - 1 file changed, 8 insert

[PATCH 1/2] Btrfs: fix kernel oops while reading compressed data

2017-09-20 Thread Liu Bo
n %cb->orig_bio. With this, btrfs's read-repair can work properly on reading compressed data. Signed-off-by: Liu Bo Reported-by: Paul Jones --- fs/btrfs/compression.c | 9 + 1 file changed, 9 insertions(+) diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c index d2ef9a

Re: kernel BUG at fs/btrfs/extent_io.c:1989

2017-09-20 Thread Liu Bo
On Wed, Sep 20, 2017 at 02:53:57PM +0200, David Sterba wrote: > On Tue, Sep 19, 2017 at 10:12:39AM -0600, Liu Bo wrote: > > On Tue, Sep 19, 2017 at 05:07:25PM +0200, David Sterba wrote: > > > On Tue, Sep 19, 2017 at 11:32:46AM +, Paul Jones wrote: > > > > >

[PATCH] Btrfs: use btrfs_op instead of bio_op in __btrfs_map_block

2017-09-19 Thread Liu Bo
This seems to be a leftover of commit cf8cddd38bab ("btrfs: don't abuse REQ_OP_* flags for btrfs_map_block"). It should use btrfs_op() helper to provide one of 'enum btrfs_map_op' types. Signed-off-by: Liu Bo --- fs/btrfs/volumes.c | 2 +- 1 file changed, 1 insertio

Re: kernel BUG at fs/btrfs/extent_io.c:1989

2017-09-19 Thread Liu Bo
On Tue, Sep 19, 2017 at 05:07:25PM +0200, David Sterba wrote: > On Tue, Sep 19, 2017 at 11:32:46AM +, Paul Jones wrote: > > > This 'mirror 0' looks fishy, (as mirror comes from > > > btrfs_io_bio->mirror_num, > > > which should be at least 1 if raid1 setup is in use.) > > > > > > Not sure if

Re: kernel BUG at fs/btrfs/extent_io.c:1989

2017-09-18 Thread Liu Bo
On Mon, Sep 18, 2017 at 08:55:29AM +, Paul Jones wrote: > Hi > I have a system that crashed during a defrag, upon reboot I got the > following trace while resuming the defrag. > Filesystem is BTRFS Raid1 on lvm+cache, kernel 4.13.2 > Check --repair gives lots of warnings about parent transid

Re: [PATCH] Btrfs: fix unexpected result when dio reading corrupted blocks

2017-09-18 Thread Liu Bo
On Sat, Sep 16, 2017 at 01:58:34PM +0200, Goffredo Baroncelli wrote: > On 09/15/2017 11:06 PM, Liu Bo wrote: > > commit 4246a0b63bd8 ("block: add a bi_error field to struct bio") > > changed the logic of how dio read endio reports errors. > > > > For si

Re: [PATCH] Btrfs: fix unexpected result when dio reading corrupted blocks

2017-09-18 Thread Liu Bo
On Mon, Sep 18, 2017 at 03:49:34PM +0200, Holger Hoffstätte wrote: > > Hello, quick question for backporting.. > > On 09/15/17 23:06, Liu Bo wrote: > > commit 4246a0b63bd8 ("block: add a bi_error field to struct bio") > > changed the logic of how dio read endio

Re: [PATCH] Btrfs: do not backup tree roots when fsync

2017-09-15 Thread Liu Bo
On Thu, Sep 14, 2017 at 02:49:03PM +0200, David Sterba wrote: > On Thu, Sep 14, 2017 at 09:55:48AM +0800, Qu Wenruo wrote: > > > > > > On 2017年09月14日 02:25, Liu Bo wrote: > > > It doens't make sense to backup tree roots when doing fsync, since > > >

[PATCH] Btrfs: fix unexpected result when dio reading corrupted blocks

2017-09-15 Thread Liu Bo
it, we need to report its checksum mismatch error to the upper layer (dio layer in this case) as well. Signed-off-by: Liu Bo Reported-by: Goffredo Baroncelli cc: Goffredo Baroncelli --- fs/btrfs/inode.c | 7 ++- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/inode

Re: BUG: BTRFS and O_DIRECT could lead to wrong checksum and wrong data

2017-09-15 Thread Liu Bo
On Fri, Sep 15, 2017 at 08:57:41PM +0200, Goffredo Baroncelli wrote: > On 09/15/2017 07:01 PM, Liu Bo wrote: > >> Conclusion: even if the file is corrupted and normally BTRFS prevent to > >> access it, using O_DIRECT > >> a) no error is returned to the caller > &g

Re: BUG: BTRFS and O_DIRECT could lead to wrong checksum and wrong data

2017-09-15 Thread Liu Bo
On Fri, Sep 15, 2017 at 12:00:19AM +0200, Goffredo Baroncelli wrote: > Hi all, > > I discovered two bugs when O_DIRECT is used... > > 1) a corrupted file doesn't return -EIO when O_DIRECT is used > > Normally BTRFS prevents to access the contents of a corrupted file; however I > was able read t

[PATCH] Btrfs: do not backup tree roots when fsync

2017-09-13 Thread Liu Bo
It doens't make sense to backup tree roots when doing fsync, since during fsync those tree roots have not been consistent on disk. Signed-off-by: Liu Bo --- fs/btrfs/disk-io.c | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk

[PATCH 2/2 v2] Btrfs: remove bio_flags which indicates a meta block of log-tree

2017-09-13 Thread Liu Bo
g one while the win is that we unify the two places that needs synchronous way and remove a special hack/flag. This removes the bio_flags related stuff for writing log-tree. Signed-off-by: Liu Bo --- v2: Improve the commit log to offer more details why this change makes sense. fs/btrfs/disk

[PATCH] Btrfs: fix confusing worker helper info

2017-09-13 Thread Liu Bo
ne for our helper function, adding a noinline tag can fix that. Signed-off-by: Liu Bo cc: David Sterba --- fs/btrfs/async-thread.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/async-thread.c b/fs/btrfs/async-thread.c index ff0b0be..593709a 100644 --- a/fs/btr

Re: [PATCH 2/2] Btrfs: remove bio_flags which indicates a meta block of log-tree

2017-09-13 Thread Liu Bo
On Wed, Sep 13, 2017 at 06:43:30PM +0200, David Sterba wrote: > On Mon, Aug 21, 2017 at 03:50:00PM -0600, Liu Bo wrote: > > Since both committing transaction and writing log-tree are doing > > plugging on metadata IO, we can unify to use %sync_writers to benefit > > both cases

Re: [PATCH 1/2 RESEND] Btrfs: make plug in writing meta blocks really work

2017-09-13 Thread Liu Bo
On Wed, Sep 13, 2017 at 05:51:55PM +0200, David Sterba wrote: > On Mon, Aug 21, 2017 at 03:49:59PM -0600, Liu Bo wrote: > > We have started plug in btrfs_write_and_wait_marked_extents() but the > > generated IOs actually go to device's schedule IO list where the work > >

[PATCH RFC] Btrfs: fix confusing worker helper info

2017-09-08 Thread Liu Bo
27;s caused by compiler doing inline for our helper function, adding a noinline tag can fix that. Signed-off-by: Liu Bo --- fs/btrfs/async-thread.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/async-thread.c b/fs/btrfs/async-thread.c index ff0b0be..593709a 100644 --

[PATCH 3/3] Btrfs: remove nr_async_submits and async_submit_draining

2017-09-07 Thread Liu Bo
es is still required by shrink_delalloc() as that function doesn't flush twice in the normal case (just issues a writeback with WB_REASON_FS_FREE_SPACE). Signed-off-by: Liu Bo --- fs/btrfs/ctree.h | 2 -- fs/btrfs/disk-io.c | 24 fs/btrfs/inode.c | 28 --

[PATCH 2/3] Btrfs: do not make defrag wait on async_delalloc_pages

2017-09-07 Thread Liu Bo
ore, the behavior of defrag option '-c' remains unchanged. Signed-off-by: Liu Bo --- fs/btrfs/ioctl.c | 15 --- 1 file changed, 15 deletions(-) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index fa1b78c..6dbedd8 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioct

[PATCH 0/3] kill async counters

2017-09-07 Thread Liu Bo
dopts bdi_write_congested() to do the same job, it's not needed, either. Liu Bo (3): Btrfs: remove nr_async_bios Btrfs: do not make defrag wait on async_delalloc_pages Btrfs: remove nr_async_submits and async_submit_draining fs/btrfs/ctree.h | 3 --- fs/btrfs/disk

[PATCH 1/3] Btrfs: remove nr_async_bios

2017-09-07 Thread Liu Bo
This was intended to congest higher layers to not send bios, but as 1) the congested bit has been taken by writeback 2) and no one is waiting for %nr_async_bios down to zero, we can safely remove this now. Signed-off-by: Liu Bo --- fs/btrfs/ctree.h | 1 - fs/btrfs/disk-io.c | 1 - fs

Re: [PATCH] Btrfs: avoid wake_up if possible

2017-09-06 Thread Liu Bo
On Wed, Sep 06, 2017 at 04:19:04PM +0200, David Sterba wrote: > On Fri, Sep 01, 2017 at 04:14:27PM -0600, Liu Bo wrote: > > wake_up() will go to check whether someone is on the waiting list with > > holding spin_lock(). > > > > Around some btrfs code, we don't chec

Re: [PATCH] Btrfs: use the new helper wbc_to_write_flags

2017-09-06 Thread Liu Bo
On Wed, Sep 06, 2017 at 04:38:06PM +0200, David Sterba wrote: > On Thu, Aug 24, 2017 at 06:19:48PM -0600, Liu Bo wrote: > > This updates btrfs to use the helper wbc_to_write_flags which has been > > applied in ext4/xfs/f2fs/block. > > Added in commit 7637241e651ec36e4094 in

Re: Workqueue: events_unbound btrfs_async_reclaim_metadata_space [btrfs]

2017-09-05 Thread Liu Bo
On Tue, Sep 05, 2017 at 11:47:26AM +0200, Marco Lorenzo Crociani wrote: > Hi, > I was transferring some data with rsync to a btrfs filesystem when I got: > > set 04 14:59:05 kernel: INFO: task kworker/u33:2:25015 blocked for more > than 120 seconds. > set 04 14:59:05 kernel: Not tainted 4.

[PATCH] Btrfs: move finish_wait out of the loop

2017-09-01 Thread Liu Bo
If we're still going to wait after schedule(), we don't have to do finish_wait() to remove our %wait_queue_entry since prepare_to_wait() won't add the same %wait_queue_entry twice. Signed-off-by: Liu Bo --- fs/btrfs/ioctl.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)

[PATCH] Btrfs: remove batch plug in run_scheduled_IO

2017-09-01 Thread Liu Bo
Block layer has a limit on plug, ie. BLK_MAX_REQUEST_COUNT == 16, so we don't gain benefits by batching 64 bios here. Signed-off-by: Liu Bo --- fs/btrfs/volumes.c | 6 -- 1 file changed, 6 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index e8b9a26..de55024 100644

[PATCH] Btrfs: use wait_event instead of a single function

2017-09-01 Thread Liu Bo
Since TASK_UNINTERRUPTIBLE has been used here, wait_event() can do the same job. Signed-off-by: Liu Bo --- fs/btrfs/ioctl.c | 21 +++-- 1 file changed, 3 insertions(+), 18 deletions(-) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 19e4dec..1c8bdde 100644 --- a/fs/btrfs

[PATCH] Btrfs: protect conditions within root->log_mutex while waiting

2017-09-01 Thread Liu Bo
Both wait_for_commit() and wait_for_writer() are checking the condition out of the mutex lock. This refactors code a bit to be lock safe. Signed-off-by: Liu Bo --- fs/btrfs/tree-log.c | 30 -- 1 file changed, 16 insertions(+), 14 deletions(-) diff --git a/fs/btrfs

[PATCH] Btrfs: avoid wake_up if possible

2017-09-01 Thread Liu Bo
ctive(), but these two are the hottest one I've run into so far. Signed-off-by: Liu Bo --- fs/btrfs/extent_io.c| 9 - fs/btrfs/ordered-data.c | 8 +++- 2 files changed, 15 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 825fad

[PATCH] Btrfs: use the new helper wbc_to_write_flags

2017-08-24 Thread Liu Bo
go to get a request or wait. Signed-off-by: Liu Bo --- fs/btrfs/extent_io.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index a454a10..825fad6 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -3467,8 +3

Re: [PATCH] Btrfs: do not reset bio->bi_ops while writing bio

2017-08-24 Thread Liu Bo
On Thu, Aug 24, 2017 at 07:10:41AM -0700, Christoph Hellwig wrote: > On Wed, Aug 23, 2017 at 12:15:09PM -0600, Liu Bo wrote: > > flush_epd_write_bio() sets bio->bi_ops by itself to honor REQ_SYNC, > > but it's not needed at all since bio->bi_ops has set up properly in &

[PATCH] Btrfs: do not reset bio->bi_ops while writing bio

2017-08-23 Thread Liu Bo
This remove this unnecessary bio->bi_ops setting. Signed-off-by: Liu Bo --- fs/btrfs/extent_io.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index b61d68f..a454a10 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -4054,9 +40

Re: [PATCH 1/6] btrfs: index check-integrity state hash by a dev_t

2017-08-23 Thread Liu Bo
On Wed, Aug 23, 2017 at 07:10:27PM +0200, Christoph Hellwig wrote: > We won't have the struct block_device available in the bio soon, so switch > to the numerical dev_t instead of the block_device pointer for looking up > the check-integrity state. Reviewed-by: Liu Bo T

Re: [PATCH 1/7] Btrfs: fix blk_status_t/errno confusion

2017-08-23 Thread Liu Bo
struct bio *bio, > + int mirror_num) > { > struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb); > - int ret; > + blk_status_t ret; > > BUG_ON(bio_op(bio) == REQ_OP_WRITE); >

[PATCH 1/2 RESEND] Btrfs: make plug in writing meta blocks really work

2017-08-21 Thread Liu Bo
s the same case as writing log tree, doing sync submit can merge more IOs. Signed-off-by: Liu Bo --- fs/btrfs/disk-io.c | 6 -- fs/btrfs/transaction.c | 2 ++ 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 080e2eb..8d097ba 100644

[PATCH 2/2] Btrfs: remove bio_flags which indicates a meta block of log-tree

2017-08-21 Thread Liu Bo
: Liu Bo --- fs/btrfs/disk-io.c | 6 ++ fs/btrfs/extent_io.c | 13 ++--- fs/btrfs/extent_io.h | 1 - 3 files changed, 4 insertions(+), 16 deletions(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 8d097ba..b0e353e 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c

<    1   2   3   4   5   6   7   8   9   10   >