Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-05 Thread Johannes Hirte
On Tue, 4 Feb 2014 09:12:54 -0500 Josef Bacik jba...@fb.com wrote: Hrm I was hoping that was going to be more helpful. Can you get perf record -ag and then perf report while it's at full cpu and get the first 3 or 4 things with their traces? Here it comes: # # captured on: Wed Feb

Re: [PATCH] Btrfs: convert to add transaction protection for btrfs send

2014-02-05 Thread Wang Shilong
Hi Josef, [..SNIP..] On 01/31/2014 11:37 AM, Wang Shilong wrote: Hello Josef, 2) Remove the per-root rwsem for the commit root and just make one big rwsem that covers all commit root switching. This way everybody who wants to search with the commit root can just use this semaphore

[PATCH 2/2] btrfs/035: add new clone overwrite regression test

2014-02-05 Thread David Disseldorp
This test uses the newly added cloner binary to dispatch full file and range specific clone (reflink) requests. Signed-off-by: David Disseldorp dd...@suse.de --- tests/btrfs/035 | 76 + tests/btrfs/035.out | 3 +++ tests/btrfs/group | 1

[PATCH 0/2] __btrfs_drop_extents() BUG_ON reproducer

2014-02-05 Thread David Disseldorp
This patch-set provides a reproducer for hitting the 3.14.0-rc1 BUG_ON() at: 692 int __btrfs_drop_extents(struct btrfs_trans_handle *trans, ... 839 /* 840 * | range to drop - | 841 * | extent | 842

[PATCH 1/2] btrfs: add small program for clone testing

2014-02-05 Thread David Disseldorp
The cloner program is capable of cloning files using the BTRFS_IOC_CLONE and BTRFS_IOC_CLONE_RANGE ioctls. Signed-off-by: David Disseldorp dd...@suse.de --- src/Makefile | 2 +- src/cloner.c | 168 +++ 2 files changed, 169 insertions(+),

Re: [PATCH 1/2] Btrfs: switch to btrfs_previous_extent_item()

2014-02-05 Thread Filipe David Manana
On Fri, Jan 31, 2014 at 4:42 PM, Wang Shilong wangshilong1...@gmail.com wrote: From: Wang Shilong wangsl.f...@cn.fujitsu.com Since we have introduced btrfs_previous_extent_item() to search previous extent item, just switch into it. Signed-off-by: Wang Shilong wangsl.f...@cn.fujitsu.com Hi

[PATCH] btrfs: looping across fs_devices isn't necessary

2014-02-05 Thread Anand Jain
btrfs_show_devname() is trying to know dev name with lowest devid for a given FSID, so looping across the FSID isn't necessary Signed-off-by: Anand Jain anand.j...@oracle.com --- fs/btrfs/super.c | 18 +++--- 1 files changed, 7 insertions(+), 11 deletions(-) diff --git

[PATCH] btrfs-progs: fix typo in reported error

2014-02-05 Thread Anand Jain
Signed-off-by: Anand Jain anand.j...@oracle.com --- utils.c |4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/utils.c b/utils.c index a045ffd..1e72b77 100644 --- a/utils.c +++ b/utils.c @@ -1340,7 +1340,7 @@ static int set_label_mounted(const char *mount_path, const

Re: [PATCH] btrfs: looping across fs_devices isn't necessary

2014-02-05 Thread Anand Jain
This patch was incomplete. Kindly ignore. Thanks, Anand On 02/05/14 08:57 PM, Anand Jain wrote: btrfs_show_devname() is trying to know dev name with lowest devid for a given FSID, so looping across the FSID isn't necessary Signed-off-by: Anand Jain anand.j...@oracle.com ---

Re: [PATCH 1/2] Btrfs: switch to btrfs_previous_extent_item()

2014-02-05 Thread Wang Shilong
Hi Filipe, On Fri, Jan 31, 2014 at 4:42 PM, Wang Shilong wangshilong1...@gmail.com wrote: From: Wang Shilong wangsl.f...@cn.fujitsu.com Since we have introduced btrfs_previous_extent_item() to search previous extent item, just switch into it. Signed-off-by: Wang Shilong

Re: [PATCH 1/2] Btrfs: switch to btrfs_previous_extent_item()

2014-02-05 Thread Wang Shilong
So i knew what was wrong here, we need found_key while btrfs_previous_extent_item() did set it properly..^_^ I will send a v2 to fix this, thanks! On Fri, Jan 31, 2014 at 4:42 PM, Wang Shilong wangshilong1...@gmail.com wrote: From: Wang Shilong wangsl.f...@cn.fujitsu.com Since we have

Frequent error messages (block group X has wrong amount of free space)

2014-02-05 Thread Elifarley Callado Coelho Cruz
I've recently installed ArchLinux on a Lenovo Ideapad laptop. Right on the first day of use, I received error messages at boot time like these: BTRFS error (device dm-0): block group 11840520192 has wrong amount of free space BTRFS error (device dm-0): failed to load free space cache for block

Re: [PATCH] Btrfs: convert to add transaction protection for btrfs send

2014-02-05 Thread Josef Bacik
On 02/05/2014 03:59 AM, Wang Shilong wrote: Hi Josef, [..SNIP..] On 01/31/2014 11:37 AM, Wang Shilong wrote: Hello Josef, 2) Remove the per-root rwsem for the commit root and just make one big rwsem that covers all commit root switching. This way everybody who wants to search with the

Re: hitting BUG_ON on troublesome FS

2014-02-05 Thread Josef Bacik
On 02/05/2014 01:34 AM, Remco Hosman - Yerf-it.com wrote: How can i tell? Label: data uuid: a8626d67-4684-4b23-99b3-8d5fa8e7fd69 Total devices 5 FS bytes used 820.00KiB devid2 size 1.82TiB used 1.00GiB path /dev/sdb2 devid3 size 1.82TiB used 1.00GiB path /dev/sdf2 devid5 size

[PATCH v4] btrfs: add simple debugfs interface

2014-02-05 Thread David Sterba
Help during debugging to export various interesting infromation and tunables without the need of extra mount options or ioctls. Usage: * declare your variable in sysfs.h, and include where you need it * define the variable in sysfs.c and make it visible via debugfs_create_TYPE Depends on

Re: Booting with syslinux not possible

2014-02-05 Thread Alex
Duncan 1i5t5.duncan at cox.net writes: Alex posted on Tue, 04 Feb 2014 17:19:09 + as excerpted: I have quite an (overly) complicated setup. I had to chuckle at that one. Fits my setup to a T, altho they're different complications than yours. I'll have to remember it the next

[PATCH v2] btrfs: send: lower memory requirements in common case

2014-02-05 Thread David Sterba
The fs_path structure uses an inline buffer and falls back to a chain of allocations, but vmalloc is not necessary because PATH_MAX fits into PAGE_SIZE. The size of fs_path has been reduced to 256 bytes from PAGE_SIZE, usually 4k. Experimental measurements show that most paths on a single

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-05 Thread Josef Bacik
On 02/05/2014 03:14 AM, Johannes Hirte wrote: On Tue, 4 Feb 2014 09:12:54 -0500 Josef Bacik jba...@fb.com wrote: Hrm I was hoping that was going to be more helpful. Can you get perf record -ag and then perf report while it's at full cpu and get the first 3 or 4 things with their traces?

Re: [PATCH 1/2] Btrfs: switch to btrfs_previous_extent_item()

2014-02-05 Thread Filipe David Manana
On Wed, Feb 5, 2014 at 4:20 PM, Josef Bacik jba...@fb.com wrote: On 02/05/2014 11:14 AM, Wang Shilong wrote: Hi Filipe, So i knew what was wrong here, we need found_key while btrfs_previous_extent_item() did set it properly..^_^ I will send a v2 to fix this, thanks! On Fri, Jan 31,

Re: [PATCH 1/2] Btrfs: switch to btrfs_previous_extent_item()

2014-02-05 Thread Wang Shilong
Hi Filipe, So i knew what was wrong here, we need found_key while btrfs_previous_extent_item() did set it properly..^_^ I will send a v2 to fix this, thanks! On Fri, Jan 31, 2014 at 4:42 PM, Wang Shilong wangshilong1...@gmail.com wrote: From: Wang Shilong wangsl.f...@cn.fujitsu.com

[PATCH 1/2] Btrfs: use right extent item position in send when finding extent clones

2014-02-05 Thread Filipe David Borba Manana
This was a leftover from the commit: 74dd17fbe3d65829e75d84f00a9525b2ace93998 (Btrfs: fix btrfs send for inline items and compression) Signed-off-by: Filipe David Borba Manana fdman...@gmail.com --- fs/btrfs/send.c |2 -- 1 file changed, 2 deletions(-) diff --git a/fs/btrfs/send.c

[PATCH 2/2] Btrfs: make some tree searches in send.c more efficient

2014-02-05 Thread Filipe David Borba Manana
We have this pattern where we do search for a contiguous group of items in a tree and everytime we find an item, we process it, then we release our path, increment the offset of the search key, do another full tree search and repeat these steps until a tree search can't find more items we're

Re: [PATCH] Btrfs: convert to add transaction protection for btrfs send

2014-02-05 Thread Wang Shilong
Hi Josef, On 02/05/2014 03:59 AM, Wang Shilong wrote: Hi Josef, [..SNIP..] On 01/31/2014 11:37 AM, Wang Shilong wrote: Hello Josef, 2) Remove the per-root rwsem for the commit root and just make one big rwsem that covers all commit root switching. This way everybody who wants to

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-05 Thread Johannes Hirte
On Wed, 5 Feb 2014 10:49:15 -0500 Josef Bacik jba...@fb.com wrote: Ok none of those make sense which makes me think it may be the ktime bits, instead of un-applying the whole patch could you just comment out the parts ktime_t start = ktime_get(); and if

Re: Are nocow files snapshot-aware

2014-02-05 Thread Kai Krakow
David Sterba dste...@suse.cz schrieb: On Tue, Feb 04, 2014 at 08:22:05PM -0500, Josef Bacik wrote: On 02/04/2014 03:52 PM, Kai Krakow wrote: Hi! I'm curious... The whole snapshot thing on btrfs is based on its COW design. But you can make individual files and directory contents nocow by

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-05 Thread Josef Bacik
On 02/05/2014 12:34 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 10:49:15 -0500 Josef Bacik jba...@fb.com wrote: Ok none of those make sense which makes me think it may be the ktime bits, instead of un-applying the whole patch could you just comment out the parts ktime_t start =

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-05 Thread Josef Bacik
On 02/05/2014 02:30 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 14:00:57 -0500 Josef Bacik jba...@fb.com wrote: On 02/05/2014 12:34 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 10:49:15 -0500 Josef Bacik jba...@fb.com wrote: Ok none of those make sense which makes me think it may be the

Provide a better free space estimate on RAID1

2014-02-05 Thread Roman Mamedov
Hello, On a freshly-created RAID1 filesystem of two 1TB disks: # df -h /mnt/p2/ Filesystem Size Used Avail Use% Mounted on /dev/sda2 1.8T 1.1M 1.8T 1% /mnt/p2 I cannot write 2TB of user data to that RAID1, so this estimate is clearly misleading. I got tired of looking at the

Re: [PATCH 1/2] Btrfs: switch to btrfs_previous_extent_item()

2014-02-05 Thread Josef Bacik
On 02/05/2014 11:14 AM, Wang Shilong wrote: Hi Filipe, So i knew what was wrong here, we need found_key while btrfs_previous_extent_item() did set it properly..^_^ I will send a v2 to fix this, thanks! On Fri, Jan 31, 2014 at 4:42 PM, Wang Shilong wangshilong1...@gmail.com wrote: From:

Re: [PATCH] Btrfs: convert to add transaction protection for btrfs send

2014-02-05 Thread Josef Bacik
On 02/05/2014 12:23 PM, Wang Shilong wrote: Hi Josef, On 02/05/2014 03:59 AM, Wang Shilong wrote: Hi Josef, [..SNIP..] On 01/31/2014 11:37 AM, Wang Shilong wrote: Hello Josef, 2) Remove the per-root rwsem for the commit root and just make one big rwsem that covers all commit root

[PATCH] Btrfs: fix assert screwup for the pending move stuff

2014-02-05 Thread Josef Bacik
Wang noticed that he was failing btrfs/030 even though me and Filipe couldn't reproduce. Turns out this is because Wang didn't have CONFIG_BTRFS_ASSERT set, which meant that a key part of Filipe's original patch was not being built in. This appears to be a mess up with merging Filipe's patch as

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-05 Thread Johannes Hirte
On Wed, 5 Feb 2014 14:36:39 -0500 Josef Bacik jba...@fb.com wrote: On 02/05/2014 02:30 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 14:00:57 -0500 Josef Bacik jba...@fb.com wrote: On 02/05/2014 12:34 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 10:49:15 -0500 Josef Bacik

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-05 Thread Josef Bacik
On 02/05/2014 04:42 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 14:36:39 -0500 Josef Bacik jba...@fb.com wrote: On 02/05/2014 02:30 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 14:00:57 -0500 Josef Bacik jba...@fb.com wrote: On 02/05/2014 12:34 PM, Johannes Hirte wrote: On Wed, 5 Feb

Re: [PATCH] Btrfs: fix assert screwup for the pending move stuff

2014-02-05 Thread Filipe David Manana
On Wed, Feb 5, 2014 at 9:19 PM, Josef Bacik jba...@fb.com wrote: Wang noticed that he was failing btrfs/030 even though me and Filipe couldn't reproduce. Turns out this is because Wang didn't have CONFIG_BTRFS_ASSERT set, which meant that a key part of Filipe's original patch was not being

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-05 Thread Josef Bacik
On 02/05/2014 04:42 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 14:36:39 -0500 Josef Bacik jba...@fb.com wrote: On 02/05/2014 02:30 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 14:00:57 -0500 Josef Bacik jba...@fb.com wrote: On 02/05/2014 12:34 PM, Johannes Hirte wrote: On Wed, 5 Feb

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-05 Thread Johannes Hirte
On Wed, 5 Feb 2014 16:46:57 -0500 Josef Bacik jba...@fb.com wrote: On 02/05/2014 04:42 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 14:36:39 -0500 Josef Bacik jba...@fb.com wrote: On 02/05/2014 02:30 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 14:00:57 -0500 Josef Bacik

Re: [PATCH 1/2] btrfs: add small program for clone testing

2014-02-05 Thread Dave Chinner
On Wed, Feb 05, 2014 at 12:16:48PM +0100, David Disseldorp wrote: The cloner program is capable of cloning files using the BTRFS_IOC_CLONE and BTRFS_IOC_CLONE_RANGE ioctls. Signed-off-by: David Disseldorp dd...@suse.de Hi Dave - long time since I've seen your head pop up around here ;) A

Re: [PATCH 2/2] btrfs/035: add new clone overwrite regression test

2014-02-05 Thread Dave Chinner
On Wed, Feb 05, 2014 at 12:16:49PM +0100, David Disseldorp wrote: This test uses the newly added cloner binary to dispatch full file and range specific clone (reflink) requests. A couple of small nits: +CLONER_PROG=$here/src/cloner Need to test that the binary was build and is present. +

Re: Are nocow files snapshot-aware

2014-02-05 Thread Duncan
Kai Krakow posted on Wed, 05 Feb 2014 19:17:10 +0100 as excerpted: David Sterba dste...@suse.cz schrieb: On Tue, Feb 04, 2014 at 08:22:05PM -0500, Josef Bacik wrote: On 02/04/2014 03:52 PM, Kai Krakow wrote: Hi! I'm curious... The whole snapshot thing on btrfs is based on its COW

device delete missing panic

2014-02-05 Thread Thermionix
openSUSE 13.1 i686 8 device raid 10 when replacing a failed disk (new device is added) ~ # uname -r 3.11.6-4-pae ~ # btrfs --version Btrfs v3.12+20131125 ~ # mount -o degraded /pool ~ # journalctl | tail Feb 06 12:22:51 store03 kernel: device label pool devid 4 transid 55050 /dev/sde Feb 06

Re: device delete missing panic

2014-02-05 Thread Thermionix
Yep, not planning on using btrfsck unless told ;) hmm now I can't even mount it; ~ # dmesg | tail [ 351.083338] btrfs: failed to read chunk tree on sdg [ 351.088283] btrfs: open_ctree failed [ 392.541465] device label pool devid 7 transid 55056 /dev/sdh [ 393.558496] btrfs: allowing degraded

Re: device delete missing panic

2014-02-05 Thread Anand Jain
your test case is same as in the patch below and the panic was due to null bdev (which matches in your logs). [RFC PATCH] btrfs: fix null pointer deference at btrfs_sysfs_add_one+0x105 But in your logs below, there isn't a panic right ? wrong cut and paste ? or what did I miss? Thanks,

Re: device delete missing panic

2014-02-05 Thread Thermionix
those are the last useful log outputs before the server locks up digging in /var/log/messages - you can see it stopped logging at 12:47, and I hard reset at 3:07 maybe I should have specified hard-lock-up instead of panic 2014-02-06T12:47:47.590784+11:00 store03 kernel: [ 4619.769346]

Re: device delete missing panic

2014-02-05 Thread Anand Jain
as I now can't mount (open_ctree failed) at around open_ctree failed message should help. Thanks, Anand -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at

Re: device delete missing panic

2014-02-05 Thread Thermionix
~ # dmesg | tail [ 351.083338] btrfs: failed to read chunk tree on sdg [ 351.088283] btrfs: open_ctree failed [ 392.541465] device label pool devid 7 transid 55056 /dev/sdh [ 393.558496] btrfs: allowing degraded mounts [ 393.558503] btrfs: disk space caching is enabled [ 393.605461] BTRFS

Re: Provide a better free space estimate on RAID1

2014-02-05 Thread Brendan Hide
On 2014/02/05 10:15 PM, Roman Mamedov wrote: Hello, On a freshly-created RAID1 filesystem of two 1TB disks: # df -h /mnt/p2/ Filesystem Size Used Avail Use% Mounted on /dev/sda2 1.8T 1.1M 1.8T 1% /mnt/p2 I cannot write 2TB of user data to that RAID1, so this estimate is