Re: [PATCH 06/17] Btrfs: introduce grab/put functions for the root of the fs/file tree

2013-05-16 Thread Liu Bo
On Thu, May 16, 2013 at 01:34:11PM +0800, Miao Xie wrote: On Thu, 16 May 2013 13:15:57 +0800, Liu Bo wrote: On Thu, May 16, 2013 at 12:31:11PM +0800, Miao Xie wrote: On thu, 16 May 2013 11:36:46 +0800, Liu Bo wrote: On Wed, May 15, 2013 at 03:48:20PM +0800, Miao Xie wrote: The grab/put

Re: [PATCH v2 1/8] Btrfs: introduce a tree for items that map UUIDs to something

2013-05-16 Thread Liu Bo
On Tue, May 14, 2013 at 11:36:53AM +0200, Stefan Behrens wrote: Mapping UUIDs to subvolume IDs is an operation with a high effort today. Today, the algorithm even has quadratic effort (based on the number of existing subvolumes), which means, that it takes minutes to send/receive a single

Re: [RFC 0/5] BTRFS hot relocation support

2013-05-16 Thread Kai Krakow
Hi! I think such a solution as part of the filesystem could do much better than something outside of it (like bcache). But I'm not sure: What makes data hot? I think the most benefit is detecting random read access and mark only those data as hot, also writes should go to the SSD first and

Re: hard freezes with 3.9.0 during io-intensive loads

2013-05-16 Thread Kai Krakow
Jan Schmidt list.bt...@jan-o-sch.net schrieb: Apparently, it's not fixed. The system does not freeze now but it threw multiple backtraces right in front of my Xorg session. The backtraces look a little bit different now. Here's what I got: https://gist.github.com/kakra/8a340f006d01e146865d

Re: [PATCH 06/17] Btrfs: introduce grab/put functions for the root of the fs/file tree

2013-05-16 Thread Miao Xie
On thu, 16 May 2013 14:15:52 +0800, Liu Bo wrote: On Thu, May 16, 2013 at 01:34:11PM +0800, Miao Xie wrote: On Thu, 16 May 2013 13:15:57 +0800, Liu Bo wrote: On Thu, May 16, 2013 at 12:31:11PM +0800, Miao Xie wrote: On thu, 16 May 2013 11:36:46 +0800, Liu Bo wrote: On Wed, May 15, 2013 at

Re: [PATCH 06/17] Btrfs: introduce grab/put functions for the root of the fs/file tree

2013-05-16 Thread Chris Mason
Quoting Miao Xie (2013-05-16 03:22:37) I must say that the patch itself looks harmless, the reason is not good enough. I don't agree with you. It is perishing that The memory reclaim task is blocked for a long time. We should avoid this problem. synchronize_rcu and friends can take a

Re: [PATCH 06/17] Btrfs: introduce grab/put functions for the root of the fs/file tree

2013-05-16 Thread Liu Bo
On Thu, May 16, 2013 at 03:22:37PM +0800, Miao Xie wrote: On thu, 16 May 2013 14:15:52 +0800, Liu Bo wrote: On Thu, May 16, 2013 at 01:34:11PM +0800, Miao Xie wrote: On Thu, 16 May 2013 13:15:57 +0800, Liu Bo wrote: On Thu, May 16, 2013 at 12:31:11PM +0800, Miao Xie wrote: On thu, 16 May

Re: [PATCH 06/17] Btrfs: introduce grab/put functions for the root of the fs/file tree

2013-05-16 Thread Chris Mason
Quoting Liu Bo (2013-05-16 10:31:39) On Thu, May 16, 2013 at 07:54:17AM -0400, Chris Mason wrote: Quoting Miao Xie (2013-05-16 03:22:37) I must say that the patch itself looks harmless, the reason is not good enough. I don't agree with you. It is perishing that The memory

[PATCH v3 2/8] Btrfs: support printing UUID tree elements

2013-05-16 Thread Stefan Behrens
This commit adds support to print UUID tree elements to print-tree.c. Signed-off-by: Stefan Behrens sbehr...@giantdisaster.de --- fs/btrfs/print-tree.c | 81 +++ 1 file changed, 81 insertions(+) diff --git a/fs/btrfs/print-tree.c

[PATCH v3 6/8] Btrfs: introduce uuid-tree-gen field

2013-05-16 Thread Stefan Behrens
In order to be able to detect the case that a filesystem is mounted with an old kernel, add a uuid-tree-gen field like the free space cache is doing it. It is part of the super block and written with each commit. Old kernels do not know this field and don't update it. Signed-off-by: Stefan

[PATCH v3 4/8] Btrfs: maintain subvolume items in the UUID tree

2013-05-16 Thread Stefan Behrens
When a new subvolume or snapshot is created, a new UUID item is added to the UUID tree. Such items are removed when the subvolume is deleted. The ioctl to set the received subvolume UUID is also touched and will now also add this received UUID into the UUID tree together with the ID of the

[PATCH v3 1/8] Btrfs: introduce a tree for items that map UUIDs to something

2013-05-16 Thread Stefan Behrens
Mapping UUIDs to subvolume IDs is an operation with a high effort today. Today, the algorithm even has quadratic effort (based on the number of existing subvolumes), which means, that it takes minutes to send/receive a single subvolume if 10,000 subvolumes exist. But even linear effort would be

[PATCH v3 5/8] Btrfs: fill UUID tree initially

2013-05-16 Thread Stefan Behrens
When the UUID tree is initially created, a task is spawned that walks through the root tree. For each found subvolume root_item, the uuid and received_uuid entries in the UUID tree are added. This is such a quick operation so that in case somebody wants to unmount the filesystem while the task is

[PATCH v3 0/8] Btrfs: introduce a tree for UUID to subvol ID mapping

2013-05-16 Thread Stefan Behrens
Mapping UUIDs to subvolume IDs is an operation with a high effort today. Today, the algorithm even has quadratic effort (based on the number of existing subvolumes), which means, that it takes minutes to send/receive a single subvolume if 10,000 subvolumes exist. But even linear effort would be

[PATCH v3 7/8] Btrfs: check UUID tree during mount if required

2013-05-16 Thread Stefan Behrens
If the filesystem was mounted with an old kernel that was not aware of the UUID tree, this is detected by looking at the uuid_tree_generation field of the superblock (similar to how the free space cache is doing it). If a mismatch is detected at mount time, a thread is started that does two

[PATCH v3 8/8] Btrfs: add mount option to force UUID tree checking

2013-05-16 Thread Stefan Behrens
This should never be needed, but since all functions are there to check and rebuild the UUID tree, a mount option is added that allows to force this check and rebuild procedure. Signed-off-by: Stefan Behrens sbehr...@giantdisaster.de --- fs/btrfs/ctree.h | 1 + fs/btrfs/disk-io.c | 3 ++-

[PATCH v3 3/8] Btrfs: create UUID tree if required

2013-05-16 Thread Stefan Behrens
This tree is not created by mkfs.btrfs. Therefore when a filesystem is mounted writable and the UUID tree does not exist, this tree is created if required. The tree is also added to the fs_info structure and initialized, but this commit does not yet read or write UUID tree elements.

[PATCH v3 0/4] Btrfs-progs: speedup btrfs send/receive

2013-05-16 Thread Stefan Behrens
The addressed issue is that Btrfs send / receive does not work as it is today when a high number of subvolumes exist. This commit changes the btrfs send/receive commands to use the UUID tree to map UUIDs to subvolumes, and to use the root tree to map subvolume IDs to paths. Now these tools start

[PATCH v3 4/4] Btrfs-progs: add uuid_tree_gen field to btrfs-show-super

2013-05-16 Thread Stefan Behrens
Signed-off-by: Stefan Behrens sbehr...@giantdisaster.de --- btrfs-show-super.c | 2 ++ ctree.h| 5 - 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/btrfs-show-super.c b/btrfs-show-super.c index f587f10..c815469 100644 --- a/btrfs-show-super.c +++

[PATCH v3 2/4] Btrfs-progs: add UUID tree lookup methods

2013-05-16 Thread Stefan Behrens
This commit adds UUID tree lookup methods that make use of the search ioctl. The code is based on the kernel code. Signed-off-by: Stefan Behrens sbehr...@giantdisaster.de --- Makefile| 5 +- ctree.h | 5 ++ uuid-tree.c | 174

[PATCH v3 1/4] Btrfs-progs: Support UUID tree and UUID items in btrfs-debug-tree

2013-05-16 Thread Stefan Behrens
Support printing these things. Signed-off-by: Stefan Behrens sbehr...@giantdisaster.de --- ctree.h | 29 +++ print-tree.c | 95 ++-- 2 files changed, 121 insertions(+), 3 deletions(-) diff --git a/ctree.h b/ctree.h

[PATCH v3 3/4] Btrfs-progs: use UUID tree for send/receive

2013-05-16 Thread Stefan Behrens
This commit changes the btrfs send/receive commands to use the UUID tree to map UUIDs to subvolumes, and to use the root tree to map subvolume IDs to paths. Now these tools start fast and are independent on the number of subvolules/snapshot that exist. Before this commit, mapping UUIDs to

[PATCH] Btrfs: explicitly use global_block_rsv for quota_tree

2013-05-16 Thread Stefan Behrens
The quota_tree was set up to use the empty_block_rsv before which would be problematic when the filesystem is filled up and ENOSPC happens during internal operations while the quota tree is updated and COWed (when the btrfs_qgroup_info_item items) are written. In fact, use_block_rsv() which is

Re: [PATCH 06/17] Btrfs: introduce grab/put functions for the root of the fs/file tree

2013-05-16 Thread Liu Bo
On Thu, May 16, 2013 at 10:34:55AM -0400, Chris Mason wrote: Quoting Liu Bo (2013-05-16 10:31:39) On Thu, May 16, 2013 at 07:54:17AM -0400, Chris Mason wrote: Quoting Miao Xie (2013-05-16 03:22:37) I must say that the patch itself looks harmless, the reason is not good enough.

[PATCH] btrfs-progs: mkfs: add -O option to specify fs features

2013-05-16 Thread David Sterba
Extend mkfs options to specify optional or potentially backwards incompatible features. Signed-off-by: David Sterba dste...@suse.cz --- man/mkfs.btrfs.8.in | 9 mkfs.c | 124 +++- 2 files changed, 123 insertions(+), 10

[PATCH] Btrfs: do delay iput in sync_fs

2013-05-16 Thread Josef Bacik
We get lock inversion with umount if we allow iputs from sync_fs, so use the delay iput flag to keep this from happening. Thanks, Reported-by: David Sterba dste...@suse.cz Signed-off-by: Josef Bacik jba...@fusionio.com --- fs/btrfs/super.c |2 +- 1 files changed, 1 insertions(+), 1

kernel 3.8.8: btrfs still crashes on boot when it can't replay a log

2013-05-16 Thread Marc MERLIN
I've reported this bug a few times over different kernel versions over the last year now, and unfortunately it's still not fixed as of 3.8 (yes, I know 3.9 is out, I'm just about to switch). What happens as far as I know: I have btrfs on top of dmcrypt on an SDD. The SSD on occasion seems to

Re: [PATCH v3 1/4] Btrfs-progs: Support UUID tree and UUID items in btrfs-debug-tree

2013-05-16 Thread David Sterba
On Thu, May 16, 2013 at 04:45:55PM +0200, Stefan Behrens wrote: +struct btrfs_uuid_item { + __le16 type;/* refer to BTRFS_UUID_ITEM_TYPE* defines above */ + __le32 len; /* number of following 64bit values */ + __le64 data[0]; /* data aligned to 64bit */ +} __attribute__

Re: nocow 'C' flag ignored after balance

2013-05-16 Thread Kyle Gates
On Fri, May 10, 2013 Liu Bo wrote: On Thu, May 09, 2013 at 03:41:49PM -0500, Kyle Gates wrote: I'll preface that I'm running Ubuntu 13.04 with the standard 3.8 series kernel so please disregard if this has been fixed in higher versions. This is on a btrfs RAID1 with 3 then 4 disks. My use case

Re: [RFC v0 1/4] vfs: add copy_range syscall and vfs entry point

2013-05-16 Thread Ric Wheeler
On 05/15/2013 04:03 PM, Zach Brown wrote: On Wed, May 15, 2013 at 07:44:05PM +, Eric Wong wrote: Why introduce a new syscall instead of extending sys_splice? Personally, I think it's ugly to have different operations use the same syscall just because their arguments match. I agree with

Re: [PATCH v3 1/4] Btrfs-progs: Support UUID tree and UUID items in btrfs-debug-tree

2013-05-16 Thread Stefan Behrens
On 05/16/2013 18:19, David Sterba wrote: On Thu, May 16, 2013 at 04:45:55PM +0200, Stefan Behrens wrote: +struct btrfs_uuid_item { + __le16 type;/* refer to BTRFS_UUID_ITEM_TYPE* defines above */ + __le32 len; /* number of following 64bit values */ + __le64 data[0]; /*

Re: [PATCH 06/17] Btrfs: introduce grab/put functions for the root of the fs/file tree

2013-05-16 Thread Miao Xie
On thu, 16 May 2013 22:57:07 +0800, Liu Bo wrote: On Thu, May 16, 2013 at 10:34:55AM -0400, Chris Mason wrote: Quoting Liu Bo (2013-05-16 10:31:39) On Thu, May 16, 2013 at 07:54:17AM -0400, Chris Mason wrote: Quoting Miao Xie (2013-05-16 03:22:37) I must say that the patch itself looks

[PATCH V2 17/17] Btrfs: make the state of the transaction more readable

2013-05-16 Thread Miao Xie
We used 3 variants to track the state of the transaction, it was complex and wasted the memory space. Besides that, it was hard to understand that which types of the transaction handles should be blocked in each transaction state, so the developers often made mistakes. This patch improved the