[PATCH 00/10] several fixes and cleanups

2012-03-27 Thread Liu Bo
This patchset consists of a bug fix from allocating chunk, six bug fixes from autodefrag, and other cleanups. I've tested it with xfstests plus autodefrag option. Liu Bo (10): Btrfs: show useful info in space reservation tracepoint Btrfs: fix deadlock during allocating chunks Btrfs: fix

[PATCH 01/10] Btrfs: show useful info in space reservation tracepoint

2012-03-27 Thread Liu Bo
o For space info, the type of space info is useful for debug. o For transaction handle, its transid is useful. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/extent-tree.c | 29 ++--- fs/btrfs/inode-map.c |6 ++ fs/btrfs/transaction.c |3 +--

[PATCH 02/10][RESEND] Btrfs: fix deadlock during allocating chunks

2012-03-27 Thread Liu Bo
This deadlock comes from xfstests 251. We'll hold the chunk_mutex throughout the whole of a chunk allocation. But if we find that we've used up system chunk space, we need to allocate a new system chunk, but this will lead to a recursion of chunk allocation and end up with a deadlock on

[PATCH 04/10] Btrfs: fix the mismatch of page-mapping

2012-03-27 Thread Liu Bo
commit 600a45e1d5e376f679ff9ecc4ce9452710a6d27c (Btrfs: fix deadlock on page lock when doing auto-defragment) fixes the deadlock on page, but it also introduces another bug. A page may have been truncated after unlock lock. So we need to find it again to get the right one. And since we've held

[PATCH 06/10] Btrfs: add a check to decide if we should defrag the range

2012-03-27 Thread Liu Bo
If our file's layout is as follows: | hole | data1 | hole | data2 | we do not need to defrag this file, because this file has holes and cannot be merged into one extent. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/ioctl.c | 36 +++- 1 files

[PATCH 05/10][RESEND] Btrfs: fix recursive defragment with autodefrag option

2012-03-27 Thread Liu Bo
Reproduce: $ mkfs.btrfs disk $ mount disk /mnt -o autodefrag $ dd if=/dev/zero of=/mnt/foobar bs=4k count=10 2/dev/null sync $ for i in `seq 9 -2 0`; do dd if=/dev/zero of=/mnt/foobar bs=4k count=1 \ seek=$i conv=notrunc 2 /dev/null; done sync then we'll get to defrag foobar again and again.

[PATCH 08/10] Btrfs: update to the right index of defragment

2012-03-27 Thread Liu Bo
When we use autodefrag, we forget to update the index which indicates the last page we've dirty. And we'll set dirty flags on a same set of pages again and again. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/ioctl.c |3 +++ 1 files changed, 3 insertions(+), 0 deletions(-)

[PATCH 10/10] Btrfs: drop cache with VACANCY em when we fail to start a transaction

2012-03-27 Thread Liu Bo
We need to clean a VACANCY em(if we have) when we fail to start a transaction. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/inode.c |6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index bacf441..2b2f0b6 100644 ---

[PATCH 09/10] Btrfs: use PagePrivate2 to check ordered data

2012-03-27 Thread Liu Bo
If a page has PagePrivate2 flag, it still remains as ordered data, so we can check this flag directly instead of looking up an ordered extent. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/inode.c | 45 +++-- 1 files changed, 15

[PATCH 07/10] Btrfs: do not bother to defrag an extent if it is a big real extent

2012-03-27 Thread Liu Bo
$ mkfs.btrfs /dev/sdb7 $ mount /dev/sdb7 /mnt/btrfs/ -oautodefrag $ dd if=/dev/zero of=/mnt/btrfs/foobar bs=4k count=10 oflag=direct 2/dev/null $ filefrag -v /mnt/btrfs/foobar Filesystem type is: 9123683e File size of /mnt/btrfs/foobar is 40960 (10 blocks, blocksize 4096) ext logical physical

[PATCH 03/10] Btrfs: fix race between direct io and autodefrag

2012-03-27 Thread Liu Bo
The bug is from running xfstests 209 with autodefrag. The race is as follows: t1 t2(autodefrag) direct IO invalidate pagecache dio(old data) add_inode_defrag invalidate pagecache endio direct IO invalidate pagecache

btrfs csum failed, scrub ok

2012-03-27 Thread Christoph Groth
I have a freshly installed system with btrfs as the root file system. The machine is running linux 3.2. The raid1 btrfs file system lives on two new hard drives. About one day after installation the following message appeared in kern.log. There were no other errors. root@mim:/var/log# grep

Re: btrfs csum failed, scrub ok

2012-03-27 Thread Roman Mamedov
On Tue, 27 Mar 2012 12:57:31 +0200 Christoph Groth c...@falma.de wrote: root@mim:/# find / -inum 453509 -ls 453509 1976 -rw-r--r-- 1 root root 2020832 Mar 7 21:11 /usr/lib/libreoffice/basis3.4/program/libsblx.so That file seems to be ok, there are no errors when re-reading it.

Re: btrfs csum failed, scrub ok

2012-03-27 Thread Christoph Groth
Roman Mamedov r...@romanrm.ru writes: On Tue, 27 Mar 2012 12:57:31 +0200 Christoph Groth c...@falma.de wrote: root@mim:/# find / -inum 453509 -ls 453509 1976 -rw-r--r-- 1 root root 2020832 Mar 7 21:11 /usr/lib/libreoffice/basis3.4/program/libsblx.so That file seems to be ok,

Re: btrfs: open_ctree failed

2012-03-27 Thread Not Zippy
One entire subvolume was restored. But there were 4 subvolumes on that partition. Is there a way to specify/force the restore of a different subvolume ? find-root seems to only find a single root. thanks On Mon, Mar 26, 2012 at 3:47 PM, Hugo Mills h...@carfax.org.uk wrote: On Mon, Mar 26, 2012

Re: btrfs: open_ctree failed

2012-03-27 Thread Hugo Mills
On Tue, Mar 27, 2012 at 05:58:17AM -0700, Not Zippy wrote: One entire subvolume was restored. But there were 4 subvolumes on that partition. Is there a way to specify/force the restore of a different subvolume ? find-root seems to only find a single root. There is only a single root

[PATCH 1/3] Btrfs: actually call btrfs_init_lockdep

2012-03-27 Thread Jan Schmidt
btrfs_init_lockdep only makes our lockdep class names look prettier, thus it did never hurt we forgot to actually call it. This turns our lockdep identifier strings from lockdep auto-set #[id] into really pretty btrfs-fs-01 or btrfs-csum-03. Signed-off-by: Jan Schmidt list.bt...@jan-o-sch.net ---

[GIT PULL] Btrfs fixes for 3.4

2012-03-27 Thread Jan Schmidt
Hi Chris, please pull my three current patches from my repo, based on your for-linus branch (I can rebase them to the integration branch if that helps): git://git.jan-o-sch.net/btrfs-unstable for-chris It's two really small fixes both mentioned earlier and a more or less imporant fixup

[PATCH 2/3] Btrfs: check return value of btrfs_cow_block()

2012-03-27 Thread Jan Schmidt
The two helper functions commit_cowonly_roots() and create_pending_snapshot() failed to check the return value from btrfs_cow_block(), which could at least in theory fail with -ENOSPC from btrfs_alloc_free_block(). This commit adds the missing checks. Signed-off-by: Jan Schmidt

[PATCH 3/3] Btrfs: fix regression in scrub path resolving

2012-03-27 Thread Jan Schmidt
In commit 4692cf58 we introduced new backref walking code for btrfs. This assumes we're searching live roots, which requires a transaction context. While scrubbing, however, we must not join a transaction because this could deadlock with the commit path. Additionally, what scrub really wants to do

[PATCH 0/8] Restriper fixes

2012-03-27 Thread Ilya Dryomov
Hi Chris, The main one here is the improvement to btrfs_can_relocate(), which is now a tiny bit smarter and does not return ENOSPC when there's plenty of unallocated space for target chunks. This, in addition to my patch which disables silent profile upgrades, should lower a number of corner

[PATCH 3/8] Btrfs: move alloc_profile_is_valid() to volumes.c

2012-03-27 Thread Ilya Dryomov
Header file is not a good place to define functions. This also moves a call to alloc_profile_is_valid() down the stack and removes a redundant check from __btrfs_alloc_chunk() - alloc_profile_is_valid() takes it into account. Signed-off-by: Ilya Dryomov idryo...@gmail.com --- fs/btrfs/ctree.h

[PATCH 1/8] Btrfs: add wrappers for working with alloc profiles

2012-03-27 Thread Ilya Dryomov
Add functions to abstract the conversion between chunk and extended allocation profile formats and switch everybody to use them. Signed-off-by: Ilya Dryomov idryo...@gmail.com --- fs/btrfs/ctree.h | 15 +++ fs/btrfs/extent-tree.c | 25 +++--

[PATCH 4/8] Btrfs: add get_restripe_target() helper

2012-03-27 Thread Ilya Dryomov
Add get_restripe_target() helper and switch everybody to use it. Signed-off-by: Ilya Dryomov idryo...@gmail.com --- fs/btrfs/extent-tree.c | 94 +-- 1 files changed, 50 insertions(+), 44 deletions(-) diff --git a/fs/btrfs/extent-tree.c

[PATCH 5/8] Btrfs: add __get_block_group_index() helper

2012-03-27 Thread Ilya Dryomov
Add __get_block_group_index() helper to be able to derive block group index from an arbitary set of flags. Implement get_block_group_index() in terms of it. Signed-off-by: Ilya Dryomov idryo...@gmail.com --- fs/btrfs/extent-tree.c | 17 - 1 files changed, 12 insertions(+), 5

[PATCH 2/8] Btrfs: make profile_is_valid() check more strict

2012-03-27 Thread Ilya Dryomov
0 is a valid value for an on-disk chunk profile, but it is not a valid extended profile. (We have a separate bit for single chunks in extended case) Also rename it to alloc_profile_is_valid() for clarity. Signed-off-by: Ilya Dryomov idryo...@gmail.com --- fs/btrfs/ctree.h | 21

[PATCH 6/8] Btrfs: improve the logic in btrfs_can_relocate()

2012-03-27 Thread Ilya Dryomov
Currently if we don't have enough space allocated we go ahead and loop though devices in the hopes of finding enough space for a chunk of the *same* type as the one we are trying to relocate. The problem with that is that if we are trying to restripe the chunk its target type can be more relaxed

[PATCH 7/8] Btrfs: validate target profiles only if we are going to use them

2012-03-27 Thread Ilya Dryomov
Do not run sanity checks on all target profiles unless they all will be used. This came up because alloc_profile_is_valid() is now more strict than it used to be. Signed-off-by: Ilya Dryomov idryo...@gmail.com --- fs/btrfs/volumes.c | 27 +++ 1 files changed, 11

[PATCH 8/8] Btrfs: allow dup for data chunks in mixed mode

2012-03-27 Thread Ilya Dryomov
Generally we don't allow dup for data, but mixed chunks are special and people seem to think this has its use cases. Signed-off-by: Ilya Dryomov idryo...@gmail.com --- fs/btrfs/volumes.c | 13 + 1 files changed, 9 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/volumes.c

[PATCH] Btrfs: fix infinite loop in btrfs_shrink_device()

2012-03-27 Thread Ilya Dryomov
If relocate of block group 0 fails with ENOSPC we end up infinitely looping because key.offset -= 1 statement in that case brings us back to where we started. Signed-off-by: Ilya Dryomov idryo...@gmail.com --- fs/btrfs/volumes.c |5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-)

Re: btrfs: open_ctree failed

2012-03-27 Thread Not Zippy
I had found that note on the restore but my restore.c does not allow that flag (it is also missing the m flag as well), I used the branch dangerousdonteveruse on https://git.kernel.org/?p=linux/kernel/git/mason/btrfs-progs.git I switched to the master branch to see if there was a difference but it

Re: btrfs csum failed, scrub ok

2012-03-27 Thread cwillu
On Tue, Mar 27, 2012 at 4:57 AM, Christoph Groth c...@falma.de wrote: I have a freshly installed system with btrfs as the root file system. The machine is running linux 3.2.  The raid1 btrfs file system lives on two new hard drives. About one day after installation the following message

Re: btrfs csum failed, scrub ok

2012-03-27 Thread Jan Schmidt
On 27.03.2012 18:24, cwillu wrote: On Tue, Mar 27, 2012 at 4:57 AM, Christoph Groth c...@falma.de wrote: A scrub done the morning after the incident also didn't find any problems: root@mim:/home/cwg# btrfs scrub status / scrub status for 2da00153-f9ea-4d6c-a6cc-10c913d22686 scrub

Re: btrfs csum failed, scrub ok

2012-03-27 Thread Christoph Groth
Jan Schmidt list.bt...@jan-o-sch.net writes: On 27.03.2012 18:24, cwillu wrote: On Tue, Mar 27, 2012 at 4:57 AM, Christoph Groth c...@falma.de wrote: A scrub done the morning after the incident also didn't find any problems: root@mim:/home/cwg# btrfs scrub status / scrub status for

Create subvolume from a directory?

2012-03-27 Thread Alex
Hi all, Just a quick question but can't find an obvious answer. Can I create/convert a existing (btrfs) directory into a subvolume? It would be very helpful when transferring 'partitions' into btrfs. I found a similar question way back in google, but that site is down now generally. Thanks in

Re: btrfs: open_ctree failed

2012-03-27 Thread Not Zippy
Thought I would let you know I did get things figured out. I used btrfs-progs from github https://github.com/josefbacik/btrfs-progs I also used the findroot function from there which generated more possibilities for the root objectid. By pluging in the guesses from findroot into -r objectid for

Re: Create subvolume from a directory?

2012-03-27 Thread Chester
On Tue, Mar 27, 2012 at 12:19 PM, Alex a...@bpmit.com wrote: Hi all, Just a quick question but can't find an obvious answer. Can I create/convert a existing (btrfs) directory into a subvolume? It would be very helpful when transferring 'partitions' into btrfs. I found a similar question

Re: Create subvolume from a directory?

2012-03-27 Thread Matthias G. Eckermann
Hello Alex and all, On 2012-03-27 T 17:19 + Alex wrote: Just a quick question but can't find an obvious answer. Can I create/convert a existing (btrfs) directory into a subvolume? It would be very helpful when transferring 'partitions' into btrfs. I found a similar question way

Re: btrfs crash after disk reconnect

2012-03-27 Thread Jan Engelhardt
On Monday 2012-03-26 03:42, Liu Bo wrote: On 03/23/2012 08:07 PM, Jan Engelhardt wrote: Observed on Linux 3.2.9 after the controller/disk flaked in-out. (The world still needs a SCSI error decoding tool to tell normal people what cmd and res are about.) I'm not that sure if your

Re: Create subvolume from a directory?

2012-03-27 Thread Fajar A. Nugraha
On Wed, Mar 28, 2012 at 5:24 AM, Matthias G. Eckermann m...@suse.com wrote: While the time measurement might be flawed due to the subvol actions inbetween, caching etc.: I tried several times, and cp --reflinks always is multiple times faster than mv in my environment. So this is

Re: Create subvolume from a directory?

2012-03-27 Thread Liu Bo
On 03/28/2012 06:24 AM, Matthias G. Eckermann wrote: Hello Alex and all, On 2012-03-27 T 17:19 + Alex wrote: Just a quick question but can't find an obvious answer. Can I create/convert a existing (btrfs) directory into a subvolume? It would be very helpful when transferring