Re: [RFC PATCH 0/3] apply the Probabilistic Skiplist on btrfs

2012-01-06 Thread Liu Bo
On 01/06/2012 07:51 AM, David Sterba wrote: Hi, I've let it run through xfstests and ended at 091, patches applied on top of 3.2, mount options compress-force=lzo,discard,inode_cache,space_cache,autodefrag fresh mkfs with defaults. Hi David, Thanks a lot for your work! I also find this

Re: [PATCH 0/2] btrfs: allow cross-subvolume BTRFS_IOC_CLONE

2012-01-06 Thread Konstantinos Skarlatos
On 22/12/2011 2:24 μμ, Chris Samuel wrote: Christoph, On Sat, 2 Apr 2011 12:40:11 AM Chris Mason wrote: Excerpts from Christoph Hellwig's message of 2011-04-01 09:34:05 -0400: I don't think it's a good idea to introduce any user visible operations over subvolume boundaries. Currently we

How long does it take to balance a 2x1TB RAID1 ?

2012-01-06 Thread Dirk Lutzebäck
Hi, I have setup up a btrfs RAID1 using two 1TB drives. How long should a 'btrfs filesystem balance' take? It is running now for more than 3 days on about 30% CPU and 40% wait state. I am using stock btrfs from ubuntu 11.10 kernel 3.0.0 Regards Dirk -- To unsubscribe from this list:

[PATCH 01/21] Btrfs: get rid of *_alloc_profile fields

2012-01-06 Thread Ilya Dryomov
{data,metadata,system}_alloc_profile fields have been unused for a long time now. Get rid of them. Signed-off-by: Ilya Dryomov idryo...@gmail.com --- fs/btrfs/ctree.h |3 --- fs/btrfs/disk-io.c |3 --- fs/btrfs/extent-tree.c | 10 -- fs/btrfs/volumes.c |6

[PATCH 02/21] Btrfs: introduce masks for chunk type and profile

2012-01-06 Thread Ilya Dryomov
Chunk's type and profile are encoded in u64 flags field. Introduce masks to easily access them. Also fix the type of BTRFS_BLOCK_GROUP_* constants, it should be ULL. Signed-off-by: Ilya Dryomov idryo...@gmail.com --- fs/btrfs/ctree.h | 26 +-

[PATCH 03/21] Btrfs: add BTRFS_AVAIL_ALLOC_BIT_SINGLE bit

2012-01-06 Thread Ilya Dryomov
Right now on-disk BTRFS_BLOCK_GROUP_* profile bits are used for avail_{data,metadata,system}_alloc_bits fields, which gather info about available allocation profiles in the FS. When chunk is created or read from disk, its profile is OR'ed with the corresponding avail_alloc_bits field. Since

[PATCH 04/21] Btrfs: make avail_*_alloc_bits fields dynamic

2012-01-06 Thread Ilya Dryomov
Currently when new chunks are created respective avail_alloc_bits field is updated to reflect profiles of all chunks present in the system. However when chunks are removed profile bits are never cleared. This patch clears profile bit of respective avail_alloc_bits field when the last chunk with

[PATCH 06/21] Btrfs: add basic infrastructure for selective balancing

2012-01-06 Thread Ilya Dryomov
This allows to have a separate set of filters for each chunk type (data,meta,sys). The code however is generic and switch on chunk type is only done once. This commit also adds a type filter: it allows to balance for example meta and system chunks w/o touching data ones. Signed-off-by: Ilya

[PATCH 07/21] Btrfs: profiles filter

2012-01-06 Thread Ilya Dryomov
Select chunks based on a given profile mask. Signed-off-by: Ilya Dryomov idryo...@gmail.com --- fs/btrfs/volumes.c | 24 fs/btrfs/volumes.h |4 2 files changed, 28 insertions(+), 0 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index

[PATCH 08/21] Btrfs: usage filter

2012-01-06 Thread Ilya Dryomov
Select chunks that are less than X percent full. Signed-off-by: Ilya Dryomov idryo...@gmail.com --- fs/btrfs/volumes.c | 36 fs/btrfs/volumes.h |1 + 2 files changed, 37 insertions(+), 0 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c

[PATCH 05/21] Btrfs: add basic restriper infrastructure

2012-01-06 Thread Ilya Dryomov
Add basic restriper infrastructure: extended balancing ioctl and all related ioctl data structures, add data structure for tracking restriper's state to fs_info, etc. The semantics of the old balancing ioctl are fully preserved. Explicitly disallow any volume operations when balance is in

[PATCH 09/21] Btrfs: devid filter

2012-01-06 Thread Ilya Dryomov
Relocate chunks which have at least one stripe located on a device with devid X. Signed-off-by: Ilya Dryomov idryo...@gmail.com --- fs/btrfs/volumes.c | 23 +++ fs/btrfs/volumes.h |1 + 2 files changed, 24 insertions(+), 0 deletions(-) diff --git a/fs/btrfs/volumes.c

[PATCH 10/21] Btrfs: devid subset filter

2012-01-06 Thread Ilya Dryomov
Select chunks which have at least one byte of at least one stripe located on a device with devid X in a given [pstart,pend) physical address range. This filter only works when devid filter is turned on. Signed-off-by: Ilya Dryomov idryo...@gmail.com --- fs/btrfs/volumes.c | 45

[PATCH 11/21] Btrfs: virtual address space subset filter

2012-01-06 Thread Ilya Dryomov
Select chunks which have at least one byte located inside a given [vstart, vend) virtual address space range. Signed-off-by: Ilya Dryomov idryo...@gmail.com --- fs/btrfs/volumes.c | 20 fs/btrfs/volumes.h |1 + 2 files changed, 21 insertions(+), 0 deletions(-) diff

[PATCH 12/21] Btrfs: do not reduce profile in do_chunk_alloc()

2012-01-06 Thread Ilya Dryomov
Every caller of do_chunk_alloc() feeds it the reduced allocation profile, so stop trying to reduce it one more time. Instead check the validity of the passed profile. Signed-off-by: Ilya Dryomov idryo...@gmail.com --- fs/btrfs/ctree.h | 18 ++ fs/btrfs/extent-tree.c |

[PATCH 13/21] Btrfs: implement online profile changing

2012-01-06 Thread Ilya Dryomov
Profile changing is done by launching a balance with BTRFS_BALANCE_CONVERT bits set and target fields of respective btrfs_balance_args structs initialized. Profile reducing code in this case will pick restriper's target profile if it's available instead of doing a blind reduce. If target profile

[PATCH 14/21] Btrfs: soft profile changing mode (aka soft convert)

2012-01-06 Thread Ilya Dryomov
When doing convert from one profile to another if soft mode is on restriper won't touch chunks that already have the profile we are converting to. This is useful if e.g. half of the FS was converted earlier. The soft mode switch is (like every other filter) per-type. This means that we can

[PATCH 15/21] Btrfs: save balance parameters to disk

2012-01-06 Thread Ilya Dryomov
Introduce a new btree objectid for storing balance item. The reason is to be able to resume restriper after a crash with the same parameters. Balance item has a very high objectid and goes into tree of tree roots. The key for the new item is as follows: [ BTRFS_BALANCE_OBJECTID ;

[PATCH 16/21] Btrfs: recover balance on mount

2012-01-06 Thread Ilya Dryomov
On mount, if balance item is found, resume balance in a separate kernel thread. Try to be smart to continue roughly where previous balance (or convert) was interrupted. For chunk types that were being converted to some profile we turn on soft convert, in case of a simple balance we turn on usage

[PATCH 17/21] Btrfs: add skip_balance mount option

2012-01-06 Thread Ilya Dryomov
Since restriper kthread starts involuntarily on mount and can suck cpu and memory bandwidth add a mount option to forcefully skip it. The restriper in that case hangs around in paused state and can be resumed from userspace when it's convenient. Signed-off-by: Ilya Dryomov idryo...@gmail.com ---

[PATCH 18/21] Btrfs: allow for pausing restriper

2012-01-06 Thread Ilya Dryomov
Implement an ioctl for pausing restriper. This pauses the relocation, but balance is still considered to be in progress: balance item is not deleted, other volume operations cannot be started, etc. If paused in the middle of profile changing operation we will continue making allocations with the

[PATCH 19/21] Btrfs: allow for cancelling restriper

2012-01-06 Thread Ilya Dryomov
Implement an ioctl for cancelling restriper. Currently we wait until relocation of the current block group is finished, in future this can be done by triggering a commit. Balance item is deleted and no memory about the interrupted balance is kept. Signed-off-by: Ilya Dryomov idryo...@gmail.com

[PATCH 20/21] Btrfs: allow for resuming restriper after it was paused

2012-01-06 Thread Ilya Dryomov
Implement an ioctl for resuming restriper. We use the same heuristics used when recovering balance after a crash to try to start where we left off last time. Signed-off-by: Ilya Dryomov idryo...@gmail.com --- fs/btrfs/ioctl.c |2 ++ fs/btrfs/ioctl.h |1 + fs/btrfs/volumes.c | 27

[PATCH 21/21] Btrfs: add balance progress reporting

2012-01-06 Thread Ilya Dryomov
Signed-off-by: Ilya Dryomov idryo...@gmail.com --- fs/btrfs/ioctl.c | 51 +++ fs/btrfs/ioctl.h |6 ++ fs/btrfs/volumes.c | 35 +-- fs/btrfs/volumes.h |3 +++ 4 files changed, 93 insertions(+), 2

[PATCH] Btrfs-progs: restriper interface

2012-01-06 Thread Ilya Dryomov
Hello, This is an update of userspace restriper interface. The main change is that restriper commands have been moved under balance prefix. So now we have: btrfs fi balance start btrfs fi balance pause btrfs fi balance cancel btrfs fi balance resume btrfs fi balance status This breaks

[PATCH] Btrfs-progs: add restriper commands

2012-01-06 Thread Ilya Dryomov
Import restriper commands under btrfs fi balance: btrfs fi balance start btrfs fi balance cancel btrfs fi balance pause btrfs fi balance resume btrfs fi balance status NOTE: Backwards compatibility is broken for now, to get the old balance everything behaviour one has to call 'btrfs fi

Re: [RFC][PATCH 3/3] Btrfs: improve truncation of btrfs

2012-01-06 Thread Josef Bacik
On Fri, Jan 06, 2012 at 11:51:16AM +0800, Miao Xie wrote: On thu, 5 Jan 2012 10:15:50 -0500, Josef Bacik wrote: + trans = btrfs_start_transaction(root, 2); + if (IS_ERR(trans)) + return PTR_ERR(trans); /* * setattr is responsible for setting the ordered_data_close

[PATCH] btrfs: silence warning in raid array setup

2012-01-06 Thread David Sterba
Raid array setup code creates an extent buffer in an usual way. When the PAGE_CACHE_SIZE is super block size, the extent pages are not marked up-to-date, which triggers a WARN_ON in the following write_extent_buffer call. Add an explicit SetPageUptodate call to silence the warning.

Re: Crash in io_ctl_drop_pages after mount with csum errors

2012-01-06 Thread David Sterba
On Fri, Jan 06, 2012 at 03:17:59PM +0800, Li Zefan wrote: [ 1499.946409] BUG: unable to handle kernel NULL pointer dereference at 0001 [ 1499.946437] IP: [a0456dd7] io_ctl_drop_pages+0x37/0x70 [btrfs] 0x01 is weired, don't know how it occured. Nevertheless we need

Re: [PATCH 0/2] btrfs: allow cross-subvolume BTRFS_IOC_CLONE

2012-01-06 Thread David Sterba
On Fri, Jan 06, 2012 at 02:04:12PM +0200, Konstantinos Skarlatos wrote: Me too wants cp --reflink across subvolumes. Please make this feature available to us, as its a poor man's dedupe and would give big space savings for many use cases. The simple case of 'cp --reflink' works fine, the

[PATCH v3] btrfs-progs: Add ioctl to read compressed size of a file

2012-01-06 Thread David Sterba
Signed-off-by: David Sterba dste...@suse.cz --- v2-v3: manpage updated, added ... around file in command description btrfs.c|9 ++- btrfs_cmds.c | 68 btrfs_cmds.h |1 + ioctl.h| 13 ++

Re: Honest timeline for btrfsck

2012-01-06 Thread Danny Piccirillo
Chris Mason chris.mason at oracle.com writes: So over the next two weeks I'm juggling the merge window and the fsck release. My goal is to demo fsck at linuxcon europe. Thanks again for all of your patience and help with Btrfs! So we have a lot of new features which is awesome but still

Re: Btrfs partition lost after RAID1 mirror disk failure?

2012-01-06 Thread C Anthony Risinger
On Wed, Jan 4, 2012 at 2:30 PM, Dan Garton dan.gar...@gmail.com wrote: Assuming that this is the case, do I stand a chance of retrieving that volume and accessing that data again? Or does destructive imply total loss? (In which case, I'll cut my losses) unfortunately i really don't know