[PATCH 1/3] Btrfs: do not log extents when we only log new names

2012-11-01 Thread Liu Bo
When we log new names, we need to log just enough to recreate the inode during log replay, and there is no need to log extents along with it. This actually fixes a bug revealed by xfstests 241, where it shows that we're logging some extents that have not updated metadata, so we don't get proper

[PATCH 2/3] Btrfs: get right arguments for btrfs_wait_ordered_range

2012-11-01 Thread Liu Bo
btrfs_wait_ordered_range expects for 'len' instead of 'end'. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/file.c |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index 9ab1bed..d2df981 100644 --- a/fs/btrfs/file.c +++

[PATCH 3/3] Btrfs: cleanup for btrfs_wait_order_range

2012-11-01 Thread Liu Bo
Variable 'found' is no more used. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/ordered-data.c |3 --- 1 files changed, 0 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/ordered-data.c b/fs/btrfs/ordered-data.c index 7772f02..7f75bea 100644 --- a/fs/btrfs/ordered-data.c +++

kernel BUG at fs/btrfs/ctree.c:2950

2012-11-01 Thread Darrick J. Wong
Hi, I have a torture test[1] that I run to test stable page writeback. When I run it against a btrfs (3.7.0-rc3) I observe the kernel bug messsage[2] that I've attached at the end of this message. The test program spawns a bunch of threads, which try to rewrite file blocks either through mmap

Re: kernel BUG at fs/btrfs/ctree.c:2950

2012-11-01 Thread Miao Xie
On thu, 1 Nov 2012 00:04:13 -0700, Darrick J. Wong wrote: Hi, I have a torture test[1] that I run to test stable page writeback. When I run it against a btrfs (3.7.0-rc3) I observe the kernel bug messsage[2] that I've attached at the end of this message. The test program spawns a bunch of

[PATCH 1/5] Btrfs: fix joining the same transaction handler more than 2 times

2012-11-01 Thread Miao Xie
If we flush inodes with pending delalloc in a transaction, we may join the same transaction handler more than 2 times. The reason is: Task use_count of trans handle commit_transaction1 |- btrfs_start_delalloc_inodes

[PATCH 2/5] Btrfs: fix missing flush when committing a transaction

2012-11-01 Thread Miao Xie
Consider the following case: Task1 Task2 start_transaction commit_transaction check pending snapshots list and the list is empty.

[PATCH 3/5] Btrfs: fix wrong file extent length

2012-11-01 Thread Miao Xie
There are two types of the file extent - inline extent and regular extent, When we log file extents, we didn't take inline extent into account, fix it. Signed-off-by: Miao Xie mi...@cn.fujitsu.com --- fs/btrfs/ctree.h |1 + fs/btrfs/file-item.c | 21 -

[PATCH 4/5] Btrfs: fix unprotected extent map operation when logging file extents

2012-11-01 Thread Miao Xie
We forget to protect the modified_extents list, fix it. Signed-off-by: Miao Xie mi...@cn.fujitsu.com --- fs/btrfs/tree-log.c |2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c index cbb544e..f7e9387 100644 --- a/fs/btrfs/tree-log.c

[PATCH 5/5] Btrfs: fix missing log when BTRFS_INODE_NEEDS_FULL_SYNC is set

2012-11-01 Thread Miao Xie
If we set BTRFS_INODE_NEEDS_FULL_SYNC, we should log all the extent, but now we forget to take it into account, and set a wrong max key, if so, we will skip the file extent metadata when doing logging. Fix it. Signed-off-by: Miao Xie mi...@cn.fujitsu.com --- fs/btrfs/tree-log.c |5 - 1

Re: [PATCH 0/3] flush delalloc by multi-task

2012-11-01 Thread Miao Xie
Hi, Josef Please drop this patchset from your btrfs-next tree because it may cause the performance regression in some cases. I'll improve it later. Thanks Miao On thu, 25 Oct 2012 17:20:29 +0800, Miao Xie wrote: This patchset introduce multi-task delalloc flush, it can make the delalloc flush

Re: [PATCH 2/5] Btrfs: fix missing flush when committing a transaction

2012-11-01 Thread Liu Bo
(sorry, forgot to cc linux-btrfs.) On Thu, Nov 01, 2012 at 03:51:41PM +0800, Miao Xie wrote: On Thu, 1 Nov 2012 15:44:43 +0800, Liu Bo wrote: On Thu, Nov 01, 2012 at 03:33:14PM +0800, Miao Xie wrote: Consider the following case: Task1 Task2

Re: [PATCH 3/5] Btrfs: fix wrong file extent length

2012-11-01 Thread Liu Bo
On Thu, Nov 01, 2012 at 03:33:59PM +0800, Miao Xie wrote: There are two types of the file extent - inline extent and regular extent, When we log file extents, we didn't take inline extent into account, fix it. Good catch. Reviewed-by: Liu Bo bo.li@oracle.com thanks, liubo

Re: [PATCH 4/5] Btrfs: fix unprotected extent map operation when logging file extents

2012-11-01 Thread Liu Bo
On Thu, Nov 01, 2012 at 03:34:54PM +0800, Miao Xie wrote: We forget to protect the modified_extents list, fix it. Looks good to me. Reviewed-by: Liu Bo bo.li@oracle.com thanks, liubo Signed-off-by: Miao Xie mi...@cn.fujitsu.com --- fs/btrfs/tree-log.c |2 ++ 1 files changed, 2

Re: [PATCH 2/5] Btrfs: fix missing flush when committing a transaction

2012-11-01 Thread Miao Xie
On thu, 1 Nov 2012 16:04:27 +0800, Liu Bo wrote: (sorry, forgot to cc linux-btrfs.) On Thu, Nov 01, 2012 at 03:51:41PM +0800, Miao Xie wrote: On Thu, 1 Nov 2012 15:44:43 +0800, Liu Bo wrote: On Thu, Nov 01, 2012 at 03:33:14PM +0800, Miao Xie wrote: Consider the following case: Task1

Re: [PATCH 2/5] Btrfs: fix missing flush when committing a transaction

2012-11-01 Thread Liu Bo
On Thu, Nov 01, 2012 at 04:16:43PM +0800, Miao Xie wrote: On thu, 1 Nov 2012 16:04:27 +0800, Liu Bo wrote: (sorry, forgot to cc linux-btrfs.) On Thu, Nov 01, 2012 at 03:51:41PM +0800, Miao Xie wrote: On Thu, 1 Nov 2012 15:44:43 +0800, Liu Bo wrote: On Thu, Nov 01, 2012 at 03:33:14PM

Re: [PATCH 5/5] Btrfs: fix missing log when BTRFS_INODE_NEEDS_FULL_SYNC is set

2012-11-01 Thread Liu Bo
On Thu, Nov 01, 2012 at 03:35:23PM +0800, Miao Xie wrote: If we set BTRFS_INODE_NEEDS_FULL_SYNC, we should log all the extent, but now we forget to take it into account, and set a wrong max key, if so, we will skip the file extent metadata when doing logging. Fix it. But it's along with

[PATCH] shared compression workspaces limits doesnot match

2012-11-01 Thread Rock Lee
In function find_workspace, it's allowed to alloc cpus + 1 workspaces at most, but in function free_workspace, it will freed the workspace if there exists more then cpus' workspaces. The two limits doesn't match. I thought the original itention is allowed to alloc cpus compression workspaces at

[Request for review] [RFC] Add label support for snapshots and subvols

2012-11-01 Thread Anand jain
From: Anand Jain anand.j...@oracle.com (This patch is for the review/test not yet for the integration). Here is an implementation of the feature to add label to the subvolume and snapshots. Which would help sysadmin to better manager the subvol and snapshots. This can be done in two ways,

[PATCH 2/2] Btrfs-progs: add feature to label subvol and snapshot

2012-11-01 Thread Anand jain
From: Anand Jain anand.j...@oracle.com Signed-off-by: Anand Jain anand.j...@oracle.com --- btrfslabel.c | 40 btrfslabel.h |4 +++- cmds-filesystem.c | 34 +- ioctl.h |2 ++ 4 files changed,

[PATCH 1/2] Btrfs-progs: move open_file_or_dir() to utils.c

2012-11-01 Thread Anand jain
From: Anand Jain anand.j...@oracle.com The definition of the function open_file_or_dir() is moved from common.c to utils.c in order to be able to share some common code between scrub and the device stats in the following step. That common code uses open_file_or_dir(). Since open_file_or_dir()

[PATCH] Btrfs: add label to snapshot and subvol

2012-11-01 Thread Anand jain
From: Anand Jain anand.j...@oracle.com This modifies the struct btrfs_root_item to hold the label, and make it v3 of this structure. Signed-off-by: Anand Jain anand.j...@oracle.com --- fs/btrfs/ctree.h | 14 ++ fs/btrfs/ioctl.c | 32

Re: [PATCH 2/5] Btrfs: fix missing flush when committing a transaction

2012-11-01 Thread Miao Xie
On Thu, 1 Nov 2012 17:00:00 +0800, Liu Bo wrote: On Thu, Nov 01, 2012 at 04:16:43PM +0800, Miao Xie wrote: On thu, 1 Nov 2012 16:04:27 +0800, Liu Bo wrote: (sorry, forgot to cc linux-btrfs.) On Thu, Nov 01, 2012 at 03:51:41PM +0800, Miao Xie wrote: On Thu, 1 Nov 2012 15:44:43 +0800, Liu Bo

Re: Need help mounting laptop corrupted root btrfs. Kernel BUG at fs/btrfs/volumes.c:3707 - FIXED

2012-11-01 Thread Sander
Marc MERLIN wrote (ao): That said, it's working fine again for now after I went back to kernel 3.5.3 (down from 3.6.3). It hasn't been long enough to say for sure, but there is a remote possibility that changes in 3.6 actually caused my drive to freeze after several hours of use. When that

Re: find-new possibility of showing modified and deleted files/directories

2012-11-01 Thread Gabriel
On Thu, 01 Nov 2012 06:06:57 +0100, Arne Jansen wrote: On 11/01/2012 02:28 AM, Shane Spencer wrote: That's Plan B. I'll be making a btrfs stream decoder and doing in place edits. I need to move stuff around to other filesystem types otherwise I'd just store the stream or apply the stream to

Re: [PATCH 1/2 v4] Btrfs: snapshot-aware defrag

2012-11-01 Thread Itaru Kitayama
Hi Liubo, I couldn't apply your V4 patch against the btrfs-next HEAD. Do you have a github branch which I can checkout? Thanks, Itaru On Wed, Oct 31, 2012 at 9:55 PM, Liu Bo bo.li@oracle.com wrote: On 10/31/2012 08:13 PM, Itaru Kitayama wrote: Hi LiuBo: I am seeing another warning with

Re: [PATCH 1/2 v4] Btrfs: snapshot-aware defrag

2012-11-01 Thread Liu Bo
On Thu, Nov 01, 2012 at 08:08:52PM +0900, Itaru Kitayama wrote: Hi Liubo, I couldn't apply your V4 patch against the btrfs-next HEAD. Do you have a github branch which I can checkout? The current btrfs-next HEAD actually have included this v4 patch, so just pull btrfs-next and give it a

Re: find-new possibility of showing modified and deleted files/directories

2012-11-01 Thread Arne Jansen
On 01.11.2012 12:00, Gabriel wrote: On Thu, 01 Nov 2012 06:06:57 +0100, Arne Jansen wrote: On 11/01/2012 02:28 AM, Shane Spencer wrote: That's Plan B. I'll be making a btrfs stream decoder and doing in place edits. I need to move stuff around to other filesystem types otherwise I'd just

Re: Crashes in extent_io.c after btrfs bad mapping eb notice

2012-11-01 Thread Jan Schmidt
On Thu, November 01, 2012 at 03:39 (+0100), Liu Bo wrote: On 11/01/2012 04:00 AM, Franke wrote: Hi, since yesterday I have run a balance while asleep/at work. Now I experimented a bit, and the situation has changed. I am now getting hard hangs ( system is gone without even writing anything

[PATCH] Btrfs-progs: check block group used count and fix if specified

2012-11-01 Thread Josef Bacik
A user reported a problem where all of his block groups had invalid used counts in the block group item. This patch walks the extent tree and counts up the used amount for each block group. If the user specifies repair we can set the correct used value and when the transaction commits we're all

no space left on device.

2012-11-01 Thread Kenneth Johansson
So I have ended up in a state where I can't delete files with rm. the error I get is no space on device. however I'm not even close to empty. /dev/sdb1 38G 27G 9.5G 75% there is about 800k files/dirs in this filesystem extra strange is that I can in another directory create and delete files.

Re: [PATCH] Btrfs-progs: check block group used count and fix if specified

2012-11-01 Thread Chris Mason
On Thu, Nov 01, 2012 at 06:34:54AM -0600, Josef Bacik wrote: A user reported a problem where all of his block groups had invalid used counts in the block group item. This patch walks the extent tree and counts up the used amount for each block group. If the user specifies repair we can set

Re: [PATCH 2/2] Btrfs: make snapshot-aware defrag as a mount option

2012-11-01 Thread Chris Mason
On Sat, Oct 27, 2012 at 04:28:41AM -0600, Liu Bo wrote: This feature works on our crucial write endio path, so if we've got lots of fragments to process, it will be kind of a disaster to the performance, so I make such a change. One can benifit from it while mounting with '-o

[PATCH 1/2] Btrfs-progs: correcting misnamed parameter options for btrfs send

2012-11-01 Thread Jan Schmidt
Unfortunately, the command line options for btrfs send were misnamed. To specify a base for an incremental snapshot transfer, the best choice is -i for incremental (was: -p). To optionally add snapshots existing on the receiver as clone sources, the best choice is -c (was: -i). Compatibily note:

[PATCH 0/2] Btrfs-progs: urgent fixes for btrfs send

2012-11-01 Thread Jan Schmidt
Hi everybody, We made a bad mistake with btrfs send command line arguments and we'd better fix it before it's being widely used (read: *now*). When using btrfs send as in the current master, the -i option does *not* give you an incremental stream as you'd expect. There are two problems in the

[PATCH 2/2] Btrfs-progs: bugfix for subvolume parent determination in btrfs send

2012-11-01 Thread Jan Schmidt
We missed to add the default subvolume, because it has no ROOT_BACKREF_ITEM. This made get_parent always fail for direct decendants of the default subvolume, resulting in lots of full streams where incremental streams were requested. Signed-off-by: Jan Schmidt list.bt...@jan-o-sch.net

Re: find-new possibility of showing modified and deleted files/directories

2012-11-01 Thread Gabriel
On Thu, 01 Nov 2012 12:29:36 +0100, Arne Jansen wrote: On 01.11.2012 12:00, Gabriel wrote: On Thu, 01 Nov 2012 06:06:57 +0100, Arne Jansen wrote: On 11/01/2012 02:28 AM, Shane Spencer wrote: That's Plan B. I'll be making a btrfs stream decoder and doing in place edits. I need to move stuff

Re: [PATCH 0/2] Btrfs-progs: urgent fixes for btrfs send

2012-11-01 Thread Chris Mason
On Thu, Nov 01, 2012 at 09:01:24AM -0600, Jan Schmidt wrote: Hi everybody, We made a bad mistake with btrfs send command line arguments and we'd better fix it before it's being widely used (read: *now*). Ok, I do agree that -i was confusing. I didn't end up using it in my backup scripts

Re: [PATCH 0/2] Btrfs-progs: urgent fixes for btrfs send

2012-11-01 Thread Jan Schmidt
On Thu, November 01, 2012 at 16:07 (+0100), Chris Mason wrote: On Thu, Nov 01, 2012 at 09:01:24AM -0600, Jan Schmidt wrote: Hi everybody, We made a bad mistake with btrfs send command line arguments and we'd better fix it before it's being widely used (read: *now*). Ok, I do agree that -i

Re: [PATCH 2/2] Btrfs: make snapshot-aware defrag as a mount option

2012-11-01 Thread Liu Bo
On 11/01/2012 10:43 PM, Chris Mason wrote: On Sat, Oct 27, 2012 at 04:28:41AM -0600, Liu Bo wrote: This feature works on our crucial write endio path, so if we've got lots of fragments to process, it will be kind of a disaster to the performance, so I make such a change. One can benifit from

Re: Crashes in extent_io.c after btrfs bad mapping eb notice

2012-11-01 Thread Jan Schmidt
On Thu, November 01, 2012 at 12:57 (+0100), Jan Schmidt wrote: I'm trying to reproduce the problems in the meantime. Looks like it worked :-/ And it also looks like it can either bug or deadlock, depending on the things going on in the kernel at the same time. I did a parallel fsmark on a

Re: [PATCH 1/2 v4] Btrfs: snapshot-aware defrag

2012-11-01 Thread Liu Bo
On 11/01/2012 10:05 PM, Itaru Kitayama wrote: Hi Liubo: The V4 leaves only warnings from btrfs_destroy_inode(). So, you think it's normal an old extent recorded can be removed from the extent tree by the time relink_file_extents() invoked? Yeah, it could be if only we run delayed refs in

Re: Need help mounting laptop corrupted root btrfs. Kernel BUG at fs/btrfs/volumes.c:3707 - FIXED

2012-11-01 Thread Marc MERLIN
On Thu, Nov 01, 2012 at 11:56:18AM +0100, Sander wrote: For now, I'll stick with 3.5.3 for a while to make sure my drive is actually ok (it seems to be afterall), and once I'm happy that it's the case, I'll go back to 3.6.3 with serial console remote logging and try to capture the full

Re: find-new possibility of showing modified and deleted files/directories

2012-11-01 Thread Shane Spencer
On Wed, Oct 31, 2012 at 9:06 PM, Arne Jansen sensi...@gmx.net wrote: On 11/01/2012 02:28 AM, Shane Spencer wrote: That's Plan B. I'll be making a btrfs stream decoder and doing in place edits. I need to move stuff around to other filesystem types otherwise I'd just store the stream or

Re: [PATCH 0/2] Btrfs-progs: urgent fixes for btrfs send

2012-11-01 Thread Chris Mason
On Thu, Nov 01, 2012 at 09:48:25AM -0600, Jan Schmidt wrote: On Thu, November 01, 2012 at 16:07 (+0100), Chris Mason wrote: On Thu, Nov 01, 2012 at 09:01:24AM -0600, Jan Schmidt wrote: Hi everybody, We made a bad mistake with btrfs send command line arguments and we'd better fix it

[PATCH] Btrfs: use radix tree tagging to keep track of dirty ebs

2012-11-01 Thread Josef Bacik
Currently we set dirty bits in the transactions dirty eb's in order to know what we have to write out on transaction commit. This means for every eb we allocate we have to allocate the corresponding extent state in the dirty pages tree. We also only change this tree on commit, so we could end up

Re: [Request for review] [RFC] Add label support for snapshots and subvols

2012-11-01 Thread cwillu
Below is a demo of this new feature. btrfs fi label -t /btrfs/sv1 Prod-DB btrfs fi label -t /btrfs/sv1 Prod-DB btrfs su snap /btrfs/sv1 /btrfs/snap1-sv1 Create a snapshot of '/btrfs/sv1' in '/btrfs/snap1-sv1' btrfs fi label -t /btrfs/snap1-sv1 btrfs fi label -t

Re: [Request for review] [RFC] Add label support for snapshots and subvols

2012-11-01 Thread Fajar A. Nugraha
On Fri, Nov 2, 2012 at 5:16 AM, cwillu cwi...@cwillu.com wrote: btrfs fi label -t /btrfs/snap1-sv1 Prod-DB-sand-box-testing Why is this better than: # btrfs su snap /btrfs/Prod-DB /btrfs/Prod-DB-sand-box-testing # mv /btrfs/Prod-DB-sand-box-testing /btrfs/Prod-DB-production-test # ls

Re: [Request for review] [RFC] Add label support for snapshots and subvols

2012-11-01 Thread Hugo Mills
On Fri, Nov 02, 2012 at 05:28:01AM +0700, Fajar A. Nugraha wrote: On Fri, Nov 2, 2012 at 5:16 AM, cwillu cwi...@cwillu.com wrote: btrfs fi label -t /btrfs/snap1-sv1 Prod-DB-sand-box-testing Why is this better than: # btrfs su snap /btrfs/Prod-DB /btrfs/Prod-DB-sand-box-testing # mv

Re: [Request for review] [RFC] Add label support for snapshots and subvols

2012-11-01 Thread Fajar A. Nugraha
On Fri, Nov 2, 2012 at 5:32 AM, Hugo Mills h...@carfax.org.uk wrote: On Fri, Nov 02, 2012 at 05:28:01AM +0700, Fajar A. Nugraha wrote: On Fri, Nov 2, 2012 at 5:16 AM, cwillu cwi...@cwillu.com wrote: btrfs fi label -t /btrfs/snap1-sv1 Prod-DB-sand-box-testing Why is this better than:

Re: [Request for review] [RFC] Add label support for snapshots and subvols

2012-11-01 Thread Goffredo Baroncelli
On 11/01/2012 11:28 PM, Fajar A. Nugraha wrote: On Fri, Nov 2, 2012 at 5:16 AM, cwillu cwi...@cwillu.com wrote: btrfs fi label -t /btrfs/snap1-sv1 Prod-DB-sand-box-testing Why is this better than: # btrfs su snap /btrfs/Prod-DB /btrfs/Prod-DB-sand-box-testing # mv

Re: [PATCH 5/5] Btrfs: fix missing log when BTRFS_INODE_NEEDS_FULL_SYNC is set

2012-11-01 Thread Miao Xie
On thu, 1 Nov 2012 17:21:12 +0800, Liu Bo wrote: On Thu, Nov 01, 2012 at 03:35:23PM +0800, Miao Xie wrote: If we set BTRFS_INODE_NEEDS_FULL_SYNC, we should log all the extent, but now we forget to take it into account, and set a wrong max key, if so, we will skip the file extent metadata