Re: [PATCH V4 07/12] Btrfs: fix corrupted metadata in the snapshot

2012-09-06 Thread Miao Xie
On Thu, 6 Sep 2012 09:09:14 -0400, Josef Bacik wrote: > On Thu, Sep 06, 2012 at 04:03:04AM -0600, Miao Xie wrote: >> When we delete a inode, we will remove all the delayed items including >> delayed >> inode update, and then truncate all the relative metadata. If there is lots >> of >> metadata,

Re: [PATCH] Btrfs: wait on async pages when shrinking delalloc

2012-09-06 Thread Mitch Harder
On Thu, Sep 6, 2012 at 3:51 PM, Josef Bacik wrote: > Mitch reported a problem where you could get an ENOSPC error when untarring > a kernel git tree onto a 16gb file system with compress-force=zlib. This is > because compression is a huge pain, it will return from ->writepages() > without having

Re: [PATCH] Btrfs: fix our overcommit math

2012-09-06 Thread Zach Brown
> + used = space_info->bytes_used + space_info->bytes_reserved + > + space_info->bytes_pinned + space_info->bytes_readonly + > + space_info->bytes_may_use; > + spin_lock(&root->fs_info->free_chunk_lock); > + avail = root->fs_info->free_chunk_space; Is th

Re: [PATCH] Btrfs: wait on async pages when shrinking delalloc

2012-09-06 Thread Zach Brown
> + if (atomic_read(&root->fs_info->async_delalloc_pages)) > + wait_event(root->fs_info->async_submit_wait, > + !atomic_read(&root->fs_info->async_delalloc_pages)); The very first thing wait_event_*() family does is test the condition. You don'

[PATCH] Btrfs: fix our overcommit math

2012-09-06 Thread Josef Bacik
I noticed I was seeing large lags when running my torrent test in a vm on my laptop. While trying to make it lag less I noticed that our overcommit math was taking into account the number of bytes we wanted to reclaim, not the number of bytes we actually wanted to allocate, which means we wouldn't

[PATCH] Btrfs: wait on async pages when shrinking delalloc

2012-09-06 Thread Josef Bacik
Mitch reported a problem where you could get an ENOSPC error when untarring a kernel git tree onto a 16gb file system with compress-force=zlib. This is because compression is a huge pain, it will return from ->writepages() without having actually created any ordered extents. To get around this we

Re: [PATCH 1/6] Btrfs: fix a bug of per-file nocow

2012-09-06 Thread David Sterba
On Fri, Aug 31, 2012 at 01:36:54PM +0800, Liu Bo wrote: > The new created file should not only inherit the NODATACOW flag, but also > honor NODATASUM flag, because we must do COW on a file extent with checksum. I was not entierly correct when suggesting to move NODATASUM to inherit_iflags. Se belo

Re: [PATCH v2 2/2] Btrfs: snapshot-aware defrag

2012-09-06 Thread Liu Bo
On Thu, Sep 06, 2012 at 09:11:53AM -0400, Josef Bacik wrote: > On Wed, Sep 05, 2012 at 07:10:52PM -0600, Liu Bo wrote: > > This comes from one of btrfs's project ideas, > > As we defragment files, we break any sharing from other snapshots. > > The balancing code will preserve the sharing, and defra

Re: [PATCH v2 2/2] Btrfs: snapshot-aware defrag

2012-09-06 Thread Liu Bo
On Thu, Sep 06, 2012 at 09:11:53AM -0400, Josef Bacik wrote: > On Wed, Sep 05, 2012 at 07:10:52PM -0600, Liu Bo wrote: > > This comes from one of btrfs's project ideas, > > As we defragment files, we break any sharing from other snapshots. > > The balancing code will preserve the sharing, and defra

Re: [PATCH 1/3] btrfs: remove unnecessary -ENOMEM BUG_ON check in extent-tree.c/exclude_super_stripes

2012-09-06 Thread Wang Sheng-Hui
On 2012年09月06日 18:09, David Sterba wrote: > On Thu, Sep 06, 2012 at 02:40:41PM +0800, Wang Sheng-Hui wrote: >> The memory allocation failure is BUG_ON in add_excluded_extent (following >> the code path) and btrfs_rmap_block. No need to BUG_ON -ENOMEM inside >> exclude_super_stripes itself. > > No

Re: Seagate Momentus XT and btrfs corruption

2012-09-06 Thread Ronnie Collinson
Since btrfs is still under heavy and constant development, it is recommended to run either the latest kernel release (3.5) or even better the latest rc (3.6-rc4), if you could try running one of these and see if the issue has allready been fixed, or is still an outstanding issue. Ronnie -- To unsu

Re: [PATCH v2 2/2] Btrfs: snapshot-aware defrag

2012-09-06 Thread Josef Bacik
On Wed, Sep 05, 2012 at 07:10:52PM -0600, Liu Bo wrote: > This comes from one of btrfs's project ideas, > As we defragment files, we break any sharing from other snapshots. > The balancing code will preserve the sharing, and defrag needs to grow this > as well. > > Now we're able to fill the blank

Re: [PATCH V4 07/12] Btrfs: fix corrupted metadata in the snapshot

2012-09-06 Thread Josef Bacik
On Thu, Sep 06, 2012 at 04:03:04AM -0600, Miao Xie wrote: > When we delete a inode, we will remove all the delayed items including delayed > inode update, and then truncate all the relative metadata. If there is lots of > metadata, we will end the current transaction, and start a new transaction to

[PATCH V4 03/12] Btrfs: fix file extent discount problem in the, snapshot

2012-09-06 Thread Miao Xie
If a snapshot is created while we are writing some data into the file, the i_size of the corresponding file in the snapshot will be wrong, it will be beyond the end of the last file extent. And btrfsck will report: root 256 inode 257 errors 100 Steps to reproduce: # mkfs.btrfs # mount # cd

[PATCH V3 7/7] Btrfs-progs: update the manpage entries for the btrfs subvolume list

2012-09-06 Thread Miao Xie
From: Zhou Bo This patch adds the introduction of the new option '-r' into the man page of 'btrfs subvolume list' command. Signed-off-by: Zhou Bo --- Sorry, forgot to add "From: ", so re-send this patch. Changelog v2 -> v3: - No change. Changelog v1 -> v2: - new patch --- man/btrfs.8.in

[PATCH V3 1/7] Btrfs-progs: move the function declarations to a new head file

2012-09-06 Thread Miao Xie
From: Zhou Bo Move the function declarations to a new head file. Signed-off-by: Zhou Bo --- Sorry, forgot to add "From: ", so re-send this patch. Changelog v2 - v3: - remove path_for_root() in send-utils.c Changelog v1 -> v2: - new patch. --- btrfs-list.c |1 + btrfs-list.h |

[PATCH V3 7/7] Btrfs-progs: update the manpage entries for the btrfs, subvolume list

2012-09-06 Thread Miao Xie
This patch adds the introduction of the new option '-r' into the man page of 'btrfs subvolume list' command. Signed-off-by: Zhou Bo --- Changelog v2 -> v3: - No change. Changelog v1 -> v2: - new patch --- man/btrfs.8.in |5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git

[PATCH V3 5/7] Btrfs-progs: restructure list_subvolumes

2012-09-06 Thread Miao Xie
The current code of list_subvols() has very bad scalability, if we want to add new filter conditions or new sort methods, we have to modify lots of code. Beside that, the most code of list_snapshots() is similar to list_subvols(), So I restructure list_subvols(), and split the subvolume filter fu

[PATCH V3 6/7] Btrfs-progs: enhance btrfs subvol list only to show read-only snapshots

2012-09-06 Thread Miao Xie
We want 'btrfs subvolume list' only to list readonly subvolumes, this patch set introduces a new option 'r' to implement it. You can use the command like that: btrfs subvolume list -r Original-Signed-off-by: Zhou Bo Signed-off-by: Miao Xie --- Changelog v2 -> v3: - re-implement this f

[PATCH V3 4/7] Btrfs-progs: fix wrong way to check if the root item contains otime and uuid

2012-09-06 Thread Miao Xie
Now we check if the root item contains otime and uuid or not by comparing ->generation_v2 and ->generation of the btrfs_root_item structure, it is wrong because it is possbile that ->generation may equal to the first variant of the next item. We fix this problem by comparing the size of btrfs_root_

[PATCH V3 3/7] Btrfs-progs: fix wrong usage of btrfs subvolume list command

2012-09-06 Thread Miao Xie
Since the uuid output function has been implemented, we should update the usage to tell the users. Signed-off-by: Miao Xie --- Changelog v1 -> v3: - new patch --- cmds-subvolume.c |3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/cmds-subvolume.c b/cmds-subvolume.c index

[PATCH V3 2/7] Btrfs-progs: fix compile warning of implicit declaration, of "list_snapshots"

2012-09-06 Thread Miao Xie
This patch fixes the following warning: cmds-subvolume.c:283:3: warning: implicit declaration of function "list_snapshots" Signed-off-by: Miao Xie --- Changelog v1 -> v3: - new patch. --- btrfs-list.h |1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/btrfs-list.h b/btrfs-

[PATCH V3 1/7] Btrfs-progs: move the function declarations to a new head file

2012-09-06 Thread Miao Xie
Move the function declarations to a new head file. Signed-off-by: Zhou Bo --- Changelog v2 - v3: - remove path_for_root() in send-utils.c Changelog v1 -> v2: - new patch. --- btrfs-list.c |1 + btrfs-list.h | 21 + cmds-inspect.c |4 +--- cmds-subvolume.c

[PATCH V3 0/7 ] Btrfs-progs: enhance btrfs subvol list only to show read-only snapshots

2012-09-06 Thread Miao Xie
We want 'btrfs subvolume list' only to list readonly subvolumes, this patch set introduces a new option 'r' to implement it. You can use the command like that: btrfs subvolume list -r Changelog v2 -> v3: - re-implement list_subvols() - re-implement this read-only subvolume list function

[PATCH V4 0/12] random bug fixes

2012-09-06 Thread Miao Xie
This patchset contains 12 bug-fix patches. - 01-03, 05-09 fix some bugs of the snapshot creation. - 04 just improves the memory allocation of the ordered data extent object. - 10 fixes a bug of the tree log that we forgot to protect ->log_batch. - 11 improve error path handle, makes it show more in

[PATCH V4 04/12] Btrfs: use a slab for ordered extents allocation

2012-09-06 Thread Miao Xie
The ordered extent allocation is in the fast path of the IO, so use a slab to improve the speed of the allocation. "Size of the struct is 280, so this will fall into the size-512 bucket, giving 8 objects per page, while own slab will pack 14 objects into a page. Another benefit I see is to c

[PATCH V4 02/12] Btrfs: fix full backref problem when inserting shared block reference

2012-09-06 Thread Miao Xie
If we create several snapshots at the same time, the following BUG_ON() will be triggered. kernel BUG at fs/btrfs/extent-tree.c:6047! Steps to reproduce: # mkfs.btrfs # mount # cd # for ((i=0;i<2400;i++)); do touch long_name_to_make_tree_more_deep$i; done # for ((i=0; i<4; i++))

[PATCH V4 07/12] Btrfs: fix corrupted metadata in the snapshot

2012-09-06 Thread Miao Xie
When we delete a inode, we will remove all the delayed items including delayed inode update, and then truncate all the relative metadata. If there is lots of metadata, we will end the current transaction, and start a new transaction to truncate the left metadata. In this way, we will leave a inode

[PATCH V4 10/12] Btrfs: fix unprotected ->log_batch

2012-09-06 Thread Miao Xie
We forget to protect ->log_batch when syncing a file, this patch fix this problem by atomic operation. And ->log_batch is used to check if there are parallel sync operations or not, so it is unnecessary to reset it to 0 after the sync operation of the current log tree complete. Signed-off-by: Miao

[PATCH V4 11/12] Btrfs: output more information when aborting a unused transaction handle

2012-09-06 Thread Miao Xie
Though we dump the stack information when aborting a unused transaction handle, we don't know the correct place where we decide to abort the transaction handle if one function has several place where the transaction abort function is invoked and jumps to the same place after this call. And beside t

[PATCH V4 08/12] Btrfs: fix the snapshot that should not exist

2012-09-06 Thread Miao Xie
The snapshot should be the image of the fs tree before it was created, so the metadata of the snapshot should not exist in the its tree. But now, we found the directory item and directory name index is in both the snapshot tree and the fs tree. It introduces some problems and makes the users feel s

[PATCH V4 06/12] Btrfs: add a new "type" field into the block reservation structure

2012-09-06 Thread Miao Xie
Sometimes we need choose the method of the reservation according to the type of the block reservation, such as the reservation for the delayed inode update. Now we identify the type just by comparing the address of the reservation variants, it is very ugly if it is a temporary one because we need c

[PATCH V4 01/12] Btrfs: fix error path in create_pending_snapshot()

2012-09-06 Thread Miao Xie
This patch fixes the following problem: - If we failed to deal with the delayed dir items, we should abort transaction, just as its comment said. Fix it. - If root reference or root back reference insertion failed, we should abort transaction. Fix it. - Fix the double free problem of pending->i

[PATCH V4 09/12] Btrfs: fix wrong size for the reservation of the, snapshot creation

2012-09-06 Thread Miao Xie
We should insert/update 6 items(root ref, root backref, dir item, dir index, root item and parent inode) when creating a snapshot, not 5 items, fix it. Signed-off-by: Miao Xie --- Changelog v1 -> v4: - new patch. --- fs/btrfs/extent-tree.c |6 +++--- fs/btrfs/ioctl.c |2 +- 2 files

[PATCH V4 12/12] Btrfs: fix wrong size for the reservation when doing, file pre-allocation.

2012-09-06 Thread Miao Xie
When we ran fsstress(a program in xfstests), the filesystem hung up when it is full. It was because the space reserved in btrfs_fallocate() was wrong, btrfs_fallocate() just used the size of the pre-allocation to reserve the space, didn't took the block size aligning into account, so the size of th

[PATCH V4 05/12] Btrfs: fix wrong orphan count of the fs/file tree

2012-09-06 Thread Miao Xie
If we add a new orphan item, we should increase the atomic counter, not decrease it. Fix it. Signed-off-by: Miao Xie --- Changelog v1 -> v4: - No change. --- fs/btrfs/inode.c |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 6ba80

Re: [PATCH 2/3] btrfs: remove unnecessary -ENOMEM BUG_ON check in extent-tree.c/btrfs_alloc_logged_file_extent

2012-09-06 Thread David Sterba
On Thu, Sep 06, 2012 at 02:41:02PM +0800, Wang Sheng-Hui wrote: > The memory allocation failure is BUG_ON in add_excluded_extent > (following the code path). No need to BUG_ON -ENOMEM inside > btrfs_alloc_logged_file_extent. This indirectly calls __set_extent_bit that does BUG_ON on memory allocat

Re: [PATCH 1/3] btrfs: remove unnecessary -ENOMEM BUG_ON check in extent-tree.c/exclude_super_stripes

2012-09-06 Thread David Sterba
On Thu, Sep 06, 2012 at 02:40:41PM +0800, Wang Sheng-Hui wrote: > The memory allocation failure is BUG_ON in add_excluded_extent (following > the code path) and btrfs_rmap_block. No need to BUG_ON -ENOMEM inside > exclude_super_stripes itself. No please. > Its return value is always 0, and useles

[PATCH] Btrfs: fix error handling in delete_block_group_cache()

2012-09-06 Thread Tsutomu Itoh
btrfs_iget() never return NULL. So, NULL check is unnecessary. Signed-off-by: Tsutomu Itoh --- fs/btrfs/relocation.c |4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c index c5dbd91..738e869 100644 --- a/fs/btrfs/relocation.c