Re: [PATCH v3 7/7] Btrfs: incremental send, avoid the overhead of allocating an orphan_dir_info object unnecessarily

2015-06-23 Thread David Sterba
On Tue, Jun 23, 2015 at 06:39:51PM +0800, Robbie Ko wrote: Avoid the overhead of allocating an orphan_dir_info object unnecessarily. Signed-off-by: Robbie Ko robbi...@synology.com Reviewed-by: David Sterba dste...@suse.cz -- To unsubscribe from this list: send the line unsubscribe linux-btrfs

Re: [PATCH 3/5] btrfs: fix clone / extent-same deadlocks

2015-06-23 Thread David Sterba
On Mon, Jun 22, 2015 at 03:47:40PM -0700, Mark Fasheh wrote: Clone and extent same lock their source and target inodes in opposite order. In addition to this, the range locking in clone doesn't take ordering into account. Fix this by having clone use the same locking helpers as

Re: [PATCH 5/5] btrfs: add no_mtime flag to btrfs-extent-same

2015-06-23 Thread David Sterba
On Mon, Jun 22, 2015 at 03:47:42PM -0700, Mark Fasheh wrote: --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -87,7 +87,8 @@ struct btrfs_ioctl_received_subvol_args_32 { static int btrfs_clone(struct inode *src, struct inode *inode, -u64 off, u64 olen, u64

Re: [PATCH v3 1/7] Revert Btrfs: incremental send, remove dead code

2015-06-23 Thread David Sterba
On Tue, Jun 23, 2015 at 04:09:07PM +0100, Filipe David Manana wrote: On Tue, Jun 23, 2015 at 11:39 AM, Robbie Ko robbi...@synology.com wrote: This reverts commit 5f806c3ae2ff6263a10a6901f97abb74dac03d36. This is a necessary patch for a subsequent patch in the series (patch 3). Still

Re: Btrfs progs release 4.1

2015-06-23 Thread David Sterba
On Tue, Jun 23, 2015 at 11:03:47AM +0200, Sjoerd wrote: On Monday 22 June 2015 17:00:23 David Sterba wrote: btrfs-progs 4.1 have been released (in time with kernel 4.1). Unusual load of changes. Would it be beneficial to update when using older kernels (3.19.x in my case)? I can't upgrade

Re: [PATCH 4/5] btrfs: allow dedupe of same inode

2015-06-23 Thread David Sterba
On Mon, Jun 22, 2015 at 03:47:41PM -0700, Mark Fasheh wrote: clone() supports cloning within an inode so extent-same can do the same now. This patch fixes up the locking in extent-same to know about the single-inode case. In addition to that, we add a check for overlapping ranges, which clone

Re: [PATCH v3 1/7] Revert Btrfs: incremental send, remove dead code

2015-06-23 Thread Filipe David Manana
On Tue, Jun 23, 2015 at 11:39 AM, Robbie Ko robbi...@synology.com wrote: This reverts commit 5f806c3ae2ff6263a10a6901f97abb74dac03d36. This is a necessary patch for a subsequent patch in the series (patch 3). Still confused. Reverting that commit (5f806c3ae2ff6263a10a6901f97abb74dac03d36)

Re: [PATCH v3 0/7] Btrfs incremental send fix serval case for rename and rm directory

2015-06-23 Thread Filipe David Manana
On Tue, Jun 23, 2015 at 11:39 AM, Robbie Ko robbi...@synology.com wrote: Patch for fix btrfs send receive. These patches base on v4.1 plus following patches. [PATCH] Btrfs: incremental send, don't delay directory renames unnecessarily [PATCH] Btrfs: incremental send, check if orphanized dir

Re: [PATCH v3 3/7] Btrfs: incremental send, avoid ancestor rename to descendant

2015-06-23 Thread Filipe David Manana
On Tue, Jun 23, 2015 at 11:39 AM, Robbie Ko robbi...@synology.com wrote: There's one more case where we can't issue a rename operation for a directory as soon as we process it. We move a directory from ancestor to descendant. | a | b | c | d Move a

Re: [PATCH] btrfs-progs: alias btrfs device delete to btrfs device remove

2015-06-23 Thread David Sterba
On Thu, Jun 18, 2015 at 03:07:09PM -0700, Omar Sandoval wrote: @@ -586,12 +606,13 @@ out: const struct cmd_group device_cmd_group = { device_cmd_group_usage, NULL, { { add, cmd_add_dev, cmd_add_dev_usage, NULL, 0 }, - { delete, cmd_rm_dev, cmd_rm_dev_usage,

Re: Btrfs progs release 4.1

2015-06-23 Thread David Sterba
On Mon, Jun 22, 2015 at 10:45:32PM +0200, Martin Steigerwald wrote: Wow, nice collection of changes! Thanks. Am Montag, 22. Juni 2015, 17:00:23 schrieb David Sterba: * new - rescure zero-log - btrfsune: - rewrite uuid on a filesystem image - new option to turn on NO_HOLES

Re: i_version vs iversion (Was: Re: [RFC PATCH v2 1/2] Btrfs: add noi_version option to disable MS_I_VERSION)

2015-06-23 Thread Theodore Ts'o
On Thu, Jun 18, 2015 at 04:38:56PM +0200, David Sterba wrote: Moving the discussion to fsdevel. Summary: disabling MS_I_VERSION brings some speedups to btrfs, but the generic 'noiversion' option cannot be used to achieve that. It is processed before it reaches btrfs superblock callback,

Re: Corrupted btrfs partition (converted from ext4) after balance

2015-06-23 Thread Chris Murphy
On Mon, Jun 22, 2015 at 7:44 PM, Qu Wenruo quwen...@cn.fujitsu.com wrote: Without the full output of btrfsck, it's quite hard to call it a clear bug report if you want to save your data in the corrupted partition. What I read was that he included everything btrfs check reported, which wasn't a

Re: [PATCH 5/5] btrfs: add no_mtime flag to btrfs-extent-same

2015-06-23 Thread Mark Fasheh
On Tue, Jun 23, 2015 at 05:11:56PM +0200, David Sterba wrote: On Mon, Jun 22, 2015 at 03:47:42PM -0700, Mark Fasheh wrote: --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -87,7 +87,8 @@ struct btrfs_ioctl_received_subvol_args_32 { static int btrfs_clone(struct inode *src, struct

[PATCH 4/5] btrfs: allow dedupe of same inode

2015-06-23 Thread Mark Fasheh
clone() supports cloning within an inode so extent-same can do the same now. This patch fixes up the locking in extent-same to know about the single-inode case. In addition to that, we add a check for overlapping ranges, which clone does not allow. Signed-off-by: Mark Fasheh mfas...@suse.de

[PATCH 5/5] btrfs: add no_mtime flag to btrfs-extent-same

2015-06-23 Thread Mark Fasheh
One issue users have reported is that dedupe changes mtime on files, resulting in tools like rsync thinking that their contents have changed when in fact the data is exactly the same. Clone still wants an mtime change, so we special case this in the code. With this patch an application can pass

Re: [PATCH] btrfs-progs: alias btrfs device delete to btrfs device remove

2015-06-23 Thread Omar Sandoval
On Tue, Jun 23, 2015 at 05:40:39PM +0200, David Sterba wrote: On Thu, Jun 18, 2015 at 03:07:09PM -0700, Omar Sandoval wrote: @@ -586,12 +606,13 @@ out: const struct cmd_group device_cmd_group = { device_cmd_group_usage, NULL, { { add, cmd_add_dev, cmd_add_dev_usage, NULL,

[PATCH 2/5] btrfs: fix deadlock with extent-same and readpage

2015-06-23 Thread Mark Fasheh
-readpage() does page_lock() before extent_lock(), we do the opposite in extent-same. We want to reverse the order in btrfs_extent_same() but it's not quite straightforward since the page locks are taken inside btrfs_cmp_data(). So I split btrfs_cmp_data() into 3 parts with a small context

[PATCH 1/5] btrfs: pass unaligned length to btrfs_cmp_data()

2015-06-23 Thread Mark Fasheh
In the case that we dedupe the tail of a file, we might expand the dedupe len out to the end of our last block. We don't want to compare data past i_size however, so pass the original length to btrfs_cmp_data(). Signed-off-by: Mark Fasheh mfas...@suse.de Reviewed-by: David Sterba dste...@suse.cz

[PATCH 3/5] btrfs: fix clone / extent-same deadlocks

2015-06-23 Thread Mark Fasheh
Clone and extent same lock their source and target inodes in opposite order. In addition to this, the range locking in clone doesn't take ordering into account. Fix this by having clone use the same locking helpers as btrfs-extent-same. In addition, I do a small cleanup of the locking helpers,

[PATCH 0/5] btrfs: dedupe fixes, features V3

2015-06-23 Thread Mark Fasheh
Hi Chris, The following patches are based on top of my patch titled btrfs: Handle unaligned length in extent_same which you have in your 'integration-4.2' branch: https://git.kernel.org/cgit/linux/kernel/git/mason/linux-btrfs.git/commit/?id=e1d227a42ea2b4664f94212bd1106b9a3413ffb8 The series

Re: [PATCH 1/1] Btrfs: Remove redundant NULL check before kfree

2015-06-23 Thread David Sterba
On Tue, Jun 23, 2015 at 11:28:00AM +0530, Maninder Singh wrote: There is no need of NULL check before kfree, removing the same Signed-off-by: Maninder Singh maninder...@samsung.com Reviewed-by: Akhilesh Kumar akhiles...@samsung.com Reviewed-by: David Sterba dste...@suse.cz -- To unsubscribe

Re: counting fragments takes more time than defragmenting

2015-06-23 Thread Marc MERLIN
Hello again, Just curious, is anyone seeing similar things with big VM images or other DBs? I forgot to mention that my vdi file is 88GB. It's surprising that it took longer to count the fragments than to actually defragment the file. Or that it took 3 defrag runs to get down to 11K extents from

Re: Question about btrfs send/receive

2015-06-23 Thread Marc MERLIN
On Tue, Jun 23, 2015 at 03:15:23PM +0800, Cao jin wrote: Hi all, I am considering implement the btrfs send/receive command in libguestfs(http://libguestfs.org/), and found that incremental send seems pretty complicated. So my questions is: 1. What is the most usually usage of btrfs

Re: Question about btrfs send/receive

2015-06-23 Thread Cao jin
Hi Marc, I can`t access /perso/btrfs/2014-03.html on that server. But I guess it has similar content as https://btrfs.wiki.kernel.org/index.php/Incremental_Backup ? If so, I am afraid incremental backup not suitable for a API implementation in libguestfs. Is the simplest form a

Re: [PATCH v2] fstests: shared test for fsync after adding xattrs to a file

2015-06-23 Thread Eryu Guan
On Sat, Jun 20, 2015 at 12:45:01AM +0100, fdman...@kernel.org wrote: From: Filipe Manana fdman...@suse.com This test is motivated by an issue found in btrfs. It test that after syncing the filesystem, adding many xattrs to a file, syncing the filesystem again, writing to the file and then

Re: Btrfs progs release 4.1

2015-06-23 Thread Sjoerd
On Monday 22 June 2015 17:00:23 David Sterba wrote: Hi, btrfs-progs 4.1 have been released (in time with kernel 4.1). Unusual load of changes. Would it be beneficial to update when using older kernels (3.19.x in my case)? I can't upgrade to kernel 4.1 since my nvidia (propriety) drivers

Re: Btrfs progs release 4.1

2015-06-23 Thread Qu Wenruo
Sjoerd wrote on 2015/06/23 11:03 +0200: On Monday 22 June 2015 17:00:23 David Sterba wrote: Hi, btrfs-progs 4.1 have been released (in time with kernel 4.1). Unusual load of changes. Would it be beneficial to update when using older kernels (3.19.x in my case)? Part(most) of. Some one

Question about btrfs send/receive

2015-06-23 Thread Cao jin
Hi all, I am considering implement the btrfs send/receive command in libguestfs(http://libguestfs.org/), and found that incremental send seems pretty complicated. So my questions is: 1. What is the most usually usage of btrfs send/receive? 2. Does it make sense to implement the simplest

Re: [RFC PATCH V11 01/21] Btrfs: subpagesize-blocksize: Fix whole page read.

2015-06-23 Thread Liu Bo
On Fri, Jun 19, 2015 at 03:15:01PM +0530, Chandan Rajendra wrote: On Friday 19 Jun 2015 12:45:37 Liu Bo wrote: On Mon, Jun 01, 2015 at 08:52:36PM +0530, Chandan Rajendra wrote: For the subpagesize-blocksize scenario, a page can contain multiple blocks. In such cases, this patch handles

[PATCH v3 5/7] Btrfs: incremental send, fix rmdir but dir have a unprocess item

2015-06-23 Thread Robbie Ko
There's one case where we attempt to rmdir a directory prematurely. Example: Parent snapshot: | a/ (ino 279) | c (ino 282) | del/ (ino 281) | tmp/ (ino 280) | long/ (ino 283) Send snapshot: | a/ (ino 279) | long (ino 283) | c/ (ino 282) |

[PATCH v3 7/7] Btrfs: incremental send, avoid the overhead of allocating an orphan_dir_info object unnecessarily

2015-06-23 Thread Robbie Ko
Avoid the overhead of allocating an orphan_dir_info object unnecessarily. Signed-off-by: Robbie Ko robbi...@synology.com --- fs/btrfs/send.c | 13 ++--- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c index dcf384d..7c61365 100644 ---

[PATCH v3 4/7] Btrfs: incremental send, fix orphan_dir_info leak

2015-06-23 Thread Robbie Ko
There's one case where we leak a orphan_dir_info structure. Example: Parent snapshot: | a/ (ino 279) | c (ino 282) | del/ (ino 281) | tmp/ (ino 280) | long/ (ino 283) | longlong/ (ino 284) Send snapshot: | a/ (ino 279) | long (ino 283)

[PATCH v3 6/7] Btrfs: incremental send, don't send utimes for non-existing directory

2015-06-23 Thread Robbie Ko
There's one where we attempt to get utimes from a directory that doesn't exist in the send snapshot. Example: Parent snapshot: | a/ (ino 259) | c (ino 264) | b/ (ino 260) | d (ino 265) | del/ (ino 263) | item1/ (ino 261) | item2/ (ino 262) Send

[PATCH v3 0/7] Btrfs incremental send fix serval case for rename and rm directory

2015-06-23 Thread Robbie Ko
Patch for fix btrfs send receive. These patches base on v4.1 plus following patches. [PATCH] Btrfs: incremental send, don't delay directory renames unnecessarily [PATCH] Btrfs: incremental send, check if orphanized dir inode needs delayed rename Thanks. Robbie Ko (7): Revert Btrfs:

[PATCH v3 3/7] Btrfs: incremental send, avoid ancestor rename to descendant

2015-06-23 Thread Robbie Ko
There's one more case where we can't issue a rename operation for a directory as soon as we process it. We move a directory from ancestor to descendant. | a | b | c | d Move a directory from ancestor to descendant means moving dir. a into dir. c This case

[PATCH v3 2/7] Btrfs: incremental send, avoid circular waiting and descendant overwrite ancestor need to update path

2015-06-23 Thread Robbie Ko
There are several more cases where we can't do the renames immediately. Example1: Parent snapshot: | d/ (ino 257) | p1 (ino 258) | p1/ (ino 259) Send snapshot: | d/ (ino 257) | p1 (ino 259) | p1/ (ino 258) Inode 258 became a child of inode 259 and both

[PATCH v3 1/7] Revert Btrfs: incremental send, remove dead code

2015-06-23 Thread Robbie Ko
This reverts commit 5f806c3ae2ff6263a10a6901f97abb74dac03d36. This is a necessary patch for a subsequent patch in the series (patch 3). [PATCH] Btrfs: incremental send, don't delay directory renames unnecessarily changes behavior of wait_for_parent_move function. It may cause path loop problem,

Re: [PATCH v2 0/5] Btrfs: RAID 5/6 missing device scrub+replace

2015-06-23 Thread Omar Sandoval
On Tue, Jun 23, 2015 at 11:07:00AM +0800, wangyf wrote: Hi, I have tested your PATCH v2 , but something wrong happened. kernel: 4.1.0-rc7+ with your five patches vitrualBox ubuntu14.10-server + LVM I make a new btrfs.ko with your patches, rmmod original module and insmod the new. When

Re: Question about btrfs send/receive

2015-06-23 Thread Marc MERLIN
On Wed, Jun 24, 2015 at 11:40:21AM +0800, Cao jin wrote: Hi Marc, I can`t access /perso/btrfs/2014-03.html on that server. But I guess it has similar content as https://btrfs.wiki.kernel.org/index.php/Incremental_Backup ? If so, I am afraid incremental backup not suitable for a API

[PATCH] btrfs-progs: man mkfs.btrfs: document -O^

2015-06-23 Thread Adam Borowski
Signed-off-by: Adam Borowski kilob...@angband.pl --- Documentation/mkfs.btrfs.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/mkfs.btrfs.asciidoc b/Documentation/mkfs.btrfs.asciidoc index fd20ca5..a9aa3cb 100644 --- a/Documentation/mkfs.btrfs.asciidoc

Re: Corrupted btrfs partition (converted from ext4) after balance

2015-06-23 Thread Qu Wenruo
Chris Murphy wrote on 2015/06/23 11:30 -0600: On Mon, Jun 22, 2015 at 7:44 PM, Qu Wenruo quwen...@cn.fujitsu.com wrote: Without the full output of btrfsck, it's quite hard to call it a clear bug report if you want to save your data in the corrupted partition. What I read was that he