btrfs-progs v3.14 mkfs.btrfs bug: --features long-option segfaults, -O short-option OK

2014-04-16 Thread Duncan
Using btrfs-progs v3.14: mkfs.btrfs ... --features ... segfaults. mkfs.btrfs ... -O ... works fine. * I used other long options, so it's not simply a problem parsing long options. * I tried the long option with various features; none worked, including --features list-all with no other

Re: btrfs-progs v3.14 mkfs.btrfs bug: --features long-option segfaults, -O short-option OK

2014-04-16 Thread Holger Hoffstätte
On Wed, 16 Apr 2014 06:41:44 +, Duncan wrote: Using btrfs-progs v3.14: mkfs.btrfs ... --features ... segfaults. Can reproduce (also with glibc 2.19 on Gentoo ;-) and building with debug found: (gdb) bt #0 0x76f3aaea in strlen () from /lib64/libc.so.6 #1 0x76f3a82e

Re: btrfs-progs v3.14 mkfs.btrfs bug: --features long-option segfaults, -O short-option OK

2014-04-16 Thread Holger Hoffstätte
On Wed, 16 Apr 2014 07:48:53 +, Holger Hoffstätte wrote: On Wed, 16 Apr 2014 06:41:44 +, Duncan wrote: Using btrfs-progs v3.14: mkfs.btrfs ... --features ... segfaults. Can reproduce (also with glibc 2.19 on Gentoo ;-) and building with debug found: (gdb) bt #0

[PATCH RFC] btrfs: Add ctime/mtime update for btrfs device add/remove.

2014-04-16 Thread Qu Wenruo
Btrfs will send uevent to udev inform the device change, but ctime/mtime for the block device inode is not udpated, which cause libblkid used by btrfs-progs unable to detect device change and use old cache, causing 'btrfs dev scan; btrfs dev rmove; btrfs dev scan' give an error message.

Re: [PATCH] Btrfs: implement inode_operations callback tmpfile

2014-04-16 Thread Chandan Rajendra
On Tuesday 01 Apr 2014 11:53:19 PM Filipe David Borba Manana wrote: +static int btrfs_tmpfile(struct inode *dir, struct dentry *dentry, umode_t mode) +{ + struct btrfs_trans_handle *trans; + struct btrfs_root *root = BTRFS_I(dir)-root; + struct inode *inode = NULL; + u64

Re: [PATCH] Btrfs: implement inode_operations callback tmpfile

2014-04-16 Thread Filipe David Manana
On Wed, Apr 16, 2014 at 10:00 AM, Chandan Rajendra chan...@linux.vnet.ibm.com wrote: On Tuesday 01 Apr 2014 11:53:19 PM Filipe David Borba Manana wrote: +static int btrfs_tmpfile(struct inode *dir, struct dentry *dentry, umode_t mode) +{ + struct btrfs_trans_handle *trans; + struct

Re: [PATCH] Btrfs: implement inode_operations callback tmpfile

2014-04-16 Thread Christoph Hellwig
On Wed, Apr 16, 2014 at 11:36:23AM +0100, Filipe David Manana wrote: The xattr is needed for the case where an acl is inherited. And 5 units are required for orphan insertion (see comment on top of btrfs_orphan_add). I'll update the comment. I don't think think a tmpfile should inherit any

Re: Copying a disk containing a btrfs filesystem

2014-04-16 Thread Michael Schuerig
On Friday 11 April 2014 12:39:31 Brendan Hide wrote: If you're 100% happy with your old disk's *content*/layout/etc (just not happy with the disk's reliability), try an overnight/over-weekend ddrescue instead Thanks again to everyone who replied and especially for suggesting ddrescue. In the

Re: [PATCH] Btrfs: implement inode_operations callback tmpfile

2014-04-16 Thread Filipe David Manana
On Wed, Apr 16, 2014 at 12:09 PM, Christoph Hellwig h...@infradead.org wrote: On Wed, Apr 16, 2014 at 11:36:23AM +0100, Filipe David Manana wrote: The xattr is needed for the case where an acl is inherited. And 5 units are required for orphan insertion (see comment on top of btrfs_orphan_add).

Re: [PATCH] Btrfs: implement inode_operations callback tmpfile

2014-04-16 Thread Christoph Hellwig
On Wed, Apr 16, 2014 at 12:25:07PM +0100, Filipe David Manana wrote: Interesting Christoph. I was following the ext4 implementation initially. So it seems the question is still open, and none of the following alternatives is decided yet (unless I missed something in the thread at fsdevel):

How do I find the physical block number?

2014-04-16 Thread Aastha Mehta
Hello, I have created a 500GB partition on my HDD and formatted it for btrfs. I created a file on it. # echo tmp data in the tmp file.. /mnt/btrfs/tmp-file # umount /mnt/btrfs Next I want to know the blocks allocated for the file and I used filefrag for it. I get some information as follows -

Re: [PATCH 1/2] btrfs: protect snapshots from deleting during send

2014-04-16 Thread David Sterba
On Tue, Apr 15, 2014 at 01:21:46PM -0400, Chris Mason wrote: On 04/15/2014 12:27 PM, David Sterba wrote: On Tue, Apr 15, 2014 at 12:00:49PM -0400, Chris Mason wrote: I'm worried about the use case where we have: * periodic automated snapshots * periodic automated deletion of old

Re: [PATCH 1/2] btrfs: protect snapshots from deleting during send

2014-04-16 Thread Chris Mason
On 04/16/2014 09:32 AM, David Sterba wrote: On Tue, Apr 15, 2014 at 01:21:46PM -0400, Chris Mason wrote: On 04/15/2014 12:27 PM, David Sterba wrote: On Tue, Apr 15, 2014 at 12:00:49PM -0400, Chris Mason wrote: I'm worried about the use case where we have: * periodic automated

[PATCH 1/4 v2] Btrfs: send, bump stream version

2014-04-16 Thread Filipe David Borba Manana
This increases the send stream version from version 1 to version 2, adding 2 new commands: 1) total data size - used to tell the receiver how much file data the stream will add or update; 2) fallocate - used to pre-allocate space for files and to punch holes in files. This is preparation

[PATCH 3/4 v2] Btrfs: send, use fallocate command to punch holes

2014-04-16 Thread Filipe David Borba Manana
Instead of sending a write command with a data buffer filled with 0 value bytes, use the fallocate command, introduced in the send stream version 2, to tell the receiver to punch a file hole using the fallocate system call. Signed-off-by: Filipe David Borba Manana fdman...@gmail.com --- V2: A v2

[PATCH 4/4 v2] Btrfs: send, use fallocate command to allocate extents

2014-04-16 Thread Filipe David Borba Manana
The send stream version 2 adds the fallocate command, which can be used to allocate extents for a file or punch holes in a file. Previously we were ignoring file prealloc extents or treating them as extents filled with 0 bytes and sending a regular write command to the stream. After this change,

[PATCH 1/4 v2] Btrfs-progs: send, bump stream version

2014-04-16 Thread Filipe David Borba Manana
This increases the send stream version from version 1 to version 2, adding 2 new commands: 1) total data size - used to tell the receiver how much file data the stream will add or update; 2) fallocate - used to pre-allocate space for files and to punch holes in files. This is preparation

[PATCH 2/4 v2] Btrfs-progs: send, implement total data size callback and progress report

2014-04-16 Thread Filipe David Borba Manana
This is a followup to the kernel patch titled: Btrfs: send, implement total data size command to allow for progress estimation This makes the btrfs send and receive commands aware of the new send flag, named BTRFS_SEND_C_TOTAL_DATA_SIZE, which tells us the amount of file data that is new

[PATCH 3/4 v2] Btrfs-progs: send, implement fallocate command callback

2014-04-16 Thread Filipe David Borba Manana
The fallocate send stream command, added in stream version 2, is used to pre-allocate space for files and punch file holes. This change implements the callback for that new command, using the fallocate function from the standard C library to carry out the specified action (allocate file space or

[PATCH 2/4 v2] Btrfs: send, implement total data size command to allow for progress estimation

2014-04-16 Thread Filipe David Borba Manana
This new send flag makes send calculate first the amount of new file data (in bytes) the send root has relatively to the parent root, or for the case of a non-incremental send, the total amount of file data the stream will create (including holes and prealloc extents). In other words, it

[PATCH 4/4 v2] Btrfs-progs: add write and clone commands debug info to receive

2014-04-16 Thread Filipe David Borba Manana
When specifying -vv print information about received write and clone commands too, as we do this for other commands already and it's very useful for debugging and troubleshooting. Signed-off-by: Filipe David Borba Manana fdman...@gmail.com --- V2: Added new send ioctl flag

[PATCH v2] xfstests: btrfs, test send's ability to punch holes and prealloc extents

2014-04-16 Thread Filipe David Borba Manana
This test verifies that after an incremental btrfs send the replicated file has the same exact hole and data structure as in the origin filesystem. This didn't use to be the case before the send stream version 2 - holes were sent as write operations of 0 valued bytes instead of punching holes with

[PATCH v2] xfstests: btrfs, add test for btrfs properties

2014-04-16 Thread Filipe David Borba Manana
This test case verifies the btrfs properties feature, a new feature introduced in the linux kernel version 3.14. Signed-off-by: Filipe David Borba Manana fdman...@gmail.com --- V2: Addressed Dave's comments, removed function to check for existence of the btrfs-progs property command and use

Re: [PATCH] xfstests: btrfs, test send's ability to punch holes and prealloc extents

2014-04-16 Thread Filipe David Manana
On Wed, Apr 16, 2014 at 1:23 AM, Dave Chinner da...@fromorbit.com wrote: On Tue, Apr 15, 2014 at 05:43:21PM +0100, Filipe David Borba Manana wrote: This test verifies that after an incremental btrfs send the replicated file has the same exact hole and data structure as in the origin

Re: Rebalance makes BTRFS 10x slower

2014-04-16 Thread Clemens Eisserer
Hi Ducan, But as I said, I'd sure like to get to the bottom of this one, since I do believe it has other potential implications in terms of bugs, etc. In theory, a balance should either not affect performance or should improve it, so getting to the bottom of why it's having such a bad

Lost /home subvolume after btrfs crash

2014-04-16 Thread Martin Wilck
Hello, I have a broken btrfs file system on a laptop. Debug material is available here: https://www.dropbox.com/sh/utv8b3qd0do6a04/zTwGQCrN9x Most importantly, the /home subvolume is lost. All attempts to recover data from it (btrfs-restore, mount -o recovery, btrfsck) have failed so far

[PATCH v2] btrfs: replace error code from btrfs_drop_extents

2014-04-16 Thread David Sterba
There's a case which clone does not handle and used to BUG_ON instead, (testcase xfstests/btrfs/035), now returns EINVAL. This error code is confusing to the ioctl caller, as it normally signifies errorneous arguments. Change it to EOPNOTSUPP which allows a fall back to copy instead of clone.

[PATCH 3/4 v3] Btrfs: send, use fallocate command to punch holes

2014-04-16 Thread Filipe David Borba Manana
Instead of sending a write command with a data buffer filled with 0 value bytes, use the fallocate command, introduced in the send stream version 2, to tell the receiver to punch a file hole using the fallocate system call. Signed-off-by: Filipe David Borba Manana fdman...@gmail.com --- V2: A v2

Re: [PATCH 1/2] btrfs: protect snapshots from deleting during send

2014-04-16 Thread Brendan Hide
On 2014/04/16 03:40 PM, Chris Mason wrote: So in my example with the automated tool, the tool really shouldn't be deleting a snapshot where send is in progress. The tool should be told that snapshot is busy and try to delete it again later. It makes more sense now, 'll queue this up for 3.16

Re: [PATCH 1/2] btrfs: protect snapshots from deleting during send

2014-04-16 Thread David Sterba
On Wed, Apr 16, 2014 at 09:40:41AM -0400, Chris Mason wrote: It makes more sense now, 'll queue this up for 3.16 and we can try it out in -next. Thanks. 3.16 is fine for me. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to

Re: [PATCH 1/2] btrfs: protect snapshots from deleting during send

2014-04-16 Thread David Sterba
On Wed, Apr 16, 2014 at 04:59:09PM +0200, Brendan Hide wrote: On 2014/04/16 03:40 PM, Chris Mason wrote: So in my example with the automated tool, the tool really shouldn't be deleting a snapshot where send is in progress. The tool should be told that snapshot is busy and try to delete it

Re: Lost /home subvolume after btrfs crash

2014-04-16 Thread Chris Murphy
On Apr 16, 2014, at 7:53 AM, Martin Wilck mwi...@arcor.de wrote: The crash happened with a rather old OpenSUSE 12.2 kernel (3.4.11-2.16). The user says she was just surfing the web normally when the crash occured (no screenshot of the original crash, unfortunately). On the next boot, the

[PATCH 4/4 v3] Btrfs: send, use fallocate command to allocate extents

2014-04-16 Thread Filipe David Borba Manana
The send stream version 2 adds the fallocate command, which can be used to allocate extents for a file or punch holes in a file. Previously we were ignoring file prealloc extents or treating them as extents filled with 0 bytes and sending a regular write command to the stream. After this change,

Re: [PATCH 00/27] Replace the old man page with asciidoc and man page for each btrfs subcommand.

2014-04-16 Thread David Sterba
On Wed, Apr 02, 2014 at 04:29:11PM +0800, Qu Wenruo wrote: Convert the old btrfs man pages to new asciidoc and split the huge btrfs man page into subcommand man page. I'm merging this patchset into the base series of integration because several patches need to update the docs and it's no longer

[PATCH] btrfs-progs: doc: link btrfsck to btrfs-check

2014-04-16 Thread David Sterba
The 'btrfsck' command has been deprecated in favor of 'btrfs check'. For compatibility install a symlink to the btrfs-check.8 manpage. CC: Qu Wenruo quwen...@cn.fujitsu.com Signed-off-by: David Sterba dste...@suse.cz --- Documentation/Makefile| 2 ++ Documentation/btrfs-check.txt | 3 ++-

Re: [PATCH 2/4 v2] Btrfs-progs: send, implement total data size callback and progress report

2014-04-16 Thread David Sterba
On Wed, Apr 16, 2014 at 03:56:15PM +0100, Filipe David Borba Manana wrote: V2: Added new send ioctl flag BTRFS_SEND_FLAG_SUPPORT_FALLOCATE. A version 2 stream is now only produced is the ioctl caller specifies at least one of the new send flags (BTRFS_SEND_FLAG_SUPPORT_FALLOCATE or

[PATCH] btrfs-progs: fix typo in subvol list usage

2014-04-16 Thread David Disseldorp
Signed-off-by: David Disseldorp dd...@suse.de --- cmds-subvolume.c | 2 +- man/btrfs.8.in | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/cmds-subvolume.c b/cmds-subvolume.c index 5e821c7..75a7385 100644 --- a/cmds-subvolume.c +++ b/cmds-subvolume.c @@ -390,7 +390,7 @@

Re: Kernel crash triggered by dd to file with memcg, worst on btrfs

2014-04-16 Thread Marian Marinov
Hi, what kernel version are you running? Marian On 04/16/2014 08:42 PM, Richard Davies wrote: Hi all, I have a test case in which I can often crash an entire machine by running dd to a file with a memcg with relatively generous limits. This is simplified from real world problems with heavy

Re: Kernel crash triggered by dd to file with memcg, worst on btrfs

2014-04-16 Thread Richard Davies
Richard Davies wrote: I have a test case in which I can often crash an entire machine by running dd to a file with a memcg with relatively generous limits. This is simplified from real world problems with heavy disk i/o inside containers. The crashes are easy to trigger when dding to create a

Kernel crash triggered by dd to file with memcg, worst on btrfs

2014-04-16 Thread Richard Davies
Hi all, I have a test case in which I can often crash an entire machine by running dd to a file with a memcg with relatively generous limits. This is simplified from real world problems with heavy disk i/o inside containers. The crashes are easy to trigger when dding to create a file on btrfs.

Re: [PATCH 2/4 v2] Btrfs-progs: send, implement total data size callback and progress report

2014-04-16 Thread Filipe David Manana
On Wed, Apr 16, 2014 at 6:43 PM, David Sterba dste...@suse.cz wrote: On Wed, Apr 16, 2014 at 03:56:15PM +0100, Filipe David Borba Manana wrote: V2: Added new send ioctl flag BTRFS_SEND_FLAG_SUPPORT_FALLOCATE. A version 2 stream is now only produced is the ioctl caller specifies at least one

[PATCH 2/4 v3] Btrfs-progs: send, implement total data size callback and progress report

2014-04-16 Thread Filipe David Borba Manana
This is a followup to the kernel patch titled: Btrfs: send, implement total data size command to allow for progress estimation This makes the btrfs send and receive commands aware of the new send flag, named BTRFS_SEND_C_TOTAL_DATA_SIZE, which tells us the amount of file data that is new

[PATCH 3/4 v3] Btrfs-progs: send, implement fallocate command callback

2014-04-16 Thread Filipe David Borba Manana
The fallocate send stream command, added in stream version 2, is used to pre-allocate space for files and punch file holes. This change implements the callback for that new command, using the fallocate function from the standard C library to carry out the specified action (allocate file space or

Re: How do I find the physical block number?

2014-04-16 Thread Aastha Mehta
On 16 April 2014 15:27, Aastha Mehta aasth...@gmail.com wrote: Hello, I have created a 500GB partition on my HDD and formatted it for btrfs. I created a file on it. # echo tmp data in the tmp file.. /mnt/btrfs/tmp-file # umount /mnt/btrfs Next I want to know the blocks allocated for the

Re: very slow btrfs filesystem: any data needed before I wipe it?

2014-04-16 Thread Marc MERLIN
On Mon, Apr 14, 2014 at 10:28:36AM +, Duncan wrote: But you might well be the first report where the devs have good access to enough detail to actually trace down the problem. I'll wait until tomorrow night to see if the devs want anything else out of it, but otherwise I'll wipe it

Re: [PATCH] xfstests: btrfs, test send's ability to punch holes and prealloc extents

2014-04-16 Thread Dave Chinner
On Wed, Apr 16, 2014 at 03:39:18PM +0100, Filipe David Manana wrote: On Wed, Apr 16, 2014 at 1:23 AM, Dave Chinner da...@fromorbit.com wrote: On Tue, Apr 15, 2014 at 05:43:21PM +0100, Filipe David Borba Manana wrote: This test verifies that after an incremental btrfs send the replicated

Re: [PATCH] xfstests: btrfs, test send's ability to punch holes and prealloc extents

2014-04-16 Thread Filipe David Manana
On Thu, Apr 17, 2014 at 12:13 AM, Dave Chinner da...@fromorbit.com wrote: On Wed, Apr 16, 2014 at 03:39:18PM +0100, Filipe David Manana wrote: On Wed, Apr 16, 2014 at 1:23 AM, Dave Chinner da...@fromorbit.com wrote: On Tue, Apr 15, 2014 at 05:43:21PM +0100, Filipe David Borba Manana wrote:

[PATCH v3] xfstests: btrfs, test send's ability to punch holes and prealloc extents

2014-04-16 Thread Filipe David Borba Manana
This test verifies that after an incremental btrfs send the replicated file has the same exact hole and data structure as in the origin filesystem. This didn't use to be the case before the send stream version 2 - holes were sent as write operations of 0 valued bytes instead of punching holes with

Re: [PATCH] btrfs-progs: doc: link btrfsck to btrfs-check

2014-04-16 Thread Qu Wenruo
Original Message Subject: [PATCH] btrfs-progs: doc: link btrfsck to btrfs-check From: David Sterba dste...@suse.cz To: linux-btrfs@vger.kernel.org Date: 2014年04月17日 01:16 The 'btrfsck' command has been deprecated in favor of 'btrfs check'. For compatibility install a symlink