Re: Problems with incremental btrfs send (out of order instructions?)

2013-08-01 Thread Stefan Behrens
On Wed, 31 Jul 2013 23:33:59 +0200, Mathijs Kwik wrote: Hi all, For some time, I've successfully deployed btrfs send/receive as a viable backup solution. It's fast and flexible and nicely scriptable =) However, every once in a while, trouble strikes on the receiving end with a message

Re: [PATCH 01/10] Btrfs-progs: add missing man page information for btrfsck

2013-08-01 Thread Stefan Behrens
On Thu, 1 Aug 2013 13:35:25 +0800, Wang Shilong wrote: Signed-off-by: Wang Shilong wangsl.f...@cn.fujitsu.com Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com --- man/btrfsck.8.in | 35 ++- 1 file changed, 30 insertions(+), 5 deletions(-) diff --git

Re: [RFC PATCH v5 5/5] Btrfs: online data deduplication

2013-08-01 Thread Liu Bo
On Wed, Jul 31, 2013 at 03:50:50PM -0700, Zach Brown wrote: +#define BTRFS_DEDUP_HASH_SIZE 32 /* 256bit = 32 * 8bit */ +#define BTRFS_DEDUP_HASH_LEN 4 + +struct btrfs_dedup_hash_item { + /* FIXME: put a hash type field here */ + + __le64 hash[BTRFS_DEDUP_HASH_LEN]; +}

Re: [RFC PATCH v5 0/5] Online data deduplication

2013-08-01 Thread Liu Bo
On Wed, Jul 31, 2013 at 05:20:27PM -0400, Josef Bacik wrote: On Wed, Jul 31, 2013 at 11:37:40PM +0800, Liu Bo wrote: Data deduplication is a specialized data compression technique for eliminating duplicate copies of repeating data.[1] This patch set is also related to Content based

Re: [PATCH] Btrfs-progs: add dedup subcommand

2013-08-01 Thread Liu Bo
On Wed, Jul 31, 2013 at 06:30:37PM +0200, Stefan Behrens wrote: On Wed, 31 Jul 2013 23:37:46 +0800, Liu Bo wrote: This aims to add deduplication subcommand, 'btrfs dedup command path', ie. register/unregister'. It can be used to enable or disable dedup support for a filesystem.

How to merge two partitions?

2013-08-01 Thread Andrew Stubbs
If I have two partitions, /dev/sda1 and /dev/sda2, one btrfs, and one ext4 (but I could convert it first), how can I merge them into one filesystem without moving all the data onto an external device and then moving it all back again? (I do have a backup, of course, but transferring the data

Re: How to merge two partitions?

2013-08-01 Thread Hugo Mills
On Thu, Aug 01, 2013 at 11:53:34AM +0100, Andrew Stubbs wrote: If I have two partitions, /dev/sda1 and /dev/sda2, one btrfs, and one ext4 (but I could convert it first), how can I merge them into one filesystem without moving all the data onto an external device and then moving it all back

Re: How to merge two partitions?

2013-08-01 Thread Andrew Stubbs
On 01/08/13 12:08, Hugo Mills wrote: I thought of using btrfs device add and just living with the untidy underlying devices, but an experiment with loopback filesystems shows that any data on the new device is silently obliterated (it might be nice if the docs mentioned this!) You would

Re: [PATCH] Btrfs: stop all workers before cleaning up roots

2013-08-01 Thread Alex Lyakas
Hi Josef, On Thu, May 30, 2013 at 11:58 PM, Josef Bacik jba...@fusionio.com wrote: Dave reported a panic because the extent_root-commit_root was NULL in the caching kthread. That is because we just unset it in free_root_pointers, which is not the correct thing to do, we have to either wait

Re: Is the checkpoint interval adjustable?

2013-08-01 Thread David Sterba
On Wed, Jul 31, 2013 at 03:56:40PM -0700, Zach Brown wrote:  I am NO programmer by any stretch.  Let's say I want them to be once every 5 min (300 sec).  Is the attached patch sane to acheive this? I think it's a reasonable patch to try, yeah. There were a few requests to tune the

[PATCH] btrfs: add mount option to set commit interval

2013-08-01 Thread David Sterba
I'ts hardcoded to 30 seconds which is fine for most users. Higher values defer data being synced to permanent storage with obvious consequences when the system crashes. The upper bound is not forced, but a warning is printed if it's more than 300 seconds (5 minutes). Signed-off-by: David Sterba

[PATCH v2] btrfs: add mount option to set commit interval

2013-08-01 Thread David Sterba
I'ts hardcoded to 30 seconds which is fine for most users. Higher values defer data being synced to permanent storage with obvious consequences when the system crashes. The upper bound is not forced, but a warning is printed if it's more than 300 seconds (5 minutes). Signed-off-by: David Sterba

Re: [3.10.3] kernel BUG at fs/btrfs/ctree.c:3000

2013-08-01 Thread David Sterba
On Wed, Jul 31, 2013 at 09:35:00AM -0400, Josef Bacik wrote: On Wed, Jul 31, 2013 at 02:19:25PM +0200, tim wrote: running btrfs on kernel 3.10.3, i got the following two backtraces, first an info about a hung task and a kernel bug. are these known issues? That's a crash that appears

Re: [PATCH] Btrfs: don't bug_on when we fail when cleaning up transactions

2013-08-01 Thread David Sterba
On Wed, Jul 31, 2013 at 10:29:23AM -0400, Josef Bacik wrote: There is no reason for this sort of jackassery. Thanks, Signed-off-by: Josef Bacik jba...@fusionio.com --- fs/btrfs/transaction.c |3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diff --git

Re: Is the checkpoint interval adjustable?

2013-08-01 Thread Zach Brown
There were a few requests to tune the interval. This finally made me to finish the patch and will send it in a second. Great, thanks. That's a good point and lowers my worries a bit, though it would be interesting to see in what way a beefy machine blows with 300 seconds set. Agreed.

Re: [RFC PATCH v5 5/5] Btrfs: online data deduplication

2013-08-01 Thread Zach Brown
So do you mean that our whole hash value will be (key.objectid + bytes) because key.objectid is a part of hash value? I think so, if I understood your question. The idea is to not store the bytes of the hash that make up the objectid more than once so the tree items are smaller. For example:

Re: [PATCH 09/12] Btrfs-progs: Add chunk recover function - using old chunk items

2013-08-01 Thread David Sterba
On Wed, Jul 03, 2013 at 09:25:17PM +0800, Miao Xie wrote: --- a/btrfs.c +++ b/btrfs.c @@ -247,6 +247,7 @@ const struct cmd_group btrfs_cmd_group = { { device, cmd_device, NULL, device_cmd_group, 0 }, { scrub, cmd_scrub, NULL, scrub_cmd_group, 0 }, {

Re: [PATCH v2] btrfs-progs: fix segfault when listing column OTIME on big endian host

2013-08-01 Thread David Sterba
On Thu, Jul 25, 2013 at 10:12:01PM +0800, Eryu Guan wrote: The second btrfs command segfaults on big endian host(ppc64) btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snap btrfs subvolume list -s /mnt/btrfs And ltrace shows localtime(0x10029c482d0) = 0

filefrag and btrfs filesystem defragment and maybe snapshots

2013-08-01 Thread Sandy McArthur
While exploring some btrfs maintenance with respect to defragmenting I ran the following commands: # filefrag /path/to/34G.file /path/to/5.7G.file /path/to/34G.file: 2406 extents found /path/to/5.7G.file: 572 extents found Thinking those mostly static files could be less fragmented I ran: #

Re: [PATCH] Btrfs-progs: add dedup subcommand

2013-08-01 Thread Mark Fasheh
On Wed, Jul 31, 2013 at 11:37:46PM +0800, Liu Bo wrote: This aims to add deduplication subcommand, 'btrfs dedup command path', ie. register/unregister'. It can be used to enable or disable dedup support for a filesystem. This seems to me like it should be a switch on btrfstune instead of a

Re: [PATCH] Btrfs-progs: fix closing of opendir()

2013-08-01 Thread David Sterba
On Mon, Jul 15, 2013 at 07:36:50PM +0800, Wang Shilong wrote: valgrind complains open_file_or_dir() causes a memory leak.That is because if we open a directoy by opendir(), and then we should call closedir() to free memory. I've reviewed this and don't see a better way how to fix the leaks

Re: [PATCH] Btrfs-progs: fix closing of opendir()

2013-08-01 Thread David Sterba
--- a/utils.c +++ b/utils.c @@ -1476,11 +1475,11 @@ u64 parse_size(char *s) return strtoull(s, NULL, 10) * mult; } -int open_file_or_dir(const char *fname) +int open_file_or_dir(const char *fname, DIR **dirstream) { int ret; struct stat st; - DIR *dirstream =

Re: filefrag and btrfs filesystem defragment and maybe snapshots

2013-08-01 Thread Liu Bo
On Thu, Aug 01, 2013 at 05:18:50PM -0400, Sandy McArthur wrote: While exploring some btrfs maintenance with respect to defragmenting I ran the following commands: # filefrag /path/to/34G.file /path/to/5.7G.file /path/to/34G.file: 2406 extents found /path/to/5.7G.file: 572 extents found

Re: [PATCH] Btrfs-progs: add dedup subcommand

2013-08-01 Thread Liu Bo
On Thu, Aug 01, 2013 at 03:01:37PM -0700, Mark Fasheh wrote: On Wed, Jul 31, 2013 at 11:37:46PM +0800, Liu Bo wrote: This aims to add deduplication subcommand, 'btrfs dedup command path', ie. register/unregister'. It can be used to enable or disable dedup support for a filesystem.

Re: filefrag and btrfs filesystem defragment and maybe snapshots

2013-08-01 Thread Sandy McArthur
On Thu, Aug 1, 2013 at 10:27 PM, Duncan 1i5t5.dun...@cox.net wrote: Sandy McArthur posted on Thu, 01 Aug 2013 17:18:50 -0400 as excerpted: While exploring some btrfs maintenance with respect to defragmenting I ran the following commands: # filefrag /path/to/34G.file /path/to/5.7G.file