[RFC PATCH 1/7] Btrfs: use _IOR for BTRFS_IOC_SUBVOL_GETFLAGS

2012-07-04 Thread Alexander Block
We used the wrong ioctl macro for the getflags ioctl before. As we don't have the set/getflags ioctls in the user space ioctl.h at the moment, it's safe to fix it now. Reviewed-by: David Sterba dste...@suse.cz Signed-off-by: Alexander Block abloc...@googlemail.com --- fs/btrfs/ioctl.h |2

[RFC PATCH 4/7] Btrfs: introduce subvol uuids and times

2012-07-04 Thread Alexander Block
the fields. Signed-off-by: Alexander Block abloc...@googlemail.com --- fs/btrfs/ctree.h | 43 ++ fs/btrfs/disk-io.c |2 + fs/btrfs/inode.c |4 ++ fs/btrfs/ioctl.c | 96 ++-- fs/btrfs/ioctl.h

[RFC PATCH 3/7] Btrfs: make iref_to_path non static

2012-07-04 Thread Alexander Block
Make iref_to_path non static (needed in send) and rename it to btrfs_iref_to_path Signed-off-by: Alexander Block abloc...@googlemail.com --- fs/btrfs/backref.c | 10 +- fs/btrfs/backref.h |4 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/backref.c b

[RFC PATCH 2/7] Btrfs: add helper for tree enumeration

2012-07-04 Thread Alexander Block
From: Arne Jansen sensi...@gmx.net Often no exact match is wanted but just the next lower or higher item. There's a lot of duplicated code throughout btrfs to deal with the corner cases. This patch adds a helper function that can facilitate searching. Signed-off-by: Arne Jansen sensi...@gmx.net

[RFC PATCH 5/7] Btrfs: add btrfs_compare_trees function

2012-07-04 Thread Alexander Block
This function is used to find the differences between two trees. The tree compare skips whole subtrees if it detects shared tree blocks and thus is pretty fast. Signed-off-by: Alexander Block abloc...@googlemail.com --- fs/btrfs/ctree.c | 425

[RFC PATCH 6/7] Btrfs: introduce BTRFS_IOC_SEND for btrfs send/receive (part 1)

2012-07-04 Thread Alexander Block
the send.h header and the helper functions for TLV handling and long path name handling and some other helpers. The second part contains the actual send logic from send.c Signed-off-by: Alexander Block abloc...@googlemail.com --- fs/btrfs/Makefile |2 +- fs/btrfs/ioctl.h | 10 + fs/btrfs

[RFC PATCH 0/6] Experimental btrfs send/receive (btrfs-progs)

2012-07-04 Thread Alexander Block
the kernel side email as well, especially the warnings! Alex. Alexander Block (6): Btrfs-progs: add BTRFS_IOC_SUBVOL_GET/SETFLAGS to ioctl.h Btrfs-progs: update ioctl.h to support clone range ioctl Btrfs-progs: print inode transid and dir item data field in debug-tree Btrfs-progs: update

[RFC PATCH 1/6] Btrfs-progs: add BTRFS_IOC_SUBVOL_GET/SETFLAGS to ioctl.h

2012-07-04 Thread Alexander Block
Btrfs send/receive and btrfs props needs this ioctl. This patch requires a recent kernel with the Btrfs: use _IOR for BTRFS_IOC_SUBVOL_GETFLAGS patch applied. Signed-off-by: Alexander Block abloc...@googlemail.com --- ioctl.h |2 ++ 1 file changed, 2 insertions(+) diff --git a/ioctl.h b

[RFC PATCH 2/6] Btrfs-progs: update ioctl.h to support clone range ioctl

2012-07-04 Thread Alexander Block
Added missing btrfs_ioctl_clone_range_args and BTRFS_IOC_CLONE_RANGE to ioctl.h Signed-off-by: Alexander Block abloc...@googlemail.com --- ioctl.h | 10 +- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/ioctl.h b/ioctl.h index 6670e08..023ca4c 100644 --- a/ioctl.h +++ b

[RFC PATCH 3/6] Btrfs-progs: print inode transid and dir item data field in debug-tree

2012-07-04 Thread Alexander Block
Add printing of inode transid and dir item data field. Signed-off-by: Alexander Block abloc...@googlemail.com --- print-tree.c |9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/print-tree.c b/print-tree.c index fc134c0..1377732 100644 --- a/print-tree.c +++ b/print

[RFC PATCH 4/6] Btrfs-progs: update btrfs-progs for subvol uuid+times support

2012-07-04 Thread Alexander Block
Update ctree.h and ioctl.h for the new uuid+times for subvolumes. Signed-off-by: Alexander Block abloc...@googlemail.com --- ctree.h | 40 - ioctl.h | 12 + print-tree.c | 79 +- 3 files

[RFC PATCH 5/6] Btrfs-progs: update ioctl.h to support btrfs send ioctl

2012-07-04 Thread Alexander Block
Add btrfs_ioctl_send_args and BTRFS_IOC_SEND to ioctl.h Signed-off-by: Alexander Block abloc...@googlemail.com --- ioctl.h | 11 +++ 1 file changed, 11 insertions(+) diff --git a/ioctl.h b/ioctl.h index 77503e6..d35710c 100644 --- a/ioctl.h +++ b/ioctl.h @@ -282,6 +282,15 @@ struct

Re: [RFC PATCH 5/7] Btrfs: add btrfs_compare_trees function

2012-07-04 Thread Alexander Block
On Wed, Jul 4, 2012 at 8:27 PM, Alex Lyakas alex.bolshoy.bt...@gmail.com wrote: Hi Alex, + spin_lock(left_root-root_times_lock); + ctransid = btrfs_root_ctransid(left_root-root_item); + spin_unlock(left_root-root_times_lock);

Re: [RFC PATCH 5/7] Btrfs: add btrfs_compare_trees function

2012-07-04 Thread Alexander Block
On Wed, Jul 4, 2012 at 9:13 PM, Alex Lyakas alex.bolshoy.bt...@gmail.com wrote: Hi Alex, +static int tree_compare_item(struct btrfs_root *left_root, +struct btrfs_path *left_path, +struct btrfs_path *right_path, +

Re: [PATCH] Btrfs: Add code to support file creation time.

2012-07-04 Thread Alexander Block
On Thu, Jul 5, 2012 at 3:07 AM, Li Zefan lize...@huawei.com wrote: On 2012/7/4 19:04, Alexander Block wrote: On Wed, Jul 4, 2012 at 9:56 AM, Li Zefan lize...@huawei.com wrote: On 2012/7/4 15:18, chandan r wrote: This patch adds a new member to the 'struct btrfs_inode' structure to hold

Re: [RFC PATCH 4/7] Btrfs: introduce subvol uuids and times

2012-07-05 Thread Alexander Block
On Wed, Jul 4, 2012 at 3:38 PM, Alexander Block abloc...@googlemail.com wrote: This patch introduces uuids for subvolumes. [...] Stefan and Jan pointed out a problem with this patch that would result in read_extent_buffer calls that read beyond the leaf size when an old root item is found

Re: [RFC PATCH 5/7] Btrfs: add btrfs_compare_trees function

2012-07-05 Thread Alexander Block
On Thu, Jul 5, 2012 at 2:19 PM, Alex Lyakas alex.bolshoy.bt...@gmail.com wrote: Alexander, + if (advance_left !left_end_reached) { + ret = tree_advance(left_root, left_path, left_level, + left_root_level, +

[RFC] Btrfs sendshots and hidden snapshots

2012-07-05 Thread Alexander Block
Hello all, in IRC we had a discussion on how we could solve sending live subvolumes and how to send subvolumes without the need to administrate/keep old snapshots for incremental sends. One of the ideas was to introduce sendshots, which are basically snapshots where no refs are counted for file

Re: [RFC PATCH 4/7] Btrfs: introduce subvol uuids and times

2012-07-05 Thread Alexander Block
On Thu, Jul 5, 2012 at 7:08 PM, Zach Brown z...@zabbo.net wrote: +static long btrfs_ioctl_set_received_subvol(struct file *file, + void __user *arg) +{ + struct btrfs_ioctl_received_subvol_args *sa = NULL; + ret = copy_to_user(arg, sa,

Re: [RFC PATCH 4/7] Btrfs: introduce subvol uuids and times

2012-07-05 Thread Alexander Block
On Thu, Jul 5, 2012 at 9:24 PM, Ilya Dryomov idryo...@gmail.com wrote: On Thu, Jul 05, 2012 at 09:18:41PM +0200, Alexander Block wrote: On Thu, Jul 5, 2012 at 9:01 PM, Zach Brown z...@zabbo.net wrote: On 07/05/2012 11:59 AM, Ilya Dryomov wrote: What if you are on a big-endian machine

Re: [RFC] Btrfs sendshots and hidden snapshots

2012-07-06 Thread Alexander Block
On Fri, Jul 6, 2012 at 12:34 AM, Goffredo Baroncelli kreij...@libero.it wrote: On 07/05/2012 06:51 PM, Alexander Block wrote: Hello all, in IRC we had a discussion on how we could solve sending live subvolumes and how to send subvolumes without the need to administrate/keep old snapshots

Re: [RFC] Btrfs sendshots and hidden snapshots

2012-07-06 Thread Alexander Block
On Fri, Jul 6, 2012 at 2:03 PM, Goffredo Baroncelli kreij...@libero.it wrote: On 07/06/2012 01:55 PM, Chris Mason wrote: On Fri, Jul 06, 2012 at 02:51:43AM -0600, Alexander Block wrote: On Fri, Jul 6, 2012 at 12:34 AM, Goffredo Baroncelli kreij...@libero.it wrote: On 07/05/2012 06:51 PM

Re: [PATCH] fix unaligned memory accesses (Closes: #656955)

2012-07-07 Thread Alexander Block
On Sat, Jul 7, 2012 at 3:37 AM, Shawn Landden shawnland...@gmail.com wrote: From: Shawn Landen shawnland...@gmail.com Fix creation of volumes using mkfs.btrfs on armv5. Signed-off-by: Shawn Landen shawnland...@gmail.com --- ctree.h | 26 -- volumes.c |5

Re: [PATCH v1 04/15] Btrfs: add helper for tree enumeration

2012-07-12 Thread Alexander Block
On Thu, Jul 12, 2012 at 11:43 AM, Jan Schmidt list.bt...@jan-o-sch.net wrote: From: Arne Jansen sensi...@gmx.net Often no exact match is wanted but just the next lower or higher item. There's a lot of duplicated code throughout btrfs to deal with the corner cases. This patch adds a helper

Re: [RFC PATCH 4/7] Btrfs: introduce subvol uuids and times

2012-07-23 Thread Alexander Block
On Mon, Jul 16, 2012 at 4:56 PM, Arne Jansen sensi...@gmx.net wrote: On 04.07.2012 15:38, Alexander Block wrote: This patch introduces uuids for subvolumes. Each subvolume has it's own uuid. In case it was snapshotted, it also contains parent_uuid. In case it was received, it also contains

Re: btrfs send/receive: if new inode ino is less than its new directory ino, incorrect path is sent

2012-07-24 Thread Alexander Block
On Wed, Jul 18, 2012 at 7:45 PM, Alex Lyakas alex.bolshoy.bt...@gmail.com wrote: Hi Alexander, I am testing different scenarios in order to better understand the non-trivial magic of get_cur_path()/will_overwrite_ref()/did_overwrite_ref()/did_overwrite_first_ref(). I hit the following issue,

Re: [RFC PATCH 6/6] Btrfs-progs: add btrfs send/receive commands

2012-07-24 Thread Alexander Block
On Thu, Jul 19, 2012 at 3:25 PM, Alex Lyakas alex.bolshoy.bt...@gmail.com wrote: +static int process_link(const char *path, const char *lnk, void *user) +{ + int ret; + struct btrfs_receive *r = user; + char *full_path = path_cat(r-full_subvol_path, path); + + if

Re: [RFC PATCH 0/6] Experimental btrfs send/receive (btrfs-progs)

2012-07-25 Thread Alexander Block
On Mon, Jul 23, 2012 at 2:29 PM, Arne Jansen sensi...@gmx.net wrote: On 04.07.2012 15:39, Alexander Block wrote: Hello all, This is the user space side of btrfs send/receive. You can apply them manually or use my git repo: git://github.com/ablock84/btrfs-progs.git (branch send

Re: [RFC PATCH 4/7] Btrfs: introduce subvol uuids and times

2012-07-25 Thread Alexander Block
On Tue, Jul 24, 2012 at 7:55 AM, Arne Jansen sensi...@gmx.net wrote: On 23.07.2012 21:41, Alexander Block wrote: On Mon, Jul 16, 2012 at 4:56 PM, Arne Jansen sensi...@gmx.net wrote: On 04.07.2012 15:38, Alexander Block wrote: + + ret = btrfs_update_root(trans, root-fs_info-tree_root

Re: [RFC PATCH 7/7] Btrfs: introduce BTRFS_IOC_SEND for btrfs send/receive (part 2)

2012-07-25 Thread Alexander Block
On Tue, Jul 10, 2012 at 5:26 PM, Alex Lyakas alex.bolshoy.bt...@gmail.com wrote: Alexander, this focuses on area of sending file extents: +static int is_extent_unchanged(struct send_ctx *sctx, + struct btrfs_path *left_path, + struct

Re: [RFC PATCH 0/6] Experimental btrfs send/receive (btrfs-progs)

2012-07-25 Thread Alexander Block
On Wed, Jul 25, 2012 at 4:00 PM, Hugo Mills h...@carfax.org.uk wrote: On Wed, Jul 25, 2012 at 12:41:56PM +0200, Alexander Block wrote: On Mon, Jul 23, 2012 at 2:29 PM, Arne Jansen sensi...@gmx.net wrote: On 04.07.2012 15:39, Alexander Block wrote: Hello all, This is the user space side

Re: [RFC PATCH 0/6] Experimental btrfs send/receive (btrfs-progs)

2012-07-25 Thread Alexander Block
On Wed, Jul 25, 2012 at 7:10 PM, Alex Lyakas alex.bolshoy.bt...@gmail.com wrote: Alexander, can you pls let know like a day or two before you run out of time? I have compiled a list of questions, but also want to do more testing before I publish them all. My flight goes on 6. August...after

Re: [RFC PATCH 6/7] Btrfs: introduce BTRFS_IOC_SEND for btrfs send/receive (part 1)

2012-07-25 Thread Alexander Block
Thanks for the review :) On 07/18/2012 08:59 AM, Arne Jansen wrote: On 04.07.2012 15:38, Alexander Block wrote: This patch introduces the BTRFS_IOC_SEND ioctl that is required for send. It allows btrfs-progs to implement full and incremental sends. Patches for btrfs-progs will follow. I had

Re: [RFC PATCH 7/7] Btrfs: introduce BTRFS_IOC_SEND for btrfs send/receive (part 2)

2012-07-25 Thread Alexander Block
On Wed, Jul 25, 2012 at 7:20 PM, Alex Lyakas alex.bolshoy.bt...@gmail.com wrote: Alexander, Same is true for BTRFS_FILE_EXTENT_PREALLOC extents, I think. Those also don't contain real data. So something like: if (left_disknr == 0 || left_type == BTRFS_FILE_EXTENT_REG) { ret = 1;

Re: btrfs send/receive: if new inode ino is less than its new directory ino, incorrect path is sent

2012-07-26 Thread Alexander Block
will continue testing more complicated cases now. Thanks, Alex. On Tue, Jul 24, 2012 at 11:26 PM, Alexander Block abloc...@googlemail.com wrote: On Wed, Jul 18, 2012 at 7:45 PM, Alex Lyakas alex.bolshoy.bt...@gmail.com wrote: Hi Alexander, I am testing different scenarios in order to better

Re: btrfs send/receive: if new inode ino is less than its new directory ino, incorrect path is sent

2012-07-26 Thread Alexander Block
I have pushed a for-alex branch to github with a new approach for the whole problem. Can you test this? On Thu, Jul 26, 2012 at 4:07 PM, Alexander Block abloc...@googlemail.com wrote: I'm currently working on another solution for the initial problem. I will create a for-alex branch for you

Re: btrfs send/receive: if new inode ino is less than its new directory ino, incorrect path is sent

2012-07-28 Thread Alexander Block
On Fri, Jul 27, 2012 at 4:37 PM, Alex Lyakas alex.bolshoy.bt...@gmail.com wrote: Hi Alexander, your solution is simple and elegant. I this this issue is solved now. Thanks! Two minor issues: 1) /* * We need some special handling for inodes that get processed before the parent *

Re: btrfs send/receive: if new inode ino is less than its new directory ino, incorrect path is sent

2012-07-30 Thread Alexander Block
On Mon, Jul 30, 2012 at 7:35 PM, Alex Lyakas alex.bolshoy.bt...@gmail.com wrote: Hi, I did not apply the patch but instead added a check for dir != tmp_dir only. The reason to not check for gen is that I have a rule in my mind: I only pass the generation number to functions where I want to

Re: Questions and notes about send/receive

2012-08-01 Thread Alexander Block
On Tue, Jul 31, 2012 at 6:32 PM, Alex Lyakas alex.bolshoy.bt...@gmail.com wrote: Hi Alexander, I relooked at my list of questions, and it seems that there are more general questions, and more focused questions. So here I list the more focused ones. I really appreciate if you can address them.

btrfs send/receive review by vfs folks

2012-08-01 Thread Alexander Block
Hello, I'm sending this to fsdevel as we're using vfs functions to open inodes and read/write from them in btrfs send/receive and a review by vfs folks would probably be a good thing here. I extracted the vfs related functions from the current version and copied them into this mail below.

Re: [PATCH] add crtime to the snapshot list

2012-08-01 Thread Alexander Block
On Wed, Aug 1, 2012 at 2:01 PM, Anand jain anand.j...@oracle.com wrote: From: Anand anand.j...@oracle.com This patch adds creation-time to the snapshot list display, which would help user to better manage the snapshots when number of snapshots grow substantially. This patch is developed

Re: [patch] Btrfs: fix some endian bugs handling the root times

2012-08-01 Thread Alexander Block
On Mon, Jul 30, 2012 at 10:10 AM, Dan Carpenter dan.carpen...@oracle.com wrote: trans-transid is cpu endian but we want to store the data as little endian. item-ctime.nsec is only 32 bits, not 64. Signed-off-by: Dan Carpenter dan.carpen...@oracle.com --- Applies to linux-next. diff --git

Re: [RFC PATCH 7/7] Btrfs: introduce BTRFS_IOC_SEND for btrfs send/receive (part 2)

2012-08-01 Thread Alexander Block
On Mon, Jul 23, 2012 at 5:17 PM, Alex Lyakas alex.bolshoy.bt...@gmail.com wrote: Hi Alexander, I did some testing of the case where same inode, but with a different generation, exists both in send_root and in parent_root. I know that this can happen primarily when inode_cache option is

Btrfs send/receive fixes and PULL request

2012-08-02 Thread Alexander Block
. Alexander Block (23): Btrfs: add rdev to get_inode_info in send/receive Btrfs: fix cur_ino parent_ino case for send/receive Btrfs: add missing check for dir != tmp_dir to is_first_ref Btrfs: remove unused code with #if 0 Btrfs: add correct parent to check_dirs

Re: Btrfs send/receive fixes and PULL request

2012-08-02 Thread Alexander Block
On Thu, Aug 2, 2012 at 11:55 AM, Alexander Block abloc...@googlemail.com wrote: Hello Chris, You can find and pull a lot of fixes for btrfs send/receive in my git repo: git://github.com/ablock84/linux-btrfs.git for-chris These fixes are mostly the results of the reviews from Arne

Re: [current linus/master] deadlock during snapshot, sb_writers

2012-08-02 Thread Alexander Block
On Fri, Aug 3, 2012 at 12:16 AM, David Sterba d...@jikos.cz wrote: Hi, top commit (v3.5-9237) fc6bdb59a501740b28ed3b616641a22c8dc5dd31, observed with btrfs-next/master as well. mount: /dev/sdb on /mnt/sdb type btrfs (rw,relatime,space_cache) fresh mkfs, snapshot stresstest, stuck on first

[PATCH] Btrfs: remove mnt_want_write call in btrfs_mksubvol

2012-08-02 Thread Alexander Block
however tried to move all calls to mnt_want_write above i_mutex. So somewhere while merging this, it got mixed up. The solution is to remove the mnt_want_write call completely from mksubvol. Reported-by: David Sterba d...@jikos.cz Signed-off-by: Alexander Block abloc...@googlemail.com --- fs/btrfs

Re: [PATCH] Btrfs: remove mnt_want_write call in btrfs_mksubvol

2012-08-03 Thread Alexander Block
On Fri, Aug 3, 2012 at 11:13 PM, Josef Bacik jba...@fusionio.com wrote: On Thu, Aug 02, 2012 at 05:16:20PM -0600, Alexander Block wrote: We got a recursive lock in mksubvol because the caller already held a lock. I think we got into this due to a merge error. Commit a874a63 removed

Re: [RFC PATCH 0/5] btrfs-progs: snapshot diff function

2012-08-25 Thread Alexander Block
On Fri, Aug 24, 2012 at 4:15 PM, Jeff Liu jeff@oracle.com wrote: Hi Alex, Thanks for taking a look. On 08/24/2012 09:09 PM, Alex Lyakas wrote: Hi Jeff, how do you see this snapshot-diff functionality vs the send/receive functionality that was recently added? I think that the binary

Re: [ablock84-btrfs:btrfs-far 20/20] fs/btrfs/send.c:1683:51: sparse: incorrect type in argument 5 (different base types)

2012-11-06 Thread Alexander Block
[usertype] ctransid vim +1683 fs/btrfs/send.c 31db9f7c Alexander Block 2012-07-25 1677 read_extent_buffer(leaf, name, (unsigned long)(ref + 1), namelen); 31db9f7c Alexander Block 2012-07-25 1678 btrfs_release_path(path); 31db9f7c Alexander Block 2012-07-25 1679 f9a5bb45 Alexander

[PATCH] Btrfs: merge inode_list in __merge_refs

2012-11-08 Thread Alexander Block
When __merge_refs merges two refs, it is also needed to merge the inode_list of both refs. Otherwise we have missed backrefs and memory leaks. This happens for example if two inodes share an extent and both lie in the same leaf and thus also have the same parent. Signed-off-by: Alexander Block

Re: [PATCH] Btrfs: merge inode_list in __merge_refs

2012-11-08 Thread Alexander Block
On Thu, Nov 8, 2012 at 10:27 PM, Alexander Block abloc...@googlemail.com wrote: When __merge_refs merges two refs, it is also needed to merge the inode_list of both refs. Otherwise we have missed backrefs and memory leaks. This happens for example if two inodes share an extent and both lie

Re: Bug in a couple of btrfs-progs components

2012-11-12 Thread Alexander Block
On Sat, Nov 10, 2012 at 9:31 AM, Alun alun.bt...@ty-penguin.org.uk wrote: Hi, (I'm not a list member and may not see any responses to this) I've been using btrfs on my Raspberry Pi. Yesterday I tried to use the btrfs scrub function and, while the scrub started OK, I couldn't run btrfs scrub

Re: [PATCH] Btrfs: send - sparse file support for btrfs-send mechanism

2012-11-29 Thread Alexander Block
On Thu, Nov 29, 2012 at 4:11 AM, Chen Yang chenyang.f...@cn.fujitsu.com wrote: when send/receive a sparse file, the holes of the original file will be filled with zero. The holes will be sent as ZERO streams, and it's unnecessary. So, I improved this by skipping the hole of file while

Re: [PATCH] Btrfs: send - pre-allocated file support for btrfs-send mechanism

2012-11-29 Thread Alexander Block
On Thu, Nov 29, 2012 at 7:06 AM, Chen Yang chenyang.f...@cn.fujitsu.com wrote: From 4222aa3a32cf4db161fcbbb87ad29b714f1bfaf2 Mon Sep 17 00:00:00 2001 From: Chen Yang chenyang.f...@cn.fujitsu.com Date: Thu, 29 Nov 2012 13:22:09 +0800 Subject: [PATCH] Btrfs: send - pre-allocated file support

Re: btrfs and backups

2012-03-26 Thread Alexander Block
On Mon, Mar 26, 2012 at 4:26 PM, Duncan 1i5t5.dun...@cox.net wrote: Fajar A. Nugraha posted on Mon, 26 Mar 2012 16:01:54 +0700 as excerpted: On Mon, Mar 26, 2012 at 3:56 PM, Felix Blanke felixbla...@gmail.com wrote: On 3/26/12 10:30 AM, James Courtier-Dutton wrote: Is there some tool like

State of btrfs send/receive

2012-03-26 Thread Alexander Block
Hello, I used btrfs some time to do incremental backups of large (200GB) VM images, which worked fine using snapshots and rsync (with --inplace to preserve COW). The only problem with this solution is that it takes extremely long due to rsync's performance on large files. For this reason, I can't

Details about compression and extents

2012-04-12 Thread Alexander Block
Hello, I'm currently trying to understand how compression in btrfs works. I could not find any detailed description about it. So here are my questions. 1. How is decided what to compress and what not? After a fast test with a 2g image file, I've looked into the extents of that file with find-new

Re: Details about compression and extents

2012-04-12 Thread Alexander Block
, Alexander Block abloc...@googlemail.com wrote: Hello, I'm currently trying to understand how compression in btrfs works. I could not find any detailed description about it. So here are my questions. 1. How is decided what to compress and what not? After a fast test with a 2g image file, I've

Questions about file extents and alignment

2012-04-13 Thread Alexander Block
Hello, Using find-new, I've seen that all regular extents seem to be 4k aligned. Inline extents were not aligned. Is it guaranteed that all regular extents are 4k aligned in disk_bytenr, disk_num_bytes, offset and num_bytes? Even if COW or what ever may happen happens? Thanks, Alex. -- To

[PATCH 1/1] btrfs: Fix mismatching struct members in ioctl.h

2012-05-03 Thread Alexander Block
Fix the size members of btrfs_ioctl_ino_path_args and btrfs_ioctl_logical_ino_args. The user space btrfs-progs utilities used __u64 and the kernel headers used __u32 before. Signed-off-by: Alexander Block abloc...@googlemail.com --- fs/btrfs/ioctl.h |4 ++-- 1 files changed, 2 insertions

[PATCH 1/3] btrfs: Fix missed backrefs in backref walking code

2012-05-04 Thread Alexander Block
__merge_refs was deleting unresolved prelim refs resulting in missed backrefs in the backref walking code. Thanks to Arne and Jan for finding this. Signed-off-by: Alexander Block abloc...@googlemail.com --- fs/btrfs/backref.c |2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff

[PATCH 2/3] btrfs: Fix ulist related problems in backref walking code

2012-05-04 Thread Alexander Block
, which then allocates a new ulist for the roots. In case the first while loops loops more then once, we leak the previous ulist as it is not freed. Point 1 was also causing missed backrefs because the iteration stopped too early. Signed-off-by: Alexander Block abloc...@googlemail.com --- fs

[PATCH 3/3] btrfs: Update comment above ulist_next

2012-05-04 Thread Alexander Block
The comment above ulist_next stated that it's allowed to call ulist_add while enumerating. This is actually not allowed as an add may realocate the nodes buffer und thus make the prev pointer invalid. Signed-off-by: Alexander Block abloc...@googlemail.com --- fs/btrfs/ulist.c |4 ++-- 1

[BUG] atime on ro snapshots is updated when it should not

2012-05-24 Thread Alexander Block
Hello, if a snapshot was created with -r and thus is read only, accessing files in it will update the atime. I would expect that atime is not updated on ro snapshots. I tried to find out where the ro check is missing. The problem seems to be that the vfs is only checking the mount, super block

Re: [BUG] atime on ro snapshots is updated when it should not

2012-05-25 Thread Alexander Block
David just pointed me to the update_time patches from Josef. When these get into the kernel, the RO snapshot + atime problem could be easily fixed. Any idea when those get into the kernel? On Thu, May 24, 2012 at 7:12 PM, Alexander Block abloc...@googlemail.com wrote: Hello, if a snapshot

[PATCH] Btrfs: don't update atime on RO subvolumes

2012-05-25 Thread Alexander Block
Before the update_time inode operation was indroduced, it was not possible to prevent updates of atime on RO subvolumes. btrfs_update_time does now check if the root is RO and skip updating of atime. This patch requires the update_time patches from Josef Bacik. Signed-off-by: Alexander Block

Btrfs, snapshots and atime problems

2012-05-25 Thread Alexander Block
Hello, I would like to start a discussion on atime in Btrfs (and possibly other filesystems with snapshot support). As atime is updated on every access of a file or directory, we get many changes to the trees in btrfs that as always trigger cow operations. This is no problem as long as the

Re: Btrfs, snapshots and atime problems

2012-05-25 Thread Alexander Block
On Fri, May 25, 2012 at 5:15 PM, Alexander Block abloc...@googlemail.com wrote: Hello, I would like to start a discussion on atime in Btrfs (and possibly other filesystems with snapshot support). As atime is updated on every access of a file or directory, we get many changes to the trees

atime and filesystems with snapshots (especially Btrfs)

2012-05-25 Thread Alexander Block
Hello, (this is a resend with proper CC for linux-fsdevel and linux-kernel) I would like to start a discussion on atime in Btrfs (and other filesystems with snapshot support). As atime is updated on every access of a file or directory, we get many changes to the trees in btrfs that as always

Re: atime and filesystems with snapshots (especially Btrfs)

2012-05-25 Thread Alexander Block
On Fri, May 25, 2012 at 5:42 PM, Josef Bacik jo...@redhat.com wrote: On Fri, May 25, 2012 at 05:35:37PM +0200, Alexander Block wrote: Hello, (this is a resend with proper CC for linux-fsdevel and linux-kernel) I would like to start a discussion on atime in Btrfs (and other filesystems

Re: atime and filesystems with snapshots (especially Btrfs)

2012-05-25 Thread Alexander Block
On Fri, May 25, 2012 at 6:28 PM, Andreas Dilger adil...@dilger.ca wrote: On 2012-05-25, at 9:59, Alexander Block abloc...@googlemail.com wrote: Are you talking about the atime for the primary copy, or the atime for the snapshots?  IMHO, the atime should not be updated for a snapshot unless

Re: atime and filesystems with snapshots (especially Btrfs)

2012-05-25 Thread Alexander Block
On Fri, May 25, 2012 at 6:32 PM, Freddie Cash fjwc...@gmail.com wrote: On May 25, 2012 9:00 AM, Alexander Block abloc...@googlemail.com wrote: On Fri, May 25, 2012 at 5:42 PM, Josef Bacik jo...@redhat.com wrote: On Fri, May 25, 2012 at 05:35:37PM +0200, Alexander Block wrote: Hello

Re: atime and filesystems with snapshots (especially Btrfs)

2012-05-25 Thread Alexander Block
On Fri, May 25, 2012 at 5:35 PM, Alexander Block abloc...@googlemail.com wrote: Hello, (this is a resend with proper CC for linux-fsdevel and linux-kernel) I would like to start a discussion on atime in Btrfs (and other filesystems with snapshot support). As atime is updated on every

Re: atime and filesystems with snapshots (especially Btrfs)

2012-05-25 Thread Alexander Block
On Fri, May 25, 2012 at 10:27 PM, Peter Maloney peter.malo...@brockmann-consult.de wrote: On 05/25/2012 09:10 PM, Alexander Block wrote: Just to show some numbers I made a simple test on a fresh btrfs fs. I copied my hosts /usr (4 gig) folder to that fs and checked metadata usage with btrfs fi

Re: atime and filesystems with snapshots (especially Btrfs)

2012-05-25 Thread Alexander Block
On Fri, May 25, 2012 at 10:42 PM, Alexander Block abloc...@googlemail.com wrote: On Fri, May 25, 2012 at 10:27 PM, Peter Maloney peter.malo...@brockmann-consult.de wrote: On 05/25/2012 09:10 PM, Alexander Block wrote: Just to show some numbers I made a simple test on a fresh btrfs fs. I

Re: atime and filesystems with snapshots (especially Btrfs)

2012-05-29 Thread Alexander Block
On Tue, May 29, 2012 at 10:14 AM, Boaz Harrosh bharr...@panasas.com wrote: Sounds like a real problem. I would suggest a few remedies. 1. Make a filesystem persistent parameter that says noatime/relatime/atime   So the default if not specified on mount is taken as a property of   the FS (mkfs

Re: inquiry on btrfs send/receive

2012-06-04 Thread Alexander Block
On Mon, Jun 4, 2012 at 2:39 PM, Alex Lyakas alex.bolshoy.bt...@gmail.com wrote: Hi Jan, Alex, I have seen some discussions about btrfs send/receive functionality being developed by you. I have also been interested in this. I spent some time coding a prototype doing something like Alex

Re: inquiry on btrfs send/receive

2012-06-04 Thread Alexander Block
On Mon, Jun 4, 2012 at 5:10 PM, shyam btrfs shyam.bt...@gmail.com wrote: Hi Alex, Jan, I was also interested in send/receive semantics was thinking that if we adhere to the semantics as in http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg07482.html of: it is impossible to track

Re: inquiry on btrfs send/receive

2012-06-05 Thread Alexander Block
On Mon, Jun 4, 2012 at 7:33 PM, Alex Lyakas alex.bolshoy.bt...@gmail.com wrote: Yes, I also noticed that sometimes transid gets bumped up, but there is no actual change. So let's say you identify that a particular part of the tree is not shared anymore, and, say, eventually you get to a

[PATCH v2] Btrfs: don't update atime on RO subvolumes

2012-06-15 Thread Alexander Block
. Signed-off-by: Alexander Block abloc...@googlemail.com --- fs/btrfs/inode.c |5 + fs/inode.c |2 ++ 2 files changed, 7 insertions(+) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index f6ab6f5..4d0ceed 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -4478,6 +4478,11

[PATCH 0/2] Fix bugs in backref resolving code

2012-06-19 Thread Alexander Block
to Jan Schmidt for his help here. Alexander Block (2): Btrfs: introduce btrfs_next_old_item Btrfs: don't assume to be on the correct extent in add_all_parents fs/btrfs/backref.c | 94 +--- fs/btrfs/ctree.h |9 +++-- 2 files changed, 59

[PATCH 1/2] Btrfs: introduce btrfs_next_old_item

2012-06-19 Thread Alexander Block
We introduce btrfs_next_old_item that uses btrfs_next_old_leaf instead of btrfs_next_leaf. btrfs_next_item is also changed to simply call btrfs_next_old_item with time_seq being 0. Signed-off-by: Alexander Block abloc...@googlemail.com --- fs/btrfs/ctree.h |9 +++-- 1 file changed, 7

[PATCH 2/2] Btrfs: don't assume to be on the correct extent in add_all_parents

2012-06-19 Thread Alexander Block
) in add_all_parents (leading to missed backrefs or even crashes). Signed-off-by: Alexander Block abloc...@googlemail.com Signed-off-by: Jan Schmidt list.bt...@jan-o-sch.net --- fs/btrfs/backref.c | 94 +--- 1 file changed, 52 insertions(+), 42

[PATCH] Allow cross subvolume reflinks (2nd attempt)

2012-06-20 Thread Alexander Block
Hello, This is the second attempt to bring in cross subvolume reflinks into btrfs. The first attempt was NAKed due to missing vfs mount checks and a clear description of what btrfs subvolumes are and probably also why cross subvolume reflinks are ok in the case of btrfs. This version of the

[PATCH] btrfs: allow cross-subvolume file clone

2012-06-20 Thread Alexander Block
From: David Sterba dste...@suse.cz Lift the EXDEV condition and allow different root trees for files being cloned, then pass source inode's root when searching for extents. Cloning is not allowed to cross vfsmounts, ie. when two subvolumes from one filesystem are mounted separately.

Re: [PATCH] Allow cross subvolume reflinks (2nd attempt)

2012-06-20 Thread Alexander Block
On Wed, Jun 20, 2012 at 7:18 PM, Goffredo Baroncelli kreij...@libero.it wrote: Hi Alexander, On 06/20/2012 12:35 PM, Alexander Block wrote: The patch also does proper vfs mount checks, so cross mount point reflinks are not possible with this patch. It only allows cross reflinks between two

Re: [PATCH] Allow cross subvolume reflinks (2nd attempt)

2012-06-20 Thread Alexander Block
On Wed, Jun 20, 2012 at 8:11 PM, Goffredo Baroncelli kreij...@libero.it wrote: Yes please, could someone explain the reason behind this decision ? May be there are valid reasons, I am asking only to know which ones ? The reason is that at the moment no user visible operations span mount

[PATCH 0/5] introduce btrfs filesystem property command

2012-06-24 Thread Alexander Block
. Alexander Block (5): Btrfs-progs: add BTRFS_IOC_SUBVOL_GET/SETFLAGS to ioctl.h Btrfs-progs: move skip_prefix and prefixcmp to utils.c Btrfs-progs: let get_label return the label instead of of printing it Btrfs-progs: make filesystem_cmd_group non const Btrfs-progs: introduce btrfs

[PATCH 1/5] Btrfs-progs: add BTRFS_IOC_SUBVOL_GET/SETFLAGS to ioctl.h

2012-06-24 Thread Alexander Block
Btrfs send/receive needs this ioctl to make a subvolume r/o after it was received. --- ioctl.h |2 ++ 1 file changed, 2 insertions(+) diff --git a/ioctl.h b/ioctl.h index f2e5d8d..d6311f6 100644 --- a/ioctl.h +++ b/ioctl.h @@ -312,6 +312,8 @@ struct btrfs_ioctl_logical_ino_args {

[PATCH 2/5] Btrfs-progs: move skip_prefix and prefixcmp to utils.c

2012-06-24 Thread Alexander Block
Needed by btrfs fi prop. Signed-off-by: Alexander Block abloc...@googlemail.com --- btrfs.c| 17 ++--- commands.h |2 -- help.c |2 ++ utils.c| 15 +++ utils.h|3 +++ 5 files changed, 22 insertions(+), 17 deletions(-) diff --git a/btrfs.c

[PATCH 5/5] Btrfs-progs: introduce btrfs filesystem property command

2012-06-24 Thread Alexander Block
property for devices. Signed-off-by: Alexander Block abloc...@googlemail.com --- Makefile |3 +- btrfs.c |2 + cmds-filesystem.c | 98 commands.h|3 + props.c | 460 + props.h

[PATCH 4/5] Btrfs-progs: make filesystem_cmd_group non const

2012-06-24 Thread Alexander Block
Make filesystem_cmd_group non const so that btrfs fi prop can generate the usage string at runtime. The commands member of struct cmd_group is also non const now. Signed-off-by: Alexander Block abloc...@googlemail.com --- cmds-filesystem.c |2 +- commands.h|4 ++-- 2 files

[PATCH 3/5] Btrfs-progs: let get_label return the label instead of of printing it

2012-06-24 Thread Alexander Block
get_label prints the label at the moment. Change this so that the label is returned and printing is done by the caller. Also bail out when open_ctree failed to avoid a crash when btrfs fi label is called on a device with no btrfs on it. Signed-off-by: Alexander Block abloc...@googlemail.com

Re: [PATCH 4/5] Btrfs-progs: make filesystem_cmd_group non const

2012-06-24 Thread Alexander Block
I've removed this patch in the mean time. It was needed to build the usage string for btrfs fi props at runtime but I found a solution to do this at compile time now. On Sun, Jun 24, 2012 at 11:20 PM, Alexander Block abloc...@googlemail.com wrote: Make filesystem_cmd_group non const so

[PATCH] Btrfs: use _IOR for BTRFS_IOC_SUBVOL_GETFLAGS

2012-06-25 Thread Alexander Block
We used the wrong ioctl macro for the getflags ioctl before. As we don't have the set/getflags ioctls in the user space ioctl.h at the moment, it's safe to fix it now. Signed-off-by: Alexander Block abloc...@googlemail.com --- fs/btrfs/ioctl.h |2 +- 1 file changed, 1 insertion(+), 1

Re: [PATCH 0/5] introduce btrfs filesystem property command

2012-06-27 Thread Alexander Block
On Wed, Jun 27, 2012 at 4:25 AM, Liu Bo liubo2...@cn.fujitsu.com wrote: On 06/25/2012 05:20 AM, Alexander Block wrote: This patchset introduces the btrfs filesystem property command. It is the result of a discussion we had on IRC. I tried to make the properties interface as generic

[RFC PATCHv2 0/3] Btrfs-progs: introduce btrfs property subgroup

2012-06-27 Thread Alexander Block
and David. Alex. Alexander Block (3): Btrfs-progs: add BTRFS_IOC_SUBVOL_GET/SETFLAGS to ioctl.h Btrfs-progs: let get_label return the label instead of of printing it Btrfs-progs: introduce btrfs property subgroup Makefile |5 +- btrfs.c |1 + btrfslabel.c | 13

[RFC PATCHv2 1/3] Btrfs-progs: add BTRFS_IOC_SUBVOL_GET/SETFLAGS to ioctl.h

2012-06-27 Thread Alexander Block
Btrfs send/receive and btrfs props needs this ioctl. This patch requires a recent kernel with the Btrfs: use _IOR for BTRFS_IOC_SUBVOL_GETFLAGS patch applied. Signed-off-by: Alexander Block abloc...@googlemail.com --- ioctl.h |2 ++ 1 file changed, 2 insertions(+) diff --git a/ioctl.h b

[RFC PATCHv2 2/3] Btrfs-progs: let get_label return the label instead of of printing it

2012-06-27 Thread Alexander Block
get_label prints the label at the moment. Change this so that the label is returned and printing is done by the caller. Also bail out when open_ctree failed to avoid a crash when btrfs fi label is called on a device with no btrfs on it. Signed-off-by: Alexander Block abloc...@googlemail.com

  1   2   >