Re: [PATCH] Btrfs: clear received_uuid field for new writable snapshots

2013-05-22 Thread Alex Lyakas
Hi Stephan, I fully understand the first part of your fix, and I believe it's quite critical. Indeed, a writable snapshot should have no evidence that it has an ancestor that was once received. Can you pls let me know that I understand the second part of your fix. In btrfs-progs the following

[no subject]

2013-05-28 Thread Alex Lyakas
Hello all, I have the following unresponsive btrfs: btrfs_end_transaction() is called and is stuck in btrfs_tree_lock(): May 27 16:13:55 vc kernel: [ 7130.421159] kworker/u:85D 0 19859 2 0x May 27 16:13:55 vc kernel: [ 7130.421159] 880095335568

wait_block_group_cache_progress() waits forever in case of drive failure

2013-06-04 Thread Alex Lyakas
Greetings all, when testing drive failures, I occasionally hit the following hang: # Block group is being cached-in by caching_thread() # caching_thread() experiences an error, e.g., in btrfs_search_slot, because of drive failure: ret = btrfs_search_slot(NULL, extent_root, key, path, 0,

Re: [PATCH 2/3] Btrfs: fix the deadlock between the transaction start/attach and commit

2013-06-12 Thread Alex Lyakas
kernel (manually, of course)? Or some additional things are needed that are missing in this kernel? Thanks, Alex. Thanks Miao On Wed, 10 Apr 2013 21:45:43 +0300, Alex Lyakas wrote: Hi Miao, I attempted to fix the issue by not joining a transaction that has trans-in_commit set. I did

Re: [PATCH 2/3] Btrfs: fix the deadlock between the transaction start/attach and commit

2013-06-16 Thread Alex Lyakas
Hi Miao, On Thu, Jun 13, 2013 at 6:08 AM, Miao Xie mi...@cn.fujitsu.com wrote: On wed, 12 Jun 2013 23:11:02 +0300, Alex Lyakas wrote: I reviewed the code starting from: 69aef69a1bc154 Btrfs: don't wait for all the writers circularly during the transaction commit until 2ce7935bf4cdf3 Btrfs

question about transaction-abort-related commits

2013-06-23 Thread Alex Lyakas
Hello Josef, Liu, I am reviewing commits in the mainline tree: e4a2bcaca9643e7430207810653222fc5187f2be Btrfs: if we aren't committing just end the transaction if we error out (call end_transaction() instead of goto cleanup_transaction) - Josef f094ac32aba3a51c00e970a2ea029339af2ca048 Btrfs: fix

Re: question about transaction-abort-related commits

2013-06-24 Thread Alex Lyakas
. On Mon, Jun 24, 2013 at 7:24 PM, Josef Bacik jba...@fusionio.com wrote: On Sun, Jun 23, 2013 at 09:52:14PM +0300, Alex Lyakas wrote: Hello Josef, Liu, I am reviewing commits in the mainline tree: e4a2bcaca9643e7430207810653222fc5187f2be Btrfs: if we aren't committing just end the transaction if we

Re: [PATCH 2/3] Btrfs: fix the deadlock between the transaction start/attach and commit

2013-06-26 Thread Alex Lyakas
Hi Miao, On Mon, Jun 17, 2013 at 4:51 AM, Miao Xie mi...@cn.fujitsu.com wrote: On sun, 16 Jun 2013 13:38:42 +0300, Alex Lyakas wrote: Hi Miao, On Thu, Jun 13, 2013 at 6:08 AM, Miao Xie mi...@cn.fujitsu.com wrote: On wed, 12 Jun 2013 23:11:02 +0300, Alex Lyakas wrote: I reviewed

Re: question about transaction-abort-related commits

2013-06-30 Thread Alex Lyakas
Hi Josef, On Wed, Jun 26, 2013 at 5:16 PM, Alex Lyakas alex.bt...@zadarastorage.com wrote: Hi Josef, Can you please help me with another question. I am looking at your patch: Btrfs: fix chunk allocation error handling https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit

Re: question about transaction-abort-related commits

2013-07-02 Thread Alex Lyakas
On Sun, Jun 30, 2013 at 2:36 PM, Josef Bacik jba...@fusionio.com wrote: On Sun, Jun 30, 2013 at 11:29:14AM +0300, Alex Lyakas wrote: Hi Josef, On Wed, Jun 26, 2013 at 5:16 PM, Alex Lyakas alex.bt...@zadarastorage.com wrote: Hi Josef, Can you please help me with another question. I am

Re: [PATCH v3] btrfs: clean snapshots one by one

2013-07-04 Thread Alex Lyakas
Hi David, I believe this patch has the following problem: On Tue, Mar 12, 2013 at 5:13 PM, David Sterba dste...@suse.cz wrote: Each time pick one dead root from the list and let the caller know if it's needed to continue. This should improve responsiveness during umount and balance which at

Re: [PATCH v3] btrfs: clean snapshots one by one

2013-07-04 Thread Alex Lyakas
Hi David, On Thu, Jul 4, 2013 at 8:03 PM, David Sterba dste...@suse.cz wrote: On Thu, Jul 04, 2013 at 06:29:23PM +0300, Alex Lyakas wrote: @@ -7363,6 +7365,12 @@ int btrfs_drop_snapshot(struct btrfs_root *root, wc-reada_count = BTRFS_NODEPTRS_PER_BLOCK(root); while (1

Re: [PATCH v3] btrfs: clean snapshots one by one

2013-07-14 Thread Alex Lyakas
Hi, On Thu, Jul 4, 2013 at 10:52 PM, Alex Lyakas alex.bt...@zadarastorage.com wrote: Hi David, On Thu, Jul 4, 2013 at 8:03 PM, David Sterba dste...@suse.cz wrote: On Thu, Jul 04, 2013 at 06:29:23PM +0300, Alex Lyakas wrote: @@ -7363,6 +7365,12 @@ int btrfs_drop_snapshot(struct btrfs_root

Re: [PATCH] Btrfs: fix lock leak when resuming snapshot deletion

2013-07-16 Thread Alex Lyakas
the extent buffer from cache. Thanks, Reported-by: Alex Lyakas alex.bt...@zadarastorage.com Signed-off-by: Josef Bacik jba...@fusionio.com --- fs/btrfs/extent-tree.c |2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index

Re: [PATCH] Btrfs: update drop progress before stopping snapshot dropping

2013-07-30 Thread Alex Lyakas
the snapshot dropping without updating the drop progress. So the next time we go to resume we will BUG_ON() because we can't find the extent we left off at because we never updated it. This patch fixes the problem. Cc: sta...@vger.kernel.org Reported-by: Alex Lyakas alex.bt

Re: [PATCH] Btrfs: fix all callers of read_tree_block

2013-07-30 Thread Alex Lyakas
Hi Josef, On Tue, Apr 23, 2013 at 9:20 PM, Josef Bacik jba...@fusionio.com wrote: We kept leaking extent buffers when mounting a broken file system and it turns out it's because not everybody uses read_tree_block properly. You need to check and make sure the extent_buffer is uptodate before

Re: [PATCH] Btrfs: stop all workers before cleaning up roots

2013-08-01 Thread Alex Lyakas
Hi Josef, On Thu, May 30, 2013 at 11:58 PM, Josef Bacik jba...@fusionio.com wrote: Dave reported a panic because the extent_root-commit_root was NULL in the caching kthread. That is because we just unset it in free_root_pointers, which is not the correct thing to do, we have to either wait

[PATCH] Notify caching_thread()s to give up on extent_commit_sem when needed.

2013-08-29 Thread Alex Lyakas
this problem happens, and whether the fix goes in the right direction. Signed-off-by: Alex Lyakas alex.bt...@zadarastorage.com --- fs/btrfs/ctree.h |7 +++ fs/btrfs/disk-io.c |1 + fs/btrfs/extent-tree.c |9 + fs/btrfs/transaction.c |2 +- 4 files changed, 14

Re: [PATCH] Notify caching_thread()s to give up on extent_commit_sem when needed.

2013-08-29 Thread Alex Lyakas
On Thu, Aug 29, 2013 at 10:55 PM, Josef Bacik jba...@fusionio.com wrote: On Thu, Aug 29, 2013 at 10:09:29PM +0300, Alex Lyakas wrote: Hi Josef, On Thu, Aug 29, 2013 at 5:38 PM, Josef Bacik jba...@fusionio.com wrote: On Thu, Aug 29, 2013 at 01:31:05PM +0300, Alex Lyakas wrote

Re: btrfs:async-thread: atomic_start_pending=1 is set, but it's too late

2013-08-29 Thread Alex Lyakas
Thanks, Chris, Josef, for confirming! On Thu, Aug 29, 2013 at 11:08 PM, Chris Mason clma...@fusionio.com wrote: Quoting Josef Bacik (2013-08-29 16:03:06) On Mon, Aug 26, 2013 at 05:16:42PM +0300, Alex Lyakas wrote: Greetings all, I see a following issue with spawning new threads

Re: [PATCH] btrfs: Fix 32/64-bit problem with BTRFS_SET_RECEIVED_SUBVOL ioctl

2014-02-15 Thread Alex Lyakas
Hello Hugo, Is this issue specific to the receive ioctl? Because what you are describing applies to any user-kernel ioctl-based interface, when you compile the user-space as 32-bit, which the kernel space has been compiled as 64-bit. For that purpose, I believe, there exists the compat_ioctl

Re: [PATCH] Btrfs: fix a deadlock on chunk mutex

2014-02-18 Thread Alex Lyakas
Hello Josef, On Tue, Dec 18, 2012 at 3:52 PM, Josef Bacik jba...@fusionio.com wrote: On Wed, Dec 12, 2012 at 06:52:37PM -0700, Liu Bo wrote: An user reported that he has hit an annoying deadlock while playing with ceph based on btrfs. Current updating device tree requires space from METADATA

Re: [PATCH] Btrfs: fix a deadlock on chunk mutex

2014-02-18 Thread Alex Lyakas
: SHA1 On 02/18/2014 10:47 AM, Alex Lyakas wrote: Hello Josef, On Tue, Dec 18, 2012 at 3:52 PM, Josef Bacik jba...@fusionio.com wrote: On Wed, Dec 12, 2012 at 06:52:37PM -0700, Liu Bo wrote: An user reported that he has hit an annoying deadlock while playing with ceph based on btrfs

Re: [RFC PATCH 5/5] Btrfs: fix broken free space cache after the system crashed

2014-03-01 Thread Alex Lyakas
Hi Miao, On Wed, Jan 15, 2014 at 2:00 PM, Miao Xie mi...@cn.fujitsu.com wrote: When we mounted the filesystem after the crash, we got the following message: BTRFS error (device xxx): block group 4315938816 has wrong amount of free space BTRFS error (device xxx): failed to load free

Re: [RFC PATCH 5/5] Btrfs: fix broken free space cache after the system crashed

2014-03-08 Thread Alex Lyakas
, the transaction commits and we crash, then yes, the allocation is lost. Alex. On Tue, Mar 4, 2014 at 8:04 AM, Miao Xie mi...@cn.fujitsu.com wrote: On sat, 1 Mar 2014 20:05:01 +0200, Alex Lyakas wrote: Hi Miao, On Wed, Jan 15, 2014 at 2:00 PM, Miao Xie mi...@cn.fujitsu.com wrote: When we mounted

Re: [PATCH] Btrfs: fix memory leak in btrfs_create_tree()

2014-03-27 Thread Alex Lyakas
Hi Tsutomu Itoh, On Thu, Mar 21, 2013 at 6:32 AM, Tsutomu Itoh t-i...@jp.fujitsu.com wrote: We should free leaf and root before returning from the error handling code. Signed-off-by: Tsutomu Itoh t-i...@jp.fujitsu.com --- fs/btrfs/disk-io.c | 12 +--- 1 file changed, 9

Re: [PATCH] Btrfs: attach delayed ref updates to delayed ref heads

2014-03-30 Thread Alex Lyakas
Hi Josef, I have a question about update_existing_head_ref() logic. The question is not specific to the rework that you have done. You have a code like this: if (ref-must_insert_reserved) { /* if the extent was freed and then * reallocated before the delayed ref *

Re: [PATCH] Btrfs: correctly determine if blocks are shared in btrfs_compare_trees

2014-04-05 Thread Alex Lyakas
Hi Filipe, Can you please explain more what is the scenario you are worried about. Let's say we have two FS trees (subvolumes) subv1 and subv2, subv2 being a RO snapshot of subv1. And they have a shared subtree at logical==X. Now we change subv1, so its subtree is COW'ed and some other logical

Re: snapshot send with parent question

2014-05-31 Thread Alex Lyakas
Michael, btrfs-send doesn't really know or care how did you manage to get from a to c. It is able to compare any two RO subvolumes (not necessarily related by snapshot operations), and produce a stream of commands that transfer a-content into c-content. Send assumes that at a receive side, you

Re: [RFC PATCH 5/7] Btrfs: add btrfs_compare_trees function

2012-07-04 Thread Alex Lyakas
Hi Alex, + spin_lock(left_root-root_times_lock); + ctransid = btrfs_root_ctransid(left_root-root_item); + spin_unlock(left_root-root_times_lock); + if (ctransid != left_start_ctransid) +

Re: [RFC PATCH 5/7] Btrfs: add btrfs_compare_trees function

2012-07-04 Thread Alex Lyakas
Hi Alex, +static int tree_compare_item(struct btrfs_root *left_root, +struct btrfs_path *left_path, +struct btrfs_path *right_path, +char *tmp_buf) +{ + int cmp; + int len1, len2; +

Re: [RFC PATCH 5/7] Btrfs: add btrfs_compare_trees function

2012-07-05 Thread Alex Lyakas
Alexander, + if (advance_left !left_end_reached) { + ret = tree_advance(left_root, left_path, left_level, + left_root_level, + advance_left != ADVANCE_ONLY_NEXT, +

Re: [RFC PATCH 5/7] Btrfs: add btrfs_compare_trees function

2012-07-05 Thread Alex Lyakas
On Thu, Jul 5, 2012 at 3:47 PM, Alexander Block abloc...@googlemail.com wrote: On Thu, Jul 5, 2012 at 2:19 PM, Alex Lyakas alex.bolshoy.bt...@gmail.com wrote: Alexander, + if (advance_left !left_end_reached) { + ret = tree_advance(left_root, left_path

Re: [RFC PATCH 6/6] Btrfs-progs: add btrfs send/receive commands

2012-07-09 Thread Alex Lyakas
Hi Alexander, I studied all the kernel + user code, and I have a long list of questions. Meanwhile, I want to report two small bugs: # BTRFS_SEND_C_MKNOD command: the receive path expects BTRFS_SEND_A_MODE, but kernel doesn't send it. So currently it errors. It looks like kernel need to send

Re: [RFC PATCH 7/7] Btrfs: introduce BTRFS_IOC_SEND for btrfs send/receive (part 2)

2012-07-10 Thread Alex Lyakas
Alexander, this focuses on area of sending file extents: +static int is_extent_unchanged(struct send_ctx *sctx, + struct btrfs_path *left_path, + struct btrfs_key *ekey) +{ + int ret = 0; + struct btrfs_key key; +

Re: [PATCH v1 00/15] Btrfs: subvolume quota groups (qgroups)

2012-07-17 Thread Alex Lyakas
Jan, I have studied to some extent the PDF and the code. I have some questions; perhaps you can address them? 1) btrfs_qgroup_account_ref() calling btrfs_find_all_roots(): I understand that bytenr indicates some EXTENT_ITEM, which is a back-reference for extent, which is perhaps a tree block

Re: [PATCH v1 00/15] Btrfs: subvolume quota groups (qgroups)

2012-07-18 Thread Alex Lyakas
Arne, thanks for your comments. On Tue, Jul 17, 2012 at 10:25 PM, Arne Jansen sensi...@gmx.net wrote: On 07/17/2012 08:33 PM, Alex Lyakas wrote: Jan, I have studied to some extent the PDF and the code. I have some questions; perhaps you can address them? 1) btrfs_qgroup_account_ref

Re: [PATCH 1/2] Btrfs: fix btrfs_is_free_space_inode to recognize btree inode

2012-07-18 Thread Alex Lyakas
Hi Liu, can you pls explain this a bit more to me. I see in my fs that there are (FREE_SPACE,0, offset) items in the root tree. It seems that offset here is the objectid of the appropriate BLOCK_GROUP_ITEM. Also, I see that each FREE_SPACE points at some INODE_ITEM within the root tree, and this

Re: [PATCH 1/2] Btrfs: fix btrfs_is_free_space_inode to recognize btree inode

2012-07-18 Thread Alex Lyakas
On Wed, Jul 18, 2012 at 3:02 PM, Liu Bo liubo2...@cn.fujitsu.com wrote: On 07/18/2012 06:02 PM, Alex Lyakas wrote: More or less. As its name shows, a free space inode's data (you name it extents) consists of free space info, meanwhile, a free space inode is issued to a block group, so

btrfs send/receive: if new inode ino is less than its new directory ino, incorrect path is sent

2012-07-18 Thread Alex Lyakas
Hi Alexander, I am testing different scenarios in order to better understand the non-trivial magic of get_cur_path()/will_overwrite_ref()/did_overwrite_ref()/did_overwrite_first_ref(). I hit the following issue, when testing full-send: This is my source subvolume (inode numbers are written): tree

Re: [RFC PATCH 6/6] Btrfs-progs: add btrfs send/receive commands

2012-07-19 Thread Alex Lyakas
+static int process_link(const char *path, const char *lnk, void *user) +{ + int ret; + struct btrfs_receive *r = user; + char *full_path = path_cat(r-full_subvol_path, path); + + if (g_verbose = 1) + fprintf(stderr, link %s - %s\n, path, lnk); + + ret =

Re: [RFC PATCH 7/7] Btrfs: introduce BTRFS_IOC_SEND for btrfs send/receive (part 2)

2012-07-23 Thread Alex Lyakas
Hi Alexander, I did some testing of the case where same inode, but with a different generation, exists both in send_root and in parent_root. I know that this can happen primarily when inode_cache option is enabled. So first I just tested some differential sends, where parent and root are unrelated

Re: [RFC PATCH 7/7] Btrfs: introduce BTRFS_IOC_SEND for btrfs send/receive (part 2)

2012-07-23 Thread Alex Lyakas
Hi Arne, (pls don't take this as if I pretend to have understood the code better than you, because I have a list of questions for Alexander too). +/* + * Helper function to generate a file name that is unique in the root of + * send_root and parent_root. This is used to generate names for

Re: [RFC PATCH 6/6] Btrfs-progs: add btrfs send/receive commands

2012-07-25 Thread Alex Lyakas
...@googlemail.com wrote: On Thu, Jul 19, 2012 at 3:25 PM, Alex Lyakas alex.bolshoy.bt...@gmail.com wrote: +static int process_link(const char *path, const char *lnk, void *user) +{ + int ret; + struct btrfs_receive *r = user; + char *full_path = path_cat(r-full_subvol_path, path

Re: [RFC PATCH 0/6] Experimental btrfs send/receive (btrfs-progs)

2012-07-25 Thread Alex Lyakas
Alexander, can you pls let know like a day or two before you run out of time? I have compiled a list of questions, but also want to do more testing before I publish them all. Thanks for your work, Alex. On Wed, Jul 25, 2012 at 7:56 PM, Alexander Block abloc...@googlemail.com wrote: On Wed, Jul

Re: [RFC PATCH 7/7] Btrfs: introduce BTRFS_IOC_SEND for btrfs send/receive (part 2)

2012-07-25 Thread Alex Lyakas
Alexander, Same is true for BTRFS_FILE_EXTENT_PREALLOC extents, I think. Those also don't contain real data. So something like: if (left_disknr == 0 || left_type == BTRFS_FILE_EXTENT_REG) { ret = 1; goto out; } Do you mean || left_type == BTRFS_FILE_EXTENT_PREALLOC? I see

Re: btrfs send/receive: if new inode ino is less than its new directory ino, incorrect path is sent

2012-07-26 Thread Alex Lyakas
at 7:45 PM, Alex Lyakas alex.bolshoy.bt...@gmail.com wrote: Hi Alexander, I am testing different scenarios in order to better understand the non-trivial magic of get_cur_path()/will_overwrite_ref()/did_overwrite_ref()/did_overwrite_first_ref(). I hit the following issue, when testing full-send

Re: btrfs send/receive: if new inode ino is less than its new directory ino, incorrect path is sent

2012-07-26 Thread Alex Lyakas
is redundant (and probably makes sense to drop ino/gen parameters of them?). Thanks, Alex. On Thu, Jul 26, 2012 at 1:52 PM, Alex Lyakas alex.bolshoy.bt...@gmail.com wrote: Hi Alexander, I did some very initial testing, and there is still an issue. The logic of finish_outoforder_dir works as expected

Re: btrfs send/receive: if new inode ino is less than its new directory ino, incorrect path is sent

2012-07-27 Thread Alex Lyakas
problem. I will create a for-alex branch for you to test later. On Thu, Jul 26, 2012 at 4:04 PM, Alex Lyakas alex.bolshoy.bt...@gmail.com wrote: Alexander, (pls let me know when this gets annoying:). Parent: /mnt/src/v2_snap0/ └── [257] file1 Send: /mnt/src/v2_snap1 └── [259] dir1

Re: btrfs send/receive: if new inode ino is less than its new directory ino, incorrect path is sent

2012-07-29 Thread Alex Lyakas
:37 PM, Alex Lyakas alex.bolshoy.bt...@gmail.com wrote: Hi Alexander, your solution is simple and elegant. I this this issue is solved now. Thanks! Two minor issues: 1) /* * We need some special handling for inodes that get processed before the parent * directory got created. See

Re: btrfs send/receive: if new inode ino is less than its new directory ino, incorrect path is sent

2012-07-30 Thread Alex Lyakas
Hi, I did not apply the patch but instead added a check for dir != tmp_dir only. The reason to not check for gen is that I have a rule in my mind: I only pass the generation number to functions where I want to know the *current* state. is_first_ref is for permanent state, the return value

Questions and notes about send/receive

2012-07-31 Thread Alex Lyakas
Hi Alexander, I relooked at my list of questions, and it seems that there are more general questions, and more focused questions. So here I list the more focused ones. I really appreciate if you can address them. I hope it's ok that I opened a separate thread for them. So here goes...

Re: Questions and notes about send/receive

2012-08-01 Thread Alex Lyakas
Alexander, thanks for addressing the issues. __get_cur_name_and_parent() if we decided to call get_first_ref() on the send_root (due to ino sctx-send_progress), do we really need to call did_overwrite_ref()? Because it will just lookup the send root again. I mean if we know that this inode

Re: [RFC PATCH 0/5] btrfs-progs: snapshot diff function

2012-08-24 Thread Alex Lyakas
Hi Jeff, how do you see this snapshot-diff functionality vs the send/receive functionality that was recently added? I think that the binary stream that the send code produces, can be interpreted by printing out text messages, which will essentially give the same information (although much more

Re: [PATCH] Btrfs-progs: replace find_mount_root from send code

2012-08-27 Thread Alex Lyakas
Hi Robert, do you think you can make the root_snap_send file of yours available anywhere (not sure how big it is)? Also, I am not sure I understand how you btrfs tree looks like WRT to where all subvolumes are mounted (but all others seem to understand well). You mentioned: /dev/loop0 on

Question about commit: Btrfs: fix btrfs send for inline items and compression, a9fa6848e4aae8d777f7af5e9f2d5b11331252dd

2012-08-27 Thread Alex Lyakas
Hi Chris, I am looking at your fix in find_extent_clone(), and the code now looks like this: if (compressed == BTRFS_COMPRESS_NONE) extent_item_pos = logical - found_key.objectid; else extent_item_pos = 0; extent_item_pos = logical - found_key.objectid; So the compressed flag

Re: [PATCH] Btrfs-progs: replace find_mount_root from send code

2012-08-28 Thread Alex Lyakas
Thank you, Robert for the detailed data. I will debug get back to you before Friday. Alex. On Mon, Aug 27, 2012 at 9:37 PM, Robert Buhren rob...@robertbuhren.de wrote: Hi Alex, i've made a list of all commands i executed for this test. You can find it here: http://pastebin.com/y8PBgmMZ

Re: [PATCH] Btrfs-progs: replace find_mount_root from send code

2012-08-28 Thread Alex Lyakas
, + fprintf(stderr, ERROR: failed to open %s. %s\n, r-root_path, strerror(-ret)); goto out; } On Tue, Aug 28, 2012 at 11:27 AM, Alex Lyakas alex.bolshoy.bt...@gmail.com wrote: Thank you, Robert for the detailed data. I

Re: enquiry about defrag

2012-09-10 Thread Alex Lyakas
Hi, You need to compile the latest version of filefrag, it has some fixes by Liu Bo wrt to that. However, even then, filefrag may not report the correct the extent count for btrfs. According to filefrag_fiemap(), it tries to check whether physical extents are consecutive on disk and then

Re: btrfs send/receive review by vfs folks

2012-09-24 Thread Alex Lyakas
Hi, write_buf: Used to write the stream to a user space supplied pipe. Please note the ERESTARTSYS comment there, I need some help here as I don't know how to handle that correctly. If I ignore the return value, it loops forever. If I bail out to user space, it reenters the ioctl and starts

Re: btrfs send/receive review by vfs folks

2012-09-27 Thread Alex Lyakas
Hi Jan, I hope to get my proposal working soon, then expect for some code from me to look at. Thanks! Alex. On Mon, Sep 24, 2012 at 11:27 AM, Jan Schmidt list.bt...@jan-o-sch.net wrote: Hi Alex, On Mon, September 24, 2012 at 11:13 (+0200), Alex Lyakas wrote: Hi, write_buf: Used to write

Re: [PATCH] Btrfs: try to avoid doing a search in btrfs_next_leaf

2012-09-30 Thread Alex Lyakas
Hi Josef, I guess I am missing something, but I thought btrfs_next_leaf() should just jump to the next leaf (or item, if it was added meanwhile) irrespective of the key that is in the last slot of the current leaf. The change you added is effective when there is a next leaf, but you refuse to go

Re: [PATCH] Btrfs: try to avoid doing a search in btrfs_next_leaf

2012-10-02 Thread Alex Lyakas
was somewhat puzzled by your change. If you decide to keep your change, can you pls update the comment of btrfs_next_leaf()/next_old_leaf(). Thanks again, Alex. On Mon, Oct 1, 2012 at 6:45 PM, Josef Bacik jba...@fusionio.com wrote: On Sun, Sep 30, 2012 at 05:28:23AM -0600, Alex Lyakas wrote: Hi

Re: btrfs send/receive review by vfs folks

2012-10-04 Thread Alex Lyakas
Hi Jan, as I promised, here is some code for you to look at. First I will describe the approach in general. # Get rid of the pipe. Instead, user-space passes a buffer and kernel fills the specified user-space buffer with commands. # When the buffer is full, kernel stops generating commands and

Re: btrfs send/receive review by vfs folks

2012-10-07 Thread Alex Lyakas
Matrin, On Sat, Oct 6, 2012 at 11:40 AM, Martin Steigerwald mar...@lichtvoll.de wrote: Am Donnerstag, 4. Oktober 2012 schrieb Alex Lyakas: Hi Jan, as I promised, here is some code for you to look at. First I will describe the approach in general. # Get rid of the pipe. Instead, user-space

Re: btrfs send/receive review by vfs folks

2012-10-07 Thread Alex Lyakas
and handles them. After it is done with the buffer, it puts it back on a free-buffer queue and signals the first thread. This is exactly producer-consumer. Alex. On Sun, Oct 7, 2012 at 12:53 PM, Martin Steigerwald mar...@lichtvoll.de wrote: Am Sonntag, 7. Oktober 2012 schrieb Alex Lyakas: Matrin

Re: btrfs send/receive review by vfs folks

2012-10-08 Thread Alex Lyakas
Hi Jan, thanks for taking time to look at the code. On Mon, Oct 8, 2012 at 11:26 AM, Jan Schmidt list.bt...@jan-o-sch.net wrote: Hi Alex, On Thu, October 04, 2012 at 17:59 (+0200), Alex Lyakas wrote: as I promised, here is some code for you to look at. And quite a lot of it. I hadn't

Re: btrfs receive to subdirectory

2012-10-09 Thread Alex Lyakas
Hi Rory, Arne, I think the problem is that currently mnt_fd in struct btrfs_receive is used both as mount root and directory in which the subvolume/snapshot needs to be created. Arne, does the following patch make sense? It uses Jan's find_mount_root function. With this patch both Rory's tests

Re: btrfs send/receive review by vfs folks

2012-10-10 Thread Alex Lyakas
Hi Jan, On Mon, Oct 8, 2012 at 3:37 PM, Jan Schmidt list.bt...@jan-o-sch.net wrote: On Mon, October 08, 2012 at 13:38 (+0200), Alex Lyakas wrote: I realize this is a big change, and a new IOCTL has to be introduced in order not to break current user-kernel protocol. The pros as I see them

Re: btrfs receive to subdirectory

2012-10-11 Thread Alex Lyakas
, 2012 at 12:11 AM, Rory Campbell-Lange r...@campbell-lange.net wrote: On 09/10/12, Alex Lyakas (alex.bt...@zadarastorage.com) wrote: Hi Rory, Arne, I think the problem is that currently mnt_fd in struct btrfs_receive is used both as mount root and directory in which the subvolume/snapshot needs

Re: [PATCH] Btrfs: extended inode refs support for send mechanism

2012-10-11 Thread Alex Lyakas
Hi Jan, On Wed, Oct 10, 2012 at 1:21 PM, Jan Schmidt list.bt...@jan-o-sch.net wrote: This adds support for the new extended inode refs to btrfs send. Signed-off-by: Jan Schmidt list.bt...@jan-o-sch.net --- fs/btrfs/backref.c | 22 - fs/btrfs/backref.h |4 ++ fs/btrfs/send.c

Re: btrfs receive to subdirectory

2012-10-11 Thread Alex Lyakas
-lange.net wrote: On 11/10/12, Alex Lyakas (alex.bt...@zadarastorage.com) wrote: I am using the latest code from Chris's repo ( git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git), which has been updated recently (latest commit 91d9eec1ff044394f2b98ee7fcb76713dd33b994). Do you think

Re: Two Issues with Btrfs Delayed Cleaner Process (linux-next)

2012-10-12 Thread Alex Lyakas
Hi Mitch, for issue (1) I proposed a small patch here: http://www.spinics.net/lists/linux-btrfs/msg19574.html which resolves it for me. The problem is that there are subvolumes that still exist in the root tree, but don't have their ROOT_BACKREF entry anymore. However, Miao's rework of the

Re: btrfs receive to subdirectory

2012-10-12 Thread Alex Lyakas
) and then with patch2 (instead of any previous). Both will do some amount of prints, pls post them. With both patches your tests work fine for me. Thanks, Alex. On Thu, Oct 11, 2012 at 4:56 PM, Rory Campbell-Lange r...@campbell-lange.net wrote: On 11/10/12, Alex Lyakas (alex.bt...@zadarastorage.com) wrote

Re: btrfs receive to subdirectory

2012-10-13 Thread Alex Lyakas
Rory, thanks for confirming. I will put together a more formal patch to the list. Thanks, Alex. On Fri, Oct 12, 2012 at 6:10 PM, Rory Campbell-Lange r...@campbell-lange.net wrote: On 12/10/12, Alex Lyakas (alex.bt...@zadarastorage.com) wrote: Hi Rory, can you pls check if you have a file

Re: problems with btrfs send / restore

2012-10-15 Thread Alex Lyakas
Hi Stefan, Is /btrfs/target/\@snapshot/ a subvolume or a directory? can you pls try the patch that I posted here: http://www.spinics.net/lists/linux-btrfs/msg19583.html I feel that you're hitting a similar issue here. Before you apply the patch, please verify that you have /etc/mtab on your

Re: [PATCH] Fits: tool to parse stream

2012-10-15 Thread Alex Lyakas
Thanks, Arne, it will definitely help to debug issues. Alex. On Mon, Oct 15, 2012 at 4:32 PM, Chris Mason chris.ma...@fusionio.com wrote: On Sat, Oct 13, 2012 at 09:41:28AM -0600, David Sterba wrote: On Sat, Oct 13, 2012 at 09:08:57AM +0100, Rory Campbell-Lange wrote: Perhaps BTRFS

Re: problems with btrfs send / restore

2012-10-15 Thread Alex Lyakas
, Oct 15, 2012 at 10:06 PM, Stefan Priebe s.pri...@profihost.ag wrote: Am 15.10.2012 21:42, schrieb Alex Lyakas: Is /btrfs/target/\@snapshot/ a subvolume or a directory? A simple directory. can you pls try the patch that I posted here: http://www.spinics.net/lists/linux-btrfs/msg19583.html I

[PATCH] btrfs-progs: Fix the receive code pathing

2012-10-17 Thread Alex Lyakas
...@campbell-lange.net Reported-by: Stefan Priebe - Profihost AG s.pri...@profihost.ag Signed-off-by: Alex Lyakas alex.bt...@zadarastorage.com diff --git a/cmds-receive.c b/cmds-receive.c index a8be6fa..6b7cf12 100644 --- a/cmds-receive.c +++ b/cmds-receive.c @@ -52,11 +52,13 @@ static int

[PATCH] Btrfs: Send: preserve ownership (uid and gid) also for symlinks.

2012-10-17 Thread Alex Lyakas
This patch also requires a change in the user-space part of receive. We need to use lchown instead of chown. We will do this in the following patch. Signed-off-by: Alex Lyakas alex.bt...@zadarastorage.com diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c index c7beb54..7d1eed2 100644 --- a/fs/btrfs

[PATCH] btrfs-progs: Receive: preserve ownership for symlinks, by using 'lchown'

2012-10-17 Thread Alex Lyakas
Signed-off-by: Alex Lyakas alex.bt...@zadarastorage.com diff --git a/cmds-receive.c b/cmds-receive.c index 6b7cf12..a6a6a5b 100644 --- a/cmds-receive.c +++ b/cmds-receive.c @@ -731,7 +731,7 @@ static int process_chown(const char *path, u64 uid, u64 gid, void *user) fprintf(stderr

[PATCH] Btrfs-progs: Filter out deleting or already-deleted subvolumes in lookup_ino_path.

2012-10-17 Thread Alex Lyakas
not do a search if dirid is BTRFS_FIRST_FREE_OBJECTID, so such a subvolume will be considered as still existing, but subvolume listing will not fail at least). Reported-by: Stefan Priebe s.pri...@profihost.ag Signed-off-by: Alex Lyakas alex.bt...@zadarastorage.com diff --git a/btrfs-list.c b/btrfs

Re: [PATCH] Btrfs-progs: Filter out deleting or already-deleted subvolumes in lookup_ino_path.

2012-10-18 Thread Alex Lyakas
Hi Miao, On Thu, Oct 18, 2012 at 4:00 AM, Miao Xie mi...@cn.fujitsu.com wrote: On Wed, 17 Oct 2012 18:06:52 +0200, Alex Lyakas wrote: -static int __list_subvol_fill_paths(int fd, struct root_lookup *root_lookup) +static int __list_subvol_fill_paths(int fd, struct root_lookup *root_lookup

Questions about btrfs-convert code

2012-10-23 Thread Alex Lyakas
Greetings all, I am studying the conversion code, and maybe somebody can help me with these questions: # btrfs_make_block_groups() I see that this function uses some heuristics to create block groups for BTRFS_BLOCK_GROUP_METADATA and BTRFS_BLOCK_GROUP_DATA. Later, ext2 data blocks are mapped

Re: Questions about mkfs -A option

2012-10-24 Thread Alex Lyakas
wrote: On 2012年10月23日 01:49, Alex Lyakas wrote: Wang, I would say that zeroing is done to prevent from seeing some other superblock (besides btrfs) there. That make sense. Alex. But I still confused by the -A option? In mkfs, we always write the sb info

Re: inode_cache

2012-10-24 Thread Alex Lyakas
Swami, I believe the only thing that inode_cache mount option does is allowing to reuse inode numbers. Like if you had a file with inode number=X and you delete this file, then you will never have inode=X in the same file tree (subvolume) again. You can look at btrfs_find_free_ino() that allocates

Re: Help understanding btrfsck output...

2012-10-24 Thread Alex Lyakas
Swami, code is your friend again:) #define I_ERR_FILE_NBYTES_WRONG (1 10) // 400 ... if (rec-found_size != rec-nbytes) rec-errors |= I_ERR_FILE_NBYTES_WRONG; From what I recently learned nbytes for an inode is the total sum of extents this inode has (assuming this is a regular

Re: btrfs seems to do COW while inode has NODATACOW set

2012-10-25 Thread Alex Lyakas
inconsistencies there, although btrfsck passes fine. Any other reason that COW should happen? Any hint on how to debug deeper is appreciated:) Alex. On Thu, Oct 25, 2012 at 8:40 PM, cwillu cwi...@cwillu.com wrote: On Thu, Oct 25, 2012 at 12:35 PM, Alex Lyakas alex.bt...@zadarastorage.com wrote: Hi everybody

Re: btrfs seems to do COW while inode has NODATACOW set

2012-10-25 Thread Alex Lyakas
have better luck figuring out what's happening using the 'filefrag -vfile' command. Hope this helps/answers your question. Regards, Wade On 10/25/2012 11:35 AM, Alex Lyakas wrote: Hi everybody, I need some help understanding the nodatacow behavior. I have set up a large file (5GiB

Re: btrfs seems to do COW while inode has NODATACOW set

2012-10-28 Thread Alex Lyakas
Hi, it appears that I found why the COW is happening. The code in the kernel that triggers this is: check_committed_ref(): if (btrfs_extent_generation(leaf, ei) = btrfs_root_last_snapshot(root-root_item)) goto out; It appears that both extent_generation and

Re: btrfs seems to do COW while inode has NODATACOW set

2012-10-29 Thread Alex Lyakas
. Is this the intended thing to do --- to bump up bytes_may_use even though we won't need any new space for data eventually? Thanks, Alex. On Sun, Oct 28, 2012 at 2:12 PM, Alex Lyakas alex.bt...@zadarastorage.com wrote: Hi, it appears that I found why the COW is happening. The code in the kernel

Re: [PATCH] Add btrfs-show-super

2012-11-03 Thread Alex Lyakas
Hi Goffredo, do you think it could be useful to add an optional argument like bytenr, which tells where to read the superblock from? For example, btrfs-convert first places the superblock on some block unused by ext2, and only at the end it migrates the superblock to the standard location. Not

bytes_may_use is incremented with NOCOW [was: btrfs seems to do COW while inode has NODATACOW set]

2012-11-04 Thread Alex Lyakas
? Thanks, Alex. On Mon, Oct 29, 2012 at 7:18 PM, Alex Lyakas alex.bt...@zadarastorage.com wrote: FWIW, I have found when I am hitting ENOSPC. btrfs_check_data_free_space() has this code: ... /* make sure we have enough space to handle the data first */ spin_lock(data_sinfo

Re: [PATCH] Btrfs: send - pre-allocated file support for btrfs-send mechanism

2012-12-01 Thread Alex Lyakas
Hi, and also, I think, for PREALLOC extents, it is not enough just to skip sending the data, but possibly need to perform preallocation on the receive side, so a new SEND command might be needed. Alex. On Thu, Nov 29, 2012 at 10:55 AM, Alexander Block abloc...@gmail.com wrote: On Thu, Nov 29,

Re: Snapper snapshot comparison algorithm

2012-12-01 Thread Alex Lyakas
Hi nafisa, in order to understand how btrfs send code compares two btrfs file trees, you may read this: http://www.spinics.net/lists/linux-btrfs/msg17731.html (where I am trying to understand it) and further down the thread. It's a very nice algorithm, actually. Thanks, Alex. On Sat, Dec 1,

Re: send/receive questions

2012-12-09 Thread Alex Lyakas
Hi, 4. support this scene, When doing the receive process, if it suddently be stopped, how to continue the receive process. I sent a proposal and a patch to implement this in http://www.spinics.net/lists/linux-btrfs/msg19387.html. However, it is not easy to make this patch

Re: Snapper snapshot comparison algorithm - send/receive questions

2012-12-09 Thread Alex Lyakas
Arvin, On Mon, Dec 3, 2012 at 1:35 PM, Arvin Schnell aschn...@suse.de wrote: On Sat, Dec 01, 2012 at 01:24:20PM +0530, nafisa mandliwala wrote: I needed help with understanding the snapshot comparison algorithm that snapper uses and its shortcomings. From reading the code, what I understood

Re: Snapper snapshot comparison algorithm

2012-12-09 Thread Alex Lyakas
to the function where the instructions are generated. Thanks On Sat, Dec 1, 2012 at 2:00 PM, Alex Lyakas alex.bt...@zadarastorage.com wrote: Hi nafisa, in order to understand how btrfs send code compares two btrfs file trees, you may read this: http://www.spinics.net/lists/linux-btrfs/msg17731

Re: Snapper snapshot comparison algorithm

2012-12-17 Thread Alex Lyakas
, and not necessarily to the order of user-operatons between snapshots. See you next week:) Alex. Thanks! On Sun, Dec 9, 2012 at 2:44 PM, Alex Lyakas alex.bt...@zadarastorage.com wrote: Mohit, Nafisa, you should start reading from changed_cb function, which is the one that notifies

Newbie questions on some of btrfs code...

2012-05-18 Thread Alex Lyakas
Greetings everybody, I have been studying some of the btrfs code and the developer documentation on the wiki. My primary interest at this point, is to be able to search within fs tree of a btrfs subvolume, which was created as a snapshot of another subvolume. For that I have been using the

  1   2   3   >