[RFC v0 2/4] x86: add sys_copy_range to syscall tables

2013-05-14 Thread Zach Brown
Add sys_copy_range to the x86 syscall tables. Happily, it doesn't require compat helpers. Signed-off-by: Zach Brown z...@redhat.com --- arch/x86/syscalls/syscall_32.tbl | 1 + arch/x86/syscalls/syscall_64.tbl | 1 + 2 files changed, 2 insertions(+) diff --git a/arch/x86/syscalls/syscall_32.tbl

[RFC v0 0/4] sys_copy_range() rough draft

2013-05-14 Thread Zach Brown
We've been talking about implementing some form of bulk data copy offloading for a while now. BTRFS and OCFS2 implement forms of copy offloading with ioctls, NFS 4.2 will include a byte-granular COPY operation, and the SCSI XCOPY command is being implemented now that Windows can issue it. In the

[RFC v0 4/4] nfs, nfsd: rough sys_copy_range and COPY support

2013-05-14 Thread Zach Brown
This crude patch illustrates the simplest plumbing involved in supporting sys_call_range with the NFS COPY operation that's pending in the 4.2 draft spec. The patch is based on a previous prototype that used the COPY op to implement sys_copyfileat which created a new file (based on the ocfs2

[RFC v0 3/4] btrfs: add .copy_range file operation

2013-05-14 Thread Zach Brown
the CLONE_RANGE ioctl and copy_range syscall. Signed-off-by: Zach Brown z...@redhat.com --- fs/btrfs/ctree.h | 3 ++ fs/btrfs/file.c | 1 + fs/btrfs/ioctl.c | 122 +-- 3 files changed, 77 insertions(+), 49 deletions(-) diff --git a/fs/btrfs/ctree.h

[RFC v0 1/4] vfs: add copy_range syscall and vfs entry point

2013-05-14 Thread Zach Brown
mpage.o ioprio.o diff --git a/fs/copy_range.c b/fs/copy_range.c new file mode 100644 index 000..3000b9f --- /dev/null +++ b/fs/copy_range.c @@ -0,0 +1,127 @@ +/* + * copy_range: offload data copying between existing files + * + * Copyright (C) 2013 Zach Brown z...@redhat.com + */ +#include linux/fs.h

Re: [RFC v0 0/4] sys_copy_range() rough draft

2013-05-14 Thread Zach Brown
On Wed, May 15, 2013 at 07:42:51AM +1000, Dave Chinner wrote: On Tue, May 14, 2013 at 02:15:22PM -0700, Zach Brown wrote: I'm going to keep hacking away at this. My next step is to get ext4 supporting .copy_range, probably with a quick hack to copy the contents of bios. Hopefully that'll

Re: [RFC v0 1/4] vfs: add copy_range syscall and vfs entry point

2013-05-15 Thread Zach Brown
On Wed, May 15, 2013 at 07:44:05PM +, Eric Wong wrote: Why introduce a new syscall instead of extending sys_splice? Personally, I think it's ugly to have different operations use the same syscall just because their arguments match. But that preference aside, sure, if the consensus is that

Re: [RFC v0 1/4] vfs: add copy_range syscall and vfs entry point

2013-05-21 Thread Zach Brown
On Tue, May 21, 2013 at 07:47:19PM +, Eric Wong wrote: Zach Brown z...@redhat.com wrote: On Wed, May 15, 2013 at 07:44:05PM +, Eric Wong wrote: Why introduce a new syscall instead of extending sys_splice? Personally, I think it's ugly to have different operations use the same

testing stable pages being modified

2013-05-30 Thread Zach Brown
'stable' pages have always been a bit of a fiction. It's easy to intentionally modify stable pages under io with some help from page references that ignore mappings and page state. Here's little test that uses O_DIRECT to get the pinned aio ring pages under IO and then has event completion

Re: testing stable pages being modified

2013-05-31 Thread Zach Brown
Changing O_DIRECT in flight has always been a deep dark corner case, and crc errors are the expected result. Have you found anyone doing this in real life? Agreed; and no, I haven't heard of people accidentally modifying stable pages. I do like the small test program though, we should

[PATCH 0/6] fix INT_MAX readdir hang, plus cleanups

2013-06-04 Thread Zach Brown
Hi gang, I finally sat down to fix that readdir hang that has been in the back of my mind for a while. I *hope* that the fix is pretty simple: just don't manufacture a fake f_pos, I *think* we can abuse f_version as an indicator that we shouldn't return entries. Does this look reasonable? We

[PATCH 1/6] btrfs: set readdir f_pos only after filldir

2013-06-04 Thread Zach Brown
The only time we need to advance f_pos is after we've successfully given a result to userspace via filldir. This simplification gets rid of the is_curr variable used to update f_pos for the delayed item readdir entries. Signed-off-by: Zach Brown z...@redhat.com --- fs/btrfs/delayed-inode.c | 5

[PATCH 2/6] btrfs: fix readdir hang with offsets past INT_MAX

2013-06-04 Thread Zach Brown
of the stack, let's use f_version to indicate that readdir should stop returning entries until seek changes f_pos and clears f_version. Signed-off-by: Zach Brown z...@redhat.com --- fs/btrfs/inode.c | 24 1 file changed, 16 insertions(+), 8 deletions(-) diff --git a/fs

[PATCH 4/6] btrfs: simplify finding next/prev delayed items

2013-06-04 Thread Zach Brown
recent next and prev items as we descend and return those when the search fails. Signed-off-by: Zach Brown z...@redhat.com --- fs/btrfs/delayed-inode.c | 54 +++- 1 file changed, 21 insertions(+), 33 deletions(-) diff --git a/fs/btrfs/delayed-inode.c b/fs

[PATCH 5/6] btrfs: add helper to get delayed item root

2013-06-04 Thread Zach Brown
-by: Zach Brown z...@redhat.com --- fs/btrfs/delayed-inode.c | 28 +--- 1 file changed, 13 insertions(+), 15 deletions(-) diff --git a/fs/btrfs/delayed-inode.c b/fs/btrfs/delayed-inode.c index 67e0f9f..fcce951 100644 --- a/fs/btrfs/delayed-inode.c +++ b/fs/btrfs/delayed

[PATCH 3/6] btrfs: trivial delayed item readdir list cleanups

2013-06-04 Thread Zach Brown
needed and the rest of the uses just find empty lists. Signed-off-by: Zach Brown z...@redhat.com --- fs/btrfs/delayed-inode.c | 17 ++--- fs/btrfs/delayed-inode.h | 3 +-- fs/btrfs/inode.c | 17 ++--- 3 files changed, 9 insertions(+), 28 deletions(-) diff --git a/fs

[PATCH 6/6] btrfs: get fewer delayed item refs during readdir

2013-06-04 Thread Zach Brown
, let's use the delayed item's key-sorted rbtree to skip items that are before f_pos and will never be used. Second, let's only acquire the new delayed items after we've exausted the existing in-tree items and still have room in the readdir buffer for more entries. Signed-off-by: Zach Brown z

Re: [PATCH 0/6] fix INT_MAX readdir hang, plus cleanups

2013-06-04 Thread Zach Brown
On Tue, Jun 04, 2013 at 07:16:53PM -0400, Chris Mason wrote: Quoting Zach Brown (2013-06-04 18:17:54) Hi gang, I finally sat down to fix that readdir hang that has been in the back of my mind for a while. I *hope* that the fix is pretty simple: just don't manufacture a fake f_pos, I

Re: [PATCH 2/2 v2] btrfs: add framework to read fs info and dev info from the kernel

2013-06-10 Thread Zach Brown
On Mon, Jun 10, 2013 at 10:59:15PM +0800, Anand Jain wrote: This adds two ioctl BTRFS_IOC_GET_FSIDS and BTRFS_IOC_GET_DEVS which reads the btrfs_fs_devices and btrfs_device structure from the kernel respectively. Why not just use sysfs to export the device lists? The information in these

Re: [PATCH 0/6] fix INT_MAX readdir hang, plus cleanups

2013-06-10 Thread Zach Brown
On Tue, Jun 04, 2013 at 04:26:57PM -0700, Zach Brown wrote: On Tue, Jun 04, 2013 at 07:16:53PM -0400, Chris Mason wrote: Quoting Zach Brown (2013-06-04 18:17:54) Hi gang, I finally sat down to fix that readdir hang that has been in the back of my mind for a while. I *hope

Re: [PATCH 2/2 v2] btrfs: add framework to read fs info and dev info from the kernel

2013-06-11 Thread Zach Brown
1) Have a simple single fixed input structure for each ioctl. Maybe with some extra padding and a flags argument if you think stuff is going to be added over time. No generic header. No casting. The ioctl number defines the input structure. If you need a different structure later, use a

Re: [PATCH 0/3] Btrfs-progs: make some subdirs

2013-06-11 Thread Zach Brown
On Tue, Jun 11, 2013 at 06:15:16PM -0500, Eric Sandeen wrote: These are some patches I've been sitting on for a while that I think clean up the btrfs-progs tree a bit; there are still quite a few files left in the top-level dir, but moving tests cmd files into cmd/ seems to clean things up a

Re: [PATCH] Btrfs: use a percpu to keep track of possibly pinned bytes

2013-06-20 Thread Zach Brown
@@ -3380,6 +3382,10 @@ static int update_space_info(struct btrfs_fs_info *info, u64 flags, if (!found) return -ENOMEM; + ret = percpu_counter_init(found-total_bytes_pinned, 0); + if (ret) + return ret; + Leaks *found if percpu_counter_init()

Re: [PATCH v5 1/8] Btrfs: introduce a tree for items that map UUIDs to something

2013-06-20 Thread Zach Brown
+/* for items that use the BTRFS_UUID_KEY */ +#define BTRFS_UUID_ITEM_TYPE_SUBVOL 0 /* for UUIDs assigned to subvols */ +#define BTRFS_UUID_ITEM_TYPE_RECEIVED_SUBVOL 1 /* for UUIDs assigned to +* received subvols */ + +/* a sequence of such

Re: [PATCH v5 1/8] Btrfs: introduce a tree for items that map UUIDs to something

2013-06-21 Thread Zach Brown
Why do we need this btrfs_uuid_item structure? Why not set the key type to either _SUBVOL or _RECEIVED_SUBVOL instead of embedding structs with those types under items with the constant BTRFS_UUID_KEY. Then use the item size to determine the number of u64 subids. Then the item has a

Re: [PATCH v5 1/8] Btrfs: introduce a tree for items that map UUIDs to something

2013-06-24 Thread Zach Brown
This would save some CPU cycles for the repeated le64_to_cpu() and for the memcpy(). The number of lines of code is equal for both ways. Hmm? It would be many fewer lines of code. Are you thinking of something shorter than the following? That's better, yeah, but there's still some

Re: [PATCH v6 1/8] Btrfs: introduce a tree for items that map UUIDs to something

2013-06-26 Thread Zach Brown
+ if (!uuid_root) { + WARN_ON_ONCE(1); + ret = -ENOENT; + goto out; + } WARN_ON_ONCE specifically returns the condition so that you can write: if (WARN_ON_ONCE(!uuid_root)) { ret = -ENOENT; goto out;

Re: [PATCH] Btrfs: fix crash regarding to ulist_add_merge

2013-06-26 Thread Zach Brown
On Wed, Jun 26, 2013 at 12:02:51PM +0800, Liu Bo wrote: Several users reported this crash of NULL pointer or general protection, the story is that we add a rbtree for speedup ulist iteration, and we use krealloc() to address ulist growth, and krealloc() use memcpy to copy old data to new

Re: [PATCH] Btrfs: fix crash regarding to ulist_add_merge

2013-06-26 Thread Zach Brown
ptrdiff_t diff = new_nodes - old; ulist-root.rb_node += diff; for (i = 0; i ulist-nnodes; i++) { ulist-nodes[i].rb_node.rb_left += diff; ulist-nodes[i].rb_node.rb_left += diff; } (rb_right

Re: [PATCH v6 1/8] Btrfs: introduce a tree for items that map UUIDs to something

2013-06-26 Thread Zach Brown
Please tell me when you are done with the full review. And please also stop the bikeshedding. I won't commit to a full review, and I won't try and guess which comments you would choose to dismiss as bikeshedding. I'm free to share what occurs to me and you're free to tell me to go jump in a

Re: [PATCH] Btrfs: fix crash regarding to ulist_add_merge

2013-06-26 Thread Zach Brown
But in the krealloc() case the rb_erase() will be trying to reference freed memmory because krealloc() frees the old pointer on success. Yeah, I realize that you're absolutely right, but my box didn't complain about the abused old pointers when we're not in int_nodes case, which is

Re: [PATCH] Btrfs: make the chunk allocator completely tree lockless

2013-06-27 Thread Zach Brown
Reviewing as requested! It certainly looks reasonable, but I don't have enough history with the code to really say much more than that. Some questions: @@ -8423,6 +8432,10 @@ void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans,

Re: [PATCH v3] Btrfs: fix crash regarding to ulist_add_merge

2013-06-28 Thread Zach Brown
fix the probem. Thanks for being persistent. Reviewed-by: Zach Brown z...@redhat.com + for (i = 0; i ulist-nnodes; i++) + rb_erase(ulist-nodes[i].rb_node, ulist-root); (still twitching over here because this is a bunch of work that achieves nothing :)) - z

Re: [PATCH 0/6] fix INT_MAX readdir hang, plus cleanups

2013-07-01 Thread Zach Brown
code. It's all lightly tested with xfstests but it wouldn't surprise me if I missed something so review is appreciated. *mmm, hmmm* One of these patches is making new entries not show up in readdir. This was discovered while running stress.sh overnight, it complained about files not

Re: [RFC PATCH] Btrfs: rework ulist with list operation

2013-07-01 Thread Zach Brown
On Mon, Jul 01, 2013 at 10:14:44PM +0800, Liu Bo wrote: This is actually from Zach Brown's idea. Thanks for giving this a try. Instead of ulist of array+rbtree, here we introduce ulist of list+rbtree, memory is dynamic allocation and we can get rid of memory re-allocation dance and special

Re: [PATCH V2 1/2] Btrfs-progs: make pretty_sizes() work less error prone

2013-07-09 Thread Zach Brown
to free allocated memory. But I think that we can do better and not have callers need to worry about per-call string storage at all. How about something like this? - z From bea92d06d98827af30518aa800428c0b2a8be101 Mon Sep 17 00:00:00 2001 From: Zach Brown z...@redhat.com Date: Tue, 9 Jul 2013 12

Re: [PATCH V2 1/2] Btrfs-progs: make pretty_sizes() work less error prone

2013-07-10 Thread Zach Brown
Neat trick! A few neat-picks below. Indeed, those are all good fixes. As these are only trivial changes I'll fix them at commit time. Great, thanks David! - z -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More

Re: [PATCH 4/4] btrfs: offline dedupe

2013-07-15 Thread Zach Brown
Josef asked that I check out the offline dedup patches. I hope that I found the most recent posting in my archives :). The first three patches seemed fine. I have questions about the read/compare/clone core: + addr = kmap(page); + memcpy(buffer + bytes_copied, addr,

Re: [PATCH] Btrfs: Print key type in decimal everywhere

2013-07-16 Thread Zach Brown
On Tue, Jul 16, 2013 at 03:38:33PM +0200, Stefan Behrens wrote: This is confusing, sometimes the key type is printed in hex (without a leading 0x which makes things even more complicated), sometimes in decimal... Change it to be in decimal everywhere. This seems particularly reasonable to me

Re: [bug] label cli hangs until balance is completed

2013-07-17 Thread Zach Brown
On Wed, Jul 17, 2013 at 11:09:46AM +0200, Stefan Behrens wrote: On Wed, 17 Jul 2013 16:29:05 +0800, Anand Jain wrote: 'btrfs fi label /btrfs' will hang until balance is completed. (and probably even set label would hang). This is because we are trying to hold volume_mutex lock which

Re: [PATCH v2] btrfs-progs: fix segfault when listing column OTIME on big endian host

2013-07-25 Thread Zach Brown
btrfs-list.c: case BTRFS_LIST_OTIME: if (subv-otime) strftime(tstr, 256, %Y-%m-%d %X, localtime(subv-otime)); else strcpy(tstr, -); printf(%s, tstr);

Re: [PATCH] Btrfs: check to see if root_list is empty before adding it to dead roots

2013-07-25 Thread Zach Brown
-int btrfs_add_dead_root(struct btrfs_root *root) +void btrfs_add_dead_root(struct btrfs_root *root) { spin_lock(root-fs_info-trans_lock); - list_add_tail(root-root_list, root-fs_info-dead_roots); + if (list_empty(root-root_list)) + list_add_tail(root-root_list,

Re: [PATCH 4/4] btrfs: offline dedupe

2013-07-26 Thread Zach Brown
+static struct page *extent_same_get_page(struct inode *inode, u64 off) +{ + struct page *page; + pgoff_t index; + struct extent_io_tree *tree = BTRFS_I(inode)-io_tree; + + index = off PAGE_CACHE_SHIFT; + + page = grab_cache_page(inode-i_mapping, index); + if

Re: [PATCH] Btrfs: add missing error check to find_parent_nodes

2013-07-30 Thread Zach Brown
diff --git a/fs/btrfs/backref.c b/fs/btrfs/backref.c index 8bc5e8c..1ba87c5 100644 --- a/fs/btrfs/backref.c +++ b/fs/btrfs/backref.c @@ -935,6 +935,8 @@ again: } ret = find_extent_in_eb(eb, bytenr,

Re: Is the checkpoint interval adjustable?

2013-07-31 Thread Zach Brown
On Wed, Jul 31, 2013 at 04:02:29PM -0400, Mike Audia wrote: I believe 30 sec is the default for the checkpoint interval.  Is this adjustable? It doesn't look like it. It looks like it's implemented with raw '30's in the code. delay = HZ * 30; ... (now

Re: [RFC PATCH v5 3/5] Btrfs: introduce a head ref rbtree

2013-07-31 Thread Zach Brown
+ BUG_ON(!locked_ref); + BUG_ON(!btrfs_delayed_ref_is_head(head-node)); Please don't add more BUG_ON()s. They're unreliable because they can kill the system. If you *must* have logic asserts, use WARN_ON_ONCE and return errors. If the path is so dire

Re: [RFC PATCH v5 5/5] Btrfs: online data deduplication

2013-07-31 Thread Zach Brown
+#define BTRFS_DEDUP_HASH_SIZE 32 /* 256bit = 32 * 8bit */ +#define BTRFS_DEDUP_HASH_LEN 4 + +struct btrfs_dedup_hash_item { + /* FIXME: put a hash type field here */ + + __le64 hash[BTRFS_DEDUP_HASH_LEN]; +} __attribute__ ((__packed__)); The handling of hashes in this patch

Re: Is the checkpoint interval adjustable?

2013-07-31 Thread Zach Brown
Thank you kindly for the prompt reply.  My goal is to make them _less_ frequent. I assumed as much. I should have added some sympathy smileys :).  I am NO programmer by any stretch.  Let's say I want them to be once every 5 min (300 sec).  Is the attached patch sane to acheive this? I

Re: Is the checkpoint interval adjustable?

2013-08-01 Thread Zach Brown
There were a few requests to tune the interval. This finally made me to finish the patch and will send it in a second. Great, thanks. That's a good point and lowers my worries a bit, though it would be interesting to see in what way a beefy machine blows with 300 seconds set. Agreed.

Re: [RFC PATCH v5 5/5] Btrfs: online data deduplication

2013-08-01 Thread Zach Brown
So do you mean that our whole hash value will be (key.objectid + bytes) because key.objectid is a part of hash value? I think so, if I understood your question. The idea is to not store the bytes of the hash that make up the objectid more than once so the tree items are smaller. For example:

Re: [PATCH v2] btrfs: add mount option to set commit interval

2013-08-02 Thread Zach Brown
Another newbie question is which version of the kernel do I need to have in order to cleanly apply this patch?  I am finding that it fails to apply to the current stable kernel code (as of now it is v3.10.4) which makes me think your patch has to be applied to a newer one?  Are you patching

Re: [PATCH] Btrfs: race free update of super flags

2013-08-05 Thread Zach Brown
On Sat, Aug 03, 2013 at 07:43:17PM +0100, Filipe David Borba Manana wrote: Before updating the super block's flags, which is a non-atomic operation, grab the super_lock in the fs_info structure. At the moment only 2 different code paths can update these flags in parallel: Hmm. You say the

Re: [PATCH] Btrfs: race free update of super flags

2013-08-05 Thread Zach Brown
I guess got confused, must have thought both referred to root-fs_info-super_copy. No worries. I honestly wasn't sure if I was just missing something :) - z (you are in a maze of twisty little super blocks, all alike) -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in

Re: [PATCH] btrfs-progs: don't overrun answer array in cmds-chunk.c

2013-08-05 Thread Zach Brown
On Mon, Aug 05, 2013 at 09:52:57PM -0500, Eric Sandeen wrote: If an array is 5 chars in size: char answer[5]; and we write the 6th char (counting from 0)... answer[5] = '\0'; *high fives* - answer[5] = '\0'; + answer[4] = '\0'; I went to see which way of

Re: [PATCH 1/2] btrfs-progs: introduce command namespace for development features

2013-08-06 Thread Zach Brown
On Tue, Aug 06, 2013 at 02:25:20PM +0200, David Sterba wrote: We'd like to make it easier to preview a new feature and remove the burden to invent sane user interface (command name, placement, arguments, man) from the beginning. My biggest worry about this is that it complicates the

Re: [PATCH] btrfs-progs: don't overrun answer array in cmds-chunk.c

2013-08-06 Thread Zach Brown
2c46c2b81061f1c55de07a80d9d177a7df7b33cb Mon Sep 17 00:00:00 2001 From: Zach Brown z...@redhat.com Date: Tue, 6 Aug 2013 11:30:21 -0700 Subject: [PATCH] btrfs-progs: simplify ask_user() Eric noticed the trivial stack overflow bug in ask_user(). I went to see the context for that fix and found

Re: [PATCH 4/4] btrfs: offline dedupe

2013-08-06 Thread Zach Brown
for taking the time to fix it up. Reviewed-by: Zach Brown z...@redhat.com - z -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/2] btrfs-progs: introduce command namespace for development features

2013-08-07 Thread Zach Brown
On Wed, Aug 07, 2013 at 02:38:12PM +0200, David Sterba wrote: On Wed, Aug 07, 2013 at 01:26:58PM +0200, David Sterba wrote: I have to re-organize integration branch(es) better, so there is eg. a branch without unstable stuff, possibly always in a pullable state. On top of that a bunch of

Re: btrfs filesystem hang after page allocation failure: order:0 + kernel BUG at fs/btrfs/extent_io.c:4151

2013-08-07 Thread Zach Brown
On Thu, Aug 08, 2013 at 02:17:05AM +0800, Tomasz Chmielewski wrote: One of btrfs filesystems hanged on my server. Yeah, that's the consequence of using BUG_ON() for error handling. It forcefully tears down the task that executes it without returning back up the stack. Any locks that are held

Re: [PATCH] Btrfs: deal with enomem in the rewind path V2

2013-08-07 Thread Zach Brown
--- a/fs/btrfs/ctree.c +++ b/fs/btrfs/ctree.c @@ -1211,7 +1211,8 @@ tree_mod_log_rewind(struct btrfs_fs_info *fs_info, struct extent_buffer *eb, BUG_ON(tm-slot != 0); eb_rewin = alloc_dummy_extent_buffer(eb-start,

Re: [PATCH 6/7] btrfs: cleanup: removed unused 'btrfs_reada_detach'

2013-08-08 Thread Zach Brown
even though the function is currently unused, I'm hesitating to remove it as it's part of the reada-API and might be handy for anyone going to use the API in the future. I agree. As replied here, http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg24047.html please keep the

Re: [PATCH 2/2] btrfs-progs: introduce btrfs filesystem show --kernel

2013-08-08 Thread Zach Brown
On Thu, Aug 08, 2013 at 04:07:07PM +0800, Anand Jain wrote: As of now btrfs filesystem show reads directly from disks. So sometimes output can be stale, mainly when user want to verify their last operation like, labeling or device delete or add... etc. This patch adds --kernel option to the

Re: Why does btrfs benchmark so badly in this case?

2013-08-08 Thread Zach Brown
I also don't know if any common use fs has an optimization whereby just the modified sector(s) is overwritten, rather than all sectors making up the file system block being modified. Most of them do. The generic direct io path allows sector sized dio. The very first bit of

Re: btrfs zero divide

2013-08-09 Thread Zach Brown
On Fri, Aug 09, 2013 at 02:26:36PM +0200, Andreas Schwab wrote: Josef Bacik jba...@fusionio.com writes: So stripe_len shouldn't be 0, if it is you have bigger problems :). The bigger problem is that stripe_nr is u64, this is completely bogus. The first operand of do_div must be u32. This

[PATCH 2/2] btrfs: don't double brelse on device rm

2013-08-09 Thread Zach Brown
the supers don't fit in the device. In that case it releases the bh assuming that it's going to read a new one, finds that it won't read, and goes to a label that releases the bh again. All it needed to do was only brelse() right before overwriting the current bh with __bread(). Signed-off-by: Zach

[PATCH 1/2] btrfs: brelse in error path in open_ctree

2013-08-09 Thread Zach Brown
Commit 1104a8855 added an error branch that forgot to release the super bh. I found this while looking for an unrelated double brelse. Signed-off-by: Zach Brown z...@redhat.com --- fs/btrfs/disk-io.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c

[RFC] btrfs-progs: fix sparse checking and warnings

2013-08-14 Thread Zach Brown
Hi gang, I was a little surprised to see that patch go by recently which fixed an endian bug. I went to see how sparse checking looked and it was.. broken. I got it going again in my Fedora environment. Most of the patches are just cleanups, but there *were* three real bugs lurking in all

[PATCH 11/15] btrfs-progs: make many private symbols static

2013-08-14 Thread Zach Brown
Signed-off-by: Zach Brown z...@redhat.com --- btrfs-convert.c| 2 +- btrfs.c| 6 +++--- cmds-dedup.c | 2 +- crc32c.c | 4 ++-- ctree.c| 4 ++-- free-space-cache.c | 4 ++-- help.c | 2 +- radix-tree.c | 2 +- 8 files changed, 13

[PATCH 04/15] btrfs-progs: fix shadow symbols

2013-08-14 Thread Zach Brown
symbols instead of pulling their blocks out into functions. Signed-off-by: Zach Brown z...@redhat.com --- cmds-check.c | 7 +++ cmds-device.c | 10 +- cmds-restore.c | 8 ctree.c| 2 -- volumes.c | 2 +- 5 files changed, 13 insertions(+), 16 deletions(-) diff

[PATCH 15/15] btrfs-progs: use NULL instead of 0

2013-08-14 Thread Zach Brown
These were mostly in option structs but there were a few gross string pointer arguments given as 0. Signed-off-by: Zach Brown z...@redhat.com --- btrfs-map-logical.c | 2 +- btrfs.c | 2 +- cmds-balance.c | 6 +++--- cmds-check.c| 2 +- cmds-dedup.c| 4

[PATCH 03/15] btrfs-progs: add ULL to u64 constant

2013-08-14 Thread Zach Brown
This silences a sparse warning: warning: constant 0x4D5F53665248425F is so big it is long from commit 52162700bb59663add809a6465ce2769d80b3664 Author: Zach Brown z...@redhat.com Date: Thu Jan 17 11:54:47 2013 -0800 btrfs-progs: treat super.magic as an le64 High fives

[PATCH 12/15] btrfs-progs: fix unaligned compat endian warnings

2013-08-14 Thread Zach Brown
] [usertype] x ctree.h:1616:1:got restricted __le64 [usertype] noident Signed-off-by: Zach Brown z...@redhat.com --- kerncompat.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/kerncompat.h b/kerncompat.h index 4d78288..1fc2b34 100644 --- a/kerncompat.h +++ b

[PATCH 05/15] btrfs-progs: remove variable length stack arrays

2013-08-14 Thread Zach Brown
sparse hates variable length array definitions on the stack: btrfs-show-super.c:155:21: warning: Variable length array is used. And it's right to. They're a fragile construct that doesn't handle bad input well at all. Signed-off-by: Zach Brown z...@redhat.com --- btrfs-show-super.c | 2

[PATCH 07/15] btrfs-progs: fix endian bugs in chunk rebuilding

2013-08-14 Thread Zach Brown
A disk_key was set by hand instead of using the endian helpers. I *think* the second one is just a typo. The chunk's num_stripes was already initialized from the record, but it's le16. So we'll set the item's size based on the record's native num_stripes. Signed-off-by: Zach Brown z

[PATCH 02/15] btrfs-progs: remove __CHECKER__ from main code

2013-08-14 Thread Zach Brown
. Signed-off-by: Zach Brown z...@redhat.com --- btrfs-convert.c | 2 -- btrfs-list.c| 2 -- cmds-device.c | 9 - mkfs.c | 3 --- utils.c | 7 --- 5 files changed, 23 deletions(-) diff --git a/btrfs-convert.c b/btrfs-convert.c index a4608ec..9a7da57 100644

[PATCH 08/15] btrfs-progs: fix extent key endian bug in repair

2013-08-14 Thread Zach Brown
Extents rebuilt from backrefs can have their objectid mangled. The code tried to build a disk_key by hand and got the swabbing backwards. Signed-off-by: Zach Brown z...@redhat.com --- cmds-check.c | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/cmds-check.c b/cmds

[PATCH 10/15] btrfs-progs: fix qgroup realloc inheritance

2013-08-14 Thread Zach Brown
to do :). Signed-off-by: Zach Brown z...@redhat.com --- qgroup.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/qgroup.c b/qgroup.c index 038c4dc..86fe2b2 100644 --- a/qgroup.c +++ b/qgroup.c @@ -74,7 +74,7 @@ qgroup_inherit_realloc(struct btrfs_qgroup_inherit **inherit, int

[PATCH 06/15] btrfs-print: define void function args

2013-08-14 Thread Zach Brown
This silences (reasonable) sparse warnings of the form: warning: non-ANSI function declaration of .. Signed-off-by: Zach Brown z...@redhat.com --- btrfs-find-root.c | 2 +- btrfs-list.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/btrfs-find-root.c b/btrfs

[PATCH 01/15] btrfs-progs: get C=1 sparse checking working again

2013-08-14 Thread Zach Brown
with FORTIFY turned on that sparse gives up and doesn't show us any errors from our code. It's a questionable hack to always turn on FORTIFY ourselves, so we'll just not do that when building with sparse. And add a nice '[SP]' quiet output line for sparse checks. Signed-off-by: Zach Brown z

[PATCH 13/15] btrfs-progs: don't use linux/fs.h

2013-08-14 Thread Zach Brown
actually need to include the low-level linux/fs.h for anything. One assumes it was just carried over from kernel space. Signed-off-by: Zach Brown z...@redhat.com --- btrfs-convert.c | 1 - mkfs.c | 1 - 2 files changed, 2 deletions(-) diff --git a/btrfs-convert.c b/btrfs-convert.c index

[PATCH 14/15] btrfs-progs: give raid6.c its exported prototypes

2013-08-14 Thread Zach Brown
that it has a chance to check that its exported prototypes are correct. Signed-off-by: Zach Brown z...@redhat.com --- raid6.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/raid6.c b/raid6.c index ce0f655..a6ee483 100644 --- a/raid6.c +++ b/raid6.c @@ -20,6 +20,8 @@ #include stdint.h

[PATCH 09/15] btrfs-progs: fix in-place byte swapping

2013-08-14 Thread Zach Brown
Storing fixed-endian values in native cpu types defeats the purpose of using sparse endian types to find endian conversion bugs. Signed-off-by: Zach Brown z...@redhat.com --- print-tree.c | 6 +++--- uuid-tree.c | 5 +++-- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/print

Re: How btrfs resize should work ?

2013-08-16 Thread Zach Brown
So it appears that it only resized part of the file system which lies on /dev/sda to 3G. This behavior is confusing as hell, is this really desirable behaviour ? Yeah, 'btrfs filesystem resize' is misnamed. It only resizes a specified device which defaults to devid 1 if you don't specify one.

Re: [PATCH 1/3] btrfs-progs: move out print in cmd_df to another function

2013-08-19 Thread Zach Brown
+static void print_df(struct btrfs_ioctl_space_args *sargs) +{ + char description[80]; + char *total_bytes; + char *used_bytes; + u64 flags; + u64 i; + int written; + char g_str[64]; + int g_sz; + for (i = 0; i sargs-total_spaces; i++) { +

Re: [PATCH] Btrfs: add support for asserts

2013-08-26 Thread Zach Brown
With this we can go through and convert any BUG_ON()'s that we have to catch actual programming mistakes to the new ASSERT() and then fix everybody else to return errors. I like the sound of that! --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -3814,6 +3814,22 @@ void btrfs_printk(const

Re: [PATCH] Btrfs: add support for asserts

2013-08-26 Thread Zach Brown
#ifdef BTRFS_ASSERT #define btrfs_assert(cond) BUG_ON(!(cond)) #else #define btrfs_assert(cond) do { if (cond) ; } while (0) #endif I think the only downside is that the BUG_ON() won't print the conditional that failed, IIRC. Sure, if you wanted to go the

Re: Question: How can I recover this partition? (unable to find logical $hugenum len 4096)

2013-08-29 Thread Zach Brown
If those fail, then look in dmesg for errors relating to the log tree -- if that's corrupt and can't be read (or causes a crash), use btrfs-zero-log. In a bit of a tangent: btrfs-zero-log throws away data that fsync/sync could have previously claimed was stable on disk. Given how often

Re: [PATCH v3] Btrfs: optimize key searches in btrfs_search_slot

2013-08-29 Thread Zach Brown
+ if (level == 0) + offset = offsetof(struct btrfs_leaf, items); + else + offset = offsetof(struct btrfs_node, ptrs); (+10 fragility points for assuming that the key starts each struct instead of using [0].key) Ok. I just copied that from

Re: [PATCH 15/15] btrfs-progs: use NULL instead of 0

2013-08-30 Thread Zach Brown
On Fri, Aug 30, 2013 at 06:25:54PM +0200, David Sterba wrote: On Wed, Aug 14, 2013 at 04:16:45PM -0700, Zach Brown wrote: + { NULL, 0, NULL, 0} + NULL_CMD_STRUCT, It's always the last item, so I think the comma should go away, currently it's mixed. I'll fix it during commit

Re: [PATCH 01/15] btrfs-progs: get C=1 sparse checking working again

2013-08-30 Thread Zach Brown
Very nice, thanks. I'm getting this error, for each built object file: $ make V=1 C=1 [SP] ctree.o Heh, I goofed when building the echo and actual rules, might as well update that to $ to have it output .c? @$(check_echo) [SP] $@ $(Q)$(check) $(AM_CFLAGS)

Re: [PATCH] Btrfs: more efficient inode tree replace operation

2013-09-03 Thread Zach Brown
On Mon, Sep 02, 2013 at 12:19:13PM +0100, Filipe David Borba Manana wrote: Instead of removing the current inode from the red black tree and then add the new one, just use the red black tree replace operation, which is more efficient. Yup, this looks correct to me. Reviewed-by: Zach Brown z

Re: [PATCH 8/9] btrfs: Cleanup the old btrfs workqueue

2013-09-11 Thread Zach Brown
On Wed, Sep 11, 2013 at 04:52:37PM +0800, Qu Wenruo wrote: Since the patches before implemented the new kernel workqueue based btrfs_worqueue_struct, the old btrfs workqueue(btrfs_worker) can be removed without any problem. Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com ---

Re: csum questions

2013-09-27 Thread Zach Brown
2) How may I tell btrfs to ignore all csums and just assume they are all correct? The reason for wanting this is in case the csum is garbled and the file is intact, or the csum is correct and the file is only partially garbled, but may still contain useful data. You can't, right now.

Re: [PATCH] xfstests: btrfs/015 regression test for prealloc with balance

2013-09-27 Thread Zach Brown
+nr_csum_no_found=$(dmesg | grep -c no csum found) +nr_csum_failed=$(dmesg | grep -c csum failed) + +_check_csum_error() +{ + new_csum_no_found=$(dmesg | grep -c no csum found) + new_csum_failed=$(dmesg | grep -c csum failed) + + if [ $nr_csum_no_found -eq $new_csum_no_found

Re: [PATCH v2] btrfs-progs: device add should check existing FS before adding

2013-09-27 Thread Zach Brown
@@ -49,14 +50,17 @@ static int cmd_add_dev(int argc, char **argv) int i, fdmnt, ret=0, e; DIR *dirstream = NULL; int discard = 1; + int force = 0; + char estr[100]; + res = test_dev_for_mkfs(argv[i], force, estr); + if (res) { +

Re: [PATCH] btrfs-progs: calculate disk space that a subvol could free

2013-09-27 Thread Zach Brown
diff --git a/cmds-subvolume.c b/cmds-subvolume.c index de246ab..0f36cde 100644 --- a/cmds-subvolume.c +++ b/cmds-subvolume.c @@ -809,6 +809,7 @@ static int cmd_subvol_show(int argc, char **argv) int fd = -1, mntfd = -1; int ret = 1; DIR *dirstream1 = NULL, *dirstream2 =

Re: csum questions

2013-09-27 Thread Zach Brown
On Fri, Sep 27, 2013 at 12:47:57PM -0600, Chris Murphy wrote: On Sep 27, 2013, at 11:51 AM, Zach Brown z...@redhat.com wrote: 2) How may I tell btrfs to ignore all csums and just assume they are all correct? The reason for wanting this is in case the csum is garbled and the file

Re: [PATCH] Btrfs: don't delete ordered roots from list during cleanup

2013-09-27 Thread Zach Brown
On Fri, Sep 27, 2013 at 04:37:46PM -0400, Josef Bacik wrote: During transaction cleanup after an abort we are just removing roots from the ordered roots list which is incorrect. We have a BUG_ON() to make sure that the root is still part of the ordered roots list when we put our ordered

Re: [PATCH] Btrfs: don't delete ordered roots from list during cleanup

2013-09-30 Thread Zach Brown
So I _think_ we may need to truncate the ordered range in the inode as well, but I haven't had a consistent reproducer for this case. I want to leave it like this for now until I'm sure we don't need the truncate and then we could probably just replace this with a test for FS_ERROR in

Re: [PATCH] btrfs: init device stats for new devices

2013-09-30 Thread Zach Brown
On Tue, Oct 01, 2013 at 12:03:05AM +0200, Ondřej Kunc wrote: Hi Zach, thank you for your answer and clarification. I cannot just unmount and mount that filesystem, because it is running busy NFS server now, so I will just try it on some testbench server. Can mount -o remount be sufficient

  1   2   3   4   >