Re: [RFC] btrfs-progs: Add recursive defrag using -r option

[Duncan]

Frank Holton posted on Mon, 16 Sep 2013 22:21:24 -0400 as excerpted:

 I'm working on a patch to add the recursive option to the btrfs
 filesystem defrag command. I'd like some feedback on how the patch looks
 as written. I've added two helper functions, which might need to be
 renamed, one to call the ioctl and one to actually handle the recursion
 into the directory.
 
 Let me know what you think.

Not being a dev I won't comment on that angle, but this will sure help 
decomplicate general defragging from a sysadmin angle.  Thanks. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
Every nonfree program has a lord, a master --
and if you use the program, he is your master.  Richard Stallman

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] rwsem: add rwsem_is_contended

[Ingo Molnar]

* Andrew Morton a...@linux-foundation.org wrote:

 On Fri, 30 Aug 2013 10:14:01 -0400 Josef Bacik jba...@fusionio.com wrote:
 
  Btrfs uses an rwsem to control access to its extent tree.  Threads 
  will hold a read lock on this rwsem while they scan the extent tree, 
  and if need_resched() they will drop the lock and schedule.  The 
  transaction commit needs to take a write lock for this rwsem for a 
  very short period to switch out the commit roots.  If there are a lot 
  of threads doing this caching operation we can starve out the 
  committers which slows everybody out.  To address this we want to add 
  this functionality to see if our rwsem has anybody waiting to take a 
  write lock so we can drop it and schedule for a bit to allow the 
  commit to continue. Thanks,
 
 This sounds rather nasty and hacky.  Rather then working around a 
 locking shortcoming in a caller it would be better to fix/enhance the 
 core locking code.  What would such a change need to do?
 
 Presently rwsem waiters are fifo-queued, are they not?  So the commit 
 thread will eventually get that lock.  Apparently that's not working 
 adequately for you but I don't fully understand what it is about these 
 dynamics which is causing observable problems.

It would be nice to see the whole solution, together with the btrfs patch.

The problem I have is that this new primitive is only superficially like 
spin_is_contended(): in the spinlock case dropping the lock will guarantee 
some sort of progress, because another CPU will almost certainly pick up 
the lock if we cpu_relax().

In the rwsem case there's no such guarantee of progress, especially if a 
read-lock is dropped. So I'd like to see how it's implemented in practice.

Thanks,

Ingo
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Drop unused parameter from btrfs_item_nr

[cwillu]

On Mon, Sep 16, 2013 at 8:58 AM, Ross Kirk ross.k...@gmail.com wrote:
 Unused parameter cleanup

 Ross Kirk (1):
   btrfs: drop unused parameter from btrfs_item_nr

  fs/btrfs/backref.c|2 +-
  fs/btrfs/ctree.c  |   34 +-
  fs/btrfs/ctree.h  |   13 ++---
  fs/btrfs/dir-item.c   |2 +-
  fs/btrfs/inode-item.c |2 +-
  fs/btrfs/inode.c  |4 ++--
  fs/btrfs/print-tree.c |2 +-
  fs/btrfs/send.c   |4 ++--
  8 files changed, 31 insertions(+), 32 deletions(-)

 --
 1.7.7.6

 --
 To unsubscribe from this list: send the line unsubscribe linux-btrfs in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html

Something appears to be missing...
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: rootfs crash

[Jogi Hofmüller]

Dear all,

After trying all the suggested options from

  http://permalink.gmane.org/gmane.comp.file-systems.btrfs/27999

I still get no mountable file system :(

Trying to mount I get

mount -oro,recovery,compress=lzo /dev/sda3 /rootfs

[  906.835314] btrfs: enabling auto recovery
[  906.836977] btrfs: use lzo compression
[  906.838611] btrfs: disk space caching is enabled
[  906.843788] btrfs: mismatching generation and generation_v2 found in 
root item. This root was probably mounted with an older kernel. 
Resetting all new fields.

[  906.898440] Btrfs detected SSD devices, enabling SSD mode
[  906.900352] parent transid verify failed on 71744598016 wanted 103531 
found 103528
[  906.902288] parent transid verify failed on 71744598016 wanted 103531 
found 103528

[  906.986653] btrfs: open_ctree failed
mount: mounting /dev/sda3 on /rootfs failed: Invalid argument

I am limited to working with the tools the Debian initramfs provides.  
This means kernel 3.10.2 (Debian 3.10.7-1) and btrfs-tools  
0.19+20130705-1.  The latter seems to be up to date with git although 
`btrfs version` says v0.20 rc1. All this is happening on an Asus Zen 
book  UX32V with two 128GB SSDs.


If anyone is interested in images produced by btrfs-image, they are 
available at


  http://plagi.at/images/

I am fresh out of ideas at the moment, so if anyone has a suggestion I 
am willing to try.


Regards!
--
j.hofmüller


--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: rootfs crash

[Sander]

Jogi Hofmüller wrote (ao):
 I am limited to working with the tools the Debian initramfs
 provides.  This means kernel 3.10.2 (Debian 3.10.7-1) and
 btrfs-tools  0.19+20130705-1.  The latter seems to be up to date
 with git although `btrfs version` says v0.20 rc1. All this is
 happening on an Asus Zen book  UX32V with two 128GB SSDs.
 
 If anyone is interested in images produced by btrfs-image, they are
 available at
 
   http://plagi.at/images/
 
 I am fresh out of ideas at the moment, so if anyone has a suggestion
 I am willing to try.

Did you try btrfs chunk-recover ?

Sander
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: rootfs crash

[Jogi Hofmüller]

Hi,

Am 2013-09-17 13:18, schrieb Sander:

Jogi Hofmüller wrote (ao):



I am fresh out of ideas at the moment, so if anyone has a suggestion
I am willing to try.


Did you try btrfs chunk-recover ?


Yes, did so.  It stopped with an error.  I will send the error message 
later since it resides on a different machine that I cannot access from 
where I am now.


Regards!
--
j.hofmüller
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] btrfs-progs: Add recursive defrag using -r option

[David Sterba]

On Mon, Sep 16, 2013 at 10:21:24PM -0400, Frank Holton wrote:
 I'm working on a patch to add the recursive option to the 
 btrfs filesystem defrag command.

Great!

 I'd like some feedback on how the patch looks as written. I've added
 two helper functions, which might need to be renamed, one to call the
 ioctl and one to actually handle the recursion into the directory. 

Please use the 'ftw' function from glibc
http://man7.org/linux/man-pages/man3/ftw.3.html

(it's used in mkfs.c for example)

 @@ -333,6 +336,7 @@ static const char * const cmd_defrag_usage[] = {
   Defragment a file or a directory,
   ,
   -v be verbose,
 + -r defragment directories and files recursively,

Directories do not get defragmented. Giving any directory to defragment
will actually deframgent the subvolume fs_tree and then the shared
extent tree.  This is hidden in implementation and has confused people.

For now, I suggest to recursively defrag only files, that's IMHO the
most frequent usecase.

Deframgenting the subvol root or the extent tree are special cases and
should be enabled only when requested, by a commandline option. This
needs updating kernel. I can go into details later if needed.

(I'ts possible to implement a per-directory defrag by extending
btrfs_defrag_leaves with start/end key ranges that represent the
directory items.)

A few comments on the code follow, I hope you convert it to the 'ftw'
callback, this will remove half of the code from this patch, so I'd
rather see the result first. Basically, the contents of the ftw callback
is done inside this loop:

 + while ((dent = readdir(dir)) != NULL) {
 + int fd = 0;
 + if (!strcmp(dent-d_name,.) || !strcmp(dent-d_name, ..))
 + continue;
 + 
 + fn[len] = 0;
 + strncat(fn+len, dent-d_name, FILENAME_MAX - len);
 + 
 + if (lstat(fn, st) == -1) {
 + fprintf(stderr,ERROR: cannot stat %s\n, fn);
 + error++;
 + continue;
 + }
 + 
 + /* ignore symlinks ??*/
 + if (S_ISLNK(st.st_mode))
 + continue;
 + 
 + if(verbose)
 + printf(%s\n, fn);
 + 
 + /* directory entry */
 + if (S_ISDIR(st.st_mode)) {
 + ret = walk_dir(fn, verbose, fancy_ioctl, range);
 + errors += ret;
 + }
 + else {
 + fd = open(fn,O_RDWR);
 + e = errno;
 + if (fd  0) {
 + fprintf(stderr, ERROR: defrag failed on %s - 
 %s\n,
 + fn, strerror(e));
 + error++;
 + continue;
 + }
 + 
 + ret = do_defrag(fd, fancy_ioctl, range);
 + e = errno;
 + 
 + if (ret) {
 + errors++;
 + fprintf(stderr, ERROR: defrag failed on %s - 
 %s\n,
 + fn, strerror(e));
 + error++;
 + }
 + close(fd);
 + } 
 + }
 +
 + close(dir_fd);
 + closedir(dir);
 +
 + return(errors);
 +}

The command line argument handling is ok.

Please get familiar with the coding style
https://www.kernel.org/doc/Documentation/CodingStyle
eg. spacing and { } placement. It's not pointless to keep the style
consistent, people who have to read the code appreciate if it looks the
same all over the place.

thanks,
david
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Btrfs: allocate the free space by the existed max extent size when ENOSPC

[David Sterba]

On Mon, Sep 09, 2013 at 02:21:58PM +0800, Miao Xie wrote:
 On fri, 06 Sep 2013 15:47:08 +0200, Stefan Behrens wrote:
  This patch makes the xfstest generic/256 lock up. It's quite
  reliably reproducible on one of my test boxes, and not at all
  visible on a second test box.
  
  And yes, I'm using the V2 patch although you haven't tagged it as V2
  in the subject line of the mail :)
 
 According to my debug, the machine was not locked up, it seems the
 patch makes the test run very slow(90s -850s on my machine).

The massive slowdown is present in current 3.12-rc code, which means V3
did not make it into the pull.

Miao please send an incremental fix for rc2.

thanks,
david
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/2] btrfs-progs: use better name for nodiscard variable and flip the logic

[David Sterba]

Signed-off-by: David Sterba dste...@suse.cz
---
 mkfs.c  | 8 
 utils.c | 4 ++--
 utils.h | 2 +-
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/mkfs.c b/mkfs.c
index 3ac00ba..40da546 100644
--- a/mkfs.c
+++ b/mkfs.c
@@ -1324,7 +1324,7 @@ int main(int ac, char **av)
int mixed = 0;
int data_profile_opt = 0;
int metadata_profile_opt = 0;
-   int nodiscard = 0;
+   int discard = 1;
int ssd = 0;
int force_overwrite = 0;
 
@@ -1409,7 +1409,7 @@ int main(int ac, char **av)
source_dir_set = 1;
break;
case 'K':
-   nodiscard=1;
+   discard = 0;
break;
default:
print_usage();
@@ -1507,7 +1507,7 @@ int main(int ac, char **av)
}
first_file = file;
ret = btrfs_prepare_device(fd, file, zero_end, dev_block_count,
-  block_count, mixed, nodiscard);
+  block_count, mixed, discard);
if (block_count  block_count  dev_block_count) {
fprintf(stderr, %s is smaller than requested size\n, 
file);
exit(1);
@@ -1614,7 +1614,7 @@ int main(int ac, char **av)
continue;
}
ret = btrfs_prepare_device(fd, file, zero_end, dev_block_count,
-  block_count, mixed, nodiscard);
+  block_count, mixed, discard);
mixed = old_mixed;
BUG_ON(ret);
 
diff --git a/utils.c b/utils.c
index 134bf80..bb1bcdb 100644
--- a/utils.c
+++ b/utils.c
@@ -569,7 +569,7 @@ int btrfs_add_to_fsid(struct btrfs_trans_handle *trans,
 }
 
 int btrfs_prepare_device(int fd, char *file, int zero_end, u64 
*block_count_ret,
-  u64 max_block_count, int *mixed, int nodiscard)
+  u64 max_block_count, int *mixed, int discard)
 {
u64 block_count;
u64 bytenr;
@@ -596,7 +596,7 @@ int btrfs_prepare_device(int fd, char *file, int zero_end, 
u64 *block_count_ret,
*mixed = 1;
}
 
-   if (!nodiscard) {
+   if (discard) {
/*
 * We intentionally ignore errors from the discard ioctl.  It is
 * not necessary for the mkfs functionality but just an 
optimization.
diff --git a/utils.h b/utils.h
index 6c2553a..6bf7880 100644
--- a/utils.h
+++ b/utils.h
@@ -34,7 +34,7 @@ int make_btrfs(int fd, const char *device, const char *label,
 int btrfs_make_root_dir(struct btrfs_trans_handle *trans,
struct btrfs_root *root, u64 objectid);
 int btrfs_prepare_device(int fd, char *file, int zero_end, u64 
*block_count_ret,
-u64 max_block_count, int *mixed, int nodiscard);
+u64 max_block_count, int *mixed, int discard);
 int btrfs_add_to_fsid(struct btrfs_trans_handle *trans,
  struct btrfs_root *root, int fd, char *path,
  u64 block_count, u32 io_width, u32 io_align,
-- 
1.8.3.1

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] Btrfs: fix worst case calculator for space usage

[Josef Bacik]

Forever ago I made the worst case calculator say that we could potentially split
into 3 blocks for every level on the way down, which isn't right.  If we split
we're only going to get two new blocks, the one we originally cow'ed and the new
one we're going to split.  Thanks,

Signed-off-by: Josef Bacik jba...@fusionio.com
---
 fs/btrfs/ctree.h |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index 2491ba0..36490b9 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -3135,7 +3135,7 @@ static inline u64 btrfs_calc_trans_metadata_size(struct 
btrfs_root *root,
 unsigned num_items)
 {
return (root-leafsize + root-nodesize * (BTRFS_MAX_LEVEL - 1)) *
-   3 * num_items;
+   2 * num_items;
 }
 
 /*
-- 
1.7.7.6

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] Revert Btrfs: rework the overcommit logic to be based on the total size

[Josef Bacik]

This reverts commit 70afa3998c9baed4186df38988246de1abdab56d.  It is causing
performance issues and wasn't actually correct.  There were problems with the
way we flushed delalloc and that was the real cause of the early enospc.
Thanks,

Signed-off-by: Josef Bacik jba...@fusionio.com
---
 fs/btrfs/extent-tree.c |   15 +++
 1 files changed, 3 insertions(+), 12 deletions(-)

diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 2f03181..07f7f69 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -3925,7 +3925,6 @@ static int can_overcommit(struct btrfs_root *root,
u64 space_size;
u64 avail;
u64 used;
-   u64 to_add;
 
used = space_info-bytes_used + space_info-bytes_reserved +
space_info-bytes_pinned + space_info-bytes_readonly;
@@ -3959,25 +3958,17 @@ static int can_overcommit(struct btrfs_root *root,
   BTRFS_BLOCK_GROUP_RAID10))
avail = 1;
 
-   to_add = space_info-total_bytes;
-
/*
 * If we aren't flushing all things, let us overcommit up to
 * 1/2th of the space. If we can flush, don't let us overcommit
 * too much, let it overcommit up to 1/8 of the space.
 */
if (flush == BTRFS_RESERVE_FLUSH_ALL)
-   to_add = 3;
+   avail = 3;
else
-   to_add = 1;
-
-   /*
-* Limit the overcommit to the amount of free space we could possibly
-* allocate for chunks.
-*/
-   to_add = min(avail, to_add);
+   avail = 1;
 
-   if (used + bytes  space_info-total_bytes + to_add)
+   if (used + bytes  space_info-total_bytes + avail)
return 1;
return 0;
 }
-- 
1.7.7.6

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] Btrfs: kill delay_iput arg to the wait_ordered functions

[Josef Bacik]

This is a left over of how we used to wait for ordered extents, which was to
grab the inode and then run filemap flush on it.  However if we have an ordered
extent then we already are holding a ref on the inode, and we just use
btrfs_start_ordered_extent anyway, so there is no reason to have an extra ref on
the inode to start work on the ordered extent.  Thanks,

Signed-off-by: Josef Bacik jba...@fusionio.com
---
 fs/btrfs/dev-replace.c  |4 ++--
 fs/btrfs/extent-tree.c  |6 +++---
 fs/btrfs/ioctl.c|2 +-
 fs/btrfs/ordered-data.c |   25 +++--
 fs/btrfs/ordered-data.h |5 ++---
 fs/btrfs/relocation.c   |2 +-
 fs/btrfs/super.c|2 +-
 fs/btrfs/transaction.c  |2 +-
 8 files changed, 14 insertions(+), 34 deletions(-)

diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c
index af800ef..5d84443 100644
--- a/fs/btrfs/dev-replace.c
+++ b/fs/btrfs/dev-replace.c
@@ -400,7 +400,7 @@ int btrfs_dev_replace_start(struct btrfs_root *root,
args-result = BTRFS_IOCTL_DEV_REPLACE_RESULT_NO_ERROR;
btrfs_dev_replace_unlock(dev_replace);
 
-   btrfs_wait_all_ordered_extents(root-fs_info, 0);
+   btrfs_wait_all_ordered_extents(root-fs_info);
 
/* force writing the updated state information to disk */
trans = btrfs_start_transaction(root, 0);
@@ -475,7 +475,7 @@ static int btrfs_dev_replace_finishing(struct btrfs_fs_info 
*fs_info,
mutex_unlock(dev_replace-lock_finishing_cancel_unmount);
return ret;
}
-   btrfs_wait_all_ordered_extents(root-fs_info, 0);
+   btrfs_wait_all_ordered_extents(root-fs_info);
 
trans = btrfs_start_transaction(root, 0);
if (IS_ERR(trans)) {
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 07f7f69..88c6db6 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -3991,7 +3991,7 @@ static void btrfs_writeback_inodes_sb_nr(struct 
btrfs_root *root,
 */
btrfs_start_all_delalloc_inodes(root-fs_info, 0);
if (!current-journal_info)
-   btrfs_wait_all_ordered_extents(root-fs_info, 0);
+   btrfs_wait_all_ordered_extents(root-fs_info);
}
 }
 
@@ -4021,7 +4021,7 @@ static void shrink_delalloc(struct btrfs_root *root, u64 
to_reclaim, u64 orig,
if (delalloc_bytes == 0) {
if (trans)
return;
-   btrfs_wait_all_ordered_extents(root-fs_info, 0);
+   btrfs_wait_all_ordered_extents(root-fs_info);
return;
}
 
@@ -4049,7 +4049,7 @@ static void shrink_delalloc(struct btrfs_root *root, u64 
to_reclaim, u64 orig,
 
loops++;
if (wait_ordered  !trans) {
-   btrfs_wait_all_ordered_extents(root-fs_info, 0);
+   btrfs_wait_all_ordered_extents(root-fs_info);
} else {
time_left = schedule_timeout_killable(1);
if (time_left)
diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
index 1a5b946..fd6557e 100644
--- a/fs/btrfs/ioctl.c
+++ b/fs/btrfs/ioctl.c
@@ -574,7 +574,7 @@ static int create_snapshot(struct btrfs_root *root, struct 
inode *dir,
if (ret)
return ret;
 
-   btrfs_wait_ordered_extents(root, 0);
+   btrfs_wait_ordered_extents(root);
 
pending_snapshot = kzalloc(sizeof(*pending_snapshot), GFP_NOFS);
if (!pending_snapshot)
diff --git a/fs/btrfs/ordered-data.c b/fs/btrfs/ordered-data.c
index 966b413..8b597c9 100644
--- a/fs/btrfs/ordered-data.c
+++ b/fs/btrfs/ordered-data.c
@@ -563,11 +563,10 @@ static void btrfs_run_ordered_extent_work(struct 
btrfs_work *work)
  * wait for all the ordered extents in a root.  This is done when balancing
  * space between drives.
  */
-void btrfs_wait_ordered_extents(struct btrfs_root *root, int delay_iput)
+void btrfs_wait_ordered_extents(struct btrfs_root *root)
 {
struct list_head splice, works;
struct btrfs_ordered_extent *ordered, *next;
-   struct inode *inode;
 
INIT_LIST_HEAD(splice);
INIT_LIST_HEAD(works);
@@ -580,15 +579,6 @@ void btrfs_wait_ordered_extents(struct btrfs_root *root, 
int delay_iput)
   root_extent_list);
list_move_tail(ordered-root_extent_list,
   root-ordered_extents);
-   /*
-* the inode may be getting freed (in sys_unlink path).
-*/
-   inode = igrab(ordered-inode);
-   if (!inode) {
-   cond_resched_lock(root-ordered_extent_lock);
-   continue;
-   }
-
atomic_inc(ordered-refs);
spin_unlock(root-ordered_extent_lock);
 
@@ -605,21 +595,12 @@ void btrfs_wait_ordered_extents(struct btrfs_root *root, 
int delay_iput)

[PATCH] Btrfs: remove space_info-reservation_progress

[Josef Bacik]

This isn't used for anything anymore, just remove it.

Signed-off-by: Josef Bacik jba...@fusionio.com
---
 fs/btrfs/ctree.h   |9 -
 fs/btrfs/extent-tree.c |3 ---
 2 files changed, 0 insertions(+), 12 deletions(-)

diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index 36490b9..0506f40 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -1118,15 +1118,6 @@ struct btrfs_space_info {
 */
struct percpu_counter total_bytes_pinned;
 
-   /*
-* we bump reservation progress every time we decrement
-* bytes_reserved.  This way people waiting for reservations
-* know something good has happened and they can check
-* for progress.  The number here isn't to be trusted, it
-* just shows reclaim activity
-*/
-   unsigned long reservation_progress;
-
unsigned int full:1;/* indicates that we cannot allocate any more
   chunks for this space */
unsigned int chunk_alloc:1; /* set if we are allocating a chunk */
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 88c6db6..d58bef1 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -4456,7 +4456,6 @@ static void block_rsv_release_bytes(struct btrfs_fs_info 
*fs_info,
space_info-bytes_may_use -= num_bytes;
trace_btrfs_space_reservation(fs_info, space_info,
space_info-flags, num_bytes, 0);
-   space_info-reservation_progress++;
spin_unlock(space_info-lock);
}
}
@@ -4657,7 +4656,6 @@ static void update_global_block_rsv(struct btrfs_fs_info 
*fs_info)
sinfo-bytes_may_use -= num_bytes;
trace_btrfs_space_reservation(fs_info, space_info,
  sinfo-flags, num_bytes, 0);
-   sinfo-reservation_progress++;
block_rsv-reserved = block_rsv-size;
block_rsv-full = 1;
}
@@ -5437,7 +5435,6 @@ static int btrfs_update_reserved_bytes(struct 
btrfs_block_group_cache *cache,
space_info-bytes_readonly += num_bytes;
cache-reserved -= num_bytes;
space_info-bytes_reserved -= num_bytes;
-   space_info-reservation_progress++;
}
spin_unlock(cache-lock);
spin_unlock(space_info-lock);
-- 
1.7.7.6

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 0/2] nodiscard for device add

[David Sterba]

David Sterba (2):
  btrfs-progs: use better name for nodiscard variable and flip the logic
  btrfs-progs: add nodiscard option to device add

 cmds-device.c  | 34 +-
 man/btrfs.8.in | 11 +--
 mkfs.c |  8 
 utils.c|  6 --
 utils.h|  2 +-
 5 files changed, 47 insertions(+), 14 deletions(-)

-- 
1.8.3.1

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 0/3] btrfs-progs: Add command group for rescue operations

[David Sterba]

David Sterba (3):
  btrfs-progs: introduce rescue command group
  btrfs-progs: move chunk-recover to rescue group
  btrfs-progs: separate command ant implementation of chunk-recover code

 Makefile|3 +-
 btrfs.c |2 +-
 chunk-recover.c | 1745 +++
 cmds-chunk.c|2 +-
 cmds-rescue.c   |  100 
 commands.h  |3 +
 man/btrfs.8.in  |4 +-
 7 files changed, 1854 insertions(+), 5 deletions(-)
 create mode 100644 chunk-recover.c
 create mode 100644 cmds-rescue.c

-- 
1.8.3.1

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/3] btrfs-progs: introduce rescue command group

[David Sterba]

Add an empty 1st level command namespace that will collect specialized
recovery tools like chunk-recover, zero-log, select-super and similar.

Signed-off-by: David Sterba dste...@suse.cz
---
 Makefile   |  2 +-
 btrfs.c|  1 +
 cmds-rescue.c  | 37 +
 commands.h |  3 +++
 man/btrfs.8.in |  6 ++
 5 files changed, 48 insertions(+), 1 deletion(-)
 create mode 100644 cmds-rescue.c

diff --git a/Makefile b/Makefile
index 3d715d8..9e58028 100644
--- a/Makefile
+++ b/Makefile
@@ -13,7 +13,7 @@ objects = ctree.o disk-io.o radix-tree.o extent-tree.o 
print-tree.o \
 cmds_objects = cmds-subvolume.o cmds-filesystem.o cmds-device.o cmds-scrub.o \
   cmds-inspect.o cmds-balance.o cmds-send.o cmds-receive.o \
   cmds-quota.o cmds-qgroup.o cmds-replace.o cmds-check.o \
-  cmds-restore.o cmds-chunk.o cmds-dedup.o
+  cmds-restore.o cmds-chunk.o cmds-dedup.o cmds-rescue.o
 libbtrfs_objects = send-stream.o send-utils.o rbtree.o btrfs-list.o crc32c.o \
   uuid-tree.o
 libbtrfs_headers = send-stream.h send-utils.h send.h rbtree.h btrfs-list.h \
diff --git a/btrfs.c b/btrfs.c
index c85af14..14a27cb 100644
--- a/btrfs.c
+++ b/btrfs.c
@@ -247,6 +247,7 @@ static const struct cmd_group btrfs_cmd_group = {
{ device, cmd_device, NULL, device_cmd_group, 0 },
{ scrub, cmd_scrub, NULL, scrub_cmd_group, 0 },
{ check, cmd_check, cmd_check_usage, NULL, 0 },
+   { rescue, cmd_rescue, NULL, rescue_cmd_group, 0 },
{ chunk-recover, cmd_chunk_recover, cmd_chunk_recover_usage, 
NULL, 0},
{ restore, cmd_restore, cmd_restore_usage, NULL, 0 },
{ inspect-internal, cmd_inspect, NULL, inspect_cmd_group, 0 
},
diff --git a/cmds-rescue.c b/cmds-rescue.c
new file mode 100644
index 000..35a61f8
--- /dev/null
+++ b/cmds-rescue.c
@@ -0,0 +1,37 @@
+/*
+ * Copyright (C) 2013 SUSE.  All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include kerncompat.h
+
+#include commands.h
+
+static const char * const rescue_cmd_group_usage[] = {
+   btrfs rescue command [options] path,
+   NULL
+};
+
+const struct cmd_group rescue_cmd_group = {
+   rescue_cmd_group_usage, NULL, {
+   { 0, 0, 0, 0, 0 }
+   }
+};
+
+int cmd_rescue(int argc, char **argv)
+{
+   return handle_command_group(rescue_cmd_group, argc, argv);
+}
diff --git a/commands.h b/commands.h
index fa7a8ce..d25b8c1 100644
--- a/commands.h
+++ b/commands.h
@@ -91,12 +91,14 @@ extern const struct cmd_group quota_cmd_group;
 extern const struct cmd_group qgroup_cmd_group;
 extern const struct cmd_group replace_cmd_group;
 extern const struct cmd_group dedup_cmd_group;
+extern const struct cmd_group rescue_cmd_group;
 
 extern const char * const cmd_send_usage[];
 extern const char * const cmd_receive_usage[];
 extern const char * const cmd_check_usage[];
 extern const char * const cmd_chunk_recover_usage[];
 extern const char * const cmd_restore_usage[];
+extern const char * const cmd_rescue_usage[];
 
 int cmd_subvolume(int argc, char **argv);
 int cmd_filesystem(int argc, char **argv);
@@ -116,6 +118,7 @@ int cmd_select_super(int argc, char **argv);
 int cmd_dump_super(int argc, char **argv);
 int cmd_debug_tree(int argc, char **argv);
 int cmd_dedup(int argc, char **argv);
+int cmd_rescue(int argc, char **argv);
 
 /* subvolume exported functions */
 int test_issubvolume(char *path);
diff --git a/man/btrfs.8.in b/man/btrfs.8.in
index 61ffe8d..24777e6 100644
--- a/man/btrfs.8.in
+++ b/man/btrfs.8.in
@@ -69,6 +69,8 @@ btrfs \- control a btrfs filesystem
 .PP
 \fBbtrfs\fP \fBcheck\fP [\fIoptions\fP] \fIdevice\fP
 .PP
+\fBbtrfs\fP \fBrescue\fP [\fIoptions\fP] \fIpath\fP
+.PP
 \fBbtrfs\fP \fBchunk-recover\fP [\fIoptions\fP] \fIdevice\fP
 .PP
 \fBbtrfs\fP \fBrestore\fP [\fIoptions\fP] \fIdevice\fP
@@ -500,6 +502,10 @@ create a new extent tree.
 .RE
 .TP
 
+\fBrescue\fR [\fIoptions\fP] path\fR
+Rescue toolbox.
+.TP
+
 \fBchunk-recover\fR [\fIoptions\fP] device\fR
 Recover the chunk tree by scanning the devices one by one.
 .RS
-- 
1.8.3.1

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/3] btrfs-progs: move chunk-recover to rescue group

[David Sterba]

Signed-off-by: David Sterba dste...@suse.cz
---
 btrfs.c|  1 -
 cmds-chunk.c   |  2 +-
 cmds-rescue.c  |  1 +
 man/btrfs.8.in | 10 ++
 4 files changed, 4 insertions(+), 10 deletions(-)

diff --git a/btrfs.c b/btrfs.c
index 14a27cb..dfae35f 100644
--- a/btrfs.c
+++ b/btrfs.c
@@ -248,7 +248,6 @@ static const struct cmd_group btrfs_cmd_group = {
{ scrub, cmd_scrub, NULL, scrub_cmd_group, 0 },
{ check, cmd_check, cmd_check_usage, NULL, 0 },
{ rescue, cmd_rescue, NULL, rescue_cmd_group, 0 },
-   { chunk-recover, cmd_chunk_recover, cmd_chunk_recover_usage, 
NULL, 0},
{ restore, cmd_restore, cmd_restore_usage, NULL, 0 },
{ inspect-internal, cmd_inspect, NULL, inspect_cmd_group, 0 
},
{ send, cmd_send, cmd_send_usage, NULL, 0 },
diff --git a/cmds-chunk.c b/cmds-chunk.c
index 115db61..985f676 100644
--- a/cmds-chunk.c
+++ b/cmds-chunk.c
@@ -1745,7 +1745,7 @@ fail_rc:
 }
 
 const char * const cmd_chunk_recover_usage[] = {
-   btrfs chunk-recover [options] device,
+   btrfs rescue chunk-recover [options] device,
Recover the chunk tree by scanning the devices one by one.,
,
-y Assume an answer of `yes' to all questions,
diff --git a/cmds-rescue.c b/cmds-rescue.c
index 35a61f8..35b3bb1 100644
--- a/cmds-rescue.c
+++ b/cmds-rescue.c
@@ -27,6 +27,7 @@ static const char * const rescue_cmd_group_usage[] = {
 
 const struct cmd_group rescue_cmd_group = {
rescue_cmd_group_usage, NULL, {
+   { chunk-recover, cmd_chunk_recover, cmd_chunk_recover_usage, 
NULL, 0},
{ 0, 0, 0, 0, 0 }
}
 };
diff --git a/man/btrfs.8.in b/man/btrfs.8.in
index 24777e6..2888941 100644
--- a/man/btrfs.8.in
+++ b/man/btrfs.8.in
@@ -69,9 +69,7 @@ btrfs \- control a btrfs filesystem
 .PP
 \fBbtrfs\fP \fBcheck\fP [\fIoptions\fP] \fIdevice\fP
 .PP
-\fBbtrfs\fP \fBrescue\fP [\fIoptions\fP] \fIpath\fP
-.PP
-\fBbtrfs\fP \fBchunk-recover\fP [\fIoptions\fP] \fIdevice\fP
+\fBbtrfs\fP \fBrescue chunk-recover\fP [\fIoptions\fP] \fIpath\fP
 .PP
 \fBbtrfs\fP \fBrestore\fP [\fIoptions\fP] \fIdevice\fP
 .PP
@@ -502,11 +500,7 @@ create a new extent tree.
 .RE
 .TP
 
-\fBrescue\fR [\fIoptions\fP] path\fR
-Rescue toolbox.
-.TP
-
-\fBchunk-recover\fR [\fIoptions\fP] device\fR
+\fBrescue chunk-recover\fR [\fIoptions\fP] device\fR
 Recover the chunk tree by scanning the devices one by one.
 .RS
 
-- 
1.8.3.1

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs-progs: add list_sort and use it to sort devices by id

[David Sterba]

The devices in 'btrfs filesystem show' are now sorted by the device id,
currently the order was undefined.

Signed-off-by: David Sterba dste...@suse.cz
---
 Makefile  |   2 +-
 cmds-filesystem.c |  19 ++-
 list_sort.c   | 144 ++
 list_sort.h   |  14 ++
 4 files changed, 176 insertions(+), 3 deletions(-)
 create mode 100644 list_sort.c
 create mode 100644 list_sort.h

diff --git a/Makefile b/Makefile
index 3d715d8..8678b5c 100644
--- a/Makefile
+++ b/Makefile
@@ -9,7 +9,7 @@ CFLAGS = -g -O1
 objects = ctree.o disk-io.o radix-tree.o extent-tree.o print-tree.o \
  root-tree.o dir-item.o file-item.o inode-item.o inode-map.o \
  extent-cache.o extent_io.o volumes.o utils.o repair.o \
- qgroup.o raid6.o free-space-cache.o uuid-tree.o
+ qgroup.o raid6.o free-space-cache.o uuid-tree.o list_sort.o
 cmds_objects = cmds-subvolume.o cmds-filesystem.o cmds-device.o cmds-scrub.o \
   cmds-inspect.o cmds-balance.o cmds-send.o cmds-receive.o \
   cmds-quota.o cmds-qgroup.o cmds-replace.o cmds-check.o \
diff --git a/cmds-filesystem.c b/cmds-filesystem.c
index 815d59a..93fd0eb 100644
--- a/cmds-filesystem.c
+++ b/cmds-filesystem.c
@@ -28,10 +28,9 @@
 #include ioctl.h
 #include utils.h
 #include volumes.h
-
 #include version.h
-
 #include commands.h
+#include list_sort.h
 
 static const char * const filesystem_cmd_group_usage[] = {
btrfs filesystem [group] command [args],
@@ -183,6 +182,21 @@ static int uuid_search(struct btrfs_fs_devices 
*fs_devices, char *search)
return 0;
 }
 
+/*
+ * Sort devices by devid, ascending
+ */
+static int cmp_device_id(void *priv, struct list_head *a,
+   struct list_head *b)
+{
+   const struct btrfs_device *da = list_entry(a, struct btrfs_device,
+   dev_list);
+   const struct btrfs_device *db = list_entry(b, struct btrfs_device,
+   dev_list);
+
+   return da-devid  db-devid ? -1 :
+   da-devid  db-devid ? 1 : 0;
+}
+
 static void print_one_uuid(struct btrfs_fs_devices *fs_devices)
 {
char uuidbuf[37];
@@ -205,6 +219,7 @@ static void print_one_uuid(struct btrfs_fs_devices 
*fs_devices)
   (unsigned long long)total,
   pretty_size(device-super_bytes_used));
 
+   list_sort(NULL, fs_devices-devices, cmp_device_id);
list_for_each(cur, fs_devices-devices) {
device = list_entry(cur, struct btrfs_device, dev_list);
 
diff --git a/list_sort.c b/list_sort.c
new file mode 100644
index 000..f526b40
--- /dev/null
+++ b/list_sort.c
@@ -0,0 +1,144 @@
+/*
+ * taken from linux kernel lib/list_sort.c, removed uneeded code and adapted
+ * for btrfsprogs
+ */
+
+#include kerncompat.h
+#include list_sort.h
+#include list.h
+
+#define MAX_LIST_LENGTH_BITS 20
+
+/*
+ * Returns a list organized in an intermediate format suited
+ * to chaining of merge() calls: null-terminated, no reserved or
+ * sentinel head node, prev links not maintained.
+ */
+static struct list_head *merge(void *priv,
+   int (*cmp)(void *priv, struct list_head *a,
+   struct list_head *b),
+   struct list_head *a, struct list_head *b)
+{
+   struct list_head head, *tail = head;
+
+   while (a  b) {
+   /* if equal, take 'a' -- important for sort stability */
+   if ((*cmp)(priv, a, b) = 0) {
+   tail-next = a;
+   a = a-next;
+   } else {
+   tail-next = b;
+   b = b-next;
+   }
+   tail = tail-next;
+   }
+   tail-next = a?:b;
+   return head.next;
+}
+
+/*
+ * Combine final list merge with restoration of standard doubly-linked
+ * list structure.  This approach duplicates code from merge(), but
+ * runs faster than the tidier alternatives of either a separate final
+ * prev-link restoration pass, or maintaining the prev links
+ * throughout.
+ */
+static void merge_and_restore_back_links(void *priv,
+   int (*cmp)(void *priv, struct list_head *a,
+   struct list_head *b),
+   struct list_head *head,
+   struct list_head *a, struct list_head *b)
+{
+   struct list_head *tail = head;
+
+   while (a  b) {
+   /* if equal, take 'a' -- important for sort stability */
+   if ((*cmp)(priv, a, b) = 0) {
+   tail-next = a;
+   a-prev = tail;
+   a = a-next;
+   } else {
+   tail-next = b;
+   b-prev = tail;
+   b = b-next;
+   }
+   tail = tail-next;
+   }
+   tail-next = a ? : b;
+
+   do {
+

[PATCH] Btrfs: iput inode on allocation failure

[Josef Bacik]

We don't do the iput when we fail to allocate our delayed delalloc work in
__start_delalloc_inodes, fix this.

Signed-off-by: Josef Bacik jba...@fusionio.com
---
 fs/btrfs/inode.c |4 
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 85a4900..7988113 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -8211,6 +8211,10 @@ static int __start_delalloc_inodes(struct btrfs_root 
*root, int delay_iput)
 
work = btrfs_alloc_delalloc_work(inode, 0, delay_iput);
if (unlikely(!work)) {
+   if (delay_iput)
+   btrfs_add_delayed_iput(inode);
+   else
+   iput(inode);
ret = -ENOMEM;
goto out;
}
-- 
1.7.7.6

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC v2] btrfs-progs: Add recursive defrag using -r option

[Frank Holton]

Thanks for that hint to use ftw. I've updated the code to use it and
tried to make sure
that I caught all of the styling errors.

Since the ftw callback doesn't take any additional options I had to add several
global variables to pass the fancy_ioctl and range parameters. Should
I replace all
of the uses of those variables with the globals or just copy into the
globals like I did in
the code below.

It does not attempt to defrag directories anymore in the recursive
mode, however, the
non recursive mode will still attempt to defrag directories. I figured
since that only works
when you run as root that it is acceptable for now.

Thanks again for looking over it.

- Frank

---
 cmds-filesystem.c |  119 ++---
 1 file changed, 96 insertions(+), 23 deletions(-)

diff --git a/cmds-filesystem.c b/cmds-filesystem.c
index f41a72a..9635066 100644
--- a/cmds-filesystem.c
+++ b/cmds-filesystem.c
@@ -22,6 +22,8 @@
 #include errno.h
 #include uuid/uuid.h
 #include ctype.h
+#include fcntl.h
+#include ftw.h

 #include kerncompat.h
 #include ctree.h
@@ -333,6 +335,7 @@ static const char * const cmd_defrag_usage[] = {
  Defragment a file or a directory,
  ,
  -v be verbose,
+ -r defragment files recursively,
  -c[zlib,lzo]   compress the file while defragmenting,
  -f flush data to disk immediately after defragmenting,
  -s start   defragment only from byte onward,
@@ -341,6 +344,57 @@ static const char * const cmd_defrag_usage[] = {
  NULL
 };

+static int do_defrag(int fd, int fancy_ioctl,
+ struct btrfs_ioctl_defrag_range_args *range)
+{
+ int ret;
+
+ if (!fancy_ioctl) {
+ ret = ioctl(fd, BTRFS_IOC_DEFRAG, NULL);
+ } else {
+ ret = ioctl(fd, BTRFS_IOC_DEFRAG_RANGE, range);
+ }
+ return ret;
+}
+
+static int global_fancy_ioctl;
+static struct btrfs_ioctl_defrag_range_args global_range;
+static int global_verbose;
+static int global_errors;
+static int defrag_callback(const char *fpath, const struct stat *sb,
int typeflag)
+{
+ int ret = 0;
+ int e = 0;
+ int fd = 0;
+
+ if (typeflag == FTW_F) {
+ if (global_verbose)
+ printf(%s\n, fpath);
+ fd = open(fpath,O_RDWR);
+ e = errno;
+ if (fd  0)
+ goto error;
+ ret = do_defrag(fd, global_fancy_ioctl, global_range);
+ e = errno;
+ close(fd);
+ if (ret  e == ENOTTY) {
+ fprintf(stderr, ERROR: defrag range ioctl not 
+ supported in this kernel, please try 
+ without any options.\n);
+ global_errors++;
+ return ENOTTY;
+ }
+ if (ret)
+ goto error;
+ }
+ return 0;
+
+error:
+ fprintf(stderr, ERROR: defrag failed on %s - %s\n, fpath, strerror(e));
+ global_errors++;
+ return 0;
+}
+
 static int cmd_defrag(int argc, char **argv)
 {
  int fd;
@@ -349,7 +403,8 @@ static int cmd_defrag(int argc, char **argv)
  u64 len = (u64)-1;
  u32 thresh = 0;
  int i;
- int errors = 0;
+ int recursive = 0;
+ global_errors = 0;
  int ret = 0;
  int verbose = 0;
  int fancy_ioctl = 0;
@@ -359,7 +414,7 @@ static int cmd_defrag(int argc, char **argv)

  optind = 1;
  while(1) {
- int c = getopt(argc, argv, vc::fs:l:t:);
+ int c = getopt(argc, argv, vrc::fs:l:t:);
  if (c  0)
  break;

@@ -389,6 +444,9 @@ static int cmd_defrag(int argc, char **argv)
  thresh = parse_size(optarg);
  fancy_ioctl = 1;
  break;
+ case 'r':
+ recursive = 1;
+ break;
  default:
  usage(cmd_defrag_usage);
  }
@@ -407,47 +465,62 @@ static int cmd_defrag(int argc, char **argv)
  }
  if (flush)
  range.flags |= BTRFS_DEFRAG_RANGE_START_IO;
+
+ memcpy(global_range, range, sizeof(range));
+ global_verbose = verbose;
+ global_fancy_ioctl = fancy_ioctl;

  for (i = optind; i  argc; i++) {
- if (verbose)
- printf(%s\n, argv[i]);
  fd = open_file_or_dir(argv[i]);
  if (fd  0) {
- fprintf(stderr, failed to open %s\n, argv[i]);
+ fprintf(stderr, ERROR: failed to open %s\n, argv[i]);
  perror(open:);
- errors++;
+ global_errors++;
  continue;
  }
- if (!fancy_ioctl) {
- ret = ioctl(fd, BTRFS_IOC_DEFRAG, NULL);
- e=errno;
- } else {
- ret = ioctl(fd, BTRFS_IOC_DEFRAG_RANGE, range);
- if (ret  errno == ENOTTY) {
- fprintf(stderr, ERROR: defrag range ioctl not 
- supported in this kernel, please try 
- without any options.\n);
- errors++;
- close(fd);
- break;
+ if (recursive) {
+ struct stat st;
+ fstat(fd, st);
+ if (S_ISDIR(st.st_mode)) {
+ ret = ftw(argv[i], defrag_callback, 10);
+ if (ret == ENOTTY)
+ exit(1);
+ /* errors are handled in the callback */
+ ret = 0;
+ } else {
+ if (verbose)
+ printf(%s\n, argv[i]);
+ ret = do_defrag(fd, fancy_ioctl, range);
+ e = errno;
  }
+ } else {
+ if (verbose)
+ printf(%s\n, argv[i]);
+ ret = do_defrag(fd, fancy_ioctl, range);
  e = errno;
  }
+ close(fd);
+ if (ret  e == ENOTTY) {
+ fprintf(stderr, ERROR: defrag range ioctl not 
+ supported in this kernel, please try 
+ without any options.\n);
+ global_errors++;
+ break;
+ }
  if (ret) {
  fprintf(stderr, ERROR: defrag failed on %s - %s\n,
  argv[i], strerror(e));
- errors++;
+ global_errors++;
  }
- close(fd);
  }
  if (verbose)
  printf(%s\n, 

Re: rootfs crash

[Jogi Hofmüller]

Hi,

Am 2013-09-17 14:24, schrieb Jogi Hofmüller:

 Yes, did so.  It stopped with an error.  I will send the error
 message later since it resides on a different machine that I cannot
 access from where I am now.

This is the promised output of

btrfs chunk-recover -y -v /dev/sda3

[ 1090.896935] device fsid f75ca841-f09b-4341-8b59-8fcab921d8ce devid 2
transid 103536 /dev/sdb3
[ 1090.900656] device fsid f75ca841-f09b-4341-8b59-8fcab921d8ce devid 1
transid 103536 /dev/sda3
[ 1090.905844] device fsid f75ca841-f09b-4341-8b59-8fcab921d8ce devid 2
transid 103536 /dev/sdb3
[ 1090.909114] device fsid f75ca841-f09b-4341-8b59-8fcab921d8ce devid 1
transid 103536 /dev/sda3

All Devices:
Device: id = 2, name = /dev/sdb3
Device: id = 1, name = /dev/sda3

btrfs: cmds-chunk.c:125: process_extent_buffer: Assertion
`!(exists-nmirrors =2)´ failed.

Regards!
-- 
j.hofmüllerhttp://users.mur.at/thesix/



signature.asc
Description: OpenPGP digital signature

[PATCH 3/3] btrfs-progs: separate command ant implementation of chunk-recover code

[David Sterba]

The command has been moved and we should rename the files accordingly,
so the entry point is now in cmds-rescue.c and the core functionality
in it's own file.

Return codes of btrfs_recover_chunk_tree have been simplified not to
require a define and another file for defintion.

CC: Miao Xie mi...@cn.fujitsu.com
Signed-off-by: David Sterba dste...@suse.cz
---
 Makefile|3 +-
 chunk-recover.c | 1745 +++
 cmds-rescue.c   |   62 ++
 3 files changed, 1809 insertions(+), 1 deletion(-)
 create mode 100644 chunk-recover.c

diff --git a/Makefile b/Makefile
index 9e58028..bd32121 100644
--- a/Makefile
+++ b/Makefile
@@ -13,7 +13,8 @@ objects = ctree.o disk-io.o radix-tree.o extent-tree.o 
print-tree.o \
 cmds_objects = cmds-subvolume.o cmds-filesystem.o cmds-device.o cmds-scrub.o \
   cmds-inspect.o cmds-balance.o cmds-send.o cmds-receive.o \
   cmds-quota.o cmds-qgroup.o cmds-replace.o cmds-check.o \
-  cmds-restore.o cmds-chunk.o cmds-dedup.o cmds-rescue.o
+  cmds-restore.o cmds-dedup.o cmds-rescue.o \
+  chunk-recover.o
 libbtrfs_objects = send-stream.o send-utils.o rbtree.o btrfs-list.o crc32c.o \
   uuid-tree.o
 libbtrfs_headers = send-stream.h send-utils.h send.h rbtree.h btrfs-list.h \
diff --git a/chunk-recover.c b/chunk-recover.c
new file mode 100644
index 000..45da9a6
--- /dev/null
+++ b/chunk-recover.c
@@ -0,0 +1,1745 @@
+/*
+ * Copyright (C) 2013 Fujitsu.  All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+#define _XOPEN_SOURCE 500
+#define _GNU_SOURCE
+
+#include stdio.h
+#include stdio_ext.h
+#include stdlib.h
+#include sys/types.h
+#include sys/stat.h
+#include fcntl.h
+#include unistd.h
+#include uuid/uuid.h
+
+#include kerncompat.h
+#include list.h
+#include radix-tree.h
+#include ctree.h
+#include extent-cache.h
+#include disk-io.h
+#include volumes.h
+#include transaction.h
+#include crc32c.h
+#include utils.h
+#include version.h
+#include btrfsck.h
+#include commands.h
+
+#define BTRFS_STRIPE_LEN   (64 * 1024)
+#define BTRFS_NUM_MIRRORS  2
+
+struct recover_control {
+   int verbose;
+   int yes;
+
+   u16 csum_size;
+   u32 sectorsize;
+   u32 leafsize;
+   u64 generation;
+   u64 chunk_root_generation;
+
+   struct btrfs_fs_devices *fs_devices;
+
+   struct cache_tree chunk;
+   struct block_group_tree bg;
+   struct device_extent_tree devext;
+   struct cache_tree eb_cache;
+
+   struct list_head good_chunks;
+   struct list_head bad_chunks;
+   struct list_head unrepaired_chunks;
+};
+
+struct extent_record {
+   struct cache_extent cache;
+   u64 generation;
+   u8 csum[BTRFS_CSUM_SIZE];
+   struct btrfs_device *devices[BTRFS_NUM_MIRRORS];
+   u64 offsets[BTRFS_NUM_MIRRORS];
+   int nmirrors;
+};
+
+static struct extent_record *btrfs_new_extent_record(struct extent_buffer *eb)
+{
+   struct extent_record *rec;
+
+   rec = malloc(sizeof(*rec));
+   if (!rec) {
+   fprintf(stderr, Fail to allocate memory for extent record.\n);
+   exit(1);
+   }
+
+   memset(rec, 0, sizeof(*rec));
+   rec-cache.start = btrfs_header_bytenr(eb);
+   rec-cache.size = eb-len;
+   rec-generation = btrfs_header_generation(eb);
+   read_extent_buffer(eb, rec-csum, (unsigned long)btrfs_header_csum(eb),
+  BTRFS_CSUM_SIZE);
+   return rec;
+}
+
+static int process_extent_buffer(struct cache_tree *eb_cache,
+struct extent_buffer *eb,
+struct btrfs_device *device, u64 offset)
+{
+   struct extent_record *rec;
+   struct extent_record *exist;
+   struct cache_extent *cache;
+   int ret = 0;
+
+   rec = btrfs_new_extent_record(eb);
+   if (!rec-cache.size)
+   goto free_out;
+again:
+   cache = lookup_cache_extent(eb_cache,
+   rec-cache.start,
+   rec-cache.size);
+   if (cache) {
+   exist = container_of(cache, struct extent_record, cache);
+
+   if (exist-generation  rec-generation)
+   goto free_out;
+   if 

Re: rootfs crash

[Clemens Eisserer]

Hi Jogi,

When I tried to recovered a broken btrfs-parition, I encountered the
same errors you did - btrfs restore seems to be a bit broken.

What helped was to:
a. comment out the free()-calls which led to crashes
b. re-run btrfs restore to account for the crashes which happen
elsewhere, hopefully it will progress each run a little further as it
did for me.

Regards, Clemens
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs-progs: Add dependencies explicitly to fix a parallel build issue

[rongqing.li]

From: Roy Li rongqing...@windriver.com

The dependencies of all: version.h or other similar ones can not
fix the parallel build failure, only reduce the times; In fact,
many *.o files require version.h file.

#grep '#include version.h' ./ -r
./btrfs-corrupt-block.c:#include version.h
./btrfs.c:#include version.h
./btrfs-image.c:#include version.h
./cmds-filesystem.c:#include version.h
./btrfs-show-super.c:#include version.h
./btrfs-select-super.c:#include version.h
./cmds-restore.c:#include version.h
./btrfs-find-root.c:#include version.h
./mkfs.c:#include version.h
./btrfs-zero-log.c:#include version.h
./btrfs-defrag.c:#include version.h
./cmds-chunk.c:#include version.h
./btrfstune.c:#include version.h
./btrfs-calc-size.c:#include version.h
./btrfs-map-logical.c:#include version.h
./cmds-check.c:#include version.h
./btrfs-debug-tree.c:#include version.h

Signed-off-by: Roy Li rongqing...@windriver.com
---
 Sorry, The patch [btrfs-progs: fix parallel build] sent by me on Sep 3
can not fix the build failure, when build enough times on a 16 core cpu,
the build failure happens again, so I refix it again.


 Makefile |6 ++
 1 file changed, 6 insertions(+)

diff --git a/Makefile b/Makefile
index c43cb68..a7c259c 100644
--- a/Makefile
+++ b/Makefile
@@ -81,6 +81,12 @@ endif
@echo [CC] $@
$(Q)$(CC) $(DEPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c $
 
+btrfs-corrupt-block.o btrfs.o btrfs-image.o cmds-filesystem.o:version.h
+btrfs-show-super.o btrfs-select-super.o cmds-restore.o:version.h
+btrfs-find-root.o mkfs.o btrfs-zero-log.o btrfs-defrag.o cmds-chunk.o:version.h
+btrfstune.o btrfs-calc-size.o btrfs-map-logical.o cmds-check.o:version.h
+btrfs-debug-tree.o:version.h
+
 %.static.o: %.c
@echo [CC] $@
$(Q)$(CC) $(DEPFLAGS) $(AM_CFLAGS) $(STATIC_CFLAGS) -c $ -o $@
-- 
1.7.10.4

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs-progs: Add dependencies explicitly to fix a parallel build issue

[Eric Sandeen]

On 9/17/13 8:11 PM, rongqing...@windriver.com wrote:
 From: Roy Li rongqing...@windriver.com
 
 The dependencies of all: version.h or other similar ones can not
 fix the parallel build failure, only reduce the times; In fact,
 many *.o files require version.h file.
 
   #grep '#include version.h' ./ -r
   ./btrfs-corrupt-block.c:#include version.h
   ./btrfs.c:#include version.h
   ./btrfs-image.c:#include version.h
   ./cmds-filesystem.c:#include version.h
   ./btrfs-show-super.c:#include version.h
   ./btrfs-select-super.c:#include version.h
   ./cmds-restore.c:#include version.h
   ./btrfs-find-root.c:#include version.h
   ./mkfs.c:#include version.h
   ./btrfs-zero-log.c:#include version.h
   ./btrfs-defrag.c:#include version.h
   ./cmds-chunk.c:#include version.h
   ./btrfstune.c:#include version.h
   ./btrfs-calc-size.c:#include version.h
   ./btrfs-map-logical.c:#include version.h
   ./cmds-check.c:#include version.h
   ./btrfs-debug-tree.c:#include version.h
 
 Signed-off-by: Roy Li rongqing...@windriver.com
 ---
  Sorry, The patch [btrfs-progs: fix parallel build] sent by me on Sep 3
 can not fix the build failure, when build enough times on a 16 core cpu,
 the build failure happens again, so I refix it again.
 
 
  Makefile |6 ++
  1 file changed, 6 insertions(+)
 
 diff --git a/Makefile b/Makefile
 index c43cb68..a7c259c 100644
 --- a/Makefile
 +++ b/Makefile
 @@ -81,6 +81,12 @@ endif
   @echo [CC] $@
   $(Q)$(CC) $(DEPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c $
  
 +btrfs-corrupt-block.o btrfs.o btrfs-image.o cmds-filesystem.o:version.h
 +btrfs-show-super.o btrfs-select-super.o cmds-restore.o:version.h
 +btrfs-find-root.o mkfs.o btrfs-zero-log.o btrfs-defrag.o 
 cmds-chunk.o:version.h
 +btrfstune.o btrfs-calc-size.o btrfs-map-logical.o cmds-check.o:version.h
 +btrfs-debug-tree.o:version.h
 +
  %.static.o: %.c
   @echo [CC] $@
   $(Q)$(CC) $(DEPFLAGS) $(AM_CFLAGS) $(STATIC_CFLAGS) -c $ -o $@
 

I think this can be done more cleanly, I'll send a patch.

-Eric
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] btrfs-progs: Add dependencies explicitly to fix a parallel build issue

[Rongqing Li]

On 09/18/2013 09:51 AM, Eric Sandeen wrote:

@echo [CC] $@
$(Q)$(CC) $(DEPFLAGS) $(AM_CFLAGS) $(STATIC_CFLAGS) -c $ -o $@


I think this can be done more cleanly, I'll send a patch.

-Eric




Thanks, expect your patch


-Roy

--
Best Reagrds,
Roy | RongQing Li
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs-progs: Add version.h dependency to fix parallel builds

[Eric Sandeen]

Change the suffix rule to ensure that version.h is 
built before we try to create any .o file.

Reported-by: Roy Li rongqing...@windriver.com
Signed-off-by: Eric Sandeen sand...@redhat.com
---

I dunno, does this work?  My gnu make is rusty.

diff --git a/Makefile b/Makefile
index 3d715d8..d25054f 100644
--- a/Makefile
+++ b/Makefile
@@ -107,13 +107,13 @@ endif
 %.o.d: %.c
$(Q)$(CC) -MM -MG -MF $@ -MT $(@:.o.d=.o) -MT $(@:.o.d=.static.o) -MT 
$@ $(AM_CFLAGS) $(CFLAGS) $
 
-.c.o:
+%.o: %.c version.h
@$(check_echo) [SP] $
$(Q)$(check) $(AM_CFLAGS) $(CFLAGS) $
@echo [CC] $@
$(Q)$(CC) $(AM_CFLAGS) $(CFLAGS) -c $
 
-%.static.o: %.c
+%.static.o: %.c version.h
@echo [CC] $@
$(Q)$(CC) $(AM_CFLAGS) $(STATIC_CFLAGS) -c $ -o $@
 
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] btrfs-progs: Add version.h dependency to fix parallel builds

[Eric Sandeen]

On 9/17/13 8:56 PM, Eric Sandeen wrote:
 Change the suffix rule to ensure that version.h is 
 built before we try to create any .o file.
 
 Reported-by: Roy Li rongqing...@windriver.com
 Signed-off-by: Eric Sandeen sand...@redhat.com
 ---
 
 I dunno, does this work?  My gnu make is rusty.

I'm actually confused by what the makefile is doing with the
dependency generation, maybe my patch isn't right.

this:

%.o.d: %.c
$(Q)$(CC) -MM -MG -MF $@ -MT $(@:.o.d=.o) -MT $(@:.o.d=.static.o) -MT 
$@ $(AM_CFLAGS) $(CFLAGS) $

completely confuses me :)

-Eric

 diff --git a/Makefile b/Makefile
 index 3d715d8..d25054f 100644
 --- a/Makefile
 +++ b/Makefile
 @@ -107,13 +107,13 @@ endif
  %.o.d: %.c
   $(Q)$(CC) -MM -MG -MF $@ -MT $(@:.o.d=.o) -MT $(@:.o.d=.static.o) -MT 
 $@ $(AM_CFLAGS) $(CFLAGS) $
  
 -.c.o:
 +%.o: %.c version.h
   @$(check_echo) [SP] $
   $(Q)$(check) $(AM_CFLAGS) $(CFLAGS) $
   @echo [CC] $@
   $(Q)$(CC) $(AM_CFLAGS) $(CFLAGS) -c $
  
 -%.static.o: %.c
 +%.static.o: %.c version.h
   @echo [CC] $@
   $(Q)$(CC) $(AM_CFLAGS) $(STATIC_CFLAGS) -c $ -o $@
  
 --
 To unsubscribe from this list: send the line unsubscribe linux-btrfs in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
 

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Btrfs: allocate the free space by the existed max extent size when ENOSPC

[Miao Xie]

On Tue, 17 Sep 2013 15:13:12 +0200, David Sterba wrote:
 On Mon, Sep 09, 2013 at 02:21:58PM +0800, Miao Xie wrote:
 On fri, 06 Sep 2013 15:47:08 +0200, Stefan Behrens wrote:
 This patch makes the xfstest generic/256 lock up. It's quite
 reliably reproducible on one of my test boxes, and not at all
 visible on a second test box.

 And yes, I'm using the V2 patch although you haven't tagged it as V2
 in the subject line of the mail :)

 According to my debug, the machine was not locked up, it seems the
 patch makes the test run very slow(90s -850s on my machine).
 
 The massive slowdown is present in current 3.12-rc code, which means V3
 did not make it into the pull.
 
 Miao please send an incremental fix for rc2.

This patch has not been merged into 3.12-rc, so the above problem should be
introduced by other patch.

Thanks
Miao

 
 thanks,
 david
 

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] xfstests/btrfs: do not test btrfs/010 with autodefrag

[Liu Bo]

btrfs/010 is going to create a fragmented file, however, with autodefrag
this is impossible, so just skip the test when we're with autodefrag.

Signed-off-by: Liu Bo bo.li@oracle.com
---
 tests/btrfs/010 |   12 
 1 files changed, 12 insertions(+), 0 deletions(-)

diff --git a/tests/btrfs/010 b/tests/btrfs/010
index 371f1b8..608942d 100755
--- a/tests/btrfs/010
+++ b/tests/btrfs/010
@@ -37,6 +37,14 @@ _cleanup()
 rm -f $tmp.*
 }
 
+is_autodefrag_set()
+{
+   case $MOUNT_OPTIONS in
+   *autodefrag*) return 0;;
+   esac
+   return 1
+}
+
 # get standard environment, filters and checks
 . ./common/rc
 . ./common/filter
@@ -49,6 +57,10 @@ _supported_os Linux
 _require_scratch
 _require_command /usr/sbin/filefrag
 
+if is_autodefrag_set; then
+   _notrun this test makes no sense with autodefrag
+fi
+
 rm -f $seqres.full
 
 _scratch_mkfs  /dev/null 21
-- 
1.7.7

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html