Re: Premature ENOSPC only with zlib Compression

2012-02-16 Thread Ahmet Inan
On Wed, Jan 18, 2012 at 5:13 PM, Mitch Harder
mitch.har...@sabayonlinux.org wrote:
 I have a Btrfs partition that is reliably reproducing premature ENOSPC
 when restoring the disk from a tar file, but it is only happening with
 zlib compression (lzo or no compression proceeds normally).

Ive just unsquashfs'ed an 5GiB image on a fresh btrfs and a much
faster System using:

compress=lzo,noatime

and got ENOSPC again :(

so it doesnt matter if lzo or zlib: ENOSPC with compression enabled!

my kernel: vanilla 3.2.5

for testing, i suggest making a squashfs of your whole system / and
unsquashfs that image into a new subvolume.
this really seems to trigger this ENOSPC problem, because it creates
the files fast enough.

Ahmet
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[RFC][PATCH] Btrfs: introduce extent buffer cache for each i-node

2012-02-16 Thread Miao Xie
This patch introduce extent buffer cache for every i-node. By this
way, we needn't search the item from the root of b+ tree, and can
save the search time. Besides that we can also reduce the lock contention
of the root.

Implementation:
- add two pointers of extent buffer into btrfs_inode struct, one for
  nodes/leaves of fs/file tree, the other for nodes/leaves of the log tree.
- add a BTRFS_HEADER_FLAG_COWED flag, which is used to mark the cowed
  nodes/leaves
- When we want to search fs/file trees or the relative log trees, we must
  try to search the cached extent buffer at first, if we can not find the
  item, we will do the common search. At some condition, we will jump to
  the common search directly:
  I. a new transaction starts.
  II. the cached extent buffer was cowed.
  III. the cached extent buffer's level is below the level we must lock.
  and so on.
  And beside that, if we can not find the item, and the slot points to the
  first the item or the last item in the leaf, we must jump out the cached
  extent buffer search, and do the common search.
- After the common search (use btrfs_search_slot), we will cache the leaf
  or the level-1 node into the btrfs i-node object.

I have do small file write performance (inline file) by sysbench for the patch,
and found it can make btrfs 5%~10% faster.

Surely, I will do function test(such as xfstests) and run more performance test
next to validate this patch. And there is some redundant code in this patch. I
will cleanup it in the next version.

Signed-off-by: Miao Xie mi...@cn.fujitsu.com
---
 fs/btrfs/btrfs_inode.h |3 +
 fs/btrfs/ctree.c   |  358 ++--
 fs/btrfs/ctree.h   |   41 +-
 fs/btrfs/dir-item.c|   28 +++--
 fs/btrfs/extent_io.c   |2 +-
 fs/btrfs/file-item.c   |   14 +-
 fs/btrfs/file.c|5 +-
 fs/btrfs/inode-item.c  |   11 +-
 fs/btrfs/inode.c   |   44 ---
 fs/btrfs/ioctl.c   |2 +-
 fs/btrfs/relocation.c  |2 +-
 fs/btrfs/super.c   |3 +-
 fs/btrfs/tree-log.c|   19 ++-
 fs/btrfs/xattr.c   |2 +-
 14 files changed, 463 insertions(+), 71 deletions(-)

diff --git a/fs/btrfs/btrfs_inode.h b/fs/btrfs/btrfs_inode.h
index 9b9b15f..2374920 100644
--- a/fs/btrfs/btrfs_inode.h
+++ b/fs/btrfs/btrfs_inode.h
@@ -164,6 +164,9 @@ struct btrfs_inode {
 
struct btrfs_delayed_node *delayed_node;
 
+   struct extent_buffer *fs_eb;
+   struct extent_buffer *log_eb;
+
struct inode vfs_inode;
 };
 
diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c
index 0639a55..693e499 100644
--- a/fs/btrfs/ctree.c
+++ b/fs/btrfs/ctree.c
@@ -458,6 +458,7 @@ static noinline int __btrfs_cow_block(struct 
btrfs_trans_handle *trans,
btrfs_set_header_backref_rev(cow, BTRFS_MIXED_BACKREF_REV);
btrfs_clear_header_flag(cow, BTRFS_HEADER_FLAG_WRITTEN |
 BTRFS_HEADER_FLAG_RELOC);
+   btrfs_set_header_flag(buf, BTRFS_HEADER_FLAG_COWED);
if (root-root_key.objectid == BTRFS_TREE_RELOC_OBJECTID)
btrfs_set_header_flag(cow, BTRFS_HEADER_FLAG_RELOC);
else
@@ -1533,6 +1534,22 @@ read_block_for_search(struct btrfs_trans_handle *trans,
return ret;
 }
 
+static int check_nodes_need_balance(struct btrfs_root *root,
+   struct btrfs_path *p,
+   struct extent_buffer *b,
+   int level, int ins_len)
+{
+   if ((p-search_for_split || ins_len  0)  btrfs_header_nritems(b) =
+   BTRFS_NODEPTRS_PER_BLOCK(root) - 3)
+   return 1;
+
+   if (ins_len  0 
+   btrfs_header_nritems(b)  BTRFS_NODEPTRS_PER_BLOCK(root) / 2)
+   return 1;
+
+   return 0;
+}
+
 /*
  * helper function for btrfs_search_slot.  This does all of the checks
  * for node-level blocks and does any balancing required based on
@@ -1874,6 +1891,299 @@ done:
return ret;
 }
 
+static int btrfs_search_cached_extent_buffer(struct btrfs_trans_handle *trans,
+   struct btrfs_root *root, struct btrfs_inode *inode,
+   struct btrfs_key *key, struct btrfs_path *p,
+   int ins_len, int cow)
+{
+   struct extent_buffer *b;
+   int slot;
+   int ret;
+   int err;
+   int level;
+   int lowest_unlock = 1;
+   int write_lock_level = 0;
+   u8 lowest_level = 0;
+
+   if (cow  (p-keep_locks || p-lowest_level))
+   return -EAGAIN;
+
+   if (unlikely(p-search_commit_root))
+   return -EAGAIN;
+
+   lowest_level = p-lowest_level;
+   WARN_ON(lowest_level  ins_len  0);
+   WARN_ON(p-nodes[0] != NULL);
+
+   if (ins_len  0) {
+   lowest_unlock = 2;
+   write_lock_level = 1;
+   }
+
+   if (!cow)
+   write_lock_level = -1;
+
+again:
+   if (unlikely(root-objectid == BTRFS_TREE_LOG_OBJECTID))
+   

[PATCH 2/4] Btrfs: skip states when they does not contain bits to clear

2012-02-16 Thread Liu Bo
Clearing a range's bits is different with setting them, since we don't
need to touch them when states do not contain bits we want.

Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com
---
 fs/btrfs/extent_io.c |   15 ++-
 1 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
index e941cc4..e791fef 100644
--- a/fs/btrfs/extent_io.c
+++ b/fs/btrfs/extent_io.c
@@ -513,6 +513,15 @@ hit_next:
WARN_ON(state-end  start);
last_end = state-end;
 
+   if (state-end  end  !need_resched())
+   next_node = rb_next(state-rb_node);
+   else
+   next_node = NULL;
+
+   /* the state doesn't have the wanted bits, go ahead */
+   if (!(state-state  bits))
+   goto next;
+
/*
 * |  desired range  |
 *  | state | or
@@ -565,12 +574,8 @@ hit_next:
goto out;
}
 
-   if (state-end  end  prealloc  !need_resched())
-   next_node = rb_next(state-rb_node);
-   else
-   next_node = NULL;
-
set |= clear_state_bit(tree, state, bits, wake);
+next:
if (last_end == (u64)-1)
goto out;
start = last_end + 1;
-- 
1.6.5.2

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 1/4] Btrfs: be less strict on finding next node in clear_extent_bit

2012-02-16 Thread Liu Bo
In clear_extent_bit, it is enough that next node is adjacent in tree level.

Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com
---
 fs/btrfs/extent_io.c |3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
index fcf77e1..e941cc4 100644
--- a/fs/btrfs/extent_io.c
+++ b/fs/btrfs/extent_io.c
@@ -577,8 +577,7 @@ hit_next:
if (start = end  next_node) {
state = rb_entry(next_node, struct extent_state,
 rb_node);
-   if (state-start == start)
-   goto hit_next;
+   goto hit_next;
}
goto search_again;
 
-- 
1.6.5.2

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 3/4] Btrfs: kick out redundant stuff in convert_extent_bit

2012-02-16 Thread Liu Bo
clear_state_bit will do merge_state for us, so kick out the redundant one.

Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com
---
 fs/btrfs/extent_io.c |5 -
 1 files changed, 0 insertions(+), 5 deletions(-)

diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
index e791fef..039c5c6 100644
--- a/fs/btrfs/extent_io.c
+++ b/fs/btrfs/extent_io.c
@@ -965,8 +965,6 @@ hit_next:
 
set_state_bits(tree, state, bits);
clear_state_bit(tree, state, clear_bits, 0);
-
-   merge_state(tree, state);
if (last_end == (u64)-1)
goto out;
 
@@ -1011,7 +1009,6 @@ hit_next:
if (state-end = end) {
set_state_bits(tree, state, bits);
clear_state_bit(tree, state, clear_bits, 0);
-   merge_state(tree, state);
if (last_end == (u64)-1)
goto out;
start = last_end + 1;
@@ -1072,8 +1069,6 @@ hit_next:
 
set_state_bits(tree, prealloc, bits);
clear_state_bit(tree, prealloc, clear_bits, 0);
-
-   merge_state(tree, prealloc);
prealloc = NULL;
goto out;
}
-- 
1.6.5.2

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Problems Implementing Snappy Patches

2012-02-16 Thread Mitch Harder
On Wed, Feb 15, 2012 at 8:06 PM, Li Zefan l...@cn.fujitsu.com wrote:
 Mitch Harder wrote:
 When I copy directory containing a kernel sources git repository to a
 freshly formated partition mounted with snappy, I get a corrupted
 copy.  If I mount with lzo or lz4 compression, I don't see any
 corruptions from the copy.

 I'm not showing any errors in dmesg.


 You're not seeing any errors, right?

That is correct.

I'm not seeing any errors thrown out directly from the copy command,
nor am I seeing anything else in the dmesg log.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [RFB] add LZ4 compression method to btrfs

2012-02-16 Thread Hugo Chevrain
You're right about the broken links, my bad.


The first link related to a few statements by Mark Ruijter, copy/pasted below :

 I just looked into it and it appears to be promising. 
 On average the speeds appear to be 48% faster then snappy.
 This is amazing since snappy has proved to be 30% faster then QuickLZ 
 which once was the fastest compression protocol.

== http://www.lessfs.com/wordpress/?p=684#comments

 I did not test LZ4 on high end hardware yet. 
 However even on my laptop it is clear that LZ4 does outperform snappy. 
 With the hardware being the bottleneck 
 LZ4 still manages to speed things up by 2~5%. 
 Most likely the difference will be larger when fast hardware is used

== http://www.lessfs.com/wordpress/?p=688

There is no precise benchmark though. Just this sentence :

 I will post the exact performance numbers on low and high end hardware 
 after testing has finished.

So maybe someone has to ask Mark about these results.


The second link initially linked to Hadoop, which recently decided to integrate
LZ4 compression :
== https://issues.apache.org/jira/browse/HADOOP-7657

However, since then, i've discovered the website of the developper in charge of
this patch (Binglin Chang), and he has come nice comparison figures available.
The main summary is copy/pasted below :

 Perf.RawCompressionLZ4Block size: 64K
 Compress:  428M/s Decompress:   681M/s( 1584M/s) ratio: 43.0% - Total

 Perf.RawCompressionSnappy Block size: 64K
 Compress:  400M/s Decompress:   442M/s(  979M/s) ratio: 45.2% - Total

== https://github.com/decster/jnicompressions/blob/master/README.md


--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Problems Implementing Snappy Patches

2012-02-16 Thread David Sterba
On Thu, Feb 16, 2012 at 10:06:13AM +0800, Li Zefan wrote:
 Mitch Harder wrote:
  I've been trying to test the snappy compression patches, but I'm
  getting corruptions when trying to use snappy as built on my system.
  
  I've also tried sourcing my snappy patches from Chris Mason's snappy
  branch on kernel.org with the same result.
  
 
 A month ago I saved emails as patches and applied them on linux-btrfs'
 for-linus branch, and I got the same problem. Then I used the snappy
 branch (plus my fix), and it was fine. Don't know why.

The decompression corruptions Mitch reported look same as what you sent.
And I don't think they are caused by misapplied patches, I've checked
both 'mbox via mutt' - git or pull from Chris' branch, + your patches to
fix the inlined extents, no other changes.

I did this test:
* mkfs
* mount with compress-force=snappy
* copy linux-3.2
* drop caches  (or umount/mount)
* check md5sums from original

Result:
md5sum: WARNING: 32986 computed checksums did NOT match

there are 37617 files in linux-3.2/

syslog says just:
[  728.744179] Btrfs loaded
[  728.746332] device fsid 194bbb84-1f87-4a37-8091-957d557633f1 devid 1 transid 
4 /dev/sda9
[  728.747693] btrfs: enabling auto defrag
[  728.747698] btrfs: enabling inode map caching
[  728.747702] btrfs: use snappy compression
[  728.747704] btrfs: disk space caching is enabled

  My system is a Core 2 Duo x86_64 Sabayon based system (Gentoo is our
  parent distro).  My target btrfs/snappy partition is a 16 GB partition
  I use for testing on a 500GB Western Digital Hard Drive.

I use the same testbox and disk as always, no related hw problems in
syslog.


david
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


NOCOW + compress-force = bug

2012-02-16 Thread Roman Mamedov
Hello,

Please be aware that there seems to be a possible problem with using NOCOW
flag on files situated on a filesystem mounted with compress-force(=lzo, in my
case).

Since experimenting with NOCOW, I started regularly hitting this BUG at
extent-tree.c:5813 

  5813 BUG_ON(!(flags  BTRFS_BLOCK_FLAG_FULL_BACKREF));

I was unable to make netconsole work over a bridged interface, so can only
post screenshots of this OOPS:
  http://romanrm.ru/pics/2012/2012-02-16-btrfs-bug-1.jpg
  http://romanrm.ru/pics/2012/2012-02-16-btrfs-bug-2.jpg

This happened four times already, and always on snapshot creation (but not
every case). I have hourly snapshots in crontab, and only one case out of about
ten fails with this problem. Did not try to deliberately reproduce it yet by
manually making snapshots very often, etc.

On Mon, 13 Feb 2012 15:10:40 +0100
David Sterba d...@jikos.cz wrote:

 Hi,
 
 On Mon, Feb 13, 2012 at 04:40:03PM +0900, dima wrote:
  Actually it is possible. Check out David's response to my question from 
  some time ago:
  http://permalink.gmane.org/gmane.comp.file-systems.btrfs/14227
 
 this was a quick aid, please see attached file for an updated tool to set
 the file flags, now added 'z' for NOCOMPRESS flag, and supports chattr
 syntax plus all of the standard file flags.
 
 Setting and unsetting nocow is done like 'fileflags +C file' or -C for
 unseting. Without any + or - options it prints current state.
 
 
 david


-- 
With respect,
Roman

~~~
Stallman had a printer,
with code he could not see.
So he began to tinker,
and set the software free.


signature.asc
Description: PGP signature


[PATCH] btrfs/check-integrity.c: included disk-io.h twice

2012-02-16 Thread Danny Kukawka
fs/btrfs/check-integrity.c included disk-io.h twice, remove the
duplicate.

Signed-off-by: Danny Kukawka danny.kuka...@bisect.de
---
 fs/btrfs/check-integrity.c |1 -
 1 files changed, 0 insertions(+), 1 deletions(-)

diff --git a/fs/btrfs/check-integrity.c b/fs/btrfs/check-integrity.c
index b669a7d..064b29b 100644
--- a/fs/btrfs/check-integrity.c
+++ b/fs/btrfs/check-integrity.c
@@ -89,7 +89,6 @@
 #include disk-io.h
 #include transaction.h
 #include extent_io.h
-#include disk-io.h
 #include volumes.h
 #include print-tree.h
 #include locking.h
-- 
1.7.8.3

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: NOCOW + compress-force = bug

2012-02-16 Thread David Sterba
On Thu, Feb 16, 2012 at 07:55:15PM +0600, Roman Mamedov wrote:
   5813 BUG_ON(!(flags  BTRFS_BLOCK_FLAG_FULL_BACKREF));
 
 I was unable to make netconsole work over a bridged interface, so can only
 post screenshots of this OOPS:
   http://romanrm.ru/pics/2012/2012-02-16-btrfs-bug-1.jpg
   http://romanrm.ru/pics/2012/2012-02-16-btrfs-bug-2.jpg
 
 This happened four times already, and always on snapshot creation (but not
 every case). I have hourly snapshots in crontab, and only one case out of 
 about
 ten fails with this problem. Did not try to deliberately reproduce it yet by
 manually making snapshots very often, etc.

I've hit a different bug with

* a process doing random reads/writes to a file
* looped toggling of NOCOW flag on the file every 10 seconds
* snapshotting of the fs every 10 seconds

and it triggered after a few minutes. It's an ENOSPC at

920 btrfs_i_size_write(parent_inode, parent_inode-i_size +
921  dentry-d_name.len * 2);
922 ret = btrfs_update_inode(trans, parent_root, parent_inode);
923 BUG_ON(ret);


[49220.539979] [ cut here ]
[49220.543580] kernel BUG at fs/btrfs/transaction.c:923!
[49220.543580] invalid opcode:  [#1] SMP
[49220.543580] CPU 0
[49220.543580] Modules linked in: btrfs aoe [last unloaded: btrfs]
[49220.543580]
[49220.543580] Pid: 12646, comm: btrfs Not tainted 3.2.0-default+ #114 Intel 
Corporation Santa Rosa platform/Matanzas
[49220.543580] RIP: 0010:[a01086da]  [a01086da] 
create_pending_snapshot+0x4ca/0x4d0 [btrfs]
[49220.543580] RSP: 0018:88869b48  EFLAGS: 00010286
[49220.543580] RAX: ffe4 RBX: 8800191afdf8 RCX: 88000e8679d8
[49220.543580] RDX: 88006cddd440 RSI: 8187d42d RDI: 0246
[49220.543580] RBP: 88869c08 R08:  R09: 
[49220.543580] R10:  R11:  R12: 8800347b6740
[49220.543580] R13: 880054eb5180 R14: 880062b56000 R15: 880078f45510
[49220.543580] FS:  7f6637f5c740() GS:88007dc0() 
knlGS:
[49220.543580] CS:  0010 DS:  ES:  CR0: 8005003b
[49220.543580] CR2: 7f80a26c7000 CR3: 1edad000 CR4: 06f0
[49220.543580] DR0:  DR1:  DR2: 
[49220.543580] DR3:  DR6: 0ff0 DR7: 0400
[49220.543580] Process btrfs (pid: 12646, threadinfo 88868000, task 
88006cddd440)
[49220.543580] Stack:
[49220.543580]  8802 0083 880078394000 
8800105d8390
[49220.543580]  88001016bcd0 880062b56000 88007a15e070 
8800105d8390
[49220.543580]  880079c14000 8800347b6758 0175 
ff84
[49220.543580] Call Trace:
[49220.543580]  [a00e65fa] ? btrfs_free_path+0x2a/0x40 [btrfs]
[49220.543580]  [a0108726] create_pending_snapshots+0x46/0x70 [btrfs]
[49220.543580]  [a0109a57] btrfs_commit_transaction+0x3e7/0x8d0 
[btrfs]
[49220.543580]  [81077aa0] ? wake_up_bit+0x40/0x40
[49220.543580]  [81357eee] ? do_raw_spin_unlock+0x5e/0xb0
[49220.543580]  [a0139e48] btrfs_mksubvol+0x358/0x360 [btrfs]
[49220.543580]  [a0139f50] 
btrfs_ioctl_snap_create_transid+0x100/0x160 [btrfs]
[49220.543580]  [81113f73] ? might_fault+0x53/0xb0
[49220.543580]  [a013a12d] 
btrfs_ioctl_snap_create_v2.clone.0+0xfd/0x110 [btrfs]
[49220.543580]  [a013ba47] btrfs_ioctl+0x4a7/0x1270 [btrfs]
[49220.543580]  [81883400] ? do_page_fault+0x2d0/0x580
[49220.543580]  [8187f150] ? _raw_spin_unlock_irq+0x30/0x50
[49220.543580]  [8103ff23] ? finish_task_switch+0x83/0xf0
[49220.543580]  [8103fee6] ? finish_task_switch+0x46/0xf0
[49220.543580]  [8114d5e8] do_vfs_ioctl+0x98/0x560
[49220.543580]  [8108b7a9] ? trace_hardirqs_off_caller+0x29/0xc0
[49220.543580]  [8187f919] ? retint_swapgs+0x13/0x1b
[49220.543580]  [8114daff] sys_ioctl+0x4f/0x80
[49220.543580]  [81887c02] system_call_fastpath+0x16/0x1b
[49220.543580] Code: 75 fd ff ff 0f 1f 44 00 00 0f b7 16 83 e8 02 48 83 c6 02 
66 89 17 48 83 c7 02 e9 64 fd ff ff 0f 0b 0f 0b 0f 0b 0f 0b 0f 0b 0f 0b 0f 0b 
0f 1f 40 00 55 48 89 e5 41 56 41 55 41 54 53 66 66 66 66
[49220.543580] RIP  [a01086da] create_pending_snapshot+0x4ca/0x4d0 
[btrfs]
[49220.543580]  RSP 88869b48
[49220.937724] ---[ end trace 8f8b8a074f8e75d4 ]---
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Premature ENOSPC only with zlib Compression

2012-02-16 Thread Ahmet Inan
On Thu, Feb 16, 2012 at 11:31 AM, Chris Samuel ch...@csamuel.org wrote:
 On Thursday 16 February 2012 19:06:52 Ahmet Inan wrote:

 and got ENOSPC again :(

 so it doesnt matter if lzo or zlib: ENOSPC with compression
 enabled!

 my kernel: vanilla 3.2.5

 I know that there has been an ENOSPC fix go in to the 3.3 RC kernels
 since the 3.2 release, any chance you'd be able to see if it still
 happens with the current 3.3 RC ?

just tested 3.3.0-rc3, and it works like a charm!

but there are lots of issues with external packages not compiling with 3.3,
why i wont be able to deploy it to our systems here yet.

could this one bugfix be backported to 3.2?

here are some results from my unsquashfs'ing just to show how
awesome btrfs with lzo is. these results are _not_ from an ssd btw!
(spinning disks, rebooted between tests)

# time unsquashfs /dev/nbd0
Parallel unsquashfs: Using 8 processors
792630 inodes (870469 blocks) to write

[==/] 870469/870469 100%
created 759645 files
created 67271 directories
created 27250 symlinks
created 5141 devices
created 0 fifos

Timing Results:

zlib:
real6m38.586s
user2m16.021s
sys 0m31.138s

lzo:
real1m53.429s
user1m55.622s
sys 0m46.664s

no compression:
real1m16.975s
user1m47.173s
sys 0m53.520s

complete system installation in under 5minutes, that simply rocks!

Ahmet
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] btrfs/check-integrity.c: included disk-io.h twice

2012-02-16 Thread Danny Kukawka
Hi,

Ignore this one, it's already in linux-next.

Danny

On Donnerstag, 16. Februar 2012, Danny Kukawka wrote:
 fs/btrfs/check-integrity.c included disk-io.h twice, remove the
 duplicate.

 Signed-off-by: Danny Kukawka danny.kuka...@bisect.de
 ---
  fs/btrfs/check-integrity.c |1 -
  1 files changed, 0 insertions(+), 1 deletions(-)

 diff --git a/fs/btrfs/check-integrity.c b/fs/btrfs/check-integrity.c
 index b669a7d..064b29b 100644
 --- a/fs/btrfs/check-integrity.c
 +++ b/fs/btrfs/check-integrity.c
 @@ -89,7 +89,6 @@
  #include disk-io.h
  #include transaction.h
  #include extent_io.h
 -#include disk-io.h
  #include volumes.h
  #include print-tree.h
  #include locking.h


--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: can't read superblock (but could mount)

2012-02-16 Thread Timo Nentwig



On Wed, 15 Feb 2012, Chris Mason wrote:


Date: Wed, 15 Feb 2012 08:23:22 -0500
From: Chris Mason chris.ma...@oracle.com
To: Timo Nentwig bt...@nentwig.biz
Cc: linux-btrfs@vger.kernel.org
Subject: Re: can't read superblock (but could mount)

On Wed, Feb 15, 2012 at 07:03:49AM +0100, Timo Nentwig wrote:

On Tue, 14 Feb 2012, Chris Mason wrote:


Ok, 3.2 shouldn't have done this.  Was this an external drive?  What
else do you have on the system?


Nothing special actually. Standard arch linux with virtualbox kernel modules.
It's a SSD if this should matter. Mounted with ssd,compress=lzo,noatime.


Ok, it sounds like we've got some memory corruption problems in here.
Hopefully not from virtualbox, but I'd start with an memtest.


Wow, now I'm really impressed! :) You are probably right. I overclocked memory
and ran memtest for like 1-2h without errors and had a rock solid system for 
quite a while when I recently started to witness all kinds of random and 
reoccuring crashes. Actually blamed the broken FS rather than to run memtest 
again. Mea culpa.


So, I hit 1 error after a while, somewhat lowered clock freq and no error in 
17h straight. I'll recreate the FS on that box.


Thanks a lot for your support!


Is this the same FS that was corrupted or a different one?


Different one. Created from an old 3.2.1 gentoo install (btrfs 0.19-dirty).


Different FS, same machine?  Please include all the kernel messages.


Yes, same machine, same SSD drive, new mkfs.btrfs.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: NOCOW + compress-force = bug

2012-02-16 Thread Chris Mason
On Thu, Feb 16, 2012 at 07:55:15PM +0600, Roman Mamedov wrote:
 Hello,
 
 Please be aware that there seems to be a possible problem with using NOCOW
 flag on files situated on a filesystem mounted with compress-force(=lzo, in my
 case).
 
 Since experimenting with NOCOW, I started regularly hitting this BUG at
 extent-tree.c:5813 
 
   5813 BUG_ON(!(flags  BTRFS_BLOCK_FLAG_FULL_BACKREF));
 
 I was unable to make netconsole work over a bridged interface, so can only
 post screenshots of this OOPS:
   http://romanrm.ru/pics/2012/2012-02-16-btrfs-bug-1.jpg
   http://romanrm.ru/pics/2012/2012-02-16-btrfs-bug-2.jpg
 
 This happened four times already, and always on snapshot creation (but not
 every case). I have hourly snapshots in crontab, and only one case out of 
 about
 ten fails with this problem. Did not try to deliberately reproduce it yet by
 manually making snapshots very often, etc.

Interesting, NOCOW and compression don't really mix.  We always cow for
compression.  I'll try to reproduce it.

-chris
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: can't read superblock (but could mount)

2012-02-16 Thread Chris Mason
On Thu, Feb 16, 2012 at 06:04:47PM +0100, Timo Nentwig wrote:
 
 
 On Wed, 15 Feb 2012, Chris Mason wrote:
 
 Date: Wed, 15 Feb 2012 08:23:22 -0500
 From: Chris Mason chris.ma...@oracle.com
 To: Timo Nentwig bt...@nentwig.biz
 Cc: linux-btrfs@vger.kernel.org
 Subject: Re: can't read superblock (but could mount)
 
 On Wed, Feb 15, 2012 at 07:03:49AM +0100, Timo Nentwig wrote:
 On Tue, 14 Feb 2012, Chris Mason wrote:
 
 Ok, 3.2 shouldn't have done this.  Was this an external drive?  What
 else do you have on the system?
 
 Nothing special actually. Standard arch linux with virtualbox kernel 
 modules.
 It's a SSD if this should matter. Mounted with ssd,compress=lzo,noatime.
 
 Ok, it sounds like we've got some memory corruption problems in here.
 Hopefully not from virtualbox, but I'd start with an memtest.
 
 Wow, now I'm really impressed! :) You are probably right. I overclocked memory
 and ran memtest for like 1-2h without errors and had a rock solid
 system for quite a while when I recently started to witness all
 kinds of random and reoccuring crashes. Actually blamed the broken
 FS rather than to run memtest again. Mea culpa.
 
 So, I hit 1 error after a while, somewhat lowered clock freq and no
 error in 17h straight. I'll recreate the FS on that box.

Grin, never be impressed when kernel guys blame the memory.  We might as
well be saying its cosmic rays.

Really though, for this particular type of corruption it is usually some
hardware problem, or a memory corruption bug in the kernel.  Thanks for
humoring me and running the memtest.

-chris
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] Cleanup useless lines

2012-02-16 Thread Vincenzo Laurenziello
This patch removes two useless calls of btrfs_header_nritems() in 
balance_level().

Signed-off-by: Vincenzo Laurenziello vin...@gmail.com
Signed-off-by: Josef Bacik jo...@redhat.com
---
--- a/fs/btrfs/ctree.c  2012-02-13 20:17:29.0 +0100
+++ b/fs/btrfs/ctree.c  2012-02-16 19:44:59.830817118 +0100
@@ -966,8 +966,6 @@ static noinline int balance_level(struct
BTRFS_NODEPTRS_PER_BLOCK(root) / 4)
return 0;
 
-   btrfs_header_nritems(mid);
-
left = read_node_slot(root, parent, pslot - 1);
if (left) {
btrfs_tree_lock(left);
@@ -997,7 +995,6 @@ static noinline int balance_level(struct
wret = push_node_left(trans, root, left, mid, 1);
if (wret  0)
ret = wret;
-   btrfs_header_nritems(mid);
}
 
/*
---


--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Problems Implementing Snappy Patches

2012-02-16 Thread Mitch Harder
Just to add another data point, I built Chris Mason's 'snappy' branch
from git.kernel.org as-is (a 3.2.0 kernel plus the snappy patches)
with no additional patches.

This 3.2.0 snappy kernel also exhibits the problem copying the kernel
sources to a snappy-compressed partition.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html