[PATCH v3] xfstests/btrfs: add qgroup rescan stress test

2014-05-09 Thread Wang Shilong
Test flow is to run fsstress after triggering quota rescan. the ruler is simple, we just remove all files and directories, sync filesystem and see if qgroup's ref and excl are nodesize. Signed-off-by: Wang Shilong wangsl.f...@cn.fujitsu.com --- v2-v3: addressed comments from josef: -

Re: [PATCH] xfstests: fix flink test V2

2014-05-09 Thread Christoph Hellwig
Looks good, Reviewed-by: Christoph Hellwig h...@lst.de -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: btrfs snapshot sizes

2014-05-09 Thread David Pottage
On 07/05/14 12:19, Marc MERLIN wrote: So have others found a good way to have an idea about how much space is taken by each snapshot? I've tried quota trees, but I'm not sure how to read the output, or if it's correct (including the negative numbers some have mentioned). Are there other

Re: [PATCH v2] Btrfs: fix hang on error (such as ENOSPC) when writing extent pages

2014-05-09 Thread Liu Bo
On Fri, May 09, 2014 at 03:42:01AM +0100, Filipe David Borba Manana wrote: When running low on available disk space and having several processes doing buffered file IO, I got the following trace in dmesg: [ 4202.720152] INFO: task kworker/u8:1:5450 blocked for more than 120 seconds. [

Re: btrfs issues in 3.14

2014-05-09 Thread Liu Bo
On Thu, May 08, 2014 at 10:51:03AM -0300, Kenny MacDermid wrote: On Wed, May 7, 2014 at 11:48 PM, Liu Bo bo.li@oracle.com wrote: On Wed, May 07, 2014 at 09:35:06AM -0300, Kenny MacDermid wrote: On Tue, May 6, 2014 at 11:22 PM, Liu Bo bo.li@oracle.com wrote: What does

Re: [PATCH v2] Btrfs: fix hang on error (such as ENOSPC) when writing extent pages

2014-05-09 Thread Filipe David Manana
On Fri, May 9, 2014 at 11:52 AM, Liu Bo bo.li@oracle.com wrote: On Fri, May 09, 2014 at 03:42:01AM +0100, Filipe David Borba Manana wrote: When running low on available disk space and having several processes doing buffered file IO, I got the following trace in dmesg: [ 4202.720152] INFO:

re: Btrfs: add optional integrity check code

2014-05-09 Thread Dan Carpenter
Hello Stefan Behrens, The patch 5db0276014b8: Btrfs: add optional integrity check code from Nov 1, 2011, leads to the following static checker warning: fs/btrfs/check-integrity.c:1099 btrfsic_process_metablock() warn: missing error code here? 'btrfsic_stack_frame_alloc()' failed.

Re: Btrfs: add optional integrity check code

2014-05-09 Thread Stefan Behrens
On Fri, 9 May 2014 15:00:07 +0300, Dan Carpenter wrote: Hello Stefan Behrens, The patch 5db0276014b8: Btrfs: add optional integrity check code from Nov 1, 2011, leads to the following static checker warning: fs/btrfs/check-integrity.c:1099 btrfsic_process_metablock() warn:

[PATCH] btrfs: check_int: propagate out-of-memory error upwards

2014-05-09 Thread Stefan Behrens
This issue was not causing any harm but IMO (and in the opinion of the static code checker) it is better to propagate this error status upwards. Signed-off-by: Stefan Behrens sbehr...@giantdisaster.de Reported-by: Dan Carpenter dan.carpen...@oracle.com --- fs/btrfs/check-integrity.c | 5 - 1

Re: [PATCH 1/3] btrfs: return errno instead of -1 from compression

2014-05-09 Thread David Sterba
On Thu, May 08, 2014 at 07:16:17PM -0400, Zach Brown wrote: The compression layer seems to have been built to return -1 and have callers make up errors that make sense. This isn't great because there are different classes of errors that originate down in the compression layer. Allocation

Fwd: Re: URGENT: my laptop's boot ssd btrfs crashed, what do you need off it?

2014-05-09 Thread Marc MERLIN
Howdy, I won't have the time to rebuild my laptop tonight, so I'll wait one more day to see if anyone would like data from that fs to see why it crashed and why btrfs recovery doesn't even seem able to open it. Also I'm not sure if I should risk 3.15rc to rebuild the filesystem and I'd love not

Re: [PATCH 2/3] btrfs: return ptr error from compression workspace

2014-05-09 Thread David Sterba
On Thu, May 08, 2014 at 07:16:18PM -0400, Zach Brown wrote: The btrfs compression wrappers translated errors from workspace allocation to either -ENOMEM or -1. The compression type workspace allocators are already returning a ERR_PTR(-ENOMEM). Just return that and get rid of the magical -1.

Re: [PATCH 3/3] btrfs: fix inline compressed read err corruption

2014-05-09 Thread David Sterba
On Thu, May 08, 2014 at 07:16:19PM -0400, Zach Brown wrote: uncompress_inline() is silently dropping an error from btrfs_decompress() after testing it and zeroing the page that was supposed to hold decompressed data. This can silently turn compressed inline data in to zeros if decompression

Re: btrfs snapshot sizes

2014-05-09 Thread Marc MERLIN
On Fri, May 09, 2014 at 08:42:22AM +0100, David Pottage wrote: On 07/05/14 12:19, Marc MERLIN wrote: So have others found a good way to have an idea about how much space is taken by each snapshot? I've tried quota trees, but I'm not sure how to read the output, or if it's correct (including

Re: Fwd: [suggestion] Add verbose notification about inode-cache rebuild to kernel log

2014-05-09 Thread David Sterba
On Thu, May 08, 2014 at 04:22:13PM +0200, Tomáš Pružina wrote: I ran into some troubles with inode-cache rebuilding on root fs after filesystem was mounted without inode_cache, which stalls boot of my box by several minutes. I boot from commandline like: root=/dev/sda4 rootfstype=btrfs

[PATCH v3] Btrfs: fix hang on error (such as ENOSPC) when writing extent pages

2014-05-09 Thread Filipe David Borba Manana
When running low on available disk space and having several processes doing buffered file IO, I got the following trace in dmesg: [ 4202.720152] INFO: task kworker/u8:1:5450 blocked for more than 120 seconds. [ 4202.720401] Not tainted 3.13.0-fdm-btrfs-next-26+ #1 [ 4202.720596] echo 0

Sending incremental backup causes deadlock (3.14.3)

2014-05-09 Thread Abbas
Has anyone else experienced a deadlock while doing an incremental send / receive? # uname -r 3.14.3-1.el6.elrepo.x86_64 # btrfs version Btrfs v3.14.1 Initial send works well but sending incremental snapshots just HALTS: # btrfs send - -p test/snap-2014-05-09-13:29:04

Re: URGENT: my laptop's boot ssd btrfs crashed, what do you need off it?

2014-05-09 Thread Chris Murphy
On May 9, 2014, at 4:35 AM, Marc MERLIN m...@merlins.org wrote: Howdy, I won't have the time to rebuild my laptop tonight, so I'll wait one more day to see if anyone would like data from that fs to see why it crashed and why btrfs recovery doesn't even seem able to open it. There's some

Re: superfluous else if ()

2014-05-09 Thread David Sterba
On Sat, Apr 26, 2014 at 08:57:00PM +0200, Toralf Förster wrote: /me wonders if this if (ret = 0) { /* Add an item for the type for the first time */ eb = path-nodes[0]; slot = path-slots[0]; offset =

Re: File capabilities are lost when sending/receiving a btrfs subvolume

2014-05-09 Thread David Sterba
On Wed, Apr 30, 2014 at 10:00:29AM -0600, Chris Murphy wrote: On Apr 29, 2014, at 1:21 PM, Juan Orti Alcaine juan.o...@miceliux.com wrote: Hello, I noticed that file capabilites are lost on received subvolumes, so I opened the bug report #68891 [1]. I don't know if other xattrs are

Re: btrfs snapshot sizes

2014-05-09 Thread Josef Bacik
On 05/07/2014 07:19 AM, Marc MERLIN wrote: So have others found a good way to have an idea about how much space is taken by each snapshot? I've tried quota trees, but I'm not sure how to read the output, or if it's correct (including the negative numbers some have mentioned). Are there other

Re: [PATCH v2 1/3] xfstests/btrfs: add qgroup rescan stress test

2014-05-09 Thread Josef Bacik
On 05/08/2014 10:43 PM, Wang Shilong wrote: On 05/09/2014 10:13 AM, Josef Bacik wrote: The inode cache is saved in the FS tree itself for every individual FS tree, that affects the sizes reported by qgroup show so we need to explicitly turn it off to get consistent values. Thanks, Right, so

Re: destroyed disk in btrfs raid

2014-05-09 Thread Hugo Mills
On Fri, May 09, 2014 at 08:02:45PM +0200, laie wrote: Hello! I've some trouble with my btrf filesystem. I've lost one backing raid device, it's luks header is overwritten and not restoreable. The lost disk was recently added. 'btrfs filesystem balace' was running for some time, but the

Re: destroyed disk in btrfs raid

2014-05-09 Thread Hugo Mills
On Fri, May 09, 2014 at 06:58:27PM +0100, Hugo Mills wrote: On Fri, May 09, 2014 at 08:02:45PM +0200, laie wrote: Now I'm looking for a way to tell btrfs to provide me with a list of the corrupted files and delete them afterwards. This would be great, because otherwise it would take very

Re: Fwd: [suggestion] Add verbose notification about inode-cache rebuild to kernel log

2014-05-09 Thread Goffredo Baroncelli
On 05/09/2014 04:51 PM, David Sterba wrote: On Thu, May 08, 2014 at 04:22:13PM +0200, Tomáš Pružina wrote: I ran into some troubles with inode-cache rebuilding on root fs after filesystem was mounted without inode_cache, which stalls boot of my box by several minutes. I boot from commandline

[PATCH] xfstests: filter out selinux xattrs for generic/062

2014-05-09 Thread Josef Bacik
If you have selinux enabled getfattr will show the selinux xattrs, which screws with the golden output of generic/062. To make matters worse you can't just greap it out because we'll still get the preamble and newline from getfattr when the selinux attr is the only attr. So this is the voodoo I

Re: [PATCH] xfstests: filter out selinux xattrs for generic/062

2014-05-09 Thread Josef Bacik
On 05/09/2014 04:17 PM, Eric Sandeen wrote: On 5/9/14, 2:44 PM, Josef Bacik wrote: If you have selinux enabled getfattr will show the selinux xattrs, which screws with the golden output of generic/062. To make matters worse you can't just greap it out because we'll still get the preamble and

Re: [PATCH] xfstests: filter out selinux xattrs for generic/062

2014-05-09 Thread Eric Sandeen
On 5/9/14, 3:20 PM, Josef Bacik wrote: I have MOUNT_OPTIONS set to something else so I lose the context bit. Thanks, Seems like we need to add to them, then, not overwrite them, for selinux? Otherwise I imagine many other tests will fail. -Eric -- To unsubscribe from this list: send the line

Re: [PATCH 3/3] btrfs: fix inline compressed read err corruption

2014-05-09 Thread Zach Brown
On Fri, May 09, 2014 at 03:58:00PM +0200, David Sterba wrote: On Thu, May 08, 2014 at 07:16:19PM -0400, Zach Brown wrote: uncompress_inline() is silently dropping an error from btrfs_decompress() after testing it and zeroing the page that was supposed to hold decompressed data. This can

[PATCH] xfstests: fix selinux context handling

2014-05-09 Thread Josef Bacik
With the new config stuff we lost the selinux options being set for systems with selinux turned on. We want the selinux context set all the time, wether we provide a MOUNT_OPTIONS value or not, so take this logic out of _mount_opts() and just put it in the body of common/config Signed-off-by:

Re: [PATCH] Btrfs: add a extent ref verify tool V2

2014-05-09 Thread Josef Bacik
On 05/08/2014 07:34 PM, Zach Brown wrote: +#ifdef CONFIG_BTRFS_FS_REF_VERIFY +int btrfs_build_ref_tree(struct btrfs_fs_info *fs_info); +void btrfs_free_ref_cache(struct btrfs_fs_info *fs_info); +int btrfs_ref_tree_mod(struct btrfs_root *root, u64 bytenr, u64 num_bytes, + u64

[PATCH] Btrfs: add a extent ref verify tool V3

2014-05-09 Thread Josef Bacik
We were having corruption issues that were tied back to problems with the extent tree. In order to track them down I built this tool to try and find the culprit, which was pretty successful. If you compile with this tool on it will live verify every ref update that the fs makes and make sure it

Re: [PATCH] Btrfs: add a extent ref verify tool V2

2014-05-09 Thread Zach Brown
On Fri, May 09, 2014 at 04:45:05PM -0400, Josef Bacik wrote: On 05/08/2014 07:34 PM, Zach Brown wrote: +#ifdef CONFIG_BTRFS_FS_REF_VERIFY +int btrfs_build_ref_tree(struct btrfs_fs_info *fs_info); +void btrfs_free_ref_cache(struct btrfs_fs_info *fs_info); +int btrfs_ref_tree_mod(struct

Re: [PATCH] xfstests: fix selinux context handling

2014-05-09 Thread Eric Sandeen
On 5/9/14, 3:40 PM, Josef Bacik wrote: With the new config stuff we lost the selinux options being set for systems with selinux turned on. We want the selinux context set all the time, wether we provide a MOUNT_OPTIONS value or not, so take this logic out of _mount_opts() and just put it in

[PATCH v2 1/3] btrfs: return errno instead of -1 from compression

2014-05-09 Thread Zach Brown
The compression layer seems to have been built to return -1 and have callers make up errors that make sense. This isn't great because there are different errors that originate down in the compression layer. Let's return real negative errnos from the compression layer so that callers can pass on

[PATCH v2 3/3] btrfs: fix inline compressed read err corruption

2014-05-09 Thread Zach Brown
uncompress_inline() is dropping the error from btrfs_decompress() after testing it and zeroing the page that was supposed to hold decompressed data. This can silently turn compressed inline data in to zeros if decompression fails due to corrupt compressed data or memory allocation failure. I

Re: [PATCH 1/3] btrfs: return errno instead of -1 from compression

2014-05-09 Thread Zach Brown
On Fri, May 09, 2014 at 03:39:26PM +0200, David Sterba wrote: On Thu, May 08, 2014 at 07:16:17PM -0400, Zach Brown wrote: The compression layer seems to have been built to return -1 and have callers make up errors that make sense. This isn't great because there are different classes of

Re: btrfs cleaner failure - fs/btrfs/extent-tree.c:5748 (3.14.0)

2014-05-09 Thread Marc MERLIN
Ok, first for the devs, I found the real trace that happened just before the system went read only My apologies for pasting the bad one first. I'll wipe/rebuild the FS tonight unless you ask me to wait for one more day and/or data off it. Please advise if I should rebuilt with 3.14.3 or

Re: btrfs cleaner failure - fs/btrfs/extent-tree.c:5748 (3.14.0)

2014-05-09 Thread Chris Murphy
On May 9, 2014, at 4:36 PM, Marc MERLIN m...@merlins.org wrote: Details: It looks like my corruption came from there. I'm still not sure why it's apparently so severe that btrfs recovery cannot open the FS now. WARNING: CPU: 6 PID: 555 at fs/btrfs/extent-tree.c:5748

Re: btrfs cleaner failure - fs/btrfs/extent-tree.c:5748 (3.14.0)

2014-05-09 Thread Chris Samuel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Marc, On Fri, 9 May 2014 03:36:59 PM Marc MERLIN wrote: Oh, I missed that. May 2 14:23:06 legolas kernel: [283268.319035] CPU: 0 PID: 25726 Comm: watchdog/0 Tainted: GW3.14.0-amd64-i915-preempt-20140216 #2 This is weird because I

Re: btrfs cleaner failure - fs/btrfs/extent-tree.c:5748 (3.14.0)

2014-05-09 Thread Marc MERLIN
On Sat, May 10, 2014 at 10:13:43AM +1000, Chris Samuel wrote: Right now, I do see: legolas:~# cat /proc/sys/kernel/tainted 512 IIUC that's an array of bit flags, and that value means you've had a previous kernel warning at that point according to:

Re: btrfs cleaner failure - fs/btrfs/extent-tree.c:5748 (3.14.0)

2014-05-09 Thread Hugo Mills
On Fri, May 09, 2014 at 05:42:54PM -0700, Marc MERLIN wrote: On Sat, May 10, 2014 at 10:13:43AM +1000, Chris Samuel wrote: Right now, I do see: legolas:~# cat /proc/sys/kernel/tainted 512 IIUC that's an array of bit flags, and that value means you've had a previous kernel

Re: btrfs cleaner failure - fs/btrfs/extent-tree.c:5748 (3.14.0)

2014-05-09 Thread Hugo Mills
On Fri, May 09, 2014 at 06:00:50PM -0600, Chris Murphy wrote: Well I'm sorta dense, so I only find a complete dmesg useful because with storage problems it seems much is due to some other problem happening earlier. Life would be so much easier if filesystems didn't store any persistent

Re: btrfs cleaner failure - fs/btrfs/extent-tree.c:5748 (3.14.0)

2014-05-09 Thread Chris Murphy
On May 9, 2014, at 7:05 PM, Hugo Mills h...@carfax.org.uk wrote: On Fri, May 09, 2014 at 05:42:54PM -0700, Marc MERLIN wrote: On Sat, May 10, 2014 at 10:13:43AM +1000, Chris Samuel wrote: Right now, I do see: legolas:~# cat /proc/sys/kernel/tainted 512 IIUC that's an array of bit flags,

Re: btrfs cleaner failure - fs/btrfs/extent-tree.c:5748 (3.14.0)

2014-05-09 Thread Duncan
Hugo Mills posted on Sat, 10 May 2014 02:09:02 +0100 as excerpted: Life would be so much easier if filesystems didn't store any persistent state... :) The number of people who don't quite get that that's the function and natural behaviour of a filesystem is... surprising. As in, Your