Re: [PATCH 0/4] [RFC] btrfs: offline dedupe

2013-04-22 Thread Gabriel de Perthuis
On Sat, Apr 20, 2013 at 05:49:25PM +0200, Gabriel de Perthuis wrote: Hi, The following series of patches implements in btrfs an ioctl to do offline deduplication of file extents. I am a fan of this patch, the API is just right. I just have a few tweaks to suggest to the argument checking.

Re: [PATCH 1/2] btrfs-progs: set generation_v2 any time we write a new root

2013-04-22 Thread Anand Jain
yeah we should set the v2 parameter at all the place where we call btrfs_set_root_generation. Sorry it slipped my mind. Thanks for the fix. Thanks, Anand On 04/22/2013 01:01 PM, Eric Sandeen wrote: With this integration branch commit in place: 2bd1169 btrfs-progs: root_item

Re: [PATCH 2/2 V2] btrfs-progs: update generation_v2 in btrfs_update_root

2013-04-22 Thread Anand Jain
/* - * If the btrfs-progs is newer and kernel is at - * generation_v1 then we don't touch v2 items - * otherwise when kernel is at same or greater - * version compared with btrfs-progs then update - * the needed - */ - old_size =

[PATCH] Btrfs-progs: btrfs-crc: support specifying checksum in hex

2013-04-22 Thread Stefan Behrens
Signed-off-by: Stefan Behrens sbehr...@giantdisaster.de --- btrfs-crc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/btrfs-crc.c b/btrfs-crc.c index 6e21a0e..e4cda43 100644 --- a/btrfs-crc.c +++ b/btrfs-crc.c @@ -49,7 +49,7 @@ int main(int argc, char **argv)

[PATCH] Btrfs: return free space in cow error path

2013-04-22 Thread Liu Bo
Replace some BUG_ONs with proper handling and take allocated space back to free space cache for later use. We don't have to worry about extent maps since they'd be freed in releasepage path. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/inode.c | 12 +--- 1 files changed, 9

[PATCH] Btrfs: fix possible infinite loop in slow caching

2013-04-22 Thread Josef Bacik
So I noticed there is an infinite loop in the slow caching code. If we return 1 when we hit the end of the tree, so we could end up caching the last block group the slow way and suddenly we're looping forever because we just keep re-searching and trying again. Fix this by only doing

[PATCH] Btrfs: use REQ_META for all metadata IO

2013-04-22 Thread Josef Bacik
We need to tag metadata io with REQ_META to avoid priority inversion when using io throttling cqroups. Thanks, Signed-off-by: Josef Bacik jba...@fusionio.com --- fs/btrfs/extent_io.c | 18 ++ 1 files changed, 10 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/extent_io.c

[PATCH] Btrfs: deal with bad mappings in btrfs_map_block

2013-04-22 Thread Josef Bacik
Martin Steigerwald reported a BUG_ON() in btrfs_map_block where we didn't find a chunk for a particular block we were trying to map. This happened because the block was bogus. We shouldn't be BUG_ON()'ing in this case, just print a message and return an error. This came from reada_add_block and

[PATCH] Btrfs: don't call readahead hook until we have read the entire eb

2013-04-22 Thread Josef Bacik
Martin Steigerwald reported a BUG_ON() where we were given a bogus bytenr to map. Turns out he is using PAGESIZE leafsizes. The readahead stuff is called every time we do a completion, but we may not have finished reading in all the pages, so the bytenr we read off the node could be completely

BTRFS 3.8.7 Kernel Crash Report‏

2013-04-22 Thread Mark Ridley
Hi, I have been having a kernel crash since kernel 3.3.2 and hoped it had been fixed by the time 3.8.7 (FC18) but no joy. I take backups of large image files (around 100GB) and btrfs snapshot them. If I then use rsync --inplace to update the images, I get a btrfs stack trace and btrfs

Re: data DUP

2013-04-22 Thread David Sterba
On Sat, Apr 20, 2013 at 01:17:06PM -0700, Roger Binns wrote: Is there any particular reason why I can't use DUP for data? IIRC separate DUP for metadata and data is not possible due to some technical reason (I don't remember nor can find the mail where this was mentioned). DUP data is possible

Re: BTRFS 3.8.7 Kernel Crash Report‏

2013-04-22 Thread David Sterba
On Mon, Apr 22, 2013 at 01:19:41PM +, Mark Ridley wrote: If I then use rsync --inplace to update the images, I get a btrfs stack trace and btrfs hangs: This happens every night. [a00bd5c1] btrfs_set_item_key_safe+0x141/0x150 [btrfs] The check fails because it finds keys in

Re: [PATCH V3] xfstests: fix common filter include in filter.btrfs

2013-04-22 Thread Rich Johnston
On 04/21/2013 05:51 PM, Eric Sandeen wrote: Not sure how this happened, but filter.btrfs including itself leads to immense sadness for any file that includes it. (I got a segfault when I tried to run 307) It should be including ./common/filter not ./common/filter.btrfs Signed-off-by: Eric

Re: btrfs scrub gives unable to find logical $hugenum len 16384

2013-04-22 Thread Martin Steigerwald
Am Samstag, 20. April 2013 schrieb Josef Bacik: So I found your bug on the plane ride, as soon as I get home I'll email it. Thanks, Did you get home yet? I would like to know the impact of the bug. I am running from a restauration of a backup via rsync which went without any errors, but

Re: btrfs scrub gives unable to find logical $hugenum len 16384

2013-04-22 Thread Martin Steigerwald
Am Montag, 22. April 2013 schrieb Martin Steigerwald: Am Samstag, 20. April 2013 schrieb Josef Bacik: So I found your bug on the plane ride, as soon as I get home I'll email it. Thanks, Did you get home yet? I would like to know the impact of the bug. I am running from a restauration

Re: BTRFS 3.8.7 Kernel Crash Report

2013-04-22 Thread Harald Glatt
On Mon, Apr 22, 2013 at 4:01 PM, Mark Ridley m...@backupsystems.co.uk wrote: Thanks, David. What causes this corruption and how can I fix it? I'm very worried about running btrfs.fsck as last time it made a slight corruption like this worse and the whole volume had to be trashed. After

Re: BTRFS 3.8.7 Kernel Crash Report

2013-04-22 Thread Mark Ridley
No. Without --repair pointed out some problems, but whats the point of knowing about the problems if they can't be fixed so I ran with --repair and it broke the volume. On 22/04/2013 15:02, Harald Glatt m...@hachre.de wrote: On Mon, Apr 22, 2013 at 4:01 PM, Mark Ridley m...@backupsystems.co.uk

Re: BTRFS 3.8.7 Kernel Crash Report

2013-04-22 Thread Harald Glatt
Yeah, --repair is not recommended as of now. Maybe posting the btrfsck output just from checking would be helpful in this case. On Mon, Apr 22, 2013 at 4:04 PM, Mark Ridley m...@backupsystems.co.uk wrote: No. Without --repair pointed out some problems, but whats the point of knowing about the

Re: BTRFS 3.8.7 Kernel Crash Report

2013-04-22 Thread Mark Ridley
What does btrfs scrub do? Is that meant to detect and fix problems? Thanks, Mark On 22/04/2013 15:13, Harald Glatt m...@hachre.de wrote: Yeah, --repair is not recommended as of now. Maybe posting the btrfsck output just from checking would be helpful in this case. On Mon, Apr 22, 2013 at

Re: BTRFS 3.8.7 Kernel Crash Report

2013-04-22 Thread Harald Glatt
Only data errors (from CRC checks), maybe also some structure errors - I'm not sure. A remount should fix all errors. If it doesn't I think it's considered a bug - ultimately the goal is that no --repair should ever be required... Scrub is generally safe though, unlike btrfsck --repair. On Mon,

Re: [PATCH] Btrfs: deal with bad mappings in btrfs_map_block

2013-04-22 Thread David Sterba
On Mon, Apr 22, 2013 at 09:14:11AM -0400, Josef Bacik wrote: Martin Steigerwald reported a BUG_ON() in btrfs_map_block where we didn't find a chunk for a particular block we were trying to map. This happened because the block was bogus. We shouldn't be BUG_ON()'ing in this case, just print

Re: [PATCH] xfstests: remove recursive include in filter.btrfs

2013-04-22 Thread Rich Johnston
Thanks for the patch Eric, V3 has been committed. --Rich commit e771c54c45a5c475bedbc5a2d6becc05be940dba Author: Eric Sandeen sand...@sandeen.net Date: Sun Apr 21 22:51:54 2013 + xfstests: fix common filter include in filter.btrfs -- To unsubscribe from this list: send the line

Re: BTRFS 3.8.7 Kernel Crash Report

2013-04-22 Thread Hugo Mills
On Mon, Apr 22, 2013 at 03:40:00PM +0100, Mark Ridley wrote: Any idea when the 3.9 kernel will be out. Most likely next Sunday/Monday, unless something drastic shows up this week (or there are too many fixes going in). I see there are lots of btrfs fixes going into it. I'm currently

Re: BTRFS 3.8.7 Kernel Crash Report

2013-04-22 Thread Harald Glatt
I heard it's coming out next week. On Mon, Apr 22, 2013 at 4:40 PM, Mark Ridley m...@backupsystems.co.uk wrote: Any idea when the 3.9 kernel will be out. I see there are lots of btrfs fixes going into it. I'm currently running FC18 on 3.8.7, although 10 minutes ago it got updated to 3.8.8.

Re: BTRFS 3.8.7 Kernel Crash Report

2013-04-22 Thread Mark Ridley
For me anyway, I hope the fedora team don't take too long to make it available. Thanks for all your help today. On 22/04/2013 15:41, Harald Glatt m...@hachre.de wrote: I heard it's coming out next week. On Mon, Apr 22, 2013 at 4:40 PM, Mark Ridley m...@backupsystems.co.uk wrote: Any idea when

Re: BTRFS 3.8.7 Kernel Crash Report

2013-04-22 Thread Harald Glatt
I think it won't... That would just be the goal eventually. If scrub sees no errors all the data should be in tact and your best bet to get things working perfectly again is to create a new filesystem and transfer the data into it. I'm not a dev and I can't say if a btrfs-image for debugging

Re: [PATCH] btrfs: make static code static remove dead code

2013-04-22 Thread David Sterba
On Fri, Apr 19, 2013 at 12:21:22PM -0700, Eric Sandeen wrote: Big patch, but all it does is add statics to functions which are in fact static, then remove the associated dead-code fallout. I support removing dead code, though nearly all of the functions make me wonder why they're not used. A

Re: [PATCH V2] btrfs: move leak debug code to functions

2013-04-22 Thread David Sterba
On Mon, Apr 22, 2013 at 12:18:37AM -0500, Eric Sandeen wrote: There was a time when sprinkling #ifdefs around was bad form. We can clean up the code a bit by moving the leak-list adds/removes into functions and make things more readable. Signed-off-by: Eric Sandeen sand...@redhat.com ---

Re: [PATCH] btrfs: make static code static remove dead code

2013-04-22 Thread Eric Sandeen
On 4/22/13 10:05 AM, David Sterba wrote: On Fri, Apr 19, 2013 at 12:21:22PM -0700, Eric Sandeen wrote: Big patch, but all it does is add statics to functions which are in fact static, then remove the associated dead-code fallout. I support removing dead code, though nearly all of the

Re: btrfs scrub gives unable to find logical $hugenum len 16384

2013-04-22 Thread Dan van der Ster
Quick question, do you agree that the traceback in the P.S. is the same BUG? I hit this during a scrub yesterday (with 3.8.7), though re-scrubbing succeeded. Cheers, Dan Apr 21 09:31:44 dvanders-webserver kernel: [505979.695932] btrfs: unable to find logical 8653227934915758829 len 16384 Apr 21

Re: [PATCH V2] btrfs: move leak debug code to functions

2013-04-22 Thread Eric Sandeen
On 4/22/13 10:22 AM, David Sterba wrote: On Mon, Apr 22, 2013 at 12:18:37AM -0500, Eric Sandeen wrote: There was a time when sprinkling #ifdefs around was bad form. We can clean up the code a bit by moving the leak-list adds/removes into functions and make things more readable.

Re: [PATCH V2] btrfs: move leak debug code to functions

2013-04-22 Thread David Sterba
On Mon, Apr 22, 2013 at 10:27:11AM -0500, Eric Sandeen wrote: Yep I noticed that too, I'd be happy to elevate it to a CONFIG_ option if it's really something that will be kept around for a while. We have (and will have) more code for debugging aid. Could maybe be lumped in to a

Re: [PATCH] btrfs: make static code static remove dead code

2013-04-22 Thread David Sterba
On Mon, Apr 22, 2013 at 10:24:12AM -0500, Eric Sandeen wrote: btrfs_reada_detach() That's an API for readahead, thhugh maybe not used now as RA is not used and at all scenarios where it could. Ok, if it's useful to keep it around just for symmetry I could understand that. reada.c:

Re: [PATCH] btrfs: make static code static remove dead code

2013-04-22 Thread Eric Sandeen
On 4/22/13 10:55 AM, David Sterba wrote: On Mon, Apr 22, 2013 at 10:24:12AM -0500, Eric Sandeen wrote: btrfs_reada_detach() That's an API for readahead, thhugh maybe not used now as RA is not used and at all scenarios where it could. Ok, if it's useful to keep it around just for symmetry I

3.8.8 fc18 kernel crash when editing a file

2013-04-22 Thread Mark Ridley
kernel: [13453.678952] [ cut here ] kernel: [13453 .679136] kernel BUG at fs/btrfs/file.c:861! kernel: [13453.679380] invalid opcode: [#1] SMP kernel: [13453.679612] Modules linked in: coretemp shpchp ppdev vmw_balloon i2c_piix4 microcode vmxnet3 parport_pc parport

Re: 3.8.8 kernel crash while editing a file

2013-04-22 Thread Josef Bacik
On Mon, Apr 22, 2013 at 01:33:12PM -0600, Mark Ridley wrote: Hi, This stack trace came out earlier today when I edited a file in perl that had a btrfs snapshot. Any idea why? or what I can do to get you more info. 3.8.8-202.fc18.x86_64 The device has 21TB of used data on it. Can

[PATCH] xfstests 311: test fsync with dm flakey

2013-04-22 Thread Josef Bacik
This test sets up a dm flakey target and then runs my fsync tester I've been using to verify btrfs's fsync() is working properly. It will create a dm flakey device, mount it, run my test, make the flakey device start dropping writes, and then unmount the fs. Then we mount it back up and make

Re: [PATCH] xfstests 311: test fsync with dm flakey

2013-04-22 Thread Zach Brown
+static void drop_all_caches() +{ + int value = 3; + int fd; + + if ((fd = open(/proc/sys/vm/drop_caches, O_WRONLY)) 0) { + fprintf(stderr, Error opening drop caches: %d\n, errno); + return; + } + + write(fd, value, sizeof(int)); +