Re: [PATCH 2/2] Btrfs: fix use-after-free in the finishing procedure of the device replace

2014-02-07 Thread Stefan Behrens
On Thu, 30 Jan 2014 16:46:55 +0800, Miao Xie wrote: During device replace test, we hit a null pointer deference (It was very easy to reproduce it by running xfstests' btrfs/011 on the devices with the virtio scsi driver). There were two bugs that caused this problem: - We might allocate new

Re: [PATCH 2/2] btrfs-progs: Add -p/--print-missing options for btrfs fi show

2014-02-07 Thread Anand Jain
Whats needed is more comprehensive btrfs fi show which shows the flags (including missing) per disk. And also show the FS/Raid status. Which I am working on. sorry -p feature would be covered by default in the coming revamp of btrfs fi show. Thanks, Anand On 02/07/2014 02:46 PM, Qu

Re: [PATCH 1/2] btrfs-progs: Add missing devices check for mounted btrfs.

2014-02-07 Thread Anand Jain
IMO btrfs-progs shouldn't add its intelligence to know if disk is missing. If btrfs-kernel doesn't know when disk is missing that's a bug to fix in btrfs-kernel. yes that indeed true as of now in btrfs-kernel. btrfs kernel has no idea when disk goes missing, just -EIO doesn't tell btrfs

Re: [PATCH 2/2] Btrfs: fix use-after-free in the finishing procedure of the device replace

2014-02-07 Thread Miao Xie
On Fri, 07 Feb 2014 09:14:42 +0100, Stefan Behrens wrote: On Thu, 30 Jan 2014 16:46:55 +0800, Miao Xie wrote: During device replace test, we hit a null pointer deference (It was very easy to reproduce it by running xfstests' btrfs/011 on the devices with the virtio scsi driver). There were two

Re: [PATCH 2/2] Btrfs: fix use-after-free in the finishing procedure of the device replace

2014-02-07 Thread Stefan Behrens
On Fri, 07 Feb 2014 17:17:45 +0800, Miao Xie wrote: On Fri, 07 Feb 2014 09:14:42 +0100, Stefan Behrens wrote: On Thu, 30 Jan 2014 16:46:55 +0800, Miao Xie wrote: During device replace test, we hit a null pointer deference (It was very easy to reproduce it by running xfstests' btrfs/011 on

Re: [PATCH] btrfs: introduce BTRFS_IOC_GET_DEVS

2014-02-07 Thread Anand Jain
Thanks for the comments. mainly here sysfs way defeats the purpose - debug as mentioned. Sysfs would/should show only mounted disks, the ioctl way doesn't have such a limitation (of course as I commented memory dump way would have been best choice but I got stuck with that approach if

Re: Provide a better free space estimate on RAID1

2014-02-07 Thread Martin Steigerwald
Am Donnerstag, 6. Februar 2014, 22:30:46 schrieb Chris Murphy: On Feb 6, 2014, at 9:40 PM, Roman Mamedov r...@romanrm.net wrote: On Thu, 06 Feb 2014 20:54:19 +0100 Goffredo Baroncelli kreij...@libero.it wrote: I agree with you about the needing of a solution. However your patch to

Re: [btrfs] BUG: unable to handle kernel NULL pointer dereference at 0000000000000038

2014-02-07 Thread David Rientjes
On Fri, 7 Feb 2014, Fengguang Wu wrote: [1.625020] BTRFS: selftest: Running btrfs_split_item tests [1.627004] BTRFS: selftest: Running find delalloc tests [2.289182] tsc: Refined TSC clocksource calibration: 2299.967 MHz [ 292.084537] kthreadd invoked oom-killer:

Re: [PATCH v2 3/3] btrfs/035: add new clone overwrite regression test

2014-02-07 Thread David Disseldorp
On Fri, 7 Feb 2014 09:45:09 +1100, Dave Chinner wrote: Not exactly what I intended. If echo fails, it will output some kind of error message, and that will cause the golden image mismatch. Fair enough. I'll resend with the two ' || echo failed to create src' checks removed. Thanks again

[PATCH v3 3/3] btrfs/035: add new clone overwrite regression test

2014-02-07 Thread David Disseldorp
This test uses the newly added cloner binary to dispatch full file and range specific clone (reflink) requests. Signed-off-by: David Disseldorp dd...@suse.de --- tests/btrfs/035 | 77 + tests/btrfs/035.out | 3 +++ tests/btrfs/group | 1

[PATCH v3 0/3] __btrfs_drop_extents() BUG_ON reproducer

2014-02-07 Thread David Disseldorp
This patch-set provides a reproducer for hitting the 3.14.0-rc1 BUG_ON() at: 692 int __btrfs_drop_extents(struct btrfs_trans_handle *trans, ... 839 /* 840 * | range to drop - | 841 * | extent | 842

[PATCH v3 1/3] btrfs: add small program for clone testing

2014-02-07 Thread David Disseldorp
The cloner program is capable of cloning files using the BTRFS_IOC_CLONE and BTRFS_IOC_CLONE_RANGE ioctls. Signed-off-by: David Disseldorp dd...@suse.de --- .gitignore | 1 + src/Makefile | 2 +- src/cloner.c | 188 +++ 3 files

Re: [PATCH v3] Btrfs: add regression test for running snapshot and send concurrently

2014-02-07 Thread Wang Shilong
Hi Dave, On Fri, Feb 07, 2014 at 12:18:31PM +0800, Wang Shilong wrote: Hi Dave, On Fri, Feb 07, 2014 at 12:10:08AM +0800, Wang Shilong wrote: +$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT \ + $SCRATCH_MNT/snap_1 $seqres.full 21 + +do_snapshots +snapshots_pid=$! +

Re: lost with degraded RAID1

2014-02-07 Thread Johan Kröckel
Is there anything else I should do with this setup or may I nuke the two partitions and reuse them? 2014-02-03 Johan Kröckel johan.kroec...@gmail.com: State is: I wont use this filesystem again. I have a backup. So I am interested to give the necessary information for debuging it and

[PATCH v3 2/3] src/cloner: use btrfs/ioctl.h header if present

2014-02-07 Thread David Disseldorp
Check for the btrfsprogs-devel ioctl.h header at configure time. Use it in src/cloner if present, otherwise fall back to using the copied clone ioctl definitions. Signed-off-by: David Disseldorp dd...@suse.de --- configure.ac | 1 + src/cloner.c | 4 2 files changed, 5 insertions(+) diff

Re: [btrfs] BUG: unable to handle kernel NULL pointer dereference at 0000000000000038

2014-02-07 Thread Fengguang Wu
On Fri, Feb 07, 2014 at 02:13:59AM -0800, David Rientjes wrote: On Fri, 7 Feb 2014, Fengguang Wu wrote: [1.625020] BTRFS: selftest: Running btrfs_split_item tests [1.627004] BTRFS: selftest: Running find delalloc tests [2.289182] tsc: Refined TSC clocksource calibration:

Can i trust you?

2014-02-07 Thread ivan . zoric
I am Mrs. Aisha Al Mutasim from England, I have a business proposal worth of $10.6M US Dollars get back for more info. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at

[PATCH 2/4] btrfs: reserve no transaction units in btrfs_ioctl_set_features

2014-02-07 Thread David Sterba
Added in patch btrfs: add ioctls to query/change feature bits online modifications to superblock don't need to reserve metadata blocks when starting a transaction. Signed-off-by: David Sterba dste...@suse.cz --- fs/btrfs/ioctl.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git

[PATCH 3/4][RFC] btrfs: export global block reserve size as space_info

2014-02-07 Thread David Sterba
Introduce a block group type bit for a global reserve and fill the space info for SPACE_INFO ioctl. This should replace the newly added ioctl (01e219e8069516cdb98594d417b8bb8d906ed30d) to get just the 'size' part of the global reserve, while the actual usage can be now visible in the 'btrfs fi df'

[PATCH 1/4] btrfs: commit transaction after setting label and features

2014-02-07 Thread David Sterba
From: Jeff Mahoney je...@suse.com The set_fslabel ioctl uses btrfs_end_transaction, which means it's possible that the change will be lost if the system crashes, same for the newly set features. Let's use btrfs_commit_transaction instead. Signed-off-by: Jeff Mahoney je...@suse.com Signed-off-by:

[PATCH 4/4] Revert btrfs: add ioctl to export size of global metadata reservation

2014-02-07 Thread David Sterba
This reverts commit 01e219e8069516cdb98594d417b8bb8d906ed30d. The information from the removed ioctl is newly provided by the existing SPACE_INFO ioctl where it logically belongs and needs only minor changes to the userspace code. The GLOBAL_RSV ioctl hasn't been used anywhere yet, introduced

[PATCH 0/4] Btrfs updates for 3.14-rc - sysfs, ioctl

2014-02-07 Thread David Sterba
Two minor fixes to the new sysfs code and a RFC change to replace the GLOBAL_RSV ioctl with the same information exported through SPACE_INFO ioctl. This requires to reserve a blockgroup type that will become part of ABI. David Sterba (3): btrfs: reserve no transaction units in

Re: Provide a better free space estimate on RAID1

2014-02-07 Thread Frank Kingswood
On 06/02/14 19:54, Goffredo Baroncelli wrote: Hi Roman On 02/06/2014 01:45 PM, Roman Mamedov wrote: There's not a lot of code to include (as my 3-line patch demonstrates), it could just as easily be removed when it's obsolete. But I did not have any high hopes of defeating the broken by design

[PATCH v4] xfstests/btrfs: add a regression test for running snapshot and send concurrently

2014-02-07 Thread Wang Shilong
From: Wang Shilong wangsl.f...@cn.fujitsu.com Btrfs would fail to send if snapshot run concurrently, this test is to make sure we have fixed the bug. Signed-off-by: Wang Shilong wangsl.f...@cn.fujitsu.com --- v3-v4: avoid to use silent send command(thanks to Dave Chinner) rename patch's

Re: [btrfs] BUG: unable to handle kernel NULL pointer dereference at 0000000000000038

2014-02-07 Thread Chris Mason
On Fri 07 Feb 2014 07:10:38 AM EST, Fengguang Wu wrote: On Fri, Feb 07, 2014 at 02:13:59AM -0800, David Rientjes wrote: On Fri, 7 Feb 2014, Fengguang Wu wrote: [1.625020] BTRFS: selftest: Running btrfs_split_item tests [1.627004] BTRFS: selftest: Running find delalloc tests [

[PATCH] btrfs-progs: Change BUG() to use assert.

2014-02-07 Thread Mitch Harder
Change the definition of BUG() to use assert instead of abort to provide information about the location of the issue. Signed-off-by: Mitch Harder mitch.har...@sabayonlinux.org --- kerncompat.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kerncompat.h b/kerncompat.h index

Re: [btrfs] BUG: unable to handle kernel NULL pointer dereference at 0000000000000038

2014-02-07 Thread Filipe David Manana
On Fri, Feb 7, 2014 at 3:10 PM, Chris Mason c...@fb.com wrote: On Fri 07 Feb 2014 07:10:38 AM EST, Fengguang Wu wrote: On Fri, Feb 07, 2014 at 02:13:59AM -0800, David Rientjes wrote: On Fri, 7 Feb 2014, Fengguang Wu wrote: [1.625020] BTRFS: selftest: Running btrfs_split_item tests [

[PATCH] btrfs-progs: add dry-run option to restore command

2014-02-07 Thread Justin Maggard
Sometimes it is useful to see what btrfs restore is going to do before provisioning enough external storage to restore onto. Add a dry-run option so we can see what files and paths are found by restore, without actually restoring any data. --- cmds-restore.c | 14 -- 1 file changed,

[PATCH] xfstests: Btrfs: add test for large metadata blocks

2014-02-07 Thread Koen De Wit
Tests Btrfs filesystems with all possible metadata block sizes, by setting large extended attributes on files. Signed-off-by: Koen De Wit koen.de@oracle.com --- tests/btrfs/036 | 128 +++ tests/btrfs/036.out |7 +++ tests/btrfs/group

[PATCH] Btrfs: unlock extent and pages on error in cow_file_range

2014-02-07 Thread Josef Bacik
When I converted the BUG_ON() for the free_space_cache_inode in cow_file_range I made it so we just return an error instead of unlocking all of our various stuff. This is a mistake and causes us to hang when we run into this. This patch fixes this problem. Thanks, Signed-off-by: Josef Bacik

Re: integration-20140206 build failure

2014-02-07 Thread WorMzy Tykashi
failure from integration-20140207: [CC] cmds-filesystem.o cmds-filesystem.c: In function 'handle_print': cmds-filesystem.c:434:5: error: too few arguments to function 'print_one_fs' space_info_arg, label, mnt); ^ cmds-filesystem.c:361:12: note: declared here static int print_one_fs

Re: Provide a better free space estimate on RAID1

2014-02-07 Thread Chris Murphy
On Feb 6, 2014, at 11:08 PM, Roman Mamedov r...@romanrm.net wrote: And what if I am accessing that partition on a server via a network CIFS/NFS share and don't even *have a way to find out* any of that. That's the strongest argument. And if the user is using Explorer/Finder/Nautilus to

[PATCH] btrfs-progs: Preserve process_one_leaf return value.

2014-02-07 Thread Mitch Harder
The return value in process_one_leaf could be over-written while looping over the items in the leaf. This patch will preserve a non-zero return value to the calling function if a non-zero return value is encountered in the loop. The return value of one (1) is consistent with non-zero values that

[PATCH] Btrfs: don't loop forever if we can't run because of the tree mod log

2014-02-07 Thread Josef Bacik
A user reported a 100% cpu hang with my new delayed ref code. Turns out I forgot to increase the count check when we can't run a delayed ref because of the tree mod log. If we can't run any delayed refs during this there is no point in continuing to look, and we need to break out. Thanks,

Re: integration-20140206 build failure

2014-02-07 Thread David Sterba
On Fri, Feb 07, 2014 at 12:38:22AM +, WorMzy Tykashi wrote: Building the latest integration snapshot, I get the following failure: ... [CC] cmds-filesystem.o cmds-filesystem.c: In function 'cmd_show': cmds-filesystem.c:740:12: error: invalid storage class for function

Re: lost with degraded RAID1

2014-02-07 Thread Chris Murphy
On Feb 7, 2014, at 4:34 AM, Johan Kröckel johan.kroec...@gmail.com wrote: Is there anything else I should do with this setup or may I nuke the two partitions and reuse them? Well I'm pretty sure once you run 'btrfs check --repair' that you've hit the end of the road. Possibly btrfs restore

Re: Provide a better free space estimate on RAID1

2014-02-07 Thread Kai Krakow
Josef Bacik jba...@fb.com schrieb: On 02/05/2014 03:15 PM, Roman Mamedov wrote: Hello, On a freshly-created RAID1 filesystem of two 1TB disks: # df -h /mnt/p2/ Filesystem Size Used Avail Use% Mounted on /dev/sda2 1.8T 1.1M 1.8T 1% /mnt/p2 I cannot write 2TB of user

Re: Are nocow files snapshot-aware

2014-02-07 Thread Kai Krakow
Chris Murphy li...@colorremedies.com schrieb: If the database/virtual machine/whatever is crash safe, then the atomic state that a snapshot grabs will be useful. How fast is this state fixed on disk from the time of the snapshot command? Loosely speaking. I'm curious if this is 1 second; a

Re: [btrfs] BUG: unable to handle kernel NULL pointer dereference at 0000000000000038

2014-02-07 Thread David Rientjes
On Fri, 7 Feb 2014, Fengguang Wu wrote: On Fri, Feb 07, 2014 at 02:13:59AM -0800, David Rientjes wrote: On Fri, 7 Feb 2014, Fengguang Wu wrote: [1.625020] BTRFS: selftest: Running btrfs_split_item tests [1.627004] BTRFS: selftest: Running find delalloc tests [2.289182]

Re: Are nocow files snapshot-aware

2014-02-07 Thread Chris Murphy
On Feb 7, 2014, at 2:07 PM, Kai Krakow hurikhan77+bt...@gmail.com wrote: Chris Murphy li...@colorremedies.com schrieb: If the database/virtual machine/whatever is crash safe, then the atomic state that a snapshot grabs will be useful. How fast is this state fixed on disk from the time of

Re: Are nocow files snapshot-aware

2014-02-07 Thread Kai Krakow
Duncan 1i5t5.dun...@cox.net schrieb: The question here is: Does it really make sense to create such snapshots of disk images currently online and running a system. They will probably be broken anyway after rollback - or at least I'd not fully trust the contents. VM images should not be

btrfs snapshot is killing IO and hanging my device with delayed writes for 10mn+

2014-02-07 Thread Marc MERLIN
On my workstation, which unfortunately I can't easily upgrade the kernel on, so it's running 3.8.0 for now, I've had pretty perpelexing hangs from btrfs snapshot when I make a new snapshot every hour. I see the btrfs snapshot command in iotop for over a minute, and it completes eventually.

Re: Are nocow files snapshot-aware

2014-02-07 Thread Kai Krakow
Chris Murphy li...@colorremedies.com schrieb: On Feb 7, 2014, at 2:07 PM, Kai Krakow hurikhan77+bt...@gmail.com wrote: Chris Murphy li...@colorremedies.com schrieb: If the database/virtual machine/whatever is crash safe, then the atomic state that a snapshot grabs will be useful. How

Re: [PATCH] xfstests: Btrfs: add test for large metadata blocks

2014-02-07 Thread Dave Chinner
On Fri, Feb 07, 2014 at 06:14:45PM +0100, Koen De Wit wrote: Tests Btrfs filesystems with all possible metadata block sizes, by setting large extended attributes on files. Signed-off-by: Koen De Wit koen.de@oracle.com There's a few things here that need fixing.

Re: Can you keep reflink relationship during a copy/backup to another filesystem?

2014-02-07 Thread Marc MERLIN
On Wed, Jan 29, 2014 at 08:05:14AM +, Hugo Mills wrote: On Tue, Jan 28, 2014 at 11:50:25PM -0800, Marc MERLIN wrote: So I used to use hardlinks to do historical backups of the same filesystem but I know it's preferable to use refllink with btrfs to avoid having too many hardlinks.

Re: [PATCH] Btrfs: convert to add transaction protection for btrfs send

2014-02-07 Thread Wang Shilong
Hi Josef, [..SNIP..] Yeah, very reasonable, if you don't mind, i would give a patch for this issue. Go for it, you'll be faster than I will be, all I do is run xfstests and try to reproduce things that will never reproduce for me. So Finally, i've figured everything out. Previous send is

Re: Are nocow files snapshot-aware

2014-02-07 Thread Duncan
Kai Krakow posted on Fri, 07 Feb 2014 23:26:34 +0100 as excerpted: So the question is: Do btrfs snapshots give the same guarantees on the filesystem level that write-barriers give on the storage level which exactly those processes rely upon? The cleanest solution would be if processes could

[PATCH] Btrfs: fix a lockdep warning when cleaning up aborted transaction

2014-02-07 Thread Liu Bo
Given now we have 2 spinlock for management of delayed refs, CONFIG_DEBUG_SPINLOCK=y helped me find this, [ 4723.413809] BUG: spinlock wrong CPU on CPU#1, btrfs-transacti/2258 [ 4723.414882] lock: 0x880048377670, .magic: dead4ead, .owner: btrfs-transacti/2258, .owner_cpu: 2 [ 4723.417146]

[PATCH v2] Btrfs: fix a spinlock warning when cleaning up aborted transaction

2014-02-07 Thread Liu Bo
Given now we have 2 spinlock for management of delayed refs, CONFIG_DEBUG_SPINLOCK=y helped me find this, [ 4723.413809] BUG: spinlock wrong CPU on CPU#1, btrfs-transacti/2258 [ 4723.414882] lock: 0x880048377670, .magic: dead4ead, .owner: btrfs-transacti/2258, .owner_cpu: 2 [ 4723.417146]