Re: [PATCH 1/2] btrfs: catch cow on deleting snapshots

2018-11-30 Thread Hans van Kranenburg
On 12/1/18 1:19 AM, Qu Wenruo wrote: > > > On 2018/12/1 上午12:52, Josef Bacik wrote: >> From: Josef Bacik >> >> When debugging some weird extent reference bug I suspected that we were >> changing a snapshot while we were deleting it, which could explain my >> bug. > > May I ask under which case

Re: [PATCH 1/2] btrfs: catch cow on deleting snapshots

2018-11-30 Thread Qu Wenruo
On 2018/12/1 上午12:52, Josef Bacik wrote: > From: Josef Bacik > > When debugging some weird extent reference bug I suspected that we were > changing a snapshot while we were deleting it, which could explain my > bug. May I ask under which case we're going to modify an unlinked snapshot? Maybe

Re: Need help with potential ~45TB dataloss

2018-11-30 Thread Qu Wenruo
On 2018/11/30 下午9:53, Patrick Dijkgraaf wrote: > Hi all, > > I have been a happy BTRFS user for quite some time. But now I'm facing > a potential ~45TB dataloss... :-( > I hope someone can help! > > I have Server A and Server B. Both having a 20-devices BTRFS RAID6 > filesystem. Because of

Re: [PATCH 2/2] btrfs: run delayed items before dropping the snapshot

2018-11-30 Thread Filipe Manana
On Fri, Nov 30, 2018 at 5:12 PM Filipe Manana wrote: > > On Fri, Nov 30, 2018 at 4:53 PM Josef Bacik wrote: > > > > From: Josef Bacik > > > > With my delayed refs patches in place we started seeing a large amount > > of aborts in __btrfs_free_extent > > > > BTRFS error (device sdb1): unable to

Re: [PATCH 1/2] btrfs: catch cow on deleting snapshots

2018-11-30 Thread Josef Bacik
On Fri, Nov 30, 2018 at 05:14:54PM +, Filipe Manana wrote: > On Fri, Nov 30, 2018 at 4:53 PM Josef Bacik wrote: > > > > From: Josef Bacik > > > > When debugging some weird extent reference bug I suspected that we were > > changing a snapshot while we were deleting it, which could explain my

Re: [PATCH 1/2] btrfs: catch cow on deleting snapshots

2018-11-30 Thread Filipe Manana
On Fri, Nov 30, 2018 at 4:53 PM Josef Bacik wrote: > > From: Josef Bacik > > When debugging some weird extent reference bug I suspected that we were > changing a snapshot while we were deleting it, which could explain my > bug. This was indeed what was happening, and this patch helped me >

Re: [PATCH 2/2] btrfs: run delayed items before dropping the snapshot

2018-11-30 Thread Filipe Manana
On Fri, Nov 30, 2018 at 4:53 PM Josef Bacik wrote: > > From: Josef Bacik > > With my delayed refs patches in place we started seeing a large amount > of aborts in __btrfs_free_extent > > BTRFS error (device sdb1): unable to find ref byte nr 91947008 parent 0 root > 35964 owner 1 offset 0 >

[PATCH 0/2] Fix aborts when dropping snapshots

2018-11-30 Thread Josef Bacik
With more machines running my recent delayed ref rsv patches we started seeing a spike in boxes aborting in __btrfs_free_extent with being unable to find the extent ref. The full details are in 2/2, but the summary is there's been a bug ever since the original delayed inode item stuff was

[PATCH 1/2] btrfs: catch cow on deleting snapshots

2018-11-30 Thread Josef Bacik
From: Josef Bacik When debugging some weird extent reference bug I suspected that we were changing a snapshot while we were deleting it, which could explain my bug. This was indeed what was happening, and this patch helped me verify my theory. It is never correct to modify the snapshot once

[PATCH 2/2] btrfs: run delayed items before dropping the snapshot

2018-11-30 Thread Josef Bacik
From: Josef Bacik With my delayed refs patches in place we started seeing a large amount of aborts in __btrfs_free_extent BTRFS error (device sdb1): unable to find ref byte nr 91947008 parent 0 root 35964 owner 1 offset 0 Call Trace: ? btrfs_merge_delayed_refs+0xaf/0x340

Re: btrfs development - question about crypto api integration

2018-11-30 Thread Nikolay Borisov
On 30.11.18 г. 17:22 ч., Chris Mason wrote: > On 29 Nov 2018, at 12:37, Nikolay Borisov wrote: > >> On 29.11.18 г. 18:43 ч., Jean Fobe wrote: >>> Hi all, >>> I've been studying LZ4 and other compression algorithms on the >>> kernel, and seen other projects such as zram and ubifs using the

Re: btrfs development - question about crypto api integration

2018-11-30 Thread Chris Mason
On 29 Nov 2018, at 12:37, Nikolay Borisov wrote: > On 29.11.18 г. 18:43 ч., Jean Fobe wrote: >> Hi all, >> I've been studying LZ4 and other compression algorithms on the >> kernel, and seen other projects such as zram and ubifs using the >> crypto api. Is there a technical reason for not

Need help with potential ~45TB dataloss

2018-11-30 Thread Patrick Dijkgraaf
Hi all, I have been a happy BTRFS user for quite some time. But now I'm facing a potential ~45TB dataloss... :-( I hope someone can help! I have Server A and Server B. Both having a 20-devices BTRFS RAID6 filesystem. Because of known RAID5/6 risks, Server B was a backup of Server A. After

[PATCH RESEND v12] btrfs-progs: add cli to forget one or all scanned devices

2018-11-30 Thread Anand Jain
This patch adds cli btrfs device forget [dev] to remove the given device structure in the kernel if the device is unmounted. If no argument is given it shall remove all stale (device which are not mounted) from the kernel. Signed-off-by: Anand Jain Reviewed-by: Nikolay Borisov --- v11->v12:

[PATCH RESEND] fstests: btrfs use forget if not reload

2018-11-30 Thread Anand Jain
[I will send this the xfstest ML after kernel and progs patch has been integrated]. btrfs reload was introduced to cleanup the device list inside the btrfs kernel module. The problem with the reload approach is that you can't run btrfs test cases 124,125, 154 and 164 on the system with btrfs as

[PATCH RESEND v12] Add cli and ioctl to forget scanned device(s)

2018-11-30 Thread Anand Jain
v12: Fixed coding style - leave space between " : ". v11: btrfs-progs: Bring the code into the else part of if(forget). Use strerror to print the erorr instead of ret. v10: Make btrfs-progs changes more readable. With an effort to keep the known bug [1] as it is.. [1] The

[PATCH RESEND] fstests: btrfs use forget if not reload

2018-11-30 Thread Anand Jain
[I will send this the xfstest ML after kernel and progs patch has been integrated]. btrfs reload was introduced to cleanup the device list inside the btrfs kernel module. The problem with the reload approach is that you can't run btrfs test cases 124,125, 154 and 164 on the system with btrfs as

[PATCH RESEND v12] btrfs: introduce feature to forget a btrfs device

2018-11-30 Thread Anand Jain
Support for a new command 'btrfs dev forget [dev]' is proposed here to undo the effects of 'btrfs dev scan [dev]'. For this purpose this patch proposes to use ioctl #5 as it was empty. IOW(BTRFS_IOCTL_MAGIC, 5, ..) This patch adds new ioctl BTRFS_IOC_FORGET_DEV which can be sent from the