Re: Recover btrfs volume which can only be mounded in read-only mode

On Mon, Oct 26, 2015 at 09:14:00AM +, Duncan wrote: > Dmitry Katsubo posted on Sun, 18 Oct 2015 11:44:08 +0200 as excerpted: > > >> Meanwhile, the present btrfs raid1 read-scheduler is both pretty simple > >> to code up and pretty simple to arrange tests for that run either one > >> side or

Re: [RFC PATCH 2/3] btrfs-progs: kernel based default features for mkfs

Thanks Jeff for the comments. On 10/23/2015 11:24 PM, Jeff Mahoney wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/21/15 4:45 AM, Anand Jain wrote: mkfs from latest btrfs-progs will enable latest default features, and if the kernel is down-rev and does not support a latest default

Re: [PATCH 1/2 v2] Btrfs: fix regression when running delayed references

On Mon, Oct 26, 2015 at 9:22 AM, Liu Bo wrote: > On 10/25/2015 06:04 PM, fdman...@kernel.org wrote: >> >> From: Filipe Manana >> >> In the kernel 4.2 merge window we had a refactoring/rework of the delayed >> references implementation in order to fix

RE: [PATCH] btrfs: Remove code for no-cow in scrub/replace

Hi, Filipe Manana > -Original Message- > From: Filipe Manana [mailto:fdman...@gmail.com] > Sent: Friday, October 23, 2015 11:17 PM > To: Jeff Mahoney > Cc: Zhao Lei ; linux-btrfs@vger.kernel.org > Subject: Re: [PATCH] btrfs: Remove code for no-cow

[PATCH] btrfs: clear PF_NOFREEZE in cleaner_kthread()

From: Jiri Kosina cleaner_kthread() kthread calls try_to_freeze() at the beginning of every cleanup attempt. This operation can't ever succeed though, as the kthread hasn't marked itself as freezable. Before (hopefully eventually) kthread freezing gets converted to fileystem

Re: [PATCH v5] btrfs: qgroup: Don't copy extent buffer to do qgroup rescan

On Mon, Oct 26, 2015 at 1:19 AM, Qu Wenruo wrote: > Ancient qgroup code call memcpy() on a extent buffer and use it for leaf > iteration. > > As extent buffer contains lock, pointers to pages, it's never sane to do > such copy. > > The following bug may be caused by this

RE: [PATCH] btrfs: Remove code for no-cow in scrub/replace

Hi, Jeff Mahoney Thanks for review! > -Original Message- > From: Jeff Mahoney [mailto:je...@suse.com] > Sent: Friday, October 23, 2015 11:11 PM > To: Zhao Lei ; linux-btrfs@vger.kernel.org > Subject: Re: [PATCH] btrfs: Remove code for no-cow in scrub/replace > >

[PATCH v2 0/5] btrfs-progs: Add all missing close_ctree and btrfs_close_all_devices

This patch add all missing close_ctree and btrfs_close_all_devices to several tools in btrfs progs, to avoid memory leak. Changelog v1->v2: Move btrfs_close_all_devices() from cmd-XXX into btrfs.c to make code simple, and avoid similar problem in cmd-XXX in future. Zhao Lei (5): btrfs-progs:

[PATCH v2 1/5] btrfs-progs: btrfs: Add missing btrfs_close_all_devices for btrfs command

Adding a btrfs_close_all_devices() after command callback in btrfs.c can force-close all opened device before program exit, to avoid memory leak in all btrfs sub-command. Suggested-by: David Sterba Signed-off-by: Zhao Lei --- btrfs.c | 9 - 1

[PATCH v2 5/5] btrfs-progs: use system's default path for math.h

Line of #include "math.h" in extent-tree.c using quotas is history reason, (we have cuseom math.h in source before) Now it is better to use "<>" instead of quotas for this header file. Signed-off-by: Zhao Lei --- extent-tree.c | 2 +- 1 file changed, 1 insertion(+), 1

[PATCH v2 4/5] btrfs-progs: Add missing close_ctree to btrfs-select-super.c

Add missing close_ctree() to btrfs-select-super.c to avoid memory leak. Signed-off-by: Zhao Lei --- btrfs-select-super.c | 1 + 1 file changed, 1 insertion(+) diff --git a/btrfs-select-super.c b/btrfs-select-super.c index bd44978..df74153 100644 ---

[PATCH v2 2/5] btrfs-progs: Remove all btrfs_close_all_devices in sub-command

Since we have btrfs_close_all_devices() in btrfs's main entrance, it is not necessary to call btrfs_close_all_devices() separately in each sub-command. Signed-off-by: Zhao Lei --- cmds-check.c | 1 - cmds-device.c | 3 --- cmds-replace.c | 2 -- 3 files changed, 6

[PATCH v2 3/5] btrfs-progs: Add all missing btrfs_close_all_devices to standalone tools

This patch add all missing btrfs_close_all_devices() to standalone tools in btrfs progs, to avoid memory leak. Signed-off-by: Zhao Lei --- btrfs-calc-size.c| 1 + btrfs-debug-tree.c | 5 - btrfs-find-root.c| 1 + btrfs-map-logical.c | 1 +

Re: [PATCH 1/2 v2] Btrfs: fix regression when running delayed references

On 10/25/2015 06:04 PM, fdman...@kernel.org wrote: From: Filipe Manana In the kernel 4.2 merge window we had a refactoring/rework of the delayed references implementation in order to fix certain problems with qgroups. However that rework introduced one more regression that

[GIT PULL] Btrfs fixes for delayed refs regression and a deadlock

From: Filipe Manana Hi Chris, please consider the following fixes for the 4.4 merge window (they were previously sent to the mailing list already). They fix an issue with delayed references that makes us hit some BUG_ONs as of the 4.2 kernel release. A lot of people have

random i/o error without error in dmesg

Hi, I have this error for a time, It's not easy to reproduce, i write everything i know at the moment. I maintain some servers running xen (4.5.1) and gentoo dom0 with recent kernels (3.18.*, 4.1.6, 4.2.3, 4.2.4). I use gentoo-sources patchset. Running xen domu s, for www and mysql. I have

[PATCH 1/3] btrfs: Cleanup no_quota parameter

No_quota parameter for delayed_ref related function are meaningless after 4.2-rc1, as any new delayed_ref_head will cause qgroup to scan extent for its rfer/excl change without checking no_quota flag. So this patch will clean them up. Signed-off-by: Qu Wenruo ---

[PATCH 2/3] btrfs: qgroup: Fix a race in delayed_ref which leads to abort trans

Between btrfs_allocerved_file_extent() and btrfs_add_delayed_qgroup_reserve(), there is a window that delayed_refs are run and delayed ref head maybe freed before btrfs_add_delayed_qgroup_reserve(). This will cause btrfs_dad_delayed_qgroup_reserve() to return -ENOENT, and cause transaction to be

[4.4][PATCH 0/3] btrfs: Qgroup hotfix

This patchset fixes 2 bugs: 1. Race condition leading to abort transaction Reported by Filipe, fixed by 2nd patch. 2. Qgroup low level double free leading to EDQUOT In fact, I hit such bug several times during internal rebase, but I'm so stupid to forgot to include it in v3 patchset. Fixed in 3rd

[PATCH 3/3] btrfs: qgroup: Fix a rebase bug which will cause qgroup double free

When rebasing my patchset, I forgot to pick up a cleanup patch to remove old hotfix in 4.2 release. Witouth the cleanup, it will screw up new qgroup reserve framework and always cause minus reserved number. Signed-off-by: Qu Wenruo --- fs/btrfs/qgroup.c | 4 1

Re: [PATCH 2/2 v3] Btrfs: fix regression running delayed references when using qgroups

wrote on 2015/10/25 18:51 +: From: Filipe Manana In the kernel 4.2 merge window we had a big changes to the implementation of delayed references and qgroups which made the no_quota field of delayed references not used anymore. More specifically the no_quota field is

Re: corrupted RAID1: unsuccessful recovery / help needed

Lukas Pirl posted on Mon, 26 Oct 2015 19:19:50 +1300 as excerpted: > TL;DR: RAID1 does not recover, I guess the interesting part in the stack > trace is: [elided, I'm not a dev so it's little help to me] > > I'd appreciate some help for repairing a corrupted RAID1. > > Setup: > * Linux

Re: [PATCH 1/3] btrfs: Cleanup no_quota parameter

Filipe Manana wrote on 2015/10/26 08:14 +: On Mon, Oct 26, 2015 at 6:11 AM, Qu Wenruo wrote: No_quota parameter for delayed_ref related function are meaningless after 4.2-rc1, as any new delayed_ref_head will cause qgroup to scan extent for its rfer/excl change

[PATCH 1/2 v3] Btrfs: fix regression when running delayed references

From: Filipe Manana In the kernel 4.2 merge window we had a refactoring/rework of the delayed references implementation in order to fix certain problems with qgroups. However that rework introduced one more regression that leads to the following trace when running delayed

Re: [PATCH 1/3] btrfs: Cleanup no_quota parameter

On Mon, Oct 26, 2015 at 8:25 AM, Qu Wenruo wrote: > > > Filipe Manana wrote on 2015/10/26 08:14 +: >> >> On Mon, Oct 26, 2015 at 6:11 AM, Qu Wenruo >> wrote: >>> >>> No_quota parameter for delayed_ref related function are meaningless >>>

Re: Recover btrfs volume which can only be mounded in read-only mode

Dmitry Katsubo posted on Sun, 18 Oct 2015 11:44:08 +0200 as excerpted: >> Meanwhile, the present btrfs raid1 read-scheduler is both pretty simple >> to code up and pretty simple to arrange tests for that run either one >> side or the other, but not both, or that are well balanced to both. >>

Re: Recover btrfs volume which can only be mounded in read-only mode

Dmitry Katsubo posted on Sun, 18 Oct 2015 11:44:08 +0200 as excerpted: [Regarding the btrfs raid1 "device-with-the-most-space" chunk-allocation strategy.] > I think the mentioned strategy (fill in the device with most free space) > is not most effective. If the data is spread equally, the read

Re: [PATCH 1/3] btrfs: Cleanup no_quota parameter

Qu Wenruo wrote on 2015/10/26 16:25 +0800: Filipe Manana wrote on 2015/10/26 08:14 +: On Mon, Oct 26, 2015 at 6:11 AM, Qu Wenruo wrote: No_quota parameter for delayed_ref related function are meaningless after 4.2-rc1, as any new delayed_ref_head will cause

[4.4][PATCH 0/3] btrfs: Qgroup hotfix

This patchset fixes 2 bugs: 1. Race condition leading to abort transaction Reported by Filipe, fixed by 2nd patch. 2. Qgroup low level double free leading to EDQUOT In fact, I hit such bug several times during internal rebase, but I'm so stupid to forgot to include it in v3 patchset. Fixed in 3rd

Re: [PATCH 1/3] btrfs: Cleanup no_quota parameter

On Mon, Oct 26, 2015 at 6:11 AM, Qu Wenruo wrote: > No_quota parameter for delayed_ref related function are meaningless > after 4.2-rc1, as any new delayed_ref_head will cause qgroup to scan > extent for its rfer/excl change without checking no_quota flag. > > So this

[PATCH 2/2 v3] Btrfs: fix regression running delayed references when using qgroups

From: Filipe Manana In the kernel 4.2 merge window we had a big changes to the implementation of delayed references and qgroups which made the no_quota field of delayed references not used anymore. More specifically the no_quota field is not used anymore as of: commit

corrupted RAID1: unsuccessful recovery / help needed

TL;DR: RAID1 does not recover, I guess the interesting part in the stack trace is: Call Trace: [] __del_reloc_root+0x30/0x100 [btrfs] [] free_reloc_roots+0x25/0x40 [btrfs] [] merge_reloc_roots+0x18e/0x240 [btrfs] [] btrfs_recover_relocation+0x374/0x420 [btrfs] []

Re: Exclusive quota of snapshot exceeded despite no space used

Thanks a lot for your reply! While remounting the filesystem fixes the issue temporary, it doesn't take very long for the bug to happen again so it's not really a workaround I can work with. I did recompile the kernel using your patches, but unfortunately the problem still appears. Thanks,

Re: BTRFS with 8TB SMR drives

I decided to give this ST8000AS0002 a try for storing old snapshots, although standardization for more optimal/native contol of SMR drives is still ongoing. I saw people got it working with 3.18 kernel, so that gave confidence. I wanted to see if i could get it running with 4.3.0-rc6 kernel (and

Re: BTRFS with 8TB SMR drives

I decided to give this ST8000AS0002 a try for backups / storing old snapshots, although standardization for more optimal/native contol of SMR drives is still ongoing. I saw people got it working with 3.18 kernel, so that gave confidence. I wanted to see if i could get it running with 4.3.0-rc6

Re: [PATCH v7 5/4] copy_file_range.2: New page documenting copy_file_range()

On Mon, Oct 26, 2015 at 12:19:33PM +, Pádraig Brady wrote: > On 26/10/15 03:39, Christoph Hellwig wrote: > > On Sat, Oct 24, 2015 at 01:02:21PM +0100, P??draig Brady wrote: > >> I'm a bit worried about the sparse expansion and default reflinking > >> which might preclude cp(1) from using this

Re: Exclusive quota of snapshot exceeded despite no space used

Johannes Henninger wrote on 2015/10/27 01:15 +0100: On 26.10.2015 08:12, Qu Wenruo wrote: Thanks a lot for your reply! While remounting the filesystem fixes the issue temporary, it doesn't take very long for the bug to happen again so it's not really a workaround I can work with. I did

Re: Bad fs performance, IO freezes

Hello, currently my computer freezes every several seconds for half a second or so. Using it feels like I'm playing musical chairs with the kernel. I have just one download happening on utorrent right now - this is what the graph looks like: http://i.imgur.com/LqhMtrJ.png and every time a new

[PATCH] btrfs: Print Warning only if ENOSPC_DEBUG is enabled

Signed-off-by : Ashish Samant --- fs/btrfs/delayed-inode.c | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/delayed-inode.c b/fs/btrfs/delayed-inode.c index a2ae427..b86cfd9 100644 --- a/fs/btrfs/delayed-inode.c +++

Re: [PATCH v3 06/21] btrfs: delayed_ref: Add new function to record reserved space into delayed ref

Filipe Manana wrote on 2015/10/25 14:39 +: On Tue, Oct 13, 2015 at 3:20 AM, Qu Wenruo wrote: Add new function btrfs_add_delayed_qgroup_reserve() function to record how much space is reserved for that extent. As btrfs only accounts qgroup at run_delayed_refs()

Re: [PATCH v3 06/21] btrfs: delayed_ref: Add new function to record reserved space into delayed ref

Chris Mason wrote on 2015/10/27 01:14 -0400: On Tue, Oct 27, 2015 at 12:13:11PM +0800, Qu Wenruo wrote: Filipe Manana wrote on 2015/10/25 14:39 +: On Tue, Oct 13, 2015 at 3:20 AM, Qu Wenruo wrote: Add new function btrfs_add_delayed_qgroup_reserve() function

Re: [PATCH v3 06/21] btrfs: delayed_ref: Add new function to record reserved space into delayed ref

On Tue, Oct 27, 2015 at 12:13:11PM +0800, Qu Wenruo wrote: > > > Filipe Manana wrote on 2015/10/25 14:39 +: > >On Tue, Oct 13, 2015 at 3:20 AM, Qu Wenruo wrote: > >>Add new function btrfs_add_delayed_qgroup_reserve() function to record > >>how much space is reserved

Re: Recover btrfs volume which can only be mounded in read-only mode

Hugo Mills posted on Mon, 26 Oct 2015 09:24:57 + as excerpted: > On Mon, Oct 26, 2015 at 09:14:00AM +, Duncan wrote: >> Dmitry Katsubo posted on Sun, 18 Oct 2015 11:44:08 +0200 as excerpted: >> >>> I think PID-based solution is not the best one. Why not simply take a >>> random device?

Re: [PATCH] Btrfs-progs: Prevent creation of filesystem with 'mixed bgs' and having differing sectorsize and nodesize.

On Wed, Oct 14, 2015 at 11:10:38PM +0530, Chandan Rajendra wrote: > mkfs.btrfs allows creation of Btrfs filesystem instances with mixed block > group feature enabled and having a sectorsize different from nodesize. > For e.g: > > [root@localhost btrfs-progs]# mkfs.btrfs -f -M -s 4096 -n 16384

Bad fs performance, IO freezes

Hi guys, I am running into really bad performance. Here's my setup: WD Red 6 TB connected over USB2 to my core i7 laptop, running Ubuntu 32-bit with kernel 4.0.4-040004-generic #201505171336. Single btrfs partition covering whole disk. Autodefrag is on. fstab line: UUID=... /media/X btrfs

Re: Bad fs performance, IO freezes

I get the same kind of muddy errors when I do a quota rescan on the filesystem or qgroup show on any subvolume on mine, and I know I don't have them enabled. On Mon, Oct 26, 2015 at 8:56 AM, cheater00 . wrote: > fwiw, I did this: > > sudo btrfs qgroup show /media/X > ERROR:

Re: random i/o error without error in dmesg

Hi FWIW, this sounds like what I've been seeing with dovecot. In case it's relevant, I'll try to explain. After some uptime, I'll see log messages like this: Okt 26 12:05:46 thetick dovecot[467]: imap(marcec): Error: pread() failed with file

Re: Bad fs performance, IO freezes

On 10/26/2015 08:16 PM, cheater00 . wrote: Hi guys, I am running into really bad performance. Here's my setup: WD Red 6 TB connected over USB2 to my core i7 laptop, running Ubuntu 32-bit with kernel 4.0.4-040004-generic #201505171336. Single btrfs partition covering whole disk. Autodefrag is

Re: Bad fs performance, IO freezes

Thanks for the reply. What version did this go into? I'll try getting a prebuilt backport of the kernel, building source could slow things down considerably, but debs will not be available for the latest few minor versions I guess. So if you can tell me a min version, I'll try to find the latest

Re: [PATCH v2 0/5] btrfs-progs: Add all missing close_ctree and btrfs_close_all_devices

On Mon, Oct 26, 2015 at 06:28:17PM +0800, Zhao Lei wrote: > This patch add all missing close_ctree and btrfs_close_all_devices > to several tools in btrfs progs, to avoid memory leak. > > Changelog v1->v2: > Move btrfs_close_all_devices() from cmd-XXX into btrfs.c to make > code simple, and

Re: Bad fs performance, IO freezes

I don't remember doing that, but just to exclude everything, how do I check? On Mon, Oct 26, 2015 at 2:45 PM, Donald Pearson wrote: > AFAIK quotas aren't a mount option, but if you never enabled them and > created the qgroups by hand that's your answer and the issue

Re: Bad fs performance, IO freezes

Accidentally didn't reply to the list the 1st time. I see the same issue when I have quotas enabled. If you have quotas on, see if turning them off helps. On Mon, Oct 26, 2015 at 7:16 AM, cheater00 . wrote: > Hi guys, > I am running into really bad performance. Here's my

Re: Bad fs performance, IO freezes

fwiw, I did this: sudo btrfs qgroup show /media/X ERROR: can't perform the search - No such file or directory ERROR: can't list qgroups: No such file or directory I assume this means no qgroups present, which means no quotas present. Please correct me if I'm wrong. So yes, the issue must lie

Re: [PATCH v7 5/4] copy_file_range.2: New page documenting copy_file_range()

On 26/10/15 03:39, Christoph Hellwig wrote: > On Sat, Oct 24, 2015 at 01:02:21PM +0100, P??draig Brady wrote: >> I'm a bit worried about the sparse expansion and default reflinking >> which might preclude cp(1) from using this call in most cases, but I will >> test and try to use it. coreutils has

Re: Bad fs performance, IO freezes

There are no quotas. I haven't enabled them. I believe the fstab says that - could they be enabled in another way? How do I check for sure? The man page doesn't say how to check the status: https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs-quota On Mon, Oct 26, 2015 at 2:32 PM, Donald Pearson

Re: Bad fs performance, IO freezes

AFAIK quotas aren't a mount option, but if you never enabled them and created the qgroups by hand that's your answer and the issue must be something else. On Mon, Oct 26, 2015 at 8:36 AM, cheater00 . wrote: > There are no quotas. I haven't enabled them. I believe the fstab

Re: Bad fs performance, IO freezes

I have located 4.3.0-rc7 binaries which I will now try. On Mon, Oct 26, 2015 at 3:38 PM, cheater00 . wrote: > Thanks for the reply. What version did this go into? I'll try getting > a prebuilt backport of the kernel, building source could slow things > down considerably, but

Re: Bad fs performance, IO freezes

So far I cannot reproduce. If I don't post again this means the issue has been fixed by updating the kernel. On Mon, Oct 26, 2015 at 4:40 PM, cheater00 . wrote: > I have located 4.3.0-rc7 binaries which I will now try. > > On Mon, Oct 26, 2015 at 3:38 PM, cheater00 .

Re: Bad fs performance, IO freezes

I do not experience btrfs-transacti going up to 100% for minutes at a time now (not reproduced yet) but I have it spiking up to say 30% for a short while and everything jags during that time. So, say, if I am watching youtube, the sound cuts out and the video drops out for a bit. And if I'm