Re: How to recover a filesystem without formatting nor using the btrfs check command.

At 10/25/2016 01:54 AM, none wrote: So do you mean lowmem is also low cpu ? Not sure, but lowmem is high IO. And by design, it won't cause dead look unless there is a looping tree block. But that will be detected by check_tree_block(). So, it just avoids any possible dead loop AFAIK.

Re: [PATCH v13.1 00/14] Btrfs In-band De-duplication

At 10/25/2016 01:46 AM, David Sterba wrote: On Thu, Oct 20, 2016 at 10:03:39AM +0800, Qu Wenruo wrote: Qu Wenruo (4): btrfs: delayed-ref: Add support for increasing data ref under spinlock btrfs: dedupe: Inband in-memory only de-duplication implement btrfs: relocation: Enhance error

[PATCH v2 3/4] btrfs-progs: raid56: Introduce new function to calculate raid5 parity or data stripe

Introduce new function raid5_gen_result() to calculate parity or data stripe. Signed-off-by: Qu Wenruo Signed-off-by: David Sterba --- Changelog: v2: None --- disk-io.h | 1 + raid56.c | 63

[PATCH v2 2/4] btrfs-progs: volumes: Remove BUG_ON in raid56 write routine

Remove various BUG_ON in raid56 write routine, including: 1) Memory allocation error Old codes allocates memory when code needs new memory in a loop, and catch the error using BUG_ON(). New codes allocates memory in a allocation loop first, if any failure is caught, freeing already

[PATCH v2 4/4] btrfs-progs: volumes: Use new raid5_gen_result to calculate raid5 parity

Use thew raid5_gen_result() function to calculate raid5 parity. Signed-off-by: Qu Wenruo Signed-off-by: David Sterba --- Changelog: v2: None --- volumes.c | 19 ++- 1 file changed, 6 insertions(+), 13 deletions(-) diff --git

[PATCH v2 1/4] btrfs-progs: rename raid6.c to raid56.c

This allows us to put raid5 codes into that file other than creating a new raid5.c. Signed-off-by: Qu Wenruo Signed-off-by: David Sterba --- Changelog v2: None --- Makefile.in | 2 +- disk-io.h | 2 +- raid6.c => raid56.c | 0 3

Re: Scrubbing Errors after restoring backup

On 24 October 2016 at 17:53, Stefan Malte Schumacher wrote: > Hello > > For reference please see this post. > https://mail-archive.com/linux-btrfs@vger.kernel.org/msg58461.html > Please note that I downgraded to btrfs-progs 4.6.1 as advised. > > After exchanging

Re: bio linked list corruption.

On Oct 24, 2016 5:00 PM, "Linus Torvalds" wrote: > > On Mon, Oct 24, 2016 at 3:42 PM, Andy Lutomirski wrote: > > > Now the fallocate thread catches up and *exits*. Dave's test makes a > > new thread that reuses the stack (the vmap area or the

Re: bio linked list corruption.

On Mon, Oct 24, 2016 at 3:42 PM, Andy Lutomirski wrote: > > Here's my theory: I think you're looking at the right code but the > wrong stack. shmem_fault_wait is fine, but shmem_fault_waitq looks > really dicey. Hmm. > Consider: > > fallocate calls wake_up_all(), which

Re: bio linked list corruption.

On Mon, Oct 24, 2016 at 1:46 PM, Linus Torvalds wrote: > On Mon, Oct 24, 2016 at 1:06 PM, Andy Lutomirski wrote: >>> >>> [69943.450108] Oops: 0003 [#1] PREEMPT SMP DEBUG_PAGEALLOC >> >> This is an unhandled kernel page fault. The string "Oops"

Re: bio linked list corruption.

On 10/24/2016 05:50 PM, Linus Torvalds wrote: On Mon, Oct 24, 2016 at 2:17 PM, Linus Torvalds wrote: The vmalloc/vfree code itself is a bit scary. In particular, we have a rather insane model of TLB flushing. We leave the virtual area on a lazy purge-list, and

Scrubbing Errors after restoring backup

Hello For reference please see this post. https://mail-archive.com/linux-btrfs@vger.kernel.org/msg58461.html Please note that I downgraded to btrfs-progs 4.6.1 as advised. After exchanging the malfunctioning drive I re-created the filesystem and restored the backup from my NAS. (I didnt entirely

Re: bio linked list corruption.

On Mon, Oct 24, 2016 at 2:17 PM, Linus Torvalds wrote: > > The vmalloc/vfree code itself is a bit scary. In particular, we have a > rather insane model of TLB flushing. We leave the virtual area on a > lazy purge-list, and we delay flushing the TLB and actually

Re: bio linked list corruption.

On Mon, Oct 24, 2016 at 1:46 PM, Linus Torvalds wrote: > > So this is all some really subtle code, but I'm not seeing that it > would be wrong. Ahh... Except maybe.. The vmalloc/vfree code itself is a bit scary. In particular, we have a rather insane model of TLB

Re: bio linked list corruption.

On Mon, Oct 24, 2016 at 1:06 PM, Andy Lutomirski wrote: >> >> [69943.450108] Oops: 0003 [#1] PREEMPT SMP DEBUG_PAGEALLOC > > This is an unhandled kernel page fault. The string "Oops" is so helpful :-/ I think there was a line above it that DaveJ just didn't include. > >>

[PATCH 4/5] writeback: introduce super_operations->write_metadata

Now that we have metadata counters in the VM, we need to provide a way to kick writeback on dirty metadata. Introduce super_operations->write_metadata. This allows file systems to deal with writing back any dirty metadata we need based on the writeback needs of the system. Since there is no

[PATCH 1/5] remove mapping from balance_dirty_pages*()

The only reason we pass in the mapping is to get the inode in order to see if writeback cgroups is enabled, and even then it only checks the bdi and a super block flag. balance_dirty_pages() doesn't even use the mapping. Since balance_dirty_pages*() works on a bdi level, just pass in the bdi and

[PATCH 0/5] Support for metadata specific accounting

(Dave again I apologize, for some reason our email server hates you and so I didn't get your previous responses again, and didn't notice until I was looking at the patchwork history for my previous submissions, so I'll try and answer all your questions here, and then I'll be much more judicious

Re: bio linked list corruption.

On Sun, Oct 23, 2016 at 9:40 PM, Dave Jones wrote: > On Sun, Oct 23, 2016 at 05:32:21PM -0400, Chris Mason wrote: > > > > > > On 10/22/2016 11:20 AM, Dave Jones wrote: > > > On Fri, Oct 21, 2016 at 04:02:45PM -0400, Dave Jones wrote: > > > > > > > > It could be

Re: [PATCH] btrfs: imporve delayed refs iterations

On Fri, Oct 21, 2016 at 05:05:07PM +0800, Wang Xiaoguang wrote: > This issue was found when I tried to delete a heavily reflinked file, > when deleting such files, other transaction operation will not have a > chance to make progress, for example, start_transaction() will blocked > in

Re: Btrfs: fix free space tree bitmaps on big-endian systems

Hi Omar, On Mon, Oct 24, 2016 at 5:16 PM, Omar Sandoval wrote: > On Mon, Oct 24, 2016 at 09:23:04AM +0200, Geert Uytterhoeven wrote: >> On Sat, Oct 15, 2016 at 2:46 AM, Linux Kernel Mailing List >> wrote: >> > Web: >> >

Re: [PATCH 1/2] btrfs-progs: fsck: Add support to clear v1 free space cache.

On Thu, Oct 20, 2016 at 09:04:23AM +0800, Qu Wenruo wrote: > >> + /* Clear all free space cache inodes and its extent data */ > >> + while (1) { > >> + bg_cache = btrfs_lookup_first_block_group(fs_info, current); > >> + if (!bg_cache) > >> + break; > >> +

Re: [PATCH v9.1 0/5] In-band de-duplication for btrfs-progs

On Thu, Oct 20, 2016 at 10:06:59AM +0800, Qu Wenruo wrote: > Patchset can be fetched from github: > https://github.com/adam900710/btrfs-progs.git dedupe_latest Branch added to progs intergration, for preview and testing. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in

Re: [PATCH v13.1 00/14] Btrfs In-band De-duplication

On Thu, Oct 20, 2016 at 10:03:39AM +0800, Qu Wenruo wrote: > Qu Wenruo (4): > btrfs: delayed-ref: Add support for increasing data ref under spinlock > btrfs: dedupe: Inband in-memory only de-duplication implement > btrfs: relocation: Enhance error handling to avoid BUG_ON > btrfs: dedupe:

Re: [PATCH] btrfs: imporve delayed refs iterations

On 10/24/16 18:46, David Sterba wrote: > On Fri, Oct 21, 2016 at 05:05:07PM +0800, Wang Xiaoguang wrote: >> This issue was found when I tried to delete a heavily reflinked file, >> when deleting such files, other transaction operation will not have a >> chance to make progress, for example,

Re: [PATCH] btrfs-progs: check: release path after usage

On Mon, Oct 24, 2016 at 10:18:14AM -0500, Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues > > While performing an fsck, an assertion failure occurs because of reusing path > in a loop. > ctree.c:1112: btrfs_search_slot: Warning: assertion `p->nodes[0] != NULL` > failed,

Re: [PATCH] btrfs-progs: Fix wrong tree block alignment for unalianged block group

On Mon, Oct 24, 2016 at 03:22:33PM +0800, Qu Wenruo wrote: > Commit 854437ca(btrfs-progs: extent-tree: avoid allocating tree block > that crosses stripe boundary) introduces check for logical bytenr not > crossing stripe boundary. > > However that check is not completely correct. > It only checks

Re: [PATCH v2] btrfs: change btrfs_csum_final result param type to u8

On Fri, Oct 21, 2016 at 12:47:02PM +0100, Domagoj Tršan wrote: > csum member of struct btrfs_super_block has array type of u8. It makes sense > that function btrfs_csum_final should be also declared to accept u8 *. I > changed the declaration of method void btrfs_csum_final(u32 crc, char >

Re: [PATCH] btrfs: imporve delayed refs iterations

On Fri, Oct 21, 2016 at 05:05:07PM +0800, Wang Xiaoguang wrote: > This issue was found when I tried to delete a heavily reflinked file, > when deleting such files, other transaction operation will not have a > chance to make progress, for example, start_transaction() will blocked > in

Re: [PATCH v2] btrfs: introduce priority based delalloc shrink mechanism

Hi Josef, are you fine with V2? On Thu, Oct 13, 2016 at 05:31:25PM +0800, Wang Xiaoguang wrote: > Since commit b02441999efcc6152b87cd58e7970bb7843f76cf, we don't wait all > ordered extents, but I run into some enospc errors when doing large file > create and delete tests, it's because

Re: Btrfs: fix free space tree bitmaps on big-endian systems

On Mon, Oct 24, 2016 at 09:23:04AM +0200, Geert Uytterhoeven wrote: > On Sat, Oct 15, 2016 at 2:46 AM, Linux Kernel Mailing List > wrote: > > Web: > > https://git.kernel.org/torvalds/c/2fe1d55134fce05c17ea118a2e37a4af771887bc > > Commit:

Re: [PATCH] btrfs-progs: Initialize btrfs_path before use

On 10/24/2016 09:57 AM, Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues > > While performing an fsck, an assertion failure occurs because of reusing path > in a loop. > ctree.c:1112: btrfs_search_slot: Warning: assertion `p->nodes[0] != NULL` > failed, value 0 > >

[PATCH] Release path after usage.

From: Goldwyn Rodrigues While performing an fsck, an assertion failure occurs because of reusing path in a loop. ctree.c:1112: btrfs_search_slot: Warning: assertion `p->nodes[0] != NULL` failed, value 0 Signed-off-by: Goldwyn Rodrigues diff --git

Re: Btrfs: fix free space tree bitmaps on big-endian systems

On Mon, Oct 24, 2016 at 09:23:04AM +0200, Geert Uytterhoeven wrote: > On Sat, Oct 15, 2016 at 2:46 AM, Linux Kernel Mailing List > wrote: > > Web: > > https://git.kernel.org/torvalds/c/2fe1d55134fce05c17ea118a2e37a4af771887bc > > Commit:

Re: [PATCH] btrfs-progs: Initialize btrfs_path before use

On Mon, Oct 24, 2016 at 3:57 PM, Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues > > While performing an fsck, an assertion failure occurs because of reusing path > in a loop. > ctree.c:1112: btrfs_search_slot: Warning: assertion `p->nodes[0] != NULL` >

[PATCH] btrfs-progs: Initialize btrfs_path before use

From: Goldwyn Rodrigues While performing an fsck, an assertion failure occurs because of reusing path in a loop. ctree.c:1112: btrfs_search_slot: Warning: assertion `p->nodes[0] != NULL` failed, value 0 Signed-off-by: Goldwyn Rodrigues diff --git

Btrfs progs pre-release 4.8.2-rc1

Hi, a pre-release has been tagged. There are build and misc bugfixes. I'm doing a rc for a minor release as it proved to be useful in the past when dealing with the build errors. Please test. ETA for 4.8.2 is in +2 days. Changes since 4.8.1: * convert: also convert file attributes * check:

Re: bio linked list corruption.

On 10/24/2016 12:40 AM, Dave Jones wrote: On Sun, Oct 23, 2016 at 05:32:21PM -0400, Chris Mason wrote: > > > On 10/22/2016 11:20 AM, Dave Jones wrote: > > On Fri, Oct 21, 2016 at 04:02:45PM -0400, Dave Jones wrote: > > > > > > It could be worth trying this, too: > > > > > > > >

Re: [PATCH] ioctl.h: add missing kernel compatibility header for BUILD_ASSERT

Hi, On Mon, Oct 24, 2016 at 09:29:12AM +0100, sly...@gmail.com wrote: > From: Sergei Trofimovich > > Header breakage noticed by cynede. Reproducible as: > > $ gcc -c /usr/include/btrfs/ioctl.h -o /tmp/a.o > /usr/include/btrfs/ioctl.h:42:14: error: expected

Re: Drive Replacement

On 2016-10-21 18:13, Peter Becker wrote: if you have >750 GB free you can simply remove one of the drives. btrfs device delete /dev/sd[x] /mnt #power off, replace device btrfs device add /dev/sd[y] /mnt Make sure to balance afterwards if you do this, the new disk will be pretty much unused

[PATCH] ioctl.h: add missing kernel compatibility header for BUILD_ASSERT

From: Sergei Trofimovich Header breakage noticed by cynede. Reproducible as: $ gcc -c /usr/include/btrfs/ioctl.h -o /tmp/a.o /usr/include/btrfs/ioctl.h:42:14: error: expected declaration specifiers or '...' before 'sizeof' BUILD_ASSERT(sizeof(struct

Re: Btrfs: fix free space tree bitmaps on big-endian systems

On Sat, Oct 15, 2016 at 2:46 AM, Linux Kernel Mailing List wrote: > Web: > https://git.kernel.org/torvalds/c/2fe1d55134fce05c17ea118a2e37a4af771887bc > Commit: 2fe1d55134fce05c17ea118a2e37a4af771887bc 520f16abf003952d in v4.7.10 1ff6341b5d92dd6b in

[PATCH] btrfs-progs: Fix wrong tree block alignment for unalianged block group

Commit 854437ca(btrfs-progs: extent-tree: avoid allocating tree block that crosses stripe boundary) introduces check for logical bytenr not crossing stripe boundary. However that check is not completely correct. It only checks if the logical bytenr and length agaist absolute logical offset.