Re: [BUG] Btrfs scrub sometime recalculate wrong parity in raid5

At 06/25/2016 08:21 PM, Goffredo Baroncelli wrote: Hi all, following the thread "Adventures in btrfs raid5 disk recovery", I investigated a bit the BTRFS capability to scrub a corrupted raid5 filesystem. To test it, I first find where a file was stored, and then I tried to corrupt the data

Re: [PATCH 3/3] btrfs-progs: send: fix handling of -c option

On 2016/11/02 21:22, David Sterba wrote: > On Wed, Oct 19, 2016 at 11:35:40AM +0900, Tsutomu Itoh wrote: >> When two or more -c options are specified, cannot find a suitable >> parent. So, output stream is bigger than correct one. >> >> [before] >> # btrfs send -f /tmp/data1 -c Snap0 -c ../SnapX

[PATCH v14 10/15] btrfs: ordered-extent: Add support for dedupe

From: Wang Xiaoguang Add ordered-extent support for dedupe. Note, current ordered-extent support only supports non-compressed source extent. Support for compressed source extent will be added later. Signed-off-by: Qu Wenruo Signed-off-by:

[PATCH v14 01/15] btrfs: improve inode's outstanding_extents computation

From: Wang Xiaoguang This issue was revealed by modifying BTRFS_MAX_EXTENT_SIZE(128MB) to 64KB, When modifying BTRFS_MAX_EXTENT_SIZE(128MB) to 64KB, fsstress test often gets these warnings from btrfs_destroy_inode():

[PATCH v14 04/15] btrfs: dedupe: Introduce function to initialize dedupe info

From: Wang Xiaoguang Add generic function to initialize dedupe info. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang Reviewed-by: Josef Bacik --- fs/btrfs/Makefile | 2 +-

[PATCH v14 07/15] btrfs: delayed-ref: Add support for increasing data ref under spinlock

For in-band dedupe, btrfs needs to increase data ref with delayed_ref locked, so add a new function btrfs_add_delayed_data_ref_lock() to increase extent ref with delayed_refs already locked. Signed-off-by: Qu Wenruo Reviewed-by: Josef Bacik ---

[PATCH v14 00/15] Btrfs In-band De-duplication

This patchset can be fetched from github: https://github.com/adam900710/linux.git wang_dedupe_latest This version is just another update mainly related to ENOSPC fix, rebased to Chris' for-linus-4.9 branch. The main part is the ENOSPC fix, which is first exposed in dedupe development, and thanks

[PATCH v14 13/15] btrfs: relocation: Enhance error handling to avoid BUG_ON

Since the introduce of btrfs dedupe tree, it's possible that balance can race with dedupe disabling. When this happens, dedupe_enabled will make btrfs_get_fs_root() return PTR_ERR(-ENOENT). But due to a bug in error handling branch, when this happens backref_cache->nr_nodes is increased but the

[PATCH v14 14/15] btrfs: dedupe: Introduce new reconfigure ioctl

Introduce new reconfigure ioctl, and new FORCE flag for in-band dedupe ioctls. Now dedupe enable and reconfigure ioctl are stateful. | Current state | Ioctl| Next state | | Disabled | enable

[PATCH v14 11/15] btrfs: dedupe: Inband in-memory only de-duplication implement

Core implement for inband de-duplication. It reuse the async_cow_start() facility to do the calculate dedupe hash. And use dedupe hash to do inband de-duplication at extent level. The work flow is as below: 1) Run delalloc range for an inode 2) Calculate hash for the delalloc range at the unit of

[PATCH v14 15/15] btrfs: fix false enospc for in-band dedupe

From: Wang Xiaoguang When testing in-band dedupe, sometimes we got ENOSPC error, but fs still has much free space. After some debuging work, we found that it's btrfs_delalloc_reserve_metadata() which sometimes tries to reserve plenty of metadata space, even for very

[PATCH v14 03/15] btrfs: dedupe: Introduce dedupe framework and its header

From: Wang Xiaoguang Introduce the header for btrfs in-band(write time) de-duplication framework and needed header. The new de-duplication framework is going to support 2 different dedupe methods and 1 dedupe hash. Signed-off-by: Qu Wenruo

[PATCH v14 09/15] btrfs: dedupe: Implement btrfs_dedupe_calc_hash interface

From: Wang Xiaoguang Unlike in-memory or on-disk dedupe method, only SHA256 hash method is supported yet, so implement btrfs_dedupe_calc_hash() interface using SHA256. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang

[PATCH v14 06/15] btrfs: dedupe: Introduce function to remove hash from in-memory tree

From: Wang Xiaoguang Introduce static function inmem_del() to remove hash from in-memory dedupe tree. And implement btrfs_dedupe_del() and btrfs_dedup_disable() interfaces. Also for btrfs_dedupe_disable(), add new functions to wait existing writer and block incoming

[PATCH v14 12/15] btrfs: dedupe: Add ioctl for inband dedupelication

From: Wang Xiaoguang Add ioctl interface for inband dedupelication, which includes: 1) enable 2) disable 3) status And a pseudo RO compat flag, to imply that btrfs now supports inband dedup. However we don't add any ondisk format change, it's just a pseudo RO compat

[PATCH v14 02/15] btrfs: fix false enospc for compression

From: Wang Xiaoguang When testing btrfs compression, sometimes we got ENOSPC error, though fs still has much free space, xfstests generic/171, generic/172, generic/173, generic/174, generic/175 can reveal this bug in my test environment when compression is enabled.

[PATCH v14 08/15] btrfs: dedupe: Introduce function to search for an existing hash

From: Wang Xiaoguang Introduce static function inmem_search() to handle the job for in-memory hash tree. The trick is, we must ensure the delayed ref head is not being run at the time we search the for the hash. With inmem_search(), we can implement the

[PATCH v14 05/15] btrfs: dedupe: Introduce function to add hash into in-memory tree

From: Wang Xiaoguang Introduce static function inmem_add() to add hash into in-memory tree. And now we can implement the btrfs_dedupe_add() interface. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang

Re: [Bug 186671] New: OOM on system with just rsync running 32GB of ram 30GB of pagecache

On 11/03/2016 07:53 PM, Andrew Morton wrote: > > (switched to email. Please respond via emailed reply-to-all, not via the > bugzilla web interface). +CC also btrfs just in case it's a problem in page reclaim there > On Wed, 02 Nov 2016 13:02:39 + bugzilla-dae...@bugzilla.kernel.org wrote:

Re: [PATCH 3/4] btrfs: Expoert and move leaf/subtree qgroup helpers to qgroup.c

Reviewed-and-Tested-by: Goldwyn Rodrigues On 10/17/2016 08:31 PM, Qu Wenruo wrote: > Move account_shared_subtree() to qgroup.c and rename it to > btrfs_qgroup_trace_subtree(). > > Do the same thing for account_leaf_items() and rename it to > btrfs_qgroup_trace_leaf_items(). >

Re: [PATCH 4/4] btrfs: qgroup: Fix qgroup data leaking by using subtree tracing

Reviewed-and-Tested-by: Goldwyn Rodrigues On 10/17/2016 08:31 PM, Qu Wenruo wrote: > Commit 62b99540a1d91e464 (btrfs: relocation: Fix leaking qgroups numbers > on data extents) only fixes the problem partly. > > The previous fix is to trace all new data extents at transaction

Re: [PATCH 1/4] btrfs: qgroup: Add comments explaining how btrfs qgroup works

Reviewed-by: Goldwyn Rodrigues On 10/17/2016 08:31 PM, Qu Wenruo wrote: > Add explain how btrfs qgroup works. > > Qgroup is split into 3 main phrases: > 1) Reserve >To ensure qgroup doesn't exceed its limit > > 2) Trace >To info qgroup to trace which extent > > 3)

Re: [PATCH 2/4] btrfs: qgroup: Rename functions to make it follow reserve,trace,account steps

Reviewed-and-Tested-by: Goldwyn Rodrigues On 10/17/2016 08:31 PM, Qu Wenruo wrote: > Rename btrfs_qgroup_insert_dirty_extent(_nolock) to > btrfs_qgroup_trace_extent(_nolock), according to the new > reserve/trace/account naming schema. > > Signed-off-by: Qu Wenruo

Re: [PATCH] btrfs: Remove some dead code

Le 03/11/2016 à 13:43, Tobias Klauser a écrit : On 2016-11-01 at 11:26:06 +0100, Christophe JAILLET wrote: 'btrfs_iget()' can not return an error pointer, so this test can be removed. This descrption does not match what the patch actually does. Shouldn't it say

recover btrfs with kernel 4.9-rc3 but btrfs progs fails

Hi, currently i've an fs which triggers this one on mount while originally having 50% disk free - but btrfs progs fails too. # btrfs check --repair -p /dev/vdb1 enabling repair mode couldn't open RDWR because of unsupported option features (3). ERROR: cannot open file system [ 164.378512]

Re: Problem with btrfs snapshots

On 2016-11-03 10:21, Peter Becker wrote: (copy for mainlinglist) 2016-11-03 15:16 GMT+01:00 Дмитрий Нечаев : Yes. We tried "sync" in our script but it doesn't help. It works only then we make one snapshot at a time. Even if we use "sync" before and after creating

linux 4.9-rc3 OOM's anyone else?

Looks like 4.9-rc3 still has the OOM issue since 4.7 as far as I can tell. Anyone else still seeing it? I created a kernel bug report for it: https://bugzilla.kernel.org/show_bug.cgi?id=186671. Not sure if it's btrfs that's making it worse or what not. -- To unsubscribe from this list: send the

Re: Problem with btrfs snapshots

(copy for mainlinglist) 2016-11-03 15:16 GMT+01:00 Дмитрий Нечаев : Yes. We tried "sync" in our script but it doesn't help. It works only then we make one snapshot at a time. Even if we use "sync" before and after creating snapshot, it doesn't help. -- To unsubscribe from

Re: Problem with btrfs snapshots

Have you tryed "sync" between create several snapshots commands? 2016-11-03 13:22 GMT+01:00 Дмитрий Нечаев : > Hello. > We are have a strange situation with btrfs snapshot. We have a special > script to create snapshot and if we create several snapshots in the same > time

Re: [PATCH] btrfs: Remove some dead code

On 2016-11-01 at 11:26:06 +0100, Christophe JAILLET wrote: > 'btrfs_iget()' can not return an error pointer, so this test can be > removed. This descrption does not match what the patch actually does. Shouldn't it say "...can not return NULL, so this test can be

Problem with btrfs snapshots

Hello. We are have a strange situation with btrfs snapshot. We have a special script to create snapshot and if we create several snapshots in the same time we get ENOSPC error. But if we create snapshots one after another with a little pause between (about 10 minutes), then everything looks ok. Is

Re: btrfs scrub with unexpected results

On 2016-11-02 17:55, Tom Arild Naess wrote: Hello, I have been running btrfs on a file server and backup server for a couple of years now, both set up as RAID 10. The file server has been running along without any problems since day one. My problems has been with the backup server. A little