Re: btrfs-replace OOM on 2GB machine

On 2015-11-13 11:15, Georg Lukas wrote: Hi, while evaluating btrfs for production use I ended up with a degraded two-disk RAID1 with one disk missing, and wanted to perform a "btrfs replace" to rebuild the RAID1. However, the replace operation causes most of my userland to be OOM-killed and

Re: btrfs-replace OOM on 2GB machine

* Austin S Hemmelgarn [2015-11-17 13:56]: > While it probably isn't related to the OOM issue, I would be particularly > wary of using BTRFS on SMR disks, we've had multiple reports of serious > issues with them (and IIRC, they were all the same model of 8TB Seagate SMR >

[PATCH v2] btrfs: Continue replace when set_block_ro failed

xfstests/011 failed in node with small_size filesystem. Can be reproduced by following script: DEV_LIST="/dev/vdd /dev/vde" DEV_REPLACE="/dev/vdf" do_test() { local mkfs_opt="$1" local size="$2" dmesg -c >/dev/null umount $SCRATCH_MNT &>/dev/null echo

[PATCH] Fix balance regression in 4.4-rc

There's a regression in 4.4-rc since commit bc3094673f22 (btrfs: extend balance filter usage to take minimum and maximum) in that existing (non-ranged) balance with -dusage=x no longer works; all chunks are skipped. After staring at the code for a while and wondering why a non-ranged balance

Re: BTRFS Error - Rockstor

On 2015-11-17 03:08, Scotty Edmonds wrote: Sorry, I'm not at all familiar with backtrace. General procedure to get a backtrace on a system without core files: 1. Make sure you have debugging symbols installed for the program you want the backtrace for, and ideally any libraries it uses.

Re: [PATCH 00/15] btrfs: Hot spare and Auto replace

On 2015-11-16 17:07, Anand Jain wrote: On 11/16/2015 09:41 PM, Austin S Hemmelgarn wrote: On 2015-11-09 05:56, Anand Jain wrote: These set of patches provides btrfs hot spare and auto replace support for you review and comments. First, here below are the simple example steps to configure

[PATCH RESEND] fstests: test for btrfs direct IO write against compressed extent

From: Filipe Manana Test that doing a direct IO write against a file range that contains one prealloc extent and one compressed extent works correctly. >From the linux kernel 4.0 onwards, this either triggered an assertion failure (leading to a BUG_ON) when

Re: [PATCH] Fix balance regression in 4.4-rc

Holger Hoffstätte posted on Tue, 17 Nov 2015 12:29:32 +0100 as excerpted: > There's a regression in 4.4-rc since commit bc3094673f22 (btrfs: extend > balance filter usage to take minimum and maximum) in that existing > (non-ranged) balance with -dusage=x no longer works; all chunks are > skipped.

Re: Transaction aborted (error -17) after crash

B.H. On Wed, Nov 18, 2015 at 5:02 AM, Qu Wenruo wrote: > > > 在 2015年11月17日 13:12, Mordechay Kaganer 写道: >> >> B.H. >> >> >> [ 836.026606] BTRFS warning (device md1): block group 12969790406656 >> has wrong amount of free space >> [ 836.026610] BTRFS warning (device

Re: Transaction aborted (error -17) after crash

Mordechay Kaganer wrote on 2015/11/18 07:18 +0200: B.H. On Wed, Nov 18, 2015 at 5:02 AM, Qu Wenruo wrote: 在 2015年11月17日 13:12, Mordechay Kaganer 写道: B.H. [ 836.026606] BTRFS warning (device md1): block group 12969790406656 has wrong amount of free space [

Re: Transaction aborted (error -17) after crash

在 2015年11月17日 13:12, Mordechay Kaganer 写道: B.H. Hello. I have btrfs volume used for backups. The configuration is as follows: # uname -a Linux yemot-4u 4.2.5-040205-generic #201510270124 SMP Tue Oct 27 01:25:49 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux # btrfs --version btrfs-progs v4.2.3

Re: BTRFS Error - Rockstor

Sorry, I'm not at all familiar with backtrace. Thanks, Scotty Edmonds sco...@scottyedmonds.com From: Qu Wenruo Sent: November-17-15 3:35 AM To: Scotty Edmonds; Hugo Mills; Donald Pearson Cc: Btrfs BTRFS Subject: Re: BTRFS Error

[RFC PATCH 09/22] btrfs-progs: Introduce function to setup temporary tree root

Introduce new function, setup_temp_tree_root(), to initialize temporary tree root for make_btrfs_v2(). The new function will setup tree root at metadata chunk and ensure data won't be written into metadata chunk. Also, new make_btrfs_v2() will have a much better code structure than old

[RFC PATCH 20/22] btrfs-progs: Enhance record_file_blocks to handle reserved ranges

Enhance record_file_blocks() function to handle reserved ranges. Unlike most of file extents, ext* data in reserved ranges are not mapped on disk with 1:1 bytenr. So we can't use bytenr directly. But thanks for calling create_ext2_image_v2() before copying inodes, we have a image in convert

[RFC PATCH 07/22] btrfs-progs: Reserve space for system/meta chunks and superblock

Now we have a free space cache tree, we can reserve space for system and metadata chunks and super blocks. With this patch, even for the temporary fs, metadata tree blocks will be prevent from being allocated into possible data chunks. This provides the basis for later btrfs-convert enhancement.

[RFC PATCH 00/22] Btrfs-convert rework part 1

Here comes the first RFC version of btrfs-convert rework. Any test is welcomed, and it can already pass the convert test from btrfs-progs. (Since the test doesn't test rollback function) The new btrfs-convert has the following bugfix/feature: 1. True separate meta/data chunk The converted

[RFC PATCH 17/22] btrfs-progs: extent-tree: Enhance btrfs_record_file_extent

Btrfs_record_file_extent() has some small problems like: 1) Can't handle overlap extent 2) May create extent larger than BTRFS_MAX_EXTENT_SIZE So enhance it using previous added facilites. Signed-off-by: Qu Wenruo --- extent-tree.c | 160

[RFC PATCH 13/22] btrfs-progs: Introduce function to initialize csum tree

Introduce new function, setup_temp_csum_tree(), to setup temporary csum tree for make_btrfs_v2(). Signed-off-by: Qu Wenruo --- utils.c | 25 + 1 file changed, 25 insertions(+) diff --git a/utils.c b/utils.c index edd8ac1..ee2a538 100644 ---

[RFC PATCH 16/22] btrfs-progs: extent-tree: Introduce function to find the first overlap extent.

Introduce a new function, btrfs_search_overlap_extent() to find the first overlap extent. It's useful for later btrfs-convert rework. Signed-off-by: Qu Wenruo --- ctree.c | 24 +++ ctree.h | 2 ++ extent-tree.c | 63

[RFC PATCH 05/22] btrfs-progs: utils: Introduce new function to remove reserved ranges

Introduce functions to remove reserved ranges for later btrfs-convert rework. The reserved ranges includes: 1. [0,1M) 2. [btrfs_sb_offset(1), +BTRFS_STRIP_LEN) 3. [btrfs_sb_offset(2), +BTRFS_STRIP_LEN) Signed-off-by: Qu Wenruo --- utils.c | 115

[RFC PATCH 08/22] btrfs-progs: Introduce function to setup temporary superblock

Introduce a new function, setup_temp_super(), to setup temporary super for make_btrfs_v2(). Unlike the old codes in make_btrfs(), it will also initialize system_chunk_array before chunk tree allocation. Signed-off-by: Qu Wenruo --- utils.c | 117

[RFC PATCH 10/22] btrfs-progs: Introduce function to setup temporary chunk root

Introduce new function, setup_temp_chunk_root(), to initialize temporary chunk root for make_btrfs_v2(). Signed-off-by: Qu Wenruo --- utils.c | 169 1 file changed, 169 insertions(+) diff --git a/utils.c

[RFC PATCH 03/22] btrfs-progs: Add new init/free function and member for mkfs_config

Add new members for mkfs_config: 1. super_bytenr For convert case to restore where super block is allocated. Has no use for normal mkfs case. 2. convert_used A cache tree to record which ranges are used in original filesystem. This will gives the guide for later convert implement to

[RFC PATCH 11/22] btrfs-progs: Introduce function to initialize device tree

Introduce new function, setup_temp_device_tree(), to setup temporary device tree for make_btrfs_v2(). Signed-off-by: Qu Wenruo --- utils.c | 56 +++- 1 file changed, 55 insertions(+), 1 deletion(-) diff --git

[RFC PATCH 04/22] btrfs-progs: convert: Read and build up used space tree

Before we do real convert, we'd better read and build up used space cache tree for later data/meta chunk layout calculation. This patch will iterate all used blocks in ext2 filesystem and record it into mkfs_cfg.used cache tree for later used. This provides the basis for later btrfs-convert

[RFC PATCH 19/22] btrfs-progs: convert: Introduce function to migrate reserved ranges

Introduce new function, migrate_reserved_ranges() to migrate used ext2 data in btrfs reserved space. Unlike old implement, which will need to relocate all the complicated csum and reference relocation, above patches already ensure such reserved ranges won't be allocated. So here we only need copy

[RFC PATCH 21/22] btrfs-progs: convert: Introduce init_btrfs_v2 function.

Introduce new init_btrfs_v2() function for later newer do_convert(). Since we have good enough chunk allocation, a lot of wired chunk hack won't ever be used. We only need to insert data chunks and create needed subvolume/inode. Signed-off-by: Qu Wenruo ---

[RFC PATCH 14/22] btrfs-progs: Introduce function to setup temporary extent tree

Introduce new function, setup_temporary_extent_tree() to build a temporary extent tree for make_btrfs_v2(). Signed-off-by: Qu Wenruo --- utils.c | 198 +++- 1 file changed, 197 insertions(+), 1 deletion(-)

[RFC PATCH 18/22] btrfs-progs: convert: Introduce new function to create ext2 image

Use new function, create_ext2_image_v2() to create ext2 image. Unlike old function which is called after copying all inodes, this function need to be called before copying inodes. Signed-off-by: Qu Wenruo --- btrfs-convert.c | 153

[RFC PATCH 12/22] btrfs-progs: Introduce function to initialize fs tree

Introudce new function, setup_temp_fs_tree(), to setup temporary fs tree for make_btrfs_v2(). Signed-off-by: Qu Wenruo --- utils.c | 26 ++ 1 file changed, 26 insertions(+) diff --git a/utils.c b/utils.c index 3be484a..edd8ac1 100644 ---

[RFC PATCH 01/22] btrfs-progs: extent-cache: Add comments for search/lookup functions

There are quite a lot search/lookup functions with different behavior, add comments for them, as it will take extra time to view source to understand the behavior difference. Signed-off-by: Qu Wenruo --- extent-cache.h | 31 +++ 1 file

[RFC PATCH 06/22] btrfs-progs: utils: Introduce function to calculate the available space

Introduce a new function, calculate_available_space() to get available space for convert. Unlike old implement, this function will do the new work: 1) batch used ext* data space. To ensure data chunks will recovery them all. And restore the result into mkfs_cfg->convert_data_chunks for

[RFC PATCH 02/22] btrfs-progs: extent-tree: Add add_merge_cache_extent function

This add_merge_cache_extent() function will try to merge adjusted cache_extent. This is used for later btrfs-convert ext2 free space cache. Signed-off-by: Qu Wenruo --- extent-cache.c | 57 + extent-cache.h | 8

[RFC PATCH 15/22] btrfs-progs: Introduce function to create convert data chunks

Introduce new function, make_convert_data_chunks(), to build up data chunks for convert. It will call a modified verion of btrfs_alloc_data_chunk() to force data chunks to cover all known ext* data. Signed-off-by: Qu Wenruo --- btrfs-convert.c | 49

Re: [PATCH 01/12] test-scripts: test migration scripts

On Mon, Nov 16, 2015 at 03:51:11PM -0800, Darrick J. Wong wrote: > On Tue, Nov 17, 2015 at 07:58:45AM +1100, Dave Chinner wrote: > > On Fri, Nov 13, 2015 at 01:36:50PM -0800, Darrick J. Wong wrote: > > > Add two scripts: "nextid" finds the next available test ID number in a > > > group, and