Re: A user cannot remove his readonly snapshots?!

On 2017-09-15 15:32, Ulli Horlacher wrote: On Fri 2017-09-15 (13:08), Austin S. Hemmelgarn wrote: On 2017-09-15 12:37, Ulli Horlacher wrote: I have my btrfs filesystem mounted with option user_subvol_rm_allowed tux@xerus: btrfs subvolume delete /test/tux/zz/.snapshot/2017-09-15_1824.test

kernel BUG at fs/btrfs/extent_io.c:1989

Hi I have a system that crashed during a defrag, upon reboot I got the following trace while resuming the defrag. Filesystem is BTRFS Raid1 on lvm+cache, kernel 4.13.2 Check --repair gives lots of warnings about parent transid verify failed, but otherwise completes without issue. Ran scrub

Re: A user cannot remove his readonly snapshots?!

On 2017-09-16 10:28, Ulli Horlacher wrote: On Sat 2017-09-16 (13:47), Kai Krakow wrote: Or you do "btrfs device stats .", it shows the associated device(s). tux@xerus:/test/tux/zz: btrfs device stats . ERROR: getting dev info for devstats failed: Operation not permitted Not possible for a

Re: ERROR: parent determination failed (btrfs send-receive)

On 18/09/17 07:10, Dave wrote: > For my understanding, what are the restrictions on deleting snapshots? > > What scenarios can lead to "ERROR: parent determination failed"? The man page for btrfs-send is reasonably clear on the requirements btrfs imposes. If you want to use incremental sends

Re: snapshots of encrypted directories?

On 2017-09-15 15:41, Ulli Horlacher wrote: On Fri 2017-09-15 (13:16), Austin S. Hemmelgarn wrote: And then mount enryptfs: mount.ecryptfs / / This only possible by root. For a user it is not possible to have access for his own snapshots. Bad. Which is why you use EncFS (which is a FUSE

Re: qemu-kvm VM died during partial raid1 problems of btrfs

On Wed, Sep 13, 2017 at 08:21:01AM -0400, Austin S. Hemmelgarn wrote: > On 2017-09-12 17:13, Adam Borowski wrote: > > On Tue, Sep 12, 2017 at 04:12:32PM -0400, Austin S. Hemmelgarn wrote: > > > On 2017-09-12 16:00, Adam Borowski wrote: > > > > Noted. Both Marat's and my use cases, though, involve

Re: how to run balance successfully (No space left on device)?

On 2017-09-18 17:29, Andrei Borzenkov wrote: On Mon, Sep 18, 2017 at 11:20 AM, Tomasz Chmielewski wrote: # df -h /var/lib/lxd FWIW, standard (aka util-linux) df is effectively useless in a situation such as this, as it really doesn't give you the information you need (it

Re: [PATCH] Btrfs: fix unexpected result when dio reading corrupted blocks

Hello, quick question for backporting.. On 09/15/17 23:06, Liu Bo wrote: > commit 4246a0b63bd8 ("block: add a bi_error field to struct bio") > changed the logic of how dio read endio reports errors. [snip] I've tried to merge this into my 4.9.x++ tree but have a question since the DIO APIs

Re: [PATCH 2/2] Remove misleading BCP 78 boilerplate

On Sun, Sep 17, 2017 at 07:52:27PM -0400, Nicholas D Steeves wrote: > BCP 78 applies to RFC 6234, but sha224-256.c is Simplified BSD. > > This causes the following lintian error when building on Debian and > Debian derivatives: > > E: btrfs-progs source: license-problem-non-free-RFC-BCP78 >

Re: using fio to test btrfs compression

2017-09-18 16:28 GMT+03:00 shally verma : > On Mon, Sep 18, 2017 at 1:56 PM, Timofey Titovets > wrote: >> 2017-09-18 10:36 GMT+03:00 shally verma : >>> Hi >>> >>> I wanted to test btrfs compression using fio command

Re: using fio to test btrfs compression

On Mon, Sep 18, 2017 at 1:56 PM, Timofey Titovets wrote: > 2017-09-18 10:36 GMT+03:00 shally verma : >> Hi >> >> I wanted to test btrfs compression using fio command but somehow >> during fio writes, I don't see code taking route of compression

Re: how to run balance successfully (No space left on device)?

On 2017-09-18 22:44, Peter Becker wrote: i'm not sure if it would help, but maybe you could try adding an 8GB (or more) USB flash drive to the pool and try to start balance. if it works out, you can throw him out of the pool after that. I really can't, it's an "online server". But I've

Re: how to run balance successfully (No space left on device)?

i'm not sure if it would help, but maybe you could try adding an 8GB (or more) USB flash drive to the pool and try to start balance. if it works out, you can throw him out of the pool after that. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to

Help Recovering BTRFS array

Hello, I will try to provide all information pertinent to the situation i find myself in. Yesterday while trying to write some data to a BTRFS filesystem on top of a mdadm raid5 array encrypted with dmcrypt comprising of 4 1tb HDD my system became unresponsive and i had no choice but to hard

Re: [PATCH] Btrfs: fix unexpected result when dio reading corrupted blocks

On Sat, Sep 16, 2017 at 01:58:34PM +0200, Goffredo Baroncelli wrote: > On 09/15/2017 11:06 PM, Liu Bo wrote: > > commit 4246a0b63bd8 ("block: add a bi_error field to struct bio") > > changed the logic of how dio read endio reports errors. > > > > For single stripe dio read, %bio->bi_status

Re: ERROR: parent determination failed (btrfs send-receive)

18.09.2017 11:45, Graham Cobb пишет: > On 18/09/17 07:10, Dave wrote: >> For my understanding, what are the restrictions on deleting snapshots? >> >> What scenarios can lead to "ERROR: parent determination failed"? > > The man page for btrfs-send is reasonably clear on the requirements > btrfs

Re: cp --reflink and qgroup limit

Hello Qu, Le 18/09/2017 à 01:33, Qu Wenruo a écrit : That's a bug, and should be fixed. OK. But it's a little complicated to fix. The biggest problem is, until reflink is done and we commit a transaction, we don't know if the the reflink operation will increase "rfer" or not. (Considering

Re: kernel BUG at fs/btrfs/extent_io.c:1989

On 09/18/17 19:09, Liu Bo wrote: > This 'mirror 0' looks fishy, (as mirror comes from > btrfs_io_bio->mirror_num, which should be at least 1 if raid1 setup is > in use.) > > Not sure if 4.13.2-gentoo made any changes on btrfs, but can you No, it did not; Gentoo always strives to be as close to

Re: [PATCH] Btrfs: fix unexpected result when dio reading corrupted blocks

On Mon, Sep 18, 2017 at 03:49:34PM +0200, Holger Hoffstätte wrote: > > Hello, quick question for backporting.. > > On 09/15/17 23:06, Liu Bo wrote: > > commit 4246a0b63bd8 ("block: add a bi_error field to struct bio") > > changed the logic of how dio read endio reports errors. > [snip] > > I've

Re: kernel BUG at fs/btrfs/extent_io.c:1989

Am Mon, 18 Sep 2017 20:30:41 +0200 schrieb Holger Hoffstätte : > On 09/18/17 19:09, Liu Bo wrote: > > This 'mirror 0' looks fishy, (as mirror comes from > > btrfs_io_bio->mirror_num, which should be at least 1 if raid1 setup > > is in use.) > > > > Not sure if

Re: kernel BUG at fs/btrfs/extent_io.c:1989

On Mon, Sep 18, 2017 at 08:55:29AM +, Paul Jones wrote: > Hi > I have a system that crashed during a defrag, upon reboot I got the > following trace while resuming the defrag. > Filesystem is BTRFS Raid1 on lvm+cache, kernel 4.13.2 > Check --repair gives lots of warnings about parent transid

Re: [PATCH] Btrfs: fix unexpected result when dio reading corrupted blocks

On 09/15/2017 11:06 PM, Liu Bo wrote: > commit 4246a0b63bd8 ("block: add a bi_error field to struct bio") > changed the logic of how dio read endio reports errors. > > For single stripe dio read, %bio->bi_status reflects the error before > verifying checksum, and now we're updating it when data

Re: Help Recovering BTRFS array

grondinm posted on Mon, 18 Sep 2017 14:14:08 -0300 as excerpted: > superblock: bytenr=65536, device=/dev/md0 > - > ERROR: bad magic on superblock on /dev/md0 at 65536 > > superblock: bytenr=67108864, device=/dev/md0 >

Re: difference between -c and -p for send-receive?

Dave posted on Mon, 18 Sep 2017 20:41:45 -0400 as excerpted: >> Well, I do not immediately see why -c must imply incremental send. We >> want to reduce amount of data that is transferred, so reuse data from >> existing snapshots, but it is really orthogonal to whether we send full >> subvolume or

Re: ERROR: parent determination failed (btrfs send-receive)

On Mon, Sep 18, 2017 at 12:23 AM, Andrei Borzenkov wrote: > 18.09.2017 05:31, Dave пишет: >> Sometimes when using btrfs send-receive, I get errors like this: >> >> ERROR: parent determination failed for >> >> When this happens, btrfs send-receive backups fail. And all

[PATCH v3 03/14] btrfs-progs: mkfs: Rework rootdir option to avoid custom chunk layout

mkfs.btrfs --rootdir uses its own custom chunk layout. This provides the possibility to limit the filesystem to a minimal size. However this custom chunk allocation has several problems. The most obvious problem is that it will allocate chunk from device offset 0. Both kernel and normal mkfs will

[PATCH v3 04/14] btrfs-progs: mkfs: Update allocation info before verbose output

Since new --rootdir can allocate chunk, it will modify the chunk allocation result. This patch will update allocation info before verbose output to reflect such info. Signed-off-by: Qu Wenruo --- mkfs/main.c | 33 + 1 file changed, 33

[PATCH v3 06/14] btrfs-progs: mkfs: Workaround BUG_ON caused by rootdir option

--rootdir option will start a transaction to fill the fs, however if something goes wrong, from ENOSPC to lack of permission, we won't commit transaction and cause BUG_ON trigger by uncommitted transaction: -- extent buffer leak: start 29392896 len 16384 extent_io.c:579: free_extent_buffer:

[PATCH v3 10/14] btrfs-progs: tests/common: Make checksum, permission and acl check path independent

convert_test_gen_checksums(), convert_test_perm() and convert_test_acl() all uses absolute path, which is good enough for convert test. However for "mkfs --rootdir" test, we want all above function to use relative path, making the output path independent. This patch modified all these functions

[PATCH v3 11/14] btrfs-progs: tests/mkfs: Add basic test case for rootdir parameter

Add a new test case to check if "--rootdir" option of mkfs.btrfs can handle the file content, perrmission and xattr correctly. The new test case reuses the convert facility, and looks just like convert-tests/001-ext2-basic. Signed-off-by: Qu Wenruo ---

[PATCH v3 01/14] btrfs-progs: Refactor find_next_chunk() to get rid of parameter root and objectid

Function find_next_chunk() is used to find next chunk start position, which should only do search on chunk tree and objectid is fixed to BTRFS_FIRST_CHUNK_TREE_OBJECTID. So refactor the parameter list to get rid of @root, which should be get from fs_info->chunk_root, and @objectid, which is fixed

[PATCH v3 08/14] btrfs-progs: tests/common: Split user xattr into its own branch for generate_dataset

generate_dataset() combines file acl and file xattr into "acls" type, since both of them are implemented by using file xattr. However sometimes we don't want user file attr under certain case, for example to populate files on tmpfs, which doesn't support user file xattr. So this patch split

[PATCH v3 00/14] Mkfs: Rework --rootdir to a more generic behavior

The patchset can be fetched from github: https://github.com/adam900710/btrfs-progs/tree/mkfs_rootdir_rework mkfs.btrfs --rootdir provides user a method to generate btrfs with pre-written content while without the need of root privilege to mount the fs. However the code is quite old and doesn't

[PATCH v3 14/14] btrfs-progs: tests/mkfs: Check error handler for rootdir parameter

Error handlers for the following case must fail gracefully: 1) No permission to read content of rootdir 2) Too small destination file 3) Non-existent destination file 4) Zero sized destination file Signed-off-by: Qu Wenruo ---

[PATCH v3 02/14] btrfs-progs: Fix one-byte overlap bug in free_block_group_cache

free_block_group_cache() calls clear_extent_bits() with wrong end, which is one byte larger than the correct range. This will cause the next adjacent cache state be split. And due to the split, private pointer (which points to block group cache) will be reset to NULL. This is very hard to detect

[PATCH v3 05/14] btrfs-progs: Avoid BUG_ON for chunk allocation when ENOSPC happens

When passing directory larger than block device using --rootdir parameter, we get the following backtrace: -- extent-tree.c:2693: btrfs_reserve_extent: BUG_ON `ret` triggered, value -28 ./mkfs.btrfs(+0x1a05d)[0x557939e6b05d] ./mkfs.btrfs(btrfs_reserve_extent+0xb5a)[0x557939e710c8]

[PATCH v3.1 11/14] btrfs-progs: tests/mkfs: Add basic test case for rootdir parameter

Add a new test case to check if "--rootdir" option of mkfs.btrfs can handle the file content, perrmission and xattr correctly. The new test case reuses the convert facility, and looks just like convert-tests/001-ext2-basic. Signed-off-by: Qu Wenruo --- changelog: v3.1:

[PATCH v3 13/14] btrfs-progs: mkfs: Fix overwritten return value for mkfs

For mkfs failure, especially --rootdir errors like EPERM/ENOSPC, the out branch will overwrite return value, causing wrong status code. Fix it so it can pass incoming test cases. Signed-off-by: Qu Wenruo --- mkfs/main.c | 5 +++-- 1 file changed, 3 insertions(+), 2

[PATCH v3 12/14] btrfs-progs: tests/common: Detect ungraceful failure case

run_mustfail() checks the return value but doesn't check if the failure is graceful. Add such ungraceful failure check, for incoming "mkfs.btrfs --rootdir" test cases, as old "--rootdir" never fail gracefully. Signed-off-by: Qu Wenruo --- tests/common | 5 + 1 file

[PATCH v3 07/14] btrfs-progs: Doc/mkfs: Add extra condition for rootdir option

Add extra limitation explained for --rootdir option, including: 1) Size limitation Now I decide to follow "mkfs.ext4 -d" behavior, so user is responsible to make sure the block device/file is large enough. 2) Read permission If user can't read the content, mkfs will just fail. So user

[PATCH v3 09/14] btrfs-progs: tests/common: Introduce optional parameter to specify destination directory for generate_dataset

Normally generate_dataset() will create data into $TEST_MNT/$dataset_type. This is OK since most tests are doing their operation in $TEST_MNT. However this is not the case for "mkfs --rootdir" test. This patch will adds an optional parameter for generate_dataset() to specify the destination

using fio to test btrfs compression

Hi I wanted to test btrfs compression using fio command but somehow during fio writes, I don't see code taking route of compression blocks where as If I do a copy to btrfs compression enabled mount point then I can easily see code falling through compression.c. Here's how I do my setup 1.

[PATCH 1/2] xfstests: Split MOUNT_OPTIONS to TEST_FS_MOUNT_OPTS and MOUNT_OPTIONS

Resovle the inconsistent of mount option. Btrfs use MOUNT_OPTIONS for both scrath_dev and test_dev. Change to MOUNT_OPTIONS for scratch mount, and TEST_FS_MOUNT_OPTS for test dev mount. Signed-off-by: Gu Jinxiang --- As mentioned by

[PATCH 2/2] xfstests: Add echo logic to show the information of TEST_FS_MOUNT_OPTS

There is message to show user the scrath mount, but no message to point TEST_FS_MOUNT_OPTS of test mount. Add logic to show test mount. Signed-off-by: Gu Jinxiang --- check | 1 + common/rc | 13 + 2 files changed, 14 insertions(+) diff --git a/check

Re: how to run balance successfully (No space left on device)?

# df -h /var/lib/lxd FWIW, standard (aka util-linux) df is effectively useless in a situation such as this, as it really doesn't give you the information you need (it can say you have lots of space available, but if btrfs has all of it allocated into chunks, even if the chunks have space in

Re: using fio to test btrfs compression

2017-09-18 10:36 GMT+03:00 shally verma : > Hi > > I wanted to test btrfs compression using fio command but somehow > during fio writes, I don't see code taking route of compression blocks > where as If I do a copy to btrfs compression enabled mount point then > I can

Re: how to run balance successfully (No space left on device)?

On Mon, Sep 18, 2017 at 11:20 AM, Tomasz Chmielewski wrote: >>> # df -h /var/lib/lxd >>> >>> FWIW, standard (aka util-linux) df is effectively useless in a situation >>> such as this, as it really doesn't give you the information you need (it >>> can say you have lots of space

Re: how to run balance successfully (No space left on device)?

Tomasz Chmielewski posted on Mon, 18 Sep 2017 18:27:09 +0900 as excerpted: > And perhaps more important - can I assume that right now, with the > latest stable kernel (4.13.2 right now), running "btrfs balance" is not > safe and can lead to data corruption or loss? > > > Consider the following

[PATCH] btrfs: cleanup mount path

Summary: Cleanup mount path by avoiding calling btrfs_mount() twice. This is for more understandable code and no functional change. Explanation: btrfs uses mount_subtree() to mount a subvolume directly. This function needs a vfsmount* of device's root (/), which is a return value of

difference between -c and -p for send-receive?

new subject for new question On Mon, Sep 18, 2017 at 1:37 PM, Andrei Borzenkov wrote: > >> What scenarios can lead to "ERROR: parent determination failed"? > > > > The man page for btrfs-send is reasonably clear on the requirements > > btrfs imposes. If you want to use