Re: btrfs-image hash collision option, super slow

Piotr Pawłow posted on Sun, 12 Nov 2017 15:05:44 +0100 as excerpted: >> The other part I'm not groking is that some filenames fail with: >> >> WARNING: cannot find a hash collision for 'Tool', generating garbage, >> it won't match indexes > > Unfortunately there are no CRC32 collisions for any

[PATCH 1/2] btrfs-progs: qgroup: move btrfs_show_qgroups's error handler to __qgroup_search

We have to process the return value of BTRFS_IOC_TREE_SEARCH ioctl in advance, so that we can distinguish between the two case where quota is not enabled (ioctl return -ENOENT) and either parent qgroup or child qgroup does not exist (update_qgroup_relation return -ENOENT). Besides this, any error

[PATCH 2/2] btrfs: reset SB MOVED_ON flag for dynamically reappearing missing device

When the missing device reappears and joins the RAID group, and if there are no more missing device at the volume level, then reset the BTRFS_SUPER_FLAG_VOL_MOVED_ON flag. This patch is on top of the patch [1] in the ML. [1] btrfs: handle dynamically reappearing missing device Signed-off-by:

[PATCH 1/2] btrfs: handle volume split brain scenario

In two device configs of RAID1/RAID5 where one device can be missing in the degraded mount, or in the configs such as four devices RAID6 where two devices can be missing, in these type of configs it can form two separate set of devices where each of the set can be mounted without the other set.

Re: Theoretical Question about commit=n

On 2017年11月13日 06:01, Hans van Kranenburg wrote: > On 11/12/2017 09:58 PM, Robert White wrote: >> Is the commit interval monotonic, or is it seconds after sync? >> >> What I mean is that if I manually call sync(2) does the commit timer >> reset? I'm thinking it does not, but I can imagine a

Re: btrfs check lowmem vs original

Hi On 11/11/2017 07:16 AM, Chris Murphy wrote: Resurrecting this old thread because I'm still seeing these errors. On Thu, Mar 16, 2017 at 6:54 PM, Qu Wenruo wrote: At 03/17/2017 07:22 AM, Chris Murphy wrote: With kernel 4.10.3, and btrfs-progs 4.10, I'm still

[PATCH 1/2] btrfs: refactor btrfs_free_stale_device() to get device list delete

We need to delete a device from the dev_list, so refactor btrfs_free_stale_device() for delete_device_from_list(). Signed-off-by: Anand Jain --- fs/btrfs/volumes.c | 27 +-- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git

[PATCH] btrfs-progs: add 'btrfs device ignore' cli

This patch adds btrfs device ignode so that a device can be ignored/missed during mount if the device is already been scanned. Basically, this command will undo the effect of the command btrfs device scan This change is compatible with older kernel without the ioctl BTRFS_IOC_IGNORE_DEV

Re: Theoretical Question about commit=n

On 11/13/2017 01:41 AM, Qu Wenruo wrote: > > On 2017年11月13日 06:01, Hans van Kranenburg wrote: >> On 11/12/2017 09:58 PM, Robert White wrote: >>> Is the commit interval monotonic, or is it seconds after sync? >>> >>> What I mean is that if I manually call sync(2) does the commit timer >>> reset?

[PATCH 2/3] btrfs-progs: lowmem fsck: Fix false backref lost warning for keyed extent data ref

For keyed extent ref, its offset is calculated offset (file offset - file extent offset), just like inlined extent data ref. However the code is using file offset to hash extent data ref offset, causing false backref lost warning like: -- ERROR: data extent[16913485824 7577600] backref lost

[PATCH 1/3] btrfs-progs: lowmem fsck: Fix inlined data extent ref lookup

When lowmem fsck tries to find backref of a specified file extent, it searches inlined data ref first. However, extent data ref contains owner root objectid, inode number and calculated offset (file offset - extent offset). The code only checks owner root objectid, not checking inode number nor

[PATCH 3/3] btrfs-progs: fsck-test: Introduce test case for false data extent backref lost

Introduce a new test image, which has an extent item with no inlined extent data ref, but all keyed extent data ref. Only in this case we can trigger fase data extent backref lost bug in lowmem mode. Signed-off-by: Qu Wenruo --- .../020-extent-ref-cases/keyed_data_ref_only.img

[PATCH v2 2/2] btrfs-progs: qgroup: cleanup __qgroup_search

Replace the if statement with the switch statement, and return the appropriate value for the future use rather than directly exit. Signed-off-by: Lu Fengqi --- Changelog: v2: 1. revoke incorrect goto pattern 2. split the error handler movement to another patch

[PATCH 2/4] btrfs-progs: lowmem check: Fix function call stack overflow caused by wrong tree reloc tree detection

For tree reloc tree root, its backref points to it self. So for such case, we should finish the lookup. Previous end condition is to ensure it's tree reloc tree *and* needs its root bytenr to match the bytenr passed in. However the @root passed in can be other tree, e.g. other tree reloc tree

[PATCH 1/4] btrfs-progs: backref: Allow backref walk to handle direct parent ref

[BUG] Btrfs lowmem mode fails with the following ASSERT() on certain valid image. -- backref.c:466: __add_missing_keys: Assertion `ref->root_id` failed, value 0 -- [REASON] Lowmem mode uses btrfs_find_all_roots() when walking down fs trees. However if a tree block with only shared parent

[PATCH 3/4] btrfs-progs: lowmem check: Fix false alerts for image with shared block ref only backref

[BUG] For image with shared block ref only metadata item like: -- item 66 key (21573632 METADATA_ITEM 0) itemoff 3971 itemsize 24 refs 66 gen 9 flags TREE_BLOCK|FULL_BACKREF tree block skinny level 0 item 0 key (21573632 SHARED_BLOCK_REF

[PATCH 0/4] Lowmem mode btrfs fixes exposed by complex tree

The patchset (along with "backref lost" bug fixes and test cases) can be fetched from github: https://github.com/adam900710/btrfs-progs/tree/lowmem_fix Despite the backref lost false alerts reported by Chris Murphy, there are still some other bugs to be fixed. One is also exposed by Chris

Re: btrfs-image hash collision option, super slow

On Sun, Nov 12, 2017 at 7:05 AM, Piotr Pawłow wrote: > >>>It could definitely be improved -- I believe there are some good >>> (but non-trivial) algorithms for finding preimages for CRC32 checksums >>> out there. It's just that btrfs-image doesn't use them. > > I implemented

[PATCH] btrfs/154: test for device dynamic rescan

Make sure missing device is included in the alloc list when it is scanned on a mounted FS. This test case needs btrfs kernel patch which is in the ML [PATCH] btrfs: handle dynamically reappearing missing device Without the kernel patch, the test will run, but reports as failed, as the device

Re: btrfs check lowmem vs original

On Sun, Nov 12, 2017 at 7:25 PM, Su Yue wrote: > > I have some patches for lowmem mode. But they are not fully tested and > unstable. > It will be so nice if you can provide some images. Sorry, Qu got the image offline, I wasn't paying attention that btrfs list wasn't

Re: btrfs check lowmem vs original

On 2017年11月13日 11:37, Chris Murphy wrote: > On Sun, Nov 12, 2017 at 7:25 PM, Su Yue wrote: > >> >> I have some patches for lowmem mode. But they are not fully tested and >> unstable. >> It will be so nice if you can provide some images. > > Sorry, Qu got the image

Re: Theoretical Question about commit=n

On 2017年11月13日 09:17, Hans van Kranenburg wrote: > On 11/13/2017 01:41 AM, Qu Wenruo wrote: >> >> On 2017年11月13日 06:01, Hans van Kranenburg wrote: >>> On 11/12/2017 09:58 PM, Robert White wrote: Is the commit interval monotonic, or is it seconds after sync? What I mean is that if

[PATCH 2/2] btrfs: introduce feature to ignore a btrfs device

Support for a new command is being added here: btrfs dev ignore Which shall undo the effects of the command btrfs dev scan This cli/ioctl is needed as there is no way to continue to mount in degraded mode if the device is already scanned, which is required to recover from the split brain raid

[PATCH 0/2] Add cli and ioctl to ignore a scanned device

Patch 1/2 is preparatory patch to get a function to delete a device from the list. Patch 2/2 adds the ioctl and feature to delete a device from the device list. Anand Jain (2): btrfs: refactor btrfs_free_stale_device() to get device list delete btrfs: introduce feature to ignore a btrfs

Re: [PATCH v4] btrfs: Remove received_uuid during received snapshot ro->rw switch

On 10/05/2017 10:22 AM, Nikolay Borisov wrote: > Currently when a read-only snapshot is received and subsequently its ro > property > is set to false i.e. switched to rw-mode the received_uuid of that subvol > remains > intact. However, once the received volume is switched to RW mode we cannot >

Re: btrfs-image hash collision option, super slow

>>It could definitely be improved -- I believe there are some good >> (but non-trivial) algorithms for finding preimages for CRC32 checksums >> out there. It's just that btrfs-image doesn't use them. I implemented a faster method using a reverse CRC32 table, which is in btrfs-progs since

[PATCH] btrfs: handle dynamically reappearing missing device

If the device is not present at the time of (-o degrade) mount the mount context will create a dummy missing struct btrfs_device. Later this device may reappear after the FS is mounted. So this patch handles that case by going through the open_device steps which this device missed and finally adds

Re: Theoretical Question about commit=n

On 11/12/2017 09:58 PM, Robert White wrote: > Is the commit interval monotonic, or is it seconds after sync? > > What I mean is that if I manually call sync(2) does the commit timer > reset? I'm thinking it does not, but I can imagine a workload where it > ideally would. The magic happens inside

Theoretical Question about commit=n

Is the commit interval monotonic, or is it seconds after sync? What I mean is that if I manually call sync(2) does the commit timer reset? I'm thinking it does not, but I can imagine a workload where it ideally would. (Again, this is purely theoretical, I have no such workload as I am about to