Re: [PATCH] Btrfs : send, truncate first to enhance many small files

2017-12-03 Thread Qu Wenruo
On 2017年12月04日 15:02, robbieko wrote: > From: Robbie Ko > > The commands generated by send contain the following step: > 1. mkfile o1851-19-0 > 2. rename o1851-19-0 -> alsa-driver/alsa-kernel/isa/es1688/es1688.c > 3. set_xattr alsa-driver/alsa-kernel/isa/es1688/es1688.c

[PATCH] btrfs: fix inconsistency during missing device rejoin

2017-12-03 Thread Anand Jain
When device is missing its not necessary that btrfs_device::name is null or the path is different when it reappears. Its possible that device can go missing after its been scanned where neither of btrfs_device::name == NULL OR btrfs_device::name != reappear_dev_path, is true. So just check for

[PATCH v3] btrfs: handle dynamically reappearing missing device

2017-12-03 Thread Anand Jain
If the device is not present at the time of (-o degrade) mount, the mount context will create a dummy missing struct btrfs_device. Later this device may reappear after the FS is mounted and then device is included in the device list but it missed the open_device part. So this patch handles that

Re: [PATCH v2] btrfs: handle dynamically reappearing missing device

2017-12-03 Thread Anand Jain
[..] would be better an explicit user intervention instead of an automatic one ? What is the user intervention method steps that you have in mind ? Just curious. Pls remember downtime is not a choice of recovery from this context which means FS should be available for the applications

[PATCH] Btrfs : send, truncate first to enhance many small files

2017-12-03 Thread robbieko
From: Robbie Ko The commands generated by send contain the following step: 1. mkfile o1851-19-0 2. rename o1851-19-0 -> alsa-driver/alsa-kernel/isa/es1688/es1688.c 3. set_xattr alsa-driver/alsa-kernel/isa/es1688/es1688.c - name=user.xattr data_len=4 data=test 4. write

Re: exclusive subvolume space missing

2017-12-03 Thread Chris Murphy
On Sun, Dec 3, 2017 at 3:47 AM, Adam Borowski wrote: > I'd say that the only good use for nocow is "I wish I have placed this file > on a non-btrfs, but it'd be too much hassle to repartition". > > If you snapshot nocow at all, you get the worst of both worlds. I think it's

Re: exclusive subvolume space missing

2017-12-03 Thread Chris Murphy
On Fri, Dec 1, 2017 at 5:53 PM, Tomasz Pala wrote: > # btrfs fi usage / > Overall: > Device size: 128.00GiB > Device allocated:117.19GiB > Device unallocated: 10.81GiB > Device missing: 0.00B > Used:

[PATCH v2 4/5] btrfs: cleanup device states define BTRFS_DEV_STATE_REPLACE_TGT

2017-12-03 Thread Anand Jain
Currently device state is being managed by each individual int variable such as struct btrfs_device::is_tgtdev_for_dev_replace. Instead of that declare btrfs_device::dev_state BTRFS_DEV_STATE_MISSING and use the bit operations. Signed-off-by: Anand Jain ---

[PATCH v2 3/5] btrfs: cleanup device states define BTRFS_DEV_STATE_MISSING

2017-12-03 Thread Anand Jain
Currently device state is being managed by each individual int variable such as struct btrfs_device::missing. Instead of that declare btrfs_device::dev_state BTRFS_DEV_STATE_MISSING and use the bit operations. Signed-off-by: Anand Jain Reviewed-by : Nikolay Borisov

[PATCH v2 5/5] btrfs: cleanup device states define BTRFS_DEV_STATE_FLUSH_SENT

2017-12-03 Thread Anand Jain
Currently device state is being managed by each individual int variable such as struct btrfs_device::is_tgtdev_for_dev_replace. Instead of that declare btrfs_device::dev_state BTRFS_DEV_STATE_FLUSH_SENT and use the bit operations. Signed-off-by: Anand Jain ---

[PATCH v2 1/5] btrfs: cleanup device states define BTRFS_DEV_STATE_WRITEABLE

2017-12-03 Thread Anand Jain
Currently device state is being managed by each individual int variable such as struct btrfs_device::writeable. Instead of that declare device state BTRFS_DEV_STATE_WRITEABLE and use the bit operations. Signed-off-by: Anand Jain --- v2: Remove a unrelated change. Start

[PATCH v2 0/5] define BTRFS_DEV_STATE

2017-12-03 Thread Anand Jain
As of now device properties and states are being represented as int variable. So clean that up using bitwise operations. Also patches in the ML such as device failed state needs this cleanup as well. V2: Accepts all comments from Nikolay. Drops can_discard. Adds BTRFS_DEV_STATE_REPLACE_TGT and

[PATCH v2 2/5] btrfs: cleanup device states define BTRFS_DEV_STATE_IN_FS_METADATA

2017-12-03 Thread Anand Jain
Currently device state is being managed by each individual int variable such as struct btrfs_device::in_fs_metadata. Instead of that declare device state BTRFS_DEV_STATE_IN_FS_METADATA and use the bit operations. Signed-off-by: Anand Jain Reviewed-by: Nikolay Borisov

Re: [PATCH 5/5] btrfs: Greatly simplify btrfs_read_dev_super

2017-12-03 Thread Anand Jain
On 12/03/2017 05:43 PM, Nikolay Borisov wrote: On 2.12.2017 01:23, Anand Jain wrote: On 12/01/2017 05:19 PM, Nikolay Borisov wrote: Currently this function executes the inner loop at most 1 due to the i = 0; i < 1 condition. Furthermore, the btrfs_super_generation(super) > transid code

Re: exclusive subvolume space missing

2017-12-03 Thread Qu Wenruo
On 2017年12月02日 17:33, Tomasz Pala wrote: > OK, I seriously need to address that, as during the night I lost > 3 GB again: > > On Sat, Dec 02, 2017 at 10:35:12 +0800, Qu Wenruo wrote: > >>> # btrfs fi sh / >>> Label: none uuid: 17a3de25-6e26-4b0b-9665-ac267f6f6c4a >>> Total devices 2

[PATCH v2] Btrfs: heuristic replace heap sort with radix sort

2017-12-03 Thread Timofey Titovets
Slowest part of heuristic for now is kernel heap sort() It's can take up to 55% of runtime on sorting bucket items. As sorting will always call on most data sets to get correctly byte_core_set_size, the only way to speed up heuristic, is to speed up sort on bucket. Add a general radix_sort

Re: [PATCH 0/7] retry write on error

2017-12-03 Thread Peter Grandi
> [ ... ] btrfs incorporates disk management which is actually a > version of md layer, [ ... ] As far as I know Btrfs has no disk management, and was wisely designed without any, just like MD: Btrfs volumes and MD sets can be composed from "block devices", not disks, and block devices are quite

Re: again "out of space" and remount read only, with 4.14

2017-12-03 Thread Martin Raiber
Am 26.11.2017 um 17:02 schrieb Tomasz Chmielewski: > On 2017-11-27 00:37, Martin Raiber wrote: >> On 26.11.2017 08:46 Tomasz Chmielewski wrote: >>> Got this one on a 4.14-rc7 filesystem with some 400 GB left: >> I guess it is too late now, but I guess the "btrfs fi usage" output of >> the file

Re: exclusive subvolume space missing

2017-12-03 Thread Adam Borowski
On Sun, Dec 03, 2017 at 01:45:45AM +, Duncan wrote: > Tomasz Pala posted on Sat, 02 Dec 2017 18:18:19 +0100 as excerpted: > >> I got ~500 small files (100-500 kB) updated partially in regular > >> intervals: > >> > >> # du -Lc **/*.rrd | tail -n1 > >> 105Mtotal > > FWIW, I've no idea

Re: [PATCH 5/5] btrfs: Greatly simplify btrfs_read_dev_super

2017-12-03 Thread Nikolay Borisov
On 2.12.2017 01:23, Anand Jain wrote: > > > On 12/01/2017 05:19 PM, Nikolay Borisov wrote: >> Currently this function executes the inner loop at most 1 due to the i >> = 0; >> i < 1 condition. Furthermore, the btrfs_super_generation(super) > >> transid code >> in the if condition is never