Re: [PATCH] Btrfs: incremental send, fix infinite loop when apply children dir moves

robbieko 於 2018-11-06 20:23 寫到: Hi, I can reproduce the infinite loop, the following will describe the reason and example. Example: tree --inodes parent/ send/ parent/ `-- [261] 261 `-- [271] 271 `-- [266] 266 |-- [259] 259 |-- [260

Re: [PATCH] Btrfs: incremental send, fix infinite loop when apply children dir moves

as not removed from pending_dir_moves. Then we create a new pending_dir and join the ino 259, because the ino 259 is currently in the stack, the new pending_dir ino 266 is also indirectly added to the stack, placed between 267 and 259. So we fix this problem by remove node from pending_dir_

[PATCH] Btrfs: fix cur_offset in the error case for nocow

From: Robbie Ko When the cow_file_range fail, the related resources are unlocked according to the range (start-end), so the unlock cannot be repeated in run_delalloc_nocow. In some cases (e.g. cur_offset <= end && cow_start!= -1), cur_offset is not updated correctly, so move the cur_offset

[PATCH] Btrfs: incremental send, fix infinite loop when apply children dir moves

From: Robbie Ko In apply_children_dir_moves, we first create an empty list (stack), then we get an entry from pending_dir_moves and add it to the stack, but we didn't delete the entry from rb_tree. So, in add_pending_dir_move, we create a new entry and then use the parent_ino in the current

[PATCH] btrfs-progs: super-recover: fix double free fs_devices memory

From: Robbie Ko super-recover collects btrfs devices information using existed functions scan_one_devices(). Problem is fs_devices is freed twice. One in __open_ctree_fd() when error happens and the other in btrfs_close_devices(recover.fs_devices) when root is NULL. Commit

Re: [PATCH] Btrfs: optimization to avoid ENOSPC for nocow writes after snapshot when low on data space

Filipe Manana 於 2018-08-03 18:22 寫到: On Fri, Aug 3, 2018 at 10:13 AM, robbieko wrote: From: Robbie Ko Commit e9894fd3e3b3 ("Btrfs: fix snapshot vs nocow writting") forced writeback fallback to COW when subvolume is snapshotted. Commit e9894fd3e3b3 ("Btrfs: fix snapshot vs

[PATCH] Btrfs: fix unexpected failure of nocow buffered writes after snapshotting when low on space

From: Robbie Ko Commit e9894fd3e3b3 ("Btrfs: fix snapshot vs nocow writting") forced nocow writes to fallback to COW, during writeback, when a snapshot is created. This resulted in writes made before creating the snapshot to unexpectedly fail with ENOSPC during writeback when success (0) was

Re: [PATCH] Btrfs: fix data lose with snapshot when nospace

Filipe Manana 於 2018-08-02 01:55 寫到: On Wed, Aug 1, 2018 at 1:54 PM, Filipe Manana wrote: On Wed, Aug 1, 2018 at 11:20 AM, robbieko wrote: Filipe Manana 於 2018-07-31 19:33 寫到: On Tue, Jul 31, 2018 at 11:17 AM, robbieko wrote: Filipe Manana 於 2018-07-30 20:34 寫到: On Mon, Jul 30, 2018

[PATCH] Btrfs: optimization to avoid ENOSPC for nocow writes after snapshot when low on data space

From: Robbie Ko Commit e9894fd3e3b3 ("Btrfs: fix snapshot vs nocow writting") forced writeback fallback to COW when subvolume is snapshotted. 1. When the space is full, write syscall will check if can nocow, and space reservation will not happen. 2. Then snapshot happens before flushing IO

Re: [PATCH] Btrfs: fix data lose with snapshot when nospace

Filipe Manana 於 2018-08-02 01:55 寫到: On Wed, Aug 1, 2018 at 1:54 PM, Filipe Manana wrote: On Wed, Aug 1, 2018 at 11:20 AM, robbieko wrote: Filipe Manana 於 2018-07-31 19:33 寫到: On Tue, Jul 31, 2018 at 11:17 AM, robbieko wrote: Filipe Manana 於 2018-07-30 20:34 寫到: On Mon, Jul 30, 2018

Re: [PATCH] Btrfs: fix data lose with snapshot when nospace

Filipe Manana 於 2018-07-31 19:33 寫到: On Tue, Jul 31, 2018 at 11:17 AM, robbieko wrote: Filipe Manana 於 2018-07-30 20:34 寫到: On Mon, Jul 30, 2018 at 12:28 PM, Filipe Manana wrote: On Mon, Jul 30, 2018 at 12:08 PM, Filipe Manana wrote: On Mon, Jul 30, 2018 at 11:21 AM, robbieko wrote

Re: [PATCH] Btrfs: fix data lose with snapshot when nospace

Filipe Manana 於 2018-07-30 20:34 寫到: On Mon, Jul 30, 2018 at 12:28 PM, Filipe Manana wrote: On Mon, Jul 30, 2018 at 12:08 PM, Filipe Manana wrote: On Mon, Jul 30, 2018 at 11:21 AM, robbieko wrote: From: Robbie Ko Commit e9894fd3e3b3 ("Btrfs: fix snapshot vs nocow writting"

[PATCH] Btrfs: fix data lose with snapshot when nospace

From: Robbie Ko Commit e9894fd3e3b3 ("Btrfs: fix snapshot vs nocow writting") modified the nocow writeback mechanism, if you create a snapshot, it will always switch to cow writeback. This will cause data loss when there is no space, because when the space is full, the write will not reserve

Re: [PATCH] Btrfs: fix physical offset reported by fiemap for inline extents

fdman...@kernel.org 於 2018-06-19 19:31 寫到: From: Filipe Manana Commit 9d311e11fc1f ("Btrfs: fiemap: pass correct bytenr when fm_extent_count is zero") introduced a regression where we no longer report 0 as the physical offset for inline extents. This is because it always sets the variable used

Re: [PATCH] Btrfs: implement unlocked buffered write

Chris Mason 於 2018-05-23 23:56 寫到: On 23 May 2018, at 3:26, robbieko wrote: Chris Mason 於 2018-05-23 02:31 寫到: On 22 May 2018, at 14:08, Christoph Hellwig wrote: On Wed, May 16, 2018 at 11:52:37AM +0800, robbieko wrote: From: Robbie Ko <robbi...@synology.com> This idea is from dir

Re: [PATCH] Btrfs: implement unlocked buffered write

Chris Mason 於 2018-05-23 02:31 寫到: On 22 May 2018, at 14:08, Christoph Hellwig wrote: On Wed, May 16, 2018 at 11:52:37AM +0800, robbieko wrote: From: Robbie Ko <robbi...@synology.com> This idea is from direct io. By this patch, we can make the buffered write parallel, and i

Re: [PATCH] Btrfs: implement unlocked buffered write

Omar Sandoval 於 2018-05-23 01:28 寫到: On Wed, May 16, 2018 at 11:52:37AM +0800, robbieko wrote: From: Robbie Ko <robbi...@synology.com> This idea is from direct io. By this patch, we can make the buffered write parallel, and improve the performance and latency. But because we can not

[PATCH] Btrfs: implement unlocked buffered write

From: Robbie Ko This idea is from direct io. By this patch, we can make the buffered write parallel, and improve the performance and latency. But because we can not update isize without i_mutex, the unlocked buffered write just can be done in front of the EOF. We needn't

[PATCH v3] btrfs: incremental send, fix BUG when invalid memory access

From: Robbie Ko [BUG] btrfs incremental send BUG happens when creating a snapshot of snapshot that is being used by send. [REASON] The problem can happen if while we are doing a send one of the snapshots used (parent or send) is snapshotted, because snapshoting implies

Re: [PATCH v2] btrfs: incremental send, fix BUG when invalid memory access

Filipe Manana 於 2018-05-11 17:59 寫到: On Fri, May 11, 2018 at 7:34 AM, robbieko <robbi...@synology.com> wrote: From: Robbie Ko <robbi...@synology.com> [BUG] btrfs incremental send BUG happens when creating a snapshot of snapshot that is being used by send. [REASON] The proble

[PATCH v2] btrfs: incremental send, fix BUG when invalid memory access

From: Robbie Ko [BUG] btrfs incremental send BUG happens when creating a snapshot of snapshot that is being used by send. [REASON] The problem can happen if while we are doing a send one of the snapshots used (parent or send) is snapshotted, because snapshoting implies

Re: [PATCH] btrfs: fix invalid memory access with journal_info

Omar Sandoval 於 2018-05-10 12:53 寫到: On Wed, May 09, 2018 at 06:35:25PM +0800, robbieko wrote: From: Robbie Ko <robbi...@synology.com> When send process requires memory allocation, shrinker may be triggered due to insufficient memory. Then evict_inode gets called when inode is

[PATCH] btrfs: fix invalid memory access with journal_info

From: Robbie Ko When send process requires memory allocation, shrinker may be triggered due to insufficient memory. Then evict_inode gets called when inode is freed, and this function may need to start transaction. However, the journal_info is already points to

Re: [PATCH] btrfs: incremental send, fix BUG when parent commit_root changed

Hi Filipe Manana, Ok. I will add all this information, in detail, to the changelog, and than send a patch V2 later. Thanks. Robbie Ko Filipe Manana 於 2018-05-09 16:29 寫到: On Wed, May 9, 2018 at 2:10 AM, robbieko <robbi...@synology.com> wrote: Filipe Manana 於 2018-05-08 19:12 寫到:

Re: [PATCH] btrfs: incremental send, fix BUG when parent commit_root changed

Filipe Manana 於 2018-05-08 19:12 寫到: On Tue, May 8, 2018 at 11:30 AM, robbieko <robbi...@synology.com> wrote: Hi Filipe Manana, Although the snapshot is readonly, when the snapshot is created, in order to modify the last_snapshot, it will cause generation changes, and it will

Re: [PATCH] btrfs: incremental send, fix BUG when parent commit_root changed

will be updated. This means that as long as the send or parent snapshot is taken during the btrfs send, commit_root will change, and send process may access invalid memory. Thanks. Robbie Ko Filipe Manana 於 2018-05-08 17:15 寫到: On Tue, May 8, 2018 at 9:15 AM, robbieko <robbi...@synology.com> wrote:

[PATCH v2 1/2] btrfs: incremental send, optimization add orphan_dir_info

From: Robbie Ko moving the allocation to the end in order to avoid unnecessary allocations Signed-off-by: Robbie Ko --- fs/btrfs/send.c | 13 ++--- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/send.c

[PATCH v2 2/2] btrfs: incremental send, improve rmdir performance for large directory

From: Robbie Ko Currently when checking if we can delete a directory, we always check if all its children have been processed. Example: A directory with 2,000,000 files was deleted Result: original : 1994m57.071s patch : 1m38.554s [FIX] Instead of checking all children

[PATCH v2 0/2] btrfs :incremental send, improve rmdir performance

From: Robbie Ko The following patch is to improve the btrfs send, the speed of large directory deletion. 1. Optimization, avoid unnecessary allocations. 2. Increase the speed of can_rmdir. Robbie Ko (2): btrfs: incremental send, optimization add orphan_dir_info

Re: [PATCH] btrfs: send, improve rmdir performance for large directory

Hi Manana, I will fix and modify and then send patch V2. Thanks. Robbie Ko Filipe Manana 於 2018-05-08 17:10 寫到: On Tue, May 8, 2018 at 3:50 AM, robbieko <robbi...@synology.com> wrote: From: Robbie Ko <robbi...@synology.com> At present, we check if the directory can be deleted.

Re: [PATCH] btrfs: incremental send, fix BUG when parent commit_root changed

Hi Manana, I will fix and modify and then send patch V2. Thanks. Robbie Ko Filipe Manana 於 2018-05-08 17:15 寫到: On Tue, May 8, 2018 at 9:15 AM, robbieko <robbi...@synology.com> wrote: From: Robbie Ko <robbi...@synology.com> [BUG] btrfs send BUG on parent commit_root

[PATCH] btrfs: incremental send, fix BUG when parent commit_root changed

From: Robbie Ko [BUG] btrfs send BUG on parent commit_root node receive destination is at the same volume. [Example] btrfs send -e -p /mnt/snap1/ /mnt/snap2/ | btrfs receive -e /mnt/temp [REASON] 1. When send with the parent, the send process will get the parent

[PATCH] btrfs: send, improve rmdir performance for large directory

From: Robbie Ko At present, we check if the directory can be deleted. We need to check whether all the files under this directory have been processed every time we finished processing an entry under that directory. Example: 2,000,000 files in 1 directory. Parent snapshot:

[PATCH v2 0/2] btrfs fiemap related BUG fix.

From: Robbie Ko This patchset intends to fix btrfs fiemap related bug. The fiemap has the following problems: 1) fiemap: pass correct bytenr when fm_extent_count is zero When user space wants to get the number of file extents, set fm_extent_count to 0 to run fiemap and

[PATCH v2 1/2] Btrfs: fiemap: pass correct bytenr when fm_extent_count is zero

From: Robbie Ko [BUG] fm_mapped_extents is not correct when fm_extent_count is 0 Like: # mount /dev/vdb5 /mnt/btrfs # dd if=/dev/zero bs=16K count=4 oflag=dsync of=/mnt/btrfs/file # xfs_io -c "fiemap -v" /mnt/btrfs/file /mnt/btrfs/file: EXT: FILE-OFFSET

[PATCH v2 2/2] Btrfs: fix fiemap extent SHARED flag error with range clone.

From: Robbie Ko [BUG] Range clone can cause fiemap to return error result. Like: # mount /dev/vdb5 /mnt/btrfs # dd if=/dev/zero bs=16K count=2 oflag=dsync of=/mnt/btrfs/file # btrfs-debugfs -f file (276 0): ram 16384 disk 2171609088 disk_size 16384 (276 16384): ram

Problem with current->journal_info for BTRFS, Null pointer dereference in start_transaction

Hi, When send process requires memory allocation, shrinker may be triggered due to insufficient memory. Then evict_inode gets called when inode is dropped, and this function may need to start transaction. However, the journal_info is already points to BTRFS_SEND_TRANS_STUB, it passed the if

Re: [PATCH 1/2] Btrfs: fiemap: pass correct bytenr when fm_extent_count is zero

Nikolay Borisov 於 2018-03-07 19:15 寫到: On 7.03.2018 12:27, robbieko wrote: Nikolay Borisov 於 2018-03-07 18:19 寫到: On  7.03.2018 10:20, robbieko wrote: From: Robbie Ko <robbi...@synology.com>  # mount /dev/vdb5 /mnt/btrfs  # dd if=/dev/zero bs=16K count=4 oflag=dsync of=/mnt/btrf

Re: [PATCH 2/2] Btrfs: fix fiemap extent SHARED flag error with range clone.

Qu Wenruo 於 2018-03-07 18:42 寫到: On 2018年03月07日 18:33, Qu Wenruo wrote: On 2018年03月07日 16:20, robbieko wrote: From: Robbie Ko <robbi...@synology.com> [BUG] Range clone can cause fiemap to return error result. Like: # mount /dev/vdb5 /mnt/btrfs # dd if=/dev/zero bs=16K count=2 oflag

Re: [PATCH 1/2] Btrfs: fiemap: pass correct bytenr when fm_extent_count is zero

Nikolay Borisov 於 2018-03-07 18:19 寫到: On 7.03.2018 10:20, robbieko wrote: From: Robbie Ko <robbi...@synology.com> # mount /dev/vdb5 /mnt/btrfs # dd if=/dev/zero bs=16K count=4 oflag=dsync of=/mnt/btrfs/file # xfs_io -c "fiemap -v" /mnt/btrfs/file /mnt/btrfs/file:

Re: [PATCH 0/2] btrfs fiemap related BUG fix.

Qu Wenruo 於 2018-03-07 17:27 寫到: On 2018年03月07日 16:20, robbieko wrote: From: Robbie Ko <robbi...@synology.com> This patchset intends to fix btrfs fiemap related bug. The fiemap has the following problems: 1) Wrong extent count when fm_extent_count is zero. 2) SHARED bit is not cor

[PATCH 0/2] btrfs fiemap related BUG fix.

From: Robbie Ko This patchset intends to fix btrfs fiemap related bug. The fiemap has the following problems: 1) Wrong extent count when fm_extent_count is zero. 2) SHARED bit is not correct I have two ideas, but I do not know which one is the best. Like: # dd

[PATCH 1/2] Btrfs: fiemap: pass correct bytenr when fm_extent_count is zero

From: Robbie Ko # mount /dev/vdb5 /mnt/btrfs # dd if=/dev/zero bs=16K count=4 oflag=dsync of=/mnt/btrfs/file # xfs_io -c "fiemap -v" /mnt/btrfs/file /mnt/btrfs/file: EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS 0: [0..127]:25088..25215 128 0x1

[PATCH 2/2] Btrfs: fix fiemap extent SHARED flag error with range clone.

From: Robbie Ko [BUG] Range clone can cause fiemap to return error result. Like: # mount /dev/vdb5 /mnt/btrfs # dd if=/dev/zero bs=16K count=2 oflag=dsync of=/mnt/btrfs/file # xfs_io -c "fiemap -v" /mnt/btrfs/file /mnt/btrfs/file: EXT: FILE-OFFSET BLOCK-RANGE

Re: Btrfs reserve metadata problem

to only 1MB dirty pages 3. cow enabled is also the same problem It is a serious performance issue. Thanks. robbieko p...@btrfs.list.sabi.co.uk 於 2018-01-02 21:08 寫到: When testing Btrfs with fio 4k random write, That's an exceptionally narrowly defined workload. Also it is narrower than

Btrfs reserve metadata problem

Hi All, When testing Btrfs with fio 4k random write, I found that volume with smaller free space available has lower performance. It seems that the smaller the free space of volume is, the smaller amount of dirty page filesystem could have. There are only 6 MB dirty pages when free space of

Re: [PATCH] Btrfs : send, truncate first to enhance many small files

: On Mon, Dec 4, 2017 at 11:28 AM, Qu Wenruo <quwenruo.bt...@gmx.com> wrote: On 2017年12月04日 19:13, Filipe Manana wrote: On Mon, Dec 4, 2017 at 7:19 AM, Qu Wenruo <quwenruo.bt...@gmx.com> wrote: On 2017年12月04日 15:02, robbieko wrote: From: Robbie Ko <robbi...@synology.com> The

[PATCH] Btrfs : send, truncate first to enhance many small files

From: Robbie Ko The commands generated by send contain the following step: 1. mkfile o1851-19-0 2. rename o1851-19-0 -> alsa-driver/alsa-kernel/isa/es1688/es1688.c 3. set_xattr alsa-driver/alsa-kernel/isa/es1688/es1688.c - name=user.xattr data_len=4 data=test 4. write

Re: [PATCH] btrfs: fix offset in test Btrfs delalloc accounting overflow

include Omar Sandoval. robbieko 於 2017-10-26 14:02 寫到: From: Robbie Ko <robbi...@synology.com> Found it when test btrfs delalloc accounting overflow, Fix offset error. We will fill in the gaps between the created extents, then outstanding extents will all be merged into 1. Sign

[PATCH] btrfs: fix offset in test Btrfs delalloc accounting overflow

From: Robbie Ko Found it when test btrfs delalloc accounting overflow, Fix offset error. We will fill in the gaps between the created extents, then outstanding extents will all be merged into 1. Signed-off-by: Robbie Ko --- tests/btrfs/010 | 2 +-

Extent map generation question

Hi, I have a question in the function "clone_update_extent_map". If em->generation is set -1, stands for we can't ignore it in incremental fsync. Why em->generation is set -1 when there is a path exists, and the generation is always -1. It can not set to trans-> transid ? Thanks. Robbie Ko

[PATCH v3 0/6] Btrfs: incremental send, fix serval case failure

From: Robbie Ko Patch for fix btrfs incremental send. These patches base on v4.8.0-rc8 V3: Improve the change log V2: Add a new patch "add generation check in existence demtermination for the parent directory" Robbie Ko (6): Btrfs: incremental send, fix failure to

[PATCH v3 2/6] Btrfs: incremental send, fix invalid path for truncate operations

From: Robbie Ko Under certain situations, an incremental send operation can a truncate operation that will make the receiving end fail when attempting to execute it, because the path is not exist. Example scenario: Parent snapshot: | dir258/ (ino 258, gen 15, dir)

[PATCH v3 1/6] Btrfs: incremental send, fix failure to rename with the name collision

From: Robbie Ko Under certain situations, an incremental send operation can a rename operation that will make the receiving end fail when attempting to execute it, because the target is exist. Example scenario: Parent snapshot: |.(ino 256, gen 5) | a1/

[PATCH v3 4/6] Btrfs: incremental send, fix invalid path for rmdir operations

From: Robbie Ko Under certain situations, an incremental send operation can a rmdir operation that will make the receiving end fail when attempting to execute it, because the path is not exist. Example scenario: Parent snapshot: | d259_old/ (ino 259, gen 96)

[PATCH v3 5/6] Btrfs: incremental send, fix invalid rename operations

From: Robbie Ko Under certain situations, an incremental send operation can a rename operation that will make the receiving end fail when attempting to execute it, because the source has been deleted. Example scenario: Parent snapshot: | d1/ (ino 257, gen

[PATCH v3 3/6] Btrfs: incremental send, fix not necessary waiting for rmdir operation

From: Robbie Ko Under certain situations, an incremental send operation can delay rmdir operation when processing inode 258, but it is not necessary, because dir258 is empty. Example scenario: Parent snapshot: | dir258/ (ino 258, gen 27) | dir257/ (ino 257,

[PATCH v3 6/6] Btrfs: incremental send, fix invalid utime operations

From: Robbie Ko Under certain situations, an incremental send operation can a utime operation that will make the receiving end fail when attempting to execute it, because the path has been deleted. Exampla scenario: Parent snapshot: | dir258/ (ino 258, gen 7,

Re: [PATCH v3 4/4] xfstests: btrfs/134: add test for incremental send which renames a directory already being deleted

Filipe Manana 於 2017-01-04 21:07 寫到: On Wed, Jan 4, 2017 at 10:53 AM, robbieko <robbi...@synology.com> wrote: From: Robbie Ko <robbi...@synology.com> Test that an incremental send operation dosen't work because dosen't -> doesn't it tries to rename a directory which is

Re: [PATCH v3 2/4] xfstests: btrfs/132: add test for invaild update time by an incremental send

Filipe Manana 於 2017-01-04 21:09 寫到: On Wed, Jan 4, 2017 at 10:53 AM, robbieko <robbi...@synology.com> wrote: From: Robbie Ko <robbi...@synology.com> Test that an incremental send operation dosen't' work because it tries to update the time to a deleted directory after it fin

[PATCH v3 4/4] xfstests: btrfs/134: add test for incremental send which renames a directory already being deleted

From: Robbie Ko Test that an incremental send operation dosen't work because it tries to rename a directory which is already deleted. This test exercises scenarios used to fail in btrfs and are fixed by the following patch for the linux kernel: "Btrfs: incremental send,

[PATCH v3 2/4] xfstests: btrfs/132: add test for invaild update time by an incremental send

From: Robbie Ko Test that an incremental send operation dosen't' work because it tries to update the time to a deleted directory after it finishes a move operation. The other one is that an operation is applied to a file using the old name not the new name. This test

[PATCH v3 1/4] xfstests: btrfs/131: add test for an incremental send with name collision

From: Robbie Ko Test that an incremental send operation doesn't work because there's a name collision in the destination and it's not checked corretly before the rename operation applies. This test exercises scenarios used to fail in btrfs and are fixed by the following

[PATCH v3 3/4] xfstests: btrfs/133: add test for incremental send with rmdir applied on wrong name

From: Robbie Ko Test that an incremental send operation doesn't work for rmdir because it uses the wrong name to delete. This test exercises scenarios used to fail in btrfs and are fixed by the following patch for the linux kernel: "Btrfs: incremental send, skip check

[PATCH v3 0/4] Btrfs: add serval test case for incremental send

From: Robbie Ko Patch for test btrfs incremental send. V3: remove "run_" based helpers V2: improve the change log Robbie Ko (4): xfstests: btrfs/131: add test for an incremental send with name collision xfstests: btrfs/132: add test for invaild update time by an

Re: [PATCH] Btrfs: fix infinite loop when tree log recovery

Hi Filipe, Thank you for your help. I will make up the incremental send change log as soon as possible. Thanks. robbieko Filipe Manana 於 2016-12-01 19:14 寫到: On Thu, Dec 1, 2016 at 1:42 AM, robbieko <robbi...@synology.com> wrote: Hi Filipe, Thank you for your review. I have see

Re: [PATCH] Btrfs: fix infinite loop when tree log recovery

? modify patch change log and then send again ? Thanks. robbieko Filipe Manana 於 2016-12-01 00:53 寫到: On Fri, Oct 7, 2016 at 10:30 AM, robbieko <robbi...@synology.com> wrote: From: Robbie Ko <robbi...@synology.com> if log tree like below: leaf N: ... item 240 key (282 DI

Re: [PATCH v2 4/4] xfstests: btrfs/134: add test for incremental send which renames a directory already being deleted

ri, Oct 28, 2016 at 09:44:06AM +0800, robbieko wrote: From: Robbie Ko <robbi...@synology.com> Test that an incremental send operation dosen't work because it tries to rename a directory which is already deleted. This test exercises scenarios used to fail in btrfs and are fixed by the

Re: [PATCH v2 1/4] xfstests: btrfs/131: add test for an incremental send with name collision

Hi Eryu Guan, OK, I will modify it. Thanks. Robbie Ko Eryu Guan 於 2016-11-01 15:17 寫到: On Fri, Oct 28, 2016 at 09:44:03AM +0800, robbieko wrote: From: Robbie Ko <robbi...@synology.com> Test that an incremental send operation doesn't work because there's a name collision in the desti

[PATCH] Btrfs: fix fsync deadlock in log_new_dir_dentries

From: Robbie Ko We found a fsync deadlock in log_new_dir_dentries, because btrfs_search_forward get path lock, then call btrfs_iget will get another extent_buffer lock, maybe occur deadlock. Fix this by release path before call btrfs_iget, avoid deadlock occur. Example:

[PATCH v2] Btrfs: fix enospc in hole punching

From: Robbie Ko The hole punching can result in adding new leafs (and as a consequence new nodes) to the tree because when we find file extent items that span beyond the hole range we may end up not deleting them (just adjusting them) and add new file extent items

[PATCH v2 3/4] xfstests: btrfs/133: add test for incremental send with rmdir applied on wrong name

From: Robbie Ko Test that an incremental send operation doesn't work for rmdir because it uses the wrong name to delete. This test exercises scenarios used to fail in btrfs and are fixed by the following patch for the linux kernel: "Btrfs: incremental send, skip check

[PATCH v2 0/4] Btrfs: add serval test case for incremental send

From: Robbie Ko Patch for test btrfs incremental send. v2: * Improve the change log Robbie Ko (4): xfstests: btrfs/131: add test for an incremental send with name collision xfstests: btrfs/132: add test for invaild update time by an incremental send

[PATCH v2 4/4] xfstests: btrfs/134: add test for incremental send which renames a directory already being deleted

From: Robbie Ko Test that an incremental send operation dosen't work because it tries to rename a directory which is already deleted. This test exercises scenarios used to fail in btrfs and are fixed by the following patch for the linux kernel: "Btrfs: incremental send,

[PATCH v2 1/4] xfstests: btrfs/131: add test for an incremental send with name collision

From: Robbie Ko Test that an incremental send operation doesn't work because there's a name collision in the destination and it's not checked corretly before the rename operation applies. This test exercises scenarios used to fail in btrfs and are fixed by the following

[PATCH v2 2/4] xfstests: btrfs/132: add test for invaild update time by an incremental send

From: Robbie Ko Test that an incremental send operation dosen't' work because it tries to update the time to a deleted directory after it finishes a move operation. The other one is that an operation is applied to a file using the old name not the new name. This test

[PATCH v2 5/6] Btrfs: incremental send, add generation check for inode is waiting for move.

From: Robbie Ko Example scenario: Parent snapshot: | d1/ (ino 257, gen 44) | d4/ (ino 258, gen 44) | d3/ (ino 259, gen 44) Send snapshot: | d1/ (ino 258, gen 47) | d4/ (ino 260, gen 47)

[PATCH v2 0/6] Btrfs: incremental send, fix serval case for inode 256 and generation

From: Robbie Ko Patch for fix btrfs incremental send. These patches base on v4.8.0-rc8 v2: * Improve the change log * Add a new patch "add generation check in existence demtermination for the parent directory" Robbie Ko (6): Btrfs: incremental send, do not skip

[PATCH v2 2/6] Btrfs: incremental send, add generation check for the inode waiting for rmdir operation.

From: Robbie Ko Example scenario: Parent snapshot: | dir258/ (ino 258, gen 15, dir) |--- dir257/ (ino 257, gen 15, dir) | dir259/ (ino 259, gen 15, dir) Send snapshot: | file258 (ino 258, gen 21, file) | new_dir259/ (ino 259, gen 21, dir) |---

[PATCH v2 4/6] Btrfs: incremental send, skip check overwritten if parents' generation are different

From: Robbie Ko Example scenario: Parent snapshot: | d259_old/ (ino 259, gen 96) | d1/ (ino 258, gen 96) | f (ino 257, gen 96) Send snapshot: | d258/ (ino 258, gen 98) | d259/ (ino 259, gen

[PATCH v2 6/6] Btrfs: incremental send, add generation check in existence demtermination for the parent directory

From: Robbie Ko Exampla scenario: Parent snapshot: | dir258/ (ino 258, gen 7, dir) |--- dir257/ (ino 257, gen 7, dir) | dir259/ (ino 259, gen 7, dir) Send snapshot: | file258 (ino 258, gen 10, file) | new_dir259/ (ino

[PATCH v2 3/6] Btrfs: incremental send, add generation for waiting_dir_move struct and check it in can_rmdir.

From: Robbie Ko Example scenario: Parent snapshot: | dir258/ (ino 258, gen 27) | dir257/ (ino 257, gen 27) | dir259/ (ino 259, gen 27) Send snapshot: | new_dir259/ (ino 259, gen 38) | new_dir258/ (ino 258, gen 38) | new_dir257/ (ino

[PATCH v2 1/6] Btrfs: incremental send, do not skip generation inconsistency check for inode 256.

From: Robbie Ko Example scenario: Parent snapshot: |.(ino 256, gen 5) | a1/ (ino 257, gen 5) | a2/ (ino 258, gen 5) Send snapshot: |.(ino 256, gen 7) | a2 (ino 257, gen 7) rmdir a1 mkfile o257-7-0

[PATCH 1/4] btrfs: add test for an incremental send don't skip overwrite ref for inode 256

From: Robbie Ko Test that an incremental send operation does not skip overwrite ref for inode 256, it's always exist. This test exercises scenarios used to fail in btrfs and are fixed by the following patches for the linux kernel: "Btrfs: incremental send, fix don't skip

[PATCH 4/4] btrfs: add test for an incremental send add gen check in will_overwrite_ref

From: Robbie Ko There a some case for overwrite ref, because when new gen is waiting, and old name conflict, but old name has been deleted. This test exercises scenarios used to fail in btrfs and are fixed by the following patches for the linux kernel: "Btrfs:

[PATCH 3/4] btrfs: add test for an incremental send add gen check in did_overwrite_ref

From: Robbie Ko There a some case, the old gen directory can not have new data to write, can not cause overwitre ref, except inode 256. This test exercises scenarios used to fail in btrfs and are fixed by the following patches for the linux kernel: "Btrfs: incremental

[PATCH 0/4] Btrfs: add serval test case for incremental send

From: Robbie Ko Patch for test btrfs incremental send. Robbie Ko (4): btrfs: add test for an incremental send don't skip overwrite ref for inode 256 btrfs: add test for an incremental send add gen for is_waiting_for_rm when some corner case btrfs: add test

[PATCH 2/4] btrfs: add test for an incremental send add gen for is_waiting_for_rm when some corner case

From: Robbie Ko There a one case for old_gen waiting_for_rm, but new_gen use get_cur_path with the same inode. This test exercises scenarios used to fail in btrfs and are fixed by the following patches for the linux kernel: "Btrfs: incremental send, add gen for

[PATCH 4/5] Btrfs: incremental send, add gen check in did_overwrite_ref

From: Robbie Ko There a some case similar as before. add check parent generation in the did_overwrite_ref. Signed-off-by: Robbie Ko --- fs/btrfs/send.c | 13 + 1 file changed, 13 insertions(+) diff --git a/fs/btrfs/send.c

[PATCH 2/5] Btrfs: incremental send, add gen for is_waiting_for_rm when some corner case

From: Robbie Ko There a one case for old_gen waiting_for_rm, but new_gen use get_cur_path with the same inode. Example: Parent snapshot: | dir258/ (ino 258, dir) |--- dir257/ (ino 257, dir) | dir259/ (ino 259, dir) Send snapshot: |

[PATCH 3/5] Btrfs: incremental send, add gen in waiting_dir_move for some corner case

From: Robbie Ko There a some case similar as before. add compare generation with dir items, and waiting_dir_move in the can_rmdir. Signed-off-by: Robbie Ko --- fs/btrfs/send.c | 14 +++--- 1 file changed, 11 insertions(+), 3 deletions(-)

[PATCH 5/5] Btrfs: incremental send, add gen check if has waiting_dir_move in the will_overwrite_ref

From: Robbie Ko There a some case similar as before. add check generation if has waiting_dir_move in the will_overwrite_ref Signed-off-by: Robbie Ko --- fs/btrfs/send.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git

[PATCH 1/5] Btrfs: incremental send, fix don't skip root inode in overwrite_ref

From: Robbie Ko When root dir item change, don't skip will_overwrite_ref, because root inode always exist. Example: Parent snapshot: | a1/ (ino 257, dir) | a2/ (ino 258, dir) Send snapshot: | a2 (ino 257, file) ERROR: rename o257-29-0 -> a2 failed: Is a

[PATCH 0/5] Btrfs: incremental send, fix serval case for root and gen

From: Robbie Ko Patch for fix btrfs incremental send. These patches base on v4.8.0-rc8 Robbie Ko (5): Btrfs: incremental send, fix don't skip root inode in overwrite_ref Btrfs: incremental send, add gen for is_waiting_for_rm when some corner case Btrfs:

[PATCH] Btrfs: fix -EINVEL in tree log recovering

From: Robbie Ko when tree log recovery, space_cache rebuild or dirty maybe save the cache. and then replay extent with disk_bytenr and disk_num_bytes, but disk_bytenr and disk_num_bytes maybe had been use for free space inode, will lead to -EINVEL. BTRFS: error in

Re: [PATCH] Btrfs: fix enospc in punch hole

, is 8 nodesize. but if extent_tree is greater level 4, worst case is need (leafsize + nodesize * 7) * 2, is bigger than resvered size, so we should use btrfs_calc_trans_metadata_size, is taken into account split leaf case. Thanks. robbieko Filipe Manana 於 2016-10-07 18:18 寫到: On Fri, Oct 7, 2016

Re: [PATCH] Btrfs: fix fsync deadlock in log_new_dir_dentries

8:46 寫到: On Fri, Oct 7, 2016 at 11:43 AM, robbieko <robbi...@synology.com> wrote: Hi Filipe, I am sorry, I express not clear enough. This number is pid, and the above are their call trace respectively. And why did you replace the continue statement with a break statement? Also pleas

[PATCH] Btrfs: fix fsync deadlock in log_new_dir_dentries

From: Robbie Ko We found a fsync deadlock ie. 32021->32020->32028->14431->14436->32021, in log_new_dir_dentries, because btrfs_search_forward get path lock, then call btrfs_iget will get another extent_buffer lock, maybe occur deadlock. we can release path before call

[PATCH] Btrfs: fix infinite loop when tree log recovery

From: Robbie Ko if log tree like below: leaf N: ... item 240 key (282 DIR_LOG_ITEM 0) itemoff 8189 itemsize 8 dir log end 1275809046 leaf N+1: item 0 key (282 DIR_LOG_ITEM 3936149215) itemoff 16275 itemsize 8 dir log

[PATCH] Btrfs: fix enospc in punch hole

From: Robbie Ko when extent-tree level > BTRFS_MAX_LEVEL / 2, __btrfs_drop_extents -> btrfs_duplicate_item -> setup_leaf_for_split -> split_leaf maybe enospc, because min_size is too small, need use btrfs_calc_trans_metadata_size. Signed-off-by: Robbie Ko

  1   2   >