Re: Btrfs progs pre-release 4.9-rc1

2016-12-15 Thread Qu Wenruo



At 12/15/2016 04:41 PM, Tsutomu Itoh wrote:

On 2016/12/15 15:45, Tsutomu Itoh wrote:

On 2016/12/14 23:42, David Sterba wrote:

Hi,

a pre-release has been tagged. Contains almost the entire devel branch from
today. There are small fixes, the lowmem mode of check gets more updates but
still does not work in the --repair mode and is considered experimental.

ETA for 4.9 is in +6 days (2016-12-20).

Minor fixes, docs improvements or more testcases will be still considered for
4.9 release.


xfstests btrfs/{108,109,117} that was working in 4.8.5 will not work properly.

+ ./check btrfs/108
FSTYP -- btrfs
PLATFORM  -- Linux/x86_64 luna 4.9.0
MKFS_OPTIONS  -- /dev/sdb3
MOUNT_OPTIONS -- /dev/sdb3 /test6

btrfs/108 1s ... [failed, exit status 1] - output mismatch (see 
/xfstests/results//btrfs/108.out.bad)
--- tests/btrfs/108.out 2015-10-19 09:55:52.0 +0900
+++ /xfstests/results//btrfs/108.out.bad2016-12-15 15:41:43.771411349 
+0900
@@ -8,6 +8,6 @@
 File digests in the original filesystem:
 fbf36a062ffcbd644b5739c4d683ccc7  SCRATCH_MNT/snap/foo
 5d2c92827a70aad932cfe7363105c55e  SCRATCH_MNT/snap/bar
-File digests in the new filesystem:
-fbf36a062ffcbd644b5739c4d683ccc7  SCRATCH_MNT/snap/foo
-5d2c92827a70aad932cfe7363105c55e  SCRATCH_MNT/snap/bar
+./common/rc: line 2784: 22352 Segmentation fault  (core dumped) "$@" >> 
$seqres.full 2>&1
...
(Run 'diff -u tests/btrfs/108.out /xfstests/results//btrfs/108.out.bad'  to 
see the entire diff)
Ran: btrfs/108
Failures: btrfs/108
Failed 1 of 1 tests


Another problem was found. xfstests btrfs/012 will not succeed.

btrfs/012 58s ... [failed, exit status 1] - output mismatch (see 
/xfstests/results//btrfs/012.out.bad)
--- tests/btrfs/012.out 2015-08-04 16:09:38.0 +0900
+++ /xfstests/results//btrfs/012.out.bad2016-12-15 17:38:10.305009249 
+0900
@@ -1 +1,3 @@
 == QA output created by 012
+btrfs-convert rollback failed
+(see /xfstests/results//btrfs/012.full for details)
...
(Run 'diff -u tests/btrfs/012.out /xfstests/results//btrfs/012.out.bad'  to 
see the entire diff)

Thanks,
Tsutomu


Thanks for the test.

Chandan has reported such regression, after his migrate_super_block() 
fix for 64K nodesize.
But that's not really caused by this patch, just too restrict rollback 
condition.


This can be fixed by the following patches:
[1/3] btrfs-progs: file-item: Fix wrong file extents inserted
[2/3] btrfs-progs: convert: Rework rollback to handle new convert image
[3/3] btrfs-progs: convert-test: trigger chunk allocation after convert

Thanks,
Qu




Thanks,
Tsutomu



Changes:
  * check: many lowmem mode updates
  * send: use splice syscall to copy buffer from kernel
  * receive: new option to dump the stream in textual form
  * convert:
* move sources to own directory
* prevent accounting of blocks beyond end of the device
* make it work with 64k sectorsize
  * mkfs: move sources to own directory
  * defrag: warns if directory used without -r
  * dev stats:
* new option to check stats for non-zero values
* add long option for -z
  * library: version bump to 0.1.2, added subvol_uuid_search2
  * other:
* cleanups
* docs updates

Tarballs: https://www.kernel.org/pub/linux/kernel/people/kdave/btrfs-progs/
Git: git://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git

Shortlog:

Adam Borowski (1):
  btrfs-progs: man mkfs: warn about RAID5/6 being experimental

Anand Jain (1):
  btrfs-progs: recursive defrag cleanup duplicate code

Austin S. Hemmelgarn (1):
  btrfs-progs: dev stats: add dev stats returncode option

Chandan Rajendra (3):
  btrfs-progs: Use helper function to access 
btrfs_super_block->sys_chunk_array_size
  btrfs-progs: convert: Prevent accounting blocks beyond end of device
  btrfs-progs: convert: Fix migrate_super_block() to work with 64k 
sectorsize

David Sterba (35):
  btrfs-progs: remove extra newline from messages
  btrfs-progs: use symbolic name for first inode number when searching
  btrfs-progs: send: use splice syscall instead of read/write to transfer 
buffer
  btrfs-progs: send: rename thread callback to read data from kernel
  btrfs-progs: make incompat bit wrappers more compact
  btrfs-progs: receive: rename receive context variable
  btrfs-progs: check: use on-stack path buffer in check_fs_first_inode
  btrfs-progs: check: use on-stack path buffer in check_fs_root_v2
  btrfs-progs: check: use on-stack path buffer in check_fs_roots_v2
  btrfs-progs: send dump: introduce helper for printing escaped path
  btrfs-progs: send dump: print escaped path
  btrfs-progs: send dump: use reentrant variant of localtime
  btrfs-progs: tests: add more gobal option to test 001-btrfs
  btrfs-progs: docs: update receive help and manual page
  btrfs-progs: build: extend pattern rules for standalone directories
  

Re: Btrfs progs pre-release 4.9-rc1

2016-12-15 Thread Tsutomu Itoh
On 2016/12/15 15:45, Tsutomu Itoh wrote:
> On 2016/12/14 23:42, David Sterba wrote:
>> Hi,
>>
>> a pre-release has been tagged. Contains almost the entire devel branch from
>> today. There are small fixes, the lowmem mode of check gets more updates but
>> still does not work in the --repair mode and is considered experimental.
>>
>> ETA for 4.9 is in +6 days (2016-12-20).
>>
>> Minor fixes, docs improvements or more testcases will be still considered for
>> 4.9 release.
> 
> xfstests btrfs/{108,109,117} that was working in 4.8.5 will not work properly.
> 
> + ./check btrfs/108
> FSTYP -- btrfs
> PLATFORM  -- Linux/x86_64 luna 4.9.0
> MKFS_OPTIONS  -- /dev/sdb3
> MOUNT_OPTIONS -- /dev/sdb3 /test6
> 
> btrfs/108 1s ... [failed, exit status 1] - output mismatch (see 
> /xfstests/results//btrfs/108.out.bad)
> --- tests/btrfs/108.out 2015-10-19 09:55:52.0 +0900
> +++ /xfstests/results//btrfs/108.out.bad2016-12-15 15:41:43.771411349 
> +0900
> @@ -8,6 +8,6 @@
>  File digests in the original filesystem:
>  fbf36a062ffcbd644b5739c4d683ccc7  SCRATCH_MNT/snap/foo
>  5d2c92827a70aad932cfe7363105c55e  SCRATCH_MNT/snap/bar
> -File digests in the new filesystem:
> -fbf36a062ffcbd644b5739c4d683ccc7  SCRATCH_MNT/snap/foo
> -5d2c92827a70aad932cfe7363105c55e  SCRATCH_MNT/snap/bar
> +./common/rc: line 2784: 22352 Segmentation fault  (core dumped) "$@" 
> >> $seqres.full 2>&1
> ...
> (Run 'diff -u tests/btrfs/108.out /xfstests/results//btrfs/108.out.bad'  
> to see the entire diff)
> Ran: btrfs/108
> Failures: btrfs/108
> Failed 1 of 1 tests

Another problem was found. xfstests btrfs/012 will not succeed.

btrfs/012 58s ... [failed, exit status 1] - output mismatch (see 
/xfstests/results//btrfs/012.out.bad)
--- tests/btrfs/012.out 2015-08-04 16:09:38.0 +0900
+++ /xfstests/results//btrfs/012.out.bad2016-12-15 17:38:10.305009249 
+0900
@@ -1 +1,3 @@
 == QA output created by 012
+btrfs-convert rollback failed
+(see /xfstests/results//btrfs/012.full for details)
...
(Run 'diff -u tests/btrfs/012.out /xfstests/results//btrfs/012.out.bad'  to 
see the entire diff)

Thanks,
Tsutomu

> 
> Thanks,
> Tsutomu
> 
>>
>> Changes:
>>   * check: many lowmem mode updates
>>   * send: use splice syscall to copy buffer from kernel
>>   * receive: new option to dump the stream in textual form
>>   * convert:
>> * move sources to own directory
>> * prevent accounting of blocks beyond end of the device
>> * make it work with 64k sectorsize
>>   * mkfs: move sources to own directory
>>   * defrag: warns if directory used without -r
>>   * dev stats:
>> * new option to check stats for non-zero values
>> * add long option for -z
>>   * library: version bump to 0.1.2, added subvol_uuid_search2
>>   * other:
>> * cleanups
>> * docs updates
>>
>> Tarballs: https://www.kernel.org/pub/linux/kernel/people/kdave/btrfs-progs/
>> Git: git://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git
>>
>> Shortlog:
>>
>> Adam Borowski (1):
>>   btrfs-progs: man mkfs: warn about RAID5/6 being experimental
>>
>> Anand Jain (1):
>>   btrfs-progs: recursive defrag cleanup duplicate code
>>
>> Austin S. Hemmelgarn (1):
>>   btrfs-progs: dev stats: add dev stats returncode option
>>
>> Chandan Rajendra (3):
>>   btrfs-progs: Use helper function to access 
>> btrfs_super_block->sys_chunk_array_size
>>   btrfs-progs: convert: Prevent accounting blocks beyond end of device
>>   btrfs-progs: convert: Fix migrate_super_block() to work with 64k 
>> sectorsize
>>
>> David Sterba (35):
>>   btrfs-progs: remove extra newline from messages
>>   btrfs-progs: use symbolic name for first inode number when searching
>>   btrfs-progs: send: use splice syscall instead of read/write to 
>> transfer buffer
>>   btrfs-progs: send: rename thread callback to read data from kernel
>>   btrfs-progs: make incompat bit wrappers more compact
>>   btrfs-progs: receive: rename receive context variable
>>   btrfs-progs: check: use on-stack path buffer in check_fs_first_inode
>>   btrfs-progs: check: use on-stack path buffer in check_fs_root_v2
>>   btrfs-progs: check: use on-stack path buffer in check_fs_roots_v2
>>   btrfs-progs: send dump: introduce helper for printing escaped path
>>   btrfs-progs: send dump: print escaped path
>>   btrfs-progs: send dump: use reentrant variant of localtime
>>   btrfs-progs: tests: add more gobal option to test 001-btrfs
>>   btrfs-progs: docs: update receive help and manual page
>>   btrfs-progs: build: extend pattern rules for standalone directories
>>   btrfs-progs: move btrfs-convert to own directory
>>   btrfs-progs: move mkfs.btrfs sources to own directory
>>   btrfs-progs: tests: check for partscan support in 
>> misc/006-partitioned-loopdev
>>   btrfs-progs: run mkfs tests in CI
>>   

Re: Btrfs progs pre-release 4.9-rc1

2016-12-14 Thread Tsutomu Itoh
On 2016/12/14 23:42, David Sterba wrote:
> Hi,
> 
> a pre-release has been tagged. Contains almost the entire devel branch from
> today. There are small fixes, the lowmem mode of check gets more updates but
> still does not work in the --repair mode and is considered experimental.
> 
> ETA for 4.9 is in +6 days (2016-12-20).
> 
> Minor fixes, docs improvements or more testcases will be still considered for
> 4.9 release.

xfstests btrfs/{108,109,117} that was working in 4.8.5 will not work properly.

+ ./check btrfs/108
FSTYP -- btrfs
PLATFORM  -- Linux/x86_64 luna 4.9.0
MKFS_OPTIONS  -- /dev/sdb3
MOUNT_OPTIONS -- /dev/sdb3 /test6

btrfs/108 1s ... [failed, exit status 1] - output mismatch (see 
/xfstests/results//btrfs/108.out.bad)
--- tests/btrfs/108.out 2015-10-19 09:55:52.0 +0900
+++ /xfstests/results//btrfs/108.out.bad2016-12-15 15:41:43.771411349 
+0900
@@ -8,6 +8,6 @@
 File digests in the original filesystem:
 fbf36a062ffcbd644b5739c4d683ccc7  SCRATCH_MNT/snap/foo
 5d2c92827a70aad932cfe7363105c55e  SCRATCH_MNT/snap/bar
-File digests in the new filesystem:
-fbf36a062ffcbd644b5739c4d683ccc7  SCRATCH_MNT/snap/foo
-5d2c92827a70aad932cfe7363105c55e  SCRATCH_MNT/snap/bar
+./common/rc: line 2784: 22352 Segmentation fault  (core dumped) "$@" 
>> $seqres.full 2>&1
...
(Run 'diff -u tests/btrfs/108.out /xfstests/results//btrfs/108.out.bad'  to 
see the entire diff)
Ran: btrfs/108
Failures: btrfs/108
Failed 1 of 1 tests

Thanks,
Tsutomu

> 
> Changes:
>   * check: many lowmem mode updates
>   * send: use splice syscall to copy buffer from kernel
>   * receive: new option to dump the stream in textual form
>   * convert:
> * move sources to own directory
> * prevent accounting of blocks beyond end of the device
> * make it work with 64k sectorsize
>   * mkfs: move sources to own directory
>   * defrag: warns if directory used without -r
>   * dev stats:
> * new option to check stats for non-zero values
> * add long option for -z
>   * library: version bump to 0.1.2, added subvol_uuid_search2
>   * other:
> * cleanups
> * docs updates
> 
> Tarballs: https://www.kernel.org/pub/linux/kernel/people/kdave/btrfs-progs/
> Git: git://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git
> 
> Shortlog:
> 
> Adam Borowski (1):
>   btrfs-progs: man mkfs: warn about RAID5/6 being experimental
> 
> Anand Jain (1):
>   btrfs-progs: recursive defrag cleanup duplicate code
> 
> Austin S. Hemmelgarn (1):
>   btrfs-progs: dev stats: add dev stats returncode option
> 
> Chandan Rajendra (3):
>   btrfs-progs: Use helper function to access 
> btrfs_super_block->sys_chunk_array_size
>   btrfs-progs: convert: Prevent accounting blocks beyond end of device
>   btrfs-progs: convert: Fix migrate_super_block() to work with 64k 
> sectorsize
> 
> David Sterba (35):
>   btrfs-progs: remove extra newline from messages
>   btrfs-progs: use symbolic name for first inode number when searching
>   btrfs-progs: send: use splice syscall instead of read/write to transfer 
> buffer
>   btrfs-progs: send: rename thread callback to read data from kernel
>   btrfs-progs: make incompat bit wrappers more compact
>   btrfs-progs: receive: rename receive context variable
>   btrfs-progs: check: use on-stack path buffer in check_fs_first_inode
>   btrfs-progs: check: use on-stack path buffer in check_fs_root_v2
>   btrfs-progs: check: use on-stack path buffer in check_fs_roots_v2
>   btrfs-progs: send dump: introduce helper for printing escaped path
>   btrfs-progs: send dump: print escaped path
>   btrfs-progs: send dump: use reentrant variant of localtime
>   btrfs-progs: tests: add more gobal option to test 001-btrfs
>   btrfs-progs: docs: update receive help and manual page
>   btrfs-progs: build: extend pattern rules for standalone directories
>   btrfs-progs: move btrfs-convert to own directory
>   btrfs-progs: move mkfs.btrfs sources to own directory
>   btrfs-progs: tests: check for partscan support in 
> misc/006-partitioned-loopdev
>   btrfs-progs: run mkfs tests in CI
>   btrfs-progs: mkfs: annotation of a case
>   btrfs-progs: docs: clarify trim after mkfs -K
>   btrfs-progs: docs: make documentation updates workflow more clear
>   btrfs-progs: dev stats: adjust some error messages
>   btrfs-progs: dev stats: use char type path
>   btrfs-progs: dev stats: use table based printing of items
>   btrfs-progs: dev stats: add long option for -z
>   btrfs-progs: docs: update dev stats help and manual page
>   btrfs-progs: help: fix printing of aliased commands
>   btrfs-progs: fixup API after change in subvol_uuid_search
>   btrfs-progs: library: bump to 0.1.2
>   btrfs-progs: handle failed strdup in subvol_uuid_search2
>   btrfs-progs: dev stats: update option name for checking non-zero