Re: [PATCH v3] generic: test fiemap offsets and < 512 byte ranges
On Wed, Apr 07, 2021 at 01:13:26PM -0700, Boris Burkov wrote:
> btrfs trims fiemap extents to the inputted offset, which leads to
> inconsistent results for most inputs, and downright bizarre outputs like
> [7..6] when the trimmed extent is at the end of an extent and shorter
> than 512 bytes.
>
> The test writes out one extent of the file system's block size and tries
> fiemaps at various offsets. It expects that all the fiemaps return the
> full single extent.
>
> I ran it under the following fs, block size combinations:
> ext2: 1024, 2048, 4096
> ext3: 1024, 2048, 4096
> ext4: 1024, 2048, 4096
> xfs: 512, 1024, 2048, 4096
> f2fs: 4096
> btrfs: 4096
>
> This test is fixed for btrfs by:
> btrfs: return whole extents in fiemap
> (https://lore.kernel.org/linux-btrfs/274e5bcebdb05a8969fc300b4802f33da2fbf218.1617746680.git.bo...@bur.io/)
>
> Signed-off-by: Boris Burkov
generic/473, which tests fiemap, has been marked as broken, as fiemap
behavior is not consistent across filesystems, and the specific behavior
tested by generic/473 is not defined and filesystems could have
different implementations.
I'm not sure if this test fits into the undefined-behavior fiemap
categary. I think it's fine if it tests a well-defined & consistent
behavior.
> ---
> v3: make the block size more generic, use test dev instead of scratch,
> cleanup style issues.
> v2: fill out copyright and test description
> ---
> tests/generic/623 | 94 +++
> tests/generic/623.out | 2 +
> tests/generic/group | 1 +
> 3 files changed, 97 insertions(+)
> create mode 100755 tests/generic/623
> create mode 100644 tests/generic/623.out
>
> diff --git a/tests/generic/623 b/tests/generic/623
> new file mode 100755
> index ..a5ef369a
> --- /dev/null
> +++ b/tests/generic/623
> @@ -0,0 +1,94 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2021 Facebook. All Rights Reserved.
> +#
> +# FS QA Test 623
> +#
> +# Test fiemaps with offsets into small parts of extents.
> +# Expect to get the whole extent, anyway.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> + rm -f $fiemap_file
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs generic
> +_require_test
> +_require_xfs_io_command "fiemap"
> +
> +rm -f $seqres.full
> +
> +fiemap_file=$TEST_DIR/foo.$$
> +
> +do_fiemap() {
> + off=$1
> + len=$2
> + echo $off $len >> $seqres.full
> + $XFS_IO_PROG -c "fiemap $off $len" $fiemap_file | tee -a $seqres.full
> +}
> +
> +check_fiemap() {
> + off=$1
> + len=$2
> + actual=$(do_fiemap $off $len)
> + [ "$actual" == "$expected" ] || _fail "unexpected fiemap on $off $len"
> +}
> +
> +# write a file with one extent
> +block_size=$(_get_block_size $TEST_DIR)
> +$XFS_IO_PROG -f -s -c "pwrite -S 0xcf 0 $block_size" $fiemap_file >/dev/null
> +
> +# since the exact extent location is unpredictable especially when
> +# varying file systems, just test that they are all equal, which is
> +# what we really expect.
> +expected=$(do_fiemap)
> +
> +mid=$((block_size / 2))
> +almost=$((block_size - 5))
> +past=$((block_size + 1))
> +
> +check_fiemap 0 $mid
> +check_fiemap 0 $block_size
> +check_fiemap 0 $past
> +check_fiemap $mid $almost
> +check_fiemap $mid $block_size
> +check_fiemap $mid $past
> +check_fiemap $almost 5
> +check_fiemap $almost 6
> +
> +# fiemap output explicitly deals in 512 byte increments,
> +# so exercise some cases where len is 512.
> +# Naturally, some of these can't work if block size is 512.
> +one_short=$((block_size - 512))
> +check_fiemap 0 512
> +check_fiemap $one_short 512
> +check_fiemap $almost 512
> +
> +_test_unmount
Any reason to umount TEST_DEV?
Thanks,
Eryu
> +
> +echo "Silence is golden"
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/generic/623.out b/tests/generic/623.out
> new file mode 100644
> index ..6f774f19
> --- /dev/null
> +++ b/tests/generic/623.out
> @@ -0,0 +1,2 @@
> +QA output created by 623
> +Silence is golden
> diff --git a/tests/generic/group b/tests/generic/group
> index b10fdea4..39e02383 100644
> --- a/tests/generic/group
> +++ b/tests/generic/group
> @@ -625,3 +625,4 @@
> 620 auto mount quick
> 621 auto quick encrypt
> 622 auto shutdown metadata atime
> +623 auto quick
> --
> 2.30.2
Re: [PATCH v3 2/3] generic/574: corrupt btrfs merkle tree data
On Thu, Apr 08, 2021 at 11:57:50AM -0700, Boris Burkov wrote:
> generic/574 has tests for corrupting the merkle tree data stored by the
> filesystem. Since btrfs uses a different scheme for storing this data,
> the existing logic for corrupting it doesn't work out of the box. Adapt
> it to properly corrupt btrfs merkle items.
>
> Note that there is a bit of a kludge here: since btrfs_corrupt_block
> doesn't handle streaming corruption bytes from stdin (I could change
> that, but it feels like overkill for this purpose), I just read the
> first corruption byte and duplicate it for the desired length. That is
> how the test is using the interface in practice, anyway.
>
> This relies on the following kernel patch for btrfs verity support:
>
> And the following btrfs-progs patch for btrfs_corrupt_block support:
>
>
> Signed-off-by: Boris Burkov
> ---
> common/verity | 15 +--
> 1 file changed, 13 insertions(+), 2 deletions(-)
>
> diff --git a/common/verity b/common/verity
> index d2c1ea24..fdd05783 100644
> --- a/common/verity
> +++ b/common/verity
> @@ -3,8 +3,7 @@
> #
> # Functions for setting up and testing fs-verity
>
> -_require_scratch_verity()
> -{
> +_require_scratch_verity() {
No need to change this.
> _require_scratch
> _require_command "$FSVERITY_PROG" fsverity
>
> @@ -315,6 +314,18 @@ _fsv_scratch_corrupt_merkle_tree()
> (( offset += ($(_get_filesize $file) + 65535) & ~65535 ))
> _fsv_scratch_corrupt_bytes $file $offset
> ;;
> + btrfs)
> + ino=$(ls -i $file | awk '{print $1}')
stat -c %i $1
And declare local variables with local.
> + sync
Why a system wide sync is needed here?
> + cat > $tmp.bytes
I think this cat would just hang there.
> + sz=$(_get_filesize $tmp.bytes)
> + read -n 1 byte < $tmp.bytes
> + ascii=$(printf "%d" "'$byte'")
> + _scratch_unmount
> + $BTRFS_CORRUPT_BLOCK_PROG -r 5 -I $ino,37,0 -v $ascii -o
> $offset -b $sz $SCRATCH_DEV
It'd be better to explain this command in comments.
> + sync
Again, is this sync really needed?
Thanks,
Eryu
> + _scratch_mount
> + ;;
> *)
> _fail "_fsv_scratch_corrupt_merkle_tree() unimplemented on
> $FSTYP"
> ;;
> --
> 2.30.2
Re: [PATCH v3 1/3] btrfs: test btrfs specific fsverity corruption
On Thu, Apr 08, 2021 at 11:57:49AM -0700, Boris Burkov wrote:
> There are some btrfs specific fsverity scenarios that don't map
> neatly onto the tests in generic/574 like holes, inline extents,
> and preallocated extents. Cover those in a btrfs specific test.
>
> This test relies on the btrfs implementation of fsverity in:
>
> and it relies on btrfs-corrupt-block for corruption, with the patches:
>
I assume you'll fill in the patch lists when they're merged.
>
> Signed-off-by: Boris Burkov
> ---
> common/config | 1 +
> common/verity | 7 ++
> tests/btrfs/290 | 190
> tests/btrfs/290.out | 17
> tests/btrfs/group | 1 +
> 5 files changed, 216 insertions(+)
> create mode 100755 tests/btrfs/290
> create mode 100644 tests/btrfs/290.out
>
> diff --git a/common/config b/common/config
> index a47e462c..003b2a88 100644
> --- a/common/config
> +++ b/common/config
> @@ -256,6 +256,7 @@ export BTRFS_UTIL_PROG=$(type -P btrfs)
> export BTRFS_SHOW_SUPER_PROG=$(type -P btrfs-show-super)
> export BTRFS_CONVERT_PROG=$(type -P btrfs-convert)
> export BTRFS_TUNE_PROG=$(type -P btrfstune)
> +export BTRFS_CORRUPT_BLOCK_PROG=$(type -P btrfs-corrupt-block)
> export XFS_FSR_PROG=$(type -P xfs_fsr)
> export MKFS_NFS_PROG="false"
> export MKFS_CIFS_PROG="false"
> diff --git a/common/verity b/common/verity
> index 38eea157..d2c1ea24 100644
> --- a/common/verity
> +++ b/common/verity
> @@ -8,6 +8,10 @@ _require_scratch_verity()
> _require_scratch
> _require_command "$FSVERITY_PROG" fsverity
>
> + if [ $FSTYP == "btrfs" ]; then
> + _require_command "$BTRFS_CORRUPT_BLOCK_PROG" btrfs_corrupt_block
> + fi
Does the fsverity function depend on btrfs-corrupt-block command? I
think btrfs-corrupt-block is only needed in this specific test, other
btrfs fsverity tests may not depend on it. So add this require rule in
the test if that's the case.
> +
> if ! _scratch_mkfs_verity &>>$seqres.full; then
> # ext4: need e2fsprogs v1.44.5 or later (but actually v1.45.2+
> # is needed for some tests to pass, due to an e2fsck bug)
> @@ -147,6 +151,9 @@ _scratch_mkfs_verity()
> ext4|f2fs)
> _scratch_mkfs -O verity
> ;;
> + btrfs)
> + _scratch_mkfs
> + ;;
> *)
> _notrun "No verity support for $FSTYP"
> ;;
> diff --git a/tests/btrfs/290 b/tests/btrfs/290
> new file mode 100755
> index ..5aff7648
> --- /dev/null
> +++ b/tests/btrfs/290
> @@ -0,0 +1,190 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (C) 2021 Facebook, Inc. All Rights Reserved.
> +#
> +# FS QA Test 290
> +#
> +# Test btrfs support for fsverity.
> +# This test extends the generic fsverity testing by corrupting inline
> extents,
> +# preallocated extents, holes, and the Merkle descriptor in a btrfs-aware
> way.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "cleanup; exit \$status" 0 1 2 3 15
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/verity
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +_supported_fs btrfs
> +_require_scratch
> +_require_scratch_verity
> +
> +cleanup()
Better to name it as _cleanup(), though usually we name local functions
without the leading underscore, but _cleanup function is a bit special,
almost all tests name it with "_" and it's in template in 'check'
script. And it's easier to do a global s/_cleanup/cleanup/ if it's
consistent with other tests.
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +get_ino() {
> + file=$1
Declare local variables with local keyword.
> + ls -i $file | awk '{print $1}'
stat -c "%i"
> +}
> +
> +validate() {
> + f=$1
> + sz=$2
sz is not used, but xfs_io below would use $sz
> + # buffered io
> + cat $f > /dev/null
> + # direct io
> + dd if=$f iflag=direct of=/dev/null status=none
$XFS_IO_PROG -dc "pread -q 0 $sz" $f
Direct I/O is used, then we need
_require_odirect
> +}
> +
> +# corrupt the data portion of an inline extent
> +corrupt_inline() {
> + f=$SCRATCH_MNT/inl
> + head -c 42 /dev/zero | tr '\0' X > $f
$XFS_IO_PROG -fc "pwrite -q -S 0x58 0 42" $f
And all usages in other functions.
> + ino=$(get_ino $f)
> + _fsv_enable $f
> + $XFS_IO_PROG -c sync $SCRATCH_MNT
> + _scratch_unmount
> + # inline data starts at disk_bytenr
> + # overwrite the first u64 with random bogus junk
> + $BTRFS_CORRUPT_BLOCK_PROG -i $ino -x 0 -f disk_bytenr $SCRATCH_DEV >
> /dev/null
> + _scratch_mount
> + validate $f
Then all validte calls should pass file size as second param.
> +}
> +
> +# preallocate a file, then corrupt it by changing it to a regular file
> +co
Re: [PATCH] btrfs/232: fix umount failure due to fsstress still running
On Thu, Mar 18, 2021 at 11:48:15AM +, fdman...@kernel.org wrote:
> From: Filipe Manana
>
> We start a process that runs fsstress, then kill the process, wait for it
> to die and then end the test, where we attempt to unmount the fs which
> often fails because the fsstress subcommand started by the process is
> still running and using the mount point. This results in a test failure:
>
> btrfs/232 1s ... umount: /home/fdmanana/btrfs-tests/scratch_1: target is
> busy.
> _check_btrfs_filesystem: filesystem on /dev/sdc is inconsistent
> (see /home/fdmanana/git/hub/xfstests/results//btrfs/232.full for details)
>
> Fix that by adding a trap to the writer() function.
>
> Signed-off-by: Filipe Manana
Thanks for the fix! I missed that in review..
Thanks,
Eryu
> ---
> tests/btrfs/232 | 4
> 1 file changed, 4 insertions(+)
>
> diff --git a/tests/btrfs/232 b/tests/btrfs/232
> index b0a04a61..b9841410 100755
> --- a/tests/btrfs/232
> +++ b/tests/btrfs/232
> @@ -32,6 +32,10 @@ _cleanup()
>
> writer()
> {
> + # Wait for running fsstress subcommand before exitting so that
> + # mountpoint is not busy when we try to unmount it.
> + trap "wait; exit" SIGTERM
> +
> while true; do
> args=`_scale_fsstress_args -p 20 -n 1000 $FSSTRESS_AVOID -d
> $SCRATCH_MNT/stressdir`
> $FSSTRESS_PROG $args >/dev/null 2>&1
> --
> 2.28.0
Re: [PATCH] btrfs: add test for cases when a dio write has to fallback to a buffered write
On Wed, Mar 10, 2021 at 10:48:35AM +, Filipe Manana wrote:
> On Sun, Mar 7, 2021 at 3:24 PM Eryu Guan wrote:
> >
> > On Sun, Mar 07, 2021 at 03:07:43PM +, Filipe Manana wrote:
> > > On Sun, Mar 7, 2021 at 2:41 PM Eryu Guan wrote:
> > > >
> > > > On Thu, Feb 11, 2021 at 05:01:18PM +, fdman...@kernel.org wrote:
> > > > > From: Filipe Manana
> > > > >
> > > > > Test cases where a direct IO write, with O_DSYNC, can not be done and
> > > > > has
> > > > > to fallback to a buffered write.
> > > > >
> > > > > This is motivated by a regression that was introduced in kernel 5.10
> > > > > by
> > > > > commit 0eb79294dbe328 ("btrfs: dio iomap DSYNC workaround")) and was
> > > > > fixed in kernel 5.11 by commit ecfdc08b8cc65d ("btrfs: remove dio
> > > > > iomap
> > > > > DSYNC workaround").
> > > > >
> > > > > Signed-off-by: Filipe Manana
> > > >
> > > > Sorry for the late review..
> > > >
> > > > So this is supposed to fail with v5.10 kernel, right? But I got it
> > > > passed
> > >
> > > Because either you are testing with a patched 5.10.x kernel, or you
> > > don't have CONFIG_BTRFS_ASSERT=y in your config.
> > > The fix landed in 5.10.18:
> >
> > You're right, I don't have CONFIG_BTRFS_ASSERT=y. As the test dumps the
> > od output of the file content, so I thought the failure would be a data
> > corruption, and expected a od output diff failure.
>
> I see the test was not merged yet, do you expect me to update anything
> in the patch?
Sorry, it was lost in my to-review queue.. But I'd be great if you could
add more descriptions about the CONFIG_BTRFS_ASSERT and the compress
case either in commit log or in test description.
Thanks,
Eryu
Re: [PATCH] fstest: random read fio test for read policy
On Mon, Feb 22, 2021 at 06:48:18AM -0800, Anand Jain wrote:
> This test case runs fio for raid1/10/1c3/1c4 profiles and all the
> available read policies in the system. At the end of the test case,
> a comparative summary of the result is in the $seqresfull-file.
>
> LOAD_FACTOR parameter controls the fio scalability. For the
> LOAD_FACTOR = 1 (default), this runs fio for file size = 1G and num
> of jobs = 1, which approximately takes 65s to finish.
>
> There are two objectives of this test case. 1. by default, with
> LOAD_FACTOR = 1, it sanity tests the read policies. And 2. Run the
> test case individually with a larger LOAD_FACTOR. For example, 10
> for the comparative study of the read policy performance.
>
> I find tests/btrfs as the placeholder for this test case. As it
> contains many things which are btrfs specific and didn't fit well
> under perf.
>
> Signed-off-by: Anand Jain
I'd like to let btrfs folks to review as well, to see if this is a sane
test for btrfs.
Thanks,
Eryu
> ---
> tests/btrfs/231 | 145
> tests/btrfs/231.out | 2 +
> tests/btrfs/group | 1 +
> 3 files changed, 148 insertions(+)
> create mode 100755 tests/btrfs/231
> create mode 100644 tests/btrfs/231.out
>
> diff --git a/tests/btrfs/231 b/tests/btrfs/231
> new file mode 100755
> index ..c08b5826f60a
> --- /dev/null
> +++ b/tests/btrfs/231
> @@ -0,0 +1,145 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2021 Anand Jain. All Rights Reserved.
> +#
> +# FS QA Test 231
> +#
> +# Random read fio test for raid1(10)(c3)(c4) with available
> +# read policy.
> +#
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +fio_config=$tmp.fio
> +fio_results=$tmp.fio_out
> +
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs btrfs
> +_require_scratch_dev_pool 4
> +
> +njob=$LOAD_FACTOR
> +size=$LOAD_FACTOR
> +_require_scratch_size $(($size * 2 * 1024 * 1024))
> +echo size=$size njob=$njob >> $seqres.full
> +
> +make_fio_config()
> +{
> + #Set direct IO to true, help to avoid buffered IO so that read happens
> + #from the devices.
> + cat >$fio_config < +[global]
> +bs=64K
> +iodepth=64
> +direct=1
> +invalidate=1
> +allrandrepeat=1
> +ioengine=libaio
> +group_reporting
> +size=${size}G
> +rw=randread
> +EOF
> +}
> +#time_based
> +#runtime=5
> +
> +make_fio_config
> +for job in $(seq 0 $njob); do
> + echo "[foo$job]" >> $fio_config
> + echo "filename=$SCRATCH_MNT/$job/file" >> $fio_config
> +done
> +_require_fio $fio_config
> +cat $fio_config >> $seqres.full
> +
> +work()
> +{
> + raid=$1
> +
> + echo - profile: $raid -- >> $seqres.full
> + echo >> $seqres.full
> + _scratch_pool_mkfs $raid >> $seqres.full 2>&1
> + _scratch_mount
> +
> + fsid=$($BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT | grep uuid: | \
> + $AWK_PROG '{print $4}')
> + readpolicy_path="/sys/fs/btrfs/$fsid/read_policy"
> + policies=$(cat $readpolicy_path | sed 's/\[//g' | sed 's/\]//g')
> +
> + for policy in $policies; do
> + echo $policy > $readpolicy_path || _fail "Fail to set
> readpolicy"
> + echo -n "activating readpolicy: " >> $seqres.full
> + cat $readpolicy_path >> $seqres.full
> + echo >> $seqres.full
> +
> + > $fio_results
> + $FIO_PROG --output=$fio_results $fio_config
> + cat $fio_results >> $seqres.full
> + done
> +
> + _scratch_unmount
> + _scratch_dev_pool_put
> +}
> +
> +_scratch_dev_pool_get 2
> +work "-m raid1 -d single"
> +
> +_scratch_dev_pool_get 2
> +work "-m raid1 -d raid1"
> +
> +_scratch_dev_pool_get 4
> +work "-m raid10 -d raid10"
> +
> +_scratch_dev_pool_get 3
> +work "-m raid1c3 -d raid1c3"
> +
> +_scratch_dev_pool_get 4
> +work "-m raid1c4 -d raid1c4"
> +
> +
> +# Now benchmark the raw device performance
> +> $fio_config
> +make_fio_config
> +_scratch_dev_pool_get 4
> +for dev in $SCRATCH_DEV_POOL; do
> + echo "[$dev]" >> $fio_config
> + echo "filename=$dev" >> $fio_config
> +done
> +_require_fio $fio_config
> +cat $fio_config >> $seqres.full
> +
> +echo - profile: raw disk -- >> $seqres.full
> +echo >> $seqres.full
> +> $fio_results
> +$FIO_PROG --output=$fio_results $fio_config
> +cat $fio_results >> $seqres.full
> +
> +echo >> $seqres.full
> +echo = Summary == >> $seqres.full
> +cat $seqres.full | egrep -A1 "Run status|Disk stats|profile:|readpolicy" >>
> $seqres.full
> +
> +echo "Silence is golden"
> +
> +# succe
Re: [PATCH] btrfs: add test for cases when a dio write has to fallback to a buffered write
On Sun, Mar 07, 2021 at 03:07:43PM +, Filipe Manana wrote:
> On Sun, Mar 7, 2021 at 2:41 PM Eryu Guan wrote:
> >
> > On Thu, Feb 11, 2021 at 05:01:18PM +, fdman...@kernel.org wrote:
> > > From: Filipe Manana
> > >
> > > Test cases where a direct IO write, with O_DSYNC, can not be done and has
> > > to fallback to a buffered write.
> > >
> > > This is motivated by a regression that was introduced in kernel 5.10 by
> > > commit 0eb79294dbe328 ("btrfs: dio iomap DSYNC workaround")) and was
> > > fixed in kernel 5.11 by commit ecfdc08b8cc65d ("btrfs: remove dio iomap
> > > DSYNC workaround").
> > >
> > > Signed-off-by: Filipe Manana
> >
> > Sorry for the late review..
> >
> > So this is supposed to fail with v5.10 kernel, right? But I got it
> > passed
>
> Because either you are testing with a patched 5.10.x kernel, or you
> don't have CONFIG_BTRFS_ASSERT=y in your config.
> The fix landed in 5.10.18:
You're right, I don't have CONFIG_BTRFS_ASSERT=y. As the test dumps the
od output of the file content, so I thought the failure would be a data
corruption, and expected a od output diff failure.
>
> https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=v5.10.18&id=a6703c71153438d3ebdf58a75d53dd5e57b33095
>
> >
> > [root@fedoravm xfstests]# ./check -s btrfs btrfs/231
> > SECTION -- btrfs
> > RECREATING-- btrfs on /dev/mapper/testvg-lv1
> > FSTYP -- btrfs
> > PLATFORM -- Linux/x86_64 fedoravm 5.10.0 #6 SMP Sun Mar 7 22:25:35
> > CST 2021
> > MKFS_OPTIONS -- /dev/mapper/testvg-lv2
> > MOUNT_OPTIONS -- /dev/mapper/testvg-lv2 /mnt/scratch
> >
> > btrfs/231 13s ... 8s
> > Ran: btrfs/231
> > Passed all 1 tests
> >
> > SECTION -- btrfs
> > =
> > Ran: btrfs/231
> > Passed all 1 tests
> >
> > > ---
> > > tests/btrfs/231 | 61 +
> > > tests/btrfs/231.out | 21
> > > tests/btrfs/group | 1 +
> > > 3 files changed, 83 insertions(+)
> > > create mode 100755 tests/btrfs/231
> > > create mode 100644 tests/btrfs/231.out
> > >
> > > diff --git a/tests/btrfs/231 b/tests/btrfs/231
> > > new file mode 100755
> > > index ..9a404f57
> > > --- /dev/null
> > > +++ b/tests/btrfs/231
> > > @@ -0,0 +1,61 @@
> > > +#! /bin/bash
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +# Copyright (C) 2021 SUSE Linux Products GmbH. All Rights Reserved.
> > > +#
> > > +# FS QA Test No. btrfs/231
> > > +#
> > > +# Test cases where a direct IO write, with O_DSYNC, can not be done and
> > > has to
> > > +# fallback to a buffered write.
> > > +#
> > > +seq=`basename $0`
> > > +seqres=$RESULT_DIR/$seq
> > > +echo "QA output created by $seq"
> > > +
> > > +tmp=/tmp/$$
> > > +status=1 # failure is the default!
> > > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > > +
> > > +_cleanup()
> > > +{
> > > + cd /
> > > + rm -f $tmp.*
> > > +}
> > > +
> > > +# get standard environment, filters and checks
> > > +. ./common/rc
> > > +. ./common/filter
> > > +. ./common/attr
> > > +
> > > +# real QA test starts here
> > > +_supported_fs btrfs
> > > +_require_scratch
> > > +_require_odirect
> > > +_require_chattr c
> > > +
> > > +rm -f $seqres.full
> > > +
> > > +_scratch_mkfs >>$seqres.full 2>&1
> > > +_scratch_mount
> > > +
> > > +# First lets test with an attempt to write into a file range with
> > > compressed
> > > +# extents.
> > > +touch $SCRATCH_MNT/foo
> > > +$CHATTR_PROG +c $SCRATCH_MNT/foo
> >
> > It's not so clear to me why writing into a compressed file is required,
> > would you please add more comments?
>
> The test is meant to test cases where we can deterministically make a
> direct IO write fallback to buffered IO.
> There are 2 such cases:
>
> 1) Attempting to write to an unaligned offset - this was the bug in
> 5.10 that resulted in a crash when CONFIG_BTRFS_ASSERT=y (default in
> many distros, such as openSUSE).
>
> 2) Writing to a range that has compressed extents. This has nothing to
> do with the 5.10 regression, I just added it since there's no existing
> test that explicitly and deterministically triggers this.
> So yes, I decided to add a test case for all possible cases of
> direct IO falling back to buffered instead of adding one just to test
> a regression (and help detecting any possible future regressions).
Sounds good, thanks!
Eryu
Re: [PATCH] btrfs: add test for cases when a dio write has to fallback to a buffered write
On Thu, Feb 11, 2021 at 05:01:18PM +, fdman...@kernel.org wrote:
> From: Filipe Manana
>
> Test cases where a direct IO write, with O_DSYNC, can not be done and has
> to fallback to a buffered write.
>
> This is motivated by a regression that was introduced in kernel 5.10 by
> commit 0eb79294dbe328 ("btrfs: dio iomap DSYNC workaround")) and was
> fixed in kernel 5.11 by commit ecfdc08b8cc65d ("btrfs: remove dio iomap
> DSYNC workaround").
>
> Signed-off-by: Filipe Manana
Sorry for the late review..
So this is supposed to fail with v5.10 kernel, right? But I got it
passed
[root@fedoravm xfstests]# ./check -s btrfs btrfs/231
SECTION -- btrfs
RECREATING-- btrfs on /dev/mapper/testvg-lv1
FSTYP -- btrfs
PLATFORM -- Linux/x86_64 fedoravm 5.10.0 #6 SMP Sun Mar 7 22:25:35 CST
2021
MKFS_OPTIONS -- /dev/mapper/testvg-lv2
MOUNT_OPTIONS -- /dev/mapper/testvg-lv2 /mnt/scratch
btrfs/231 13s ... 8s
Ran: btrfs/231
Passed all 1 tests
SECTION -- btrfs
=
Ran: btrfs/231
Passed all 1 tests
> ---
> tests/btrfs/231 | 61 +
> tests/btrfs/231.out | 21
> tests/btrfs/group | 1 +
> 3 files changed, 83 insertions(+)
> create mode 100755 tests/btrfs/231
> create mode 100644 tests/btrfs/231.out
>
> diff --git a/tests/btrfs/231 b/tests/btrfs/231
> new file mode 100755
> index ..9a404f57
> --- /dev/null
> +++ b/tests/btrfs/231
> @@ -0,0 +1,61 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (C) 2021 SUSE Linux Products GmbH. All Rights Reserved.
> +#
> +# FS QA Test No. btrfs/231
> +#
> +# Test cases where a direct IO write, with O_DSYNC, can not be done and has
> to
> +# fallback to a buffered write.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/attr
> +
> +# real QA test starts here
> +_supported_fs btrfs
> +_require_scratch
> +_require_odirect
> +_require_chattr c
> +
> +rm -f $seqres.full
> +
> +_scratch_mkfs >>$seqres.full 2>&1
> +_scratch_mount
> +
> +# First lets test with an attempt to write into a file range with compressed
> +# extents.
> +touch $SCRATCH_MNT/foo
> +$CHATTR_PROG +c $SCRATCH_MNT/foo
It's not so clear to me why writing into a compressed file is required,
would you please add more comments?
Thanks,
Eryu
> +
> +$XFS_IO_PROG -s -c "pwrite -S 0xab -b 1M 0 1M" $SCRATCH_MNT/foo |
> _filter_xfs_io
> +# Now do the direct IO write...
> +$XFS_IO_PROG -d -s -c "pwrite -S 0xcd 512K 512K" $SCRATCH_MNT/foo |
> _filter_xfs_io
> +
> +# Now try doing a write into an unaligned offset...
> +$XFS_IO_PROG -f -d -s -c "pwrite -S 0xef 512K" $SCRATCH_MNT/bar |
> _filter_xfs_io
> +
> +# Unmount, mount again, and verify we have the expected data.
> +_scratch_cycle_mount
> +
> +echo "File foo data:"
> +od -A d -t x1 $SCRATCH_MNT/foo
> +echo "File bar data:"
> +od -A d -t x1 $SCRATCH_MNT/bar
> +
> +status=0
> +exit
> diff --git a/tests/btrfs/231.out b/tests/btrfs/231.out
> new file mode 100644
> index ..def05769
> --- /dev/null
> +++ b/tests/btrfs/231.out
> @@ -0,0 +1,21 @@
> +QA output created by 231
> +wrote 1048576/1048576 bytes at offset 0
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +wrote 524288/524288 bytes at offset 524288
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +wrote 524288/524288 bytes at offset
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +File foo data:
> +000 ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab
> +*
> +0524288 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd
> +*
> +1048576
> +File bar data:
> +000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> +*
> +0001104 00 00 00 00 00 00 00 ef ef ef ef ef ef ef ef ef
> +0001120 ef ef ef ef ef ef ef ef ef ef ef ef ef ef ef ef
> +*
> +0525392 ef ef ef ef ef ef ef
> +0525399
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index a7c65983..9f63db69 100644
> --- a/tests/btrfs/group
> +++ b/tests/btrfs/group
> @@ -233,3 +233,4 @@
> 228 auto quick volume
> 229 auto quick send clone
> 230 auto quick qgroup limit
> +231 auto quick compress rw
> --
> 2.28.0
Re: [PATCH] generic/473: fix expectation properly in out file
On Wed, Feb 24, 2021 at 05:37:20PM +0800, Chengguang Xu wrote:
> 在 星期三, 2021-02-24 17:22:35 Su Yue 撰写
> >
> > On Wed 24 Feb 2021 at 16:51, Chengguang Xu
> > wrote:
> >
> > > 在 星期三, 2021-02-24 15:52:17 Su Yue 撰写
> > >
> > > >
> > > > Cc to the author and linux-xfs, since it's xfsprogs related.
> > > >
> > > > On Tue 23 Feb 2021 at 21:40, Chengguang Xu
> > > >
> > > > wrote:
> > > >
> > > > > It seems the expected result of testcase of "Hole + Data"
> > > > > in generic/473 is not correct, so just fix it properly.
> > > > >
> > > >
> > > > But it's not proper...
> > > >
> > > > > Signed-off-by: Chengguang Xu
> > > > > ---
> > > > > tests/generic/473.out | 2 +-
> > > > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > > > >
> > > > > diff --git a/tests/generic/473.out b/tests/generic/473.out
> > > > > index 75816388..f1ee5805 100644
> > > > > --- a/tests/generic/473.out
> > > > > +++ b/tests/generic/473.out
> > > > > @@ -6,7 +6,7 @@ Data + Hole
> > > > > 1: [256..287]: hole
> > > > > Hole + Data
> > > > > 0: [0..127]: hole
> > > > > -1: [128..255]: data
> > > > > +1: [128..135]: data
> > > > >
> > > > The line is produced by `$XFS_IO_PROG -c "fiemap -v 0 65k"
> > > > $file |
> > > > _filter_fiemap`.
> > > > 0-64k is a hole and 64k-128k is a data extent.
> > > > fiemap ioctl always returns *complete* ranges of extents.
> > >
> > > Manual testing result in latest kernel like below.
> > >
> > > [root@centos test]# uname -a
> > > Linux centos 5.11.0+ #5 SMP Tue Feb 23 21:02:27 CST 2021 x86_64
> > > x86_64 x86_64 GNU/Linux
> > >
> > > [root@centos test]# xfs_io -V
> > > xfs_io version 5.0.0
> > >
> > > [root@centos test]# stat a
> > > File: a
> > > Size: 4194304 Blocks: 0 IO Block: 4096
> > > regular file
> > > Device: fc01h/64513dInode: 140 Links: 1
> > > Access: (0644/-rw-r--r--) Uid: (0/root) Gid: (0/
> > > root)
> > > Access: 2021-02-24 16:33:20.235654140 +0800
> > > Modify: 2021-02-24 16:33:25.070641521 +0800
> > > Change: 2021-02-24 16:33:25.070641521 +0800
> > > Birth: -
> > >
> > > [root@centos test]# xfs_io -c "pwrite 64k 64k" a
> > > wrote 65536/65536 bytes at offset 65536
> > > 64 KiB, 16 ops; 0. sec (992.063 MiB/sec and 253968.2540
> > > ops/sec)
> > >
> > > [root@VM-8-4-centos test]# xfs_io -c "fiemap -v 0 65k" a
> > > a:
> > > EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS
> > >0: [0..127]:hole 128
> > >1: [128..135]: 360..367 8 0x1
> > >
> >
> > Sorry, my carelessness. I only checked btrfs implementation but
> > xfs
> > and ext4 do return the change you made.
> >
>
> Yeah, it seems there is no bad side effect to show only specified range of
> extents
> and keep all the same behavior is also good for testing. I can post a fix
> patch for
> this but before that let us to wait some feedback from maintainers and
> experts.
generic/473 is marked as broken by commit 715eac1a9e66 ("generic/47[23]:
remove from auto/quick groups").
Thanks,
Eryu
Re: [PATCH] btrfs/220: fix clear_cache and inode_cache option tests
On Tue, Jan 12, 2021 at 01:17:47PM -0800, Boris Burkov wrote:
> I recently changed clear_cache to not appear in mount options, as it has
> one shot semantics, which was breaking this test. Test explicitly that
> it _doesn't_ appear, which properly fails on old filesystems and passes
> on misc-next.
>
> The patch that changed this behavior was:
> 8b228324a8ce btrfs: clear free space tree on ro->rw remount
>
> Separately, inode_cache is deprecated and will never appear in mount
> options; remove it entirely.
>
> Signed-off-by: Boris Burkov
Missing email address in above tag.
Also cc'ed linux-btrfs list for review.
Thanks,
Eryu
> ---
> tests/btrfs/220 | 10 +++---
> 1 file changed, 7 insertions(+), 3 deletions(-)
>
> diff --git a/tests/btrfs/220 b/tests/btrfs/220
> index c84c7065..1242460f 100755
> --- a/tests/btrfs/220
> +++ b/tests/btrfs/220
> @@ -215,11 +215,8 @@ test_optional_kernel_features()
>
> test_non_revertible_options()
> {
> - test_mount_opt "clear_cache" "clear_cache"
> test_mount_opt "degraded" "degraded"
>
> - test_mount_opt "inode_cache" "inode_cache"
> -
> # nologreplay should be used only with
> test_should_fail "nologreplay"
> test_mount_opt "nologreplay,ro" "ro,rescue=nologreplay"
> @@ -238,6 +235,11 @@ test_non_revertible_options()
> test_mount_opt "rescue=nologreplay,ro" "ro,rescue=nologreplay"
> }
>
> +test_one_shot_options()
> +{
> + test_mount_opt "clear_cache" ""
> +}
> +
> # All these options can be reverted (with their "no" counterpart), or can
> have
> # their values set to default on remount
> test_revertible_options()
> @@ -321,6 +323,8 @@ test_optional_kernel_features
>
> test_non_revertible_options
>
> +test_one_shot_options
> +
> test_revertible_options
>
> test_subvol
> --
> 2.24.1
Re: [PATCH v2] btrfs: test if rename handles dir item collision correctly
On Thu, Dec 17, 2020 at 11:47:49AM +, Filipe Manana wrote:
> On Tue, Dec 15, 2020 at 4:16 AM ethanwu wrote:
> >
> > This is a regression test for the issue fixed by the kernel commit titled
> > "btrfs: correctly calculate item size used when item key collision happens"
> >
> > In this case, we'll simply rename many forged filename that cause collision
> > under a directory to see if rename failed and filesystem is forced readonly.
> >
> > Signed-off-by: ethanwu
> > ---
> > v2:
> > - Add a python script to generate the forged name at run-time rather than
> > from hardcoded names
> > - Fix , Btrfs->btrfs, and typo mentioned in v1
> >
> > src/btrfs_crc32c_forged_name.py | 92 +
> > tests/btrfs/228 | 72 ++
> > tests/btrfs/228.out | 2 +
> > tests/btrfs/group | 1 +
> > 4 files changed, 167 insertions(+)
> > create mode 100755 src/btrfs_crc32c_forged_name.py
> > create mode 100755 tests/btrfs/228
> > create mode 100644 tests/btrfs/228.out
> >
> > diff --git a/src/btrfs_crc32c_forged_name.py
> > b/src/btrfs_crc32c_forged_name.py
> > new file mode 100755
> > index ..d8abedde
> > --- /dev/null
> > +++ b/src/btrfs_crc32c_forged_name.py
> > @@ -0,0 +1,92 @@
> > +# SPDX-License-Identifier: GPL-2.0
> > +
> > +import struct
> > +import sys
> > +import os
> > +import argparse
> > +
> > +class CRC32(object):
> > + """A class to calculate and manipulate CRC32."""
> > + def __init__(self):
> > +self.polynom = 0x82F63B78
> > +self.table, self.reverse = [0]*256, [0]*256
> > +self._build_tables()
> > +
> > + def _build_tables(self):
> > +for i in range(256):
> > + fwd = i
> > + rev = i << 24
> > + for j in range(8, 0, -1):
> > +# build normal table
> > +if (fwd & 1) == 1:
> > + fwd = (fwd >> 1) ^ self.polynom
> > +else:
> > + fwd >>= 1
> > +self.table[i] = fwd & 0x
> > +# build reverse table =)
> > +if rev & 0x8000 == 0x8000:
> > + rev = ((rev ^ self.polynom) << 1) | 1
> > +else:
> > + rev <<= 1
> > +rev &= 0x
> > +self.reverse[i] = rev
> > +
> > + def calc(self, s):
> > +"""Calculate crc32 of a string.
> > + Same crc32 as in (binascii.crc32)&0x.
> > +"""
> > +crc = 0x
> > +for c in s:
> > + crc = (crc >> 8) ^ self.table[(crc ^ ord(c)) & 0xff]
> > +return crc^0x
> > +
> > + def forge(self, wanted_crc, s, pos=None):
> > +"""Forge crc32 of a string by adding 4 bytes at position pos."""
> > +if pos is None:
> > + pos = len(s)
> > +
> > +# forward calculation of CRC up to pos, sets current forward CRC state
> > +fwd_crc = 0x
> > +for c in s[:pos]:
> > + fwd_crc = (fwd_crc >> 8) ^ self.table[(fwd_crc ^ ord(c)) & 0xff]
> > +
> > +# backward calculation of CRC up to pos, sets wanted backward CRC state
> > +bkd_crc = wanted_crc^0x
> > +for c in s[pos:][::-1]:
> > + bkd_crc = ((bkd_crc << 8) & 0x) ^ self.reverse[bkd_crc >> 24]
> > + bkd_crc ^= ord(c)
> > +
> > +# deduce the 4 bytes we need to insert
> > +for c in struct.pack(' > + bkd_crc = ((bkd_crc << 8) & 0x) ^ self.reverse[bkd_crc >> 24]
> > + bkd_crc ^= ord(c)
> > +
> > +res = s[:pos] + struct.pack(' > +return res
> > +
> > + def parse_args(self):
> > +parser = argparse.ArgumentParser()
> > +parser.add_argument("-d", default=os.getcwd(), dest='dir',
> > +help="directory to generate forged names")
> > +parser.add_argument("-c", default=1, type=int, dest='count',
> > +help="number of forged names to create")
> > +return parser.parse_args()
> > +
> > +if __name__=='__main__':
> > +
> > + crc = CRC32()
> > + wanted_crc = 0x
> > + count = 0
> > + args = crc.parse_args()
> > + dirpath=args.dir
> > + while count < args.count :
> > +origname = os.urandom (89).encode ("hex")[:-1].strip ("\x00")
> > +forgename = crc.forge(wanted_crc, origname, 4)
> > +if ("/" not in forgename) and ("\x00" not in forgename):
> > + srcpath=dirpath + '/' + str(count)
> > + dstpath=dirpath + '/' + forgename
> > + file (srcpath, 'a').close()
> > + os.rename(srcpath, dstpath)
> > + os.system('btrfs fi sync %s' % (dirpath))
> > + count+=1;
> > +
> > diff --git a/tests/btrfs/228 b/tests/btrfs/228
> > new file mode 100755
> > index ..e38da19b
> > --- /dev/null
> > +++ b/tests/btrfs/228
> > @@ -0,0 +1,72 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2020 Synology. All Rights Reserved.
> > +#
> > +# FS QA Test 228
> > +#
> > +# Test if btrfs rename handle dir item collision correctly
> > +# Without patch fix, rename will fail with EOVERFLOW, and filesystem
> > +# is forced readonly.
> > +#
> > +# This bug is g
Re: [RESEND PATCH] btrfs: Add test 154
On Mon, Dec 07, 2020 at 06:19:00PM +0200, Nikolay Borisov wrote:
> This test verifies btrfs' free objectid management. I.e it ensures that
> the first objectid is always 256 in an fs tree.
>
> Signed-off-by: Nikolay Borisov
Some minor issues below, but I'd like btrfs folks to help review to see
if the free objectid management test is reasonable.
> ---
>
> Resend it as I fudged btrfs' mailing list address so the patch didn't get to
> it.
> tests/btrfs/154 | 80 +
> tests/btrfs/154.out | 2 ++
> tests/btrfs/group | 1 +
> 3 files changed, 83 insertions(+)
> create mode 100755 tests/btrfs/154
> create mode 100644 tests/btrfs/154.out
>
> diff --git a/tests/btrfs/154 b/tests/btrfs/154
> new file mode 100755
> index ..6aee204e05cb
> --- /dev/null
> +++ b/tests/btrfs/154
> @@ -0,0 +1,80 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (C) 2020 SUSE Linux Products GmbH. All Rights Reserved.
> +#
> +# FS QA Test 154
> +#
> +# Test correct operation of free objectid related functionality
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs btrfs
> +_require_scratch
> +
> +
> +_scratch_mkfs > /dev/null
> +_scratch_mount
> +
> +# create a new subvolume to validate its objectid is initialized accordingly
> +$BTRFS_UTIL_PROG subvolume create $SCRATCH_MNT/newvol >> $seqres.full 2>&1 \
> + || _fail "couldn't create subvol"
> +
> +$BTRFS_UTIL_PROG inspect-internal dump-tree -t1 $SCRATCH_DEV \
> + | grep -q "256 ROOT_ITEM" || _fail "First subvol with id 256 doesn't
> exist"
This also requires
_require_btrfs_command inspect-internal dump-tree
And it's better to explain why '256' is the expected value, where does
it come from.
> +
> +# create new file in the new subvolume to validate its objectid is set as
> +# expected
> +touch $SCRATCH_MNT/newvol/file1
> +
> +# ensure we have consistent view on-disk
> +sync
> +
> +# get output related to the new root's dir entry
> +output=$($BTRFS_UTIL_PROG inspect-internal dump-tree -t5 $SCRATCH_DEV | grep
> -A2 "256 DIR_ITEM 1903355334")
> +
> +# get the objectid of the new root
> +new_root_id=$(echo "$output" | awk '/location key/{printf $3}' | tr -d '(')
I'd dump the output to a tmp file (and the following outputs in the
test), as saving the output in a variable may cause unexpected results,
something like ignoring "\n", and make it harder to debug.
> +[ $new_root_id -eq 256 ] || _fail "New root id not equal to 256"
> +
> +# the given root should always be item number 2, since it's the only item
> +item_seq=$(echo "$output" | awk '/item/ {printf $2}')
$AWK_PROG
Thanks,
Eryu
> +[ $item_seq -eq 2 ] || _fail "New root not at item idx 2"
> +
> +# now parse the structure of the new subvol's tree
> +output=$($BTRFS_UTIL_PROG inspect-internal dump-tree -t256 $SCRATCH_DEV)
> +
> +# this is the subvol's own ino
> +first_ino=$(echo "$output" | awk '/item 0/{printf $4}' | tr -d '(')
> +[ $first_ino -eq 256 ] || _fail "First ino objectid in subvol not 256"
> +
> +# this is ino of first file in subvol
> +second_ino=$(echo "$output" | awk '/item 4/{printf $4}' | tr -d '(')
> +[ $second_ino -eq 257 ] || _fail "Second ino objectid in subvol not 257"
> +
> +# success, all done
> +echo "Silence is golden"
> +status=0
> +exit
> diff --git a/tests/btrfs/154.out b/tests/btrfs/154.out
> new file mode 100644
> index ..a18c304305c4
> --- /dev/null
> +++ b/tests/btrfs/154.out
> @@ -0,0 +1,2 @@
> +QA output created by 154
> +Silence is golden
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index d18450c7552e..44d33222def0 100644
> --- a/tests/btrfs/group
> +++ b/tests/btrfs/group
> @@ -156,6 +156,7 @@
> 151 auto quick volume
> 152 auto quick metadata qgroup send
> 153 auto quick qgroup limit
> +154 auto quick
> 155 auto quick send
> 156 auto quick trim balance
> 157 auto quick raid
> --
> 2.17.1
Re: [PATCH 2/2] fstest: btrfs/197: test for alien devices
On Mon, Oct 07, 2019 at 05:41:01PM +0800, Anand Jain wrote:
> Test if btrfs.ko sucessfully identifies and reports the missing device,
> if the missed device contians no btrfs magic string.
>
> Signed-off-by: Anand Jain
> ---
> tests/btrfs/197 | 79
> +
> tests/btrfs/197.out | 25 +
> tests/btrfs/group | 1 +
> 3 files changed, 105 insertions(+)
> create mode 100755 tests/btrfs/197
> create mode 100644 tests/btrfs/197.out
>
> diff --git a/tests/btrfs/197 b/tests/btrfs/197
> new file mode 100755
> index ..82e1a299ca43
> --- /dev/null
> +++ b/tests/btrfs/197
> @@ -0,0 +1,79 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2019 Oracle. All Rights Reserved.
> +#
> +# FS QA Test 197
> +#
> +# Test stale and alien device in the fs devices list.
> +# Similar to the testcase btrfs/196 except that here the alien device no more
> +# contains the btrfs superblock.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/filter.btrfs
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs generic
> +_supported_os Linux
> +_require_command "$WIPEFS_PROG" wipefs
> +_require_scratch
> +_require_scratch_dev_pool 4
> +
> +workout()
> +{
> + raid=$1
> + device_nr=$2
> +
> + echo $raid
> + _scratch_dev_pool_get $device_nr
> +
> + _scratch_pool_mkfs "-d$raid -m$raid" >> $seqres.full 2>&1 || \
> + _fail "mkfs failed"
> +
> + # Make device_1 an alien btrfs device for the raid created above by
> + # adding it to the $TEST_DIR
Stale comments above.
Otherwise looks fine to me.
> +
> + # don't test with the first device as auto fs check (_check_scratch_fs)
> + # picks the first device
> + device_1=$(echo $SCRATCH_DEV_POOL | awk '{print $2}')
> + $WIPEFS_PROG -a $device_1 >> $seqres.full 2>&1
If creating a new btrfs works for btrfs/196, I wonder if we could merge
the two tests into one test, firstly create a new fs & degraded mount,
then wipefs & degraded mount.
Thanks,
Eryu
> +
> + device_2=$(echo $SCRATCH_DEV_POOL | awk '{print $1}')
> + _mount -o degraded $device_2 $SCRATCH_MNT
> + # Check if missing device is reported as in the 196.out
> + $BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT | \
> + _filter_btrfs_filesystem_show
> +
> + _scratch_unmount
> + _scratch_dev_pool_put
> +}
> +
> +workout "raid1" "2"
> +workout "raid5" "3"
> +workout "raid6" "4"
> +workout "raid10" "4"
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/btrfs/197.out b/tests/btrfs/197.out
> new file mode 100644
> index ..79237b854b5a
> --- /dev/null
> +++ b/tests/btrfs/197.out
> @@ -0,0 +1,25 @@
> +QA output created by 197
> +raid1
> +Label: none uuid:
> + Total devices FS bytes used
> + devid size used path SCRATCH_DEV
> + *** Some devices missing
> +
> +raid5
> +Label: none uuid:
> + Total devices FS bytes used
> + devid size used path SCRATCH_DEV
> + *** Some devices missing
> +
> +raid6
> +Label: none uuid:
> + Total devices FS bytes used
> + devid size used path SCRATCH_DEV
> + *** Some devices missing
> +
> +raid10
> +Label: none uuid:
> + Total devices FS bytes used
> + devid size used path SCRATCH_DEV
> + *** Some devices missing
> +
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index c86ea2516397..f2eac5c20712 100644
> --- a/tests/btrfs/group
> +++ b/tests/btrfs/group
> @@ -199,3 +199,4 @@
> 194 auto volume
> 195 auto volume
> 196 auto quick volume
> +197 auto quick volume
> --
> 1.8.3.1
>
Re: [PATCH 1/2] fstest: btrfs/196: test for alien btrfs-devices
On Mon, Oct 07, 2019 at 05:41:00PM +0800, Anand Jain wrote:
> Test if btrfs.ko sucessfully identifies and reports the missing device,
> if the missed device contians someother btrfs.
>
> Signed-off-by: Anand Jain
> ---
> tests/btrfs/196 | 77
> +
> tests/btrfs/196.out | 25 +
> tests/btrfs/group | 1 +
> 3 files changed, 103 insertions(+)
> create mode 100755 tests/btrfs/196
> create mode 100644 tests/btrfs/196.out
>
> diff --git a/tests/btrfs/196 b/tests/btrfs/196
> new file mode 100755
> index ..e35cdce492e5
> --- /dev/null
> +++ b/tests/btrfs/196
> @@ -0,0 +1,77 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2019 Oracle. All Rights Reserved.
> +#
> +# FS QA Test 196
> +#
> +# Test stale and alien btrfs-device in the fs devices list.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/filter.btrfs
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs generic
> +_supported_os Linux
_supported_os btrfs
> +_require_scratch
> +_require_scratch_dev_pool 4
> +
> +workout()
> +{
> + raid=$1
> + device_nr=$2
> +
> + echo $raid
> + _scratch_dev_pool_get $device_nr
> +
> + _scratch_pool_mkfs "-d$raid -m$raid" >> $seqres.full 2>&1 || \
> + _fail "mkfs failed"
> +
> + # Make device_1 an alien btrfs device for the raid created above by
> + # adding it to the $TEST_DIR
Instead of adding $device_1 to $TEST_DIR, does creating a new btrfs on
$device_1 work? I'm a bit worried that changing $TEST_DIR/$TEST_DEV
configuration would have some side effects, e.g. we add it to $TEST_DIR
and cancle the test before removing it, the $TEST_DEV will end up in an
unexpected state in next test run. (Though we could remove the device in
_cleanup, this is just an example.)
> +
> + # don't test with the first device as auto fs check (_check_scratch_fs)
> + # picks the first device
> + device_1=$(echo $SCRATCH_DEV_POOL | awk '{print $2}')
> + $BTRFS_UTIL_PROG device add -f "$device_1" "$TEST_DIR"
> +
> + device_2=$(echo $SCRATCH_DEV_POOL | awk '{print $1}')
> + _mount -o degraded $device_2 $SCRATCH_MNT
> + # Check if missing device is reported as in the 196.out
Test could be renumbered, no need to hardcode 196.out, just ".out file"
would be fine.
> + $BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT | \
> + _filter_btrfs_filesystem_show
> +
> + $BTRFS_UTIL_PROG device remove "$device_1" "$TEST_DIR"
I think we should remove $device_1 in _cleanup if it's not empty in case
test exits unexpectly. But as mentioned above, better to avoid adding
new device to $TEST_DIR.
Thanks,
Eryu
> + _scratch_unmount
> + _scratch_dev_pool_put
> +}
> +
> +workout "raid1" "2"
> +workout "raid5" "3"
> +workout "raid6" "4"
> +workout "raid10" "4"
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/btrfs/196.out b/tests/btrfs/196.out
> new file mode 100644
> index ..311ae9e2f46a
> --- /dev/null
> +++ b/tests/btrfs/196.out
> @@ -0,0 +1,25 @@
> +QA output created by 196
> +raid1
> +Label: none uuid:
> + Total devices FS bytes used
> + devid size used path SCRATCH_DEV
> + *** Some devices missing
> +
> +raid5
> +Label: none uuid:
> + Total devices FS bytes used
> + devid size used path SCRATCH_DEV
> + *** Some devices missing
> +
> +raid6
> +Label: none uuid:
> + Total devices FS bytes used
> + devid size used path SCRATCH_DEV
> + *** Some devices missing
> +
> +raid10
> +Label: none uuid:
> + Total devices FS bytes used
> + devid size used path SCRATCH_DEV
> + *** Some devices missing
> +
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index 3ce6fa4628d8..c86ea2516397 100644
> --- a/tests/btrfs/group
> +++ b/tests/btrfs/group
> @@ -198,3 +198,4 @@
> 193 auto quick qgroup enospc limit
> 194 auto volume
> 195 auto volume
> +196 auto quick volume
> --
> 1.8.3.1
>
Re: [PATCH v2 2/2] btrfs: Add test for btrfs balance convert functionality
Hi Qu,
On Tue, Oct 01, 2019 at 12:04:19PM +0300, Nikolay Borisov wrote:
> Add basic test to ensure btrfs conversion functionality is tested. This test
> exercies conversion to all possible types of the data portion. This is
> sufficient
> since from the POV of relocation we are only moving blockgroups.
>
> Signed-off-by: Nikolay Borisov
Would you please help review this v2 as well? Thanks a lot!
Eryu
> ---
> tests/btrfs/194 | 84
> +
> tests/btrfs/194.out | 2 ++
> tests/btrfs/group | 1 +
> 3 files changed, 87 insertions(+)
> create mode 100755 tests/btrfs/194
> create mode 100644 tests/btrfs/194.out
>
> diff --git a/tests/btrfs/194 b/tests/btrfs/194
> new file mode 100755
> index ..39b6e0a969c1
> --- /dev/null
> +++ b/tests/btrfs/194
> @@ -0,0 +1,84 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2019 SUSE Linux Products GmbH. All Rights Reserved.
> +#
> +# FS QA Test 194
> +#
> +# Test raid profile conversion. It's sufficient to test all dest profiles as
> +# source profiles just rely on being able to read the metadata.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch_dev_pool 4
> +
> +
> +declare -a TEST_VECTORS=(
> +# $nr_dev_min:$data:$metadata:$data_convert:$metadata_convert
> +"4:single:raid1"
> +"4:single:raid0"
> +"4:single:raid10"
> +"4:single:dup"
> +"4:single:raid5"
> +"4:single:raid6"
> +"2:raid1:single"
> +)
> +
> +run_testcase() {
> + IFS=':' read -ra args <<< $1
> + num_disks=${args[0]}
> + src_type=${args[1]}
> + dst_type=${args[2]}
> +
> + _scratch_dev_pool_get $num_disks
> +
> + echo "=== Running test: $1 ===" >> $seqres.full
> +
> + _scratch_pool_mkfs -d$src_type >> $seqres.full 2>&1
> + _scratch_mount
> +
> + # Create random filesystem with 20k write ops
> + run_check $FSSTRESS_PROG -d $SCRATCH_MNT -w -n 1 $FSSTRESS_AVOID
> +
> + $BTRFS_UTIL_PROG balance start -f -dconvert=$dst_type $SCRATCH_MNT >>
> $seqres.full 2>&1
> + [ $? -eq 0 ] || echo "$1: Failed convert"
> +
> + $BTRFS_UTIL_PROG scrub start -B $SCRATCH_MNT >>$seqres.full 2>&1
> + [ $? -eq 0 ] || echo "$1: Scrub failed"
> +
> + _scratch_unmount
> + _check_btrfs_filesystem $SCRATCH_DEV
> + _scratch_dev_pool_put
> +}
> +
> +for i in "${TEST_VECTORS[@]}"; do
> + run_testcase $i
> +done
> +
> +echo "Silence is golden"
> +status=0
> +exit
> diff --git a/tests/btrfs/194.out b/tests/btrfs/194.out
> new file mode 100644
> index ..7bfd50ffb5a4
> --- /dev/null
> +++ b/tests/btrfs/194.out
> @@ -0,0 +1,2 @@
> +QA output created by 194
> +Silence is golden
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index b92cb12ca66f..a2c0ad87d0f6 100644
> --- a/tests/btrfs/group
> +++ b/tests/btrfs/group
> @@ -196,3 +196,4 @@
> 191 auto quick send dedupe
> 192 auto replay snapshot stress
> 193 auto quick qgroup enospc limit
> +194 auto volume balance
> --
> 2.7.4
>
Re: [PATCH][v2] btrfs/194: add a test for multi-subvolume fsyncing
On Wed, Oct 02, 2019 at 02:41:33PM -0400, Josef Bacik wrote:
> I discovered a problem in btrfs where we'd end up pointing at a block we
> hadn't written out yet. This is triggered by a race when two different
> files on two different subvolumes fsync. This test exercises this path
> with dm-log-writes, and then replays the log at every FUA to verify the
> file system is still mountable and the log is replayable.
>
> This test is to verify the fix
>
> btrfs: fix incorrect updating of log root tree
>
> actually fixed the problem.
>
> Signed-off-by: Josef Bacik
> ---
> v1->v2:
> - added the patchname related to this test in the comments and changelog.
> - running fio makes it use 400mib of shared memory, so running 50 of them is
> impossible on boxes that don't have hundreds of gib of RAM. Fixed this to
> just generate a fio config so we can run 1 fio instance with 50 threads
> which
> makes it not OOM boxes with tiny amounts of RAM.
> - fixed some formatting things that Filipe pointed out.
>
> tests/btrfs/194 | 111
> tests/btrfs/194.out | 2 +
> tests/btrfs/group | 1 +
> 3 files changed, 114 insertions(+)
> create mode 100755 tests/btrfs/194
> create mode 100644 tests/btrfs/194.out
>
> diff --git a/tests/btrfs/194 b/tests/btrfs/194
> new file mode 100755
> index ..b98064e2
> --- /dev/null
> +++ b/tests/btrfs/194
> @@ -0,0 +1,111 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2019 Facebook. All Rights Reserved.
> +#
> +# FS QA Test 194
> +#
> +# Test multi subvolume fsync to test a bug where we'd end up pointing at a
> block
> +# we haven't written. This was fixed by the patch
> +#
> +# btrfs: fix incorrect updating of log root tree
> +#
> +# Will do log replay and check the filesystem.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +fio_config=$tmp.fio
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + _log_writes_cleanup &> /dev/null
> + _dmthin_cleanup
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/dmthin
> +. ./common/dmlogwrites
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs generic
> +_supported_os Linux
> +
> +# Use thin device as replay device, which requires $SCRATCH_DEV
> +_require_scratch_nocheck
> +# and we need extra device as log device
> +_require_log_writes
> +_require_dm_target thin-pool
> +
> +cat >$fio_config < +[global]
> +readwrite=write
> +fallocate=none
> +bs=4k
> +fsync=1
> +size=128k
> +EOF
> +
> +for i in $(seq 0 49); do
> + echo "[foo$i]" >> $fio_config
> + echo "filename=$SCRATCH_MNT/$i/file" >> $fio_config
> +done
> +
> +_require_fio $fio_config
> +
> +cat $fio_config >> $seqres.full
> +
> +# Use a thin device to provide deterministic discard behavior. Discards are
> used
> +# by the log replay tool for fast zeroing to prevent out-of-order replay
> issues.
> +_test_unmount
Why umount $TEST_DEV here?
> +_dmthin_init $devsize $devsize $csize $lowspace
'devsize' 'csize' and 'lowspace' are not defined, and _dmthin_init uses
all defaults. Define them or just use the defaults?
Thanks,
Eryu
> +_log_writes_init $DMTHIN_VOL_DEV
> +_log_writes_mkfs >> $seqres.full 2>&1
> +_log_writes_mark mkfs
> +
> +_log_writes_mount
> +
> +# First create all the subvolumes
> +for i in $(seq 0 49); do
> + $BTRFS_UTIL_PROG subvolume create "$SCRATCH_MNT/$i" > /dev/null
> +done
> +
> +$FIO_PROG $fio_config > /dev/null 2>&1
> +_log_writes_unmount
> +
> +_log_writes_remove
> +prev=$(_log_writes_mark_to_entry_number mkfs)
> +[ -z "$prev" ] && _fail "failed to locate entry mark 'mkfs'"
> +cur=$(_log_writes_find_next_fua $prev)
> +[ -z "$cur" ] && _fail "failed to locate next FUA write"
> +
> +while [ ! -z "$cur" ]; do
> + _log_writes_replay_log_range $cur $DMTHIN_VOL_DEV >> $seqres.full
> +
> + # We need to mount the fs because btrfsck won't bother checking the log.
> + _dmthin_mount
> + _dmthin_check_fs
> +
> + prev=$cur
> + cur=$(_log_writes_find_next_fua $(($cur + 1)))
> + [ -z "$cur" ] && break
> +done
> +
> +echo "Silence is golden"
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/btrfs/194.out b/tests/btrfs/194.out
> new file mode 100644
> index ..7bfd50ff
> --- /dev/null
> +++ b/tests/btrfs/194.out
> @@ -0,0 +1,2 @@
> +QA output created by 194
> +Silence is golden
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index b92cb12c..0d0e1bba 100644
> --- a/tests/btrfs/group
> +++ b/tests/btrfs/group
> @@ -196,3 +196,4 @@
> 191 auto quick send dedupe
> 192 auto replay snapshot stress
> 193 auto quick qgroup enospc limit
> +194 auto metadata log volume
> --
> 2.21.0
>
Re: [PATCH][v2] btrfs/194: add a test for multi-subvolume fsyncing
On Thu, Oct 03, 2019 at 12:12:36PM +0100, Filipe Manana wrote:
> On Thu, Oct 3, 2019 at 11:59 AM Filipe Manana wrote:
> >
> > On Wed, Oct 2, 2019 at 7:44 PM Josef Bacik wrote:
> > >
> > > I discovered a problem in btrfs where we'd end up pointing at a block we
> > > hadn't written out yet. This is triggered by a race when two different
> > > files on two different subvolumes fsync. This test exercises this path
> > > with dm-log-writes, and then replays the log at every FUA to verify the
> > > file system is still mountable and the log is replayable.
> > >
> > > This test is to verify the fix
> > >
> > > btrfs: fix incorrect updating of log root tree
> > >
> > > actually fixed the problem.
> > >
> > > Signed-off-by: Josef Bacik
> >
> > Reviewed-by: Filipe Manana
> >
> > It's working now.
> > Confirmed this triggers the bug, and after about 4 hours of this
> > running with the btrfs patch, it doesn't trigger the bug anymore.
> >
> > Thanks!
> >
> > > ---
> > > v1->v2:
> > > - added the patchname related to this test in the comments and changelog.
> > > - running fio makes it use 400mib of shared memory, so running 50 of them
> > > is
> > > impossible on boxes that don't have hundreds of gib of RAM. Fixed this
> > > to
> > > just generate a fio config so we can run 1 fio instance with 50 threads
> > > which
> > > makes it not OOM boxes with tiny amounts of RAM.
> > > - fixed some formatting things that Filipe pointed out.
> > >
> > > tests/btrfs/194 | 111
> > > tests/btrfs/194.out | 2 +
> > > tests/btrfs/group | 1 +
> > > 3 files changed, 114 insertions(+)
> > > create mode 100755 tests/btrfs/194
> > > create mode 100644 tests/btrfs/194.out
> > >
> > > diff --git a/tests/btrfs/194 b/tests/btrfs/194
> > > new file mode 100755
> > > index ..b98064e2
> > > --- /dev/null
> > > +++ b/tests/btrfs/194
> > > @@ -0,0 +1,111 @@
> > > +#! /bin/bash
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +# Copyright (c) 2019 Facebook. All Rights Reserved.
> > > +#
> > > +# FS QA Test 194
> > > +#
> > > +# Test multi subvolume fsync to test a bug where we'd end up pointing at
> > > a block
> > > +# we haven't written. This was fixed by the patch
> > > +#
> > > +# btrfs: fix incorrect updating of log root tree
> > > +#
> > > +# Will do log replay and check the filesystem.
> > > +#
> > > +seq=`basename $0`
> > > +seqres=$RESULT_DIR/$seq
> > > +echo "QA output created by $seq"
> > > +
> > > +here=`pwd`
> > > +tmp=/tmp/$$
> > > +fio_config=$tmp.fio
> > > +status=1 # failure is the default!
> > > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > > +
> > > +_cleanup()
> > > +{
> > > + cd /
> > > + _log_writes_cleanup &> /dev/null
> > > + _dmthin_cleanup
> > > + rm -f $tmp.*
> > > +}
> > > +
> > > +# get standard environment, filters and checks
> > > +. ./common/rc
> > > +. ./common/filter
> > > +. ./common/dmthin
> > > +. ./common/dmlogwrites
> > > +
> > > +# remove previous $seqres.full before test
> > > +rm -f $seqres.full
> > > +
> > > +# real QA test starts here
> > > +
> > > +# Modify as appropriate.
> > > +_supported_fs generic
>
> Btw, only forgot about this.
> Should be: _supported_fs btrfs
>
> Eryu can probably fix that at commit time.
> Thanks.
Sure. Thanks for the review!
Eryu
Re: [PATCH] btrfs: Add regression test for SINGLE profile conversion
On Thu, Sep 26, 2019 at 10:26:35AM +0300, Nikolay Borisov wrote:
> This is a regression test for the bug fixed by
> 'btrfs: Fix a regression which we can't convert to SINGLE profile'
>
> Signed-off-by: Nikolay Borisov
> ---
> tests/btrfs/194 | 52
> tests/btrfs/194.out | 2 ++
> tests/btrfs/group | 1 +
> 3 files changed, 55 insertions(+)
> create mode 100755 tests/btrfs/194
> create mode 100644 tests/btrfs/194.out
>
> diff --git a/tests/btrfs/194 b/tests/btrfs/194
> new file mode 100755
> index ..8935defd3f5e
> --- /dev/null
> +++ b/tests/btrfs/194
> @@ -0,0 +1,52 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2019 SUSE Linux Products GmbH. All Rights Reserved.
> +#
> +# FS QA Test 194
> +#
> +# Test that block groups profile can be converted to SINGLE. This is a
> regression
> +# test for 'btrfs: Fix a regression which we can't convert to SINGLE profile'
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch_dev_pool 2
> +
> +_scratch_dev_pool_get 2
> +_scratch_pool_mkfs -draid1
> +
> +_scratch_mount
> +
> +$BTRFS_UTIL_PROG balance start -dconvert=single $SCRATCH_MNT > $seqres.full
> 2>&1
> +[ $? -eq 0 ] || _fail "Convert failed"
This indicates we're missing profile conversion tests in fstests. I
think it's better to add a test framework/helpers and a full set of
profile conversion tests instead of this raid1->single special case.
e.g.
Define a test case array, which describes what's the src/dst of this
conversion and how many devices this case requires, e.g.
test_cases=(
# $nr_dev_min:$metadata:$data:$metadata_convert:$data_convert
"2:single:single:raid0:raid0"
"2:single:single:raid1:raid1"
"2:single:single:raid1:raid0"
"3:single:single:raid5:raid5"
"4:single:single:raid6:raid6"
"4:single:single:raid10:raid10"
"2:raid0:raid0:raid1:raid0"
"2:raid0:raid0:raid1:raid1"
"3:raid0:raid0:raid5:raid5"
"4:raid0:raid0:raid6:raid6"
"4:raid0:raid0:raid10:raid10"
"2:raid1:raid0:raid1:raid1"
"3:raid1:raid0:raid5:raid5"
"4:raid1:raid0:raid6:raid6"
"4:raid1:raid1:raid10:raid10"
"4:raid5:raid5:raid1:raid1"
"4:raid5:raid5:raid6:raid6"
"4:raid5:raid5:raid10:raid10"
"4:raid6:raid6:raid1:raid1"
"4:raid6:raid6:raid5:raid5"
"4:raid6:raid6:raid10:raid10"
"4:raid10:raid10:raid5:raid5"
"4:raid10:raid10:raid6:raid6"
)
Then, create btrfs in source profile and populate fs with different
kinds of files, compute sha1 digests of these files(fssum?), start
conversion, after conversion compare sha1 digests.
And perhaps we could split the test cases into group and add them to
different tests, in case putting all cases in a single test makes the
test time too long.
Thanks,
Eryu
> +
> +_scratch_umount
> +_scratch_dev_pool_put
> +
> +echo "Silence is golden"
> +status=0
> +exit
> diff --git a/tests/btrfs/194.out b/tests/btrfs/194.out
> new file mode 100644
> index ..7bfd50ffb5a4
> --- /dev/null
> +++ b/tests/btrfs/194.out
> @@ -0,0 +1,2 @@
> +QA output created by 194
> +Silence is golden
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index b92cb12ca66f..6a11eb1b8230 100644
> --- a/tests/btrfs/group
> +++ b/tests/btrfs/group
> @@ -196,3 +196,4 @@
> 191 auto quick send dedupe
> 192 auto replay snapshot stress
> 193 auto quick qgroup enospc limit
> +194 auto quick volume balance
> --
> 2.7.4
>
Re: [PATCH 2/2] fstests: btrfs/011: Handle finished scrub/replace operation gracefully
On Wed, Sep 18, 2019 at 02:56:26PM +0800, Qu Wenruo wrote:
> [BUG]
> When btrfs/011 is executed on a fast enough system (fully memory backed
> VM, with test device has unsafe cache mode), the test can fail like
> this:
>
> btrfs/011 43s ... [failed, exit status 1]- output mismatch (see
> /home/adam/xfstests-dev/results//btrfs/011.out.bad)
> --- tests/btrfs/011.out 2019-07-22 14:13:44.64326 +0800
> +++ /home/adam/xfstests-dev/results//btrfs/011.out.bad 2019-09-18
> 14:49:28.308798022 +0800
> @@ -1,3 +1,4 @@
> QA output created by 011
> *** test btrfs replace
> -*** done
> +failed: '/usr/bin/btrfs replace cancel /mnt/scratch'
> +(see /home/adam/xfstests-dev/results//btrfs/011.full for details)
> ...
>
> [CAUSE]
> Looking into the full output, it shows:
> ...
> Replace from /dev/mapper/test-scratch1 to /dev/mapper/test-scratch2
>
> # /usr/bin/btrfs replace start -f /dev/mapper/test-scratch1
> /dev/mapper/test-scratch2 /mnt/scratch
> # /usr/bin/btrfs replace cancel /mnt/scratch
> INFO: ioctl(DEV_REPLACE_CANCEL)"/mnt/scratch": not started
> failed: '/usr/bin/btrfs replace cancel /mnt/scratch'
>
> So this means the replace is already finished before we cancel it.
> For fast system, it's very common.
Does generate heavier load & more data make replace operation last
longer? e.g. make more 'noise' by running fsstress instead of dumping
/dev/urandom before starting replace.
And does sleep shorter time (0.5s?) before cancel work?
Thanks,
Eryu
>
> [FIX]
> Instead of using _run_btrfs_util_prog which requires 0 as return value,
> we just call "$BTRFS_UTIL_PROG replace cancel" and ignore all its
> stderr/stdout, and completely rely on "$BTRFS_UTIL_PROG replace status"
> output to verify the work.
>
> Furthermore if we finished replac before cancelling it, we should
> replace again to switch the device back, or after the test case, btrfs
> check will fail as there is no valid btrfs on that replaced device.
>
> Signed-off-by: Qu Wenruo
> ---
> tests/btrfs/011 | 16 ++--
> 1 file changed, 14 insertions(+), 2 deletions(-)
>
> diff --git a/tests/btrfs/011 b/tests/btrfs/011
> index 89bb4d11..858b00e8 100755
> --- a/tests/btrfs/011
> +++ b/tests/btrfs/011
> @@ -148,13 +148,25 @@ btrfs_replace_test()
> # background the replace operation (no '-B' option given)
> _run_btrfs_util_prog replace start -f $replace_options
> $source_dev $target_dev $SCRATCH_MNT
> sleep 1
> - _run_btrfs_util_prog replace cancel $SCRATCH_MNT
> + # 1s is enough for fast system to finish replace, so here we
> + # ignore all the output, completely rely on later status
> + # output to determine
> + $BTRFS_UTIL_PROG replace cancel $SCRATCH_MNT &> /dev/null
>
> # 'replace status' waits for the replace operation to finish
> # before the status is printed
> $BTRFS_UTIL_PROG replace status $SCRATCH_MNT > $tmp.tmp 2>&1
> cat $tmp.tmp >> $seqres.full
> - grep -q canceled $tmp.tmp || _fail "btrfs replace status
> (canceled) failed"
> + grep -q -e canceled -e finished $tmp.tmp ||\
> + _fail "btrfs replace status (canceled) failed"
> +
> + # If replace finished before cancel, replace them back or
> + # the final fsck after test case will fail as there is no btrfs
> + # on the $source_dev anymore
> + if grep -q -e finished $tmp.tmp ; then
> + $BTRFS_UTIL_PROG replace start -Bf $replace_options \
> + $target_dev $source_dev $SCRATCH_MNT
> + fi
> else
> if [ "${quick}Q" = "thoroughQ" ]; then
> # On current hardware, the thorough test runs
> --
> 2.22.0
Re: [PATCH v2] fstests: btrfs: Verify falloc on multiple holes won't cause qgroup reserved data space leak
On Sun, Sep 15, 2019 at 10:21:14AM +0100, Filipe Manana wrote:
> On Sun, Sep 15, 2019 at 8:32 AM Qu Wenruo wrote:
> >
> > Add a test case where falloc is called on multiple holes with qgroup
> > enabled.
> >
> > This can cause qgroup reserved data space leak and false EDQUOT error
> > even we're not reaching the limit.
> >
> > The fix is titled:
> > "btrfs: qgroup: Fix the wrong target io_tree when freeing
> > reserved data space"
> >
> > Signed-off-by: Qu Wenruo
> > ---
> > changelog:
> > v2:
> > - Remove the unnecessary loop
> > The loop itself is just to ensure we leak as much space as possible.
> > However even one loop is already enough to fail the final
> > verification write, so remove the loop and modify the golden output.
> > ---
> > tests/btrfs/192 | 70 +
> > tests/btrfs/192.out | 8 ++
> > tests/btrfs/group | 1 +
> > 3 files changed, 79 insertions(+)
> > create mode 100755 tests/btrfs/192
> > create mode 100644 tests/btrfs/192.out
> >
> > diff --git a/tests/btrfs/192 b/tests/btrfs/192
> > new file mode 100755
> > index ..1abd7838
> > --- /dev/null
> > +++ b/tests/btrfs/192
> > @@ -0,0 +1,70 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (C) 2019 SUSE Linux Products GmbH. All Rights Reserved.
> > +#
> > +# FS QA Test 192
> > +#
> > +# Test if btrfs is going to leak qgroup reserved data space when
> > +# falloc on multiple holes fails.
> > +# The fix is titled:
> > +# "btrfs: qgroup: Fix the wrong target io_tree when freeing reserved data
> > space"
> > +#
> > +seq=`basename $0`
> > +seqres=$RESULT_DIR/$seq
> > +echo "QA output created by $seq"
> > +
> > +here=`pwd`
> > +tmp=/tmp/$$
> > +status=1 # failure is the default!
> > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > +
> > +_cleanup()
> > +{
> > + cd /
> > + rm -f $tmp.*
> > +}
> > +
> > +# get standard environment, filters and checks
> > +. ./common/rc
> > +. ./common/filter
> > +
> > +# remove previous $seqres.full before test
> > +rm -f $seqres.full
> > +
> > +# real QA test starts here
> > +
> > +# Modify as appropriate.
> > +_supported_fs btrfs
> > +_supported_os Linux
> > +_require_scratch
> > +_require_xfs_io_command falloc
> > +
> > +_scratch_mkfs > /dev/null
> > +_scratch_mount
> > +
> > +$BTRFS_UTIL_PROG quota enable "$SCRATCH_MNT" > /dev/null
> > +$BTRFS_UTIL_PROG quota rescan -w "$SCRATCH_MNT" > /dev/null
> > +$BTRFS_UTIL_PROG qgroup limit -e 256M "$SCRATCH_MNT"
> > +
> > +# Create a file with the following layout:
> > +# 0 128M 256M 384M
> > +# | Hole |4K| Hole |4K| Hole |
> > +# The total hole size will be 384M - 8k
> > +truncate -s 384m "$SCRATCH_MNT/file"
> > +$XFS_IO_PROG -c "pwrite 128m 4k" -c "pwrite 256m 4k" \
> > + "$SCRATCH_MNT/file" | _filter_xfs_io
> > +
> > +# Falloc 0~384M range, it's going to fail due to the qgroup limit
> > +$XFS_IO_PROG -c "falloc 0 384m" "$SCRATCH_MNT/file" |\
> > + _filter_xfs_io_error
>
> This can be in a single line, as it doesn't go beyond 80 characters.
> It doesn't hurt, but the use of the error filter isn't necessary here
> at the moment.
>
> > +rm "$SCRATCH_MNT/file"
>
> rm -f, in case one has " alias rm='rm -i' " in its environment.
>
> > +
> > +# Ensure above delete reaches disk and free some space
> > +sync
> > +
> > +# We should be able to write at least 3/4 of the limit
> > +$XFS_IO_PROG -f -c "pwrite 0 192m" "$SCRATCH_MNT/file" | _filter_xfs_io
> > +
> > +# success, all done
> > +status=0
> > +exit
> > diff --git a/tests/btrfs/192.out b/tests/btrfs/192.out
> > new file mode 100644
> > index ..654adf48
> > --- /dev/null
> > +++ b/tests/btrfs/192.out
> > @@ -0,0 +1,8 @@
> > +QA output created by 192
> > +wrote 4096/4096 bytes at offset 134217728
> > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> > +wrote 4096/4096 bytes at offset 268435456
> > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> > +fallocate: Disk quota exceeded
> > +wrote 201326592/201326592 bytes at offset 0
> > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> > diff --git a/tests/btrfs/group b/tests/btrfs/group
> > index 2474d43e..160fe927 100644
> > --- a/tests/btrfs/group
> > +++ b/tests/btrfs/group
> > @@ -194,3 +194,4 @@
> > 189 auto quick send clone
> > 190 auto quick replay balance qgroup
> > 191 auto quick send dedupe
> > +192 auto qgroup fast enospc limit
>
> fast -> quick?
> (we don't have a group named "fast" yet)
Yeah, I've changed it to 'quick' on commit. And I can fix your other
comments on commit as well.
>
> Anyway, small things that can be fixed at commit time.
>
> Reviewed-by: Filipe Manana
Thanks for the review!
Eryu
Re: [PATCH] fstests: btrfs: Verify falloc on multiple holes won't cause qgroup reserved data space leak
On Fri, Sep 13, 2019 at 09:51:51AM +0800, Qu Wenruo wrote:
> Add a test case where falloc is called on multiple holes with qgroup
> enabled.
>
> This can cause qgroup reserved data space leak and false EDQUOT error
> even we're not reaching the limit.
>
> The fix is titled:
> "btrfs: qgroup: Fix the wrong target io_tree when freeing
> reserved data space"
>
> Signed-off-by: Qu Wenruo
> ---
> tests/btrfs/192 | 72 +
> tests/btrfs/192.out | 18
> tests/btrfs/group | 1 +
> 3 files changed, 91 insertions(+)
> create mode 100755 tests/btrfs/192
> create mode 100644 tests/btrfs/192.out
>
> diff --git a/tests/btrfs/192 b/tests/btrfs/192
> new file mode 100755
> index ..361b6d92
> --- /dev/null
> +++ b/tests/btrfs/192
> @@ -0,0 +1,72 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (C) 2019 SUSE Linux Products GmbH. All Rights Reserved.
> +#
> +# FS QA Test 192
> +#
> +# Test if btrfs is going to leak qgroup reserved data space when
> +# falloc on multiple holes fails.
> +# The fix is titled:
> +# "btrfs: qgroup: Fix the wrong target io_tree when freeing reserved data
> space"
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch
> +_require_xfs_io_command falloc
> +
> +_scratch_mkfs > /dev/null
> +_scratch_mount
> +
> +$BTRFS_UTIL_PROG quota enable "$SCRATCH_MNT" > /dev/null
> +$BTRFS_UTIL_PROG quota rescan -w "$SCRATCH_MNT" > /dev/null
> +$BTRFS_UTIL_PROG qgroup limit -e 256M "$SCRATCH_MNT"
> +
> +for i in $(seq 3); do
Why do we need to loop 3 times? Some comments would be good, or we could
just remove the loop?
Other than that the test looks fine to me.
Thanks,
Eryu
> + # Create a file with the following layout:
> + # 0 128M 256M 384M
> + # | Hole |4K| Hole |4K| Hole |
> + # The total hole size will be 384M - 8k
> + truncate -s 384m "$SCRATCH_MNT/file"
> + $XFS_IO_PROG -c "pwrite 128m 4k" -c "pwrite 256m 4k" \
> + "$SCRATCH_MNT/file" | _filter_xfs_io
> +
> + # Falloc 0~384M range, it's going to fail due to the qgroup limit
> + $XFS_IO_PROG -c "falloc 0 384m" "$SCRATCH_MNT/file" |\
> + _filter_xfs_io_error
> + rm "$SCRATCH_MNT/file"
> +
> + # Ensure above delete reaches disk and free some space
> + sync
> +done
> +
> +# We should be able to write at least 3/4 of the limit
> +$XFS_IO_PROG -f -c "pwrite 0 192m" "$SCRATCH_MNT/file" | _filter_xfs_io
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/btrfs/192.out b/tests/btrfs/192.out
> new file mode 100644
> index ..13bc6036
> --- /dev/null
> +++ b/tests/btrfs/192.out
> @@ -0,0 +1,18 @@
> +QA output created by 192
> +wrote 4096/4096 bytes at offset 134217728
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +wrote 4096/4096 bytes at offset 268435456
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +fallocate: Disk quota exceeded
> +wrote 4096/4096 bytes at offset 134217728
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +wrote 4096/4096 bytes at offset 268435456
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +fallocate: Disk quota exceeded
> +wrote 4096/4096 bytes at offset 134217728
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +wrote 4096/4096 bytes at offset 268435456
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +fallocate: Disk quota exceeded
> +wrote 201326592/201326592 bytes at offset 0
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index 2474d43e..160fe927 100644
> --- a/tests/btrfs/group
> +++ b/tests/btrfs/group
> @@ -194,3 +194,4 @@
> 189 auto quick send clone
> 190 auto quick replay balance qgroup
> 191 auto quick send dedupe
> +192 auto qgroup fast enospc limit
> --
> 2.22.0
>
Re: [PATCH v3] fstests: btrfs: Check snapshot creation and deletion with dm-logwrites
On Mon, Aug 26, 2019 at 02:20:45PM +0800, Qu Wenruo wrote: > We have generic dm-logwrites with fsstress test case (generic/482), but > it doesn't cover fs specific operations like btrfs snapshot creation and > deletion. > > Furthermore, that test is not heavy enough to bump btrfs tree height by > its short runtime. > > And finally, btrfs check doesn't consider dirty log as an error, unlike > ext*/xfs, that's to say we don't need to mount the fs to replay the log, > but just run btrfs check on the fs is enough. > > So introduce a similar test case but for btrfs only. > > The test case will stress btrfs by: > - Use small nodesize to bump tree height > - Create a base tree which is already high enough > - Trim tree blocks to find possible trim bugs > - Call snapshot creation and deletion along with fsstress > > To utilize replay-log --check and --fsck command, we fix one bug in > replay-log first: > - Return 1 when fsck failed > Original when fsck failed, run_fsck() returns -1, but to make > replay_log prog to return 1, we need to return a minus value, so > fix it by setting @ret to -EUCLEAN when run_fsck() failed, so that > we can detect the fsck failure by simply checking the return value > of replay-log. Sorry, I didn't quite get this. run_fsck() already returns a negative value (-1) on fsck failure (thus @ret is -1 in this case), and replay_log exits with 1 if @ret < 0. All seem fine to me, setting -EUCLEAN doesn't seem necessary to me. Did I miss anything? Anyway, I think this bugfix could be in a separate patch. > > Also it includes certain workaround for btrfs: > - Use no-holes feature > To avoid missing hole file extents. > Although that behavior doesn't follow the on-disk format spec, it > doesn't cause data loss. And will follow the new on-disk format spec > of no-holes feature, so it's better to workaround it. > > And an optimization for btrfs only: > - Use replay-log --fsck/--check command > Since dm-log-writes records bios sequentially, there is no way to > locate certain entry unless we iterate all entries. > This is becoming a big performance penalty if we replay certain a > range, check the fs, then re-execute replay-log to replay another > range. > > We need to records the previous entry location, or we need to > re-iterate all previous entries. > > Thankfully, replay-log has already address it by providing --fsck and > --check command, thus we don't need to break replay-log command. > > Please note, for fast storage (e.g. fast NVME or unsafe cache mode), > it's recommended to use log devices larger than 15G, or we can't record > the full log of just 30s fsstress run. > > Signed-off-by: Qu Wenruo > --- > For the log devices size problem, I have submitted dm-logwrites bio flag > filter support, to filter out data bios. > > But that is not yet merged into kernel, thus we need a large log device > for short run. > > For reference, if using unsafe cache mode for all test devices, on a > system with 32G dual-channel DDR4 3200 RAM, 5G log device will be > filled up in less than 15 seconds. > So to ensure dm-log-writes covers all the operations, one needs at least > 15G log device, and even more if using RAM with more channels. > > Changelog: > v2 > - Better expression/words for comment > - Add requirement for no-holes features > - Use xattr to bump up tree height > So no need for max_inline mount option > - Coding style fixes for function definition > - Add -f for rm to avoid user alias setting > - Add new workload (update time stamp and create new files) for snapshot > workload > - Remove an unnecessary sync call > - Get rid of wrong 2>&1 redirection > - Add to group "snapshot" and "stress" > > v3: > - Add '_require_attrs' and source common/attr > - Introduce '_require_fsck_not_report_dirty_logs_as_error' > - Add comment for the replay-log code fix > - Wait after killing all background fsstress > - Use $BLKDISCARD_PROG instead of plain 'blkdiscard' > - Add trap for snapshot and delete workload > --- > common/config | 1 + > common/dmlogwrites | 44 ++ > src/log-writes/replay-log.c | 2 + > tests/btrfs/192 | 156 > tests/btrfs/192.out | 2 + > tests/btrfs/group | 1 + > 6 files changed, 206 insertions(+) > create mode 100755 tests/btrfs/192 > create mode 100644 tests/btrfs/192.out > > diff --git a/common/config b/common/config > index bd64be62..4c86a492 100644 > --- a/common/config > +++ b/common/config > @@ -183,6 +183,7 @@ export LOGGER_PROG="$(type -P logger)" > export DBENCH_PROG="$(type -P dbench)" > export DMSETUP_PROG="$(type -P dmsetup)" > export WIPEFS_PROG="$(type -P wipefs)" > +export BLKDISCARD_PROG="$(type -P blkdiscard)" > export DUMP_PROG="$(type -P dump)" > export RESTORE_PROG="$(type -P restore)" > export LVM_PROG="$(type -P lvm)" > diff --git a/common/dmlogwrites b/common/dmlogwrites > index ae2cbc6a..474ec5
Re: [PATCH v2] fstests: btrfs: Check snapshot creation and deletion with dm-logwrites
On Mon, Aug 19, 2019 at 04:18:06PM +0800, Qu Wenruo wrote:
> We have generic dm-logwrites with fsstress test case (generic/482), but
> it doesn't cover fs specific operations like btrfs snapshot creation and
> deletion.
>
> Furthermore, that test is not heavy enough to bump btrfs tree height by
> its short runtime.
>
> And finally, btrfs check doesn't consider dirty log as an error, unlike
> ext*/xfs, that's to say we don't need to mount the fs to replay the log,
> but just run btrfs check on the fs is enough.
>
> So introduce a similar test case but for btrfs only.
>
> The test case will stress btrfs by:
> - Use small nodesize to bump tree height
> - Create a base tree which is already high enough
> - Trim tree blocks to find possible trim bugs
> - Call snapshot creation and deletion along with fsstress
>
> Also it includes certain workaround for btrfs:
> - Use no-holes feature
> To avoid missing hole file extents.
> Although that behavior doesn't follow the on-disk format spec, it
> doesn't cause data loss. And will follow the new on-disk format spec
> of no-holes feature, so it's better to workaround it.
>
> And an optimization for btrfs only:
> - Use replay-log --fsck/--check command
> Since dm-log-writes records bios sequentially, there is no way to
> locate certain entry unless we iterate all entries.
> This is becoming a big performance penalty if we replay certain a
> range, check the fs, then re-execute replay-log to replay another
> range.
>
> We need to records the previous entry location, or we need to
> re-iterate all previous entries.
>
> Thankfully, replay-log has already address it by providing --fsck and
> --check command, thus we don't need to break replay-log command.
>
> Please note, for fast storage (e.g. fast NVME or unsafe cache mode),
> it's recommended to use log devices larger than 15G, or we can't record
> the full log of just 30s fsstress run.
>
> Signed-off-by: Qu Wenruo
> ---
> For the log devices size problem, I have submitted dm-logwrites bio flag
> filter support, to filter out data bios.
>
> But that is not yet merged into kernel, thus we need a large log device
> for short run.
>
> Changelog:
> - Better expression/words for comment
> - Add requirement for no-holes features
> - Use xattr to bump up tree height
> So no need for max_inline mount option
> - Coding style fixes for function definition
> - Add -f for rm to avoid user alias setting
> - Add new workload (update time stamp and create new files) for snapshot
> workload
> - Remove an unnecessary sync call
> - Get rid of wrong 2>&1 redirection
> - Add to group "snapshot" and "stress"
> ---
> common/config | 1 +
> common/dmlogwrites | 28 +++
> src/log-writes/replay-log.c | 2 +
> tests/btrfs/192 | 149
> tests/btrfs/192.out | 2 +
> tests/btrfs/group | 1 +
> 6 files changed, 183 insertions(+)
> create mode 100755 tests/btrfs/192
> create mode 100644 tests/btrfs/192.out
>
> diff --git a/common/config b/common/config
> index bd64be62..4c86a492 100644
> --- a/common/config
> +++ b/common/config
> @@ -183,6 +183,7 @@ export LOGGER_PROG="$(type -P logger)"
> export DBENCH_PROG="$(type -P dbench)"
> export DMSETUP_PROG="$(type -P dmsetup)"
> export WIPEFS_PROG="$(type -P wipefs)"
> +export BLKDISCARD_PROG="$(type -P blkdiscard)"
> export DUMP_PROG="$(type -P dump)"
> export RESTORE_PROG="$(type -P restore)"
> export LVM_PROG="$(type -P lvm)"
> diff --git a/common/dmlogwrites b/common/dmlogwrites
> index ae2cbc6a..3292544a 100644
> --- a/common/dmlogwrites
> +++ b/common/dmlogwrites
> @@ -175,3 +175,31 @@ _log_writes_replay_log_range()
> >> $seqres.full 2>&1
> [ $? -ne 0 ] && _fail "replay failed"
> }
> +
> +# Replay and check each fua/flush (specified by $2) point.
> +#
> +# Since dm-log-writes records bio sequentially, even just replaying a range
> +# still needs to iterate all records before the end point.
> +# When number of records grows, it will be unacceptably slow, thus we need
> +# to use relay-log itself to trigger fsck, avoid unnecessary seek.
> +_log_writes_fast_replay_check()
> +{
> + local check_point=$1
> + local blkdev=$2
> + local fsck_command
> +
> + [ -z "$check_point" -o -z "$blkdev" ] && _fail \
> + "check_point and blkdev must be specified for
> _log_writes_fast_replay_check"
> +
> + # Check program must not treat dirty log as an error.
> + # So only btrfs can use this feature
> + if [ $FSTYP = "btrfs" ]; then
> + fsck_command="$BTRFS_UTIL_PROG check $blkdev"
> + else
> + _notrun "fsck of $FSTYP reports dirty jounal/log as error,
> skipping test"
> + fi
Hmm, it'd better to introduce a new require rule and call it in the test
instead of calling _notrun in a do_something function.
> + $here/src/log-writes/replay-log --log $LOGWRITES_DEV \
> + --re
Re: [PATCH] fstests: generic/500 doesn't work for btrfs
On Thu, Aug 15, 2019 at 02:26:59PM -0400, Josef Bacik wrote: > Btrfs does COW, so when we unlink the file we need to update metadata > and write it to a new location, which we can't do because the thinp is > full. This results in an EIO during a metadata write, which makes us > flip read only, thus making it impossible to fstrim the fs. Just make > it so we skip this test for btrfs. > > Signed-off-by: Josef Bacik > --- > tests/generic/500 | 6 ++ > 1 file changed, 6 insertions(+) > > diff --git a/tests/generic/500 b/tests/generic/500 > index 201d8b9f..5cd7126f 100755 > --- a/tests/generic/500 > +++ b/tests/generic/500 > @@ -49,6 +49,12 @@ _supported_os Linux > _require_scratch_nocheck > _require_dm_target thin-pool > > +# The unlink below will result in new metadata blocks for btrfs because of > CoW, > +# and since we've filled the thinp device it'll return EIO, which will make > +# btrfs flip read only, making it fail this test when it just won't work > right > +# for us in the first place. > +test $FSTYP == "btrfs" && _notrun "btrfs doesn't work that way lol" I'm wondering if we could introduce a proper _require rule to cover this case? e.g. require the fs doesn't allocate new blocks on unlink? or something like that. But I'm not sure what's the proper fs feature to require here, any suggestions? Thanks, Eryu > + > # Require underlying device support discard > _scratch_mkfs >>$seqres.full 2>&1 > _scratch_mount > -- > 2.21.0 >
Re: [PATCH] fstests: btrfs: Check snapshot creation and deletion with dm-logwrites
On Fri, Aug 16, 2019 at 05:47:33PM +0800, Qu Wenruo wrote: [...] > >> +$KILLALL_PROG -q $FSSTRESS_PROG &> /dev/null > > > > You're very inconsistent within the same test :) Using both "> > > /dev/null 2>&1" and "&> /dev/null". > > My bad, I mean 2>&1 > /dev/null. > What I mean is output stderr while skip stdout in previous calls. Then just do "2>/dev/null", the test harness will capture both stdout and stderr. "2>&1 > /dev/null" is less used and usually indicates a mis-usage :) Thanks, Eryu
Re: [PATCH v2] generic: test cloning large exents to a file with many small extents
On Fri, Jul 05, 2019 at 11:09:38AM +0100, Filipe Manana wrote:
> On Fri, Jul 5, 2019 at 8:43 AM Eryu Guan wrote:
> >
> > On Fri, Jun 28, 2019 at 11:08:36PM +0100, fdman...@kernel.org wrote:
> > > From: Filipe Manana
> > >
> > > Test that if we clone a file with some large extents into a file that has
> > > many small extents, when the fs is nearly full, the clone operation does
> > > not fail and produces the correct result.
> > >
> > > This is motivated by a bug found in btrfs wich is fixed by the following
> > > patches for the linux kernel:
> > >
> > > [PATCH 1/2] Btrfs: factor out extent dropping code from hole punch
> > > handler
> > > [PATCH 2/2] Btrfs: fix ENOSPC errors, leading to transaction aborts, when
> > > cloning extents
> > >
> > > The test currently passes on xfs.
> > >
> > > Signed-off-by: Filipe Manana
> > > ---
> > >
> > > V2: Use _scratch_cycle_mount instead of _scratch_remount, as we want to
> > > see
> > > if the operation was durably persisted (otherwise we are seeing
> > > content
> > > from the page cache).
> > > Use _reflink instead of calling xfs_io with the reflink command.
> > > Make the comment before filling the filesystem more clear about why it
> > > is done the way it is instead of using _fill_fs.
> > >
> > > tests/generic/558 | 80
> > > +++
> > > tests/generic/558.out | 5
> > > tests/generic/group | 1 +
> > > 3 files changed, 86 insertions(+)
> > > create mode 100755 tests/generic/558
> > > create mode 100644 tests/generic/558.out
> > >
> > > diff --git a/tests/generic/558 b/tests/generic/558
> > > new file mode 100755
> > > index ..f982930d
> > > --- /dev/null
> > > +++ b/tests/generic/558
> > > @@ -0,0 +1,80 @@
> > > +#! /bin/bash
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +# Copyright (C) 2019 SUSE Linux Products GmbH. All Rights Reserved.
> > > +#
> > > +# FSQA Test No. 558
> > > +#
> > > +# Test that if we clone a file with some large extents into a file that
> > > has
> > > +# many small extents, when the fs is nearly full, the clone operation
> > > does
> > > +# not fail and produces the correct result.
> > > +#
> > > +seq=`basename $0`
> > > +seqres=$RESULT_DIR/$seq
> > > +echo "QA output created by $seq"
> > > +tmp=/tmp/$$
> > > +status=1 # failure is the default!
> > > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > > +
> > > +_cleanup()
> > > +{
> > > + cd /
> > > + rm -f $tmp.*
> > > +}
> > > +
> > > +# get standard environment, filters and checks
> > > +. ./common/rc
> > > +. ./common/filter
> > > +. ./common/reflink
> > > +
> > > +# real QA test starts here
> > > +_supported_fs generic
> > > +_supported_os Linux
> > > +_require_scratch_reflink
> > > +
> > > +rm -f $seqres.full
> > > +
> > > +_scratch_mkfs_sized $((512 * 1024 * 1024)) >>$seqres.full 2>&1
> > > +_scratch_mount
> > > +
> > > +file_size=$(( 128 * 1024 * 1024 )) # 128Mb
> > > +extent_size=4096
> > > +num_extents=$(( $file_size / $extent_size ))
> > > +
> > > +# Create a file with many small extents.
> > > +for ((i = 0; i < $num_extents; i++)); do
> > > + offset=$(( $i * $extent_size ))
> > > + $XFS_IO_PROG -f -s -c "pwrite -S 0xe5 $offset $extent_size" \
> > > + $SCRATCH_MNT/foo >>/dev/null
> > > +done
> >
> > This is taking too long time (1000+s) to finish when testing on XFS.
>
> Hum, for on 5.2-rc, debug kernel (lockdep, plenty of other debug stuff
> enabled that slows things down), it takes about 350 seconds for me.
>
> > I'm
> > wondering if we could take use of src/punch-alternating to punch out
> > every other block to create a file with many small extents.
>
> That's fine, I didn't remember about that.
>
> >
> > i.e. with the following diffs, test runs & passes with XFS within 90s
> > while still reproduces the btrfs bug (note that I have to increase the
> > file_size to 200M to reproduce the btrfs bug, while 256M seems bring
> > test time back to 1000+s).
Re: [PATCH v2] generic: test cloning large exents to a file with many small extents
On Fri, Jun 28, 2019 at 11:08:36PM +0100, fdman...@kernel.org wrote:
> From: Filipe Manana
>
> Test that if we clone a file with some large extents into a file that has
> many small extents, when the fs is nearly full, the clone operation does
> not fail and produces the correct result.
>
> This is motivated by a bug found in btrfs wich is fixed by the following
> patches for the linux kernel:
>
> [PATCH 1/2] Btrfs: factor out extent dropping code from hole punch handler
> [PATCH 2/2] Btrfs: fix ENOSPC errors, leading to transaction aborts, when
> cloning extents
>
> The test currently passes on xfs.
>
> Signed-off-by: Filipe Manana
> ---
>
> V2: Use _scratch_cycle_mount instead of _scratch_remount, as we want to see
> if the operation was durably persisted (otherwise we are seeing content
> from the page cache).
> Use _reflink instead of calling xfs_io with the reflink command.
> Make the comment before filling the filesystem more clear about why it
> is done the way it is instead of using _fill_fs.
>
> tests/generic/558 | 80
> +++
> tests/generic/558.out | 5
> tests/generic/group | 1 +
> 3 files changed, 86 insertions(+)
> create mode 100755 tests/generic/558
> create mode 100644 tests/generic/558.out
>
> diff --git a/tests/generic/558 b/tests/generic/558
> new file mode 100755
> index ..f982930d
> --- /dev/null
> +++ b/tests/generic/558
> @@ -0,0 +1,80 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (C) 2019 SUSE Linux Products GmbH. All Rights Reserved.
> +#
> +# FSQA Test No. 558
> +#
> +# Test that if we clone a file with some large extents into a file that has
> +# many small extents, when the fs is nearly full, the clone operation does
> +# not fail and produces the correct result.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/reflink
> +
> +# real QA test starts here
> +_supported_fs generic
> +_supported_os Linux
> +_require_scratch_reflink
> +
> +rm -f $seqres.full
> +
> +_scratch_mkfs_sized $((512 * 1024 * 1024)) >>$seqres.full 2>&1
> +_scratch_mount
> +
> +file_size=$(( 128 * 1024 * 1024 )) # 128Mb
> +extent_size=4096
> +num_extents=$(( $file_size / $extent_size ))
> +
> +# Create a file with many small extents.
> +for ((i = 0; i < $num_extents; i++)); do
> + offset=$(( $i * $extent_size ))
> + $XFS_IO_PROG -f -s -c "pwrite -S 0xe5 $offset $extent_size" \
> + $SCRATCH_MNT/foo >>/dev/null
> +done
This is taking too long time (1000+s) to finish when testing on XFS. I'm
wondering if we could take use of src/punch-alternating to punch out
every other block to create a file with many small extents.
i.e. with the following diffs, test runs & passes with XFS within 90s
while still reproduces the btrfs bug (note that I have to increase the
file_size to 200M to reproduce the btrfs bug, while 256M seems bring
test time back to 1000+s). Would you please check if this works for you?
Thanks,
Eryu
diff --git a/tests/generic/558 b/tests/generic/558
index f982930d65a2..40f8a7a98d3f 100755
--- a/tests/generic/558
+++ b/tests/generic/558
@@ -30,22 +30,22 @@ _cleanup()
_supported_fs generic
_supported_os Linux
_require_scratch_reflink
+_require_test_program "punch-alternating"
+_require_xfs_io_command "fpunch"
rm -f $seqres.full
_scratch_mkfs_sized $((512 * 1024 * 1024)) >>$seqres.full 2>&1
_scratch_mount
-file_size=$(( 128 * 1024 * 1024 )) # 128Mb
+file_size=$(( 200 * 1024 * 1024 )) # 200MB
extent_size=4096
num_extents=$(( $file_size / $extent_size ))
# Create a file with many small extents.
-for ((i = 0; i < $num_extents; i++)); do
- offset=$(( $i * $extent_size ))
- $XFS_IO_PROG -f -s -c "pwrite -S 0xe5 $offset $extent_size" \
- $SCRATCH_MNT/foo >>/dev/null
-done
+$XFS_IO_PROG -f -c "pwrite -S 0xe5 -b $file_size 0 $file_size" \
+ $SCRATCH_MNT/foo >>/dev/null
+$here/src/punch-alternating $SCRATCH_MNT/foo >> $seqres.full
# Create file bar with the same size that file foo has but with large extents.
$XFS_IO_PROG -f -c "pwrite -S 0xc7 -b $file_size 0 $file_size" \
diff --git a/tests/generic/558.out b/tests/generic/558.out
index d1e8e70f5b79..00cb5a9744aa 100644
--- a/tests/generic/558.out
+++ b/tests/generic/558.out
@@ -2,4 +2,4 @@ QA output created by 558
File foo data after cloning and remount:
000 c7 c7 c7 c7 c7 c7 c7 c7 c7 c7 c7 c7 c7 c7 c7 c7
*
-134217728
+209715200
> +
> +# Create file bar with the same size that file foo has but with large
> extents.
> +$XFS_IO_PROG -f -c "pwrite -S 0xc7 -b $file_size 0 $file_size" \
> + $SCRATCH_MNT/bar >>/dev/null
> +
> +# Fill the fs (F
Re: [PATCH 2/2] generic/059: also test that the file's mtime and ctime are updated
On Fri, Jun 21, 2019 at 11:48:57AM +0100, Filipe Manana wrote: > On Fri, Jun 21, 2019 at 11:36 AM Eryu Guan wrote: > > > > On Wed, Jun 19, 2019 at 01:06:24PM +0100, fdman...@kernel.org wrote: > > > From: Filipe Manana > > > > > > Test as well that hole punch operations that affect a single file block > > > also update the file's mtime and ctime. > > > > > > This is motivated by a bug a found in btrfs which is fixed by the > > > following patch for the linux kernel: > > > > > > "Btrfs: add missing inode version, ctime and mtime updates when > > > punching hole" > > > > > > Signed-off-by: Filipe Manana > > > --- > > > tests/generic/059 | 18 ++ > > > 1 file changed, 18 insertions(+) > > > > > > diff --git a/tests/generic/059 b/tests/generic/059 > > > index e8cb93d8..fd44b2ea 100755 > > > --- a/tests/generic/059 > > > +++ b/tests/generic/059 > > > @@ -18,6 +18,9 @@ > > > # > > > # Btrfs: add missing inode update when punching hole > > > # > > > +# Also test the mtime and ctime properties of the file change after > > > punching > > > +# holes with ranges that operate only on a single block of the file. > > > +# > > > seq=`basename $0` > > > seqres=$RESULT_DIR/$seq > > > echo "QA output created by $seq" > > > @@ -68,6 +71,13 @@ $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/foo > > > # fsync log. > > > sync > > > > > > +# Sleep for 1 second, because we want to check that the next punch > > > operations we > > > +# do update the file's mtime and ctime. > > > +sleep 1 > > > > Is this supposed to be after recording the initial c/mtime? i.e. moving > > it after c/mtime_before? > > Either way is fine. Capturing the times right before or right after > the sleep, gives the same values as nothing changed the file. Ah, you're right. > > Btw, I had noticed the other day that the second "echo" has > $mtime_after instead of $ctime_after (copy-paste mistake). > Do you want me to send a v2 fixing that typo, or you can do it > yourself when you pick the patch? I can fix it on commit, thanks for pointing it out! Thanks, Eryu
Re: [PATCH 2/2] generic/059: also test that the file's mtime and ctime are updated
On Wed, Jun 19, 2019 at 01:06:24PM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > Test as well that hole punch operations that affect a single file block > also update the file's mtime and ctime. > > This is motivated by a bug a found in btrfs which is fixed by the > following patch for the linux kernel: > > "Btrfs: add missing inode version, ctime and mtime updates when > punching hole" > > Signed-off-by: Filipe Manana > --- > tests/generic/059 | 18 ++ > 1 file changed, 18 insertions(+) > > diff --git a/tests/generic/059 b/tests/generic/059 > index e8cb93d8..fd44b2ea 100755 > --- a/tests/generic/059 > +++ b/tests/generic/059 > @@ -18,6 +18,9 @@ > # > # Btrfs: add missing inode update when punching hole > # > +# Also test the mtime and ctime properties of the file change after punching > +# holes with ranges that operate only on a single block of the file. > +# > seq=`basename $0` > seqres=$RESULT_DIR/$seq > echo "QA output created by $seq" > @@ -68,6 +71,13 @@ $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/foo > # fsync log. > sync > > +# Sleep for 1 second, because we want to check that the next punch > operations we > +# do update the file's mtime and ctime. > +sleep 1 Is this supposed to be after recording the initial c/mtime? i.e. moving it after c/mtime_before? Thanks, Eryu > + > +mtime_before=$(stat -c %Y $SCRATCH_MNT/foo) > +ctime_before=$(stat -c %Z $SCRATCH_MNT/foo) > + > # Punch a hole in our file. This small range affects only 1 page. > # This made the btrfs hole punching implementation write only some zeroes in > # one page, but it did not update the btrfs inode fields used to determine if > @@ -94,5 +104,13 @@ _flakey_drop_and_remount > echo "File content after:" > od -t x1 $SCRATCH_MNT/foo > > +mtime_after=$(stat -c %Y $SCRATCH_MNT/foo) > +ctime_after=$(stat -c %Z $SCRATCH_MNT/foo) > + > +[ $mtime_after -gt $mtime_before ] || \ > + echo "mtime did not increase (before: $mtime_before after: $mtime_after" > +[ $ctime_after -gt $ctime_before ] || \ > + echo "ctime did not increase (before: $ctime_before after: $mtime_after" > + > status=0 > exit > -- > 2.11.0 >
Re: [PATCH v2] fstests: btrfs: Validate that balance and qgroups work correctly when balance needs to be resumed on mount
On Thu, May 23, 2019 at 09:11:01AM +0800, Qu Wenruo wrote: > There are two regressions related to balance resume: > - Kernel NULL pointer dereference at mount time > Introduced in v5.0 > - Kernel BUG_ON() just after mount > Introduced in v5.1 > > The kernel fixes are: > "btrfs: qgroup: Check if @bg is NULL to avoid NULL pointer > dereference" > "btrfs: reloc: Also queue orphan reloc tree for cleanup to > avoid BUG_ON()" > > Signed-off-by: Qu Wenruo The summary seems a bit long, does this look like a proper summary to describe the test? "btrfs: resume balance on mount with qgroup" If so, I'll use this summary on commit. Thanks, Eryu
Re: [PATCH] fstests: generic, fsync fuzz tester with fsstress
On Wed, May 15, 2019 at 04:02:21PM +0100, fdman...@kernel.org wrote:
> From: Filipe Manana
>
> Run fsstress, fsync every file and directory, simulate a power failure and
> then verify the all files and directories exist, with the same data and
> metadata they had before the power failure.
>
> This tes has found already 2 bugs in btrfs, that caused mtime and ctime of
> directories not being preserved after replaying the log/journal and loss
> of a directory's attributes (such a UID and GID) after replaying the log.
> The patches that fix the btrfs issues are titled:
>
> "Btrfs: fix wrong ctime and mtime of a directory after log replay"
> "Btrfs: fix fsync not persisting changed attributes of a directory"
>
> Running this test 1000 times:
>
> - on xfs, no issues were found
>
> - on ext4 it has resulted in about a dozen journal checksum errors (on a
> 5.0 kernel) that resulted in failure to mount the filesystem after the
> simulated power failure with dmflakey, which produces the following
> error in dmesg/syslog:
>
> [Mon May 13 12:51:37 2019] JBD2: journal checksum error
> [Mon May 13 12:51:37 2019] EXT4-fs (dm-0): error loading journal
>
> Signed-off-by: Filipe Manana
> ---
> tests/generic/547 | 72
> +++
> tests/generic/547.out | 2 ++
> tests/generic/group | 1 +
> 3 files changed, 75 insertions(+)
> create mode 100755 tests/generic/547
> create mode 100644 tests/generic/547.out
>
> diff --git a/tests/generic/547 b/tests/generic/547
> new file mode 100755
> index ..577b0e9b
> --- /dev/null
> +++ b/tests/generic/547
> @@ -0,0 +1,72 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (C) 2019 SUSE Linux Products GmbH. All Rights Reserved.
> +#
> +# FS QA Test No. 547
> +#
> +# Run fsstress, fsync every file and directory, simulate a power failure and
> +# then verify the all files and directories exist, with the same data and
> +# metadata they had before the power failure.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + _cleanup_flakey
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/dmflakey
> +
> +# real QA test starts here
> +_supported_fs generic
> +_supported_os Linux
> +_require_scratch
As we save fssum file to $TEST_DIR, it'd be better to _require_test too.
> +_require_fssum
> +_require_dm_target flakey
> +
> +rm -f $seqres.full
> +
> +fssum_files_dir=$TEST_DIR/generic-test-$seq
> +rm -fr $fssum_files_dir
> +mkdir $fssum_files_dir
> +
> +_scratch_mkfs >>$seqres.full 2>&1
> +_require_metadata_journaling $SCRATCH_DEV
> +_init_flakey
> +_mount_flakey
> +
> +mkdir $SCRATCH_MNT/test
> +args=`_scale_fsstress_args -p 4 -n 100 $FSSTRESS_AVOID -d $SCRATCH_MNT/test`
> +args="$args -f mknod=0 -f symlink=0"
> +echo "Running fsstress with arguments: $args" >>$seqres.full
> +$FSSTRESS_PROG $args >>$seqres.full
> +
> +# Fsync every file and directory.
> +find $SCRATCH_MNT/test -type f,d -exec $XFS_IO_PROG -c "fsync" {} \;
My 'find' on Fedora 29 vm (find (GNU findutils) 4.6.0) doesn't support
"-type f,d" syntax
find: Arguments to -type should contain only one letter
I have to change this to
find $SCRATCH_MNT/test \( -type f -o -type d \) -exec $XFS_IO_PROG -c "fsync"
{} \;
Otherwise looks good to me, thanks!
Eryu
> +# Compute a digest of the filesystem (using the test directory only, to skip
> +# fs specific directories such as "lost+found" on ext4 for example).
> +$FSSUM_PROG -A -f -w $fssum_files_dir/fs_digest $SCRATCH_MNT/test
> +
> +# Simulate a power failure and mount the filesystem to check that all files
> and
> +# directories exist and have all data and metadata preserved.
> +_flakey_drop_and_remount
> +
> +# Compute a new digest and compare it to the one we created previously, they
> +# must match.
> +$FSSUM_PROG -r $fssum_files_dir/fs_digest $SCRATCH_MNT/test
> +
> +_unmount_flakey
> +
> +status=0
> +exit
> diff --git a/tests/generic/547.out b/tests/generic/547.out
> new file mode 100644
> index ..0f6f1131
> --- /dev/null
> +++ b/tests/generic/547.out
> @@ -0,0 +1,2 @@
> +QA output created by 547
> +OK
> diff --git a/tests/generic/group b/tests/generic/group
> index 47e81d96..49639fc9 100644
> --- a/tests/generic/group
> +++ b/tests/generic/group
> @@ -549,3 +549,4 @@
> 544 auto quick clone
> 545 auto quick cap
> 546 auto quick clone enospc log
> +547 auto quick log
> --
> 2.11.0
>
Re: [PATCH v2] fstests: btrfs: try use forget to unregister device
On Fri, Apr 26, 2019 at 06:35:42PM +0200, David Sterba wrote: > On Tue, Apr 02, 2019 at 04:19:46PM +0800, Anand Jain wrote: > > Some btrfs test cases use btrfs module-reload to unregister devices in > > the btrfs kernel. The problem with the module-reload approach is, if test > > system contains btrfs as rootfs, then you can't run these test cases. > > > > Patches [1] introduced btrfs forget feature which can unregister devices > > without the module-reload approach. > > > > [1] > > btrfs-progs: device scan: add new option to forget one or all scanned > > devices > > btrfs: introduce new ioctl to unregister a btrfs device > > > > And this patch makes relevant changes in the fstests to use this new > > feature, when available. > > > > Signed-off-by: Anand Jain > > --- > > v2: > > Update change log. > > Rename _require_btrfs_forget_if_not_fs_loadable() to > > _require_btrfs_forget_or_module_loadable() > > Rename _btrfs_forget_if_not_fs_reload() to _btrfs_forget_or_module_reload() > > Reviewed-by: David Sterba The patch has already been applied & pushed to upstream, thanks for the review all the same! Thanks, Eryu
Re: [PATCH v2] btrfs: fix filtering of scratch device in test case 048
On Thu, Apr 25, 2019 at 01:37:09AM +0100, fdman...@kernel.org wrote:
> From: Filipe Manana
>
> The recent commit 4529b20e1aa8f9 ("btrfs/048: amend property validation
> cases"), does not properly filter the scratch device because the error
> messages are sent to stderr and not to stdout, and the pipe filter only
> gets input from the stdout of the btrfs utility. We need to redirect the
> stderr of the btrfs utility to its stdout.
>
> Further, the golden output had the path "/mnt/scratch" hardcoded, instead
> of using SCRATCH_MNT. Fix that as well.
>
> The test was failing on any setup where the scratch device is not mounted
> at "/mnt/scratch".
>
> Signed-off-by: Filipe Manana
Thanks for fixing it! Sorry that I missed the issues in review..
Eryu
Re: [PATCH] btrfs: test send with deduplication running concurrently
On Sat, Apr 20, 2019 at 02:14:25PM +, Filipe Manana wrote: > On Mon, Apr 15, 2019 at 9:32 AM wrote: > > > > From: Filipe Manana > > > > Stress send running in parallel with deduplication against files that > > belong to the snapshots used by send. The goal is to hit assertion failures > > and BUG_ONs when send is running, or send finding an inconsistent snapshot > > that leads to a failure (reported in dmesg/syslog) and results in an EIO > > error returned to user space. The test needs big trees (snapshots) with > > large differences between the parent and send snapshots in order to hit > > such issues with a good probability. > > > > This currently fails in btrfs, and there is a patch for the linux kernel > > that fixes it and is titled: > > > > "Btrfs: fix race between send and deduplication that lead to failures > >and crashes" > > > > Signed-off-by: Filipe Manana > > Eryu, can you please skip this patch for now? I want to do a v2, > likely next week, which tests a few more things. > Thanks. Sure, thanks for the heads-up! Eryu
Re: [PATCH v3 3/7] fsstress: add operation for setting xattrs on files and directories
On Thu, Apr 04, 2019 at 05:30:18PM +0100, fdman...@kernel.org wrote:
> From: Filipe Manana
>
> Currently fsstress does not exercise creating, reading or deleting xattrs
> on files or directories. This change adds support for setting xattrs on
> files and directories, using only the xattr user namespace (the other
> namespaces are not general purpose and are used for security, capabilities,
> ACLs, etc). This adds a counter for each file entry structure that keeps
> track of the number of xattrs set for the file entry, and each new xattr
> has a name that includes the counter's value (example: "user.x4").
> Values for the xattrs have at most 100 bytes, which is more than the
> maximum size supported for all major filesystems.
>
> Signed-off-by: Filipe Manana
I've went through the xattr patches, and they look good to me overall,
thanks! Just some minor comments in this patch below.
> ---
>
> V2: Use a different name for the operation (setfattr instead of the already
> taken setxattr) and make use of the helper functions for opening and
> closing files or directories, introduced in the first patch of this
> series.
> V3: Simplified implementation to not need to open a file descriptor and
> use a path string instead.
>
> ltp/fsstress.c | 140
> +
> 1 file changed, 131 insertions(+), 9 deletions(-)
>
> diff --git a/ltp/fsstress.c b/ltp/fsstress.c
> index 3ec19143..d1db19da 100644
> --- a/ltp/fsstress.c
> +++ b/ltp/fsstress.c
> @@ -27,6 +27,7 @@
> io_context_t io_ctx;
> #endif
> #include
> +#include
>
> #ifndef FS_IOC_GETFLAGS
> #define FS_IOC_GETFLAGS _IOR('f', 1, long)
> @@ -84,6 +85,7 @@ typedef enum {
> OP_RESVSP,
> OP_RMDIR,
> OP_SETATTR,
> + OP_SETFATTR,
> OP_SETXATTR,
I think it'd be better to document above three *ATTR operations in
comments.
> OP_SPLICE,
> OP_STAT,
> @@ -110,6 +112,7 @@ typedef struct opdesc {
> typedef struct fent {
> int id;
> int parent;
> + int xattr_counter;
> } fent_t;
>
> typedef struct flist {
> @@ -156,6 +159,8 @@ struct print_string {
> #define MAXFSIZE((1ULL << 63) - 1ULL)
> #define MAXFSIZE32 ((1ULL << 40) - 1ULL)
>
> +#define XATTR_NAME_BUF_SIZE 18
> +
> void afsync_f(int, long);
> void allocsp_f(int, long);
> void aread_f(int, long);
> @@ -176,6 +181,7 @@ void fdatasync_f(int, long);
> void fiemap_f(int, long);
> void freesp_f(int, long);
> void fsync_f(int, long);
> +char *gen_random_string(int);
> void getattr_f(int, long);
> void getdents_f(int, long);
> void link_f(int, long);
> @@ -194,6 +200,7 @@ void rename_f(int, long);
> void resvsp_f(int, long);
> void rmdir_f(int, long);
> void setattr_f(int, long);
> +void setfattr_f(int, long);
> void setxattr_f(int, long);
> void splice_f(int, long);
> void stat_f(int, long);
> @@ -204,6 +211,7 @@ void unlink_f(int, long);
> void unresvsp_f(int, long);
> void write_f(int, long);
> void writev_f(int, long);
> +char *xattr_flag_to_string(int);
>
> opdesc_t ops[] = {
> /* { OP_ENUM, "name", function, freq, iswrite }, */
> @@ -245,6 +253,7 @@ opdesc_t ops[] = {
> { OP_RESVSP, "resvsp", resvsp_f, 1, 1 },
> { OP_RMDIR, "rmdir", rmdir_f, 1, 1 },
> { OP_SETATTR, "setattr", setattr_f, 0, 1 },
> + { OP_SETFATTR, "setfattr", setfattr_f, 4, 1 },
The frequency of setfattr looks a bit high to me, a buffer "write" op
has 4. Maybe 2 is more reasonable and change get/delfattr op's freq to 1?
(And rename delfattr to removefattr to match the removexattr syscall
name?)
> { OP_SETXATTR, "setxattr", setxattr_f, 1, 1 },
> { OP_SPLICE, "splice", splice_f, 1, 1 },
> { OP_STAT, "stat", stat_f, 1, 0 },
> @@ -296,7 +305,7 @@ char *execute_cmd = NULL;
> int execute_freq = 1;
> struct print_string flag_str = {0};
>
> -void add_to_flist(int, int, int);
> +void add_to_flist(int, int, int, int);
> void append_pathname(pathname_t *, char *);
> int attr_list_path(pathname_t *, char *, const int, int, attrlist_cursor_t
> *);
> int attr_remove_path(pathname_t *, const char *, int);
> @@ -315,6 +324,7 @@ int fent_to_name(pathname_t *, flist_t *, fent_t *);
> void fix_parent(int, int);
> void free_pathname(pathname_t *);
> int generate_fname(fent_t *, int, pathname_t *, int *, int *);
> +void generate_xattr_name(int, char *);
> int get_fname(int, long, pathname_t *, flist_t **, fent_t **, int *);
> void init_pathname(pathname_t *);
> int lchown_path(pathname_t *, uid_t, gid_t);
> @@ -716,7 +726,7 @@ translate_flags(int flags, const char *delim,
> }
>
> void
> -add_to_flist(int ft, int id, int parent)
> +add_to_flist(int ft, int id, int parent, int xattr_counter)
> {
> fent_t *fep;
> flist_t *ftp;
> @@ -729,6 +739,7 @@ add_to_flist(int ft, int id, int parent)
> fep = &ftp->fents[ftp->nfiles++];
>
Re: [PATCH v2] fstests: btrfs/048: amend property validation cases
On Sun, Apr 07, 2019 at 03:45:36PM +0300, Nikolay Borisov wrote: > > > On 7.04.19 г. 14:54 ч., Anand Jain wrote: > > On 6/4/19 8:02 pm, Eryu Guan wrote: > >> On Fri, Apr 05, 2019 at 04:21:10PM +0300, Nikolay Borisov wrote: > >>> > >>> > >>> On 3.04.19 г. 20:04 ч., Anand Jain wrote: > >>>> Add more property validation cases which are fixed by the patches [1] > >>>> [1] > >>>> btrfs: fix vanished compression property after failed set > >>>> btrfs: fix zstd compression parameter > >>>> > >>>> Signed-off-by: Anand Jain > >>> > >>> Reviewed-by: Nikolay Borisov > >> > >> Thanks for the test and the review! > >> > >> But this looks like a targeted regression test that may fail an existing > >> test. It's better to write a new test for this. > > > > Regression is only when fstests is upgraded. This > > test case mentions the prerequisite kernel patches [1]. > > So that should suffice the concern? > > I agree with Anand here, this is an extension to an existing test, which > covers specific feature. IMO it's not good to always introduce new tests > because every invocation of a test comes with an overhead of spawning > processes and whatnot. THis is not a problem for 10 tests, but currently > for btrfs we execute around 600 tests each one "wasting" some cycles to > spawn bash processes to execute the actual test. Fair enough. We're having more tests now, and we do consider "merging" some tests into one case. Thanks, Eryu
Re: [PATCH] fstests: log-writes: Add new option to replay/find next write to sector
On Thu, Mar 28, 2019 at 01:35:07PM +0800, Qu Wenruo wrote:
> For kernel operation we have METADATA/FUA/FLUSH flags as a beacon, but
> for user space write we don't have any useful flag unless the user space
> tool call fsync() to generate a FLUSH bio.
>
> This means for user space write, we don't really have an equivalent of
> --next-fua to find super block write.
>
> So this patch will add a new option, --next-write to
> replay/find next write to certain sector.
> And normally the should be super block sector number.
>
> With that, we can replay to super block write even it's user space
> triggering the write.
>
> Signed-off-by: Qu Wenruo
This lools fine to me. But I'd like Josef or Amir to take a look at it
too, I think they're more familiar with log-writes infrastructure than
me :)
Thanks,
Eryu
> ---
> src/log-writes/replay-log.c | 16 +++-
> 1 file changed, 15 insertions(+), 1 deletion(-)
>
> diff --git a/src/log-writes/replay-log.c b/src/log-writes/replay-log.c
> index c16df40e5741..0d0ec49c8a00 100644
> --- a/src/log-writes/replay-log.c
> +++ b/src/log-writes/replay-log.c
> @@ -9,6 +9,7 @@
> enum option_indexes {
> NEXT_FLUSH,
> NEXT_FUA,
> + NEXT_WRITE,
> START_ENTRY,
> END_MARK,
> LOG,
> @@ -28,6 +29,7 @@ enum option_indexes {
> static struct option long_options[] = {
> {"next-flush", no_argument, NULL, 0},
> {"next-fua", no_argument, NULL, 0},
> + {"next-write", required_argument, NULL, 0},
> {"start-entry", required_argument, NULL, 0},
> {"end-mark", required_argument, NULL, 0},
> {"log", required_argument, NULL, 0},
> @@ -53,6 +55,8 @@ static void usage(void)
> fprintf(stderr, "\t--limit - number of entries to replay\n");
> fprintf(stderr, "\t--next-flush - replay to/find the next flush\n");
> fprintf(stderr, "\t--next-fua - replay to/find the next fua\n");
> + fprintf(stderr, "\t--next-write - replay to/find the next "
> + "write to \n");
> fprintf(stderr, "\t--start-entry - start at the given "
> "entry #\n");
> fprintf(stderr, "\t--start-mark - mark to start from\n");
> @@ -139,6 +143,7 @@ int main(int argc, char **argv)
> char *logfile = NULL, *replayfile = NULL, *fsck_command = NULL;
> struct log_write_entry *entry;
> u64 stop_flags = 0;
> + u64 stop_sector = 0;
> u64 start_entry = 0;
> u64 start_sector = 0;
> u64 end_sector = -1ULL;
> @@ -173,6 +178,14 @@ int main(int argc, char **argv)
> case NEXT_FUA:
> stop_flags |= LOG_FUA_FLAG;
> break;
> + case NEXT_WRITE:
> + stop_sector = strtoull(optarg, &tmp, 0);
> + if (tmp && *tmp != '\0') {
> + fprintf(stderr, "Invalid sector number\n");
> + exit(1);
> + }
> + tmp = NULL;
> + break;
> case START_ENTRY:
> start_entry = strtoull(optarg, &tmp, 0);
> if (tmp && *tmp != '\0') {
> @@ -324,7 +337,8 @@ int main(int argc, char **argv)
> while ((ret = log_seek_next_entry(log, entry, 1)) == 0) {
> num_entries++;
> if ((run_limit && num_entries == run_limit) ||
> - should_stop(entry, stop_flags, end_mark)) {
> + should_stop(entry, stop_flags, end_mark) ||
> + (stop_sector && entry->sector == stop_sector)) {
> printf("%llu@%llu\n",
> (unsigned long long)log->cur_entry - 1,
> log->cur_pos / log->sectorsize);
> --
> 2.21.0
>
Re: [PATCH v2] fstests: btrfs/048: amend property validation cases
On Fri, Apr 05, 2019 at 04:21:10PM +0300, Nikolay Borisov wrote: > > > On 3.04.19 г. 20:04 ч., Anand Jain wrote: > > Add more property validation cases which are fixed by the patches [1] > > [1] > > btrfs: fix vanished compression property after failed set > > btrfs: fix zstd compression parameter > > > > Signed-off-by: Anand Jain > > Reviewed-by: Nikolay Borisov Thanks for the test and the review! But this looks like a targeted regression test that may fail an existing test. It's better to write a new test for this. Thanks, Eryu > > > --- > > v2: correct kernel patch titles in the test header and change log > > > > tests/btrfs/048 | 23 +++ > > tests/btrfs/048.out | 16 > > 2 files changed, 39 insertions(+) > > > > diff --git a/tests/btrfs/048 b/tests/btrfs/048 > > index 588219579cc6..f6de0b8ca8b1 100755 > > --- a/tests/btrfs/048 > > +++ b/tests/btrfs/048 > > @@ -6,6 +6,9 @@ > > # > > # Btrfs properties test. The btrfs properties feature was introduced in the > > # linux kernel 3.14. > > +# Fails without the kernel patches: > > +# btrfs: fix vanished compression property after failed set > > +# btrfs: fix zstd compression parameter > > # > > seq=`basename $0` > > seqres=$RESULT_DIR/$seq > > @@ -34,6 +37,7 @@ _supported_os Linux > > _require_test > > _require_scratch > > _require_btrfs_command "property" > > +_require_btrfs_command inspect-internal dump-super > > > > send_files_dir=$TEST_DIR/btrfs-test-$seq > > > > @@ -203,5 +207,24 @@ $BTRFS_UTIL_PROG property get $SCRATCH_MNT/sv1 > > compression > > touch $SCRATCH_MNT/sv1/file2 > > $BTRFS_UTIL_PROG property get $SCRATCH_MNT/sv1/file2 compression > > > > +echo -e "\nTesting argument validation, should fail" > > +$BTRFS_UTIL_PROG property set $SCRATCH_MNT compression 'lz' | > > _filter_scratch > > +echo "***" > > +$BTRFS_UTIL_PROG property set $SCRATCH_MNT compression 'zli' | > > _filter_scratch > > +echo "***" > > +$BTRFS_UTIL_PROG property set $SCRATCH_MNT compression 'zst' | > > _filter_scratch > > + > > +echo -e "\nTesting if property is persistent across failed validation" > > +$BTRFS_UTIL_PROG property set $SCRATCH_MNT compression 'lzo' > > +$BTRFS_UTIL_PROG property set $SCRATCH_MNT compression 'zli' | > > _filter_scratch > > +$BTRFS_UTIL_PROG property get $SCRATCH_MNT compression > > + > > +echo -e "\nTesting generation is unchanged after failed validation" > > +$BTRFS_UTIL_PROG filesystem sync $SCRATCH_MNT > > +$BTRFS_UTIL_PROG inspect-internal dump-super $SCRATCH_DEV | grep > > '^generation' > > +$BTRFS_UTIL_PROG property set $SCRATCH_MNT compression 'lz' | > > _filter_scratch > > +$BTRFS_UTIL_PROG filesystem sync $SCRATCH_MNT > > +$BTRFS_UTIL_PROG inspect-internal dump-super $SCRATCH_DEV | grep > > '^generation' > > + > > status=0 > > exit > > diff --git a/tests/btrfs/048.out b/tests/btrfs/048.out > > index 3e4e3d28950a..00f39bc01227 100644 > > --- a/tests/btrfs/048.out > > +++ b/tests/btrfs/048.out > > @@ -76,3 +76,19 @@ compression=zlib > > Testing subvolume property inheritance > > compression=lzo > > compression=lzo > > + > > +Testing argument validation, should fail > > +ERROR: failed to set compression for /mnt/scratch: Invalid argument > > +*** > > +ERROR: failed to set compression for /mnt/scratch: Invalid argument > > +*** > > +ERROR: failed to set compression for /mnt/scratch: Invalid argument > > + > > +Testing if property is persistent across failed validation > > +ERROR: failed to set compression for /mnt/scratch: Invalid argument > > +compression=lzo > > + > > +Testing generation is unchanged after failed validation > > +generation 7 > > +ERROR: failed to set compression for /mnt/scratch: Invalid argument > > +generation 7 > >
Re: [PATCH v3] fstests: btrfs verify hardening agaist duplicate fsid
On Tue, Apr 02, 2019 at 01:58:23PM +0800, Anand Jain wrote: > > Eryu, > > This patch isn't integrated yet. Sorry for missing that, and thanks for reminding! I've applied the patch. Thanks! Eryu
Re: [PATCH] btrfs: Enable btrfs/003
On Tue, Mar 19, 2019 at 12:58:51PM +0200, Nikolay Borisov wrote:
> For a long time this test has been failing on all kinds of VM configuration,
> which are using virtio_blk devices. This is due to the fact that scsi
> devices are deletable and virtio_blk are not. However, this only prevents
> device replace case to run and has no negative effect on the other
> useful test cases.
>
> Re-enable btrfs/003 to run by making _require_deletable_scratch_dev_pool
> private to the test case and modifying it to return success (0) or
> failure (1) if devices are not deletable. Further modify the replace
> test case to check the return value of this function and skip it if
> devices are not deletable.
>
> Signed-off-by: Nikolay Borisov
> ---
> common/rc | 12
> tests/btrfs/003 | 19 ++-
> 2 files changed, 18 insertions(+), 13 deletions(-)
>
> diff --git a/common/rc b/common/rc
> index 1c42515ff0ea..5693ba3cad18 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -2961,18 +2961,6 @@ _require_scratch_dev_pool_equal_size()
> done
> }
>
> -# We will check if the device is deletable
> -_require_deletable_scratch_dev_pool()
> -{
> - local i
> - local x
> - for i in $SCRATCH_DEV_POOL; do
> - x=`echo $i | cut -d"/" -f 3`
> - if [ ! -f /sys/class/block/${x}/device/delete ]; then
> - _notrun "$i is a device which is not deletable"
> - fi
> - done
> -}
>
> # Check that fio is present, and it is able to execute given jobfile
> _require_fio()
> diff --git a/tests/btrfs/003 b/tests/btrfs/003
> index 938030ef4c65..2aeb9fe6325a 100755
> --- a/tests/btrfs/003
> +++ b/tests/btrfs/003
> @@ -17,6 +17,21 @@ dev_removed=0
> removed_dev_htl=""
> trap "_cleanup; exit \$status" 0 1 2 3 15
>
> +# Check if all scratch dev pools are deletable
> +_require_deletable_scratch_dev_pool()
Then it's not a _require rule anymore, maybe we could just rename it to
something like "deletable_scratch_dev_pool"? Also remove the leading "_"
if it becomes a private function.
Thanks,
Eryu
> +{
> + local i
> + local x
> + for i in $SCRATCH_DEV_POOL; do
> + x=`echo $i | cut -d"/" -f 3`
> + if [ ! -f /sys/class/block/${x}/device/delete ]; then
> + return 1
> + fi
> + done
> +
> + return 0
> +}
> +
> _cleanup()
> {
> cd /
> @@ -35,7 +50,6 @@ _supported_fs btrfs
> _supported_os Linux
> _require_scratch
> _require_scratch_dev_pool 4
> -_require_deletable_scratch_dev_pool
> _require_command "$WIPEFS_PROG" wipefs
>
> rm -f $seqres.full
> @@ -111,6 +125,9 @@ _test_replace()
> local ds
> local d
>
> + # If scratch devs are not deletable skip this test
> + if ! _require_deletable_scratch_dev_pool; then return 0; fi
> +
> # exclude the first and the last disk in the disk pool
> n=$(($n-1))
> ds=${devs[@]:1:$(($n-1))}
> --
> 2.7.4
>
Re: [PATCH v2] btrfs: Extend btrfs/003 device removal test
On Tue, Mar 19, 2019 at 01:04:04PM +0200, Nikolay Borisov wrote:
> When a device is deleted/removed from a btrfs filesystem the kernel
> ensures all superblocks on said device are zeroed out. Test for this
> behavior. Since btrfs inspect-internal dump-super always return success
> I cannot test for the return value of the command. Instead there are 2
> cases to handle:
>
> 1. When the device is smaller than the requested super block copy, i.e.
> super block copy 2 resides at 256GB. In such cases btrfs command just
> returns blank screen
>
> 2. When the device is removed and a valid offset of the super block is
> queried btrfs command returns a textual error to stderr.
>
> Signed-off-by: Nikolay Borisov
I think this test deserves a new test itself, not being embedded in an
existing test randomly.
Thanks,
Eryu
> ---
> V2:
> * properly assign the return value of the last command expression to ret
>
> tests/btrfs/003 | 8
> 1 file changed, 8 insertions(+)
>
> diff --git a/tests/btrfs/003 b/tests/btrfs/003
> index 22aa57aad0b9..a805826ce891 100755
> --- a/tests/btrfs/003
> +++ b/tests/btrfs/003
> @@ -161,6 +161,14 @@ _test_remove()
> dev_del=`echo ${SCRATCH_DEV_POOL} | awk '{print $NF}'`
> $BTRFS_UTIL_PROG device delete $dev_del $SCRATCH_MNT || _fail "btrfs
> device delete failed"
> $BTRFS_UTIL_PROG filesystem show $SCRATCH_DEV 2>&1 | grep $dev_del >>
> $seqres.full && _fail "btrfs still shows the deleted dev"
> + for i in {0..2}; do
> + local output=$($BTRFS_UTIL_PROG inspect-internal dump-super -s
> $i $dev_del 2>&1)
> + $BTRFS_UTIL_PROG inspect-internal dump-super -s $i $dev_del
> 2>&1 | grep -q "bad magic"
> + ret=$?
> + if [[ "$output" != "" && $ret -eq 1 ]]; then
> + _fail "Delete dev superblocks not scratched"
> + fi
> + done
> _scratch_unmount
> }
>
> --
> 2.7.4
>
Re: [PATCH] fstests: btrfs try use forget to unregister device
On Tue, Mar 19, 2019 at 05:49:40PM +0800, Anand Jain wrote:
> btrfs module reload was introduced to unregister devices in the btrfs
> kernel module.
>
> The problem with the module reload approach is that you can't run btrfs
> test cases 124, 125, 154 and 164 on the system with btrfs as root fs.
>
> Patches [1] introduced btrfs forget feature which lets to cleanup the
> kernel device list without kernel module reload.
>
> [1]
> btrfs-progs: add cli to forget one or all scanned devices
> btrfs: introduce new ioctl to unregister a btrfs device
>
> So this patch uses forget feature instead of kernel module reload, if
> the forget feature is available.
>
> Signed-off-by: Anand Jain
Looks fine to me. But I'd like an explicit review from btrfs folks too.
Thanks,
Eryu
> ---
> common/btrfs| 20
> tests/btrfs/124 | 6 +++---
> tests/btrfs/125 | 6 +++---
> tests/btrfs/154 | 6 +++---
> tests/btrfs/164 | 4 ++--
> 5 files changed, 31 insertions(+), 11 deletions(-)
>
> diff --git a/common/btrfs b/common/btrfs
> index f6513c06f95f..e94e011db04e 100644
> --- a/common/btrfs
> +++ b/common/btrfs
> @@ -382,3 +382,23 @@ _scratch_btrfs_sectorsize()
> $BTRFS_UTIL_PROG inspect-internal dump-super $SCRATCH_DEV |\
> grep sectorsize | awk '{print $2}'
> }
> +
> +_btrfs_supports_forget()
> +{
> + $BTRFS_UTIL_PROG device scan --help | grep -wq forget && \
> + $BTRFS_UTIL_PROG device scan --forget > /dev/null 2>&1
> +}
> +
> +_require_btrfs_forget_if_not_fs_loadable()
> +{
> + _btrfs_supports_forget && return
> +
> + _require_loadable_fs_module "btrfs"
> +}
> +
> +_btrfs_forget_if_not_fs_reload()
> +{
> + _btrfs_supports_forget && return
> +
> + _reload_fs_module "btrfs"
> +}
> diff --git a/tests/btrfs/124 b/tests/btrfs/124
> index a52c65f608ff..9341dcea8896 100755
> --- a/tests/btrfs/124
> +++ b/tests/btrfs/124
> @@ -51,7 +51,7 @@ _supported_fs btrfs
> _supported_os Linux
> _require_scratch_dev_pool 2
> _test_unmount
> -_require_loadable_fs_module "btrfs"
> +_require_btrfs_forget_if_not_fs_loadable
>
> _scratch_dev_pool_get 2
>
> @@ -86,7 +86,7 @@ echo "clean btrfs ko" >> $seqres.full
> _scratch_unmount
>
> # un-scan the btrfs devices
> -_reload_fs_module "btrfs"
> +_btrfs_forget_if_not_fs_reload
>
> echo >> $seqres.full
> echo "-Write degraded mount fill upto $max_fs_sz bytes-" >>
> $seqres.full
> @@ -125,7 +125,7 @@ echo
> echo "Mount degraded with the other dev"
> _scratch_unmount
> # un-scan the btrfs devices
> -_reload_fs_module "btrfs"
> +_btrfs_forget_if_not_fs_reload
> _mount -o degraded $dev2 $SCRATCH_MNT >>$seqres.full 2>&1
> _run_btrfs_util_prog filesystem show
> checkpoint3=`md5sum $SCRATCH_MNT/tf2`
> diff --git a/tests/btrfs/125 b/tests/btrfs/125
> index 847fa62ad25f..3d847033f1a2 100755
> --- a/tests/btrfs/125
> +++ b/tests/btrfs/125
> @@ -50,7 +50,7 @@ _supported_fs btrfs
> _supported_os Linux
> _require_scratch_dev_pool 3
> _test_unmount
> -_require_loadable_fs_module "btrfs"
> +_require_btrfs_forget_if_not_fs_loadable
> _require_btrfs_fs_feature raid56
>
> _scratch_dev_pool_get 3
> @@ -103,7 +103,7 @@ echo "unmount" >> $seqres.full
> _scratch_unmount
> echo "clean btrfs ko" >> $seqres.full
> # un-scan the btrfs devices
> -_reload_fs_module "btrfs"
> +_btrfs_forget_if_not_fs_reload
> _mount -o degraded,device=$dev2 $dev1 $SCRATCH_MNT >>$seqres.full 2>&1
> dd if=/dev/zero of="$SCRATCH_MNT"/tf2 bs=$bs count=$count \
> >>$seqres.full 2>&1
> @@ -139,7 +139,7 @@ echo "Mount degraded but with other dev"
>
> _scratch_unmount
> # un-scan the btrfs devices
> -_reload_fs_module "btrfs"
> +_btrfs_forget_if_not_fs_reload
>
> _mount -o degraded,device=${dev2} $dev3 $SCRATCH_MNT >>$seqres.full 2>&1
>
> diff --git a/tests/btrfs/154 b/tests/btrfs/154
> index cd6c688fb9fe..e39f54ac6ab8 100755
> --- a/tests/btrfs/154
> +++ b/tests/btrfs/154
> @@ -36,7 +36,7 @@ rm -f $seqres.full
> _supported_fs btrfs
> _supported_os Linux
> _require_scratch_dev_pool 2
> -_require_loadable_fs_module "btrfs"
> +_require_btrfs_forget_if_not_fs_loadable
>
> _scratch_dev_pool_get 2
>
> @@ -90,7 +90,7 @@ degrade_mount_write()
>
> echo "clean btrfs ko" >> $seqres.full
> # un-scan the btrfs devices
> - _reload_fs_module "btrfs"
> + _btrfs_forget_if_not_fs_reload
> _mount -o degraded $DEV1 $SCRATCH_MNT >>$seqres.full 2>&1
> cnt=$(( $COUNT/10 ))
> dd if=/dev/urandom of="$SCRATCH_MNT"/tf1 bs=$bs count=$cnt \
> @@ -142,7 +142,7 @@ verify()
> echo "unmount" >> $seqres.full
>
> _scratch_unmount
> - _reload_fs_module "btrfs"
> + _btrfs_forget_if_not_fs_reload
> _mount -o degraded $DEV2 $SCRATCH_MNT >>$seqres.full 2>&1
> verify_checkpoint1=`md5sum $SCRATCH_MNT/tf1`
> verify_checkpoint2=`md5sum $SCRATCH_MNT/tf2`
> diff --git a/tests/btrfs/164 b/tests/btrfs/164
> index 097191a0e493..55042c4035e0 10
Re: [PATCH v2] shared/298: Wire btrfs support in get_free_sectors
On Fri, Feb 08, 2019 at 01:44:04PM +0200, Nikolay Borisov wrote:
> Add support for btrfs in shared/298. Achieve this by introducing 2
> new awk scripts that parse relevant btrfs structures and print holes.
> Additionally modify the test to create larger - 3gb filesystem in the
> case of btrfs. This is needed so that distinct block groups are used
> for data and metadata.
>
> Signed-off-by: Nikolay Borisov
Sorry for the late review.. I find that parsing btrfs extent and dev
tree is very btrfs-specific, it'd be great if btrfs folks could help
review the two awk scripts as well!
> ---
>
> V2:
> * Changed the way args are passed to mkfs.btrfs to preserve existing
> options, yet override data/metadata profile settings
>
> parse-dev-tree.awk| 47 +++
> parse-extent-tree.awk | 125
> ++
I'd prefer placing these files in src dir, instead of dumping them in
top dir directly.
> tests/shared/298 | 36 +--
> 3 files changed, 205 insertions(+), 3 deletions(-)
> create mode 100755 parse-dev-tree.awk
> create mode 100755 parse-extent-tree.awk
>
> diff --git a/parse-dev-tree.awk b/parse-dev-tree.awk
> new file mode 100755
> index ..52f9c0aadc25
> --- /dev/null
> +++ b/parse-dev-tree.awk
> @@ -0,0 +1,47 @@
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2019 Nikolay Borisov, SUSE LLC. All Rights Reserved.
> +#
> +# Parses btrfs' device tree for holes, required parameters passed on command
I find this description not very useful, would you please describe the
expected output and format as well?
> +# line:
> +# * spb - how many bytes per sector, this is used so that the numbers
^^^ This is misleading, in shared/298 spb represents "sector
per block", but here it's really sector size.
> +#returned by the script are in sectors.
> +# * devsize - size of the device in bytes, used to output the final
This line is not aligned with above line, it contains leading tab.
> +#hole (if any)
> +
> +function get_extent_size(line, tmp) {
Would you please document the expected intput and output in comment as
well? So it's easier to review.
Also, is the 'tmp' argument really needed?
> + split(line, tmp)
> + return tmp[6]
> +}
> +
> +function get_extent_offset(line, tmp) {
Same here.
> + split(line, tmp)
> + gsub(/\)/,"", tmp[6])
> + return tmp[6]
> +}
> +
> +BEGIN {
> + dev_extent_match="^.item [0-9]* key \\([0-9]* DEV_EXTENT [0-9]*\\).*"
> + dev_extent_len_match="^\\s*chunk_objectid [0-9]* chunk_offset [0-9]*
> length [0-9]*$"
> +}
> +
> +{
> + if (match($0,dev_extent_match)) {
> + extent_offset = get_extent_offset($0)
> + if (prev_extent_end) {
> + hole_size = extent_offset - prev_extent_end
> + if (hole_size > 0) {
> + print prev_extent_end / spb, int((extent_offset
> - 1) / spb)
> + }
> + }
> + } else if (match($0, dev_extent_len_match)) {
> + extent_size = get_extent_size($0)
> + prev_extent_end = extent_offset + extent_size
> + }
> +}
> +
> +END {
> + if (prev_extent_end) {
> + print prev_extent_end / spb, int((devsize - 1) / spb)
> + }
> +}
> +
> diff --git a/parse-extent-tree.awk b/parse-extent-tree.awk
> new file mode 100755
> index ..01c61254cfef
> --- /dev/null
> +++ b/parse-extent-tree.awk
> @@ -0,0 +1,125 @@
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2019 Nikolay Borisov, SUSE LLC. All Rights Reserved.
> +#
> +# Parses btrfs' extent tree for holes, required parameters passed on command
Same here, please provide more details.
> +# line:
> +# * spb - how many bytes per sector, this is used so that the numbers
And replace 'spb' with a more proper variable name.
> +#returned by the script are in sectors.
> +# * nodesize - size of metadata extents, used for internal calculations
Indention issue too.
> +
> +function get_extent_size(line, tmp) {
> + if (line ~ data_match || line ~ bg_match) {
> + split(line, tmp)
> + gsub(/\)/,"", tmp[6])
> + return tmp[6]
> + } else if (line ~ metadata_match) {
> + return nodesize
> + }
> +}
> +
> +function get_extent_offset(line, tmp) {
> + split(line, tmp)
> + gsub(/\(/,"",tmp[4])
> + return tmp[4]
> +}
> +
> +function print_array( base_offset, bg_line)
Document the expected input and output of these functions too.
And why there're so many spaces before 'base_offset' argument?
> +{
> + if (match(lines[0], bg_match)) {
> + #we don't have an extent at the beginning of of blockgroup, so
> we
Add a space after '#' for comments.
> + #have a hole between blockgroup offset and first extent offset
> + bg_line = lines[0]
Re: [PATCH 0/7] fstests: test Btrfs swapfile support
On Tue, Nov 06, 2018 at 02:06:30PM +0100, David Sterba wrote:
> On Mon, Nov 05, 2018 at 12:09:31AM +0800, Eryu Guan wrote:
> > On Fri, Nov 02, 2018 at 02:29:35PM -0700, Omar Sandoval wrote:
> > > From: Omar Sandoval
> > >
> > > This series fixes a couple of generic swapfile tests and adds some
> > > Btrfs-specific swapfile tests. Btrfs swapfile support is scheduled for
> > > 4.21 [1].
> > >
> > > 1: https://www.spinics.net/lists/linux-btrfs/msg83454.html
> > >
> > > Thanks!
> >
> > Thanks for the fixes and new tests!
> >
> > >
> > > Omar Sandoval (7):
> > > generic/{472,496,497}: fix $seeqres typo
> > > generic/{472,496}: fix swap file creation on Btrfs
> >
> > I've merged above two patches, they're two obvious bug fixes.
> >
> > > btrfs: test swap file activation restrictions
> > > btrfs: test invalid operations on a swap file
> > > btrfs: test swap files on multiple devices
> > > btrfs: test device add/remove/replace with an active swap file
> > > btrfs: test balance and resize with an active swap file
> >
> > These tests look fine to me, but it'd be really great if btrfs folks
> > could help review above tests and provide Reviewed-by tags.
>
> All look good to me,
>
> Reviewed-by: David Sterba
Thanks a lot for the review!
>
> A few nits I saw:
>
> - the command names should not be shortened, ie. 'btrfs subvolume
> snapshot' instead of 'btrfs subvol snap'
> - the test description (eg. 3/7 and 4/7) could mention which case is
> tested, eg. swapfile with compression or COW or snapshot
> - 4/7 has typo 'nowcow' in a comment
I've addressed these nits on commit.
Thanks,
Eryu
Re: [PATCH 0/7] fstests: test Btrfs swapfile support
On Fri, Nov 02, 2018 at 02:29:35PM -0700, Omar Sandoval wrote:
> From: Omar Sandoval
>
> This series fixes a couple of generic swapfile tests and adds some
> Btrfs-specific swapfile tests. Btrfs swapfile support is scheduled for
> 4.21 [1].
>
> 1: https://www.spinics.net/lists/linux-btrfs/msg83454.html
>
> Thanks!
Thanks for the fixes and new tests!
>
> Omar Sandoval (7):
> generic/{472,496,497}: fix $seeqres typo
> generic/{472,496}: fix swap file creation on Btrfs
I've merged above two patches, they're two obvious bug fixes.
> btrfs: test swap file activation restrictions
> btrfs: test invalid operations on a swap file
> btrfs: test swap files on multiple devices
> btrfs: test device add/remove/replace with an active swap file
> btrfs: test balance and resize with an active swap file
These tests look fine to me, but it'd be really great if btrfs folks
could help review above tests and provide Reviewed-by tags.
And perhaps we could add test 17[56] to 'volume' group, as they do
device operations. And similarly, add the last test to 'balance' group.
Thanks,
Eryu
>
> tests/btrfs/173 | 55 ++
> tests/btrfs/173.out | 5 +++
> tests/btrfs/174 | 66
> tests/btrfs/174.out | 10 ++
> tests/btrfs/175 | 73
> tests/btrfs/175.out | 8 +
> tests/btrfs/176 | 82 +
> tests/btrfs/176.out | 5 +++
> tests/btrfs/177 | 64 +++
> tests/btrfs/177.out | 6
> tests/btrfs/group | 5 +++
> tests/generic/472 | 16 -
> tests/generic/496 | 8 ++---
> tests/generic/497 | 2 +-
> 14 files changed, 391 insertions(+), 14 deletions(-)
> create mode 100755 tests/btrfs/173
> create mode 100644 tests/btrfs/173.out
> create mode 100755 tests/btrfs/174
> create mode 100644 tests/btrfs/174.out
> create mode 100755 tests/btrfs/175
> create mode 100644 tests/btrfs/175.out
> create mode 100755 tests/btrfs/176
> create mode 100644 tests/btrfs/176.out
> create mode 100755 tests/btrfs/177
> create mode 100644 tests/btrfs/177.out
>
> --
> 2.19.1
>
Re: [PATCH v2 rev log added] fstests: btrfs verify hardening agaist duplicate fsid
On Tue, Oct 09, 2018 at 02:28:21AM +0800, Anand Jain wrote:
> We have a known bug in btrfs, that we let the device path be changed
> after the device has been mounted. So using this loop hole the new
> copied device would appears as if its mounted immediately after its
> been copied. So this test case reproduces this issue.
>
> For example:
>
> Initially.. /dev/mmcblk0p4 is mounted as /
>
> lsblk
> NAMEMAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> mmcblk0 179:00 29.2G 0 disk
> |-mmcblk0p4 179:404G 0 part /
> |-mmcblk0p2 179:20 500M 0 part /boot
> |-mmcblk0p3 179:30 256M 0 part [SWAP]
> `-mmcblk0p1 179:10 256M 0 part /boot/efi
>
> btrfs fi show
> Label: none uuid: 07892354-ddaa-4443-90ea-f76a06accaba
> Total devices 1 FS bytes used 1.40GiB
> devid1 size 4.00GiB used 3.00GiB path /dev/mmcblk0p4
>
> Copy mmcblk0 to sda
> dd if=/dev/mmcblk0 of=/dev/sda
>
> And immediately after the copy completes the change in the device
> superblock is notified which the automount scans using
> btrfs device scan and the new device sda becomes the mounted root
> device.
>
> lsblk
> NAMEMAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> sda 8:01 14.9G 0 disk
> |-sda48:414G 0 part /
> |-sda28:21 500M 0 part
> |-sda38:31 256M 0 part
> `-sda18:11 256M 0 part
> mmcblk0 179:00 29.2G 0 disk
> |-mmcblk0p4 179:404G 0 part
> |-mmcblk0p2 179:20 500M 0 part /boot
> |-mmcblk0p3 179:30 256M 0 part [SWAP]
> `-mmcblk0p1 179:10 256M 0 part /boot/efi
> btrfs fi show /
> Label: none uuid: 07892354-ddaa-4443-90ea-f76a06accaba
> Total devices 1 FS bytes used 1.40GiB
> devid1 size 4.00GiB used 3.00GiB path /dev/sda4
>
> The bug is quite nasty that you can't either unmount /dev/sda4 or
> /dev/mmcblk0p4. And the problem does not get solved until you take
> the sda out of the system on to another system to change its fsid using
> the 'btrfstune -u' command.
>
> Signed-off-by: Anand Jain
Hi btrfs folks,
Please help review if this patch adds a valid test for btrfs. It looks
fine to me from fstests' perspective of view, though it needs some
really minor tweaks that I can fix on commit (e.g. format of for loop,
adding space after '#' in comments line). Thanks a lot!
Eryu
> ---
> v1->v2:
> dont play around with dev patch use it as it is.
> do not use SCRATCH_MNT instead create it at the TEST_DIR and its related
>changes.
> golden out changes
>
> tests/btrfs/173 | 88
> +
> tests/btrfs/173.out | 6
> tests/btrfs/group | 1 +
> 3 files changed, 95 insertions(+)
> create mode 100755 tests/btrfs/173
> create mode 100644 tests/btrfs/173.out
>
> diff --git a/tests/btrfs/173 b/tests/btrfs/173
> new file mode 100755
> index ..b466ae921e19
> --- /dev/null
> +++ b/tests/btrfs/173
> @@ -0,0 +1,88 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2018 Oracle. All Rights Reserved.
> +#
> +# FS QA Test 173
> +#
> +# Fuzzy test for FS image duplication.
> +# Could be fixed by
> +#[patch] btrfs: harden agaist duplicate fsid
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +mnt=$TEST_DIR/$seq.mnt
> +_cleanup()
> +{
> + rm -rf $mnt > /dev/null 2>&1
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch_dev_pool 2
> +_scratch_dev_pool_get 2
> +
> +dev_foo=$(echo $SCRATCH_DEV_POOL | awk '{print $1}')
> +dev_bar=$(echo $SCRATCH_DEV_POOL | awk '{print $2}')
> +
> +echo dev_foo=$dev_foo >> $seqres.full
> +echo dev_bar=$dev_bar >> $seqres.full
> +echo | tee -a $seqres.full
> +
> +rm -rf $mnt > /dev/null 2>&1
> +mkdir $mnt
> +_mkfs_dev $dev_foo
> +_mount $dev_foo $mnt
> +
> +check_btrfs_mount()
> +{
> + local x=$(findmnt $mnt | grep -v TARGET | awk '{print $2}')
> + [[ $x == $dev_foo ]] && echo DEV_FOO
> + [[ $x == $dev_bar ]] && echo DEV_BAR
> +}
> +
> +echo MNT $(check_btrfs_mount)
> +
> +for sb_bytenr in 65536 67108864
> +do
> + echo -n "dd status=none if=$dev_foo of=$dev_bar bs=1 "\
> + "seek=$sb_bytenr skip=$sb_bytenr count=4096" >> $seqres.full
> + dd status=none if=$dev_foo of=$dev_bar bs=1 seek=$sb_bytenr \
> + skip=$sb_bytenr count=4096 >> $seqres.full 2>&1
> + echo ..:$? >> $seqres.full
> +done
> +
> +#Original device is mounted, scan of its clone should fail
> +$BTRFS_UTIL_PROG device scan $dev_bar >> $seqres.full 2>&1
> +echo btrfs device scan dev_bar ...:$?| te
Re: [PATCH v2 5/9] generic/102 open code dev_size _scratch_mkfs_sized()
On Wed, Sep 26, 2018 at 12:08:56PM +0800, Anand Jain wrote:
>
>
> On 09/25/2018 06:54 PM, Nikolay Borisov wrote:
> >
> >
> > On 25.09.2018 07:24, Anand Jain wrote:
> > > Open code helps to grep and find out parameter sent to the
> > > _scratch_mkfs_sized here.
> > >
> > > Signed-off-by: Anand Jain
> >
> > IMO this is noise, you can just as simply do
> > "grep _scratch_mkfs_sized" and then open the file to inspect the actual
> > argument. But it's up to the xfstest maintainers
>
> I am ok. Its just a nice cleanup.
>
> Thanks, Anand
I prefer dropping patch 5/6/7, as I don't think they're that necessary.
BTW, other patches from this series but patch 3 ("geneirc/077 fix min
size for btrfs") look fine to me, I'm taking them in this week's update.
Thanks,
Eryu
>
> > > ---
> > > tests/generic/102 | 3 +--
> > > 1 file changed, 1 insertion(+), 2 deletions(-)
> > >
> > > diff --git a/tests/generic/102 b/tests/generic/102
> > > index faf940ac5070..aad496a5bc69 100755
> > > --- a/tests/generic/102
> > > +++ b/tests/generic/102
> > > @@ -31,8 +31,7 @@ _require_scratch
> > > rm -f $seqres.full
> > > -dev_size=$((512 * 1024 * 1024)) # 512MB filesystem
> > > -_scratch_mkfs_sized $dev_size >>$seqres.full 2>&1
> > > +_scratch_mkfs_sized $((512 * 1024 * 1024)) >>$seqres.full 2>&1
> > > _scratch_mount
> > > for ((i = 0; i < 10; i++)); do
> > >
Re: [PATCH v2 3/9] geneirc/077 fix min size for btrfs
On Tue, Sep 25, 2018 at 12:24:16PM +0800, Anand Jain wrote: > If btrfs need to be tested at its default blockgroup which is non-mixed, > then it needs at least 256mb. > > Signed-off-by: Anand Jain (Sorry for the late review..) > --- > tests/generic/077 | 3 +-- > 1 file changed, 1 insertion(+), 2 deletions(-) > > diff --git a/tests/generic/077 b/tests/generic/077 > index ef6af18c83e3..ec236992513f 100755 > --- a/tests/generic/077 > +++ b/tests/generic/077 > @@ -49,8 +49,7 @@ rm -f $seqres.full > _scratch_unmount >/dev/null 2>&1 > echo "*** MKFS ***" >>$seqres.full > echo "" >>$seqres.full > -SIZE=`expr 50 \* 1024 \* 1024` > -_scratch_mkfs_sized $SIZE >>$seqres.full 2>&1 \ > +_scratch_mkfs_sized $((256 * 1024 *1024)) >>$seqres.full 2>&1 \ > || _fail "mkfs failed" Hmm, this test copies "/lib/modules/" to fill the original 50M filesystem (which seems a bad way to me) " # Something w/ enough data to fill 50M of fs... filler=/lib/modules/ ... echo "*** populate filesystem, pass #1" | tee -a $seqres.full cp -rf $filler $SCRATCH_MNT/subdir >$seqres.full 2>&1 ... " It works most of the time as "/lib/modules" is usually larger than 50M, but it may not fullfil the fs with 256M size. I think we should fix the way to fill the fs too. Thanks, Eryu > _scratch_mount > mkdir $SCRATCH_MNT/subdir > -- > 1.8.3.1 >
Re: [PATCH] fstests: btrfs verify hardening agaist duplicate fsid
On Mon, Oct 01, 2018 at 04:44:35PM +0800, Anand Jain wrote:
> We have a known bug in btrfs, that we let the device path be changed
> after the device has been mounted. So using this loop hole the new
> copied device would appears as if its mounted immediately after its
> been copied. So this test case reproduces this issue.
>
> For example:
>
> Initially.. /dev/mmcblk0p4 is mounted as /
>
> lsblk
> NAMEMAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> mmcblk0 179:00 29.2G 0 disk
> |-mmcblk0p4 179:404G 0 part /
> |-mmcblk0p2 179:20 500M 0 part /boot
> |-mmcblk0p3 179:30 256M 0 part [SWAP]
> `-mmcblk0p1 179:10 256M 0 part /boot/efi
>
> btrfs fi show
> Label: none uuid: 07892354-ddaa-4443-90ea-f76a06accaba
> Total devices 1 FS bytes used 1.40GiB
> devid1 size 4.00GiB used 3.00GiB path /dev/mmcblk0p4
>
> Copy mmcblk0 to sda
> dd if=/dev/mmcblk0 of=/dev/sda
>
> And immediately after the copy completes the change in the device
> superblock is notified which the automount scans using
> btrfs device scan and the new device sda becomes the mounted root
> device.
>
> lsblk
> NAMEMAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> sda 8:01 14.9G 0 disk
> |-sda48:414G 0 part /
> |-sda28:21 500M 0 part
> |-sda38:31 256M 0 part
> `-sda18:11 256M 0 part
> mmcblk0 179:00 29.2G 0 disk
> |-mmcblk0p4 179:404G 0 part
> |-mmcblk0p2 179:20 500M 0 part /boot
> |-mmcblk0p3 179:30 256M 0 part [SWAP]
> `-mmcblk0p1 179:10 256M 0 part /boot/efi
> btrfs fi show /
> Label: none uuid: 07892354-ddaa-4443-90ea-f76a06accaba
> Total devices 1 FS bytes used 1.40GiB
> devid1 size 4.00GiB used 3.00GiB path /dev/sda4
>
> The bug is quite nasty that you can't either unmount /dev/sda4 or
> /dev/mmcblk0p4. And the problem does not get solved until you take
> the sda out of the system on to another system to change its fsid using
> the 'btrfstune -u' command.
>
> Signed-off-by: Anand Jain
Looks like that the test will break the whole test env as it leaves an
unmountable $SCRATCH_MNT. I'd wait for the fix to get in first before
merging the test, in case it breaks normal regression tests. (I noticed
that the test is not in 'auto' group, so it's not that dangerous.)
Also, it'd be great if test can be reviewed by btrfs folks too!
> ---
> tests/btrfs/173 | 72
> +
> tests/btrfs/173.out | 5
> tests/btrfs/group | 1 +
> 3 files changed, 78 insertions(+)
> create mode 100755 tests/btrfs/173
> create mode 100644 tests/btrfs/173.out
>
> diff --git a/tests/btrfs/173 b/tests/btrfs/173
> new file mode 100755
> index ..f59a62e206c3
> --- /dev/null
> +++ b/tests/btrfs/173
> @@ -0,0 +1,72 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2018 Oracle. All Rights Reserved.
> +#
> +# FS QA Test 173
> +#
> +# Fuzzy test for FS image duplication.
> +# Could be fixed by
> +#[patch] btrfs: harden agaist duplicate fsid
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch_dev_pool 2
> +_scratch_dev_pool_get 2
> +
> +dev_foo=$(echo $SCRATCH_DEV_POOL | awk '{print $1}' | rev | cut -d"/" -f1 |
> rev)
> +dev_bar=$(echo $SCRATCH_DEV_POOL | awk '{print $2}' | rev | cut -d"/" -f1 |
> rev)
This doesn't work if the devices in SCRATCH_DEV_POOL are symlinks, e.g.
lvm devices: /dev/mapper/testvg-testlv1, dev_foo is "testvg-testlv1" in
this case.
> +
> +_mkfs_dev /dev/$dev_foo
But /dev/testvg-testlv1 isn't existed.
_short_dev and/or _real_dev is useful in this case. e.g.
dev_foo=$(echo $SCRATCH_DEV_POOL | awk '{print $1}')
# dev_foo is like "dm-1"
dev_foo=$(_short_dev $dev_foo)
# dev_foo is like "/dev/dm-1"
dev_foo=$(_real_dev $dev_foo)
> +_mount /dev/$dev_foo $SCRATCH_MNT
It'd better to mount non-SCRATCH_DEV to other mount point, e.g.
$TEST_DIR/$seq.mnt
Thanks,
Eryu
> +
> +echo mount before btrfs image clone | tee -a $seqres.full
> +findmnt /dev/$dev_foo | grep -v TARGET | awk '{print $1" "$2}' | \
> + sed -e "s/$dev_foo/dev_foo/g" | _filter_scratch | tee -a $seqres.full
> +findmnt /dev/$dev_bar | grep -v TARGET | awk '{print $1" "$2}' | \
> + sed -e "s/$dev_bar/dev_bar/g" | _filter_scratch | tee -a $seqres.full
> +
> +for sb_bytenr in 65536 67108864
> +do
> + echo -n "dd status=none if=/dev/$dev_foo of=/dev/$dev_bar bs=1 "\
> + "seek=$sb_bytenr skip=$sb_bytenr c
Re: [PATCH] test unaligned punch hole at ENOSPC
On Mon, Sep 24, 2018 at 07:47:39PM +0800, Anand Jain wrote:
> Try to punch hole with unaligned size and offset when the FS
> returns ENOSPC
>
> Signed-off-by: Anand Jain
> ---
> This test case fails on btrfs as of now.
>
> tests/btrfs/172 | 66
> +
> tests/btrfs/172.out | 2 ++
> tests/btrfs/group | 1 +
> 3 files changed, 69 insertions(+)
> create mode 100755 tests/btrfs/172
> create mode 100644 tests/btrfs/172.out
>
> diff --git a/tests/btrfs/172 b/tests/btrfs/172
> new file mode 100755
> index ..9c32a173f912
> --- /dev/null
> +++ b/tests/btrfs/172
> @@ -0,0 +1,66 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2018 Oracle. All Rights Reserved.
> +#
> +# FS QA Test 172
> +#
> +# Test if the unaligned (by size and offset) punch hole is successful when FS
> +# is at ENOSPC.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs generic
> +_supported_os Linux
> +_require_scratch
> +
> +_scratch_mkfs_sized $((200 * 1024 *1024)) >> $seqres.full
> +
> +# max_inline helps to create regular extent
> +_scratch_mount "-o max_inline=0,nodatacow"
> +
> +echo "Fill fs upto ENOSPC" >> $seqres.full
> +dd status=none if=/dev/zero of=$SCRATCH_MNT/filler bs=512 >> $seqres.full
> 2>&1
> +
> +extent_size=$(_scratch_btrfs_sectorsize)
> +unalign_by=512
> +echo extent_size=$extent_size unalign_by=$unalign_by >> $seqres.full
> +
> +hole_offset=0
> +hole_len=$unalign_by
> +run_check fallocate -p -o $hole_offset -l $hole_len $SCRATCH_MNT/filler
Please don't introduce new run_check/_run_btrfs_util_prog users, just
redirect output to /dev/null if the outputs don't matter. Please refer
to this thread
https://www.spinics.net/lists/linux-btrfs/msg80996.html
And use xfs_io fpunch command instead of bare 'fallocate -p', and check
xfs_io and kernel support on fpunch by calling
_require_xfs_io_comand "fpunch"
> +
> +hole_offset=$(($extent_size + $unalign_by))
> +hole_len=$(($extent_size - $unalign_by))
> +run_check fallocate -p -o $hole_offset -l $hole_len $SCRATCH_MNT/filler
> +
> +hole_offset=$(($extent_size * 2 + $unalign_by))
> +hole_len=$(($extent_size * 5))
> +run_check fallocate -p -o $hole_offset -l $hole_len $SCRATCH_MNT/filler
> +
> +# success, all done
> +echo "Silence is golden"
> +status=0
> +exit
> diff --git a/tests/btrfs/172.out b/tests/btrfs/172.out
> new file mode 100644
> index ..ce2de3f0d107
> --- /dev/null
> +++ b/tests/btrfs/172.out
> @@ -0,0 +1,2 @@
> +QA output created by 172
> +Silence is golden
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index feffc45b6564..7e1a638ab7e1 100644
> --- a/tests/btrfs/group
> +++ b/tests/btrfs/group
> @@ -174,3 +174,4 @@
> 169 auto quick send
> 170 auto quick snapshot
> 171 auto quick qgroup
> +172 auto quick
Add 'punch' group too.
Thanks,
Eryu
> --
> 1.8.3.1
>
Re: [PATCH] fstests: btrfs/149 make it sectorsize independent
On Wed, Sep 05, 2018 at 02:35:12PM +0800, Anand Jain wrote:
> Originally this test case was designed to work with only 4K sectorsize.
> Now enhance it to work with any sector sizes and makes the following
> changes:
> Output file not to contain any traces of sector size.
> Use max_inline=0 mount option so that it meets the requisite of non inline
> regular extent.
> Don't log the md5sum results to the output file as the data size vary by
> the sectorsize.
>
> Signed-off-by: Anand Jain
> ---
> common/btrfs| 7 +++
> common/filter | 5 +
> tests/btrfs/149 | 29 -
> tests/btrfs/149.out | 12 ++--
> 4 files changed, 38 insertions(+), 15 deletions(-)
>
> diff --git a/common/btrfs b/common/btrfs
> index 79c687f73376..e6a218d6b63a 100644
> --- a/common/btrfs
> +++ b/common/btrfs
> @@ -367,3 +367,10 @@ _run_btrfs_balance_start()
>
> run_check $BTRFS_UTIL_PROG balance start $bal_opt $*
> }
> +
> +#return the sector size of the btrfs scratch fs
> +_scratch_sectorsize()
The function name would indicate that it's a general helper function and
works for all filesystems, but it only works for btrfs. I'd be better to
name it as "_scratch_btrfs_sectorsize" or similar.
> +{
> + $BTRFS_UTIL_PROG inspect-internal dump-super $SCRATCH_DEV |\
> + grep sectorsize | awk '{print $2}'
> +}
> diff --git a/common/filter b/common/filter
> index 3965c2eb752b..e87740ddda3f 100644
> --- a/common/filter
> +++ b/common/filter
> @@ -271,6 +271,11 @@ _filter_xfs_io_pages_modified()
> _filter_xfs_io_units_modified "Page" $PAGE_SIZE
> }
>
> +_filter_xfs_io_numbers()
> +{
> +_filter_xfs_io | sed -E 's/[0-9]+//g'
> +}
> +
> _filter_test_dir()
> {
> # TEST_DEV may be a prefix of TEST_DIR (e.g. /mnt, /mnt/ovl-mnt)
> diff --git a/tests/btrfs/149 b/tests/btrfs/149
> index 3e955a305e0f..3958fa844c8b 100755
> --- a/tests/btrfs/149
> +++ b/tests/btrfs/149
> @@ -44,21 +44,27 @@ rm -fr $send_files_dir
> mkdir $send_files_dir
>
> _scratch_mkfs >>$seqres.full 2>&1
> -_scratch_mount "-o compress"
> +# On 64K pagesize systems the compression is more efficient, so max_inline
> +# helps to create regular (non inline) extent irrespective of the final
> +# write size.
> +_scratch_mount "-o compress -o max_inline=0"
>
> # Write to our file using direct IO, so that this way the write ends up not
> # getting compressed, that is, we get a regular extent which is neither
> # inlined nor compressed.
> # Alternatively, we could have mounted the fs without compression enabled,
> # which would result as well in an uncompressed regular extent.
> -$XFS_IO_PROG -f -d -c "pwrite -S 0xab 0 4K" $SCRATCH_MNT/foobar |
> _filter_xfs_io
> +sectorsize=$(_scratch_sectorsize)
This new helper takes use of "btrfs inspect-internal dump-super", so the
test needs a new _require rule:
_require_btrfs_command inspect-internal dump-super
Otherwise the patch looks fine to me. But I'd be great if other btrfs
folks could help review the change and/or help actually test the changes
with 64k sectorsize.
Thanks,
Eryu
> +$XFS_IO_PROG -f -d -c "pwrite -S 0xab 0 $sectorsize" $SCRATCH_MNT/foobar |\
> + _filter_xfs_io_numbers
>
> $BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT \
> $SCRATCH_MNT/mysnap1 > /dev/null
>
> # Clone the regular (not inlined) extent.
> -$XFS_IO_PROG -c "reflink $SCRATCH_MNT/foobar 0 8K 4K" $SCRATCH_MNT/foobar \
> - | _filter_xfs_io
> +$XFS_IO_PROG -c \
> + "reflink $SCRATCH_MNT/foobar 0 $((2 * $sectorsize)) $sectorsize" \
> + $SCRATCH_MNT/foobar | _filter_xfs_io_numbers
>
> $BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT \
> $SCRATCH_MNT/mysnap2 > /dev/null
> @@ -76,21 +82,26 @@ $BTRFS_UTIL_PROG send -p $SCRATCH_MNT/mysnap1 -f
> $send_files_dir/2.snap \
>$SCRATCH_MNT/mysnap2 2>&1 >/dev/null | _filter_scratch
>
> echo "File digests in the original filesystem:"
> -md5sum $SCRATCH_MNT/mysnap1/foobar | _filter_scratch
> -md5sum $SCRATCH_MNT/mysnap2/foobar | _filter_scratch
> +sum_src_snap1=$(md5sum $SCRATCH_MNT/mysnap1/foobar | awk '{print $1}')
> +sum_src_snap2=$(md5sum $SCRATCH_MNT/mysnap2/foobar | awk '{print $1}')
> +echo "src checksum created"
>
> # Now recreate the filesystem by receiving both send streams and verify we
> get
> # the same file content that the original filesystem had.
> _scratch_unmount
> _scratch_mkfs >>$seqres.full 2>&1
> -_scratch_mount "-o compress"
> +_scratch_mount "-o compress,max_inline=0"
>
> $BTRFS_UTIL_PROG receive -f $send_files_dir/1.snap $SCRATCH_MNT > /dev/null
> $BTRFS_UTIL_PROG receive -f $send_files_dir/2.snap $SCRATCH_MNT > /dev/null
>
> echo "File digests in the new filesystem:"
> -md5sum $SCRATCH_MNT/mysnap1/foobar | _filter_scratch
> -md5sum $SCRATCH_MNT/mysnap2/foobar | _filter_scratch
> +sum_dest_snap1=$(md5sum $SCRATCH_MNT/mysnap1/foobar | awk '{print $1}')
> +sum_dest_snap2=$(md5sum $SCRATCH_MNT/mysnap2/foobar | awk '{
Re: [PATCH] generic: test for deduplication between different files
On Sun, Aug 19, 2018 at 04:41:31PM +0100, Filipe Manana wrote:
> On Sun, Aug 19, 2018 at 3:07 PM, Eryu Guan wrote:
> > On Fri, Aug 17, 2018 at 09:39:24AM +0100, fdman...@kernel.org wrote:
> >> From: Filipe Manana
> >>
> >> Test that deduplication of an entire file that has a size that is not
> >> aligned to the filesystem's block size into a different file does not
> >> corrupt the destination's file data.
> >>
> >> This test is motivated by a bug found in Btrfs which is fixed by the
> >> following patch for the linux kernel:
> >>
> >> "Btrfs: fix data corruption when deduplicating between different files"
> >>
> >> XFS also fails this test, at least as of linux kernel 4.18-rc7, exactly
> >> with the same corruption as in Btrfs - some bytes of a block get replaced
> >> with zeroes after the deduplication.
> >>
> >> Signed-off-by: Filipe Manana
> >> ---
> >> tests/generic/505 | 84
> >> +++
> >> tests/generic/505.out | 33
> >> tests/generic/group | 1 +
> >> 3 files changed, 118 insertions(+)
> >> create mode 100755 tests/generic/505
> >> create mode 100644 tests/generic/505.out
> >>
> >> diff --git a/tests/generic/505 b/tests/generic/505
> >> new file mode 100755
> >> index ..5ee232a2
> >> --- /dev/null
> >> +++ b/tests/generic/505
> >> @@ -0,0 +1,84 @@
> >> +#! /bin/bash
> >> +# SPDX-License-Identifier: GPL-2.0
> >> +# Copyright (C) 2018 SUSE Linux Products GmbH. All Rights Reserved.
> >> +#
> >> +# FS QA Test No. 505
> >> +#
> >> +# Test that deduplication of an entire file that has a size that is not
> >> aligned
> >> +# to the filesystem's block size into a different file does not corrupt
> >> the
> >> +# destination's file data.
> >> +#
> >> +seq=`basename $0`
> >> +seqres=$RESULT_DIR/$seq
> >> +echo "QA output created by $seq"
> >> +tmp=/tmp/$$
> >> +status=1 # failure is the default!
> >> +trap "_cleanup; exit \$status" 0 1 2 3 15
> >> +
> >> +_cleanup()
> >> +{
> >> + cd /
> >> + rm -f $tmp.*
> >> +}
> >> +
> >> +# get standard environment, filters and checks
> >> +. ./common/rc
> >> +. ./common/filter
> >> +. ./common/reflink
> >> +
> >> +# real QA test starts here
> >> +_supported_fs generic
> >> +_supported_os Linux
> >> +_require_scratch_dedupe
> >> +
> >> +rm -f $seqres.full
> >> +
> >> +_scratch_mkfs >>$seqres.full 2>&1
> >> +_scratch_mount
> >> +
> >> +# The first byte with a value of 0xae starts at an offset (2518890) which
> >> is not
> >> +# a multiple of the block size.
> >> +$XFS_IO_PROG -f \
> >> + -c "pwrite -S 0x6b 0 2518890" \
> >> + -c "pwrite -S 0xae 2518890 102398" \
> >> + $SCRATCH_MNT/foo | _filter_xfs_io
> >> +
> >> +# Create a second file with a length not aligned to the block size, whose
> >> bytes
> >> +# all have the value 0x6b, so that its extent(s) can be deduplicated with
> >> the
> >> +# first file.
> >> +$XFS_IO_PROG -f -c "pwrite -S 0x6b 0 557771" $SCRATCH_MNT/bar |
> >> _filter_xfs_io
> >> +
> >> +# The file is filled with bytes having the value 0x6b from offset 0 to
> >> offset
> >> +# 2518889 and with the value 0xae from offset 2518890 to offset 2621287.
> >> +echo "File content before deduplication:"
> >> +od -t x1 $SCRATCH_MNT/foo
> >> +
> >> +# Now deduplicate the entire second file into a range of the first file
> >> that
> >> +# also has all bytes with the value 0x6b. The destination range's end
> >> offset
> >> +# must not be aligned to the block size and must be less then the offset
> >> of
> >> +# the first byte with the value 0xae (byte at offset 2518890).
> >> +$XFS_IO_PROG -c "dedupe $SCRATCH_MNT/bar 0 1957888 557771"
> >> $SCRATCH_MNT/foo \
> >> + | _filter_xfs_io
> >> +
> >> +# The bytes in the range starting at offset 2515659 (end of the
> >> deduplication
> >> +# range) and ending at offset 2519040 (start offset rounded up to the
> >> block
> >> +# size) must all have the value 0xae (and not replaced with 0x00 values).
> >
> > This doesn't seem right to me, range [2515659, 2518890) should be 0x6b
> > not 0xae, while range [2518890, 2519040) indeed should contain 0xae.
>
> Yes, indeed. My mistake (got it right in the comment before the first
> call to "od").
> Can you fix it up (if there's nothing else to fix), or do you need me
> to send a new version?
Sure, I can fix it on commit. But I've already pushed this week's update
to upstream, so you won't see it until next week :)
Thanks,
Eryu
Re: [PATCH] generic: test for deduplication between different files
On Fri, Aug 17, 2018 at 09:39:24AM +0100, fdman...@kernel.org wrote:
> From: Filipe Manana
>
> Test that deduplication of an entire file that has a size that is not
> aligned to the filesystem's block size into a different file does not
> corrupt the destination's file data.
>
> This test is motivated by a bug found in Btrfs which is fixed by the
> following patch for the linux kernel:
>
> "Btrfs: fix data corruption when deduplicating between different files"
>
> XFS also fails this test, at least as of linux kernel 4.18-rc7, exactly
> with the same corruption as in Btrfs - some bytes of a block get replaced
> with zeroes after the deduplication.
>
> Signed-off-by: Filipe Manana
> ---
> tests/generic/505 | 84
> +++
> tests/generic/505.out | 33
> tests/generic/group | 1 +
> 3 files changed, 118 insertions(+)
> create mode 100755 tests/generic/505
> create mode 100644 tests/generic/505.out
>
> diff --git a/tests/generic/505 b/tests/generic/505
> new file mode 100755
> index ..5ee232a2
> --- /dev/null
> +++ b/tests/generic/505
> @@ -0,0 +1,84 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (C) 2018 SUSE Linux Products GmbH. All Rights Reserved.
> +#
> +# FS QA Test No. 505
> +#
> +# Test that deduplication of an entire file that has a size that is not
> aligned
> +# to the filesystem's block size into a different file does not corrupt the
> +# destination's file data.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/reflink
> +
> +# real QA test starts here
> +_supported_fs generic
> +_supported_os Linux
> +_require_scratch_dedupe
> +
> +rm -f $seqres.full
> +
> +_scratch_mkfs >>$seqres.full 2>&1
> +_scratch_mount
> +
> +# The first byte with a value of 0xae starts at an offset (2518890) which is
> not
> +# a multiple of the block size.
> +$XFS_IO_PROG -f \
> + -c "pwrite -S 0x6b 0 2518890" \
> + -c "pwrite -S 0xae 2518890 102398" \
> + $SCRATCH_MNT/foo | _filter_xfs_io
> +
> +# Create a second file with a length not aligned to the block size, whose
> bytes
> +# all have the value 0x6b, so that its extent(s) can be deduplicated with the
> +# first file.
> +$XFS_IO_PROG -f -c "pwrite -S 0x6b 0 557771" $SCRATCH_MNT/bar |
> _filter_xfs_io
> +
> +# The file is filled with bytes having the value 0x6b from offset 0 to offset
> +# 2518889 and with the value 0xae from offset 2518890 to offset 2621287.
> +echo "File content before deduplication:"
> +od -t x1 $SCRATCH_MNT/foo
> +
> +# Now deduplicate the entire second file into a range of the first file that
> +# also has all bytes with the value 0x6b. The destination range's end offset
> +# must not be aligned to the block size and must be less then the offset of
> +# the first byte with the value 0xae (byte at offset 2518890).
> +$XFS_IO_PROG -c "dedupe $SCRATCH_MNT/bar 0 1957888 557771" $SCRATCH_MNT/foo \
> + | _filter_xfs_io
> +
> +# The bytes in the range starting at offset 2515659 (end of the deduplication
> +# range) and ending at offset 2519040 (start offset rounded up to the block
> +# size) must all have the value 0xae (and not replaced with 0x00 values).
This doesn't seem right to me, range [2515659, 2518890) should be 0x6b
not 0xae, while range [2518890, 2519040) indeed should contain 0xae.
Thanks,
Eryu
> +# In other words, we should have exactly the same data we had before we asked
> +# for deduplication.
> +echo "File content after deduplication and before unmounting:"
> +od -t x1 $SCRATCH_MNT/foo
> +
> +# Unmount the filesystem and mount it again. This guarantees any file data in
> +# the page cache is dropped.
> +_scratch_cycle_mount
> +
> +# The bytes in the range starting at offset 2515659 (end of the deduplication
> +# range) and ending at offset 2519040 (start offset rounded up to the block
> +# size) must all have the value 0xae (and not replaced with 0x00 values).
> +# In other words, we should have exactly the same data we had before we asked
> +# for deduplication.
> +echo "File content after unmounting:"
> +od -t x1 $SCRATCH_MNT/foo
> +
> +status=0
> +exit
> diff --git a/tests/generic/505.out b/tests/generic/505.out
> new file mode 100644
> index ..7556b9fb
> --- /dev/null
> +++ b/tests/generic/505.out
> @@ -0,0 +1,33 @@
> +QA output created by 505
> +wrote 2518890/2518890 bytes at offset 0
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +wrote 102398/102398 bytes at offset 2518890
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +wrote 557771/557771 bytes at offset 0
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +File content before dedu
Re: [PATCH] fstests: btrfs: Add test for corrupted orphan qgroup numbers
On Fri, Aug 10, 2018 at 05:10:29PM +0800, Qu Wenruo wrote:
>
>
> On 8/10/18 4:54 PM, Filipe Manana wrote:
> > On Fri, Aug 10, 2018 at 9:46 AM, Qu Wenruo wrote:
> >>
> >>
> >> On 8/9/18 5:26 PM, Filipe Manana wrote:
> >>> On Thu, Aug 9, 2018 at 8:45 AM, Qu Wenruo wrote:
> This bug is exposed by populating a high level qgroup, and then make it
> orphan (high level qgroup without child)
> >>>
> >>> Same comment as in the kernel patch:
> >>>
> >>> "That sentence is confusing. An orphan, by definition [1], is someone
> >>> (or something in this case) without parents.
> >>> But you mention a group without children, so that should be named
> >>> "childless" or simply say "without children".
> >>> So one part of the sentence is wrong, either what is in parenthesis or
> >>> what comes before them.
> >>>
> >>> [1] https://www.thefreedictionary.com/orphan
> >>> "
> >>>
> with old qgroup numbers, and
> finally do rescan.
>
> Normally rescan should zero out all qgroups' accounting number, but due
> to a kernel bug which won't mark orphan qgroups dirty, their on-disk
> data is not updated, thus old numbers remain and cause qgroup
> corruption.
>
> Fixed by the following kernel patch:
> "btrfs: qgroup: Dirty all qgroups before rescan"
>
> Reported-by: Misono Tomohiro
> Signed-off-by: Qu Wenruo
> ---
> tests/btrfs/170 | 82 +
> tests/btrfs/170.out | 3 ++
> tests/btrfs/group | 1 +
> 3 files changed, 86 insertions(+)
> create mode 100755 tests/btrfs/170
> create mode 100644 tests/btrfs/170.out
>
> diff --git a/tests/btrfs/170 b/tests/btrfs/170
> new file mode 100755
> index ..bcf8b5c0e4f3
> --- /dev/null
> +++ b/tests/btrfs/170
> @@ -0,0 +1,82 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2018 SUSE Linux Products GmbH. All Rights Reserved.
> +#
> +# FS QA Test 170
> +#
> +# Test if btrfs can clear orphan (high level qgroup without child)
> qgroup's
> +# accounting numbers during rescan.
> +# Fixed by the following kernel patch:
> +# "btrfs: qgroup: Dirty all qgroups before rescan"
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch
> +
> +_scratch_mkfs > /dev/null 2>&1
> +_scratch_mount
> +
> +
> +# Populate the fs
> +_run_btrfs_util_prog subvolume create "$SCRATCH_MNT/subvol"
> +_pwrite_byte 0xcdcd 0 1M "$SCRATCH_MNT/subvol/file1" | _filter_xfs_io >
> /dev/null
> +
> +# Ensure that file reach disk, so it will also appear in snapshot
> >>>
> >>> # Ensure that buffered file data is persisted, so we won't have an
> >>> empty file in the snapshot.
> +sync
> +_run_btrfs_util_prog subvolume snapshot "$SCRATCH_MNT/subvol"
> "$SCRATCH_MNT/snapshot"
> +
> +
> +_run_btrfs_util_prog quota enable "$SCRATCH_MNT"
> +_run_btrfs_util_prog quota rescan -w "$SCRATCH_MNT"
> +
> +# Create high level qgroup
> +_run_btrfs_util_prog qgroup create 1/0 "$SCRATCH_MNT"
> +
> +# Don't use _run_btrfs_util_prog here, as it can return 1 to info user
> +# that qgroup is marked inconsistent, this is a bug in btrfs-progs, but
> +# to ensure it will work, we just ignore the return value.
> >>>
> >>> Comment should go away IMHO. The preferred way is to call
> >>> $BTRFS_UTIL_PROG and have failures noticed
> >>> through differences in the golden output. There's no point in
> >>> mentioning something that currently doesn't work
> >>> if it's not used here.
> >>
> >> In this case, I think we still need to mention why we don't use
> >> _run_btrfs_util_progs, in fact if we use _run_btrfs_util_progs, the test
> >> will just fail due to the return value.
> >>
> >> In fact, it's a workaround and worthy noting IIRC.
> >
> > Still disagree, because we are not checking the return value and rely
> > on errors printing something to stderr/stdout.
>
> OK, either way I'll introduce a new filter here for filtering out either
> "Quota data changed, rescan scheduled" or "quotas may be inconsistent,
> rescan needed".
>
> As there is patch floating around to change the default behavior of
> "btrf
Re: [PATCH v2] fstests: btrfs/168 verify device ready after device delete
On Fri, Jul 06, 2018 at 02:01:37PM +0800, Anand Jain wrote:
> This test case verifies if the device ready return success after the
> device delete.
>
> Signed-off-by: Anand Jain
Need some helps from btrfs folks to see if it's a valid test. Thanks!
Eryu
> ---
> v1->v2: use _run_btrfs_util_prog instead of open coding it.
>
> tests/btrfs/168 | 68
> +
> tests/btrfs/168.out | 2 ++
> tests/btrfs/group | 1 +
> 3 files changed, 71 insertions(+)
> create mode 100755 tests/btrfs/168
> create mode 100644 tests/btrfs/168.out
>
> diff --git a/tests/btrfs/168 b/tests/btrfs/168
> new file mode 100755
> index ..0d3e99839209
> --- /dev/null
> +++ b/tests/btrfs/168
> @@ -0,0 +1,68 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (C) 2018 Oracle. All Rights Reserved.
> +#
> +# FS QA Test 168
> +#
> +# Test if btrfs is still reported ready after the device delete
> +#
> +# This could be fixed by the following kernel commit:
> +# btrfs: fix missing superblock update in the device delete commit
> transaction
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_command "$BTRFS_TUNE_PROG" btrfstune
> +_require_scratch_dev_pool 2
> +
> +_scratch_dev_pool_get 2
> +dev_1=$(echo $SCRATCH_DEV_POOL | awk '{print $1}')
> +dev_2=$(echo $SCRATCH_DEV_POOL | awk '{print $2}')
> +
> +# normal delete device and then check for ready
> +run_check _scratch_pool_mkfs "-d single -m single"
> +_scratch_mount
> +_run_btrfs_util_prog device delete $dev_1 $SCRATCH_MNT
> +_run_btrfs_util_prog device ready $dev_2
> +
> +_scratch_unmount
> +# delete a seed device and then check for ready
> +run_check $BTRFS_TUNE_PROG -S 1 $dev_2
> +run_check _mount $dev_2 $SCRATCH_MNT
> +_run_btrfs_util_prog device add -f $dev_1 $SCRATCH_MNT
> +run_check mount -o rw,remount $dev_1 $SCRATCH_MNT
> +_run_btrfs_util_prog device delete $dev_2 $SCRATCH_MNT
> +_run_btrfs_util_prog device ready $dev_1
> +
> +_scratch_unmount
> +_scratch_dev_pool_put
> +
> +echo "Silence is golden"
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/btrfs/168.out b/tests/btrfs/168.out
> new file mode 100644
> index ..893a41d859c8
> --- /dev/null
> +++ b/tests/btrfs/168.out
> @@ -0,0 +1,2 @@
> +QA output created by 168
> +Silence is golden
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index 5cff3bd6cc03..7bc3ea457992 100644
> --- a/tests/btrfs/group
> +++ b/tests/btrfs/group
> @@ -170,3 +170,4 @@
> 165 auto quick subvol
> 166 auto quick qgroup
> 167 auto quick replace volume
> +168 auto quick volume
> --
> 2.15.0
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] fstests: btrfs/168 verify device ready after device delete
On Tue, Jul 03, 2018 at 04:47:53PM +0800, Anand Jain wrote:
> This test case verifies if the device ready return success after the
> device delete.
>
> Signed-off-by: Anand Jain
Looks fine to me overall, but I may need some helps from btrfs folks :)
> ---
> tests/btrfs/168 | 68
> +
> tests/btrfs/168.out | 2 ++
> tests/btrfs/group | 1 +
> 3 files changed, 71 insertions(+)
> create mode 100755 tests/btrfs/168
> create mode 100644 tests/btrfs/168.out
>
> diff --git a/tests/btrfs/168 b/tests/btrfs/168
> new file mode 100755
> index ..cb5b8eb4b5a8
> --- /dev/null
> +++ b/tests/btrfs/168
> @@ -0,0 +1,68 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (C) 2018 Oracle. All Rights Reserved.
> +#
> +# FS QA Test 168
> +#
> +# Test if btrfs is still reported ready after the device delete
> +#
> +# This could be fixed by the following kernel commit:
> +# btrfs: fix missing superblock update in the device delete commit
> transaction
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_command "$BTRFS_TUNE_PROG" btrfstune
> +_require_scratch_dev_pool 2
> +
> +_scratch_dev_pool_get 2
> +dev_1=$(echo $SCRATCH_DEV_POOL | awk '{print $1}')
> +dev_2=$(echo $SCRATCH_DEV_POOL | awk '{print $2}')
> +
> +# normal delete device and then check for ready
> +run_check _scratch_pool_mkfs "-d single -m single"
> +_scratch_mount
> +run_check $BTRFS_UTIL_PROG device delete $dev_1 $SCRATCH_MNT
> +run_check $BTRFS_UTIL_PROG device ready $dev_2
Why not "_run_btrfs_util_prog device delete "
> +
> +_scratch_unmount
> +# delete a seed device and then check for ready
> +run_check $BTRFS_TUNE_PROG -S 1 $dev_2
> +run_check _mount $dev_2 $SCRATCH_MNT
> +_run_btrfs_util_prog device add -f $dev_1 $SCRATCH_MNT
Like this one?
But I still prefer dropping run_check family functions completely, just
using bare command, if the output of these commands are not
deterministic we could simply through them away.
Thanks,
Eryu
> +run_check mount -o rw,remount $dev_1 $SCRATCH_MNT
> +run_check $BTRFS_UTIL_PROG device delete $dev_2 $SCRATCH_MNT
> +run_check $BTRFS_UTIL_PROG device ready $dev_1
> +
> +_scratch_unmount
> +_scratch_dev_pool_put
> +
> +echo "Silence is golden"
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/btrfs/168.out b/tests/btrfs/168.out
> new file mode 100644
> index ..893a41d859c8
> --- /dev/null
> +++ b/tests/btrfs/168.out
> @@ -0,0 +1,2 @@
> +QA output created by 168
> +Silence is golden
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index 5cff3bd6cc03..7bc3ea457992 100644
> --- a/tests/btrfs/group
> +++ b/tests/btrfs/group
> @@ -170,3 +170,4 @@
> 165 auto quick subvol
> 166 auto quick qgroup
> 167 auto quick replace volume
> +168 auto quick volume
> --
> 2.15.0
>
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] fstests: btrfs: Test if btrfs will corrupt nodatasum compressed extent when replacing device
On Thu, Jun 28, 2018 at 08:11:00AM +0300, Nikolay Borisov wrote: > > > On 1.06.2018 04:34, Qu Wenruo wrote: > > This is a long existing bug (from 2012) but exposed by a reporter > > recently, that when compressed extent without data csum get written to > > device-replace target device, the written data is in fact uncompressed data > > other than the original compressed data. > > > > And since btrfs still consider the data is compressed and will try to read > > it > > as compressed, it can cause read error. > > > > The root cause is located, and one RFC patch already sent to fix it, > > titled "[PATCH RFC] btrfs: scrub: Don't use inode pages for device replace". > > (The RFC is only for the extra possible way to fix the bug, the fix > > itself should work without problem) > > > > Reported-by: James Harvey > > Signed-off-by: Qu Wenruo > > Reviewed-by: Nikolay Borisov Thanks for the review! I assume the v3 patch also passes your review :) Eryu -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] fstests: btrfs/085: replace btrfs-debug-tree with btrfs inspect-internal dump-tree
On Thu, Jun 21, 2018 at 03:04:22PM +0800, Lu Fengqi wrote:
> Since btrfs-dump-tree has been removed from btrfs-progs, use btrfs
> inspect-internal dump-tree instead of btrfs-dump-tree.
>
> Signed-off-by: Lu Fengqi
Then there's no user of $BTRFS_DEBUG_TREE_PROG, I think we could remove
the definition too in common/config
export BTRFS_DEBUG_TREE_PROG=$(type -P btrfs-debug-tree)
I'll remove it on commit.
Thanks,
Eryu
> ---
> tests/btrfs/085 | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/tests/btrfs/085 b/tests/btrfs/085
> index 8cc69c21c0dd..4773ed8041fd 100755
> --- a/tests/btrfs/085
> +++ b/tests/btrfs/085
> @@ -39,14 +39,14 @@ _supported_fs btrfs
> _supported_os Linux
> _require_scratch
> _require_dm_target flakey
> -_require_command "$BTRFS_DEBUG_TREE_PROG" btrfs-debug-tree
> +_require_btrfs_command inspect-internal dump-tree
>
> rm -f $seqres.full
>
> has_orphan_item()
> {
> INO=$1
> - if $BTRFS_DEBUG_TREE_PROG $SCRATCH_DEV | \
> + if $BTRFS_UTIL_PROG inspect-internal dump-tree $SCRATCH_DEV | \
> grep -q "key (ORPHAN ORPHAN_ITEM $INO)"; then
> return 0
> fi
> --
> 2.17.1
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] fstests: btrfs: Test if btrfs will corrupt nodatasum compressed extent when replacing device
On Fri, Jun 08, 2018 at 02:17:23PM +0800, Qu Wenruo wrote:
> This is a long existing bug (from 2012) but exposed by a reporter
> recently, that when compressed extent without data csum get written to
> device-replace target device, the written data is in fact uncompressed data
> other than the original compressed data.
>
> And since btrfs still consider the data is compressed and will try to read it
> as compressed, it can cause read error.
>
> The root cause is located, and one RFC patch already sent to fix it,
> titled "[PATCH] btrfs: scrub: Don't use inode pages for device replace".
> (The RFC is only for the extra possible way to fix the bug, the fix
> itself should work without problem)
>
> Reported-by: James Harvey
> Signed-off-by: Qu Wenruo
> ---
> changelog:
> v2:
> Now the fix patch is no longer RFC.
> Remove _require_test as we don't really touch it.
> Add comment on the mount cycle.
> Add the test to group 'volume'.
Thanks for the revision! But again, I'd like to get some explicit
reviews from btrfs folks.
> ---
> tests/btrfs/161 | 91 +
> tests/btrfs/161.out | 2 +
> tests/btrfs/group | 1 +
> 3 files changed, 94 insertions(+)
> create mode 100755 tests/btrfs/161
> create mode 100644 tests/btrfs/161.out
>
> diff --git a/tests/btrfs/161 b/tests/btrfs/161
> new file mode 100755
> index ..ce1b0e04
> --- /dev/null
> +++ b/tests/btrfs/161
> @@ -0,0 +1,91 @@
> +#! /bin/bash
> +# FS QA Test 161
> +#
> +# Test if btrfs will corrupt compressed data extent without data csum
> +# by replacing it with uncompressed data, when doing replacing device.
> +#
> +# This could be fixed by the following RFC patch:
can be dropped?
Thanks,
Eryu
P.S.
*IF* you're going to send v3, could you please follow the new test
template (create new test with './new btrfs' would do the work) and
rebase against latest master? That'd be easier for me to apply the
patch, but I'm also fine with taking it as-is (after we get Reviewed-by
tag), I can convert the test and re-number it on commit as always.
> +# "[PATCH] btrfs: scrub: Don't use inode pages for device replace"
> +#
> +#---
> +# Copyright (C) 2018 SUSE Linux Products GmbH. All Rights Reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> +#---
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch_dev_pool 2
> +_require_scratch_dev_pool_equal_size
> +
> +
> +_scratch_dev_pool_get 1
> +_spare_dev_get
> +_scratch_pool_mkfs >> $seqres.full 2>&1
> +
> +# Create nodatasum inode
> +_scratch_mount "-o nodatasum"
> +touch $SCRATCH_MNT/nodatasum_file
> +_scratch_remount "datasum,compress"
> +_pwrite_byte 0xcd 0 128K $SCRATCH_MNT/nodatasum_file > /dev/null
> +
> +# Write the compressed data back to disk
> +sync
> +
> +# Replace the device
> +_run_btrfs_util_prog replace start -Bf 1 $SPARE_DEV $SCRATCH_MNT
> +
> +# Unmount to drop all cache so next read will read from disk
> +_scratch_unmount
> +_mount $SPARE_DEV $SCRATCH_MNT
> +
> +# Now the EXTENT_DATA item still marks the extent as compressed,
> +# but the on-disk data is uncompressed, thus reading it as compressed
> +# will definitely cause EIO.
> +cat $SCRATCH_MNT/nodatasum_file > /dev/null
> +
> +_scratch_unmount
> +_spare_dev_put
> +_scratch_dev_pool_put
> +
> +echo "Silence is golden"
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/btrfs/161.out b/tests/btrfs/161.out
> new file mode 100644
> index ..1752a243
> --- /dev/null
> +++ b/tests/btrfs/161.out
> @@ -0,0 +1,2 @@
> +QA output created by 161
> +Silence is golden
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index f04ee8d5..9195b368 100644
> --- a/tests/btrfs/group
Re: [PATCH v3] btrfs: Add test that checks rmdir(2) can delete a subvolume
On Wed, Jun 13, 2018 at 03:06:45PM +0900, Misono Tomohiro wrote:
> Add btrfs test that checks "rmdir" or "rm -r" command can delete a
> subvolume like an ordinary directory.
>
> This behavior has been restricted long time but becomes allowed by
> following commit in the kernel:
> btrfs: Allow rmdir(2) to delete an empty subvolume
As the patch is already upstream, I'll add commit id in reference too.
a79a464d5675 ("btrfs: Allow rmdir(2) to delete an empty subvolume")
>
> The test will be skipped if kernel does not support the feature,
> which can be checked whether /sys/fs/btrfs/features/rmdir_subvol
> exists or not.
>
> Reviewed-by: David Sterba
> Signed-off-by: Misono Tomohiro
> ---
> changelog:
> v2 -> v3 - Skip test if kernel does not support the feature by
> checking sysfs
> - Update license notation
Thanks a lot for the revision and using new template!
>
> This test should pass on kernel 4.18-rc1~ (or in current linus' master),
> otherwise it will be skipped. Please change the test number appropriately
> when applied as other pending tests exists.
>
> Thanks,
> Tomohiro Misono
>
> tests/btrfs/200 | 128
>
> tests/btrfs/200.out | 2 +
> tests/btrfs/group | 1 +
> 3 files changed, 131 insertions(+)
> create mode 100755 tests/btrfs/200
> create mode 100644 tests/btrfs/200.out
>
> diff --git a/tests/btrfs/200 b/tests/btrfs/200
> new file mode 100755
> index ..15213eed
> --- /dev/null
> +++ b/tests/btrfs/200
> @@ -0,0 +1,128 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2018 Fujitsu. All Rights Reserved.
> +#
> +# FS QA Test btrfs/200
> +#
> +# QA test that checks rmdir(2) works for subvolumes like ordinary
> directories.
> +#
> +# This behavior has been restricted long time but becomes allowed by
> following
> +# patch in the kernel:
> +# btrfs: Allow rmdir(2) to delete an empty subvolume
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +create_subvol()
> +{
> + $BTRFS_UTIL_PROG subvolume create $1 >> $seqres.full 2>&1
> +}
> +
> +create_snapshot()
> +{
> + $BTRFS_UTIL_PROG subvolume snapshot $@ >> $seqres.full 2>&1
> +}
> +
> +rmdir_subvol()
> +{
> + rmdir $1 >> $seqres.full 2>&1
> +}
> +
> +rm_r_subvol() {
> + rm -r $1 >> $seqres.full 2>&1
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch
> +
> +if [ ! -e /sys/fs/btrfs/features/rmdir_subvol ]; then
> + _notrun "The kernel does not support the deletion of subvolume by rmdir"
> +fi
Just notice that there's a _require_btrfs_fs_feature helper that could
do this exact check. I'll fix it on commit.
_require_btrfs_fs_feature "rmdir_subvol"
Thanks,
Eryu
> +
> +_scratch_mkfs > /dev/null 2>&1 || _fail "mkfs failed"
> +_scratch_mount
> +
> +# Check that an empty subvolume can be deleted by rmdir
> +create_subvol $SCRATCH_MNT/sub1
> +rmdir_subvol $SCRATCH_MNT/sub1 || \
> + echo "rmdir should delete an empty subvolume"
> +
> +# Check that non-empty subvolume cannot be deleted by rmdir
> +create_subvol $SCRATCH_MNT/sub2
> +touch $SCRATCH_MNT/sub2/file
> +rmdir_subvol $SCRATCH_MNT/sub2 && \
> + echo "rmdir should fail for non-empty subvolume"
> +rm $SCRATCH_MNT/sub2/file
> +rmdir_subvol $SCRATCH_MNT/sub2 || \
> + echo "rmdir should delete an empty subvolume"
> +
> +# Check that read-only empty subvolume can be deleted by rmdir
> +create_subvol $SCRATCH_MNT/sub3
> +create_snapshot -r $SCRATCH_MNT/sub3 $SCRATCH_MNT/snap
> +$BTRFS_UTIL_PROG property set $SCRATCH_MNT/sub3 ro true >> $seqres.full 2>&1
> +rmdir_subvol $SCRATCH_MNT/sub3 || \
> + echo "rmdir should delete an empty subvolume"
> +rmdir_subvol $SCRATCH_MNT/snap || \
> + echo "rmdir should delete a readonly empty subvolume"
> +
> +# Check that the default subvolume cannot be deleted by rmdir
> +create_subvol $SCRATCH_MNT/sub4
> +subvolid=$(_btrfs_get_subvolid $SCRATCH_MNT sub4)
> +$BTRFS_UTIL_PROG subvolume set-default $subvolid $SCRATCH_MNT \
> + >> $seqres.full 2>&1
> +rmdir_subvol $SCRATCH_MNT/sub4 && \
> + echo "rmdir should fail for the default subvolume"
> +
> +# Check that subvolume stub (created by snapshot) can be deleted by rmdir
> +# (Note: this has been always allowed)
> +create_subvol $SCRATCH_MNT/sub5
> +create_subvol $SCRATCH_MNT/sub5/sub6
> +create_snapshot $SCRATCH_MNT/sub5 $SCRATCH_MNT/snap2
> +rmdir $SCRATCH_MNT/snap2/sub6 || \
> + echo "rmdir should delete a subvolume stub (ino number == 2)"
> +
> +# Check that rm -r works for both non-snapshot subv
Re: [PATCH] generic: add test for fsync of directory after creating hard link
On Mon, Jun 11, 2018 at 07:24:35PM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > Test that if we create a new hard link for a file which was previously > fsync'ed, fsync a parent directory of the new hard link and power fail, > the parent directory exists after mounting the filesystem again. The > parent directory must be a new directory, not yet persisted. > > This test is motivated by a bug found in btrfs, where the fsync'ed parent > directory was lost after a power failure. The bug in btrfs is fixed by a > patch for the linux kernel titled: > > "Btrfs: sync log after logging new name" > > Signed-off-by: Filipe Manana Looks good to me. > --- > tests/generic/498 | 65 > +++ > tests/generic/498.out | 2 ++ > tests/generic/group | 1 + > 3 files changed, 68 insertions(+) > create mode 100755 tests/generic/498 > create mode 100644 tests/generic/498.out > > diff --git a/tests/generic/498 b/tests/generic/498 > new file mode 100755 > index ..1cf73bda > --- /dev/null > +++ b/tests/generic/498 > @@ -0,0 +1,65 @@ > +#! /bin/bash > +# SPDX-License-Identifier: GPL-2.0 > +# Copyright (C) 2018 SUSE Linux Products GmbH. All Rights Reserved. > +# > +# FS QA Test No. 498 > +# > +# Test that if we create a new hard link for a file which was previously > +# fsync'ed, fsync a parent directory of the new hard link and power fail, > +# the parent directory exists after mounting the filesystem again. > +# Thanks a lot for using new test template! Eryu -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] fstests: btrfs: Test if btrfs will corrupt nodatasum compressed extent when replacing device
On Fri, Jun 01, 2018 at 09:34:48AM +0800, Qu Wenruo wrote:
> This is a long existing bug (from 2012) but exposed by a reporter
> recently, that when compressed extent without data csum get written to
> device-replace target device, the written data is in fact uncompressed data
> other than the original compressed data.
>
> And since btrfs still consider the data is compressed and will try to read it
> as compressed, it can cause read error.
>
> The root cause is located, and one RFC patch already sent to fix it,
> titled "[PATCH RFC] btrfs: scrub: Don't use inode pages for device replace".
> (The RFC is only for the extra possible way to fix the bug, the fix
> itself should work without problem)
>
> Reported-by: James Harvey
> Signed-off-by: Qu Wenruo
Looks fine to me overall, some minor issues inline. But I'd really like
an explicit ACK from btrfs folks.
> ---
> tests/btrfs/161 | 91 +
> tests/btrfs/161.out | 2 +
> tests/btrfs/group | 1 +
> 3 files changed, 94 insertions(+)
> create mode 100755 tests/btrfs/161
> create mode 100644 tests/btrfs/161.out
>
> diff --git a/tests/btrfs/161 b/tests/btrfs/161
> new file mode 100755
> index ..d4a2b474
> --- /dev/null
> +++ b/tests/btrfs/161
> @@ -0,0 +1,91 @@
> +#! /bin/bash
> +# FS QA Test 161
> +#
> +# Test if btrfs will corrupt compressed data extent without data csum
> +# by replacing it with uncompressed data, when doing replacing device.
> +#
> +# This could be fixed by the following RFC patch:
> +# "[PATCH RFC] btrfs: scrub: Don't use inode pages for device replace"
> +#
> +#---
> +# Copyright (C) 2018 SUSE Linux Products GmbH. All Rights Reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> +#---
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_test
This is not needed, there's no point running fsck on $TEST_DEV after
test when it's not used at all.
> +_require_scratch_dev_pool 2
> +_require_scratch_dev_pool_equal_size
> +
> +
> +_scratch_dev_pool_get 1
> +_spare_dev_get
> +_scratch_pool_mkfs >> $seqres.full 2>&1
> +
> +# Create nodatasum inode
> +_scratch_mount "-o nodatasum"
> +touch $SCRATCH_MNT/nodatasum_file
> +_scratch_remount "datasum,compress"
> +_pwrite_byte 0xcd 0 128K $SCRATCH_MNT/nodatasum_file > /dev/null
> +
> +# Write the compressed data back to disk
> +sync
> +
> +# Replace the device
> +_run_btrfs_util_prog replace start -Bf 1 $SPARE_DEV $SCRATCH_MNT
> +
> +_scratch_unmount
> +
> +_mount $SPARE_DEV $SCRATCH_MNT
Better to explain the cycle mount with comments.
> +
> +# Since now the compressed extent contains *UNCOMPRESSED* data, reading it
> will
> +# easily trigger a EIO error
> +cat $SCRATCH_MNT/nodatasum_file > /dev/null
> +
> +_scratch_unmount
> +_spare_dev_put
> +_scratch_dev_pool_put
> +
> +echo "Silence is golden"
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/btrfs/161.out b/tests/btrfs/161.out
> new file mode 100644
> index ..1752a243
> --- /dev/null
> +++ b/tests/btrfs/161.out
> @@ -0,0 +1,2 @@
> +QA output created by 161
> +Silence is golden
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index f04ee8d5..f900b3d0 100644
> --- a/tests/btrfs/group
> +++ b/tests/btrfs/group
> @@ -163,3 +163,4 @@
> 158 auto quick raid scrub
> 159 auto quick
> 160 auto quick
> +161 auto quick replace
Anand introduced a new 'volume' group in commit 144c8463d38b ("btrfs:
introduce btrfs/volume group"), all 'replace' tests should be in
'volume' group now.
Thanks,
Eryu
> --
> 2.17.0
>
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsu
Re: [PATCH v2 1/4] fstests: btrfs: add seed sprout functionality test
On Mon, May 28, 2018 at 05:51:45PM +0800, Anand Jain wrote:
> Create a seed device and add the sprout device to it.
>
> Signed-off-by: Anand Jain
This series looks fine to me from fstests' point of view, there're just
some really minor common issues.
But I'd like some reviews from other btrfs folks.
> ---
> v1->v2: Use functions to do the respective operations. Add data
> verification.
>
> common/config | 1 +
> tests/btrfs/161 | 90
> +
> tests/btrfs/161.out | 9 ++
> tests/btrfs/group | 1 +
> 4 files changed, 101 insertions(+)
> create mode 100755 tests/btrfs/161
> create mode 100644 tests/btrfs/161.out
>
> diff --git a/common/config b/common/config
> index af360cefc804..f0735cb0153d 100644
> --- a/common/config
> +++ b/common/config
> @@ -235,6 +235,7 @@ case "$HOSTOS" in
> export BTRFS_UTIL_PROG="`set_prog_path btrfs`"
> export BTRFS_SHOW_SUPER_PROG="`set_prog_path btrfs-show-super`"
> export BTRFS_CONVERT_PROG="`set_prog_path btrfs-convert`"
> + export BTRFS_TUNE_PROG="`set_prog_path btrfstune`"
> export XFS_FSR_PROG="`set_prog_path xfs_fsr`"
> export MKFS_NFS_PROG="false"
> export MKFS_CIFS_PROG="false"
> diff --git a/tests/btrfs/161 b/tests/btrfs/161
> new file mode 100755
> index ..009e95354e85
> --- /dev/null
> +++ b/tests/btrfs/161
> @@ -0,0 +1,90 @@
> +#! /bin/bash
> +# FS QA Test 161 seed sprout functionality test
It'd be good to put the all test descriptions below the test seq number,
as all other tests do.
> +#
> +# Create a seed device, mount it and, add a new device to create a
> +# sprout filesystem.
> +#
> +#---
> +# Copyright (c) 2018 Oracle. All Rights Reserved.
> +# Author: Anand Jain
The author info is not needed.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> +#---
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch_dev_pool 2
_require_command "$BTRFS_TUNE_PROG" btrfstune
> +
> +_scratch_dev_pool_get 2
> +
> +DEV_SEED=$(echo $SCRATCH_DEV_POOL | awk '{print $1}')
> +DEV_SPROUT=$(echo $SCRATCH_DEV_POOL | awk '{print $2}')
Better to use lower case for local variables.
Thanks,
Eryu
> +
> +create_seed()
> +{
> + _mkfs_dev $DEV_SEED
> + run_check _mount $DEV_SEED $SCRATCH_MNT
> + $XFS_IO_PROG -f -d -c "pwrite -S 0xab 0 256K" $SCRATCH_MNT/foobar >\
> + /dev/null
> + echo -- golden --
> + od -x $SCRATCH_MNT/foobar
> + _run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
> + _scratch_unmount
> + $BTRFS_TUNE_PROG -S 1 $DEV_SEED
> + run_check _mount $DEV_SEED $SCRATCH_MNT
> +}
> +
> +create_sprout()
> +{
> + _run_btrfs_util_prog device add -f $DEV_SPROUT $SCRATCH_MNT
> + _scratch_unmount
> + run_check _mount $DEV_SPROUT $SCRATCH_MNT
> + echo -- sprout --
> + od -x $SCRATCH_MNT/foobar
> + _scratch_unmount
> +}
> +
> +create_seed
> +create_sprout
> +
> +_scratch_dev_pool_put
> +
> +status=0
> +exit
> diff --git a/tests/btrfs/161.out b/tests/btrfs/161.out
> new file mode 100644
> index ..363b8217f45c
> --- /dev/null
> +++ b/tests/btrfs/161.out
> @@ -0,0 +1,9 @@
> +QA output created by 161
> +-- golden --
> +000 abab abab abab abab abab abab abab abab
> +*
> +100
> +-- sprout --
> +000 abab abab abab abab abab abab abab abab
> +*
> +100
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index f04ee8d5297c..fe83631f0f33 100644
> --- a/tests/btrfs/group
> +++ b/tests/btrfs/group
> @@ -163,3 +163,4 @@
> 158 auto quick raid scrub
> 159 auto quick
> 160 auto quick
> +161 auto quick
> --
> 2.7.0
>
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body o
Re: [PATCH v2 5/5] generic: test invalid swap file activation
On Fri, May 18, 2018 at 07:37:07AM -0700, Darrick J. Wong wrote:
> On Wed, May 16, 2018 at 01:38:49PM -0700, Omar Sandoval wrote:
> > From: Omar Sandoval
> >
> > Swap files cannot have holes, and they must at least two pages.
> > swapon(8) and mkswap(8) have stricter restrictions, so add versions of
> > those commands without any restrictions.
> >
> > Signed-off-by: Omar Sandoval
> > ---
> > .gitignore| 2 ++
> > src/Makefile | 2 +-
> > src/mkswap.c | 83 +++
> > src/swapon.c | 24 +
> > tests/generic/490 | 77 +++
> > tests/generic/490.out | 5 +++
> > tests/generic/group | 1 +
> > 7 files changed, 193 insertions(+), 1 deletion(-)
> > create mode 100644 src/mkswap.c
> > create mode 100644 src/swapon.c
> > create mode 100755 tests/generic/490
> > create mode 100644 tests/generic/490.out
> >
> > diff --git a/.gitignore b/.gitignore
> > index 53029e24..efc73a7c 100644
> > --- a/.gitignore
> > +++ b/.gitignore
> > @@ -92,6 +92,7 @@
> > /src/lstat64
> > /src/makeextents
> > /src/metaperf
> > +/src/mkswap
> > /src/mmapcat
> > /src/multi_open_unlink
> > /src/nametest
> > @@ -111,6 +112,7 @@
> > /src/seek_sanity_test
> > /src/stale_handle
> > /src/stat_test
> > +/src/swapon
> > /src/t_access_root
> > /src/t_dir_offset
> > /src/t_dir_offset2
> > diff --git a/src/Makefile b/src/Makefile
> > index c42d3bb1..01fe99ef 100644
> > --- a/src/Makefile
> > +++ b/src/Makefile
> > @@ -26,7 +26,7 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize
> > preallo_rw_pattern_reader \
> > renameat2 t_getcwd e4compact test-nextquota punch-alternating \
> > attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type \
> > dio-invalidate-cache stat_test t_encrypted_d_revalidate \
> > - attr_replace_test
> > + attr_replace_test swapon mkswap
> >
> > SUBDIRS = log-writes perf
> >
> > diff --git a/src/mkswap.c b/src/mkswap.c
> > new file mode 100644
> > index ..d0bce2bd
> > --- /dev/null
> > +++ b/src/mkswap.c
> > @@ -0,0 +1,83 @@
> > +/* mkswap(8) without any sanity checks */
> > +
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +
> > +struct swap_header {
> > + charbootbits[1024];
> > + uint32_tversion;
> > + uint32_tlast_page;
> > + uint32_tnr_badpages;
> > + unsigned char sws_uuid[16];
> > + unsigned char sws_volume[16];
> > + uint32_tpadding[117];
> > + uint32_tbadpages[1];
> > +};
> > +
> > +int main(int argc, char **argv)
> > +{
> > + struct swap_header *hdr;
> > + FILE *file;
> > + struct stat st;
> > + long page_size;
> > + int ret;
> > +
> > + if (argc != 2) {
> > + fprintf(stderr, "usage: %s PATH\n", argv[0]);
> > + return EXIT_FAILURE;
> > + }
> > +
> > + page_size = sysconf(_SC_PAGESIZE);
> > + if (page_size == -1) {
> > + perror("sysconf");
> > + return EXIT_FAILURE;
> > + }
> > +
> > + hdr = calloc(1, page_size);
> > + if (!hdr) {
> > + perror("calloc");
> > + return EXIT_FAILURE;
> > + }
> > +
> > + file = fopen(argv[1], "r+");
> > + if (!file) {
> > + perror("fopen");
> > + free(hdr);
> > + return EXIT_FAILURE;
> > + }
> > +
> > + ret = fstat(fileno(file), &st);
> > + if (ret) {
> > + perror("fstat");
> > + free(hdr);
> > + fclose(file);
> > + return EXIT_FAILURE;
> > + }
> > +
> > + hdr->version = 1;
> > + hdr->last_page = st.st_size / page_size - 1;
> > + memset(&hdr->sws_uuid, 0x99, sizeof(hdr->sws_uuid));
> > + memcpy((char *)hdr + page_size - 10, "SWAPSPACE2", 10);
> > +
> > + if (fwrite(hdr, page_size, 1, file) != 1) {
> > + perror("fwrite");
> > + free(hdr);
> > + fclose(file);
> > + return EXIT_FAILURE;
> > + }
> > +
> > + if (fclose(file) == EOF) {
> > + perror("fwrite");
> > + free(hdr);
> > + return EXIT_FAILURE;
> > + }
> > +
> > + free(hdr);
> > +
> > + return EXIT_SUCCESS;
> > +}
> > diff --git a/src/swapon.c b/src/swapon.c
> > new file mode 100644
> > index ..0cb7108a
> > --- /dev/null
> > +++ b/src/swapon.c
> > @@ -0,0 +1,24 @@
> > +/* swapon(8) without any sanity checks; simply calls swapon(2) directly. */
> > +
> > +#include
> > +#include
> > +#include
> > +#include
> > +
> > +int main(int argc, char **argv)
> > +{
> > + int ret;
> > +
> > + if (argc != 2) {
> > + fprintf(stderr, "usage: %s PATH\n", argv[0]);
> > + return EXIT_FAILURE;
> > + }
> > +
> > + ret = swapon(argv[1], 0);
> > + if (ret) {
> > + perror("swapon");
> > + return EXIT_FAILURE;
> > + }
> > +
> > + return EXIT_SUCCESS;
> > +}
> > diff --git a/tests/generic/490 b/tests/generic/490
> > new f
Re: [PATCH v2 3/5] generic: add test for dedupe on an active swapfile
On Wed, May 16, 2018 at 01:38:47PM -0700, Omar Sandoval wrote: > From: Omar Sandoval > > Similar to generic/356 that makes sure we can't reflink an active ^^^ dedupe I'll fix it on commit. Thanks, Eryu -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 2/5] generic: enable swapfile tests on Btrfs
On Wed, May 16, 2018 at 01:38:46PM -0700, Omar Sandoval wrote:
> From: Omar Sandoval
>
> Commit 8c96cfbfe530 ("generic/35[67]: disable swapfile tests on Btrfs")
> disabled the swapfile tests on Btrfs because it did not support
> swapfiles at the time. Now that we're adding support, we want these
So currently btrfs has no swapfile support yet, and tests still _notrun
with current v4.17-rc5 kernel? Just want to make sure that's expected.
> tests to run, but they don't. _require_scratch_swapfile always fails for
> Btrfs because swapfiles on Btrfs must be set to nocow. After fixing
> that, generic/356 and generic/357 fail for the same reason. After fixing
> _that_, both tests still fail because we don't allow reflinking a
> non-checksummed extent (which nocow implies) to a checksummed extent.
> Add a helper for formatting a swap file which does the chattr, and
> chattr the second file, which gets these tests running on kernels
> supporting Btrfs swapfiles.
>
> Signed-off-by: Omar Sandoval
> ---
> common/rc | 15 ---
> tests/generic/356 | 7 ---
> tests/generic/357 | 6 +++---
> 3 files changed, 19 insertions(+), 9 deletions(-)
>
> diff --git a/common/rc b/common/rc
> index ffe53236..814b8b5c 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -,6 +,17 @@ _require_odirect()
> rm -f $testfile 2>&1 > /dev/null
> }
>
> +_format_swapfile() {
> + local fname="$1"
> + local sz="$2"
> +
> + touch "$fname"
Better to remove $fname first, just in case it's an existing file with
some blocks allocated, as chattr(1) says
"Note: For btrfs, the 'C' flag should be set on new or empty files. If
it is set on a file which already has data blocks, it is undefined when
the blocks assigned to the file will be fully stable"
> + chmod 0600 "$fname"
> + $CHATTR_PROG +C "$fname" > /dev/null 2>&1
It'd be good to have some comments on this chattr +C.
> + _pwrite_byte 0x61 0 "$sz" "$fname" >> $seqres.full
> + mkswap -U 27376b42-ff65-42ca-919f-6c9b62292a5c "$fname" >> $seqres.full
Is the label really needed?
Thanks,
Eryu
> +}
> +
> # Check that the filesystem supports swapfiles
> _require_scratch_swapfile()
> {
> @@ -2231,10 +2242,8 @@ _require_scratch_swapfile()
> _scratch_mount
>
> # Minimum size for mkswap is 10 pages
> - local size=$(($(get_page_size) * 10))
> + _format_swapfile "$SCRATCH_MNT/swap" $(($(get_page_size) * 10))
>
> - _pwrite_byte 0x61 0 "$size" "$SCRATCH_MNT/swap" >/dev/null 2>&1
> - mkswap "$SCRATCH_MNT/swap" >/dev/null 2>&1
> if ! swapon "$SCRATCH_MNT/swap" >/dev/null 2>&1; then
> _scratch_unmount
> _notrun "swapfiles are not supported"
> diff --git a/tests/generic/356 b/tests/generic/356
> index 51eeb652..b4a38f84 100755
> --- a/tests/generic/356
> +++ b/tests/generic/356
> @@ -59,11 +59,12 @@ blocks=160
> blksz=65536
>
> echo "Initialize file"
> -echo >> $seqres.full
> -_pwrite_byte 0x61 0 $((blocks * blksz)) $testdir/file1 >> $seqres.full
> -mkswap -U 27376b42-ff65-42ca-919f-6c9b62292a5c $testdir/file1 >> $seqres.full
> +_format_swapfile "$testdir/file1" $((blocks * blksz))
> swapon $testdir/file1
>
> +touch "$testdir/file2"
> +$CHATTR_PROG +C "$testdir/file2" >/dev/null 2>&1
> +
> echo "Try to reflink"
> _cp_reflink $testdir/file1 $testdir/file2 2>&1 | _filter_scratch
>
> diff --git a/tests/generic/357 b/tests/generic/357
> index 0dd0c10f..9a83a283 100755
> --- a/tests/generic/357
> +++ b/tests/generic/357
> @@ -59,9 +59,9 @@ blocks=160
> blksz=65536
>
> echo "Initialize file"
> -echo >> $seqres.full
> -_pwrite_byte 0x61 0 $((blocks * blksz)) $testdir/file1 >> $seqres.full
> -mkswap -U 27376b42-ff65-42ca-919f-6c9b62292a5c $testdir/file1 >> $seqres.full
> +_format_swapfile "$testdir/file1" $((blocks * blksz))
> +touch "$testdir/file2"
> +$CHATTR_PROG +C "$testdir/file2" >/dev/null 2>&1
> _cp_reflink $testdir/file1 $testdir/file2 2>&1 | _filter_scratch
>
> echo "Try to swapon"
> --
> 2.17.0
>
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs: test ENOSPC caused by many orphan items
On Tue, May 15, 2018 at 07:14:02PM -0700, Omar Sandoval wrote:
> On Wed, May 16, 2018 at 09:48:58AM +0800, Eryu Guan wrote:
> > On Wed, May 09, 2018 at 11:21:55PM -0700, Omar Sandoval wrote:
> > > From: Omar Sandoval
> > >
> > > Btrfs has a bug where we can prematurely ENOSPC if we have lots of
> > > orphaned files, i.e., deleted files which are still open. Add a test
> > > which repeatedly creates and deletes a file while keeping all of the
> > > file descriptors open. This should succeed but doesn't on Btrfs without
> > > the fix.
> > >
> > > Signed-off-by: Omar Sandoval
> > > ---
> > > tests/generic/479 | 0
> > > tests/generic/487 | 65 +++
> > > tests/generic/487.out | 2 ++
> > > tests/generic/group | 1 +
> > > 4 files changed, 68 insertions(+)
> > > mode change 100644 => 100755 tests/generic/479
> > > create mode 100755 tests/generic/487
> > > create mode 100644 tests/generic/487.out
> > >
> > > diff --git a/tests/generic/479 b/tests/generic/479
> > > old mode 100644
> > > new mode 100755
> > > diff --git a/tests/generic/487 b/tests/generic/487
> > > new file mode 100755
> > > index ..66379cf0
> > > --- /dev/null
> > > +++ b/tests/generic/487
> > > @@ -0,0 +1,65 @@
> > > +#! /bin/bash
> > > +# FS QA Test 487
> > > +#
> > > +# Test having many file descriptors referring to deleted files open.
> > > Regression
> > > +# test for patch "Btrfs: fix ENOSPC caused by orphan items reservations".
> > > +#
> > > +#---
> > > +# Copyright (c) 2018 Omar Sandoval. All Rights Reserved.
> > > +#
> > > +# This program is free software; you can redistribute it and/or
> > > +# modify it under the terms of the GNU General Public License as
> > > +# published by the Free Software Foundation.
> > > +#
> > > +# This program is distributed in the hope that it would be useful,
> > > +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> > > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > > +# GNU General Public License for more details.
> > > +#
> > > +# You should have received a copy of the GNU General Public License
> > > +# along with this program; if not, write the Free Software Foundation,
> > > +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> > > +#---
> > > +#
> > > +
> > > +seq=`basename $0`
> > > +seqres=$RESULT_DIR/$seq
> > > +echo "QA output created by $seq"
> > > +
> > > +here=`pwd`
> > > +tmp=/tmp/$$
> > > +status=1 # failure is the default!
> > > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > > +
> > > +_cleanup()
> > > +{
> > > + cd /
> > > + rm -f $tmp.*
> > > +}
> > > +
> > > +. ./common/rc
> > > +. ./common/filter
> > > +
> > > +rm -f $seqres.full
> > > +
> > > +_supported_fs generic
> > > +_supported_os Linux
> > > +_require_scratch
> > > +
> > > +_scratch_mkfs_sized $((1024 * 1024 * 1024)) >> $seqres.full 2>&1
> > > +_scratch_mount
> > > +
> > > +test_file="$SCRATCH_MNT/$seq"
> > > +
> > > +(
> > > +ulimit -n $((16 * 1024))
> > > +# ~1 files on a 1 GB filesystem should be no problem.
> > > +for ((i = 1000; i < 1; i++)); do
> > > + eval "exec $i<> \"$test_file\"" && rm "$test_file" || break
> > > +done
> >
> > There's a helper command in src that does exactly this job, e.g.
> >
> > $here/src/multi_open_unlink -f $SCRATCH_MNT/$seq -n 1 -s 0
> >
> > which creates & unlinks 1 files and keeps them open for 0 second in
> > $SCRATCH_MNT using "$seq" as name prefix. This reduces the test run time
> > from 13s to 1s for me.
> >
> > It's a straightforward change, I'll just update on commit, please let me
> > know if you have different thoughts.
> >
> > Thanks,
> > Eryu
>
> Great, as long as it still reproduces the bug without the fix applied,
> that's perfect. Thanks!
Yes, I've verified that it still reproduces the bug on btrfs and test
passes with the fix(es) applied.
Thanks,
Eryu
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs: test ENOSPC caused by many orphan items
On Wed, May 09, 2018 at 11:21:55PM -0700, Omar Sandoval wrote:
> From: Omar Sandoval
>
> Btrfs has a bug where we can prematurely ENOSPC if we have lots of
> orphaned files, i.e., deleted files which are still open. Add a test
> which repeatedly creates and deletes a file while keeping all of the
> file descriptors open. This should succeed but doesn't on Btrfs without
> the fix.
>
> Signed-off-by: Omar Sandoval
> ---
> tests/generic/479 | 0
> tests/generic/487 | 65 +++
> tests/generic/487.out | 2 ++
> tests/generic/group | 1 +
> 4 files changed, 68 insertions(+)
> mode change 100644 => 100755 tests/generic/479
> create mode 100755 tests/generic/487
> create mode 100644 tests/generic/487.out
>
> diff --git a/tests/generic/479 b/tests/generic/479
> old mode 100644
> new mode 100755
> diff --git a/tests/generic/487 b/tests/generic/487
> new file mode 100755
> index ..66379cf0
> --- /dev/null
> +++ b/tests/generic/487
> @@ -0,0 +1,65 @@
> +#! /bin/bash
> +# FS QA Test 487
> +#
> +# Test having many file descriptors referring to deleted files open.
> Regression
> +# test for patch "Btrfs: fix ENOSPC caused by orphan items reservations".
> +#
> +#---
> +# Copyright (c) 2018 Omar Sandoval. All Rights Reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> +#---
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +. ./common/rc
> +. ./common/filter
> +
> +rm -f $seqres.full
> +
> +_supported_fs generic
> +_supported_os Linux
> +_require_scratch
> +
> +_scratch_mkfs_sized $((1024 * 1024 * 1024)) >> $seqres.full 2>&1
> +_scratch_mount
> +
> +test_file="$SCRATCH_MNT/$seq"
> +
> +(
> +ulimit -n $((16 * 1024))
> +# ~1 files on a 1 GB filesystem should be no problem.
> +for ((i = 1000; i < 1; i++)); do
> + eval "exec $i<> \"$test_file\"" && rm "$test_file" || break
> +done
There's a helper command in src that does exactly this job, e.g.
$here/src/multi_open_unlink -f $SCRATCH_MNT/$seq -n 1 -s 0
which creates & unlinks 1 files and keeps them open for 0 second in
$SCRATCH_MNT using "$seq" as name prefix. This reduces the test run time
from 13s to 1s for me.
It's a straightforward change, I'll just update on commit, please let me
know if you have different thoughts.
Thanks,
Eryu
> +)
> +
> +echo "Silence is golden"
> +
> +status=0
> +exit
> diff --git a/tests/generic/487.out b/tests/generic/487.out
> new file mode 100644
> index ..5f31fd97
> --- /dev/null
> +++ b/tests/generic/487.out
> @@ -0,0 +1,2 @@
> +QA output created by 487
> +Silence is golden
> diff --git a/tests/generic/group b/tests/generic/group
> index 505383f7..93581257 100644
> --- a/tests/generic/group
> +++ b/tests/generic/group
> @@ -489,3 +489,4 @@
> 484 auto quick
> 485 auto quick insert
> 486 auto quick attr
> +487 auto quick
> --
> 2.17.0
>
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] test online label ioctl
On Mon, Apr 30, 2018 at 04:43:18PM -0500, Eric Sandeen wrote:
> This tests the online label ioctl that btrfs has, which has been
> recently proposed for XFS.
>
> To run, it requires an updated xfs_io with the label command and a
> filesystem that supports it
>
> A slight change here to _require_xfs_io_command as well, so that tests
> which simply fail with "Inappropriate ioctl" can be caught in the
> common case.
>
> Signed-off-by: Eric Sandeen
> ---
>
> this passes on btrfs, _notruns on xfs/ext4 of yore, and passes
> on xfs w/ my online label patchset (as long as xfs_io has the new
> capability)
>
> diff --git a/common/rc b/common/rc
> index 9ffab7f..c53a721 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -2158,6 +2158,9 @@ _require_xfs_io_command()
> echo $testio | grep -q "Inappropriate ioctl" && \
> _notrun "xfs_io $command support is missing"
> ;;
> + "label")
> + testio=`$XFS_IO_PROG -c "label" $TEST_DIR 2>&1`
> + ;;
> "open")
> # -c "open $f" is broken in xfs_io <= 4.8. Along with the fix,
> # a new -C flag was introduced to execute one shot commands.
> @@ -2196,7 +2199,7 @@ _require_xfs_io_command()
> rm -f $testfile 2>&1 > /dev/null
> echo $testio | grep -q "not found" && \
> _notrun "xfs_io $command support is missing"
> - echo $testio | grep -q "Operation not supported" && \
> + echo $testio | grep -q "Operation not supported\|Inappropriate ioctl"
> && \
> _notrun "xfs_io $command failed (old kernel/wrong fs?)"
> echo $testio | grep -q "Invalid" && \
> _notrun "xfs_io $command failed (old kernel/wrong fs/bad args?)"
> diff --git a/tests/generic/485 b/tests/generic/485
> new file mode 100755
> index 000..79902c2
> --- /dev/null
> +++ b/tests/generic/485
> @@ -0,0 +1,99 @@
> +#! /bin/bash
> +# FS QA Test 485
> +#
> +# Test the online filesystem label set/get ioctls
> +#
> +#---
> +# Copyright (c) 2018 Red Hat, Inc. All Rights Reserved.
> +# Author: Eric Sandeen
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> +#---
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +_supported_fs generic
> +_supported_os Linux
> +_require_scratch
> +_require_xfs_io_command "label"
> +
> +_scratch_mkfs > $seqres.full 2>&1
> +_scratch_mount
> +
> +# Make sure we can set & clear the label
> +$XFS_IO_PROG -c "label label.$seq" $SCRATCH_MNT
> +$XFS_IO_PROG -c "label" $SCRATCH_MNT
> +
> +# And that userspace can see it now, while mounted
> +blkid -s LABEL $SCRATCH_DEV | _filter_scratch
> +
> +# And that the it is still there when it's unmounted
> +_scratch_unmount
> +blkid -s LABEL $SCRATCH_DEV | _filter_scratch
> +
> +# And that it persists after a remount
> +_scratch_mount
> +$XFS_IO_PROG -c "label" $SCRATCH_MNT
> +
> +# And that a too-long label is rejected, beyond the interface max:
> +LABEL=$(perl -e "print 'l' x 257;")
> +$XFS_IO_PROG -c "label $LABEL" $SCRATCH_MNT
> +
> +# And that it succeeds right at the filesystem max:
> +case $FSTYP in
> +xfs)
> + MAXLEN=12;
> + ;;
> +btrfs)
> + MAXLEN=256
Seems this should be 255, otherwise I got failure like:
-label = "MAXLABEL"
+label: Invalid argument
and MAXLEN=255 makes the test pass with btrfs.
> + ;;
> +*)
> + MAXLEN=256
> + echo "Your filesystem supports online label, please add max length"
Perhaps we can introduce a new helper similar to _require_acl_get_max()
and _notrun the test if current $FSTYP doesn't define a maxlen on
filesystem label?
> + ;;
> +esac
> +LABEL=$(perl -e "print 'o' x $MAXLEN;")
> +$XFS_IO_PROG -c "label $LABEL" $SCRATCH_MNT | sed -e 's/o\+/MAXLABEL/'
> +
> +# And that it fails just past the filesystem max:
> +let TOOLONG=MAXLEN+1
> +LABEL=$(perl -e "print 'o' x
Re: [PATCH v2] btrfs: Add test that checks rmdir(2) can delete a subvolume
On Fri, Apr 27, 2018 at 06:26:35PM +0200, David Sterba wrote: > On Fri, Apr 27, 2018 at 05:02:45PM +0900, Misono Tomohiro wrote: > > Add btrfs test that checks "rmdir" or "rm -r" command can delete a > > subvolume like an ordinary drectory. > > > > This behavior has been restricted long time but becomes allowed by > > following patch in the kernel: > > btrfs: Allow rmdir(2) to delete an empty subvolume > > AFAICS this test will fail on older kernels, eg. stable versions, but > this is not a bug that needs to be fixed by backporting patches. The > usual way is to use the expunges, but as this is a feature coverage, I > think the test should detect if the kernel has the feature at all. > Similar to the fallocate modes etc. Agreed, if the change is treated as a behavior change not a bug on old kernels, we probably need a way to detect the case and _notrun if kernel doesn't support it. > > For the test itself, the scenarios look sufficient, so > Reviewed-by: David Sterba Thanks for the review! Eryu -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] btrfs: Add test that checks rmdir(2) can delete a subvolume
On Fri, Apr 27, 2018 at 05:02:45PM +0900, Misono Tomohiro wrote:
> Add btrfs test that checks "rmdir" or "rm -r" command can delete a
> subvolume like an ordinary drectory.
>
> This behavior has been restricted long time but becomes allowed by
> following patch in the kernel:
> btrfs: Allow rmdir(2) to delete an empty subvolume
>
> Signed-off-by: Tomohiro Misono
Looks good to me, thanks! But still, I'd like to see a Reviewed-by tag
from btrfs developers.
Thanks,
Eryu
> ---
> v1 -> v2
> - rebase to current master
> - remove underscore from local function
> - remove source common/btrfs
> - remove unnecessary _filter_scratch and echo messages when fail
>
> As I will take a whole week vacation next week, my replay will be late.
> Thanks,
> Tomohiro Misono
>
>
> tests/btrfs/160 | 141
>
> tests/btrfs/160.out | 2 +
> tests/btrfs/group | 1 +
> 3 files changed, 144 insertions(+)
> create mode 100755 tests/btrfs/160
> create mode 100644 tests/btrfs/160.out
>
> diff --git a/tests/btrfs/160 b/tests/btrfs/160
> new file mode 100755
> index ..86d2dca7
> --- /dev/null
> +++ b/tests/btrfs/160
> @@ -0,0 +1,141 @@
> +#! /bin/bash
> +# FS QA Test btrfs/159
> +#
> +# QA test that checks rmdir(2) works for subvolumes like ordinary
> directories.
> +#
> +# This behavior has been restricted long time but becomes allowed by
> following
> +# patch in the kernel:
> +# btrfs: Allow rmdir(2) to delete an empty subvolume
> +#
> +#---
> +# Copyright (c) 2018 Fujitsu. All Rights Reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> +#---
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +create_subvol()
> +{
> + $BTRFS_UTIL_PROG subvolume create $1 >> $seqres.full 2>&1
> +}
> +
> +create_snapshot()
> +{
> + $BTRFS_UTIL_PROG subvolume snapshot $@ >> $seqres.full 2>&1
> +}
> +
> +rmdir_subvol()
> +{
> + rmdir $1 >> $seqres.full 2>&1
> +}
> +
> +rm_r_subvol() {
> + rm -r $1 >> $seqres.full 2>&1
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch
> +
> +_scratch_mkfs > /dev/null 2>&1 || _fail "mkfs failed"
> +_scratch_mount
> +
> +# Check that an empty subvolume can be deleted by rmdir
> +create_subvol $SCRATCH_MNT/sub1
> +rmdir_subvol $SCRATCH_MNT/sub1 || \
> + echo "rmdir should delete an empty subvolume"
> +
> +# Check that non-empty subvolume cannot be deleted by rmdir
> +create_subvol $SCRATCH_MNT/sub2
> +touch $SCRATCH_MNT/sub2/file
> +rmdir_subvol $SCRATCH_MNT/sub2 && \
> + echo "rmdir should fail for non-empty subvolume"
> +rm $SCRATCH_MNT/sub2/file
> +rmdir_subvol $SCRATCH_MNT/sub2 || \
> + echo "rmdir should delete an empty subvolume"
> +
> +# Check that read-only empty subvolume can be deleted by rmdir
> +create_subvol $SCRATCH_MNT/sub3
> +create_snapshot -r $SCRATCH_MNT/sub3 $SCRATCH_MNT/snap
> +$BTRFS_UTIL_PROG property set $SCRATCH_MNT/sub3 ro true >> $seqres.full 2>&1
> +rmdir_subvol $SCRATCH_MNT/sub3 || \
> + echo "rmdir should delete an empty subvolume"
> +rmdir_subvol $SCRATCH_MNT/snap || \
> + echo "rmdir should delete a readonly empty subvolume"
> +
> +# Check that the default subvolume cannot be deleted by rmdir
> +create_subvol $SCRATCH_MNT/sub4
> +subvolid=$(_btrfs_get_subvolid $SCRATCH_MNT sub4)
> +$BTRFS_UTIL_PROG subvolume set-default $subvolid $SCRATCH_MNT \
> + >> $seqres.full 2>&1
> +rmdir_subvol $SCRATCH_MNT/sub4 && \
> + echo "rmdir should fail for the default subvolume"
> +
> +# Check that subvolume stub (created by snapshot) can be deleted by rmdir
> +# (Note: this has been always allowed)
> +create_subvol $SCRATCH_MNT/sub5
> +create_subvol $SCRATCH_MNT/sub5/sub6
> +create_snapshot $SCRATCH_MNT/sub5 $SCRATCH_MNT/snap2
> +rmdir $SCRATCH_MNT/snap2/su
Re: [PATCH] btrfs: Add test that checks rmdir(2) can delete a subvolume
On Thu, Apr 19, 2018 at 04:23:35PM +0900, Misono Tomohiro wrote:
> Add btrfs test that checks "rmdir" or "rm -r" command can delete a
> subvolume like an ordinary drectory.
>
> This behavior has been restricted long time but becomes allowed by
> following patch in the kernel:
> btrfs: Allow rmdir(2) to delete an empty subvolume
>
> Signed-off-by: Tomohiro Misono
Looks fine to me overall from fstests' perspective of view, some minor
issues inline.
But I'd like a Reviewed-by tag from btrfs developers too.
> ---
> tests/btrfs/159 | 140
>
> tests/btrfs/159.out | 2 +
> tests/btrfs/group | 1 +
> 3 files changed, 143 insertions(+)
> create mode 100755 tests/btrfs/159
> create mode 100644 tests/btrfs/159.out
>
> diff --git a/tests/btrfs/159 b/tests/btrfs/159
> new file mode 100755
> index ..2d830502
> --- /dev/null
> +++ b/tests/btrfs/159
> @@ -0,0 +1,140 @@
> +#! /bin/bash
> +# FS QA Test btrfs/159
> +#
> +# QA test that checks rmdir(2) works for subvolumes like ordinary
> directories.
> +#
> +# This behavior has been restricted long time but becomes allowed by
> following
> +# patch in the kernel:
> +# btrfs: Allow rmdir(2) to delete an empty subvolume
> +#
> +#---
> +# Copyright (c) 2018 Fujitsu. All Rights Reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> +#---
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +_create_subvol()
There's no need to name these local helper functions with the leading
underscore.
> +{
> + $BTRFS_UTIL_PROG subvolume create $1 2>&1 | \
> + _filter_scratch >> $seqres.full
No need to do _filter_scratch if the output won't go to stdout/stderr,
it's fine or even recommended to not filter the output when dumping them
to $seqres.full for debug purpose.
There're a few other places that _filter_scratch can be dropped.
> +}
> +
> +_create_snapshot()
> +{
> + $BTRFS_UTIL_PROG subvolume snapshot $@ 2>&1 | \
> + _filter_scratch >> $seqres.full
> +}
> +
> +_rmdir_subvol()
> +{
> + rmdir $1 2>&1 | _filter_scratch
I notice that we don't expect any output from the test script ("Silence
is golden" in .out file), so I don't think it's necessary to do
redirection and filter here, just do "rmdir $1" and any error message
would break golden image and fail the test.
> + return ${PIPESTATUS[0]}
If you only care about the return value, just dump the rmdir output to
$seqres.full or /dev/null.
> +}
> +
> +_rm_r_subvol() {
> + rm -r $1 2>&1 | _filter_scratch
> + return ${PIPESTATUS[0]}
Same here.
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/btrfs
No need to source common/btrfs, it's already sourced by common/rc based
on current $FSTYP.
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch
> +
> +_scratch_mkfs > /dev/null 2>&1 || _fail "mkfs failed"
> +_scratch_mount
> +
> +# Check that an empty subvolume can be deleted by rmdir
> +_create_subvol $SCRATCH_MNT/sub1
> +_rmdir_subvol $SCRATCH_MNT/sub1
> +
> +# Check that non-empty subvolume cannot be deleted by rmdir
> +_create_subvol $SCRATCH_MNT/sub2
> +touch $SCRATCH_MNT/sub2/file
> +_rmdir_subvol $SCRATCH_MNT/sub2 >> $seqres.full && \
> + _fail "rmdir should fail for non-empty subvolume"
Just echo the error message, instead of exiting the test, test could
continue. Or even simpler, don't rely on the return status of
_rmdir_subvol, just put any expected output from _rmdir_subvol to .out
file, e.g.
_rmdir_subvol $SCRATCH_MNT/sub2 | _filter_scratch
So that, again, any failure message could break the golden image and
fail the test.
> +rm $SCRATCH_MNT/sub2/file
> +_rmdir_subvol $SCRATCH_MNT/sub2
> +
> +# Check that read-only empty subvolume can be deleted by rmdir
> +_create_subvol $SCRATCH_MNT/sub3
> +_create_snapshot
Re: [PATCH] btrfs: add verify chattr support for send/receive test
[adding linux-btrfs list to cc]
On Tue, Apr 17, 2018 at 04:44:42PM -0700, Howard McLauchlan wrote:
> This test aims to verify correct behaviour with chattr operations and
> btrfs send/receive. The intent is to check general correctness as well
> as special interactions with troublesome flags(immutable, append only).
>
> This test is motivated by a bug in btrfs which demonstrates a lack of
> chattr support in btrfs send/receive.
>From the discussion of the kernel patch thread, it looks more like a
"new feature" or a behavior change than a bug. If it's a regression test
for a pure bug fix, it's fine to fail the test on old
kernels/btrfs-progs. But if it's a "new feature" or a behavior change,
it's better to add new _require rules to make sure the kernel and/or
userspace does have chattr support in send/receive.
I'll comment from fstests' point of view and defer above question to
btrfs developers :)
>
> A kernel patch to fix this can be found at:
>
> btrfs: add chattr support for send/receive
>
> The accompanying userspace patch can be found at:
>
> btrfs-progs: add chattr support for send/receive
>
> Signed-off-by: Howard McLauchlan
> ---
> tests/btrfs/159 | 202
> tests/btrfs/159.out | 38 +
> tests/btrfs/group | 1 +
> 3 files changed, 241 insertions(+)
> create mode 100755 tests/btrfs/159
> create mode 100644 tests/btrfs/159.out
>
> diff --git a/tests/btrfs/159 b/tests/btrfs/159
> new file mode 100755
> index ..b3a32594
> --- /dev/null
> +++ b/tests/btrfs/159
> @@ -0,0 +1,202 @@
> +#! /bin/bash
> +# FS QA Test 159
> +#
> +# This test verifies the correct behaviour of chattr support for btrfs
> +# send/receive; 6 cases will be tested:
> +# 1. New inode created with an inode flag set
> +# 2. Existing inode with BTRFS_INODE_APPEND is written to with sequence:
> +#chattr -a
> +#pwrite something
> +#chattr +a
> +# 3. Existing inode with BTRFS_INODE_APPEND is written to with sequence:
> +#chattr -a
> +#pwrite something
> +#chattr +d
> +# 4. Existing inode is written to with sequence:
> +#setfattr something
> +#chattr +a
> +# 5. Existing inode with BTRFS_INODE_APPEND is written to with sequence:
> +#chattr -a
> +#setfattr something
> +#setfattr something else
> +#chattr +a
> +# 6. As above, but with pwrite instead of setfattr
> +# The goal of 5 and 6 is not to test correctness, but to ensure we
> +# don't send extra chattrs that are unnecessary.
> +#
> +# We verify the md5sum of the snapshots in the receive directory to ensure
> file
> +# contents have changed appropriately. We also observe the flags changing (or
> +# not changing) as appropriate.
> +#
> +#---
> +# Copyright (c) 2018 Facebook. All Rights Reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> +#---
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch
> +
> +send_files_dir=$TEST_DIR/btrfs-test-$seq
Take use of $TEST_DIR, so we need _require_test too.
And we may need _require_attrs and _require_test_lsattr too.
> +
> +rm -rf $send_files_dir
> +mkdir $send_files_dir
> +
> +_scratch_mkfs >>$seqres.full 2>&1
> +_scratch_mount
> +
> +# Create receive directory
> +mkdir $SCRATCH_MNT/receive
> +
> +# Create test file and set chattr flag
> +_run_btrfs_util_prog subvolume create $SCRATCH_MNT/parent
> +$XFS_IO_PROG -f -c "pwrite -S 0xaa 0K 32K" $SCRATCH_MNT/parent/foo |
> _filter_xfs_io
> +$CHATTR_PROG +a $SCRATCH_MNT/parent/foo
> +
> +# Send/Receive initial snapshot
> +_run_btrfs_util_prog subvolume snapshot -r $SCRATCH_MNT/parent \
> + $SCRATCH_MNT/old_parent
Re: [PATCH v2] fstests: test btrfs fsync after hole punching with no-holes mode
On Mon, Apr 16, 2018 at 12:28:59PM +0100, Filipe Manana wrote: > On Mon, Apr 9, 2018 at 2:05 PM, Eryu Guan wrote: > > On Sun, Apr 08, 2018 at 09:46:24AM +0100, Filipe Manana wrote: > >> On Sun, Apr 8, 2018 at 8:46 AM, Eryu Guan wrote: > >> > On Wed, Mar 28, 2018 at 12:55:30PM +0100, fdman...@kernel.org wrote: > >> >> From: Filipe Manana > >> >> > >> >> Test that when we have the no-holes mode enabled and a specific metadata > >> >> layout, if we punch a hole and fsync the file, at replay time the whole > >> >> hole was preserved. > >> >> > >> >> This issue is fixed by the following btrfs patch for the linux kernel: > >> >> > >> >> "Btrfs: fix fsync after hole punching when using no-holes feature" > >> >> > >> >> Signed-off-by: Filipe Manana > >> >> --- > >> >> > >> >> V2: Made the test work when selinux is enabled, and made it use direct > >> >> IO > >> >> writes to ensure 256K extents. > >> > > >> > Test fails with selinux enabled now on unpatched kernel. But I found > >> > that, in my release testing, test still fails when testing with current > >> > Linus tree (HEAD is 642e7fd23353, without selinux this time), which > >> > should contain the mentioned fix. Does that mean the bug is not fully > >> > fixed? > >> > >> The bug is fully fixed. But HEAD 642e7fd23353 does not contain the > >> fix. The current linus' master has it. > > > > I'll double check.. Thanks for the heads-up! > > Hi Eryu, any reason this didn't go in the last update? > Thanks. Sorry, I thought I queued it for last update, but actually I queued "generic: test for fsync after fallocate" not this one (and I double checked that test not this one..). I'll give it another try and queue it for next update if all look well. Thanks, Eryu -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3] fstests: btrfs/159 superblock corruption test case
On Sat, Apr 14, 2018 at 06:43:49AM +0800, Anand Jain wrote: > > > > +# Test if the superblock corruption is handled correctly: > > > +#- Test fsid miss-match (csum ok) between primary and copy > > > superblock > > > +#Fixed by the ML patch: > > > +#btrfs: check if the fsid in the primary sb and copy sb are same > > > +#- Test if the mount fails if the primary superblock csum is > > > +#corrupted on any disk > > > +#- Test if the mount does not fail if the copy1 sb csum is > > > corrupted > > > +#Fixed by the ML patches: > > > +#btrfs: verify superblock checksum during scan > > > +#btrfs: verify checksum for all devices in mount context > > > > Do you have a tree that I can pull from? I want to make sure the test > > does pass on patched kernel, but the patchset doesn't apply on v4.16 > > kernel. > > We have new discussions on whether to check for the alien-superblock and > the superblock-checksum at the mount and scan time respectively. And > depending on its outcome this test-case should be modified as well. So > can you please defer this fstest patch, for now, I shall send a revised > fstest patch when kernel patches gets integrated. Thanks for the heads-up, I'll drop it for now. > > In any case, if you want to give a try, those patches are base on kdave repo > at [1]. > [1] > https://github.com/kdave/btrfs-devel.git misc-next Thanks! Eryu -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3] fstests: btrfs/159 superblock corruption test case
On Mon, Apr 09, 2018 at 01:28:30PM +0800, Anand Jain wrote:
> Verify if the superblock corruption is handled correctly.
>
> Signed-off-by: Anand Jain
> ---
> v2->v3:
> Provide the disk to be corrupted as an arg, instead of swapping the devices,
> so drop mount_opt_minus_args().
> 159.out slightly changed.
> v1->v2:
> $subject slightly changed
> Added more info about the test-case
> Keep the stuff from the ./new btrfs
> Add mount_opt_minus_args() to get the options (if) set at the config file
> Move DEV_GOOD & DEV_BAD to where it starts to use
> To help debugging added run_check where possible
> Remove {} in the out file
> Use _filter_error_mount for mount fail cases other than -EINVAL
>
> tests/btrfs/159 | 149
>
> tests/btrfs/159.out | 21
> tests/btrfs/group | 1 +
> 3 files changed, 171 insertions(+)
> create mode 100755 tests/btrfs/159
> create mode 100644 tests/btrfs/159.out
>
> diff --git a/tests/btrfs/159 b/tests/btrfs/159
> new file mode 100755
> index ..c3a50b58b0b9
> --- /dev/null
> +++ b/tests/btrfs/159
> @@ -0,0 +1,149 @@
> +#! /bin/bash
> +# FS QA Test 159
> +#
> +# Test if the superblock corruption is handled correctly:
> +#- Test fsid miss-match (csum ok) between primary and copy superblock
> +#Fixed by the ML patch:
> +#btrfs: check if the fsid in the primary sb and copy sb are same
> +#- Test if the mount fails if the primary superblock csum is
> +#corrupted on any disk
> +#- Test if the mount does not fail if the copy1 sb csum is corrupted
> +#Fixed by the ML patches:
> +#btrfs: verify superblock checksum during scan
> +#btrfs: verify checksum for all devices in mount context
Do you have a tree that I can pull from? I want to make sure the test
does pass on patched kernel, but the patchset doesn't apply on v4.16
kernel.
Thanks,
Eryu
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] fstests: test btrfs fsync after hole punching with no-holes mode
On Sun, Apr 08, 2018 at 09:46:24AM +0100, Filipe Manana wrote: > On Sun, Apr 8, 2018 at 8:46 AM, Eryu Guan wrote: > > On Wed, Mar 28, 2018 at 12:55:30PM +0100, fdman...@kernel.org wrote: > >> From: Filipe Manana > >> > >> Test that when we have the no-holes mode enabled and a specific metadata > >> layout, if we punch a hole and fsync the file, at replay time the whole > >> hole was preserved. > >> > >> This issue is fixed by the following btrfs patch for the linux kernel: > >> > >> "Btrfs: fix fsync after hole punching when using no-holes feature" > >> > >> Signed-off-by: Filipe Manana > >> --- > >> > >> V2: Made the test work when selinux is enabled, and made it use direct IO > >> writes to ensure 256K extents. > > > > Test fails with selinux enabled now on unpatched kernel. But I found > > that, in my release testing, test still fails when testing with current > > Linus tree (HEAD is 642e7fd23353, without selinux this time), which > > should contain the mentioned fix. Does that mean the bug is not fully > > fixed? > > The bug is fully fixed. But HEAD 642e7fd23353 does not contain the > fix. The current linus' master has it. I'll double check.. Thanks for the heads-up! Eryu -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] fstests: test btrfs fsync after hole punching with no-holes mode
On Wed, Mar 28, 2018 at 12:55:30PM +0100, fdman...@kernel.org wrote:
> From: Filipe Manana
>
> Test that when we have the no-holes mode enabled and a specific metadata
> layout, if we punch a hole and fsync the file, at replay time the whole
> hole was preserved.
>
> This issue is fixed by the following btrfs patch for the linux kernel:
>
> "Btrfs: fix fsync after hole punching when using no-holes feature"
>
> Signed-off-by: Filipe Manana
> ---
>
> V2: Made the test work when selinux is enabled, and made it use direct IO
> writes to ensure 256K extents.
Test fails with selinux enabled now on unpatched kernel. But I found
that, in my release testing, test still fails when testing with current
Linus tree (HEAD is 642e7fd23353, without selinux this time), which
should contain the mentioned fix. Does that mean the bug is not fully
fixed?
--- tests/btrfs/159.out 2018-04-03 18:25:41.566105514 -0700
+++ /root/xfstests/results//btrfs/159.out.bad 2018-04-08 00:28:26.921968949
-0700
@@ -6,4 +6,4 @@
File digest before power failure:
c5c0a13588a639529c979c57c336f441 SCRATCH_MNT/foobar
File digest after power failure and log replay:
-c5c0a13588a639529c979c57c336f441 SCRATCH_MNT/foobar
+c84746bbd1b97420f076d417a6a7d81c SCRATCH_MNT/foobar
I'll drop this patch for now.
Thanks,
Eryu
>
> tests/btrfs/159 | 115
>
> tests/btrfs/159.out | 9
> tests/btrfs/group | 1 +
> 3 files changed, 125 insertions(+)
> create mode 100755 tests/btrfs/159
> create mode 100644 tests/btrfs/159.out
>
> diff --git a/tests/btrfs/159 b/tests/btrfs/159
> new file mode 100755
> index ..eb667692
> --- /dev/null
> +++ b/tests/btrfs/159
> @@ -0,0 +1,115 @@
> +#! /bin/bash
> +# FSQA Test No. 159
> +#
> +# Test that when we have the no-holes mode enabled and a specific metadata
> +# layout, if we punch a hole and fsync the file, at replay time the whole
> +# hole was preserved.
> +#
> +#---
> +#
> +# Copyright (C) 2018 SUSE Linux Products GmbH. All Rights Reserved.
> +# Author: Filipe Manana
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> +#---
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + _cleanup_flakey
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/dmflakey
> +
> +# real QA test starts here
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch
> +_require_dm_target flakey
> +_require_xfs_io_command "fpunch"
> +_require_odirect
> +
> +rm -f $seqres.full
> +
> +run_test()
> +{
> + local punch_offset=$1
> +
> + # We create the filesystem with a node size of 64Kb because we need to
> + # create a specific metadata layout in order to trigger the bug we are
> + # testing. At the moment the node size can not be smaller then the
> + # system's page size, so given that the largest possible page size is
> + # 64Kb and by default the node size is set to the system's page size
> + # value, we explicitly create a filesystem with a 64Kb node size.
> + _scratch_mkfs -O no-holes -n $((64 * 1024)) >>$seqres.full 2>&1
> + _require_metadata_journaling $SCRATCH_DEV
> + _init_flakey
> + _mount_flakey
> +
> + # Create our test file with 832 extents of 256Kb each. Before each
> + # extent, there is a 256Kb hole (except for the first extent, which
> + # starts at offset 0). This creates two leafs in the filesystem tree.
> + # We use direct IO to ensure we get exactly 256K extents (with buffered
> + # IO we can get writeback triggered at any time and therefore get
> + # extents smaller than 256K).
> + for ((i = 0; i <= 831; i++)); do
> + local offset=$((i * 2 * 256 * 1024))
> + $XFS_IO_PROG -f -d -c "pwrite -S 0xab -b 256K $offset 256K" \
> + $SCRATCH_MNT/foobar >/dev/null
> + done
> +
> + # Make sure everything done so far is durably persisted.
> + sync
> +
> + # Now punch a hole that covers part of the extent
Re: [PATCH v2] fstests: btrfs/159 superblock corruption test case
On Thu, Apr 05, 2018 at 02:28:49PM +0800, Anand Jain wrote:
> Verify if the superblock corruption is handled correctly.
>
> Signed-off-by: Anand Jain
> ---
> v1->v2:
> $subject slightly changed
> Added more info about the test-case
> Keep the stuff from the ./new btrfs
> Add mount_opt_minus_args() to get the options (if) set at the config file
> Move DEV_GOOD & DEV_BAD to where it starts to use
> To help debugging added run_check where possible
> Remove {} in the out file
> Use _filter_error_mount for mount fail cases other than -EINVAL
>
> tests/btrfs/159 | 177
>
> tests/btrfs/159.out | 23 +++
> tests/btrfs/group | 1 +
> 3 files changed, 201 insertions(+)
> create mode 100755 tests/btrfs/159
> create mode 100644 tests/btrfs/159.out
>
> diff --git a/tests/btrfs/159 b/tests/btrfs/159
> new file mode 100755
> index ..521cfdab0242
> --- /dev/null
> +++ b/tests/btrfs/159
> @@ -0,0 +1,177 @@
> +#! /bin/bash
> +# FS QA Test 159
> +#
> +# Test if the superblock corruption is handled correctly:
> +#- Test fsid miss-match (csum ok) between primary and copy superblock
> +#Fixed by the ML patch:
> +#btrfs: check if the fsid in the primary sb and copy sb are same
> +#- Test if the mount fails if the primary superblock csum is
> +#corrupted on any disk
> +#- Test if the mount does not fail if the copy1 sb csum is corrupted
> +#Fixed by the ML patches:
> +#btrfs: verify superblock checksum during scan
> +#btrfs: verify checksum for all devices in mount context
> +#
> +#-
> +# Copyright (c) 2018 Oracle. All Rights Reserved.
> +# Author: Anand Jain
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> +#---
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> + _scratch_dev_pool_put
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/module
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch_dev_pool 2
> +_require_loadable_fs_module "btrfs"
> +_require_command "$WIPEFS_PROG" wipefs
> +
> +_scratch_dev_pool_get 2
> +
> +mount_opt_minus_args()
> +{
> + local nr
> + local mnt_opt=""
> +
> + nr=`_scratch_mount_options | awk '{print NF}'`
> + if [ $nr -eq 4 ]; then
Seems this only works with "scratch_mount_options" returns something
like:
"-o opt dev mnt" or "-o opt1,opt2 dev mnt"
but not
"-oopt dev mnt" nor
"-o opt1 -o opt2 dev mnt" nor if $SELINUX_MOUNT_OPTIONS not empty.
Also if MOUNT_OPTIONS is "-oopt1 -oopt2", mount_opt_minus_args would
return something like "-o -oopt2,device=", which are not valid
mount options.
> + #gets only the mount option set in the config file
> + mnt_opt=`_scratch_mount_options | awk '{print $2}'`
> + fi
> + #Append the additional opts provide as func args.
> + #Make sure -o is not echo-ed if both config file mnt-option
> + #and the test case mnt-option are null.
> + if [ -z $mnt_opt ]; then
> + if [ ! -z $* ]; then
> + echo "-o $*"
> + fi
> + else
> + if [ -z $* ]; then
> + echo "-o $mnt_opt"
> + else
> + echo "-o $mnt_opt,$*"
> + fi
> + fi
> +}
> +
> +wipe()
> +{
> + $WIPEFS_PROG -a $DEV_GOOD > /dev/null 2>&1
> + $WIPEFS_PROG -a $DEV_BAD > /dev/null 2>&1
> +}
> +
> +# Test for fsid miss-match (csum ok) with primary and copy superblock.
> +check_copy1_fsid()
> +{
> + local bytenr=67108864
> + echo -e "\\ncheck_copy1_fsid\\n" | tee -a $seqres.full
> +
> + wipe
> + $MKFS_BTRFS_PROG -fq $DEV_GOOD
> + $MKFS_BTRFS_PROG -fq $DEV_BAD
> + _reload_fs_module "btrfs"
> +
> + run_check dd status=none of=$DEV_BAD if=$DEV_GOOD ibs=1 obs=1\
> + skip=$bytenr seek=$bytenr count=4K
> +
> +
Re: [PATCH] fstests: generic test for fsync after fallocate
On Thu, Apr 05, 2018 at 10:56:14PM +0100, fdman...@kernel.org wrote:
> From: Filipe Manana
>
> Test that fsync operations preserve extents allocated with fallocate(2)
> that are placed beyond a file's size.
>
> This test is motivated by a bug found in btrfs where unwritten extents
> beyond the inode's i_size were not preserved after a fsync and power
> failure. The btrfs bug is fixed by the following patch for the linux
> kernel:
>
> "Btrfs: fix loss of prealloc extents past i_size after fsync log replay"
>
> Signed-off-by: Filipe Manana
Hmm, xfs fails this test, while ext4 passes.
# diff -u tests/generic/483.out
/root/workspace/xfstests/results//xfs_4k_crc/generic/483.out.bad
--- tests/generic/483.out 2018-04-07 23:35:00.55511 +0800
+++ /root/workspace/xfstests/results//xfs_4k_crc/generic/483.out.bad
2018-04-07 23:39:48.780659707 +0800
@@ -6,5 +6,5 @@
0: [0..511]: data
1: [512..2559]: unwritten
File baz fiemap:
-0: [0..511]: data
-1: [512..6143]: unwritten
+0: [0..895]: data
+1: [896..6143]: unwritten
I'm not sure what the problem is yet, but IMHO controlling on-disk
layout of a file from userspace is hard and should be avoided if
possible.
Why not dumping md5sum to .out file like other dmflakey tests? I've
checked the md5sum of all the three test files, and they're the same on
xfs as on ext4, so the files are not corrupted on xfs.
Thanks,
Eryu
> ---
> tests/generic/482 | 118
> ++
> tests/generic/482.out | 10 +
> tests/generic/group | 1 +
> 3 files changed, 129 insertions(+)
> create mode 100755 tests/generic/482
> create mode 100644 tests/generic/482.out
>
> diff --git a/tests/generic/482 b/tests/generic/482
> new file mode 100755
> index ..43bbc913
> --- /dev/null
> +++ b/tests/generic/482
> @@ -0,0 +1,118 @@
> +#! /bin/bash
> +# FSQA Test No. 482
> +#
> +# Test that fsync operations preserve extents allocated with fallocate(2)
> that
> +# are placed beyond a file's size.
> +#
> +#---
> +#
> +# Copyright (C) 2018 SUSE Linux Products GmbH. All Rights Reserved.
> +# Author: Filipe Manana
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> +#---
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + _cleanup_flakey
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/dmflakey
> +. ./common/punch
> +
> +# real QA test starts here
> +_supported_fs generic
> +_supported_os Linux
> +_require_scratch
> +_require_dm_target flakey
> +_require_xfs_io_command "falloc" "-k"
> +_require_xfs_io_command "fiemap"
> +
> +rm -f $seqres.full
> +
> +_scratch_mkfs >>$seqres.full 2>&1
> +_require_metadata_journaling $SCRATCH_DEV
> +_init_flakey
> +_mount_flakey
> +
> +# Create our test files.
> +$XFS_IO_PROG -f -c "pwrite -S 0xea 0 256K" $SCRATCH_MNT/foo >/dev/null
> +
> +# Create a file with many extents. We later want to shrink truncate it and
> +# add a prealloc extent beyond its new size.
> +for ((i = 1; i <= 500; i++)); do
> + offset=$(((i - 1) * 4 * 1024))
> + $XFS_IO_PROG -f -s -c "pwrite -S 0xcf $offset 4K" \
> + $SCRATCH_MNT/bar >/dev/null
> +done
> +
> +# A file which already has a prealloc extent beyond its size.
> +# The fsync done on it is motivated by differences in the btrfs
> implementation
> +# of fsync (first fsync has different logic from subsequent fsyncs).
> +$XFS_IO_PROG -f -c "pwrite -S 0xf1 0 256K" \
> + -c "falloc -k 256K 768K" \
> + -c "fsync" \
> + $SCRATCH_MNT/baz >/dev/null
> +
> +# Make sure everything done so far is durably persisted.
> +sync
> +
> +# Allocate an extent beyond the size of the first test file and fsync it.
> +$XFS_IO_PROG -c "falloc -k 256K 1M"\
> + -c "fsync" \
> + $SCRATCH_MNT/foo
> +
> +# Do a shrinking truncate of our test file, add a prealloc extent to it after
> +# its new size and fsync it.
> +$XFS_IO_PROG -c "truncate 256K" \
> + -c "falloc -k 256K 1M"\
> + -c "fsync" \
> + $S
Re: [PATCH typo-fixed] fstests: btrfs: 159 superblock corruption test case
On Thu, Mar 29, 2018 at 06:28:48PM +0800, Anand Jain wrote:
> Verify if the superblock corruption is handled correctly.
>
> Signed-off-by: Anand Jain
> ---
> tests/btrfs/159 | 142
>
> tests/btrfs/159.out | 35 +
> tests/btrfs/group | 1 +
> 3 files changed, 178 insertions(+)
> create mode 100755 tests/btrfs/159
> create mode 100644 tests/btrfs/159.out
>
> diff --git a/tests/btrfs/159 b/tests/btrfs/159
> new file mode 100755
> index ..f42ba4b777e7
> --- /dev/null
> +++ b/tests/btrfs/159
> @@ -0,0 +1,142 @@
> +#! /bin/bash
> +# FS QA Test 159
> +#
> +# Test if the superblock corruption is handled correctly.
> +# - Check and validate superblock csum in mount and scan context
> +# - Check and validate superblock for all disks in the fs
> +# - Make sure if the copies are really a copy of the primary superblock
> +#
> +#-
> +# Copyright (c) 2018 Oracle. All Rights Reserved.
> +# Author: Anand Jain
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> +#-
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +status=1 # failure is the default!
Please use './new btrfs' to generate new test template, which defines
variables like 'tmp' and 'here', these variables may not be used in the
test explicitly, but they could be used by some helper functions,
especially the "$tmp" var. (Or please don't remove them from template :)
> +
> +_cleanup()
> +{
> + _scratch_dev_pool_put
> +}
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/module
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch_dev_pool 2
> +_require_loadable_fs_module "btrfs"
> +_require_command "$WIPEFS_PROG" wipefs
> +
> +_scratch_dev_pool_get 2
> +
> +MNT_OPT=$(echo $MOUNT_OPTIONS | cut -d" " -f2-)
I'm not sure about the purpose of this MNT_OPT, and it could be empty
and causes problems in test. Please see below.
> +DEV_GOOD=$(echo $SCRATCH_DEV_POOL | awk '{print $1}')
> +DEV_BAD=$(echo $SCRATCH_DEV_POOL | awk '{print $2}')
> +
> +wipe()
> +{
> + $WIPEFS_PROG -a $DEV_GOOD > /dev/null 2>&1
> + $WIPEFS_PROG -a $DEV_BAD > /dev/null 2>&1
> +}
> +
> +# If copy1 fsid does not match with primary fail the scan thus the mount as
> well
> +check_copy1_fsid()
> +{
> + local bytenr=67108864
> + echo -e "\\ncheck_copy1_fsid\\n{" | tee -a $seqres.full
Hmm, I don't think the "{ }" block is necessary in .out file, IMHO it
makes the diff context harder to read when test fails. Just echo the
test name would be fine.
> +
> + wipe
> + $MKFS_BTRFS_PROG -fq $DEV_GOOD
> + $MKFS_BTRFS_PROG -fq $DEV_BAD
> + _reload_fs_module "btrfs"
> +
> + dd status=none of=$DEV_BAD if=$DEV_GOOD ibs=1 obs=1 skip=$bytenr
> seek=$bytenr count=4K|\
> + tee -a $seqres.full
> +
> + _mount -o $MNT_OPT $DEV_BAD $SCRATCH_MNT 2>&1 | _filter_scratch_pool
When $MOUNT_OPTIONS is empty, this mount fails as
+mount: can't find /mnt/scratch in /etc/fstab
And I think we need to umount $SCRATCH_MNT here in case a buggy btrfs
module allowed the mount, as the next test assumes the device is not
mounted anywhere. Otherwise I see failures like:
+ERROR: /dev/loop1 is mounted
+mount failed
> +
> + echo -e "good\\n}" | tee -a $seqres.full
> +}
> +
> +# As in Linux kernel 4.16 if the primary is corrupted mount will fail.
> +# Which might change in the long run.
> +check_primary()
> +{
> + local bytenr=65536
> + echo -e "\\ncheck_primary\\n{" | tee -a $seqres.full
> +
> + wipe
> + _scratch_pool_mkfs "-mraid1 -draid1"
> + _reload_fs_module "btrfs"
> +
> + #To corrupt a disk block, read in hex, write in dec
> + od -j$bytenr -N1 -An -x $DEV_BAD |\
> + dd status=none of=$DEV_BAD ibs=1 obs=1 seek=$bytenr count=1|\
> + tee -a $seqres.full
> + _mount -o $MNT_OPT,device=$DEV_GOOD $DEV_BAD $SCRATCH_MNT 2>&1 |
> _filter_scratch_pool
Same here, need to umount $SCRATCH_MNT.
Also, if a mount failure is expe
Re: [PATCH] fstests: test btrfs fsync after hole punching with no-holes mode
On Thu, Mar 29, 2018 at 02:45:26PM +0100, Filipe Manana wrote: > On Wed, Mar 28, 2018 at 11:33 AM, Eryu Guan wrote: > > On Wed, Mar 28, 2018 at 09:48:17AM +0100, Filipe Manana wrote: > >> On Wed, Mar 28, 2018 at 3:17 AM, Eryu Guan wrote: > >> > On Mon, Mar 26, 2018 at 11:59:21PM +0100, fdman...@kernel.org wrote: > >> >> From: Filipe Manana > >> >> > >> >> Test that when we have the no-holes mode enabled and a specific metadata > >> >> layout, if we punch a hole and fsync the file, at replay time the whole > >> >> hole was preserved. > >> >> > >> >> This issue is fixed by the following btrfs patch for the linux kernel: > >> >> > >> >> "Btrfs: fix fsync after hole punching when using no-holes feature" > >> > > >> > I'd expect a test failure with 4.16-rc6 kernel, as the mentioned fix > >> > above is not there. But test always passes for me. Did I miss anything? > >> > btrfs-progs version is btrfs-progs-4.11.1-3.fc27. > >> > >> It should fail on any kernel, with any btrfs-progs version (which > >> should be irrelevant). > >> Somehow on your system we are not getting the specific metadata layout > >> needed to trigger the issue. > >> > >> Can you apply the following patch on top of the test and provide the > >> result 159.full file? > >> > >> https://friendpaste.com/6xAuLeN4xl1AGjO9Qc5I8L > >> > >> So that I can see what metadata layout you are getting. > >> Thanks! > > > > Sure, please see attachment. > > Thanks! > So you indeed get a different metadata layout, and that is because you > have SELinux enabled which causes some xattr to be added to the test > file (foobar): > > item 6 key (257 XATTR_ITEM 3817753667) itemoff 64932 itemsize 83 > location key (0 UNKNOWN.0 0) type XATTR > transid 7 data_len 37 name_len 16 > name: security.selinux > data unconfined_u:object_r:unlabeled_t:s0 > > I can make the test work with and without selinux enabled (by punching > holes on a few extents that are candidates to be at leaf boundaries). > Is it worth it? (I assume most people run the tests without selinux) Yes, please make it work with selinux if possible (but if that requires too much complexity, we can give it a second thought). I'm not sure about others, but I run fstests with selinux almost all the time, because Fedora/RHEL distros have selinux on by default :) so are all other people using Fedora/RHEL/Centos as test hosts, I guess. Thanks, Eryu -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] fstests: test btrfs fsync after hole punching with no-holes mode
On Wed, Mar 28, 2018 at 09:48:17AM +0100, Filipe Manana wrote: > On Wed, Mar 28, 2018 at 3:17 AM, Eryu Guan wrote: > > On Mon, Mar 26, 2018 at 11:59:21PM +0100, fdman...@kernel.org wrote: > >> From: Filipe Manana > >> > >> Test that when we have the no-holes mode enabled and a specific metadata > >> layout, if we punch a hole and fsync the file, at replay time the whole > >> hole was preserved. > >> > >> This issue is fixed by the following btrfs patch for the linux kernel: > >> > >> "Btrfs: fix fsync after hole punching when using no-holes feature" > > > > I'd expect a test failure with 4.16-rc6 kernel, as the mentioned fix > > above is not there. But test always passes for me. Did I miss anything? > > btrfs-progs version is btrfs-progs-4.11.1-3.fc27. > > It should fail on any kernel, with any btrfs-progs version (which > should be irrelevant). > Somehow on your system we are not getting the specific metadata layout > needed to trigger the issue. > > Can you apply the following patch on top of the test and provide the > result 159.full file? > > https://friendpaste.com/6xAuLeN4xl1AGjO9Qc5I8L > > So that I can see what metadata layout you are getting. > Thanks! Sure, please see attachment. Thanks, Eryu btrfs-159.full.bz2 Description: BZip2 compressed data
Re: [PATCH v2 3/3] fstests: generic: Check the fs after each FUA writes
On Wed, Mar 28, 2018 at 01:51:44PM +0800, Qu Wenruo wrote:
>
>
> On 2018年03月28日 13:04, Amir Goldstein wrote:
> > On Wed, Mar 28, 2018 at 7:40 AM, Qu Wenruo wrote:
> >> Basic test case which triggers fsstress with dm-log-writes, and then
> >> check the fs after each FUA writes.
> >> With needed infrastructure and special handlers for journal based fs.
> >>
> >> Signed-off-by: Qu Wenruo
> >> ---
> >> changelog:
> >> v2:
> >> Use proper "SUSE Linux Products GmbH" instead of "SuSE"
> >> Get rid of dm-snapshot which is pretty slow if we're creating and
> >> deleting snapshots repeatedly.
> >> (Maybe LVM thin provision would be much better, but current replay
> >>solution is good so far, and no slower than dm-snapshot)
> >> Add back run_check so that we can get the seed.
> >> ---
> >> Unfortunately, neither xfs nor ext4 survies this test for even single
> >> success, while btrfs survives.
> >> (Although not what I want, I'm just trying my luck
> >> to reproduce a serious btrfs corruption situation)
> >>
> >> Although btrfs may be the fastest fs for the test, since it has fixed
> >> small amount of write in mkfs and almost nothing to replay, it still
> >> takes about 240s~300s to finish (the same level using snapshot).
> >>
> >> It would take longer time for ext4 for its large amount of write during
> >> mkfs, if it can survive the test in the first space.
> >
> > Hmm, how much time would the total test would take if you don't fail
> > it on fsck? I am asking because it may be possible to improve this with
> > only a single snapshot after mkfs.
>
> The only fs which can pass the test right now is btrfs, so other
> estimation is mostly based on guess.
>
> >
> > Anyway, if total test run time is expected to be under 10min I wouldn't
> > bother with this optimization, at least not right now. IMO it is more
> > important to get the test out there to get the corruption bugs floating.
>
> I'd say from current status, if XFS doesn't fail, it would definitely
> finish in 10min.
> For EXT4 I'm not pretty sure.
10min might be a bit long, 5min would be good enough. I may need to
adjust the fsstress "-n" param based on test results when I got some
time, hopefully this week..
And I noticed that fsstress "-p" is based on nr_cpus, I'd like to cap it
with a max allowed number, so test won't run for too long on hosts with
hundreds of cpus. It could always be scaled with _scale_fsstress_args.
+nr_cpus=$("$here/src/feature" -o)
+fsstress_args=$(_scale_fsstress_args -w -d $SCRATCH_MNT -n 512 -p $nr_cpus \
+ $FSSTRESS_AVOID)
>
> I'd like to keep current test case as simple as possible right now, and
> for later enhancement, I have several different ideas:
Please make new tests then :)
>
> 1) Reflink fs + loopback
>Yep, use btrfs/xfs as base filesystem and create copy using reflink,
>then use such files as loopback device.
>The good thing is, AFAIK btrfs/xfs reflink is really fast.
>Much much faster than dm-snapshot or even LVM thin.
>
>The much much smaller block size (4K default) makes CoW overhead
>(LVM thin is 64K, not sure about dm-snapshot though).
>
>The problem is, such setup needs extra mount point and can be a
>little hard to setup, and we're introducing another layer of fs,
>if the fs itself has some hidden bug, it would screw up the test
>case.
>
> 2) LVM thin provision
>LVM thin provision looks much like btrfs/xfs for block level, and
>smaller default block size (64K vs original 2M) makes CoW overhead
>smaller.
>
>I'm currently testing this method, the good thing is it's a little
>easier to setup and we can use single mount point.
>
> Anyway, with proper and efficient snapshot ability implemented, I will
> definitely convert this test case, and add Flush test case.
>
> Thanks for your review too,
> Qu
>
> >
> > Thanks for working on this!
> > You can add
> > Reviewed-by: Amir Goldstein
Thank you both!
Eryu
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] fstests: test btrfs fsync after hole punching with no-holes mode
On Mon, Mar 26, 2018 at 11:59:21PM +0100, fdman...@kernel.org wrote:
> From: Filipe Manana
>
> Test that when we have the no-holes mode enabled and a specific metadata
> layout, if we punch a hole and fsync the file, at replay time the whole
> hole was preserved.
>
> This issue is fixed by the following btrfs patch for the linux kernel:
>
> "Btrfs: fix fsync after hole punching when using no-holes feature"
I'd expect a test failure with 4.16-rc6 kernel, as the mentioned fix
above is not there. But test always passes for me. Did I miss anything?
btrfs-progs version is btrfs-progs-4.11.1-3.fc27.
>
> Signed-off-by: Filipe Manana
> ---
> tests/btrfs/159 | 100
>
> tests/btrfs/159.out | 5 +++
> tests/btrfs/group | 1 +
> 3 files changed, 106 insertions(+)
> create mode 100755 tests/btrfs/159
> create mode 100644 tests/btrfs/159.out
>
> diff --git a/tests/btrfs/159 b/tests/btrfs/159
> new file mode 100755
> index ..6083975a
> --- /dev/null
> +++ b/tests/btrfs/159
> @@ -0,0 +1,100 @@
> +#! /bin/bash
> +# FSQA Test No. 159
> +#
> +# Test that when we have the no-holes mode enabled and a specific metadata
> +# layout, if we punch a hole and fsync the file, at replay time the whole
> +# hole was preserved.
> +#
> +#---
> +#
> +# Copyright (C) 2018 SUSE Linux Products GmbH. All Rights Reserved.
> +# Author: Filipe Manana
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> +#---
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + _cleanup_flakey
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/dmflakey
> +
> +# real QA test starts here
> +_supported_fs generic
^^^ btrfs
> +_supported_os Linux
> +_require_scratch
> +_require_dm_target flakey
> +_require_xfs_io_command "fpunch"
> +
> +rm -f $seqres.full
> +
> +# We create the filesystem with a node size of 64Kb because we need to
> create a
> +# specific metadata layout in order to trigger the bug we are testing. At the
> +# moment the node size can not be smaller then the system's page size, so
> given
> +# that the largest possible page size is 64Kb and by default the node size is
> +# set to the system's page size value, we explictly create a filesystem with
> a
> +# 64Kb node size.
> +_scratch_mkfs -O no-holes -n $((64 * 1024)) >>$seqres.full 2>&1
> +_require_metadata_journaling $SCRATCH_DEV
> +_init_flakey
> +_mount_flakey
> +
> +# Create our test file with 831 extents of 256Kb each. Before each extent,
> there
I think it's 832 extents in total? As the index starts with 0.
Otherwise looks good to me.
Thanks,
Eryu
> +# is a 256Kb hole (except for the first extent, which starts at offset 0).
> This
> +# creates two leafs in the filesystem tree, with the extent at offset
> 216530944
> +# being the last item in the first leaf and the extent at offset 217055232
> being
> +# the first item in the second leaf.
> +for ((i = 0; i <= 831; i++)); do
> + offset=$((i * 2 * 256 * 1024))
> + $XFS_IO_PROG -f -c "pwrite -S 0xab -b 256K $offset 256K" \
> + $SCRATCH_MNT/foobar >/dev/null
> +done
> +
> +# Now persist everything done so far.
> +sync
> +
> +# Now punch a hole that covers part of the extent at offset 216530944.
> +$XFS_IO_PROG -c "fpunch $((216530944 + 128 * 1024 - 4000)) 256K" \
> + -c "fsync" \
> + $SCRATCH_MNT/foobar
> +
> +echo "File digest before power failure:"
> +md5sum $SCRATCH_MNT/foobar | _filter_scratch
> +
> +# Simulate a power failure and mount the filesystem to check that replaying
> the
> +# fsync log/journal succeeds and our test file has the expected content.
> +_flakey_drop_and_remount
> +
> +echo "File digest after power failure and log replay:"
> +md5sum $SCRATCH_MNT/foobar | _filter_scratch
> +
> +_unmount_flakey
> +_cleanup_flakey
> +
> +status=0
> +exit
> diff --git a/tests/btrfs/159.out b/tests/btrfs/159.out
> new file mode 100644
> index ..3317e516
> --- /dev/null
> +++ b/tests/b
Re: [PATCH 3/3] fstests: generic: Check the fs after each FUA writes
On Wed, Mar 21, 2018 at 02:22:29PM +0200, Amir Goldstein wrote: > > + > > +_log_writes_mount > > +$FSSTRESS_PROG $fsstress_args > /dev/null 2>&1 > > You should run fsstress with run_check() so output will go to $seqres.full > this way if you are able to catch a bug, you can take the random seed > from fsstress output and repeat the same event sequence, which > doesn't guaranty, but can increase the chances of reproducing the bug. I suggested dropping run_check, as I don't think we care about the fsstress return value here (and I always try to avoid new run_check usage), but I agree that it might be useful to save the fsstress output to $seqres.full. Thanks, Eryu -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH RFC 3/3] fstests: generic: Check the fs after each FUA writes
On Wed, Mar 14, 2018 at 05:02:30PM +0800, Qu Wenruo wrote:
> Basic test case which triggers fsstress with dm-log-writes, and then
> check the fs after each FUA writes.
> With needed infrastructure and special handlers for journal based fs.
>
> Signed-off-by: Qu Wenruo
> ---
> In my test, xfs and btrfs survives while ext4 would report error during fsck.
>
> My current biggest concern is, we abuse $TEST_DEV and mkfs on it all by
> ourselves. Not sure if it's allowed.
> ---
> common/dmlogwrites| 119
> tests/generic/481 | 124
> ++
> tests/generic/481.out | 2 +
> tests/generic/group | 1 +
> 4 files changed, 246 insertions(+)
> create mode 100755 tests/generic/481
> create mode 100644 tests/generic/481.out
>
> diff --git a/common/dmlogwrites b/common/dmlogwrites
> index 467b872e..258f5887 100644
> --- a/common/dmlogwrites
> +++ b/common/dmlogwrites
> @@ -126,3 +126,122 @@ _log_writes_cleanup()
> $UDEV_SETTLE_PROG >/dev/null 2>&1
> _log_writes_remove
> }
> +
> +# Convert log writes mark to entry number
> +# Result entry number is output to stdout, could be empty if not found
> +_log_writes_mark_to_entry_number()
> +{
> + local _mark=$1
> + local ret
> +
> + [ -z "$_mark" ] && _fatal \
> + "mark must be given for _log_writes_mark_to_entry_number"
> +
> + ret=$($here/src/log-writes/replay-log --find --log $LOGWRITES_DEV \
> + --end-mark $_mark 2> /dev/null)
> + [ -z "$ret" ] && return
> + ret=$(echo "$ret" | cut -f1 -d\@)
> + echo "mark $_mark has entry number $ret" >> $seqres.full
> + echo "$ret"
> +}
> +
> +# Find next fua write entry number
> +# Result entry number is output to stdout, could be empty if not found
> +_log_writes_find_next_fua()
> +{
> + local _start_entry=$1
> + local ret
> +
> + [ -z "$_start_entry" ] && _start_entry=0
> + ret=$($here/src/log-writes/replay-log --find --log $LOGWRITES_DEV \
> + --next-fua --start-entry $_start_entry 2> /dev/null)
> + [ -z "$ret" ] && return
> +
> + ret=$(echo "$ret" | cut -f1 -d\@)
> + echo "next fua is entry number $ret" >> $seqres.full
> + echo "$ret"
> +}
> +
> +# Replay log range to specified entry
> +# $1:End entry. The entry with this number *WILL* be replayed
> +# $2:Start entry. If not specified, start from the first entry.
> +# $3:Verbose. If set to 'y' do verbose output
> +_log_writes_replay_log_entry_range()
_log_writes_replay_log_range() would be fine.
> +{
> + local _end=$1
> + local _start=$2
This arguments order ($end comes first) makes me confused when I first
read the test code. Reverse the order?
> + local _verbose=$3
> +
> + [ -z "$_end" ] && _fatal \
> + "end entry must be specified for _log_writes_replay_log_entry_range"
> +
> + [ "x$verbose" == "xy" ] && _verbose="-v"
> + [ -z "$_start" ] && _start=0
> + [ "$_start" -gt "$_end" ] && _fatal \
> + "wrong parameter order for
> _log_writes_replay_log_entry_range:start=$_start end=$_end"
> +
> + # To ensure we replay the last entry, for _start == 0 case,
> + # we need to manually increase the end entry number to ensure
> + # it's played
> + echo "=== replay from $_start to $_end ===" >> $seqres.full
> + $here/src/log-writes/replay-log --log $LOGWRITES_DEV \
> + --replay $SCRATCH_DEV --start-entry $_start \
> + --limit $(($_end - $_start + 1)) $_verbose \
> + >> $seqres.full 2>&1
> + [ $? -ne 0 ] && _fatal "replay failed"
> +}
> +
> +_log_writes_cleanup_snapshot()
> +{
> + $UDEV_SETTLE_PROG > /dev/null 2>&1
> + $DMSETUP_PROG remove "$DMLOGWRITES_SNAPSHOT_NAME" > /dev/null 2>&1
> + $DMSETUP_PROG remove "$DMLOGWRITES_ORIGIN_NAME" > /dev/null 2>&1
> +}
> +
> +# Helper to create snapshot of a the replayed device
> +# Useful for journal based filesystem such as XFS and Ext4 to replay
> +# their journal without touching the replay device, so that we can
> +# continue replaying other than replay from the beginning.
> +# $1:Snapshot device
> +_log_writes_create_snapshot()
> +{
> + _require_dm_target snapshot
This doesn't belong here, call it in the test.
> +
> + local snapshot_dev=$1
> + local cow_base=""
Unused variable.
> +
> + [ -z "$snapshot_dev" ] && _fatal \
> + "@device must be specified for _log_writes_create_snapshot"
> + local size=$(blockdev --getsz $SCRATCH_DEV)
> + [ -z "$size" ] && _fatal \
> + "failed to get device size for _log_writes_create_snapshot"
> +
> + _log_writes_cleanup_snapshot
> +
> + DMLOGWRITES_ORIGIN_NAME="log-writes-origin"
> + DMLOGWRITES_SNAPSHOT_NAME="log-writes-snapshot"
> + DMLOGWRITES_ORIGIN_TABLE="0 $size snapshot-origin $SCRATCH_DEV"
> + DMLOGWRITES_SNAPSHOT_TABLE="0 $size snapshot
> /dev/mapper/$DMLOGWRITES_ORIGIN_
Re: [PATCH RFC 3/3] fstests: generic: Check the fs after each FUA writes
On Fri, Mar 16, 2018 at 01:17:07PM +0800, Qu Wenruo wrote: > > > On 2018年03月16日 12:01, Eryu Guan wrote: > > On Wed, Mar 14, 2018 at 05:02:30PM +0800, Qu Wenruo wrote: > >> Basic test case which triggers fsstress with dm-log-writes, and then > >> check the fs after each FUA writes. > >> With needed infrastructure and special handlers for journal based fs. > > > > It's not clear to me why the existing infrastructure is not sufficient > > for the test. It'd be great if you could provide more information and/or > > background in commit log. > > The main problem of current infrastructure is we don't have the > following things: > > 1) Way to take full advantage of dm-log-writes >The main thing is, we don't have test cases to check each FUA (this >patch) and flush (later patch after clearing all the RFC comments). > >We have some dm-flakey test cases to emulate power loss, but they are >mostly for fsync. >Here we are not only testing fsync, but also every superblock update. >Which should be a super set of dm-flakey tests. > > 2) Workaround for journal replay >In fact, if we only test btrfs, we don't even need such complicated >work, just 'replay-log --fsck "btrfs check" --check fua' will be >enough. As btrfs check doesn't report dirty journal (log tree) as >problem. >But for journal based fses, their fsck all report dirty journal as >error, which needs current snapshot works to replay them before >running fsck. And replay-to-fua doesn't guarantee a consistent filesystem state, that's why we need to mount/umount the target device to replay the filesystem journal, and to avoid replaying already-replayed-log over and over again, we create a snapshot of the target device and mount cycle & fsck the snapshot, right? I'm wondering if the overhead of repeatly create & destroy snapshots is larger than replaying log from start. Maybe snapshots take more time? > > I would add them in next version if there is no extra comment on this. > > > > >> > >> Signed-off-by: Qu Wenruo > >> --- > >> In my test, xfs and btrfs survives while ext4 would report error during > >> fsck. > >> > >> My current biggest concern is, we abuse $TEST_DEV and mkfs on it all by > >> ourselves. Not sure if it's allowed. > > > > As Amir already replied, that's not allowed, any destructive operations > > should be done on $SCRATCH_DEV. > > Yep, I'm looking for similar case who uses $SCRATCH_DEV as LVM pv do get > extra device. > > Or can we reuse the scratch_dev_pool even for ext4/xfs? I think so, IMO pool devices are not limited to btrfs. But I think we could use a loop device reside on $TEST_DIR? Or if snapshots take longer time, then we don't need this extra device at all :) I have some other comments, will reply to the RFC patch in another email. Thanks, Eryu -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH RFC 3/3] fstests: generic: Check the fs after each FUA writes
On Wed, Mar 14, 2018 at 05:02:30PM +0800, Qu Wenruo wrote:
> Basic test case which triggers fsstress with dm-log-writes, and then
> check the fs after each FUA writes.
> With needed infrastructure and special handlers for journal based fs.
It's not clear to me why the existing infrastructure is not sufficient
for the test. It'd be great if you could provide more information and/or
background in commit log.
>
> Signed-off-by: Qu Wenruo
> ---
> In my test, xfs and btrfs survives while ext4 would report error during fsck.
>
> My current biggest concern is, we abuse $TEST_DEV and mkfs on it all by
> ourselves. Not sure if it's allowed.
As Amir already replied, that's not allowed, any destructive operations
should be done on $SCRATCH_DEV.
> ---
> common/dmlogwrites| 119
> tests/generic/481 | 124
> ++
> tests/generic/481.out | 2 +
> tests/generic/group | 1 +
> 4 files changed, 246 insertions(+)
> create mode 100755 tests/generic/481
> create mode 100644 tests/generic/481.out
>
> diff --git a/common/dmlogwrites b/common/dmlogwrites
> index 467b872e..258f5887 100644
> --- a/common/dmlogwrites
> +++ b/common/dmlogwrites
> @@ -126,3 +126,122 @@ _log_writes_cleanup()
> $UDEV_SETTLE_PROG >/dev/null 2>&1
> _log_writes_remove
> }
> +
> +# Convert log writes mark to entry number
> +# Result entry number is output to stdout, could be empty if not found
> +_log_writes_mark_to_entry_number()
> +{
> + local _mark=$1
No need to name a local variable with leading underscore.
> + local ret
> +
> + [ -z "$_mark" ] && _fatal \
> + "mark must be given for _log_writes_mark_to_entry_number"
Please use _fail instead of _fatal (and all the other places in this
patch). I think _fatal is only used at the initializing phase of the
test harness (e.g. check required tools, env etc.) and abort the test
run in early stage. _fail is used to indicate a failure in test phase,
it also logs the error messages in $seqres.full.
> +
> + ret=$($here/src/log-writes/replay-log --find --log $LOGWRITES_DEV \
> + --end-mark $_mark 2> /dev/null)
Is this output long or short? If it's a short one, e.g. a number or a
simple string, I think it's OK to save it in a variable. But if it's
long/complex enough and/or has multiple lines, I think we'd better to
save it with a tmp file, e.g. $tmp..
> + [ -z "$ret" ] && return
> + ret=$(echo "$ret" | cut -f1 -d\@)
Because I want to avoid echoing such a variable to a pipe (even it's
quoted), it used to cause subtle problems if not quoted properly.
Also, it'd be better to give a sample output of the replay-log command
in comment if it's short enough, so people could have an idea what it
looks like and know what you're looking for in the subsequent steps
(cut -f1 -d\@, in this example), and it's easier to review.
Above comments apply to other helper functions too.
Thanks for all the work!
Eryu
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] fstests: test regression of -EEXIST on creating new file after log replay
On Sat, Mar 10, 2018 at 04:56:04PM -0700, Liu Bo wrote:
> The regression is introduced to btrfs in linux v4.4 and it refuses to create
> new files after log replay by returning -EEXIST.
>
> Although the problem is on btrfs only, there is no btrfs stuff in terms of
> test, so this makes it generic.
>
> The kernel fix is
> Btrfs: fix unexpected -EEXIST when creating new inode
>
> Reviewed-by: Filipe Manana
> Signed-off-by: Liu Bo
> ---
> v2: - Remove failed message from 481.out
> - Drop the unnecessary write in creating a file
>
> tests/generic/481 | 75
> +++
> tests/generic/481.out | 2 ++
> tests/generic/group | 1 +
> 3 files changed, 78 insertions(+)
> create mode 100755 tests/generic/481
> create mode 100644 tests/generic/481.out
>
> diff --git a/tests/generic/481 b/tests/generic/481
> new file mode 100755
> index 000..6a7e9dd
> --- /dev/null
> +++ b/tests/generic/481
> @@ -0,0 +1,75 @@
> +#! /bin/bash
> +# FSQA Test No. 481
> +#
> +# Reproduce a regression of btrfs that leads to -EEXIST on creating new files
> +# after log replay.
> +#
> +# The kernel fix is
> +# Btrfs: fix unexpected -EEXIST when creating new inode
> +#
> +#---
> +#
> +# Copyright (C) 2018 Oracle. All Rights Reserved.
> +# Author: Bo Liu
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> +#---
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + _cleanup_flakey
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/dmflakey
> +
> +# real QA test starts here
> +_supported_fs generic
> +_supported_os Linux
> +_require_scratch
> +_require_dm_target flakey
> +
> +rm -f $seqres.full
> +
> +_scratch_mkfs >>$seqres.full 2>&1
I added a "_require_metadata_journaling $SCRATCH_DEV" here, otherwise
test fails with ext2, which doesn't have journal.
Thanks,
Eryu
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] fstests: btrfs/146: make sure hit all stripes in the case of compression
On Thu, Mar 08, 2018 at 01:56:45PM +0800, Lu Fengqi wrote: > In the case of compression, each 128K input data chunk will be compressed > to 4K (because of the characters written are duplicate). Therefore we have > to write (128K * 16) to make sure every stripe can be hit. > > Signed-off-by: Lu Fengqi > --- > > V2: Modify the regular expression to ensure that it matches various > compress mount options. > > tests/btrfs/146 | 11 +-- > 1 file changed, 9 insertions(+), 2 deletions(-) > > diff --git a/tests/btrfs/146 b/tests/btrfs/146 > index 7071c128ca0a..a51eda68eaf3 100755 > --- a/tests/btrfs/146 > +++ b/tests/btrfs/146 > @@ -74,9 +74,16 @@ _scratch_pool_mkfs "-d raid0 -m raid1" > $seqres.full 2>&1 > _scratch_mount > > # How much do we need to write? We need to hit all of the stripes. btrfs uses > -# a fixed 64k stripesize, so write enough to hit each one > +# a fixed 64k stripesize, so write enough to hit each one. In the case of > +# compression, each 128K input data chunk will be compressed to 4K (because > of > +# the characters written are duplicate). Therefore we have to write (128K * > 16) > +# to make sure every stripe can be hit. > number_of_devices=`echo $SCRATCH_DEV_POOL | wc -w` > -write_kb=$(($number_of_devices * 64)) > +if ! echo $MOUNT_OPTIONS | grep -qoP 'compress(-force)?(=(?!no)|,|$)'; then I'm wondering if we could just write ($number_of_devices * 2048)K data unconditionally, so we could get rid of this case switch and the fancy perl style regexp? Thanks, Eryu > + write_kb=$(($number_of_devices * 64)) > +else > + write_kb=$(($number_of_devices * 2048)) > +fi > _require_fs_space $SCRATCH_MNT $write_kb > > testfile=$SCRATCH_MNT/fsync-err-test > -- > 2.16.2 > > > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] fstests: btrfs/004: increase the buffer size of logical-resolve to the maximum value 64K
On Tue, Mar 06, 2018 at 03:02:31PM +0800, Lu Fengqi wrote:
> Because of commit e76e13ce8c0b ("fsstress: implement the
> clonerange/deduperange ioctls"), dedupe makes the number of references to
> the same extent item increase so much that the default 4K buffer of
> logical-resolve is no longer sufficient.
>
> Signed-off-by: Lu Fengqi
This looks fine to me. But I'd like an explicit ack from btrfs
developers.
Thanks,
Eryu
> ---
> tests/btrfs/004 | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tests/btrfs/004 b/tests/btrfs/004
> index de583cc355d4..0d2efb91dba7 100755
> --- a/tests/btrfs/004
> +++ b/tests/btrfs/004
> @@ -103,7 +103,7 @@ _btrfs_inspect_addr()
> expect_addr=$3
> expect_inum=$4
> file=$5
> - cmd="$BTRFS_UTIL_PROG inspect-internal logical-resolve -P $addr $mp"
> + cmd="$BTRFS_UTIL_PROG inspect-internal logical-resolve -s 65536 -P
> $addr $mp"
> echo "# $cmd" >> $seqres.full
> out=`$cmd`
> echo "$out" >> $seqres.full
> --
> 2.16.2
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] fstests: common/rc: fix device still mounted error with SCRATCH_DEV_POOL
On Mon, Jan 15, 2018 at 11:10:20PM -0800, Liu Bo wrote:
> On Mon, Jan 15, 2018 at 02:22:28PM +0800, Eryu Guan wrote:
> > On Fri, Jan 12, 2018 at 06:04:59PM -0700, Liu Bo wrote:
> > > One of btrfs tests, btrfs/011, uses SCRATCH_DEV_POOL and puts a
> > > non-SCRATCH_DEV
> > > device as the first one when doing mkfs, and this makes
> > > _require_scratch{_nocheck} fail to umount $SCRATCH_MNT since it checks
> > > mount
> > > point with SCRATCH_DEV only, and for sure it finds nothing to umount and
> > > the
> > > following tests complain about 'device still mounted' alike errors.
> > >
> > > Introduce a helper to address this special case where both btrfs and
> > > scratch
> > > dev pool are in use.
> > >
> > > Signed-off-by: Liu Bo
> >
> > Hmm, I didn't see this problem, I ran btrfs/011 then another tests that
> > uses $SCRATCH_DEV, and the second test ran fine too. Can you please
> > provide more details?
>
> Sure, so I was using 4 devices of size being 2500M, btrfs/011 bailed
> out when doing a cp due to enospc then _fail is called to abort the
> test, and the mount point now is associated with a different device
> other than SCRATCH_DEV, so that _require_scratch_nocheck in btrfs/012
> was not able to umount SCRATCH_MNT.
Yeah, that's the exact case I described as below. I think adding
_scratch_umount >/dev/null 2>&1 in _cleanup() would resolve your issue.
>
> >
> > Anyway, I think we should fix btrfs/011 to either not use $SCRATCH_DEV
> > in replace operations (AFAIK, other btrfs replace tests do this) or
> > umount all devices before exit. And I noticed btrfs/011 does umount
> > $SCRATCH_MNT at the end of workout(), so usually all should be fine
> > (perhaps it would leave a device mounted if interrupted in the middle of
> > test run, because _cleanup() doesn't do umount).
>
> That's true, if you want, I could fix all btrfs replace tests to
> umount SCRATCH_MNT right before exit.
I think only the tests that replace $SCRATCH_DEV (as what btrfs/011
does) need fixes, _require_scratch would umount $SCRATCH_MNT for other
tests.
Thanks,
Eryu
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] fstests: common/rc: fix device still mounted error with SCRATCH_DEV_POOL
On Fri, Jan 12, 2018 at 06:04:59PM -0700, Liu Bo wrote:
> One of btrfs tests, btrfs/011, uses SCRATCH_DEV_POOL and puts a
> non-SCRATCH_DEV
> device as the first one when doing mkfs, and this makes
> _require_scratch{_nocheck} fail to umount $SCRATCH_MNT since it checks mount
> point with SCRATCH_DEV only, and for sure it finds nothing to umount and the
> following tests complain about 'device still mounted' alike errors.
>
> Introduce a helper to address this special case where both btrfs and scratch
> dev pool are in use.
>
> Signed-off-by: Liu Bo
Hmm, I didn't see this problem, I ran btrfs/011 then another tests that
uses $SCRATCH_DEV, and the second test ran fine too. Can you please
provide more details?
Anyway, I think we should fix btrfs/011 to either not use $SCRATCH_DEV
in replace operations (AFAIK, other btrfs replace tests do this) or
umount all devices before exit. And I noticed btrfs/011 does umount
$SCRATCH_MNT at the end of workout(), so usually all should be fine
(perhaps it would leave a device mounted if interrupted in the middle of
test run, because _cleanup() doesn't do umount).
Thanks,
Eryu
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/5] fstests: filter: Introduce filter to filter out offset for xfs_io
On Thu, Jan 11, 2018 at 02:55:57PM +0800, Qu Wenruo wrote:
> Some test cases (AFAIK, btrfs RAID recovery test cases) read out certain
> location to verify its data.
>
> Such read is mostly OK, but the golden output contains the on-disk
> offset, which can differ due to underlying chunk change.
> (This time is mkfs chunk layout change for btrfs)
>
> So introduce macro _filter_xfs_io_offset to filter out the offset part
> wrote 65536/65536 bytes at offset 136708096
>
> And offset from "pread -v"
> 0826: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa
> ^
>
> Reported-by: Nikolay Borisov
> Signed-off-by: Qu Wenruo
> ---
> common/filter | 10 +-
> 1 file changed, 9 insertions(+), 1 deletion(-)
>
> diff --git a/common/filter b/common/filter
> index 9c33efac..77afcbc4 100644
> --- a/common/filter
> +++ b/common/filter
> @@ -226,6 +226,15 @@ _filter_xfs_io()
> sed -e "s/[0-9/.]* [GMKiBbytes]*, [0-9]* ops\; [0-9/:. sec]*
> ([infa0-9/.]* [EPGMKiBbytes]*\/sec and [infa0-9/.]* ops\/sec)/XXX Bytes, X
> ops\; XX:XX:XX.X (XXX YYY\/sec and XXX ops\/sec)/"
> }
>
> +# Also filter out the offset part of xfs_io output
> +# Some test cases may be affected by underlaying extent/chunk layout change,
> +# so wipe out this part to avoid golden output difference
> +_filter_xfs_io_offset()
> +{
> +# filter out " at offset XXX" and offset of "pread -v"
> +sed -e "s/ at offset [0-9]*$//" -e "s/^[0-9a-f]\+:/:/"
> +}
> +
I'd put _filter_xfs_io part of _filter_xfs_io_offset, so the callers
don't have to chain the filter themselves, e.g.
- $XFS_IO_PROG ... | _filter_xfs_io | _filter_xfs_io_offset
+ $XFS_IO_PROG ... | _filter_xfs_io_offset
And I perfer folding this 5-patch patchset into a single patch, so the
introduction and the usage of the new helper function together provide a
complete context in git log.
Thanks,
Eryu
> # stderr filter for xfs_io to handle change of error output format (e.g.
> # pwrite64 -> pwrite).
> _filter_xfs_io_error()
> @@ -277,7 +286,6 @@ _filter_xfs_io_pages_modified()
> _filter_xfs_io_units_modified "Page" $PAGE_SIZE
> }
>
> -
> _filter_test_dir()
> {
> # TEST_DEV may be a prefix of TEST_DIR (e.g. /mnt, /mnt/ovl-mnt)
> --
> 2.15.1
>
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] generic/015: Change the test filesystem size to 101mb
On Mon, Jan 08, 2018 at 10:43:30AM +0200, Nikolay Borisov wrote: > This test has been failing for btrfs for quite some time, > at least since 4.7. There are 2 implementation details of btrfs that > it exposes: > > 1. Currently btrfs filesystem under 100mb are created in Mixed block > group mode. Freespace accounting for it is not 100% accurate - I've mkfs.btrfs does this too? Because I noticed _scratch_mkfs_sized adds '--mixed' mkfs option explicitly. > observed around 100-200kb discrepancy between a newly created filesystem, > then writing a file and deleting it and checking the free space. This > falls within %3 and not %1 as hardcoded in the test. > > 2. BTRFS won't flush it's delayed allocation on file deletion if less > than 32mb are deleted. On such files we need to perform sync (missing > in the test) or wait until time elapses for transaction commit. > > Since mixed mode is somewhat deprecated and btrfs is not really intended > to be used on really small devices let's just adjust the test to > create a 101mb fs, which doesn't use mixed mode and really test > freespace accounting. > > Signed-off-by: Nikolay Borisov > --- > tests/generic/015 | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/tests/generic/015 b/tests/generic/015 > index 78f2b13..416c4ae 100755 > --- a/tests/generic/015 > +++ b/tests/generic/015 > @@ -53,7 +53,7 @@ _supported_os Linux > _require_scratch > _require_no_large_scratch_dev > > -_scratch_mkfs_sized `expr 50 \* 1024 \* 1024` >/dev/null 2>&1 \ > +_scratch_mkfs_sized `expr 101 \* 1024 \* 1024` >/dev/null 2>&1 \ Better to have some comments in the code too, to explain why we choose 101m filesystem size. Thanks, Eryu > || _fail "mkfs failed" > _scratch_mount || _fail "mount failed" > out=$SCRATCH_MNT/fillup.$$ > -- > 2.7.4 > > -- > To unsubscribe from this list: send the line "unsubscribe fstests" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] fstests: btrfs/158: reproduce a scrub bug on raid6 corruption
On Tue, Jan 02, 2018 at 01:35:00PM -0700, Liu Bo wrote:
> This is to reproduce a bug of scrub, with which scrub is unable to
> repair raid6 corruption as expected.
>
> The kernel side fixes are
> Btrfs: make raid6 rebuild retry more
> Btrfs: fix scrub to repair raid6 corruption
>
> Signed-off-by: Liu Bo
Looks fine overall, I tested it with 4.15-rc6 kernel and test failed as
expected, re-tested successfully after applying the patches mentioned in
commit log.
Just some really minor issues below, and I can fix them on commit if
what I suggest looks sane to you.
> ---
> tests/btrfs/158 | 114
>
> tests/btrfs/158.out | 10 +
> tests/btrfs/group | 1 +
> 3 files changed, 125 insertions(+)
> create mode 100755 tests/btrfs/158
> create mode 100644 tests/btrfs/158.out
>
> diff --git a/tests/btrfs/158 b/tests/btrfs/158
> new file mode 100755
> index 000..43afc2d
> --- /dev/null
> +++ b/tests/btrfs/158
> @@ -0,0 +1,114 @@
> +#! /bin/bash
> +# FS QA Test 158
> +#
> +# The test case is check if scrub is able fix raid6 data corruption,
> +# ie. if there is data corruption on two disks in the same horizontal
> +# stripe, e.g. due to bitrot.
> +#
> +# The kernel fixes are
> +#Btrfs: make raid6 rebuild retry more
> +#Btrfs: fix scrub to repair raid6 corruption
> +#
> +#---
> +# Copyright (c) 2017 Oracle. All Rights Reserved.
2018?
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> +#---
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch_dev_pool 4
> +_require_btrfs_command inspect-internal dump-tree
> +
> +get_physical_stripe0()
> +{
> + $BTRFS_UTIL_PROG inspect-internal dump-tree -t 3 $SCRATCH_DEV | \
> + grep " DATA\|RAID6" -A 10 | \
> + $AWK_PROG '($1 ~ /stripe/ && $3 ~ /devid/ && $2 ~ /0/) { print $6 }'
> +}
> +
> +get_physical_stripe1()
> +{
> + $BTRFS_UTIL_PROG inspect-internal dump-tree -t 3 $SCRATCH_DEV | \
> + grep " DATA\|RAID6" -A 10 | \
> + $AWK_PROG '($1 ~ /stripe/ && $3 ~ /devid/ && $2 ~ /1/) { print $6 }'
> +}
> +
> +_scratch_dev_pool_get 4
> +# step 1: create a raid6 btrfs and create a 4K file
> +echo "step 1..mkfs.btrfs" >>$seqres.full
> +
> +mkfs_opts="-d raid6 -b 1G"
> +_scratch_pool_mkfs $mkfs_opts >>$seqres.full 2>&1
> +
> +# -o nospace_cache makes sure data is written to the start position of the
> data
> +# chunk
> +_scratch_mount -o nospace_cache
> +
> +# [0,64K) is written to stripe 0 and [64K, 128K) is written to stripe 1
> +$XFS_IO_PROG -f -d -c "pwrite -S 0xaa 0 128K" -c "fsync" \
> + "$SCRATCH_MNT/foobar" | _filter_xfs_io
> +
> +_scratch_unmount
> +
> +stripe_0=`get_physical_stripe0`
> +stripe_1=`get_physical_stripe1`
> +dev4=`echo $SCRATCH_DEV_POOL | awk '{print $4}'`
> +dev3=`echo $SCRATCH_DEV_POOL | awk '{print $3}'`
> +
> +# step 2: corrupt the 1st and 2nd stripe (stripe 0 and 1)
> +echo "step 2..simulate bitrot at offset $stripe_0 of device_4($dev4) and
> offset $stripe_1 of device_3($dev3)" >>$seqres.full
> +
> +$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev4 | _filter_xfs_io
> +$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev3 | _filter_xfs_io
> +
> +# step 3: read foobar to repair the bitrot
Comment meant to be "scrub to repair the bitrot"?
> +echo "step 3..repair the bitrot" >> $seqres.full
> +_scratch_mount -o nospace_cache
> +
> +btrfs scrub start -B $SCRATCH_MNT >> $seqres.full 2>&1
$BTRFS_UTIL_PROG ...
> +
> +od -x $SCRATCH_MNT/foobar
> +
> +_scratch_dev_pool_put
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/btrfs/158.out b/tests/btrfs/158.out
> new file mode 100644
> index 000..1f5ad3f
> --- /dev/null
> +++ b/tests/btrfs/158
Re: [PATCH] fstests: btrfs: Add test case to check if btrfs can handle full fs trim correctly
On Thu, Dec 07, 2017 at 08:43:43AM +0800, Qu Wenruo wrote: > Ping. > > Any comment on this? It's been pushed out to upstream, see commit 88231c0c0b9d Thanks, Eryu -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] fstests: btrfs: reproduce a read failure on raid6 setup
On Mon, Dec 04, 2017 at 03:33:23PM -0700, Liu Bo wrote: > This test case is to reproduce a bug of raid6 reconstruction process. > > The kernel fix are > Btrfs: do not merge rbios if their fail stripe index are not identical > Btrfs: make raid6 rebuild retry more > > Signed-off-by: Liu Bo Test failed as expected with v4.15-rc2 kernel and passed with above patches applied. I fixed two trailing whitespace issues and some of the indentions to avoid too-long line, also replaced 'repair' group with a new 'raid' group. I think 'repair' group is meant for testing filesystem repair tools, not the internal raid repair mechanism, so a new 'raid' group looks better to me, as we're having more raid-specific tests recently. Thanks, Eryu -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs/057: Fix test case to work on 64K page size
On Wed, Dec 06, 2017 at 12:15:45PM +0530, Harish wrote: > On platforms with a page size greater than 4Kb, at the moment btrfs > doesn't support a node/leaf size smaller than the page size, but it > supports a larger one. So use the max supported node size (64Kb) so > that the test runs on any platform currently supported by Linux. > > Signed-off-by: Harish Looks fine to me. Also cc btrfs list for review. Thanks, Eryu > --- > tests/btrfs/057 | 7 +-- > tests/btrfs/057.out | 4 ++-- > 2 files changed, 7 insertions(+), 4 deletions(-) > > diff --git a/tests/btrfs/057 b/tests/btrfs/057 > index 8d35579..06b2be9 100755 > --- a/tests/btrfs/057 > +++ b/tests/btrfs/057 > @@ -48,8 +48,11 @@ _require_scratch > > rm -f $seqres.full > > -# use small leaf size to get higher btree height. > -run_check _scratch_mkfs "-b 1g --nodesize 4096" > +# Currently in btrfs the node/leaf size can not be smaller than the page > +# size (but it can be greater than the page size). So use the largest > +# supported node/leaf size (64Kb) so that the test can run on any platform > +# that Linux supports. > +run_check _scratch_mkfs "-b 1g --nodesize 65536" > > # inode cache is saved in the FS tree itself for every > # individual FS tree,that affects the sizes reported by qgroup show > diff --git a/tests/btrfs/057.out b/tests/btrfs/057.out > index 60cb92d..f2d9e9c 100644 > --- a/tests/btrfs/057.out > +++ b/tests/btrfs/057.out > @@ -1,3 +1,3 @@ > QA output created by 057 > -4096 4096 > -4096 4096 > +65536 65536 > +65536 65536 > -- > 2.7.4 > > -- > To unsubscribe from this list: send the line "unsubscribe fstests" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
