Re: BTRFS thinks that a device is mounted
Hallo, Nikos, Du meintest am 21.10.11: > What went wrong in the following scenario? BTRFS thinks that a device > is mounted while it is not. [...] > btrfs device scan /dev/sdh > Scanning for Btrfs filesystems in '/dev/sdh' > ERROR: unable to scan the device '/dev/sdh' - Invalid argument > ## Comment: OK, we give up. Try to recreate btrfs from scratch > mkfs.btrfs -m raid1 -d raid1 /dev/sdg /dev/sdh /dev/sdl > WARNING! - Btrfs Btrfs v0.19 IS EXPERIMENTAL > WARNING! - see http://btrfs.wiki.kernel.org before using > /dev/sdh is mounted > ## Comment: Of course /dev/sdh is not mounted. > mount |grep /dev/sdh > root@lxc:~# Two days ago I've seen a perhaps similar problem with an ext3 disk: Connected via USB running fsck (with some options) mount /dev/$disk $mountpoint also told that the disk was mounted (it wasn't really). Maybe some other operation (like - in this case - fsck) on/with the disk leads to this wrong message. Viele Gruesse! Helmut -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3/3] Add snappy interface to crypto API
From: Andi Kleen Mainly so that ubifs can use it. Snappy is a better compressor in the same niche as LZO. Only lightly tested so far. Experiences welcome. Cc: herb...@gondor.apana.org.au Cc: dedeki...@gmail.com Cc: adrian.hun...@intel.com Signed-off-by: Andi Kleen --- crypto/Kconfig |9 + crypto/Makefile |1 + crypto/snappy.c | 99 +++ 3 files changed, 109 insertions(+), 0 deletions(-) create mode 100644 crypto/snappy.c diff --git a/crypto/Kconfig b/crypto/Kconfig index ae27b753..2c85991 100644 --- a/crypto/Kconfig +++ b/crypto/Kconfig @@ -823,6 +823,15 @@ config CRYPTO_LZO help This is the LZO algorithm. +config CRYPTO_SNAPPY + tristate "Snappy compression algorithm" + select CRYPTO_ALGAPI + select SNAPPY + help + snappy is a faster alternative to the lzo compression algorithm + with comparable compression. It is very fast on 64bit systems, but also + good on 32bit systems. It especially excels at already compressed data. + comment "Random Number Generation" config CRYPTO_ANSI_CPRNG diff --git a/crypto/Makefile b/crypto/Makefile index ce5a813..cf92f6f 100644 --- a/crypto/Makefile +++ b/crypto/Makefile @@ -80,6 +80,7 @@ obj-$(CONFIG_CRYPTO_MICHAEL_MIC) += michael_mic.o obj-$(CONFIG_CRYPTO_CRC32C) += crc32c.o obj-$(CONFIG_CRYPTO_AUTHENC) += authenc.o authencesn.o obj-$(CONFIG_CRYPTO_LZO) += lzo.o +obj-$(CONFIG_CRYPTO_SNAPPY) += snappy.o obj-$(CONFIG_CRYPTO_RNG2) += rng.o obj-$(CONFIG_CRYPTO_RNG2) += krng.o obj-$(CONFIG_CRYPTO_ANSI_CPRNG) += ansi_cprng.o diff --git a/crypto/snappy.c b/crypto/snappy.c new file mode 100644 index 000..2f44d30 --- /dev/null +++ b/crypto/snappy.c @@ -0,0 +1,99 @@ +/* + * Cryptographic API. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published by + * the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; if not, write to the Free Software Foundation, Inc., 51 + * Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + * + */ + +#include +#include +#include +#include +#include + +struct snappy_ctx { + struct snappy_env env; +}; + +/* Only needed for compression actually */ +static int snappy_init(struct crypto_tfm *tfm) +{ + struct snappy_ctx *ctx = crypto_tfm_ctx(tfm); + + return snappy_init_env(&ctx->env); +} + +static void snappy_exit(struct crypto_tfm *tfm) +{ + struct snappy_ctx *ctx = crypto_tfm_ctx(tfm); + + snappy_free_env(&ctx->env); +} + +static int snp_compress(struct crypto_tfm *tfm, const u8 *src, + unsigned int slen, u8 *dst, unsigned int *dlen) +{ + struct snappy_ctx *ctx = crypto_tfm_ctx(tfm); + size_t olen; + int err; + + /* very pessimistic. check in snappy? */ + if (*dlen < snappy_max_compressed_length(*dlen)) + return -EINVAL; + err = snappy_compress(&ctx->env, src, slen, dst, &olen); + *dlen = olen; + return err; +} + +static int snp_decompress(struct crypto_tfm *tfm, const u8 *src, + unsigned int slen, u8 *dst, unsigned int *dlen) +{ + size_t ulen; + + if (!snappy_uncompressed_length(src, slen, &ulen)) + return -EIO; + if (*dlen < ulen) + return -EINVAL; + *dlen = ulen; + return snappy_uncompress(src, slen, dst) ? 0 : -EIO; +} + +static struct crypto_alg alg = { + .cra_name = "snappy", + .cra_flags = CRYPTO_ALG_TYPE_COMPRESS, + .cra_ctxsize= sizeof(struct snappy_ctx), + .cra_module = THIS_MODULE, + .cra_list = LIST_HEAD_INIT(alg.cra_list), + .cra_init = snappy_init, + .cra_exit = snappy_exit, + .cra_u = { .compress = { + .coa_compress = snp_compress, + .coa_decompress = snp_decompress } } +}; + +static int __init snappy_mod_init(void) +{ + return crypto_register_alg(&alg); +} + +static void __exit snappy_mod_fini(void) +{ + crypto_unregister_alg(&alg); +} + +module_init(snappy_mod_init); +module_exit(snappy_mod_fini); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Snappy Compression Algorithm"); -- 1.7.4.4 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/3] BTRFS: Add snappy support
From: Andi Kleen Add support in btrfs for snappy compression. This is based on the lzo code with minor modifications. The btrfs glue code could be significantly improved over LZO by exploiting some snappy features, but hasn't so far. Open: implement scatter-gather support and get rid of the temporary buffers. Some performance numbers (thanks to Jacob Sowles for running them) bonnie++, Core i7-64bit block output rewrite block input random K/sec K/secK/sec K/sec None 100% 100% 100%100% zlib+6.3%+5.4%+6.6% +11.6% lzo+11.5%+4.6%+12.4% +6.7% snappy +19.3%+28.1% +32.6% +9.3% Snappy does extremly well on the 64bit architecture, outperforming everything else, sometimes with a healthy margin. bonnie++, Atom-32bit block output rewrite block input random K/sec K/secK/sec K/sec None 100% 100% 100%100% zlib -43.1%-24.2% -19.0% +12.0% lzo+0.8% +2.6%+6.8% +14.8% snappy +19.5%+16.2% +24.0% +15.7% zlib does very poorly on Atom, actually degrading performance. snappy is generally faster or similar to LZO. The difference is not as big as on the 64bit CPU though, but still visible. bonnie++, files, Core-i7-64bit sequential createdelete random create delete files/sec None 100% 100% 100% 100% zlib +8.3% +10.5%+10.3% +1.4% lzo+3.8% +3.3% +5.4% -3.89% snappy +23.7%+37.2%+21.8% +23.8% bonnie++, files, Atom-32bit sequential createdelete random create delete files/sec None 100% 100% 100% 100% zlib +3.0% +7.9% +5.2% +5.1% lzo+8.2% +5.9% +4.8% +4.6% snappy +3.1% +8.5% +5.7% +1.3% Creation/Deletion on Atom is a case where snappy loses to LZO, however the loss is small. On 64bit Core it's a win. I should add that these benchmarks mainly use 0 filled IO, however FFSB was also quickly tested with more random data and the differences were similar. See also the micro benchmarks in the algorithm description for the behaviour with different data types. FFSB, 4 threads, stair case data pattern, Reads MB/s Core i7-64bit Atom-32bit MB/s MB/s None 100% 100% zlib +8.0% +4.2% lzo+9.3% +4.8% snappy +12.4% +7.9% In general snappy is a better replacement for LZO, especially on 64bit, but even on 32bit. Cc: chris.ma...@oracle.com Signed-off-by: Andi Kleen --- fs/btrfs/Kconfig |1 + fs/btrfs/Makefile |3 +- fs/btrfs/compression.c |1 + fs/btrfs/compression.h |1 + fs/btrfs/ctree.h |9 +- fs/btrfs/disk-io.c |2 + fs/btrfs/ioctl.c |4 + fs/btrfs/snappy.c | 435 fs/btrfs/super.c |9 +- lib/Makefile |1 + 10 files changed, 461 insertions(+), 5 deletions(-) create mode 100644 fs/btrfs/snappy.c diff --git a/fs/btrfs/Kconfig b/fs/btrfs/Kconfig index ecb9fd3..d55df9c 100644 --- a/fs/btrfs/Kconfig +++ b/fs/btrfs/Kconfig @@ -6,6 +6,7 @@ config BTRFS_FS select ZLIB_DEFLATE select LZO_COMPRESS select LZO_DECOMPRESS + select SNAPPY help Btrfs is a new filesystem with extents, writable snapshotting, support for multiple devices and many more features. diff --git a/fs/btrfs/Makefile b/fs/btrfs/Makefile index 40e6ac0..7cd86e7 100644 --- a/fs/btrfs/Makefile +++ b/fs/btrfs/Makefile @@ -7,6 +7,7 @@ btrfs-y += super.o ctree.o extent-tree.o print-tree.o root-tree.o dir-item.o \ extent_map.o sysfs.o struct-funcs.o xattr.o ordered-data.o \ extent_io.o volumes.o async-thread.o ioctl.o locking.o orphan.o \ export.o tree-log.o free-space-cache.o zlib.o lzo.o \ - compression.o delayed-ref.o relocation.o delayed-inode.o scrub.o + compression.o delayed-ref.o relocation.o delayed-inode.o scrub.o \ + snappy.o btrfs-$(CONFIG_BTRFS_FS_POSIX_ACL) += acl.o diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c index 8ec5d86..b171858 100644 --- a/fs/btrfs/compression.c +++ b/fs/btrfs/compression.c @@ -729,6 +729,7 @@ static wait_queue_head_t comp_workspace_wait[BTRFS_COMPRESS_TYPES]; struct btrfs_compress_op *btrfs_compress_op[] = { &btrfs_zlib_compress, &btrfs_lzo_compress, + &btrfs_snappy_compress, }; int __init btrfs_init_compress(void) diff --git a/fs/btrfs/compression.h b/fs/btrfs/compression.h index a12059f..971a425 100644 --- a/fs/btrfs/compression.h +++ b/fs/btrfs/compression.h @@ -79,5 +79,6 @@ struct btrfs_compress_op { extern struct btrfs_compress_op btrfs_zlib_compress; extern struct btrfs_compress_op btrfs_lzo_compress; +extern struct btrfs_compress_op btrfs_snappy_compress; #endif di
[PATCH 1/3] Add the snappy-c compressor to lib
From: Andi Kleen This is a C port of the google snappy compressor. It has roughly comparable compression to LZO, but is significantly faster on many file types. For example it beats all other compressors on already compressed data. I ported the original C++ code over to C and did some changes to make it better fit into the kernel. It preallocates the worst case memory consumption now. While the code being larger than lzo it is still reasonable (about 5K on x86). Decompression needs very little memory, Compression currently 192K on 64bit and 128K on 32bit. For comparison LZO compression needs 128K on 64bit and 64K on 32bit. [This could be lowered significantly by not preallocating for most use cases, typically the footprint is much lower. The original C++ version only allocated most of this when (rarely) needed, but this is more problematic in the kernel] There are some minor divergences from the Linux coding standards: in particular I kept the C++/C99 style mixed statement/declarations. This was mainly to not diverge too much from the reference C++ source, so that improvements from there can be easily ported. There are some other left overs from the google style, but very little now. checkpatch.pl has some complaints, but they are either caused by the above or checkpatch.pl bugs (it misparsed #define foo do {} while(0)) Performance: The compressor performs best on 64bit-LE systems, but is also quite good on 32bit. I haven't tested BE, but I don't expect that to add a lot of overhead. Here is some performance data (32bit, Nehalem): c/b = cycles/byte; lower numbers are better. x86-64 executable: (compression minimally slower than qlz, but much better at decompression, lzo is left in the dust): snappy: emacs-gtk: 11007968 b: ratio 0.38: comp 8.13 uncomp 2.65 c/b lzo : emacs-gtk: 11007968 b: ratio 0.33: comp 12.74 uncomp 4.70 c/b zlib1 : emacs-gtk: 11007968 b: ratio 0.27: comp 49.96 uncomp 13.14 c/b zlib3 : emacs-gtk: 11007968 b: ratio 0.26: comp 64.17 uncomp 12.33 c/b lzf : emacs-gtk: 11007968 b: ratio 0.37: comp 9.85 uncomp 4.33 c/b qlz : emacs-gtk: 11007968 b: ratio 0.34: comp 7.51 uncomp 6.28 c/b fastlz: emacs-gtk: 11007968 b: ratio 0.37: comp 10.73 uncomp 4.97 c/b Compressed data (beats everything else): snappy: udev-151.tar.gz: 634842 b: ratio 1.00: comp 0.99 uncomp 0.33 c/b lzo : udev-151.tar.gz: 634842 b: ratio 1.00: comp 41.44 uncomp 0.66 c/b zlib1 : udev-151.tar.gz: 634842 b: ratio 1.00: comp 116.99 uncomp 3.94 c/b zlib3 : udev-151.tar.gz: 634842 b: ratio 1.00: comp 117.68 uncomp 3.94 c/b lzf : udev-151.tar.gz: 634842 b: ratio 1.03: comp 16.32 uncomp 1.14 c/b qlz : udev-151.tar.gz: 634842 b: ratio 1.00: comp 10.42 uncomp 0.42 c/b fastlz: udev-151.tar.gz: 634842 b: ratio 1.03: comp 19.35 uncomp 2.07 c/b Text file (compression somewhat slower than qlz, but decompression much better, lzo is much worse): snappy: manual.txt: 445343 b: ratio 0.47: comp 12.01 uncomp 3.12 c/b lzo : manual.txt: 445343 b: ratio 0.44: comp 16.32 uncomp 7.53 c/b zlib1 : manual.txt: 445343 b: ratio 0.35: comp 56.37 uncomp 15.59 c/b zlib3 : manual.txt: 445343 b: ratio 0.31: comp 73.45 uncomp 13.99 c/b lzf : manual.txt: 445343 b: ratio 0.46: comp 13.43 uncomp 5.47 c/b qlz : manual.txt: 445343 b: ratio 0.44: comp 9.16 uncomp 8.19 c/b fastlz: manual.txt: 445343 b: ratio 0.46: comp 14.22 uncomp 7.28 c/b As you can see snappy is a good all-around compressor. On 64bit the compression is even faster and beats everything else easily: snappy: emacs-gtk: 11007968 b: ratio 0.38: comp 4.90 uncomp 2.65 c/b lzo : emacs-gtk: 11007968 b: ratio 0.33: comp 11.24 uncomp 4.46 c/b zlib1 : emacs-gtk: 11007968 b: ratio 0.27: comp 41.67 uncomp 11.13 c/b zlib3 : emacs-gtk: 11007968 b: ratio 0.26: comp 51.80 uncomp 10.54 c/b lzf : emacs-gtk: 11007968 b: ratio 0.37: comp 8.79 uncomp 4.05 c/b qlz : emacs-gtk: 11007968 b: ratio 0.34: comp 5.44 uncomp 5.46 c/b fastlz: emacs-gtk: 11007968 b: ratio 0.37: comp 9.91 uncomp 4.77 c/b On 64bit it's now nearly as fast as qlz on the text file too: snappy: manual.txt: 445343 b: ratio 0.47: comp 7.79 uncomp 3.47 c/b lzo : manual.txt: 445343 b: ratio 0.44: comp 15.46 uncomp 7.27 c/b zlib1 : manual.txt: 445343 b: ratio 0.35: comp 45.79 uncomp 12.78 c/b zlib3 : manual.txt: 445343 b: ratio 0.31: comp 60.52 uncomp 11.72 c/b lzf : manual.txt: 445343 b: ratio 0.46: comp 12.62 uncomp 5.30 c/b qlz : manual.txt: 445343 b: ratio 0.44: comp 6.81 uncomp 7.65 c/b fastlz: manual.txt: 445343 b: ratio 0.46: comp 13.75 uncomp 6.52 c/b Overall it's a good alternative to lzo, with the only drawback being the somewhat higher memory use. The memory use can be fixed for most cases (e.g. some of the current buffers are only used for SG), but isn't yet in this version. Open: it's pretty easy to add scatter-gather support since input/output is quite abstracted. This would benefit some users who could avoid temporary buffers. Signed-off-by: Andi Kleen --- include/linux/snappy.h | 26 + lib/
BTRFS thinks that a device is mounted
Hello all, What went wrong in the following scenario? BTRFS thinks that a device is mounted while it is not. BTRFS tools ver: Btrfs v0.19-101-g3891d2d Uname: 3.1.0-rc7-686-pae ## Comment: Initially we had only root (/) mounted as btrfs mount |grep btrfs /dev/sdc2 on / type btrfs (rw,autodefrag) ## Comment: Create a new btrfs fs on /dev/sdg /dev/sdh /dev/sdl mkfs.btrfs -m raid1 -d raid1 /dev/sdg /dev/sdh /dev/sdl btrfs filesystem show Label: none uuid: f0425f63-e486-4c10-adab-47a23085c8db Total devices 3 FS bytes used 28.00KB devid3 size 2.00GB used 200.00MB path /dev/sdl devid2 size 1.00GB used 200.00MB path /dev/sdh devid1 size 768.00MB used 404.00MB path /dev/sdg Label: none uuid: cc8211ba-6f62-46d6-ba17-49250f06a941 Total devices 1 FS bytes used 512.98MB devid1 size 6.09GB used 1.75GB path /dev/sdc2 ## Comment: unorthodox, but looks possible, add one of the devices from the newly created fs to root (/) btrfs device add /dev/sdh / ## Comment: Remove it back. btrfs device delete /dev/sdh / ## Comment: We ended up with "*** Some devices missing" on both devices btrfs filesystem show Label: none uuid: 5910f378-0784-4e7f-b918-de35067252e9 Total devices 3 FS bytes used 28.00KB devid3 size 2.00GB used 200.00MB path /dev/sdl devid1 size 768.00MB used 404.00MB path /dev/sdg *** Some devices missing Label: none uuid: cc8211ba-6f62-46d6-ba17-49250f06a941 Total devices 2 FS bytes used 512.59MB devid1 size 6.09GB used 1.50GB path /dev/sdc2 *** Some devices missing ## Comment: Use btrfs balance to fix root (/) btrfs filesystem balance / btrfs filesystem show Label: none uuid: 5910f378-0784-4e7f-b918-de35067252e9 Total devices 3 FS bytes used 28.00KB devid3 size 2.00GB used 200.00MB path /dev/sdl devid1 size 768.00MB used 404.00MB path /dev/sdg *** Some devices missing Label: none uuid: cc8211ba-6f62-46d6-ba17-49250f06a941 Total devices 1 FS bytes used 513.40MB devid1 size 6.09GB used 1.75GB path /dev/sdc2 ## Comment: try to mount the initially created fs on /mnt. It fails mount -t btrfs /dev/sdg /mnt mount: wrong fs type, bad option, bad superblock on /dev/sdg, missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail or so ## Comment: See if device scan can do the trick. Nope... btrfs device scan /dev/sdh Scanning for Btrfs filesystems in '/dev/sdh' ERROR: unable to scan the device '/dev/sdh' - Invalid argument ## Comment: OK, we give up. Try to recreate btrfs from scratch mkfs.btrfs -m raid1 -d raid1 /dev/sdg /dev/sdh /dev/sdl WARNING! - Btrfs Btrfs v0.19 IS EXPERIMENTAL WARNING! - see http://btrfs.wiki.kernel.org before using /dev/sdh is mounted ## Comment: Of course /dev/sdh is not mounted. mount |grep /dev/sdh root@lxc:~# Thanks, Nikos -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 5/5] _populate_fs should use OPTIND when getopts is used
Signed-off-by: Anand Jain --- common.rc |1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/common.rc b/common.rc index e3c4e67..cab0b64 100644 --- a/common.rc +++ b/common.rc @@ -1547,6 +1547,7 @@ _populate_fs() root=root # path of initial root of directory tree randomdata=false # -x data type urandom or zero +OPTIND=1 while getopts "d:f:n:r:s:v:x" c do case $c in -- 1.7.7 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3/5] 264: Functional test case for the btrfs snapshot
Create snapshots in various ways, modify the data around the block and file boundaries and verify the data integrity. Signed-off-by: Anand Jain --- 264 | 193 +++ 264.out |2 + group |1 + 3 files changed, 196 insertions(+), 0 deletions(-) create mode 100755 264 create mode 100644 264.out diff --git a/264 b/264 new file mode 100755 index 000..d655d1e --- /dev/null +++ b/264 @@ -0,0 +1,193 @@ +#!/bin/bash +# FS QA Test No. 264 +# +# Extented btrfs snapshot test cases +# +#--- +# Copyright (c) 2011 Oracle All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#--- +# +# creator +owner=anand.j...@oracle.com + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! + +_cleanup() +{ +rm -f $tmp.* +} + +trap "_cleanup ; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_need_to_be_root +_supported_fs btrfs +_supported_os Linux +_require_scratch + +_scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed" +_scratch_mount + +# Create and save sha256sum +# arg1 FS to generate sha256 +# arg2 File name to save the sha256 output +_save_checksum() +{ + local i=0 + >$2 + cd $1 + for i in `find . -type f`; do sha256sum $i >> $2; done + cd $OLDPWD +} + +# Verify the sha256sum for a FS +# arg1 FS to be tested +# arg2 sha256 file +_verify_checksum() +{ + cd $1 + [ -f $2 ] || _fail "checksum file $2 not found" + sha256sum -c $2 | grep "FAILED" + cd $OLDPWD +} + +# Create a snapshot +# arg1 dest dir +# Return snapshot name in the SNAPNAME +_create_snap() +{ + local x + [ -d $1 ] || _fail "Destination dir $1 not present" + SNAPNAME=`mktemp -u $SCRATCH_MNT/snap.XX` + btrfs subvolume snapshot $1 $SNAPNAME > /dev/null || _fail "snapshot create failed" +} + +# Reads and writes new data but does not allocate new blocks +# arg1 FS to be modified +_read_modify_write() +{ + local i + local FSIZE + for i in `find $1 -type f` + do + FSIZE=`stat -t $i | cut -d" " -f2` + dd if=$i of=/dev/null obs=$FSIZE count=1 status=noxfer 2>/dev/null & + dd if=/dev/urandom of=$i obs=$FSIZE count=1 status=noxfer 2>/dev/null & + done + wait $! +} + +# Fills the allocated blocks +# arg1 FS in question +_fill_blk() +{ + local FSIZE + local BLKS + local NBLK + local FALLOC + local WS + + for i in `find /$1 -type f` + do + FSIZE=`stat -t $i | cut -d" " -f2` + BLKS=`stat -c "%B" $i` + NBLK=`stat -c "%b" $i` + FALLOC=$(($BLKS * $NBLK)) + WS=$(($FALLOC - $FSIZE)) + dd if=/dev/urandom of=$i oseek=$FSIZE obs=$WS count=1 status=noxfer 2>/dev/null & + done + wait $! +} + + +# Append a random size to the files +# arg1 : FS in question +_append_file() +{ + local FSIZE + local X + local N + local i + N=0 + for i in `find $1 -type f` + do + if [ $N == 0 ]; then + X=$i + FSIZE=`stat -t $X | cut -d" " -f2` + dd if=$X of=$X seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer 2>/dev/null & + N=$(($N+1)) + continue + fi + FSIZE=`stat -t $i | cut -d" " -f2` + dd if=$X of=$i seek=1 bs=$FSIZE obs=$FSIZE count=1 status=noxfer 2>/dev/null & + X=$i + done + wait $! +} + +# real QA test starts here### +# sv1 - is just a name nothing spl +firstvol="$SCRATCH_MNT/sv1" +btrfs subvolume create $firstvol > /dev/null || _fail "btrfs subvolume create $firstvol failed" +dirp=`mktemp -duq $firstvol/dir.XX` +_populate_fs -n 1 -f 20 -d 10 -r $dirp -s 10 -x +SNAPNAME=0 +_create_snap $firstvol +_save_checksum $firstvol $tmp.sv1.sum +_verify_checksum $SNAPNAME $tmp.sv1.sum + +#Append1 the files +_fill_blk $SNAPNAME +_verify_checksum $firstvol $tmp.sv1.
[PATCH 4/5] 265: Functional test case for the btrfs raid operations
This will verify the various raid features in btrfs and device replacement functionality. Signed-off-by: Anand Jain --- 265 | 187 +++ 265.out |2 + group |1 + 3 files changed, 190 insertions(+), 0 deletions(-) create mode 100755 265 create mode 100644 265.out diff --git a/265 b/265 new file mode 100755 index 000..10f214f --- /dev/null +++ b/265 @@ -0,0 +1,187 @@ +#! /bin/bash +# FS QA Test No. 265 +# +# btrfs vol tests +# +#--- +# Copyright (c) 2011 Oracle. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#--- +# +# creator +owner=anand.j...@oracle.com + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +dev_removed=0 +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ +cd / +rm -f $tmp.* +if [ $dev_removed == 1 ]; then +umount $SCRATCH_MNT +_devmgt_add "${DEVHTL}" +fi +} + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_need_to_be_root +_supported_fs btrfs +_supported_os Linux +_require_scratch +_require_scratch_dev_pool +_require_deletable_scratch_dev_pool + +# Test cases related to raid in btrfs +_test_raid0() +{ + export MKFS_OPTIONS="-m raid0 -d raid0" + _scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed" + _scratch_mount + dirp=`mktemp -duq $SCRATCH_MNT/dir.XX` + _populate_fs -n 1 -f 20 -d 10 -r $dirp -s 10 + umount $SCRATCH_MNT +} + +_test_raid1() +{ + export MKFS_OPTIONS="-m raid1 -d raid1" + _scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed" + _scratch_mount + dirp=`mktemp -duq $SCRATCH_MNT/dir.XX` + _populate_fs -n 1 -f 20 -d 10 -r $dirp -s 10 + umount $SCRATCH_MNT +} + +_test_raid10() +{ + export MKFS_OPTIONS="-m raid10 -d raid10" + _scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed" + _scratch_mount + dirp=`mktemp -duq $SCRATCH_MNT/dir.XX` + _populate_fs -n 1 -f 20 -d 10 -r $dirp -s 10 + umount $SCRATCH_MNT +} + +_test_single() +{ + export MKFS_OPTIONS="-m single -d single" + _scratch_mkfs $SCRATCH_DEV_POOL > /dev/null 2>&1 || _fail "mkfs failed" + _scratch_mount + dirp=`mktemp -duq $SCRATCH_MNT/dir.XX` + _populate_fs -n 1 -f 20 -d 10 -r $dirp -s 10 + umount $SCRATCH_MNT +} + +_test_add() +{ + local i + local devs[]="( $SCRATCH_DEV_POOL )" + local n=${#devs[@]} + + n=$(($n-1)) + + export MKFS_OPTIONS="" + _scratch_mkfs > /dev/null 2>&1 || _fail "mkfs failed" + _scratch_mount + dirp=`mktemp -duq $SCRATCH_MNT/dir.XX` + _populate_fs -n 1 -f 20 -d 10 -r $dirp -s 10 + for i in `seq 1 $n`; do + btrfs device add ${devs[$i]} $SCRATCH_MNT > /dev/null 2>&1 || _fail "device add failed" + done + btrfs filesystem balance $SCRATCH_MNT || _fail "balance failed" + umount $SCRATCH_MNT +} + +_test_replace() +{ + local i + local devs=( $SCRATCH_DEV_POOL ) + local n=${#devs[@]} + local ds + local d + local DEVHTL="" + + # exclude the last disk in the disk pool + n=$(($n-1)) + ds=${devs[@]:0:$n} + + export MKFS_OPTIONS="-m raid1 -d raid1" + _scratch_mkfs "$ds" > /dev/null 2>&1 || _fail "tr: mkfs failed" + _scratch_mount + dirp=`mktemp -duq $SCRATCH_MNT/dir.XX` + _populate_fs -n 1 -f 20 -d 10 -r $dirp -s 10 + + #pick the 2nd last disk + ds=${devs[@]:$(($n-1)):1} + + # retrive the HTL for this scsi disk + d=`echo $ds|cut -d"/" -f3` + DEVHTL=`ls -l /sys/class/block/${d} | rev | cut -d "/" -f 3 | rev` + + #fail disk + _devmgt_remove ${DEVHTL} + dev_removed=1 + + btrfs fi show $SCRATCH_DEV | grep "Some devices missing" > /dev/null || _fail \ + "btrfs did not report device missing" + + # add a new disk to btrfs + ds=${devs[@]:$(($n)):1} + btrfs device add ${ds} $SCRATCH_MNT > /dev/null 2>&1 || _fail "dev add failed" + #
[PATCH 2/5] Added SCRATCH_DEV_POOL to specify multiple disks for the btrfs RAID
SCRATCH_DEV takes single disk as the scratch place for testing. New SCRATCH_DEV_POOL can used to specify multiple disks for the scratch btrfs filesystem. Using SCRATCH_DEV and or SCRATCH_DEV_POOL will follow the following logic. btrfs FS OR any FS SCRATCH_DEV_POOL is unset and SCRATCH_DEV is set . test-case with _require_scratch_dev_pool will not run . test-case without _require_scratch_dev_pool will run SCRATCH_DEV_POOL is set and SCRATCH_DEV is unset . test-case with _require_scratch_dev_pool - runs only if FSTYP=btrfs . test-case without _require_scratch_dev_pool will run using first dev in the SCRATCH_DEV_POOL as a SCRATCH_DEV - if FSTYP=btrfs it includes SCRATCH_DEV_POOL disks to the FS - if FSTYP=non-btrfs SCRATCH_DEV_POOL is ignored SCRATCH_DEV_POOL is set and SCRATCH_DEV is set . reports error in the config SCRATCH_DEV_POOL is unset and SCRATCH_DEV is unset . no change Signed-off-by: Anand Jain --- README| 16 +--- common.config | 11 common.rc | 76 - 3 files changed, 98 insertions(+), 5 deletions(-) diff --git a/README b/README index 5367be6..0f52f02 100644 --- a/README +++ b/README @@ -34,14 +34,22 @@ Preparing system for tests (IRIX and Linux): - leave empty and expect this partition to be clobbered by some tests. If this is not provided, many tests will not be run. - -(these must be two DIFFERENT partitions) + (SCRATCH and TEST must be two DIFFERENT partitions) + OR +- for btrfs only: some btrfs test cases will need 3 or more independent + SCRATCH disks which should be set using SCRATCH_DEV_POOL (for eg: + SCRATCH_DEV_POOL="/dev/sda /dev/sdb /dev/sdc") with which + SCRATCH_DEV should be unused by the tester, and for the legacy + support SCRATCH_DEV will be set to the first disk of the + SCRATCH_DEV_POOL by xfstests script. - setup your environment - setenv TEST_DEV "device containing TEST PARTITION" - setenv TEST_DIR "mount point of TEST PARTITION" - optionally: - - setenv SCRATCH_DEV "device containing SCRATCH PARTITION" + - setenv SCRATCH_DEV "device containing SCRATCH PARTITION" OR + (btrfs only) setenv SCRATCH_DEV_POOL "to 3 or more SCRATCH disks for + testing btrfs raid concepts" - setenv SCRATCH_MNT "mount point for SCRATCH PARTITION" - setenv TAPE_DEV "tape device for testing xfsdump" - setenv RMT_TAPE_DEV "remote tape device for testing xfsdump" @@ -63,7 +71,7 @@ Preparing system for tests (IRIX and Linux): tape which can be overwritten. - make sure $TEST_DEV is a mounted XFS partition -- make sure that $SCRATCH_DEV contains nothing useful +- make sure that $SCRATCH_DEV or $SCRATCH_DEV_POOL contains nothing useful Running tests: diff --git a/common.config b/common.config index e94624e..1df2bbd 100644 --- a/common.config +++ b/common.config @@ -229,6 +229,17 @@ if [ ! -d "$TEST_DIR" ]; then exit 1 fi +# a btrfs tester will set only SCRATCH_DEV_POOL, we will put first of its dev +# to SCRATCH_DEV and rest to SCRATCH_DEV_POOL to maintain the backward compatibility +if [ ! -z "$SCRATCH_DEV_POOL" ]; then +if [ ! -z "$SCRATCH_DEV" ]; then +echo "common.config: Error: \$SCRATCH_DEV should be unset when \$SCRATCH_DEV_POOL is set" +exit 1 +fi +SCRATCH_DEV=`echo $SCRATCH_DEV_POOL | awk '{print $1}'` +SCRATCH_DEV_POOL=`echo $SCRATCH_DEV_POOL | awk '{ for (i = 2; i <= NF; i++) print $i}'` +fi + echo $SCRATCH_DEV | grep -q ":" > /dev/null 2>&1 if [ ! -z "$SCRATCH_DEV" -a ! -b "$SCRATCH_DEV" -a "$?" != "0" ]; then echo "common.config: Error: \$SCRATCH_DEV ($SCRATCH_DEV) is not a block device or a NFS filesystem" diff --git a/common.rc b/common.rc index 37379a5..e3c4e67 100644 --- a/common.rc +++ b/common.rc @@ -1499,7 +1499,7 @@ _nfiles() file=f$f echo > $file if [ $size -gt 0 ]; then -if [ $2 == false ]; then +if [ "$2" == "false" ]; then dd if=/dev/zero of=$file bs=1024 count=$size 2>&1 | _filter_dd else dd if=/dev/urandom of=$file bs=1024 count=$size 2>&1 | _filter_dd @@ -1598,6 +1598,80 @@ _test_inode_extsz() echo $blocks } +# scratch_dev_pool should contain the disks pool for the btrfs raid +_require_scratch_dev_pool() +{ + local i + if [ -z "$SCRATCH_DEV_POOL" ]; then + _notrun "this test requires a valid \$SCRATCH_DEV_POOL" + fi + + # btrfs test case needs 2 or more scratch_dev_pool; other FS not sure +
[PATCH 1/5] fill files with random data
snapshot data integrity test-case needs filesystem with random data. Signed-off-by: Anand Jain --- common.rc | 21 ++--- 1 files changed, 14 insertions(+), 7 deletions(-) diff --git a/common.rc b/common.rc index e948169..37379a5 100644 --- a/common.rc +++ b/common.rc @@ -1490,6 +1490,7 @@ _die() exit 1 } +#takes files, randomdata _nfiles() { f=0 @@ -1498,20 +1499,24 @@ _nfiles() file=f$f echo > $file if [ $size -gt 0 ]; then -dd if=/dev/zero of=$file bs=1024 count=$size +if [ $2 == false ]; then +dd if=/dev/zero of=$file bs=1024 count=$size 2>&1 | _filter_dd +else +dd if=/dev/urandom of=$file bs=1024 count=$size 2>&1 | _filter_dd +fi fi let f=$f+1 done } -# takes dirname, depth +# takes dirname, depth, randomdata _descend() { -dirname=$1; depth=$2 +dirname=$1; depth=$2; randomdata=$3 mkdir $dirname || die "mkdir $dirname failed" cd $dirname -_nfiles $files # files for this dir +_nfiles $files $randomdata # files for this dir and data type [ $depth -eq 0 ] && return let deep=$depth-1 # go 1 down @@ -1529,7 +1534,7 @@ _descend() # Populate a filesystem with inodes for performance experiments # -# usage: populate [-v] [-n ndirs] [-f nfiles] [-d depth] [-r root] [-s size] +# usage: populate [-v] [-n ndirs] [-f nfiles] [-d depth] [-r root] [-s size] [-x] # _populate_fs() { @@ -1540,8 +1545,9 @@ _populate_fs() depth=2 # depth of tree from root to leaves verbose=false root=root # path of initial root of directory tree +randomdata=false # -x data type urandom or zero -while getopts "d:f:n:r:s:v" c +while getopts "d:f:n:r:s:v:x" c do case $c in d) depth=$OPTARG;; @@ -1550,10 +1556,11 @@ _populate_fs() s) size=$OPTARG;; v) verbose=true;; r) root=$OPTARG;; +x) randomdata=true;; esac done -_descend $root $depth +_descend $root $depth $randomdata wait cd $here -- 1.7.7 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/5] xfstests enhancement and bug fix
Anand Jain (5): updating to fill files with random data Added SCRATCH_DEV_POOL to specify multiple disks for the btrfs RAID 264: Functional test case for the btrfs snapshot 265: Functional test case for the btrfs raid operations _populate_fs should use OPTIND when getopts is used 264 | 193 + 264.out |2 + 265 | 187 +++ 265.out |2 + README| 16 - common.config | 11 +++ common.rc | 96 ++-- group |2 + 8 files changed, 498 insertions(+), 11 deletions(-) create mode 100755 264 create mode 100644 264.out create mode 100755 265 create mode 100644 265.out -- 1.7.7 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 3/3] 266: Functional test case for the btrfs raid operations
On 19/10/2011 17:45, Christoph Hellwig wrote: +if [ $dev_removed == 1 ]; then + umount $SCRATCH_MNT +_devmgt_add "${DEVHTL}" +fi messy indendation. got it. thanks. + btrfs filesystem balance $SCRATCH_MNT || _fail "balance failed" + #btrfs filesystem show $SCRATCH_DEV 2>&1 | egrep devid |awk '{ + # if ( $6 == "0.00" ) { exit 1 } + #}' || _fail "test6_add... failed" what is the point of the commented out code? otherwise looks good to me. yes that has to be removed. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2/3] 265: Functional test case for the btrfs de-fragmentation
On 19/10/2011 17:43, Christoph Hellwig wrote: On Tue, Oct 18, 2011 at 02:28:55PM +0800, Anand Jain wrote: To verify the btrfs de-fragmentation does not fail. Any reason you can't simply fold btrfs defragmentation testing into the existing common defragmentation test (218)? oh! 218 ! let me defer defrag for now. will get the other test-cases going. thanks anand -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/3] 264: Functional test case for the btrfs snapshot
comments in line. On 19/10/2011 17:42, Christoph Hellwig wrote: On Tue, Oct 18, 2011 at 02:28:54PM +0800, Anand Jain wrote: Create snapshots in various ways, modify the data around the block and file boundaries and verify the data integrity. The test itselt looks good enough, but I have some comments on the pool infrastructure changes. I also think they should probably be a separate preparatory patch, or at least documented in the changelog as well. index 5367be6..7c135c7 100644 --- a/README +++ b/README @@ -36,12 +36,17 @@ Preparing system for tests (IRIX and Linux): not be run. (these must be two DIFFERENT partitions) + +- for btrfs only: some tests would need 3 or more independent SCRATCH disks, + which should be setenv SCRATCH_DEV_POOL instead of SCRATCH_DEV + - setup your environment - setenv TEST_DEV "device containing TEST PARTITION" - setenv TEST_DIR "mount point of TEST PARTITION" - optionally: - setenv SCRATCH_DEV "device containing SCRATCH PARTITION" + - setenv SCRATCH_DEV_POOL "pool of SCRATCH disks for testing btrfs" How does one find out what the pool name is? You'll also need to document how to create the pool from disks. agreed. @@ -229,6 +229,20 @@ if [ ! -d "$TEST_DIR" ]; then exit 1 fi +# a btrfs tester will set only SCRATCH_DEV_POOL, we will put first of its dev +# to SCRATCH_DEV and rest to SCRATCH_DEV_POOL to maintain the backward compatibility +if [ "$HOSTOS" == "Linux" ]; then +FSTYP_tmp=`blkid -c /dev/null -s TYPE -o value $TEST_DEV` +else +FSTYP_tmp=xfs +fi Why do we need a second FSTYP detection? If the existing one isn't early enough make sure it's done early enough instead of duplicating it. looks like its ok not to have FSTYP checked here, it will follow the following logic.. btrfs FS OR any FS SCRATCH_DEV_POOL is unset and SCRATCH_DEV is set . test-case with _require_scratch_dev_pool will not run . test-case without _require_scratch_dev_pool will run SCRATCH_DEV_POOL is set and SCRATCH_DEV is unset . test-case with _require_scratch_dev_pool - runs only if FSTYP=btrfs . test-case without _require_scratch_dev_pool will run using first dev in the SCRATCH_DEV_POOL as a SCRATCH_DEV - if FSTYP=btrfs it includes SCRATCH_DEV_POOL disks to the FS - if FSTYP=non-btrfs SCRATCH_DEV_POOL is ignored SCRATCH_DEV_POOL is set and SCRATCH_DEV is set . reports error in the config SCRATCH_DEV_POOL is unset and SCRATCH_DEV is unset . no change --- a/common.rc +++ b/common.rc @@ -1498,7 +1498,11 @@ _nfiles() file=f$f echo > $file if [ $size -gt 0 ]; then -dd if=/dev/zero of=$file bs=1024 count=$size + if [ $randomdata == false ]; then + dd if=/dev/zero of=$file bs=1024 count=$size 2>&1 | _filter_dd + else + dd if=/dev/urandom of=$file bs=1024 count=$size 2>&1 | _filter_dd + fi I'd rather see the randomdata flag passed down explicitly to _descend and _nfiles rather than setting a magic environment variable. makes sense. added. @@ -1508,7 +1512,11 @@ _nfiles() _descend() { dirname=$1; depth=$2 -mkdir $dirname || die "mkdir $dirname failed" + if [ -d $dirname ]; then + dirname=`mktemp -dq $dirname/dir.XX` + else + mkdir $dirname || die "mkdir $dirname failed" + fi Why would the directory here already exist? This at least needs very good documentation. Also the indentation seems off compared to the surrounding code. Hmm. Changing this back to the original design - where calling function has to ensure a new directory name which does not exists. @@ -1550,6 +1559,7 @@ _populate_fs() s) size=$OPTARG;; v) verbose=true;; r) root=$OPTARG;; + x) randomdata=true;; indendation is off again. oh! thks for pointing. +# scratch_dev_pool should contain the disks pool for the btrfs raid +_require_scratch_dev_pool() +{ + local i + case "$FSTYP" in + btrfs) + if [ -z "$SCRATCH_DEV_POOL" ] + then For new code I'd generally prefer the more readable if [ ... ]; then although the above form unfortunately still is fairly common in xfsprogs. got it. +# We will check if the device is virtual (eg: loop device) since it does not +# have the delete entry-point. Otherwise SCSI and USB devices are fine. +_require_deletable_scratch_dev_pool() +{ + local i + local x + for i in $SCRATCH_DEV_POOL; do + x=`echo $i | cut -d"/" -f 3` + ls -l /sys/class/block/${x} | grep -q "virtual" + if [ $? == "0" ]; then + _notrun "$i is a
Re: WARNING: at fs/btrfs/inode.c:2114
2011/10/20 Liu Bo : > On 10/17/2011 11:23 PM, Christian Brunner wrote: >> 2011/10/11 Christian Brunner : >> >> I have updated to a 3.0.6 kernel, with all the btrfs patches from >> josef's git repo this weekend. But I'm still seeing the following >> warning: >> > > Hi, > > Would you try with this patch: > > http://permalink.gmane.org/gmane.comp.file-systems.btrfs/13728 > I have now applied the patch josef sent to the list (Btrfs: use the global reserve when truncating the free space cache inode), but the warning is still there: [ 69.153400] [ cut here ] [ 69.158669] WARNING: at fs/btrfs/inode.c:2114 btrfs_orphan_commit_root+0xb0/0xc0 [btrfs]() [ 69.167984] Hardware name: ProLiant DL180 G6 [ 69.173037] Modules linked in: btrfs zlib_deflate libcrc32c sunrpc bonding ipv6 sg serio_raw pcspkr ghes hed iTCO_wdt iTCO_vendor_support ixgbe dca mdio i7core_edac edac_core iomemory_vsl(P) hpsa squashfs [last unloaded: scsi_wait_scan] [ 69.197502] Pid: 3426, comm: ceph-osd Tainted: P 3.0.6-1.fits.8.el6.x86_64 #1 [ 69.206591] Call Trace: [ 69.209389] [] warn_slowpath_common+0x7f/0xc0 [ 69.216144] [] warn_slowpath_null+0x1a/0x20 [ 69.222647] [] btrfs_orphan_commit_root+0xb0/0xc0 [btrfs] [ 69.230550] [] commit_fs_roots+0xc5/0x1b0 [btrfs] [ 69.237698] [] ? mutex_lock+0x31/0x60 [ 69.243707] [] ? btrfs_free_path+0x2a/0x40 [btrfs] [ 69.250966] [] btrfs_commit_transaction+0x3c6/0x820 [btrfs] [ 69.259087] [] ? wait_current_trans+0x28/0x110 [btrfs] [ 69.266720] [] ? join_transaction+0x25/0x250 [btrfs] [ 69.274136] [] ? wake_up_bit+0x40/0x40 [ 69.280210] [] btrfs_sync_fs+0x59/0xd0 [btrfs] [ 69.287072] [] btrfs_ioctl+0x495/0xd50 [btrfs] [ 69.293922] [] ? inode_has_perm+0x30/0x40 [ 69.300286] [] ? file_has_perm+0xdc/0xf0 [ 69.306558] [] do_vfs_ioctl+0x9a/0x5a0 [ 69.312657] [] sys_ioctl+0xa1/0xb0 [ 69.318352] [] system_call_fastpath+0x16/0x1b [ 69.325107] ---[ end trace 2fd1a5665203d8e3 ]--- Thanks, Christian -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Btrfs: use the global reserve when truncating the free space cache inode
We no longer use the orphan block rsv for holding the reservation for truncating the inode, so instead use the global block rsv and check to make sure it has enough space for us to truncate the space. Thanks, Signed-off-by: Josef Bacik --- fs/btrfs/free-space-cache.c | 22 +- 1 files changed, 17 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c index de205d5..95ada1c 100644 --- a/fs/btrfs/free-space-cache.c +++ b/fs/btrfs/free-space-cache.c @@ -198,14 +198,24 @@ int btrfs_truncate_free_space_cache(struct btrfs_root *root, struct inode *inode) { struct btrfs_block_rsv *rsv; + u64 needed_bytes; loff_t oldsize; int ret = 0; rsv = trans->block_rsv; - trans->block_rsv = root->orphan_block_rsv; - ret = btrfs_block_rsv_check(root, root->orphan_block_rsv, 5); - if (ret) - return ret; + trans->block_rsv = &root->fs_info->global_block_rsv; + + /* 1 for slack space, 1 for updating the inode */ + needed_bytes = btrfs_calc_trunc_metadata_size(root, 1) + + btrfs_calc_trans_metadata_size(root, 1); + + spin_lock(&trans->block_rsv->lock); + if (trans->block_rsv->reserved < needed_bytes) { + spin_unlock(&trans->block_rsv->lock); + trans->block_rsv = rsv; + return -ENOSPC; + } + spin_unlock(&trans->block_rsv->lock); oldsize = i_size_read(inode); btrfs_i_size_write(inode, 0); @@ -218,13 +228,15 @@ int btrfs_truncate_free_space_cache(struct btrfs_root *root, ret = btrfs_truncate_inode_items(trans, root, inode, 0, BTRFS_EXTENT_DATA_KEY); - trans->block_rsv = rsv; if (ret) { + trans->block_rsv = rsv; WARN_ON(1); return ret; } ret = btrfs_update_inode(trans, root, inode); + trans->block_rsv = rsv; + return ret; } -- 1.7.5.2 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel BUG at fs/btrfs/inode.c:1163
Hi Anand, I changed the replication level of the rbd pool, from one to two. ceph osd pool set rbd size 2 And then during the sync the bug happened, but today I could not reproduce it. So I do not have a testcase for you. Best Regards, martin Am 19.10.2011 17:02, schrieb Anand Jain: I tried to play with ceph here and not a complete success yet. any idea what was done on the system at the time of the problem ? and any specific command that could trigger this again ? Thanks. anand -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs: add a missing block_rsv reset
On Thu, Oct 20, 2011 at 05:25:18PM +0800, Liu Bo wrote: > In commit ab1ca99b51df63901617b9f10f9a36d5d4972d78 > (Btrfs: reset to appropriate block rsv after orphan operations), > we miss a block_rsv reset and this sometimes leads us to > the WARNING of btrfs_orphan_commit_root(). > > Signed-off-by: Liu Bo Actually this needs to be converted over to the proper way to truncate things. I'll write that up and send a patch. Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: WARNING: at fs/btrfs/inode.c:2114
On 10/17/2011 11:23 PM, Christian Brunner wrote: > 2011/10/11 Christian Brunner : >> 2011/10/11 Liu Bo : >>> On 10/10/2011 12:41 AM, Christian Brunner wrote: I just realized that this is still the same warning I reported some month ago. I thought that this had been fixed with 25d37af374263243214be9d912cbb46a8e469bc7 which is included in the kernel I'm using. So I think there must be another Problem. >>> Would you try with this patch: >>> >>> http://marc.info/?l=linux-btrfs&m=131547325515336&w=2 >>> >> This one is already included in my tree. > > I have updated to a 3.0.6 kernel, with all the btrfs patches from > josef's git repo this weekend. But I'm still seeing the following > warning: > Hi, Would you try with this patch: http://permalink.gmane.org/gmane.comp.file-systems.btrfs/13728 thanks, liubo > [75532.763336] [ cut here ] > [75532.768570] WARNING: at fs/btrfs/inode.c:2114 > btrfs_orphan_commit_root+0xb0/0xc0 [btrfs]() > [75532.777807] Hardware name: ProLiant DL180 G6 > [75532.782798] Modules linked in: btrfs zlib_deflate libcrc32c sunrpc > bonding ipv6 sg serio_raw pcspkr ghes hed iTCO_wdt iTCO_vendor_support > i7core_edac edac_core ixgbe dca mdio iomemory_vsl(P) hpsa squashfs > [last unloaded: scsi_wait_scan] > [75532.806891] Pid: 1858, comm: ceph-osd Tainted: P > 3.0.6-1.fits.5.el6.x86_64 #1 > [75532.815990] Call Trace: > [75532.818772] [] warn_slowpath_common+0x7f/0xc0 > [75532.825514] [] warn_slowpath_null+0x1a/0x20 > [75532.832076] [] btrfs_orphan_commit_root+0xb0/0xc0 > [btrfs] > [75532.840028] [] commit_fs_roots+0xc5/0x1b0 [btrfs] > [75532.847196] [] ? mutex_lock+0x31/0x60 > [75532.853198] [] ? btrfs_free_path+0x2a/0x40 [btrfs] > [75532.860476] [] > btrfs_commit_transaction+0x3c6/0x820 [btrfs] > [75532.868599] [] ? wait_current_trans+0x28/0x110 [btrfs] > [75532.876264] [] ? join_transaction+0x25/0x250 [btrfs] > [75532.883762] [] ? wake_up_bit+0x40/0x40 > [75532.889839] [] btrfs_sync_fs+0x59/0xd0 [btrfs] > [75532.896703] [] btrfs_ioctl+0x495/0xd50 [btrfs] > [75532.903544] [] ? inode_has_perm+0x30/0x40 > [75532.909902] [] ? file_has_perm+0xdc/0xf0 > [75532.916205] [] do_vfs_ioctl+0x9a/0x5a0 > [75532.922276] [] sys_ioctl+0xa1/0xb0 > [75532.927988] [] system_call_fastpath+0x16/0x1b > [75532.934755] ---[ end trace a10c532625ad12af ]--- > > Regards, > Christian > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Btrfs: add a missing block_rsv reset
In commit ab1ca99b51df63901617b9f10f9a36d5d4972d78 (Btrfs: reset to appropriate block rsv after orphan operations), we miss a block_rsv reset and this sometimes leads us to the WARNING of btrfs_orphan_commit_root(). Signed-off-by: Liu Bo --- fs/btrfs/free-space-cache.c |4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c index 41ac927..d96815c 100644 --- a/fs/btrfs/free-space-cache.c +++ b/fs/btrfs/free-space-cache.c @@ -199,8 +199,10 @@ int btrfs_truncate_free_space_cache(struct btrfs_root *root, ret = btrfs_block_rsv_check(trans, root, root->orphan_block_rsv, 0, 5); - if (ret) + if (ret) { + trans->block_rsv = rsv; return ret; + } oldsize = i_size_read(inode); btrfs_i_size_write(inode, 0); -- 1.6.5.2 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html