Re: btrfs hang on brd
On 03/06/11 12:41, Adrian Hunter wrote: No improvement on 3.0-rc1+ (commit 5c6cce92bc8aee751aafe82c5d9caf7553226a3d). And on 59c5f46fbe01a00eedf54a23789634438bb80603 Linux 3.0-rc2 btrfs quickly fails to mount due to no space e.g. Script -- #!/bin/sh sudo modprobe brd rd_size=262144 sudo umount /mnt/test/ 2> /dev/null echo 'mkfs.btrfs /dev/ram0' sudo mkfs.btrfs /dev/ram0 sudo mkdir -p /mnt/test echo 'mount -t btrfs /dev/ram0 /mnt/test' sudo mount -t btrfs /dev/ram0 /mnt/test sudo mkdir -p /mnt/test/test sudo chown $USER /mnt/test/test sudo chgrp $USER /mnt/test/test sudo umount /mnt/test full=0 i=0 while true; do sudo mount -t btrfs /dev/ram0 /mnt/test if df | grep ram0 | grep 100% > /dev/null; then full=`expr $full \+ 1` if test $full -gt 6;then rm -rf /mnt/test/test/* full=0 fi else full=0 fi fsstress -c -r -d /mnt/test/test -p 3 -n 1000 -l 10 sudo umount /mnt/test i=`expr $i \+ 1` echo $i done Script Output - mkfs.btrfs /dev/ram0 WARNING! - Btrfs Btrfs v0.19 IS EXPERIMENTAL WARNING! - see http://btrfs.wiki.kernel.org before using fs created label (null) on /dev/ram0 nodesize 4096 leafsize 4096 sectorsize 4096 size 256.00MB Btrfs Btrfs v0.19 mount -t btrfs /dev/ram0 /mnt/test 1 2 3 4 mount: No space left on device ^C Kernel messages --- [ 95.084429] brd: module loaded [ 95.368708] Btrfs loaded [ 95.369455] device fsid 9644bae59f78285c-8db3a698a8a82293 devid 1 transid 7 /dev/ram0 [ 95.370582] SELinux: initialized (dev ram0, type btrfs), uses xattr [ 95.487905] device fsid 9644bae59f78285c-8db3a698a8a82293 devid 1 transid 10 /dev/ram0 [ 95.49] SELinux: initialized (dev ram0, type btrfs), uses xattr [ 107.336869] device fsid 9644bae59f78285c-8db3a698a8a82293 devid 1 transid 2664 /dev/ram0 [ 107.338873] SELinux: initialized (dev ram0, type btrfs), uses xattr [ 115.861909] device fsid 9644bae59f78285c-8db3a698a8a82293 devid 1 transid 6816 /dev/ram0 [ 115.864089] SELinux: initialized (dev ram0, type btrfs), uses xattr [ 122.586606] device fsid 9644bae59f78285c-8db3a698a8a82293 devid 1 transid 10872 /dev/ram0 [ 122.587779] SELinux: initialized (dev ram0, type btrfs), uses xattr [ 128.824401] BTRFS: inode 19622 still on the orphan list [ 128.844945] [ cut here ] [ 128.844969] WARNING: at fs/btrfs/extent-tree.c:6878 btrfs_free_block_groups+0x1e6/0x228 [btrfs]() [ 128.844973] Hardware name: XPS 8300 [ 128.844974] Modules linked in: btrfs zlib_deflate libcrc32c brd tun fuse cpufreq_ondemand acpi_cpufreq freq_table mperf ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables ipv6 uinput snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm broadcom snd_timer snd tg3 pcspkr joydev iTCO_wdt iTCO_vendor_support soundcore snd_page_alloc i2c_i801 serio_raw dcdbas microcode usb_storage i915 drm_kms_helper drm i2c_algo_bit i2c_core video [last unloaded: scsi_wait_scan] [ 128.845024] Pid: 2503, comm: umount Not tainted 3.0.0-rc2-2011-06-06-02+ #13 [ 128.845026] Call Trace: [ 128.845035] [] warn_slowpath_common+0x85/0x9d [ 128.845050] [] ? btrfs_free_block_groups+0x204/0x228 [btrfs] [ 128.845056] [] warn_slowpath_null+0x1a/0x1c [ 128.845069] [] btrfs_free_block_groups+0x1e6/0x228 [btrfs] [ 128.845087] [] close_ctree+0x22c/0x346 [btrfs] [ 128.845092] [] ? should_resched+0xe/0x2e [ 128.845097] [] ? _cond_resched+0xe/0x22 [ 128.845100] [] ? down_write+0x29/0x49 [ 128.845110] [] btrfs_put_super+0x1d/0x2c [btrfs] [ 128.845116] [] generic_shutdown_super+0x74/0xe9 [ 128.845120] [] kill_anon_super+0x16/0x50 [ 128.845125] [] deactivate_locked_super+0x26/0x4b [ 128.845129] [] deactivate_super+0x3a/0x3f [ 128.845133] [] mntput_no_expire+0xd0/0xd5 [ 128.845136] [] sys_umount+0x2dc/0x30a [ 128.845141] [] ? path_put+0x22/0x27 [ 128.845145] [] system_call_fastpath+0x16/0x1b [ 128.845149] ---[ end trace e535ebf83dd43354 ]--- [ 128.845152] space_info has 8814592 free, is full [ 128.845156] space_info total=41943040, used=24576000, pinned=0, reserved=98304, may_use=0, readonly=8454144 [ 128.897312] device fsid 9644bae59f78285c-8db3a698a8a82293 devid 1 transid 14722 /dev/ram0 [ 128.910740] BTRFS: inode 19622 still on the orphan list [ 128.910747] btrfs: could not do orphan cleanup -28 [ 128.948912] btrfs: open_ctree failed -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs hang on brd
On 01/06/11 13:07, Adrian Hunter wrote: On 01/06/11 11:54, David Sterba wrote: On Tue, May 31, 2011 at 10:03:12AM +0300, Adrian Hunter wrote: Hi I seem to be able to get btrfs reproducibly to produce warnings and finally hang when running a stress test on a ramdisk. Testing was done using the "integration-test" branch of btrfs-unstable. Note that I also tested v2.6.39 and "integration-test" took much longer to hang i.e. it is an improvement The test script and stack dumps are below. Is this a valid test? Is it worth me investigating these? I've tried to reproduce myself, but the fsstress utility (taken from latest LTP suite) crashes sometimes and I cannot take it as a proper reproduction. Can you point me to the exact version you used? The LTP version does not compile properly: make[4]: Entering directory `/home/ahunter/Desktop/Projects/ltp/ltp-full-20110228/testcases/kernel/fs/fsstress' gcc -g -O2 -g -O2 -fno-strict-aliasing -pipe -Wall -DNO_XFS -I/home/ahunter/Desktop/Projects/ltp/ltp-full-20110228/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -Wno-error -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress fsstress.c: In function 'dread_f': fsstress.c:1829:2: warning: implicit declaration of function 'memalign' fsstress.c:1829:6: warning: assignment makes pointer from integer without a cast fsstress.c: In function 'dwrite_f': fsstress.c:1912:6: warning: assignment makes pointer from integer without a cast fsstress.c:1844:17: warning: 'diob.d_miniosz' may be used uninitialized in this function fsstress.c:1844:17: warning: 'diob.d_maxiosz' may be used uninitialized in this function fsstress.c:1844:17: warning: 'diob.d_mem' may be used uninitialized in this function fsstress.c: In function 'dread_f': fsstress.c:1750:17: warning: 'diob.d_miniosz' may be used uninitialized in this function fsstress.c:1750:17: warning: 'diob.d_maxiosz' may be used uninitialized in this function fsstress.c:1750:17: warning: 'diob.d_mem' may be used uninitialized in this function I hacked a couple of changes but I need to check them before mailing to the ltp-list: From: Adrian Hunter Date: Wed, 1 Jun 2011 13:01:48 +0300 Subject: [PATCH] fsstress: quick fix for compile errors Signed-off-by: Adrian Hunter --- testcases/kernel/fs/fsstress/fsstress.c |2 ++ testcases/kernel/fs/fsstress/global.h |1 + 2 files changed, 3 insertions(+), 0 deletions(-) diff --git a/testcases/kernel/fs/fsstress/fsstress.c b/testcases/kernel/fs/fsstress/fsstress.c index e3b48ea..83c23ed 100644 --- a/testcases/kernel/fs/fsstress/fsstress.c +++ b/testcases/kernel/fs/fsstress/fsstress.c @@ -1757,6 +1757,7 @@ dread_f(int opno, long r) struct stat64stb; intv; +memset(&diob, 0, sizeof(struct dioattr)); init_pathname(&f); if (!get_fname(FT_REGFILE, r, &f, NULL, NULL, &v)) { if (v) @@ -1851,6 +1852,7 @@ dwrite_f(int opno, long r) struct stat64stb; intv; +memset(&diob, 0, sizeof(struct dioattr)); init_pathname(&f); if (!get_fname(FT_REGFILE, r, &f, NULL, NULL, &v)) { if (v) diff --git a/testcases/kernel/fs/fsstress/global.h b/testcases/kernel/fs/fsstress/global.h index f788395..5ab5d56 100644 --- a/testcases/kernel/fs/fsstress/global.h +++ b/testcases/kernel/fs/fsstress/global.h @@ -58,6 +58,7 @@ #include #include #include +#include #ifndef O_DIRECT #define O_DIRECT 04 -- 1.7.4.4 (But no warning or hang observed, on top of 3.0-rc1 + cmason/for-linus) I will try it tonight. No improvement on 3.0-rc1+ (commit 5c6cce92bc8aee751aafe82c5d9caf7553226a3d). Logs follow: Warnings [ 2857.023360] WARNING: at fs/btrfs/extent-tree.c:5648 btrfs_alloc_free_block+0x14e/0x357 [btrfs]() [ 2857.023364] Hardware name: XPS 8300 [ 2857.023367] Modules linked in: tun btrfs zlib_deflate libcrc32c brd fuse cpufreq_ondemand acpi_cpufreq freq_table mperf ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables ipv6 uinput snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device broadcom snd_pcm snd_timer snd tg3 iTCO_wdt serio_raw dcdbas iTCO_vendor_support microcode soundcore pcspkr snd_page_alloc i2c_i801 joydev usb_storage i915 drm_kms_helper drm i2c_algo_bit i2c_core video [last unloaded: scsi_wait_scan] [ 2857.023431] Pid: 8809, comm: btrfs-endio-wri Not tainted 3.0.0-rc1-work-2011-06-01-01+ #11 [ 2857.023435] Call Trace: [ 2857.023461] [] warn_slowpath_common+0x85/0x9d [ 2857.023471] [] warn_slowpath_null+0x1a/0x1c [ 2857.023494] [] btrfs_alloc_free_block+0x14e/0x357 [btrfs] [ 2857.023526] [] ? map_private_extent_buffer+0xb1/0xd5 [btrfs] [ 2857.023547] [] __btrfs_cow_block+0x102/0x31e [btrfs] [ 2857.023565] [] ? unlock_up+0xd9/0xe4 [btrfs] [ 2857.023585] [] btrfs_cow_block+0x104/0x14e [btrfs] [ 2857.023604] [] btrfs_search_slot+0x162/0x502 [btrfs] [ 2857.023625] [] btrf
Re: btrfs hang on brd
On 01/06/11 13:07, Adrian Hunter wrote: On 01/06/11 11:54, David Sterba wrote: On Tue, May 31, 2011 at 10:03:12AM +0300, Adrian Hunter wrote: Hi I seem to be able to get btrfs reproducibly to produce warnings and finally hang when running a stress test on a ramdisk. Testing was done using the "integration-test" branch of btrfs-unstable. Note that I also tested v2.6.39 and "integration-test" took much longer to hang i.e. it is an improvement The test script and stack dumps are below. Is this a valid test? Is it worth me investigating these? I've tried to reproduce myself, but the fsstress utility (taken from latest LTP suite) crashes sometimes and I cannot take it as a proper reproduction. Can you point me to the exact version you used? The LTP version does not compile properly: make[4]: Entering directory `/home/ahunter/Desktop/Projects/ltp/ltp-full-20110228/testcases/kernel/fs/fsstress' gcc -g -O2 -g -O2 -fno-strict-aliasing -pipe -Wall -DNO_XFS -I/home/ahunter/Desktop/Projects/ltp/ltp-full-20110228/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -Wno-error -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress fsstress.c: In function 'dread_f': fsstress.c:1829:2: warning: implicit declaration of function 'memalign' fsstress.c:1829:6: warning: assignment makes pointer from integer without a cast fsstress.c: In function 'dwrite_f': fsstress.c:1912:6: warning: assignment makes pointer from integer without a cast fsstress.c:1844:17: warning: 'diob.d_miniosz' may be used uninitialized in this function fsstress.c:1844:17: warning: 'diob.d_maxiosz' may be used uninitialized in this function fsstress.c:1844:17: warning: 'diob.d_mem' may be used uninitialized in this function fsstress.c: In function 'dread_f': fsstress.c:1750:17: warning: 'diob.d_miniosz' may be used uninitialized in this function fsstress.c:1750:17: warning: 'diob.d_maxiosz' may be used uninitialized in this function fsstress.c:1750:17: warning: 'diob.d_mem' may be used uninitialized in this function I hacked a couple of changes but I need to check them before mailing to the ltp-list: In fact there is already a fix here: http://sourceforge.net/mailarchive/message.php?msg_id=27212868 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs hang on brd
On 01/06/11 11:54, David Sterba wrote: On Tue, May 31, 2011 at 10:03:12AM +0300, Adrian Hunter wrote: Hi I seem to be able to get btrfs reproducibly to produce warnings and finally hang when running a stress test on a ramdisk. Testing was done using the "integration-test" branch of btrfs-unstable. Note that I also tested v2.6.39 and "integration-test" took much longer to hang i.e. it is an improvement The test script and stack dumps are below. Is this a valid test? Is it worth me investigating these? I've tried to reproduce myself, but the fsstress utility (taken from latest LTP suite) crashes sometimes and I cannot take it as a proper reproduction. Can you point me to the exact version you used? The LTP version does not compile properly: make[4]: Entering directory `/home/ahunter/Desktop/Projects/ltp/ltp-full-20110228/testcases/kernel/fs/fsstress' gcc -g -O2 -g -O2 -fno-strict-aliasing -pipe -Wall -DNO_XFS -I/home/ahunter/Desktop/Projects/ltp/ltp-full-20110228/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -Wno-error -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress fsstress.c: In function 'dread_f': fsstress.c:1829:2: warning: implicit declaration of function 'memalign' fsstress.c:1829:6: warning: assignment makes pointer from integer without a cast fsstress.c: In function 'dwrite_f': fsstress.c:1912:6: warning: assignment makes pointer from integer without a cast fsstress.c:1844:17: warning: 'diob.d_miniosz' may be used uninitialized in this function fsstress.c:1844:17: warning: 'diob.d_maxiosz' may be used uninitialized in this function fsstress.c:1844:17: warning: 'diob.d_mem' may be used uninitialized in this function fsstress.c: In function 'dread_f': fsstress.c:1750:17: warning: 'diob.d_miniosz' may be used uninitialized in this function fsstress.c:1750:17: warning: 'diob.d_maxiosz' may be used uninitialized in this function fsstress.c:1750:17: warning: 'diob.d_mem' may be used uninitialized in this function I hacked a couple of changes but I need to check them before mailing to the ltp-list: From: Adrian Hunter Date: Wed, 1 Jun 2011 13:01:48 +0300 Subject: [PATCH] fsstress: quick fix for compile errors Signed-off-by: Adrian Hunter --- testcases/kernel/fs/fsstress/fsstress.c |2 ++ testcases/kernel/fs/fsstress/global.h |1 + 2 files changed, 3 insertions(+), 0 deletions(-) diff --git a/testcases/kernel/fs/fsstress/fsstress.c b/testcases/kernel/fs/fsstress/fsstress.c index e3b48ea..83c23ed 100644 --- a/testcases/kernel/fs/fsstress/fsstress.c +++ b/testcases/kernel/fs/fsstress/fsstress.c @@ -1757,6 +1757,7 @@ dread_f(int opno, long r) struct stat64 stb; int v; + memset(&diob, 0, sizeof(struct dioattr)); init_pathname(&f); if (!get_fname(FT_REGFILE, r, &f, NULL, NULL, &v)) { if (v) @@ -1851,6 +1852,7 @@ dwrite_f(int opno, long r) struct stat64 stb; int v; + memset(&diob, 0, sizeof(struct dioattr)); init_pathname(&f); if (!get_fname(FT_REGFILE, r, &f, NULL, NULL, &v)) { if (v) diff --git a/testcases/kernel/fs/fsstress/global.h b/testcases/kernel/fs/fsstress/global.h index f788395..5ab5d56 100644 --- a/testcases/kernel/fs/fsstress/global.h +++ b/testcases/kernel/fs/fsstress/global.h @@ -58,6 +58,7 @@ #include #include #include +#include #ifndef O_DIRECT #define O_DIRECT 04 -- 1.7.4.4 (But no warning or hang observed, on top of 3.0-rc1 + cmason/for-linus) I will try it tonight. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs hang on brd
On Tue, May 31, 2011 at 10:03:12AM +0300, Adrian Hunter wrote: > Hi > > I seem to be able to get btrfs reproducibly to > produce warnings and finally hang when running > a stress test on a ramdisk. > > Testing was done using the "integration-test" > branch of btrfs-unstable. Note that I also tested > v2.6.39 and "integration-test" took much longer to > hang i.e. it is an improvement > > The test script and stack dumps are below. > > Is this a valid test? > > Is it worth me investigating these? I've tried to reproduce myself, but the fsstress utility (taken from latest LTP suite) crashes sometimes and I cannot take it as a proper reproduction. Can you point me to the exact version you used? (But no warning or hang observed, on top of 3.0-rc1 + cmason/for-linus) > Test > > > #!/bin/sh > > sudo modprobe brd rd_size=262144 this is minimal size possible, 256MB > > sudo umount /mnt/test/ 2> /dev/null > > echo 'mkfs.btrfs /dev/ram0' > > sudo mkfs.btrfs /dev/ram0 > > sudo mkdir -p /mnt/test > > echo 'mount -t btrfs /dev/ram0 /mnt/test' > > sudo mount -t btrfs /dev/ram0 /mnt/test > > sudo mkdir -p /mnt/test/test > > sudo chown $USER /mnt/test/test > sudo chgrp $USER /mnt/test/test > > sudo umount /mnt/test > > full=0 > i=0 > while true; do > sudo mount -t btrfs /dev/ram0 /mnt/test > > if df | grep ram0 | grep 100% > /dev/null; then > full=`expr $full \+ 1` > if test $full -gt 6;then > rm -rf /mnt/test/test/* > full=0 > fi > else > full=0 > fi > > fsstress -c -r -d /mnt/test/test -p 3 -n 1000 -l 10 > > sudo umount /mnt/test > > i=`expr $i \+ 1` > echo $i > done > > > > Stack dumps for warnings > > > > [ 7481.520750] WARNING: at fs/btrfs/extent-tree.c:5648 5644 ret = block_rsv_use_bytes(block_rsv, blocksize); 5645 if (!ret) 5646 return block_rsv; 5647 if (ret) { 5648 WARN_ON(1); 5649 ret = reserve_metadata_bytes(trans, root, block_rsv, blocksize, 5650 0); and block_rsv_use_bytes() returns nonzero in case of ENOSPC. > [ 7481.521176] WARNING: at fs/btrfs/extent-tree.c:5648 > btrfs_alloc_free_block+0x14e/0x357 [btrfs]() > [ 7481.521178] Hardware name: XPS 8300 > [ 7481.521180] Modules linked in: tcp_lp tun btrfs zlib_deflate > libcrc32c brd fuse cpufreq_ondemand acpi_cpufreq freq_table mperf > ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter > ip6_tables ipv6 uinput snd_hda_codec_hdmi snd_hda_codec_realtek > snd_hda_intel snd_hda_codec broadcom tg3 snd_hwdep snd_seq > snd_seq_device snd_pcm joydev pcspkr iTCO_wdt iTCO_vendor_support > dcdbas serio_raw i2c_i801 snd_timer snd microcode soundcore > snd_page_alloc usb_storage i915 drm_kms_helper drm i2c_algo_bit > i2c_core video [last unloaded: scsi_wait_scan] > [ 7481.521237] Pid: 3980, comm: btrfs-endio-wri Tainted: GW > 2.6.39-integration-test-20110526-01+ #2 > [ 7481.521240] Call Trace: > [ 7481.521245] [] warn_slowpath_common+0x85/0x9d > [ 7481.521250] [] warn_slowpath_null+0x1a/0x1c > [ 7481.521288] [] btrfs_alloc_free_block+0x14e/0x357 > [btrfs] > [ 7481.521303] [] ? map_private_extent_buffer+0xb1/0xd5 > [btrfs] > [ 7481.521313] [] __btrfs_cow_block+0x102/0x31e [btrfs] > [ 7481.521322] [] ? btrfs_set_item_key+0x3/0x20 [btrfs] > [ 7481.521341] [] btrfs_cow_block+0x104/0x14d [btrfs] > [ 7481.521353] [] btrfs_search_slot+0x162/0x502 [btrfs] > [ 7481.521378] [] btrfs_lookup_file_extent+0x3c/0x3e > [btrfs] > [ 7481.521388] [] ? btrfs_alloc_path+0x1a/0x2b [btrfs] > [ 7481.521405] [] btrfs_drop_extents+0x10e/0x731 [btrfs] > [ 7481.521410] [] ? need_resched+0x23/0x2d > [ 7481.521415] [] ? _cond_resched+0xe/0x22 > [ 7481.521420] [] ? slab_pre_alloc_hook.clone.32+0x2d/0x31 > [ 7481.521426] [] ? kmem_cache_alloc+0x29/0xf7 > [ 7481.521441] [] > insert_reserved_file_extent.clone.34+0x70/0x1fc [btrfs] > [ 7481.521470] [] ? lock_extent_bits+0x5e/0xa8 [btrfs] > [ 7481.521496] [] btrfs_endio_direct_write+0x171/0x29a > [btrfs] > [ 7481.521511] [] ? end_workqueue_fn+0xf6/0x10e [btrfs] > [ 7481.521516] [] bio_endio+0x2d/0x2f > [ 7481.521539] [] end_workqueue_fn+0x101/0x10e [btrfs] > [ 7481.521565] [] worker_loop+0x193/0x4ca [btrfs] > [ 7481.521581] [] ? btrfs_queue_worker+0x214/0x214 [btrfs] > [ 7481.521586] [] kthread+0x82/0x8a > [ 7481.521591] [] kernel_thread_helper+0x4/0x10 > [ 7481.521596] [] ? kthread_worker_fn+0x14b/0x14b > [ 7481.521601] [] ? gs_change+0x13/0x13 > [ 7481.521604] ---[ end trace abb147a5624a0a25 ]--- > [ 7481.521639] [ cut here ] > > Stack dumps for more warnings > -- > > [21983.399906] WARNING: at fs/btrfs/extent-tree.c:3832 3829 static void release_global_block_rsv(struct btrfs_fs_info *fs_info) 3830 { 3831 block_rsv_re