Re: how to understand "btrfs fi show" output? "No space left" issues
On 2016 Sep 20, Peter Becker wrote: > Data, RAID1: total=417.12GiB, used=131.33GiB > > You have 417(total)-131(used) blocks wo are only partial filled. > You should balance your file-system. > > At first you need some free space. You could remove some files / old > snapshots etc. or you add a empty USB-Stick with min. 4 GB to your > BTRFS-Pool (after balancing complete you can remove the stick from the > pool). He has plenty of space. What you're describing is the case that either data pool or metadata pool is full, the other has enough space and nothing is left that could be allocated to the full pool. In this case rebalancing would help. But in Tomasz' case there is enough space in every pool, so the allocator should use it. This really sounds like a bug. > But at first you should try to free emty data and meta data blocks: > > btrfs balance start -musage=0 /mnt > btrfs balance start -dusage=0 /mnt Since kernel 3.18 this is done automatically. regards, Johannes -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Btrfs: device_list_add() should not update list when mounted breaks subvol mount
On Sun, 14 Sep 2014 00:45:49 + (UTC) Duncan 1i5t5.dun...@cox.net wrote: Johannes Hirte posted on Sat, 13 Sep 2014 23:23:20 +0200 as excerpted: On Sat, 13 Sep 2014 19:55:25 +0200 Johannes Hirte johannes.hi...@datenkhaos.de wrote: On Sat, 13 Sep 2014 13:36:37 +0800 Anand Jain anand.j...@oracle.com wrote: The quickest workaround for you will be to try to match the device path as in the btrfs fi show -m /mnt output to your probably fstab/mnttab entry. Doesn't work here. I don't even get a path with the affected kernels. I'll get: Label: none uuid: 02edbd6b-f044-4800-b21e-ca8982c2c2e5 Total devices 1 FS bytes used 270.10GiB *** Some devices missing Btrfs v3.16 with a working kernel: Label: none uuid: 02edbd6b-f044-4800-b21e-ca8982c2c2e5 Total devices 1 FS bytes used 270.10GiB devid 1 size 293.89GiB used 289.06GiB path /dev/sda1 Btrfs v3.16 And now I was able to reproduce on a second machine. The main difference between the affected and the unaffected systems is initramfs. On the affected systems, I don't use one. On the working systems, the rootfs is mounted via initramfs before. I'll test, if an initramfs will solve the issue. Seems likely, cause if I put the disk of an affected system into a working system and mount it there, everything works. Of course, with the initramfs it works. I see a btrfs device scan in the initramfs script. What happens if you simply run btrfs device scan manually, before doing btrfs filesystem show? I'm guessing that'll fix it. Not tested, but I doubt it will fix it. In this initramfs it is a leftover from a time, when the system was multi-device. On two other systems, initramfs works without the scan. Additionally I can put the affected HDD from the laptop and put it into/on another system, that is affected without initramfs. I can mount it there without any scan before. But I'm not 100% sure if udev takes responsibility for scanning here. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Btrfs: device_list_add() should not update list when mounted breaks subvol mount
On Mon, 15 Sep 2014 20:32:58 +0800
Anand Jain anand.j...@oracle.com wrote:
Hi Johannes,
Can I have you this tested.. ? Thanks.
---
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index e9676a4..1224b61 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -533,7 +533,7 @@ static noinline int device_list_add(const char
*path,
* the btrfs dev scan cli, after FS has been mounted.
*/
if (fs_devices-opened) {
- return -EBUSY;
+ goto out;
} else {
/*
* That is if the FS is _not_ mounted and if
you @@ -566,6 +566,7 @@ static noinline int device_list_add(const
char *path, if (!fs_devices-opened)
device-generation = found_transid;
+out:
*fs_devices_ret = fs_devices;
return ret;
---
With this change, it works again without initramfs.
regards,
Johannes
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Btrfs: device_list_add() should not update list when mounted breaks subvol mount
On Tue, 16 Sep 2014 01:39:49 +0800
Anand Jain anand.j...@oracle.com wrote:
On 16/09/2014 01:14, Johannes Hirte wrote:
On Mon, 15 Sep 2014 20:32:58 +0800
Anand Jain anand.j...@oracle.com wrote:
Hi Johannes,
Can I have you this tested.. ? Thanks.
---
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index e9676a4..1224b61 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -533,7 +533,7 @@ static noinline int device_list_add(const char
*path,
* the btrfs dev scan cli, after FS has been
mounted. */
if (fs_devices-opened) {
- return -EBUSY;
+ goto out;
} else {
/*
* That is if the FS is _not_ mounted
and if you @@ -566,6 +566,7 @@ static noinline int
device_list_add(const char *path, if (!fs_devices-opened)
device-generation = found_transid;
+out:
*fs_devices_ret = fs_devices;
return ret;
---
With this change, it works again without initramfs.
Thanks. I am bit confused, is there any configuration that is
still not working, even with the above changes.
Anand
At moment, I'm not aware of any. But I will test the remaining systems
I have.
regards,
Johannes
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Btrfs: device_list_add() should not update list when mounted breaks subvol mount
On Sat, 13 Sep 2014 13:36:37 +0800 Anand Jain anand.j...@oracle.com wrote: Xavier, Johannes, The quickest workaround for you will be to try to match the device path as in the btrfs fi show -m /mnt output to your probably fstab/mnttab entry. Doesn't work here. I don't even get a path with the affected kernels. I'll get: Label: none uuid: 02edbd6b-f044-4800-b21e-ca8982c2c2e5 Total devices 1 FS bytes used 270.10GiB *** Some devices missing Btrfs v3.16 with a working kernel: Label: none uuid: 02edbd6b-f044-4800-b21e-ca8982c2c2e5 Total devices 1 FS bytes used 270.10GiB devid1 size 293.89GiB used 289.06GiB path /dev/sda1 Btrfs v3.16 Filesystem layout is: subvolid 0 contains only the different subvolumes ID 257 gen 414674 top level 5 path rootfs ID 269 gen 414615 top level 5 path home-USER1 ID 317 gen 411498 top level 5 path home-USER2 ID 363 gen 410939 top level 5 path home-USER3 ID 382 gen 315844 top level 5 path home-USER4 ID 933 gen 410514 top level 5 path home-USER5 ID 995 gen 315756 top level 5 path homefs-USER6 subvol rootfs (ID 257) is set to the default subvolume, mounted at start. Grub commandline is like following: root=/dev/sda1 ro rootflags=subvol=rootfs,inode_cache,autodefrag It doesn't matter, if the subvol parameter is set. I've tried with, without and with subvolid=0 parameter. Everytime the same result. And now I was able to reproduce on a second machine. The main difference between the affected and the unaffected systems is initramfs. On the affected systems, I don't use one. On the working systems, the rootfs is mounted via initramfs before. I'll test, if an initramfs will solve the issue. Seems likely, cause if I put the disk of an affected system into a working system and mount it there, everything works. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Btrfs: device_list_add() should not update list when mounted breaks subvol mount
On Sat, 13 Sep 2014 19:55:25 +0200 Johannes Hirte johannes.hi...@datenkhaos.de wrote: On Sat, 13 Sep 2014 13:36:37 +0800 Anand Jain anand.j...@oracle.com wrote: Xavier, Johannes, The quickest workaround for you will be to try to match the device path as in the btrfs fi show -m /mnt output to your probably fstab/mnttab entry. Doesn't work here. I don't even get a path with the affected kernels. I'll get: Label: none uuid: 02edbd6b-f044-4800-b21e-ca8982c2c2e5 Total devices 1 FS bytes used 270.10GiB *** Some devices missing Btrfs v3.16 with a working kernel: Label: none uuid: 02edbd6b-f044-4800-b21e-ca8982c2c2e5 Total devices 1 FS bytes used 270.10GiB devid1 size 293.89GiB used 289.06GiB path /dev/sda1 Btrfs v3.16 Filesystem layout is: subvolid 0 contains only the different subvolumes ID 257 gen 414674 top level 5 path rootfs ID 269 gen 414615 top level 5 path home-USER1 ID 317 gen 411498 top level 5 path home-USER2 ID 363 gen 410939 top level 5 path home-USER3 ID 382 gen 315844 top level 5 path home-USER4 ID 933 gen 410514 top level 5 path home-USER5 ID 995 gen 315756 top level 5 path homefs-USER6 subvol rootfs (ID 257) is set to the default subvolume, mounted at start. Grub commandline is like following: root=/dev/sda1 ro rootflags=subvol=rootfs,inode_cache,autodefrag It doesn't matter, if the subvol parameter is set. I've tried with, without and with subvolid=0 parameter. Everytime the same result. And now I was able to reproduce on a second machine. The main difference between the affected and the unaffected systems is initramfs. On the affected systems, I don't use one. On the working systems, the rootfs is mounted via initramfs before. I'll test, if an initramfs will solve the issue. Seems likely, cause if I put the disk of an affected system into a working system and mount it there, everything works. Of course, with the initramfs it works. Content of the init-script: #!/bin/sh mount -t devtmpfs devtmpfs /dev mount -t proc proc /proc mount -t sysfs sysfs /sys mount -t tmpfs tmpfs /run sleep 3 # wait for kernel msgs to quiet echo loading initrd btrfs dev scan sleep 5 mount -o ro,subvol=rootfs,inode_cache,autodefrag /dev/sda1 /newroot if [[ -x /newroot/sbin/init ]]; then umount /sys /proc exec switch_root /newroot /sbin/init fi #rescue shell exec sh -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Btrfs: device_list_add() should not update list when mounted breaks subvol mount
commit b96de000bc8bc9688b3a2abea4332bd57648a49f breaks subvolume mount on one of my systems. I've bisected a mount problem to this commit. Situation is: - one hdd with btrfs - default subvolume (rootfs) is different from subovlid=0 - at boot, several subvols are mounted at /home/$DIR after commit b96de000bc8bc9688b3a2abea4332bd57648a49f this is not possible anymore. Trying to mount results in (example): mount: /dev/sda1 is already mounted or /home/video busy The output of btrfs show is curious too: btrfs fi show Label: none uuid: 43438ef5-adac-46a9-823e-14951ee6866a Total devices 1 FS bytes used 150.05GiB *** Some devices missing Btrfs v3.16 As this is a laptop with only one drive bay, this was never a multi device setup. I've two more systems with kernel version 3.17-rc3 running and no problem like this. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: BTRFS setup advice for laptop performance ?
On Fri, 04 Apr 2014 08:33:10 -0400 Austin S Hemmelgarn ahferro...@gmail.com wrote: On 2014-04-04 04:02, Swâmi Petaramesh wrote: - Is it still recommended to mkfs with a nodesize or leafsize different (bigger) than the default ? I wouldn't like to lose too much disk space anyway (1/2 nodesize per file on average ?), as it will be limited... This depends on many things, the average size of the files on the disk is the biggest factor. In general you should get the best disk utilization by setting nodesize so that a majority of the files are less than the leafsize minus 256 bytes, and all but a few are smaller than two times the leafsize minus 256 bytes. However, if you want to really benefit from the data compression, you should just use the smallest leaf/nodesize for your system (which is what mkfs defaults to), as data that gets as BTRFS stores files whose size is at least (roughly) 256 bytes less than the leafsize inline with the metadata, and doesn't compress such files. With commit c652e4efb8e2dd76ef1627d8cd649c6af5905902 the default node-/leafsize has changed: commit c652e4efb8e2dd76ef1627d8cd649c6af5905902 Author: Chris Mason chris.ma...@fusionio.com Date: Fri Nov 8 13:51:52 2013 -0500 mkfs: change default metadata blocksize to 16KB -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs: throttle delayed refs better
On Fri, 14 Feb 2014 14:29:35 -0500 Josef Bacik jba...@fb.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/14/2014 02:25 PM, Johannes Hirte wrote: On Thu, 6 Feb 2014 16:19:46 -0500 Josef Bacik jba...@fb.com wrote: Ok so I thought I reproduced the problem but I just reproduced a different problem. Please undo any changes you've made and apply this patch and reproduce and then provide me with any debug output that gets spit out. I'm sending this via thunderbird with 6 different extensions to make sure it comes out right so if it doesn't work let me know and I'll just paste it somewhere. Thanks, Sorry for the long delay. Was to busy last week. Ok perfect this is fixed by [PATCH] Btrfs: don't loop forever if we can't run because of the tree mod log and it went into -rc2 iirc, so give that a whirl and make sure it fixes your problem. Thanks, Yes, seems to be fixed now. I wasn't able to reproduce it anymore. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs: throttle delayed refs better
On Thu, 6 Feb 2014 16:19:46 -0500 Josef Bacik jba...@fb.com wrote: Ok so I thought I reproduced the problem but I just reproduced a different problem. Please undo any changes you've made and apply this patch and reproduce and then provide me with any debug output that gets spit out. I'm sending this via thunderbird with 6 different extensions to make sure it comes out right so if it doesn't work let me know and I'll just paste it somewhere. Thanks, Sorry for the long delay. Was to busy last week. Here is the output: [ 25.240971] looped a lot, count 14, nr 32, no_selected_ref 99986 [ 25.267639] looped a lot, count 14, nr 32, no_selected_ref 199987 [ 25.294308] looped a lot, count 14, nr 32, no_selected_ref 299988 [ 25.320605] looped a lot, count 14, nr 32, no_selected_ref 399989 [ 25.346639] looped a lot, count 14, nr 32, no_selected_ref 40 [ 25.372517] looped a lot, count 14, nr 32, no_selected_ref 51 [ 25.398924] looped a lot, count 14, nr 32, no_selected_ref 62 [ 25.425443] looped a lot, count 14, nr 32, no_selected_ref 73 [ 25.451344] looped a lot, count 14, nr 32, no_selected_ref 84 [ 25.477350] looped a lot, count 14, nr 32, no_selected_ref 95 [ 25.503069] looped a lot, count 14, nr 32, no_selected_ref 106 [ 25.529372] looped a lot, count 14, nr 32, no_selected_ref 117 [ 25.49] looped a lot, count 14, nr 32, no_selected_ref 128 [ 25.581418] looped a lot, count 14, nr 32, no_selected_ref 139 [ 25.607514] looped a lot, count 14, nr 32, no_selected_ref 150 [ 25.633794] looped a lot, count 14, nr 32, no_selected_ref 161 [ 25.659699] looped a lot, count 14, nr 32, no_selected_ref 172 [ 25.686095] looped a lot, count 14, nr 32, no_selected_ref 183 [ 25.711906] looped a lot, count 14, nr 32, no_selected_ref 194 [ 25.752255] looped a lot, count 14, nr 32, no_selected_ref 205 [ 25.788077] looped a lot, count 0, nr 32, no_selected_ref 10 [ 25.811966] looped a lot, count 14, nr 32, no_selected_ref 216 [ 360.749227] looped a lot, count 8, nr 32, no_selected_ref 2 [ 360.770434] looped a lot, count 8, nr 32, no_selected_ref 13 [ 360.792136] looped a lot, count 8, nr 32, no_selected_ref 24 [ 360.813571] looped a lot, count 8, nr 32, no_selected_ref 35 [ 360.834932] looped a lot, count 8, nr 32, no_selected_ref 46 [ 360.856085] looped a lot, count 8, nr 32, no_selected_ref 57 [ 360.877374] looped a lot, count 8, nr 32, no_selected_ref 68 [ 360.899455] looped a lot, count 8, nr 32, no_selected_ref 79 [ 360.921175] looped a lot, count 8, nr 32, no_selected_ref 90 [ 360.942409] looped a lot, count 8, nr 32, no_selected_ref 101 [ 360.963800] looped a lot, count 8, nr 32, no_selected_ref 112 [ 360.985397] looped a lot, count 8, nr 32, no_selected_ref 123 [ 361.007148] looped a lot, count 8, nr 32, no_selected_ref 134 [ 361.028789] looped a lot, count 8, nr 32, no_selected_ref 145 [ 361.050564] looped a lot, count 8, nr 32, no_selected_ref 156 [ 361.072008] looped a lot, count 8, nr 32, no_selected_ref 167 [ 361.093269] looped a lot, count 8, nr 32, no_selected_ref 178 [ 361.114645] looped a lot, count 8, nr 32, no_selected_ref 189 [ 361.136099] looped a lot, count 8, nr 32, no_selected_ref 1900010 [ 361.157566] looped a lot, count 8, nr 32, no_selected_ref 211 [ 361.178969] looped a lot, count 8, nr 32, no_selected_ref 2100012 [ 361.200397] looped a lot, count 8, nr 32, no_selected_ref 2200013 [ 361.221980] looped a lot, count 8, nr 32, no_selected_ref 2300014 [ 361.243435] looped a lot, count 8, nr 32, no_selected_ref 2400015 [ 361.264777] looped a lot, count 8, nr 32, no_selected_ref 2500016 [ 361.286518] looped a lot, count 8, nr 32, no_selected_ref 2600017 [ 361.308240] looped a lot, count 8, nr 32, no_selected_ref 2700018 [ 361.329850] looped a lot, count 8, nr 32, no_selected_ref 2800019 [ 361.351420] looped a lot, count 8, nr 32, no_selected_ref 2900020 [ 361.372633] looped a lot, count 8, nr 32, no_selected_ref 321 [ 361.394330] looped a lot, count 8, nr 32, no_selected_ref 3100022 [ 361.416039] looped a lot, count 8, nr 32, no_selected_ref 3200023 [ 361.437659] looped a lot, count 8, nr 32, no_selected_ref 3300024 [ 361.459181] looped a lot, count 8, nr 32, no_selected_ref 3400025 [ 361.481058] looped a lot, count 8, nr 32, no_selected_ref 3500026 [ 361.502441] looped a lot, count 8, nr 32, no_selected_ref 3600027 [ 361.523964] looped a lot, count 8, nr 32, no_selected_ref 3700028 [ 361.545387] looped a lot, count 8, nr 32, no_selected_ref 3800029 [ 361.566717] looped a lot, count 8, nr 32, no_selected_ref 3900030 [ 361.588079] looped a lot, count 8, nr 32, no_selected_ref 431 [ 361.609673] looped a lot, count 8, nr 32, no_selected_ref 4100032 [ 361.631028] looped a lot, count 8, nr 32, no_selected_ref 4200033 [ 361.652498] looped a lot, count 8, nr 32,
Re: [PATCH] Btrfs: throttle delayed refs better
On Tue, 4 Feb 2014 09:12:54 -0500 Josef Bacik jba...@fb.com wrote: Hrm I was hoping that was going to be more helpful. Can you get perf record -ag and then perf report while it's at full cpu and get the first 3 or 4 things with their traces? Here it comes: # # captured on: Wed Feb 5 00:11:41 2014 # # no symbols found in /usr/sbin/acpid, maybe install a debug package? unexpected end of event stream # Samples: 168K of event 'cycles' # Event count (approx.): 126847081763 # # Overhead Command Shared Object Symbol # ... .. ... # 18.48% btrfs-freespace [kernel.kallsyms] [k] state_store | --- state_store 10.25% btrfs-freespace [kernel.kallsyms] [k] sys_sched_rr_get_interval | --- sys_sched_rr_get_interval 9.02% btrfs-freespace [kernel.kallsyms] [k] rt_mutex_slowunlock | --- rt_mutex_slowunlock 8.76% btrfs-freespace [kernel.kallsyms] [k] btrfs_submit_compressed_write | --- btrfs_submit_compressed_write 6.63% btrfs-freespace [kernel.kallsyms] [k] sched_show_task | --- sched_show_task 5.19% btrfs-freespace [kernel.kallsyms] [k] find_free_extent | --- find_free_extent 5.15% btrfs-freespace [kernel.kallsyms] [k] trace_print_graph_duration | --- trace_print_graph_duration I'm going to try and reproduce today, is there anything special about your fs? Compression, large blocksizes, skinny metadata? Thanks, Filesystem was created with -l 32768 -n 32768 and skinny metadata enabled. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs: throttle delayed refs better
On Wed, 5 Feb 2014 10:49:15 -0500
Josef Bacik jba...@fb.com wrote:
Ok none of those make sense which makes me think it may be the ktime
bits, instead of un-applying the whole patch could you just comment
out the parts
ktime_t start = ktime_get();
and
if (actual_count 0) {
u64 runtime = ktime_to_ns(ktime_sub(ktime_get(),
start)); u64 avg;
/*
* We weigh the current average higher than our
current runtime
* to avoid large swings in the average.
*/
spin_lock(delayed_refs-lock);
avg = fs_info-avg_delayed_ref_runtime * 3 + runtime;
avg = div64_u64(avg, 4);
fs_info-avg_delayed_ref_runtime = avg;
spin_unlock(delayed_refs-lock);
}
in __btrfs_run_delayed_refs and see if that makes the problem stop?
If it does will you try chris's for-linus branch to see if it still
reproduces there? Maybe some patch changed ktime_get() in -rc1 that
is causing issues and we're just now exposing it. Thanks,
With the ktime bits disabled, I wasn't able to reproduce the
problem anymore. With Chris' for-linus branch it took longer but still
appeared.
regards,
Johannes
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs: throttle delayed refs better
On Wed, 5 Feb 2014 14:36:39 -0500
Josef Bacik jba...@fb.com wrote:
On 02/05/2014 02:30 PM, Johannes Hirte wrote:
On Wed, 5 Feb 2014 14:00:57 -0500
Josef Bacik jba...@fb.com wrote:
On 02/05/2014 12:34 PM, Johannes Hirte wrote:
On Wed, 5 Feb 2014 10:49:15 -0500
Josef Bacik jba...@fb.com wrote:
Ok none of those make sense which makes me think it may be the
ktime bits, instead of un-applying the whole patch could you just
comment out the parts
ktime_t start = ktime_get();
and
if (actual_count 0) {
u64 runtime =
ktime_to_ns(ktime_sub(ktime_get(), start)); u64 avg;
/*
* We weigh the current average higher than
our current runtime
* to avoid large swings in the average.
*/
spin_lock(delayed_refs-lock);
avg = fs_info-avg_delayed_ref_runtime * 3 +
runtime; avg = div64_u64(avg, 4);
fs_info-avg_delayed_ref_runtime = avg;
spin_unlock(delayed_refs-lock);
}
in __btrfs_run_delayed_refs and see if that makes the problem
stop? If it does will you try chris's for-linus branch to see if
it still reproduces there? Maybe some patch changed ktime_get()
in -rc1 that is causing issues and we're just now exposing it.
Thanks,
With the ktime bits disabled, I wasn't able to reproduce the
problem anymore. With Chris' for-linus branch it took longer but
still appeared.
Ok can you send your .config, maybe there's some weird time bug
being exposed. What kind of CPU do you have? Thanks,
Josef
It's a Core i5-540M, dualcore + hyperthreading
Ok while I'm doing this can you change
btrfs_should_throttle_delayed_refs to _always_ return 1, still with
all the ktime stuff commented out, and see if that causes the problem
to happen? Thanks,
Yes it does. Same behavior as without ktime stuff commented out.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs: throttle delayed refs better
On Wed, 5 Feb 2014 16:46:57 -0500
Josef Bacik jba...@fb.com wrote:
On 02/05/2014 04:42 PM, Johannes Hirte wrote:
On Wed, 5 Feb 2014 14:36:39 -0500
Josef Bacik jba...@fb.com wrote:
On 02/05/2014 02:30 PM, Johannes Hirte wrote:
On Wed, 5 Feb 2014 14:00:57 -0500
Josef Bacik jba...@fb.com wrote:
On 02/05/2014 12:34 PM, Johannes Hirte wrote:
On Wed, 5 Feb 2014 10:49:15 -0500
Josef Bacik jba...@fb.com wrote:
Ok none of those make sense which makes me think it may be the
ktime bits, instead of un-applying the whole patch could you
just comment out the parts
ktime_t start = ktime_get();
and
if (actual_count 0) {
u64 runtime =
ktime_to_ns(ktime_sub(ktime_get(), start)); u64 avg;
/*
* We weigh the current average higher than
our current runtime
* to avoid large swings in the average.
*/
spin_lock(delayed_refs-lock);
avg = fs_info-avg_delayed_ref_runtime * 3
+ runtime; avg = div64_u64(avg, 4);
fs_info-avg_delayed_ref_runtime = avg;
spin_unlock(delayed_refs-lock);
}
in __btrfs_run_delayed_refs and see if that makes the problem
stop? If it does will you try chris's for-linus branch to see
if it still reproduces there? Maybe some patch changed
ktime_get() in -rc1 that is causing issues and we're just now
exposing it. Thanks,
With the ktime bits disabled, I wasn't able to reproduce the
problem anymore. With Chris' for-linus branch it took longer but
still appeared.
Ok can you send your .config, maybe there's some weird time bug
being exposed. What kind of CPU do you have? Thanks,
Josef
It's a Core i5-540M, dualcore + hyperthreading
Ok while I'm doing this can you change
btrfs_should_throttle_delayed_refs to _always_ return 1, still with
all the ktime stuff commented out, and see if that causes the
problem to happen? Thanks,
Yes it does. Same behavior as without ktime stuff commented out.
Ok perfect, can you send me a btrfs fi df of that volume, and do you
have any snapshots or anything? Thanks,
btrfs fi df /
Data, single: total=220.01GiB, used=210.85GiB
System, DUP: total=8.00MiB, used=32.00KiB
System, single: total=4.00MiB, used=0.00
Metadata, DUP: total=4.00GiB, used=2.93GiB
Metadata, single: total=8.00MiB, used=0.00
No snapshots but several subvolumes. / itself is a seperate subvolume
and subvol 0 only contains the other subvolumes (5 at moment). qgroups
aren't enabled.
mount options are noatime,inode_cache, if this matters
regards,
Johannes
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs: throttle delayed refs better
On Thu, 23 Jan 2014 13:07:52 -0500 Josef Bacik jba...@fb.com wrote: On one of our gluster clusters we noticed some pretty big lag spikes. This turned out to be because our transaction commit was taking like 3 minutes to complete. This is because we have like 30 gigs of metadata, so our global reserve would end up being the max which is like 512 mb. So our throttling code would allow a ridiculous amount of delayed refs to build up and then they'd all get run at transaction commit time, and for a cold mounted file system that could take up to 3 minutes to run. So fix the throttling to be based on both the size of the global reserve and how long it takes us to run delayed refs. This patch tracks the time it takes to run delayed refs and then only allows 1 seconds worth of outstanding delayed refs at a time. This way it will auto-tune itself from cold cache up to when everything is in memory and it no longer has to go to disk. This makes our transaction commits take much less time to run. Thanks, Signed-off-by: Josef Bacik jba...@fb.com This one breaks my system. Shortly after boot the btrfs-freespace thread goes up to 100% CPU usage and the system is nearly unresponsive. I've seen it first with the full pull request for 3.14-rc1 and was able to track it down to this patch. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs: throttle delayed refs better
On Mon, 3 Feb 2014 16:08:08 -0500 Josef Bacik jba...@fb.com wrote: On 02/03/2014 01:28 PM, Johannes Hirte wrote: On Thu, 23 Jan 2014 13:07:52 -0500 Josef Bacik jba...@fb.com wrote: On one of our gluster clusters we noticed some pretty big lag spikes. This turned out to be because our transaction commit was taking like 3 minutes to complete. This is because we have like 30 gigs of metadata, so our global reserve would end up being the max which is like 512 mb. So our throttling code would allow a ridiculous amount of delayed refs to build up and then they'd all get run at transaction commit time, and for a cold mounted file system that could take up to 3 minutes to run. So fix the throttling to be based on both the size of the global reserve and how long it takes us to run delayed refs. This patch tracks the time it takes to run delayed refs and then only allows 1 seconds worth of outstanding delayed refs at a time. This way it will auto-tune itself from cold cache up to when everything is in memory and it no longer has to go to disk. This makes our transaction commits take much less time to run. Thanks, Signed-off-by: Josef Bacik jba...@fb.com This one breaks my system. Shortly after boot the btrfs-freespace thread goes up to 100% CPU usage and the system is nearly unresponsive. I've seen it first with the full pull request for 3.14-rc1 and was able to track it down to this patch. Could you turn on the softlockup timer and see if you can get a backtrace of where it is stuck? In the meantime I will go through and see if I can pinpoint where it may be happening. Thanks, Josef This is what I've got with CONFIG_LOCKUP_DETECTOR=y CONFIG_HARDLOCKUP_DETECTOR=y # CONFIG_BOOTPARAM_HARDLOCKUP_PANIC is not set CONFIG_BOOTPARAM_HARDLOCKUP_PANIC_VALUE=0 # CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC is not set CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC_VALUE=0 CONFIG_DETECT_HUNG_TASK=y CONFIG_DEFAULT_HUNG_TASK_TIMEOUT=120 # CONFIG_BOOTPARAM_HUNG_TASK_PANIC is not set CONFIG_BOOTPARAM_HUNG_TASK_PANIC_VALUE=0 # CONFIG_PANIC_ON_OOPS is not set CONFIG_PANIC_ON_OOPS_VALUE=0 CONFIG_PANIC_TIMEOUT=0 CONFIG_SCHED_DEBUG=y CONFIG_SCHEDSTATS=y CONFIG_TIMER_STATS=y CONFIG_DEBUG_PREEMPT=y [ 203.610758] perf samples too long (2513 2500), lowering kernel.perf_event_max_sample_rate to 5 [ 360.625822] INFO: task btrfs-endio-wri:1075 blocked for more than 120 seconds. [ 360.625826] Not tainted 3.14.0-rc1 #19 [ 360.625828] echo 0 /proc/sys/kernel/hung_task_timeout_secs disables this message. [ 360.625829] btrfs-endio-wri D 880137c12d00 0 1075 2 0x [ 360.625833] 8800b6b10950 0002 00012d00 8800b6b10950 [ 360.625837] 8801325b3fd8 8800a2dcc000 8801325719e8 [ 360.625840] 880132571800 8800b635ba00 81256192 [ 360.625844] Call Trace: [ 360.625854] [81256192] ? wait_current_trans.isra.19+0xbb/0xdf [ 360.625858] [81089730] ? finish_wait+0x65/0x65 [ 360.625860] [8125740a] ? start_transaction+0x2f1/0x4e3 [ 360.625864] [81260b8a] ? btrfs_finish_ordered_io+0x44c/0x7b2 [ 360.625869] [81062dab] ? try_to_del_timer_sync+0x53/0x5e [ 360.625871] [81062ddc] ? del_timer_sync+0x26/0x43 [ 360.625875] [815a9a7b] ? schedule_timeout+0xeb/0x104 [ 360.625877] [81061f10] ? rcu_read_unlock_sched_notrace+0x11/0x11 [ 360.625882] [8127bd1c] ? worker_loop+0x162/0x4c3 [ 360.625884] [8127bbba] ? btrfs_queue_worker+0x275/0x275 [ 360.625888] [81072f8b] ? kthread+0xa3/0xab [ 360.625893] [810da9c1] ? trace_preempt_on+0xd/0x2a [ 360.625895] [8107] ? freeze_workqueues_begin+0x8/0x11e [ 360.625897] [81072ee8] ? __kthread_parkme+0x5a/0x5a [ 360.625901] [815ad8ec] ? ret_from_fork+0x7c/0xb0 [ 360.625903] [81072ee8] ? __kthread_parkme+0x5a/0x5a [ 360.625906] INFO: task btrfs-transacti:1084 blocked for more than 120 seconds. [ 360.625908] Not tainted 3.14.0-rc1 #19 [ 360.625909] echo 0 /proc/sys/kernel/hung_task_timeout_secs disables this message. [ 360.625910] btrfs-transacti D 880137c52d00 0 1084 2 0x [ 360.625912] 880132428950 0002 00012d00 880132428950 [ 360.625915] 8800b5a35fd8 8801331a5a70 8801331a5ae8 [ 360.625918] 8800aba981b8 00015000 0001 8126b986 [ 360.625921] Call Trace: [ 360.625925] [8126b986] ? btrfs_start_ordered_extent+0x91/0xdf [ 360.625928] [81089730] ? finish_wait+0x65/0x65 [ 360.625931] [8126bbff] ? btrfs_wait_ordered_range+0xab/0x10a [ 360.625934] [8128ac45] ? __btrfs_write_out_cache+0x43c/0x67f [ 360.625939] [8112d81b] ? kmem_cache_free+0x66/0x10d [ 360.625942] [8125970a] ? btrfs_update_inode_item+0xb9/0xcd [ 360.625944
Re: [PATCH] Btrfs: relocate csums properly with prealloc extents
On Fri, 27 Sep 2013 09:37:00 -0400 Josef Bacik jba...@fusionio.com wrote: A user reported a problem where they were getting csum errors when running a balance and running systemd's journal. This is because systemd is awesome and fallocate()'s its log space and writes into it. Unfortunately we assume that when we read in all the csums for an extent that they are sequential starting at the bytenr we care about. This obviously isn't the case for prealloc extents, where we could have written to the middle of the prealloc extent only, which means the csum would be for the bytenr in the middle of our range and not the front of our range. Fix this by offsetting the new bytenr we are logging to based on the original bytenr the csum was for. With this patch I no longer see the csum errors I was seeing. Thanks, Any assessment when this goes upstream? Until it hit Linus tree it won't won't appear in stable. And this seems rather important. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: balance induced csum errors, systemd-journal
On Tue, 24 Sep 2013 22:34:20 -0600 Chris Murphy li...@colorremedies.com wrote: OK so I think I'm narrowing this down to just the systemd journal, and it's not checksums that are corrupted, it's the journal itself. I doubt it's systemd-dependent, cause I've seen similar behaviour on a Gentoo system without systemd. Before balance the filesystem was ok, after I get root 257 inode 2875 errors 1800 root 257 inode 2881 errors 1800 root 257 inode 2969 errors 1800 root 257 inode 3063 errors 1800 root 257 inode 3120 errors 1800 root 257 inode 12407 errors 1800 root 257 inode 19496 errors 1800 root 257 inode 19500 errors 1800 root 257 inode 19564 errors 1800 root 257 inode 19643 errors 1800 root 257 inode 19693 errors 1800 root 257 inode 19949 errors 1800 root 257 inode 20178 errors 1800 root 257 inode 20320 errors 1800 root 257 inode 20406 errors 1800 root 257 inode 20512 errors 1800 root 257 inode 20586 errors 1800 root 257 inode 20654 errors 1800 root 257 inode 20727 errors 1800 root 257 inode 20728 errors 1800 root 257 inode 20821 errors 1800 root 257 inode 20843 errors 1800 root 257 inode 21062 errors 1800 root 257 inode 21078 errors 1800 root 257 inode 21222 errors 1800 root 257 inode 21356 errors 1800 root 257 inode 21437 errors 1800 root 257 inode 55082 errors 1800 root 257 inode 65343 errors 1800 root 257 inode 72413 errors 1800 on a fsck and scrub tells me that there are unfixable csum errors. Kernel is 3.12.0-rc2-00083-g4b97280. I've observed this two times, and every time only the first subvolume (root 257) was affected. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: hang on 3.9, 3.10-rc5
On Tue, 18 Jun 2013 17:19:04 + (UTC) Jon Nelson jnelson+bt...@jamponi.net wrote: Josef Bacik jbacik at fusionio.com writes: On Tue, Jun 11, 2013 at 11:43:30AM -0400, Sage Weil wrote: I'm also seeing this hang regularly with both 3.9 and 3.10-rc5. Is this is a known problem? In this case there is no powercycling; just a regular ceph-osd workload. .. I'm able to cause a complete kernel hang by defrag'ing even one file on 3.9.X (3.9.0 through 3.9.4, so far). I see similar behavior with autodefrag enabled. When fetching mails with claws (piped through bogofilter) the whole system got stuck more or less. I can switch between the tasks but everything involving I/O is hanging. Most time I was able to solve this with the sync command in a shell. I got only one time a backtrace from hung task checker: Jun 20 12:37:47 localhost kernel: INFO: task btrfs-cleaner:771 blocked for more than 120 seconds. Jun 20 12:37:47 localhost kernel: echo 0 /proc/sys/kernel/hung_task_timeout_secs disables this message. Jun 20 12:37:47 localhost kernel: btrfs-cleaner D 88011fc91740 0 771 2 0x Jun 20 12:37:47 localhost kernel: 88011aaff530 0002 880119929fd8 00011740 Jun 20 12:37:47 localhost kernel: 880119929fd8 88011aabddc0 880119928000 880119929cb8 Jun 20 12:37:47 localhost kernel: 8800aaac22b4 8800aaac22b0 8800aaac22b8 Jun 20 12:37:47 localhost kernel: Call Trace: Jun 20 12:37:47 localhost kernel: [81487fd4] ? schedule_preempt_disabled+0x19/0x1e Jun 20 12:37:47 localhost kernel: [81486fa1] ? __mutex_lock_common.isra.9+0x19d/0x283 Jun 20 12:37:47 localhost kernel: [8109856d] ? ondemand_readahead+0x15d/0x200 Jun 20 12:37:47 localhost kernel: [81486dbf] ? mutex_lock+0xe/0x1d Jun 20 12:37:47 localhost kernel: [81171579] ? btrfs_defrag_file+0x41a/0xa4e Jun 20 12:37:47 localhost kernel: [814889b0] ? _raw_spin_unlock+0x27/0x31 Jun 20 12:37:47 localhost kernel: [81159eb9] ? btrfs_run_defrag_inodes+0x1f7/0x2d3 Jun 20 12:37:47 localhost kernel: [8115100e] ? btrfs_run_delayed_iputs+0x44/0xbe Jun 20 12:37:47 localhost kernel: [811459d5] ? cleaner_kthread+0x89/0xf3 Jun 20 12:37:47 localhost kernel: [8114594c] ? transaction_kthread+0x17a/0x17a Jun 20 12:37:47 localhost kernel: [81042c00] ? kthread+0x7d/0x85 Jun 20 12:37:47 localhost kernel: [8104] ? thaw_workqueues+0xd3/0xff Jun 20 12:37:47 localhost kernel: [81042b83] ? __kthread_parkme+0x59/0x59 Jun 20 12:37:47 localhost kernel: [8148942c] ? ret_from_fork+0x7c/0xb0 Jun 20 12:37:47 localhost kernel: [81042b83] ? __kthread_parkme+0x59/0x59 Jun 20 12:37:47 localhost kernel: INFO: task konqueror:2384 blocked for more than 120 seconds. Jun 20 12:37:47 localhost kernel: echo 0 /proc/sys/kernel/hung_task_timeout_secs disables this message. Jun 20 12:37:47 localhost kernel: konqueror D 88011fc11740 0 2384 2133 0x Jun 20 12:37:47 localhost kernel: 8800c82dce20 0002 8800c0c55fd8 00011740 Jun 20 12:37:47 localhost kernel: 8800c0c55fd8 81a10400 8800c0c54000 8800c0c55e90 Jun 20 12:37:47 localhost kernel: 8800c40aeb44 8800c40aeb40 8800c40aeb48 Jun 20 12:37:47 localhost kernel: Call Trace: Jun 20 12:37:47 localhost kernel: [81487fd4] ? schedule_preempt_disabled+0x19/0x1e Jun 20 12:37:47 localhost kernel: [81486fa1] ? __mutex_lock_common.isra.9+0x19d/0x283 Jun 20 12:37:47 localhost kernel: [81486dbf] ? mutex_lock+0xe/0x1d Jun 20 12:37:47 localhost kernel: [810d3fb1] ? do_unlinkat+0x88/0x17a Jun 20 12:37:47 localhost kernel: [814889b0] ? _raw_spin_unlock+0x27/0x31 Jun 20 12:37:47 localhost kernel: [8100ae75] ? syscall_trace_enter+0xcf/0x145 Jun 20 12:37:47 localhost kernel: [814896cb] ? tracesys+0xdd/0xe2 Jun 20 12:37:47 localhost kernel: INFO: task konqueror:2385 blocked for more than 120 seconds. Jun 20 12:37:47 localhost kernel: echo 0 /proc/sys/kernel/hung_task_timeout_secs disables this message. Jun 20 12:37:47 localhost kernel: konqueror D 88011fc91740 0 2385 2133 0x Jun 20 12:37:47 localhost kernel: 8800c82dd5f0 0002 8800c0d1dfd8 00011740 Jun 20 12:37:47 localhost kernel: 8800c0d1dfd8 88011aabddc0 8800c0d1c000 8800c0d1de90 Jun 20 12:37:47 localhost kernel: 8800c40aeb44 8800c40aeb40 8800c40aeb48 Jun 20 12:37:47 localhost kernel: Call Trace: Jun 20 12:37:47 localhost kernel: [81487fd4] ? schedule_preempt_disabled+0x19/0x1e Jun 20 12:37:47 localhost kernel: [81486fa1] ? __mutex_lock_common.isra.9+0x19d/0x283 Jun 20 12:37:47 localhost kernel: [81486dbf] ? mutex_lock+0xe/0x1d Jun 20 12:37:47 localhost kernel: [810d3fb1] ?
Re: WARNING: at fs/btrfs/extent_map.c:77 free_extent_map
On Tue, 12 Mar 2013 09:39:35 +0800 Liu Bo bo.li@oracle.com wrote: Hi Johannes, Could you please tell us what mount options you're with? thanks, liubo The Filesystem has six subvolumes, so mount options are: noatime,inode_cache,autodefrag,subvolid=... for each subvol. I was able to reproduce it and getting a full backtrace with netconsole: [ cut here ] WARNING: at fs/btrfs/extent_map.c:77 free_extent_map+0x64/0x76() Hardware name: EasyNote TK81 Modules linked in: netconsole configfs nfsd exportfs auth_rpcgss nfs_acl fuse nfs lockd sunrpc snd_hda_codec_hdmi ath9k sr_mod snd_hda_codec_realtek acpi_cpufreq broadcom tg3 acer_wmi k10temp mperf ath9k_common ath9k_hw ath snd_hda_intel snd_hda_codec snd_hwdep snd_pcm cdrom snd_page_alloc snd_timer snd wmi soundcore i2c_piix4 ohci_hcd Pid: 15870, comm: bogofilter Not tainted 3.9.0-rc2-00112-g7c6baa3 #294 Call Trace: [8102abc2] ? warn_slowpath_common+0x76/0x8c [8115dcff] ? free_extent_map+0x64/0x76 [8115bc57] ? btrfs_drop_extent_cache+0x363/0x39f [81152db4] ? __cow_file_range+0x175/0x3c1 [8108ef08] ? find_get_pages_contig+0x100/0x115 [8114bb02] ? join_transaction.isra.34+0x30f/0x31a [8114d9f7] ? start_transaction+0x2d8/0x3e8 [8115383e] ? cow_file_range+0xa9/0xc5 [811538f7] ? run_delalloc_range+0x9d/0x33b [8116139b] ? free_extent_state+0x12/0x21 [81163fa3] ? __extent_writepage+0x1a8/0x5d8 [811635ae] ? end_extent_writepage+0x5d/0x5d [8116451d] ? extent_write_cache_pages.isra.29.constprop.47+0x14a/0x255 [81164836] ? extent_writepages+0x49/0x60 [81150146] ? btrfs_update_inode_item+0xde/0xde [8108fc58] ? __filemap_fdatawrite_range+0x4d/0x52 [8115a192] ? btrfs_sync_file+0x48/0x203 [810e783b] ? do_fsync+0x2b/0x50 [810e7a42] ? sys_fdatasync+0xb/0xf [814877d2] ? system_call_fastpath+0x16/0x1b ---[ end trace a3b02a44716bacc5 ]--- [ cut here ] WARNING: at fs/btrfs/extent_map.c:77 free_extent_map+0x64/0x76() Hardware name: EasyNote TK81 Modules linked in: netconsole configfs nfsd exportfs auth_rpcgss nfs_acl fuse nfs lockd sunrpc snd_hda_codec_hdmi ath9k sr_mod snd_hda_codec_realtek acpi_cpufreq broadcom tg3 acer_wmi k10temp mperf ath9k_common ath9k_hw ath snd_hda_intel snd_hda_codec snd_hwdep snd_pcm cdrom snd_page_alloc snd_timer snd wmi soundcore i2c_piix4 ohci_hcd Pid: 15870, comm: bogofilter Tainted: GW3.9.0-rc2-00112-g7c6baa3 #294 Call Trace: [8102abc2] ? warn_slowpath_common+0x76/0x8c [8115dcff] ? free_extent_map+0x64/0x76 [8115bc57] ? btrfs_drop_extent_cache+0x363/0x39f [81152db4] ? __cow_file_range+0x175/0x3c1 [8108ef08] ? find_get_pages_contig+0x100/0x115 [8114bb02] ? join_transaction.isra.34+0x30f/0x31a [8114d9f7] ? start_transaction+0x2d8/0x3e8 [8115383e] ? cow_file_range+0xa9/0xc5 [811538f7] ? run_delalloc_range+0x9d/0x33b [8116139b] ? free_extent_state+0x12/0x21 [81163fa3] ? __extent_writepage+0x1a8/0x5d8 [811635ae] ? end_extent_writepage+0x5d/0x5d [8116451d] ? extent_write_cache_pages.isra.29.constprop.47+0x14a/0x255 [81164836] ? extent_writepages+0x49/0x60 [81150146] ? btrfs_update_inode_item+0xde/0xde [8108fc58] ? __filemap_fdatawrite_range+0x4d/0x52 [8115a192] ? btrfs_sync_file+0x48/0x203 [810e783b] ? do_fsync+0x2b/0x50 [810e7a42] ? sys_fdatasync+0xb/0xf [814877d2] ? system_call_fastpath+0x16/0x1b ---[ end trace a3b02a44716bacc6 ]--- [ cut here ] WARNING: at fs/btrfs/extent_map.c:77 free_extent_map+0x64/0x76() Hardware name: EasyNote TK81 Modules linked in: netconsole configfs nfsd exportfs auth_rpcgss nfs_acl fuse nfs lockd sunrpc snd_hda_codec_hdmi ath9k sr_mod snd_hda_codec_realtek acpi_cpufreq broadcom tg3 acer_wmi k10temp mperf ath9k_common ath9k_hw ath snd_hda_intel snd_hda_codec snd_hwdep snd_pcm cdrom snd_page_alloc snd_timer snd wmi soundcore i2c_piix4 ohci_hcd Pid: 15887, comm: bogofilter Tainted: GW3.9.0-rc2-00112-g7c6baa3 #294 Call Trace: [8102abc2] ? warn_slowpath_common+0x76/0x8c [8115dcff] ? free_extent_map+0x64/0x76 [8115bc57] ? btrfs_drop_extent_cache+0x363/0x39f [81152db4] ? __cow_file_range+0x175/0x3c1 [8114bb02] ? join_transaction.isra.34+0x30f/0x31a [8114d9f7] ? start_transaction+0x2d8/0x3e8 [8115383e] ? cow_file_range+0xa9/0xc5 [811538f7] ? run_delalloc_range+0x9d/0x33b [8116139b] ? free_extent_state+0x12/0x21 [81163fa3] ? __extent_writepage+0x1a8/0x5d8 [811635ae] ? end_extent_writepage+0x5d/0x5d [8116451d] ? extent_write_cache_pages.isra.29.constprop.47+0x14a/0x255 [81164836] ? extent_writepages+0x49/0x60 [81150146] ?
WARNING: at fs/btrfs/extent_map.c:77 free_extent_map
Since the updates for linux-3.9 I've had three or four times a system freeze and only a reset (Magic SysRq) helped. After the reboot I found a bunch of this in syslog: Mar 11 21:56:09 localhost kernel: [ cut here ] Mar 11 21:56:09 localhost kernel: WARNING: at fs/btrfs/extent_map.c:77 free_extent_map+0x64/0x76() Mar 11 21:56:09 localhost kernel: Hardware name: EasyNote TK81 Mar 11 21:56:09 localhost kernel: Modules linked in: nfsv4 nfsd exportfs auth_rpcgss nfs_acl fuse nfs lockd sunrpc snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_intel ath9k snd_hda_codec ath9k_common ath9k_hw acer_wmi snd_hwdep snd_pcm ath sr_mod wmi broadcom snd_page_alloc snd_timer cdrom tg3 k10temp snd acpi_cpufreq ohci_hcd soundcore i2c_piix4 mperf Mar 11 21:56:09 localhost kernel: Pid: 11260, comm: bogofilter Tainted: G W3.9.0-rc2 #293 Mar 11 21:56:09 localhost kernel: Call Trace: Mar 11 21:56:09 localhost kernel: [8102abc2] ? warn_slowpath_common+0x76/0x8c Mar 11 21:56:09 localhost kernel: [8115dcff] ? free_extent_map+0x64/0x76 Mar 11 21:56:09 localhost kernel: [8115bc57] ? btrfs_drop_extent_cache+0x363/0x39f Mar 11 21:56:09 localhost kernel: [81152db4] ? __cow_file_range+0x175/0x3c1 Mar 11 21:56:09 localhost kernel: [8114bb02] ? join_transaction.isra.34+0x30f/0x31a Mar 11 21:56:09 localhost kernel: [8114d9f7] ? start_transaction+0x2d8/0x3e8 Mar 11 21:56:09 localhost kernel: [8115383e] ? cow_file_range+0xa9/0xc5 Mar 11 21:56:09 localhost kernel: [811538f7] ? run_delalloc_range+0x9d/0x33b Mar 11 21:56:09 localhost kernel: [8116139b] ? free_extent_state+0x12/0x21 Mar 11 21:56:09 localhost kernel: [81163fa3] ? __extent_writepage+0x1a8/0x5d8 Mar 11 21:56:09 localhost kernel: [811635ae] ? end_extent_writepage+0x5d/0x5d Mar 11 21:56:09 localhost kernel: [8116451d] ? extent_write_cache_pages.isra.29.constprop.47+0x14a/0x255 Mar 11 21:56:09 localhost kernel: [81164836] ? extent_writepages+0x49/0x60 Mar 11 21:56:09 localhost kernel: [81150146] ? btrfs_update_inode_item+0xde/0xde Mar 11 21:56:09 localhost kernel: [8108fc58] ? __filemap_fdatawrite_range+0x4d/0x52 Mar 11 21:56:09 localhost kernel: [8115a192] ? btrfs_sync_file+0x48/0x203 Mar 11 21:56:09 localhost kernel: [810c85ff] ? vfs_write+0xaf/0xf8 Mar 11 21:56:09 localhost kernel: [810e783b] ? do_fsync+0x2b/0x50 Mar 11 21:56:09 localhost kernel: [810e7a42] ? sys_fdatasync+0xb/0xf Mar 11 21:56:09 localhost kernel: [814877d2] ? system_call_fastpath+0x16/0x1b Mar 11 21:56:09 localhost kernel: ---[ end trace 3eaea449d8d56f92 ]--- As far as I remeber, it happend when fetching emails with claws. But it's not a reliable testcase. Another trace from the first time I found it in the logs. But here the system didn't hang: Mar 4 14:28:35 localhost kernel: [ cut here ] Mar 4 14:28:35 localhost kernel: WARNING: at fs/btrfs/extent_map.c:77 free_extent_map+0x64/0x76() Mar 4 14:28:35 localhost kernel: Hardware name: EasyNote TK81 Mar 4 14:28:35 localhost kernel: Modules linked in: nfsd exportfs auth_rpcgss nfs_acl fuse nfs lockd sunrpc snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_intel ath9k snd_hda_codec ath9k_common snd_hwdep snd_pcm broadcom ath9k_hw snd_page_alloc ath sr_mod snd_timer acer_wmi snd cdrom wmi tg3 ohci_hcd soundcore k10temp edac_core acpi_cpufreq i2c_piix4 mperf Mar 4 14:28:35 localhost kernel: Pid: 1574, comm: flush-btrfs-1 Not tainted 3.9.0-rc1 #289 Mar 4 14:28:35 localhost kernel: Call Trace: Mar 4 14:28:35 localhost kernel: [8102ab92] ? warn_slowpath_common+0x76/0x8c Mar 4 14:28:35 localhost kernel: [8115dc7b] ? free_extent_map+0x64/0x76 Mar 4 14:28:35 localhost kernel: [8115bbd3] ? btrfs_drop_extent_cache+0x363/0x39f Mar 4 14:28:35 localhost kernel: [81152d2d] ? __cow_file_range+0x175/0x3c1 Mar 4 14:28:36 localhost kernel: [81487830] ? _raw_spin_unlock+0x1c/0x28 Mar 4 14:28:36 localhost kernel: [81160de3] ? release_extent_buffer.isra.25+0x90/0x97 Mar 4 14:28:36 localhost kernel: [81153673] ? run_delalloc_nocow+0x6fa/0x795 Mar 4 14:28:36 localhost kernel: [81153837] ? run_delalloc_range+0x64/0x33b Mar 4 14:28:36 localhost kernel: [81161317] ? free_extent_state+0x12/0x21 Mar 4 14:28:36 localhost kernel: [81163f1f] ? __extent_writepage+0x1a8/0x5d8 Mar 4 14:28:36 localhost kernel: [8116352a] ? end_extent_writepage+0x5d/0x5d Mar 4 14:28:36 localhost kernel: [811d4b69] ? cpumask_any_but+0x25/0x34 Mar 4 14:28:36 localhost kernel: [810a5259] ? vma_interval_tree_subtree_search+0x33/0x55 Mar 4 14:28:36 localhost kernel: [810b07b8] ? page_mkclean+0x107/0x119 Mar 4 14:28:36 localhost kernel: [81164499] ? extent_write_cache_pages.isra.29.constprop.47+0x14a/0x255 Mar 4 14:28:36
Re: Another defrag question
On Thu, 21 Feb 2013 18:47:28 +0100 Swâmi Petaramesh sw...@petaramesh.org wrote: Le 21/02/2013 18:25, Hugo Mills a écrit : Correct. But btrfs isn't at that stage yet. It's getting visibly closer, but it's not quite there. Hence the very strong recommendation to keep up with the latest code. Hugo. The matter is that BTRFS had many early adopters just because it is - and has been for long now - in the mainline Linux kernel, so supposed stable and good choice for the future. And it's marked as EXPERIMENTAL. So if you want to join the game, you have to accept the rules. OTOH my 6th machine runs native ZFS on Linux, and I have to tell that it shows orders of magnitude better performance and never gave me a single problem in several (3 ?) years. Only upgrading the distro is always a big frightening and problematic. And initial installation was a bit tricky. You didn't, many other had. I remember a lot of threads in the OpenSolaris forum, where the solution for problems was: recreate your filesystem, replay your backup. Everytime I show my Linux machines to friends and say : “Hey, I got the most advanced filesystem on earth !” I soon get the answer “Oh boy, that's the slowest machine boot and FS I've ever seen since I was reading floppy disks on my 386SX in 1991 ! Can you really live with this ?” Did you presented it while (re)creating the inode_cache? Sounds a little like that. So, for not quite there and the return codes +20 that have been a minor pain in the arse for a couple years but the line is still in the code... I can understand developer's PoV, been there, done that, but still, BTRFS might in the end lose a numer of its early adopters if it keeps being not quite there too long. Shitfing to ZFS is just a PPA and 2 apt-get install commands away... It will definitely be easier than start playing with mainline PPA Ubuntu kernels... So why do you bother with btrfs, if ZFS fit your needs? regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs and 1 billion small files
Am Mon, 7 May 2012 12:39:28 +0100 schrieb Hugo Mills h...@carfax.org.uk: On Mon, May 07, 2012 at 01:15:26PM +0200, Alessio Focardi wrote: ... That's a very clever suggestion, I'm preparing a test server right now: going to use the -m single option. Any other suggestion regarding format options? pagesize? leafsize? I'm not sure about these -- some values of them definitely break things. I think they are required to be the same, and that you could take them up to 64k with no major problems, but do check that first with someone who actually knows. First, if you have this filesystem as rootfs, a separate /boot partition is needed. Grub is unable to boot from btrfs with different node-/leafsize. Second a very recent kernel is needed (linux-3.4-rc1 at least). regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel BUG at fs/btrfs/delayed-inode.c:1466!
Am Mon, 12 Mar 2012 15:21:49 +0100 schrieb Jacek Luczak difrost.ker...@gmail.com: 2) A *regression* in 3.3.0-rc6-00197-g9f8050c - completely unusable as reports ENOSPC - to reproduce, mount volume and issue: # CNT=1 ; while [ $CNT -lt 1 ] ; do rm -f /btrfs/dd ; ! touch /btrfs/dd echo $CNT break ; CNT=$(( $CNT + 1 )) ; done On my host this shows: # CNT=1 ; while [ $CNT -lt 1 ] ; do rm -f /btrfs/dd ; ! touch /btrfs/dd echo $CNT break ; CNT=$(( $CNT + 1 )) ; done touch: cannot touch `/btrfs/dd': No space left on device 423 - remount to reset: # CNT=1 ; while [ $CNT -lt 1 ] ; do rm -f /btrfs/dd ; ! touch /btrfs/dd echo $CNT break ; CNT=$(( $CNT + 1 )) ; done touch: cannot touch `/btrfs/dd': No space left on device 1 # umount /btrfs/ # mount -t btrfs /dev/vg00/btrfs /btrfs/ -o noatime,nodatacow,defaults # CNT=1 ; while [ $CNT -lt 1 ] ; do rm -f /btrfs/dd ; ! touch /vdd echo $CNT break ; CNT=$(( $CNT + 1 )) ; done touch: cannot touch `/btrfs/dd': No space left on device 423 - bisected down to 5500cdb (Btrfs: increase the global block reserve estimates). After reverting this one Linus master works for me again. With above patch reverted after a longer run I've got ENOSPC again: 1) # df -hP /btrfs FilesystemSize Used Avail Use% Mounted on /dev/mapper/vg00-btrfs 195G 179G 11G 95% /btrfs 2) # rm -f /btrfs/dd rm: cannot remove `/btrfs/dd': No space left on device 3) strace unlink(/btrfs/dd)= -1 ENOSPC (No space left on device) 4) last message from kernel (except WARN_ONs): btrfs: fail to dirty inode 116882385 I've remouted volume and after that I've been able to remove dd file from volume. In dmesg there's bunch on new WARN_ONs: [ cut here ] WARNING: at fs/btrfs/extent-tree.c:4185 btrfs_free_block_groups+0x17d/0x2b8 [btrfs]() Hardware name: ProLiant BL460c G6 Modules linked in: btrfs zlib_deflate lzo_compress ipmi_devintf autofs4 be2iscsi iscsi_boot_sysfs ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_addr iscsi_tcp bnx2i cnic uio ipv6 cxgb3i libcxgbi iw_cxgb3 ib_core cxgb3 libiscsi_tcp libiscsi scsi_transport_iscsi dm_mirror dm_region_hash dm_log dm_multipath video battery acpi_pad acpi_ipmi ac parport usbhid evdev acpi_power_meter radeon ttm drm_kms_helper drm hwmon ipmi_si bnx2x backlight i2c_algo_bit ipmi_msghandler i2c_core hpilo mdio hpwdt psmouse uhci_hcd ehci_hcd Pid: 9518, comm: umount Tainted: GW 3.3.0-rc6-00197-g9f8050c-dirty #1 Call Trace: [8105caca] ? print_oops_end_marker+0x9/0x20 [a031a8a7] ? btrfs_free_block_groups+0x17d/0x2b8 [btrfs] [8105cc92] ? warn_slowpath_common+0x78/0x8d [a031a8a7] ? btrfs_free_block_groups+0x17d/0x2b8 [btrfs] [a0327473] ? close_ctree+0x1e1/0x380 [btrfs] [811320c2] ? dispose_list+0x27/0x31 [8113248c] ? evict_inodes+0xc5/0xcc [8112098c] ? generic_shutdown_super+0x4d/0xc1 [81120a67] ? kill_anon_super+0x9/0x11 [a030a3aa] ? btrfs_kill_super+0xd/0x73 [btrfs] [81120c81] ? deactivate_locked_super+0x2f/0x5f [81135d5f] ? sys_umount+0x2c1/0x30b [813ef0f9] ? system_call_fastpath+0x16/0x1b ---[ end trace fd6da849e53b77dd ]--- [ cut here ] WARNING: at fs/btrfs/extent-tree.c:4186 btrfs_free_block_groups+0x198/0x2b8 [btrfs]() Hardware name: ProLiant BL460c G6 Modules linked in: btrfs zlib_deflate lzo_compress ipmi_devintf autofs4 be2iscsi iscsi_boot_sysfs ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_addr iscsi_tcp bnx2i cnic uio ipv6 cxgb3i libcxgbi iw_cxgb3 ib_core cxgb3 libiscsi_tcp libiscsi scsi_transport_iscsi dm_mirror dm_region_hash dm_log dm_multipath video battery acpi_pad acpi_ipmi ac parport usbhid evdev acpi_power_meter radeon ttm drm_kms_helper drm hwmon ipmi_si bnx2x backlight i2c_algo_bit ipmi_msghandler i2c_core hpilo mdio hpwdt psmouse uhci_hcd ehci_hcd Pid: 9518, comm: umount Tainted: GW 3.3.0-rc6-00197-g9f8050c-dirty #1 Call Trace: [8105caca] ? print_oops_end_marker+0x9/0x20 [a031a8c2] ? btrfs_free_block_groups+0x198/0x2b8 [btrfs] [8105cc92] ? warn_slowpath_common+0x78/0x8d [a031a8c2] ? btrfs_free_block_groups+0x198/0x2b8 [btrfs] [a0327473] ? close_ctree+0x1e1/0x380 [btrfs] [811320c2] ? dispose_list+0x27/0x31 [8113248c] ? evict_inodes+0xc5/0xcc [8112098c] ? generic_shutdown_super+0x4d/0xc1 [81120a67] ? kill_anon_super+0x9/0x11 [a030a3aa] ? btrfs_kill_super+0xd/0x73 [btrfs] [81120c81] ? deactivate_locked_super+0x2f/0x5f [81135d5f] ? sys_umount+0x2c1/0x30b [813ef0f9] ? system_call_fastpath+0x16/0x1b ---[ end trace fd6da849e53b77de ]--- [ cut here ] WARNING: at fs/btrfs/extent-tree.c:4187 btrfs_free_block_groups+0x1b3/0x2b8 [btrfs]() Hardware name: ProLiant BL460c G6 Modules
Re: [PATCH] Btrfs: hold enough space for global_rsv
Am Fri, 09 Mar 2012 09:28:56 +0800 schrieb Liu Bo liubo2...@cn.fujitsu.com: On 03/09/2012 03:22 AM, Johannes Hirte wrote: Am Tue, 6 Mar 2012 14:50:32 +0100 schrieb Johannes Hirte johannes.hi...@fem.tu-ilmenau.de: I've backed up the filesystem, deleted the subvolumes, recreated them and copied the data back. Now everything seems to work again. I've also a full image of the damaged filesystem for further investigation. If someone has an idea for testing, I'm happy to try it. It's much worse than I thought. After a short time the same error happened again (no space left on device). So recreated the filesystem (mkbtrfs with default values) and copied the data from the backup back, but the error still came back. I'm now on kernel 3.2 which seems to work. I'll try to bisect the bad commit. For info, df says: OK, plz show us the results after your bisect, let's narrow down where goes wrong. thanks, liubo Bisect points again to: 5500cdbe14d7435e04f66ff3cfb8ecd8b8e44ebf is the first bad commit commit 5500cdbe14d7435e04f66ff3cfb8ecd8b8e44ebf Author: Liu Bo liubo2...@cn.fujitsu.com Date: Thu Feb 23 10:49:04 2012 -0500 Btrfs: increase the global block reserve estimates When doing IO with large amounts of data fragmentation, the global block reserve calulations are too low. This increases them to avoid ENOSPC crashes. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com Signed-off-by: Chris Mason chris.ma...@oracle.com The revision before is working and reverting this commit from master works too. But as mentioned before, I'm not sure if this is root cause. First time I've seen the error it happened without this patch too later on. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: filesystem full when it's not? out of inodes? huh?
Am Sat, 25 Feb 2012 20:05:13 -0800 schrieb Fahrzin Hemmati fahh...@gmail.com: No, at least not yet, nor am I aware of any plans for subvolume quotas, though I could be wrong. Arne Jansen is working on it, IIRC. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs: hold enough space for global_rsv
Am Tue, 6 Mar 2012 14:50:32 +0100 schrieb Johannes Hirte johannes.hi...@fem.tu-ilmenau.de: I've backed up the filesystem, deleted the subvolumes, recreated them and copied the data back. Now everything seems to work again. I've also a full image of the damaged filesystem for further investigation. If someone has an idea for testing, I'm happy to try it. It's much worse than I thought. After a short time the same error happened again (no space left on device). So recreated the filesystem (mkbtrfs with default values) and copied the data from the backup back, but the error still came back. I'm now on kernel 3.2 which seems to work. I'll try to bisect the bad commit. For info, df says: Filesystem Size Used Avail Use% Mounted on rootfs 200G 128G 69G 66% / /dev/sda1 200G 128G 69G 66% / rc-svcdir 1.0M 128K 896K 13% /lib64/rc/init.d cgroup_root 10M 52K 10M 1% /sys/fs/cgroup udev 10M 168K 9.9M 2% /dev shm 2.0G 0 2.0G 0% /dev/shm /dev/sda1 200G 128G 69G 66% /home and btrfs fi df: Data: total=149.01GB, used=118.57GB System, DUP: total=8.00MB, used=24.00KB System: total=4.00MB, used=0.00 Metadata, DUP: total=6.38GB, used=4.55GB Metadata: total=8.00MB, used=0.0 Kernel 3.3-rc6 fails on this with no space left on device. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs: hold enough space for global_rsv
Am Tue, 28 Feb 2012 10:06:14 +0800 schrieb Liu Bo liubo2...@cn.fujitsu.com: On 02/27/2012 09:29 PM, Johannes Hirte wrote: Am Tue, 17 Jan 2012 17:51:59 +0800 schrieb Liu Bo liubo2...@cn.fujitsu.com: I've kept hitting enospc warnings of global_rsv while running defragment on files: btrfs: block rsv returned -28 WARNING: at fs/btrfs/extent-tree.c:5984 btrfs_alloc_free_block+0x333/0x340 [btrfs]() ... I used a fio jobs to create a file with lots of fragments: $ filefrag /mnt/btrfs/foobar /mnt/btrfs/foobar: 66964 extents found and then btrfs fi defrag /mnt/btrfs/foobar sync would pop the warnings. I found that the global_rsv size is just not enough for defragment, and didn't find any space leak in using global_rsv, so double it and go ahead. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/extent-tree.c |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 8603ee4..77ea23c 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -3979,7 +3979,7 @@ static u64 calc_global_metadata_size(struct btrfs_fs_info *fs_info) num_bytes += div64_u64(data_used + meta_used, 50); if (num_bytes * 3 meta_used) - num_bytes = div64_u64(meta_used, 3); + num_bytes = div64_u64(meta_used, 3) * 2; return ALIGN(num_bytes, fs_info-extent_root-leafsize 10); } This patch breakes my system. With this applied all services fail on boot with no space left messages. It's weird since this patch is just aiming to enlarge our metadata reservation count. so you've tried a revert or a bisect, right? Can you show me the environment or any log messages? thanks, liubo Sorry for the long delay. My system was really screwed up and took time to fix it. First, it wasn't your patch that made the system fail. At this time, it was the first revision that didn't work anymore. I don't know why this one. Short time later also earlier revisions showed that error. I was able to boot with a live system from USB stick. The filesystem was mountable and readable, but I couldn't modify or create a single file. Two or three times I got a btrfs: fail to dirty inode 256 error -28 but most times nothing was reported in the logs. The filesystem consists of three subvolumes, the default one, one for rootfs and one for home. If I did a defrag on the rootfs, I was able to create files. But after unmounting and remounting the filesystem, the same error appeared again. Also a balance of the filesystem resulted in no space error after some time. I've backed up the filesystem, deleted the subvolumes, recreated them and copied the data back. Now everything seems to work again. I've also a full image of the damaged filesystem for further investigation. If someone has an idea for testing, I'm happy to try it. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Btrfs: hold enough space for global_rsv
Am Tue, 17 Jan 2012 17:51:59 +0800 schrieb Liu Bo liubo2...@cn.fujitsu.com: I've kept hitting enospc warnings of global_rsv while running defragment on files: btrfs: block rsv returned -28 WARNING: at fs/btrfs/extent-tree.c:5984 btrfs_alloc_free_block+0x333/0x340 [btrfs]() ... I used a fio jobs to create a file with lots of fragments: $ filefrag /mnt/btrfs/foobar /mnt/btrfs/foobar: 66964 extents found and then btrfs fi defrag /mnt/btrfs/foobar sync would pop the warnings. I found that the global_rsv size is just not enough for defragment, and didn't find any space leak in using global_rsv, so double it and go ahead. Signed-off-by: Liu Bo liubo2...@cn.fujitsu.com --- fs/btrfs/extent-tree.c |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 8603ee4..77ea23c 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -3979,7 +3979,7 @@ static u64 calc_global_metadata_size(struct btrfs_fs_info *fs_info) num_bytes += div64_u64(data_used + meta_used, 50); if (num_bytes * 3 meta_used) - num_bytes = div64_u64(meta_used, 3); + num_bytes = div64_u64(meta_used, 3) * 2; return ALIGN(num_bytes, fs_info-extent_root-leafsize 10); } This patch breakes my system. With this applied all services fail on boot with no space left messages. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
how reliable is btrfsck?
After some problems with btrfs (Oopses), I've checked now the btrfs filesystems on my systems as a precaution. On the first system I got root 256 inode 404596 errors 400 root 256 inode 404603 errors 400 root 256 inode 409540 errors 400 root 256 inode 409562 errors 400 root 258 inode errors 400 root 258 inode 5556 errors 400 root 258 inode 27624 errors 400 root 258 inode 27659 errors 400 root 258 inode 29243 errors 400 root 258 inode 29255 errors 400 root 258 inode 29256 errors 400 root 258 inode 90400 errors 400 root 258 inode 90401 errors 400 root 258 inode 91155 errors 400 root 258 inode 166025 errors 400 found 71626346496 bytes used err is 1 total csum bytes: 68939428 total tree bytes: 1028964352 total fs tree bytes: 901152768 btree space waste bytes: 274375495 file data blocks allocated: 73391845376 referenced 70346993664 Btrfs v0.19-50-ge6bd18d This is a single disk system with three subvolumes additional to the default. On a second system with a RAID1 setup with two 500G disks I get: failed to read /dev/sr0 failed to read /dev/sr0 root 5 inode 1891143 errors 400 root 5 inode 1891166 errors 400 root 5 inode 1915207 errors 400 root 5 inode 1915214 errors 400 root 5 inode 1915531 errors 400 root 5 inode 1915547 errors 400 root 5 inode 1915599 errors 400 root 5 inode 1915750 errors 400 root 5 inode 1915777 errors 400 found 391921790976 bytes used err is 1 total csum bytes: 379086532 total tree bytes: 3737182208 total fs tree bytes: 3013656576 btree space waste bytes: 1024370594 file data blocks allocated: 388184608768 referenced 388184051712 Btrfs v0.19-36-g28da90f The output is identical for both disks. A third disk in this system doesn't show any errors. So the question is, is btrfsck miss-reporting something here? Or are these real errors? regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: defrag makes fragmentation worse
On Friday 10 June 2011 01:53:41 David Sterba wrote: On Fri, Jun 10, 2011 at 12:48:36AM +0200, Johannes Hirte wrote: I've observed several times that after a btrfs filesystem defrag a file was way more fragmented than before. For example, a file that was recently written, had 10 extents (output from filefrag). After a defrag filefrag showed more than 1900 extents. For curiosity, a simple copy of this defragmented file reduced the number of fragments to 1. With a different file I got 63 extents before and over 3000 extents after defrag. Do you have compression enabled? Or autodefrag mount option? No compression, only autodefrag was enabled this time. The last time before I saw this, autodefrag didn't exist. 'filefrag -v' will tell you size of the extents, would be interesting to see. Needed some tries but now I have one. Before defrag the file consisted of 174 extents. Now after defrag there are 786 extents. filefrag -v shows: Filesystem type is: 9123683e File size of test1 is 106857600 (26089 blocks, blocksize 4096) ext logical physical expected length flags 0 0 7037185 299 1 299 7989102 7037483 64 2 363 7037548 7989165 1 3 364 7989538 7037548 64 4 428 7037613 7989601 1 5 429 7990288 7037613 64 6 493 7037678 7990351 1 7 494 7992819 7037678 64 8 558 7037743 7992882 1 9 559 7993037 7037743 64 10 623 7037809 7993100 1 11 624 7993171 7037809 64 12 688 9547471 7993234 1 13 689 9547947 9547471 64 14 753 9547536 9548010 1 ... 89 16159 9696920 9696590 64 490 16223 9696655 9696983 1 491 16224 9700654 9696655 64 492 16288 9697127 9700717276 493 16564 9700718 9697402 64 494 16628 9697467 9700781 1 ... 781 25924 12575962 12535459 64 782 25988 12535524 12576025 1 783 25989 12576026 12535524 64 784 26053 12531601 12576089 1 785 26054 12541596 12531601 35 eof test1: 786 extents found It's no problem if defrag can't reduce the fragmentation. But in this case it shouldn't be done at all. AFAIK defragmentation just reads the file, marks all pages dirty and lets it be written back. If the free space is fragmented, so will be the newly written copy. I do not know if there is some logic comparing old and new extent layout (or if it's even possible). If there is some comparison between old and new, it seems to be broken (or the fiemap from btrfs). regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
defrag makes fragmentation worse
I've observed several times that after a btrfs filesystem defrag a file was way more fragmented than before. For example, a file that was recently written, had 10 extents (output from filefrag). After a defrag filefrag showed more than 1900 extents. For curiosity, a simple copy of this defragmented file reduced the number of fragments to 1. With a different file I got 63 extents before and over 3000 extents after defrag. It's no problem if defrag can't reduce the fragmentation. But in this case it shouldn't be done at all. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Quota Implementation
On Friday 03 June 2011 18:24:41 Arne Jansen wrote: Hi, If no one is already working on it, I'd like to take the Quota lock and see how far I come. Let me sketch out in short what I'm planning to do: - Quota will be subvolume based. Only the FS-trees and data extents will be accounted. - Quota Groups can be defined. Every quota group can comprise any number of subvolumes. A subvolume can be assigned to any number of quota groups. - A Quota Group can account/limit the total amount of space that is referenced by it and/or the amount of space that is exclusively referenced (i.e. referenced by no other quota group). - With this it is possible to define a hierarchical quota that need not necessarily reflect the filesystem hierarchy. - It is also possible to decide for each snapshot if it should be accounted into the parent group. So in a scenario where each subvolume reflect a user home, it's possible to have some snapshots accounted to the user and others not (e.g. the ones needed for system backups). - Quota information will be stored in new records, possibly in a separate tree. - It should be possible to change the Quota config and group assignments online, though this might need a full re-scan of the fs. - It does NOT include any kind of user/group (UID/GID) quota. Any addenda or arguments why it's impossible or insane welcome. What's the benefit of this complexity? Why not a more simple quota/reservation per subvolume? The semantics you described, can be achived by user/group quotas too. And we need them anyway. Perhaps this can be implemented together, reusing the code. Then we have the question if user/group quotas are per filesystem or per subvolume. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Having parent transid verify failed
On Thursday 05 May 2011 22:32:42 Chris Mason wrote: Excerpts from Konstantinos Skarlatos's message of 2011-05-05 16:27:54 -0400: I think i made some progress. When i tried to remove the directory that i suspect contains the problematic file, i got this on the console rm -rf serverloft/ Ok, our one bad block is in the extent allocation tree. This is going to be the very hardest thing to fix. Until I finish off the code to rebuild parts of the extent allocation tree, I think your best bet is to copy the files off. The big question is, what happened to make this error? Can you describe your setup in more detail? -chris It seems that I run into the same problem: parent transid verify failed on 32940560384 wanted 210334 found 210342 BUG: scheduling while atomic: chrome/17058/0x0002 Modules linked in: snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss fuse dm_crypt dm_mod usbhid snd_intel8x0 snd_ac97_codec sr_mod cdrom ac97_bus snd_pcm sg snd_timer snd e1000 fschmd uhci_hcd snd_page_alloc i2c_i801 [last unloaded: microcode] Pid: 17058, comm: chrome Tainted: GW 2.6.39 #29 Call Trace: [c13cf70c] ? schedule+0x78/0x6ef [c11acabb] ? generic_make_request+0x1d5/0x22f [c11acbad] ? submit_bio+0x98/0x9f [c118026a] ? btrfs_map_bio+0x1ab/0x1b5 [c13cfdc2] ? io_schedule+0x3f/0x50 [c105723d] ? sleep_on_page+0x5/0x8 [c13d0292] ? __wait_on_bit+0x31/0x58 [c1057238] ? __lock_page+0x52/0x52 [c1057388] ? wait_on_page_bit+0x5a/0x62 [c1037f92] ? autoremove_wake_function+0x29/0x29 [c117ab39] ? read_extent_buffer_pages+0x33a/0x3b5 [c115891f] ? btree_read_extent_buffer_pages.clone.51+0x44/0x9e [c11578b0] ? verify_parent_transid+0x147/0x147 [c11593aa] ? read_tree_block+0x2d/0x3e [c1144f90] ? read_block_for_search.clone.36+0xc3/0x35d [c11863bf] ? btrfs_tree_unlock+0x19/0x3a [c11420bb] ? unlock_up+0x88/0x9f [c1146f7e] ? btrfs_search_slot+0x39d/0x4fe [c1149fa1] ? lookup_inline_extent_backref+0x116/0x49b [c11773b0] ? set_extent_dirty+0x19/0x1d [c114cbd0] ? __btrfs_free_extent+0xe2/0x6c6 [c114fa28] ? run_clustered_refs+0x6ad/0x720 [c1191330] ? btrfs_find_ref_cluster+0x53/0x11f [c114fb53] ? btrfs_run_delayed_refs+0xb8/0x18d [c115d395] ? __btrfs_end_transaction+0x5a/0x17f [c115d4dc] ? btrfs_end_transaction+0x9/0xb [c1165e19] ? btrfs_evict_inode+0x190/0x1a7 [c1092c45] ? evict+0x56/0xeb [c108baa8] ? do_unlinkat+0xc3/0x103 [c13d1c90] ? sysenter_do_call+0x12/0x26 [c13d] ? console_conditional_schedule+0x8/0xf parent transid verify failed on 32940560384 wanted 210334 found 210342 BUG: scheduling while atomic: chrome/17058/0x0002 Modules linked in: snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss fuse dm_crypt dm_mod usbhid snd_intel8x0 snd_ac97_codec sr_mod cdrom ac97_bus snd_pcm sg snd_timer snd e1000 fschmd uhci_hcd snd_page_alloc i2c_i801 [last unloaded: microcode] Pid: 17058, comm: chrome Tainted: GW 2.6.39 #29 Call Trace: [c13cf70c] ? schedule+0x78/0x6ef [c11acabb] ? generic_make_request+0x1d5/0x22f [c11acbad] ? submit_bio+0x98/0x9f [c118026a] ? btrfs_map_bio+0x1ab/0x1b5 [c13cfdc2] ? io_schedule+0x3f/0x50 [c105723d] ? sleep_on_page+0x5/0x8 [c13d0292] ? __wait_on_bit+0x31/0x58 [c1057238] ? __lock_page+0x52/0x52 [c1057388] ? wait_on_page_bit+0x5a/0x62 [c1037f92] ? autoremove_wake_function+0x29/0x29 [c117ab39] ? read_extent_buffer_pages+0x33a/0x3b5 [c116bd50] ? lookup_extent_mapping+0x5a/0x148 [c115891f] ? btree_read_extent_buffer_pages.clone.51+0x44/0x9e [c11578b0] ? verify_parent_transid+0x147/0x147 [c11593aa] ? read_tree_block+0x2d/0x3e [c1144f90] ? read_block_for_search.clone.36+0xc3/0x35d [c11863bf] ? btrfs_tree_unlock+0x19/0x3a [c11420bb] ? unlock_up+0x88/0x9f [c1146f7e] ? btrfs_search_slot+0x39d/0x4fe [c1149fa1] ? lookup_inline_extent_backref+0x116/0x49b [c11773b0] ? set_extent_dirty+0x19/0x1d [c114cbd0] ? __btrfs_free_extent+0xe2/0x6c6 [c114fa28] ? run_clustered_refs+0x6ad/0x720 [c1191330] ? btrfs_find_ref_cluster+0x53/0x11f [c114fb53] ? btrfs_run_delayed_refs+0xb8/0x18d [c115d395] ? __btrfs_end_transaction+0x5a/0x17f [c115d4dc] ? btrfs_end_transaction+0x9/0xb [c1165e19] ? btrfs_evict_inode+0x190/0x1a7 [c1092c45] ? evict+0x56/0xeb [c108baa8] ? do_unlinkat+0xc3/0x103 [c13d1c90] ? sysenter_do_call+0x12/0x26 [c13d] ? console_conditional_schedule+0x8/0xf parent transid verify failed on 32940560384 wanted 210334 found 210342 BUG: scheduling while atomic: chrome/17058/0x0002 Modules linked in: snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss fuse dm_crypt dm_mod usbhid snd_intel8x0 snd_ac97_codec sr_mod cdrom ac97_bus snd_pcm sg snd_timer snd e1000 fschmd uhci_hcd snd_page_alloc i2c_i801 [last unloaded: microcode] Pid: 17058, comm: chrome Tainted: GW 2.6.39 #29 Call Trace: [c13cf70c] ? schedule+0x78/0x6ef [c11acabb] ? generic_make_request+0x1d5/0x22f [c11acbad] ? submit_bio+0x98/0x9f [c118026a] ?
Re: BUG: unable to handle kernel NULL pointer dereference at (null)
On Wednesday 06 April 2011 19:15:41 Josef Bacik wrote:
On Wed, Apr 06, 2011 at 01:10:38PM +0200, Johannes Hirte wrote:
On Tuesday 05 April 2011 23:57:53 Josef Bacik wrote:
Now it hit
Man I cannot catch a break. I hope this is the last one. Thanks,
Ok I give up, I just cleaned it all up and don't mark the pages as dirty
unless we're actually going to succeed at writing them. This should fix
everything
---
fs/btrfs/ctree.h|5 ++
fs/btrfs/file.c | 21 +++
fs/btrfs/free-space-cache.c | 117
--- 3 files changed, 69
insertions(+), 74 deletions(-)
diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index 3458b57..0d00a07 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -2576,6 +2576,11 @@ int btrfs_drop_extents(struct btrfs_trans_handle
*trans, struct inode *inode, int btrfs_mark_extent_written(struct
btrfs_trans_handle *trans,
struct inode *inode, u64 start, u64 end);
int btrfs_release_file(struct inode *inode, struct file *file);
+void btrfs_drop_pages(struct page **pages, size_t num_pages);
+int btrfs_dirty_pages(struct btrfs_root *root, struct inode *inode,
+ struct page **pages, size_t num_pages,
+ loff_t pos, size_t write_bytes,
+ struct extent_state **cached);
/* tree-defrag.c */
int btrfs_defrag_leaves(struct btrfs_trans_handle *trans,
diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c
index e621ea5..75899a0 100644
--- a/fs/btrfs/file.c
+++ b/fs/btrfs/file.c
@@ -104,7 +104,7 @@ static noinline int btrfs_copy_from_user(loff_t pos,
int num_pages, /*
* unlocks pages after btrfs_file_write is done with them
*/
-static noinline void btrfs_drop_pages(struct page **pages, size_t
num_pages) +void btrfs_drop_pages(struct page **pages, size_t num_pages)
{
size_t i;
for (i = 0; i num_pages; i++) {
@@ -127,16 +127,13 @@ static noinline void btrfs_drop_pages(struct page
**pages, size_t num_pages) * this also makes the decision about creating
an inline extent vs * doing real data extents, marking pages dirty and
delalloc as required. */
-static noinline int dirty_and_release_pages(struct btrfs_root *root,
- struct file *file,
- struct page **pages,
- size_t num_pages,
- loff_t pos,
- size_t write_bytes)
+int btrfs_dirty_pages(struct btrfs_root *root, struct inode *inode,
+ struct page **pages, size_t num_pages,
+ loff_t pos, size_t write_bytes,
+ struct extent_state **cached)
{
int err = 0;
int i;
- struct inode *inode = fdentry(file)-d_inode;
u64 num_bytes;
u64 start_pos;
u64 end_of_last_block;
@@ -149,7 +146,7 @@ static noinline int dirty_and_release_pages(struct
btrfs_root *root,
end_of_last_block = start_pos + num_bytes - 1;
err = btrfs_set_extent_delalloc(inode, start_pos, end_of_last_block,
- NULL);
+ cached);
if (err)
return err;
@@ -992,9 +989,9 @@ static noinline ssize_t __btrfs_buffered_write(struct
file *file, }
if (copied 0) {
- ret = dirty_and_release_pages(root, file, pages,
- dirty_pages, pos,
- copied);
+ ret = btrfs_dirty_pages(root, inode, pages,
+ dirty_pages, pos, copied,
+ NULL);
if (ret) {
btrfs_delalloc_release_space(inode,
dirty_pages PAGE_CACHE_SHIFT);
diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c
index f561c95..a3f420d 100644
--- a/fs/btrfs/free-space-cache.c
+++ b/fs/btrfs/free-space-cache.c
@@ -508,6 +508,7 @@ int btrfs_write_out_cache(struct btrfs_root *root,
struct inode *inode;
struct rb_node *node;
struct list_head *pos, *n;
+ struct page **pages;
struct page *page;
struct extent_state *cached_state = NULL;
struct btrfs_free_cluster *cluster = NULL;
@@ -517,13 +518,13 @@ int btrfs_write_out_cache(struct btrfs_root *root,
u64 start, end, len;
u64 bytes = 0;
u32 *crc, *checksums;
- pgoff_t index = 0, last_index = 0;
unsigned long first_page_offset;
- int num_checksums;
+ int index = 0, num_pages = 0;
int entries = 0;
int bitmaps = 0;
int ret = 0;
bool next_page = false;
+ bool out_of_space = false;
root = root-fs_info-tree_root
Re: BUG: unable to handle kernel NULL pointer dereference at (null)
On Tuesday 05 April 2011 23:57:53 Josef Bacik wrote:
Now it hit
Man I cannot catch a break. I hope this is the last one. Thanks,
Josef
---
fs/btrfs/free-space-cache.c | 32
1 files changed, 32 insertions(+), 0 deletions(-)
diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c
index 74bc432..b8052be 100644
--- a/fs/btrfs/free-space-cache.c
+++ b/fs/btrfs/free-space-cache.c
@@ -522,6 +522,7 @@ int btrfs_write_out_cache(struct btrfs_root *root,
int bitmaps = 0;
int ret = 0;
bool next_page = false;
+ bool out_of_space = false;
root = root-fs_info-tree_root;
@@ -629,6 +630,11 @@ int btrfs_write_out_cache(struct btrfs_root *root,
offset = start_offset;
}
+ if (index last_index) {
+ out_of_space = true;
+ break;
+ }
+
page = find_get_page(inode-i_mapping, index);
addr = kmap(page);
@@ -732,6 +738,10 @@ int btrfs_write_out_cache(struct btrfs_root *root,
struct btrfs_free_space *entry =
list_entry(pos, struct btrfs_free_space, list);
+ if (index last_index) {
+ out_of_space = true;
+ break;
+ }
page = find_get_page(inode-i_mapping, index);
addr = kmap(page);
@@ -754,6 +764,28 @@ int btrfs_write_out_cache(struct btrfs_root *root,
index++;
}
+ if (out_of_space) {
+ page = find_get_page(inode-i_mapping, 0);
+
+ /*
+ * Have to do the normal stuff in case writeback gets started on
+ * this page before we invalidate it.
+ */
+ ClearPageChecked(page);
+ set_page_extent_mapped(page);
+ SetPageUptodate(page);
+ set_page_dirty(page);
+ unlock_page(page);
+ page_cache_release(page);
+ page_cache_release(page);
+
+ ret = 0;
+ unlock_extent_cached(BTRFS_I(inode)-io_tree, 0,
+ i_size_read(inode) - 1, cached_state,
+ GFP_NOFS);
+ goto out_free;
+ }
+
/* Zero out the rest of the pages just to make sure */
while (index = last_index) {
void *addr;
Sorry no, it still hits the BUG() in inode.c (line 1565). It takes longer to
hit than before but is still reproducible.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: BUG: unable to handle kernel NULL pointer dereference at (null)
On Wednesday 06 April 2011 22:47:28 Jordan Patterson wrote: Hi Josef: I tried your latest patch, since I had the same issue from the first email. With the patch applied, I am now hitting the BUG_ON(block_group-total_bitmaps = max_bitmaps); in add_new_bitmap in fs/btrfs/free-space-cache.c:1246 as soon as I mount the filesystem, with or without -o clear_cache. It works fine in 2.6.38. I get the same error after mounting with clear_cache under 2.6.38 and rebooting into the current kernel with your patch. Jordan What filesystem is it and how did you mount it with -o clear_cache? If it is your rootfs did you applied clear_cache to /etc/fstab or your bootloader? If it was the latter it won't work. For the rootfs you need to add it to the boot options. For me this worked every time. Josef, is there any way to detect a wrong cache, saved by an pre-2.6.39 kernel and discard it? regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
BUG: unable to handle kernel NULL pointer dereference at (null)
With the latest btrfs changes, I got this Oops when doing rm on a large directory: BUG: unable to handle kernel NULL pointer dereference at (null) IP: [c101c838] kunmap+0x46/0x46 *pdpt = 34a85001 *pde = Oops: [#1] PREEMPT SMP last sysfs file: /sys/devices/virtual/vtconsole/vtcon1/uevent Modules linked in: snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss fuse dm_crypt dm_mod usbhid snd_intel8x0 snd_ac97_codec ac97_bus snd_pcm snd_timer sr_mod cdrom sg snd fschmd e1000 uhci_hcd snd_page_alloc i2c_i801 [last unloaded: microcode] Pid: 1156, comm: btrfs-transacti Tainted: GW 2.6.39-rc1-00262- gc53813f #20 FUJITSU SIEMENS SCENIC P / SCENICO P/D1561 EIP: 0060:[c101c838] EFLAGS: 00010296 CPU: 1 EIP is at kmap+0x0/0x38 EAX: EBX: ECX: EDX: 0010 ESI: f5bc6400 EDI: f3c75520 EBP: f3c755f0 ESP: f58f9e10 DS: 007b ES: 007b FS: 00d8 GS: SS: 0068 Process btrfs-transacti (pid: 1156, ti=f58f8000 task=f6516f40 task.ti=f58f8000) Stack: c1186d15 ffc22000 f58f9ec0 0010 f3c75610 f5885780 f52339e8 0009 f5bc6400 0001 f6415800 f3c75638 08bb f5bc63c0 f58857b4 f60b68a0 0040 f52338e8 ffc22000 0008 0010 Call Trace: [c1186d15] ? btrfs_write_out_cache+0x60c/0xa3c [c114a815] ? btrfs_write_dirty_block_groups+0x400/0x494 [c11566a7] ? commit_cowonly_roots+0xa9/0x180 [c1157799] ? btrfs_commit_transaction+0x2ee/0x59c [c1037c85] ? wake_up_bit+0x16/0x16 [c1152a83] ? transaction_kthread+0x149/0x1d6 [c101d1b9] ? complete+0x28/0x36 [c115293a] ? btrfs_congested_fn+0x5d/0x5d [c10379c4] ? kthread+0x63/0x68 [c1037961] ? kthread_worker_fn+0xeb/0xeb [c13cba36] ? kernel_thread_helper+0x6/0xd Code: 8d 8a 00 e4 54 c1 2b 8a 8c e7 54 c1 81 f9 00 08 00 00 74 11 81 f9 00 0c 00 00 75 0e 83 3d 10 2f 60 c1 02 75 05 e9 5e a3 04 00 c3 8b 10 c1 ea 1e c1 e2 0a 8d 8a 00 e4 54 c1 2b 8a 8c e7 54 c1 81 EIP: [c101c838] kmap+0x0/0x38 SS:ESP 0068:f58f9e10 CR2: ---[ end trace c8511126ee91dfdf ]--- This is the second Oops. On the first one I wasn't able to catch the backtrace, but IIRC the bug happend on kmap not kunmap the first time. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: BUG: unable to handle kernel NULL pointer dereference at (null)
On Tuesday 05 April 2011 19:42:03 Josef Bacik wrote:
On Tue, Apr 05, 2011 at 07:38:13PM +0200, Johannes Hirte wrote:
With the latest btrfs changes, I got this Oops when doing rm on a large
directory:
BUG: unable to handle kernel NULL pointer dereference at (null)
IP: [c101c838] kunmap+0x46/0x46
*pdpt = 34a85001 *pde =
Oops: [#1] PREEMPT SMP
last sysfs file: /sys/devices/virtual/vtconsole/vtcon1/uevent
Modules linked in: snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device
snd_pcm_oss snd_mixer_oss fuse dm_crypt dm_mod usbhid snd_intel8x0
snd_ac97_codec ac97_bus snd_pcm snd_timer sr_mod cdrom sg snd fschmd
e1000 uhci_hcd snd_page_alloc i2c_i801 [last unloaded: microcode]
Pid: 1156, comm: btrfs-transacti Tainted: GW 2.6.39-rc1-00262-
gc53813f #20 FUJITSU SIEMENS SCENIC P / SCENICO P/D1561
EIP: 0060:[c101c838] EFLAGS: 00010296 CPU: 1
EIP is at kmap+0x0/0x38
EAX: EBX: ECX: EDX: 0010
ESI: f5bc6400 EDI: f3c75520 EBP: f3c755f0 ESP: f58f9e10
DS: 007b ES: 007b FS: 00d8 GS: SS: 0068
Process btrfs-transacti (pid: 1156, ti=f58f8000 task=f6516f40
task.ti=f58f8000)
Stack:
c1186d15 ffc22000 f58f9ec0 0010 f3c75610 f5885780 f52339e8
0009 f5bc6400 0001 f6415800 f3c75638 08bb f5bc63c0
f58857b4 f60b68a0 0040 f52338e8 ffc22000 0008 0010
Call Trace:
[c1186d15] ? btrfs_write_out_cache+0x60c/0xa3c
[c114a815] ? btrfs_write_dirty_block_groups+0x400/0x494
[c11566a7] ? commit_cowonly_roots+0xa9/0x180
[c1157799] ? btrfs_commit_transaction+0x2ee/0x59c
[c1037c85] ? wake_up_bit+0x16/0x16
[c1152a83] ? transaction_kthread+0x149/0x1d6
[c101d1b9] ? complete+0x28/0x36
[c115293a] ? btrfs_congested_fn+0x5d/0x5d
[c10379c4] ? kthread+0x63/0x68
[c1037961] ? kthread_worker_fn+0xeb/0xeb
[c13cba36] ? kernel_thread_helper+0x6/0xd
Code: 8d 8a 00 e4 54 c1 2b 8a 8c e7 54 c1 81 f9 00 08 00 00 74 11 81 f9
00 0c 00 00 75 0e 83 3d 10 2f 60 c1 02 75 05 e9 5e a3 04 00 c3 8b 10
c1 ea 1e c1 e2 0a 8d 8a 00 e4 54 c1 2b 8a 8c e7 54 c1 81
EIP: [c101c838] kmap+0x0/0x38 SS:ESP 0068:f58f9e10
CR2:
---[ end trace c8511126ee91dfdf ]---
This is the second Oops. On the first one I wasn't able to catch the
backtrace, but IIRC the bug happend on kmap not kunmap the first time.
Yeah I think I know what this is but I need somebody to verify it for me.
Can you run with this patch and let me know what happens? Thanks,
Josef
diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c
index 74bc432..5e6f4b3 100644
--- a/fs/btrfs/free-space-cache.c
+++ b/fs/btrfs/free-space-cache.c
@@ -624,6 +624,7 @@ int btrfs_write_out_cache(struct btrfs_root *root,
next_page = false;
+ BUG_ON(index last_index);
if (index == 0) {
start_offset = first_page_offset;
offset = start_offset;
@@ -732,6 +733,7 @@ int btrfs_write_out_cache(struct btrfs_root *root,
struct btrfs_free_space *entry =
list_entry(pos, struct btrfs_free_space, list);
+ BUG_ON(index last_index);
page = find_get_page(inode-i_mapping, index);
addr = kmap(page);
Hm, I tried but now I hit the
BUG_ON(block_group-total_bitmaps = max_bitmaps); in add_new_bitmap in
fs/btrfs/free-space-cache.c:1255 when booting the system.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: BUG: unable to handle kernel NULL pointer dereference at (null)
On Tuesday 05 April 2011 23:12:27 Josef Bacik wrote:
On Tue, Apr 05, 2011 at 11:08:52PM +0200, Johannes Hirte wrote:
On Tuesday 05 April 2011 21:31:43 Josef Bacik wrote:
On Tue, Apr 05, 2011 at 09:21:55PM +0200, Johannes Hirte wrote:
On Tuesday 05 April 2011 20:53:24 Josef Bacik wrote:
On Tue, Apr 05, 2011 at 08:52:21PM +0200, Johannes Hirte wrote:
On Tuesday 05 April 2011 19:42:03 Josef Bacik wrote:
On Tue, Apr 05, 2011 at 07:38:13PM +0200, Johannes Hirte wrote:
With the latest btrfs changes, I got this Oops when doing rm
on a large directory:
BUG: unable to handle kernel NULL pointer dereference at
(null) IP: [c101c838] kunmap+0x46/0x46
*pdpt = 34a85001 *pde =
Oops: [#1] PREEMPT SMP
last sysfs file: /sys/devices/virtual/vtconsole/vtcon1/uevent
Modules linked in: snd_seq_oss snd_seq_midi_event snd_seq
snd_seq_device snd_pcm_oss snd_mixer_oss fuse dm_crypt dm_mod
usbhid snd_intel8x0 snd_ac97_codec ac97_bus snd_pcm snd_timer
sr_mod cdrom sg snd fschmd e1000 uhci_hcd snd_page_alloc
i2c_i801 [last unloaded: microcode]
Pid: 1156, comm: btrfs-transacti Tainted: GW
2.6.39-rc1-00262- gc53813f #20 FUJITSU SIEMENS SCENIC P /
SCENICO P/D1561
EIP: 0060:[c101c838] EFLAGS: 00010296 CPU: 1
EIP is at kmap+0x0/0x38
EAX: EBX: ECX: EDX: 0010
ESI: f5bc6400 EDI: f3c75520 EBP: f3c755f0 ESP: f58f9e10
DS: 007b ES: 007b FS: 00d8 GS: SS: 0068
Process btrfs-transacti (pid: 1156, ti=f58f8000 task=f6516f40
task.ti=f58f8000)
Stack:
c1186d15 ffc22000 f58f9ec0 0010 f3c75610
f5885780 f52339e8 0009 f5bc6400 0001
f6415800 f3c75638 08bb f5bc63c0 f58857b4 f60b68a0
0040 f52338e8 ffc22000 0008 0010
Call Trace:
[c1186d15] ? btrfs_write_out_cache+0x60c/0xa3c
[c114a815] ? btrfs_write_dirty_block_groups+0x400/0x494
[c11566a7] ? commit_cowonly_roots+0xa9/0x180
[c1157799] ? btrfs_commit_transaction+0x2ee/0x59c
[c1037c85] ? wake_up_bit+0x16/0x16
[c1152a83] ? transaction_kthread+0x149/0x1d6
[c101d1b9] ? complete+0x28/0x36
[c115293a] ? btrfs_congested_fn+0x5d/0x5d
[c10379c4] ? kthread+0x63/0x68
[c1037961] ? kthread_worker_fn+0xeb/0xeb
[c13cba36] ? kernel_thread_helper+0x6/0xd
Code: 8d 8a 00 e4 54 c1 2b 8a 8c e7 54 c1 81 f9 00 08 00 00
74 11 81 f9 00 0c 00 00 75 0e 83 3d 10 2f 60 c1 02 75 05 e9
5e a3 04 00 c3 8b 10 c1 ea 1e c1 e2 0a 8d 8a 00 e4 54 c1
2b 8a 8c e7 54 c1 81 EIP: [c101c838] kmap+0x0/0x38 SS:ESP
0068:f58f9e10 CR2:
---[ end trace c8511126ee91dfdf ]---
This is the second Oops. On the first one I wasn't able to
catch the backtrace, but IIRC the bug happend on kmap not
kunmap the first time.
Yeah I think I know what this is but I need somebody to verify
it for me. Can you run with this patch and let me know what
happens? Thanks,
Josef
diff --git a/fs/btrfs/free-space-cache.c
b/fs/btrfs/free-space-cache.c index 74bc432..5e6f4b3 100644
--- a/fs/btrfs/free-space-cache.c
+++ b/fs/btrfs/free-space-cache.c
@@ -624,6 +624,7 @@ int btrfs_write_out_cache(struct btrfs_root
*root,
next_page = false;
+ BUG_ON(index last_index);
if (index == 0) {
start_offset = first_page_offset;
offset = start_offset;
@@ -732,6 +733,7 @@ int btrfs_write_out_cache(struct btrfs_root
*root,
struct btrfs_free_space *entry =
list_entry(pos, struct btrfs_free_space, list);
+ BUG_ON(index last_index);
page = find_get_page(inode-i_mapping, index);
addr = kmap(page);
Hm, I tried but now I hit the
BUG_ON(block_group-total_bitmaps = max_bitmaps); in
add_new_bitmap in fs/btrfs/free-space-cache.c:1255 when booting
the system.
Can you mount -o clear_cache to make sure it's not the cache thats
causing that? Thanks,
Josef
Mounting with clear_cache under 2.6.38 helped. I was able to boot
and test with your patch an hit the second BUG_ON on
free-space-cache.c:738.
Perfect can you try this and verify you don't panic anymore please?
Thanks,
Josef
---
fs/btrfs/free-space-cache.c | 18 ++
1 files changed, 18 insertions(+), 0
Re: [PATCH] btrfs file write debugging patch
On Monday 07 March 2011 20:56:50 Maria Wikström wrote: mån 2011-03-07 klockan 00:07 -0600 skrev Mitch Harder: On Sun, Mar 6, 2011 at 6:58 PM, Chris Mason chris.ma...@oracle.com wrote: Excerpts from Chris Mason's message of 2011-03-06 13:00:27 -0500: Excerpts from Mitch Harder's message of 2011-03-05 11:50:14 -0500: I've constructed a test patch that is currently addressing all the issues on my system. The portion of Openmotif that was having issues with page faults works correctly with this patch, and gcc-4.4.5 builds without issue. I extracted only the portion of the first patch that corrects the handling of dirty_pages when copied==0, and incorporated the second patch that falls back to one-page-at-a-time if there are troubles with page faults. Just to make sure I understand, could you please post the full combined path that was giving you trouble with gcc? We do need to make sure the pages are properly up to date if we fall back to partial writes. Ok, I was able to reproduce this easily with fsx. The problem is that I wasn't making sure the last partial page in the write was up to date when it was also the first page in the write. Here is the updated patch, it has all the fixes we've found so far: This latest patch that Chris has sent out fixes the issues I've been encountering. I can build gcc-4.4.5 without problems. Also, the portion of Openmotif that was having issues with page faults is working correctly. Let me know if you still would like to see the path names for the portions of the gcc-4.4.5 build that were giving me issues. I didn't save that information, but I can regenerate it. But it sounds like it's irrelevant now. With the patch I can compile libgcrypt without any problem, so it solves my problems to. Can confirm this. And the bug seems to be hardware-related. On my Pentium4 system it was 100% reproducible, on my Atom-based system I couldn't trigger it. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs file write debugging patch
On Monday 28 February 2011 02:46:05 Chris Mason wrote: Excerpts from Mitch Harder's message of 2011-02-25 13:43:37 -0500: Some clarification on my previous message... After looking at my ftrace log more closely, I can see where Btrfs is trying to release the allocated pages. However, the calculation for the number of dirty_pages is equal to 1 when copied == 0. So I'm seeing at least two problems: (1) It keeps looping when copied == 0. (2) One dirty page is not being released on every loop even though copied == 0 (at least this problem keeps it from being an infinite loop by eventually exhausting reserveable space on the disk). Hi everyone, There are actually tow bugs here. First the one that Mitch hit, and a second one that still results in bad file_write results with my debugging hunks (the first two hunks below) in place. My patch fixes Mitch's bug by checking for copied == 0 after btrfs_copy_from_user and going the correct delalloc accounting. This one looks solved, but you'll notice the patch is bigger. First, I add some random failures to btrfs_copy_from_user() by failing everyone once and a while. This was much more reliable than trying to use memory pressure than making copy_from_user fail. If copy_from_user fails and we partially update a page, we end up with a page that may go away due to memory pressure. But, btrfs_file_write assumes that only the first and last page may have good data that needs to be read off the disk. This patch ditches that code and puts it into prepare_pages instead. But I'm still having some errors during long stress.sh runs. Ideas are more than welcome, hopefully some other timezones will kick in ideas while I sleep. At least it doesn't fix the emerge-problem for me. The behavior is now the same as with 2.6.38-rc3. It needs a 'emerge --oneshot dev-libs/libgcrypt' with no further interaction to get the emerge-process hang with a svn-process consuming 100% CPU. I can cancel the emerge-process with ctrl-c but the spawned svn-process stays and it needs a reboot to get rid of it. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page
On Friday 28 January 2011 02:26:43 Zhong, Xin wrote: Please try the fix in below link: http://www.spinics.net/lists/linux-btrfs/msg08051.html Thanks! This doesn't fix it for me. At least there is a difference. Whereas the svn process started consuming 100% CPU without any further interaction before, the system just hang now. The svn process starts eating the CPU when I cancel the emerge via ctrl-c. Additional I see a flush-btrfs task now consuming CPU time. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel BUG at fs/btrfs/inode.c:806
On Thursday 02 December 2010 17:19:56 Chris Mason wrote: Excerpts from Johannes Hirte's message of 2010-12-01 08:11:01 -0500: On one of my machines with btrfs I got this bug: entry offset 29085974528, bytes 4096, bitmap no entry offset 29162995712, bytes 20480, bitmap yes entry offset 29171744768, bytes 4096, bitmap no block group has cluster?: no 0 blocks of free space at or bigger than bytes is block group 29834084352 has 1073741824 bytes, 1072648192 used 0 pinned 0 reserved Well, you've had an ENOSPC explosion. The block group messages where way more, too much for the dmesg log buffer. Kernel is a 2.6.37-rc3+ without the latest btrfs-fixes. The bug occurred when compiling openoffice.org. After the bug a 'df -h' showed: df -h: FilesystemSize Used Avail Use% Mounted on rootfs 21G 17G 770M 96% / /dev/root 21G 17G 770M 96% / rc-svcdir 1.0M 108K 916K 11% /lib/rc/init.d udev 10M 116K 9.9M 2% /dev shm 1013M 0 1013M 0% /dev/shm /dev/sda2 66G 46G 20G 71% /home /dev/sdb1 75G 56G 19G 75% /mnt/windows Which of these filesystems were you compiling on? On /. It's a gentoo system and the bug happened during an 'emerge openoffice'. The compilation ist usually done under /var/tmp/portage. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel BUG at fs/btrfs/inode.c:806
On Thursday 02 December 2010 17:52:50 Johannes Hirte wrote: On Thursday 02 December 2010 17:19:56 Chris Mason wrote: Excerpts from Johannes Hirte's message of 2010-12-01 08:11:01 -0500: On one of my machines with btrfs I got this bug: entry offset 29085974528, bytes 4096, bitmap no entry offset 29162995712, bytes 20480, bitmap yes entry offset 29171744768, bytes 4096, bitmap no block group has cluster?: no 0 blocks of free space at or bigger than bytes is block group 29834084352 has 1073741824 bytes, 1072648192 used 0 pinned 0 reserved Well, you've had an ENOSPC explosion. The block group messages where way more, too much for the dmesg log buffer. Kernel is a 2.6.37-rc3+ without the latest btrfs-fixes. The bug occurred when compiling openoffice.org. After the bug a 'df -h' showed: df -h: FilesystemSize Used Avail Use% Mounted on rootfs 21G 17G 770M 96% / /dev/root 21G 17G 770M 96% / rc-svcdir 1.0M 108K 916K 11% /lib/rc/init.d udev 10M 116K 9.9M 2% /dev shm 1013M 0 1013M 0% /dev/shm /dev/sda2 66G 46G 20G 71% /home /dev/sdb1 75G 56G 19G 75% /mnt/windows Which of these filesystems were you compiling on? On /. It's a gentoo system and the bug happened during an 'emerge openoffice'. The compilation ist usually done under /var/tmp/portage. Btw, I was able to reproduce this with a second try to emerge openoffice. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: disk space caching generation missmatch
On Thursday 02 December 2010 21:34:10 Josef Bacik wrote: On Wed, Dec 01, 2010 at 10:40:29PM +0100, Johannes Hirte wrote: On Wednesday 01 December 2010 22:22:45 Johannes Hirte wrote: On Wednesday 01 December 2010 21:03:13 Josef Bacik wrote: On Wed, Dec 01, 2010 at 08:56:14PM +0100, Johannes Hirte wrote: On Wednesday 01 December 2010 18:40:18 Josef Bacik wrote: On Wed, Dec 01, 2010 at 05:46:14PM +0100, Johannes Hirte wrote: After enabling disk space caching I've observed several log entries like this: btrfs: free space inode generation (0) did not match free space cache generation (169594) for block group 15464398848 I'm not sure, but it seems this happens on every reboot. Is this something to worry about? So that usually means 1 of a couple of things 1) You didn't have space for us to save the free space cache 2) When trying to write out the cache we hit one of those cases where we would deadlock so we couldn't write the cache out It's nothing to worry about, it's doing what it is supposed to. However I'd like to know why we're not able to write out the cache. Are you running close to full? Thanks, Josef I think there should be enough free space: Ok it doesn't look like theres an actual problem, we're just being sub-optimal. Take out the other patch and apply this one, boot into that kernel and then reboot and then give me the dmesg. Here it comes: Initializing cgroup subsys cpuset Linux version 2.6.37-rc4-space-cache-dbg-00022-g620731b-dirty (r...@netbook) (gcc version 4.5.1 (Gentoo 4.5.1-r1 p1.3, pie-0.4.5) ) #126 SMP PREEMPT Fri Dec 3 00:40:04 CET 2010 Atom PSE erratum detected, BIOS microcode update recommended BIOS-provided physical RAM map: BIOS-e820: - 0009f800 (usable) BIOS-e820: 0009f800 - 000a (reserved) BIOS-e820: 000dc000 - 000e4000 (reserved) BIOS-e820: 000e8000 - 0010 (reserved) BIOS-e820: 0010 - 7f6d (usable) BIOS-e820: 7f6d - 7f6e2000 (ACPI data) BIOS-e820: 7f6e2000 - 7f6e3000 (ACPI NVS) BIOS-e820: 7f6e3000 - 8000 (reserved) BIOS-e820: e000 - f000 (reserved) BIOS-e820: fec0 - fec1 (reserved) BIOS-e820: fed0 - fed00400 (reserved) BIOS-e820: fed14000 - fed1a000 (reserved) BIOS-e820: fed1c000 - fed9 (reserved) BIOS-e820: fee0 - fee01000 (reserved) BIOS-e820: ff00 - 0001 (reserved) NX (Execute Disable) protection: active DMI present. DMI: M912/M912, BIOS R02 05/04/2009 e820 update range: - 0001 (usable) == (reserved) e820 remove range: 000a - 0010 (usable) last_pfn = 0x7f6d0 max_arch_pfn = 0x100 MTRR default type: uncachable MTRR fixed ranges enabled: 0-9 write-back A-B uncachable C-C write-protect D-D uncachable E-F write-protect MTRR variable ranges enabled: 0 base 0 mask 08000 write-back 1 base 07F70 mask 0FFF0 uncachable 2 base 07F80 mask 0FF80 uncachable 3 disabled 4 disabled 5 disabled 6 disabled 7 disabled x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106 Scanning 0 areas for low memory corruption initial memory mapped : 0 - 01a0 init_memory_mapping: -37bfe000 00 - 0037bfe000 page 4k kernel direct mapping tables up to 37bfe000 @ 183f000-1a0 ACPI: RSDP 000f7e40 00024 (v02 GBT ) ACPI: XSDT 7f6dc705 00084 (v01 GBTGBTUACPI 0604 LTP ) ACPI: FACP 7f6e1bd2 000F4 (v03 INTEL CALISTGA 0604 ALAN 0001) ACPI: DSDT 7f6dd907 04257 (v01 INTEL CALISTGA 0604 INTL 20050624) ACPI: FACS 7f6e2fc0 00040 ACPI: APIC 7f6e1cc6 00068 (v01 INTEL CALISTGA 0604 LOHR 005A) ACPI: HPET 7f6e1d2e 00038 (v01 INTEL CALISTGA 0604 LOHR 005A) ACPI: MCFG 7f6e1d66 0003C (v01 INTEL CALISTGA 0604 LOHR 005A) ACPI: SLIC 7f6e1da2 00176 (v01 GBTGBTUACPI 0604 TBD 0001) ACPI: TCPA 7f6e1f18 00032 (v01 PTLTD CALISTGA 0604 PTL 0001) ACPI: TMOR 7f6e1f4a 00026 (v01 PTLTD 0604 PTL 0003) ACPI: APIC 7f6e1f70 00068 (v01 PTLTD ? APIC 0604 LTP ) ACPI: BOOT 7f6e1fd8 00028 (v01 PTLTD $SBFTBL$ 0604 LTP 0001) ACPI: SSDT 7f6dcd25 0025F (v01 PmRef Cpu0Tst 3000 INTL 20050624) ACPI: SSDT 7f6dcc7f 000A6 (v01 PmRef Cpu1Tst 3000 INTL 20050624) ACPI: SSDT 7f6dc789 004F6 (v02 PmRefCpuPm 3000 INTL 20050624) ACPI: BIOS bug: multiple APIC/MADT found, using 0 ACPI: If acpi_apic_instance=2 works better, notify linux-a...@vger.kernel.org ACPI: Local APIC address
Re: disk space caching generation missmatch
On Friday 03 December 2010 01:44:49 C Anthony Risinger wrote: Did you fix that typo I posted? C Anthony [mobile] Yes, without fix it wouldn't compile. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel BUG at fs/btrfs/inode.c:806
On Thursday 02 December 2010 20:21:30 Chris Mason wrote: Excerpts from Johannes Hirte's message of 2010-12-02 12:02:16 -0500: On Thursday 02 December 2010 17:52:50 Johannes Hirte wrote: On Thursday 02 December 2010 17:19:56 Chris Mason wrote: Excerpts from Johannes Hirte's message of 2010-12-01 08:11:01 -0500: On one of my machines with btrfs I got this bug: entry offset 29085974528, bytes 4096, bitmap no entry offset 29162995712, bytes 20480, bitmap yes entry offset 29171744768, bytes 4096, bitmap no block group has cluster?: no 0 blocks of free space at or bigger than bytes is block group 29834084352 has 1073741824 bytes, 1072648192 used 0 pinned 0 reserved Well, you've had an ENOSPC explosion. The block group messages where way more, too much for the dmesg log buffer. Kernel is a 2.6.37-rc3+ without the latest btrfs-fixes. The bug occurred when compiling openoffice.org. After the bug a 'df -h' showed: df -h: FilesystemSize Used Avail Use% Mounted on rootfs 21G 17G 770M 96% / /dev/root 21G 17G 770M 96% / rc-svcdir 1.0M 108K 916K 11% /lib/rc/init.d udev 10M 116K 9.9M 2% /dev shm 1013M 0 1013M 0% /dev/shm /dev/sda2 66G 46G 20G 71% /home /dev/sdb1 75G 56G 19G 75% /mnt/windows Which of these filesystems were you compiling on? On /. It's a gentoo system and the bug happened during an 'emerge openoffice'. The compilation ist usually done under /var/tmp/portage. Btw, I was able to reproduce this with a second try to emerge openoffice. Ok, there is one related fix in the git tree right now that you don't have. I'm not 100% sure it'll fix this, but it can't hurt. -chris Unfortunately it didn't fixed the bug. The system crashed again on emerging openoffice. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
disk space caching generation missmatch
After enabling disk space caching I've observed several log entries like this: btrfs: free space inode generation (0) did not match free space cache generation (169594) for block group 15464398848 I'm not sure, but it seems this happens on every reboot. Is this something to worry about? regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: disk space caching generation missmatch
On Wednesday 01 December 2010 18:40:18 Josef Bacik wrote: On Wed, Dec 01, 2010 at 05:46:14PM +0100, Johannes Hirte wrote: After enabling disk space caching I've observed several log entries like this: btrfs: free space inode generation (0) did not match free space cache generation (169594) for block group 15464398848 I'm not sure, but it seems this happens on every reboot. Is this something to worry about? So that usually means 1 of a couple of things 1) You didn't have space for us to save the free space cache 2) When trying to write out the cache we hit one of those cases where we would deadlock so we couldn't write the cache out It's nothing to worry about, it's doing what it is supposed to. However I'd like to know why we're not able to write out the cache. Are you running close to full? Thanks, Josef I think there should be enough free space: df -h FilesystemSize Used Avail Use% Mounted on rootfs 41G 29G 8.4G 78% / /dev/root 41G 29G 8.4G 78% / rc-svcdir 1.0M 112K 912K 11% /lib/rc/init.d udev 10M 284K 9.8M 3% /dev shm 1008M 0 1008M 0% /dev/shm /dev/sda3 108G 90G 15G 87% /home btrfs filesystem df / Data: total=34.48GB, used=26.13GB System, DUP: total=8.00MB, used=12.00KB System: total=4.00MB, used=0.00 Metadata, DUP: total=2.75GB, used=1.26GB Metadata: total=8.00MB, used=0.00 btrfs filesystem df /home Data: total=88.01GB, used=84.84GB System, DUP: total=8.00MB, used=20.00KB System: total=4.00MB, used=0.00 Metadata, DUP: total=4.00GB, used=2.43GB Metadata: total=8.00MB, used=0.00 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: disk space caching generation missmatch
On Wednesday 01 December 2010 21:03:13 Josef Bacik wrote:
On Wed, Dec 01, 2010 at 08:56:14PM +0100, Johannes Hirte wrote:
On Wednesday 01 December 2010 18:40:18 Josef Bacik wrote:
On Wed, Dec 01, 2010 at 05:46:14PM +0100, Johannes Hirte wrote:
After enabling disk space caching I've observed several log entries
like this:
btrfs: free space inode generation (0) did not match free space cache
generation (169594) for block group 15464398848
I'm not sure, but it seems this happens on every reboot. Is this
something to
worry about?
So that usually means 1 of a couple of things
1) You didn't have space for us to save the free space cache
2) When trying to write out the cache we hit one of those cases where we
would
deadlock so we couldn't write the cache out
It's nothing to worry about, it's doing what it is supposed to. However
I'd
like to know why we're not able to write out the cache. Are you running
close
to full? Thanks,
Josef
I think there should be enough free space:
Hmm well then we're hitting one of the other corner cases. Can you run with
this debug thread and reboot. Hopefully it will tell me why we're not saving
the free space cache. Thanks,
Josef
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 87aae66..4fd5659 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -2794,13 +2794,17 @@ again:
if (i_size_read(inode) 0) {
ret = btrfs_truncate_free_space_cache(root, trans, path,
inode);
- if (ret)
+ if (ret) {
+ printk(KERN_ERR truncate free space cache failed for
%llu, %d\n,
+block_group-key.objectid, ret);
goto out_put;
+ }
}
spin_lock(block_group-lock);
if (block_group-cached != BTRFS_CACHE_FINISHED) {
spin_unlock(block_group-lock);
+ printk(KERN_ERR block group %llu not cached\n,
block_group-key.objectid);
goto out_put;
}
spin_unlock(block_group-lock);
@@ -2820,8 +2824,10 @@ again:
num_pages *= PAGE_CACHE_SIZE;
ret = btrfs_check_data_free_space(inode, num_pages);
- if (ret)
+ if (ret) {
+ printk(KERN_ERR not enough free space for cache %llu\n,
block_group-key.objectid);
goto out_put;
+ }
ret = btrfs_prealloc_file_range_trans(inode, trans, 0, 0, num_pages,
num_pages, num_pages,
diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c
index 22ee0dc..0078172 100644
--- a/fs/btrfs/free-space-cache.c
+++ b/fs/btrfs/free-space-cache.c
@@ -511,6 +511,8 @@ int btrfs_write_out_cache(struct btrfs_root *root,
spin_lock(block_group-lock);
if (block_group-disk_cache_state BTRFS_DC_SETUP) {
spin_unlock(block_group-lock);
+ printk(KERN_ERR block group %llu, wrong dcs %d\n,
block_group-key.objectid,
+block_group-disk_cache_state);
return 0;
}
spin_unlock(block_group-lock);
@@ -520,6 +522,7 @@ int btrfs_write_out_cache(struct btrfs_root *root,
return 0;
if (!i_size_read(inode)) {
+ printk(KERN_ERR no allocated space for block group %llu\n,
block_group-key.objectid);
iput(inode);
return 0;
}
@@ -771,6 +774,7 @@ out_free:
block_group-disk_cache_state = BTRFS_DC_ERROR;
spin_unlock(block_group-lock);
BTRFS_I(inode)-generation = 0;
+ printk(KERN_ERR problem writing out block group cache for
%llu\n, block_group-key.objectid);
}
kfree(checksums);
btrfs_update_inode(trans, root, inode);
This is from dmesg shortly after reboot with the debug patch:
btrfs: free space inode generation (0) did not match free space cache
generation (116974) for block group 14256439296
btrfs: free space inode generation (0) did not match free space cache
generation (116974) for block group 14256439296
btrfs: free space inode generation (0) did not match free space cache
generation (116974) for block group 14256439296
btrfs: free space inode generation (0) did not match free space cache
generation (116974) for block group 14256439296
btrfs: free space inode generation (0) did not match free space cache
generation (177986) for block group 5398069248
btrfs: free space inode generation (0) did not match free space cache
generation (177986) for block group 5398069248
block group 5398069248 not cached
block group 19893583872 not cached
no allocated space for block group 5398069248
no allocated space for block group 19893583872
btrfs: free space inode generation (0) did not match free space cache
generation (116974) for block
Re: disk space caching generation missmatch
On Wednesday 01 December 2010 22:22:45 Johannes Hirte wrote:
On Wednesday 01 December 2010 21:03:13 Josef Bacik wrote:
On Wed, Dec 01, 2010 at 08:56:14PM +0100, Johannes Hirte wrote:
On Wednesday 01 December 2010 18:40:18 Josef Bacik wrote:
On Wed, Dec 01, 2010 at 05:46:14PM +0100, Johannes Hirte wrote:
After enabling disk space caching I've observed several log entries
like this:
btrfs: free space inode generation (0) did not match free space cache
generation (169594) for block group 15464398848
I'm not sure, but it seems this happens on every reboot. Is this
something to
worry about?
So that usually means 1 of a couple of things
1) You didn't have space for us to save the free space cache
2) When trying to write out the cache we hit one of those cases where
we would
deadlock so we couldn't write the cache out
It's nothing to worry about, it's doing what it is supposed to.
However I'd
like to know why we're not able to write out the cache. Are you
running close
to full? Thanks,
Josef
I think there should be enough free space:
Hmm well then we're hitting one of the other corner cases. Can you run with
this debug thread and reboot. Hopefully it will tell me why we're not
saving
the free space cache. Thanks,
Josef
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 87aae66..4fd5659 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -2794,13 +2794,17 @@ again:
if (i_size_read(inode) 0) {
ret = btrfs_truncate_free_space_cache(root, trans, path,
inode);
- if (ret)
+ if (ret) {
+ printk(KERN_ERR truncate free space cache failed for
%llu, %d\n,
+ block_group-key.objectid, ret);
goto out_put;
+ }
}
spin_lock(block_group-lock);
if (block_group-cached != BTRFS_CACHE_FINISHED) {
spin_unlock(block_group-lock);
+ printk(KERN_ERR block group %llu not cached\n,
block_group-key.objectid);
goto out_put;
}
spin_unlock(block_group-lock);
@@ -2820,8 +2824,10 @@ again:
num_pages *= PAGE_CACHE_SIZE;
ret = btrfs_check_data_free_space(inode, num_pages);
- if (ret)
+ if (ret) {
+ printk(KERN_ERR not enough free space for cache %llu\n,
block_group-key.objectid);
goto out_put;
+ }
ret = btrfs_prealloc_file_range_trans(inode, trans, 0, 0, num_pages,
num_pages, num_pages,
diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c
index 22ee0dc..0078172 100644
--- a/fs/btrfs/free-space-cache.c
+++ b/fs/btrfs/free-space-cache.c
@@ -511,6 +511,8 @@ int btrfs_write_out_cache(struct btrfs_root *root,
spin_lock(block_group-lock);
if (block_group-disk_cache_state BTRFS_DC_SETUP) {
spin_unlock(block_group-lock);
+ printk(KERN_ERR block group %llu, wrong dcs %d\n,
block_group-key.objectid,
+ block_group-disk_cache_state);
return 0;
}
spin_unlock(block_group-lock);
@@ -520,6 +522,7 @@ int btrfs_write_out_cache(struct btrfs_root *root,
return 0;
if (!i_size_read(inode)) {
+ printk(KERN_ERR no allocated space for block group %llu\n,
block_group-key.objectid);
iput(inode);
return 0;
}
@@ -771,6 +774,7 @@ out_free:
block_group-disk_cache_state = BTRFS_DC_ERROR;
spin_unlock(block_group-lock);
BTRFS_I(inode)-generation = 0;
+ printk(KERN_ERR problem writing out block group cache for
%llu\n, block_group-key.objectid);
}
kfree(checksums);
btrfs_update_inode(trans, root, inode);
This is from dmesg shortly after reboot with the debug patch:
btrfs: free space inode generation (0) did not match free space cache
generation (116974) for block group 14256439296
btrfs: free space inode generation (0) did not match free space cache
generation (116974) for block group 14256439296
btrfs: free space inode generation (0) did not match free space cache
generation (116974) for block group 14256439296
btrfs: free space inode generation (0) did not match free space cache
generation (116974) for block group 14256439296
btrfs: free space inode generation (0) did not match free space cache
generation (177986) for block group 5398069248
btrfs: free space inode generation (0) did not match free space cache
generation (177986) for block group 5398069248
block group 5398069248 not cached
block group 19893583872 not cached
no allocated space for block group 5398069248
no allocated space for block group 19893583872
Re: machine gets unresponsive during btrfs balance
On Saturday 14 August 2010 00:11:55 Andreas Philipp wrote: On 12.08.2010 10:04, Yan, Zheng wrote: On Thu, Aug 12, 2010 at 3:14 PM, Andreas Philipp philipp.andr...@gmail.com wrote: Hi, I am using a btrfs filesystem created with raid0 for data and metadata for (temporary) storage of tv recordings from my vdr. The filesystem was created under kernel version 2.6.34. An initial btrfs balance command succeeded. Since I upgraded to 2.6.35-rcX and 2.6.35 btrfs balance no longer finishes but puts the machine in some unresponsive state. Unfortunately, I do not see any kernel oops or other debug information because even the display freezes. The last thing that happens are that those two lines are written to /var/log/messages: Aug 11 21:42:23 thor kernel: btrfs: found 62911 extents Aug 11 21:42:24 thor kernel: btrfs: relocating block group 1723913469952 flags 9 After that the machine becomes immediately unresponsive. As I did not see anything that might be related to my problem in the changelog for 2.6.35.1 I did not try again with this version. Do you have more than one machines? would you please setup netconsole to see what happen. I have reproduced the error on v2.6.35.1 and recorded all kernel output with netconsole. The interesting point is that this time the machine did not crash but the btrfs balance segfaulted at exact the same position where the previous crashes had happened. Looks like another manifestation of the csum bug. Are you able to read all files from the affected volume? Did you tried a balance with an 2.6.34 kernel after the test with 2.6.35? regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: machine gets unresponsive during btrfs balance
On Thursday 26 August 2010 15:39:25 Andreas Philipp wrote: On 26.08.2010 15:27, Johannes Hirte wrote: Looks like another manifestation of the csum bug. Are you able to read all files from the affected volume? Did you tried a balance with an 2.6.34 kernel after the test with 2.6.35? Till now I did not see any unreadable files but I did not do a complete test. No, I did not try to balance with an 2.6.34 kernel. If it helps I can switch back and try. I hope it helps to localize the error. It's still not clear where this starts an what kernels are affected. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 2.6.36-rc1 btrfs still unstable
On Monday 16 August 2010 16:17:54 Morten P.D. Stevens wrote: Hi Chris, the other big question is: Is btrfs with 2.6.36 really rockstable and ready to use in productive environments? Thanks Morten I don't think so. There is at least one checksum bug and ENOSPC problems are also still present. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel BUG at fs/btrfs/extent-tree.c:1353
Am Donnerstag 22 Juli 2010, 20:07:23 schrieb Johannes Hirte: Am Montag 19 Juli 2010, 10:01:46 schrieb Miao Xie: On Thu, 15 Jul 2010 20:14:51 +0200, Johannes Hirte wrote: Am Donnerstag 15 Juli 2010, 02:11:04 schrieb Dave Chinner: On Wed, Jul 14, 2010 at 05:25:23PM +0200, Johannes Hirte wrote: Am Donnerstag 08 Juli 2010, 16:31:09 schrieb Chris Mason: I'm not sure if btrfs is to blame for this error. After the errors I switched to XFS on this system and got now this error: ls -l .kde4/share/apps/akregator/data/ ls: cannot access .kde4/share/apps/akregator/data/feeds.opml: Structure needs cleaning total 4 ?? ? ???? feeds.opml What is the error reported in dmesg when the XFS filesytem shuts down? Nothing. I double checked the logs. There are only the messages when mounting the filesystem. No other errors are reported than the inaccessible file and the output from xfs_check. Is there anything wrong with your disks or memory? Sometimes the bad memory can break the filesystem. I have met this kind of problem some time ago. I don't think that's the case. I've checked the RAM with memtest86+ and got no errors. I got the errors with two different disks, the first one with btrfs the second one now with XFS. Before changing to the second disk, I've run badblocks on it to be sure it has no errors. I think I've found it. The bug was introduced by commit 7f0e7bed936a0c422641a046551829a01341dd80 Author: Christoph Hellwig h...@lst.de Date: Tue Jun 8 18:14:34 2010 +0200 writeback: fix writeback completion notifications The code dealing with bdi_work-state and completion of a bdi_work is a major mess currently. This patch makes sure we directly use one set of flags to deal with it, and use it consistently, which means: - always notify about completion from the rcu callback. We only ever wait for it from on-stack callers, so this simplification does not even cause a theoretical slowdown currently. It also makes sure we don't miss out on the notification if we ever add other callers to wait for it. - make earlier completion notification depending on the on-stack allocation, not the sync mode. If we introduce new callers that want to do WB_SYNC_NONE writeback from on-stack callers this will be nessecary. Also rename bdi_wait_on_work_clear to bdi_wait_on_work_done and inline a few small functions into their only caller to make the code understandable. Signed-off-by: Christoph Hellwig h...@lst.de Signed-off-by: Jens Axboe jax...@fusionio.com and seems to be fixed by commit 83ba7b071f30f7c01f72518ad72d5cd203c27502 Author: Christoph Hellwig h...@lst.de Date: Tue Jul 6 08:59:53 2010 +0200 writeback: simplify the write back thread queue First remove items from work_list as soon as we start working on them.This means we don't have to track any pending or visited state and can get rid of all the RCU magic freeing the work items - we can simply free them once the operation has finished. Second use a real completion for tracking synchronous requests - if the caller sets the completion pointer we complete it, otherwise use it as a boolean indicator that we can free the work item directly. Third unify struct wb_writeback_args and struct bdi_work into a single data structure, wb_writeback_work. Previous we set all parameters into a struct wb_writeback_args, copied it into struct bdi_work, copied it again on the stack to use it there. Instead of just allocate one structure dynamically or on the stack and use it all the way through the stack. Signed-off-by: Christoph Hellwig h...@lst.de Signed-off-by: Jens Axboe jax...@fusionio.com I was able to reproduce the bug by unpacking a big tar-file and deleting this files multiple times. Normally with btrfs the kernel crashed within 20 runs. After commit 83ba7b071f30f7c01f72518ad72d5cd203c27502 it survived more than 500 runs. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel BUG at fs/btrfs/extent-tree.c:1353
Am Montag 19 Juli 2010, 10:01:46 schrieb Miao Xie: On Thu, 15 Jul 2010 20:14:51 +0200, Johannes Hirte wrote: Am Donnerstag 15 Juli 2010, 02:11:04 schrieb Dave Chinner: On Wed, Jul 14, 2010 at 05:25:23PM +0200, Johannes Hirte wrote: Am Donnerstag 08 Juli 2010, 16:31:09 schrieb Chris Mason: I'm not sure if btrfs is to blame for this error. After the errors I switched to XFS on this system and got now this error: ls -l .kde4/share/apps/akregator/data/ ls: cannot access .kde4/share/apps/akregator/data/feeds.opml: Structure needs cleaning total 4 ?? ? ???? feeds.opml What is the error reported in dmesg when the XFS filesytem shuts down? Nothing. I double checked the logs. There are only the messages when mounting the filesystem. No other errors are reported than the inaccessible file and the output from xfs_check. Is there anything wrong with your disks or memory? Sometimes the bad memory can break the filesystem. I have met this kind of problem some time ago. I don't think that's the case. I've checked the RAM with memtest86+ and got no errors. I got the errors with two different disks, the first one with btrfs the second one now with XFS. Before changing to the second disk, I've run badblocks on it to be sure it has no errors. If there is no problem with your disk and memory, Could you tell us the parameter of mkfs.btrfs and mount? I'm not sure what parameters I've used for mkbtrfs. It was either none ore '-m single'. mount parameters are only noatime. Some time ago I've played a little with max_inline. On the actual disk with XFS I got now some more errors on my root-fs. Similar error on one file: ls: cannot access /var/tmp/portage/app- office/krita-2.2.1/work/krita-2.2.1/krita/image/tiles3/tests/dm_consistancy_test/dm_consistancy_test.pr: Invalid argument xfs_check shows on this fs: localhost ~ # xfs_check /dev/sda1 agi unlinked bucket 10 is 7279754 in ag 0 (inode=7279754) agi unlinked bucket 11 is 7279755 in ag 0 (inode=7279755) dir 91466358 entry dm_consistancy_test.pr bad inode number 1862628266 dir 91466358 size is 36, should be 35 agi unlinked bucket 48 is 11677104 in ag 2 (inode=78785968) agi unlinked bucket 49 is 11677105 in ag 2 (inode=78785969) agi unlinked bucket 50 is 11677106 in ag 2 (inode=78785970) agi unlinked bucket 51 is 11677107 in ag 2 (inode=78785971) agi unlinked bucket 52 is 11677108 in ag 2 (inode=78785972) agi unlinked bucket 53 is 11677109 in ag 2 (inode=78785973) agi unlinked bucket 54 is 11677110 in ag 2 (inode=78785974) agi unlinked bucket 55 is 11677111 in ag 2 (inode=78785975) agi unlinked bucket 58 is 11677114 in ag 2 (inode=78785978) agi unlinked bucket 59 is 11677115 in ag 2 (inode=78785979) agi unlinked bucket 60 is 11677116 in ag 2 (inode=78785980) agi unlinked bucket 61 is 11677117 in ag 2 (inode=78785981) allocated inode 7279754 has 0 link count allocated inode 7279755 has 0 link count disconnected inode 91466360, nlink 1 allocated inode 78785968 has 0 link count allocated inode 78785969 has 0 link count allocated inode 78785970 has 0 link count allocated inode 78785971 has 0 link count allocated inode 78785972 has 0 link count allocated inode 78785973 has 0 link count allocated inode 78785974 has 0 link count allocated inode 78785975 has 0 link count allocated inode 78785978 has 0 link count allocated inode 78785979 has 0 link count allocated inode 78785980 has 0 link count allocated inode 78785981 has 0 link count And again I don't find any related message in dmesg. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel BUG at fs/btrfs/extent-tree.c:1353
Am Donnerstag 15 Juli 2010, 20:14:51 schrieb Johannes Hirte:
Am Donnerstag 15 Juli 2010, 02:11:04 schrieb Dave Chinner:
On Wed, Jul 14, 2010 at 05:25:23PM +0200, Johannes Hirte wrote:
Am Donnerstag 08 Juli 2010, 16:31:09 schrieb Chris Mason:
I'm not sure if btrfs is to blame for this error. After the errors I
switched to XFS on this system and got now this error:
ls -l .kde4/share/apps/akregator/data/
ls: cannot access .kde4/share/apps/akregator/data/feeds.opml: Structure
needs cleaning
total 4
?? ? ???? feeds.opml
What is the error reported in dmesg when the XFS filesytem shuts down?
Nothing. I double checked the logs. There are only the messages when
mounting the filesystem. No other errors are reported than the
inaccessible file and the output from xfs_check.
I'm running now a kernel with more debug options enabled and got this:
[ 6794.810935]
[ 6794.810941] =
[ 6794.810955] [ INFO: inconsistent lock state ]
[ 6794.810966] 2.6.35-rc4-btrfs-debug #7
[ 6794.810975] -
[ 6794.810984] inconsistent {RECLAIM_FS-ON-W} - {IN-RECLAIM_FS-W} usage.
[ 6794.810996] kswapd0/361 [HC0[0]:SC0[0]:HE1:SE1] takes:
[ 6794.811006] ((ip-i_iolock)-mr_lock#2){?+}, at: [c10fa82d]
xfs_ilock+0x22/0x67
[ 6794.811039] {RECLAIM_FS-ON-W} state was registered at:
[ 6794.811046] [c104ebc1] mark_held_locks+0x42/0x5e
[ 6794.811046] [c104f1f7] lockdep_trace_alloc+0x99/0xb0
[ 6794.811046] [c10740b8] __alloc_pages_nodemask+0x6a/0x4a1
[ 6794.811046] [c106edc2] __page_cache_alloc+0x11/0x13
[ 6794.811046] [c106fb43] grab_cache_page_write_begin+0x47/0x81
[ 6794.811046] [c10b2050] block_write_begin_newtrunc+0x2e/0x9c
[ 6794.811046] [c10b233a] block_write_begin+0x23/0x5d
[ 6794.811046] [c1114a9d] xfs_vm_write_begin+0x26/0x28
[ 6794.811046] [c106f15d] generic_file_buffered_write+0xb5/0x1bd
[ 6794.811046] [c1117e31] xfs_file_aio_write+0x40e/0x66d
[ 6794.811046] [c10950b4] do_sync_write+0x8b/0xc6
[ 6794.811046] [c109568b] vfs_write+0x77/0xa4
[ 6794.811046] [c10957f3] sys_write+0x3c/0x5e
[ 6794.811046] [c1002690] sysenter_do_call+0x12/0x36
[ 6794.811046] irq event stamp: 141369
[ 6794.811046] hardirqs last enabled at (141369): [c13639d2]
_raw_spin_unlock_irqrestore+0x36/0x5b
[ 6794.811046] hardirqs last disabled at (141368): [c13634c5]
_raw_spin_lock_irqsave+0x14/0x68
[ 6794.811046] softirqs last enabled at (141300): [c1032d69]
__do_softirq+0xfe/0x10d
[ 6794.811046] softirqs last disabled at (141295): [c1032da7]
do_softirq+0x2f/0x47
[ 6794.811046]
[ 6794.811046] other info that might help us debug this:
[ 6794.811046] 2 locks held by kswapd0/361:
[ 6794.811046] #0: (shrinker_rwsem){..}, at: [c10774db]
shrink_slab+0x25/0x13f
[ 6794.811046] #1: (xfs_mount_list_lock){.-}, at: [c111cc78]
xfs_reclaim_inode_shrink+0x2a/0xe8
[ 6794.811046]
[ 6794.811046] stack backtrace:
[ 6794.811046] Pid: 361, comm: kswapd0 Not tainted 2.6.35-rc4-btrfs-debug #7
[ 6794.811046] Call Trace:
[ 6794.811046] [c13616c0] ? printk+0xf/0x17
[ 6794.811046] [c104e988] valid_state+0x134/0x142
[ 6794.811046] [c104ea66] mark_lock+0xd0/0x1e9
[ 6794.811046] [c104e2a7] ? check_usage_forwards+0x0/0x5f
[ 6794.811046] [c105003d] __lock_acquire+0x374/0xc80
[ 6794.811046] [c1044942] ? sched_clock_local+0x12/0x121
[ 6794.811046] [c1044c0b] ? sched_clock_cpu+0x122/0x133
[ 6794.811046] [c1050d4d] lock_acquire+0x5f/0x76
[ 6794.811046] [c10fa82d] ? xfs_ilock+0x22/0x67
[ 6794.811046] [c1043974] down_write_nested+0x32/0x63
[ 6794.811046] [c10fa82d] ? xfs_ilock+0x22/0x67
[ 6794.811046] [c10fa82d] xfs_ilock+0x22/0x67
[ 6794.811046] [c10faa48] xfs_ireclaim+0x98/0xbb
[ 6794.811046] [c1043a1e] ? up_write+0x16/0x2b
[ 6794.811046] [c111c78c] xfs_reclaim_inode+0x1a7/0x1b1
[ 6794.811046] [c111cafe] xfs_inode_ag_walk+0x77/0xbc
[ 6794.811046] [c111c5e5] ? xfs_reclaim_inode+0x0/0x1b1
[ 6794.811046] [c111cc07] xfs_inode_ag_iterator+0x52/0x99
[ 6794.811046] [c111cc78] ? xfs_reclaim_inode_shrink+0x2a/0xe8
[ 6794.811046] [c111c5e5] ? xfs_reclaim_inode+0x0/0x1b1
[ 6794.811046] [c111cc99] xfs_reclaim_inode_shrink+0x4b/0xe8
[ 6794.811046] [c1077588] shrink_slab+0xd2/0x13f
[ 6794.811046] [c1078cef] kswapd+0x37d/0x4e9
[ 6794.811046] [c104028f] ? autoremove_wake_function+0x0/0x2f
[ 6794.811046] [c1078972] ? kswapd+0x0/0x4e9
[ 6794.811046] [c103ffbc] kthread+0x60/0x65
[ 6794.811046] [c103ff5c] ? kthread+0x0/0x65
[ 6794.811046] [c1002bba] kernel_thread_helper+0x6/0x10
Don't know if this is related to the problem.
regards,
Johannes
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Status of BTRFS
Am Freitag 16 Juli 2010, 13:55:26 schrieb Edward Ned Harvey: Is this a good place to get a clue about the status of BTRFS? Like ... Is it usable yet, and stuff like that? Thank you... I wouldn't suggest to use it in productive environments. Especially as the error handling is very rudimentarily for now. If you run into errors, you won't be able to repair the filesystem. For productive environments there are also way to much situations where the whole kernel panics instead only the affected filesystem. Another construction area is still the ENOSPC handling. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel BUG at fs/btrfs/extent-tree.c:1353
Am Donnerstag 15 Juli 2010, 02:11:04 schrieb Dave Chinner: On Wed, Jul 14, 2010 at 05:25:23PM +0200, Johannes Hirte wrote: Am Donnerstag 08 Juli 2010, 16:31:09 schrieb Chris Mason: I'm not sure if btrfs is to blame for this error. After the errors I switched to XFS on this system and got now this error: ls -l .kde4/share/apps/akregator/data/ ls: cannot access .kde4/share/apps/akregator/data/feeds.opml: Structure needs cleaning total 4 ?? ? ???? feeds.opml What is the error reported in dmesg when the XFS filesytem shuts down? Nothing. I double checked the logs. There are only the messages when mounting the filesystem. No other errors are reported than the inaccessible file and the output from xfs_check. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
csum errors
Am Dienstag 13 Juli 2010, 14:23:58 schrieb Johannes Hirte: On the Opteron system I got now csum errors. I've synced some data from the netbook to the Opteron yesteray. After hitting ENOSPC with 4GB free, I've run 'btrfs-vol -b' on this fs in hope to get some more free space. It worked but the command failed and I found in dmesg: btrfs csum failed ino 339 off 935280640 csum 337776576 private 337776575 btrfs csum failed ino 339 off 935280640 csum 337776576 private 337776575 btrfs csum failed ino 339 off 935280640 csum 337776576 private 337776575 btrfs csum failed ino 339 off 935280640 csum 337776576 private 337776575 So I've tested the new synced data by syncing them to another disk on the Optoern system (XFS). As I've expected (or better feared), some data wasn't readable and I found more csum errors in dmesg: btrfs csum failed ino 1849137 off 368640 csum 3354885689 private 3354885688 btrfs csum failed ino 1849137 off 368640 csum 3354885689 private 3354885688 btrfs csum failed ino 1849137 off 368640 csum 3354885689 private 3354885688 btrfs csum failed ino 1849137 off 368640 csum 3354885689 private 3354885688 btrfs csum failed ino 1849137 off 368640 csum 3354885689 private 3354885688 btrfs csum failed ino 1849137 off 368640 csum 3354885689 private 3354885688 btrfs csum failed ino 1849137 off 368640 csum 3354885689 private 3354885688 btrfs csum failed ino 1849137 off 368640 csum 3354885689 private 3354885688 btrfs csum failed ino 1849137 off 368640 csum 3354885689 private 3354885688 btrfs csum failed ino 1849137 off 368640 csum 3354885689 private 3354885688 btrfs csum failed ino 1912210 off 5095424 csum 847944548 private 847944547 btrfs csum failed ino 1912210 off 5095424 csum 847944548 private 847944547 btrfs csum failed ino 1912210 off 5095424 csum 847944548 private 847944547 btrfs csum failed ino 1912210 off 5095424 csum 847944548 private 847944547 btrfs csum failed ino 1912210 off 5095424 csum 847944548 private 847944547 btrfs csum failed ino 1912210 off 5095424 csum 847944548 private 847944547 btrfs csum failed ino 1912210 off 5095424 csum 847944548 private 847944547 btrfs csum failed ino 1912210 off 5095424 csum 847944548 private 847944547 btrfs csum failed ino 1912210 off 5095424 csum 847944548 private 847944547 btrfs csum failed ino 1912210 off 5095424 csum 847944548 private 847944547 btrfs csum failed ino 1912210 off 5095424 csum 847944548 private 847944547 btrfs csum failed ino 1912210 off 5095424 csum 847944548 private 847944547 btrfs csum failed ino 1959333 off 252362752 csum 686735346 private 686735345 btrfs csum failed ino 1959333 off 252362752 csum 686735346 private 686735345 btrfs csum failed ino 1959333 off 252362752 csum 686735346 private 686735345 btrfs csum failed ino 1959333 off 252362752 csum 686735346 private 686735345 btrfs csum failed ino 1959333 off 252362752 csum 686735346 private 686735345 btrfs csum failed ino 1959333 off 252362752 csum 686735346 private 686735345 btrfs csum failed ino 1959333 off 651108352 csum 2851505977 private 2851505976 btrfs csum failed ino 1959333 off 651108352 csum 2851505977 private 2851505976 btrfs csum failed ino 1959333 off 651108352 csum 2851505977 private 2851505976 btrfs csum failed ino 1959333 off 651108352 csum 2851505977 private 2851505976 btrfs csum failed ino 1959333 off 651108352 csum 2851505977 private 2851505976 btrfs csum failed ino 1959333 off 651108352 csum 2851505977 private 2851505976 btrfs csum failed ino 1959333 off 898342912 csum 4271223884 private 4271223883 btrfs csum failed ino 1959333 off 898342912 csum 4271223884 private 4271223883 btrfs csum failed ino 1959333 off 898342912 csum 4271223884 private 4271223883 btrfs csum failed ino 1959333 off 898342912 csum 4271223884 private 4271223883 btrfs csum failed ino 1959333 off 898342912 csum 4271223884 private 4271223883 btrfs csum failed ino 1959333 off 898342912 csum 4271223884 private 4271223883 I think, this is a different error. I've only seen them on filesystems from my Opteron system. It seems that the recorded csums are wrong and it looks to me like rounding errors. The data itself should be correct, as I've tested one affected file via md5sum against the original on another filesystem. Any ideas what is going wrong here? regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: csum errors
Am Donnerstag 15 Juli 2010, 21:03:09 schrieb Chris Mason:
On Thu, Jul 15, 2010 at 08:30:17PM +0200, Johannes Hirte wrote:
Am Dienstag 13 Juli 2010, 14:23:58 schrieb Johannes Hirte:
ino 1959333 off 898342912 csum 4271223884 private 4271223883
I think, this is a different error. I've only seen them on filesystems
from my Opteron system. It seems that the recorded csums are wrong and
it looks to me like rounding errors. The data itself should be correct,
as I've tested one affected file via md5sum against the original on
another filesystem. Any ideas what is going wrong here?
Are you doing data mirroring?
No, I don't.
We can map that block and do a raw read off the device to see what the
data blocks actually contain.
I've modified the btrfs-source a little to get the data. In inode.c I've
changed the code to:
csum = btrfs_csum_data(root, kaddr + offset, csum, end - start + 1);
btrfs_csum_final(csum, (char *)csum);
if (csum != private)
if (printk_ratelimit()) {
printk(KERN_INFO csum != private; ino %lu off %llu
csum %u private %llu\n, page-mapping-host-i_ino,
(unsigned long long)start, csum,
(unsigned long long)private);
}
// goto zeroit;
kunmap_atomic(kaddr, KM_USER0);
This way I could read the files with wrong csum too. As I wrote, I've compared
the md5sum from one file with a copy on an other filesystem. As they are the
same, at least for this file the data should be correct. The big question is,
why do the csums differ?
regards,
Johannes
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: csum errors
Am Donnerstag 15 Juli 2010, 21:35:51 schrieb Chris Mason: On Thu, Jul 15, 2010 at 09:32:12PM +0200, Johannes Hirte wrote: Am Donnerstag 15 Juli 2010, 21:03:09 schrieb Chris Mason: On Thu, Jul 15, 2010 at 08:30:17PM +0200, Johannes Hirte wrote: Am Dienstag 13 Juli 2010, 14:23:58 schrieb Johannes Hirte: ino 1959333 off 898342912 csum 4271223884 private 4271223883 I think, this is a different error. I've only seen them on filesystems from my Opteron system. It seems that the recorded csums are wrong and it looks to me like rounding errors. The data itself should be correct, as I've tested one affected file via md5sum against the original on another filesystem. Any ideas what is going wrong here? Are you doing data mirroring? No, I don't. We can map that block and do a raw read off the device to see what the data blocks actually contain. I've modified the btrfs-source a little to get the data. In inode.c I've changed the code to: Great. The bad csums are all just one bit off, that can't be an accident. When were they written (which kernel?). Did you boot a 32 bit kernel on there at any time? No, I don't have a bootable 32bit installation on this system. I've tested it now with a 32bit system by dumping the whole filesystem to an external drive and mounting this to a 32bit system. The result was the same. The affected files were written by different kernels. I think at least 2.6.34, 2.6.35-rc3 and 2.6.35-rc4 should be involved, perhaps 2.6.33 too. I'll try to figure it out more exactly. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel BUG at fs/btrfs/extent-tree.c:1353
Am Donnerstag 08 Juli 2010, 16:31:09 schrieb Chris Mason: Neither Yan nor I have been able to reproduce this locally, but a few people have now hit it. Johannes, are you available to try out a debugging kernel to try and track this down? -chris On Thu, Jul 08, 2010 at 04:27:23PM +0200, Johannes Hirte wrote: When doing a 'rm -r /var/tmp/portage/sys-devel' I get the following Oops: [ cut here ] kernel BUG at fs/btrfs/extent-tree.c:1353! invalid opcode: [#1] PREEMPT SMP last sysfs file: /sys/devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0A:00/power_supply/BAT0/charge_ full Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss nfs lockd nfs_acl auth_rpcgss sunrpc sco rfcomm bnep l2cap crc16 xts gf128mul usb_storage dm_crypt dm_mod coretemp hwmon acpi_cpufreq mperf snd_hda_codec_realtek uvcvideo iwl3945 snd_hda_intel snd_hda_codec iwlcore videodev r8169 snd_hwdep btusb snd_pcm v4l1_compat mac80211 snd_timer bluetooth snd mii cfg80211 soundcore sg rfkill ac i2c_i801 snd_page_alloc uhci_hcd battery [last unloaded: microcode] Pid: 2358, comm: rm Not tainted 2.6.35-rc4 #32 M912/M912 EIP: 0060:[c10c383b] EFLAGS: 00010202 CPU: 1 EIP is at lookup_inline_extent_backref+0xf2/0x406 EAX: 0001 EBX: 0007 ECX: EDX: ESI: 0004 EDI: f7268150 EBP: 0004 ESP: f5aa5d08 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 Process rm (pid: 2358, ti=f5aa4000 task=f6f0fa70 task.ti=f5aa4000) Stack: f702f8c0 f744e080 f665f380 00b0 f6c80f00 0 f744e080 c10ec226 e98acfff f6c98000 1001 0e987000 0004 0 0850 040e9870 a800 1000 0007 0e987000 Call Trace: [c10ec226] ? set_extent_dirty+0x19/0x1d [c10c5081] ? __btrfs_free_extent+0xda/0x675 [c10c88bf] ? run_clustered_refs+0x699/0x6d7 [c10d239f] ? btrfs_mark_buffer_dirty+0xa3/0xef [c1101454] ? btrfs_find_ref_cluster+0xf9/0x13a [c10c89bc] ? btrfs_run_delayed_refs+0xbf/0x155 [c10d3a73] ? __btrfs_end_transaction+0x53/0x16c [c10db480] ? btrfs_delete_inode+0x166/0x17e [c102280d] ? get_parent_ip+0x8/0x19 [c108fe5c] ? generic_delete_inode+0x6f/0xbd [c108f5b3] ? iput+0x46/0x48 [c10893a8] ? do_unlinkat+0xc7/0x109 [c102280d] ? get_parent_ip+0x8/0x19 [c10822e3] ? fput+0x12/0x15c [c10a2f30] ? dnotify_flush+0x41/0xc2 [c107fe85] ? filp_close+0x4c/0x52 [c107feed] ? sys_close+0x62/0x9b [c1002550] ? sysenter_do_call+0x12/0x26 Code: 80 4e 68 02 8d 4c 24 43 89 f8 6a 01 ff 74 24 1c ff 74 24 08 8b 54 24 38 e8 01 c2 ff ff 83 c4 0c 83 f8 00 0f 8c e1 02 00 00 74 02 0f 0b 8b 04 24 8b 34 24 8b 00 8b 56 20 89 44 24 08 e8 2e fa ff EIP: [c10c383b] lookup_inline_extent_backref+0xf2/0x406 SS:ESP 0068:f5aa5d08 ---[ end trace d97601f0b455ca72 ]--- note: rm[2358] exited with preempt_count 2 BUG: scheduling while atomic: rm/2358/0x1003 Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss nfs lockd nfs_acl auth_rpcgss sunrpc sco rfcomm bnep l2cap crc16 xts gf128mul usb_storage dm_crypt dm_mod coretemp hwmon acpi_cpufreq mperf snd_hda_codec_realtek uvcvideo iwl3945 snd_hda_intel snd_hda_codec iwlcore videodev r8169 snd_hwdep btusb snd_pcm v4l1_compat mac80211 snd_timer bluetooth snd mii cfg80211 soundcore sg rfkill ac i2c_i801 snd_page_alloc uhci_hcd battery [last unloaded: microcode] Pid: 2358, comm: rm Tainted: G D 2.6.35-rc4 #32 Call Trace: [c12de6b3] ? schedule+0x88/0x332 [c10237c1] ? __cond_resched+0xf/0x19 [c12de9e2] ? _cond_resched+0x12/0x18 [c106ceec] ? unmap_vmas+0x4e7/0x534 [c1070c8f] ? exit_mmap+0x64/0xa4 [c1026089] ? mmput+0x21/0x96 [c102938e] ? exit_mm+0xe7/0xf0 [c12dfa28] ? _raw_spin_unlock_irqrestore+0x1a/0x24 [c103aaa1] ? hrtimer_try_to_cancel+0x31/0x3a [c102a42e] ? do_exit+0x17b/0x57d [c1028e78] ? kmsg_dump+0x81/0xf9 [c1002d06] ? do_invalid_op+0x0/0x76 [c1004fa0] ? oops_end+0x72/0x75 [c1002d6f] ? do_invalid_op+0x69/0x76 [c10c383b] ? lookup_inline_extent_backref+0xf2/0x406 [c10bdc9a] ? generic_bin_search.clone.0+0x145/0x150 [c10bcf30] ? btrfs_cow_block+0x106/0x112 [c10bdcdc] ? bin_search+0x37/0x3d [c10bfe33] ? btrfs_search_slot+0x405/0x477 [c12e031a] ? error_code+0x66/0x6c [c1002d06] ? do_invalid_op+0x0/0x76 [c10c383b] ? lookup_inline_extent_backref+0xf2/0x406 [c10ec226] ? set_extent_dirty+0x19/0x1d [c10c5081] ? __btrfs_free_extent+0xda/0x675 [c10c88bf] ? run_clustered_refs+0x699/0x6d7 [c10d239f] ? btrfs_mark_buffer_dirty+0xa3/0xef [c1101454] ? btrfs_find_ref_cluster+0xf9/0x13a [c10c89bc] ? btrfs_run_delayed_refs+0xbf/0x155 [c10d3a73] ? __btrfs_end_transaction+0x53/0x16c [c10db480] ? btrfs_delete_inode+0x166/0x17e [c102280d] ? get_parent_ip+0x8/0x19 [c108fe5c] ? generic_delete_inode+0x6f/0xbd
Re: kernel BUG at fs/btrfs/extent-tree.c:1353
Am Sonntag 11 Juli 2010, 14:28:09 schrieb Johannes Hirte: ... I've three systems running with btrfs, a dual Opteron (252), a Pentium 4 system and a netbook with N270 Atom. The netbook is the only one that shows the errors. It's also the only system where I'm using gcc-4.5. Perhaps it's related, but I doubt it's the only reason as I'm using gcc-4.5 since May. On the Opteron system I got now csum errors. I've synced some data from the netbook to the Opteron yesteray. After hitting ENOSPC with 4GB free, I've run 'btrfs-vol -b' on this fs in hope to get some more free space. It worked but the command failed and I found in dmesg: btrfs csum failed ino 339 off 935280640 csum 337776576 private 337776575 btrfs csum failed ino 339 off 935280640 csum 337776576 private 337776575 btrfs csum failed ino 339 off 935280640 csum 337776576 private 337776575 btrfs csum failed ino 339 off 935280640 csum 337776576 private 337776575 So I've tested the new synced data by syncing them to another disk on the Optoern system (XFS). As I've expected (or better feared), some data wasn't readable and I found more csum errors in dmesg: btrfs csum failed ino 1849137 off 368640 csum 3354885689 private 3354885688 btrfs csum failed ino 1849137 off 368640 csum 3354885689 private 3354885688 btrfs csum failed ino 1849137 off 368640 csum 3354885689 private 3354885688 btrfs csum failed ino 1849137 off 368640 csum 3354885689 private 3354885688 btrfs csum failed ino 1849137 off 368640 csum 3354885689 private 3354885688 btrfs csum failed ino 1849137 off 368640 csum 3354885689 private 3354885688 btrfs csum failed ino 1849137 off 368640 csum 3354885689 private 3354885688 btrfs csum failed ino 1849137 off 368640 csum 3354885689 private 3354885688 btrfs csum failed ino 1849137 off 368640 csum 3354885689 private 3354885688 btrfs csum failed ino 1849137 off 368640 csum 3354885689 private 3354885688 btrfs csum failed ino 1912210 off 5095424 csum 847944548 private 847944547 btrfs csum failed ino 1912210 off 5095424 csum 847944548 private 847944547 btrfs csum failed ino 1912210 off 5095424 csum 847944548 private 847944547 btrfs csum failed ino 1912210 off 5095424 csum 847944548 private 847944547 btrfs csum failed ino 1912210 off 5095424 csum 847944548 private 847944547 btrfs csum failed ino 1912210 off 5095424 csum 847944548 private 847944547 btrfs csum failed ino 1912210 off 5095424 csum 847944548 private 847944547 btrfs csum failed ino 1912210 off 5095424 csum 847944548 private 847944547 btrfs csum failed ino 1912210 off 5095424 csum 847944548 private 847944547 btrfs csum failed ino 1912210 off 5095424 csum 847944548 private 847944547 btrfs csum failed ino 1912210 off 5095424 csum 847944548 private 847944547 btrfs csum failed ino 1912210 off 5095424 csum 847944548 private 847944547 btrfs csum failed ino 1959333 off 252362752 csum 686735346 private 686735345 btrfs csum failed ino 1959333 off 252362752 csum 686735346 private 686735345 btrfs csum failed ino 1959333 off 252362752 csum 686735346 private 686735345 btrfs csum failed ino 1959333 off 252362752 csum 686735346 private 686735345 btrfs csum failed ino 1959333 off 252362752 csum 686735346 private 686735345 btrfs csum failed ino 1959333 off 252362752 csum 686735346 private 686735345 btrfs csum failed ino 1959333 off 651108352 csum 2851505977 private 2851505976 btrfs csum failed ino 1959333 off 651108352 csum 2851505977 private 2851505976 btrfs csum failed ino 1959333 off 651108352 csum 2851505977 private 2851505976 btrfs csum failed ino 1959333 off 651108352 csum 2851505977 private 2851505976 btrfs csum failed ino 1959333 off 651108352 csum 2851505977 private 2851505976 btrfs csum failed ino 1959333 off 651108352 csum 2851505977 private 2851505976 btrfs csum failed ino 1959333 off 898342912 csum 4271223884 private 4271223883 btrfs csum failed ino 1959333 off 898342912 csum 4271223884 private 4271223883 btrfs csum failed ino 1959333 off 898342912 csum 4271223884 private 4271223883 btrfs csum failed ino 1959333 off 898342912 csum 4271223884 private 4271223883 btrfs csum failed ino 1959333 off 898342912 csum 4271223884 private 4271223883 btrfs csum failed ino 1959333 off 898342912 csum 4271223884 private 4271223883 I suspect something goes horribly wrong on writing to disc within btrfs. On the netbook I got missing blocks, on the Opteron System bad csums. Both systems are running linux-2.6.35-rc4, the netbook with gcc-4.5.0 the Opteron system with gcc-4.4.4. I'll test the P4 system later, if there are similar errors too. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel BUG at fs/btrfs/extent-tree.c:1353
It's getting worse. The /home partition is now affected too. I get the Oops on simple unmounting the fs. btrfsck gives me this output on this fs: btrfsck /dev/mapper/sdb3 leaf 123780497408 items 49 free space 271 generation 62207 owner 2 fs uuid 7f013285-88d8-452f-a139-7d44bffd14b6 chunk uuid 365526c9-e209-46a1-8963-3157306d9e05 item 0 key (123780108288 EXTENT_ITEM 4096) itemoff 3944 itemsize 51 extent refs 1 gen 45309 flags 2 tree block key (836797 6c 0) level 1 tree block backref root 5 item 1 key (123780112384 EXTENT_ITEM 4096) itemoff 3893 itemsize 51 extent refs 1 gen 45309 flags 2 tree block key (836955 c 416051) level 0 tree block backref root 5 item 2 key (123780116480 EXTENT_ITEM 4096) itemoff 3842 itemsize 51 extent refs 1 gen 24599 flags 2 tree block key (18446744073709551606 80 122869407744) level 0 tree block backref root 7 item 3 key (123780120576 EXTENT_ITEM 4096) itemoff 3791 itemsize 51 extent refs 1 gen 49958 flags 2 tree block key (979249 1 0) level 0 tree block backref root 5 item 4 key (123780124672 EXTENT_ITEM 4096) itemoff 3740 itemsize 51 extent refs 1 gen 62191 flags 2 tree block key (1220 1 0) level 0 tree block backref root 5 item 5 key (123780128768 EXTENT_ITEM 4096) itemoff 3689 itemsize 51 extent refs 1 gen 54817 flags 2 tree block key (1001168 c 455590) level 0 tree block backref root 5 item 6 key (123780132864 EXTENT_ITEM 4096) itemoff 3638 itemsize 51 extent refs 1 gen 62201 flags 2 tree block key (28712 1 0) level 1 tree block backref root 5 item 7 key (123780136960 EXTENT_ITEM 4096) itemoff 3587 itemsize 51 extent refs 1 gen 62191 flags 2 tree block key (34645 c 33037) level 0 tree block backref root 5 item 8 key (123780141056 EXTENT_ITEM 4096) itemoff 3536 itemsize 51 extent refs 1 gen 50007 flags 2 tree block key (31007 60 4044) level 0 tree block backref root 5 item 9 key (123780145152 EXTENT_ITEM 4096) itemoff 3485 itemsize 51 extent refs 1 gen 62202 flags 2 tree block key (123644329984 a8 4096) level 0 tree block backref root 2 item 10 key (123780149248 EXTENT_ITEM 4096) itemoff 3434 itemsize 51 extent refs 1 gen 62202 flags 2 tree block key (123644854272 a8 4096) level 0 tree block backref root 2 item 11 key (123780153344 EXTENT_ITEM 4096) itemoff 3383 itemsize 51 extent refs 1 gen 62202 flags 2 tree block key (123645849600 a8 4096) level 0 tree block backref root 2 item 12 key (123780157440 EXTENT_ITEM 4096) itemoff 3332 itemsize 51 extent refs 1 gen 62207 flags 2 tree block key (123411308544 a8 4096) level 2 tree block backref root 2 item 13 key (123780161536 EXTENT_ITEM 4096) itemoff 3281 itemsize 51 extent refs 1 gen 62200 flags 2 tree block key (1325101 c 1264) level 0 tree block backref root 5 item 14 key (123780165632 EXTENT_ITEM 4096) itemoff 3230 itemsize 51 extent refs 1 gen 56401 flags 2 tree block key (59621 1 0) level 1 tree block backref root 5 item 15 key (123780169728 EXTENT_ITEM 4096) itemoff 3179 itemsize 51 extent refs 1 gen 24613 flags 2 tree block key (18446744073709551606 80 125996056576) level 0 tree block backref root 7 item 16 key (123780173824 EXTENT_ITEM 4096) itemoff 3128 itemsize 51 extent refs 1 gen 62189 flags 2 tree block key (1324334 1 0) level 0 tree block backref root 5 item 17 key (123780177920 EXTENT_ITEM 4095) itemoff 3077 itemsize 51 extent refs 1 gen 62207 flags 2 tree block key (123682791424 a8 4096) level 1 tree block backref root 2 item 18 key (123780182016 EXTENT_ITEM 4096) itemoff 3026 itemsize 51 extent refs 1 gen 62202 flags 2 tree block key (123648741376 a8 4096) level 0 tree block backref root 2 item 19 key (123780186112 EXTENT_ITEM 4096) itemoff 2975 itemsize 51 extent refs 1 gen 62201 flags 2 tree block key (28854 54 1781866506) level 0 tree block backref root 5 item 20 key (123780190208 EXTENT_ITEM 4096) itemoff 2924 itemsize 51 extent refs 1 gen 62192 flags 2 tree block
kernel BUG at fs/btrfs/extent-tree.c:1353
When doing a 'rm -r /var/tmp/portage/sys-devel' I get the following Oops: [ cut here ] kernel BUG at fs/btrfs/extent-tree.c:1353! invalid opcode: [#1] PREEMPT SMP last sysfs file: /sys/devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0A:00/power_supply/BAT0/charge_full Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss nfs lockd nfs_acl auth_rpcgss sunrpc sco rfcomm bnep l2cap crc16 xts gf128mul usb_storage dm_crypt dm_mod coretemp hwmon acpi_cpufreq mperf snd_hda_codec_realtek uvcvideo iwl3945 snd_hda_intel snd_hda_codec iwlcore videodev r8169 snd_hwdep btusb snd_pcm v4l1_compat mac80211 snd_timer bluetooth snd mii cfg80211 soundcore sg rfkill ac i2c_i801 snd_page_alloc uhci_hcd battery [last unloaded: microcode] Pid: 2358, comm: rm Not tainted 2.6.35-rc4 #32 M912/M912 EIP: 0060:[c10c383b] EFLAGS: 00010202 CPU: 1 EIP is at lookup_inline_extent_backref+0xf2/0x406 EAX: 0001 EBX: 0007 ECX: EDX: ESI: 0004 EDI: f7268150 EBP: 0004 ESP: f5aa5d08 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 Process rm (pid: 2358, ti=f5aa4000 task=f6f0fa70 task.ti=f5aa4000) Stack: f702f8c0 f744e080 f665f380 00b0 f6c80f00 0 f744e080 c10ec226 e98acfff f6c98000 1001 0e987000 0004 0 0850 040e9870 a800 1000 0007 0e987000 Call Trace: [c10ec226] ? set_extent_dirty+0x19/0x1d [c10c5081] ? __btrfs_free_extent+0xda/0x675 [c10c88bf] ? run_clustered_refs+0x699/0x6d7 [c10d239f] ? btrfs_mark_buffer_dirty+0xa3/0xef [c1101454] ? btrfs_find_ref_cluster+0xf9/0x13a [c10c89bc] ? btrfs_run_delayed_refs+0xbf/0x155 [c10d3a73] ? __btrfs_end_transaction+0x53/0x16c [c10db480] ? btrfs_delete_inode+0x166/0x17e [c102280d] ? get_parent_ip+0x8/0x19 [c108fe5c] ? generic_delete_inode+0x6f/0xbd [c108f5b3] ? iput+0x46/0x48 [c10893a8] ? do_unlinkat+0xc7/0x109 [c102280d] ? get_parent_ip+0x8/0x19 [c10822e3] ? fput+0x12/0x15c [c10a2f30] ? dnotify_flush+0x41/0xc2 [c107fe85] ? filp_close+0x4c/0x52 [c107feed] ? sys_close+0x62/0x9b [c1002550] ? sysenter_do_call+0x12/0x26 Code: 80 4e 68 02 8d 4c 24 43 89 f8 6a 01 ff 74 24 1c ff 74 24 08 8b 54 24 38 e8 01 c2 ff ff 83 c4 0c 83 f8 00 0f 8c e1 02 00 00 74 02 0f 0b 8b 04 24 8b 34 24 8b 00 8b 56 20 89 44 24 08 e8 2e fa ff EIP: [c10c383b] lookup_inline_extent_backref+0xf2/0x406 SS:ESP 0068:f5aa5d08 ---[ end trace d97601f0b455ca72 ]--- note: rm[2358] exited with preempt_count 2 BUG: scheduling while atomic: rm/2358/0x1003 Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss nfs lockd nfs_acl auth_rpcgss sunrpc sco rfcomm bnep l2cap crc16 xts gf128mul usb_storage dm_crypt dm_mod coretemp hwmon acpi_cpufreq mperf snd_hda_codec_realtek uvcvideo iwl3945 snd_hda_intel snd_hda_codec iwlcore videodev r8169 snd_hwdep btusb snd_pcm v4l1_compat mac80211 snd_timer bluetooth snd mii cfg80211 soundcore sg rfkill ac i2c_i801 snd_page_alloc uhci_hcd battery [last unloaded: microcode] Pid: 2358, comm: rm Tainted: G D 2.6.35-rc4 #32 Call Trace: [c12de6b3] ? schedule+0x88/0x332 [c10237c1] ? __cond_resched+0xf/0x19 [c12de9e2] ? _cond_resched+0x12/0x18 [c106ceec] ? unmap_vmas+0x4e7/0x534 [c1070c8f] ? exit_mmap+0x64/0xa4 [c1026089] ? mmput+0x21/0x96 [c102938e] ? exit_mm+0xe7/0xf0 [c12dfa28] ? _raw_spin_unlock_irqrestore+0x1a/0x24 [c103aaa1] ? hrtimer_try_to_cancel+0x31/0x3a [c102a42e] ? do_exit+0x17b/0x57d [c1028e78] ? kmsg_dump+0x81/0xf9 [c1002d06] ? do_invalid_op+0x0/0x76 [c1004fa0] ? oops_end+0x72/0x75 [c1002d6f] ? do_invalid_op+0x69/0x76 [c10c383b] ? lookup_inline_extent_backref+0xf2/0x406 [c10bdc9a] ? generic_bin_search.clone.0+0x145/0x150 [c10bcf30] ? btrfs_cow_block+0x106/0x112 [c10bdcdc] ? bin_search+0x37/0x3d [c10bfe33] ? btrfs_search_slot+0x405/0x477 [c12e031a] ? error_code+0x66/0x6c [c1002d06] ? do_invalid_op+0x0/0x76 [c10c383b] ? lookup_inline_extent_backref+0xf2/0x406 [c10ec226] ? set_extent_dirty+0x19/0x1d [c10c5081] ? __btrfs_free_extent+0xda/0x675 [c10c88bf] ? run_clustered_refs+0x699/0x6d7 [c10d239f] ? btrfs_mark_buffer_dirty+0xa3/0xef [c1101454] ? btrfs_find_ref_cluster+0xf9/0x13a [c10c89bc] ? btrfs_run_delayed_refs+0xbf/0x155 [c10d3a73] ? __btrfs_end_transaction+0x53/0x16c [c10db480] ? btrfs_delete_inode+0x166/0x17e [c102280d] ? get_parent_ip+0x8/0x19 [c108fe5c] ? generic_delete_inode+0x6f/0xbd [c108f5b3] ? iput+0x46/0x48 [c10893a8] ? do_unlinkat+0xc7/0x109 [c102280d] ? get_parent_ip+0x8/0x19 [c10822e3] ? fput+0x12/0x15c [c10a2f30] ? dnotify_flush+0x41/0xc2 [c107fe85] ? filp_close+0x4c/0x52 [c107feed] ? sys_close+0x62/0x9b [c1002550] ? sysenter_do_call+0x12/0x26 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel BUG at fs/btrfs/extent-tree.c:1353
Am Donnerstag 08 Juli 2010, 16:31:09 schrieb Chris Mason: Neither Yan nor I have been able to reproduce this locally, but a few people have now hit it. Johannes, are you available to try out a debugging kernel to try and track this down? Sure, just tell me what to do. Is it enough to recompile the kernel with debug options enabled or are special debug patches necessary? regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Still ENOSPC problems with 2.6.35-rc3
Am Donnerstag 17 Juni 2010, 02:47:07 schrieb Yan, Zheng: On Thu, Jun 17, 2010 at 7:56 AM, Johannes Hirte johannes.hi...@fem.tu-ilmenau.de wrote: Am Donnerstag 17 Juni 2010, 01:12:54 schrieb Yan, Zheng: On Thu, Jun 17, 2010 at 1:48 AM, Johannes Hirte johannes.hi...@fem.tu-ilmenau.de wrote: With kernel-2.6.34 I run into the ENOSPC problems that where reported on this list recently. The filesystem was somewhat over 90% full and most operations on it caused a Oops. I was able to delete files by trial and error and freed up half of the filesystem space. Operation on the other files still caused an Oops. For 2.6.35 there went some patches in, that addressed this problem. Sadly they don't fix it but only avoid the Oops. A simple 'ls' on this filesystem results in To avoid ENOSPC oops, btrfs in 2.6.35 reserves more metadata space for system use than older btrfs. If the FS has already ran out of metadata space, using btrfs in 2.6.35 doesn't help. Yan, Zheng So how can this be fixed/avoided? There must be some free metadata space, since I was able to delete files, more than 20Gig, mostly small files. Also from my understanding, when freeing space by deleting files, metadata space should be freed. Or do I get something wrong here? 2.6.35 does change something, since I can delete more files, where 2.6.34 does Oops. But you're right, it doesn't help at all. So, where is this space and why it can't be used? what will happen if you keep deleting files using 2.6.35? With 2.6.35 I'm able to continue deleting files, even those where 2.6.34 would Oops. It's slow and give me many of this warnings in dmesg but they get deleted. I didn't tried it to the end, but I can do if you want. I've saved the affected filesystem to a separate partition, so I can test with it. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Still ENOSPC problems with 2.6.35-rc3
With kernel-2.6.34 I run into the ENOSPC problems that where reported on this list recently. The filesystem was somewhat over 90% full and most operations on it caused a Oops. I was able to delete files by trial and error and freed up half of the filesystem space. Operation on the other files still caused an Oops. For 2.6.35 there went some patches in, that addressed this problem. Sadly they don't fix it but only avoid the Oops. A simple 'ls' on this filesystem results in [ cut here ] WARNING: at fs/btrfs/extent-tree.c:3441 btrfs_block_rsv_check+0x10c/0x13e() Hardware name: To Be Filled By O.E.M. Modules linked in: snd_seq_midi snd_emu10k1_synth snd_emux_synth snd_seq_virmidi snd_seq_midi_emul snd_seq_oss snd_seq_midi_event snd_seq snd_pcm_oss snd_mixer_oss radeon ttm drm_kms_helper drm i2c_algo_bit snd_emu10k1 snd_rawmidi snd_ac97_codec ac97_bus snd_pcm snd_seq_device snd_timer snd_page_alloc snd_util_mem snd_hwdep snd amd64_edac_mod sata_sil sg sr_mod uhci_hcd ohci_hcd edac_core edac_mce_amd k8temp i2c_amd8111 i2c_amd756 hwmon Pid: 26973, comm: ls Not tainted 2.6.35-rc3 #1 Call Trace: [81031044] ? warn_slowpath_common+0x78/0x8c [81147fdf] ? btrfs_block_rsv_check+0x10c/0x13e [81155857] ? __btrfs_end_transaction+0x9f/0x1b1 [8115aaa2] ? btrfs_dirty_inode+0x58/0xf9 [810b07ba] ? __mark_inode_dirty+0x25/0x149 [810a809a] ? touch_atime+0xfc/0x125 [810a3a32] ? filldir+0x0/0xc3 [810a3c1c] ? vfs_readdir+0x76/0x9c [810a3d7e] ? sys_getdents+0x7d/0xcd [81364f1f] ? page_fault+0x1f/0x30 [81001e2b] ? system_call_fastpath+0x16/0x1b ---[ end trace 4aa882f64f792d16 ]--- block_rsv size 654311424 reserved 67809280 freed 0 0 [ cut here ] WARNING: at fs/btrfs/extent-tree.c:3441 btrfs_block_rsv_check+0x10c/0x13e() Hardware name: To Be Filled By O.E.M. Modules linked in: snd_seq_midi snd_emu10k1_synth snd_emux_synth snd_seq_virmidi snd_seq_midi_emul snd_seq_oss snd_seq_midi_event snd_seq snd_pcm_oss snd_mixer_oss radeon ttm drm_kms_helper drm i2c_algo_bit snd_emu10k1 snd_rawmidi snd_ac97_codec ac97_bus snd_pcm snd_seq_device snd_timer snd_page_alloc snd_util_mem snd_hwdep snd amd64_edac_mod sata_sil sg sr_mod uhci_hcd ohci_hcd edac_core edac_mce_amd k8temp i2c_amd8111 i2c_amd756 hwmon Pid: 26970, comm: btrfs-transacti Tainted: GW 2.6.35-rc3 #1 Call Trace: [81031044] ? warn_slowpath_common+0x78/0x8c [81147fdf] ? btrfs_block_rsv_check+0x10c/0x13e [81155857] ? __btrfs_end_transaction+0x9f/0x1b1 [81155a7a] ? btrfs_commit_transaction+0xf4/0x5fd [8102c39f] ? enqueue_task+0x39/0x47 [81363dbb] ? mutex_lock+0xd/0x31 [81043979] ? autoremove_wake_function+0x0/0x2a [81151b5b] ? transaction_kthread+0x16d/0x213 [811519ee] ? transaction_kthread+0x0/0x213 [810435ad] ? kthread+0x75/0x7d [81002b54] ? kernel_thread_helper+0x4/0x10 [81043538] ? kthread+0x0/0x7d [81002b50] ? kernel_thread_helper+0x0/0x10 ---[ end trace 4aa882f64f792d17 ]--- block_rsv size 654311424 reserved 67809280 freed 0 0 [ cut here ] WARNING: at fs/btrfs/extent-tree.c:3441 btrfs_block_rsv_check+0x10c/0x13e() Hardware name: To Be Filled By O.E.M. Modules linked in: snd_seq_midi snd_emu10k1_synth snd_emux_synth snd_seq_virmidi snd_seq_midi_emul snd_seq_oss snd_seq_midi_event snd_seq snd_pcm_oss snd_mixer_oss radeon ttm drm_kms_helper drm i2c_algo_bit snd_emu10k1 snd_rawmidi snd_ac97_codec ac97_bus snd_pcm snd_seq_device snd_timer snd_page_alloc snd_util_mem snd_hwdep snd amd64_edac_mod sata_sil sg sr_mod uhci_hcd ohci_hcd edac_core edac_mce_amd k8temp i2c_amd8111 i2c_amd756 hwmon Pid: 26973, comm: ls Tainted: GW 2.6.35-rc3 #1 Call Trace: [81031044] ? warn_slowpath_common+0x78/0x8c [81147fdf] ? btrfs_block_rsv_check+0x10c/0x13e [81155857] ? __btrfs_end_transaction+0x9f/0x1b1 [811562fb] ? start_transaction+0x15f/0x1c4 [8115aaaf] ? btrfs_dirty_inode+0x65/0xf9 [810b07ba] ? __mark_inode_dirty+0x25/0x149 [810a809a] ? touch_atime+0xfc/0x125 [810a3a32] ? filldir+0x0/0xc3 [810a3c1c] ? vfs_readdir+0x76/0x9c [810a3d7e] ? sys_getdents+0x7d/0xcd [81364f1f] ? page_fault+0x1f/0x30 [81001e2b] ? system_call_fastpath+0x16/0x1b ---[ end trace 4aa882f64f792d18 ]--- block_rsv size 654311424 reserved 67809280 freed 0 0 btrfs: fail to dirty inode 256 error -28 [ cut here ] WARNING: at fs/btrfs/extent-tree.c:3441 btrfs_block_rsv_check+0x10c/0x13e() Hardware name: To Be Filled By O.E.M. Modules linked in: snd_seq_midi snd_emu10k1_synth snd_emux_synth snd_seq_virmidi snd_seq_midi_emul snd_seq_oss snd_seq_midi_event snd_seq snd_pcm_oss snd_mixer_oss radeon ttm drm_kms_helper drm i2c_algo_bit snd_emu10k1 snd_rawmidi snd_ac97_codec ac97_bus snd_pcm
Re: [patch 11/11] btrfs: The file argument for fsync() is never null
Am Dienstag 15 Juni 2010, 02:08:20 schrieb Chris Mason: On Mon, Jun 14, 2010 at 11:45:40PM +0200, Johannes Hirte wrote: Am Montag 14 Juni 2010, 23:16:01 schrieb Christoph Hellwig: On Mon, Jun 14, 2010 at 11:11:20PM +0200, Dan Carpenter wrote: Looks like you've applied the patch to a far too old kernel. It can't be NULL for quite a while already. You're the expert, but it looks like it could be null in 2.6.34 like he says. I'm just looking at vfs_fsync_range() in git show v2.6.34:fs/sync.c. 2.6.34 is far too old. For the changes yes, but not for working. I needed the btrfs fixes without all the other bugs introduced with 2.6.35-rc. I was to careless and pulled to much changes in. My fault. Well, my fault. I usually keep the btrfs-unstable tree against one release old, and the users have come to expect it. I'll make a .34 branch that works. -chris What about backporting only the important patches to the stable series? Or would this be to much work for a still experimental filesystem? regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Still ENOSPC problems with 2.6.35-rc3
Am Donnerstag 17 Juni 2010, 01:12:54 schrieb Yan, Zheng: On Thu, Jun 17, 2010 at 1:48 AM, Johannes Hirte johannes.hi...@fem.tu-ilmenau.de wrote: With kernel-2.6.34 I run into the ENOSPC problems that where reported on this list recently. The filesystem was somewhat over 90% full and most operations on it caused a Oops. I was able to delete files by trial and error and freed up half of the filesystem space. Operation on the other files still caused an Oops. For 2.6.35 there went some patches in, that addressed this problem. Sadly they don't fix it but only avoid the Oops. A simple 'ls' on this filesystem results in To avoid ENOSPC oops, btrfs in 2.6.35 reserves more metadata space for system use than older btrfs. If the FS has already ran out of metadata space, using btrfs in 2.6.35 doesn't help. Yan, Zheng So how can this be fixed/avoided? There must be some free metadata space, since I was able to delete files, more than 20Gig, mostly small files. Also from my understanding, when freeing space by deleting files, metadata space should be freed. Or do I get something wrong here? 2.6.35 does change something, since I can delete more files, where 2.6.34 does Oops. But you're right, it doesn't help at all. So, where is this space and why it can't be used? regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [patch 11/11] btrfs: The file argument for fsync() is never null
Am Samstag 29 Mai 2010, 11:49:07 schrieb Dan Carpenter: The file argument for fsync is never null so we can remove this check. What drew my attention here is that 7ea8085910e: drop unused dentry argument to -fsync introduced an unconditional dereference at the start of the function and that generated a smatch warning. Signed-off-by: Dan Carpenter erro...@gmail.com diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index 787b50a..e252d23 100644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -1140,7 +1140,7 @@ int btrfs_sync_file(struct file *file, int datasync) /* * ok we haven't committed the transaction yet, lets do a commit */ - if (file file-private_data) + if (file-private_data) btrfs_ioctl_trans_end(file); trans = btrfs_start_transaction(root, 0); I think you're wrong here. I've run into a kernel null pointer dereference at this point with a NFS exported btrfs filesystem: BUG: unable to handle kernel NULL pointer dereference at 0098 IP: [8115d7b3] btrfs_sync_file+0xa7/0x15a PGD 2a72e067 PUD 1c29f067 PMD 0 Oops: [#1] SMP last sysfs file: /sys/devices/pci:00/:00:01.0/:01:00.0/power_method CPU 1 Modules linked in: snd_seq_midi snd_emu10k1_synth snd_emux_synth snd_seq_virmidi snd_seq_midi_emul snd_seq_oss snd_seq_midi_event snd_seq snd_pcm_oss snd_mixer_oss radeon ttm drm_kms_helper drm i2c_algo_bit snd_emu10k1 snd_rawmidi snd_ac97_codec ac97_bus snd_pcm snd_seq_device snd_timer snd_page_alloc snd_util_mem snd_hwdep snd amd64_edac_mod edac_core uhci_hcd ohci_hcd sata_sil edac_mce_amd sg sr_mod k8temp i2c_amd756 i2c_amd8111 hwmon Pid: 2330, comm: nfsd Not tainted 2.6.34-btrfs-2-drm #1 TYAN Tiger K8W Dual AMD Opteron, S2875/To Be Filled By O.E.M. RIP: 0010:[8115d7b3] [8115d7b3] btrfs_sync_file+0xa7/0x15a RSP: :880119e9bcf0 EFLAGS: 00010202 RAX: RBX: 88011e4da000 RCX: 0020 RDX: 0df8 RSI: RDI: 88011e495b28 RBP: 88011c30eb78 R08: 8141bab0 R09: R10: 880119e9bc50 R11: 88011c30eb78 R12: 88011c305240 R13: R14: 8141bab0 R15: 880119d1d040 FS: 2ac4c0279180() GS:88000188() knlGS:f74726d0 CS: 0010 DS: ES: CR0: 8005003b CR2: 0098 CR3: 15c84000 CR4: 06f0 DR0: DR1: DR2: DR3: DR6: 0ff0 DR7: 0400 Process nfsd (pid: 2330, threadinfo 880119e9a000, task 880119e185e0) Stack: 8800273da3d8 0002 0003 0 88011c305240 88011c30ec90 810b1a99 0 880119d1d040 8800 880023e63240 Call Trace: [810b1a99] ? vfs_fsync_range+0x7a/0xa7 [8112e055] ? nfsd4_create_clid_dir+0x12d/0x15d [81129e64] ? nfsd4_open_confirm+0xec/0x118 [8111dcb1] ? nfsd4_proc_compound+0x1fd/0x3b5 [8110f1cb] ? nfsd_dispatch+0xdf/0x1b5 [8132e89a] ? svc_process+0x41d/0x703 [8102bcc3] ? default_wake_function+0x0/0xf [8110f6dd] ? nfsd+0xe1/0x125 [8110f5fc] ? nfsd+0x0/0x125 [81041fcd] ? kthread+0x75/0x7d [81002b54] ? kernel_thread_helper+0x4/0x10 [81041f58] ? kthread+0x0/0x7d [81002b50] ? kernel_thread_helper+0x0/0x10 Code: 8b bb 28 01 00 00 89 44 24 08 48 81 c7 28 1b 00 00 e8 1e 79 1f 00 8b 44 24 08 e9 b4 00 00 00 48 81 c7 28 1b 00 00 e8 09 79 1f 00 49 83 bd 98 00 00 00 00 74 08 4c 89 ef e8 06 75 01 00 31 f6 48 RIP [8115d7b3] btrfs_sync_file+0xa7/0x15a RSP 880119e9bcf0 CR2: 0098 ---[ end trace 5c7989eaf4eda923 ]--- addr2line showed me exact this change you made. It happend with a linux-2.6.34 with the latest btrfs-changes on top. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [patch 11/11] btrfs: The file argument for fsync() is never null
Am Montag 14 Juni 2010, 23:16:01 schrieb Christoph Hellwig: On Mon, Jun 14, 2010 at 11:11:20PM +0200, Dan Carpenter wrote: Looks like you've applied the patch to a far too old kernel. It can't be NULL for quite a while already. You're the expert, but it looks like it could be null in 2.6.34 like he says. I'm just looking at vfs_fsync_range() in git show v2.6.34:fs/sync.c. 2.6.34 is far too old. For the changes yes, but not for working. I needed the btrfs fixes without all the other bugs introduced with 2.6.35-rc. I was to careless and pulled to much changes in. My fault. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: task imap:2958 blocked for more than 120 seconds
Am Sonntag 10 Januar 2010 21:05:46 schrieb Johannes Hirte:
I've observed this hanging task now several times. Not sure when this
started, but 2.6.32 is affected too, IIRC. I don't have a test pattern for
this. Dovecot imap triggers this from time to time. I've enabled
CONFIG_DETECT_HUNG_TASK now and got this two tasks which hang:
INFO: task imap:2958 blocked for more than 120 seconds.
echo 0 /proc/sys/kernel/hung_task_timeout_secs disables this message.
imap D 0 2958 2653 0x
88008caf5a28 0046 810544cf
88008caf5998 0001 88008caf5fd8 88008caf9530
de78 001d2700 001d2700 88008caf9530
Call Trace:
[810544cf] ? trace_hardirqs_off+0xd/0xf
[8105613b] ? trace_hardirqs_on_caller+0x10c/0x130
[81071262] ? sync_page+0x0/0x48
[8138b46a] io_schedule+0x38/0x4d
[810712a6] sync_page+0x44/0x48
[8138b99a] __wait_on_bit_lock+0x41/0x8a
[8107124e] __lock_page+0x61/0x68
[810475a3] ? wake_bit_function+0x0/0x2e
[810718c6] filemap_fault+0xea/0x345
[81084022] __do_fault+0x50/0x3d3
[81086077] handle_mm_fault+0x32f/0x65d
[8102039d] ? do_page_fault+0xf4/0x26f
[811cacb4] ? __down_read_trylock+0x46/0x4e
[8104b0f3] ? down_read_trylock+0x3f/0x49
[8102039d] ? do_page_fault+0xf4/0x26f
[81020500] do_page_fault+0x257/0x26f
[8138e2ff] page_fault+0x1f/0x30
[81083a13] ? might_fault+0x57/0xa7
[811746e2] ? btrfs_copy_from_user+0x4f/0x113
[81174771] ? btrfs_copy_from_user+0xde/0x113
[81174cd0] btrfs_file_write+0x439/0x6fe
[810a5c34] vfs_write+0xad/0x14e
[8105613b] ? trace_hardirqs_on_caller+0x10c/0x130
[810a5d2a] sys_pwrite64+0x55/0x74
[81001eeb] system_call_fastpath+0x16/0x1b
2 locks held by imap/2958:
#0: (sb-s_type-i_mutex_key#4){+.+.+.}, at: [81174a00]
btrfs_file_write+0x169/0x6fe
#1: (mm-mmap_sem){++}, at: [8102039d]
do_page_fault+0xf4/0x26f INFO: task flush-btrfs-2:2783 blocked for more
than 120 seconds.
echo 0 /proc/sys/kernel/hung_task_timeout_secs disables this message.
flush-btrfs-2 D 0 2783 2 0x
88010cdcf9d0 0046 810544cf
88010cdcf940 88010cdcffd8 88010cd18290
de78 001d2700 001d2700 88010cd18290
Call Trace:
[810544cf] ? trace_hardirqs_off+0xd/0xf
[8105613b] ? trace_hardirqs_on_caller+0x10c/0x130
[81071262] ? sync_page+0x0/0x48
[8138b46a] io_schedule+0x38/0x4d
[810712a6] sync_page+0x44/0x48
[8138b99a] __wait_on_bit_lock+0x41/0x8a
[81070ce1] ? find_get_pages_tag+0x0/0x130
[8107124e] __lock_page+0x61/0x68
[810475a3] ? wake_bit_function+0x0/0x2e
[811871f6] T.858+0xf1/0x2cd
[8104c4f9] ? sched_clock_cpu+0xc6/0xd4
[8104c3cd] ? sched_clock_local+0x1c/0x82
[8104c4f9] ? sched_clock_cpu+0xc6/0xd4
[810544cf] ? trace_hardirqs_off+0xd/0xf
[811874d3] extent_writepages+0x3f/0x54
[8116dbd5] ? btrfs_get_extent+0x0/0x7ee
[8116da7e] btrfs_writepages+0x22/0x24
[81077d12] do_writepages+0x1f/0x28
[810c0233] writeback_single_inode+0xf1/0x2f0
[810c0f57] writeback_inodes_wb+0x3a9/0x4b2
[810c118b] wb_writeback+0x12b/0x1af
[810c1436] wb_do_writeback+0x17f/0x195
[810c1342] ? wb_do_writeback+0x8b/0x195
[810c1477] bdi_writeback_task+0x2b/0x84
[81082a45] ? bdi_start_fn+0x0/0xcf
[81082ab6] bdi_start_fn+0x71/0xcf
[81082a45] ? bdi_start_fn+0x0/0xcf
[81047197] kthread+0x7a/0x82
[81002c94] kernel_thread_helper+0x4/0x10
[8138e0fc] ? restore_args+0x0/0x30
[8104711d] ? kthread+0x0/0x82
[81002c90] ? kernel_thread_helper+0x0/0x10
1 lock held by flush-btrfs-2/2783:
#0: (type-s_umount_key#20){..}, at: [810c0e82]
writeback_inodes_wb+0x2d4/0x4b2
regards,
Johannes
It happend again, today I found in the logs:
INFO: task imap:2590 blocked for more than 120 seconds.
echo 0 /proc/sys/kernel/hung_task_timeout_secs disables this message.
imap D 88002828ddf8 0 2590 2472 0x
88011fa53000 0086 8800282923a0
88011b7cc480 880028294c58 880117a5e000
fffdcf81 880117a5e280 00012340 00012340
Call Trace:
[81061de2] ? sync_page+0x0/0x46
[8134acaa] ? io_schedule+0x35/0x48
[81061e23] ? sync_page+0x41/0x46
[8134b185] ? __wait_on_bit_lock+0x3c/0x85
[81061dcf] ? __lock_page+0x5d/0x63
[81042877] ? wake_bit_function+0x0/0x2e
Re: [btrfs] kernel BUG at include/linux/spinlock.h:376!
Am Donnerstag 14 Januar 2010 20:37:08 schrieb Chris Mason: On Thu, Jan 07, 2010 at 10:29:32PM +0100, Johannes Hirte wrote: One of my btrfs filesystems gives the following bug message on access: Jan 6 23:08:12 datengrab kernel: [ cut here ] Jan 6 23:08:12 datengrab kernel: kernel BUG at include/linux/spinlock.h:376! Jan 6 23:08:12 datengrab kernel: invalid opcode: [#1] SMP Jan 6 23:08:12 datengrab kernel: last sysfs file: /sys/devices/pci:00/:00:18.3/temp1_input Jan 6 23:08:12 datengrab kernel: CPU 1 Jan 6 23:08:12 datengrab kernel: Pid: 2837, comm: btrfs-endio-wri Not tainted 2.6.33-rc3-00033-g03b7675 #12 TYAN Tiger K8W Dual AMD Opteron, S2875/To Be Filled By O.E.M. Jan 6 23:08:12 datengrab kernel: RIP: 0010:[8118f7ea] [8118f7ea] btrfs_assert_tree_locked+0x16/0x1c Well, we really should have this tree block locked, but btrfs_mark_extent_written is doing some special things. Is the trace always the same? Sorry for the long delay. Yes the trace was always the same. I can't test patches, since I'm not working on the corrupted FS anymore. The bug only occurred on the corrupted filesystem. But as I've seen, Yan Zheng has tracked it down (commit 6c7d54ac87f338c479d9729e8392eca3f76e11e1). I still suspect that the FS corruption was caused by this bug. It didn't happened again. If so, I'll report. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [btrfs] kernel BUG at include/linux/spinlock.h:376!
Am Donnerstag 07 Januar 2010 22:29:32 schrieb Johannes Hirte: One of my btrfs filesystems gives the following bug message on access: Jan 6 23:08:12 datengrab kernel: [ cut here ] Jan 6 23:08:12 datengrab kernel: kernel BUG at include/linux/spinlock.h:376! Jan 6 23:08:12 datengrab kernel: invalid opcode: [#1] SMP Jan 6 23:08:12 datengrab kernel: last sysfs file: /sys/devices/pci:00/:00:18.3/temp1_input Jan 6 23:08:12 datengrab kernel: CPU 1 Jan 6 23:08:12 datengrab kernel: Pid: 2837, comm: btrfs-endio-wri Not tainted 2.6.33-rc3-00033-g03b7675 #12 TYAN Tiger K8W Dual AMD Opteron, S2875/To Be Filled By O.E.M. Jan 6 23:08:12 datengrab kernel: RIP: 0010:[8118f7ea] [8118f7ea] btrfs_assert_tree_locked+0x16/0x1c Jan 6 23:08:12 datengrab kernel: RSP: 0018:8800237b5a50 EFLAGS: 00010246 Jan 6 23:08:12 datengrab kernel: RAX: 0404 RBX: 88011f444ea0 RCX: 8800 Jan 6 23:08:12 datengrab kernel: RDX: 0004 RSI: 88011c219000 RDI: 8800829b3c00 Jan 6 23:08:12 datengrab kernel: RBP: 8800237b5a50 R08: 0016 R09: 8800237b5a30 Jan 6 23:08:12 datengrab kernel: R10: 8800237b5a28 R11: 0191 R12: 88011c219000 Jan 6 23:08:12 datengrab kernel: R13: 000c R14: 0001 R15: 88011981e740 Jan 6 23:08:12 datengrab kernel: FS: 7f2c79ac8700() GS:88002b40() knlGS: Jan 6 23:08:12 datengrab kernel: CS: 0010 DS: ES: CR0: 8005003b Jan 6 23:08:12 datengrab kernel: CR2: 026300a0 CR3: 000116b7f000 CR4: 06f0 Jan 6 23:08:12 datengrab kernel: DR0: DR1: DR2: Jan 6 23:08:12 datengrab kernel: DR3: DR6: 0ff0 DR7: 0400 Jan 6 23:08:12 datengrab kernel: Process btrfs-endio-wri (pid: 2837, threadinfo 8800237b4000, task 8800235037e0) Jan 6 23:08:12 datengrab kernel: Stack: Jan 6 23:08:12 datengrab kernel: 8800237b5ac0 81154ded 012c 000c Jan 6 23:08:12 datengrab kernel: 0 8816 000181150b93 0ce3 0f66 Jan 6 23:08:12 datengrab kernel: 0 88007ff44000 8800829b3d00 88011f444ea0 Jan 6 23:08:12 datengrab kernel: Call Trace: Jan 6 23:08:12 datengrab kernel: [81154ded] push_leaf_left+0x9f/0x158 Jan 6 23:08:12 datengrab kernel: [8115539e] btrfs_del_items+0x363/0x48f Jan 6 23:08:12 datengrab kernel: [81175b06] btrfs_mark_extent_written+0x53b/0x55f Jan 6 23:08:12 datengrab kernel: [8105616c] ? trace_hardirqs_on+0xd/0xf Jan 6 23:08:12 datengrab kernel: [8138bc85] ? mutex_unlock+0x9/0xb Jan 6 23:08:12 datengrab kernel: [8117422d] btrfs_finish_ordered_io+0x176/0x247 Jan 6 23:08:12 datengrab kernel: [810544cf] ? trace_hardirqs_off+0xd/0xf Jan 6 23:08:12 datengrab kernel: [81174313] btrfs_writepage_end_io_hook+0x15/0x17 Jan 6 23:08:12 datengrab kernel: [81184dd5] end_bio_extent_writepage+0xa9/0x154 Jan 6 23:08:12 datengrab kernel: [8105613b] ? trace_hardirqs_on_caller+0x10c/0x130 Jan 6 23:08:12 datengrab kernel: [810c8d27] bio_endio+0x26/0x28 Jan 6 23:08:12 datengrab kernel: [81167b8d] end_workqueue_fn+0x10c/0x11b Jan 6 23:08:12 datengrab kernel: [8118d259] worker_loop+0x175/0x44d Jan 6 23:08:12 datengrab kernel: [8118d0e4] ? worker_loop+0x0/0x44d Jan 6 23:08:12 datengrab kernel: [81047197] kthread+0x7a/0x82 Jan 6 23:08:12 datengrab kernel: [81002c94] kernel_thread_helper+0x4/0x10 Jan 6 23:08:12 datengrab kernel: [8138e13c] ? restore_args+0x0/0x30 Jan 6 23:08:12 datengrab kernel: [8104711d] ? kthread+0x0/0x82 Jan 6 23:08:12 datengrab kernel: [81002c90] ? kernel_thread_helper+0x0/0x10 Jan 6 23:08:12 datengrab kernel: Code: c8 ff 48 81 c4 88 00 00 00 5b 41 5c 41 5d 41 5e 41 5f c9 c3 90 f6 47 38 02 55 48 89 e5 75 10 8b 47 70 89 c2 c1 fa 08 38 c 2 75 04 0f 0b eb fe c9 c3 55 31 c0 65 48 8b 14 25 48 b5 00 00 48 89 e5 Jan 6 23:08:12 datengrab kernel: RIP [8118f7ea] btrfs_assert_tree_locked+0x16/0x1c Jan 6 23:08:12 datengrab kernel: RSP 8800237b5a50 Jan 6 23:08:12 datengrab kernel: ---[ end trace 96d932f09da027f6 ]--- It only happens on write access. I was able to copy all the data to another drive without any error. The filesystem is damaged, btrfsck gives root 5 inode 6969680 errors 2000 found 191511994368 bytes used err is 1 total csum bytes: 186404900 total tree bytes: 629936128 total fs tree bytes: 388333568 btree space waste bytes: 146015924 file data blocks allocated: 191957340160 referenced 190751694848 Btrfs v0.19-4-gab8fb4c It's the btrfs-code from 2.6.33 with the following additional patches: diff --git a/fs/btrfs
task imap:2958 blocked for more than 120 seconds
I've observed this hanging task now several times. Not sure when this started,
but 2.6.32 is affected too, IIRC. I don't have a test pattern for this. Dovecot
imap triggers this from time to time. I've enabled CONFIG_DETECT_HUNG_TASK now
and got this two tasks which hang:
INFO: task imap:2958 blocked for more than 120 seconds.
echo 0 /proc/sys/kernel/hung_task_timeout_secs disables this message.
imap D 0 2958 2653 0x
88008caf5a28 0046 810544cf
88008caf5998 0001 88008caf5fd8 88008caf9530
de78 001d2700 001d2700 88008caf9530
Call Trace:
[810544cf] ? trace_hardirqs_off+0xd/0xf
[8105613b] ? trace_hardirqs_on_caller+0x10c/0x130
[81071262] ? sync_page+0x0/0x48
[8138b46a] io_schedule+0x38/0x4d
[810712a6] sync_page+0x44/0x48
[8138b99a] __wait_on_bit_lock+0x41/0x8a
[8107124e] __lock_page+0x61/0x68
[810475a3] ? wake_bit_function+0x0/0x2e
[810718c6] filemap_fault+0xea/0x345
[81084022] __do_fault+0x50/0x3d3
[81086077] handle_mm_fault+0x32f/0x65d
[8102039d] ? do_page_fault+0xf4/0x26f
[811cacb4] ? __down_read_trylock+0x46/0x4e
[8104b0f3] ? down_read_trylock+0x3f/0x49
[8102039d] ? do_page_fault+0xf4/0x26f
[81020500] do_page_fault+0x257/0x26f
[8138e2ff] page_fault+0x1f/0x30
[81083a13] ? might_fault+0x57/0xa7
[811746e2] ? btrfs_copy_from_user+0x4f/0x113
[81174771] ? btrfs_copy_from_user+0xde/0x113
[81174cd0] btrfs_file_write+0x439/0x6fe
[810a5c34] vfs_write+0xad/0x14e
[8105613b] ? trace_hardirqs_on_caller+0x10c/0x130
[810a5d2a] sys_pwrite64+0x55/0x74
[81001eeb] system_call_fastpath+0x16/0x1b
2 locks held by imap/2958:
#0: (sb-s_type-i_mutex_key#4){+.+.+.}, at: [81174a00]
btrfs_file_write+0x169/0x6fe
Re: Still Problems with /dev/btrfs-control
Am Samstag 09 Januar 2010 12:05:34 schrieb Goffredo Baroncelli: Hi Michael On Saturday 09 January 2010, Dipl.-Ing. Michael Niederle wrote: Thanks for the quick reply! But I still have problems with btrfsctl: stat /dev/btrfs-control File: `/dev/btrfs-control' Size: 0 Blocks: 0 IO Block: 4096 block special file Device: ch/12d Inode: 659848 Links: 1 Device type: a,3e Ok, two things: 1) btrfs-control is a *character* device and _not_ a *block device* 2) on my system it is allocated under 10,55 (major/minor). This must be checked on the target machine, as the minor number is allocated dynamically. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: What protection does btrfs checksumming currently give? (Was Re: btrfs volume mounts and dies (was Re: Segfault in btrfsck))
Am Donnerstag 07 Januar 2010 20:29:49 schrieb jim owens: Steve Freitas wrote: Hi all, I was under the mistaken impression that btrfs checksumming, in its current default configuration, protected your data from bitrot. It appears this is not the case: On Wed, 2010-01-06 at 18:24 +0100, Johannes Hirte wrote: Am Mittwoch 06 Januar 2010 16:59:55 schrieb Steve Freitas: So please correct me if I have some mistaken assumptions. I thought btrfs would be tolerant of that -- if a block failed the checksum test, it would reconstruct and remap it. Only if enough redundancy is left. And with the default setup btrfs is only mirroring the metadata not the data. So can someone please tell me what the current state-of-the-art is of data protection with btrfs? Does it differ with single-device versus multiple-device configurations? Is it possible to enable data checksumming now? Under what conditions? And will it do what a naive user would expect it to do, namely, correct for diverse kinds of errors in your storage subsystem? If not, what does it do? Etc... First, understand that a checksum only says this block is good or bad. The checksum can not be used to reconstruct the data. Checksums are present for all btrfs blocks unless you explicitly shut them off with mount/ioctl/fcntl options. To have a good copy you can use as a replacement block, you must use either btrfs raid1 or raid10. You can use raid1 with 1 drive, in a mode called dup where both copies are made to that device. By default with 1 drive, btrfs uses dup for metadata and 1 copy (nodup) for file data blocks. To get file data dup, you just use mkfs.btrfs -d raid1. If you have btrfs raid, it will find the good block on a read, but AFAIK we don't have tools yet to automatically reallocate the bad one. jim Additionally I repeat the suggestion from Sander, check your drive for bad blocks. It sounds very likely that your drive is bad and you will get into trouble again with the new created FS. And the Oops you've posted smells like a bug in btrfs code. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[btrfs] kernel BUG at include/linux/spinlock.h:376!
One of my btrfs filesystems gives the following bug message on access: Jan 6 23:08:12 datengrab kernel: [ cut here ] Jan 6 23:08:12 datengrab kernel: kernel BUG at include/linux/spinlock.h:376! Jan 6 23:08:12 datengrab kernel: invalid opcode: [#1] SMP Jan 6 23:08:12 datengrab kernel: last sysfs file: /sys/devices/pci:00/:00:18.3/temp1_input Jan 6 23:08:12 datengrab kernel: CPU 1 Jan 6 23:08:12 datengrab kernel: Pid: 2837, comm: btrfs-endio-wri Not tainted 2.6.33-rc3-00033-g03b7675 #12 TYAN Tiger K8W Dual AMD Opteron, S2875/To Be Filled By O.E.M. Jan 6 23:08:12 datengrab kernel: RIP: 0010:[8118f7ea] [8118f7ea] btrfs_assert_tree_locked+0x16/0x1c Jan 6 23:08:12 datengrab kernel: RSP: 0018:8800237b5a50 EFLAGS: 00010246 Jan 6 23:08:12 datengrab kernel: RAX: 0404 RBX: 88011f444ea0 RCX: 8800 Jan 6 23:08:12 datengrab kernel: RDX: 0004 RSI: 88011c219000 RDI: 8800829b3c00 Jan 6 23:08:12 datengrab kernel: RBP: 8800237b5a50 R08: 0016 R09: 8800237b5a30 Jan 6 23:08:12 datengrab kernel: R10: 8800237b5a28 R11: 0191 R12: 88011c219000 Jan 6 23:08:12 datengrab kernel: R13: 000c R14: 0001 R15: 88011981e740 Jan 6 23:08:12 datengrab kernel: FS: 7f2c79ac8700() GS:88002b40() knlGS: Jan 6 23:08:12 datengrab kernel: CS: 0010 DS: ES: CR0: 8005003b Jan 6 23:08:12 datengrab kernel: CR2: 026300a0 CR3: 000116b7f000 CR4: 06f0 Jan 6 23:08:12 datengrab kernel: DR0: DR1: DR2: Jan 6 23:08:12 datengrab kernel: DR3: DR6: 0ff0 DR7: 0400 Jan 6 23:08:12 datengrab kernel: Process btrfs-endio-wri (pid: 2837, threadinfo 8800237b4000, task 8800235037e0) Jan 6 23:08:12 datengrab kernel: Stack: Jan 6 23:08:12 datengrab kernel: 8800237b5ac0 81154ded 012c 000c Jan 6 23:08:12 datengrab kernel: 0 8816 000181150b93 0ce3 0f66 Jan 6 23:08:12 datengrab kernel: 0 88007ff44000 8800829b3d00 88011f444ea0 Jan 6 23:08:12 datengrab kernel: Call Trace: Jan 6 23:08:12 datengrab kernel: [81154ded] push_leaf_left+0x9f/0x158 Jan 6 23:08:12 datengrab kernel: [8115539e] btrfs_del_items+0x363/0x48f Jan 6 23:08:12 datengrab kernel: [81175b06] btrfs_mark_extent_written+0x53b/0x55f Jan 6 23:08:12 datengrab kernel: [8105616c] ? trace_hardirqs_on+0xd/0xf Jan 6 23:08:12 datengrab kernel: [8138bc85] ? mutex_unlock+0x9/0xb Jan 6 23:08:12 datengrab kernel: [8117422d] btrfs_finish_ordered_io+0x176/0x247 Jan 6 23:08:12 datengrab kernel: [810544cf] ? trace_hardirqs_off+0xd/0xf Jan 6 23:08:12 datengrab kernel: [81174313] btrfs_writepage_end_io_hook+0x15/0x17 Jan 6 23:08:12 datengrab kernel: [81184dd5] end_bio_extent_writepage+0xa9/0x154 Jan 6 23:08:12 datengrab kernel: [8105613b] ? trace_hardirqs_on_caller+0x10c/0x130 Jan 6 23:08:12 datengrab kernel: [810c8d27] bio_endio+0x26/0x28 Jan 6 23:08:12 datengrab kernel: [81167b8d] end_workqueue_fn+0x10c/0x11b Jan 6 23:08:12 datengrab kernel: [8118d259] worker_loop+0x175/0x44d Jan 6 23:08:12 datengrab kernel: [8118d0e4] ? worker_loop+0x0/0x44d Jan 6 23:08:12 datengrab kernel: [81047197] kthread+0x7a/0x82 Jan 6 23:08:12 datengrab kernel: [81002c94] kernel_thread_helper+0x4/0x10 Jan 6 23:08:12 datengrab kernel: [8138e13c] ? restore_args+0x0/0x30 Jan 6 23:08:12 datengrab kernel: [8104711d] ? kthread+0x0/0x82 Jan 6 23:08:12 datengrab kernel: [81002c90] ? kernel_thread_helper+0x0/0x10 Jan 6 23:08:12 datengrab kernel: Code: c8 ff 48 81 c4 88 00 00 00 5b 41 5c 41 5d 41 5e 41 5f c9 c3 90 f6 47 38 02 55 48 89 e5 75 10 8b 47 70 89 c2 c1 fa 08 38 c 2 75 04 0f 0b eb fe c9 c3 55 31 c0 65 48 8b 14 25 48 b5 00 00 48 89 e5 Jan 6 23:08:12 datengrab kernel: RIP [8118f7ea] btrfs_assert_tree_locked+0x16/0x1c Jan 6 23:08:12 datengrab kernel: RSP 8800237b5a50 Jan 6 23:08:12 datengrab kernel: ---[ end trace 96d932f09da027f6 ]--- It only happens on write access. I was able to copy all the data to another drive without any error. The filesystem is damaged, btrfsck gives root 5 inode 6969680 errors 2000 found 191511994368 bytes used err is 1 total csum bytes: 186404900 total tree bytes: 629936128 total fs tree bytes: 388333568 btree space waste bytes: 146015924 file data blocks allocated: 191957340160 referenced 190751694848 Btrfs v0.19-4-gab8fb4c It's the btrfs-code from 2.6.33 with the following additional patches: diff --git a/fs/btrfs/acl.c b/fs/btrfs/acl.c index 2e9e699..3a3a96d 100644 --- a/fs/btrfs/acl.c +++ b/fs/btrfs/acl.c @@ -111,13 +111,15 @@ static int
Re: btrfs volume mounts and dies (was Re: Segfault in btrfsck)
Am Mittwoch 06 Januar 2010 16:59:55 schrieb Steve Freitas: Hi Sander, On Wed, 2010-01-06 at 08:52 +0100, Sander wrote: I don't have your original mail, but I think I remember you mentioned a lot of bad sectors on that disk reported by SMART. If that is indeed the case it might be dificult for the people who might be able to help you, to help you. Thanks for your response. You're correct about the bad sector warning. So please correct me if I have some mistaken assumptions. I thought btrfs would be tolerant of that -- if a block failed the checksum test, it would reconstruct and remap it. Only if enough redundancy is left. And with the default setup btrfs is only mirroring the metadata not the data. (Also, I assumed that if a drive hadn't filled its bad sector remapping table, it could handle it at the hardware level, and SMART's warning was just that -- a warning, not a dire pronouncement of utter unsuitability -- but that's something else.) Bad sectors are only remapped by the drive on write time. As long as this isn't the case, they are only marked as pending. As you have written, that SMART detected many bad blocks, I suspect the FS is really damaged. And as btrfsck is limited, I don't think it can fix this. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] check for NULL pointer dereference in btrfs_set_acl
Check for for NULL pointer in btrfs_set_acl and omit calling
posix_acl_equiv_mode in this case to avoid NULL pointer dereference there.
Signed-off-by: Johannes Hirte johannes.hi...@fem.tu-ilmenau.de
diff --git a/fs/btrfs/acl.c b/fs/btrfs/acl.c
index 2e9e699..3a3a96d 100644
--- a/fs/btrfs/acl.c
+++ b/fs/btrfs/acl.c
@@ -111,13 +111,15 @@ static int btrfs_set_acl(struct btrfs_trans_handle
*trans,
switch (type) {
case ACL_TYPE_ACCESS:
- mode = inode-i_mode;
- ret = posix_acl_equiv_mode(acl, mode);
- if (ret 0)
- return ret;
- ret = 0;
- inode-i_mode = mode;
name = POSIX_ACL_XATTR_ACCESS;
+ if (acl) {
+ mode = inode-i_mode;
+ ret = posix_acl_equiv_mode(acl, mode);
+ if (ret 0)
+ return ret;
+ ret = 0;
+ inode-i_mode = mode;
+ }
break;
case ACL_TYPE_DEFAULT:
if (!S_ISDIR(inode-i_mode))
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [patch 0/2] grub-0.97: btrfs support
Am Freitag 11 Dezember 2009 16:27:54 schrieb Edward Shishkin: Johannes Hirte wrote: Am Freitag 11 Dezember 2009 12:17:29 schrieb Edward Shishkin: Johannes Hirte wrote: Am Freitag 11 Dezember 2009 00:15:46 schrieb Johannes Hirte: Am Freitag 25 September 2009 00:06:23 schrieb Edward Shishkin: Hello everyone. ... The following patches are for Fedora 10(**). The distro-independent package will be put to kernel.org a bit later. All comments, bugreports, etc. are welcome as usual. Ok, I have another comment/bugreport *g*. I'm testing this patch with gentoo, so the grub sources are not identicaly the same. With this patches applied, grub is unable to detect JFS or XFS filesystems. XFS is reported as unknown, JFS is reported as btrfs. Reiserfs and ext2/3 are detected as expected. Yes, this patch is for Fedora. For other distros some issues are possible, so please be careful.. I've also tested now with the fedora sources. There is the same bug. The btrfs patch breaks the filesystem detection. All filesystems after btrfs in fsys_table aren't detected. Moving btrfs to the end of fsys_table is a workaround but will interfere with FFS. So this should better be fixed in the btrfs-part of grub, so that it: a) doesn't missdetect a JFS filesystem as btrfs b) doesn't break the detection for remaining filesystems in the array. Hello. Yes, I confirm that xfs, etc. file systems are not detected, but missdetection jfs as btrfs looks rather fantastic :) Please, try the attached patch. Report if any problems. The patch works, but the problem with misdetected JFS filesystem still persists. It happens if the device contained a btrfs filesystem before. I assume that the JFS super block starts later on the device as the btrfs one do and jfs_mkfs doesn't clean the space ahead of the JFS super block. So if a JFS filesystem is created on a device that contained a btrfs before, btrfs_mount still detects the beginning of the old btrfs super block and reads crap later on. To avoid this, btrfs detection could be placed after JFS. Are there any objections against this? regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: (Little) Patch about null dereference with acl and posix.
Am Mittwoch 18 November 2009 22:28:27 schrieb briaeros007:
Hello,
For some days, i've got oops on my system and i've investigate it a bit.
The trouble was with posix_acl_equiv_mode , and for some reason
(corrupted metadata ?) btrfs sometimes call it with acl==NULL
This function doesn't like it.
So in my patch I've first put a little error protection around the
call, and then avoid to call btrfs_set_acl with acl=NULL.
I'm not sure if it's ok with best practice, but i've done the test
which produce the oops, and know it doesn't (but some csum failed.
Well if my btrfs is corrupted, it's comprehensible).
The patch is the following.
diff --git a/fs/btrfs/acl.c b/fs/btrfs/acl.c
index 3616042..f8ade24 100644
--- a/fs/btrfs/acl.c
+++ b/fs/btrfs/acl.c
@@ -111,7 +111,8 @@ static int btrfs_set_acl(struct inode *inode,
struct posix_acl *acl, int type)
switch (type) {
case ACL_TYPE_ACCESS:
mode = inode-i_mode;
- ret = posix_acl_equiv_mode(acl, mode);
+ if (acl mode)
+ ret = posix_acl_equiv_mode(acl, mode);
if (ret 0)
return ret;
ret = 0;
@@ -165,12 +166,13 @@ static int btrfs_xattr_set_acl(struct inode
*inode, int type,
} else if (IS_ERR(acl)) {
return PTR_ERR(acl);
}
+ else
+ {
+ ret = btrfs_set_acl(inode, acl, type);
+ posix_acl_release(acl);
+ }
}
- ret = btrfs_set_acl(inode, acl, type);
-
- posix_acl_release(acl);
-
return ret;
}
Shouldn't this go upstream and into stable review?
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [patch 0/2] grub-0.97: btrfs support
Am Freitag 25 September 2009 00:06:23 schrieb Edward Shishkin: Hello everyone. ... The following patches are for Fedora 10(**). The distro-independent package will be put to kernel.org a bit later. All comments, bugreports, etc. are welcome as usual. Ok, I have another comment/bugreport *g*. I'm testing this patch with gentoo, so the grub sources are not identicaly the same. With this patches applied, grub is unable to detect JFS or XFS filesystems. XFS is reported as unknown, JFS is reported as btrfs. Reiserfs and ext2/3 are detected as expected. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [patch 2/2] grub-0.97: btrfs multidevice configuration support
Am Dienstag 03 November 2009 01:59:39 schrieb Edward Shishkin: Johannes Hirte wrote: Why is the btrfs code dealing with network devices at all? Why not? :) I don't see the possiblity to get a btrfs filesystem this way. So as far as I understand this, it's complete useless. The CD support doesn't look very usefull too to me. It's possible to put a btrfs filesystem on a CD or DVD. But that seems rather theoretical. regards, Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [patch 0/2] grub-0.97: btrfs support
Am Freitag 11 Dezember 2009 00:15:46 schrieb Johannes Hirte:
Am Freitag 25 September 2009 00:06:23 schrieb Edward Shishkin:
Hello everyone.
...
The following patches are for Fedora 10(**).
The distro-independent package will be put to kernel.org a bit later.
All comments, bugreports, etc. are welcome as usual.
Ok, I have another comment/bugreport *g*.
I'm testing this patch with gentoo, so the grub sources are not identicaly
the same. With this patches applied, grub is unable to detect JFS or XFS
filesystems. XFS is reported as unknown, JFS is reported as btrfs.
Reiserfs and ext2/3 are detected as expected.
A possible solution is to put FSYS_BTRFS on the end of struct fsys_entry
fsys_table. I've tested with FSYS_BTFS as the second last entry, the last is
still FFS.
diff -Nru grub-0.97-r9/stage2/disk_io.c grub-0.97-r10/stage2/disk_io.c
--- grub-0.97-r9/stage2/disk_io.c 2009-12-10 23:41:37.0 +0100
+++ grub-0.97-r10/stage2/disk_io.c 2009-12-11 00:50:51.555007247 +0100
@@ -79,6 +79,9 @@
# ifdef FSYS_ISO9660
{iso9660, iso9660_mount, iso9660_read, iso9660_dir, 0, 0},
# endif
+# ifdef FSYS_BTRFS
+ {btrfs, btrfs_mount, btrfs_read, btrfs_dir, 0, btrfs_embed},
+# endif
/* XX FFS should come last as it's superblock is commonly crossing tracks
on floppies from track 1 to 2, while others only use 1. */
# ifdef FSYS_FFS
With this order, XFS and JFS filesystems are identified correct. But I think,
this is just a workaround.
regards,
Johannes
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: (Little) Patch about null dereference with acl and posix.
Am Mittwoch 18 November 2009 22:28:27 schrieb briaeros007:
Hello,
For some days, i've got oops on my system and i've investigate it a bit.
The trouble was with posix_acl_equiv_mode , and for some reason
(corrupted metadata ?) btrfs sometimes call it with acl==NULL
This function doesn't like it.
So in my patch I've first put a little error protection around the
call, and then avoid to call btrfs_set_acl with acl=NULL.
I'm not sure if it's ok with best practice, but i've done the test
which produce the oops, and know it doesn't (but some csum failed.
Well if my btrfs is corrupted, it's comprehensible).
The patch is the following.
diff --git a/fs/btrfs/acl.c b/fs/btrfs/acl.c
index 3616042..f8ade24 100644
--- a/fs/btrfs/acl.c
+++ b/fs/btrfs/acl.c
@@ -111,7 +111,8 @@ static int btrfs_set_acl(struct inode *inode,
struct posix_acl *acl, int type)
switch (type) {
case ACL_TYPE_ACCESS:
mode = inode-i_mode;
- ret = posix_acl_equiv_mode(acl, mode);
+ if (acl mode)
+ ret = posix_acl_equiv_mode(acl, mode);
if (ret 0)
return ret;
ret = 0;
@@ -165,12 +166,13 @@ static int btrfs_xattr_set_acl(struct inode
*inode, int type,
} else if (IS_ERR(acl)) {
return PTR_ERR(acl);
}
+ else
+ {
+ ret = btrfs_set_acl(inode, acl, type);
+ posix_acl_release(acl);
+ }
}
- ret = btrfs_set_acl(inode, acl, type);
-
- posix_acl_release(acl);
-
return ret;
}
Thanx for this fix. I think I run into the same bug with rdiff-backup:
Dec 1 19:09:11 datengrab kernel: BUG: unable to handle kernel NULL pointer
dereference at 0004
Dec 1 19:09:11 datengrab kernel: IP: [810c6c12]
posix_acl_equiv_mode+0x0/0x90
Dec 1 19:09:11 datengrab kernel: PGD 3609f067 PUD 7163067 PMD 0
Dec 1 19:09:11 datengrab kernel: Oops: [#1] SMP
Dec 1 19:09:11 datengrab kernel: last sysfs file:
/sys/devices/pci:00/:00:18.3/temp1_input
Dec 1 19:09:11 datengrab kernel: CPU 0
Dec 1 19:09:11 datengrab kernel: Modules linked in: usb_storage snd_seq_midi
snd_emu10k1_synth snd_emux_synth snd_seq_virmidi snd_seq_midi_emul snd_seq_oss
snd_seq_midi_event snd_seq snd_pcm_oss snd_mixer_oss aes_x86_64 aes_generic
xts gf128mul dm_crypt snd_emu10k1 snd_rawmidi snd_ac97_codec ac97_bus snd_pcm
snd_seq_device snd_timer snd_page_alloc snd_util_mem snd_hwdep sr_mod snd
fglrx(P) k8temp sata_sil hwmon i2c_amd756 i2c_amd8111 ohci_hcd sg uhci_hcd
Dec 1 19:09:11 datengrab kernel: Pid: 22720, comm: rdiff-backup Tainted: P
2.6.31.6-fglrx2 #2 To Be Filled By O.E.M.
Dec 1 19:09:11 datengrab kernel: RIP: 0010:[810c6c12]
[810c6c12]
posix_acl_equiv_mode+0x0/0x90
Dec 1 19:09:11 datengrab kernel: RSP: 0018:88001e051d60 EFLAGS: 00010246
Dec 1 19:09:11 datengrab kernel: RAX: 41c0 RBX:
RCX:
Dec 1 19:09:11 datengrab kernel: RDX: 8000 RSI: 88001e051d84
RDI:
Dec 1 19:09:11 datengrab kernel: RBP: 880056b315e8 R08: 880056b315e8
R09:
880056b315e8
Dec 1 19:09:11 datengrab kernel: R10: 88001e051e38 R11: 4000
R12:
Dec 1 19:09:11 datengrab kernel: R13: 8000 R14:
R15:
Dec 1 19:09:11 datengrab kernel: FS: 7f6b1e98d700()
GS:8800015e() knlGS:
Dec 1 19:09:11 datengrab kernel: CS: 0010 DS: ES: CR0:
80050033
Dec 1 19:09:11 datengrab kernel: CR2: 0004 CR3: 4c641000
CR4: 06f0
Dec 1 19:09:11 datengrab kernel: DR0: DR1:
DR2:
Dec 1 19:09:11 datengrab kernel: DR3: DR6: 0ff0
DR7: 0400
Dec 1 19:09:11 datengrab kernel: Process rdiff-backup (pid: 22720, threadinfo
88001e05, task 88005b0f3980)
Dec 1 19:09:11 datengrab kernel: Stack:
Dec 1 19:09:11 datengrab kernel: 81171176 880056b315e8
8109d127
880013e8b033
Dec 1 19:09:11 datengrab kernel: 0 41c01e051e98 880056b315e8
880056b315e8 8000
Dec 1 19:09:11 datengrab kernel: 0 880056b316a0
Dec 1 19:09:11 datengrab kernel: Call Trace:
Dec 1 19:09:11 datengrab kernel: [81171176] ?
btrfs_set_acl+0x5a/0x1ab
Dec 1 19:09:11 datengrab kernel: [8109d127] ? dput+0x2c/0x145
Dec 1 19:09:11 datengrab kernel: [81171307] ?
btrfs_xattr_set_acl+0x40/0x68
Dec 1 19:09:11 datengrab kernel: [8115db45] ?
btrfs_removexattr+0x20/0x60
Dec 1 19:09:11 datengrab kernel: [810a448b] ?
vfs_removexattr+0x78/0x104
Dec 1 19:09:11 datengrab kernel:
Re: [patch 2/2] grub-0.97: btrfs multidevice configuration support
Am Freitag 25 September 2009 00:06:40 schrieb Edward Shishkin:
Hi Edward,
I was pointed to a problem with this patch.
+static u64 scan_grub_devices(struct btrfs_device *dev,
+int (*discerner)(struct btrfs_device **, int),
+int lookup)
+{
...
+#ifdef SUPPORT_NETBOOT
+ errnum = ERR_NONE;
+ if (network_ready
+ !get_diskinfo(NETWORK_DRIVE, geom)) {
+ dev-drive = NETWORK_DRIVE;
+ dev-part = 0;
+ dev-length = geom.total_sectors;
+ if (discerner(dev, lookup)) {
+ count++;
+ if (lookup)
+ goto exit;
+ }
+ }
+#endif /* SUPPORT_NETBOOT */
+ exit:
+ return count;
+}
This won't compile since network_ready is undeclared. Why is the btrfs code
dealing with network devices at all?
regards
Johannes
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [patch 1/2] grub-0.97: btrfs support for a singe device configuration
Am Donnerstag 29 Oktober 2009 00:02:08 schrieb Edward Shishkin: Johannes Hirte wrote: [...] This compiles and works, but only with stage2. When using stage1_5, grub hangs on boot with GRUB loading stage1.5 and blinking cursor. Is the multidevice patch also necessary for single devices? Yes, please use both patches! I have split the stuff because the mailing list refuse attachments 100K. Thanks, Edward. With both patches applied it works now with stage1_5. Thank you. The patch for gentoo is available on http://bugs.gentoo.org/show_bug.cgi?id=283637, if somebody is interested. Johannes -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [patch 1/2] grub-0.97: btrfs support for a singe device configuration
Am Mittwoch 28 Oktober 2009 00:01:39 schrieb Edward Shishkin: Johannes Hirte wrote: Am Freitag 25 September 2009 00:06:32 schrieb Edward Shishkin: I was trying the patch and got a little confused. How did you get this work without linking against libgcc? Hmm.. Actually my Fedora stuff does link it: [...] +btrfs_stage1_5_exec_LDADD = @LIBGCC@ [...] I guess you have missed it when porting to Gentoo.. Ok, I've got it. I had to add -lgcc also for some more targets in Makefile.am: if NETBOOT_SUPPORT -pre_stage2_exec_LDADD = ../netboot/libdrivers.a +pre_stage2_exec_LDADD = ../netboot/libdrivers.a -lgcc +else +pre_stage2_exec_LDADD = -lgcc endif if DISKLESS_SUPPORT @@ -197,6 +201,16 @@ -DNO_BLOCK_FILES=1 reiserfs_stage1_5_exec_LDFLAGS = $(STAGE1_5_LINK) +# For btrfs_stage1_5 target. +btrfs_stage1_5_exec_SOURCES = start.S asm.S common.c char_io.c \ + disk_io.c stage1_5.c fsys_btrfs.c bios.c +btrfs_stage1_5_exec_CFLAGS = $(STAGE1_5_COMPILE) -DFSYS_BTRFS=1 \ + -DNO_BLOCK_FILES=1 +btrfs_stage1_5_exec_CCASFLAGS = $(STAGE1_5_COMPILE) -DFSYS_BTRFS=1 \ + -DNO_BLOCK_FILES=1 +btrfs_stage1_5_exec_LDFLAGS = $(STAGE1_5_LINK) +btrfs_stage1_5_exec_LDADD = -lgcc + # For vstafs_stage1_5 target. vstafs_stage1_5_exec_SOURCES = start.S asm.S common.c char_io.c \ disk_io.c stage1_5.c fsys_vstafs.c bios.c @@ -240,7 +254,7 @@ diskless_exec_CCASFLAGS = $(STAGE2_COMPILE) $(FSYS_CFLAGS) \ -DSUPPORT_DISKLESS=1 diskless_exec_LDFLAGS = $(PRE_STAGE2_LINK) -diskless_exec_LDADD = ../netboot/libdrivers.a +diskless_exec_LDADD = ../netboot/libdrivers.a -lgcc diskless_size.h: diskless -rm -f $@ This compiles and works, but only with stage2. When using stage1_5, grub hangs on boot with GRUB loading stage1.5 and blinking cursor. Is the multidevice patch also necessary for single devices? I've omitted this patch, since my root-fs (with /boot) lies on a single partition. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [patch 1/2] grub-0.97: btrfs support for a singe device configuration
Am Freitag 25 September 2009 00:06:32 schrieb Edward Shishkin: I was trying the patch and got a little confused. How did you get this work without linking against libgcc? I've tested it with the gentoo patches for grub and get /usr/src/portage/portage/sys-boot/grub-0.97-r10/work/grub-0.97/stage2/fsys_btrfs.c:551: undefined reference to `__udivdi3' /usr/src/portage/portage/sys-boot/grub-0.97-r10/work/grub-0.97/stage2/fsys_btrfs.c:571: undefined reference to `__umoddi3' /usr/src/portage/portage/sys-boot/grub-0.97-r10/work/grub-0.97/stage2/fsys_btrfs.c:575: undefined reference to `__umoddi3' /usr/src/portage/portage/sys-boot/grub-0.97-r10/work/grub-0.97/stage2/fsys_btrfs.c:581: undefined reference to `__umoddi3' /usr/src/portage/portage/sys-boot/grub-0.97-r10/work/grub-0.97/stage2/fsys_btrfs.c:583: undefined reference to `__udivdi3' /usr/src/portage/portage/sys-boot/grub-0.97-r10/work/grub-0.97/stage2/fsys_btrfs.c:588: undefined reference to `__umoddi3' /usr/src/portage/portage/sys-boot/grub-0.97-r10/work/grub-0.97/stage2/fsys_btrfs.c:589: undefined reference to `__udivdi3' collect2: ld returned 1 exit status make[2]: *** [pre_stage2.exec] Error 1 make[2]: Leaving directory `/usr/src/portage/portage/sys-boot/grub-0.97-r10/work/grub-0.97/stage2' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/usr/src/portage/portage/sys-boot/grub-0.97-r10/work/grub-0.97' make: *** [all] Error 2 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [patch 1/2] grub-0.97: btrfs support for a singe device configuration
Am Mittwoch 28 Oktober 2009 00:01:39 schrieb Edward Shishkin: Johannes Hirte wrote: Am Freitag 25 September 2009 00:06:32 schrieb Edward Shishkin: I was trying the patch and got a little confused. How did you get this work without linking against libgcc? Hmm.. Actually my Fedora stuff does link it: [...] +btrfs_stage1_5_exec_LDADD = @LIBGCC@ [...] I guess you have missed it when porting to Gentoo.. btrfs_stage1_5_exec_LDADD = @LIBGCC@ is in place, I've didn't missed. My fault, the error log was too short. As far as I can see the error occours at pre_stage2_exec-stage2: mv -f .deps/pre_stage2_exec-stage2.Tpo .deps/pre_stage2_exec-stage2.Po mv -f .deps/pre_stage2_exec-graphics.Tpo .deps/pre_stage2_exec-graphics.Po mv -f .deps/pre_stage2_exec-tparm.Tpo .deps/pre_stage2_exec-tparm.Po x86_64-pc-linux-gnu-gcc -m32 -Os -fno-stack-protector -fno-builtin -nostdinc -DSUPPORT_SERIAL=1 -DSUPPORT_HERCULES=1 -DSUPPORT_GRAPHICS=1 -DFSYS_EXT2FS=1 -DFSYS_FAT=1 -DFSYS_FFS=1 -DFSYS_UFS2=1 -DFSYS_MINIX=1 -DFSYS_REISERFS=1 -DFSYS_BTRFS=1 -DFSYS_VSTAFS=1 -DFSYS_JFS=1 -DFSYS_XFS=1 -DFSYS_ISO9660=1 -DUSE_MD5_PASSWORDS=1 -g -nostdlib -Wl,-N -Wl,-Ttext -Wl,8200 -Wl,-O1 -Wl,--sort-common -Wl,-z,combreloc -Wl,--build-id=none -o pre_stage2.exec pre_stage2_exec-asm.o pre_stage2_exec-bios.o pre_stage2_exec-boot.o pre_stage2_exec-builtins.o pre_stage2_exec-char_io.o pre_stage2_exec-cmdline.o pre_stage2_exec-common.o pre_stage2_exec-console.o pre_stage2_exec-disk_io.o pre_stage2_exec-fsys_ext2fs.o pre_stage2_exec-fsys_fat.o pre_stage2_exec-fsys_ffs.o pre_stage2_exec-fsys_iso9660.o pre_stage2_exec-fsys_jfs.o pre_stage2_exec-fsys_minix.o pre_stage2_exec-fsys_reiserfs.o pre_stage2_exec-fsys_btrfs.o pre_stage2_exec-fsys_ufs2.o pre_stage2_exec-fsys_vstafs.o pre_stage2_exec-fsys_xfs.o pre_stage2_exec-gunzip.o pre_stage2_exec-hercules.o pre_stage2_exec-md5.o pre_stage2_exec-serial.o pre_stage2_exec-smp-imps.o pre_stage2_exec-stage2.o pre_stage2_exec-terminfo.o pre_stage2_exec-tparm.o pre_stage2_exec-graphics.o pre_stage2_exec-fsys_btrfs.o: In function `__btrfs_map_block': /usr/src/portage/portage/sys-boot/grub-0.97-r10/work/grub-0.97/stage2/fsys_btrfs.c:551: undefined reference to `__udivdi3' /usr/src/portage/portage/sys-boot/grub-0.97-r10/work/grub-0.97/stage2/fsys_btrfs.c:571: undefined reference to `__umoddi3' /usr/src/portage/portage/sys-boot/grub-0.97-r10/work/grub-0.97/stage2/fsys_btrfs.c:575: undefined reference to `__umoddi3' /usr/src/portage/portage/sys-boot/grub-0.97-r10/work/grub-0.97/stage2/fsys_btrfs.c:581: undefined reference to `__umoddi3' /usr/src/portage/portage/sys-boot/grub-0.97-r10/work/grub-0.97/stage2/fsys_btrfs.c:583: undefined reference to `__udivdi3' /usr/src/portage/portage/sys-boot/grub-0.97-r10/work/grub-0.97/stage2/fsys_btrfs.c:588: undefined reference to `__umoddi3' /usr/src/portage/portage/sys-boot/grub-0.97-r10/work/grub-0.97/stage2/fsys_btrfs.c:589: undefined reference to `__udivdi3' collect2: ld returned 1 exit status make[2]: *** [pre_stage2.exec] Error 1 make[2]: Leaving directory `/usr/src/portage/portage/sys-boot/grub-0.97-r10/work/grub-0.97/stage2' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/usr/src/portage/portage/sys-boot/grub-0.97-r10/work/grub-0.97' make: *** [all] Error 2 I'm not very familiar with autotools, but I think btrfs_stage1_5_exec_LDADD isn't the right place to catch this. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
crash with linux-2.6.31.1
I got the following crash today: Oct 2 10:30:04 datengrab kernel: [ cut here ] Oct 2 10:30:04 datengrab kernel: WARNING: at fs/btrfs/free-space-cache.c:90 tree_insert_offset+0x74/0xbb [btrfs]() Oct 2 10:30:04 datengrab kernel: Hardware name: To Be Filled By O.E.M. Oct 2 10:30:04 datengrab kernel: Modules linked in: snd_seq_midi snd_emu10k1_synth snd_emux_synth snd_seq_virmidi snd_seq_midi_emul snd_seq_oss snd_seq_midi_event snd_seq snd_pcm_oss snd_mixer_oss btrfs zlib_deflate crc32c libcrc32c aes_x86_64 aes_generic xts gf128mul dm_crypt snd_emu10k1 snd_rawmidi snd_ac97_codec ac97_bus snd_pcm snd_seq_device snd_timer snd_page_alloc snd_util_mem snd_hwdep snd sg sr_mod ohci_hcd ehci_hcd uhci_hcd fglrx(P) Oct 2 10:30:04 datengrab kernel: Pid: 258, comm: pdflush Tainted: P 2.6.31.1 #1 Oct 2 10:30:04 datengrab kernel: Call Trace: Oct 2 10:30:04 datengrab kernel: [8102b511] ? warn_slowpath_common+0x76/0x8c Oct 2 10:30:04 datengrab kernel: [a0369c42] ? tree_insert_offset+0x74/0xbb [btrfs] Oct 2 10:30:04 datengrab kernel: [a036a0c2] ? link_free_space+0x3b/0x51 [btrfs] Oct 2 10:30:04 datengrab kernel: [a036a3c4] ? btrfs_find_space_for_alloc+0x142/0x157 [btrfs] Oct 2 10:30:04 datengrab kernel: [a0335692] ? find_free_extent+0x48a/0x705 [btrfs] Oct 2 10:30:04 datengrab kernel: [81183071] ? radix_tree_gang_lookup_slot+0x6b/0x8f Oct 2 10:30:04 datengrab kernel: [a0335d27] ? __btrfs_reserve_extent+0x114/0x283 [btrfs] Oct 2 10:30:04 datengrab kernel: [a0335ec3] ? btrfs_reserve_extent+0x2d/0x54 [btrfs] Oct 2 10:30:04 datengrab kernel: [a0346b99] ? cow_file_range+0x1a7/0x302 [btrfs] Oct 2 10:30:04 datengrab kernel: [a0347359] ? run_delalloc_range+0x9e/0x2ed [btrfs] Oct 2 10:30:04 datengrab kernel: [a035c0b9] ? find_lock_delalloc_range+0x123/0x178 [btrfs] Oct 2 10:30:04 datengrab kernel: [a035c756] ? __extent_writepage+0x1d8/0x716 [btrfs] Oct 2 10:30:04 datengrab kernel: [a035ae38] ? end_bio_extent_writepage+0x0/0x1a2 [btrfs] Oct 2 10:30:04 datengrab kernel: [8118320d] ? radix_tree_gang_lookup_tag_slot+0x7a/0x9f Oct 2 10:30:04 datengrab kernel: [810525cd] ? find_get_pages_tag+0x36/0xbe Oct 2 10:30:04 datengrab kernel: [a035cff4] ? T.607+0x13f/0x287 [btrfs] Oct 2 10:30:04 datengrab kernel: [a034523d] ? btrfs_submit_bio_hook+0xd2/0x103 [btrfs] Oct 2 10:30:04 datengrab kernel: [a0345285] ? __btrfs_submit_bio_start+0x0/0x24 [btrfs] Oct 2 10:30:04 datengrab kernel: [a034526e] ? __btrfs_submit_bio_done+0x0/0x17 [btrfs] Oct 2 10:30:04 datengrab kernel: [a035d237] ? extent_writepages+0x41/0x5a [btrfs] Oct 2 10:30:04 datengrab kernel: [a0345cfd] ? btrfs_get_extent+0x0/0x70b [btrfs] Oct 2 10:30:04 datengrab kernel: [81057ecc] ? do_writepages+0x20/0x2d Oct 2 10:30:04 datengrab kernel: [8108e61f] ? writeback_single_inode+0x14c/0x325 Oct 2 10:30:04 datengrab kernel: [8108ebde] ? generic_sync_sb_inodes+0x286/0x3e7 Oct 2 10:30:04 datengrab kernel: [81026d49] ? update_curr+0x4e/0xc2 Oct 2 10:30:04 datengrab kernel: [8108ee26] ? writeback_inodes+0x6d/0xa5 Oct 2 10:30:04 datengrab kernel: [8105897b] ? pdflush+0x0/0x1c5 Oct 2 10:30:04 datengrab kernel: [81058001] ? wb_kupdate+0x94/0x108 Oct 2 10:30:04 datengrab kernel: [8105897b] ? pdflush+0x0/0x1c5 Oct 2 10:30:04 datengrab kernel: [81058a89] ? pdflush+0x10e/0x1c5 Oct 2 10:30:04 datengrab kernel: [81057f6d] ? wb_kupdate+0x0/0x108 Oct 2 10:30:04 datengrab kernel: [8103a7e9] ? kthread+0x81/0x89 Oct 2 10:30:04 datengrab kernel: [8100ba0a] ? child_rip+0xa/0x20 Oct 2 10:30:04 datengrab kernel: [8105897b] ? pdflush+0x0/0x1c5 Oct 2 10:30:04 datengrab kernel: [8103a768] ? kthread+0x0/0x89 Oct 2 10:30:04 datengrab kernel: [8100ba00] ? child_rip+0x0/0x20 Oct 2 10:30:04 datengrab kernel: ---[ end trace ef71307e17570618 ]--- Oct 2 10:30:04 datengrab kernel: [ cut here ] Oct 2 10:30:04 datengrab kernel: WARNING: at fs/btrfs/free-space-cache.c:90 tree_insert_offset+0x74/0xbb [btrfs]() Oct 2 10:30:04 datengrab kernel: Hardware name: To Be Filled By O.E.M. Oct 2 10:30:04 datengrab kernel: Modules linked in: snd_seq_midi snd_emu10k1_synth snd_emux_synth snd_seq_virmidi snd_seq_midi_emul snd_seq_oss snd_seq_midi_event snd_seq snd_pcm_oss snd_mixer_oss btrfs zlib_deflate crc32c libcrc32c aes_x86_64 aes_generic xts gf128mul dm_crypt snd_emu10k1 snd_rawmidi snd_ac97_codec ac97_bus snd_pcm snd_seq_device snd_timer snd_page_alloc snd_util_mem snd_hwdep snd sg sr_mod ohci_hcd ehci_hcd uhci_hcd fglrx(P) Oct 2 10:30:04 datengrab kernel: Pid: 258, comm: pdflush Tainted: PW 2.6.31.1 #1 Oct 2 10:30:04 datengrab kernel: Call Trace: Oct 2 10:30:04 datengrab kernel: [8102b511] ?
Oops with enospc-branch
With the enospc-branch my system oops with some workload. Netconsole gave me this trace: nc6: using datagram socket [ cut here ] kernel BUG at fs/btrfs/extent-tree.c:2829! invalid opcode: [#1] last sysfs file: /sys/devices/pci:00/:00:06.0/:02:05.0/host1/target1:0:0/1:0:0:0/block/sdb/size CPU 0 Modules linked in: netconsole configfs snd_seq_midi snd_emu10k1_synth snd_emux_synth snd_seq_virmidi snd_seq_midi_emul snd_seq_oss snd_seq_midi_event snd_seq snd_pcm_oss snd_mixer_oss btrfs zlib_deflate crc32c libcrc32c aes_x86_64 aes_generic xts gf128mul dm_crypt snd_emu10k1 snd_rawmidi snd_ac97_codec ac97_bus snd_pcm snd_seq_device snd_timer snd_page_alloc snd_util_mem snd_hwdep sg snd sr_mod ehci_hcd ohci_hcd uhci_hcd Pid: 2880, comm: ebuild.sh Not tainted 2.6.30-btrfs #3 To Be Filled By O.E.M. RIP: 0010:[a011f548] [a011f548] btrfs_unreserve_metadata_space+0x75/0x89 [btrfs] RSP: 0018:880044c67c88 EFLAGS: 00010287 RAX: RBX: 88005e5e RCX: 1000 RDX: 88005e87b900 RSI: 88005db93d48 RDI: 88005e5e RBP: 88005db92000 R08: R09: 002d R10: 0246 R11: 0d28 R12: 00087000 R13: 88004b4903b8 R14: R15: 002d FS: 7ffcbe66e6f0() GS:8060c000() knlGS: CS: 0010 DS: ES: CR0: 80050033 CR2: 004b52d8 CR3: 44c6e000 CR4: 06e0 DR0: DR1: DR2: DR3: DR6: 0ff0 DR7: 0400 Process ebuild.sh (pid: 2880, threadinfo 880044c66000, task 88004edd2d00) Stack: 88005529e618 88004b490548 88005e5e a01300ec a0144cd8 88005529e618 88004b4903e0 0021 a014639f 1000 Call Trace: [a01300ec] ? btrfs_clear_bit_hook+0xb4/0x185 [btrfs] [a0144cd8] ? alloc_extent_state+0x12/0x5e [btrfs] [a014639f] ? clear_state_bit+0x7c/0x127 [btrfs] [a01466c5] ? clear_extent_bit+0x27b/0x2f4 [btrfs] [a0148900] ? clear_extent_bits+0x1c/0x21 [btrfs] [a0136e63] ? prepare_pages+0x1c5/0x234 [btrfs] [a0137480] ? btrfs_file_write+0x3c6/0x679 [btrfs] [80275ac3] ? vfs_write+0xa6/0x131 [80275c04] ? sys_write+0x45/0x6e [8020ac28] ? system_call_fastpath+0x16/0x1b Code: 00 00 8b 8b 40 02 00 00 47 8d 24 e4 4d 63 e4 48 6b c0 0e 48 01 c8 4c 0f af e0 48 8b 42 40 4c 39 e0 73 0c 48 c7 42 40 00 00 00 00 0f 0b eb 0e 4c 29 e0 48 89 42 40 31 c0 5b 5d 41 5c c3 eb fe 55 RIP [a011f548] btrfs_unreserve_metadata_space+0x75/0x89 [btrfs] RSP 880044c67c88 ---[ end trace a0e09bb9134da1c7 ]--- [ cut here ] kernel BUG at fs/btrfs/extent-tree.c:2829! invalid opcode: [#2] last sysfs file: /sys/devices/pci:00/:00:06.0/:02:05.0/host1/target1:0:0/1:0:0:0/block/sdb/size CPU 0 Modules linked in: netconsole configfs snd_seq_midi snd_emu10k1_synth snd_emux_synth snd_seq_virmidi snd_seq_midi_emul snd_seq_oss snd_seq_midi_event snd_seq snd_pcm_oss snd_mixer_oss btrfs zlib_deflate crc32c libcrc32c aes_x86_64 aes_generic xts gf128mul dm_crypt snd_emu10k1 snd_rawmidi snd_ac97_codec ac97_bus snd_pcm snd_seq_device snd_timer snd_page_alloc snd_util_mem snd_hwdep sg snd sr_mod ehci_hcd ohci_hcd uhci_hcd Pid: 2529, comm: python2.6 Tainted: G D2.6.30-btrfs #3 To Be Filled By O.E.M. RIP: 0010:[a011f548] [a011f548] btrfs_unreserve_metadata_space+0x75/0x89 [btrfs] RSP: 0018:88005a57dc88 EFLAGS: 00010287 RAX: RBX: 88005e5e RCX: 1000 RDX: 88005e87b900 RSI: 88005db93d48 RDI: 88005e5e RBP: 88005db92000 R08: R09: 002d R10: 0246 R11: 88004b43b880 R12: 00087000 R13: 88004ea9aca0 R14: R15: 002d FS: 7f5d02dd56f0() GS:8060c000() knlGS: CS: 0010 DS: ES: CR0: 80050033 CR2: 7f5d02e06000 CR3: 5a5ed000 CR4: 06e0 DR0: DR1: DR2: DR3: DR6: 0ff0 DR7: 0400 Process python2.6 (pid: 2529, threadinfo 88005a57c000, task 88005a51) Stack: 88004b40b528 88004ea9ae30 88005e5e a01300ec 88004b40b528 88004ea9acc8 0021 a014639f Call Trace: [a01300ec] ? btrfs_clear_bit_hook+0xb4/0x185 [btrfs] [a014639f] ? clear_state_bit+0x7c/0x127 [btrfs] [a01466c5] ? clear_extent_bit+0x27b/0x2f4 [btrfs] [a0148900] ? clear_extent_bits+0x1c/0x21 [btrfs] [a0136e63] ? prepare_pages+0x1c5/0x234 [btrfs] [a0137480] ?
