Re: kernel BUG when removing missing drive (Take 2)
On Fri, Oct 29, 2010 at 11:55:49AM -0700, Erik Jensen wrote: So, I ended up just applying the relevant commit to my existing source tree, which did allow me to successfully remove the missing drive, so I seem to be back up and running. Thank you very much! Fantastic, thanks for letting us know. -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Can't remove missing drive
On Sat, Oct 30, 2010 at 06:37:06PM +1100, William Uther wrote: Hi, I have a raid1 setup with a missing device. I have added a new device and everything seems to be working fine, except I cannot remove the old, missing, device. There is no error - but the 'some devices missing' tag doesn't go away. r...@willvo:~# btrfs filesystem show failed to read /dev/sr0 Label: none uuid: f929c413-01c8-443f-b4f2-86f36702f519 Total devices 3 FS bytes used 578.39GB devid1 size 931.51GB used 604.00GB path /dev/sdb1 devid2 size 931.51GB used 604.00GB path /dev/sdc1 *** Some devices missing Btrfs Btrfs v0.19 r...@willvo:~# btrfs device delete missing /data r...@willvo:~# btrfs filesystem show failed to read /dev/sr0 Label: none uuid: f929c413-01c8-443f-b4f2-86f36702f519 Total devices 3 FS bytes used 578.39GB devid1 size 931.51GB used 604.00GB path /dev/sdb1 devid2 size 931.51GB used 604.00GB path /dev/sdc1 *** Some devices missing Btrfs Btrfs v0.19 There are a number of sub-volumes of /data that are mounted in other locations. I'm using kernel 2.6.36 (the lucid backport of the natty kernel) and similar btrfs-tools (lucid backport of natty tools). Interestingly looking at the output of `dh -h`, it appears that the 'missing' devices are no longer being counted in the filesystem size - there is just a phantom 'missing' tag in btrfs-show. Is this actually a problem, or can I just keep running as is? It seems to mount fine without -odegraded. Any ideas how I can list the missing devices? Any ideas on how I can remove the missing devices? What have you tried so far? The general formula is: mount -o degraded /dev/xxx /mnt (where xxx is one drive still in the array) btrfs-vol -r missing /mnt I'd suggest pulling the master branch of the unstable tree first, it has a fix for the btrfs-vol -r missing code. -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
btrfs and apt package manager in ubuntu
it's very slow in installtion with apt in ubuntu -- Abdullah Ansari ahe...@gmail.com -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Kernel 2.6.36 btrfs csum bugreport
Hi everybody, Today while playing around with btrfs I uncovered what must be a bug in the btrfs checksum code. My kernel log received a couple of these messages with various ino and off numbers: btrfs csum failed ino 5098 off 524288 csum 2981133980 private 959545494 [..] This happens on reading from the btrfs filesystem. The funny thing is that the files are read correct, as verified by md5sum. I have cross-checked this on another machine (with same kernel and btrfs utils): same result. A full filesystem md5sum check showed no errors. The md5sums obviously were computed before the data was copied to the btrfs. So I conclude that these messages are faulty because data is read correctly. In addition, when you have more than one btrfs you cannot see from the message which fs it is refering to. Here is my setup, maybe it has something to do with the (nowadays) unusual kernel target: - unmodified upstream 2.6.36 kernel - Debian Squeeze - Standard Debian gcc 4.3.5 with target i486 - CPU AMD Geode LX800 on ALIX board - btrfs on USB-ATA connected IDE drive Seagate Barracuda 7200.8 ST3400832A - btrs utils v0.19 - about 300GB of data of all sorts in 5+ files on the fs - data gets rsynced to another btrfs volume of 1TB when on read the csum errors occur Hope that some of this informations rings a bell on someones mind. If so, please let me know ;) bye, Andreas -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
database for device scan
Hallo, linux-btrfs, I've used for some days btrfs-progs-20100717, now I'm using btrfs- progs-201030 (built with the SlackBuild script for slackware). Works fine, but the actual version works in another way than the previous when I say mkfs.btrfs /dev/sdb1 (or how the device may be named). The previous version finished its work within some seconds, the actual version first tests all block devices in /dev. If I'd run udev that may be no big problem. But I prefer not using udev, I prefer working with static device entries, and that leads to many entries for devices which really don't exist. And scan scans all of them. Maybe it's a better idea not to use /dev as database, using proc/ partitions may be the better way. Viele Gruesse! Helmut -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
btrfs-convert fails
Hallo, linux-btrfs, I've tried to convert a 12 GByte ext2 partition (nearly full, 280 MByte free) with btrfs-convert. After about 15 minutes (700-MHz-CPU) the system tells ... creating ext2fs image file cleaning up system chunk btrfs-convert: extent-tree.c:2529: btrfs_reserve_extent: Assertion `!(ret)' failed Abgebrochen syslogd tells Oct 31 17:43:16 ElNath kernel: [ cut here ] Oct 31 17:43:17 ElNath kernel: kernel BUG at fs/btrfs/volumes.c:2831! Oct 31 17:43:17 ElNath kernel: invalid opcode: [#1] Oct 31 17:43:17 ElNath kernel: last sysfs file: /sys/devices/pci:00/:00:04.0/:08:02.0/:09:00.0/class Oct 31 17:43:17 ElNath kernel: Modules linked in: sg nf_nat_ftp nf_conntrack_ftp ipt_MASQUERADE iptable_nat nf_nat xt_DSCP xt_multiport xt_recent nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack xt_tcpudp ipt_REJECT iptable_filter iptable_mangle ip_tables xt_iprange x_tables nfsd exportfs ipv6 8139too 8139cp r8169 savagefb fb_ddc vgastate i2c_piix4 piix e100 mii intel_agp agpgart cmd64x video thermal_sys output ac battery yenta_socket pcmcia_rsrc pcmcia pcmcia_core thinkpad_acpi hwmon led_class nvram fuse Oct 31 17:43:17 ElNath kernel: Oct 31 17:43:17 ElNath kernel: Pid: 6396, comm: mount Not tainted 2.6.35.8-h1 #1 26478EG/26478EG Oct 31 17:43:17 ElNath kernel: EIP: 0060:[c1231d1a] EFLAGS: 00010206 CPU: 0 Oct 31 17:43:17 ElNath kernel: EIP is at btrfs_rmap_block+0x3ba/0x3f0 Oct 31 17:43:17 ElNath kernel: EAX: EBX: 3df0 ECX: EDX: Oct 31 17:43:17 ElNath kernel: ESI: 3df0 EDI: d7bf76c0 EBP: d1befca8 ESP: d1befc24 Oct 31 17:43:17 ElNath kernel: DS: 007b ES: 007b FS: GS: 00e0 SS: 0068 Oct 31 17:43:17 ElNath kernel: Process mount (pid: 6396, ti=d1bee000 task=c23767a0 task.ti=d1bee000) Oct 31 17:43:17 ElNath kernel: Stack: Oct 31 17:43:17 ElNath kernel: 0001 c10252f8 0004 03fa d1befc48 Oct 31 17:43:17 ElNath kernel: 0 c1025383 d1befc80 c1228f3d 1000 c1b90100 d75bd7e0 0003 d1befc78 Oct 31 17:43:17 ElNath kernel: 0 1000 0004 d75bdca8 d1befcb0 Oct 31 17:43:17 ElNath kernel: Call Trace: Oct 31 17:43:17 ElNath kernel: [c10252f8] ? kmap_atomic_prot+0x48/0xc0 Oct 31 17:43:17 ElNath kernel: [c1025383] ? kmap_atomic+0x13/0x20 Oct 31 17:43:17 ElNath kernel: [c1228f3d] ? map_private_extent_buffer+0x8d/0x120 Oct 31 17:43:17 ElNath kernel: [c1229091] ? map_extent_buffer+0xc1/0xd0 Oct 31 17:43:17 ElNath kernel: [c11f13ac] ? exclude_super_stripes+0xac/0x140 Oct 31 17:43:17 ElNath kernel: [c11fa78a] ? btrfs_read_block_groups+0x4aa/0x700 Oct 31 17:43:17 ElNath kernel: [c1205ae5] ? open_ctree+0xfe5/0x1430 Oct 31 17:43:17 ElNath kernel: [c12a0577] ? strlcpy+0x37/0x60 Oct 31 17:43:17 ElNath kernel: [c11e728a] ? btrfs_get_sb+0x5ca/0x790 Oct 31 17:43:17 ElNath kernel: [c10c70e1] ? dput+0x91/0x120 Oct 31 17:43:17 ElNath kernel: [c10bde6e] ? path_to_nameidata+0x1e/0x50 Oct 31 17:43:17 ElNath kernel: [c10cc23d] ? alloc_vfsmnt+0x6d/0x110 Oct 31 17:43:17 ElNath kernel: [c109a093] ? kstrdup+0x43/0x60 Oct 31 17:43:17 ElNath kernel: [c10b7942] ? vfs_kern_mount+0x72/0x1c0 Oct 31 17:43:17 ElNath kernel: [c10ca745] ? get_fs_type+0x35/0xc0 Oct 31 17:43:17 ElNath kernel: [c10b7aee] ? do_kern_mount+0x3e/0xe0 Oct 31 17:43:17 ElNath kernel: [c10cd2a8] ? do_mount+0x488/0x6d0 Oct 31 17:43:17 ElNath kernel: [c1099e72] ? strndup_user+0x62/0xa0 Oct 31 17:43:17 ElNath kernel: [c10cd57c] ? sys_mount+0x8c/0xb0 Oct 31 17:43:17 ElNath kernel: [c17495cc] ? syscall_call+0x7/0xb Oct 31 17:43:17 ElNath kernel: Code: 44 8b 55 dc 89 34 da 89 7c da 04 8b 4d d8 83 c3 01 89 5d ec 8b 49 18 89 4d bc e9 70 fd ff ff 89 d0 31 d2 f7 f7 89 d1 89 c6 eb 80 0f 0b eb fe 66 90 ba 2c 0b 00 00 b8 a6 a4 87 c1 e8 21 e5 df ff Oct 31 17:43:17 ElNath kernel: EIP: [c1231d1a] btrfs_rmap_block+0x3ba/0x3f0 SS:ESP 0068:d1befc24 Oct 31 17:43:17 ElNath kernel: ---[ end trace f68618b8246ff4aa ]--- What goes wrong? How much free place needs btrfs-convert? Viele Gruesse! Helmut -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Horrible btrfs performance due to fragmentation
On Mon, Oct 11, 2010 at 6:46 PM, Calvin Walton calvin.wal...@gmail.com wrote: On Mon, 2010-10-11 at 03:30 +0300, Felipe Contreras wrote: I use btrfs on most of my volumes on my laptop, and I've always felt booting was very slow, but definitely sure is slow, is starting up Google Chrome: encrypted ext4: ~20s btrfs: ~2:11s I have tried different things to find out exactly what is the issue, but haven't quite found it yet. If you've been using this volume for a while, it could just have become badly fragmented. You could try btrfs's fancy online defragmentation abilities to see if that'll give you an improvement: # btrfs filesystem defragment /mountpoint/of/volume Let us know if that helps, of course :) I finally managed to track down this issue. Indeed the fragmentation is horrible, and 'btrfs filesystem defragment' doesn't help: % cat History-old History % btrfs filesystem defragment /home % echo 3 /proc/sys/vm/drop_caches % time dd if=History of=/dev/null time dd if=History-old of=/dev/null 109664+0 records in 109664+0 records out 56147968 bytes (56 MB) copied, 1.90015 s, 29.5 MB/s dd if=History of=/dev/null 0.08s user 0.29s system 15% cpu 2.458 total 109664+0 records in 109664+0 records out 56147968 bytes (56 MB) copied, 97.772 s, 574 kB/s dd if=History-old of=/dev/null 0.07s user 0.80s system 0% cpu 1:37.79 total I think this is a serious issue that *must* be fixed for 1.0. I filed a bug for this: https://bugzilla.kernel.org/show_bug.cgi?id=21562 -- Felipe Contreras -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Kernel 2.6.36 btrfs csum bugreport
Today while playing around with btrfs I uncovered what must be a bug in the btrfs checksum code. My kernel log received a couple of these messages with various ino and off numbers: btrfs csum failed ino 5098 off 524288 csum 2981133980 private 959545494 [..] This happens on reading from the btrfs filesystem. The funny thing is that the files are read correct, as verified by md5sum. I have cross-checked this on another machine (with same kernel and btrfs utils): same result. A full filesystem md5sum check showed no errors. The md5sums obviously were computed before the data was copied to the btrfs. So I conclude that these messages are faulty because data is read correctly. In addition, when you have more than one btrfs you cannot see from the message which fs it is refering to. Is this a raid1 or a dup array? -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Horrible btrfs performance due to fragmentation
On Sun, Oct 31, 2010 at 1:58 PM, Felipe Contreras felipe.contre...@gmail.com wrote: On Mon, Oct 11, 2010 at 6:46 PM, Calvin Walton calvin.wal...@gmail.com wrote: On Mon, 2010-10-11 at 03:30 +0300, Felipe Contreras wrote: I use btrfs on most of my volumes on my laptop, and I've always felt booting was very slow, but definitely sure is slow, is starting up Google Chrome: encrypted ext4: ~20s btrfs: ~2:11s I have tried different things to find out exactly what is the issue, but haven't quite found it yet. If you've been using this volume for a while, it could just have become badly fragmented. You could try btrfs's fancy online defragmentation abilities to see if that'll give you an improvement: # btrfs filesystem defragment /mountpoint/of/volume Let us know if that helps, of course :) I finally managed to track down this issue. Indeed the fragmentation is horrible, and 'btrfs filesystem defragment' doesn't help: % cat History-old History % btrfs filesystem defragment /home % echo 3 /proc/sys/vm/drop_caches % time dd if=History of=/dev/null time dd if=History-old of=/dev/null 109664+0 records in 109664+0 records out 56147968 bytes (56 MB) copied, 1.90015 s, 29.5 MB/s dd if=History of=/dev/null 0.08s user 0.29s system 15% cpu 2.458 total 109664+0 records in 109664+0 records out 56147968 bytes (56 MB) copied, 97.772 s, 574 kB/s dd if=History-old of=/dev/null 0.07s user 0.80s system 0% cpu 1:37.79 total I think this is a serious issue that *must* be fixed for 1.0. I filed a bug for this: https://bugzilla.kernel.org/show_bug.cgi?id=21562 btrfs fi defrag isn't recursive. btrfs filesystem defrag /home will defragment the space used to store the folder, without touching the space used to store files in that folder. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Horrible btrfs performance due to fragmentation
On Mon, Nov 01, 2010 at 12:36:58AM +0200, Felipe Contreras wrote: On Mon, Nov 1, 2010 at 12:25 AM, cwillu cwi...@cwillu.com wrote: btrfs fi defrag isn't recursive. btrfs filesystem defrag /home will defragment the space used to store the folder, without touching the space used to store files in that folder. Yes, that came up on the IRC, but: 1) It doesn't make sense: btrfs filesystem doesn't allow a fileystem as argument? Why would anyone want it to be _non_ recursive? You missed the subsequent discussion on IRC about the interaction of COW with defrag. Essentially, if you've got two files that are COW copies of each other, and one has had something written to it since, it's *impossible* for both files to be defragmented, without making a full copy of both: Start with a file (A, etc are data blocks on the disk): file1 = ABCDEF Cow copy it: file1 = ABCDEF file2 = ABCDEF Now write to one of them: file1 = ABCDEF file2 = ABCDxF So, either file1 is contiguous, and file2 is fragmented (with the block x somewhere else on disk), or file2 is contiguous, and file1 is fragmented (with E somewhere else on disk). In fact, we've determined by experiment that when you defrag a file that's sharing blocks with another one, the file gets copied in its entirety, thus separating the blocks of the file and its COW duplicate. 2) The filesystem should not degrade performance so horribly no matter how long the it has been used. Even git has automatic garbage collection. Since, I believe, btrfs uses COW very heavily internally for ensuring consistency, you can end up with fragmenting files and directories very easily. You probably need some kind of scrubber that goes looking for non-COW files that are fragmented, and defrags them in the background. Hugo. -- === Hugo Mills: h...@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- No! My collection of rare, incurable diseases! Violated! --- signature.asc Description: Digital signature
Re: Can't remove missing drive
Thanks to Chris and Brian for the help! On 31/10/2010, at 11:01 PM, Chris Mason wrote: On Sat, Oct 30, 2010 at 06:37:06PM +1100, William Uther wrote: [snip - issues removing a missing drive - see below for new log] Is this actually a problem, or can I just keep running as is? It seems to mount fine without -odegraded. Any ideas how I can list the missing devices? Any ideas on how I can remove the missing devices? What have you tried so far? Well, to remove the missing drive I've tried `btrfs-vol -r missing /data` and newer `btrfs` command. I've previously tried with the system mounted in degraded mode. The wiki, https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices, suggests that you should mount the new disk before removing the missing disk. I've also tried removing the old device with `btrfs device delete /dev/loop0 /data` - i.e. giving the missing device explicitly. Also, the 'missing' device, /dev/loop0, is there - just not connected to anything. I thought that might be the issue so I moved it out of the way and tried to remove 'missing' again. No change. To list the missing devices I tried `btrfs filesystem show` - that shows 'some devices missing' but doesn't list them. Interestingly, the new log below shows that `btrfs device delete missing` shows that `btrfs delete` doesn't think there are any devices missing. The general formula is: mount -o degraded /dev/xxx /mnt (where xxx is one drive still in the array) btrfs-vol -r missing /mnt I'd suggest pulling the master branch of the unstable tree first, it has a fix for the btrfs-vol -r missing code. Ok. Is this kernel or tools or both? I'll assume both. I probably wont be able to get to that for a few days. On 31/10/2010, at 4:55 PM, Brian Rogers wrote: The lack of a message on the delete operation indicates success. What you see is the expected behavior, since 'btrfs filesystem show' is reading the partitions directly. Therefore, it won't see any changes that haven't been committed to disk yet. The 'some devices missing' message should go away after running 'sync', or rebooting, or un-mounting the file system. Thanks for the suggestion, but that doesn't seem to work. I've tried rebooting multiple times. The new log below might be more interesting - note that `btrfs device delete missing` claims that there is no missing device. r...@willvo:~# btrfs filesystem sync /data FSSync '/data' r...@willvo:~# btrfs filesystem show failed to read /dev/sr0 Label: none uuid: f929c413-01c8-443f-b4f2-86f36702f519 Total devices 3 FS bytes used 577.81GB devid1 size 931.51GB used 604.00GB path /dev/sdb1 devid2 size 931.51GB used 604.00GB path /dev/sdc1 *** Some devices missing Btrfs Btrfs v0.19 r...@willvo:~# btrfs device delete missing /data r...@willvo:~# tail -1 /var/log/syslog Nov 1 11:20:39 willvo kernel: [175031.411348] btrfs: no missing devices found to remove r...@willvo:~# btrfs filesystem show failed to read /dev/sr0 Label: none uuid: f929c413-01c8-443f-b4f2-86f36702f519 Total devices 3 FS bytes used 577.81GB devid1 size 931.51GB used 604.00GB path /dev/sdb1 devid2 size 931.51GB used 604.00GB path /dev/sdc1 *** Some devices missing Btrfs Btrfs v0.19 r...@willvo:~# btrfs filesystem sync /data FSSync '/data' r...@willvo:~# btrfs filesystem show failed to read /dev/sr0 Label: none uuid: f929c413-01c8-443f-b4f2-86f36702f519 Total devices 3 FS bytes used 577.81GB devid1 size 931.51GB used 604.00GB path /dev/sdb1 devid2 size 931.51GB used 604.00GB path /dev/sdc1 *** Some devices missing Btrfs Btrfs v0.19 Cheers, Will :-} -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs-convert fails
On Sun, Oct 31, 2010 at 11:55 PM, Helmut Hullen hul...@t-online.de wrote: Hallo, linux-btrfs, I've tried to convert a 12 GByte ext2 partition (nearly full, 280 MByte free) with btrfs-convert. After about 15 minutes (700-MHz-CPU) the system tells ... creating ext2fs image file cleaning up system chunk btrfs-convert: extent-tree.c:2529: btrfs_reserve_extent: Assertion `!(ret)' failed Abgebrochen Try btrfs-convert -r /dev/xxx, hopefully it will recover your ext2. Yan, Zheng -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
