Re: [PATCH v2] Btrfs: fix btrfs boot when compiled as built-in
>> err = btrfs_hash_init(); > Did you find out what err's value was? Its -2 Ahmet -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] Btrfs: fix btrfs boot when compiled as built-in
okay, putting late_initcall instead of module_init fixes it for me, but this is something you guys should decide how to handle: //module_init(init_btrfs_fs) late_initcall(init_btrfs_fs); Ahmet -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] Btrfs: fix btrfs boot when compiled as built-in
There is still a problem when btrfs is built-in: err = btrfs_hash_init(); Will not succeed if run on an slow cpu (intel u7300) or in qemu if btrfs is built-in: # qemu-kvm -enable-kvm -usbdevice tablet -m 2048 -smp 2 -sdl -kernel /usr/src/linux/arch/x86/boot/bzImage But it works, when btrfs is a module. I really do not want to go back using modules in my initramfs .. There must be a way to move init_btrfs_fs to the end of Linux built-in initializations .. Ahmet -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] Btrfs: fix tree mod logging
Thanks a lot Filipe! Have been testing this patch now for 5 days and it fixed this annoying Problem since 3.11.0 on 3x NFS Servers here. This is also a candidate that should be back ported, as it fixes crashes. Just for Information for others here: Your previous patch, "Btrfs: return immediately if tree log mod is not necessary" is also needed to make it apply cleanly. Ahmet -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [REGRESSION] 3.12-rc1: Trying to create snapshot corrupted filesystem
On Sat, Sep 21, 2013 at 1:20 PM, Ahmet Inan wrote: >> You will want the patch I just sent, >> >> Btrfs: create the uuid tree on remount rw >> >> and that should fix the snapshot problems. Thanks, > > thanks Josef - you can close this bug: > > https://bugzilla.kernel.org/show_bug.cgi?id=61301 > > then. will try your patch later, too. > ok, works for me, too. changed state of bug to "resolved" Ahmet -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [REGRESSION] 3.12-rc1: Trying to create snapshot corrupted filesystem
> You will want the patch I just sent, > > Btrfs: create the uuid tree on remount rw > > and that should fix the snapshot problems. Thanks, thanks Josef - you can close this bug: https://bugzilla.kernel.org/show_bug.cgi?id=61301 then. will try your patch later, too. Ahmet -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs defrag kernel 3.11
ive had the same problem and this commit helped, which i had to cherry-pick:
commit c83b3452db6e1b048312c50b23f06c9634535f80
Author: Josef Bacik
Date: Mon Jul 22 12:50:37 2013 -0400
Btrfs: reset ret in record_one_backref
I was getting warnings when running find ./ -type f -exec btrfs fi
defrag -f {}
\; from record_one_backref because ret was set. Turns out it was
because it was
set to 1 because the search slot didn't come out exact and we
never reset it.
So reset it to 0 right after the search so we don't leak this and get
uneccessary warnings. Thanks,
Ahmet
On Mon, Sep 9, 2013 at 4:41 AM, Wang Shilong wrote:
>
> Hello,
>
> This bug should have been fixed by Liu Bo, the following patch:
>
> Btrfs: fix crash regarding to ulist_add_merge
>
> https://git.kernel.org/cgit/linux/kernel/git/josef/btrfs-next.git/commit/?h=for-chris&id=35f0399db6658f465b00893bdd13b992a0acfef0
>
> You can try btrfs-next:
> git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-next.git
>
> Thanks,
> Wang
>
> On 09/07/2013 05:35 AM, Marco L. Crociani wrote:
>>
>> Hi,
>> I have run:
>> find / -xdev \( -type f -o -type d \) -exec btrfs filesystem defragment -v
>> -- {} +
>>
>> On Ubuntu 13.04 with kernel 3.11 from root recovery shell after remounting
>> rw /
>>
>> This is the dmesg output:
>>
>> [17788.691504] [ cut here ]
>> [17788.691510] [ cut here ]
>> [17788.691538] WARNING: CPU: 1 PID: 2093 at
>> /home/apw/COD/linux/fs/btrfs/inode.c:2206 record_one_backref+0x3a9/0x420
>> [btrfs]()
>> [17788.691568] WARNING: CPU: 2 PID: 2095 at
>> /home/apw/COD/linux/fs/btrfs/inode.c:2206 record_one_backref+0x3a9/0x420
>> [btrfs]()
>> [17788.691540] Modules linked in: snd_usb_audio
>> [17788.691570] Modules linked in: snd_usb_audio uvcvideo snd_usbmidi_lib
>> videobuf2_vmalloc videobuf2_memops videobuf2_core videodev
>> snd_hda_codec_realtek snd_hda_codec_hdmi snd_hda_intel snd_hda_codec
>> snd_hwdep btusb eeepc_wmi asus_wmi sparse_keymap snd_pcm snd_page_alloc
>> k10temp video snd_seq_midi mac_hid snd_seq_midi_event snd_rawmidi i2c_piix4
>> dm_multipath serio_raw snd_seq snd_seq_device snd_timer scsi_dh snd
>> ohci_pci soundcore bluetooth btrfs zlib_deflate libcrc32c raid10 raid456
>> async_memcpy async_raid6_recov async_pq async_xor async_tx xor raid6_pq
>> raid0 multipath linear hid_generic usbhid hid raid1 usb_storage radeon wmi
>> i2c_algo_bit ttm r8169 mii drm_kms_helper ahci drm libahci
>> [17788.691641] uvcvideo<4>[17788.691642] CPU: 2 PID: 2095 Comm:
>> btrfs-endio-wri Not tainted 3.11.0-031100-generic #201309021735
>> [17788.691644] Hardware name: System manufacturer System Product
>> Name/F1A75-V EVO, BIOS 0702 07/12/2011
>> [17788.691645] 089e 88011b091a58 81720fca
>> 1b8b
>> [17788.691650] snd_usbmidi_lib videobuf2_vmalloc videobuf2_memops
>> videobuf2_core videodev snd_hda_codec_realtek snd_hda_codec_hdmi
>> snd_hda_intel snd_hda_codec snd_hwdep btusb eeepc_wmi asus_wmi
>> sparse_keymap snd_pcm snd_page_alloc k10temp video snd_seq_midi mac_hid
>> snd_seq_midi_event snd_rawmidi i2c_piix4 dm_multipath serio_raw snd_seq
>> snd_seq_device snd_timer scsi_dh snd ohci_pci soundcore bluetooth btrfs
>> zlib_deflate libcrc32c raid10 raid456 async_memcpy async_raid6_recov
>> async_pq async_xor async_tx xor raid6_pq raid0 multipath linear hid_generic
>> usbhid hid raid1 usb_storage radeon wmi i2c_algo_bit ttm r8169 mii
>> drm_kms_helper ahci drm libahci
>> [17788.691717]
>> [17788.691717] 88011b091a98 8106534c
>> 88011b091a78
>> [17788.691720] 8801f3c1aea0 0001 8801056d2dc0
>> 1600
>> [17788.691722] Call Trace:
>> [17788.691727] [] dump_stack+0x46/0x58
>> [17788.691730] [] warn_slowpath_common+0x8c/0xc0
>> [17788.691732] [] warn_slowpath_null+0x1a/0x20
>> [17788.691744] [] record_one_backref+0x3a9/0x420 [btrfs]
>> [17788.691755] [] ? btrfs_submit_direct+0x190/0x190
>> [btrfs]
>> [17788.691768] [] iterate_leaf_refs+0x52/0xc0 [btrfs]
>> [17788.691779] [] ? btrfs_submit_direct+0x190/0x190
>> [btrfs]
>> [17788.691790] [] iterate_extent_inodes+0x198/0x270
>> [btrfs]
>> [17788.691801] [] iterate_inodes_from_logical+0x92/0xb0
>> [btrfs]
>> [17788.691812] [] ? btrfs_submit_direct+0x190/0x190
>> [btrfs]
>> [17788.691823] [] record_extent_backrefs+0x7c/0xf0
>> [btrfs]
>> [17788.691834] [] relink_file_extents+0x44/0x180 [btrfs]
>> [17788.691845] [] btrfs_finish_ordered_io+0x135/0x4d0
>> [btrfs]
>> [17788.691856] [] finish_ordered_fn+0x15/0x20 [btrfs]
>> [17788.691867] [] worker_loop+0xa0/0x320 [btrfs]
>> [17788.691878] [] ?
>> check_pending_worker_creates.isra.1+0xe0/0xe0 [btrfs]
>> [17788.691881] [] kthread+0xc0/0xd0
>> [17788.691884] [] ? flush_kthread_worker+0xb0/0xb0
>> [17788.691886] [] ret_from_fork+0x7c/0xb0
>> [17788.691888] [] ? flush_kthread_worker+0xb0/0xb0
>> [17788.691890] ---[ end trace b632fc27406d343c ]---
>> [17788.691892] CPU: 1
Re: kernel BUG at fs/btrfs/ctree.c:1144!
> The real problem, however, is not caused by that commit but by a tree mod log
> bug. I expect that fs/btrfs/ctree.c:1144 is this BUG_ON in your kernel from
> __tree_mod_log_rewind (my line numbers don't match):
>
> 1138 switch (tm->op) {
> 1139 case MOD_LOG_KEY_REMOVE_WHILE_FREEING:
> 1140 BUG_ON(tm->slot < n);
>
> I've got a fix for that I'm currently testing, expect it on the list soon.
Yea, thats the line.
> For the meantime I recommend to not defrag your filesystem.
Ok, already disabled and rebooted.
Will come back to You if i still get that Problem.
> As a general remark, please send your stack traces inline, not as attachment
> if
> possible.
Google mail always screws longer than 80 lines in plain text mode and
non plain text is not what You want.
Maybe i should go back to using mutt.
Thank You,
Ahmet
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
kernel BUG at fs/btrfs/ctree.c:1144!
I got this problem since 3.8.5 + for-linus (from that time). Have just tried 3.8.6 + for-linus with "git merge -X theirs btrfs/for-linus" but still same problem. Going back to 3.7.4 + for-linus (from that time) doesn't give me the problem. This is an production nfs server with 2x2TB raid1, so cant reboot it that often. Have seen this same problem on another system (also raid1) once, but rebooting helped, no problems since. Both systems use autodefrag, maybe that sometimes triggers it? I really would like to help, so i can stay on the latest kernels. What should i do? Ahmet [35103.977604] [ cut here ] [35103.977633] kernel BUG at fs/btrfs/ctree.c:1144! [35103.977653] invalid opcode: [#1] SMP [35103.977674] Modules linked in: dm_mod fuse snd_hda_codec_analog snd_hda_intel snd_hda_codec snd_pcm snd_page_alloc snd_timer snd soundcore ppdev iTCO_wdt lpc_ich mfd_core parport_pc parport kvm [35103.91] CPU 1 [35103.977783] Pid: 2989, comm: btrfs-endio-wri Not tainted 3.8.6-ainan #119 Dell Inc. OptiPlex 755 /0GM819 [35103.977825] RIP: 0010:[] [] __tree_mod_log_rewind+0x25b/0x260 [35103.977864] RSP: 0018:880107a8f7b8 EFLAGS: 00010297 [35103.977885] RAX: RBX: 88006a9094d0 RCX: 8801126124c0 [35103.977912] RDX: 0b6398ee RSI: 066c RDI: 880182953a40 [35103.977939] RBP: 880107a8f7e8 R08: 1000 R09: 880107a8f760 [35103.977965] R10: 0016ed05 R11: R12: 880222fb76c0 [35103.977993] R13: 002f R14: 8801126124c0 R15: 0cd2 [35103.978019] FS: () GS:880237c4() knlGS: [35103.978049] CS: 0010 DS: ES: CR0: 8005003b [35103.978071] CR2: 00b89308 CR3: 0001b5b2f000 CR4: 07e0 [35103.978097] DR0: DR1: DR2: [35103.978123] DR3: DR6: 0ff0 DR7: 0400 [35103.978150] Process btrfs-endio-wri (pid: 2989, threadinfo 880107a8e000, task 88014516c0c0) [35103.978182] Stack: [35103.978192] 88006a909a44 0002 88006a9094d0 88006a909a10 [35103.978227] 8801355bd400 1600 880107a8f888 812b6266 [35103.978261] 880107a8f828 880222fb76c0 88022dea8000 00010cd2 [35103.978295] Call Trace: [35103.978309] [] btrfs_search_old_slot+0x646/0x990 [35103.978336] [] __resolve_indirect_refs+0x16f/0x610 [35103.978363] [] ? free_extent_buffer+0x58/0xb0 [35103.978387] [] ? __add_missing_keys.clone.13+0x89/0x130 [35103.978415] [] ? free_extent_buffer+0x58/0xb0 [35103.978439] [] find_parent_nodes+0x586/0x9a0 [35103.978462] [] ? btrfs_set_path_blocking+0x31/0x70 [35103.978488] [] btrfs_find_all_roots+0xa1/0x100 [35103.978513] [] ? record_extent_backrefs+0xf0/0xf0 [35103.978538] [] iterate_extent_inodes+0x183/0x2d0 [35103.978562] [] ? btrfs_get_token_64+0x5f/0xf0 [35103.978587] [] ? free_extent_buffer+0x58/0xb0 [35103.978612] [] iterate_inodes_from_logical+0xa7/0xb0 [35103.978638] [] ? record_extent_backrefs+0xf0/0xf0 [35103.978664] [] record_extent_backrefs+0x78/0xf0 [35103.978689] [] btrfs_finish_ordered_io+0x186/0xa70 [35103.978715] [] ? try_to_del_timer_sync+0x76/0xa0 [35103.978740] [] ? cascade+0xf0/0x190 [35103.978762] [] finish_ordered_fn+0x10/0x20 [35103.978785] [] worker_loop+0xb8/0x540 [35103.978807] [] ? btrfs_queue_worker+0x310/0x310 [35103.978832] [] ? btrfs_queue_worker+0x310/0x310 [35103.978857] [] kthread+0xc6/0xd0 [35103.978877] [] ? kthread_freezable_should_stop+0x70/0x70 [35103.978905] [] ret_from_fork+0x7c/0xb0 [35103.978927] [] ? kthread_freezable_should_stop+0x70/0x70 [35103.978952] Code: c1 49 63 46 58 48 89 c2 48 c1 e2 05 48 8d 54 10 65 49 63 46 2c 48 89 c6 48 c1 e6 05 48 8d 74 30 65 e8 fa 9b 04 00 e9 ad fe ff ff <0f> 0b 0f 0b 90 55 48 89 e5 41 56 49 89 d6 41 55 49 89 f5 41 54 [35103.979149] RIP [] __tree_mod_log_rewind+0x25b/0x260 [35103.979178] RSP [35103.991131] ---[ end trace 3a477bfb86e5ca4a ]---
Re: btrfs crash when low on memory.
> If we're corrupting on abort that is a bug too that needs to be fixed > too. I've banged on the abort stuff a lot recently when trying to > make it not panic the box and it appears to work fine. Obviously that > kind of stuff needs to be tested as well, but so far I haven't seen > abort corrupt the file system. Thanks, thank you for the info Josef. i will report a bug next time i hit such a case then. Ahmet -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs crash when low on memory.
On Wed, Feb 27, 2013 at 7:26 PM, Josef Bacik wrote: > On Wed, Feb 27, 2013 at 07:31:11AM -0700, Ahmet Inan wrote: >> > Yeah we have a lot of >> > >> > ptr = kmalloc(); >> > BUG_ON(ptr); >> > >> > everywhere. I'll fix this one up but I really need to sit down and go >> > through >> > all of them and make sure we do the right thing in all these places. >> > Thanks, >> >> But what would be the right thing to do when you got no memory? >> Spinlock until you can kmalloc? Pre-reserve some memory? >> > > Return ENOMEM? We have a way to abort transactions now, if it's in a horrible > of enough spot we can just abort the transaction and let the user deal with > the > aftermath, it's nicer than panicing. Thanks, youre right. i am only afraid of silent corruption of data on aborts: our guys here trigger OOM all the time with their compilers and numerical codes (go figure). and until now we had no more aborts / panics because of "vm.min_free_kbytes = 65536" and thus no corruption. my point is: i like a freezing computer more than an corrupting computer, even if its a server. reboot to the rescue. Ahmet -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs crash when low on memory.
> Yeah we have a lot of > > ptr = kmalloc(); > BUG_ON(ptr); > > everywhere. I'll fix this one up but I really need to sit down and go through > all of them and make sure we do the right thing in all these places. Thanks, But what would be the right thing to do when you got no memory? Spinlock until you can kmalloc? Pre-reserve some memory? At the moment im using: vm.min_free_kbytes = 65536 Which helps most of the time and i think is the better way to handle this kind of Situation. Ahmet -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Why btrfs inline small file by default?
>> 9.08GB + 992.48MB*2 == 11.02GB >> 10.85GB + 518MB*2 == 11.86GB >> That's nearly a GB smaller. > That, too; I missed the "DUP". Not quite as pronounced as in my > calculations, then, but still a significant enough difference. great. now were down to 7-8% just FYI: ive retested with max_inline=0 but with leafsize=64K this time: # mkfs.btrfs -l 64K -L root /dev/sda2 ... real4m45.878s user1m44.730s sys 1m7.226s thats 2% slower for this one test (no big deal really) # btrfs filesystem show Label: 'root' uuid: dd2951da-2529-4320-a952-e692ea5bdbc3 Total devices 1 FS bytes used 11.37GB devid1 size 232.63GB used 13.04GB path /dev/sda2 Btrfs v0.20-rc1 # btrfs filesystem df /mnt/point/ Data: total=11.01GB, used=10.89GB System, DUP: total=8.00MB, used=64.00KB System: total=4.00MB, used=0.00 Metadata, DUP: total=1.00GB, used=487.94MB Metadata: total=8.00MB, used=0.00 (1024*10.89 + 2*487.94) / 1024 = 11.84GiB still around 7-8% now lets see what everyday use with max_inline=0 and leafsize=64K feels like :) Ahmet -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Why btrfs inline small file by default?
with defaults (inlining small files): real4m39.253s Data: total=10.01GB, used=9.08GB Metadata, DUP: total=2.00GB, used=992.48MB > > This uses 10290.40 MB total, if we pad with zeroes (9.08GB plus > 992.48MB). without inline: real4m42.085s Data: total=11.01GB, used=10.85GB Metadata, DUP: total=1.00GB, used=518.59MB > > Under the same assumption, this uses 11628.99 MB total (10.85GB + > 518.59MB). > With inlining, you're using about 1.3 GB less disk space and require a thank you for clarifying this. 10% is indeed a benefit. > few seconds less wall-clock time for the same thing. those where only 2 tests, have to make a lot more runs to make qualified judgement there. one percent difference is noise floor to me. Ahmet -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Why btrfs inline small file by default?
>> i also dont see any benefit from inlining small files: >> with defaults (inlining small files): >> real4m39.253s >> Data: total=10.01GB, used=9.08GB >> Metadata, DUP: total=2.00GB, used=992.48MB >> without inline: >> real4m42.085s >> Data: total=11.01GB, used=10.85GB >> Metadata, DUP: total=1.00GB, used=518.59MB > > I suggest you take a closer look at your numbers. both use 12GiB in total and both need 280 seconds. am i missing something? Ahmet -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Why btrfs inline small file by default?
i also dont see any benefit from inlining small files: this example is me doing a fully fledged prebuilt gentoo system installation on a fresh HDD from squashfs image on usb key in under 5 minutes: with defaults (inlining small files): # mount -o noatime,compress=lzo /dev/sda2 /mnt/point # time unsquashfs -f -d /mnt/point/ /dev/sdb2 real4m39.253s user1m37.854s sys 1m1.433s # btrfs filesystem show Label: 'root' uuid: 6fb7104d-8f4a-4f3a-aff8-fdad0a39f569 Total devices 1 FS bytes used 10.05GB devid1 size 232.63GB used 14.04GB path /dev/sda2 Btrfs v0.20-rc1 # btrfs filesystem df /mnt/point/ Data: total=10.01GB, used=9.08GB System, DUP: total=8.00MB, used=4.00KB System: total=4.00MB, used=0.00 Metadata, DUP: total=2.00GB, used=992.48MB Metadata: total=8.00MB, used=0.00 without inline: # mount -o max_inline=0,noatime,compress=lzo /dev/sda2 /mnt/point # time unsquashfs -f -d /mnt/point/ /dev/sdb2 real4m42.085s user1m36.894s sys 1m1.583s # btrfs filesystem show failed to read /dev/sr0 Label: 'root' uuid: 97ad3c97-ab03-4197-86d3-72869604b368 Total devices 1 FS bytes used 11.36GB devid1 size 232.63GB used 13.04GB path /dev/sda2 Btrfs v0.20-rc1 # btrfs filesystem df /mnt/point/ Data: total=11.01GB, used=10.85GB System, DUP: total=8.00MB, used=4.00KB System: total=4.00MB, used=0.00 Metadata, DUP: total=1.00GB, used=518.59MB Metadata: total=8.00MB, used=0.00 i will test "no inlining" for some more time though. Ahmet -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ENOSPC design issues
when testing, please also do something like this: # create big squashfs image somewhere: # mksquashfs / /big.img -noappend -no-sparse -e big.img # then unpack into fresh filesystem with (and no) compression: # unsquashfs -f -d /subvol /big.img this is how i was always able to trigger ENOSPC while trying to make a full system installation from squashfs image. you should also try different compression algos (i only use lzo) btw: i was able to trigger ENOSPC with for-linus on 3.5.4 on a i686 Pentium M Notebook with only 1GB of Memory and fresh FSthis way, otherwise havent seen ENOSPC for long time. Ahmet On Tue, Sep 25, 2012 at 7:02 PM, Josef Bacik wrote: > On Tue, Sep 25, 2012 at 10:43:36AM -0600, David Sterba wrote: >> On Thu, Sep 20, 2012 at 03:03:06PM -0400, Josef Bacik wrote: >> > I'm going to look at fixing some of the performance issues that crop up >> > because >> > of our reservation system. Before I go and do a whole lot of work I want >> > some >> > feedback. I've done a brain dump here >> > https://btrfs.wiki.kernel.org/index.php/ENOSPC >> >> Thanks for writing it down, much appreciated. >> >> My first and probably naive approach is described in the page, quoting >> here: >> >> "Attempt to address how to flush less stated below. The >> over-reservation of a 4k block can go up to 96k as the worst case >> calculation (see above). This accounts for splitting the full tree path >> from 8th level root down to the leaf plus the node splits. My question: >> how often do we need to go up to the level N+1 from current level N? >> for levels 0 and 1 it may happen within one transaction, maybe not so >> often for level 2 and with exponentially decreasing frequency for the >> higher levels. Therefore, is it possible to check the tree level first >> and adapt the calculation according to that? Let's say we can reduce >> the 4k reservation size from 96k to 32k on average (for a many-gigabyte >> filesystem), thus increasing the space available for reservations by >> some factor. The expected gain is less pressure to the flusher because >> more reservations will succeed immediately. >> The idea behind is to make the initial reservation more accurate to >> current state than blindly overcommitting by some random factor (1/2). >> Another hint to the tree root level may be the usage of the root node: >> eg. if the root is less than half full, splitting will not happen >> unless there are K concurrent reservations running where K is >> proportional to overwriting the whole subtree (same exponential >> decrease with increasing level) and this will not be possible within >> one transaction or there will not be enough space to satisfy all >> reservations. (This attempts to fine-tune the currently hardcoded level >> 8 up to the best value). The safe value for the level in the >> calculations would be like N+1, ie. as if all the possible splits >> happen with respect to current tree height." >> >> implemented as follows on top of next/master, in short: >> * disable overcommit completely >> * do the optimistically best guess for the metadata and reserve only up >> to the current tree height >> > > So I had tried to do this before, the problem is when height changes our > reserve > changes. So for things like delalloc we say we have X number of extents and > we > reserve that much space, but then when we run delalloc we re-calculate the > metadata size for X number extents we've removed and that number could come > out > differently since the height of the tree would have changed. One thing we > could > do is to store the actual reservation with the extent in the io_tree, but I > think we already use the private for something else so we'd have to add it > somewhere else. Thanks, > > Josef > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- Ahmet Inan Systemadministrator Mathematisches Institut Albert-Ludwigs-Universität Freiburg Eckerstr. 1 79104 Freiburg im Breisgau Tel.: +49(0)761 / 203-5552 Raum: 332 mailto:sys...@email.mathematik.uni-freiburg.de Abteilung für Angewandte Mathematik Albert-Ludwigs-Universität Freiburg Hermann-Herder-Str. 10 79104 Freiburg im Breisgau Tel.: +49(0)761 / 203-5626 Raum: 221 mailto:ad...@mathematik.uni-freiburg.de Abteilung für Mathematische Stochastik Albert-Ludwigs-Universität Freiburg Eckerstr. 1 79104 Freiburg im Breisgau Tel.: +49(0)761 / 203-5678 Raum: 249 mailto:helpd...@stochastik.uni-freiburg.de -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Boot speed/mount time regression with 3.4.0-rc2
On Sat, Apr 21, 2012 at 10:54 PM, Ahmet Inan wrote: > On Fri, Apr 13, 2012 at 3:47 PM, Josef Bacik wrote: >> On Fri, Apr 13, 2012 at 02:26:19PM +0200, Ahmet Inan wrote: >>> On Fri, Apr 13, 2012 at 1:22 PM, Ahmet Inan >>> wrote: >>> > On Fri, Apr 13, 2012 at 8:49 AM, cwillu wrote: >>> >>> dmesg and fstab attached as requested. >>> >> >>> >> Need dmesg after you've hit alt-sysrq-w a couple times during the slow >>> >> period. >>> > >>> > here. >>> > >>> > i guess i should also increase dmesg history size next time. >>> > other than the slow boot, everything seems normal after 10-20minutes. >>> > >>> > fyi: the space_cache option is not really helping with those >>> > twenty computers. the one thing i observed is, that sometimes >>> > they reboot fast and only to reboot slow again after that. >>> >>> sorry for spaming the list with my dmesg files, >>> please tell me, how i could do better. >>> >>> here a more complete dmesg. >> >> Hrm so you are getting blocked task warnings just trying to mount the >> filesystem, so either your disk is really really really slow or there's >> something bigger going on. Let me think about this some and I'll get back to >> you. > > Josef, i finally found out something: > > btrfs in kernel => fast boot > btrfs as module => very slow boot > > and here see the results: > http://aam.mathematik.uni-freiburg.de/IAM/homepages/ainan/bootchart_btrfs_in_kernel.png > > this is much, much better! > > I dont understand why btrfs as a module performs that bad on rotating disk. > On SSD i had no issues and also i never used space_cache before. > > Im going to deploy the new kernel on our systems next week with > btrfs in-kernel and come back with the results. FYI: a 100 computers here with all kinds of hardware constellations work perfectly since the kernel upgrade (3.3.2 + for-linus) one week ago. We are very happy with the better btrfs performance. Also boot times are very short again. Even upgrading from different and very old Kernel versions worked flawless: vanilla linux-2.6.38.8: "Old style space inode found, converting." great :-) To make a long story short: Thanks a lot! Ahmet -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Boot speed/mount time regression with 3.4.0-rc2
On Sun, Apr 22, 2012 at 11:16 AM, Ahmet Inan wrote: > On Sun, Apr 22, 2012 at 9:59 AM, Sergei Trofimovich wrote: >> On Sat, 21 Apr 2012 22:54:41 +0200 >> Ahmet Inan wrote: >> >>> On Fri, Apr 13, 2012 at 3:47 PM, Josef Bacik wrote: >>> > On Fri, Apr 13, 2012 at 02:26:19PM +0200, Ahmet Inan wrote: >>> >> On Fri, Apr 13, 2012 at 1:22 PM, Ahmet Inan >>> >> wrote: >>> >> > On Fri, Apr 13, 2012 at 8:49 AM, cwillu wrote: >>> >> >>> dmesg and fstab attached as requested. >>> >> >> >>> >> >> Need dmesg after you've hit alt-sysrq-w a couple times during the >>> >> >> slow period. >>> >> > >>> >> > here. >>> >> > >>> >> > i guess i should also increase dmesg history size next time. >>> >> > other than the slow boot, everything seems normal after 10-20minutes. >>> >> > >>> >> > fyi: the space_cache option is not really helping with those >>> >> > twenty computers. the one thing i observed is, that sometimes >>> >> > they reboot fast and only to reboot slow again after that. >>> >> >>> >> sorry for spaming the list with my dmesg files, >>> >> please tell me, how i could do better. >>> >> >>> >> here a more complete dmesg. >>> > >>> > Hrm so you are getting blocked task warnings just trying to mount the >>> > filesystem, so either your disk is really really really slow or there's >>> > something bigger going on. Let me think about this some and I'll get >>> > back to >>> > you. >>> >>> Josef, i finally found out something: >>> >>> btrfs in kernel => fast boot >>> btrfs as module => very slow boot >>> >>> and here see the results: >>> http://aam.mathematik.uni-freiburg.de/IAM/homepages/ainan/bootchart_btrfs_in_kernel.png >>> >>> this is much, much better! >>> >>> I dont understand why btrfs as a module performs that bad on rotating disk. >>> On SSD i had no issues and also i never used space_cache before. >> >> There is a suspiction that putting space_cache (or anything else) to fstab >> for rootfs >> does not get applied due to a bug: >> >> http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg16039.html > > Thats right. > > space_cache only got enabled for real, after ive set the option in the > boot argument once. > Even if it looks like it gets enabled in dmesg via /etc/fstab, > the next boot without space_cache doesnt show space_cache enabled. > > But this is another problem i will look into later. I just pulled for-linus and thank you for that patch Sergei. Now i can enable space_cache on all systems without playing with bootarguments or fstab: # mount -o remount,space_cache / it speeds up boot times after a couple reboots. perfect! > At the moment the biggest problem is having btrfs as a module. To be sure i just rebooted that troublesome computer again with btrfs as module and with space_cache enabled: http://aam.mathematik.uni-freiburg.de/IAM/homepages/ainan/bootchart_btrfs_as_module.png So it looks like btrfs as module only amplified that space_cache problem .. And now that space_cache is enabled and working even btrfs as module works ok. Conclusion: btrfs in kernel + space_cache solves all my slow boot problems. Ahmet -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Boot speed/mount time regression with 3.4.0-rc2
On Sun, Apr 22, 2012 at 9:59 AM, Sergei Trofimovich wrote: > On Sat, 21 Apr 2012 22:54:41 +0200 > Ahmet Inan wrote: > >> On Fri, Apr 13, 2012 at 3:47 PM, Josef Bacik wrote: >> > On Fri, Apr 13, 2012 at 02:26:19PM +0200, Ahmet Inan wrote: >> >> On Fri, Apr 13, 2012 at 1:22 PM, Ahmet Inan >> >> wrote: >> >> > On Fri, Apr 13, 2012 at 8:49 AM, cwillu wrote: >> >> >>> dmesg and fstab attached as requested. >> >> >> >> >> >> Need dmesg after you've hit alt-sysrq-w a couple times during the slow >> >> >> period. >> >> > >> >> > here. >> >> > >> >> > i guess i should also increase dmesg history size next time. >> >> > other than the slow boot, everything seems normal after 10-20minutes. >> >> > >> >> > fyi: the space_cache option is not really helping with those >> >> > twenty computers. the one thing i observed is, that sometimes >> >> > they reboot fast and only to reboot slow again after that. >> >> >> >> sorry for spaming the list with my dmesg files, >> >> please tell me, how i could do better. >> >> >> >> here a more complete dmesg. >> > >> > Hrm so you are getting blocked task warnings just trying to mount the >> > filesystem, so either your disk is really really really slow or there's >> > something bigger going on. Let me think about this some and I'll get back >> > to >> > you. >> >> Josef, i finally found out something: >> >> btrfs in kernel => fast boot >> btrfs as module => very slow boot >> >> and here see the results: >> http://aam.mathematik.uni-freiburg.de/IAM/homepages/ainan/bootchart_btrfs_in_kernel.png >> >> this is much, much better! >> >> I dont understand why btrfs as a module performs that bad on rotating disk. >> On SSD i had no issues and also i never used space_cache before. > > There is a suspiction that putting space_cache (or anything else) to fstab > for rootfs > does not get applied due to a bug: > > http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg16039.html Thats right. space_cache only got enabled for real, after ive set the option in the boot argument once. Even if it looks like it gets enabled in dmesg via /etc/fstab, the next boot without space_cache doesnt show space_cache enabled. But this is another problem i will look into later. At the moment the biggest problem is having btrfs as a module. > Do all of your affected computers run 3.4.0-rc kernels? They run vanilla/linux-3.3.2 with btrfs/for-linus. The unaffected computers have <= vanilla/linux-3.2.9 with btrfs/for-linus from 2 months ago or so. Ahmet -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Boot speed/mount time regression with 3.4.0-rc2
On Fri, Apr 13, 2012 at 3:47 PM, Josef Bacik wrote: > On Fri, Apr 13, 2012 at 02:26:19PM +0200, Ahmet Inan wrote: >> On Fri, Apr 13, 2012 at 1:22 PM, Ahmet Inan >> wrote: >> > On Fri, Apr 13, 2012 at 8:49 AM, cwillu wrote: >> >>> dmesg and fstab attached as requested. >> >> >> >> Need dmesg after you've hit alt-sysrq-w a couple times during the slow >> >> period. >> > >> > here. >> > >> > i guess i should also increase dmesg history size next time. >> > other than the slow boot, everything seems normal after 10-20minutes. >> > >> > fyi: the space_cache option is not really helping with those >> > twenty computers. the one thing i observed is, that sometimes >> > they reboot fast and only to reboot slow again after that. >> >> sorry for spaming the list with my dmesg files, >> please tell me, how i could do better. >> >> here a more complete dmesg. > > Hrm so you are getting blocked task warnings just trying to mount the > filesystem, so either your disk is really really really slow or there's > something bigger going on. Let me think about this some and I'll get back to > you. Josef, i finally found out something: btrfs in kernel => fast boot btrfs as module => very slow boot and here see the results: http://aam.mathematik.uni-freiburg.de/IAM/homepages/ainan/bootchart_btrfs_in_kernel.png this is much, much better! I dont understand why btrfs as a module performs that bad on rotating disk. On SSD i had no issues and also i never used space_cache before. Im going to deploy the new kernel on our systems next week with btrfs in-kernel and come back with the results. Ahmet -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Boot speed/mount time regression with 3.4.0-rc2
On Fri, Apr 13, 2012 at 2:57 PM, cwillu wrote: > These are usb disks? Does that failure at 12.241517 (or related) > happen every time? no, 0CCD:0052 is the webcam. i dont have the modules for the webcam in initramfs, thats why. the real slowness is around 33.305370 The 2 SATA Disks are fine. I have many more Computers with varying Hardware but the same Kernel and see the same Problems there too. Ahmet -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Boot speed/mount time regression with 3.4.0-rc2
On Fri, Apr 13, 2012 at 8:49 AM, cwillu wrote: >> dmesg and fstab attached as requested. > > Need dmesg after you've hit alt-sysrq-w a couple times during the slow period. here. i guess i should also increase dmesg history size next time. other than the slow boot, everything seems normal after 10-20minutes. fyi: the space_cache option is not really helping with those twenty computers. the one thing i observed is, that sometimes they reboot fast and only to reboot slow again after that. Ahmet dmesg Description: Binary data
Re: Boot speed/mount time regression with 3.4.0-rc2
On Thu, Apr 12, 2012 at 4:23 PM, Josef Bacik wrote: > On Thu, Apr 12, 2012 at 09:37:48AM -0400, Josef Bacik wrote: >> On Thu, Apr 12, 2012 at 11:22:51AM +0200, Ahmet Inan wrote: >> > On Wed, Apr 11, 2012 at 7:04 PM, Josef Bacik wrote: >> > > On Wed, Apr 11, 2012 at 05:26:29PM +0200, Ahmet Inan wrote: >> > >> On Tue, Apr 10, 2012 at 5:16 PM, Josef Bacik wrote: >> > >> > On Mon, Apr 09, 2012 at 05:20:46PM -0400, Calvin Walton wrote: >> > >> >> On Mon, 2012-04-09 at 16:54 -0400, Josef Bacik wrote: >> > >> >> > On Mon, Apr 09, 2012 at 01:10:04PM -0400, Calvin Walton wrote: >> > >> >> > > On Mon, 2012-04-09 at 11:53 -0400, Calvin Walton wrote: >> > >> >> > > > Hi, >> > >> >> > > > >> > >> >> > > > I have a system that's using a dracut-generated initramfs to >> > >> >> > > > mount a >> > >> >> > > > btrfs root. After upgrading to kernel 3.4.0-rc2 to test it >> > >> >> > > > out, I've >> > >> >> > > > noticed that the process of mounting the root filesystem takes >> > >> >> > > > much >> > >> >> > > > longer with 3.4.0-rc2 than it did with 3.3.1 - nearly 30 >> > >> >> > > > seconds slower! >> > >> >> >> > >> >> > > And the bisect results are in: >> > >> >> > > 285ff5af6ce358e73f53b55c9efadd4335f4c2ff is the first bad commit >> > >> >> > > commit 285ff5af6ce358e73f53b55c9efadd4335f4c2ff >> > >> >> > > Author: Josef Bacik >> > >> >> > > Date: Fri Jan 13 15:27:45 2012 -0500 >> > >> >> > > >> > >> >> > > Btrfs: remove the ideal caching code> >> > >> >> > >> > >> >> > Ok can you give this a whirl? You are going to have to >> > >> >> > boot/reboot a few times >> > >> >> > to let the cache get re-generated again to make sure it's taken >> > >> >> > effect, but >> > >> >> > hopefully this will help out. Thanks, >> > >> >> >> > >> >> Unfortunately, it doesn't seem to help. Even after 3 or 4 reboots >> > >> >> with >> > >> >> this patch applied I'm still seeing the same delay. >> > >> >> >> > >> > >> > >> > Ok drop that previous patch and give this one a whirl, it helped on >> > >> > my laptop. >> > >> > This is only half of the problem AFAICS, but it's the easier half to >> > >> > fix, in >> > >> > the meantime I need to lock down why we're not writing out cache for >> > >> > a bunch of >> > >> > block groups, but thats trickier since the messages I need are spit >> > >> > out while >> > >> > I'm shutting down, so I need to get creative. Let me know if/how >> > >> > much this >> > >> > helps. Thanks, >> > >> >> > >> i have tried your patch and my system still needs several minutes to >> > >> boot >> > >> until it can be used. >> > >> Also tried to reboot several times - it doesn't look like its getting >> > >> better. >> > >> The last thing the system does when its shutting down is a read-only >> > >> remount of "/" so no umount. >> > >> Booting was much faster before i pulled for-linus a few weeks ago but >> > >> i couldn't find the time to bisect it yet .. >> > >> >> > >> please also look at the attached dmesg.txt. >> > >> this is an core i3 system with 2x2TB BTRFS RAID1 and lots of >> > >> home directories and snapshots. >> > >> >> > >> I'm going to test this patch on twenty more computers but with >> > >> smaller HDDs and less files and see if it helps to speed up their >> > >> boot times. >> > >> >> > > >> > > Ok looks like you are running into a different problem. Could you maybe >> > > run >> > > bootchart and upload the resulting png somewhere so I can look and see >> > > what all >> > > is running while you boot? Thanks, >> > >> > http://aam.mathematik.uni-freiburg.de/IAM/homepages/ainan/bootchart.png >> > >> > i have tried your patch now on the twenty more computers i mentioned and >> > still it takes a minute to remount rw "/" on those, even after several >> > reboots. >> > >> >> Oops responding to the whole list this time.. >> >> Um ouch your system appears to not be doing anything for like 300 seconds but >> sitting there. Can you hook up a console and capture sysrq+w while thats >> going >> on? Also you are mounting with -o space_cache right? Can I see your dmesg >> to >> make sure it's doing what it's supposed to? Thanks, >> > > Ok you don't actually have space_cache enabled it looks like, make sure to add > space_cache to your fstab so it gets enabled, and then reboot a few times to > make sure everything gets cached right and then it should help. Thanks, now i did enable space_cache in fstab and rebooted 4 times, still no improvement: http://aam.mathematik.uni-freiburg.de/IAM/homepages/ainan/bootchart_space_cache.png is it vital to put this space_cache option to the boot argument as well? mounting "/" readonly in initramfs and booting to it (until remount "/" rw) is quite fast. dmesg and fstab attached as requested. Ahmet dmesg Description: Binary data fstab Description: Binary data
Re: Boot speed/mount time regression with 3.4.0-rc2
On Wed, Apr 11, 2012 at 7:04 PM, Josef Bacik wrote: > On Wed, Apr 11, 2012 at 05:26:29PM +0200, Ahmet Inan wrote: >> On Tue, Apr 10, 2012 at 5:16 PM, Josef Bacik wrote: >> > On Mon, Apr 09, 2012 at 05:20:46PM -0400, Calvin Walton wrote: >> >> On Mon, 2012-04-09 at 16:54 -0400, Josef Bacik wrote: >> >> > On Mon, Apr 09, 2012 at 01:10:04PM -0400, Calvin Walton wrote: >> >> > > On Mon, 2012-04-09 at 11:53 -0400, Calvin Walton wrote: >> >> > > > Hi, >> >> > > > >> >> > > > I have a system that's using a dracut-generated initramfs to mount a >> >> > > > btrfs root. After upgrading to kernel 3.4.0-rc2 to test it out, I've >> >> > > > noticed that the process of mounting the root filesystem takes much >> >> > > > longer with 3.4.0-rc2 than it did with 3.3.1 - nearly 30 seconds >> >> > > > slower! >> >> >> >> > > And the bisect results are in: >> >> > > 285ff5af6ce358e73f53b55c9efadd4335f4c2ff is the first bad commit >> >> > > commit 285ff5af6ce358e73f53b55c9efadd4335f4c2ff >> >> > > Author: Josef Bacik >> >> > > Date: Fri Jan 13 15:27:45 2012 -0500 >> >> > > >> >> > > Btrfs: remove the ideal caching code> >> >> > >> >> > Ok can you give this a whirl? You are going to have to boot/reboot a >> >> > few times >> >> > to let the cache get re-generated again to make sure it's taken effect, >> >> > but >> >> > hopefully this will help out. Thanks, >> >> >> >> Unfortunately, it doesn't seem to help. Even after 3 or 4 reboots with >> >> this patch applied I'm still seeing the same delay. >> >> >> > >> > Ok drop that previous patch and give this one a whirl, it helped on my >> > laptop. >> > This is only half of the problem AFAICS, but it's the easier half to fix, >> > in >> > the meantime I need to lock down why we're not writing out cache for a >> > bunch of >> > block groups, but thats trickier since the messages I need are spit out >> > while >> > I'm shutting down, so I need to get creative. Let me know if/how much this >> > helps. Thanks, >> >> i have tried your patch and my system still needs several minutes to boot >> until it can be used. >> Also tried to reboot several times - it doesn't look like its getting better. >> The last thing the system does when its shutting down is a read-only >> remount of "/" so no umount. >> Booting was much faster before i pulled for-linus a few weeks ago but >> i couldn't find the time to bisect it yet .. >> >> please also look at the attached dmesg.txt. >> this is an core i3 system with 2x2TB BTRFS RAID1 and lots of >> home directories and snapshots. >> >> I'm going to test this patch on twenty more computers but with >> smaller HDDs and less files and see if it helps to speed up their >> boot times. >> > > Ok looks like you are running into a different problem. Could you maybe run > bootchart and upload the resulting png somewhere so I can look and see what > all > is running while you boot? Thanks, http://aam.mathematik.uni-freiburg.de/IAM/homepages/ainan/bootchart.png i have tried your patch now on the twenty more computers i mentioned and still it takes a minute to remount rw "/" on those, even after several reboots. Ahmet -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Boot speed/mount time regression with 3.4.0-rc2
On Tue, Apr 10, 2012 at 5:16 PM, Josef Bacik wrote: > On Mon, Apr 09, 2012 at 05:20:46PM -0400, Calvin Walton wrote: >> On Mon, 2012-04-09 at 16:54 -0400, Josef Bacik wrote: >> > On Mon, Apr 09, 2012 at 01:10:04PM -0400, Calvin Walton wrote: >> > > On Mon, 2012-04-09 at 11:53 -0400, Calvin Walton wrote: >> > > > Hi, >> > > > >> > > > I have a system that's using a dracut-generated initramfs to mount a >> > > > btrfs root. After upgrading to kernel 3.4.0-rc2 to test it out, I've >> > > > noticed that the process of mounting the root filesystem takes much >> > > > longer with 3.4.0-rc2 than it did with 3.3.1 - nearly 30 seconds >> > > > slower! >> >> > > And the bisect results are in: >> > > 285ff5af6ce358e73f53b55c9efadd4335f4c2ff is the first bad commit >> > > commit 285ff5af6ce358e73f53b55c9efadd4335f4c2ff >> > > Author: Josef Bacik >> > > Date: Fri Jan 13 15:27:45 2012 -0500 >> > > >> > > Btrfs: remove the ideal caching code> >> > >> > Ok can you give this a whirl? You are going to have to boot/reboot a few >> > times >> > to let the cache get re-generated again to make sure it's taken effect, but >> > hopefully this will help out. Thanks, >> >> Unfortunately, it doesn't seem to help. Even after 3 or 4 reboots with >> this patch applied I'm still seeing the same delay. >> > > Ok drop that previous patch and give this one a whirl, it helped on my laptop. > This is only half of the problem AFAICS, but it's the easier half to fix, in > the meantime I need to lock down why we're not writing out cache for a bunch > of > block groups, but thats trickier since the messages I need are spit out while > I'm shutting down, so I need to get creative. Let me know if/how much this > helps. Thanks, i have tried your patch and my system still needs several minutes to boot until it can be used. Also tried to reboot several times - it doesn't look like its getting better. The last thing the system does when its shutting down is a read-only remount of "/" so no umount. Booting was much faster before i pulled for-linus a few weeks ago but i couldn't find the time to bisect it yet .. please also look at the attached dmesg.txt. this is an core i3 system with 2x2TB BTRFS RAID1 and lots of home directories and snapshots. I'm going to test this patch on twenty more computers but with smaller HDDs and less files and see if it helps to speed up their boot times. Ahmet [ 28.155134] btrfs: use lzo compression [ 239.698516] INFO: task btrfs-transacti:997 blocked for more than 120 seconds. [ 239.698520] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 239.698524] btrfs-transacti D 88023fcd0a00 0 997 2 0x [ 239.698530] 880233c1bce0 0046 880234bcc9c0 00010a00 [ 239.698535] 880233c1bfd8 880233c1a000 00010a00 4000 [ 239.698539] 880233c1bfd8 00010a00 880236d384c0 880234bcc9c0 [ 239.698544] Call Trace: [ 239.698554] [] ? idle_balance+0x108/0x130 [ 239.698580] [] ? do_chunk_alloc.clone.68+0x74/0x400 [btrfs] [ 239.698602] [] ? btrfs_find_ref_cluster+0x33/0x170 [btrfs] [ 239.698618] [] ? btrfs_reduce_alloc_profile+0x53/0x120 [btrfs] [ 239.698624] [] ? cache_alloc_refill+0x7c/0x280 [ 239.698630] [] schedule+0x3a/0x50 [ 239.698634] [] schedule_timeout+0x1c5/0x220 [ 239.698638] [] ? mutex_lock+0x19/0x40 [ 239.698643] [] ? prepare_to_wait+0x5b/0x90 [ 239.698662] [] btrfs_commit_transaction+0x30f/0x9a0 [btrfs] [ 239.698680] [] ? join_transaction.clone.26+0x22/0x2e0 [btrfs] [ 239.698685] [] ? wake_up_bit+0x40/0x40 [ 239.698701] [] transaction_kthread+0x25b/0x2d0 [btrfs] [ 239.698717] [] ? btrfs_alloc_root+0x30/0x30 [btrfs] [ 239.698733] [] ? btrfs_alloc_root+0x30/0x30 [btrfs] [ 239.698737] [] kthread+0x96/0xa0 [ 239.698742] [] kernel_thread_helper+0x4/0x10 [ 239.698746] [] ? kthread_freezable_should_stop+0x70/0x70 [ 239.698750] [] ? gs_change+0xb/0xb [ 359.367274] INFO: task btrfs-transacti:997 blocked for more than 120 seconds. [ 359.367278] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 359.367281] btrfs-transacti D 88023fcd0a00 0 997 2 0x [ 359.367288] 880233c1bce0 0046 880234bcc9c0 00010a00 [ 359.367292] 880233c1bfd8 880233c1a000 00010a00 4000 [ 359.367297] 880233c1bfd8 00010a00 880236d384c0 880234bcc9c0 [ 359.367302] Call Trace: [ 359.367311] [] ? idle_balance+0x108/0x130 [ 359.367341] [] ? do_chunk_alloc.clone.68+0x74/0x400 [btrfs] [ 359.367361] [] ? btrfs_find_ref_cluster+0x33/0x170 [btrfs] [ 359.367376] [] ? btrfs_reduce_alloc_profile+0x53/0x120 [btrfs] [ 359.367382] [] ? cache_alloc_refill+0x7c/0x280 [ 359.367387] [] schedule+0x3a/0x50 [ 359.367392] [] schedule_timeout+0x1c5/0x220 [ 359.367395] [] ? mutex_lock+0x19/0x40 [ 359.367401] [] ? prepare_to_wait+0x5b/0x90 [ 359.367418] [
Re: Premature ENOSPC only with zlib Compression
On Thu, Feb 16, 2012 at 6:55 PM, Chris Mason wrote: > On Thu, Feb 16, 2012 at 04:05:40PM +0100, Ahmet Inan wrote: >> On Thu, Feb 16, 2012 at 11:31 AM, Chris Samuel wrote: >> > On Thursday 16 February 2012 19:06:52 Ahmet Inan wrote: >> > >> >> and got ENOSPC again :( >> >> >> >> so it doesnt matter if lzo or zlib: ENOSPC with compression >> >> enabled! >> >> >> >> my kernel: vanilla 3.2.5 >> > >> > I know that there has been an ENOSPC fix go in to the 3.3 RC kernels >> > since the 3.2 release, any chance you'd be able to see if it still >> > happens with the current 3.3 RC ? >> >> just tested 3.3.0-rc3, and it works like a charm! >> >> but there are lots of issues with external packages not compiling with 3.3, >> why i wont be able to deploy it to our systems here yet. >> >> could this one bugfix be backported to 3.2? > > All of the btrfs patches in 3.3-rc are against 3.2 in my git tree: > > git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus thank you very much, it works as it should. had to couse some traffic on kernel.org but finally i got a local linux repo! is for-linus considered stable and can i pull regularly? here what i did for those interested: cd /usr/src/ rm linux git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git linux cd linux git checkout linux-3.2.y git pull git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus cp /boot/config .config make menuconfig # look around, save and exit make -j8 make modules_install make install and thats it, 3.2.6 stable kernel from git with latest for-linus btrfs from git Ahmet -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Premature ENOSPC only with zlib Compression
On Thu, Feb 16, 2012 at 11:31 AM, Chris Samuel wrote: > On Thursday 16 February 2012 19:06:52 Ahmet Inan wrote: > >> and got ENOSPC again :( >> >> so it doesnt matter if lzo or zlib: ENOSPC with compression >> enabled! >> >> my kernel: vanilla 3.2.5 > > I know that there has been an ENOSPC fix go in to the 3.3 RC kernels > since the 3.2 release, any chance you'd be able to see if it still > happens with the current 3.3 RC ? just tested 3.3.0-rc3, and it works like a charm! but there are lots of issues with external packages not compiling with 3.3, why i wont be able to deploy it to our systems here yet. could this one bugfix be backported to 3.2? here are some results from my unsquashfs'ing just to show how awesome btrfs with lzo is. these results are _not_ from an ssd btw! (spinning disks, rebooted between tests) # time unsquashfs /dev/nbd0 Parallel unsquashfs: Using 8 processors 792630 inodes (870469 blocks) to write [==/] 870469/870469 100% created 759645 files created 67271 directories created 27250 symlinks created 5141 devices created 0 fifos Timing Results: zlib: real6m38.586s user2m16.021s sys 0m31.138s lzo: real1m53.429s user1m55.622s sys 0m46.664s no compression: real1m16.975s user1m47.173s sys 0m53.520s complete system installation in under 5minutes, that simply rocks! Ahmet -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Premature ENOSPC only with zlib Compression
On Wed, Jan 18, 2012 at 5:13 PM, Mitch Harder wrote: > I have a Btrfs partition that is reliably reproducing premature ENOSPC > when restoring the disk from a tar file, but it is only happening with > zlib compression (lzo or no compression proceeds normally). Ive just unsquashfs'ed an 5GiB image on a fresh btrfs and a much faster System using: compress=lzo,noatime and got ENOSPC again :( so it doesnt matter if lzo or zlib: ENOSPC with compression enabled! my kernel: vanilla 3.2.5 for testing, i suggest making a squashfs of your whole system "/" and unsquashfs that image into a new subvolume. this really seems to trigger this ENOSPC problem, because it creates the files fast enough. Ahmet -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Premature ENOSPC only with zlib Compression
On Wed, Jan 18, 2012 at 5:13 PM, Mitch Harder wrote: > I have a Btrfs partition that is reliably reproducing premature ENOSPC > when restoring the disk from a tar file, but it is only happening with > zlib compression (lzo or no compression proceeds normally). thank you very much for this info. i did the same thing: compress=lzo now i can do a full system installation to an fresh btrfs filesystem without "--bwlimit 4096" to rsync again! my kernel: vanilla 3.2.1 Ahmet -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
