Re: [btrfs-transacti] btrfs-endio-wri] - WAS: Re: [btrfs-delalloc-]
On 07/01/2011 06:07 PM, Josef Bacik wrote: I found what my btrfs partition now is really slow. Something wrong happend. I try on 2.6.32 and 2.6.39 - same result. It is because partition is almost full? I run a simple cycle to clean some old files: for i in `cat /tmp/delit`; do rm -f $i ; done And it is takes about 5-10 second per file to delete. I get sysrq+w while rm is work - it is attached. Just a shot in the dark, but will you give this a shot and let me know how it goes? Thanks Sorry for late answer. Im afraid I can`t test it now - then problem occurs I have no time to wait and format partition. But I will save this patch and if I run at this problem again I will try it and report here. Thanks for your help anyway. -- Best regards, Proskurin Kirill -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [btrfs-transacti] btrfs-endio-wri] - WAS: Re: [btrfs-delalloc-]
On 06/30/2011 09:13 PM, Josef Bacik wrote: On 06/30/2011 10:12 AM, Proskurin Kirill wrote: On 06/29/2011 08:14 PM, Josef Bacik wrote: Ok - I upgrade to 2.6.39-2 but it is seems to all things get worse. Now I see [btrfs-transacti] btrfs-endio-wri] 80-100% all the time and io performance looks like lower then before. Our scribe daemon in state D most of the time with half of a normal load. Only kernel was changed. Any performance tune recommendation? Can you get sysrq+w while this problem is happening so we can see whats going on? Thanks, I attached sysrq+w. Hope it helps. Heh so it just looks like you are writing a bunch of stuff, that's not particularly helpful. What does this program do generically? How full is your fs? Do you have snapshots/subvolumes? If so how many? Thanks, I found what my btrfs partition now is really slow. Something wrong happend. I try on 2.6.32 and 2.6.39 - same result. It is because partition is almost full? I run a simple cycle to clean some old files: for i in `cat /tmp/delit`; do rm -f $i ; done And it is takes about 5-10 second per file to delete. I get sysrq+w while rm is work - it is attached. -- Best regards, Proskurin Kirill SysRq : Show Blocked State taskPC stack pid father rmD 0002 0 5662 4515 0x0080 880110d916d8 0082 a024d97b 880067324e40 45e1ab10 3000 00010015dcde 88013b6145f8 880110d91fd8 f598 88013b6145f8 Call Trace: [a024d97b] ? md_raid5_unplug_device+0x7b/0x100 [raid456] [8110d310] ? sync_page+0x0/0x50 [814db013] io_schedule+0x73/0xc0 [8110d34d] sync_page+0x3d/0x50 [814db87f] __wait_on_bit+0x5f/0x90 [8110d503] wait_on_page_bit+0x73/0x80 [8108e140] ? wake_bit_function+0x0/0x50 [a0203512] ? submit_one_bio+0x82/0xa0 [btrfs] [a020a30a] read_extent_buffer_pages+0x39a/0x430 [btrfs] [a01dd0d0] ? btree_get_extent+0x0/0x1b0 [btrfs] [a01e069d] btree_read_extent_buffer_pages+0x5d/0xc0 [btrfs] [a01e077c] read_tree_block+0x3c/0x60 [btrfs] [a01c2251] read_block_for_search+0xf1/0x3b0 [btrfs] [a01ca079] btrfs_search_slot+0x309/0x8a0 [btrfs] [a0204002] ? insert_state+0x102/0x180 [btrfs] [a01d5d0c] lookup_inline_extent_backref+0xbc/0x3f0 [btrfs] [a01d6406] __btrfs_free_extent+0xd6/0x730 [btrfs] [a01d6f3e] run_clustered_refs+0x4de/0x7f0 [btrfs] [a0223f00] ? btrfs_find_ref_cluster+0x70/0x180 [btrfs] [a01d7318] btrfs_run_delayed_refs+0xc8/0x220 [btrfs] [a01e4c0f] __btrfs_end_transaction+0x6f/0x240 [btrfs] [a01e4e35] btrfs_end_transaction+0x15/0x20 [btrfs] [a01ec077] btrfs_delete_inode+0x197/0x200 [btrfs] [a01ebee0] ? btrfs_delete_inode+0x0/0x200 [btrfs] [8118cfbe] generic_delete_inode+0xde/0x1d0 [8118d115] generic_drop_inode+0x65/0x80 [a01e593d] btrfs_drop_inode+0x3d/0x50 [btrfs] [8118bf82] iput+0x62/0x70 [81183332] do_unlinkat+0x112/0x1c0 [810d1ac2] ? audit_syscall_entry+0x272/0x2a0 [8100c635] ? math_state_restore+0x45/0x60 [81183542] sys_unlinkat+0x22/0x40 [8100b172] system_call_fastpath+0x16/0x1b Sched Debug Version: v0.09, 2.6.32-131.2.1.el6.x86_64 #1 now at 1733266.774675 msecs .jiffies : 4296400562 .sysctl_sched_latency: 20.00 .sysctl_sched_min_granularity: 4.00 .sysctl_sched_wakeup_granularity : 4.00 .sysctl_sched_child_runs_first : 0.00 .sysctl_sched_features : 15471 .sysctl_sched_tunable_scaling: 1 (logaritmic) cpu#0, 2133.291 MHz .nr_running: 0 .load : 0 .nr_switches : 372148 .nr_load_updates : 89485 .nr_uninterruptible: 0 .next_balance : 4296.400777 .curr-pid : 0 .clock : 1733558.607055 .cpu_load[0] : 0 .cpu_load[1] : 0 .cpu_load[2] : 0 .cpu_load[3] : 0 .cpu_load[4] : 6 .yld_count : 0 .sched_switch : 0 .sched_count : 457866 .sched_goidle : 145476 .avg_idle : 994900 .ttwu_count: 224174 .ttwu_local: 204302 .bkl_count : 4650 cfs_rq[0]:/ .exec_clock: 13532.801929 .MIN_vruntime : 0.01 .min_vruntime : 10873.718018 .max_vruntime : 0.01 .spread: 0.00 .spread0 : 0.00 .nr_running
Re: [btrfs-transacti] btrfs-endio-wri] - WAS: Re: [btrfs-delalloc-]
On 06/30/2011 09:13 PM, Josef Bacik wrote: On 06/30/2011 10:12 AM, Proskurin Kirill wrote: On 06/29/2011 08:14 PM, Josef Bacik wrote: Ok - I upgrade to 2.6.39-2 but it is seems to all things get worse. Now I see [btrfs-transacti] btrfs-endio-wri] 80-100% all the time and io performance looks like lower then before. Our scribe daemon in state D most of the time with half of a normal load. Only kernel was changed. Any performance tune recommendation? Can you get sysrq+w while this problem is happening so we can see whats going on? Thanks, I attached sysrq+w. Hope it helps. Heh so it just looks like you are writing a bunch of stuff, that's not particularly helpful. What does this program do generically? How full is your fs? Do you have snapshots/subvolumes? If so how many? Thanks, Scribe is taling logs by network. ~300 hosts send it to it and scribe write them to fs. I use zlib compression on btrfs partition. Mount options is noatime,noacl,compress-force=zlib iostat -x 1 tells me what util of btrfs partition is from 50% to ~90% all the time. Btrfs partition is near full: Size Used Avail Use% 28T 26T 2.1T 93% I don`t use snapshots/subvolumes at this time so none. -- Best regards, Proskurin Kirill -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [btrfs-transacti] btrfs-endio-wri] - WAS: Re: [btrfs-delalloc-]
On 06/30/2011 09:13 PM, Josef Bacik wrote: On 06/30/2011 10:12 AM, Proskurin Kirill wrote: On 06/29/2011 08:14 PM, Josef Bacik wrote: Ok - I upgrade to 2.6.39-2 but it is seems to all things get worse. Now I see [btrfs-transacti] btrfs-endio-wri] 80-100% all the time and io performance looks like lower then before. Our scribe daemon in state D most of the time with half of a normal load. Only kernel was changed. Any performance tune recommendation? Can you get sysrq+w while this problem is happening so we can see whats going on? Thanks, I attached sysrq+w. Hope it helps. Heh so it just looks like you are writing a bunch of stuff, that's not particularly helpful. What does this program do generically? How full is your fs? Do you have snapshots/subvolumes? If so how many? Thanks, I found what my btrfs partition now is really slow. Something wrong happend. I try on 2.6.32 and 2.6.39 - same result. It is because partition is almost full? I run a simple cycle to clean some old files: for i in `cat /tmp/delit`; do rm -f $i ; done And it is takes about 5-10 second per file to delete. I get sysrq+w while rm is work - it is attached. -- Best regards, Proskurin Kirill dmesg.rm Description: application/vnd.rn-realmedia
[btrfs-transacti] btrfs-endio-wri] - WAS: Re: [btrfs-delalloc-]
On 06/27/2011 05:21 PM, Hubert Kario wrote: On Monday 27 of June 2011 11:04:06 Proskurin Kirill wrote: Hello all. What we have: SL6 - kernel 2.6.32-131.2.1.el6.x86_64 btrfs on mdadm RAID5 with 8 HDD - 27T partition. I see this at top: 1182 root 20 0 000 R 100.0 0.0 16:39.73 [btrfs-delalloc-] And LA is grow. What is this and how can I fix it? delalloc is a delayed allocation kernel thread -- it probably means something is writing large amounts of data to the file system 2.6.32 is *old* as far as btrfs is concerned, there have been many bugs fixed and performance improvements since Ok - I upgrade to 2.6.39-2 but it is seems to all things get worse. Now I see [btrfs-transacti] btrfs-endio-wri] 80-100% all the time and io performance looks like lower then before. Our scribe daemon in state D most of the time with half of a normal load. Only kernel was changed. Any performance tune recommendation? -- Best regards, Proskurin Kirill -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[btrfs-delalloc-]
Hello all. What we have: SL6 - kernel 2.6.32-131.2.1.el6.x86_64 btrfs on mdadm RAID5 with 8 HDD - 27T partition. I see this at top: 1182 root 20 0 000 R 100.0 0.0 16:39.73 [btrfs-delalloc-] And LA is grow. What is this and how can I fix it? -- Best regards, Proskurin Kirill -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: compression ratio
On 06/20/2011 06:34 PM, Helmut Hullen wrote: Hallo, Proskurin, Du meintest am 20.06.11: I`m new to btrfs and do some testing now. What we have: SL6 - kernel 2.6.32-131.2.1.el6.x86_64 mdadm RAID5 with 8 HDD - 27T partition. You should take a newer kernel. On my system I needed 2.6.38.5 and newer for btrfs; older kernels lead to unreadable partitions. What kernel version is most recommended at this time? -- Best regards, Proskurin Kirill -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
