Re: [btrfs-transacti] btrfs-endio-wri] - WAS: Re: [btrfs-delalloc-]

2011-07-06 Thread Proskurin Kirill

On 07/01/2011 06:07 PM, Josef Bacik wrote:


I found what my btrfs partition now is really slow. Something wrong
happend. I try on 2.6.32 and 2.6.39 - same result. It is because
partition is almost full?

I run a simple cycle to clean some old files:
for i in `cat /tmp/delit`; do rm -f $i ; done

And it is takes about 5-10 second per file to delete.
I get sysrq+w while rm is work - it is attached.



Just a shot in the dark, but will you give this a shot and let me know
how it goes?  Thanks


Sorry for late answer. Im afraid I can`t test it now - then problem 
occurs I have no time to wait and format partition. But I will save 
this patch and if I run at this problem again I will try it and report 
here.


Thanks for your help anyway.

--
Best regards,
Proskurin Kirill
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [btrfs-transacti] btrfs-endio-wri] - WAS: Re: [btrfs-delalloc-]

2011-07-01 Thread Proskurin Kirill

On 06/30/2011 09:13 PM, Josef Bacik wrote:
 On 06/30/2011 10:12 AM, Proskurin Kirill wrote:
 On 06/29/2011 08:14 PM, Josef Bacik wrote:
 Ok - I upgrade to 2.6.39-2 but it is seems to all things get worse.
 Now I see [btrfs-transacti]   btrfs-endio-wri] 80-100% all the 
time and

 io performance looks like lower then before.

 Our scribe daemon in state D most of the time with half of a normal
 load. Only kernel was changed.

 Any performance tune recommendation?


 Can you get sysrq+w while this problem is happening so we can see whats
 going on?  Thanks,

 I attached sysrq+w. Hope it helps.


 Heh so it just looks like you are writing a bunch of stuff, that's not
 particularly helpful.  What does this program do generically?  How full
 is your fs?  Do you have snapshots/subvolumes?  If so how many?  Thanks,

I found what my btrfs partition now is really slow. Something wrong 
happend. I try on 2.6.32 and 2.6.39 - same result. It is because 
partition is almost full?


I run a simple cycle to clean some old files:
for i in `cat /tmp/delit`; do rm -f $i ; done

And it is takes about 5-10 second per file to delete.
I get sysrq+w while rm is work - it is attached.

--
Best regards,
Proskurin Kirill
SysRq : Show Blocked State
  taskPC stack   pid father
rmD 0002 0  5662   4515 0x0080
 880110d916d8 0082  a024d97b
 880067324e40 45e1ab10 3000 00010015dcde
 88013b6145f8 880110d91fd8 f598 88013b6145f8
Call Trace:
 [a024d97b] ? md_raid5_unplug_device+0x7b/0x100 [raid456]
 [8110d310] ? sync_page+0x0/0x50
 [814db013] io_schedule+0x73/0xc0
 [8110d34d] sync_page+0x3d/0x50
 [814db87f] __wait_on_bit+0x5f/0x90
 [8110d503] wait_on_page_bit+0x73/0x80
 [8108e140] ? wake_bit_function+0x0/0x50
 [a0203512] ? submit_one_bio+0x82/0xa0 [btrfs]
 [a020a30a] read_extent_buffer_pages+0x39a/0x430 [btrfs]
 [a01dd0d0] ? btree_get_extent+0x0/0x1b0 [btrfs]
 [a01e069d] btree_read_extent_buffer_pages+0x5d/0xc0 [btrfs]
 [a01e077c] read_tree_block+0x3c/0x60 [btrfs]
 [a01c2251] read_block_for_search+0xf1/0x3b0 [btrfs]
 [a01ca079] btrfs_search_slot+0x309/0x8a0 [btrfs]
 [a0204002] ? insert_state+0x102/0x180 [btrfs]
 [a01d5d0c] lookup_inline_extent_backref+0xbc/0x3f0 [btrfs]
 [a01d6406] __btrfs_free_extent+0xd6/0x730 [btrfs]
 [a01d6f3e] run_clustered_refs+0x4de/0x7f0 [btrfs]
 [a0223f00] ? btrfs_find_ref_cluster+0x70/0x180 [btrfs]
 [a01d7318] btrfs_run_delayed_refs+0xc8/0x220 [btrfs]
 [a01e4c0f] __btrfs_end_transaction+0x6f/0x240 [btrfs]
 [a01e4e35] btrfs_end_transaction+0x15/0x20 [btrfs]
 [a01ec077] btrfs_delete_inode+0x197/0x200 [btrfs]
 [a01ebee0] ? btrfs_delete_inode+0x0/0x200 [btrfs]
 [8118cfbe] generic_delete_inode+0xde/0x1d0
 [8118d115] generic_drop_inode+0x65/0x80
 [a01e593d] btrfs_drop_inode+0x3d/0x50 [btrfs]
 [8118bf82] iput+0x62/0x70
 [81183332] do_unlinkat+0x112/0x1c0
 [810d1ac2] ? audit_syscall_entry+0x272/0x2a0
 [8100c635] ? math_state_restore+0x45/0x60
 [81183542] sys_unlinkat+0x22/0x40
 [8100b172] system_call_fastpath+0x16/0x1b
Sched Debug Version: v0.09, 2.6.32-131.2.1.el6.x86_64 #1
now at 1733266.774675 msecs
  .jiffies : 4296400562
  .sysctl_sched_latency: 20.00
  .sysctl_sched_min_granularity: 4.00
  .sysctl_sched_wakeup_granularity : 4.00
  .sysctl_sched_child_runs_first   : 0.00
  .sysctl_sched_features   : 15471
  .sysctl_sched_tunable_scaling: 1 (logaritmic)

cpu#0, 2133.291 MHz
  .nr_running: 0
  .load  : 0
  .nr_switches   : 372148
  .nr_load_updates   : 89485
  .nr_uninterruptible: 0
  .next_balance  : 4296.400777
  .curr-pid : 0
  .clock : 1733558.607055
  .cpu_load[0]   : 0
  .cpu_load[1]   : 0
  .cpu_load[2]   : 0
  .cpu_load[3]   : 0
  .cpu_load[4]   : 6
  .yld_count : 0
  .sched_switch  : 0
  .sched_count   : 457866
  .sched_goidle  : 145476
  .avg_idle  : 994900
  .ttwu_count: 224174
  .ttwu_local: 204302
  .bkl_count : 4650

cfs_rq[0]:/
  .exec_clock: 13532.801929
  .MIN_vruntime  : 0.01
  .min_vruntime  : 10873.718018
  .max_vruntime  : 0.01
  .spread: 0.00
  .spread0   : 0.00
  .nr_running

Re: [btrfs-transacti] btrfs-endio-wri] - WAS: Re: [btrfs-delalloc-]

2011-06-30 Thread Proskurin Kirill

On 06/30/2011 09:13 PM, Josef Bacik wrote:

On 06/30/2011 10:12 AM, Proskurin Kirill wrote:

On 06/29/2011 08:14 PM, Josef Bacik wrote:

Ok - I upgrade to 2.6.39-2 but it is seems to all things get worse.
Now I see [btrfs-transacti]   btrfs-endio-wri] 80-100% all the time and
io performance looks like lower then before.

Our scribe daemon in state D most of the time with half of a normal
load. Only kernel was changed.

Any performance tune recommendation?



Can you get sysrq+w while this problem is happening so we can see whats
going on?  Thanks,


I attached sysrq+w. Hope it helps.



Heh so it just looks like you are writing a bunch of stuff, that's not
particularly helpful.  What does this program do generically?  How full
is your fs?  Do you have snapshots/subvolumes?  If so how many?  Thanks,


Scribe is taling logs by network. ~300 hosts send it to it and scribe 
write them to fs. I use zlib compression on btrfs partition.


Mount options is noatime,noacl,compress-force=zlib

iostat -x 1 tells me what util of btrfs partition is from 50% to ~90% 
all the time.


Btrfs partition is near full:
Size  Used Avail Use%
28T   26T  2.1T  93%

I don`t use snapshots/subvolumes at this time so none.

--
Best regards,
Proskurin Kirill
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [btrfs-transacti] btrfs-endio-wri] - WAS: Re: [btrfs-delalloc-]

2011-06-30 Thread Proskurin Kirill

On 06/30/2011 09:13 PM, Josef Bacik wrote:

On 06/30/2011 10:12 AM, Proskurin Kirill wrote:

On 06/29/2011 08:14 PM, Josef Bacik wrote:

Ok - I upgrade to 2.6.39-2 but it is seems to all things get worse.
Now I see [btrfs-transacti]   btrfs-endio-wri] 80-100% all the time and
io performance looks like lower then before.

Our scribe daemon in state D most of the time with half of a normal
load. Only kernel was changed.

Any performance tune recommendation?



Can you get sysrq+w while this problem is happening so we can see whats
going on?  Thanks,


I attached sysrq+w. Hope it helps.



Heh so it just looks like you are writing a bunch of stuff, that's not
particularly helpful.  What does this program do generically?  How full
is your fs?  Do you have snapshots/subvolumes?  If so how many?  Thanks,


I found what my btrfs partition now is really slow. Something wrong 
happend. I try on 2.6.32 and 2.6.39 - same result. It is because 
partition is almost full?


I run a simple cycle to clean some old files:
for i in `cat /tmp/delit`; do rm -f $i ; done

And it is takes about 5-10 second per file to delete.
I get sysrq+w while rm is work - it is attached.

--
Best regards,
Proskurin Kirill


dmesg.rm
Description: application/vnd.rn-realmedia


[btrfs-transacti] btrfs-endio-wri] - WAS: Re: [btrfs-delalloc-]

2011-06-29 Thread Proskurin Kirill

On 06/27/2011 05:21 PM, Hubert Kario wrote:

On Monday 27 of June 2011 11:04:06 Proskurin Kirill wrote:

Hello all.

What we have:
SL6 - kernel 2.6.32-131.2.1.el6.x86_64
btrfs on mdadm RAID5 with 8 HDD - 27T partition.

I see this at top:
1182 root  20   0 000 R 100.0  0.0  16:39.73
[btrfs-delalloc-]

And LA is grow. What is this and how can I fix it?


delalloc is a delayed allocation kernel thread -- it probably means something
is writing large amounts of data to the file system

2.6.32 is *old* as far as btrfs is concerned, there have been many bugs fixed
and performance improvements since


Ok - I upgrade to 2.6.39-2 but it is seems to all things get worse.
Now I see [btrfs-transacti]  btrfs-endio-wri] 80-100% all the time and 
io performance looks like lower then before.


Our scribe daemon in state D most of the time with half of a normal 
load. Only kernel was changed.


Any performance tune recommendation?

--
Best regards,
Proskurin Kirill
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[btrfs-delalloc-]

2011-06-27 Thread Proskurin Kirill

Hello all.

What we have:
SL6 - kernel 2.6.32-131.2.1.el6.x86_64
btrfs on mdadm RAID5 with 8 HDD - 27T partition.

I see this at top:
1182 root  20   0 000 R 100.0  0.0  16:39.73 
[btrfs-delalloc-]


And LA is grow. What is this and how can I fix it?

--
Best regards,
Proskurin Kirill
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


compression ratio

2011-06-20 Thread Proskurin Kirill

Hello all.

I`m new to btrfs and do some testing now.

What we have:
SL6 - kernel 2.6.32-131.2.1.el6.x86_64
mdadm RAID5 with 8 HDD - 27T partition.

Mount options is noatime,noacl,compress-force
I use scribe daemon to copy log files from 200 hosts to that partition 
for stress testing.


But I found what compression ratio is really small.
Partition is full of regular log files - plain text. Most of them a big 
ones(10-20Gb) and they grow in realtime.


du -hs /logs/
1.1T   /logs/

And compressed size is(df -h):
908G

Regular gzip do a significant more ratio.

There is a problem here? File size? Realtime grow of a files?


--
Best regards,
Proskurin Kirill
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: compression ratio

2011-06-20 Thread Proskurin Kirill

On 06/20/2011 06:34 PM, Helmut Hullen wrote:

Hallo, Proskurin,

Du meintest am 20.06.11:


I`m new to btrfs and do some testing now.



What we have:
SL6 - kernel 2.6.32-131.2.1.el6.x86_64
mdadm RAID5 with 8 HDD - 27T partition.


You should take a newer kernel. On my system I needed 2.6.38.5 and newer
for btrfs; older kernels lead to unreadable partitions.


What kernel version is most recommended at this time?


--
Best regards,
Proskurin Kirill
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html