Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-08 Thread Jeff Mahoney
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hans Reiser wrote:
 Guys, if you run the kernel under a debugger, and get it to where you
 see the excessive CPU usage, and then start stepping through the bitmap
 code, I am sure it will be very obvious what the error is.  Can anyone
 do that for us?  Jeff?

Apologies to everyone CC'd who've already seen this message. It was
bounced from the namesys servers and I wanted to preserve the CC list.

***

Mike sent me a copy of the metadata and I am now able to reproduce
locally. My profiling looks like this:

samples  %  image name  app namesymbol name
148596  17.8573 reiserfs.ko reiserfsreiserfs_in_journal
58194   6.9934  reiserfs.ko reiserfssearch_by_key
38937   4.6792  vmlinux vmlinux memmove
38783   4.6607  reiserfs.ko reiserfsscan_bitmap_block
38466   4.6226  jbd jbd (no symbols)
23249   2.7939  vmlinux vmlinux __find_get_block
18196   2.1867  vmlinux vmlinux tty_write
17734   2.1312  vmlinux vmlinux do_ioctl
17293   2.0782  looploop(no symbols)
15400   1.8507  vmlinux vmlinux cond_resched_lock
14836   1.7829  vmlinux vmlinux copy_user_generic_c
14143   1.6996  reiserfs.ko reiserfsdo_journal_end
13638   1.6389  vmlinux vmlinux find_next_zero_bit
13236   1.5906  vmlinux vmlinux default_llseek
12925   1.5532  vmlinux vmlinux bit_waitqueue
89211.0721  vmlinux vmlinux __delay


Hans -

My speculation about the bitmaps being fragmented was right on. I wrote
a quick little script to parse the output of debugreiserfs -m and report
on the frequency of different window sizes. Windows of 1-31 blocks are
extremely common, accounting for 99.8% of all free windows. The problem
is that in my testing, where I made the allocator report the size of
allocation requests, the most common request was for a window of 32 blocks.

What's happening is that we keep finding windows that are too small,
which results in a lot of wasted effort. The cycle goes like this:

if (unfm  is_block_in_journal(s, bmap_n, *beg, beg))
continue;
/* first zero bit found; we check next bits */
for (end = *beg + 1;; end++) {
if (end = *beg + max || end = boundary
|| reiserfs_test_le_bit(end, bi-bh-b_data)) {
next = end;
break;
}
/* finding the other end of zero bit window requires
 * looking into journal structures (in
 * case of searching for free blocks for unformatted nodes) */
if (unfm  is_block_in_journal(s, bmap_n, end, next))
break;
}

If the window is too small, we end up looping up to the top and try to
find another one. Since the overwhelming majority of the windows are too
small, we go through just about all the bitmaps without backing off the
window size.

- -Jeff

- --
Jeff Mahoney
SUSE Labs
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org

iD8DBQFEr/1QLPWxlyuTD7IRApS7AJ9FgnAIGagxeWLDxpiixZt3bW7RmQCgoYwS
+ycgwRw+I6mVATMNTeuLPQ8=
=67kl
-END PGP SIGNATURE-


Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-08 Thread Hans Reiser
By 8 iterations, I mean 8 bitmaps scanned.


Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-08 Thread Hans Reiser
So limit the number of iterations of rejecting windows that are too
small.  Say, 8.

Hans

Jeff Mahoney wrote:


 What's happening is that we keep finding windows that are too small,
 which results in a lot of wasted effort. The cycle goes like this:

 if (unfm  is_block_in_journal(s, bmap_n, *beg, beg))
 continue;
 /* first zero bit found; we check next bits */
 for (end = *beg + 1;; end++) {
 if (end = *beg + max || end = boundary
 || reiserfs_test_le_bit(end, bi-bh-b_data)) {
 next = end;
 break;
 }
 /* finding the other end of zero bit window requires
  * looking into journal structures (in
  * case of searching for free blocks for unformatted nodes) */
 if (unfm  is_block_in_journal(s, bmap_n, end, next))
 break;
 }

 If the window is too small, we end up looping up to the top and try to
 find another one. Since the overwhelming majority of the windows are too
 small, we go through just about all the bitmaps without backing off the
 window size.

 -Jeff

 --
 Jeff Mahoney
 SUSE Labs