Re: ReiserFS v3 choking when free space falls below 10% - FIXED

2006-08-18 Thread Bernd Butscheidt
Hello, I'm hope I'm doing this right here, The question is: Is or since when is the the patch which helped Mike Benoit integrated to the kernel source? Greetings bernd_b

Re: ReiserFS v3 choking when free space falls below 10% - FIXED

2006-08-18 Thread Mike Benoit
Interesting that you sent this today. Just last night I really started to notice slow down on my MythTV box. It had been going on for a while, but I finally got fed up last night enough to look in to it, and what I found was this: 0 1324 4904 10184 6732800 1408 1060 1033 1393

Re: ReiserFS v3 choking when free space falls below 10% - FIXED

2006-08-18 Thread Mike Benoit
Sorry, here is the vmstat output with column headers. procs ---memory-- ---swap-- -io --system-- cpu r b swpd free buff cache si sobibo incs us sy id wa 1 0324 5976 10196 6635600 1722 1831 759 1496 89 4 1 7 0 1

Re: ReiserFS v3 choking when free space falls below 10% - FIXED

2006-07-26 Thread Hans Reiser
David Masover wrote: As a future MythTV user a bit late to this discussion, I'm curious -- was this Reiser3 or 4? Are there any known MythTV issues with v4? I say this because the box with my capture card is running on a Reiser4 root right now... I think you get to be the one to tell

Re: ReiserFS v3 choking when free space falls below 10% - FIXED

2006-07-25 Thread David Masover
Mike Benoit wrote: Thanks for all your hard work, I'm sure many other MythTV users will be appreciate it. As a future MythTV user a bit late to this discussion, I'm curious -- was this Reiser3 or 4? Are there any known MythTV issues with v4? I say this because the box with my capture card is

Re: ReiserFS v3 choking when free space falls below 10% - FIXED

2006-07-25 Thread Mike Benoit
On Tue, 2006-07-25 at 19:10 -0500, David Masover wrote: Mike Benoit wrote: Thanks for all your hard work, I'm sure many other MythTV users will be appreciate it. As a future MythTV user a bit late to this discussion, I'm curious -- was this Reiser3 or 4? Are there any known MythTV

Re: ReiserFS v3 choking when free space falls below 10% - FIXED

2006-07-24 Thread Mike Benoit
I applied the attached patch that Jeff supplied me and so far it is working flawlessly. I currently have less than 4% free space on my drive and the CPU usage is less then 3% with two recordings going. I'll let it run until about 2% free space just to test further. It also _appears_ that overall

Re: ReiserFS v3 choking when free space falls below 10% - FIXED

2006-07-24 Thread Jeff Mahoney
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Mike Benoit wrote: I applied the attached patch that Jeff supplied me and so far it is working flawlessly. I currently have less than 4% free space on my drive and the CPU usage is less then 3% with two recordings going. I'll let it run until

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-12 Thread Hans Reiser
Jeffrey Mahoney wrote: Hans Reiser wrote: You make this way too complicated because you are trying to be way too perfect. If you scan 3 bitmap blocks and find nothing, stop trying to size match. Agreed on the trying too hard.. What about the actual algorithm suggested? I think we can

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-11 Thread Jeffrey Mahoney
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jeff Mahoney wrote: Hans Reiser wrote: Guys, if you run the kernel under a debugger, and get it to where you see the excessive CPU usage, and then start stepping through the bitmap code, I am sure it will be very obvious what the error is. Can

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-11 Thread Hans Reiser
You make this way too complicated because you are trying to be way too perfect. If you scan 3 bitmap blocks and find nothing, stop trying to size match. Hans Jeffrey Mahoney wrote: Jeff Mahoney wrote: Hans Reiser wrote: Guys, if you run the kernel under a debugger, and get it to where you

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-11 Thread Jeffrey Mahoney
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hans Reiser wrote: You make this way too complicated because you are trying to be way too perfect. If you scan 3 bitmap blocks and find nothing, stop trying to size match. Agreed on the trying too hard. I think we can find a better, less perfect

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-08 Thread Jeff Mahoney
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hans Reiser wrote: Guys, if you run the kernel under a debugger, and get it to where you see the excessive CPU usage, and then start stepping through the bitmap code, I am sure it will be very obvious what the error is. Can anyone do that for us?

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-08 Thread Hans Reiser
By 8 iterations, I mean 8 bitmaps scanned.

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-08 Thread Hans Reiser
So limit the number of iterations of rejecting windows that are too small. Say, 8. Hans Jeff Mahoney wrote: What's happening is that we keep finding windows that are too small, which results in a lot of wasted effort. The cycle goes like this: if (unfm is_block_in_journal(s, bmap_n,

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-07 Thread Mike Benoit
Hi Jeff, Like clock work the problem showed up pretty much exactly when I expected. This time however I discovered a few other interesting tidbits along the way. I tried to re-create the problem much faster by writing a little script that would append data to a file at about 5mb/s, I ran

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-07 Thread Jan Kara
Hi, just one note: I've looked to the in scan_bitmap() in bitmap.c. There is: /* When the bitmap is more than 10% free, anyone can allocate. * When it's less than 10% free, only files that already use the * bitmap are allowed. Once we pass 80% full, this restriction

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-07 Thread Jeff Mahoney
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jan Kara wrote: Hi, just one note: I've looked to the in scan_bitmap() in bitmap.c. There is: /* When the bitmap is more than 10% free, anyone can allocate. * When it's less than 10% free, only files that already use the

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-07 Thread Jeff Mahoney
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jan Kara wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jan Kara wrote: Hi, just one note: I've looked to the in scan_bitmap() in bitmap.c. There is: /* When the bitmap is more than 10% free, anyone can allocate. *

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-07 Thread Hans Reiser
Jan Kara wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jan Kara wrote: Hi, just one note: I've looked to the in scan_bitmap() in bitmap.c. There is: /* When the bitmap is more than 10% free, anyone can allocate. * When it's less than 10% free, only files that

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-07 Thread Jan Kara
Jan Kara wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jan Kara wrote: Hi, just one note: I've looked to the in scan_bitmap() in bitmap.c. There is: /* When the bitmap is more than 10% free, anyone can allocate. * When it's less than 10% free, only

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-07 Thread Hans Reiser
Jan Kara wrote: Jan Kara wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jan Kara wrote: Hi, just one note: I've looked to the in scan_bitmap() in bitmap.c. There is: /* When the bitmap is more than 10% free, anyone can allocate. * When it's less than

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-07 Thread Mike Benoit
On Fri, 2006-07-07 at 19:49 +0200, Jan Kara wrote: Hi, just one note: I've looked to the in scan_bitmap() in bitmap.c. There is: /* When the bitmap is more than 10% free, anyone can allocate. * When it's less than 10% free, only files that already use the *

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-07 Thread Mike Benoit
On Fri, 2006-07-07 at 19:49 +0200, Jan Kara wrote: Hi, just one note: I've looked to the in scan_bitmap() in bitmap.c. There is: /* When the bitmap is more than 10% free, anyone can allocate. * When it's less than 10% free, only files that already use the *

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-07 Thread Hans Reiser
Guys, if you run the kernel under a debugger, and get it to where you see the excessive CPU usage, and then start stepping through the bitmap code, I am sure it will be very obvious what the error is. Can anyone do that for us? Jeff?

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-06 Thread Jure Pečar
On Tue, 04 Jul 2006 19:37:34 -0700 Hans Reiser [EMAIL PROTECTED] wrote: Mike Benoit wrote: Hi Jeff, I just tried the patch you suggested and it didn't make a difference. The load still spikes as soon as the free space falls below ~10%. Jeff, please audit your code for what happens

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-06 Thread Mike Benoit
On Thu, 2006-07-06 at 12:58 +0200, Jure Pečar wrote: On Tue, 04 Jul 2006 19:37:34 -0700 Hans Reiser [EMAIL PROTECTED] wrote: Mike Benoit wrote: Hi Jeff, I just tried the patch you suggested and it didn't make a difference. The load still spikes as soon as the free space falls

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-06 Thread Jonathan Briggs
On Thu, 2006-07-06 at 08:43 -0700, Mike Benoit wrote: [snip] My desktop machine (v2.6.16, same as my MythTV box) is running with 9% free space right now and it is not experiencing any slow down. I think the problem is caused by the usage pattern of MythTV and how it simultaneously streams one

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-06 Thread Toby Thain
On 6-Jul-06, at 11:43 AM, Mike Benoit wrote: On Thu, 2006-07-06 at 12:58 +0200, Jure Pečar wrote: On Tue, 04 Jul 2006 19:37:34 -0700 Hans Reiser [EMAIL PROTECTED] wrote: Mike Benoit wrote: Hi Jeff, I just tried the patch you suggested and it didn't make a difference. The load still

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-06 Thread Jeff Mahoney
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Mike Benoit wrote: My desktop machine (v2.6.16, same as my MythTV box) is running with 9% free space right now and it is not experiencing any slow down. I think the problem is caused by the usage pattern of MythTV and how it simultaneously streams

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-06 Thread Hans Reiser
Jeff Mahoney wrote: Ok, if you run into the problem again, can you dump the metadata before freeing the space? The code itself looks sound, and I'm wondering if you've managed to create pathological fragmentation that's mucking things up. There should be no possible fragmentation that

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-06 Thread Jeff Mahoney
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hans Reiser wrote: There should be no possible fragmentation that would increase CPU usage like that. With the current algorithms, in which you check one field in the bitmap to see if it has any free blocks, it should not be possible for scanning

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-06 Thread Mike Benoit
On Thu, 2006-07-06 at 14:02 -0400, Jeff Mahoney wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Mike Benoit wrote: My desktop machine (v2.6.16, same as my MythTV box) is running with 9% free space right now and it is not experiencing any slow down. I think the problem is caused by

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-06 Thread Jeff Mahoney
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Mike Benoit wrote: On Thu, 2006-07-06 at 14:02 -0400, Jeff Mahoney wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Mike Benoit wrote: My desktop machine (v2.6.16, same as my MythTV box) is running with 9% free space right now and it is not

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-06 Thread Mike Benoit
On Thu, 2006-07-06 at 14:19 -0400, Jeff Mahoney wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hans Reiser wrote: There should be no possible fragmentation that would increase CPU usage like that. With the current algorithms, in which you check one field in the bitmap to see if

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-06 Thread Hans Reiser
Jeff, I am suspicious, because I know that 90% is a magic number in your code.

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-05 Thread Tom Vier
On Tue, Jul 04, 2006 at 07:37:34PM -0700, Hans Reiser wrote: Mike, thanks so much for going to this much effort. It is rather likely this is a problem affecting many users. Last weekend, i accidentally filled my /. I noticed when i heard the drives (it's a 2 drive raid 0) thrashing. I didn't

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-05 Thread Jeff Mahoney
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hans Reiser wrote: Mike Benoit wrote: Hi Jeff, I just tried the patch you suggested and it didn't make a difference. The load still spikes as soon as the free space falls below ~10%. Jeff, please audit your code for what happens when all the

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-04 Thread Mike Benoit
Hi Jeff, I just tried the patch you suggested and it didn't make a difference. The load still spikes as soon as the free space falls below ~10%. On Fri, 2006-06-30 at 12:47 -0400, Jeff Mahoney wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hans Reiser wrote: Mike Benoit

Re: ReiserFS v3 choking when free space falls below 10%?

2006-07-04 Thread Hans Reiser
Mike Benoit wrote: Hi Jeff, I just tried the patch you suggested and it didn't make a difference. The load still spikes as soon as the free space falls below ~10%. Jeff, please audit your code for what happens when all the bitmap blocks reach 90% full. Could you discuss your design and code

Re: ReiserFS v3 choking when free space falls below 10%?

2006-06-30 Thread Hans Reiser
Mike Benoit wrote: This seems strange, because to me this type of workload would lend itself to being less fragmented then most workloads. All the box does is records TV programs, so over the course of 30-60min periods I would guess 95+% of the writes are sequential. Why would the

Re: ReiserFS v3 choking when free space falls below 10%?

2006-06-30 Thread Hans Reiser
Jeff, does the code do anything funny when crossing the 90% point? You have special heuristics for that, yes? Maybe a bug is hiding in them?

Re: ReiserFS v3 choking when free space falls below 10%?

2006-06-29 Thread Vladimir V. Saveliev
Hello On Thu, 2006-06-29 at 10:41 -0700, Mike Benoit wrote: My MythTV box recently started showing odd behavior during recordings, at certain times the load of the box would spike to 10+ and recordings would start losing frames and become unwatchable. TOP would show mythbackend as using 90+%

Re: ReiserFS v3 choking when free space falls below 10%?

2006-06-29 Thread Mike Benoit
On Thu, 2006-06-29 at 23:12 +0400, Vladimir V. Saveliev wrote: Hello On Thu, 2006-06-29 at 10:41 -0700, Mike Benoit wrote: So I finally got around to profiling mythbackend when the load starts to spike. To my surprise it appears that once I have less then 10% (30GB) free on the drive

Re: ReiserFS v3 choking when free space falls below 10%?

2006-06-29 Thread Vladimir V. Saveliev
Hello On Thu, 2006-06-29 at 13:15 -0700, Mike Benoit wrote: On Thu, 2006-06-29 at 23:12 +0400, Vladimir V. Saveliev wrote: Hello On Thu, 2006-06-29 at 10:41 -0700, Mike Benoit wrote: So I finally got around to profiling mythbackend when the load starts to spike. To my surprise it