Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
Vincent Stemen wrote: > The problem is, that's not true. These problems are not slipping > through because of lack of testers. Just to add some sanity to this thread, I have been using the 2.4.x kernels ever since they came out, on my personal workstation and on some workstations that I administrate for fellow students in my department here at UCLA. They have basically worked fine for me. They are not perfect, but many of the 2.4.x releases have been a big improvement over the 2.2.x releases. For one, 2.4.x actually can tell which pages are not used, and swap out unused daemons, which helps a lot on a 64Mb box :) -BenR -- Einstein did not prove that everything is relative. Einstein explained how the speed of light could be constant. Benjamin Redelings I <>< http://www.bol.ucla.edu/~bredelin/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Wed, 30 May 2001, Vincent Stemen wrote: > On Wednesday 30 May 2001 15:17, Mike Galbraith wrote: > > On Wed, 30 May 2001, Vincent Stemen wrote: > > > On Wednesday 30 May 2001 01:02, Mike Galbraith wrote: > > > > On Tue, 29 May 2001, Vincent Stemen wrote: > > > > > On Tuesday 29 May 2001 15:16, Alan Cox wrote: > > > > > > > a reasonably stable release until 2.2.12. I do not understand > > > > > > > why code with such serious reproducible problems is being > > > > > > > introduced into the even numbered kernels. What happened to > > > > > > > the plan to use only the > > > > > > > > > > > > Who said it was introduced ?? It was more 'lurking' than > > > > > > introduced. And unfortunately nobody really pinned it down in > > > > > > 2.4test. > > > > > > > > > > I fail to see the distinction. First of all, why was it ever > > > > > released as 2.4-test? That question should probably be directed at > > > > > Linus. If it is not fully tested, then it should be released it as > > > > > an odd number. If it already existed in the odd numbered > > > > > development kernel and was known, then it should have never been > > > > > released as a production kernel until it was resolved. Otherwise, > > > > > it completely defeats the purpose of having the even/odd numbering > > > > > system. > > > > > > > > > > I do not expect bugs to never slip through to production kernels, > > > > > but known bugs that are not trivial should not, and serious bugs > > > > > like these VM problems especially should not. > > > > > > > > And you can help prevent them from slipping through by signing up as > > > > a shake and bake tester. Indeed, you can make your expectations > > > > reality absolutely free of charge, and or compensation > > > > what a bargain! > > > > > > > > X ___ ;-) > > > > > > > > -Mike > > > > > > The problem is, that's not true. These problems are not slipping > > > through because of lack of testers. As Alan said, the VM problem has > > > > Sorry, that's a copout. You (we) had many chances to notice. Don't > > push the problems back onto developers.. it's our problem. > > > > How is that a copout? The problem was noticed. I am only suggesting > that we not be in such a hurry to put code in the production kernels > until we are pretty sure it works well enough, and that we release > major production versions more often so that they do not contain 2 or > 3 years worth of new code making it so hard to debug. We probably > should have had 2 or 3 code freezes and production releases since > 2.2.x. As I mentioned in a previous posting, this way we do not have > to run a 2 or 3 year old kernel in order to have reasonable stability. I don't think you or I can do a better job of release management than Linus and friends, so there's no point in us discussing it. If you want to tell Linus, Alan et al how to do it 'right', you go do that. -Mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Marcelo Tosatti wrote: > On Wed, 30 May 2001, Mike Galbraith wrote: > > > On Wed, 30 May 2001, Rik van Riel wrote: > > > > > On Wed, 30 May 2001, Marcelo Tosatti wrote: > > > > > > > The problem is that we allow _every_ task to age pages on the system > > > > at the same time --- this is one of the things which is fucking up. > > > > > > This should not have any effect on the ratio of cache > > > reclaiming vs. swapout use, though... > > > > It shouldn't.. but when many tasks are aging, it does. > > What Rik means is that they are independant problems. Ok. > > > Excluding these guys certainly seems to make a difference. > > Sure, those guys are going to "help" kswapd to unmap pte's and allocate > swap space. > > Now even if only kswapd does this job (meaning a sane amount of cache > reclaims/swapouts), you still have to deal with the reclaim/swapout > tradeoff. > > See? Yes. -Mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Wednesday 30 May 2001 15:17, Mike Galbraith wrote: > On Wed, 30 May 2001, Vincent Stemen wrote: > > On Wednesday 30 May 2001 01:02, Mike Galbraith wrote: > > > On Tue, 29 May 2001, Vincent Stemen wrote: > > > > On Tuesday 29 May 2001 15:16, Alan Cox wrote: > > > > > > a reasonably stable release until 2.2.12. I do not understand > > > > > > why code with such serious reproducible problems is being > > > > > > introduced into the even numbered kernels. What happened to > > > > > > the plan to use only the > > > > > > > > > > Who said it was introduced ?? It was more 'lurking' than > > > > > introduced. And unfortunately nobody really pinned it down in > > > > > 2.4test. > > > > > > > > I fail to see the distinction. First of all, why was it ever > > > > released as 2.4-test? That question should probably be directed at > > > > Linus. If it is not fully tested, then it should be released it as > > > > an odd number. If it already existed in the odd numbered > > > > development kernel and was known, then it should have never been > > > > released as a production kernel until it was resolved. Otherwise, > > > > it completely defeats the purpose of having the even/odd numbering > > > > system. > > > > > > > > I do not expect bugs to never slip through to production kernels, > > > > but known bugs that are not trivial should not, and serious bugs > > > > like these VM problems especially should not. > > > > > > And you can help prevent them from slipping through by signing up as > > > a shake and bake tester. Indeed, you can make your expectations > > > reality absolutely free of charge, and or compensation > > > what a bargain! > > > > > > X ___ ;-) > > > > > > -Mike > > > > The problem is, that's not true. These problems are not slipping > > through because of lack of testers. As Alan said, the VM problem has > > Sorry, that's a copout. You (we) had many chances to notice. Don't > push the problems back onto developers.. it's our problem. > How is that a copout? The problem was noticed. I am only suggesting that we not be in such a hurry to put code in the production kernels until we are pretty sure it works well enough, and that we release major production versions more often so that they do not contain 2 or 3 years worth of new code making it so hard to debug. We probably should have had 2 or 3 code freezes and production releases since 2.2.x. As I mentioned in a previous posting, this way we do not have to run a 2 or 3 year old kernel in order to have reasonable stability. > > Here are some of the problems I see: > > > > There was far to long of a stretch with to much code dumped into both > > the 2.2 and 2.4 kernels before release. There needs to be a smaller > > number changes between major releases so that they can be more > > thoroughly tested and debugged. In the race to get it out there they > > are making the same mistakes as Microsoft, releasing production > > kernels with known serious bugs because it is taking to long and they > > want to move on forward. I enjoy criticizing Microsoft so much for > > the same thing that I do not want to have to stop in order to not > > sound hypocritical :-). The Linux community has built a lot of it's > > reputation on not making these mistakes. Please lets try not to > > destroy that. > > > > They are disregarding the even/odd versioning system. > > For example: > > There was a new 8139too driver added to the the 2.4.5 (I think) kernel > > which Alan Cox took back out and reverted to the old one in his > > 2.4.5-ac? versions because it is apparently causing lockups. > > Shouldn't this new driver have been released in a 2.5.x development > > kernel and proven there before replacing the one in the production > > kernel? I haven't even seen a 2.5.x kernel released yet. > > > > Based on Linus's original very good plan for even/odd numbers, there > > should not have been 2.4.0-test? kernels either. This was another > > example of the rush to increment to 2.4 long before it was ready. > > There was a long stretch of test kernels and and now we are all the > > way to 2.4.5 and it is still not stable. We are repeating the 2.2.x > > process all over again. It should have been 2.3.x until the > > production release was ready. If they needed to distinguish a code > > freeze for final testing, it could be done with a 4th version > > component (2.3.xx.xx), where the 4 component is incremented for final > > bug fixes. > > Sorry, I disagree with every last bit. Either you accept a situation > or you try to do something about it. > > -Mike I am spending a lot of time testing new kernels, reporting bugs and offering suggestions that I think may improve on the stability of production kernels. Is this not considered doing something about it? It is necessary to point out where one sees a problem in order to offer possible solutions for improvement. - Vincent - To unsubscribe from this list: send the line
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Wednesday 30 May 2001 15:30, Rik van Riel wrote: > On Wed, 30 May 2001, Vincent Stemen wrote: > > The problem is, that's not true. These problems are not slipping > > through because of lack of testers. As Alan said, the VM problem has > > been lurking, which means that it was known already. > > Fully agreed, it went through because of a lack of hours > per day and the fact that the priority of developers was > elsewhere. > > For me, for example, the priorities have mostly been with > bugs that bothered me or that bothered Conectiva's customers. > > If you _really_ feel this strongly about the bug, you could > either try to increase the number of hours a day for all of I sure wish I could :-). > us or you could talk to my boss about hiring me as a consultant > to fix the problem for you on an emergency basis :) > The other two alternatives would be either waiting until > somebody gets around to fixing the bug or sending in a patch > yourself. > > Trying to piss off developers has adverse effect on all four > of the methods above :) > Why should my comments piss anybody off? I am just trying to point out a problem, as I see it, an offer suggestions for improvement. Other developers will either agree with me or they wont. Contributions are not made only through writing code. I contribute through code, bug reports, ideas, and suggestions. I would love to dive in and try to help fix some of the kernel problems but my hands are just to full right now. My comments are not meant to rush anybody and I am not criticizing how long it is taking. I know everybody is doing everything they can just like I am, and they are doing a terrific job. I am just suggesting a modification to the way the kernels are distributed that is more like the early versions that I hoped would allow us to maintain a stable kernel for distributions and production machines. - Vincent Stemen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
Ronald Bultje writes: > On 30 May 2001 14:58:57 -0500, Vincent Stemen wrote: > > There was a new 8139too driver added to the the 2.4.5 (I think) kernel > > which Alan Cox took back out and reverted to the old one in his > > 2.4.5-ac? versions because it is apparently causing lockups. > > Shouldn't this new driver have been released in a 2.5.x development > > kernel and proven there before replacing the one in the production > > kernel? I haven't even seen a 2.5.x kernel released yet. > > If every driver has to go thorugh the complete development cycle (of 2+ > years), I'm sure very little driver writers will be as motivated as they > are now - it takes ages before they see their efforts "rewarded" with a > place in the kernel. > The ideal case is that odd-numbered kernels are "for testing" and > even-numbered kernels are stable. However, this is only theory. In > practice, you can't rule out all bugs. And you can't test all things for > all cases and every test case, the linux community doesn't have the > manpower for that. And to prevent a complete driver development cycle > taking 2+ years, you have to compromise. > > If you would take 2+ years for a single driver development cycle, nobody > would be interested in linux since the new devices would only be > supported by a stable kernel two years after their release. See the > point? To prevent that, you need to compromise. and thus, sometimes, you > have some crashes. I agree with everything you say up till this point, but you are arguing against a point I never made. First of all, bugs like the 8139too lockup was found within the first day or two of release in the 2.4.3 kernel. Also, most show stopper bugs such as the VM problems are found fairly quickly. Even if it takes a long time to figure out how to fix them, I do not think they should be pushed on through into production kernels until they are until they are fixed. I already said that I do not expect minor bugs not to slip through. However, if they are minor, they can usually be fixed quickly once they are discovered and it is no big deal if they make it into a production kernel. > That's why there's still 2.2.x - that's purely stable > and won't crash as fast as 2.4.x, but misses the "newest > cutting-edge-technology device support" and "newest technology" (like > new SMP handling , ReiserFS, etc... But it *is* stable. > The reason I suggested more frequent major production releases is so that you don't have to go back to a 2 or 3 year old kernel and loose out on years worth of new features to have any stability. One show stopper bug like the VM problems would not be as much of a problem if there was a stable production kernel that we could run that was only 4 or 6 months old. > > Based on Linus's original very good plan for even/odd numbers, there > > should not have been 2.4.0-test? kernels either. This was another > > example of the rush to increment to 2.4 long before it was ready. > > There was a long stretch of test kernels and and now we are all the > > way to 2.4.5 and it is still not stable. We are repeating the 2.2.x > > process all over again. > > Wrong again. > 2.3.x is for development, adding new things, testing, adding, testing, > changing, testing, etc. Which is the same point I made. > 2.4-test is for testing only, it's some sort of feature freeze. Agreed. My only point here was that it suggests that there are only minor bugs left to be solved before the production release by setting the version to 2.4-test. That is one of the reasons I made the suggestion to keep it in the 2.3 range, since there were actually serious VM problems still upon the production 2.4 release. > 2.4.x is for final/stable 2.4. > It's a standard *nix development cycle. That's how everyone does it. My point exactly. > > Regards, > > Ronald Bultje - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Mike Galbraith wrote: > On Wed, 30 May 2001, Rik van Riel wrote: > > > On Wed, 30 May 2001, Marcelo Tosatti wrote: > > > > > The problem is that we allow _every_ task to age pages on the system > > > at the same time --- this is one of the things which is fucking up. > > > > This should not have any effect on the ratio of cache > > reclaiming vs. swapout use, though... > > It shouldn't.. but when many tasks are aging, it does. What Rik means is that they are independant problems. > Excluding these guys certainly seems to make a difference. Sure, those guys are going to "help" kswapd to unmap pte's and allocate swap space. Now even if only kswapd does this job (meaning a sane amount of cache reclaims/swapouts), you still have to deal with the reclaim/swapout tradeoff. See? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Rik van Riel wrote: > On Wed, 30 May 2001, Marcelo Tosatti wrote: > > > The problem is that we allow _every_ task to age pages on the system > > at the same time --- this is one of the things which is fucking up. > > This should not have any effect on the ratio of cache > reclaiming vs. swapout use, though... It shouldn't.. but when many tasks are aging, it does. Excluding these guys certainly seems to make a difference. (could be seeing something else and interpreting it wrong...) -Mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Wed, 30 May 2001, Vincent Stemen wrote: > The problem is, that's not true. These problems are not slipping > through because of lack of testers. As Alan said, the VM problem has > been lurking, which means that it was known already. Fully agreed, it went through because of a lack of hours per day and the fact that the priority of developers was elsewhere. For me, for example, the priorities have mostly been with bugs that bothered me or that bothered Conectiva's customers. If you _really_ feel this strongly about the bug, you could either try to increase the number of hours a day for all of us or you could talk to my boss about hiring me as a consultant to fix the problem for you on an emergency basis :) The other two alternatives would be either waiting until somebody gets around to fixing the bug or sending in a patch yourself. Trying to piss off developers has adverse effect on all four of the methods above :) Rik -- Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://distro.conectiva.com/ Send all your spam to [EMAIL PROTECTED] (spam digging piggy) - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
> There was a new 8139too driver added to the the 2.4.5 (I think) kernel > which Alan Cox took back out and reverted to the old one in his > 2.4.5-ac? versions because it is apparently causing lockups. > Shouldn't this new driver have been released in a 2.5.x development > kernel and proven there before replacing the one in the production > kernel? I haven't even seen a 2.5.x kernel released yet. Nope. The 2.4.3 one is buggy too - but differently (and it turns out a little less) buggy. Welcome to software. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Wednesday 30 May 2001 01:02, Mike Galbraith wrote: > On Tue, 29 May 2001, Vincent Stemen wrote: > > On Tuesday 29 May 2001 15:16, Alan Cox wrote: > > > > a reasonably stable release until 2.2.12. I do not understand why > > > > code with such serious reproducible problems is being introduced > > > > into the even numbered kernels. What happened to the plan to use > > > > only the > > > > > > Who said it was introduced ?? It was more 'lurking' than introduced. > > > And unfortunately nobody really pinned it down in 2.4test. > > > > I fail to see the distinction. First of all, why was it ever released > > as 2.4-test? That question should probably be directed at Linus. If > > it is not fully tested, then it should be released it as an odd > > number. If it already existed in the odd numbered development kernel > > and was known, then it should have never been released as a production > > kernel until it was resolved. Otherwise, it completely defeats the > > purpose of having the even/odd numbering system. > > > > I do not expect bugs to never slip through to production kernels, but > > known bugs that are not trivial should not, and serious bugs like > > these VM problems especially should not. > > And you can help prevent them from slipping through by signing up as a > shake and bake tester. Indeed, you can make your expectations reality > absolutely free of charge, and or compensation > what a bargain! > > X ___ ;-) > > -Mike The problem is, that's not true. These problems are not slipping through because of lack of testers. As Alan said, the VM problem has been lurking, which means that it was known already. We currently have no development/production kernel distinction and I have not been able to find even one stable 2.4.x version to run on our main machines. Reverting back to 2.2.x is a real pain because of all the surrounding changes which will affect our initscripts and other system configuration issues, such as Unix98 pty's, proc filesystem differences, device numbering, etc. I have the greatest respect and appreciation for Linus, Alan, and all of the other kernel developers. My comments are not meant to criticize, but rather to point out some the problems I see that are making it so difficult to stabilize the kernel and encourage them to steer back on track. Here are some of the problems I see: There was far to long of a stretch with to much code dumped into both the 2.2 and 2.4 kernels before release. There needs to be a smaller number changes between major releases so that they can be more thoroughly tested and debugged. In the race to get it out there they are making the same mistakes as Microsoft, releasing production kernels with known serious bugs because it is taking to long and they want to move on forward. I enjoy criticizing Microsoft so much for the same thing that I do not want to have to stop in order to not sound hypocritical :-). The Linux community has built a lot of it's reputation on not making these mistakes. Please lets try not to destroy that. They are disregarding the even/odd versioning system. For example: There was a new 8139too driver added to the the 2.4.5 (I think) kernel which Alan Cox took back out and reverted to the old one in his 2.4.5-ac? versions because it is apparently causing lockups. Shouldn't this new driver have been released in a 2.5.x development kernel and proven there before replacing the one in the production kernel? I haven't even seen a 2.5.x kernel released yet. Based on Linus's original very good plan for even/odd numbers, there should not have been 2.4.0-test? kernels either. This was another example of the rush to increment to 2.4 long before it was ready. There was a long stretch of test kernels and and now we are all the way to 2.4.5 and it is still not stable. We are repeating the 2.2.x process all over again. It should have been 2.3.x until the production release was ready. If they needed to distinguish a code freeze for final testing, it could be done with a 4th version component (2.3.xx.xx), where the 4 component is incremented for final bug fixes. - Vincent Stemen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Mike Galbraith wrote: > I wouldn't go so far as to say totally broken (mostly because I've > tried like _hell_ to find a better way, and [despite minor successes] > I've not been able to come up with something which covers all cases > that even _I_ [hw tech] can think of well). The "easy way out" seems to be physical -> virtual page reverse mappings, these make it trivial to apply balanced pressure on all pages. The downside of this measure is that it costs additional overhead and can up to double the amount of memory we take in with page tables. Of course, this amount is only prohibitive if the amount of page table memory was also prohibitively large in the first place, but ... :) regards, Rik -- Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://distro.conectiva.com/ Send all your spam to [EMAIL PROTECTED] (spam digging piggy) - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Rik van Riel wrote: > On Wed, 30 May 2001, Marcelo Tosatti wrote: > > > The problem is that we allow _every_ task to age pages on the system > > at the same time --- this is one of the things which is fucking up. > > This should not have any effect on the ratio of cache > reclaiming vs. swapout use, though... Sure, who said that ? :) The current discussion between Mike/Jonathan and me is about the aging issue. > > > The another problem is that don't limit the writeout in the VM. > > This is a big problem too, but also unrelated to the > impossibility of balancing cache vs. swap in the current > scheme. ... - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Marcelo Tosatti wrote: > The problem is that we allow _every_ task to age pages on the system > at the same time --- this is one of the things which is fucking up. This should not have any effect on the ratio of cache reclaiming vs. swapout use, though... > The another problem is that don't limit the writeout in the VM. This is a big problem too, but also unrelated to the impossibility of balancing cache vs. swap in the current scheme. regards, Rik -- Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://distro.conectiva.com/ Send all your spam to [EMAIL PROTECTED] (spam digging piggy) - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Marcelo Tosatti wrote: > On Wed, 30 May 2001, Jonathan Morton wrote: > > > >The page aging logic does seems fragile as heck. You never know how > > >many folks are aging pages or at what rate. If aging happens too fast, > > >it defeats the garbage identification logic and you rape your cache. If > > >aging happens too slowly.. sigh. > > > > Then it sounds like the current algorithm is totally broken and needs > > replacement. If it's impossible to make a system stable by choosing the > > right numbers, the system needs changing, not the numbers. I think that's > > pretty much what we're being taught in Control Engineering. :) > > The problem is that we allow _every_ task to age pages on the system at > the same time --- this is one of the things which is fucking up. Yes. (I've been muttering/mumbling about this for... ever. look at the last patch I posted in this light.. make -j30 load:) > The another problem is that don't limit the writeout in the VM. And sometimes we don't start writing out soon enough. > We (me and Rik) are going to work on this later --- right now I'm busy > with the distribution release and Rik is travelling. Cool. -Mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Jonathan Morton wrote: > >The page aging logic does seems fragile as heck. You never know how > >many folks are aging pages or at what rate. If aging happens too fast, > >it defeats the garbage identification logic and you rape your cache. If > >aging happens too slowly.. sigh. > > Then it sounds like the current algorithm is totally broken and needs > replacement. If it's impossible to make a system stable by choosing the > right numbers, the system needs changing, not the numbers. I think that's > pretty much what we're being taught in Control Engineering. :) I wouldn't go so far as to say totally broken (mostly because I've tried like _hell_ to find a better way, and [despite minor successes] I've not been able to come up with something which covers all cases that even _I_ [hw tech] can think of well). Hard numbers are just plain bad, see below. I _will_ say that it is entirely too easy to break for comfort ;-) > Not having studied the code too closely, it sounds as though there are half > a dozen different "clocks" running for different types of memory, and each > one runs at a different speed and is updated at a different time. > Meanwhile, the paging-out is done on the assumption that all the clocks are > (at least roughly) in sync. Makes sense, right? (not!) No, I don't think that's the case at all. The individual zone balancing logic (or individual page [content] type logic) hasn't been inplimented yet (content type handling I don't think we want), but doesn't look like it'll be any trouble to do with the structures in place. That's just a fleshing out thing. The variations in aging rate is the most difficult problem I can see. IMHO, it needs to be either decoupled and done by an impartial bystander (tried that, ran into info flow troubles because of scheduling) or integrated tightly into the allocator proper (tried that.. interesting results but has problems of it's own wrt the massive changes in general strategy needed to make it work.. approaches rewrite) > I think it's worthwhile to think of the page/buffer caches as having a > working set of their own - if they are being heavily used, they should get > more memory than if they are only lightly used. The important point to get > right is to ensure that the "clocks" used for each memory area remain in > sync - they don't have to measure real time, just be consistent and fine > granularity. IMHO, the only thing of interest you can do with clocks is set your state sample rate. If state is changing rapidly, you must sample rapidly. As far as corrections go, you can only insert a corrective vector into the mix and then see if the sum induced the desired change in direction. The correct magnitude of this vector is not even possible to know.. that's what makes it so darn hard [defining numerical goals is utterly bogus]. > I'm working on some relatively small changes to vmscan.c which should help > improve the behaviour without upsetting the balance too much. Watch this > space... With much interest :) -Mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Jonathan Morton wrote: > >The page aging logic does seems fragile as heck. You never know how > >many folks are aging pages or at what rate. If aging happens too fast, > >it defeats the garbage identification logic and you rape your cache. If > >aging happens too slowly.. sigh. > > Then it sounds like the current algorithm is totally broken and needs > replacement. If it's impossible to make a system stable by choosing the > right numbers, the system needs changing, not the numbers. I think that's > pretty much what we're being taught in Control Engineering. :) The problem is that we allow _every_ task to age pages on the system at the same time --- this is one of the things which is fucking up. The another problem is that don't limit the writeout in the VM. We (me and Rik) are going to work on this later --- right now I'm busy with the distribution release and Rik is travelling. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
>The page aging logic does seems fragile as heck. You never know how >many folks are aging pages or at what rate. If aging happens too fast, >it defeats the garbage identification logic and you rape your cache. If >aging happens too slowly.. sigh. Then it sounds like the current algorithm is totally broken and needs replacement. If it's impossible to make a system stable by choosing the right numbers, the system needs changing, not the numbers. I think that's pretty much what we're being taught in Control Engineering. :) Not having studied the code too closely, it sounds as though there are half a dozen different "clocks" running for different types of memory, and each one runs at a different speed and is updated at a different time. Meanwhile, the paging-out is done on the assumption that all the clocks are (at least roughly) in sync. Makes sense, right? (not!) I think it's worthwhile to think of the page/buffer caches as having a working set of their own - if they are being heavily used, they should get more memory than if they are only lightly used. The important point to get right is to ensure that the "clocks" used for each memory area remain in sync - they don't have to measure real time, just be consistent and fine granularity. I'm working on some relatively small changes to vmscan.c which should help improve the behaviour without upsetting the balance too much. Watch this space... -- from: Jonathan "Chromatix" Morton mail: [EMAIL PROTECTED] (not for attachments) big-mail: [EMAIL PROTECTED] uni-mail: [EMAIL PROTECTED] The key to knowledge is not to rely on people to teach you it. Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/ -BEGIN GEEK CODE BLOCK- Version 3.12 GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*) -END GEEK CODE BLOCK- - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Tue, 29 May 2001, Craig Kulesa wrote: > Mike Galbraith ([EMAIL PROTECTED]) wrote: > > > > Emphatic yes. We went from cache collapse to cache bloat. > > Rik, I think Mike deserves his beer. ;) :) ... > So is there an ideal VM balance for everyone? I have found that low-RAM (I seriously doubt it) > systems seem to benefit from being on the "cache-collapse" side of the > curve (so I prefer the pre-2.4.5 balance more than Mike probably does) and I hate both bad behaviors equally. "cache bloat" hurts more people than "cache collapse" does though because it shows under light load. > those low-RAM systems are the first hit when, as now, we're favoring > "cache bloat". Should balance behaviors could be altered by the user > (via sysctl's maybe? Yeah, I hear the cringes)? Or better, is it > possible to dynamically choose where the watermarks in balancing should > lie, and alter them automatically? 2.5 stuff there, no doubt. Balancing > seems so *fragile* (to me). The page aging logic does seems fragile as heck. You never know how many folks are aging pages or at what rate. If aging happens too fast, it defeats the garbage identification logic and you rape your cache. If aging happens too slowly.. sigh. -Mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Tue, 29 May 2001, Craig Kulesa wrote: Mike Galbraith ([EMAIL PROTECTED]) wrote: Emphatic yes. We went from cache collapse to cache bloat. Rik, I think Mike deserves his beer. ;) :) ... So is there an ideal VM balance for everyone? I have found that low-RAM (I seriously doubt it) systems seem to benefit from being on the cache-collapse side of the curve (so I prefer the pre-2.4.5 balance more than Mike probably does) and I hate both bad behaviors equally. cache bloat hurts more people than cache collapse does though because it shows under light load. those low-RAM systems are the first hit when, as now, we're favoring cache bloat. Should balance behaviors could be altered by the user (via sysctl's maybe? Yeah, I hear the cringes)? Or better, is it possible to dynamically choose where the watermarks in balancing should lie, and alter them automatically? 2.5 stuff there, no doubt. Balancing seems so *fragile* (to me). The page aging logic does seems fragile as heck. You never know how many folks are aging pages or at what rate. If aging happens too fast, it defeats the garbage identification logic and you rape your cache. If aging happens too slowly.. sigh. -Mike - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Jonathan Morton wrote: The page aging logic does seems fragile as heck. You never know how many folks are aging pages or at what rate. If aging happens too fast, it defeats the garbage identification logic and you rape your cache. If aging happens too slowly.. sigh. Then it sounds like the current algorithm is totally broken and needs replacement. If it's impossible to make a system stable by choosing the right numbers, the system needs changing, not the numbers. I think that's pretty much what we're being taught in Control Engineering. :) The problem is that we allow _every_ task to age pages on the system at the same time --- this is one of the things which is fucking up. The another problem is that don't limit the writeout in the VM. We (me and Rik) are going to work on this later --- right now I'm busy with the distribution release and Rik is travelling. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Jonathan Morton wrote: The page aging logic does seems fragile as heck. You never know how many folks are aging pages or at what rate. If aging happens too fast, it defeats the garbage identification logic and you rape your cache. If aging happens too slowly.. sigh. Then it sounds like the current algorithm is totally broken and needs replacement. If it's impossible to make a system stable by choosing the right numbers, the system needs changing, not the numbers. I think that's pretty much what we're being taught in Control Engineering. :) I wouldn't go so far as to say totally broken (mostly because I've tried like _hell_ to find a better way, and [despite minor successes] I've not been able to come up with something which covers all cases that even _I_ [hw tech] can think of well). Hard numbers are just plain bad, see below. I _will_ say that it is entirely too easy to break for comfort ;-) Not having studied the code too closely, it sounds as though there are half a dozen different clocks running for different types of memory, and each one runs at a different speed and is updated at a different time. Meanwhile, the paging-out is done on the assumption that all the clocks are (at least roughly) in sync. Makes sense, right? (not!) No, I don't think that's the case at all. The individual zone balancing logic (or individual page [content] type logic) hasn't been inplimented yet (content type handling I don't think we want), but doesn't look like it'll be any trouble to do with the structures in place. That's just a fleshing out thing. The variations in aging rate is the most difficult problem I can see. IMHO, it needs to be either decoupled and done by an impartial bystander (tried that, ran into info flow troubles because of scheduling) or integrated tightly into the allocator proper (tried that.. interesting results but has problems of it's own wrt the massive changes in general strategy needed to make it work.. approaches rewrite) I think it's worthwhile to think of the page/buffer caches as having a working set of their own - if they are being heavily used, they should get more memory than if they are only lightly used. The important point to get right is to ensure that the clocks used for each memory area remain in sync - they don't have to measure real time, just be consistent and fine granularity. IMHO, the only thing of interest you can do with clocks is set your state sample rate. If state is changing rapidly, you must sample rapidly. As far as corrections go, you can only insert a corrective vector into the mix and then see if the sum induced the desired change in direction. The correct magnitude of this vector is not even possible to know.. that's what makes it so darn hard [defining numerical goals is utterly bogus]. I'm working on some relatively small changes to vmscan.c which should help improve the behaviour without upsetting the balance too much. Watch this space... With much interest :) -Mike - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Marcelo Tosatti wrote: On Wed, 30 May 2001, Jonathan Morton wrote: The page aging logic does seems fragile as heck. You never know how many folks are aging pages or at what rate. If aging happens too fast, it defeats the garbage identification logic and you rape your cache. If aging happens too slowly.. sigh. Then it sounds like the current algorithm is totally broken and needs replacement. If it's impossible to make a system stable by choosing the right numbers, the system needs changing, not the numbers. I think that's pretty much what we're being taught in Control Engineering. :) The problem is that we allow _every_ task to age pages on the system at the same time --- this is one of the things which is fucking up. Yes. (I've been muttering/mumbling about this for... ever. look at the last patch I posted in this light.. make -j30 load:) The another problem is that don't limit the writeout in the VM. And sometimes we don't start writing out soon enough. We (me and Rik) are going to work on this later --- right now I'm busy with the distribution release and Rik is travelling. Cool. -Mike - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Marcelo Tosatti wrote: The problem is that we allow _every_ task to age pages on the system at the same time --- this is one of the things which is fucking up. This should not have any effect on the ratio of cache reclaiming vs. swapout use, though... The another problem is that don't limit the writeout in the VM. This is a big problem too, but also unrelated to the impossibility of balancing cache vs. swap in the current scheme. regards, Rik -- Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://distro.conectiva.com/ Send all your spam to [EMAIL PROTECTED] (spam digging piggy) - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Rik van Riel wrote: On Wed, 30 May 2001, Marcelo Tosatti wrote: The problem is that we allow _every_ task to age pages on the system at the same time --- this is one of the things which is fucking up. This should not have any effect on the ratio of cache reclaiming vs. swapout use, though... Sure, who said that ? :) The current discussion between Mike/Jonathan and me is about the aging issue. The another problem is that don't limit the writeout in the VM. This is a big problem too, but also unrelated to the impossibility of balancing cache vs. swap in the current scheme. ... - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Mike Galbraith wrote: I wouldn't go so far as to say totally broken (mostly because I've tried like _hell_ to find a better way, and [despite minor successes] I've not been able to come up with something which covers all cases that even _I_ [hw tech] can think of well). The easy way out seems to be physical - virtual page reverse mappings, these make it trivial to apply balanced pressure on all pages. The downside of this measure is that it costs additional overhead and can up to double the amount of memory we take in with page tables. Of course, this amount is only prohibitive if the amount of page table memory was also prohibitively large in the first place, but ... :) regards, Rik -- Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://distro.conectiva.com/ Send all your spam to [EMAIL PROTECTED] (spam digging piggy) - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Rik van Riel wrote: On Wed, 30 May 2001, Marcelo Tosatti wrote: The problem is that we allow _every_ task to age pages on the system at the same time --- this is one of the things which is fucking up. This should not have any effect on the ratio of cache reclaiming vs. swapout use, though... It shouldn't.. but when many tasks are aging, it does. Excluding these guys certainly seems to make a difference. (could be seeing something else and interpreting it wrong...) -Mike - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Mike Galbraith wrote: On Wed, 30 May 2001, Rik van Riel wrote: On Wed, 30 May 2001, Marcelo Tosatti wrote: The problem is that we allow _every_ task to age pages on the system at the same time --- this is one of the things which is fucking up. This should not have any effect on the ratio of cache reclaiming vs. swapout use, though... It shouldn't.. but when many tasks are aging, it does. What Rik means is that they are independant problems. Excluding these guys certainly seems to make a difference. Sure, those guys are going to help kswapd to unmap pte's and allocate swap space. Now even if only kswapd does this job (meaning a sane amount of cache reclaims/swapouts), you still have to deal with the reclaim/swapout tradeoff. See? - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
Ronald Bultje writes: On 30 May 2001 14:58:57 -0500, Vincent Stemen wrote: There was a new 8139too driver added to the the 2.4.5 (I think) kernel which Alan Cox took back out and reverted to the old one in his 2.4.5-ac? versions because it is apparently causing lockups. Shouldn't this new driver have been released in a 2.5.x development kernel and proven there before replacing the one in the production kernel? I haven't even seen a 2.5.x kernel released yet. If every driver has to go thorugh the complete development cycle (of 2+ years), I'm sure very little driver writers will be as motivated as they are now - it takes ages before they see their efforts rewarded with a place in the kernel. The ideal case is that odd-numbered kernels are for testing and even-numbered kernels are stable. However, this is only theory. In practice, you can't rule out all bugs. And you can't test all things for all cases and every test case, the linux community doesn't have the manpower for that. And to prevent a complete driver development cycle taking 2+ years, you have to compromise. If you would take 2+ years for a single driver development cycle, nobody would be interested in linux since the new devices would only be supported by a stable kernel two years after their release. See the point? To prevent that, you need to compromise. and thus, sometimes, you have some crashes. I agree with everything you say up till this point, but you are arguing against a point I never made. First of all, bugs like the 8139too lockup was found within the first day or two of release in the 2.4.3 kernel. Also, most show stopper bugs such as the VM problems are found fairly quickly. Even if it takes a long time to figure out how to fix them, I do not think they should be pushed on through into production kernels until they are until they are fixed. I already said that I do not expect minor bugs not to slip through. However, if they are minor, they can usually be fixed quickly once they are discovered and it is no big deal if they make it into a production kernel. That's why there's still 2.2.x - that's purely stable and won't crash as fast as 2.4.x, but misses the newest cutting-edge-technology device support and newest technology (like new SMP handling , ReiserFS, etc... But it *is* stable. The reason I suggested more frequent major production releases is so that you don't have to go back to a 2 or 3 year old kernel and loose out on years worth of new features to have any stability. One show stopper bug like the VM problems would not be as much of a problem if there was a stable production kernel that we could run that was only 4 or 6 months old. Based on Linus's original very good plan for even/odd numbers, there should not have been 2.4.0-test? kernels either. This was another example of the rush to increment to 2.4 long before it was ready. There was a long stretch of test kernels and and now we are all the way to 2.4.5 and it is still not stable. We are repeating the 2.2.x process all over again. Wrong again. 2.3.x is for development, adding new things, testing, adding, testing, changing, testing, etc. Which is the same point I made. 2.4-test is for testing only, it's some sort of feature freeze. Agreed. My only point here was that it suggests that there are only minor bugs left to be solved before the production release by setting the version to 2.4-test. That is one of the reasons I made the suggestion to keep it in the 2.3 range, since there were actually serious VM problems still upon the production 2.4 release. 2.4.x is for final/stable 2.4. It's a standard *nix development cycle. That's how everyone does it. My point exactly. Regards, Ronald Bultje - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Wednesday 30 May 2001 15:30, Rik van Riel wrote: On Wed, 30 May 2001, Vincent Stemen wrote: The problem is, that's not true. These problems are not slipping through because of lack of testers. As Alan said, the VM problem has been lurking, which means that it was known already. Fully agreed, it went through because of a lack of hours per day and the fact that the priority of developers was elsewhere. For me, for example, the priorities have mostly been with bugs that bothered me or that bothered Conectiva's customers. If you _really_ feel this strongly about the bug, you could either try to increase the number of hours a day for all of I sure wish I could :-). us or you could talk to my boss about hiring me as a consultant to fix the problem for you on an emergency basis :) The other two alternatives would be either waiting until somebody gets around to fixing the bug or sending in a patch yourself. Trying to piss off developers has adverse effect on all four of the methods above :) Why should my comments piss anybody off? I am just trying to point out a problem, as I see it, an offer suggestions for improvement. Other developers will either agree with me or they wont. Contributions are not made only through writing code. I contribute through code, bug reports, ideas, and suggestions. I would love to dive in and try to help fix some of the kernel problems but my hands are just to full right now. My comments are not meant to rush anybody and I am not criticizing how long it is taking. I know everybody is doing everything they can just like I am, and they are doing a terrific job. I am just suggesting a modification to the way the kernels are distributed that is more like the early versions that I hoped would allow us to maintain a stable kernel for distributions and production machines. - Vincent Stemen - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Marcelo Tosatti wrote: On Wed, 30 May 2001, Mike Galbraith wrote: On Wed, 30 May 2001, Rik van Riel wrote: On Wed, 30 May 2001, Marcelo Tosatti wrote: The problem is that we allow _every_ task to age pages on the system at the same time --- this is one of the things which is fucking up. This should not have any effect on the ratio of cache reclaiming vs. swapout use, though... It shouldn't.. but when many tasks are aging, it does. What Rik means is that they are independant problems. Ok. Excluding these guys certainly seems to make a difference. Sure, those guys are going to help kswapd to unmap pte's and allocate swap space. Now even if only kswapd does this job (meaning a sane amount of cache reclaims/swapouts), you still have to deal with the reclaim/swapout tradeoff. See? Yes. -Mike - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Wednesday 30 May 2001 15:17, Mike Galbraith wrote: On Wed, 30 May 2001, Vincent Stemen wrote: On Wednesday 30 May 2001 01:02, Mike Galbraith wrote: On Tue, 29 May 2001, Vincent Stemen wrote: On Tuesday 29 May 2001 15:16, Alan Cox wrote: a reasonably stable release until 2.2.12. I do not understand why code with such serious reproducible problems is being introduced into the even numbered kernels. What happened to the plan to use only the Who said it was introduced ?? It was more 'lurking' than introduced. And unfortunately nobody really pinned it down in 2.4test. I fail to see the distinction. First of all, why was it ever released as 2.4-test? That question should probably be directed at Linus. If it is not fully tested, then it should be released it as an odd number. If it already existed in the odd numbered development kernel and was known, then it should have never been released as a production kernel until it was resolved. Otherwise, it completely defeats the purpose of having the even/odd numbering system. I do not expect bugs to never slip through to production kernels, but known bugs that are not trivial should not, and serious bugs like these VM problems especially should not. And you can help prevent them from slipping through by signing up as a shake and bake tester. Indeed, you can make your expectations reality absolutely free of charge, microfont and or compensation /microfont what a bargain! X ___ ;-) -Mike The problem is, that's not true. These problems are not slipping through because of lack of testers. As Alan said, the VM problem has Sorry, that's a copout. You (we) had many chances to notice. Don't push the problems back onto developers.. it's our problem. How is that a copout? The problem was noticed. I am only suggesting that we not be in such a hurry to put code in the production kernels until we are pretty sure it works well enough, and that we release major production versions more often so that they do not contain 2 or 3 years worth of new code making it so hard to debug. We probably should have had 2 or 3 code freezes and production releases since 2.2.x. As I mentioned in a previous posting, this way we do not have to run a 2 or 3 year old kernel in order to have reasonable stability. Here are some of the problems I see: There was far to long of a stretch with to much code dumped into both the 2.2 and 2.4 kernels before release. There needs to be a smaller number changes between major releases so that they can be more thoroughly tested and debugged. In the race to get it out there they are making the same mistakes as Microsoft, releasing production kernels with known serious bugs because it is taking to long and they want to move on forward. I enjoy criticizing Microsoft so much for the same thing that I do not want to have to stop in order to not sound hypocritical :-). The Linux community has built a lot of it's reputation on not making these mistakes. Please lets try not to destroy that. They are disregarding the even/odd versioning system. For example: There was a new 8139too driver added to the the 2.4.5 (I think) kernel which Alan Cox took back out and reverted to the old one in his 2.4.5-ac? versions because it is apparently causing lockups. Shouldn't this new driver have been released in a 2.5.x development kernel and proven there before replacing the one in the production kernel? I haven't even seen a 2.5.x kernel released yet. Based on Linus's original very good plan for even/odd numbers, there should not have been 2.4.0-test? kernels either. This was another example of the rush to increment to 2.4 long before it was ready. There was a long stretch of test kernels and and now we are all the way to 2.4.5 and it is still not stable. We are repeating the 2.2.x process all over again. It should have been 2.3.x until the production release was ready. If they needed to distinguish a code freeze for final testing, it could be done with a 4th version component (2.3.xx.xx), where the 4 component is incremented for final bug fixes. Sorry, I disagree with every last bit. Either you accept a situation or you try to do something about it. -Mike I am spending a lot of time testing new kernels, reporting bugs and offering suggestions that I think may improve on the stability of production kernels. Is this not considered doing something about it? It is necessary to point out where one sees a problem in order to offer possible solutions for improvement. - Vincent - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Wed, 30 May 2001, Vincent Stemen wrote: The problem is, that's not true. These problems are not slipping through because of lack of testers. As Alan said, the VM problem has been lurking, which means that it was known already. Fully agreed, it went through because of a lack of hours per day and the fact that the priority of developers was elsewhere. For me, for example, the priorities have mostly been with bugs that bothered me or that bothered Conectiva's customers. If you _really_ feel this strongly about the bug, you could either try to increase the number of hours a day for all of us or you could talk to my boss about hiring me as a consultant to fix the problem for you on an emergency basis :) The other two alternatives would be either waiting until somebody gets around to fixing the bug or sending in a patch yourself. Trying to piss off developers has adverse effect on all four of the methods above :) Rik -- Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://distro.conectiva.com/ Send all your spam to [EMAIL PROTECTED] (spam digging piggy) - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Wed, 30 May 2001, Vincent Stemen wrote: On Wednesday 30 May 2001 15:17, Mike Galbraith wrote: On Wed, 30 May 2001, Vincent Stemen wrote: On Wednesday 30 May 2001 01:02, Mike Galbraith wrote: On Tue, 29 May 2001, Vincent Stemen wrote: On Tuesday 29 May 2001 15:16, Alan Cox wrote: a reasonably stable release until 2.2.12. I do not understand why code with such serious reproducible problems is being introduced into the even numbered kernels. What happened to the plan to use only the Who said it was introduced ?? It was more 'lurking' than introduced. And unfortunately nobody really pinned it down in 2.4test. I fail to see the distinction. First of all, why was it ever released as 2.4-test? That question should probably be directed at Linus. If it is not fully tested, then it should be released it as an odd number. If it already existed in the odd numbered development kernel and was known, then it should have never been released as a production kernel until it was resolved. Otherwise, it completely defeats the purpose of having the even/odd numbering system. I do not expect bugs to never slip through to production kernels, but known bugs that are not trivial should not, and serious bugs like these VM problems especially should not. And you can help prevent them from slipping through by signing up as a shake and bake tester. Indeed, you can make your expectations reality absolutely free of charge, microfont and or compensation /microfont what a bargain! X ___ ;-) -Mike The problem is, that's not true. These problems are not slipping through because of lack of testers. As Alan said, the VM problem has Sorry, that's a copout. You (we) had many chances to notice. Don't push the problems back onto developers.. it's our problem. How is that a copout? The problem was noticed. I am only suggesting that we not be in such a hurry to put code in the production kernels until we are pretty sure it works well enough, and that we release major production versions more often so that they do not contain 2 or 3 years worth of new code making it so hard to debug. We probably should have had 2 or 3 code freezes and production releases since 2.2.x. As I mentioned in a previous posting, this way we do not have to run a 2 or 3 year old kernel in order to have reasonable stability. I don't think you or I can do a better job of release management than Linus and friends, so there's no point in us discussing it. If you want to tell Linus, Alan et al how to do it 'right', you go do that. -Mike - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Wednesday 30 May 2001 01:02, Mike Galbraith wrote: On Tue, 29 May 2001, Vincent Stemen wrote: On Tuesday 29 May 2001 15:16, Alan Cox wrote: a reasonably stable release until 2.2.12. I do not understand why code with such serious reproducible problems is being introduced into the even numbered kernels. What happened to the plan to use only the Who said it was introduced ?? It was more 'lurking' than introduced. And unfortunately nobody really pinned it down in 2.4test. I fail to see the distinction. First of all, why was it ever released as 2.4-test? That question should probably be directed at Linus. If it is not fully tested, then it should be released it as an odd number. If it already existed in the odd numbered development kernel and was known, then it should have never been released as a production kernel until it was resolved. Otherwise, it completely defeats the purpose of having the even/odd numbering system. I do not expect bugs to never slip through to production kernels, but known bugs that are not trivial should not, and serious bugs like these VM problems especially should not. And you can help prevent them from slipping through by signing up as a shake and bake tester. Indeed, you can make your expectations reality absolutely free of charge, microfont and or compensation /microfont what a bargain! X ___ ;-) -Mike The problem is, that's not true. These problems are not slipping through because of lack of testers. As Alan said, the VM problem has been lurking, which means that it was known already. We currently have no development/production kernel distinction and I have not been able to find even one stable 2.4.x version to run on our main machines. Reverting back to 2.2.x is a real pain because of all the surrounding changes which will affect our initscripts and other system configuration issues, such as Unix98 pty's, proc filesystem differences, device numbering, etc. I have the greatest respect and appreciation for Linus, Alan, and all of the other kernel developers. My comments are not meant to criticize, but rather to point out some the problems I see that are making it so difficult to stabilize the kernel and encourage them to steer back on track. Here are some of the problems I see: There was far to long of a stretch with to much code dumped into both the 2.2 and 2.4 kernels before release. There needs to be a smaller number changes between major releases so that they can be more thoroughly tested and debugged. In the race to get it out there they are making the same mistakes as Microsoft, releasing production kernels with known serious bugs because it is taking to long and they want to move on forward. I enjoy criticizing Microsoft so much for the same thing that I do not want to have to stop in order to not sound hypocritical :-). The Linux community has built a lot of it's reputation on not making these mistakes. Please lets try not to destroy that. They are disregarding the even/odd versioning system. For example: There was a new 8139too driver added to the the 2.4.5 (I think) kernel which Alan Cox took back out and reverted to the old one in his 2.4.5-ac? versions because it is apparently causing lockups. Shouldn't this new driver have been released in a 2.5.x development kernel and proven there before replacing the one in the production kernel? I haven't even seen a 2.5.x kernel released yet. Based on Linus's original very good plan for even/odd numbers, there should not have been 2.4.0-test? kernels either. This was another example of the rush to increment to 2.4 long before it was ready. There was a long stretch of test kernels and and now we are all the way to 2.4.5 and it is still not stable. We are repeating the 2.2.x process all over again. It should have been 2.3.x until the production release was ready. If they needed to distinguish a code freeze for final testing, it could be done with a 4th version component (2.3.xx.xx), where the 4 component is incremented for final bug fixes. - Vincent Stemen - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
There was a new 8139too driver added to the the 2.4.5 (I think) kernel which Alan Cox took back out and reverted to the old one in his 2.4.5-ac? versions because it is apparently causing lockups. Shouldn't this new driver have been released in a 2.5.x development kernel and proven there before replacing the one in the production kernel? I haven't even seen a 2.5.x kernel released yet. Nope. The 2.4.3 one is buggy too - but differently (and it turns out a little less) buggy. Welcome to software. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Tue, 29 May 2001, Vincent Stemen wrote: > On Tuesday 29 May 2001 15:16, Alan Cox wrote: > > > a reasonably stable release until 2.2.12. I do not understand why > > > code with such serious reproducible problems is being introduced into > > > the even numbered kernels. What happened to the plan to use only the > > > > Who said it was introduced ?? It was more 'lurking' than introduced. And > > unfortunately nobody really pinned it down in 2.4test. > > > > I fail to see the distinction. First of all, why was it ever released > as 2.4-test? That question should probably be directed at Linus. If > it is not fully tested, then it should be released it as an odd > number. If it already existed in the odd numbered development kernel > and was known, then it should have never been released as a production > kernel until it was resolved. Otherwise, it completely defeats the > purpose of having the even/odd numbering system. > > I do not expect bugs to never slip through to production kernels, but > known bugs that are not trivial should not, and serious bugs like > these VM problems especially should not. And you can help prevent them from slipping through by signing up as a shake and bake tester. Indeed, you can make your expectations reality absolutely free of charge, and or compensation what a bargain! X ___ ;-) -Mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
Mike Galbraith ([EMAIL PROTECTED]) wrote: > > Emphatic yes. We went from cache collapse to cache bloat. Rik, I think Mike deserves his beer. ;) Agreed. Swap reclaim doesn't seem to be the root issue here, IMHO. But instead: his box was capable of maintaining a modest cache and the desired user processes without massive allocations (and use) of swap space. There was plenty of cache to reap, but VM decided to swapout instead. Seems we're out of balance here (IMHO). I see this too, and it's only a symptom of post-2.4.4 kernels. Example: on a 128M system w/2.4.5, loading X and a simulation code of RSS=70M causes the system to drop 40-50M into swap...with 100M of cache sitting there, and some of those cache pages are fairly old. It's not just allocation; there is noticable disk activity associated with paging that causes a lag in interactivity. In 2.4.4, there is no swap activity at all. And if the application causes heavy I/O *and* memory load (think StarOffice, or Quake 3), this situation gets even worse (because there's typically more competition/demand for cache pages). And on low-memory systems (ex. 32M), even a basic Web browsing test w/ Opera drops the 2.4.5 system 25M into swap where 2.4.4 barely cracks 5 MB on the same test (and the interactivity shows). This is all independent of swap reclaim. So is there an ideal VM balance for everyone? I have found that low-RAM systems seem to benefit from being on the "cache-collapse" side of the curve (so I prefer the pre-2.4.5 balance more than Mike probably does) and those low-RAM systems are the first hit when, as now, we're favoring "cache bloat". Should balance behaviors could be altered by the user (via sysctl's maybe? Yeah, I hear the cringes)? Or better, is it possible to dynamically choose where the watermarks in balancing should lie, and alter them automatically? 2.5 stuff there, no doubt. Balancing seems so *fragile* (to me). Cheers, Craig Kulesa [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
> a reasonably stable release until 2.2.12. I do not understand why > code with such serious reproducible problems is being introduced into > the even numbered kernels. What happened to the plan to use only the Who said it was introduced ?? It was more 'lurking' than introduced. And unfortunately nobody really pinned it down in 2.4test. > By the way, The 2.4.5-ac3 kernel still fills swap and runs out of > memory during my morning NFS incremental backup. I got this message > in the syslog. 2.4.5-ac doesn't do some of the write throttling. Thats one thing I'm still working out. Linus 2.4.5 does write throttling but Im not convinced its done the right way > completely full. By that time the memory was in a reasonable state > but the swap space is still never being released. It wont be, its copied of memory already in apps. Linus said 2.4.0 would need more swap than ram when he put out 2.4.0. Alan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Tuesday 29 May 2001 10:37, elko wrote: > On Tuesday 29 May 2001 11:10, Alan Cox wrote: > > > It's not a bug. It's a feature. It only breaks systems that are run > > > w= ith "too > > > little" swap, and the only difference from 2.2 till now is, that the > > > de= finition > > > of "too little" changed. > > > > its a giant bug. Or do you want to add 128Gb of unused swap to a full > > kitted out Xeon box - or 512Gb to a big athlon ??? > > this bug is biting me too and I do NOT like it ! > > if it's a *giant* bug, then why is LK-2.4 called a *stable* kernel ?? This has been my complaint ever since the 2.2.0 kernel. I did not see a reasonably stable release until 2.2.12. I do not understand why code with such serious reproducible problems is being introduced into the even numbered kernels. What happened to the plan to use only the odd numbered kernels for debugging and refinement of the code? I never said anything because I thought the the kernel developers would eventually get back on track after the mistakes of the 2.2.x kernels but it has been years now and it still has not happened. I do not wish sound un-appreciative to those that have put so much wonderful work into the Linux kernel but this is the same thing we criticize Microsoft for. Putting out production code that obviously is not ready. Please lets not earn the same reputation of such commercial companies. By the way, The 2.4.5-ac3 kernel still fills swap and runs out of memory during my morning NFS incremental backup. I got this message in the syslog. May 29 06:39:15 (none) kernel: Out of Memory: Killed process 23502 (xteevee). For some reason xteevee is commonly the process that gets killed. My understanding is that it is part of Xscreensaver, but it was during my backup. This was the output of 'free' after I got up and found the swap completely full. By that time the memory was in a reasonable state but the swap space is still never being released. total used free sharedbuffers cached Mem:255960 220668 35292292 110960 80124 -/+ buffers/cache: 29584 226376 Swap:40124 40112 12 Configuration - AMD K6-2/450 256Mb RAM 2.4.5-ac3 Kernel compiled with egcs-1.1.2. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
On Tuesday 29 May 2001 11:10, Alan Cox wrote: > > It's not a bug. It's a feature. It only breaks systems that are run w= > > ith "too > > little" swap, and the only difference from 2.2 till now is, that the de= > > finition > > of "too little" changed. > > its a giant bug. Or do you want to add 128Gb of unused swap to a full > kitted out Xeon box - or 512Gb to a big athlon ??? this bug is biting me too and I do NOT like it ! if it's a *giant* bug, then why is LK-2.4 called a *stable* kernel ?? -- Elko Holl - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
> It's not a bug. It's a feature. It only breaks systems that are run w= > ith "too > little" swap, and the only difference from 2.2 till now is, that the de= > finition > of "too little" changed. its a giant bug. Or do you want to add 128Gb of unused swap to a full kitted out Xeon box - or 512Gb to a big athlon ??? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
> Ouch! When compiling MySql, building sql_yacc.cc results in a ~300M > cc1plus process size. Unfortunately this leads the machine with 380M of > RAM deeply into swap: > > Mem: 381608K av, 248504K used, 133104K free, 0K shrd, 192K > buff > Swap: 255608K av, 255608K used, 0K free 215744K > cached That is supposed to hapen. The pages are existing both in swap and memory but not recovered. In that state the VM hasn't even broken yet. Where you hit a problem is that the 255Mb of stuff both in memory and swap won't be flushed from swap when you need more swap space. That is a giant size special edition stupid design flaw that is on the VM hackers list. But there are only a finite number of patches you can do in a day, and things like sucking completely came first I believe - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
On Tuesday 29 May 2001 11:10, Alan Cox wrote: It's not a bug. It's a feature. It only breaks systems that are run w= ith too little swap, and the only difference from 2.2 till now is, that the de= finition of too little changed. its a giant bug. Or do you want to add 128Gb of unused swap to a full kitted out Xeon box - or 512Gb to a big athlon ??? this bug is biting me too and I do NOT like it ! if it's a *giant* bug, then why is LK-2.4 called a *stable* kernel ?? -- Elko Holl - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Tuesday 29 May 2001 10:37, elko wrote: On Tuesday 29 May 2001 11:10, Alan Cox wrote: It's not a bug. It's a feature. It only breaks systems that are run w= ith too little swap, and the only difference from 2.2 till now is, that the de= finition of too little changed. its a giant bug. Or do you want to add 128Gb of unused swap to a full kitted out Xeon box - or 512Gb to a big athlon ??? this bug is biting me too and I do NOT like it ! if it's a *giant* bug, then why is LK-2.4 called a *stable* kernel ?? This has been my complaint ever since the 2.2.0 kernel. I did not see a reasonably stable release until 2.2.12. I do not understand why code with such serious reproducible problems is being introduced into the even numbered kernels. What happened to the plan to use only the odd numbered kernels for debugging and refinement of the code? I never said anything because I thought the the kernel developers would eventually get back on track after the mistakes of the 2.2.x kernels but it has been years now and it still has not happened. I do not wish sound un-appreciative to those that have put so much wonderful work into the Linux kernel but this is the same thing we criticize Microsoft for. Putting out production code that obviously is not ready. Please lets not earn the same reputation of such commercial companies. By the way, The 2.4.5-ac3 kernel still fills swap and runs out of memory during my morning NFS incremental backup. I got this message in the syslog. May 29 06:39:15 (none) kernel: Out of Memory: Killed process 23502 (xteevee). For some reason xteevee is commonly the process that gets killed. My understanding is that it is part of Xscreensaver, but it was during my backup. This was the output of 'free' after I got up and found the swap completely full. By that time the memory was in a reasonable state but the swap space is still never being released. total used free sharedbuffers cached Mem:255960 220668 35292292 110960 80124 -/+ buffers/cache: 29584 226376 Swap:40124 40112 12 Configuration - AMD K6-2/450 256Mb RAM 2.4.5-ac3 Kernel compiled with egcs-1.1.2. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
Mike Galbraith ([EMAIL PROTECTED]) wrote: Emphatic yes. We went from cache collapse to cache bloat. Rik, I think Mike deserves his beer. ;) Agreed. Swap reclaim doesn't seem to be the root issue here, IMHO. But instead: his box was capable of maintaining a modest cache and the desired user processes without massive allocations (and use) of swap space. There was plenty of cache to reap, but VM decided to swapout instead. Seems we're out of balance here (IMHO). I see this too, and it's only a symptom of post-2.4.4 kernels. Example: on a 128M system w/2.4.5, loading X and a simulation code of RSS=70M causes the system to drop 40-50M into swap...with 100M of cache sitting there, and some of those cache pages are fairly old. It's not just allocation; there is noticable disk activity associated with paging that causes a lag in interactivity. In 2.4.4, there is no swap activity at all. And if the application causes heavy I/O *and* memory load (think StarOffice, or Quake 3), this situation gets even worse (because there's typically more competition/demand for cache pages). And on low-memory systems (ex. 32M), even a basic Web browsing test w/ Opera drops the 2.4.5 system 25M into swap where 2.4.4 barely cracks 5 MB on the same test (and the interactivity shows). This is all independent of swap reclaim. So is there an ideal VM balance for everyone? I have found that low-RAM systems seem to benefit from being on the cache-collapse side of the curve (so I prefer the pre-2.4.5 balance more than Mike probably does) and those low-RAM systems are the first hit when, as now, we're favoring cache bloat. Should balance behaviors could be altered by the user (via sysctl's maybe? Yeah, I hear the cringes)? Or better, is it possible to dynamically choose where the watermarks in balancing should lie, and alter them automatically? 2.5 stuff there, no doubt. Balancing seems so *fragile* (to me). Cheers, Craig Kulesa [EMAIL PROTECTED] - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
Ouch! When compiling MySql, building sql_yacc.cc results in a ~300M cc1plus process size. Unfortunately this leads the machine with 380M of RAM deeply into swap: Mem: 381608K av, 248504K used, 133104K free, 0K shrd, 192K buff Swap: 255608K av, 255608K used, 0K free 215744K cached That is supposed to hapen. The pages are existing both in swap and memory but not recovered. In that state the VM hasn't even broken yet. Where you hit a problem is that the 255Mb of stuff both in memory and swap won't be flushed from swap when you need more swap space. That is a giant size special edition stupid design flaw that is on the VM hackers list. But there are only a finite number of patches you can do in a day, and things like sucking completely came first I believe - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
a reasonably stable release until 2.2.12. I do not understand why code with such serious reproducible problems is being introduced into the even numbered kernels. What happened to the plan to use only the Who said it was introduced ?? It was more 'lurking' than introduced. And unfortunately nobody really pinned it down in 2.4test. By the way, The 2.4.5-ac3 kernel still fills swap and runs out of memory during my morning NFS incremental backup. I got this message in the syslog. 2.4.5-ac doesn't do some of the write throttling. Thats one thing I'm still working out. Linus 2.4.5 does write throttling but Im not convinced its done the right way completely full. By that time the memory was in a reasonable state but the swap space is still never being released. It wont be, its copied of memory already in apps. Linus said 2.4.0 would need more swap than ram when he put out 2.4.0. Alan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
It's not a bug. It's a feature. It only breaks systems that are run w= ith too little swap, and the only difference from 2.2 till now is, that the de= finition of too little changed. its a giant bug. Or do you want to add 128Gb of unused swap to a full kitted out Xeon box - or 512Gb to a big athlon ??? - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Tue, 29 May 2001, Vincent Stemen wrote: On Tuesday 29 May 2001 15:16, Alan Cox wrote: a reasonably stable release until 2.2.12. I do not understand why code with such serious reproducible problems is being introduced into the even numbered kernels. What happened to the plan to use only the Who said it was introduced ?? It was more 'lurking' than introduced. And unfortunately nobody really pinned it down in 2.4test. I fail to see the distinction. First of all, why was it ever released as 2.4-test? That question should probably be directed at Linus. If it is not fully tested, then it should be released it as an odd number. If it already existed in the odd numbered development kernel and was known, then it should have never been released as a production kernel until it was resolved. Otherwise, it completely defeats the purpose of having the even/odd numbering system. I do not expect bugs to never slip through to production kernels, but known bugs that are not trivial should not, and serious bugs like these VM problems especially should not. And you can help prevent them from slipping through by signing up as a shake and bake tester. Indeed, you can make your expectations reality absolutely free of charge, microfont and or compensation /microfont what a bargain! X ___ ;-) -Mike - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
On Tue, 29 May 2001, Jeff Garzik wrote: > > On Tuesday 29 May 2001 00:10, Jakob Østergaard wrote: > > > > > > > Mem: 381608K av, 248504K used, 133104K free, 0K shrd, 192K > > > > > buff > > > > > Swap: 255608K av, 255608K used, 0K free 215744K > > > > > cached > > > > > > > > > > Vanilla 2.4.5 VM. > > > > > It's not a bug. It's a feature. It only breaks systems that are run with > > > "too little" swap, and the only difference from 2.2 till now is, that the > > > definition of "too little" changed. > > I am surprised as many people as this are missing, > > * when you have an active process using ~300M of VM, in a ~380M machine, > 2/3 of the machine's RAM should -not- be soaked up by cache Emphatic yes. We went from cache collapse to cache bloat. IMHO, the bugfix for collapse exposed other problems. I posted a patch which I believe demonstrated that pretty well. (i also bet Rik a virtual beer that folks would knock on his mailbox when 2.4.5 was released. please cc him somebody.. i want my brewski;) -Mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
> On Tuesday 29 May 2001 00:10, Jakob Østergaard wrote: > > > > > Mem: 381608K av, 248504K used, 133104K free, 0K shrd, 192K > > > > buff > > > > Swap: 255608K av, 255608K used, 0K free 215744K > > > > cached > > > > > > > > Vanilla 2.4.5 VM. > > > It's not a bug. It's a feature. It only breaks systems that are run with > > "too little" swap, and the only difference from 2.2 till now is, that the > > definition of "too little" changed. I am surprised as many people as this are missing, * when you have an active process using ~300M of VM, in a ~380M machine, 2/3 of the machine's RAM should -not- be soaked up by cache * when you have an active process using ~300M of VM, in a ~380M machine, swap should not be full while there is 133M of RAM available. The above quoted is top output, taken during the several minutes where cc1plus process was ~300M in size. Similar numbers existed before and after my cut-n-paste, so this was not transient behavior. I can assure you, these are bugs not features :) -- Jeff Garzik | Disbelief, that's why you fail. Building 1024| MandrakeSoft | - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
On Tue, May 29, 2001 at 01:46:28PM +0900, G. Hugh Song wrote:
> Jakob,
>
> My Alpha has 2GB of physical memory. In this case how much swap space
> should
> I assign in these days of kernel 2.4.*? I had had trouble with 1GB of
> swap space
> before switching back to 2.2.20pre2aa1.
If you run a single mingetty and bash session, you need no swap.
If you run four 1GB processes concurrently, I would use ~5-6G of swap to be on
the safe side.
Swap is very cheap, even if measured in gigabytes. Go with the sum of the
largest process foot-prints you can imagine running on your system, and then
add some. Be generous. It's not like unused swap space is going to slow the
system down - it's a nice extra little safety to have. It's beyond me why
anyone would run a system with marginal swap.
On a compile box here with 392 MB physical, I have 900 MB swap. This
accomodates multiple concurrent 100-300 MB compile jobs. Never had a problem.
Oh, and I didn't have to change my swap setup between 2.2 and 2.4.
--
: [EMAIL PROTECTED] : And I see the elder races, :
:.: putrid forms of man:
: Jakob Østergaard : See him rise and claim the earth, :
:OZ9ABN : his downfall is at hand. :
:.:{Konkhra}...:
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
Jakob, My Alpha has 2GB of physical memory. In this case how much swap space should I assign in these days of kernel 2.4.*? I had had trouble with 1GB of swap space before switching back to 2.2.20pre2aa1. Thanks -- G. Hugh Song - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
On Tue, May 29, 2001 at 11:32:09AM +0900, G. Hugh Song wrote:
>
> Jeff Garzik wrote:
> >
> > Ouch! When compiling MySql, building sql_yacc.cc results in a ~300M
> > cc1plus process size. Unfortunately this leads the machine with 380M of
> > RAM deeply into swap:
> >
> > Mem: 381608K av, 248504K used, 133104K free, 0K shrd, 192K
> > buff
> > Swap: 255608K av, 255608K used, 0K free 215744K
> > cached
> >
> > Vanilla 2.4.5 VM.
> >
>
> This bug known as the swap-reclaim bug has been there for a while since
> around 2.4.4. Rick van Riel said that it is in the TO-DO list.
> Because of this, I went back to 2.2.20pre2aa1 on UP2000 SMP.
>
> IMHO, the current 2.4.* kernels should still be 2.3.*. When this bug
> is removed, I will come back to 2.4.*.
Just keep enough swap around. How hard can that be ?
Really, it's not like a memory leak or something. It's just "late reclaim".
If Linux didn't do over-commit, you wouldn't have been able to run that job
anyway.
It's not a bug. It's a feature. It only breaks systems that are run with "too
little" swap, and the only difference from 2.2 till now is, that the definition
of "too little" changed.
--
: [EMAIL PROTECTED] : And I see the elder races, :
:.: putrid forms of man:
: Jakob Østergaard : See him rise and claim the earth, :
:OZ9ABN : his downfall is at hand. :
:.:{Konkhra}...:
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
Jeff Garzik wrote: > > Ouch! When compiling MySql, building sql_yacc.cc results in a ~300M > cc1plus process size. Unfortunately this leads the machine with 380M of > RAM deeply into swap: > > Mem: 381608K av, 248504K used, 133104K free, 0K shrd, 192K > buff > Swap: 255608K av, 255608K used, 0K free 215744K > cached > > Vanilla 2.4.5 VM. > This bug known as the swap-reclaim bug has been there for a while since around 2.4.4. Rick van Riel said that it is in the TO-DO list. Because of this, I went back to 2.2.20pre2aa1 on UP2000 SMP. IMHO, the current 2.4.* kernels should still be 2.3.*. When this bug is removed, I will come back to 2.4.*. Regards, Hugh - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
Jeff Garzik wrote: > > Ouch! When compiling MySql, building sql_yacc.cc results in a ~300M > cc1plus process size. Unfortunately this leads the machine with 380M of > RAM deeply into swap: > > Mem: 381608K av, 248504K used, 133104K free, 0K shrd, 192K > buff > Swap: 255608K av, 255608K used, 0K free 215744K > cached > > Vanilla 2.4.5 VM. > Sorry. I just looked at your numbers again and saw you have 133 MB of real ram free. Is this during compile? -- = Mohammad A. Haque http://www.haque.net/ [EMAIL PROTECTED] "Alcohol and calculus don't mix. Project Lead Don't drink and derive." --Unknown http://wm.themes.org/ [EMAIL PROTECTED] = - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
Jeff Garzik wrote: Ouch! When compiling MySql, building sql_yacc.cc results in a ~300M cc1plus process size. Unfortunately this leads the machine with 380M of RAM deeply into swap: Mem: 381608K av, 248504K used, 133104K free, 0K shrd, 192K buff Swap: 255608K av, 255608K used, 0K free 215744K cached Vanilla 2.4.5 VM. Sorry. I just looked at your numbers again and saw you have 133 MB of real ram free. Is this during compile? -- = Mohammad A. Haque http://www.haque.net/ [EMAIL PROTECTED] Alcohol and calculus don't mix. Project Lead Don't drink and derive. --Unknown http://wm.themes.org/ [EMAIL PROTECTED] = - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
Jeff Garzik wrote: Ouch! When compiling MySql, building sql_yacc.cc results in a ~300M cc1plus process size. Unfortunately this leads the machine with 380M of RAM deeply into swap: Mem: 381608K av, 248504K used, 133104K free, 0K shrd, 192K buff Swap: 255608K av, 255608K used, 0K free 215744K cached Vanilla 2.4.5 VM. This bug known as the swap-reclaim bug has been there for a while since around 2.4.4. Rick van Riel said that it is in the TO-DO list. Because of this, I went back to 2.2.20pre2aa1 on UP2000 SMP. IMHO, the current 2.4.* kernels should still be 2.3.*. When this bug is removed, I will come back to 2.4.*. Regards, Hugh - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
On Tue, May 29, 2001 at 11:32:09AM +0900, G. Hugh Song wrote:
Jeff Garzik wrote:
Ouch! When compiling MySql, building sql_yacc.cc results in a ~300M
cc1plus process size. Unfortunately this leads the machine with 380M of
RAM deeply into swap:
Mem: 381608K av, 248504K used, 133104K free, 0K shrd, 192K
buff
Swap: 255608K av, 255608K used, 0K free 215744K
cached
Vanilla 2.4.5 VM.
This bug known as the swap-reclaim bug has been there for a while since
around 2.4.4. Rick van Riel said that it is in the TO-DO list.
Because of this, I went back to 2.2.20pre2aa1 on UP2000 SMP.
IMHO, the current 2.4.* kernels should still be 2.3.*. When this bug
is removed, I will come back to 2.4.*.
Just keep enough swap around. How hard can that be ?
Really, it's not like a memory leak or something. It's just late reclaim.
If Linux didn't do over-commit, you wouldn't have been able to run that job
anyway.
It's not a bug. It's a feature. It only breaks systems that are run with too
little swap, and the only difference from 2.2 till now is, that the definition
of too little changed.
--
: [EMAIL PROTECTED] : And I see the elder races, :
:.: putrid forms of man:
: Jakob Østergaard : See him rise and claim the earth, :
:OZ9ABN : his downfall is at hand. :
:.:{Konkhra}...:
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
Jakob, My Alpha has 2GB of physical memory. In this case how much swap space should I assign in these days of kernel 2.4.*? I had had trouble with 1GB of swap space before switching back to 2.2.20pre2aa1. Thanks -- G. Hugh Song - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
On Tue, May 29, 2001 at 01:46:28PM +0900, G. Hugh Song wrote:
Jakob,
My Alpha has 2GB of physical memory. In this case how much swap space
should
I assign in these days of kernel 2.4.*? I had had trouble with 1GB of
swap space
before switching back to 2.2.20pre2aa1.
If you run a single mingetty and bash session, you need no swap.
If you run four 1GB processes concurrently, I would use ~5-6G of swap to be on
the safe side.
Swap is very cheap, even if measured in gigabytes. Go with the sum of the
largest process foot-prints you can imagine running on your system, and then
add some. Be generous. It's not like unused swap space is going to slow the
system down - it's a nice extra little safety to have. It's beyond me why
anyone would run a system with marginal swap.
On a compile box here with 392 MB physical, I have 900 MB swap. This
accomodates multiple concurrent 100-300 MB compile jobs. Never had a problem.
Oh, and I didn't have to change my swap setup between 2.2 and 2.4.
--
: [EMAIL PROTECTED] : And I see the elder races, :
:.: putrid forms of man:
: Jakob Østergaard : See him rise and claim the earth, :
:OZ9ABN : his downfall is at hand. :
:.:{Konkhra}...:
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
On Tuesday 29 May 2001 00:10, Jakob Østergaard wrote: Mem: 381608K av, 248504K used, 133104K free, 0K shrd, 192K buff Swap: 255608K av, 255608K used, 0K free 215744K cached Vanilla 2.4.5 VM. It's not a bug. It's a feature. It only breaks systems that are run with too little swap, and the only difference from 2.2 till now is, that the definition of too little changed. I am surprised as many people as this are missing, * when you have an active process using ~300M of VM, in a ~380M machine, 2/3 of the machine's RAM should -not- be soaked up by cache * when you have an active process using ~300M of VM, in a ~380M machine, swap should not be full while there is 133M of RAM available. The above quoted is top output, taken during the several minutes where cc1plus process was ~300M in size. Similar numbers existed before and after my cut-n-paste, so this was not transient behavior. I can assure you, these are bugs not features :) -- Jeff Garzik | Disbelief, that's why you fail. Building 1024| MandrakeSoft | - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
On Tue, 29 May 2001, Jeff Garzik wrote: On Tuesday 29 May 2001 00:10, Jakob Østergaard wrote: Mem: 381608K av, 248504K used, 133104K free, 0K shrd, 192K buff Swap: 255608K av, 255608K used, 0K free 215744K cached Vanilla 2.4.5 VM. It's not a bug. It's a feature. It only breaks systems that are run with too little swap, and the only difference from 2.2 till now is, that the definition of too little changed. I am surprised as many people as this are missing, * when you have an active process using ~300M of VM, in a ~380M machine, 2/3 of the machine's RAM should -not- be soaked up by cache Emphatic yes. We went from cache collapse to cache bloat. IMHO, the bugfix for collapse exposed other problems. I posted a patch which I believe demonstrated that pretty well. (i also bet Rik a virtual beer that folks would knock on his mailbox when 2.4.5 was released. please cc him somebody.. i want my brewski;) -Mike - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
