Re: Replacement for grep(1) (part 2)

1999-07-17 Thread Matthew Dillon
: :It results sometimes in out of swap, too. : : Inetd is rate-limited by default nowadays, so this really doesn't apply. : :It really does apply. Inetd limits incoming connections per minute, not per :second. It is possible to use minute limit in a few seconds and cause a high :load. Sendmail

Re: Replacement for grep(1) (part 2)

1999-07-17 Thread Valentin Nechayev
Brian F. Feldman wrote: There are other ways. For example, even if a user account is resource limited, root processes (such as sendmail, popper, identd, and so forth) are not. Attacks against these servers generally result in very high loads and sometimes make it difficult

Re: Replacement for grep(1) (part 2)

1999-07-17 Thread Valentin Nechayev
Matthew Dillon wrote: Give me a shell and I can crash any machine. Oh. ;| A good example of this is sendmail. Before the MaxDaemonChildren and MaxArticleSize options, it was possible for sendmail to overcommit a machine. In this case the overcommit that can occur is with

Re: Replacement for grep(1) (part 2)

1999-07-17 Thread Matthew Dillon
: :It results sometimes in out of swap, too. : : Inetd is rate-limited by default nowadays, so this really doesn't apply. : :It really does apply. Inetd limits incoming connections per minute, not per :second. It is possible to use minute limit in a few seconds and cause a high :load. Sendmail is

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-16 Thread Narvi
[cc: list trimmed] On Thu, 15 Jul 1999 [EMAIL PROTECTED] wrote: In that scenario, the 512MB of swap I assigned to this machine would be dangerously low. With 13GB disks available for a couple of hundred bucks, my machines aren't going to run out of swap space any time soon,

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-16 Thread Daniel C. Sobral
Matthew Dillon wrote: Something is weird here. If the solaris people are using a SWAPSIZE + REALMEM VM model, they have to allow the allocated + reserved space go +REALMEM bytes over available swap space. If not they are using only a SWAPSIZE VM model. I did not check if

Re: Replacement for grep(1) (part 2)

1999-07-16 Thread Ville-Pertti Keinonen
[EMAIL PROTECTED] (Chris G. Demetriou) writes: Matthew Dillon [EMAIL PROTECTED] writes: The text size of a program is irrelevant, because swap is never allocated for it. The data and BSS are only relevant when they No, you can mprotect read-only vnode mappings to writable. Most

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-16 Thread Sean Witham
"Daniel C. Sobral" wrote: It would be nice to have a way to indicate that, a la SIGDANGER. Ok, everybody is avoiding this, so I'll comment. Yes, this would be interesting, and a good implementation will very probably be committed. *BUT*, this is not as useful as it seems. Since the

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-16 Thread Matthew Dillon
: :For those who wish to develop code for safety related systems that is :not good enough. They have to prove that all code can handle the :degradation :of resources gracefully. Such code relies on guaranteed memory :allocations :or in the very least warnings of memory shortage and prioritized

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-16 Thread David Brownlee
On Fri, 16 Jul 1999, Matthew Dillon wrote: I'm sorry, but when you write code for a safety related system you do not dynamically allocate memory at all. It's all essentially static. There is no issue with the memory resource. Besides, none of the BSD's are certified for any

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-16 Thread Alan C. Horn
On Fri, 16 Jul 1999, Matthew Dillon wrote: : Well, NetBSD is slated to be used in the 'Space Acceleration : Measurement System II', measuring the microgravity environment on : the International Space Station using a distributed system based : on several NetBSD/i386 boxes. :

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-16 Thread Daniel Eischen
I'm sorry, but when you write code for a safety related system you do not dynamically allocate memory at all. It's all essentially static. There is no issue with the memory resource. Besides, none of the BSD's are certified for any of that stuff that I know of. Sometimes

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-16 Thread Matthew Dillon
: I'm sorry, but when you write code for a safety related system you : do not dynamically allocate memory at all. It's all essentially static. : There is no issue with the memory resource. Besides, none of the BSD's are : certified for any of that stuff that I know of. :

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-16 Thread David Scheidt
On Fri, 16 Jul 1999, Daniel C. Sobral wrote: Technical follow-up: Contrary to what I previously said, a number of tests reveal that Solaris, indeed, does not overcommit. All non-read only segments, Neither does HP/UX 10.x. (Haven't got an 11 box handy to check.) The memory allocation

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-16 Thread Dominic Mitchell
On Thu, Jul 15, 1999 at 09:57:31PM -0700, Matthew Dillon wrote: Something is weird here. If the solaris people are using a SWAPSIZE + REALMEM VM model, they have to allow the allocated + reserved space go +REALMEM bytes over available swap space. If not they are using only

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-16 Thread Narvi
[cc: list trimmed] On Thu, 15 Jul 1999 lyn...@orthanc.ab.ca wrote: In that scenario, the 512MB of swap I assigned to this machine would be dangerously low. With 13GB disks available for a couple of hundred bucks, my machines aren't going to run out of swap space any time soon,

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-16 Thread Daniel C. Sobral
Matthew Dillon wrote: Something is weird here. If the solaris people are using a SWAPSIZE + REALMEM VM model, they have to allow the allocated + reserved space go +REALMEM bytes over available swap space. If not they are using only a SWAPSIZE VM model. I did not check if

Re: Replacement for grep(1) (part 2)

1999-07-16 Thread Valentin Nechayev
Daniel C. Sobral wrote: Eh? Reasonable programs *never* run into trouble. Trouble only happens when you have unreasonable programs around, or did not configure the system correctly. And if you did not configure the system correctly, why do you think you would be able to correctly estimate

Re: Replacement for grep(1) (part 2)

1999-07-16 Thread Ville-Pertti Keinonen
jul...@whistle.com (Julian Elischer) writes: If you wanted to fix this, you could add a patch to malloc that touched every page that it handed to the application. (and trapped sig11s) How would you expect that to work? Several misunderstandings seem to be common regarding this issue (most

Re: Replacement for grep(1) (part 2)

1999-07-16 Thread Ville-Pertti Keinonen
c...@netbsd.org (Chris G. Demetriou) writes: Matthew Dillon dil...@apollo.backplane.com writes: The text size of a program is irrelevant, because swap is never allocated for it. The data and BSS are only relevant when they No, you can mprotect read-only vnode mappings to writable.

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-16 Thread Patrick Welche
Matthew Dillon wrote: :On Tue, 13 Jul 1999 23:18:58 -0400 (EDT) : John Baldwin jobal...@vt.edu wrote: : : What does that have to do with overcommit? I student administrate a undergrad : CS lab at a university, and when student's programs misbehaved, they generate a : fault and are

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-16 Thread Daniel C. Sobral
Patrick Welche wrote: students != hostile users We obviously have known different students... :-) Making mistakes is part of learning. A hostile user is one which will act in a non-friendly manner. Whether intentionaly or not is irrelevant from the point of view of the administrator, as far

Re: Replacement for grep(1) (part 2)

1999-07-16 Thread Valentin Nechayev
Daniel C. Sobral wrote: 4.4BSD derived system cannot do this, and have to use different machine for such applications. Incorrect. We can set *limits* to the users, so they won't be able to crash down the system. No. Really, not all users are used system in the same time. And it is too

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-16 Thread Sean Witham
Daniel C. Sobral wrote: It would be nice to have a way to indicate that, a la SIGDANGER. Ok, everybody is avoiding this, so I'll comment. Yes, this would be interesting, and a good implementation will very probably be committed. *BUT*, this is not as useful as it seems. Since the

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-16 Thread Matthew Dillon
: :For those who wish to develop code for safety related systems that is :not good enough. They have to prove that all code can handle the :degradation :of resources gracefully. Such code relies on guaranteed memory :allocations :or in the very least warnings of memory shortage and prioritized

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-16 Thread David Brownlee
On Fri, 16 Jul 1999, Matthew Dillon wrote: I'm sorry, but when you write code for a safety related system you do not dynamically allocate memory at all. It's all essentially static. There is no issue with the memory resource. Besides, none of the BSD's are certified for

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-16 Thread Matthew Dillon
: Well, NetBSD is slated to be used in the 'Space Acceleration : Measurement System II', measuring the microgravity environment on : the International Space Station using a distributed system based : on several NetBSD/i386 boxes. : : Sometimes your 'what-if' senarios

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-16 Thread Alan C. Horn
On Fri, 16 Jul 1999, Matthew Dillon wrote: : Well, NetBSD is slated to be used in the 'Space Acceleration : Measurement System II', measuring the microgravity environment on : the International Space Station using a distributed system based : on several NetBSD/i386 boxes. : :

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-16 Thread Daniel Eischen
I'm sorry, but when you write code for a safety related system you do not dynamically allocate memory at all. It's all essentially static. There is no issue with the memory resource. Besides, none of the BSD's are certified for any of that stuff that I know of. Sometimes

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-16 Thread Matthew Dillon
: I'm sorry, but when you write code for a safety related system you : do not dynamically allocate memory at all. It's all essentially static. : There is no issue with the memory resource. Besides, none of the BSD's are : certified for any of that stuff that I know of. :

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-16 Thread Brian F. Feldman
Can we kill this thread already? This resolves nothing. The only good to come of this is all of the nice doc-proj input Matt is providing (and providing well, I might add.) There is no point that hasn't been rehashed a dozen times over, and you (the ones who want overcommitting turned off) are

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-16 Thread David Scheidt
On Fri, 16 Jul 1999, Daniel C. Sobral wrote: Technical follow-up: Contrary to what I previously said, a number of tests reveal that Solaris, indeed, does not overcommit. All non-read only segments, Neither does HP/UX 10.x. (Haven't got an 11 box handy to check.) The memory allocation

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Michael Schuster - TSC SunOS Germany
Hi everyone, I've been following this discussion almost from the beginning, and I have the feeling that we're not _really_ getting very far. There's good arguments for and against overcommit, depending on your point of view and your requirements. What I do see is a not-so-openly voiced consent

Re: Replacement for grep(1) (part 2)

1999-07-15 Thread Daniel C. Sobral
Kevin Schoedel wrote: Imagine a reasonably big program, like Netscape or Emacs, of which you usually just use a subset of features. There can easily be many megabytes of code and data in them you never actually use, or you don't _usually_ use (like the people who use emacs like it was vi

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Garance A Drosihn
At 6:29 PM -0700 7/14/99, Matthew Dillon wrote: If 1G isn't enough, spend another $30 and throw 2G of swap online. Or perhaps dedicate an entire $150 disk and throw 6+ GB of swap online. The equivalent setup using a non-overcommit model would require considerably more swap

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Noriyuki Soda
On Thu, 15 Jul 1999, Daniel C. Sobral wrote: Uh... like any modern unix, Solaris overcommits. On Thu, 15 Jul 1999 08:46:36 -0700 (PDT), "Eduardo E. Horvath" [EMAIL PROTECTED] said: Where do you guys get this misinformation? : Note the `19464k reserved'; that space has

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Matthew Dillon
:Both Dillon and Sobral mistakenly claimed that "Solaris overcommits", :this fact seems to be somewhat suggestive. : :And also, the followings are allocated memory and reserved memory :in my environment. (This table also includes Eduardo's example) : : SunOS allocated reservedtotal

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Matthew Dillon
:"pstat -s" on SunOS4, and "swap -s" on SunOS5. From Solaris man page: : ::-s Print summary information about total swap :: space usage and availability: :: :: allocated The total amount of swap space ::

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Matthew Dillon
::-s Print summary information about total swap :: space usage and availability: :: :: allocated The total amount of swap space :: (in 1024-byte blocks) ::

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Andrzej Bialecki
On Wed, 14 Jul 1999, John Nemeth wrote: On Jul 15, 2:40am, "Daniel C. Sobral" wrote: } Garance A Drosihn wrote: } At 12:20 AM +0900 7/15/99, Daniel C. Sobral wrote: } In which case the program that consumed all memory will be killed. } The program killed is +NOT+ the one demanding

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Matthew Dillon
:Before program start: :total: 2k bytes allocated + 4792k reserved = 24792k used, 191048k available : :After malloc, before touch: :total: 18756k bytes allocated + 37500k reserved = 56256k used, 159580k available : :After malloc + touch: :total: 52804k bytes allocated + 4852k reserved =

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread sthaug
If this is correct, then solaris is using a VMSPACE = SWAPSPACE model. FreeBSD uses a VMSPACE = SWAPSPACE + REALMEM model. AFAIK it has been stated quite explicitly by the Solaris folks that Solaris 2.x uses VMSPACE = SWAPSPACE + REALMEM. This is *different* from SunOS 4.1.x. Steinar

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Matthew Dillon
Here is what I get from one of BEST's mail www proxy machines. ~dillon/br adds the object size's together. 'swap' and 'default' objects refers to unbacked VM objects - and none of the processes running fork shared unbacked objects so we don't have to worry about that. The

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread lyndon
In that scenario, the 512MB of swap I assigned to this machine would be dangerously low. With 13GB disks available for a couple of hundred bucks, my machines aren't going to run out of swap space any time soon, even if I commit to disk. All I want for Christmas is a knob to disable

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Sheldon Hearn
On Thu, 15 Jul 1999 17:53:52 CST, [EMAIL PROTECTED] wrote: All I want for Christmas is a knob to disable overcommit. And what I'm pretty sure the majority of the readers on this list want is for those of you who really think it's necessary to do it yourselves. What? Nobody who wants to

re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread matthew green
All I want for Christmas is a knob to disable overcommit. And what I'm pretty sure the majority of the readers on this list want is for those of you who really think it's necessary to do it yourselves. What? Nobody who wants to disable the policy knows how to do it? Hmmm,

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Matthew Dillon
: In that scenario, the 512MB of swap I assigned to this machine would be : dangerously low. : :With 13GB disks available for a couple of hundred bucks, my machines aren't :going to run out of swap space any time soon, even if I commit to disk. : :All I want for Christmas is a knob to

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Matthew Dillon
:Technical follow-up: : :Contrary to what I previously said, a number of tests reveal that :Solaris, indeed, does not overcommit. All non-read only segments, :and all malloc()ed memory is reserved upon exec() or fork(), and the :reserved memory is not allowed to exceed the total memory. It makes

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Michael Schuster - TSC SunOS Germany
Hi everyone, I've been following this discussion almost from the beginning, and I have the feeling that we're not _really_ getting very far. There's good arguments for and against overcommit, depending on your point of view and your requirements. What I do see is a not-so-openly voiced consent

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Garance A Drosihn
At 6:29 PM -0700 7/14/99, Matthew Dillon wrote: If 1G isn't enough, spend another $30 and throw 2G of swap online. Or perhaps dedicate an entire $150 disk and throw 6+ GB of swap online. The equivalent setup using a non-overcommit model would require considerably more swap to

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Noriyuki Soda
On Thu, 15 Jul 1999, Daniel C. Sobral wrote: Uh... like any modern unix, Solaris overcommits. On Thu, 15 Jul 1999 08:46:36 -0700 (PDT), Eduardo E. Horvath e...@one-o.com said: Where do you guys get this misinformation? : Note the `19464k reserved'; that space has been

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Matthew Dillon
:Both Dillon and Sobral mistakenly claimed that Solaris overcommits, :this fact seems to be somewhat suggestive. : :And also, the followings are allocated memory and reserved memory :in my environment. (This table also includes Eduardo's example) : : SunOS allocated reservedtotal

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Noriyuki Soda
On Thu, 15 Jul 1999 11:09:01 -0700 (PDT), Matthew Dillon dil...@apollo.backplane.com said: Umm... how are you getting the reserved numbers? pstat -s on SunOS4, and swap -s on SunOS5. From Solaris man page: :-s Print summary information about total swap :

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Matthew Dillon
:pstat -s on SunOS4, and swap -s on SunOS5. From Solaris man page: : ::-s Print summary information about total swap :: space usage and availability: :: :: allocated The total amount of swap space :: (in

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Matthew Dillon
::-s Print summary information about total swap :: space usage and availability: :: :: allocated The total amount of swap space :: (in 1024-byte blocks) ::

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Andrzej Bialecki
On Wed, 14 Jul 1999, John Nemeth wrote: On Jul 15, 2:40am, Daniel C. Sobral wrote: } Garance A Drosihn wrote: } At 12:20 AM +0900 7/15/99, Daniel C. Sobral wrote: } In which case the program that consumed all memory will be killed. } The program killed is +NOT+ the one demanding

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Matthew Dillon
:Before program start: :total: 2k bytes allocated + 4792k reserved = 24792k used, 191048k available : :After malloc, before touch: :total: 18756k bytes allocated + 37500k reserved = 56256k used, 159580k available : :After malloc + touch: :total: 52804k bytes allocated + 4852k reserved =

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Jonathan Lemon
In article local.mail.freebsd-hackers/199907151825.laa11...@apollo.backplane.com you write: ::-s Print summary information about total swap :: space usage and availability: :: :: allocated The total amount of swap space ::

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread sthaug
If this is correct, then solaris is using a VMSPACE = SWAPSPACE model. FreeBSD uses a VMSPACE = SWAPSPACE + REALMEM model. AFAIK it has been stated quite explicitly by the Solaris folks that Solaris 2.x uses VMSPACE = SWAPSPACE + REALMEM. This is *different* from SunOS 4.1.x. Steinar

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Matthew Dillon
Here is what I get from one of BEST's mail www proxy machines. ~dillon/br adds the object size's together. 'swap' and 'default' objects refers to unbacked VM objects - and none of the processes running fork shared unbacked objects so we don't have to worry about that. The

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread lyndon
In that scenario, the 512MB of swap I assigned to this machine would be dangerously low. With 13GB disks available for a couple of hundred bucks, my machines aren't going to run out of swap space any time soon, even if I commit to disk. All I want for Christmas is a knob to disable

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Sheldon Hearn
On Thu, 15 Jul 1999 17:53:52 CST, lyn...@orthanc.ab.ca wrote: All I want for Christmas is a knob to disable overcommit. And what I'm pretty sure the majority of the readers on this list want is for those of you who really think it's necessary to do it yourselves. What? Nobody who wants to

re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread matthew green
All I want for Christmas is a knob to disable overcommit. And what I'm pretty sure the majority of the readers on this list want is for those of you who really think it's necessary to do it yourselves. What? Nobody who wants to disable the policy knows how to do it? Hmmm,

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread lyndon
And what I'm pretty sure the majority of the readers on this list want is for those of you who really think it's necessary to do it yourselves. What? Nobody who wants to disable the policy knows how to do it? Hmmm, I wonder whether that's significant... Sheldon, if you can't contribute

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Matthew Dillon
: In that scenario, the 512MB of swap I assigned to this machine would be : dangerously low. : :With 13GB disks available for a couple of hundred bucks, my machines aren't :going to run out of swap space any time soon, even if I commit to disk. : :All I want for Christmas is a knob to

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Daniel C. Sobral
Technical follow-up: Contrary to what I previously said, a number of tests reveal that Solaris, indeed, does not overcommit. All non-read only segments, and all malloc()ed memory is reserved upon exec() or fork(), and the reserved memory is not allowed to exceed the total memory. It makes

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-15 Thread Matthew Dillon
:Technical follow-up: : :Contrary to what I previously said, a number of tests reveal that :Solaris, indeed, does not overcommit. All non-read only segments, :and all malloc()ed memory is reserved upon exec() or fork(), and the :reserved memory is not allowed to exceed the total memory. It makes

Re: Replacement for grep(1) (part 2)

1999-07-14 Thread Doug Rabson
On Tue, 13 Jul 1999, Jon Ribbens wrote: Alfred Perlstein [EMAIL PROTECTED] wrote: You're browsing with netscape and It hits about 32megs in size, you click on a multimedia object and netscape execs a helper app. vfork() you also have to consider a program wishing to make sparse use

Re: Replacement for grep(1) (part 2)

1999-07-14 Thread Matthew Dillon
: Back on topic: : : Obviously you devote the most time to handling the most common : and serious failure modes, but if someone else if willing to : put in the work to handle nightmare cases, should you ignore or : discard that work? Of course not. But nobody

Re: Replacement for grep(1) (part 2)

1999-07-14 Thread Robert Elz
Date:Tue, 13 Jul 1999 14:14:52 -0700 (PDT) From:Matthew Dillon [EMAIL PROTECTED] Message-ID: [EMAIL PROTECTED] | If you don't have the disk necessary for a standard overcommit model to | work, you definitely do not have the disk necessary for a

Re: Replacement for grep(1) (part 2)

1999-07-14 Thread Niall Smart
Maybe if I call the sysctl "vm.crashmenow". No, that will just make more people actually try it. It might be doable as a compile-time option, since you wouldn't be able to run anything approaching standard on such a system anyway. I don't see much use for it myself. As I

Re: Replacement for grep(1) (part 2)

1999-07-14 Thread Daniel C. Sobral
Noriyuki Soda wrote: Running out of swap can be easily done by normal user privilege. Non-overcommiting system can run important application on the system which has a normal user, because it never lose critical data, even if a user on the system make a mistake. (The application might stop,

Re: Replacement for grep(1) (part 2)

1999-07-14 Thread Chris G. Demetriou
Doug Rabson [EMAIL PROTECTED] writes: Overcommit can be used for many reasons. I use it to reserve a large linear address space to mmap alpha i/o spaces [...] Overcommit can be used for many reasons, but unless you've misdescribed what you're doing, _that's not one of them_. The mapped I/O

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-14 Thread Brian F. Feldman
On Thu, 15 Jul 1999, Daniel C. Sobral wrote: "Charles M. Hannum" wrote: That's also objectively false. Most such environments I've had experience with are, in fact, multi-user systems. As you've pointed out yourself, there is no combination of resource limits and whatnot that are

Re: Replacement for grep(1) (part 2)

1999-07-14 Thread Daniel C. Sobral
"Chris G. Demetriou" wrote: ... Overcommit avoidance may not be useful for your particular uses of these UNIX-like systems. However, if you think that it's not useful to anybody who uses them (or that people who think it's useful are deluding themselves 8-), then you're sorely mistaken and

Re: Replacement for grep(1) (part 2)

1999-07-14 Thread Daniel C. Sobral
Matthew Dillon wrote: : :Heh, really? The camera ships w/ Apache running on it. : :-- Jason R. Thorpe [EMAIL PROTECTED] They obviously have a lot of memory to play with, then. Or they are crazy. Writing a web server is fairly easy to do. I've written several,

Re: Replacement for grep(1) (part 2)

1999-07-14 Thread Daniel C. Sobral
Jason Thorpe wrote: There is a lot of hidden 'potential' VM that you haven't considered. For example, if the resource limit for a process's stack is 8MB, then the process can potentially allocate 8MB of stack even though it may actually only allocate 32K of stack.

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-14 Thread Garance A Drosihn
At 12:00 PM -0400 7/14/99, Brian F. Feldman wrote: So why don't we do something else: when we're down to a certain amount of backing store, start collecting statistics. When we're out, we check the statistics and find what process has been allocating most of it. We kill that process. Not

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-14 Thread Daniel C. Sobral
"Brian F. Feldman" wrote: In which case the program that consumed all memory will be killed. The program killed is +NOT+ the one demanding memory, it's the one with most of it. So why don't we do something else: when we're down to a certain amount of backing store, start collecting

Re: Replacement for grep(1) (part 2)

1999-07-14 Thread Julian Elischer
If you wanted to fix this, you could add a patch to malloc that touched every page that it handed to the application. (and trapped sig11s) On Wed, 14 Jul 1999 [EMAIL PROTECTED] wrote: I mean, jeeze, the reservation for the program stack alone would eat up all your available swap

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-14 Thread Brian F. Feldman
You don't seem to understand that a runaway process/one designed just to take up memory will be much more active than your little IMAP servers, and be the one killed, if this scheme were used. Brian Fundakowski Feldman _ __ ___ ___ ___ ___ [EMAIL PROTECTED] _ __

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-14 Thread lyndon
You don't seem to understand that a runaway process/one designed just to take up memory will be much more active than your little IMAP servers, and be the one killed, if this scheme were used. No, what I don't understand is how the current behaviour can tell that my temporary and *valid*

Re: Replacement for grep(1) (part 2)

1999-07-14 Thread Doug Rabson
On 14 Jul 1999, Chris G. Demetriou wrote: Doug Rabson [EMAIL PROTECTED] writes: Overcommit can be used for many reasons. I use it to reserve a large linear address space to mmap alpha i/o spaces [...] Overcommit can be used for many reasons, but unless you've misdescribed what you're

Re: Replacement for grep(1) (part 2)

1999-07-14 Thread David Brownlee
On Thu, 15 Jul 1999, Daniel C. Sobral wrote: For the record, professional digital cameras go into the $100K range, so I'd be expecting it not only to run Apache, but also to come with Doom. :-) Well you have 16MB RAM, 32MB flash memory, a network interface, other bits and

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-14 Thread Michael Richardson
"John" == John Nemeth [EMAIL PROTECTED] writes: John On one system I administrate, the largest process is typically John rpc.nisd (the NIS+ server daemon). Killing that process would be a John bad thing (TM). You're talking about killing random processes. John This is no way

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-14 Thread John Nemeth
On Jul 15, 2:40am, "Daniel C. Sobral" wrote: } Garance A Drosihn wrote: } At 12:20 AM +0900 7/15/99, Daniel C. Sobral wrote: } In which case the program that consumed all memory will be killed. } The program killed is +NOT+ the one demanding memory, it's the one } with most of it. } }

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-14 Thread Michael Richardson
"Ben" == Ben Rosengart [EMAIL PROTECTED] writes: Ben On Wed, 14 Jul 1999, John Nemeth wrote: On one system I administrate, the largest process is typically rpc.nisd (the NIS+ server daemon). Killing that process would be a bad thing (TM). You're talking about killing

Re: Replacement for grep(1) (part 2)

1999-07-14 Thread Robert Elz
Date:Thu, 15 Jul 1999 00:53:17 +0900 From:"Daniel C. Sobral" [EMAIL PROTECTED] Message-ID: [EMAIL PROTECTED] | Would you care to name such systems? munnari was one (the system of the From: header, even though this mail isn't actually going anywhere near it). I

Re: Replacement for grep(1) (part 2)

1999-07-14 Thread Matthew Dillon
:Now let's look at what happens with the two methods. : :With all VM backed by real mem or swap space, processes go about allocating :memory - when there is no more left, the allocations start failing. :If the process is perl, it just collapses in a heap, and the log file :summary doesn't get

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-14 Thread Jason Thorpe
On Tue, 13 Jul 1999 23:18:58 -0400 (EDT) John Baldwin [EMAIL PROTECTED] wrote: What does that have to do with overcommit? I student administrate a undergrad CS lab at a university, and when student's programs misbehaved, they generate a fault and are killed. The only machines that

Re: Replacement for grep(1) (part 2)

1999-07-14 Thread sthaug
Also, your named is badly misconfigured if it grows to 130MB. We never allow ours to grow past 30MB. How do you know what kind of name server configuration kre is running? Here's an example of a name server running *non-recursive*, serving 11.500 zones: PID USERNAME PRI NICE SIZE

Re: Replacement for grep(1) (part 2)

1999-07-14 Thread Matthew Dillon
: : Also, your named is badly misconfigured if it grows to 130MB. We never : allow ours to grow past 30MB. : :How do you know what kind of name server configuration kre is running? :Here's an example of a name server running *non-recursive*, serving :11.500 zones: : : PID USERNAME PRI

Re: Swap overcommit (was Re: Replacement for grep(1) (part 2))

1999-07-14 Thread Matthew Dillon
:On Tue, 13 Jul 1999 23:18:58 -0400 (EDT) : John Baldwin [EMAIL PROTECTED] wrote: : : What does that have to do with overcommit? I student administrate a undergrad : CS lab at a university, and when student's programs misbehaved, they generate a : fault and are killed. The only machines

Re: Replacement for grep(1) (part 2)

1999-07-14 Thread Jason Thorpe
On Wed, 14 Jul 1999 12:43:07 + Niall Smart [EMAIL PROTECTED] wrote: Perhaps it could be an additional flag to mmap, in this way people wishing to run an overcommited system could do so but those writing programs which must not overcommit for certain memory allocations could ensure

Re: Replacement for grep(1) (part 2)

1999-07-14 Thread Jason Thorpe
On Thu, 15 Jul 1999 01:52:11 +0900 "Daniel C. Sobral" [EMAIL PROTECTED] wrote: ...um, so, make the code that deals with faulting in the stack a bit smarter. Uh? Like what? Like overcommitting, for instance? The beauty of overcommitting is that either you do it or you don't. :-) One

Re: Replacement for grep(1) (part 2)

1999-07-14 Thread Jason Thorpe
On Thu, 15 Jul 1999 01:59:12 +0900 "Daniel C. Sobral" [EMAIL PROTECTED] wrote: That's why you make it a switch. No, really, you *can* just make it a switch. So, enlighten me, please... how do you switch it in NetBSD? When the code to do it is implemented (not that hard, really,

Re: Swap subsystem overhead (was Re: Replacement for grep(1) (part 2))

1999-07-14 Thread Nik Clayton
On Tue, Jul 13, 1999 at 05:12:30PM -0700, Matthew Dillon wrote: Ok, I will be more specific. Under FreeBSD-STABLE *AND* FreeBSD-CURRENT, FreeBSD allocates metadata structures that scale to the amount of swap space assigned to the system. However, it is not *precisely* the

Re: Replacement for grep(1) (part 2)

1999-07-14 Thread Matthew Dillon
: :One option is to special-case overcommit the stack. Another is to :set the default stack limits to something more reasonable on a system :where overcommit is disabled. : :-- Jason R. Thorpe [EMAIL PROTECTED] Try setting all the resource limits to something reasonable on general

Re: Replacement for grep(1) (part 2)

1999-07-14 Thread Nate Williams
[ Trimmed CC list a bit ] :* even if you are not willing to pay that price, there _are_ people :who are quite willing to pay that price to get the benefits that they :see (whether it's a matter of perception or not, from their :perspective they may as well be real) of such a scheme.

Re: Replacement for grep(1) (part 2)

1999-07-14 Thread Matthew Dillon
: : Quite true. In the embedded world we preallocate memory and shape : the programs to what is available in the system. But if we run out : of memory we usually panic and reboot - because the code is designed : to NOT run out of memory and thus running out of memory is a

  1   2   3   4   5   >