Alvin Starr wrote:
> On Tue, 27 Apr 1999, Benno Senoner wrote:
>
> > Hi,
> >
> > My system is a Redhat 5.2 running on
> > linux 2.2.6 + raid0145-19990421.
> >
> > I tested if the system is stable while swapping heavily.
> > I tested a regular swap area and a soft-RAID1 (2 disks) swaparea.
> >
> >
> > So I wrote e little program wich does basically the following:
> >
> > allocate as much as possible blocks of about 4MB,
> >
> > then begin to write data in a linear way to each block,
> > and displaying the write-performance in MB/sec.
> > The program writes a dot on the screen after the write
> > of each 1024bytes.
> >
> > NOW TO THE VERY STRANGE RESULTS :
> >
> > 1)
> > my frist BIG QUESTION is if there is a design flaw in malloc() or not:
> >
> > when I do (number of successfully allocated blocks)* 4MB
> >
> > then I get 2GB of *SUCCESSFULLY* malloc()ed memory,
> > but my system has only about 100MB of virtual mem
> > (64MB RAM + 40MB swap).
>
> I took a look at your program. It looks as if you are not using the memory
> that you malloc. If I remember correctly Linux will not allocate the
> memory that you have requested until you use it. This is a really handy
> feature when you have sparce arrays.
I now this, fact,
without the fragment below, the program would do nothing,
just run the program and see that is writes to mem,
look at your HD LED
:-)
> What you need to do is to have your
> program touch every page in the virtual memory allocated. If the page size
> is 4096 bytes you will have to write every 4096'th byte to insure that the
> page is brought into existance.
this is the fragment that writes to the memory:
in particular buffer[u]=u does the actual write,
buffer=block_arr[i];
for(u=0;u<MYBUFSIZE;u++)
{
buffer[u]=u;
if((u & 1023)==0) { printf("."); fflush(stdout); }
}
>
>
> >
> > Does anyone know why the kernels does not limit the maximum malloc()ed
> > memory
> > to the amount of RAM+SWAP ?
>
> This is a design choice. If you do this you will limit programms that use
> virtual memory in a way that is sparsly populated.
>
> > Will this be changed in the future ?
>
> Other systems pre-commit memory and this can at times cause your system to
> stop allowing new processes to run up even though you are not using
> anywhere near the total of ram+swap avilable. The choice of using one
> allocation method or another may be a possible place for a kernel tuneing
> feature. But for me I like the current choice.
>
> >
> > 2)
> >
> > At beginning the program runs fine and when the RAM is used up,
> > swapping activity begins, and the mem-write performance drops
> > to about the write performance of the disk or RAID1 array.
> >
> > Now the problem:
> > When all RAM + SWAP are used up, then the system begin to freeze,
> > and every 10-20secs , there appear messages on the console
> > like:
> >
> > out of memory of syslog
> > out of memory of klogd
> > .
> > .
> >
> > and after a while my swapstress program exits with a Bus Error.
> >
> > Sometimes even "update" get killed, or "init"
> > which writes: PANIC SEGMENT VIOLATION !
> >
> > after the exit of with "Bus Error" of my swapstress program the system
> > continues to work, but since init got killed , you can not reboot or
> > shutdown the machine anymore.
> >
> > Note that I ran swapstress as normal user.
> > This means it's easy to crash/render unstable Linux with heavy
> > malloc()ing
> > / swapping.
>
> resource exhaustion can cause a number of problems one of which is system
> crashes. One solution is to limit the per user memory maximum so that a
> single user cannot burn up all the system memory but that still will not
> stop the problem.
>
> One possible answer is for the kernel to allways spare some swap space for
> tasks running as root and to suspend any user tasks that request memory
> when the swap limit is reached. the creation of new user processes should
> also be suspended when this limit is reached. at this point an
> administrator would be able to login to the system and kill the offending
> processes or take some other remedial action.
I agree, root processes should have some spare ressurces.
>
>
> Alvin Starr || voice: (416)585-9971
> Interlink Connectivity || fax: (416)585-9974
> [EMAIL PROTECTED] ||
