Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.

[Thies C. Arntzen]

On Mon, Jan 29, 2001 at 06:23:56PM -0600, Brian Moon wrote:
> Ok, so what are the benefits of not returning memory to the system?  It
> seems that no one who can do it wants to so there must be a good reason.Q

in order to be able to "give back" memory to the system we
would have to  switch to an mmap malloc implementation just
for all emalloc'd data - does anyone know such thing that
could be integrated (read: that has an apache-style license)? 

tc

-- 
PHP Development Mailing List 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
To contact the list administrators, e-mail: [EMAIL PROTECTED]

Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.

[Brian Moon]

> 4.
> >And, BTW, you can control it - see MAX_ constants at zend_alloc.h
>
> As I CLEARLY (as I think ...) described in
> http://marc.theaimsgroup.com/?l=php-dev&m=98080647502573&w=2
> the mechanism of memory "deadlocks" is not based on the cache size
> (it is small in fact), but in order of memory operations (problem
> appears even when the size is set to 1 !). We've tested some
> very simple fixes (turn off ZEND_FAST_CACHE and for example
> force the shutdown_memory_manager always to clean the cache, not
> only when "clean" flag is set. It needs some small modification
> in for loop in order to reinitialize cache globals) and the problem
> has dissapeared (almost, but I'll say about it later in 5).
> The CPU performance lowers down only a little bit.

I did what it says at that URL and still had the problems.  I suspect it is
related to libraries and such that we use at phorum.org but I am not sure
how to pinpoint it.  Can you tell me what you did to force the
shutdown_memory_manager.  I would like to see what that can do for me.

Brian Moon
-
Phorum Dev Team - http://phorum.org
Making better forums with PHP
-




-- 
PHP Development Mailing List 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
To contact the list administrators, e-mail: [EMAIL PROTECTED]

Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.

[Filip Sielimowicz]

OK, I'll try to summarize.

> Because it will hurt performance, and gaining performance usually costs
> much more that gaining memory (compare expense for doubling memory and
> doubling performance). Most people in most cases are willing to trade
> memory for performance - that's why caching exists. 

1. I agree, but I agree also with Brian (see
http://marc.theaimsgroup.com/?l=php-dev&m=98081424122248&w=2 ).

2. From our tests we cannot say, that CPU performance is considerably
better when cache is enabled (on RH6.2 it is less than 10%). I've
already written about it. When there are x servers to upgrade with
extra RAM memory - this COSTS (especially in Poland ;).

3. "We are free to turn it off" - Yes, but now there are no 
simply means to do that. So "we demand" some option in 
.configure script, of course cache should be on by default.
Now we can use solution described in
http://marc.theaimsgroup.com/?l=php-dev&m=98053283403331&w=2
It has been tested much, but no warranty !

4. 
>And, BTW, you can control it - see MAX_ constants at zend_alloc.h

As I CLEARLY (as I think ...) described in
http://marc.theaimsgroup.com/?l=php-dev&m=98080647502573&w=2
the mechanism of memory "deadlocks" is not based on the cache size
(it is small in fact), but in order of memory operations (problem
appears even when the size is set to 1 !). We've tested some
very simple fixes (turn off ZEND_FAST_CACHE and for example
force the shutdown_memory_manager always to clean the cache, not
only when "clean" flag is set. It needs some small modification
in for loop in order to reinitialize cache globals) and the problem
has dissapeared (almost, but I'll say about it later in 5).
The CPU performance lowers down only a little bit.

5. We've found out that not only the php memory caches are responsible
for described effect. When we use oci8 module - the effect comes back
(but in smaller size), because there's independent memory management
in that module (and in oracle libraries). We are working on it.

I hope this discussion shall result in most effective and
flexible solutions.

Filip Sielimowicz
[EMAIL PROTECTED]


-- 
PHP Development Mailing List 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
To contact the list administrators, e-mail: [EMAIL PROTECTED]

Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.

[Stanislav Malyshev]

BM>> Yes, but it would be really nice if it was returned.  Systems
BM>> with 128MB or RAM run out quickly if there is a script ( like a
BM>> long Phorum thread ) that takes a good chunk.  Of course the
BM>> argument can be made that the box needs more RAM but if the
BM>> memory can be returned (maybe even with a php.ini setting that
BM>> states the max amount PHP should keep) to the system why not do
BM>> it?

Because it will hurt performance, and gaining performance usually costs
much more that gaining memory (compare expense for doubling memory and
doubling performance). Most people in most cases are willing to trade
memory for performance - that's why caching exists. And, BTW, you can
control it - see MAX_ constants at zend_alloc.h

-- 
Stanislav Malyshev, Zend Products Engineer
[EMAIL PROTECTED]  http://www.zend.com/ +972-3-6139665 ext.115



-- 
PHP Development Mailing List 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
To contact the list administrators, e-mail: [EMAIL PROTECTED]

Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.

[Stanislav Malyshev]

BM>> It is not a leak exactly but more of a greed.  It seems to be a
BM>> semi-intentional but not necessary thing.  It seems that the way
BM>> Zend/PHP is allocating memory is not allowing it to be returned
BM>> to the system.  Once PHP has some memory it will not let it go,
BM>> but it will reuse it.  So, if a script takes up a good chunk of
BM>> memory, it is never returned to the system unless that httpd
BM>> process is killed or dies.

That effect was discussed to death already. Libc implementation of malloc
does not return memory to the system unless some very special conditions
are met (freed block is very large or there's enough consecutive free
space at the top of the memory). Unless PHP switches to some non-standard
malloc algoritm, it would be this way.

As for PHP not freeing some meory block - it was done for purpose, and the
purpose is - performance (I hope you don't think Zend authors have gone
through all that trouble of having their own allocs just because they
didn't think of using standard mallocs, right?). If memory size is more
important for you than performance - you are free to turn this solution
off.

-- 
Stanislav Malyshev, Zend Products Engineer
[EMAIL PROTECTED]  http://www.zend.com/ +972-3-6139665 ext.115



-- 
PHP Development Mailing List 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
To contact the list administrators, e-mail: [EMAIL PROTECTED]

Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.

[Boian Bonev]

hi,

i'd disagree on this - imagine you have a webserver with reasonable load and
99.9% of php page requests use max 10mb ram. you have 1% that use 100mb ram.
then it not a good idea to upgrade to 1g ram just for these 0.1% since each
httpd process will hold 100mb ram after certain time (the 0.1% will spread
on all httpd processes)

on the other side getting and returning ram to the system is not a fast
thing and should be avoided. most common solution is to tell the memory
cache how much ram to held. or preset it at compile time.

i see that my most loaded system (most complex scripts) uses about 10m per
process.

b.

> Seems to me like it would be a lot easier just to add more RAM.  $89 for
> 256M these days.  RAM is cheap.
>
> > That is what we are doing now.  We have it at 5.  Any higher and we are
> > running out.  There is only 128MB of RAM in our machine.  I am
considering
> > switching to CGI PHP just to eliviate this problem.  Of course, that
sucks.
> >
> > > > Yes, yes, I agree ! But this is the problem ! Unfortunatelly "memory
> > > > hungry script" happens from time to time and there's no chance to
prevent
> > > > httpd processes from keeping memory (which is not used again in most
> > > > cases). Maybe it is not a problem for you, but it is for us.
> > >
> > > A quick fix would be to set your MaxRequestsPerChild to a lower value.
> > >



-- 
PHP Development Mailing List 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
To contact the list administrators, e-mail: [EMAIL PROTECTED]

Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.

[Jason Greene]

What is your typical apache process size, and number of current running procs?

Jason

- Original Message - 
From: "Brian Moon" <[EMAIL PROTECTED]>
To: "Rasmus Lerdorf" <[EMAIL PROTECTED]>
Cc: "Filip Sielimowicz" <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
Sent: Monday, January 29, 2001 6:23 PM
Subject: Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.


> Ok, so what are the benefits of not returning memory to the system?  It
> seems that no one who can do it wants to so there must be a good reason.
> 
> Brian Moon
> --
> Phorum Dev Team - http://phorum.org
> Making better forums with PHP
> --
> 
> 
> - Original Message -
> From: "Rasmus Lerdorf" <[EMAIL PROTECTED]>
> To: "Brian Moon" <[EMAIL PROTECTED]>
> Cc: "Filip Sielimowicz" <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
> Sent: Monday, January 29, 2001 5:03 PM
> Subject: Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.
> 
> 
> > Seems to me like it would be a lot easier just to add more RAM.  $89 for
> > 256M these days.  RAM is cheap.
> >
> > -Rasmus
> >
> > On Mon, 29 Jan 2001, Brian Moon wrote:
> >
> > > That is what we are doing now.  We have it at 5.  Any higher and we are
> > > running out.  There is only 128MB of RAM in our machine.  I am
> considering
> > > switching to CGI PHP just to eliviate this problem.  Of course, that
> sucks.
> > >
> > > Brian Moon
> > > --
> > > Phorum Dev Team - http://phorum.org
> > > Making better forums with PHP
> > > ----------------------
> > >
> > >
> > > - Original Message -
> > > From: "Rasmus Lerdorf" <[EMAIL PROTECTED]>
> > > To: "Filip Sielimowicz" <[EMAIL PROTECTED]>
> > > Cc: "Brian Moon" <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
> > > Sent: Monday, January 29, 2001 6:56 AM
> > > Subject: Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.
> > >
> > >
> > > > > Yes, yes, I agree ! But this is the problem ! Unfortunatelly "memory
> > > > > hungry script" happens from time to time and there's no chance to
> > > prevent
> > > > > httpd processes from keeping memory (which is not used again in most
> > > > > cases). Maybe it is not a problem for you, but it is for us.
> > > >
> > > > A quick fix would be to set your MaxRequestsPerChild to a lower value.
> > > >
> > > > -Rasmus
> > > >
> > > >
> > >
> >
> >
> 
> 
> -- 
> PHP Development Mailing List <http://www.php.net/>
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
> To contact the list administrators, e-mail: [EMAIL PROTECTED]
> 


-- 
PHP Development Mailing List <http://www.php.net/>
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
To contact the list administrators, e-mail: [EMAIL PROTECTED]

Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.

[Brian Moon]

Ok, so what are the benefits of not returning memory to the system?  It
seems that no one who can do it wants to so there must be a good reason.

Brian Moon
--
Phorum Dev Team - http://phorum.org
Making better forums with PHP
--


- Original Message -
From: "Rasmus Lerdorf" <[EMAIL PROTECTED]>
To: "Brian Moon" <[EMAIL PROTECTED]>
Cc: "Filip Sielimowicz" <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
Sent: Monday, January 29, 2001 5:03 PM
Subject: Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.


> Seems to me like it would be a lot easier just to add more RAM.  $89 for
> 256M these days.  RAM is cheap.
>
> -Rasmus
>
> On Mon, 29 Jan 2001, Brian Moon wrote:
>
> > That is what we are doing now.  We have it at 5.  Any higher and we are
> > running out.  There is only 128MB of RAM in our machine.  I am
considering
> > switching to CGI PHP just to eliviate this problem.  Of course, that
sucks.
> >
> > Brian Moon
> > --
> > Phorum Dev Team - http://phorum.org
> > Making better forums with PHP
> > --
> >
> >
> > - Original Message -
> > From: "Rasmus Lerdorf" <[EMAIL PROTECTED]>
> > To: "Filip Sielimowicz" <[EMAIL PROTECTED]>
> > Cc: "Brian Moon" <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
> > Sent: Monday, January 29, 2001 6:56 AM
> > Subject: Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.
> >
> >
> > > > Yes, yes, I agree ! But this is the problem ! Unfortunatelly "memory
> > > > hungry script" happens from time to time and there's no chance to
> > prevent
> > > > httpd processes from keeping memory (which is not used again in most
> > > > cases). Maybe it is not a problem for you, but it is for us.
> > >
> > > A quick fix would be to set your MaxRequestsPerChild to a lower value.
> > >
> > > -Rasmus
> > >
> > >
> >
>
>


-- 
PHP Development Mailing List <http://www.php.net/>
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
To contact the list administrators, e-mail: [EMAIL PROTECTED]

Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.

[Rasmus Lerdorf]

> It is not a leak exactly but more of a greed.  It seems to be a
> semi-intentional but not necessary thing.  It seems that the way Zend/PHP is
> allocating memory is not allowing it to be returned to the system.  Once PHP
> has some memory it will not let it go, but it will reuse it.  So, if a
> script takes up a good chunk of memory, it is never returned to the system
> unless that httpd process is killed or dies.

Right, this is standard.  But this does not explain incremental memory
usage.  Memory is marked as free and re-used.  It does not need to be
returned to the system for it to be re-used.  So the memory usage for any
one process should never exceed the memory usage of your most memory
hungry script.

-Rasmus


-- 
PHP Development Mailing List 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
To contact the list administrators, e-mail: [EMAIL PROTECTED]

Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.

[Rasmus Lerdorf]

On Mon, 29 Jan 2001, Filip Sielimowicz wrote:

> > A quick fix would be to set your MaxRequestsPerChild to a lower value.
>
> Have you ever tried to measure the time of killing and
> starting httpd process ? You can see that with a bare human eye.

Sure.  But that isn't something that happens on an actual request, so an
end user isn't going to see that.

-Rasmus


-- 
PHP Development Mailing List 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
To contact the list administrators, e-mail: [EMAIL PROTECTED]

Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.

[Rasmus Lerdorf]

> Yes, yes, I agree ! But this is the problem ! Unfortunatelly "memory
> hungry script" happens from time to time and there's no chance to prevent
> httpd processes from keeping memory (which is not used again in most
> cases). Maybe it is not a problem for you, but it is for us.

A quick fix would be to set your MaxRequestsPerChild to a lower value.

-Rasmus


-- 
PHP Development Mailing List 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
To contact the list administrators, e-mail: [EMAIL PROTECTED]

Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.

[Rasmus Lerdorf]

Seems to me like it would be a lot easier just to add more RAM.  $89 for
256M these days.  RAM is cheap.

-Rasmus

On Mon, 29 Jan 2001, Brian Moon wrote:

> That is what we are doing now.  We have it at 5.  Any higher and we are
> running out.  There is only 128MB of RAM in our machine.  I am considering
> switching to CGI PHP just to eliviate this problem.  Of course, that sucks.
>
> Brian Moon
> --
> Phorum Dev Team - http://phorum.org
> Making better forums with PHP
> --
>
>
> - Original Message -
> From: "Rasmus Lerdorf" <[EMAIL PROTECTED]>
> To: "Filip Sielimowicz" <[EMAIL PROTECTED]>
> Cc: "Brian Moon" <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
> Sent: Monday, January 29, 2001 6:56 AM
> Subject: Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.
>
>
> > > Yes, yes, I agree ! But this is the problem ! Unfortunatelly "memory
> > > hungry script" happens from time to time and there's no chance to
> prevent
> > > httpd processes from keeping memory (which is not used again in most
> > > cases). Maybe it is not a problem for you, but it is for us.
> >
> > A quick fix would be to set your MaxRequestsPerChild to a lower value.
> >
> > -Rasmus
> >
> >
>


-- 
PHP Development Mailing List <http://www.php.net/>
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
To contact the list administrators, e-mail: [EMAIL PROTECTED]

Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.

[Brian Moon]

Yes, but it would be really nice if it was returned.  Systems with 128MB or
RAM run out quickly if there is a script ( like a long Phorum thread ) that
takes a good chunk.  Of course the argument can be made that the box needs
more RAM but if the memory can be returned (maybe even with a php.ini
setting that states the max amount PHP should keep) to the system why not do
it?

Brian Moon
-
Phorum Dev Team - http://phorum.org
Making better forums with PHP
-


- Original Message -
From: "Rasmus Lerdorf" <[EMAIL PROTECTED]>
To: "Brian Moon" <[EMAIL PROTECTED]>
Cc: "Filip Sielimowicz" <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
Sent: Sunday, January 28, 2001 12:23 AM
Subject: Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.


> > It is not a leak exactly but more of a greed.  It seems to be a
> > semi-intentional but not necessary thing.  It seems that the way
Zend/PHP is
> > allocating memory is not allowing it to be returned to the system.  Once
PHP
> > has some memory it will not let it go, but it will reuse it.  So, if a
> > script takes up a good chunk of memory, it is never returned to the
system
> > unless that httpd process is killed or dies.
>
> Right, this is standard.  But this does not explain incremental memory
> usage.  Memory is marked as free and re-used.  It does not need to be
> returned to the system for it to be re-used.  So the memory usage for any
> one process should never exceed the memory usage of your most memory
> hungry script.
>
> -Rasmus
>
>


-- 
PHP Development Mailing List <http://www.php.net/>
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
To contact the list administrators, e-mail: [EMAIL PROTECTED]

Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.

[Brian Moon]

Hi Rasmus,

It is not a leak exactly but more of a greed.  It seems to be a
semi-intentional but not necessary thing.  It seems that the way Zend/PHP is
allocating memory is not allowing it to be returned to the system.  Once PHP
has some memory it will not let it go, but it will reuse it.  So, if a
script takes up a good chunk of memory, it is never returned to the system
unless that httpd process is killed or dies.

There is a thread that describes what happens and a discussion on how to
change Zend/PHP to not do this.  Here is the link:

http://marc.theaimsgroup.com/?l=php-dev&m=97923602322593&w=2

Now, being the 133t C hacker that I am NOT, I do not dare dive into this.
However, I did use the recommendations given in this thread and it appeared
to have a definite effect on the performance of my server.  It did however,
as the author warned, make PHP unstable.

I fear that a deep look into the way that memory is being used will have to
take place to really find this problem.  I am willing to use phorum.org as a
guinea pig for test scripts that cause this or whatever I can do to help fix
this problem.

Believe me, I think you know how much I love PHP, I did everything I could
to not blame PHP for this.  I have updated Red Hat (glibc and gcc), Apache
(btw, static pages don't cause the growth), and anything else I could think
of to solve it.

Brian Moon
-
Phorum Dev Team - http://phorum.org
Making better forums with PHP
-


- Original Message -
From: "Rasmus Lerdorf" <[EMAIL PROTECTED]>
To: "Filip Sielimowicz" <[EMAIL PROTECTED]>
Cc: <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
Sent: Saturday, January 27, 2001 11:53 PM
Subject: Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.


> > PS. Hey, guys from the php-dev. Don't you have anything
> > to say ? Please do that !
>
> Please do what?
>
> You should try to figure out what is leaking memory on you.  None of my
> servers are leaking.
>
> -Rasmus
>
>


-- 
PHP Development Mailing List <http://www.php.net/>
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
To contact the list administrators, e-mail: [EMAIL PROTECTED]

Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.

[Brian Moon]

That is what we are doing now.  We have it at 5.  Any higher and we are
running out.  There is only 128MB of RAM in our machine.  I am considering
switching to CGI PHP just to eliviate this problem.  Of course, that sucks.

Brian Moon
--
Phorum Dev Team - http://phorum.org
Making better forums with PHP
--


- Original Message -
From: "Rasmus Lerdorf" <[EMAIL PROTECTED]>
To: "Filip Sielimowicz" <[EMAIL PROTECTED]>
Cc: "Brian Moon" <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
Sent: Monday, January 29, 2001 6:56 AM
Subject: Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.


> > Yes, yes, I agree ! But this is the problem ! Unfortunatelly "memory
> > hungry script" happens from time to time and there's no chance to
prevent
> > httpd processes from keeping memory (which is not used again in most
> > cases). Maybe it is not a problem for you, but it is for us.
>
> A quick fix would be to set your MaxRequestsPerChild to a lower value.
>
> -Rasmus
>
>


-- 
PHP Development Mailing List <http://www.php.net/>
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
To contact the list administrators, e-mail: [EMAIL PROTECTED]

Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.

[Filip Sielimowicz]

On Sat, 27 Jan 2001, Rasmus Lerdorf wrote:

> > It is not a leak exactly but more of a greed.  It seems to be a
> > semi-intentional but not necessary thing.  It seems that the way Zend/PHP is
> > allocating memory is not allowing it to be returned to the system.  Once PHP
> > has some memory it will not let it go, but it will reuse it.  So, if a
> > script takes up a good chunk of memory, it is never returned to the system
> > unless that httpd process is killed or dies.
> 
> Right, this is standard.  But this does not explain incremental memory

I have not said anything about _incremental_ memory usage. I try to
put your attention to that _standard_ problem. 

> usage.  Memory is marked as free and re-used.  It does not need to be
> returned to the system for it to be re-used.  So the memory usage for any
> one process should never exceed the memory usage of your most memory
> hungry script.

Yes, yes, I agree ! But this is the problem ! Unfortunatelly "memory
hungry script" happens from time to time and there's no chance to prevent
httpd processes from keeping memory (which is not used again in most
cases). Maybe it is not a problem for you, but it is for us. 

And now I can say quite exactly, why this happens and how to correct this.
For exemple, see the algorithms implemented in emalloc and efree.
After many uses of emallocs (standard situation when executing script),
the cache (i simplify a little, but we can say that it works like a LIFO
queue of "freed" memory blocks) is empty. And than there comes a time for
freeing memory, which usually goes like this: last allocated block is
beeing freed first. So it goes to the cache.

Advantage: during normal execution, where emallocs and efrees are
going next to each other, this mechanism guarantees high performance
- there are only a few blocks allocated and they are reused very
frequently.

Disadvantage: we can expect such situation (cache size=1)
M1
 M2
  M3
   M4
M5
F5 -> this goes to cache (which is empty)
   F4 -> this is freed normally
  F3 -> normally
 F2 -> ...
F1 -> ...

At the end of execution there stays M5 block in the cache and it
prevents the memory of ALL blocks from beeing returned to the system.

The last allocated block goes ALWAYS to the cache. This is 
horrible... 

I know, the model is much simplified, the shown order of freeing
blocks may be different. But ...

I think, some simple solutions should be clear now.

I haven't been analising the ZEND FAST CACHE yet, but I think
the problem is similar.


-- 
PHP Development Mailing List 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
To contact the list administrators, e-mail: [EMAIL PROTECTED]

Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.

[Filip Sielimowicz]

> There is a thread that describes what happens and a discussion on how to
> change Zend/PHP to not do this.  Here is the link:
> 
> http://marc.theaimsgroup.com/?l=php-dev&m=97923602322593&w=2
> 
> Now, being the 133t C hacker that I am NOT, I do not dare dive into this.
> However, I did use the recommendations given in this thread and it appeared
> to have a definite effect on the performance of my server.  It did however,
> as the author warned, make PHP unstable.

That solution is not good for sure. In original PHP
all allocated memory blocks are stored in a list
and there's a control on it. After my changes the
list is not supported and it may cause many memory
management problems (including the described, but in
some different circumstances).

Thanks for your response !

Filip Sielimowicz
[EMAIL PROTECTED]


-- 
PHP Development Mailing List 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
To contact the list administrators, e-mail: [EMAIL PROTECTED]

Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.

[Filip Sielimowicz]

> A quick fix would be to set your MaxRequestsPerChild to a lower value.

Have you ever tried to measure the time of killing and
starting httpd process ? You can see that with a bare human eye.



-- 
PHP Development Mailing List 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
To contact the list administrators, e-mail: [EMAIL PROTECTED]

Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.

[Rasmus Lerdorf]

> PS. Hey, guys from the php-dev. Don't you have anything
> to say ? Please do that !

Please do what?

You should try to figure out what is leaking memory on you.  None of my
servers are leaking.

-Rasmus


-- 
PHP Development Mailing List 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
To contact the list administrators, e-mail: [EMAIL PROTECTED]

Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.

[Andrzej Kiesz]

Dnia Wed, Jan 24, 2001 at 06:21:13PM -, [EMAIL PROTECTED] napisal:
> From: [EMAIL PROTECTED]
> Operating system: RH 7
> PHP version:  4.0.4
> PHP Bug Type: Performance problem
> Bug description:  Memory is not being freed.
> 
> Hi guys,
> 
> I do not have much information.  I know that my Apache processes memory is growing 
>by the minute.  If I start a separate server on another port and serve only static 
>pages and files through it, those process do not grow.

Yes, that's truth. I don't want to write the same stuff as my friend did.
Very very simple test can show that php doesn't free memory that it allocated before.



> 
> I have tried what I remembered of gdb but have not come up with anything.  I know I 
>need to attach to a process and it seems I can `gdb {pid}` but that gives me nothing. 
> How can I get some info for you guys?  I do have --enable-debug.
> 
> thanks,
> 
> Brian.
> Phorum.org
> 
> 
> -- 
> Edit Bug report at: http://bugs.php.net/?id=8889&edit=1
> 
> 
> 
> -- 
> PHP Development Mailing List 
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
> To contact the list administrators, e-mail: [EMAIL PROTECTED]

-- 

Andrzej Kiesz, [EMAIL PROTECTED], http://sport.wp.pl 


-- 
PHP Development Mailing List 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
To contact the list administrators, e-mail: [EMAIL PROTECTED]

Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.

[Filip Sielimowicz]

On 24 Jan 2001 [EMAIL PROTECTED] wrote:

> From: [EMAIL PROTECTED]
> Operating system: RH 7
> PHP version:  4.0.4
> PHP Bug Type: Performance problem
> Bug description:  Memory is not being freed.
> 
> Hi guys,
> 
> I do not have much information.  I know that my Apache processes memory is growing 
>by the minute.  If I start a separate server on another port and serve only static 
>pages and files through it, those process do not grow.
> 
> I have tried what I remembered of gdb but have not come up with anything.  I know I 
>need to attach to a process and it seems I can `gdb {pid}` but that gives me nothing. 
> How can I get some info for you guys?  I do have --enable-debug.
> 
> thanks,
> 
> Brian.
> Phorum.org
> 
Ha ! I (my company) have the same problem, there was a quite long
discussion about it (look for mails with phrase "holds memory" in title),
but now you can hear nothing about that topic here... 
I've made some mistakes there, analising the problem, but after
about a week (two ?) I know much more. We are still working on it.

Now I can serve you some simple solution.
There are two memory caches, which are responsible for the observed memory
effect (they do not vary the order of memory operations). They are
implemented in the Zend module.
If you have some experience in recompiling the php source, I can 
help you.
See the file Zend/zend_alloc.c, near line number 39 (I can not say
exactly)

#define ZEND_DISABLE_MEMORY_CACHE 0

You should set this value to 1.

#define ZEND_DISABLE_MEMORY_CACHE 1

And secondary you have to turn off so called fast cache.
To do this, you can add at line 24

#define ZEND_ENABLE_FAST_CACHE 0

to the file Zend/zend_fast_cache.h, so it looks like this now:

...
#ifndef ZEND_FAST_CACHE_H
#define ZEND_FAST_CACHE_H

#define ZEND_ENABLE_FAST_CACHE 0

#ifndef ZEND_ENABLE_FAST_CACHE
# if ZEND_DEBUG
# define ZEND_ENABLE_FAST_CACHE 0
# else
# define ZEND_ENABLE_FAST_CACHE 1
# endif
#endif 
...


Then recompile (all php source, use make clean before for safety)
and install.
This solution turns off memory caches, but from our
tests (on standard RH 6.2 with standard kernel)
we can say, that CPU performance goes down
only less then 10% (it depends on running scripts,
but you can expect MUCH less then 10%, so you may not care).

We have worked out also two more complex solutions, where
there's no need to turn off the first cache.
But this will be described when we finish tests.

Good luck !

PS. Hey, guys from the php-dev. Don't you have anything
to say ? Please do that !


-- 
PHP Development Mailing List 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
To contact the list administrators, e-mail: [EMAIL PROTECTED]

Re: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.

[Jon Tai]

I have been noticing the same problem on my server.  Apache started
consuming large amounts of memory after I upgraded to php-4.0.4pl1.  Running
ps -aux would tell me that apache was consuming 6.9 - 11.0 in the "mem"
column; restarting apache would make the numbers go back down to ~3.0.

I think the problem may have something to do with ob_gzhandler.  I tried
disabling ob_gzhandler last night and the server has been fine all day.  Is
there something I can do to get more information about what's going on?

I'm running php-4.0.4pl1 as an apache-1.3.14 module on RedHat 6.2.

- Jon

- Original Message -
From: <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Wednesday, January 24, 2001 10:21 AM
Subject: [PHP-DEV] PHP 4.0 Bug #8889: Memory is not being freed.


From: [EMAIL PROTECTED]
Operating system: RH 7
PHP version:  4.0.4
PHP Bug Type: Performance problem
Bug description:  Memory is not being freed.

Hi guys,

I do not have much information.  I know that my Apache processes memory is
growing by the minute.  If I start a separate server on another port and
serve only static pages and files through it, those process do not grow.

I have tried what I remembered of gdb but have not come up with anything.  I
know I need to attach to a process and it seems I can `gdb {pid}` but that
gives me nothing.  How can I get some info for you guys?  I do
have --enable-debug.

thanks,

Brian.
Phorum.org


--
Edit Bug report at: http://bugs.php.net/?id=8889&edit=1



--
PHP Development Mailing List 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
To contact the list administrators, e-mail: [EMAIL PROTECTED]




-- 
PHP Development Mailing List 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
To contact the list administrators, e-mail: [EMAIL PROTECTED]