[PHP-DEV] RE: [PATCH]apache_register_shutdown_function final version

2003-02-13 Thread Zeev Suraski
At 01:37 07/01/2003, Joseph Tate wrote:

Well, according to my highly technical methods of deduction (fprintf(stderr,
Inside sapi_shutdown);)  sapi_shutdown is never called when serving a PHP
request when served using mod4_php under Apache.  This is because the
shutdown_wrapper never gets initialized as a cleanup function.  This because
the php_init_handler is never called, and this happens for some reason
undivinable by me.  Perhaps it should be used, and we should execute regular
shutdown functions there, and proceed to use php_request_shutdown in the way
it was used prior to php 4.1.x i.e. after the connection has closed (on
apache anyway) and while PHP is still in memory.  We'll get the added
benefit of (on Apache systems anyway) a significant performance gain as the
cleanup phase happens without the client having to wait for it.


You must have messed up on your analysis, as php_init_handler() is 
certainly being called, and so is sapi_shutdown().  Place an abort() in 
them, it won't take more than a couple of minutes to know that for certain :)
I would guess that the source of the problem is that sapi_shutdown() is 
simply not the function you're looking for.  You're probably looking for 
sapi_deactivate().

Here is my vision based on my understanding of the inner workings of PHP.

There are at least four types of shutdown functions in PHP (not including
extensions):
PHP space (identified by the use of register_shutdown_function)
module_shutdown (called when a module is to clean up after itself)
sapi_shutdown (not used under many sapi implementations)
php_shutdown
there may also be zend and tsrm shutdown functions.


Not quite.

First off, if we're talking about register_shutdown_function(), then we're 
dealing with activate/deactivate functions, and not startup/shutdown 
functions.  In the old terms, still used by modules, we're dealing with 
RINIT/RSHTDOWN (request-init, request-shutdown).
Activate (rinit) / deactivate (rshutdown) functions get called in the 
beginning and end of each request, respectively.  In that regard, there are 
*plenty* of them.  Activate/deactivate functions are called for the engine, 
SAPI, output buffering, whatnot.  The engine, in turn, calls the 
activate/deactivate callbacks that are available in modules, if 
available.  One of the deactivate actions in php_request_shutdown is to 
call the userland-registered shutdown functions.

SAPI is just one of the many subsystems which are initialized/destroyed by 
php_requset_startup() and php_request_shutdown().  There's nothing special 
about it.

php_shutdown seems to always be a call to shutdown wheras the remainder are
called when shutdown occurs.


Didn't quite understand that..?


  Thus the cleanup/shutdown routines should be
called in this order:

main
init
load and run PHP script
call sapi_shutdown
which calls module_shutdown on all used modules
and then executes register_shutdown_functions
and then calls php_request_shutdown in a server specific 
manner (in apache
as a cleanup_function after the http communication has terminated)
which (under platforms that support it) executes
register_offline_functions
before cleaning up PHP and freeing memory.

There are many problematic things about this layout.  Suffice to say that 
intimately knowing the shutdown order in PHP, I don't think there are any 
conceptual problems with it.  Again, there's nothing inherent in SAPI that 
makes it the function of choice to be the root of all deactivation calls, 
as it is just one of the many subsystems in PHP.  PHP's main deactivation 
function is php_request_shutdown(), and it calls sapi_deactivate(), because 
it's one of the subsystems that requires deactivation.  It's the 
responsibility of the SAPI module to figure out the best way to ensure that 
php_request_shutdown() gets called at the end of each request.

SAPI was developed for abstracting the server, but it seems that instead it
has been worked around rather than extended for the individual servers.  I
speak in generalizations, but that's what it looks like.  In my book, all
php internal functions should be called through the appropriate SAPI
functions, which are called by a main function for each sapi implementation.
There is too much deviation from this, thereby crippling SAPI.  Either use
it or cut it out completly.

These changes are too large for a minor version release, i.e. 4.3.1, but
perhaps for 4.4/5.0?  I'd be willing to put some time into reworking the
SAPI system.


I think that these harsh thoughts are mostly due to the problems in the 
initial analysis.  SAPI is really working very very well as it is :)

Zeev


--
PHP Development Mailing List http://www.php.net/
To unsubscribe, visit: http://www.php.net/unsub.php



[PHP-DEV] RE: [PATCH]apache_register_shutdown_function final version

2003-01-06 Thread Joseph Tate
Well, according to my highly technical methods of deduction (fprintf(stderr,
Inside sapi_shutdown);)  sapi_shutdown is never called when serving a PHP
request when served using mod4_php under Apache.  This is because the
shutdown_wrapper never gets initialized as a cleanup function.  This because
the php_init_handler is never called, and this happens for some reason
undivinable by me.  Perhaps it should be used, and we should execute regular
shutdown functions there, and proceed to use php_request_shutdown in the way
it was used prior to php 4.1.x i.e. after the connection has closed (on
apache anyway) and while PHP is still in memory.  We'll get the added
benefit of (on Apache systems anyway) a significant performance gain as the
cleanup phase happens without the client having to wait for it.

Here is my vision based on my understanding of the inner workings of PHP.

There are at least four types of shutdown functions in PHP (not including
extensions):
PHP space (identified by the use of register_shutdown_function)
module_shutdown (called when a module is to clean up after itself)
sapi_shutdown (not used under many sapi implementations)
php_shutdown
there may also be zend and tsrm shutdown functions.

php_shutdown seems to always be a call to shutdown wheras the remainder are
called when shutdown occurs.  Thus the cleanup/shutdown routines should be
called in this order:

main
init
load and run PHP script
call sapi_shutdown
which calls module_shutdown on all used modules
and then executes register_shutdown_functions
and then calls php_request_shutdown in a server specific manner (in 
apache
as a cleanup_function after the http communication has terminated)
which (under platforms that support it) executes
register_offline_functions
before cleaning up PHP and freeing memory.

SAPI was developed for abstracting the server, but it seems that instead it
has been worked around rather than extended for the individual servers.  I
speak in generalizations, but that's what it looks like.  In my book, all
php internal functions should be called through the appropriate SAPI
functions, which are called by a main function for each sapi implementation.
There is too much deviation from this, thereby crippling SAPI.  Either use
it or cut it out completly.

These changes are too large for a minor version release, i.e. 4.3.1, but
perhaps for 4.4/5.0?  I'd be willing to put some time into reworking the
SAPI system.


 -Original Message-
 From: Zeev Suraski [mailto:[EMAIL PROTECTED]]
 Sent: Tuesday, December 31, 2002 4:21 AM
 To: Joseph Tate
 Cc: Php-Dev List; Jason Priebe
 Subject: RE: [PATCH]apache_register_shutdown_function final version


 As you can see in bug #15209, the change that introduced this problem was
 that we started calling php_request_shutdown() in apache_module_main(),
 prior to calling it from php_apache_request_shutdown().  So, the place to
 put it is clearly php_apache_request_shutdown(), where it used to
 be called
 before.

 The problem is by the time you would get to
 php_apache_request_shutdown(),
 the per-request memory manager is already deactivated, so any emalloc()'d
 memory you may have is already freed.  I'm not sure how to tackle
 this in a
 server independent way...

 Zeev

 At 21:55 30/12/2002, Joseph Tate wrote:
 That's no good.  If I remove the sapi_close stuff, and try to execute the
 shutdown functions in php_apache_request_shutdown() all I get is stuff in
 the error log listing the leaked memory.  The requested
 function does not
 get executed.
 
 Joseph
 
   -Original Message-
   From: Zeev Suraski [mailto:[EMAIL PROTECTED]]
   Sent: Monday, December 30, 2002 1:57 PM
   Subject: RE: [PATCH]apache_register_shutdown_function final version
  
  
   Try looking at php_apache_request_shutdown() in mod_php4.c.  It's
   our pool
   destructor.
  
   Zeev





-- 
PHP Development Mailing List http://www.php.net/
To unsubscribe, visit: http://www.php.net/unsub.php




[PHP-DEV] RE: [PATCH]apache_register_shutdown_function final version

2002-12-31 Thread Zeev Suraski
As you can see in bug #15209, the change that introduced this problem was 
that we started calling php_request_shutdown() in apache_module_main(), 
prior to calling it from php_apache_request_shutdown().  So, the place to 
put it is clearly php_apache_request_shutdown(), where it used to be called 
before.

The problem is by the time you would get to php_apache_request_shutdown(), 
the per-request memory manager is already deactivated, so any emalloc()'d 
memory you may have is already freed.  I'm not sure how to tackle this in a 
server independent way...

Zeev

At 21:55 30/12/2002, Joseph Tate wrote:
That's no good.  If I remove the sapi_close stuff, and try to execute the
shutdown functions in php_apache_request_shutdown() all I get is stuff in
the error log listing the leaked memory.  The requested function does not
get executed.

Joseph

 -Original Message-
 From: Zeev Suraski [mailto:[EMAIL PROTECTED]]
 Sent: Monday, December 30, 2002 1:57 PM
 Subject: RE: [PATCH]apache_register_shutdown_function final version


 Try looking at php_apache_request_shutdown() in mod_php4.c.  It's
 our pool
 destructor.

 Zeev



--
PHP Development Mailing List http://www.php.net/
To unsubscribe, visit: http://www.php.net/unsub.php




[PHP-DEV] Re: [PATCH]apache_register_shutdown_function final version

2002-12-30 Thread Zeev Suraski
That's a bit of an odd way to implement it - are you sure closing the link 
explicitly at that point won't interfere with anything?  In 4.1 (or 
whatever the last version it worked like that was), it was taking advantage 
of the fact PHP's resource pool was being destroyed after the link was 
already closed.  It sounds a bit risky to me to close the link explicitly 
at this point, even though I'm not currently aware of any immediate problem 
this may cause (maybe it would ruin keep-alive?)

Zeev

At 19:22 30/12/2002, Joseph Tate wrote:
As a reminder, this replaces the register_shutdown_function functionality
removed in 4.1.x as described at http://bugs.php.net/15209.  I've made my
final adjustments to the patch.  Please review and commit both to HEAD and
PHP_4_3.  I received no response from the SAPI guru's, so I went ahead and
added the sapi_close function to all SAPI modules, initializing it to NULL.
It's not implemented for anything but apache 1.3.  Implementing for Apache2
is not a big deal, just call ap_lingering_close from within sapi_close.

With the addition of sapi_close, it should be possible to add the
functionality of apache_register_shutdown_function to every platform
implementing the sapi_close method.

Also, those on non-Apache systems, please test to make sure that this does
not break your builds.

Thanks,
Joseph
[EMAIL PROTECTED]





--
PHP Development Mailing List http://www.php.net/
To unsubscribe, visit: http://www.php.net/unsub.php




[PHP-DEV] RE: [PATCH]apache_register_shutdown_function final version

2002-12-30 Thread Joseph Tate
There probably is a better way to do it.  I just haven't been able to figure
it out.  Most Apache modules wait until the logging stage to execute
offline code, but it doesn't seem that at that stage PHP code is still in
memory.  Thus there doesn't seem to be a satisfactory way to do this.  As
for explicitly closing the connection, I call flush before I do that, and
everything that PHP cares about has been done, except for garbage_collecting
and unloading itself.  Apache goes ahead and cleans up everything else after
the logging stage.  I haven't tried explicitly with a Connection-Type:
Keep-Alive, but I don't think it will cause browsers to crash.  I've tested
with IE 6x and Mozilla 1.2.1.  My implementation was based on code existing
in the apache sources (just not externally available in Apache 1.3.x)
lingering_close and much discussion on the Apache-Modules mailing list.

If you can point me to a place in PHP land where the Apache process has
closed the connection, and the script is still in memory, I'll be happy to
make it neater, but in my search (I spent a day or so looking for such a
place) I couldn't find anything.

Another interesting thing to note is that the Patch that worked with 4.1.x
to restore the functionality doesn't work with 4.2 or 4.3, so something has
changed so that all resource pool cleanup is happening before the connection
is closed rather than after, causing PHP to appear slower than it needs to.

Joseph

 -Original Message-
 From: Zeev Suraski [mailto:[EMAIL PROTECTED]]
 Sent: Monday, December 30, 2002 12:55 PM

 That's a bit of an odd way to implement it - are you sure closing
 the link
 explicitly at that point won't interfere with anything?  In 4.1 (or
 whatever the last version it worked like that was), it was taking
 advantage
 of the fact PHP's resource pool was being destroyed after the link was
 already closed.  It sounds a bit risky to me to close the link explicitly
 at this point, even though I'm not currently aware of any
 immediate problem
 this may cause (maybe it would ruin keep-alive?)

 Zeev



-- 
PHP Development Mailing List http://www.php.net/
To unsubscribe, visit: http://www.php.net/unsub.php




[PHP-DEV] RE: [PATCH]apache_register_shutdown_function final version

2002-12-30 Thread Zeev Suraski
Try looking at php_apache_request_shutdown() in mod_php4.c.  It's our pool 
destructor.

Zeev

At 20:55 30/12/2002, Joseph Tate wrote:
There probably is a better way to do it.  I just haven't been able to figure
it out.  Most Apache modules wait until the logging stage to execute
offline code, but it doesn't seem that at that stage PHP code is still in
memory.  Thus there doesn't seem to be a satisfactory way to do this.  As
for explicitly closing the connection, I call flush before I do that, and
everything that PHP cares about has been done, except for garbage_collecting
and unloading itself.  Apache goes ahead and cleans up everything else after
the logging stage.  I haven't tried explicitly with a Connection-Type:
Keep-Alive, but I don't think it will cause browsers to crash.  I've tested
with IE 6x and Mozilla 1.2.1.  My implementation was based on code existing
in the apache sources (just not externally available in Apache 1.3.x)
lingering_close and much discussion on the Apache-Modules mailing list.

If you can point me to a place in PHP land where the Apache process has
closed the connection, and the script is still in memory, I'll be happy to
make it neater, but in my search (I spent a day or so looking for such a
place) I couldn't find anything.

Another interesting thing to note is that the Patch that worked with 4.1.x
to restore the functionality doesn't work with 4.2 or 4.3, so something has
changed so that all resource pool cleanup is happening before the connection
is closed rather than after, causing PHP to appear slower than it needs to.

Joseph

 -Original Message-
 From: Zeev Suraski [mailto:[EMAIL PROTECTED]]
 Sent: Monday, December 30, 2002 12:55 PM

 That's a bit of an odd way to implement it - are you sure closing
 the link
 explicitly at that point won't interfere with anything?  In 4.1 (or
 whatever the last version it worked like that was), it was taking
 advantage
 of the fact PHP's resource pool was being destroyed after the link was
 already closed.  It sounds a bit risky to me to close the link explicitly
 at this point, even though I'm not currently aware of any
 immediate problem
 this may cause (maybe it would ruin keep-alive?)

 Zeev




--
PHP Development Mailing List http://www.php.net/
To unsubscribe, visit: http://www.php.net/unsub.php




[PHP-DEV] RE: [PATCH]apache_register_shutdown_function final version

2002-12-30 Thread Joseph Tate
That's no good.  If I remove the sapi_close stuff, and try to execute the
shutdown functions in php_apache_request_shutdown() all I get is stuff in
the error log listing the leaked memory.  The requested function does not
get executed.

Joseph

 -Original Message-
 From: Zeev Suraski [mailto:[EMAIL PROTECTED]]
 Sent: Monday, December 30, 2002 1:57 PM
 Subject: RE: [PATCH]apache_register_shutdown_function final version


 Try looking at php_apache_request_shutdown() in mod_php4.c.  It's
 our pool
 destructor.

 Zeev


-- 
PHP Development Mailing List http://www.php.net/
To unsubscribe, visit: http://www.php.net/unsub.php




RE: [PHP-DEV] RE: [PATCH]apache_register_shutdown_function final version

2002-12-30 Thread Joseph Tate
Upon further inspection, it is clear that shutdown_memory_manager is called
before php_apache_request_shutdown.  This, it seems, hails back from the
change that caused cleanup to happen before the connection closed, which may
or may not be related to the memory management changes.  This causes the
Zend engine to deallocate all my nice pointers.

php_apache_request_shutdown is run as an apache cleanup routine.  This is
good design, but in main.c in php_request_shutdown, the PHP code as it
exists when the process is running is destroyed.  This long before the
connection is shut down, and in turn long before php_apache_request_shutdown
is run.  Thus any attempt to move php_call_shutdown_functions any lower in
the execution order of php_request_shutdown results in a segmentation fault.
This was what I did originally to test for an execution location where the
connection was closed but PHP was still intact.  It seems, as I suspected,
that there is no place where the connection is closed and PHP code still
exists.

Back to the solution proposed, I've put a test around the sapi_close call
which means that it will only be called if there are registered apache
shutdown functions.  Will this placate you?  It will thus only break
keep-alive if there are offline functions to be run.

If this is still unacceptable, the php_request_shutdown may need to be
redesigned, and thus, I fear, all of SAPI.

Joseph

 -Original Message-
 From: Joseph Tate [mailto:[EMAIL PROTECTED]]
 Sent: Monday, December 30, 2002 2:55 PM
 Subject: [PHP-DEV] RE: [PATCH]apache_register_shutdown_function final
 version


 That's no good.  If I remove the sapi_close stuff, and try to execute the
 shutdown functions in php_apache_request_shutdown() all I get is stuff in
 the error log listing the leaked memory.  The requested
 function does not
 get executed.

 Joseph


-- 
PHP Development Mailing List http://www.php.net/
To unsubscribe, visit: http://www.php.net/unsub.php