RE: [PHP-DEV] Garbage collector patch
OK everyone's back from holidays and vacations :) Unless there are any serious objections we'll commit the GC patch on Monday to both HEAD and PHP_5_3 to get a wider network of testers. If anything looks fishy and/or people's benchmarks raise concerns we can back it out of PHP_5_3 (or tweak) if/when needed. Andi -Original Message- From: Johannes Schlüter [mailto:[EMAIL PROTECTED] Sent: Thursday, January 03, 2008 5:11 AM To: Andi Gutmans Cc: Derick Rethans; Dmitry Stogov; PHP Developers Mailing List Subject: RE: [PHP-DEV] Garbage collector patch Hi, On Tue, 2007-12-18 at 09:57 -0800, Andi Gutmans wrote: Uhm I meant 5.3. I guess we can commit it but as you know, reverting this kind of stuff later is a headache people don't like doing. With this patch it's not too bad because besides the macros which are committed already it's mostly isolated. So can anybody do the commit then? Thanks. Only enough testing by people on the list will be able to give us additional information re: to the merits of this patch and how we should handle it. My hope people find it stable with negligible performance impact so that we can always keep it compiled in by default. Right, and having it in CVS really helps having many people testing it in different environments. johannes Andi -Original Message- From: Derick Rethans [mailto:[EMAIL PROTECTED] Sent: Tuesday, December 18, 2007 9:20 AM To: Andi Gutmans Cc: Dmitry Stogov; PHP Developers Mailing List Subject: RE: [PHP-DEV] Garbage collector patch On Tue, 18 Dec 2007, Andi Gutmans wrote: Derick Rethans wrote: Maybe we just should put it in cvs then? Then we won't have this issue and other people can test it more easily as well. The only problem with that is what if people get bad benchmarks/results and we want to pull it out? It kind of defeats the purpose to do this in a more structured way. Want us to commit it to a branch? PHP 5.2.x? Neither of those solve the merging issues, so no, not to a branch or 5.2. It should just go to HEAD and PHP_5_3. *If* it seems to be really bad, we can always revert it later. regards, Derick -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP-DEV] Garbage collector patch
Hi, On Tue, 2007-12-18 at 09:57 -0800, Andi Gutmans wrote: Uhm I meant 5.3. I guess we can commit it but as you know, reverting this kind of stuff later is a headache people don't like doing. With this patch it's not too bad because besides the macros which are committed already it's mostly isolated. So can anybody do the commit then? Thanks. Only enough testing by people on the list will be able to give us additional information re: to the merits of this patch and how we should handle it. My hope people find it stable with negligible performance impact so that we can always keep it compiled in by default. Right, and having it in CVS really helps having many people testing it in different environments. johannes Andi -Original Message- From: Derick Rethans [mailto:[EMAIL PROTECTED] Sent: Tuesday, December 18, 2007 9:20 AM To: Andi Gutmans Cc: Dmitry Stogov; PHP Developers Mailing List Subject: RE: [PHP-DEV] Garbage collector patch On Tue, 18 Dec 2007, Andi Gutmans wrote: Derick Rethans wrote: Maybe we just should put it in cvs then? Then we won't have this issue and other people can test it more easily as well. The only problem with that is what if people get bad benchmarks/results and we want to pull it out? It kind of defeats the purpose to do this in a more structured way. Want us to commit it to a branch? PHP 5.2.x? Neither of those solve the merging issues, so no, not to a branch or 5.2. It should just go to HEAD and PHP_5_3. *If* it seems to be really bad, we can always revert it later. regards, Derick -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, it would really be great if this could be part of CVS alredy. The patch doesn't compile currently, I get zillions of Zend/zend_execute.o: In function `gc_zval_check_possible_root': php5.3-200712250730/Zend/zend_gc.h:175: undefined reference to `gc_zval_possible_root' Zend/zend_execute.o: In function `gc_remove_from_buffer': php5.3-200712250730/Zend/zend_gc.h:183: undefined reference to `gc_globals' php5.3-200712250730/Zend/zend_gc.h:184: undefined reference to `gc_globals' php5.3-200712250730/Zend/zend_gc.h:183: undefined reference to `gc_globals' php5.3-200712250730/Zend/zend_gc.h:184: undefined reference to `gc_globals' collect2: ld returned 1 exit status right now ... - - Markus Dmitry Stogov wrote: updated patch. Dmitry. Derick Rethans wrote: On Mon, 17 Dec 2007, Dmitry Stogov wrote: Didn't I send it to you? Maybe, maybe not :) I couldn't find it atleast. I just tried to apply this to PHP 5.3, but it gives lots of failed chucks... Are you sure this is the one against 5.3? regards, Derick -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHcMSe1nS0RcInK9ARApIzAJ4iQgfSe3bBjjdF87GEk9nlqjF42wCgtkd/ 0pmCIOSo0irNjsqHELkjmwY= =xtVx -END PGP SIGNATURE- -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
./buildconf --force ./config.nice make clean make Dmtiry. Markus Fischer wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, it would really be great if this could be part of CVS alredy. The patch doesn't compile currently, I get zillions of Zend/zend_execute.o: In function `gc_zval_check_possible_root': php5.3-200712250730/Zend/zend_gc.h:175: undefined reference to `gc_zval_possible_root' Zend/zend_execute.o: In function `gc_remove_from_buffer': php5.3-200712250730/Zend/zend_gc.h:183: undefined reference to `gc_globals' php5.3-200712250730/Zend/zend_gc.h:184: undefined reference to `gc_globals' php5.3-200712250730/Zend/zend_gc.h:183: undefined reference to `gc_globals' php5.3-200712250730/Zend/zend_gc.h:184: undefined reference to `gc_globals' collect2: ld returned 1 exit status right now ... - - Markus Dmitry Stogov wrote: updated patch. Dmitry. Derick Rethans wrote: On Mon, 17 Dec 2007, Dmitry Stogov wrote: Didn't I send it to you? Maybe, maybe not :) I couldn't find it atleast. I just tried to apply this to PHP 5.3, but it gives lots of failed chucks... Are you sure this is the one against 5.3? regards, Derick -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHcMSe1nS0RcInK9ARApIzAJ4iQgfSe3bBjjdF87GEk9nlqjF42wCgtkd/ 0pmCIOSo0irNjsqHELkjmwY= =xtVx -END PGP SIGNATURE- -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
On 25.12.2007 11:51, Markus Fischer wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, it would really be great if this could be part of CVS alredy. The patch doesn't compile currently, I get zillions of You didn't read 3 last messages in the thread, did you? You have to run ./buildconf ./configure after applying the patch. -- Wbr, Antony Dovgal -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ok, I got the message :-) Sorry for interruption, thanks, works as expected. - - Markus Dmitry Stogov wrote: ./buildconf --force ./config.nice make clean make Dmtiry. Markus Fischer wrote: Hi, it would really be great if this could be part of CVS alredy. The patch doesn't compile currently, I get zillions of Zend/zend_execute.o: In function `gc_zval_check_possible_root': php5.3-200712250730/Zend/zend_gc.h:175: undefined reference to `gc_zval_possible_root' Zend/zend_execute.o: In function `gc_remove_from_buffer': php5.3-200712250730/Zend/zend_gc.h:183: undefined reference to `gc_globals' php5.3-200712250730/Zend/zend_gc.h:184: undefined reference to `gc_globals' php5.3-200712250730/Zend/zend_gc.h:183: undefined reference to `gc_globals' php5.3-200712250730/Zend/zend_gc.h:184: undefined reference to `gc_globals' collect2: ld returned 1 exit status right now ... - Markus Dmitry Stogov wrote: updated patch. Dmitry. Derick Rethans wrote: On Mon, 17 Dec 2007, Dmitry Stogov wrote: Didn't I send it to you? Maybe, maybe not :) I couldn't find it atleast. I just tried to apply this to PHP 5.3, but it gives lots of failed chucks... Are you sure this is the one against 5.3? regards, Derick -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHcM471nS0RcInK9ARAmz+AJ42FYnr2HIQgf6BFPUlQvHIZ/e0gACfbp7c YarzVjLMClNpvTR0XNPlMLM= =9A+8 -END PGP SIGNATURE- -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
On Tue, 18 Dec 2007, Rasmus Lerdorf wrote: Ilia Alshanetsky wrote: Putting it into CVS into a good idea, but lets create a revert point via a tag, so we can always easily undo the patch if need be. Making sure it is ifdef'ed nicely would let us leave it in CVS until we get it right and if it causes problems for some people they have a way to build PHP without it. And yes, that means their build won't be binary compatible, which is fine and no different from them trying to revert the patch themselves. Both ideas sound like a good plan to me! regards, Derick -- Derick Rethans http://derickrethans.nl | http://ezcomponents.org | http://xdebug.org -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
On Wed, 19 Dec 2007, Dmitry Stogov wrote: Derick Rethans wrote: On Mon, 17 Dec 2007, Dmitry Stogov wrote: Didn't I send it to you? Maybe, maybe not :) I couldn't find it atleast. I just tried to apply this to PHP 5.3, but it gives lots of failed chucks... Are you sure this is the one against 5.3? updated patch. Great, testing it now! regards, Derick -- Derick Rethans http://derickrethans.nl | http://ezcomponents.org | http://xdebug.org -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
On 19.12.2007 13:20, Alexey Zakhlestin wrote: On 12/19/07, Dmitry Stogov [EMAIL PROTECTED] wrote: updated patch. got an error during linking phase You have to run this first: ./cvsclean ./buildconf -- Wbr, Antony Dovgal -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
On Wed, 19 Dec 2007, Alexey Zakhlestin wrote: On 12/19/07, Dmitry Stogov [EMAIL PROTECTED] wrote: updated patch. got an error during linking phase /usr/bin/ld: Undefined symbols: _gc_globals_id _gc_globals_ctor _gc_zval_possible_root _gc_collect_cycles _gc_init _gc_reset _gc_zobj_possible_root collect2: ld returned 1 exit status make: *** [sapi/cli/php] Error 1 You have to re-run buildconf, it adds a zend_gc.c file. regards, Derick -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
On 12/19/07, Dmitry Stogov [EMAIL PROTECTED] wrote: updated patch. got an error during linking phase /usr/bin/ld: Undefined symbols: _gc_globals_id _gc_globals_ctor _gc_zval_possible_root _gc_collect_cycles _gc_init _gc_reset _gc_zobj_possible_root collect2: ld returned 1 exit status make: *** [sapi/cli/php] Error 1 OS: MacOS-X 10.4.11, intel -- Alexey Zakhlestin http://blog.milkfarmsoft.com/ -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
On 12/19/07, Dmitry Stogov [EMAIL PROTECTED] wrote: updated patch. Great! I managed to compile it. Works for me. The problem mentioned here is gone: http://blog.milkfarmsoft.com/?p=52 It is now possible to continue my appserver implementation ;) -- Alexey Zakhlestin http://blog.milkfarmsoft.com/ -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
On Mon, 17 Dec 2007, Dmitry Stogov wrote: Didn't I send it to you? Maybe, maybe not :) I couldn't find it atleast. I just tried to apply this to PHP 5.3, but it gives lots of failed chucks... Are you sure this is the one against 5.3? regards, Derick -- Derick Rethans http://derickrethans.nl | http://ezcomponents.org | http://xdebug.org -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
Probably I've changed ZE after the patch was done :( I'll rebuild it, but not today. Dmitry. Derick Rethans wrote: On Mon, 17 Dec 2007, Dmitry Stogov wrote: Didn't I send it to you? Maybe, maybe not :) I couldn't find it atleast. I just tried to apply this to PHP 5.3, but it gives lots of failed chucks... Are you sure this is the one against 5.3? regards, Derick -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
On Tue, 18 Dec 2007, Dmitry Stogov wrote: Derick Rethans wrote: On Mon, 17 Dec 2007, Dmitry Stogov wrote: Didn't I send it to you? Maybe, maybe not :) I couldn't find it atleast. I just tried to apply this to PHP 5.3, but it gives lots of failed chucks... Are you sure this is the one against 5.3? Probably I've changed ZE after the patch was done :( I'll rebuild it, but not today. Maybe we just should put it in cvs then? Then we won't have this issue and other people can test it more easily as well. regards, Derick -- Derick Rethans http://derickrethans.nl | http://ezcomponents.org | http://xdebug.org -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
On 12/18/07, Derick Rethans [EMAIL PROTECTED] wrote: Maybe we just should put it in cvs then? Then we won't have this issue and other people can test it more easily as well. +1 I want to test it, but, usually, messing with large patches is too time-consuming :-( having it in CVS will give me a chance -- Alexey Zakhlestin http://blog.milkfarmsoft.com/ -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP-DEV] Garbage collector patch
-Original Message- From: Derick Rethans [mailto:[EMAIL PROTECTED] Sent: Tuesday, December 18, 2007 7:06 AM To: Dmitry Stogov Cc: PHP Developers Mailing List Subject: Re: [PHP-DEV] Garbage collector patch Maybe we just should put it in cvs then? Then we won't have this issue and other people can test it more easily as well. The only problem with that is what if people get bad benchmarks/results and we want to pull it out? It kind of defeats the purpose to do this in a more structured way. Want us to commit it to a branch? PHP 5.2.x? Andi
RE: [PHP-DEV] Garbage collector patch
On Tue, 18 Dec 2007, Andi Gutmans wrote: Derick Rethans wrote: Maybe we just should put it in cvs then? Then we won't have this issue and other people can test it more easily as well. The only problem with that is what if people get bad benchmarks/results and we want to pull it out? It kind of defeats the purpose to do this in a more structured way. Want us to commit it to a branch? PHP 5.2.x? Neither of those solve the merging issues, so no, not to a branch or 5.2. It should just go to HEAD and PHP_5_3. *If* it seems to be really bad, we can always revert it later. regards, Derick -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP-DEV] Garbage collector patch
Uhm I meant 5.3. I guess we can commit it but as you know, reverting this kind of stuff later is a headache people don't like doing. With this patch it's not too bad because besides the macros which are committed already it's mostly isolated. Only enough testing by people on the list will be able to give us additional information re: to the merits of this patch and how we should handle it. My hope people find it stable with negligible performance impact so that we can always keep it compiled in by default. Andi -Original Message- From: Derick Rethans [mailto:[EMAIL PROTECTED] Sent: Tuesday, December 18, 2007 9:20 AM To: Andi Gutmans Cc: Dmitry Stogov; PHP Developers Mailing List Subject: RE: [PHP-DEV] Garbage collector patch On Tue, 18 Dec 2007, Andi Gutmans wrote: Derick Rethans wrote: Maybe we just should put it in cvs then? Then we won't have this issue and other people can test it more easily as well. The only problem with that is what if people get bad benchmarks/results and we want to pull it out? It kind of defeats the purpose to do this in a more structured way. Want us to commit it to a branch? PHP 5.2.x? Neither of those solve the merging issues, so no, not to a branch or 5.2. It should just go to HEAD and PHP_5_3. *If* it seems to be really bad, we can always revert it later. regards, Derick
RE: [PHP-DEV] Garbage collector patch
On Tue, 18 Dec 2007, Andi Gutmans wrote: Derick Rethans wrote: On Tue, 18 Dec 2007, Andi Gutmans wrote: Derick Rethans wrote: Maybe we just should put it in cvs then? Then we won't have this issue and other people can test it more easily as well. The only problem with that is what if people get bad benchmarks/results and we want to pull it out? It kind of defeats the purpose to do this in a more structured way. Want us to commit it to a branch? PHP 5.2.x? Neither of those solve the merging issues, so no, not to a branch or 5.2. It should just go to HEAD and PHP_5_3. *If* it seems to be really bad, we can always revert it later. Uhm I meant 5.3. I guess we can commit it but as you know, reverting this kind of stuff later is a headache people don't like doing. Life's not always easy ;-) regards, Derick -- Derick Rethans http://derickrethans.nl | http://ezcomponents.org | http://xdebug.org -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
Uhm I meant 5.3. I guess we can commit it but as you know, reverting this kind of stuff later is a headache people don't like doing. Life's not always easy ;-) So why again life should be not easy for all PHP tree users and not only for 3 people that work on the specific patch? CVS has branches just for that purpose. Merging a branch is easy. Reverting a patch with changes made over it is hard. -- Stanislav Malyshev, Zend Software Architect [EMAIL PROTECTED] http://www.zend.com/ (408)253-8829 MSN: [EMAIL PROTECTED] -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
Putting it into CVS into a good idea, but lets create a revert point via a tag, so we can always easily undo the patch if need be. On 18-Dec-07, at 10:05 AM, Derick Rethans wrote: On Tue, 18 Dec 2007, Dmitry Stogov wrote: Derick Rethans wrote: On Mon, 17 Dec 2007, Dmitry Stogov wrote: Didn't I send it to you? Maybe, maybe not :) I couldn't find it atleast. I just tried to apply this to PHP 5.3, but it gives lots of failed chucks... Are you sure this is the one against 5.3? Probably I've changed ZE after the patch was done :( I'll rebuild it, but not today. Maybe we just should put it in cvs then? Then we won't have this issue and other people can test it more easily as well. regards, Derick -- Derick Rethans http://derickrethans.nl | http://ezcomponents.org | http://xdebug.org -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php Ilia Alshanetsky -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
Making sure it is ifdef'ed nicely would let us leave it in CVS until we get it right and if it causes problems for some people they have a way to build PHP without it. And yes, that means their build won't be binary compatible, which is fine and no different from them trying to revert the patch themselves. -Rasmus Ilia Alshanetsky wrote: Putting it into CVS into a good idea, but lets create a revert point via a tag, so we can always easily undo the patch if need be. On 18-Dec-07, at 10:05 AM, Derick Rethans wrote: On Tue, 18 Dec 2007, Dmitry Stogov wrote: Derick Rethans wrote: On Mon, 17 Dec 2007, Dmitry Stogov wrote: Didn't I send it to you? Maybe, maybe not :) I couldn't find it atleast. I just tried to apply this to PHP 5.3, but it gives lots of failed chucks... Are you sure this is the one against 5.3? Probably I've changed ZE after the patch was done :( I'll rebuild it, but not today. Maybe we just should put it in cvs then? Then we won't have this issue and other people can test it more easily as well. regards, Derick -- Derick Rethans http://derickrethans.nl | http://ezcomponents.org | http://xdebug.org -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php Ilia Alshanetsky -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
Putting it into CVS into a good idea, but lets create a revert point via a tag, so we can always easily undo the patch if need be. Along with undoing all patches on the way done by other people for othe reasones. People, what's going on? Branches function in CVS is created EXACTLY for this use case, why on Earth not to use it? -- Stanislav Malyshev, Zend Software Architect [EMAIL PROTECTED] http://www.zend.com/ (408)253-8829 MSN: [EMAIL PROTECTED] -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
Making sure it is ifdef'ed nicely would let us leave it in CVS until we get it right and if it causes problems for some people they have a way Why we need non-working code in main CVS ifdef'ed if we can use perfectly good CVS branch functionality? Just to make life harder to ourselves and the code harder to edit and understand? -- Stanislav Malyshev, Zend Software Architect [EMAIL PROTECTED] http://www.zend.com/ (408)253-8829 MSN: [EMAIL PROTECTED] -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
On Thu, 13 Dec 2007, Derick Rethans wrote: On Thu, 13 Dec 2007, Dmitry Stogov wrote: You test are enormous, but than you anyway. :) I've found and fixed the problem. Now your test suite is passed with the following results: PHP_5_3: Tests: 7706, Failures: 45, Errors: 807, Skipped: 347 With GC: Tests: 7706, Failures: 45, Errors: 804, Skipped: 347 I've no idea why GC fixes three tests. (I can send you logs if you like) Yes, I'd appreciate that. And thanks for running them! Could you share the updated patch as well please? I'd like to play with it / test it - could you put the patch somewhere please? And also the test logs as I am interested in why the GC patch fixes things :) regards, Derick -- Derick Rethans http://derickrethans.nl | http://ezcomponents.org | http://xdebug.org -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
On Fri, 7 Dec 2007, Dmitry Stogov wrote: all your tests passed for me yesterday with memory_limit=1G. May be it's some fresh non GC related bug :) Try the same test with -dzend.enable_gc=0 and with unpatched PHP. I would very interested in this bad test case, if it's really related to GC. I tried to run it, but unfortunately my machine ran out of memory before it (that includes a full swapspace). I'd need access to a machine which has more memory than I have for this (4gb minimal). If you have such a machine, perhaps you can try yourself? Instead of running just the Template tests, I ran all of them: ~/dev/php/php-5.3dev-gc/sapi/cli/php -dzend.enable_gc=1 UnitTest/src/runtests.php -c /home/httpd/html/report -D sqlite://:memory: For -c to work, you will need xdebug installed though - I tried without but got the same issue. regards, Derick -- Derick Rethans http://derickrethans.nl | http://ezcomponents.org | http://xdebug.org -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
On Thu, 13 Dec 2007, Dmitry Stogov wrote: You test are enormous, but than you anyway. :) I've found and fixed the problem. Now your test suite is passed with the following results: PHP_5_3: Tests: 7706, Failures: 45, Errors: 807, Skipped: 347 With GC: Tests: 7706, Failures: 45, Errors: 804, Skipped: 347 I've no idea why GC fixes three tests. (I can send you logs if you like) Yes, I'd appreciate that. And thanks for running them! Could you share the updated patch as well please? regards, Derick -- Derick Rethans http://derickrethans.nl | http://ezcomponents.org | http://xdebug.org -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
Hi Derick, You test are enormous, but than you anyway. :) I've found and fixed the problem. Now your test suite is passed with the following results: PHP_5_3: Tests: 7706, Failures: 45, Errors: 807, Skipped: 347 With GC: Tests: 7706, Failures: 45, Errors: 804, Skipped: 347 I've no idea why GC fixes three tests. (I can send you logs if you like) Thanks. Dmitry. Derick Rethans wrote: On Fri, 7 Dec 2007, Dmitry Stogov wrote: all your tests passed for me yesterday with memory_limit=1G. May be it's some fresh non GC related bug :) Try the same test with -dzend.enable_gc=0 and with unpatched PHP. I would very interested in this bad test case, if it's really related to GC. I tried to run it, but unfortunately my machine ran out of memory before it (that includes a full swapspace). I'd need access to a machine which has more memory than I have for this (4gb minimal). If you have such a machine, perhaps you can try yourself? Instead of running just the Template tests, I ran all of them: ~/dev/php/php-5.3dev-gc/sapi/cli/php -dzend.enable_gc=1 UnitTest/src/runtests.php -c /home/httpd/html/report -D sqlite://:memory: For -c to work, you will need xdebug installed though - I tried without but got the same issue. regards, Derick -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
BTW the interesting fact that during Derick's test GC cycle collector was automatically run 124 times and it collected 2,388,399 zvals. So it really works! :) Dmitry. Dmitry Stogov wrote: Hi Derick, You test are enormous, but than you anyway. :) I've found and fixed the problem. Now your test suite is passed with the following results: PHP_5_3: Tests: 7706, Failures: 45, Errors: 807, Skipped: 347 With GC: Tests: 7706, Failures: 45, Errors: 804, Skipped: 347 I've no idea why GC fixes three tests. (I can send you logs if you like) Thanks. Dmitry. Derick Rethans wrote: On Fri, 7 Dec 2007, Dmitry Stogov wrote: all your tests passed for me yesterday with memory_limit=1G. May be it's some fresh non GC related bug :) Try the same test with -dzend.enable_gc=0 and with unpatched PHP. I would very interested in this bad test case, if it's really related to GC. I tried to run it, but unfortunately my machine ran out of memory before it (that includes a full swapspace). I'd need access to a machine which has more memory than I have for this (4gb minimal). If you have such a machine, perhaps you can try yourself? Instead of running just the Template tests, I ran all of them: ~/dev/php/php-5.3dev-gc/sapi/cli/php -dzend.enable_gc=1 UnitTest/src/runtests.php -c /home/httpd/html/report -D sqlite://:memory: For -c to work, you will need xdebug installed though - I tried without but got the same issue. regards, Derick -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP-DEV] Garbage collector patch
Hi Ilia, I suggest more people test the performance difference because as you can see for us it was negligible. From my experience you see bigger deviations just by moving kernels, compilers, and even small patches which affect where in memory the code segments sit, etc... Maybe some people here who have performance harnesses (I'm sure Yahoo! has one) could test it too. Andi -Original Message- From: Ilia Alshanetsky [mailto:[EMAIL PROTECTED] Sent: Saturday, December 08, 2007 9:35 AM To: [EMAIL PROTECTED] Cc: Rasmus Lerdorf; Stas Malyshev; [EMAIL PROTECTED]; 'Antony Dovgal'; Andi Gutmans; 'Cristian Rodriguez'; internals@lists.php.net Subject: Re: [PHP-DEV] Garbage collector patch Richard, zval is such a common PHP structure that in a scope of a single script (even a trivial one) you'd have thousands of them. Therefor even an extra 4 bytes matter, and for people with large application it would matter even more. I wish the patch was such that it had no impact on the people who don't need it, but the reality is that for anyone who does not need gc cycles, it'll be a performance (speed memory) penalty. This is why I had intimately suggested making it a compile time, configuration option. On 7-Dec-07, at 9:25 PM, Richard Lynch wrote: On Tue, December 4, 2007 2:18 pm, Rasmus Lerdorf wrote: Stanislav Malyshev wrote: 1. Always compile it in but leave undocumented #ifdefs in place for performance freaks. Those same performance freaks aren't going to care about the binary compatibility issue since they are the same people who build all their own stuff. Note that breaking BC is not only about performance - one your build is not the same as mainstream PHP, you can't use any binary extension which would do anything non-performance-related - like interfacing some external system/library, debugging, profiling, testing, security and so on. Any commercial module won't be available for the user of this switch, and all open-source modules one'd have to build by oneself, which may be serious maintenance issue. I know there are a bunch of companies that compile PHP with their own options but still use commercial modules, including both performance and non-performance ones. They couldn't use this compile switch. Yes, I know what binary compatibility means. Call me crazy, but... Would it be possible to hard-code the bit that adds the 4 bytes to the zval, but make the execution bit a flag so that binary compatibility is always there, but the executable code that does the GC can come or go as needed?... Or am I just talking nonsense due to ignorance? -- Some people have a gift link here. Know what I want? I want you to buy a CD from some indie artist. http://cdbaby.com/from/lynch Yeah, I get a buck. So? -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php Ilia Alshanetsky -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
Sorry to intrude on this one! It seems that some real hard work has gone into this, and a big thanks from the community for all your hard work. Can the gc patch feasibly be improved any more? If so surely the time scales involved with improving further would mean it'd miss the boat for a 5.3.x release..? Real-life use of PHP has to be the foremost concideration; in the production environment, especially on server farms and shared hosts, how many of them are going to be upgraded to 5.3 anyways, and when so, not for a long time (sadly)! 4 bytes vs the earlier 12 bytes is a significant improvement, and much as this may seem a terrible thing to say.. Roll it out in 5.3, turned on by default (with option to disable at compile time) - that's your test right there, if there are problems then have it disabled by default in the subsequent minor release. My only (and major) concern as an end developer is the 5% slowdown and 3% memory overhead, exactly what benefit am I as an end developer going recieve for this? Andi started this series with : On one hand I think now the effort has been made it's a good thing to offer it as part of PHP; and I for one agree, with the emphasis on *offer*. Many Regards Apologies for the intrusion. Nathan Andi Gutmans wrote: Hi Ilia, I suggest more people test the performance difference because as you can see for us it was negligible. From my experience you see bigger deviations just by moving kernels, compilers, and even small patches which affect where in memory the code segments sit, etc... Maybe some people here who have performance harnesses (I'm sure Yahoo! has one) could test it too. Andi -Original Message- From: Ilia Alshanetsky [mailto:[EMAIL PROTECTED] Sent: Saturday, December 08, 2007 9:35 AM To: [EMAIL PROTECTED] Cc: Rasmus Lerdorf; Stas Malyshev; [EMAIL PROTECTED]; 'Antony Dovgal'; Andi Gutmans; 'Cristian Rodriguez'; internals@lists.php.net Subject: Re: [PHP-DEV] Garbage collector patch Richard, zval is such a common PHP structure that in a scope of a single script (even a trivial one) you'd have thousands of them. Therefor even an extra 4 bytes matter, and for people with large application it would matter even more. I wish the patch was such that it had no impact on the people who don't need it, but the reality is that for anyone who does not need gc cycles, it'll be a performance (speed memory) penalty. This is why I had intimately suggested making it a compile time, configuration option. On 7-Dec-07, at 9:25 PM, Richard Lynch wrote: On Tue, December 4, 2007 2:18 pm, Rasmus Lerdorf wrote: Stanislav Malyshev wrote: 1. Always compile it in but leave undocumented #ifdefs in place for performance freaks. Those same performance freaks aren't going to care about the binary compatibility issue since they are the same people who build all their own stuff. Note that breaking BC is not only about performance - one your build is not the same as mainstream PHP, you can't use any binary extension which would do anything non-performance-related - like interfacing some external system/library, debugging, profiling, testing, security and so on. Any commercial module won't be available for the user of this switch, and all open-source modules one'd have to build by oneself, which may be serious maintenance issue. I know there are a bunch of companies that compile PHP with their own options but still use commercial modules, including both performance and non-performance ones. They couldn't use this compile switch. Yes, I know what binary compatibility means. Call me crazy, but... Would it be possible to hard-code the bit that adds the 4 bytes to the zval, but make the execution bit a flag so that binary compatibility is always there, but the executable code that does the GC can come or go as needed?... Or am I just talking nonsense due to ignorance? -- Some people have a gift link here. Know what I want? I want you to buy a CD from some indie artist. http://cdbaby.com/from/lynch Yeah, I get a buck. So? -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php Ilia Alshanetsky -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
On Thu, 6 Dec 2007, Dmitry Stogov wrote: Derick Rethans wrote: That doesn't mean it works better for me than the old one either, where I couldn't get it to crash at all ;-) Anyway, did you try to reproduce it with the instructions that I provided? Yes I did. I'm looking into the crash. I tested the updated patches now, it it's much better. I do however still get an error: Template: ezcTemplateRegressionTest: ../home/derick/dev/php/php-5.3dev-gc/Zend/zend_hash.c(668) : ht=0x3cc79338 is being destroyed but only after running *lots* of tests[1]. I'll try to rerun it in GDB to see whether I can get a backtrace, or through valgrind. [1] ~/dev/php/php-5.3dev-gc/sapi/cli/php -dzend.enable_gc=1 UnitTest/src/runtests.php -c /home/httpd/html/report -D sqlite://:memory: regards, Derick -- Derick Rethans http://derickrethans.nl | http://ezcomponents.org | http://xdebug.org -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
hi Derick, all your tests passed for me yesterday with memory_limit=1G. May be it's some fresh non GC related bug :) Try the same test with -dzend.enable_gc=0 and with unpatched PHP. I would very interested in this bad test case, if it's really related to GC. Thanks. Dmitry. Derick Rethans wrote: On Thu, 6 Dec 2007, Dmitry Stogov wrote: Derick Rethans wrote: That doesn't mean it works better for me than the old one either, where I couldn't get it to crash at all ;-) Anyway, did you try to reproduce it with the instructions that I provided? Yes I did. I'm looking into the crash. I tested the updated patches now, it it's much better. I do however still get an error: Template: ezcTemplateRegressionTest: ../home/derick/dev/php/php-5.3dev-gc/Zend/zend_hash.c(668) : ht=0x3cc79338 is being destroyed but only after running *lots* of tests[1]. I'll try to rerun it in GDB to see whether I can get a backtrace, or through valgrind. [1] ~/dev/php/php-5.3dev-gc/sapi/cli/php -dzend.enable_gc=1 UnitTest/src/runtests.php -c /home/httpd/html/report -D sqlite://:memory: regards, Derick -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP-DEV] Garbage collector patch
On Wed, 5 Dec 2007, Andi Gutmans wrote: Yes that's basically true although Dmitry can probably elaborate. I believe if the collector kicks in there'll also be a bit of a slowdown but my main concern is to be able to turn it off because of stability. As you can see with the update patch it seems performance is quite good. From what I can see, the patch doesn't actually work... it just crashes. Did you have a look at that yet? regards, Derick -- Derick Rethans http://derickrethans.nl | http://ezcomponents.org | http://xdebug.org -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
On 06.12.2007 11:21, Derick Rethans wrote: On Wed, 5 Dec 2007, Andi Gutmans wrote: Yes that's basically true although Dmitry can probably elaborate. I believe if the collector kicks in there'll also be a bit of a slowdown but my main concern is to be able to turn it off because of stability. As you can see with the update patch it seems performance is quite good. From what I can see, the patch doesn't actually work... it just crashes. Did you have a look at that yet? To be honest, I can't see any crashes using standard PHP test suite. Do you have a reproduce case or something? -- Wbr, Antony Dovgal -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
Antony Dovgal schrieb: To be honest, I can't see any crashes using standard PHP test suite. Does the standard PHP test suite contain PHP code with reference cycles? -- Sebastian Bergmann http://sebastian-bergmann.de/ GnuPG Key: 0xB85B5D69 / 27A7 2B14 09E4 98CD 6277 0E5B 6867 C514 B85B 5D69 -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
On Thu, 6 Dec 2007, Antony Dovgal wrote: On 06.12.2007 11:21, Derick Rethans wrote: On Wed, 5 Dec 2007, Andi Gutmans wrote: Yes that's basically true although Dmitry can probably elaborate. I believe if the collector kicks in there'll also be a bit of a slowdown but my main concern is to be able to turn it off because of stability. As you can see with the update patch it seems performance is quite good. From what I can see, the patch doesn't actually work... it just crashes. Did you have a look at that yet? To be honest, I can't see any crashes using standard PHP test suite. That doesn't surprise me, as none of the tests there run really complicated code. Do you have a reproduce case or something? Sure, it was in my original mail, but here it is again: To reproduce, run the Template tests of the eZ Components: # mkdir ezc # cd ezc # svn co http://svn.ez.no/svn/ezcomponents/trunk # svn co http://svn.ez.no/svn/ezcomponents/scripts # ./scripts/setup-env.sh # php UnitTest/src/run-tests.php Template regards, Derick -- Derick Rethans http://derickrethans.nl | http://ezcomponents.org | http://xdebug.org -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
Derick Rethans wrote: On Tue, 4 Dec 2007, Andi Gutmans wrote: To clarify I meant there's a tri-state (not compiled in, compiled in but collection turned off, compiled in but collection turned on). My recommendation was to always compile it in but to keep collection turned off by default. That's totally fine with me. With David's patch you could turn it on with both an ini setting (PHP_INI_ALL) or with a function. Can the improved one do so as well? The GC has two stages. 1) GC information collection (called very often during php script execution and doesn't take a lot of time) 2) unreferenced cycle collection (called seldom but may take long time) The original patch wrapped (1) and (2) with checks for GC on/off. The improved one only (2). The checks for GC on/off in original patch took just a bit less time than the whole GC information collection in the improved. Thanks. Dmitry. regards, Derick -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
The GC patch passes the whole php test-suite with the same bugs. The fact that it crashes on some script doesn't mean that the patch doesn't actually work. Thanks. Dmitry. Derick Rethans wrote: On Wed, 5 Dec 2007, Andi Gutmans wrote: Yes that's basically true although Dmitry can probably elaborate. I believe if the collector kicks in there'll also be a bit of a slowdown but my main concern is to be able to turn it off because of stability. As you can see with the update patch it seems performance is quite good. From what I can see, the patch doesn't actually work... it just crashes. Did you have a look at that yet? regards, Derick -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
On Thu, 6 Dec 2007, Dmitry Stogov wrote: Derick Rethans wrote: On Wed, 5 Dec 2007, Andi Gutmans wrote: Yes that's basically true although Dmitry can probably elaborate. I believe if the collector kicks in there'll also be a bit of a slowdown but my main concern is to be able to turn it off because of stability. As you can see with the update patch it seems performance is quite good. From what I can see, the patch doesn't actually work... it just crashes. Did you have a look at that yet? The GC patch passes the whole php test-suite with the same bugs. The fact that it crashes on some script doesn't mean that the patch doesn't actually work. That doesn't mean it works better for me than the old one either, where I couldn't get it to crash at all ;-) Anyway, did you try to reproduce it with the instructions that I provided? regards, Derick -- Derick Rethans http://derickrethans.nl | http://ezcomponents.org | http://xdebug.org -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
Dmitry Stogov wrote: In general this patch will use more memory. (4 bytes more for each heap allocated zval). The only advantage is automatic cycle collection, but most web applications doesn't make cycles. Could it be that this code should only be enabled for the CLI sapi? That is where I would want it most. Only in my cli apps do I run into having to trick the garbage collection into working in my favor. I usually end having my loop just call a function on each pass. That seems to help with memory use in PHP currently. -- Brian Moon Senior Developer -- http://dealnews.com/ It's good to be cheap =) -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP-DEV] Garbage collector patch
On Tue, 4 Dec 2007, Andi Gutmans wrote: To clarify I meant there's a tri-state (not compiled in, compiled in but collection turned off, compiled in but collection turned on). My recommendation was to always compile it in but to keep collection turned off by default. That's totally fine with me. With David's patch you could turn it on with both an ini setting (PHP_INI_ALL) or with a function. Can the improved one do so as well? regards, Derick -- Derick Rethans http://derickrethans.nl | http://ezcomponents.org | http://xdebug.org -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
To clarify I meant there's a tri-state (not compiled in, compiled in but collection turned off, compiled in but collection turned on). My recommendation was to always compile it in but to keep collection turned off by default. Do I understand it right that being compiled in, turned on and turned off have the same performance implications (slightly slower, a little bit more memory) and the difference is mainly stability-wise? -- Stanislav Malyshev, Zend Software Architect [EMAIL PROTECTED] http://www.zend.com/ (408)253-8829 MSN: [EMAIL PROTECTED] -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
On Wed, 2007-12-05 at 10:01 -0600, Brian Moon wrote: Dmitry Stogov wrote: In general this patch will use more memory. (4 bytes more for each heap allocated zval). The only advantage is automatic cycle collection, but most web applications doesn't make cycles. Could it be that this code should only be enabled for the CLI sapi? That is where I would want it most. Only in my cli apps do I run into having to trick the garbage collection into working in my favor. I usually end having my loop just call a function on each pass. That seems to help with memory use in PHP currently. Still comes down the issue of binary compatibility with respect to modules and such. Cheers, Rob. -- ... SwarmBuy.com - http://www.swarmbuy.com Leveraging the buying power of the masses! ... -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
On Mon, 3 Dec 2007, Andi Gutmans wrote: The revised patch has the following advantages: - It fixes several crash bugs So far, my tests only see new crash bugs... :/ Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x2ba1140c6930 (LWP 21108)] 0x008be4fe in zend_std_read_property (object=0x1d7f670, member=0x1e3a1e8, type=0) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_object_handlers.c:338 338 property_info = zend_get_property_info(zobj-ce, member, (zobj-ce-__get != NULL) TSRMLS_CC); (gdb) bt #0 0x008be4fe in zend_std_read_property (object=0x1d7f670, member=0x1e3a1e8, type=0) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_object_handlers.c:338 #1 0x00907fd2 in zend_fetch_property_address_read_helper_SPEC_UNUSED_CONST (type=0, execute_data=0x7fff9c73ffa0) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:16282 #2 0x009081ca in ZEND_FETCH_OBJ_R_SPEC_UNUSED_CONST_HANDLER (execute_data=0x7fff9c73ffa0) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:16304 #3 0x008c4694 in execute (op_array=0x1e17c70) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:87 #4 0x008c4dec in zend_do_fcall_common_helper_SPEC (execute_data=0x7fff9c740df0) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:221 #5 0x008c5b0d in ZEND_DO_FCALL_BY_NAME_SPEC_HANDLER (execute_data=0x7fff9c740df0) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:309 #6 0x008c4694 in execute (op_array=0x1dce4e0) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:87 #7 0x008c4dec in zend_do_fcall_common_helper_SPEC (execute_data=0x7fff9c741450) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:221 #8 0x008c5b0d in ZEND_DO_FCALL_BY_NAME_SPEC_HANDLER (execute_data=0x7fff9c741450) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:309 #9 0x008c4694 in execute (op_array=0x1e17dc0) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:87 #10 0x008c4dec in zend_do_fcall_common_helper_SPEC (execute_data=0x7fff9c741c80) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:221 #11 0x008c5b0d in ZEND_DO_FCALL_BY_NAME_SPEC_HANDLER (execute_data=0x7fff9c741c80) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:309 #12 0x008c4694 in execute (op_array=0x1de1a70) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:87 #13 0x008c4dec in zend_do_fcall_common_helper_SPEC (execute_data=0x7fff9c743bb0) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:221 #14 0x008c5b0d in ZEND_DO_FCALL_BY_NAME_SPEC_HANDLER (execute_data=0x7fff9c743bb0) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:309 #15 0x008c4694 in execute (op_array=0x1d69390) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:87 #16 0x008c4dec in zend_do_fcall_common_helper_SPEC (execute_data=0x7fff9c745550) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:221 #17 0x008c5b0d in ZEND_DO_FCALL_BY_NAME_SPEC_HANDLER (execute_data=0x7fff9c745550) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:309 #18 0x008c4694 in execute (op_array=0x1a20250) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:87 #19 0x008c4dec in zend_do_fcall_common_helper_SPEC (execute_data=0x7fff9c7459b0) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:221 #20 0x008c5b0d in ZEND_DO_FCALL_BY_NAME_SPEC_HANDLER (execute_data=0x7fff9c7459b0) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:309 #21 0x008c4694 in execute (op_array=0x1a18ed0) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:87 #22 0x008c4dec in zend_do_fcall_common_helper_SPEC (execute_data=0x7fff9c746340) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:221 #23 0x008c5b0d in ZEND_DO_FCALL_BY_NAME_SPEC_HANDLER (execute_data=0x7fff9c746340) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:309 #24 0x008c4694 in execute (op_array=0x1a38be0) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:87 #25 0x008c4dec in zend_do_fcall_common_helper_SPEC (execute_data=0x7fff9c7472e0) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:221 #26 0x008c5b0d in ZEND_DO_FCALL_BY_NAME_SPEC_HANDLER (execute_data=0x7fff9c7472e0) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:309 #27 0x008c4694 in execute (op_array=0x11b5430) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:87 #28 0x008c4dec in zend_do_fcall_common_helper_SPEC (execute_data=0x7fff9c7475d0) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:221 #29 0x008c5b0d in ZEND_DO_FCALL_BY_NAME_SPEC_HANDLER (execute_data=0x7fff9c7475d0) at /home/derick/dev/php/php-5.3dev-gc/Zend/zend_vm_execute.h:309 #30
Re: [PHP-DEV] Garbage collector patch
On Wed, 5 Dec 2007, Stanislav Malyshev wrote: To clarify I meant there's a tri-state (not compiled in, compiled in but collection turned off, compiled in but collection turned on). My recommendation was to always compile it in but to keep collection turned off by default. Do I understand it right that being compiled in, turned on and turned off have the same performance implications (slightly slower, a little bit more memory) and the difference is mainly stability-wise? Yeah - atleast, that's how David's original patch behaved. regards, Derick -- Derick Rethans http://derickrethans.nl | http://ezcomponents.org | http://xdebug.org -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP-DEV] Garbage collector patch
Yes that's basically true although Dmitry can probably elaborate. I believe if the collector kicks in there'll also be a bit of a slowdown but my main concern is to be able to turn it off because of stability. As you can see with the update patch it seems performance is quite good. Anyway will circle back with Dmitry just to be sure. Andi -Original Message- From: Derick Rethans [mailto:[EMAIL PROTECTED] Sent: Wednesday, December 05, 2007 3:10 PM To: Stas Malyshev Cc: Andi Gutmans; Guilherme Blanco; [EMAIL PROTECTED]; Rasmus Lerdorf; Antony Dovgal; Cristian Rodriguez; internals@lists.php.net Subject: Re: [PHP-DEV] Garbage collector patch On Wed, 5 Dec 2007, Stanislav Malyshev wrote: To clarify I meant there's a tri-state (not compiled in, compiled in but collection turned off, compiled in but collection turned on). My recommendation was to always compile it in but to keep collection turned off by default. Do I understand it right that being compiled in, turned on and turned off have the same performance implications (slightly slower, a little bit more memory) and the difference is mainly stability-wise? Yeah - atleast, that's how David's original patch behaved. regards, Derick -- Derick Rethans http://derickrethans.nl | http://ezcomponents.org | http://xdebug.org
Re: [PHP-DEV] Garbage collector patch
In general this patch will use more memory. (4 bytes more for each heap allocated zval). The only advantage is automatic cycle collection, but most web applications doesn't make cycles. Dmitry. Guilherme Blanco wrote: I want to put my 2 cents, but before make any comment, I'm interested to know if it's only a performance impact. Talking about performance is ok and I agree that any penalty should be taken into account, but if the patch uses less memory resources, I think that shared hostings will care about it. I remember when you released PHP 5_2, that was around 12% slower than any version of PHP 4. None care about it, and only in PHP 5_2 you took into account and optimized it a lot. As long as it uses less memory, it should be added into 5_3. Wang spent a lot of efforts to create the patch, Dmitry too. Also, I think you should take care of 5% of performance penalty is not huge if you consider that now you have a nice implementation of what was not working correctly (referring to circular references). But... if it uses more memory... I stay in the middle of the sides. Too much more? This is a factor. Besides all of this argumentation of performance... I'd go with Andi's first suggestion. No config for it... in or out. Make it configurable may generate some BC incompatibilities. Regards, On Dec 4, 2007 5:45 PM, [EMAIL PROTECTED] wrote: I think having it configurable is a must. Turning it on / off via a compile flag will not suit everyone. Eg - I have a situation where I would not want to run garbage collection for web pages served off my server due to the performance hit, however I also have a CLI script which I run to do complex, but repetitive tasks for ~half an hour. Now this is not really a big deal to me as I managed to rid most of the leaks by nulling out cyclic references, however that took a lot of work which could have been avoided by this. Could I suggest also enabling it by default for CLI? SCOTT MCNAUGHT Software Developer Synergy 8 / +617 3397 5212 [EMAIL PROTECTED] -Original Message- From: Rasmus Lerdorf [mailto:[EMAIL PROTECTED] Sent: Wednesday, 5 December 2007 5:17 AM To: Antony Dovgal Cc: Andi Gutmans; Cristian Rodriguez; internals@lists.php.net Subject: Re: [PHP-DEV] Garbage collector patch Antony Dovgal wrote: On 04.12.2007 18:31, Andi Gutmans wrote: You may be right longer term but shorter term I still believe there may be stability issues with this patch some of which we haven't figured out. Although we did testing and found crash bugs I wouldn't trust our level of testing to deem it stable. If we go down the route of always on we could have a hidden ini file (not listed in php.ini-dist and phpinfo()) which we can advise to turn off if something goes wrong and once we can enough confidence that there aren't any lurking bugs we could remove it. You mean provide an ini setting to be able to turn it Off? But why do you call it always On then? And what's the difference comparing to what you've proposed earlier? Concerning the stability issues, I'd say we have quite good chance to make it stable enough, as (I hope) 5.3.0 is not going to be released for at least several months more. Companies that are especially concerned of performance/stability are encouraged to step forward and give us a hand. Companies concerned about performance aren't going to use it at all. A 5% hit is significant given that it doesn't fix anything that a company already concerned about performance/stability cares about. Super leaky or recursively allocating scripts have long since been weeded out of those code bases, so it is a 5% performance hit with no gain. I am not arguing that it isn't useful in the general case. It will make PHP more robust on loosely controlled servers for what amounts to only a small penalty, but at the same time it is an easy 5% win for people with dedicated servers running well-written code. -Rasmus -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
On 04.12.2007 06:00, Cristian Rodriguez wrote: such switches only add more complexity, confusion for users and addtional trouble to distributors. ... which I believe are going to enable this flag by default anyway, because they don't care about performance too much. So I agree, either do it or not, yet another engine mode is bad idea. -- Wbr, Antony Dovgal -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
I believe this has to be done and it should be always on. Having less headache is good. I will find other ways to speed up my projects On 12/4/07, Antony Dovgal [EMAIL PROTECTED] wrote: On 04.12.2007 06:00, Cristian Rodriguez wrote: such switches only add more complexity, confusion for users and addtional trouble to distributors. ... which I believe are going to enable this flag by default anyway, because they don't care about performance too much. So I agree, either do it or not, yet another engine mode is bad idea. -- Wbr, Antony Dovgal -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php -- Alexey Zakhlestin http://blog.milkfarmsoft.com/ -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
On Tue, 2007-12-04 at 00:00 -0300, Cristian Rodriguez wrote: 2007/12/3, Ilia Alshanetsky [EMAIL PROTECTED]: I think the patch does have a value, yes, it does, what worries me is the introduction of yet another non-sense ini setting that modified the very engine behaviuor.. I think we all agree that there are way too many of those do we ? Yup. :) My suggestion is that we make this feature a compile time flag. My suggestion is not to add any compile time flag, either provide it or dont. I have to agree. Either it's in or not. We've had enough trouble with giving too much rope for people already.. -- Patches/Donations: http://pecl.php.net/~jani/ -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
I could be wrong, of course, but my guess is that the ~3% speed decrease a) is not really significant to 95% of the users (I mean _really_ significant; of course, everyone is going to whine about it first) and b) is going to be eliminated over time david Am 04.12.2007 um 03:43 schrieb Ilia Alshanetsky: First of all big thanks for Dmitry and David for spending time on this project and continuing to improve the original patch. Based on the results so far, I think the patch does have a value, but certainly not in a general case. Relative simple scripts have little to gain from it and only to loose 3-5% of speed depending on a situation and given insubstantial memory gains it seems of little use in general case. That said, there are complex applications (perhaps unnecessarily so ;-) ) that could see some real benefits from this code. My suggestion is that we make this feature a compile time flag, that's off by default and people who feel that their applications warrant a garbage collector can enable it for their install and at the same time those who don't (general case) have no penalties of having it in place. On 3-Dec-07, at 4:01 PM, Andi Gutmans wrote: Hi all, Was hoping to send this off earlier but I was travelling for the past week and had very limited email access. As promised in the past few weeks we have spent a significant amount of time in reviewing the garbage collector work and testing it in our performance lab. Dmitry has been exchanging ideas and patches with David Wang during this time. Suffice to say that as I've mentioned in the past, memory management is an extremely sensitive piece of PHP, which is why it was important for us to review this work in great depth before committing it to the code base. The updated patch still retains the same algorithm as David's original implementation however the data structures have been changed in order to work faster and use less memory. The revised patch has the following advantages: - It fixes several crash bugs - Enhances performance by removing several unnecessary checks - The memory overhead was reduced (from 12 to 4 bytes for each heap-allocated zval) - The speed of clean PHP code (code that doesn't create cycles) was improved - Additional test cases were created The end result is a more stable and faster GC patch. That said we have yet to find real-life applications that create significant cycles which would benefit from this patch. In fact as you'll see from the results our tests show an increase in maximum memory use and slower execution (to be fair they are marginal). We have tested both PHP_5_3 without any patches, the original patch and the new patch. The following table shows execution time (seconds for N requests) and slowdown. PHP_5_3 Original GC patch Current GC patch slowdown slowdown bench 11,550 12,310 6,58% 12,170 5,37% hello 8,781 8,852 0,81% 8,813 0,36% xoops 128,500 135,100 5,14% 130,200 1,32% static 18,540 20,840 12,41% 18,920 2,05% qdig 29,320 30,270 3,24% 29,610 0,99% qdig2 13,960 14,100 1,00% 14,090 0,93% The next table shows memory usage in MB and overhead PHP_5_3 Original GC patch Current GC patch overhead overhead hello 13,750 13,945 1,42% 13,765 0,11% xoops 18,036 18,636 3,33% 18,568 2,95% static 15,300 16,000 4,58% 15,308 0,05% qdig 14,820 15,008 1,27% 14,828 0,05% qdig2 14,824 15,012 1,27% 14,838 0,09% To summarize the patch lead to approx. 5% slowdown and 3% memory overhead for typical applications (as always, you mileage may vary depending on your system's architecture and OS although my guesstimate is that you will see even worse results in a 64bit environment). We also tested SugarCRM to get another sanity for these results and we got similar results. I am not quite sure where this leaves us with this patch. On one hand I think now the effort has been made it's a good thing to offer it as part of PHP. The downside is of course some performance degradation and possible instabilities as this patch continues to stabilize (it took about two releases for us to stabilize the Zend Memory Manager even after in-depth testing due to edge cases in allocation patterns and various extensions, etc...).I'd be surprised if our team has found all possible bugs. Personally I think the decision should be either in or out. Adding this as a compile option is not a good idea as it would create binary compatibility issues and would be a pain for the community. I think my inclination is to commit the patch, not have it #ifdef'ed but always compiled but to have the garbage collection itself turned off by default (mainly for stability reasons. Note: the increased memory footprint and performance loss also exists with the collection itself turned off). We can turn it on when we're in dev for snapshots so that we
RE: [PHP-DEV] Garbage collector patch
You may be right longer term but shorter term I still believe there may be stability issues with this patch some of which we haven't figured out. Although we did testing and found crash bugs I wouldn't trust our level of testing to deem it stable. If we go down the route of always on we could have a hidden ini file (not listed in php.ini-dist and phpinfo()) which we can advise to turn off if something goes wrong and once we can enough confidence that there aren't any lurking bugs we could remove it. Andi -Original Message- From: Antony Dovgal [mailto:[EMAIL PROTECTED] Sent: Tuesday, December 04, 2007 12:28 AM To: Cristian Rodriguez Cc: internals@lists.php.net Subject: Re: [PHP-DEV] Garbage collector patch On 04.12.2007 06:00, Cristian Rodriguez wrote: such switches only add more complexity, confusion for users and addtional trouble to distributors. ... which I believe are going to enable this flag by default anyway, because they don't care about performance too much. So I agree, either do it or not, yet another engine mode is bad idea. -- Wbr, Antony Dovgal -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
On 04.12.2007 18:31, Andi Gutmans wrote: You may be right longer term but shorter term I still believe there may be stability issues with this patch some of which we haven't figured out. Although we did testing and found crash bugs I wouldn't trust our level of testing to deem it stable. If we go down the route of always on we could have a hidden ini file (not listed in php.ini-dist and phpinfo()) which we can advise to turn off if something goes wrong and once we can enough confidence that there aren't any lurking bugs we could remove it. You mean provide an ini setting to be able to turn it Off? But why do you call it always On then? And what's the difference comparing to what you've proposed earlier? Concerning the stability issues, I'd say we have quite good chance to make it stable enough, as (I hope) 5.3.0 is not going to be released for at least several months more. Companies that are especially concerned of performance/stability are encouraged to step forward and give us a hand. -- Wbr, Antony Dovgal -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
that could see some real benefits from this code. My suggestion is that we make this feature a compile time flag, that's off by default and What about binary compatibility? -- Stanislav Malyshev, Zend Software Architect [EMAIL PROTECTED] http://www.zend.com/ (408)253-8829 MSN: [EMAIL PROTECTED] -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
I could be wrong, of course, but my guess is that the ~3% speed decrease a) is not really significant to 95% of the users (I mean _really_ significant; of course, everyone is going to whine about it first) and GC is not really useful to 95% of the users either, so is 5% of the users important or not? b) is going to be eliminated over time What is going to be eliminated, the slowdown? How? -- Stanislav Malyshev, Zend Software Architect [EMAIL PROTECTED] http://www.zend.com/ (408)253-8829 MSN: [EMAIL PROTECTED] -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
Antony Dovgal wrote: On 04.12.2007 18:31, Andi Gutmans wrote: You may be right longer term but shorter term I still believe there may be stability issues with this patch some of which we haven't figured out. Although we did testing and found crash bugs I wouldn't trust our level of testing to deem it stable. If we go down the route of always on we could have a hidden ini file (not listed in php.ini-dist and phpinfo()) which we can advise to turn off if something goes wrong and once we can enough confidence that there aren't any lurking bugs we could remove it. You mean provide an ini setting to be able to turn it Off? But why do you call it always On then? And what's the difference comparing to what you've proposed earlier? Concerning the stability issues, I'd say we have quite good chance to make it stable enough, as (I hope) 5.3.0 is not going to be released for at least several months more. Companies that are especially concerned of performance/stability are encouraged to step forward and give us a hand. Companies concerned about performance aren't going to use it at all. A 5% hit is significant given that it doesn't fix anything that a company already concerned about performance/stability cares about. Super leaky or recursively allocating scripts have long since been weeded out of those code bases, so it is a 5% performance hit with no gain. I am not arguing that it isn't useful in the general case. It will make PHP more robust on loosely controlled servers for what amounts to only a small penalty, but at the same time it is an easy 5% win for people with dedicated servers running well-written code. -Rasmus -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP-DEV] Garbage collector patch
I think having it configurable is a must. Turning it on / off via a compile flag will not suit everyone. Eg - I have a situation where I would not want to run garbage collection for web pages served off my server due to the performance hit, however I also have a CLI script which I run to do complex, but repetitive tasks for ~half an hour. Now this is not really a big deal to me as I managed to rid most of the leaks by nulling out cyclic references, however that took a lot of work which could have been avoided by this. Could I suggest also enabling it by default for CLI? SCOTT MCNAUGHT Software Developer Synergy 8 / +617 3397 5212 [EMAIL PROTECTED] -Original Message- From: Rasmus Lerdorf [mailto:[EMAIL PROTECTED] Sent: Wednesday, 5 December 2007 5:17 AM To: Antony Dovgal Cc: Andi Gutmans; Cristian Rodriguez; internals@lists.php.net Subject: Re: [PHP-DEV] Garbage collector patch Antony Dovgal wrote: On 04.12.2007 18:31, Andi Gutmans wrote: You may be right longer term but shorter term I still believe there may be stability issues with this patch some of which we haven't figured out. Although we did testing and found crash bugs I wouldn't trust our level of testing to deem it stable. If we go down the route of always on we could have a hidden ini file (not listed in php.ini-dist and phpinfo()) which we can advise to turn off if something goes wrong and once we can enough confidence that there aren't any lurking bugs we could remove it. You mean provide an ini setting to be able to turn it Off? But why do you call it always On then? And what's the difference comparing to what you've proposed earlier? Concerning the stability issues, I'd say we have quite good chance to make it stable enough, as (I hope) 5.3.0 is not going to be released for at least several months more. Companies that are especially concerned of performance/stability are encouraged to step forward and give us a hand. Companies concerned about performance aren't going to use it at all. A 5% hit is significant given that it doesn't fix anything that a company already concerned about performance/stability cares about. Super leaky or recursively allocating scripts have long since been weeded out of those code bases, so it is a 5% performance hit with no gain. I am not arguing that it isn't useful in the general case. It will make PHP more robust on loosely controlled servers for what amounts to only a small penalty, but at the same time it is an easy 5% win for people with dedicated servers running well-written code. -Rasmus -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
I want to put my 2 cents, but before make any comment, I'm interested to know if it's only a performance impact. Talking about performance is ok and I agree that any penalty should be taken into account, but if the patch uses less memory resources, I think that shared hostings will care about it. I remember when you released PHP 5_2, that was around 12% slower than any version of PHP 4. None care about it, and only in PHP 5_2 you took into account and optimized it a lot. As long as it uses less memory, it should be added into 5_3. Wang spent a lot of efforts to create the patch, Dmitry too. Also, I think you should take care of 5% of performance penalty is not huge if you consider that now you have a nice implementation of what was not working correctly (referring to circular references). But... if it uses more memory... I stay in the middle of the sides. Too much more? This is a factor. Besides all of this argumentation of performance... I'd go with Andi's first suggestion. No config for it... in or out. Make it configurable may generate some BC incompatibilities. Regards, On Dec 4, 2007 5:45 PM, [EMAIL PROTECTED] wrote: I think having it configurable is a must. Turning it on / off via a compile flag will not suit everyone. Eg - I have a situation where I would not want to run garbage collection for web pages served off my server due to the performance hit, however I also have a CLI script which I run to do complex, but repetitive tasks for ~half an hour. Now this is not really a big deal to me as I managed to rid most of the leaks by nulling out cyclic references, however that took a lot of work which could have been avoided by this. Could I suggest also enabling it by default for CLI? SCOTT MCNAUGHT Software Developer Synergy 8 / +617 3397 5212 [EMAIL PROTECTED] -Original Message- From: Rasmus Lerdorf [mailto:[EMAIL PROTECTED] Sent: Wednesday, 5 December 2007 5:17 AM To: Antony Dovgal Cc: Andi Gutmans; Cristian Rodriguez; internals@lists.php.net Subject: Re: [PHP-DEV] Garbage collector patch Antony Dovgal wrote: On 04.12.2007 18:31, Andi Gutmans wrote: You may be right longer term but shorter term I still believe there may be stability issues with this patch some of which we haven't figured out. Although we did testing and found crash bugs I wouldn't trust our level of testing to deem it stable. If we go down the route of always on we could have a hidden ini file (not listed in php.ini-dist and phpinfo()) which we can advise to turn off if something goes wrong and once we can enough confidence that there aren't any lurking bugs we could remove it. You mean provide an ini setting to be able to turn it Off? But why do you call it always On then? And what's the difference comparing to what you've proposed earlier? Concerning the stability issues, I'd say we have quite good chance to make it stable enough, as (I hope) 5.3.0 is not going to be released for at least several months more. Companies that are especially concerned of performance/stability are encouraged to step forward and give us a hand. Companies concerned about performance aren't going to use it at all. A 5% hit is significant given that it doesn't fix anything that a company already concerned about performance/stability cares about. Super leaky or recursively allocating scripts have long since been weeded out of those code bases, so it is a 5% performance hit with no gain. I am not arguing that it isn't useful in the general case. It will make PHP more robust on loosely controlled servers for what amounts to only a small penalty, but at the same time it is an easy 5% win for people with dedicated servers running well-written code. -Rasmus -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php -- Guilherme Blanco - Web Developer CBC - Certified Bindows Consultant Cell Phone: +55 (16) 9166-6902 MSN: [EMAIL PROTECTED] URL: http://blog.bisna.com São Carlos - SP/Brazil
Re: [PHP-DEV] Garbage collector patch
[EMAIL PROTECTED] wrote: I think having it configurable is a must. Turning it on / off via a compile flag will not suit everyone. Looking at the code, that isn't really possible. You could have a switch to turn off the collection, but that won't get you your performance back. To avoid the performance penalty you need it to be a compile-time decision. I only see 2 possible workable solutions here: 1. Always compile it in but leave undocumented #ifdefs in place for performance freaks. Those same performance freaks aren't going to care about the binary compatibility issue since they are the same people who build all their own stuff. 2. Drop it -Rasmus -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
1. Always compile it in but leave undocumented #ifdefs in place for performance freaks. Those same performance freaks aren't going to care about the binary compatibility issue since they are the same people who build all their own stuff. Note that breaking BC is not only about performance - one your build is not the same as mainstream PHP, you can't use any binary extension which would do anything non-performance-related - like interfacing some external system/library, debugging, profiling, testing, security and so on. Any commercial module won't be available for the user of this switch, and all open-source modules one'd have to build by oneself, which may be serious maintenance issue. I know there are a bunch of companies that compile PHP with their own options but still use commercial modules, including both performance and non-performance ones. They couldn't use this compile switch. -- Stanislav Malyshev, Zend Software Architect [EMAIL PROTECTED] http://www.zend.com/ (408)253-8829 MSN: [EMAIL PROTECTED] -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
Stanislav Malyshev wrote: 1. Always compile it in but leave undocumented #ifdefs in place for performance freaks. Those same performance freaks aren't going to care about the binary compatibility issue since they are the same people who build all their own stuff. Note that breaking BC is not only about performance - one your build is not the same as mainstream PHP, you can't use any binary extension which would do anything non-performance-related - like interfacing some external system/library, debugging, profiling, testing, security and so on. Any commercial module won't be available for the user of this switch, and all open-source modules one'd have to build by oneself, which may be serious maintenance issue. I know there are a bunch of companies that compile PHP with their own options but still use commercial modules, including both performance and non-performance ones. They couldn't use this compile switch. Yes, I know what binary compatibility means. -Rasmus -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP-DEV] Garbage collector patch
Unfortunately it also takes more memory in the common case (at least the apps that we tested). It's not a significant amount more but it's a tiny bit more. I'd say negligible. Btw, I think I miswrote in my email. When you look at the benchmark results on the new patch it's actually 1-2% slower and 0-3% more memory. I think even on a heavy duty server that is likely not to have much impact. I'm sure just the fluctuations between hardware architectures and compilers will often create bigger differences. To clarify I meant there's a tri-state (not compiled in, compiled in but collection turned off, compiled in but collection turned on). My recommendation was to always compile it in but to keep collection turned off by default. I am also OK with trying to keep collection turned on as long as we have an INI parameter to turn it off when things go wrong (it can be hidden, i.e. doesn't show up in phpinfo() nor php.ini). Unfortunately there are certain areas of PHP which took more than a few months to stabilize. Not sure this would be the case here but better safe than sorry and have way to turn it off. You can never know what weird memory patterns in extensions and SAPI modules will lead to crashes. Andi -Original Message- From: Guilherme Blanco [mailto:[EMAIL PROTECTED] Sent: Tuesday, December 04, 2007 11:59 AM To: [EMAIL PROTECTED] Cc: Rasmus Lerdorf; Antony Dovgal; Andi Gutmans; Cristian Rodriguez; internals@lists.php.net Subject: Re: [PHP-DEV] Garbage collector patch I want to put my 2 cents, but before make any comment, I'm interested to know if it's only a performance impact. Talking about performance is ok and I agree that any penalty should be taken into account, but if the patch uses less memory resources, I think that shared hostings will care about it. I remember when you released PHP 5_2, that was around 12% slower than any version of PHP 4. None care about it, and only in PHP 5_2 you took into account and optimized it a lot. As long as it uses less memory, it should be added into 5_3. Wang spent a lot of efforts to create the patch, Dmitry too. Also, I think you should take care of 5% of performance penalty is not huge if you consider that now you have a nice implementation of what was not working correctly (referring to circular references). But... if it uses more memory... I stay in the middle of the sides. Too much more? This is a factor. Besides all of this argumentation of performance... I'd go with Andi's first suggestion. No config for it... in or out. Make it configurable may generate some BC incompatibilities. Regards, On Dec 4, 2007 5:45 PM, [EMAIL PROTECTED] wrote: I think having it configurable is a must. Turning it on / off via a compile flag will not suit everyone. Eg - I have a situation where I would not want to run garbage collection for web pages served off my server due to the performance hit, however I also have a CLI script which I run to do complex, but repetitive tasks for ~half an hour. Now this is not really a big deal to me as I managed to rid most of the leaks by nulling out cyclic references, however that took a lot of work which could have been avoided by this. Could I suggest also enabling it by default for CLI? SCOTT MCNAUGHT Software Developer Synergy 8 / +617 3397 5212 [EMAIL PROTECTED] -Original Message- From: Rasmus Lerdorf [mailto:[EMAIL PROTECTED] Sent: Wednesday, 5 December 2007 5:17 AM To: Antony Dovgal Cc: Andi Gutmans; Cristian Rodriguez; internals@lists.php.net Subject: Re: [PHP-DEV] Garbage collector patch Antony Dovgal wrote: On 04.12.2007 18:31, Andi Gutmans wrote: You may be right longer term but shorter term I still believe there may be stability issues with this patch some of which we haven't figured out. Although we did testing and found crash bugs I wouldn't trust our level of testing to deem it stable. If we go down the route of always on we could have a hidden ini file (not listed in php.ini- dist and phpinfo()) which we can advise to turn off if something goes wrong and once we can enough confidence that there aren't any lurking bugs we could remove it. You mean provide an ini setting to be able to turn it Off? But why do you call it always On then? And what's the difference comparing to what you've proposed earlier? Concerning the stability issues, I'd say we have quite good chance to make it stable enough, as (I hope) 5.3.0 is not going to be released for at least several months more. Companies that are especially concerned of performance/stability are encouraged to step forward and give us a hand. Companies concerned about performance aren't going to use it at all. A 5% hit is significant given that it doesn't fix anything that a company already concerned about performance/stability cares about. Super leaky or recursively allocating scripts have
Re: [PHP-DEV] Garbage collector patch
On Dec 3, 2007 4:01 PM, Andi Gutmans [EMAIL PROTECTED] wrote: Hi all, Was hoping to send this off earlier but I was travelling for the past week and had very limited email access. As promised in the past few weeks we have spent a significant amount of time in reviewing the garbage collector work and testing it in our performance lab. Dmitry has been exchanging ideas and patches with David Wang during this time. Suffice to say that as I've mentioned in the past, memory management is an extremely sensitive piece of PHP, which is why it was important for us to review this work in great depth before committing it to the code base. The updated patch still retains the same algorithm as David's original implementation however the data structures have been changed in order to work faster and use less memory. The revised patch has the following advantages: - It fixes several crash bugs - Enhances performance by removing several unnecessary checks - The memory overhead was reduced (from 12 to 4 bytes for each heap-allocated zval) - The speed of clean PHP code (code that doesn't create cycles) was improved - Additional test cases were created The end result is a more stable and faster GC patch. That said we have yet to find real-life applications that create significant cycles which would benefit from this patch. In fact as you'll see from the results our tests show an increase in maximum memory use and slower execution (to be fair they are marginal). We have tested both PHP_5_3 without any patches, the original patch and the new patch. The following table shows execution time (seconds for N requests) and slowdown. PHP_5_3 Original GC patch Current GC patch slowdown slowdown bench 11,550 12,310 6,58% 12,170 5,37% hello 8,781 8,852 0,81% 8,813 0,36% xoops 128,500 135,100 5,14% 130,200 1,32% static 18,540 20,840 12,41% 18,920 2,05% qdig 29,320 30,270 3,24% 29,610 0,99% qdig2 13,960 14,100 1,00% 14,090 0,93% The next table shows memory usage in MB and overhead PHP_5_3 Original GC patch Current GC patch overhead overhead hello 13,750 13,945 1,42% 13,765 0,11% xoops 18,036 18,636 3,33% 18,568 2,95% static 15,300 16,000 4,58% 15,308 0,05% qdig 14,820 15,008 1,27% 14,828 0,05% qdig2 14,824 15,012 1,27% 14,838 0,09% To summarize the patch lead to approx. 5% slowdown and 3% memory overhead for typical applications (as always, you mileage may vary depending on your system's architecture and OS although my guesstimate is that you will see even worse results in a 64bit environment). We also tested SugarCRM to get another sanity for these results and we got similar results. I am not quite sure where this leaves us with this patch. On one hand I think now the effort has been made it's a good thing to offer it as part of PHP. The downside is of course some performance degradation and possible instabilities as this patch continues to stabilize (it took about two releases for us to stabilize the Zend Memory Manager even after in-depth testing due to edge cases in allocation patterns and various extensions, etc...).I'd be surprised if our team has found all possible bugs. Personally I think the decision should be either in or out. Adding this as a compile option is not a good idea as it would create binary compatibility issues and would be a pain for the community. I think my inclination is to commit the patch, not have it #ifdef'ed but always compiled but to have the garbage collection itself turned off by default (mainly for stability reasons. Note: the increased memory footprint and performance loss also exists with the collection itself turned off). We can turn it on when we're in dev for snapshots so that we iron out bugs. That said, as you can see from the results most people and real-life applications will be worse off than today. Thanks to David Dmitry for working hard on this (and everyone else who contributed). The stage is open for ideas/thoughts/suggestions J Andi Andi, I don't know about how it worked for anyone else, but the tables didn't display properly on Gmail, so I had a hard time keeping up with the performance differences. If you have this in a separate file, could you send it to me as an attachment? I have a few real-world benchmarks I'd like to try from the angle of the shared-webhost industry (lots of users with horrible code running simultaneously, et cetera) which I'd like to compare to your notes. Thanks. -- Daniel P. Brown [office] (570-) 587-7080 Ext. 272 [mobile] (570-) 766-8107 If at first you don't succeed, stick to what you know best so that you can make enough money to pay someone else to do it for you. -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit:
Re: [PHP-DEV] Garbage collector patch
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, Daniel Brown wrote: I don't know about how it worked for anyone else, but the tables didn't display properly on Gmail, so I had a hard time keeping up with the performance differences. If you have this in a separate file, could you send it to me as an attachment? Same problem, all in one row, to table to read. We're having non-apache application which run up to one hour to do their task and I think our QA can test the new GC there (and memory consumption is an issue there definitely). thanks -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHVHWh1nS0RcInK9ARAmPlAJ9XnOzFmLSl8qDxY5bLBfJBcmgqRACfZnsn R3cFSHfkMpoffq+f5vMxI3g= =OkMW -END PGP SIGNATURE- -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP-DEV] Garbage collector patch
That'd be great. Dmitry, David, can you please send the updated patch to the list? Andi -Original Message- From: Markus Fischer [mailto:[EMAIL PROTECTED] Sent: Monday, December 03, 2007 1:31 PM To: Daniel Brown Cc: Andi Gutmans; internals@lists.php.net Subject: Re: [PHP-DEV] Garbage collector patch -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, Daniel Brown wrote: I don't know about how it worked for anyone else, but the tables didn't display properly on Gmail, so I had a hard time keeping up with the performance differences. If you have this in a separate file, could you send it to me as an attachment? Same problem, all in one row, to table to read. We're having non-apache application which run up to one hour to do their task and I think our QA can test the new GC there (and memory consumption is an issue there definitely). thanks -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHVHWh1nS0RcInK9ARAmPlAJ9XnOzFmLSl8qDxY5bLBfJBcmgqRACfZnsn R3cFSHfkMpoffq+f5vMxI3g= =OkMW -END PGP SIGNATURE- -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
That looks great, Andi. Thanks. On Dec 3, 2007 4:38 PM, Andi Gutmans [EMAIL PROTECTED] wrote: Sorry about that. Does the attached PDFed screenshot work for you? Andi -Original Message- From: Daniel Brown [mailto:[EMAIL PROTECTED] Sent: Monday, December 03, 2007 1:21 PM To: Andi Gutmans Cc: internals@lists.php.net Subject: Re: [PHP-DEV] Garbage collector patch On Dec 3, 2007 4:01 PM, Andi Gutmans [EMAIL PROTECTED] wrote: Hi all, Was hoping to send this off earlier but I was travelling for the past week and had very limited email access. As promised in the past few weeks we have spent a significant amount of time in reviewing the garbage collector work and testing it in our performance lab. Dmitry has been exchanging ideas and patches with David Wang during this time. Suffice to say that as I've mentioned in the past, memory management is an extremely sensitive piece of PHP, which is why it was important for us to review this work in great depth before committing it to the code base. The updated patch still retains the same algorithm as David's original implementation however the data structures have been changed in order to work faster and use less memory. The revised patch has the following advantages: - It fixes several crash bugs - Enhances performance by removing several unnecessary checks - The memory overhead was reduced (from 12 to 4 bytes for each heap-allocated zval) - The speed of clean PHP code (code that doesn't create cycles) was improved - Additional test cases were created The end result is a more stable and faster GC patch. That said we have yet to find real-life applications that create significant cycles which would benefit from this patch. In fact as you'll see from the results our tests show an increase in maximum memory use and slower execution (to be fair they are marginal). We have tested both PHP_5_3 without any patches, the original patch and the new patch. The following table shows execution time (seconds for N requests) and slowdown. PHP_5_3 Original GC patch Current GC patch slowdown slowdown bench 11,550 12,310 6,58% 12,170 5,37% hello 8,781 8,852 0,81% 8,813 0,36% xoops 128,500 135,100 5,14% 130,200 1,32% static 18,540 20,840 12,41% 18,920 2,05% qdig 29,320 30,270 3,24% 29,610 0,99% qdig2 13,960 14,100 1,00% 14,090 0,93% The next table shows memory usage in MB and overhead PHP_5_3 Original GC patch Current GC patch overhead overhead hello 13,750 13,945 1,42% 13,765 0,11% xoops 18,036 18,636 3,33% 18,568 2,95% static 15,300 16,000 4,58% 15,308 0,05% qdig 14,820 15,008 1,27% 14,828 0,05% qdig2 14,824 15,012 1,27% 14,838 0,09% To summarize the patch lead to approx. 5% slowdown and 3% memory overhead for typical applications (as always, you mileage may vary depending on your system's architecture and OS although my guesstimate is that you will see even worse results in a 64bit environment). We also tested SugarCRM to get another sanity for these results and we got similar results. I am not quite sure where this leaves us with this patch. On one hand I think now the effort has been made it's a good thing to offer it as part of PHP. The downside is of course some performance degradation and possible instabilities as this patch continues to stabilize (it took about two releases for us to stabilize the Zend Memory Manager even after in-depth testing due to edge cases in allocation patterns and various extensions, etc...).I'd be surprised if our team has found all possible bugs. Personally I think the decision should be either in or out. Adding this as a compile option is not a good idea as it would create binary compatibility issues and would be a pain for the community. I think my inclination is to commit the patch, not have it #ifdef'ed but always compiled but to have the garbage collection itself turned off by default (mainly for stability reasons. Note: the increased memory footprint and performance loss also exists with the collection itself turned off). We can turn it on when we're in dev for snapshots so that we iron out bugs. That said, as you can see from the results most people and real-life applications will be worse off than today
Re: [PHP-DEV] Garbage collector patch
Markus, If for some reason the PDF attachment didn't come through to you on the list, let me know and I'll upload it to one of my servers for you to download and use, as well. On Dec 3, 2007 4:40 PM, Daniel Brown [EMAIL PROTECTED] wrote: That looks great, Andi. Thanks. On Dec 3, 2007 4:38 PM, Andi Gutmans [EMAIL PROTECTED] wrote: Sorry about that. Does the attached PDFed screenshot work for you? Andi -Original Message- From: Daniel Brown [mailto:[EMAIL PROTECTED] Sent: Monday, December 03, 2007 1:21 PM To: Andi Gutmans Cc: internals@lists.php.net Subject: Re: [PHP-DEV] Garbage collector patch On Dec 3, 2007 4:01 PM, Andi Gutmans [EMAIL PROTECTED] wrote: Hi all, Was hoping to send this off earlier but I was travelling for the past week and had very limited email access. As promised in the past few weeks we have spent a significant amount of time in reviewing the garbage collector work and testing it in our performance lab. Dmitry has been exchanging ideas and patches with David Wang during this time. Suffice to say that as I've mentioned in the past, memory management is an extremely sensitive piece of PHP, which is why it was important for us to review this work in great depth before committing it to the code base. The updated patch still retains the same algorithm as David's original implementation however the data structures have been changed in order to work faster and use less memory. The revised patch has the following advantages: - It fixes several crash bugs - Enhances performance by removing several unnecessary checks - The memory overhead was reduced (from 12 to 4 bytes for each heap-allocated zval) - The speed of clean PHP code (code that doesn't create cycles) was improved - Additional test cases were created The end result is a more stable and faster GC patch. That said we have yet to find real-life applications that create significant cycles which would benefit from this patch. In fact as you'll see from the results our tests show an increase in maximum memory use and slower execution (to be fair they are marginal). We have tested both PHP_5_3 without any patches, the original patch and the new patch. The following table shows execution time (seconds for N requests) and slowdown. PHP_5_3 Original GC patch Current GC patch slowdown slowdown bench 11,550 12,310 6,58% 12,170 5,37% hello 8,781 8,852 0,81% 8,813 0,36% xoops 128,500 135,100 5,14% 130,200 1,32% static 18,540 20,840 12,41% 18,920 2,05% qdig 29,320 30,270 3,24% 29,610 0,99% qdig2 13,960 14,100 1,00% 14,090 0,93% The next table shows memory usage in MB and overhead PHP_5_3 Original GC patch Current GC patch overhead overhead hello 13,750 13,945 1,42% 13,765 0,11% xoops 18,036 18,636 3,33% 18,568 2,95% static 15,300 16,000 4,58% 15,308 0,05% qdig 14,820 15,008 1,27% 14,828 0,05% qdig2 14,824 15,012 1,27% 14,838 0,09% To summarize the patch lead to approx. 5% slowdown and 3% memory overhead for typical applications (as always, you mileage may vary depending on your system's architecture and OS although my guesstimate is that you will see even worse results in a 64bit environment). We also tested SugarCRM to get another sanity for these results and we got similar results. I am not quite sure where this leaves us with this patch. On one hand I think now the effort has been made it's a good thing to offer it as part of PHP. The downside is of course some performance degradation and possible instabilities as this patch continues to stabilize (it took about two releases for us to stabilize the Zend Memory Manager even after in-depth testing due to edge cases in allocation patterns and various extensions, etc...).I'd be surprised if our team has found all possible bugs. Personally I think the decision should be either in or out. Adding this as a compile option is not a good idea as it would create binary compatibility issues and would be a pain
Re: [PHP-DEV] Garbage collector patch
Daniel Brown wrote: If for some reason the PDF attachment didn't come through to you on the list, let me know and I'll upload it to one of my servers for you to download and use, as well. The PDF didn't make it through for me. Can you upload it? -- Edward Z. YangGnuPG: 0x869C48DA HTML Purifier http://htmlpurifier.org Anti-XSS Filter [[ 3FA8 E9A9 7385 B691 A6FC B3CB A933 BE7D 869C 48DA ]] -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, yes exactly, there was no PDF attachment. Interestingly the signature was a separate attachment ... thanks - - Markus Daniel Brown wrote: Markus, If for some reason the PDF attachment didn't come through to you on the list, let me know and I'll upload it to one of my servers for you to download and use, as well. On Dec 3, 2007 4:40 PM, Daniel Brown [EMAIL PROTECTED] wrote: That looks great, Andi. Thanks. On Dec 3, 2007 4:38 PM, Andi Gutmans [EMAIL PROTECTED] wrote: Sorry about that. Does the attached PDFed screenshot work for you? Andi -Original Message- From: Daniel Brown [mailto:[EMAIL PROTECTED] Sent: Monday, December 03, 2007 1:21 PM To: Andi Gutmans Cc: internals@lists.php.net Subject: Re: [PHP-DEV] Garbage collector patch On Dec 3, 2007 4:01 PM, Andi Gutmans [EMAIL PROTECTED] wrote: Hi all, Was hoping to send this off earlier but I was travelling for the past week and had very limited email access. As promised in the past few weeks we have spent a significant amount of time in reviewing the garbage collector work and testing it in our performance lab. Dmitry has been exchanging ideas and patches with David Wang during this time. Suffice to say that as I've mentioned in the past, memory management is an extremely sensitive piece of PHP, which is why it was important for us to review this work in great depth before committing it to the code base. The updated patch still retains the same algorithm as David's original implementation however the data structures have been changed in order to work faster and use less memory. The revised patch has the following advantages: - It fixes several crash bugs - Enhances performance by removing several unnecessary checks - The memory overhead was reduced (from 12 to 4 bytes for each heap-allocated zval) - The speed of clean PHP code (code that doesn't create cycles) was improved - Additional test cases were created The end result is a more stable and faster GC patch. That said we have yet to find real-life applications that create significant cycles which would benefit from this patch. In fact as you'll see from the results our tests show an increase in maximum memory use and slower execution (to be fair they are marginal). We have tested both PHP_5_3 without any patches, the original patch and the new patch. The following table shows execution time (seconds for N requests) and slowdown. PHP_5_3 Original GC patch Current GC patch slowdown slowdown bench 11,550 12,310 6,58% 12,170 5,37% hello 8,781 8,852 0,81% 8,813 0,36% xoops 128,500 135,100 5,14% 130,200 1,32% static 18,540 20,840 12,41% 18,920 2,05% qdig 29,320 30,270 3,24% 29,610 0,99% qdig2 13,960 14,100 1,00% 14,090 0,93% The next table shows memory usage in MB and overhead PHP_5_3 Original GC patch Current GC patch overhead overhead hello 13,750 13,945 1,42% 13,765 0,11% xoops 18,036 18,636 3,33% 18,568 2,95% static 15,300 16,000 4,58% 15,308 0,05% qdig 14,820 15,008 1,27% 14,828 0,05% qdig2 14,824 15,012 1,27% 14,838 0,09% To summarize the patch lead to approx. 5% slowdown and 3% memory overhead for typical applications (as always, you mileage may vary depending on your system's architecture and OS although my guesstimate is that you will see even worse results in a 64bit environment). We also tested SugarCRM to get another sanity for these results and we got similar results. I am not quite sure where this leaves us with this patch. On one hand I think now the effort has been made it's a good thing to offer it as part of PHP. The downside is of course some performance degradation and possible instabilities as this patch continues to stabilize (it took about two releases for us to stabilize the Zend Memory Manager even after in-depth testing due to edge cases in allocation patterns and various extensions, etc...).I'd be surprised if our team has found all possible bugs. Personally I think the decision should be either in or out. Adding this as a compile option is not a good idea as it would create binary compatibility issues and would be a pain for the community. I think my inclination is to commit the patch, not have it #ifdef'ed but always compiled but to have the garbage collection itself turned off by default (mainly for stability reasons. Note: the increased memory footprint and performance loss also exists with the collection itself turned off). We can turn it on when we're in dev for snapshots so that we iron out bugs. That said, as you can see from the results most people and real-life applications will be worse off than today. Thanks to David Dmitry for working hard on this (and everyone else who
RE: [PHP-DEV] Garbage collector patch
Argh. Here you go: http://cvs.php.net/~andi/GC_email.pdf -Original Message- From: Edward Z. Yang [mailto:[EMAIL PROTECTED] Sent: Monday, December 03, 2007 1:43 PM To: internals@lists.php.net Subject: Re: [PHP-DEV] Garbage collector patch Daniel Brown wrote: If for some reason the PDF attachment didn't come through to you on the list, let me know and I'll upload it to one of my servers for you to download and use, as well. The PDF didn't make it through for me. Can you upload it? -- Edward Z. YangGnuPG: 0x869C48DA HTML Purifier http://htmlpurifier.org Anti-XSS Filter [[ 3FA8 E9A9 7385 B691 A6FC B3CB A933 BE7D 869C 48DA ]] -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
Sorry about that. Does the attached PDFed screenshot work for you? If only we knew how to publish documents on that 'web thing. (-: S -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP-DEV] Garbage collector patch
On Mon, 3 Dec 2007, Andi Gutmans wrote: Sorry about that. Does the attached PDFed screenshot work for you? No, as you can't attach files here Derick -- Derick Rethans http://derickrethans.nl | http://ez.no | http://xdebug.org -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
On Dec 3, 2007 4:49 PM, Derick Rethans [EMAIL PROTECTED] wrote: On Mon, 3 Dec 2007, Andi Gutmans wrote: Sorry about that. Does the attached PDFed screenshot work for you? No, as you can't attach files here Derick -- Derick Rethans http://derickrethans.nl | http://ez.no | http://xdebug.org I didn't think it would work on the list, but I figured if Andi either sent it to me, or clicked Reply-All, either way it would be delivered directly to my inbox, which it was. So now it's here: http://www.pilotpig.net/pdfs/ -- Daniel P. Brown [office] (570-) 587-7080 Ext. 272 [mobile] (570-) 766-8107 If at first you don't succeed, stick to what you know best so that you can make enough money to pay someone else to do it for you. -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP-DEV] Garbage collector patch
Sorry about that. Does the attached PDFed screenshot work for you? Andi -Original Message- From: Daniel Brown [mailto:[EMAIL PROTECTED] Sent: Monday, December 03, 2007 1:21 PM To: Andi Gutmans Cc: internals@lists.php.net Subject: Re: [PHP-DEV] Garbage collector patch On Dec 3, 2007 4:01 PM, Andi Gutmans [EMAIL PROTECTED] wrote: Hi all, Was hoping to send this off earlier but I was travelling for the past week and had very limited email access. As promised in the past few weeks we have spent a significant amount of time in reviewing the garbage collector work and testing it in our performance lab. Dmitry has been exchanging ideas and patches with David Wang during this time. Suffice to say that as I've mentioned in the past, memory management is an extremely sensitive piece of PHP, which is why it was important for us to review this work in great depth before committing it to the code base. The updated patch still retains the same algorithm as David's original implementation however the data structures have been changed in order to work faster and use less memory. The revised patch has the following advantages: - It fixes several crash bugs - Enhances performance by removing several unnecessary checks - The memory overhead was reduced (from 12 to 4 bytes for each heap-allocated zval) - The speed of clean PHP code (code that doesn't create cycles) was improved - Additional test cases were created The end result is a more stable and faster GC patch. That said we have yet to find real-life applications that create significant cycles which would benefit from this patch. In fact as you'll see from the results our tests show an increase in maximum memory use and slower execution (to be fair they are marginal). We have tested both PHP_5_3 without any patches, the original patch and the new patch. The following table shows execution time (seconds for N requests) and slowdown. PHP_5_3 Original GC patch Current GC patch slowdown slowdown bench 11,550 12,310 6,58% 12,170 5,37% hello 8,781 8,852 0,81% 8,813 0,36% xoops 128,500 135,100 5,14% 130,200 1,32% static 18,540 20,840 12,41% 18,920 2,05% qdig 29,320 30,270 3,24% 29,610 0,99% qdig2 13,960 14,100 1,00% 14,090 0,93% The next table shows memory usage in MB and overhead PHP_5_3 Original GC patch Current GC patch overhead overhead hello 13,750 13,945 1,42% 13,765 0,11% xoops 18,036 18,636 3,33% 18,568 2,95% static 15,300 16,000 4,58% 15,308 0,05% qdig 14,820 15,008 1,27% 14,828 0,05% qdig2 14,824 15,012 1,27% 14,838 0,09% To summarize the patch lead to approx. 5% slowdown and 3% memory overhead for typical applications (as always, you mileage may vary depending on your system's architecture and OS although my guesstimate is that you will see even worse results in a 64bit environment). We also tested SugarCRM to get another sanity for these results and we got similar results. I am not quite sure where this leaves us with this patch. On one hand I think now the effort has been made it's a good thing to offer it as part of PHP. The downside is of course some performance degradation and possible instabilities as this patch continues to stabilize (it took about two releases for us to stabilize the Zend Memory Manager even after in-depth testing due to edge cases in allocation patterns and various extensions, etc...).I'd be surprised if our team has found all possible bugs. Personally I think the decision should be either in or out. Adding this as a compile option is not a good idea as it would create binary compatibility issues and would be a pain for the community. I think my inclination is to commit the patch, not have it #ifdef'ed but always compiled but to have the garbage collection itself turned off by default (mainly for stability reasons. Note: the increased memory footprint and performance loss also exists with the collection itself turned off). We can turn it on when we're in dev for snapshots so that we iron out bugs. That said, as you can see from the results most people and real-life applications will be worse off than today. Thanks to David Dmitry for working hard on this (and everyone else who contributed). The stage is open for ideas/thoughts/suggestions J Andi Andi, I don't know about how it worked for anyone else, but the tables didn't display properly on Gmail, so I had a hard time keeping up with the performance differences. If you have this in a separate file
Re: [PHP-DEV] Garbage collector patch
2007/12/3, Ilia Alshanetsky [EMAIL PROTECTED]: I think the patch does have a value, yes, it does, what worries me is the introduction of yet another non-sense ini setting that modified the very engine behaviuor.. I think we all agree that there are way too many of those do we ? . My suggestion is that we make this feature a compile time flag. My suggestion is not to add any compile time flag, either provide it or dont. and people who feel that their applications warrant a garbage collector can enable it for their install and at the same time those who don't (general case) have no penalties of having it in place. People more commonly uses PHP from distributors, in binary form.. they dont usually decide what is enabled or not, so you have to either reccommend this feature or mark it clearly as untrusted, experimental. such switches only add more complexity, confusion for users and addtional trouble to distributors. my 2CLP. -- http://www.kissofjudas.net/ -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
First of all big thanks for Dmitry and David for spending time on this project and continuing to improve the original patch. Based on the results so far, I think the patch does have a value, but certainly not in a general case. Relative simple scripts have little to gain from it and only to loose 3-5% of speed depending on a situation and given insubstantial memory gains it seems of little use in general case. That said, there are complex applications (perhaps unnecessarily so ;-) ) that could see some real benefits from this code. My suggestion is that we make this feature a compile time flag, that's off by default and people who feel that their applications warrant a garbage collector can enable it for their install and at the same time those who don't (general case) have no penalties of having it in place. On 3-Dec-07, at 4:01 PM, Andi Gutmans wrote: Hi all, Was hoping to send this off earlier but I was travelling for the past week and had very limited email access. As promised in the past few weeks we have spent a significant amount of time in reviewing the garbage collector work and testing it in our performance lab. Dmitry has been exchanging ideas and patches with David Wang during this time. Suffice to say that as I've mentioned in the past, memory management is an extremely sensitive piece of PHP, which is why it was important for us to review this work in great depth before committing it to the code base. The updated patch still retains the same algorithm as David's original implementation however the data structures have been changed in order to work faster and use less memory. The revised patch has the following advantages: - It fixes several crash bugs - Enhances performance by removing several unnecessary checks - The memory overhead was reduced (from 12 to 4 bytes for each heap-allocated zval) - The speed of clean PHP code (code that doesn't create cycles) was improved - Additional test cases were created The end result is a more stable and faster GC patch. That said we have yet to find real-life applications that create significant cycles which would benefit from this patch. In fact as you'll see from the results our tests show an increase in maximum memory use and slower execution (to be fair they are marginal). We have tested both PHP_5_3 without any patches, the original patch and the new patch. The following table shows execution time (seconds for N requests) and slowdown. PHP_5_3 Original GC patch Current GC patch slowdown slowdown bench 11,550 12,310 6,58% 12,170 5,37% hello 8,781 8,852 0,81% 8,813 0,36% xoops 128,500 135,100 5,14% 130,200 1,32% static 18,540 20,840 12,41% 18,920 2,05% qdig 29,320 30,270 3,24% 29,610 0,99% qdig2 13,960 14,100 1,00% 14,090 0,93% The next table shows memory usage in MB and overhead PHP_5_3 Original GC patch Current GC patch overhead overhead hello 13,750 13,945 1,42% 13,765 0,11% xoops 18,036 18,636 3,33% 18,568 2,95% static 15,300 16,000 4,58% 15,308 0,05% qdig 14,820 15,008 1,27% 14,828 0,05% qdig2 14,824 15,012 1,27% 14,838 0,09% To summarize the patch lead to approx. 5% slowdown and 3% memory overhead for typical applications (as always, you mileage may vary depending on your system's architecture and OS although my guesstimate is that you will see even worse results in a 64bit environment). We also tested SugarCRM to get another sanity for these results and we got similar results. I am not quite sure where this leaves us with this patch. On one hand I think now the effort has been made it's a good thing to offer it as part of PHP. The downside is of course some performance degradation and possible instabilities as this patch continues to stabilize (it took about two releases for us to stabilize the Zend Memory Manager even after in-depth testing due to edge cases in allocation patterns and various extensions, etc...).I'd be surprised if our team has found all possible bugs. Personally I think the decision should be either in or out. Adding this as a compile option is not a good idea as it would create binary compatibility issues and would be a pain for the community. I think my inclination is to commit the patch, not have it #ifdef'ed but always compiled but to have the garbage collection itself turned off by default (mainly for stability reasons. Note: the increased memory footprint and performance loss also exists with the collection itself turned off). We can turn it on when we're in dev for snapshots so that we iron out bugs. That said, as you can see from the results most people and real-life applications will be worse off than today. Thanks to David Dmitry for working hard on this (and everyone else who contributed). The stage is open for ideas/thoughts/suggestions J Andi Ilia Alshanetsky -- PHP Internals - PHP
Re: [PHP-DEV] Garbage collector patch
such switches only add more complexity, confusion for users and addtional trouble to distributors. FWIW, amen to that. - Steph -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
On Dec 3, 2007, at 1:01 PM, Andi Gutmans wrote: Hi all, Was hoping to send this off earlier but I was travelling for the past week and had very limited email access. As promised in the past few weeks we have spent a significant amount of time in reviewing the garbage collector work and testing it in our performance lab. Dmitry has been exchanging ideas and patches with David Wang during this time. Suffice to say that as I've mentioned in the past, memory management is an extremely sensitive piece of PHP, which is why it was important for us to review this work in great depth before committing it to the code base. The updated patch still retains the same algorithm as David's original implementation however the data structures have been changed in order to work faster and use less memory. ... Personally I think the decision should be either in or out. Adding this as a compile option is not a good idea as it would create binary compatibility issues and would be a pain for the community. ... The stage is open for ideas/thoughts/suggestions I'm really hesitant to even *mention* this idea, but Could alternate memory management systems be made available via PECL add-ons, or, more to the point: What is the *actual cost and complexity* involved in implementing (possibly many) different user-selectable memory management systems, and what other future benefits/drawbacks might we see by doing such a thing? (GC is big now, but what about memory pools per mod_auth user, or SHM/SEM pools, or tuning amounts of memory per function, etc...) I will now apologize to everybody who I just made cry, scream, or damage their furniture, as I didn't mean to hurt you, just trying to stimulate ideas. -Ronabop -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP-DEV] Garbage collector patch
-Original Message- From: Ronald Chmara [mailto:[EMAIL PROTECTED] Sent: Monday, December 03, 2007 10:06 PM To: Andi Gutmans Cc: internals@lists.php.net Subject: Re: [PHP-DEV] Garbage collector patch I'm really hesitant to even *mention* this idea, but Could alternate memory management systems be made available via PECL add-ons, or, more to the point: What is the *actual cost and complexity* involved in implementing (possibly many) different user-selectable memory management systems, and what other future benefits/drawbacks might we see by doing such a thing? (GC is big now, but what about memory pools per mod_auth user, or SHM/SEM pools, or tuning amounts of memory per function, etc...) I will now apologize to everybody who I just made cry, scream, or damage their furniture, as I didn't mean to hurt you, just trying to stimulate ideas. Hi Ronald, PHP 5.2.x already supports the ability to hook in different page managers. In PHP 5.3 you can also override the memory allocation functions. However, this would not include garbage collection like algorithms which actually require changes in the core PHP data type such as zvals. In fact the garbage collection adds memory to the basic datatypes which is why I suggested to either always make these changes, or don't make them so that we retain binary compatibility across all builds of PHP. So overriding basic memory allocation functions, yes, ability to provide various GC implementations, no. Andi -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Garbage collector patch
On Dec 3, 2007, at 10:30 PM, Andi Gutmans wrote: -Original Message- From: Ronald Chmara [mailto:[EMAIL PROTECTED] What is the *actual cost and complexity* involved in implementing (possibly many) different user-selectable memory management systems, and what other future benefits/drawbacks might we see by doing such a thing? (GC is big now, but what about memory pools per mod_auth user, or SHM/SEM pools, or tuning amounts of memory per function, etc...) I will now apologize to everybody who I just made cry, scream, or damage their furniture, as I didn't mean to hurt you, just trying to stimulate ideas. Hi Ronald, PHP 5.2.x already supports the ability to hook in different page managers. In PHP 5.3 you can also override the memory allocation functions. However, this would not include garbage collection like algorithms which actually require changes in the core PHP data type such as zvals. In fact the garbage collection adds memory to the basic datatypes which is why I suggested to either always make these changes, or don't make them so that we retain binary compatibility across all builds of PHP. So overriding basic memory allocation functions, yes, ability to provide various GC implementations, no. Okay, so, without this patch, I can allocate mem, and destroy it, on a per page level, but not on a zval level, and the choice is: a) zval (etc.) destruction with this patch, which has a 5% speed hit at times (varies per test) b) no patch, no change c) something which hasn't been thought of yet? I'd have to (sadly) ask that anything that slows down PHP by 5%, to improve performance for programmers that, uhm, leak or otherwise gobble RAM, that they, uhm, refactor their code as professional programmers. Thanks for clearing it up for me, Andi. -Bop -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
