First, this is not an internals question. But, I would be not many
people on the general list could help you.
I wrote some daemon scripts for a web crawler in PHP. Now I spent one
day to work around the growing memory consumtion of these scripts.
I have PHP daemons running for weeks that don't use up a lot of memory.
You have to be careful what you do in a PHP script. I would guess you
are running Linux. Assuming you are, the linux memory system never
frees allocated memory back to the OS. Not until the process dies at
least. So, if any part of your code ever needs a lot of memory, that
will be allocated to the process until it ends.
- Was it a mistake to use PHP for such scripts? What language should I've been
choosing instead?
Any language has its caveats. And you can waste (leak is a strong word)
memory with any language.
- My suspicion is, that either pdo_mysql or dom are not freeing their
used memory during a request. Is that possible?
Are you telling pdo to free its results? If not, that is bad
programming. You have to unset vars and free db result sets yourself to
ensure they are not building up. PHP uses a lazy garbage collector that
is optimized for short lived web scripts. You have to overcome that
when working with PHP in a non-web environment.
- I'm using http://libslack.org/daemon now to control the script
execution. This gave me the idea for a special kind of PHP binary
"php-daemon":
- php-daemon is an executable that restarts a given php script in a
loop
- php-daemon can be combined with apc/xcache to store the bytecode
Unfortunately I'm still to much a newbie to write this myself.
My standard way to handle scripts that need a lot of memory is to use
the pcntl functions to fork children that run and do the work. They can
end after a certain time or memory usage. Again, you have to make sure
the main, parent script is well written and does not waste file
descriptors, connections, etc.
Hmm, maybe I will write a blog post about this.
--
Brian Moon
Senior Web Engineer
------------------------------
When you care enough to spend the very least.
http://dealnews.com/
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
