Re: [PHP] Re: Memory investigation
you could read this http://xdebug.org/docs/execution_trace and then parse trace files to get the memory usage create a global to store user functions, I think user function do not populate get_defined_functions(); until the function is called; $t = get_defined_functions(); $userFunctions = $t['user']; $file = xdebug_get_tracefile_name(); parse the file and then create a function to call the said variable. I think you can figure it out. la...@garfieldtech.com wrote: Currently it's mostly procedural with some components that are OO. I suspect most of the memory sinks are large arrays (we have a lot of them), but I am not certain of that. Hence my interest in more accurate investigation tools. --Larry Garfield dsiemba...@gmail.com wrote: couple questions Larry is this application composed of classes or straight up no holes barred procedural code? la...@garfieldtech.com wrote: That's not really what I'm after. Let me try an example: function foo($id) { static $foos = array(); if (empty($foos[$id]) { $foos[$id] = load_foo($id); } return $foos[$id]; } When load_foo() is slow (e.g., lots of DB traffic or remote-server calls or whatever), such caching can have a significant performance boost. Sometime after foo() has been called 15 times from 30 places in code, when I get to the end of the request (or just every time I call foo() would be fine) I want to be able to do something like: $cost = get_memory_used_by($foos); So that I can determine how much memory that caching is costing me over the lifetime of the page, and determine if it's a worthwhile trade-off. --Larry Garfield dsiemba...@gmail.com wrote: function check_memory_usage($memory) { $memory[] = memory_get_usage(); return $memory; } something like this? you can put it wherever you like and returns an array for further processing. You could optionally add a second argument to set the index to a name and check if the name exists to add 1 to the end of the name so your indexes stay maintained. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Re: Memory investigation
ON Linux I have kcacheGrind setup with Xdebug and I find it is a nice little thing to have. It won't tell you the memory consumed but it will find cycles and display object maps. if you have Kcachegrind it is likely you have valgrind installed. http://www2.mandriva.com/ http://valgrind.org/ http://kcachegrind.sourceforge.net/html/Home.html http://xdebug.org/ Larry Garfield wrote: Hi folks. I have a complicated PHP app that is eating up more memory than I think it should. I have a couple of theories as to where it could be going, but I need some way to verify it. There are a number of large data structures (mostly arrays) that get built up throughout the course of the request. What I'd like to be able to do is at certain points check to see how much memory those data structures are using. Because they're not all built at once, the usual check memory before, build, check after routine won't work. Plus, that gets screwed up by PHP's use of copy-on-write at times. I know that it would result in essentially over-reporting, but I would ideally like to be able to ignore the copy-on-write issue and say if this variable were the only thing in memory (and its dependents, of course, for a nested array), what would its memory usage be? I just have no idea how to do that. Anyone have a suggestion for how to accomplish that? --Larry Garfield -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: Memory investigation
Yep, I'm familiar with XDebug and KCacheGrind. As you say, though, it doens't (as far as I am aware) offer the particular data that I'm after. We've already got cachegrind gurus working on the code who know how to use it better than I do. :-) What I'm looking for is see this big cache object we've been building up in memory for this whole page request? Now that we're done with the page, how big is it, really? --Larry Garfield u...@domain.invalid wrote: ON Linux I have kcacheGrind setup with Xdebug and I find it is a nice little thing to have. It won't tell you the memory consumed but it will find cycles and display object maps. if you have Kcachegrind it is likely you have valgrind installed. http://www2.mandriva.com/ http://valgrind.org/ http://kcachegrind.sourceforge.net/html/Home.html http://xdebug.org/ Larry Garfield wrote: Hi folks. I have a complicated PHP app that is eating up more memory than I think it should. I have a couple of theories as to where it could be going, but I need some way to verify it. There are a number of large data structures (mostly arrays) that get built up throughout the course of the request. What I'd like to be able to do is at certain points check to see how much memory those data structures are using. Because they're not all built at once, the usual check memory before, build, check after routine won't work. Plus, that gets screwed up by PHP's use of copy-on-write at times. I know that it would result in essentially over-reporting, but I would ideally like to be able to ignore the copy-on-write issue and say if this variable were the only thing in memory (and its dependents, of course, for a nested array), what would its memory usage be? I just have no idea how to do that. Anyone have a suggestion for how to accomplish that? --Larry Garfield -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: Memory investigation
On 3 March 2010 15:49, la...@garfieldtech.com la...@garfieldtech.com wrote: Yep, I'm familiar with XDebug and KCacheGrind. As you say, though, it doens't (as far as I am aware) offer the particular data that I'm after. We've already got cachegrind gurus working on the code who know how to use it better than I do. :-) What I'm looking for is see this big cache object we've been building up in memory for this whole page request? Now that we're done with the page, how big is it, really? You could try Memtrack (http://pecl.php.net/package/memtrack) but I can't comment as to what state it's in. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: Memory investigation
function check_memory_usage($memory) { $memory[] = memory_get_usage(); return $memory; } something like this? you can put it wherever you like and returns an array for further processing. You could optionally add a second argument to set the index to a name and check if the name exists to add 1 to the end of the name so your indexes stay maintained. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: Memory investigation
That's not really what I'm after. Let me try an example: function foo($id) { static $foos = array(); if (empty($foos[$id]) { $foos[$id] = load_foo($id); } return $foos[$id]; } When load_foo() is slow (e.g., lots of DB traffic or remote-server calls or whatever), such caching can have a significant performance boost. Sometime after foo() has been called 15 times from 30 places in code, when I get to the end of the request (or just every time I call foo() would be fine) I want to be able to do something like: $cost = get_memory_used_by($foos); So that I can determine how much memory that caching is costing me over the lifetime of the page, and determine if it's a worthwhile trade-off. --Larry Garfield dsiemba...@gmail.com wrote: function check_memory_usage($memory) { $memory[] = memory_get_usage(); return $memory; } something like this? you can put it wherever you like and returns an array for further processing. You could optionally add a second argument to set the index to a name and check if the name exists to add 1 to the end of the name so your indexes stay maintained. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: Memory investigation
global $fooSize = 0; function foo($id) { global $fooSize; if (empty($foos($id)) { $b = get_memory_usage(true); $foos[$id] = load_foo($id); $fooSize+= $b - get_memory_usage(true); } ... } On Wed, Mar 3, 2010 at 8:16 PM, la...@garfieldtech.com la...@garfieldtech.com wrote: That's not really what I'm after. Let me try an example: function foo($id) { static $foos = array(); if (empty($foos[$id]) { $foos[$id] = load_foo($id); } return $foos[$id]; } When load_foo() is slow (e.g., lots of DB traffic or remote-server calls or whatever), such caching can have a significant performance boost. Sometime after foo() has been called 15 times from 30 places in code, when I get to the end of the request (or just every time I call foo() would be fine) I want to be able to do something like: $cost = get_memory_used_by($foos); So that I can determine how much memory that caching is costing me over the lifetime of the page, and determine if it's a worthwhile trade-off. --Larry Garfield dsiemba...@gmail.com wrote: function check_memory_usage($memory) { $memory[] = memory_get_usage(); return $memory; } something like this? you can put it wherever you like and returns an array for further processing. You could optionally add a second argument to set the index to a name and check if the name exists to add 1 to the end of the name so your indexes stay maintained. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: Memory investigation
couple questions Larry is this application composed of classes or straight up no holes barred procedural code? la...@garfieldtech.com wrote: That's not really what I'm after. Let me try an example: function foo($id) { static $foos = array(); if (empty($foos[$id]) { $foos[$id] = load_foo($id); } return $foos[$id]; } When load_foo() is slow (e.g., lots of DB traffic or remote-server calls or whatever), such caching can have a significant performance boost. Sometime after foo() has been called 15 times from 30 places in code, when I get to the end of the request (or just every time I call foo() would be fine) I want to be able to do something like: $cost = get_memory_used_by($foos); So that I can determine how much memory that caching is costing me over the lifetime of the page, and determine if it's a worthwhile trade-off. --Larry Garfield dsiemba...@gmail.com wrote: function check_memory_usage($memory) { $memory[] = memory_get_usage(); return $memory; } something like this? you can put it wherever you like and returns an array for further processing. You could optionally add a second argument to set the index to a name and check if the name exists to add 1 to the end of the name so your indexes stay maintained. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Re: Memory investigation
Currently it's mostly procedural with some components that are OO. I suspect most of the memory sinks are large arrays (we have a lot of them), but I am not certain of that. Hence my interest in more accurate investigation tools. --Larry Garfield dsiemba...@gmail.com wrote: couple questions Larry is this application composed of classes or straight up no holes barred procedural code? la...@garfieldtech.com wrote: That's not really what I'm after. Let me try an example: function foo($id) { static $foos = array(); if (empty($foos[$id]) { $foos[$id] = load_foo($id); } return $foos[$id]; } When load_foo() is slow (e.g., lots of DB traffic or remote-server calls or whatever), such caching can have a significant performance boost. Sometime after foo() has been called 15 times from 30 places in code, when I get to the end of the request (or just every time I call foo() would be fine) I want to be able to do something like: $cost = get_memory_used_by($foos); So that I can determine how much memory that caching is costing me over the lifetime of the page, and determine if it's a worthwhile trade-off. --Larry Garfield dsiemba...@gmail.com wrote: function check_memory_usage($memory) { $memory[] = memory_get_usage(); return $memory; } something like this? you can put it wherever you like and returns an array for further processing. You could optionally add a second argument to set the index to a name and check if the name exists to add 1 to the end of the name so your indexes stay maintained. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php