php-general Digest 4 Sep 2013 09:21:22 -0000 Issue 8355
php-general Digest 4 Sep 2013 09:21:22 - Issue 8355
Topics (messages 322028 through 322033):
Re: refernces, arrays, and why does it take up so much memory? [SOLVED]
322028 by: Daevid Vincent
322029 by: Stuart Dallas
322030 by: Daevid Vincent
322031 by: Daevid Vincent
322033 by: Stuart Dallas
Re: PHP-5.5.2 +opcache segfaults with Piwik
322032 by: Daniel
Administrivia:
To subscribe to the digest, e-mail:
php-general-digest-subscr...@lists.php.net
To unsubscribe from the digest, e-mail:
php-general-digest-unsubscr...@lists.php.net
To post to the list, e-mail:
php-gene...@lists.php.net
--
---BeginMessage---
EUREKA!
-Original Message-
From: Stuart Dallas [mailto:stu...@3ft9.com]
Sent: Tuesday, September 03, 2013 6:31 AM
To: Daevid Vincent
Cc: php-gene...@lists.php.net
Subject: Re: [PHP] refernces, arrays, and why does it take up so much
memory?
On 3 Sep 2013, at 02:30, Daevid Vincent dae...@daevid.com wrote:
I'm confused on how a reference works I think.
I have a DB result set in an array I'm looping over. All I simply want
to
do
is make the array key the id of the result set row.
This is the basic gist of it:
private function _normalize_result_set()
{
foreach($this-tmp_results as $k = $v)
{
$id = $v['id'];
$new_tmp_results[$id] = $v; //2013-08-29 [dv] using
a
reference here cuts the memory usage in half!
You are assigning a reference to $v. In the next iteration of the loop, $v
will be pointing at the next item in the array, as will the reference
you're
storing here. With this code I'd expect $new_tmp_results to be an array
where the keys (i.e. the IDs) are correct, but the data in each item
matches
the data in the last item from the original array, which appears to be
what
you describe.
unset($this-tmp_results[$k]);
Doing this for every loop is likely very inefficient. I don't know how the
inner workings of PHP process something like this, but I wouldn't be
surprised if it's allocating a new chunk of memory for a version of the
array without this element. You may find it better to not unset anything
until the loop has finished, at which point you can just unset($this-
tmp_results).
/*
if ($i++ % 1000 == 0)
{
gc_enable(); // Enable Garbage Collector
var_dump(gc_enabled()); // true
var_dump(gc_collect_cycles()); // # of
elements
cleaned up
gc_disable(); // Disable Garbage Collector
}
*/
}
$this-tmp_results = $new_tmp_results;
//var_dump($this-tmp_results); exit;
unset($new_tmp_results);
}
Try this:
private function _normalize_result_set()
{
// Initialise the temporary variable.
$new_tmp_results = array();
// Loop around just the keys in the array.
foreach (array_keys($this-tmp_results) as $k)
{
// Store the item in the temporary array with the ID as the key.
// Note no pointless variable for the ID, and no use of !
$new_tmp_results[$this-tmp_results[$k]['id']] =
$this-tmp_results[$k];
}
// Assign the temporary variable to the original variable.
$this-tmp_results = $new_tmp_results;
}
I'd appreciate it if you could plug this in and see what your memory usage
reports say. In most cases, trying to control the garbage collection
through
the use of references is the worst way to go about optimising your code.
In
my code above I'm relying on PHPs copy-on-write feature where data is only
duplicated when assigned if it changes. No unsets, just using scope to
mark
a variable as able to be cleaned up.
Where is this result set coming from? You'd save yourself a lot of
memory/time by putting the data in to this format when you read it from
the
source. For example, if reading it from MySQL, $this-
tmp_results[$row['id']] = $row when looping around the result set.
Also, is there any reason why you need to process this full set of data in
one go? Can you not break it up in to smaller pieces that won't put as
much
strain on resources?
-Stuart
There were reasons I had the $id -- I only showed the relevant parts of the
code for sake of not overly complicating what I was trying to illustrate.
There is other processing that had to be done too in the loop and that is
also what I illustrated.
Here is your version effectively:
private function _normalize_result_set() //Stuart
{
if (!$this-tmp_results || count($this-tmp_results) 1)
return;
$new_tmp_results = array();
// Loop
php-general Digest 4 Sep 2013 22:25:25 -0000 Issue 8356
php-general Digest 4 Sep 2013 22:25:25 - Issue 8356 Topics (messages 322034 through 322038): Static utility class? 322034 by: Micky Hulse 322035 by: Stephen 322036 by: David Harkness 322037 by: Micky Hulse message to user after complete POST 322038 by: iccsi Administrivia: To subscribe to the digest, e-mail: php-general-digest-subscr...@lists.php.net To unsubscribe from the digest, e-mail: php-general-digest-unsubscr...@lists.php.net To post to the list, e-mail: php-gene...@lists.php.net -- ---BeginMessage--- Hi all! Example code: https://gist.github.com/mhulse/6441525 Goal: I want to have a utility class that contain utility methods which should have the option of being called multiple times on a page. I think my main goal is to avoid having to new things ... I don't really need to create an instance here (there's no need for a constructor in this case). Question: Is the above simple pattern a good way to have a class that calls static methods? To put it another way, is there any reason why I would not want to use the above code? Basically I want to wrap these simple functions in a nice class wrapper and have the ability to call them without having to jump through more hoops than is necessary. Are there any pitfalls to writing a class like this? Or, is this the standard way of writing a simple utility class for this type of situation? Please let me know if I need to clarify my questions. Thanks for your time? ---End Message--- ---BeginMessage--- On 13-09-04 03:25 PM, Micky Hulse wrote: I want to have a utility class that contain utility methods which should have the option of being called multiple times on a page. This sounds simply like a library of functions that are implemented using objects. You can use the standard require_once in your various PHP source files so you only deal with loading the library file in the first file in which it is needed. It would not be loaded from other files. Instantiate your static variables in the library file. Again, so that only happens once. -- Stephen ---End Message--- ---BeginMessage--- On Wed, Sep 4, 2013 at 12:25 PM, Micky Hulse mickyhulse.li...@gmail.comwrote: I want to have a utility class that contain utility methods which should have the option of being called multiple times on a page. ... To put it another way, is there any reason why I would not want to use the above code? The main problem caused by static methods is testability. Mocking or stubbing a static method requires using a PHP extension and ensuring that the original is reset whether the test passes or fails. As long as your utility methods don't perform actions you want to avoid during tests, this is fine. Good examples for a utility class are string-processing functions such as parsing and formatting, trimming and/or normalizing empty string to null, etc. You want tests to work against these methods directly since there's no need to avoid the work. You'll want to avoid static methods whenever you want to be able to fix the behavior (stubbing) to test scenarios in the class under test. An example is anything hitting a database or external service. In order to test that your downloader sends the correct warning email to the sysadmin when a REST call fails, you need to be able to force the REST call to fail. This is easy to do when you plug in an instance implementing the API because you can give it an implementation/mock that fails for all calls. I can't say if what you're thinking of will make a good utility class since the code you posted is fake. If you post some examples of methods you want to make static, we can give you pointers on which are good candidates and which are best left to instance methods. Peace, David ---End Message--- ---BeginMessage--- Thank you so much for the quick and very informative/educational replies Stephen and David, I really appreciate it! :) On Wed, Sep 4, 2013 at 12:36 PM, Stephen stephe...@rogers.com wrote: This sounds simply like a library of functions that are implemented using objects. Instantiate your static variables in the library file. Again, so that only happens once. functions implemented using objects sounds like exactly what I want. Just out of curiosity, and sorry in advance for my ignorance, but does the code I posted fit that type of pattern? If not, what would I need to modify and how would I call it? Actually, I should probably spend some time Googling before I ask you for more details. :D On Wed, Sep 4, 2013 at 1:15 PM, David Harkness davi...@highgearmedia.com wrote: I can't say if what you're thinking of will make a good utility class since the code you posted is fake. If you post some examples of methods you want to make static, we can give you pointers on which are good candidates and which are best left to instance methods. Sorry about the fake
Re: [PHP] refernces, arrays, and why does it take up so much memory? [SOLVED]
On 4 Sep 2013, at 00:03, Daevid Vincent dae...@daevid.com wrote: -Original Message- From: Stuart Dallas [mailto:stu...@3ft9.com] Sent: Tuesday, September 03, 2013 2:37 PM To: Daevid Vincent Cc: php-general@lists.php.net; 'Jim Giner' Subject: Re: [PHP] refernces, arrays, and why does it take up so much memory? [SOLVED] On 3 Sep 2013, at 21:47, Daevid Vincent dae...@daevid.com wrote: We get over 30,000 hits per second, and even with lots of caching, 216MB vs 70-96MB is significant and the speed hit is only about 1.5 seconds more per page. Here are three distinctly different example pages that exercise different parts of the code path: PAGE RENDERED IN 7.0466279983521 SECONDS MEMORY USED @START: 262,144 - @END: 26,738,688 = 26,476,544 BYTES MEMORY PEAK USAGE: 69,730,304 BYTES PAGE RENDERED IN 6.9327299594879 SECONDS MEMORY USED @START: 262,144 - @END: 53,739,520 = 53,477,376 BYTES MEMORY PEAK USAGE: 79,167,488 BYTES PAGE RENDERED IN 7.55816092 SECONDS MEMORY USED @START: 262,144 - @END: 50,855,936 = 50,593,792 BYTES MEMORY PEAK USAGE: 96,206,848 BYTES Knowing nothing about your application I'm obviously not in a strong position to comment, but seven seconds to generate a page would be unacceptable to me and any of my clients. It's a one time hit and the rest is served from a cache for the next 24 hours which serves very very fast after that initial rendering. It's just we have so many thousands of pages that this becomes an issue -- especially when webcrawlers hit us and thread-out so MANY pages are trying to render at the same time, especially the ones towards the end where they haven't been cached since rarely do real people get that far... Like you know, pages 900, 901, 902, etc... with new content each day, page 1 today is now page 2 tomorrow, so it's a constant thorn. At 30k requests per second, is it a one-time hit, or is it 225k hit because in the 7 seconds it takes the cache to be built you have that many clients also building the cache? Or is this already an offline script, in which case how long it takes is largely irrelevant. What is your caching strategy? What is cached? At what granularity? When is the cache updated (i.e. on demand or on change)? Why does a page need to retrieve so much data? Can that data not be summarised/processed ahead of demand? I'll put money on it being possible to cut that time by changing your caching strategy. The memory usage is also ridiculous - does a single page really display that amount of data? Granted, there are some applications that cannot be optimised beyond a certain point, but those numbers make me sad! HA! It was over 400MB per page a few weeks ago. I keep whittling it down, but I think I'm hitting the lower limit at this point. That is nuts! What's the website? It's a tough balance between database hits, cache hits, network traffic (memcached), disk i/o, page speed, load balancing, etc. All we can do is try things and tweak and see what works and what brings the servers to their binary knees. Without knowing anything about the site there's little I can say, but if you want to take this off-list I'm happy to talk about it. I have a fair amount of experience with high-traffic web applications so it's possible I might be able to help. -Stuart -- Stuart Dallas 3ft9 Ltd http://3ft9.com/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] Static utility class?
Hi all! Example code: https://gist.github.com/mhulse/6441525 Goal: I want to have a utility class that contain utility methods which should have the option of being called multiple times on a page. I think my main goal is to avoid having to new things ... I don't really need to create an instance here (there's no need for a constructor in this case). Question: Is the above simple pattern a good way to have a class that calls static methods? To put it another way, is there any reason why I would not want to use the above code? Basically I want to wrap these simple functions in a nice class wrapper and have the ability to call them without having to jump through more hoops than is necessary. Are there any pitfalls to writing a class like this? Or, is this the standard way of writing a simple utility class for this type of situation? Please let me know if I need to clarify my questions. Thanks for your time?
Re: [PHP] Static utility class?
On 13-09-04 03:25 PM, Micky Hulse wrote: I want to have a utility class that contain utility methods which should have the option of being called multiple times on a page. This sounds simply like a library of functions that are implemented using objects. You can use the standard require_once in your various PHP source files so you only deal with loading the library file in the first file in which it is needed. It would not be loaded from other files. Instantiate your static variables in the library file. Again, so that only happens once. -- Stephen -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Static utility class?
On Wed, Sep 4, 2013 at 12:25 PM, Micky Hulse mickyhulse.li...@gmail.comwrote: I want to have a utility class that contain utility methods which should have the option of being called multiple times on a page. ... To put it another way, is there any reason why I would not want to use the above code? The main problem caused by static methods is testability. Mocking or stubbing a static method requires using a PHP extension and ensuring that the original is reset whether the test passes or fails. As long as your utility methods don't perform actions you want to avoid during tests, this is fine. Good examples for a utility class are string-processing functions such as parsing and formatting, trimming and/or normalizing empty string to null, etc. You want tests to work against these methods directly since there's no need to avoid the work. You'll want to avoid static methods whenever you want to be able to fix the behavior (stubbing) to test scenarios in the class under test. An example is anything hitting a database or external service. In order to test that your downloader sends the correct warning email to the sysadmin when a REST call fails, you need to be able to force the REST call to fail. This is easy to do when you plug in an instance implementing the API because you can give it an implementation/mock that fails for all calls. I can't say if what you're thinking of will make a good utility class since the code you posted is fake. If you post some examples of methods you want to make static, we can give you pointers on which are good candidates and which are best left to instance methods. Peace, David
Re: [PHP] Static utility class?
Thank you so much for the quick and very informative/educational replies Stephen and David, I really appreciate it! :) On Wed, Sep 4, 2013 at 12:36 PM, Stephen stephe...@rogers.com wrote: This sounds simply like a library of functions that are implemented using objects. Instantiate your static variables in the library file. Again, so that only happens once. functions implemented using objects sounds like exactly what I want. Just out of curiosity, and sorry in advance for my ignorance, but does the code I posted fit that type of pattern? If not, what would I need to modify and how would I call it? Actually, I should probably spend some time Googling before I ask you for more details. :D On Wed, Sep 4, 2013 at 1:15 PM, David Harkness davi...@highgearmedia.com wrote: I can't say if what you're thinking of will make a good utility class since the code you posted is fake. If you post some examples of methods you want to make static, we can give you pointers on which are good candidates and which are best left to instance methods. Sorry about the fake code. To be honest, I have not written the code just yet ... I'm kinda wanting to find the perfect pattern before I get too far down the rabbit hole (though, this is for some simple utility functions, so refactoring things should be easy later on). I'll be sure to post real code for any follow up questions. For now, thanks to you guys, I have a ton to work with. Thanks again for the pro advice! Cheers, M -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
[PHP] message to user after complete POST
I have a POST form and action itself like following form name=MyForm id=MyForm method=POST action=index.php /form I want to show success message when POST complete or error message if there is any. I would like to know are there any property or global variable I can check to show message to users. If not, it seems that the only solution is jQuery, since it has success function that jQuery call after POST. Your help and information is great appreciated, Regards, Iccsi, -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Static utility class?
On 13-09-04 05:09 PM, Micky Hulse wrote: Thank you so much for the quick and very informative/educational replies Stephen and David, I really appreciate it! :) On Wed, Sep 4, 2013 at 12:36 PM, Stephen stephe...@rogers.com wrote: This sounds simply like a library of functions that are implemented using objects. Instantiate your static variables in the library file. Again, so that only happens once. functions implemented using objects sounds like exactly what I want. Just out of curiosity, and sorry in advance for my ignorance, but does the code I posted fit that type of pattern? If not, what would I need to modify and how would I call it? Actually, I should probably spend some time Googling before I ask you for more details. :D Well, your code does not take advantage of the features of classes, but the syntax is correct. Global, static variables are not consistent with the design concepts of OOP. But as a first step in moving from procedural code to OOP, which is learning the syntax, go for it. -- Stephen -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Static utility class?
Thanks Stephen! I really appreciate the help! :) In my PHP ventures over the years, I haven't made much use of static variables/methods/properties ... I was thinking they might be useful for this one bit of code, but based on your feedback (and David's) I think I'll be heading down a different path. Thanks again for the kick in the right direction! Much appreciated! Cheers, Micky -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] message to user after complete POST
hey, if you are just trying to give the user a feedback about the information sent, there is a Global variable for this, and it's the $_POST. When you receive the information on the script you called on the Action atribute of the form, via the $_POST global variable, you will test the information end echo the feedback on that script. Note that, unless you are using ajax (jquery, or any other javascript ajax method), when you post a form, you will be intantly redirected to the action path. then you have nothing to do in the source page. hope you got it. 2013/9/4 iccsi inu...@gmail.com I have a POST form and action itself like following form name=MyForm id=MyForm method=POST action=index.php /form I want to show success message when POST complete or error message if there is any. I would like to know are there any property or global variable I can check to show message to users. If not, it seems that the only solution is jQuery, since it has success function that jQuery call after POST. Your help and information is great appreciated, Regards, Iccsi, -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] message to user after complete POST
Thanks a million for the information and help, yes, I will use jQuery to have my success function call in jQuery.ajax. Thanks again, Regards, Iccsi, Rodrigo Santos wrote in message news:caombckqonakxoc4tnhcpn2ycdpy8503xttc7sosjywhtd5x...@mail.gmail.com... hey, if you are just trying to give the user a feedback about the information sent, there is a Global variable for this, and it's the $_POST. When you receive the information on the script you called on the Action atribute of the form, via the $_POST global variable, you will test the information end echo the feedback on that script. Note that, unless you are using ajax (jquery, or any other javascript ajax method), when you post a form, you will be intantly redirected to the action path. then you have nothing to do in the source page. hope you got it. 2013/9/4 iccsi inu...@gmail.com I have a POST form and action itself like following form name=MyForm id=MyForm method=POST action=index.php /form I want to show success message when POST complete or error message if there is any. I would like to know are there any property or global variable I can check to show message to users. If not, it seems that the only solution is jQuery, since it has success function that jQuery call after POST. Your help and information is great appreciated, Regards, Iccsi, -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Static utility class?
Hi, first, sorry for the bad English. Yes, at least, as far as I know, this is the perfect way to do what you want to do. Think like this: when you instanciate a class, you are allocating memory. If you don't need any information stored, then you don't need to allocate memory, right? So, it's is logic to have a class that you will call only one fragment when you need it, rather than load the entire class just for one method. Just be careful to organize your utility methods by by meaning. Don't put a method that make dating stuff and a method that write a random string in the same class. By doing so, you are breaking the object orientation purpose. 2013/9/4 Micky Hulse mickyhulse.li...@gmail.com Hi all! Example code: https://gist.github.com/mhulse/6441525 Goal: I want to have a utility class that contain utility methods which should have the option of being called multiple times on a page. I think my main goal is to avoid having to new things ... I don't really need to create an instance here (there's no need for a constructor in this case). Question: Is the above simple pattern a good way to have a class that calls static methods? To put it another way, is there any reason why I would not want to use the above code? Basically I want to wrap these simple functions in a nice class wrapper and have the ability to call them without having to jump through more hoops than is necessary. Are there any pitfalls to writing a class like this? Or, is this the standard way of writing a simple utility class for this type of situation? Please let me know if I need to clarify my questions. Thanks for your time?
Re: [PHP] Static utility class?
Hi Rodrigo, thanks for the help, I really appreciate it! On Wed, Sep 4, 2013 at 5:55 PM, Rodrigo Santos rodrigos.santo...@gmail.com wrote: Hi, first, sorry for the bad English. Not bad at all! Very clear and well written reply (heck, it's better than my native English writing), so thank you! :) Yes, at least, as far as I know, this is the perfect way to do what you want to do. Think like this: when you instanciate a class, you are allocating memory. If you don't need any information stored, then you don't need to allocate memory, right? So, it's is logic to have a class that you will call only one fragment when you need it, rather than load the entire class just for one method. Interesting! That makes a lot of sense. Now you've piqued my interests! :) I was going to head down a different path, but you've inspired me to further explore the use of static methods/properties/variables/other. A part of me just wants to learn more about PHP OOP, and using static members is something I've not explored much. Seems like a simple functional utility class would be a good time to play and learn more. :D Just be careful to organize your utility methods by by meaning. Don't put a method that make dating stuff and a method that write a random string in the same class. By doing so, you are breaking the object orientation purpose. Excellent tip! Thank you Rodrigo! I really appreciate the tips/advice and inspiration. :) Have a great afternoon! Cheers, Micky -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
