[PHP] Why does this script run out of memory?

2011-10-28 Thread Jim Long
I'm running PHP 5.3.8 on FreeBSD 8.2 with MySQL 5.1.55. The script below is designed to be able to WHILE it's way through a MySQL query result set, and process each row. However, it runs out of memory a little after a quarter million rows. The schema fields total to about 200 bytes per row, so

Re: [PHP] Why does this script run out of memory?

2011-10-28 Thread Eric Butera
On Fri, Oct 28, 2011 at 12:38 PM, Jim Long p...@umpquanet.com wrote: I'm running PHP 5.3.8 on FreeBSD 8.2 with MySQL 5.1.55. The script below is designed to be able to WHILE it's way through a MySQL query result set, and process each row. However, it runs out of memory a little after a

Re: [PHP] Why does this script run out of memory?

2011-10-28 Thread Jim Long
On Fri, Oct 28, 2011 at 01:21:36PM -0400, Eric Butera wrote: On Fri, Oct 28, 2011 at 12:38 PM, Jim Long p...@umpquanet.com wrote: I'm running PHP 5.3.8 on FreeBSD 8.2 with MySQL 5.1.55. The script below is designed to be able to WHILE it's way through a MySQL query result set, and process

Re: Re: [PHP] Why does this script run out of memory?

2011-10-28 Thread James
Original Message From: Eric Butera eric.but...@gmail.com To: Jim Long p...@umpquanet.com Cc: php-general@lists.php.net Sent: Fri, Oct 28, 2011, 1:22 PM Subject: Re: [PHP] Why does this script run out of memory? On Fri, Oct 28, 2011 at 12:38 PM, Jim Long p...@umpquanet.com wrote: I'm

Re: Re: [PHP] Why does this script run out of memory?

2011-10-28 Thread Ashley Sheridan
On Fri, 2011-10-28 at 13:32 -0400, James wrote: Original Message From: Eric Butera eric.but...@gmail.com To: Jim Long p...@umpquanet.com Cc: php-general@lists.php.net Sent: Fri, Oct 28, 2011, 1:22 PM Subject: Re: [PHP] Why does this script run out of memory? On Fri, Oct 28

Re: Re: [PHP] Why does this script run out of memory?

2011-10-28 Thread Jim Long
On Fri, Oct 28, 2011 at 01:32:32PM -0400, James wrote: On Fri, Oct 28, 2011 at 12:38 PM, Jim Long p...@umpquanet.com wrote: I'm running PHP 5.3.8 on FreeBSD 8.2 with MySQL 5.1.55. The script below is designed to be able to WHILE it's way through a MySQL query result set, and process

Re: [PHP] Why does this script run out of memory?

2011-10-28 Thread Jim Giner
If all you want to do is count the records, why are you not letting sql do it for you instead of doing the while loop? That's all that script is doing, if that is the exact code you ran. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php

Re: Re: [PHP] Why does this script run out of memory?

2011-10-28 Thread Daniel Brown
On Fri, Oct 28, 2011 at 14:05, Jim Long p...@umpquanet.com wrote: the tail end of the output becomes:  274695    134202232  274696    134202672  274697    134203112  274698    134203552  274699    134203992 PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to

Re: [PHP] Why does this script run out of memory?

2011-10-28 Thread Daniel Brown
On Fri, Oct 28, 2011 at 13:25, Jim Long p...@umpquanet.com wrote: Eric: Thanks for your reply. process row here is a comment.  It doesn't do anything.  The script, exactly as shown, runs out of memory, exactly as shown. My response presumes that you're planning on placing something

Re: [PHP] Why does this script run out of memory?

2011-10-28 Thread Jim Long
On Fri, Oct 28, 2011 at 03:24:37PM -0400, Jim Giner wrote: If all you want to do is count the records, why are you not letting sql do it for you instead of doing the while loop? That's all that script is doing, if that is the exact code you ran. Hi, Jim. Thank you for replying. One of

Re: [PHP] Why does this script run out of memory?

2011-10-28 Thread Eric Butera
On Fri, Oct 28, 2011 at 3:29 PM, Daniel Brown danbr...@php.net wrote: On Fri, Oct 28, 2011 at 13:25, Jim Long p...@umpquanet.com wrote: Eric: Thanks for your reply. process row here is a comment.  It doesn't do anything.  The script, exactly as shown, runs out of memory, exactly as shown.

Re: [PHP] Why does this script run out of memory?

2011-10-28 Thread Daniel Brown
On Fri, Oct 28, 2011 at 16:21, Jim Long p...@umpquanet.com wrote: I will try experimenting with Daniel's idea of unbuffered queries, but my understanding is that while an unbuffered result resource is in use, no other SQL transactions can be conducted. Maybe I can get around that by using one

Re: [PHP] Why does this script run out of memory?

2011-10-28 Thread Jim Long
On Fri, Oct 28, 2011 at 03:42:48PM -0400, Eric Butera wrote: On Fri, Oct 28, 2011 at 3:29 PM, Daniel Brown danbr...@php.net wrote: On Fri, Oct 28, 2011 at 13:25, Jim Long p...@umpquanet.com wrote: Eric: Thanks for your reply. process row here is a comment. ??It doesn't do anything.

Re: [PHP] Why does this script run out of memory?

2011-10-28 Thread Tommy Pham
On Fri, Oct 28, 2011 at 9:38 AM, Jim Long p...@umpquanet.com wrote: I'm running PHP 5.3.8 on FreeBSD 8.2 with MySQL 5.1.55. Jim, Installed from packages or standard port tree build? Did you do any tweak for the ports build? Any special compiler parameters in your make.conf? I've noticed

Re: [PHP] Why does this script run out of memory?

2011-10-28 Thread Paul Halliday
On Fri, Oct 28, 2011 at 1:38 PM, Jim Long p...@umpquanet.com wrote: I'm running PHP 5.3.8 on FreeBSD 8.2 with MySQL 5.1.55. The script below is designed to be able to WHILE it's way through a MySQL query result set, and process each row. However, it runs out of memory a little after a

[PHP] mysql_fetch_array() vs mysql_fetch_assoc() WAS: Re: [PHP] Why does this script run out of memory?

2011-10-28 Thread Daniel Brown
On Fri, Oct 28, 2011 at 18:13, Paul Halliday paul.halli...@gmail.com wrote: Whats the difference between fetch_assoc and fetch_row? I use: while ($row = mysql_fetch_row($theQuery)) {    doCartwheel; } on just under 300 million rows and nothing craps out. I have memory_limit set to 4GB

Re: [PHP] Why does this script run out of memory?

2011-10-28 Thread Jim Long
On Fri, Oct 28, 2011 at 02:57:02PM -0700, Tommy Pham wrote: On Fri, Oct 28, 2011 at 9:38 AM, Jim Long p...@umpquanet.com wrote: I'm running PHP 5.3.8 on FreeBSD 8.2 with MySQL 5.1.55. Jim, Installed from packages or standard port tree build? Did you do any tweak for the ports build?

[PHP] Re: mysql_fetch_array() vs mysql_fetch_assoc() WAS: Re: [PHP] Why does this script run out of memory?

2011-10-28 Thread Paul Halliday
On Fri, Oct 28, 2011 at 7:19 PM, Daniel Brown danbr...@php.net wrote: On Fri, Oct 28, 2011 at 18:13, Paul Halliday paul.halli...@gmail.com wrote: Whats the difference between fetch_assoc and fetch_row? I use: while ($row = mysql_fetch_row($theQuery)) {    doCartwheel; } on just under 300

Re: [PHP] mysql_fetch_array() vs mysql_fetch_assoc() WAS: Re: [PHP] Why does this script run out of memory?

2011-10-28 Thread Jim Long
On Fri, Oct 28, 2011 at 06:19:56PM -0400, Daniel Brown wrote: On Fri, Oct 28, 2011 at 18:13, Paul Halliday paul.halli...@gmail.com wrote: Whats the difference between fetch_assoc and fetch_row? I use: while ($row = mysql_fetch_row($theQuery)) { ? ?doCartwheel; } on just under

Re: [PHP] mysql_fetch_array() vs mysql_fetch_assoc() WAS: Re: [PHP] Why does this script run out of memory?

2011-10-28 Thread Daniel Brown
On Fri, Oct 28, 2011 at 18:48, Jim Long p...@umpquanet.com wrote: I'm not seeing any numeric keys in my mysql_fetch_assoc() arrays. You're absolutely correct, that's my mistake: substitute mysql_fetch_row() for mysql_fetch_assoc(). Duh. Time to call it a week -- /Daniel P. Brown

Re: [PHP] Why does this script run out of memory?

2011-10-28 Thread shiplu
I had a spider written in PHP long ago. I had similar problems. because there were millions of rows of urls and I was fetching them in one single query. See inline, could this modification help you. Please test. On Fri, Oct 28, 2011 at 10:38 PM, Jim Long p...@umpquanet.com wrote: I'm running