Re: [PHP] Exporting large data from mysql to html using php

2011-11-03 Thread Ashley Sheridan
On Wed, 2011-11-02 at 19:47 -0400, Jason Pruim wrote: Jason Pruim li...@pruimphotography.com On Oct 31, 2011, at 7:52 PM, Ashley Sheridan wrote: On Mon, 2011-10-31 at 19:29 -0400, Jason Pruim wrote: Jason Pruim li...@pruimphotography.com On Oct 31, 2011, at 7:11 PM,

Re: [PHP] Exporting large data from mysql to html using php

2011-11-02 Thread Jason Pruim
Jason Pruim li...@pruimphotography.com On Oct 31, 2011, at 7:52 PM, Ashley Sheridan wrote: On Mon, 2011-10-31 at 19:29 -0400, Jason Pruim wrote: Jason Pruim li...@pruimphotography.com On Oct 31, 2011, at 7:11 PM, Jim Lucas wrote: On 10/24/2011 5:50 PM, Jason Pruim wrote: Now

Re: [PHP] Exporting large data from mysql to html using php

2011-10-31 Thread Jim Lucas
On 10/24/2011 5:50 PM, Jason Pruim wrote: Now that I've managed to list 3 separate programming languages and somewhat tie it back into php here's the question... I have about 89 million records in mysql... the initial load of the page takes 2 to 3 minutes, I am using pagination, so I have

Re: [PHP] Exporting large data from mysql to html using php

2011-10-31 Thread Jason Pruim
Jason Pruim li...@pruimphotography.com On Oct 31, 2011, at 7:11 PM, Jim Lucas wrote: On 10/24/2011 5:50 PM, Jason Pruim wrote: Now that I've managed to list 3 separate programming languages and somewhat tie it back into php here's the question... I have about 89 million records in

Re: [PHP] Exporting large data from mysql to html using php

2011-10-31 Thread Ashley Sheridan
On Mon, 2011-10-31 at 19:29 -0400, Jason Pruim wrote: Jason Pruim li...@pruimphotography.com On Oct 31, 2011, at 7:11 PM, Jim Lucas wrote: On 10/24/2011 5:50 PM, Jason Pruim wrote: Now that I've managed to list 3 separate programming languages and somewhat tie it back into php

Re: [PHP] Exporting large data from mysql to html using php

2011-10-27 Thread Lester Caine
Jim Giner wrote: Your boss wants to give access to phone numbers to the public in general? Then what? Glad mine's unlisted. Is it? Does it start 518248 ? I often forget to get a phone number when a parcel has to go by carrier, and paypal does not include that info, but only rarely does one

Re: [PHP] Exporting large data from mysql to html using php

2011-10-27 Thread David Robley
Jim Giner wrote: David Robley robl...@aapt.net.au wrote in message news:49.50.34068.1b567...@pb1.pair.com... Consider running EXPLAIN on all your queries to see if there is something Mysql thinks could be done to improve performance. Why do so many responders seem to think the problem

Re: [PHP] Exporting large data from mysql to html using php

2011-10-27 Thread Tommy Pham
On Wed, Oct 26, 2011 at 5:47 PM, Jason Pruim li...@pruimphotography.comwrote: Jason Pruim li...@pruimphotography.com The server that's running it is a home computer with a VPS installed... It's not my dev environment :) Home computer used for a production environment? Wow.. I'm

Re: [PHP] Exporting large data from mysql to html using php

2011-10-27 Thread Nathan Nobbe
On Mon, Oct 24, 2011 at 6:50 PM, Jason Pruim li...@pruimphotography.comwrote: Now that I've managed to list 3 separate programming languages and somewhat tie it back into php here's the question... I have about 89 million records in mysql... the initial load of the page takes 2 to 3 minutes,

Re: [PHP] Exporting large data from mysql to html using php

2011-10-27 Thread Jim Giner
Good luck, that's a LOT of reading. I'd estimate that's about 3k+ pages of reading. :) Regards, Tommy nice to see someone else is finally getting the point that I'm been making. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit:

Re: [PHP] Exporting large data from mysql to html using php

2011-10-27 Thread Jason Pruim
Jason Pruim li...@pruimphotography.com On Oct 27, 2011, at 2:44 PM, Tommy Pham wrote: On Wed, Oct 26, 2011 at 5:47 PM, Jason Pruim li...@pruimphotography.com wrote: Jason Pruim li...@pruimphotography.com The server that's running it is a home computer with a VPS installed... It's

Re: [PHP] Exporting large data from mysql to html using php

2011-10-27 Thread Jason Pruim
Jason Pruim li...@pruimphotography.com On Oct 27, 2011, at 4:08 PM, Nathan Nobbe wrote: On Mon, Oct 24, 2011 at 6:50 PM, Jason Pruim li...@pruimphotography.com wrote: Now that I've managed to list 3 separate programming languages and somewhat tie it back into php here's the question...

Re: [PHP] Exporting large data from mysql to html using php

2011-10-26 Thread Lester Caine
Tommy Pham wrote: It turns out the issue was actually in the pagination... I'm reworking the whole thing and stream lining it... But in the pagination that I found on the internet it used a SELECT COUNT(*) WHERE state='{$state}'; and the COUNT was killing the time... Once that was removed,

Re: [PHP] Exporting large data from mysql to html using php

2011-10-26 Thread Tommy Pham
On Wed, Oct 26, 2011 at 12:52 AM, Lester Caine les...@lsces.co.uk wrote: Tommy Pham wrote: It turns out the issue was actually in the pagination... I'm reworking the whole thing and stream lining it... But in the pagination that I found on the internet it used a SELECT COUNT(*) WHERE

Re: [PHP] Exporting large data from mysql to html using php

2011-10-26 Thread Lester Caine
Tommy Pham wrote: I wonder ... The real question is what's the purpose of the DB? Is it for OLAP or OLTP? ;) As for dealing with DB having millions of rows, you're crossing over into DBA area. Many of my customers have coming up on 20 years of data available. There has been a debate on

Re: [PHP] Exporting large data from mysql to html using php

2011-10-26 Thread Tommy Pham
On Wed, Oct 26, 2011 at 1:40 AM, Lester Caine les...@lsces.co.uk wrote: Tommy Pham wrote: I wonder ... The real question is what's the purpose of the DB? Is it for OLAP or OLTP? ;) As for dealing with DB having millions of rows, you're crossing over into DBA area. Many of my customers

Re: [PHP] Exporting large data from mysql to html using php

2011-10-26 Thread Lester Caine
Tommy Pham wrote: Many of my customers have coming up on 20 years of data available. There has been a debate on transferring historic data to a separate database, but having it available is not causing a problem, except for some counts and larger search actions, and being able

Re: [PHP] Exporting large data from mysql to html using php

2011-10-26 Thread Tommy Pham
On Wed, Oct 26, 2011 at 4:14 AM, Lester Caine les...@lsces.co.uk wrote: Tommy Pham wrote: Many of my customers have coming up on 20 years of data available. There has been a debate on transferring historic data to a separate database, but having it available is not causing a

Re: [PHP] Exporting large data from mysql to html using php

2011-10-26 Thread Jason Pruim
Jason Pruim li...@pruimphotography.com On Oct 26, 2011, at 6:23 AM, Tommy Pham wrote: On Wed, Oct 26, 2011 at 1:40 AM, Lester Caine les...@lsces.co.uk wrote: Tommy Pham wrote: I wonder ... The real question is what's the purpose of the DB? Is it for OLAP or OLTP? ;) As for dealing

Re: [PHP] Exporting large data from mysql to html using php

2011-10-26 Thread Jim Giner
Your boss wants to give access to phone numbers to the public in general? Then what? Glad mine's unlisted. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP] Exporting large data from mysql to html using php

2011-10-26 Thread Jason Pruim
Jason Pruim li...@pruimphotography.com On Oct 26, 2011, at 9:09 PM, Jim Giner wrote: Your boss wants to give access to phone numbers to the public in general? Then what? Glad mine's unlisted. There's no identifying information on the phone numbers... Simply just the phone number...

Re: [PHP] Exporting large data from mysql to html using php

2011-10-25 Thread Jim Giner
I disagree. It's not about tuning the queries, it is more about the appl. design that currently thinks it SHOULD do such huge queries. My approach would be to prompt the user for filtering criteria that automatically would reduce the result set size. Although at this time I believe the OP

Re: [PHP] Exporting large data from mysql to html using php

2011-10-25 Thread Jason Pruim
Jason Pruim li...@pruimphotography.com On Oct 25, 2011, at 10:51 AM, Jim Giner wrote: I disagree. It's not about tuning the queries, it is more about the appl. design that currently thinks it SHOULD do such huge queries. My approach would be to prompt the user for filtering criteria

Re: [PHP] Exporting large data from mysql to html using php

2011-10-25 Thread Jim Giner
Again why even do a detail query? Nobody is going to examine pages and pages and etc. Do a summary qry if u just need a count - no pagination there jg On Oct 25, 2011, at 6:26 PM, Jason Pruim li...@pruimphotography.com wrote: Jason Pruim li...@pruimphotography.com On Oct 25,

Re: [PHP] Exporting large data from mysql to html using php

2011-10-25 Thread Jason Pruim
Jason Pruim li...@pruimphotography.com On Oct 25, 2011, at 6:35 PM, Jim Giner wrote: Again why even do a detail query? Nobody is going to examine pages and pages and etc. Do a summary qry if u just need a count - no pagination there jg The bosses wanted them to be able to page

Re: [PHP] Exporting large data from mysql to html using php

2011-10-25 Thread Bastien
On 2011-10-25, at 6:56 PM, Jason Pruim li...@pruimphotography.com wrote: Jason Pruim li...@pruimphotography.com On Oct 25, 2011, at 6:35 PM, Jim Giner wrote: Again why even do a detail query? Nobody is going to examine pages and pages and etc. Do a summary qry if u just

Re: [PHP] Exporting large data from mysql to html using php

2011-10-25 Thread Jim Giner
David Robley robl...@aapt.net.au wrote in message news:49.50.34068.1b567...@pb1.pair.com... Consider running EXPLAIN on all your queries to see if there is something Mysql thinks could be done to improve performance. Why do so many responders seem to think the problem here is in the

Re: [PHP] Exporting large data from mysql to html using php

2011-10-25 Thread David Robley
Jason Pruim wrote: Jason Pruim li...@pruimphotography.com On Oct 25, 2011, at 10:51 AM, Jim Giner wrote: I disagree. It's not about tuning the queries, it is more about the appl. design that currently thinks it SHOULD do such huge queries. My approach would be to prompt the

Re: [PHP] Exporting large data from mysql to html using php

2011-10-25 Thread Jason Pruim
Jason Pruim li...@pruimphotography.com On Oct 25, 2011, at 9:58 PM, Jim Giner wrote: David Robley robl...@aapt.net.au wrote in message news:49.50.34068.1b567...@pb1.pair.com... Consider running EXPLAIN on all your queries to see if there is something Mysql thinks could be done to

Re: [PHP] Exporting large data from mysql to html using php

2011-10-25 Thread Jim Giner
- Original Message - From: Jason Pruim li...@pruimphotography.com To: Jim Giner jim.gi...@albanyhandball.com Cc: php-general@lists.php.net Sent: Tuesday, October 25, 2011 10:06 PM Subject: Re: [PHP] Exporting large data from mysql to html using php It turns out the issue was actually

Re: [PHP] Exporting large data from mysql to html using php

2011-10-25 Thread Jason Pruim
: [PHP] Exporting large data from mysql to html using php It turns out the issue was actually in the pagination... I'm reworking the whole thing and stream lining it... But in the pagination that I found on the internet it used a SELECT COUNT(*) WHERE state='{$state}'; and the COUNT

Re: [PHP] Exporting large data from mysql to html using php

2011-10-25 Thread Tommy Pham
On Tue, Oct 25, 2011 at 7:06 PM, Jason Pruim li...@pruimphotography.comwrote: It turns out the issue was actually in the pagination... I'm reworking the whole thing and stream lining it... But in the pagination that I found on the internet it used a SELECT COUNT(*) WHERE state='{$state}'; and

[PHP] Exporting large data from mysql to html using php

2011-10-24 Thread Jason Pruim
Now that I've managed to list 3 separate programming languages and somewhat tie it back into php here's the question... I have about 89 million records in mysql... the initial load of the page takes 2 to 3 minutes, I am using pagination, so I have LIMIT's on the SQL query's... But they just

Re: [PHP] Exporting large data from mysql to html using php

2011-10-24 Thread Shiplu Mokaddim
2-3 minutes is long enough. I think your SELECT query and MySQL schema has something that can be improved. Sent from a handheld device. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP] Exporting large data from mysql to html using php

2011-10-24 Thread Bastien
On 2011-10-24, at 8:50 PM, Jason Pruim li...@pruimphotography.com wrote: Now that I've managed to list 3 separate programming languages and somewhat tie it back into php here's the question... I have about 89 million records in mysql... the initial load of the page takes 2 to 3 minutes,

Re: [PHP] Exporting large data from mysql to html using php

2011-10-24 Thread Jim Giner
Why would any user need to have access to 89M records? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP] Exporting large data from mysql to html using php

2011-10-24 Thread Bastien
On 2011-10-24, at 10:44 PM, Jim Giner jim.gi...@albanyhandball.com wrote: Why would any user need to have access to 89M records? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php History or audit trail data? I can think of lots. I

Re: [PHP] Exporting large data from mysql to html using php

2011-10-24 Thread Jason Pruim
On Oct 24, 2011, at 10:44 PM, Jim Giner wrote: Why would any user need to have access to 89M records? They don't need access to it to edit it... Just to be able to view it... Also, it will be expanding in the future to include alot more data. Jason Pruim li...@pruimphotography.com --

Re: [PHP] Exporting large data from mysql to html using php

2011-10-24 Thread Jason Pruim
Jason Pruim li...@pruimphotography.com On Oct 24, 2011, at 9:20 PM, Bastien wrote: On 2011-10-24, at 8:50 PM, Jason Pruim li...@pruimphotography.com wrote: Now that I've managed to list 3 separate programming languages and somewhat tie it back into php here's the question... I

Re: [PHP] Exporting large data from mysql to html using php

2011-10-24 Thread Jim Giner
Yes - but - we're talking about a user-app that the OP is trying to provide 89M records to. Sure - some users might have need of looking at even as much as a million records IF they were researching something that needed it. But - for the 'general' user of an app - I cannot see a need to be

Re: [PHP] Exporting large data from mysql to html using php

2011-10-24 Thread Bastien
On 2011-10-24, at 11:26 PM, Jim Giner jim.gi...@albanyhandball.com wrote: Yes - but - we're talking about a user-app that the OP is trying to provide 89M records to. Sure - some users might have need of looking at even as much as a million records IF they were researching something that

Re: [PHP] Exporting large data from mysql to html using php

2011-10-24 Thread Bastien
On 2011-10-24, at 11:14 PM, Jason Pruim li...@pruimphotography.com wrote: Jason Pruim li...@pruimphotography.com On Oct 24, 2011, at 9:20 PM, Bastien wrote: On 2011-10-24, at 8:50 PM, Jason Pruim li...@pruimphotography.com wrote: Now that I've managed to list 3 separate

Re: [PHP] Exporting large data from mysql to html using php

2011-10-24 Thread tamouse mailing lists
On Mon, Oct 24, 2011 at 7:50 PM, Jason Pruim li...@pruimphotography.com wrote: I have about 89 million records in mysql... the initial load of the page takes 2 to 3 minutes, I am using pagination, so I have LIMIT's on the SQL query's... But they just aren't going fast enough... What I would