Re: [PHP] Report generation as pdf and csv in php application for huge record set

[George Langley]

On 2011-07-14, at 12:50 AM, Midhun Girish wrote:

> 
> 
> On Thu, Jul 14, 2011 at 11:18 AM, George Langley  
> wrote:
> On 2011-07-13, at 11:20 PM, Midhun Girish wrote:
> 
> > Hi,
> >
> >
> > - Browsers generally have a 5 minute time-out. If you send the PDF directly 
> > to the browser and reach the limit, it will be lost. It is therefore 
> > advised for very big documents to generate them in a file, and to send some 
> > data to the browser from time to time (with a call to flush() to force the 
> > output). When the document is finished, you can send a redirection to it or 
> > create a link.
> > Remark: even if the browser times out, the script may continue to run on 
> > the server.
> >
> >
> > When it comes to the user end, they wont wait for more than 1 min. So 
> > redirection after generating the report is not possible. But creating link 
> > may be a good idea. How can it be done? I mean shouldn't there be a service 
> > running in the server to process these request after the execution of the 
> > php script is completed? Also shouldn't we post all the report generation 
> > request to a database so that serial processing can be done?
> ---
>We're helping each other here, as I'm facing a similar situation! Read 
> up on how to store as a file here:
> 
> 
> 
> You should then be able to generate an email notification to send once the 
> file is complete, with the link to the newly-created file.
> 
> 
> George Langley
> Multimedia Developer
> 
> 
> Suppose we are using this tool at sever side. We have set up the php page so 
> that all request will be saved in a db. And based on the conditions in 
> request a pdf CAN be generated by this tool. My question is how will we give 
> the trigger for generating this pdf? We cant do it in the php script due to 
> the timeout problem. The only solution i can think of is a cron which is 
> repeated every 5 min or so which fetches records from the db and generates 
> the pdf. But even with a cron we will face the timeout problem. Is there any 
> alternative to a cron like a server side service or something?
> 
> 
> Midhun Girish
--
As I understand it, the "server-side" timeout can be increased in the 
php.ini setting. So the PHP script can run as long as required if that is set 
sufficiently high enough. As long as you send something to the browser to 
confirm that the process has started, the browser-side timeout will not be a 
problem and your user will know to wait until notified by email that the file 
is ready. Once the script has completed, it stores the final db record and 
sends the email.

George

Re: [PHP] Report generation as pdf and csv in php application for huge record set

[George Langley]

On 2011-07-13, at 11:20 PM, Midhun Girish wrote:

> Hi,
> 
> 
> - Browsers generally have a 5 minute time-out. If you send the PDF directly 
> to the browser and reach the limit, it will be lost. It is therefore advised 
> for very big documents to generate them in a file, and to send some data to 
> the browser from time to time (with a call to flush() to force the output). 
> When the document is finished, you can send a redirection to it or create a 
> link.
> Remark: even if the browser times out, the script may continue to run on the 
> server.
> 
> 
> When it comes to the user end, they wont wait for more than 1 min. So 
> redirection after generating the report is not possible. But creating link 
> may be a good idea. How can it be done? I mean shouldn't there be a service 
> running in the server to process these request after the execution of the php 
> script is completed? Also shouldn't we post all the report generation request 
> to a database so that serial processing can be done?
---
We're helping each other here, as I'm facing a similar situation! Read 
up on how to store as a file here:



You should then be able to generate an email notification to send once the file 
is complete, with the link to the newly-created file.


George Langley
Multimedia Developer

Re: [PHP] Report generation as pdf and csv in php application for huge record set

[Midhun Girish]

Hi,


> - Browsers generally have a 5 minute time-out. If you send the PDF directly
> to the browser and reach the limit, it will be lost. It is therefore advised
> for very big documents to generate them in a file, and to send some data to
> the browser from time to time (with a call to flush() to force the output).
> When the document is finished, you can send a redirection to it or create a
> link.
> Remark: even if the browser times out, the script may continue to run on
> the server.
>  
>

When it comes to the user end, they wont wait for more than 1 min. So
redirection after generating the report is not possible. But creating link
may be a good idea. How can it be done? I mean shouldn't there be a service
running in the server to process these request after the execution of the
php script is completed? Also shouldn't we post all the report generation
request to a database so that serial processing can be done?

Re: [PHP] Report generation as pdf and csv in php application for huge record set

[George Langley]

On 2011-07-13, at 9:59 PM, Midhun Girish wrote:
> 
> I have an erp application developed in php (zend framework actually). There
> are many reports which the admin level users can take from the application.
> Some reports have more than 600,000 records. The viewing of reports in the
> browser is fine as i have paginated the result. But when it comes to
> exporting the entire report as pdf and in excel format, im in trouble. I use
> fpdf for pdf generation and  a custom class for converting result to excel.
> Both these fails when the number of records exceeds 500. Now i have
> pagianted the report generation and users can download the pdf page wise.
> But that is not the exact requirement. Im in need of a tool to generate pdf
> and excel(or csv) for large data set. I came across
> http://www.htmldoc.org/when i googled for one. Has anyone in the list
> has faced a similar problem.
> How exactly is report generation possible when the number of records is
> extremely huge? Is there a good solution which can be implemented in php?
> Please help.
-
Hi there. FAQ #16 on:



has some useful info on handling large files:

16. What's the limit of the file sizes I can generate with FPDF?

There is no particular limit. There are some constraints, however: 

- The maximum memory size allocated to PHP scripts is usually 8MB. For very big 
documents, especially with images, this limit may be reached (the file being 
built into memory). The parameter is configured in the php.ini file. 

- The maximum execution time allocated defaults to 30 seconds. This limit can 
of course be easily reached. It is configured in php.ini and may be altered 
dynamically with set_time_limit(). 

- Browsers generally have a 5 minute time-out. If you send the PDF directly to 
the browser and reach the limit, it will be lost. It is therefore advised for 
very big documents to generate them in a file, and to send some data to the 
browser from time to time (with a call to flush() to force the output). When 
the document is finished, you can send a redirection to it or create a link. 
Remark: even if the browser times out, the script may continue to run on the 
server.


HTH


George Langley
Multimedia Developer

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP] Report generation as pdf and csv in php application for huge record set

[Midhun Girish]

Hi all,

I have an erp application developed in php (zend framework actually). There
are many reports which the admin level users can take from the application.
Some reports have more than 600,000 records. The viewing of reports in the
browser is fine as i have paginated the result. But when it comes to
exporting the entire report as pdf and in excel format, im in trouble. I use
fpdf for pdf generation and  a custom class for converting result to excel.
Both these fails when the number of records exceeds 500. Now i have
pagianted the report generation and users can download the pdf page wise.
But that is not the exact requirement. Im in need of a tool to generate pdf
and excel(or csv) for large data set. I came across
http://www.htmldoc.org/when i googled for one. Has anyone in the list
has faced a similar problem.
How exactly is report generation possible when the number of records is
extremely huge? Is there a good solution which can be implemented in php?
Please help.


Midhun Girish