> Randolf Richardson wrote:
> >> I know that it's possible(and arguably best practice) to use Apache to
> >> download large files efficiently and quickly, without passing them through
> >> mod_perl. However, the data I need to download from my application is both
> >> dynamically generated and sensitive so I cannot expose it to the internet
> >> for anonymous download via Apache. So, I'm wondering if mod_perl has a
> >> capability similar to the output stream of a java servlet. Specifically, I
> >> want to return bits and pieces of the file at a time over the wire so that
> >> I can avoid loading the entire file into memory prior to sending it to the
> >> browser. Currently, I'm loading the entire file into memory before sending
> >> it and
> >>
> >> Is this possible with mod_perl and, if so, how should I go about
> >> implementing it?
> >
> > Yes, it is possible -- instead of loading the entire contents of a
> > file into RAM, just read blocks in a loop and keep sending them until
> > you reach EoF (End of File).
> >
> > You can also use $r->flush along the way if you like, but as I
> > understand it this isn't necessary because Apache HTTPd will send the
> > data as soon as its internal buffers contain enough data. Of course,
> > if you can tune your block size in your loop to match Apache's output
> > buffer size, then that will probably help. (I don't know much about
> > the details of Apache's output buffers because I've not read up too
> > much on them, so I hope my assumptions about this are correct.)
> >
> > One of the added benefits you get from using a loop is that you can
> > also implement rate limiting if that becomes useful. You can
> > certainly also implement access controls as well by cross-checking
> > the file being sent with whatever internal database queries you'd
> > normally use to ensure it's okay to send the file first.
> >
>
> You can also :
> 1) write the data to a file
> 2) $r->sendfile(...);
> 3) add a cleanup handler, to delete the file when the request has been served.
> See here for details :
> http://perl.apache.org/docs/2.0/api/Apache2/RequestIO.html#C_sendfile_
>
> For this to work, there is an Apache configuration directive which must be
> set to "on". I
> believe it is called "UseSendFile".
> Essentially what senfile() does, is to delegate the actual reading and
> sending of the file
> to Apache httpd and the underlying OS, using code which is specifically
> optimised for this
> purpose. It is much kore efficient than doing this in a read/write loop by
> yourself, at
> the cost of having less fine control over the operation.
Thank you André, this is an excellent solution.
One comment I'd like to add is that it appears that steps 1 and 3
can be eliminated if a file is basically being copied (because the
data is pre-existing in its entirety in a file already), hence
resulting in even better performance due to even fewer resources and
CPU cycles being consumed.
Randolf Richardson - rand...@inter-corporate.com
Inter-Corporate Computer & Network Services, Inc.
Beautiful British Columbia, Canada
http://www.inter-corporate.com/
