[cgiapp] Streaming Large Files using PSGI

[Mike Tonks]

For years I've used the handy CAP::Stream

return $self-stream_file( $file );

method and this does a lovely job of chunking the data so large files
are handled without memory issues.  Files could be anything from 10Mb
up to 800Mb ish.

I'm now looking to go 'production' with a PSGI, Starman  Nginx setup.

In PSGI mode we have to set $ENV{'CGI_APP_RETURN_ONLY'} to make
streaming work, because CAP::Stream uses a print direct to STDOUT
(which is a no no for the PSGI spec afaik).

This is bypassed by the env setting, but digging into the internals it
is simply loading the file contents into $output and returning that?

Digging into plack a little, it does support streaming by the return
of a subref:

http://cpansearch.perl.org/src/MIYAGAWA/Plack-1.0004/eg/dot-psgi/echo-stream-sync.psgi

where we can $w-write(..); to the response object inside the chunking
loop (example is not for binary file but I'm assuming it's the same
basic thing).

This is as far as I've got.  Can anyone help / advise?  Maybe patching
CAP::Stream is the way to go - I suppose it should be possible to
detect the running environment and 'do the right thing'.


Regards,

Mike

#  CGI::Application community mailing list  
####
##  To unsubscribe, or change your message delivery options,  ##
##  visit:  http://www.erlbaum.net/mailman/listinfo/cgiapp##
####
##  Web archive:   http://www.erlbaum.net/pipermail/cgiapp/   ##
##  Wiki:  http://cgiapp.erlbaum.net/ ##
####

Re: [cgiapp] Streaming Large Files using PSGI

[Mark Stosberg]

 This is as far as I've got.  Can anyone help / advise?  Maybe patching
 CAP::Stream is the way to go - I suppose it should be possible to
 detect the running environment and 'do the right thing'.

Mike,

My reading of the spec is that you can return a callback instead of a
body, and the Plack server should handle the rest:

https://metacpan.org/module/PSGI#Delayed-Response-and-Streaming-Body

If the server doesn't suppport streaming, a MiddleWare can provide it:

https://metacpan.org/module/Plack::Middleware::BufferedStreaming

I have not needed to do this yet myself with PSGI (but will eventually).

   Mark

#  CGI::Application community mailing list  
####
##  To unsubscribe, or change your message delivery options,  ##
##  visit:  http://www.erlbaum.net/mailman/listinfo/cgiapp##
####
##  Web archive:   http://www.erlbaum.net/pipermail/cgiapp/   ##
##  Wiki:  http://cgiapp.erlbaum.net/ ##
####