OK, I leave you to it. However, asynchronously spawning subprocesses *will* allow you to parallelise the process. I'd call it design, rather than a workaround, but there you go (:
Steve On Sun, 2013-05-26 at 22:38 -0400, B.R. wrote: > One way or another, even if an external script is called, PHP will > need to wait for the scripts completion, making the parallelization > impossible or at least useless (since, to wait for a return code of an > external script is still blocking). > > > I am not trying to find a workaround, I need to know how the > fastcgi_reload_timeout works (if I understood it properly), if I > properly disabled PHP buffering for my example case and how eventually > to control those timeouts. > > I'd like to address the central problem here, not closing my eyes on > it. > > > --- > B. R. > > > On Sun, May 26, 2013 at 10:24 PM, Steve Holdoway > <[email protected]> wrote: > Surely, you're still serialising the transfer with a loop? > > On Sun, 2013-05-26 at 22:11 -0400, B.R. wrote: > > Thanks for your answer. > > > > I didn't go into specifics because my problem doesn't rely > at the > > application-level logic. > > > > What you describe is what my script does already. > > > > > > However in this particular case I have 16 files weighting > each a few > > MB which need to be transfered back at once. > > > > > > PHP allocates 30s for each loop turn (far enough to copy the > file + > > echo some output message about successes/failed completion). > > > > Nginx cuts the execution avec fastcgi_read_timeout time even > with my > > efforts to cut down any buffering on PHP side (thus forcing > the output > > to be sent to Nginx to reinitialize the timeout counter). > > > > That Nginx action is the center of my attention right now. > How can I > > get read of it in a scalable fashion (ie no > fastcgi_read_time = > > 9999999) ? > > --- > > B. R. > > > > > > > > > > On Sun, May 26, 2013 at 9:46 PM, Steve Holdoway > > <[email protected]> wrote: > > Write a script that lists the remote files, then > checks for > > the > > existence of the file locally, and copy it if it > doesn't > > exist? That way > > no internal loop is used - use a different exit code > to note > > whether > > there was one copied, or there were none ready. > > > > That way you scale for a single file transfer. > There's nothing > > to be > > gained from looping internally - well > performance-wise that > > is. > > > > Steve > > > > On Sun, 2013-05-26 at 21:31 -0400, B.R. wrote: > > > No ideas? > > > > > > --- > > > B. R. > > > > > > > > > On Sat, May 25, 2013 at 1:01 PM, B.R. > > <[email protected]> wrote: > > > Hello, > > > > > > > > > I am trying to understand how > fastcgi_read_timout > > works in > > > Nginx. > > > > > > > > > Here is what I wanna do: > > > > > > I list files (few MB each) on a distant > place which > > I copy one > > > by one (loop) on the local disk through > PHP. > > > > > > I do not know the amount of files I need > to copy, > > thus I do > > > not know the total amount of time I need > for the > > script to > > > finish its execution. What I know is that > I can > > ensure is a > > > processing time limit per file. > > > > > > I would like my script not to be > forcefully > > interrupted by > > > either sides (PHP or Nginx) before > completion. > > > > > > > > > > > > What I did so far: > > > > > > - PHP has a 'max_execution_time' of 30s > (default?). > > In the > > > loop copying files, I use the > set_time_limit() > > procedure to > > > reinitialize the limit before each file > copy, hence > > each file > > > processing has 30s to go: way enough! > > > > > > > > > - The problem seems to lie on the Nginx > side, with > > the > > > 'fastcgi_read_timeout' configuration > entry. > > > > > > I can't ensure what maximum time I need, > and I would > > like not > > > to use way-off values such as 2 weeks or 1 > year > > there. ;o) > > > > > > What I understood from the documentation > is that the > > timeout > > > is reinitialized after a successful read: > am I > > right? > > > > > > > > > The challenge is now to cut any buffering > occurring > > on the PHP > > > side and let Nginx manage it (since the > buffering > > will occur > > > after content is being read from the > backend). Here > > is what I > > > did: > > > > > > * PHP's zlib.output_compression is > deactivated by > > default in > > > PHP > > > > > > * I deactivated PHP's output_buffering > (default is > > 4096 bytes) > > > > > > * I am using the PHP flush() procedure at > the end of > > each > > > iteration of the copying loop, after a > message is > > written to > > > the output > > > > > > > > > > > > Current state: > > > > > > * The script seems to still be cut after > the > > expiration of the > > > 'fastcgi_read_timout' limit (confirmed by > the error > > log entry > > > 'upstream timed out (110: Connection timed > out) > > while reading > > > upstream') > > > > > > * The PHP loop is entered several times > since > > multiple files > > > have been copied > > > > > > * The output sent to the browser is cut > before any > > output from > > > the loop appears > > > > > > > > > It seems that there is still some unwanted > buffering > > on the > > > PHP side. > > > > > > I also note that the PHP's flush() > procedure doesn't > > seem to > > > work since the output in the browser > doesn't contain > > any > > > message written after eahc file copy. > > > > > > > > > Am I misunderstanding something about > Nginx here > > (especially > > > about the 'fastcgi_read_timeout' > directive)? > > > > > > Have you any intel/piece of advice on hte > matter? > > > > > > Thanks, > > > > > > --- > > > B. R. > > > > > > > > > > > _______________________________________________ > > > nginx mailing list > > > [email protected] > > > http://mailman.nginx.org/mailman/listinfo/nginx > > > > -- > > Steve Holdoway BSc(Hons) MNZCS > <[email protected]> > > http://www.greengecko.co.nz > > MSN: [email protected] > > Skype: sholdowa > > > > _______________________________________________ > > nginx mailing list > > [email protected] > > http://mailman.nginx.org/mailman/listinfo/nginx > > > > > > _______________________________________________ > > nginx mailing list > > [email protected] > > http://mailman.nginx.org/mailman/listinfo/nginx > > -- > Steve Holdoway BSc(Hons) MNZCS <[email protected]> > http://www.greengecko.co.nz > MSN: [email protected] > Skype: sholdowa > > _______________________________________________ > nginx mailing list > [email protected] > http://mailman.nginx.org/mailman/listinfo/nginx > > > > _______________________________________________ > nginx mailing list > [email protected] > http://mailman.nginx.org/mailman/listinfo/nginx -- Steve Holdoway BSc(Hons) MNZCS <[email protected]> http://www.greengecko.co.nz MSN: [email protected] Skype: sholdowa _______________________________________________ nginx mailing list [email protected] http://mailman.nginx.org/mailman/listinfo/nginx
