Hello Wandenberg, Thanks for your reply.
Using proxy_cache_lock, when the second request arrive, it will wait until the object is complete in the cache (or until proxy_cache_lock_timeout expires). But, in many cases, my upstream has a really slow link and NGINX needs more than 30 minutes to download the object. In practice, probably I will see a lote of parallel downloads from the same object. Someone has other idea? Or I'm wrong about proxy_cache_lock? Regards. -- Alex Garzão Projetista de Software Azion Technologies alex.garzao (at) azion.com On Mon, Aug 26, 2013 at 6:58 PM, Wandenberg Peixoto <[email protected]> wrote: > Try to use the proxy_cache_lock configuration, I think this is what you are > looking for. > Don't forget to configure the proxy_cache_lock_timeout to your use case. > > On Aug 26, 2013 6:54 PM, "Alex Garzão" <[email protected]> wrote: >> >> Hello guys, >> >> This is my first post to nginx-devel. >> >> First of all, I would like to congratulate NGINX developers. NGINX is >> an amazing project :-) >> >> Well, I'm using NGINX as a proxy server, with cache enabled. I noted >> that, when two (or more) users trying to download the same object, in >> parallel, and the object isn't in the cache, NGINX download them from >> the upstream. In this case, NGINX creates one connection to upstream >> (per request) and download them to temp files. Ok, this works, but, in >> some situations, in one server, we saw more than 70 parallel downloads >> to the same object (in this case, an object with more than 200 MB). >> >> If possible, I would like some insights about how can I avoid this >> situation. I looked to see if it's just a configuration, but I didn't >> find nothing. >> >> IMHO, I think the best approach is share the temp file. If possible, I >> would like to known your opinions about this approach. >> >> I looked at the code in ngx_http_upstream.c and ngx_http_proxy.c, and >> I'm trying to fix the code to share the temp. I think that I need to >> do the following tasks: >> >> 1) Register the current downloads from upstreams. Probably I can >> address this with a rbtree, where each node has the unique object id >> and a list with downstreams (requests?) waiting for data from the >> temp. >> >> 2) Disassociate the read from upstream from the write to downstream. >> Today, in the ngx_event_pipe function, NGINX reads from upstream, >> writes to temp, and writes to downstream. But, as I can have N >> downstreams waiting data from the same upstream, probably I need to >> move the write to downstream to another place. The only way I think is >> implementing a polling event, but I know that this is incorrect >> because NGINX is event based, and polling waste a lote of CPU. >> >> 3) When I know that there more data in temp to be sent, which function >> I must use? ngx_http_output_filter? >> >> Suggestions will welcome :-) >> >> Thanks people! >> >> -- >> Alex Garzão >> Projetista de Software >> Azion Technologies >> alex.garzao (at) azion.com >> >> _______________________________________________ >> nginx-devel mailing list >> [email protected] >> http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > _______________________________________________ > nginx-devel mailing list > [email protected] > http://mailman.nginx.org/mailman/listinfo/nginx-devel _______________________________________________ nginx-devel mailing list [email protected] http://mailman.nginx.org/mailman/listinfo/nginx-devel
