Re: [PATCH] When cache file is expired, nginx can make a lot of upstream requests.

[Maxim Dounin]

Hello!

On Tue, Mar 31, 2020 at 01:59:31PM +0900, Sangdeuk Kwon wrote:

> Thank you for your explanation
> 
> I have a question.
> Are there any directives for preventing multiple upstream and serving new
> content?
> 
> expect case:
> 1st request: get new content from origin and serve new content.
> 2nd request: serve new content.
> 3rd request: serve new content.
> 
> "proxy_cache_use stale updating" and "proxy_cache_background_update off"
> case:
> 1st request: get new content from origin and serve new content.
> 2nd request: serve stale content.
> finish 1st request's upstream.
> 3rd request: serve new content.
> 
> "proxy_cache_use stale updating" and "proxy_cache_background_update on"
> case:
> 1st request: serve stale content and make subrequest for getting new
> content.
> 2nd request: serve stale content.
> finish 1st request's subrequest.
> 3rd request: serve new content.

As of now the answer is "no", you cannot configure nginx to 
avoid multiple upstream requests when updating cache items and 
avoid serving stale responses at the same time.

There were some attempts in the past to expand proxy_cache_lock to 
updating cache items though.  If you are interested, you may want 
to check this patch:

http://mailman.nginx.org/pipermail/nginx-devel/2018-December/011710.html

-- 
Maxim Dounin
http://mdounin.ru/
___
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel

Re: [PATCH] When cache file is expired, nginx can make a lot of upstream requests.

[Maxim Dounin]

Hello!

On Wed, Mar 25, 2020 at 04:05:37PM +0900, Sangdeuk Kwon wrote:

> # HG changeset patch
> # User Sangdeuk Kwon 
> # Date 1585118845 -32400
> #  Wed Mar 25 15:47:25 2020 +0900
> # Node ID fe5988cefe615bb0f2a8936221c8464a6664916e
> # Parent  0cb942c1c1aa98118076e72e0b89940e85e6291c
> When cache file is expired, nginx can make a lot of upstream requests.
> 
> In cache miss case, nginx makes only one upstream request
> if "proxy_cache_lock" is "on".
> When cache file is expired, nginx can make a lot of upstream requests
> even though "proxy_cache_lock" is "on".

That's expected behaviour.  Citing the "proxy_cache_lock" 
directive description in the documentation 
(http://nginx.org/r/proxy_cache_lock):

: When enabled, only one request at a time will be allowed to 
: populate a new cache element identified according to the 
: proxy_cache_key directive by passing a request to a proxied 
: server.

Note "a new cache element" part.  To limit load on backends when 
updating existing cache elements, "proxy_cache_use_stale updating" 
can be used.  See http://nginx.org/r/proxy_cache_use_stale for 
details.

> If origin's content is changed, this simultaneous upstream requests
> can make "too long header" error log.

And this claim seems to be incorrect and, even if it is correct, 
irrelevant to the expected behaviour described above.

Summing the above, it is completely unclear from the description 
what your patch tries to do.  And the code seems to be completely 
wrong, too, but it's hard to tell without understanding what it 
is expected to do in the first place.

-- 
Maxim Dounin
http://mdounin.ru/
___
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel