Faidon Liambotis has uploaded a new change for review.
https://gerrit.wikimedia.org/r/107364
Change subject: Varnish: don't inadvertently convert 500s to 503s
......................................................................
Varnish: don't inadvertently convert 500s to 503s
Currently, when retry5xx is set in the puppet config, we return(restart)
from vcl_fetch() on 500s, to have Varnish retry the request on a
different backend.
However, we have no protection against doing this indefinitely by
checking req.restart, so we end up doing so, and hitting Varnish's
max_restarts loop detection parameter and have the frontend throw its
own cryptic 503 instead of e.g. a backend (either Varnish or MediaWiki)
display its own 500 or 503.
Fix this by guarding the return(restart) with req.restarts, and
hardcoding 4 in the process. We might want to lower this more in the
future (4 retires sounds like a lot) but for now this should do it.
Change-Id: Ibf358a2b3df2e984947347138856c5c7e1b91a5e
---
M modules/varnish/templates/vcl/wikimedia.vcl.erb
1 file changed, 10 insertions(+), 1 deletion(-)
git pull ssh://gerrit.wikimedia.org:29418/operations/puppet
refs/changes/64/107364/1
diff --git a/modules/varnish/templates/vcl/wikimedia.vcl.erb
b/modules/varnish/templates/vcl/wikimedia.vcl.erb
index 25621e1..1da3391 100644
--- a/modules/varnish/templates/vcl/wikimedia.vcl.erb
+++ b/modules/varnish/templates/vcl/wikimedia.vcl.erb
@@ -305,7 +305,16 @@
}
<% if vcl_config.fetch("retry5xx", "0") == "1" -%>
if (beresp.status >= 500 && beresp.status < 505) {
- return(restart);
+ # Retry the backend request 3 times, then give up and display
+ # the backend's error page, instead of our own.
+ #
+ # Note that max_restarts is 4 by default, so Varnish would
+ # otherwise detect this as a loop and present its own 503.
+ if (req.restarts < 4) {
+ return(restart);
+ } else {
+ return(pass);
+ }
}
<% end -%>
set beresp.grace = 60m;
--
To view, visit https://gerrit.wikimedia.org/r/107364
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ibf358a2b3df2e984947347138856c5c7e1b91a5e
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Faidon Liambotis <fai...@wikimedia.org>
_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
