Ema has submitted this change and it was merged. (
https://gerrit.wikimedia.org/r/378732 )
Change subject: VCL: remove wikiScrape rate limiting
......................................................................
VCL: remove wikiScrape rate limiting
Remove the ad-hoc rate limiting for the wikiScrape User-Agent as we have
now tackled the problem in a more generic way.
Introduce UA-specific rate limiting (varnishTest) in the test version of
our VCL for VTC testing.
Change-Id: I19cd735f1dc277e6a9ec540f1dfb6428017101a3
---
M modules/varnish/files/tests/text/12-rate-limiting.vtc
M modules/varnish/templates/text-frontend.inc.vcl.erb
2 files changed, 11 insertions(+), 9 deletions(-)
Approvals:
Ema: Verified; Looks good to me, approved
diff --git a/modules/varnish/files/tests/text/12-rate-limiting.vtc
b/modules/varnish/files/tests/text/12-rate-limiting.vtc
index 541cc00..8de8aa1 100644
--- a/modules/varnish/files/tests/text/12-rate-limiting.vtc
+++ b/modules/varnish/files/tests/text/12-rate-limiting.vtc
@@ -19,14 +19,14 @@
client c1 {
# First 25 requests are fine
loop 25 {
- txreq -hdr "User-Agent: wikiScrape/0.0.0" -hdr "X-Forwarded-Proto:
https"
+ txreq -hdr "User-Agent: varnishTest" -hdr "X-Forwarded-Proto: https"
rxresp
expect resp.status == 200
expect resp.http.X-Cache ~ "pass"
}
# The 26th request gets rate limited
- txreq -hdr "User-Agent: wikiScrape/0.0.0" -hdr "X-Forwarded-Proto: https"
+ txreq -hdr "User-Agent: varnishTest" -hdr "X-Forwarded-Proto: https"
rxresp
expect resp.status == 429
expect resp.msg == "Too Many Requests"
@@ -36,7 +36,7 @@
delay 5
- txreq -hdr "User-Agent: wikiScrape/0.0.0" -hdr "X-Forwarded-Proto: https"
+ txreq -hdr "User-Agent: varnishTest" -hdr "X-Forwarded-Proto: https"
rxresp
expect resp.status == 200
} -run
diff --git a/modules/varnish/templates/text-frontend.inc.vcl.erb
b/modules/varnish/templates/text-frontend.inc.vcl.erb
index 096c386..6bb6216 100644
--- a/modules/varnish/templates/text-frontend.inc.vcl.erb
+++ b/modules/varnish/templates/text-frontend.inc.vcl.erb
@@ -180,12 +180,6 @@
sub cluster_fe_hit { }
sub cluster_fe_ratelimit {
- if (req.http.User-Agent ~ "^wikiScrape") {
- if (vsthrottle.is_denied("wikiScrape:" + req.http.X-Client-IP,
25, 5s)) {
- return (synth(429, "Too Many Requests"));
- }
- }
-
// Ratelimit miss/pass requests per IP:
// * Excluded for now:
// * all WMF IPs (including labs)
@@ -207,6 +201,14 @@
}
}
}
+
+ <%- if @varnish_testing -%>
+ if (req.http.User-Agent ~ "^varnishTest") {
+ if (vsthrottle.is_denied("varnishTest:" + req.http.X-Client-IP,
25, 5s)) {
+ return (synth(429, "Too Many Requests"));
+ }
+ }
+ <%- end -%>
}
sub cluster_fe_miss {
--
To view, visit https://gerrit.wikimedia.org/r/378732
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I19cd735f1dc277e6a9ec540f1dfb6428017101a3
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Ema <e...@wikimedia.org>
Gerrit-Reviewer: BBlack <bbl...@wikimedia.org>
Gerrit-Reviewer: Ema <e...@wikimedia.org>
Gerrit-Reviewer: jenkins-bot <>
_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
