[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-07-27 Thread gerritbot
gerritbot added a comment. Change 301374 merged by BBlack: Revert "Revert "Revert "cache_misc: do not deliver expired cached objects""" https://gerrit.wikimedia.org/r/301374TASK DETAILhttps://phabricator.wikimedia.org/T134989EMAIL

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-07-27 Thread gerritbot
gerritbot added a comment. Change 301374 had a related patch set uploaded (by BBlack): Revert "Revert "Revert "cache_misc: do not deliver expired cached objects""" https://gerrit.wikimedia.org/r/301374TASK DETAILhttps://phabricator.wikimedia.org/T134989EMAIL

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-17 Thread ema
ema added a comment. I've upgraded cache_misc to Varnish 4 again. We're now running a patched version of Varnish (4.1.2-1wm5) including the following patches: - 0005-handle-eof-http1.1.patch

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-17 Thread Stashbot
Stashbot added a comment. Mentioned in SAL [2016-05-17T09:02:23Z] upgrading esams cache_misc to varnish 4 (https://phabricator.wikimedia.org/T126206, https://phabricator.wikimedia.org/T134989) TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-17 Thread Stashbot
Stashbot added a comment. Mentioned in SAL [2016-05-17T08:31:56Z] upgrading codfw cache_misc to varnish 4 (https://phabricator.wikimedia.org/T126206, https://phabricator.wikimedia.org/T134989) TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-17 Thread gerritbot
gerritbot added a comment. Change 289148 merged by Ema: Revert "cache_misc: downgrade almost all to varnish3" https://gerrit.wikimedia.org/r/289148 TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-17 Thread gerritbot
gerritbot added a comment. Change 289148 had a related patch set uploaded (by Ema): Revert "cache_misc: downgrade almost all to varnish3" https://gerrit.wikimedia.org/r/289148 TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-16 Thread BBlack
BBlack added a comment. cache_maps cluster switched to the new varnish package today TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: BBlack Cc: jeremyb, Ronarts12, Krenair, Dzahn, GWicke,

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-15 Thread gerritbot
gerritbot added a comment. Change 288817 merged by Ema: Revert 'Allow grace-hits on hit-for-pass objects' https://gerrit.wikimedia.org/r/288817 TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-15 Thread gerritbot
gerritbot added a comment. Change 288817 had a related patch set uploaded (by Ema): Revert 'Allow grace-hits on hit-for-pass objects' https://gerrit.wikimedia.org/r/288817 TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-15 Thread ema
ema added a comment. I haven't been able to reproduce the bug so far after reverting the commit mentioned above. cp1045 and cp3007 are both running fine with an updated version of varnish. Please use cp3007:80 if you want to play around, it will only hit itself and cp1045 to ensure

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-15 Thread ema
ema added a comment. Update: setting default_grace to 0 makes the issue unreproducible. 4.1.0 is not affected, while 4.1.1 is, so I started looking for grace-related commits between the two versions. Looks like the bug was introduced by this one:

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-14 Thread ema
ema added a comment. The issue can be reproduced running two varnishd on the same host, without any VCL. A web server listening on port 80 is also needed, serving any type of file. I've used a 612 bytes HTML page. /usr/sbin/varnishd -a :3128 -b localhost:80 -p default_ttl=5 -p

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-13 Thread BBlack
BBlack added a comment. Current State: - cp3007 and cp1045 are depooled from user traffic, icinga-downtimed for several days, and have puppet disabled. Please do not re-enable puppet on these! They also have confd shut down, and are running custom configs to continue debugging this

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-13 Thread BBlack
BBlack added a comment. I forgot one of our temporary hacks in the list above in https://phabricator.wikimedia.org/T134989#2290254: 4. https://gerrit.wikimedia.org/r/#/c/288656/ - we also enabled a critical small bit here in v4 vcl_hit. I reverted this for now during the varnish3

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-13 Thread gerritbot
gerritbot added a comment. Change 288655 had a related patch set uploaded (by BBlack): cache_misc: downgrade almost all to varnish3 https://gerrit.wikimedia.org/r/288655 TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-13 Thread gerritbot
gerritbot added a comment. Change 288655 merged by BBlack: cache_misc: downgrade almost all to varnish3 https://gerrit.wikimedia.org/r/288655 TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-13 Thread Stashbot
Stashbot added a comment. Mentioned in SAL [2016-05-13T13:13:56Z] depooling cp3007 to try reproducing https://phabricator.wikimedia.org/T134989 TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-13 Thread ema
ema added a comment. In https://phabricator.wikimedia.org/T134989#2292155, @Jonas wrote: > Issue appearing again [...] > < Age: 58 > < X-Cache: cp1045 hit+miss(0), cp3009 hit+miss(0), cp3007 frontend hit(4) On @Gehel's repro from this morning

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-13 Thread Jonas
Jonas added a comment. Issue appearing again curl -v 'https://query.wikidata.org/i18n/en.json' * Hostname was NOT found in DNS cache * Trying 91.198.174.217... * Connected to query.wikidata.org (91.198.174.217) port 443 (#0) * successfully set certificate verify

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-13 Thread Stashbot
Stashbot added a comment. Mentioned in SAL [2016-05-13T10:27:01Z] depooling cp1061 trying to reproduce https://phabricator.wikimedia.org/T134989 TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-13 Thread ema
ema added a comment. Issue reproduced: https://phabricator.wikimedia.org/P3067 TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: ema Cc: Smalyshev, Heather, Nirzar, ZMcCune, ema, Stashbot,

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-13 Thread ema
ema added a comment. In https://phabricator.wikimedia.org/T134989#2290254, @BBlack wrote: > 1. We've got 2x upstream bugfixes applied to our varnishd on cache_misc: https://github.com/varnishcache/varnish-cache/commit/d828a042b3fc2c2b4f1fea83021f0d5508649e50 +

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-12 Thread Smalyshev
Smalyshev added a comment. I don't see any weirdness anymore on query.wikidata.org/ @Jonas ? TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Smalyshev, Heather, Nirzar, ZMcCune,

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-12 Thread BBlack
BBlack added a comment. Has anyone been able to reproduce any of the problems in the tickets merged into here, since roughly the timestamp of the above message? TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-12 Thread BBlack
BBlack added a comment. So we're currently have several experiments in play trying to figure this out: 1. We've got 2x upstream bugfixes applied to our varnishd on cache_misc: https://github.com/varnishcache/varnish-cache/commit/d828a042b3fc2c2b4f1fea83021f0d5508649e50 +

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-12 Thread Stashbot
Stashbot added a comment. Mentioned in SAL [2016-05-12T15:52:43Z] testing file storage on misc eqiad+esams https://phabricator.wikimedia.org/T134989 TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-12 Thread Kghbln
Kghbln added a comment. In https://phabricator.wikimedia.org/T134989#2289372, @BBlack wrote: > due to missing character encoding supposedly, but it's entirely likely it's missing everything and that's just the first thing it notices. I believe that this observation is correct.

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-12 Thread BBlack
BBlack added a comment. In the merged ticket above, it's browser access to status.wm.o, and the browser's getting a 304 Not Modified and complaining about it (due to missing character encoding supposedly, but it's entirely likely it's missing everything and that's just the first thing it

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-12 Thread Stashbot
Stashbot added a comment. Mentioned in SAL [2016-05-12T13:34:23Z] upgrading misc to varnish 4.1.2-1wm4 and wiping caches (https://phabricator.wikimedia.org/T134989) TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-12 Thread gerritbot
gerritbot added a comment. Change 288387 merged by Ema: Add debian/patches/0006-IMS-duplicate-headers-proper-handling.patch https://gerrit.wikimedia.org/r/288387 TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-12 Thread gerritbot
gerritbot added a comment. Change 288387 had a related patch set uploaded (by Ema): Add debian/patches/0006-IMS-duplicate-headers-proper-handling.patch https://gerrit.wikimedia.org/r/288387 TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-12 Thread Jonas
Jonas added a comment. Now I get content-length 0 for query.wikidata.org jonkr@C134:~$ curl -v 'https://query.wikidata.org/' * Hostname was NOT found in DNS cache * Trying 91.198.174.217... * Connected to query.wikidata.org (91.198.174.217) port 443 (#0) * successfully

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-12 Thread Stashbot
Stashbot added a comment. Mentioned in SAL [2016-05-12T08:33:24Z] wiping misc caches once again (https://phabricator.wikimedia.org/T134989) TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To:

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-12 Thread Stashbot
Stashbot added a comment. Mentioned in SAL [2016-05-12T07:51:40Z] upgrading misc ulsfo to varnish 4.1.2-1wm3 and wiping caches (https://phabricator.wikimedia.org/T134989) TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-12 Thread Stashbot
Stashbot added a comment. Mentioned in SAL [2016-05-12T07:47:03Z] upgrading misc codfw to 4.1.2-1wm3 and wiping caches (https://phabricator.wikimedia.org/T134989) TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-12 Thread Stashbot
Stashbot added a comment. Mentioned in SAL [2016-05-12T07:05:38Z] running varnish 4.1.2-1wm3 in misc esams (https://phabricator.wikimedia.org/T134989) TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-12 Thread gerritbot
gerritbot added a comment. Change 288350 merged by Ema: Add debian/patches/0005-handle-eof-http1.1.patch https://gerrit.wikimedia.org/r/288350 TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-12 Thread gerritbot
gerritbot added a comment. Change 288350 had a related patch set uploaded (by Ema): Add debian/patches/0005-handle-eof-http1.1.patch https://gerrit.wikimedia.org/r/288350 TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-11 Thread BBlack
BBlack added a comment. Status update: we've been debugging this off and on all day. It's some kind of bug fallout from cache_misc's upgrade to Varnish 4. It's a very complicated bug, and we don't really understand it yet. We've made some band-aid fixes to VCL for now which should keep

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-11 Thread gerritbot
gerritbot added a comment. Change 288213 merged by BBlack: tlsproxy: use http/1.1 only in conjunction with varnish4 https://gerrit.wikimedia.org/r/288213 TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-11 Thread gerritbot
gerritbot added a comment. Change 288213 had a related patch set uploaded (by BBlack): tlsproxy: use http/1.1 only in conjunction with varnish4 https://gerrit.wikimedia.org/r/288213 TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-11 Thread Gehel
Gehel added a comment. Seems the issue (or a similar one is still happening), I'm trying to reproduce... TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: BBlack, Aklapper, Zppix,

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-11 Thread Gehel
Gehel added a comment. I can't reproduce the issue. Since it was random, I might just be lucky. Closing this anyway for the moment, I'll reopen if I see the issue again. TASK DETAIL https://phabricator.wikimedia.org/T134989 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T134989: WDQS empty response - transfer clsoed with 15042 bytes remaining to read

2016-05-11 Thread BBlack
BBlack added a comment. Assuming there was no transient issue (which became cached) on the wdqs end of things, then this was likely a transient thing from nginx experiments or the cache_misc varnish4 upgrade. I banned all wdqs objects from cache_misc and now your test URL works fine. Can