Re: [Wikitech-l] What happened to our user agent requirements?

I'm not 100% convinced that the UA requirement is helpful, for two reasons: 1) Lots of requests will have default like "PHP" or "Python/urllib" or whatever from the tool they used to build their bot. These aren't helpful either as they contain no of how to get in touch. 2) It's trivial to work

[Wikitech-l] Delays in loading pages on Wikimedia sites

Yesterday and today, I've experienced a number of delays in getting pages to load on Wikimedia sites. My connectivity to other websites is normal, so I think something may be going on internally with Wikimedia connectivity. I recall seeing discussions during Mexico Wikimania about testing

Re: [Wikitech-l] What happened to our user agent requirements?

Specifically, the hypothesis that people are sending "-"? On 1 September 2015 at 12:58, Tomasz Finc wrote: > Let's get a task in phab for this so that we can triage next steps. > I'm curious about this as well. > > --tomasz > > On Tue, Sep 1, 2015 at 9:46 AM, Oliver Keyes

Re: [Wikitech-l] What happened to our user agent requirements?

Tracking the overall issue On Tue, Sep 1, 2015 at 9:59 AM, Oliver Keyes wrote: > Specifically, the hypothesis that people are sending "-"? > > On 1 September 2015 at 12:58, Tomasz Finc wrote: >> Let's get a task in phab for this so that we can triage

Re: [Wikitech-l] What happened to our user agent requirements?

Let's get a task in phab for this so that we can triage next steps. I'm curious about this as well. --tomasz On Tue, Sep 1, 2015 at 9:46 AM, Oliver Keyes wrote: > On 1 September 2015 at 12:42, John wrote: >> Could they be sending a non-standard

Re: [Wikitech-l] What happened to our user agent requirements?

I agree with rate-limiting those without some sort of ID (login or API key). As Oliver said, big (ab)users can massively skew our stats, often by themselves. But hordes of upper middle volume bots (way too high for a human, nowhere near the max for a superstar bot) can have a large cumulative

Re: [Wikitech-l] Delays in loading pages on Wikimedia sites

On Tue, Sep 1, 2015 at 2:13 PM, Pine W wrote: > Yesterday and today, I've experienced a number of delays in getting pages > to load on Wikimedia sites. My connectivity to other websites is normal, so > I think something may be going on internally with Wikimedia connectivity.

Re: [Wikitech-l] Delays in loading pages on Wikimedia sites

OK, thanks. Pine On Tue, Sep 1, 2015 at 12:35 PM, Keegan Peterzell wrote: > On Tue, Sep 1, 2015 at 2:13 PM, Pine W wrote: > > > Yesterday and today, I've experienced a number of delays in getting pages > > to load on Wikimedia sites. My

Re: [Wikitech-l] What happened to our user agent requirements?

Awesome; thanks for the analysis, Krinkle. Do we want to change this behaviour? From my point of view the answer is 'yes, not setting any kind of user agent is a violation of our API etiquette and we should be taking steps to alert people that it is' but if other people have different

Re: [Wikitech-l] What happened to our user agent requirements?

If people aren't capable of following UA guidelines I doubt they're going to follow voluntary login. For what it's worth I absolutely support both rate-limiting and login to get around this. In fact, I would argue that from an analytics point of view rate limiting is probably the most

Re: [Wikitech-l] What happened to our user agent requirements?

On Tue, Sep 1, 2015 at 1:18 PM, Krinkle wrote: > In the past (2012?) these were definitely being blocked. (Ran into it from > time to time on Toolserver) > It seems php file_get_contents('http://...api..' ) is > also working fine now, > without having to

Re: [Wikitech-l] What happened to our user agent requirements?

I've confirmed just now that whatever requirement there was, it doesn't seem to be in effect. Both omitting the header entirely, sending it with empty string, and sending with "-"; – all three result in a response from the MediaWiki API. $ curl -A '' --include -v

Re: [Wikitech-l] New RFC: better JS minification

Ori Livneh wrote: >On Tue, Sep 1, 2015 at 8:43 AM, Jérémie Roquet >wrote: >> Has the RFC been abandonned because of lack of interest? > >Speaking for myself: at the time the RFC was written, I was skeptical that >the benefits would be worth incurring a dependency on an

Re: [Wikitech-l] What happened to our user agent requirements?

On Tue, Sep 1, 2015 at 10:42 PM, Platonides wrote: > Brad Jorsch (Anomie) wrote: >> I wonder if it got lost in the move from Squid to Varnish, or something >> along those lines. > That's likely, given that it was enforced by squid. We could easily add it back in Varnish,

Re: [Wikitech-l] "Try the free Wikipedia app" banners

> It's been discussed previously both on this list and elsewhere, but for > better or worse the Wikimedia Foundation has an entire "Mobile apps" team > that pretty much exclusively works on closed platforms, as I understand > it. They've gone as far as to abandon Gerrit in favor of GitHub. For

Re: [Wikitech-l] New RFC: better JS minification

On Tue, Sep 1, 2015 at 8:49 AM, Ori Livneh wrote: > So +1 for reviving it. > Feel free to, however I myself have neither the time nor inclination to work on this; the components are: * The extension itself: https://github.com/wikimedia/mediawiki-extensions-Minifier *

Re: [Wikitech-l] Phabricator monthly statistics - 2015-08

Wikimedia Phabricator will be soon one year old! On Tue, Sep 1, 2015 at 2:00 AM, wrote: > > Number of accounts created in (2015-08): 288 > Kind of surprised about the fact that we keep having almost ten new Phabricator users every day, I have created a graph at

Re: [Wikitech-l] What happened to our user agent requirements?

On Tue, Sep 1, 2015 at 4:54 PM, Brandon Black wrote: > I really do like the idea of moving towards smarter ratelimiting of > APIs by default, though (and have brought this up in several contexts > recently, but I'm not really aware of whatever past work we've done in > that

Re: [Wikitech-l] What happened to our user agent requirements?

On Tue, Sep 1, 2015 at 5:54 PM, Gergo Tisza wrote: > > > Rate limiting / UA policy enforcement has to be done in Varnish, since API > responses can be cached there and so the requests don't necessarily reach > higher layers (and we wouldn't want to vary on user agent).

Re: [Wikitech-l] "Try the free Wikipedia app" banners

What was the publicising of the campaign prior to its launch? It should be pretty apparent to people with experience within the movement that this would be both entirely novel and pretty controversial. I'd expect some amount of transparency around it (a phabricator ticket is not, in and of

[Wikitech-l] What does "Director of Architecture" mean anyway?

Hi everyone, It's been a while since I've been involved here, and I've got a pretty good excuse. I'm going to avoid going into too much detail, but in short, I was involved in a pretty nasty traffic collision, and have only recently returned to work. It turned out the way that a lot of bike

Re: [Wikitech-l] "Try the free Wikipedia app" banners

Ori Livneh wrote: >We appear to be running a banner campaign on the mobile web site, driving >people to download the mobile app: > >https://en.m.wikipedia.org/wiki/?banner=Aug2015_app_banner_2 >https://en.m.wikipedia.org/wiki/?banner=Aug2015_app_banner_1 > >[...] > >This isn't cool. This isn't us.

Re: [Wikitech-l] What happened to our user agent requirements?

We recently revisited rate limiting in https://phabricator.wikimedia.org/T107934, but came to similar conclusions as reached in this thread: - Limits for weak identifiers like IPs or user agents would (at least initially) need to be high enough to render the limiting borderline useless

Re: [Wikitech-l] What happened to our user agent requirements?

On Wed, Sep 2, 2015 at 1:21 AM, Gabriel Wicke wrote: > On Tue, Sep 1, 2015 at 5:54 PM, Gergo Tisza wrote: >> >> >> Rate limiting / UA policy enforcement has to be done in Varnish, since API >> responses can be cached there and so the requests don't

Re: [Wikitech-l] What happened to our user agent requirements?

On Tue, Sep 01, 2015 at 12:42:35PM -0400, John wrote: > Could they be sending a non-standard header of "-" They could. But if a request comes in without a User-Agent header, the logging pipeline silently translates it into "-". Have fun, Christian P.S.: The relevant configuration (for

[Wikitech-l] Collaboration team reprioritization

For a while now, the Collaboration team has been working on Flow, the structured discussion system. I want to let you know about some changes in that long-term plan. While initial announcements about Flow said that it would be a universal replacement for talk pages, the features that were

Re: [Wikitech-l] What happened to our user agent requirements?

Brad Jorsch (Anomie) wrote: I wonder if it got lost in the move from Squid to Varnish, or something along those lines. That's likely, given that it was enforced by squid. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org

Re: [Wikitech-l] Phabricator monthly statistics - 2015-08

2015. 9. 1. 오후 7:18에 님이 작성: > > > Hi Community Metrics team, > > this is your automatic monthly Phabricator statistics mail. > > Number of accounts created in (2015-08): 288 > Number of active users (any activity) in (2015-08): 831 > Number of task authors in

[Wikitech-l] Phabricator monthly statistics - 2015-08

Hi Community Metrics team, this is your automatic monthly Phabricator statistics mail. Number of accounts created in (2015-08): 288 Number of active users (any activity) in (2015-08): 831 Number of task authors in (2015-08): 481 Number of users who have closed tasks in (2015-08): 258 Number of

[Wikitech-l] Semantic MediaWiki Conference Fall 2015: Call for Participation

Dear users, developers and all people interested in semantic wikis, We are very happy to announce that early bird registration to the 12th Semantic MediaWiki Conference is now open! *Important facts reminder:* * Dates: October 28th to October 30th 2015 (Wednesday to Friday) * Location:

Re: [Wikitech-l] Collaboration team reprioritization

Thanks for the update. Discussion is taking place on Wikimedia-l. Pine On Sep 1, 2015 2:27 PM, "Danny Horn" wrote: > For a while now, the Collaboration team has been working on Flow, the > structured discussion system. I want to let you know about some changes in > that

[Wikitech-l] RFC meeting this week

Hi all, I'm going to take on the RFC review meeting announcement duties for Tim. Discussion topic this week: "RFC: Master/slave datacenter strategy for MediaWiki": I've put the meeting details in Phab's event tool here:

Re: [Wikitech-l] "Try the free Wikipedia app" banners

On Tue, Sep 1, 2015 at 8:30 AM, Ori Livneh wrote: > We appear to be running a banner campaign on the mobile web site, driving > people to download the mobile app: > Just in time! http://techcrunch.com/2015/09/01/death-to-app-install-interstitials/

Re: [Wikitech-l] "Try the free Wikipedia app" banners

Il 02/09/2015 07:39, Matthew Flaschen ha scritto: On 09/01/2015 11:30 AM, Ori Livneh wrote: We appear to be running a banner campaign on the mobile web site, driving people to download the mobile app: https://en.m.wikipedia.org/wiki/?banner=Aug2015_app_banner_2

Re: [Wikitech-l] What happened to our user agent requirements?

On 09/01/2015 10:37 AM, Brion Vibber wrote: > I'm not 100% convinced that the UA requirement is helpful, for two reasons: For those of us who looked for the initial rationale on the UA requirement, the announcement and resulting discussion is at [1]. [1]

Re: [Wikitech-l] "Try the free Wikipedia app" banners

On 09/01/2015 11:30 AM, Ori Livneh wrote: We appear to be running a banner campaign on the mobile web site, driving people to download the mobile app: https://en.m.wikipedia.org/wiki/?banner=Aug2015_app_banner_2 https://en.m.wikipedia.org/wiki/?banner=Aug2015_app_banner_1 Campaign definition:

Re: [Wikitech-l] "Try the free Wikipedia app" banners

On Tue, Sep 1, 2015 at 10:09 PM, Ori Livneh wrote: > Just in time! > http://techcrunch.com/2015/09/01/death-to-app-install-interstitials/ Interstitials are full-page ads where you have to click a link to get to the actual content. These are normal banners. More importantly,

Re: [Wikitech-l] What does "Director of Architecture" mean anyway?

Welcome back. It's good to hear of your recovery. Pine On Sep 1, 2015 8:29 PM, "Rob Lanphier" wrote: > Hi everyone, > > It's been a while since I've been involved here, and I've got a pretty good > excuse. I'm going to avoid going into too much detail, but in short, I was

[Wikitech-l] Discovery Department A/B testing an alternative to prefix search next week

Hi everyone, *tl;dr: Discovery Department to run A/B test comparing new search suggester to prefix search, to see if it can reduce zero results rate.* As I'm sure you're all aware, the search box at the top right of every page on desktop uses prefix

Re: [Wikitech-l] What happened to our user agent requirements?

On Tue, Sep 1, 2015 at 9:24 AM Oliver Keyes wrote: > Is the > blocking of requests absent a user agent simply happening at a > 'higher' stage (in mediawiki itself?) and so not registering with the > varnishes, No, it's not done at the application level. > or is sending

Re: [Wikitech-l] What happened to our user agent requirements?

On 1 September 2015 at 12:41, Chad wrote: > On Tue, Sep 1, 2015 at 9:24 AM Oliver Keyes wrote: > >> Is the >> blocking of requests absent a user agent simply happening at a >> 'higher' stage (in mediawiki itself?) and so not registering with the >>

Re: [Wikitech-l] What happened to our user agent requirements?

On 1 September 2015 at 12:42, John wrote: > Could they be sending a non-standard header of "-" Perfectly possible although also impossible to detect :( > > On Tuesday, September 1, 2015, Chad wrote: > >> On Tue, Sep 1, 2015 at 9:24 AM Oliver

[Wikitech-l] What happened to our user agent requirements?

According to https://meta.wikimedia.org/wiki/User-Agent_policy and the associated mailing list threads, user agent headers are now required (and have been for some time) but on the request log side, we see a lot of requests with the user agent "-" - IOW, an empty field. Is the blocking of requests

Re: [Wikitech-l] What happened to our user agent requirements?

Could they be sending a non-standard header of "-" On Tuesday, September 1, 2015, Chad wrote: > On Tue, Sep 1, 2015 at 9:24 AM Oliver Keyes > wrote: > > > Is the > > blocking of requests absent a user agent simply happening at a >

Re: [Wikitech-l] "Try the free Wikipedia app" banners

Probably T103896 but in that case something was misconfigured (those should only show in Finland). On Tue, Sep 1, 2015 at 8:30 AM, Ori Livneh wrote: > We appear to be running a banner campaign on the mobile web site, driving >

Re: [Wikitech-l] New RFC: better JS minification

Hi everyone, I was running some experiments with UglifyJS lately and so I stumbled upon this RFC which shows that if I've not discovered anything new, we can still get ~15% smaller JS files. 2014-03-19 22:16 GMT+01:00 Max Semenik : > Hi, I'd like to present a new RFC for

[Wikitech-l] "Try the free Wikipedia app" banners

We appear to be running a banner campaign on the mobile web site, driving people to download the mobile app: https://en.m.wikipedia.org/wiki/?banner=Aug2015_app_banner_2 https://en.m.wikipedia.org/wiki/?banner=Aug2015_app_banner_1 Campaign definition:

Re: [Wikitech-l] New RFC: better JS minification

On Tue, Sep 1, 2015 at 8:43 AM, Jérémie Roquet wrote: > Has the RFC been abandonned because of lack of interest? > Speaking for myself: at the time the RFC was written, I was skeptical that the benefits would be worth incurring a dependency on an external service (or the