Re: [Wikimedia-l] Throttling (was: Re: Please can someone put 50p in the meter)

[Platonides]

On 15/10/12 16:15, WereSpielChequers wrote:
> https://www.mediawiki.org/wiki/Manual:Edit_throttling is well worth
> reading, especially the warning that "Many users sharing the same IP
> address could kick in throttling". Which seems a pretty clear indication to
> me that this is working at the IP level and looking at all edits by newbies
> and unregistered editors, rather than treating each member of the workshop
> separately. Once you get to each trainee you find that previewing and
> trying to save again will usually solve the problem, but leave you unable
> to replicate the bug.
> 
> So I think we have found our problem! Now lets see how many months it takes
> to fix it.

That's right. The ip limit applies to both anon and newbie users. The
newbie limit applies by action and user, and the ip limit by action and ip.
So if you have many newbies going out through the same ip, they all
aggregate in the same count.


> Presumably established users of some sort are whitelisted through
> this? (...) But other editors have complained that Cat a Lot doesn't
> work for them and mysteriously hangs or fails,

If you are autoconfirmed, newbie and ip limits don't apply to you.



> One obvious workaround is to use multiple IPs in the same workshop. I think
> the cost of Satellite broadband is only a few hundred quid a year per
> subscription. I've already proposed a subscription for the UK as it would
> enable  people to run editing sessions at big public events such as county
> shows, but it would also help counter this bug.
> 
> WSC

When we whitelist an ip for a workshop, we should also be increasing the
throttling limit for that ip.



___
Wikimedia-l mailing list
Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l

Re: [Wikimedia-l] Throttling (was: Re: Please can someone put 50p in the meter)

[WereSpielChequers]

https://www.mediawiki.org/wiki/Manual:Edit_throttling is well worth
reading, especially the warning that "Many users sharing the same IP
address could kick in throttling". Which seems a pretty clear indication to
me that this is working at the IP level and looking at all edits by newbies
and unregistered editors, rather than treating each member of the workshop
separately. Once you get to each trainee you find that previewing and
trying to save again will usually solve the problem, but leave you unable
to replicate the bug.

So I think we have found our problem! Now lets see how many months it takes
to fix it.

One obvious workaround is to use multiple IPs in the same workshop. I think
the cost of Satellite broadband is only a few hundred quid a year per
subscription. I've already proposed a subscription for the UK as it would
enable  people to run editing sessions at big public events such as county
shows, but it would also help counter this bug.

WSC

--
>
> Message: 1
> Date: Mon, 15 Oct 2012 10:30:25 +0200
> From: "Federico Leva (Nemo)" 
> To: Wikimedia Mailing List 
> Cc: Wikimedia developers 
> Subject: Re: [Wikimedia-l] Throttling (was: Re: Please can someone put
> 50p in the meter)
> Message-ID: <507bc9a1.7040...@gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> WereSpielChequers, 15/10/2012 09:56:
> > 60 edits a minute sounds high, and probably faster than most of these
> > sessions run at, but not if it is as I suspect, calculated every few
> > seconds.
>
> It's not, as far as I can see. This is how it works:
> <https://www.mediawiki.org/wiki/Manual:$wgRateLimits> (someone please
> expand it otherwise).
> And these are all the existing limits:
> <
> https://gerrit.wikimedia.org/r/gitweb?p=operations/mediawiki-config.git;a=blob;f=wmf-config/InitialiseSettings.php;h=f814f3b46e996d6cb33d64c43965e807dfaec810;hb=HEAD#l6437
> >
> Does Andrew's experience not fit with this?
>
> > So if the tutor says "all save now" and ten people hit enter
> > simultaneously the attempted editing rate is briefly rather more than 1
> per
> > second - hence the throttle kicks in and the tutorial collapses in chaos
> > with several students getting throttling errors at the same time. It
> would
> > be nice to think that the WiFi we used was going through the same IP as
> the
> > rest of the British library and that we merely lifted the normal editing
> > rate above 60 edits a minute, but I suspect that the rate is calculated
> > rather more frequently than every minute.
> >
> > Presumably established users of some sort are whitelisted through this?
> If
> > so it could explain a longstanding Cat a Lot problem. I frequently use
> Cat
> > a lot to categorise images on Commons and my personal editing rate there
> > has gone far above 60 edits a minute, however I'm pretty sure I'd be on
> any
> > commons whitelist. But other editors have complained that Cat a Lot
> doesn't
> > work for them and mysteriously hangs or fails, Is it possible that this
> > throttling feature could be  the cause of that problem as well?
>
> noratelimit circumvents all such limits, but on Commons only the
> standard groups plus account creators have it, and you're just
> autopatrolled.
> The only group having serious throttling problems in the past were
> rollbackers on en.wiki; it shouldn't be too hard for Commons to add
> noratelimit via some group, if that's a problem.
>
> > If so perhaps it would be a good idea to analyse some of the recent
> > incidents where this feature has kicked in, see how often it disrupts
> > goodfaith editing and how often it disrupts badfaith editing that
> wouldn't
> > have triggered the edit filter. Maybe this was once a net benefit, but
> with
> > the edit filter dealing with most badfaith editing, and increasing
> amounts
> > of editing workshops and tools like Catalot, perhaps this feature has
> > transitioned from net positive to net negative? Alternatively could we
> have
> > a process where we can whitelist the IP Addresses of places where we are
> > running training sessions, and put  note on
> >
> http://commons.wikimedia.org/wiki/MediaWiki_talk:Gadget-Cat-a-lot.jsexplaining
> > how to spot if your editing has been throttled and how to get
> > yourself Whitelisted
>
> Rate limits have never been a problem with some minimal preparation:
> <https://www.mediawiki.org/wiki/Help:Mass_account_creation> (in 6-7
> years of WMIT workshops, I've never heard of big problems with this).
>

Re: [Wikimedia-l] Throttling (was: Re: Please can someone put 50p in the meter)

[Andrew Gray]

On 15 October 2012 09:30, Federico Leva (Nemo)  wrote:
> WereSpielChequers, 15/10/2012 09:56:
>
>> 60 edits a minute sounds high, and probably faster than most of these
>> sessions run at, but not if it is as I suspect, calculated every few
>> seconds.
>
> It's not, as far as I can see. This is how it works:
>  (someone please expand
> it otherwise).
> And these are all the existing limits:
> 
> Does Andrew's experience not fit with this?

These limits confuse me a bit, I have to admit. The key one seems to be:

'edit' => array(
'ip' => array( 8, 60 ),
'newbie' => array( 8, 60 ),

but per the manual, "ip" only applies to "each anon and recent
account", and "newbie" applies to "each recent account" - surely
"each" means the rate-limiting should be applied to the accounts
individually, rather than being triggered by them all coming from the
same location?

http://www.mediawiki.org/wiki/Manual:Edit_throttling suggests it can
also be configured as something on the enwiki edit filters, but I've
had a look at those and couldn't immediately see one that seems to do
this.

> Rate limits have never been a problem with some minimal preparation:
>  (in 6-7 years of
> WMIT workshops, I've never heard of big problems with this).

I want to emphasise again that I've pretty much never had problems
with account creation rate limiting - everyone attending a workshop is
asked to create an account as part of a little bit of homework three
days earlier - it's only ever been edit throttling that's an issue.

-- 
- Andrew Gray
  andrew.g...@dunelm.org.uk

___
Wikimedia-l mailing list
Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l

Re: [Wikimedia-l] Throttling (was: Re: Please can someone put 50p in the meter)

[Federico Leva (Nemo)]

WereSpielChequers, 15/10/2012 09:56:

60 edits a minute sounds high, and probably faster than most of these
sessions run at, but not if it is as I suspect, calculated every few
seconds.


It's not, as far as I can see. This is how it works: 
 (someone please 
expand it otherwise).

And these are all the existing limits:

Does Andrew's experience not fit with this?


So if the tutor says "all save now" and ten people hit enter
simultaneously the attempted editing rate is briefly rather more than 1 per
second - hence the throttle kicks in and the tutorial collapses in chaos
with several students getting throttling errors at the same time. It would
be nice to think that the WiFi we used was going through the same IP as the
rest of the British library and that we merely lifted the normal editing
rate above 60 edits a minute, but I suspect that the rate is calculated
rather more frequently than every minute.

Presumably established users of some sort are whitelisted through this? If
so it could explain a longstanding Cat a Lot problem. I frequently use Cat
a lot to categorise images on Commons and my personal editing rate there
has gone far above 60 edits a minute, however I'm pretty sure I'd be on any
commons whitelist. But other editors have complained that Cat a Lot doesn't
work for them and mysteriously hangs or fails, Is it possible that this
throttling feature could be  the cause of that problem as well?


noratelimit circumvents all such limits, but on Commons only the 
standard groups plus account creators have it, and you're just 
autopatrolled.
The only group having serious throttling problems in the past were 
rollbackers on en.wiki; it shouldn't be too hard for Commons to add 
noratelimit via some group, if that's a problem.



If so perhaps it would be a good idea to analyse some of the recent
incidents where this feature has kicked in, see how often it disrupts
goodfaith editing and how often it disrupts badfaith editing that wouldn't
have triggered the edit filter. Maybe this was once a net benefit, but with
the edit filter dealing with most badfaith editing, and increasing amounts
of editing workshops and tools like Catalot, perhaps this feature has
transitioned from net positive to net negative? Alternatively could we have
a process where we can whitelist the IP Addresses of places where we are
running training sessions, and put  note on
http://commons.wikimedia.org/wiki/MediaWiki_talk:Gadget-Cat-a-lot.jsexplaining
how to spot if your editing has been throttled and how to get
yourself Whitelisted


Rate limits have never been a problem with some minimal preparation: 
 (in 6-7 
years of WMIT workshops, I've never heard of big problems with this).


Nemo

___
Wikimedia-l mailing list
Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l

Re: [Wikimedia-l] Throttling (was: Re: Please can someone put 50p in the meter)

[WereSpielChequers]

Hi John,

60 edits a minute sounds high, and probably faster than most of these
sessions run at, but not if it is as I suspect, calculated every few
seconds.  So if the tutor says "all save now" and ten people hit enter
simultaneously the attempted editing rate is briefly rather more than 1 per
second - hence the throttle kicks in and the tutorial collapses in chaos
with several students getting throttling errors at the same time. It would
be nice to think that the WiFi we used was going through the same IP as the
rest of the British library and that we merely lifted the normal editing
rate above 60 edits a minute, but I suspect that the rate is calculated
rather more frequently than every minute.

Presumably established users of some sort are whitelisted through this? If
so it could explain a longstanding Cat a Lot problem. I frequently use Cat
a lot to categorise images on Commons and my personal editing rate there
has gone far above 60 edits a minute, however I'm pretty sure I'd be on any
commons whitelist. But other editors have complained that Cat a Lot doesn't
work for them and mysteriously hangs or fails, Is it possible that this
throttling feature could be  the cause of that problem as well?

If so perhaps it would be a good idea to analyse some of the recent
incidents where this feature has kicked in, see how often it disrupts
goodfaith editing and how often it disrupts badfaith editing that wouldn't
have triggered the edit filter. Maybe this was once a net benefit, but with
the edit filter dealing with most badfaith editing, and increasing amounts
of editing workshops and tools like Catalot, perhaps this feature has
transitioned from net positive to net negative? Alternatively could we have
a process where we can whitelist the IP Addresses of places where we are
running training sessions, and put  note on
http://commons.wikimedia.org/wiki/MediaWiki_talk:Gadget-Cat-a-lot.jsexplaining
how to spot if your editing has been throttled and how to get
yourself Whitelisted

WSC

>
> Message: 6
> Date: Sun, 14 Oct 2012 11:36:03 -0400
> From: John 
> To: Wikimedia Mailing List 
> Subject: Re: [Wikimedia-l] Throttling (was: Re: Please can someone put
> 50p in the meter)
> Message-ID:
> <
> cap-jhpmcqpx1sw5cmfqtwtmxp6mxf2z5nq_pjtp2ry3meq1...@mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> Next time you get said message can you take a screenshot and let us
> know, (it is by default somewhere over 60/edits per minute)
>
> On Sun, Oct 14, 2012 at 11:25 AM, Andrew Gray 
> wrote:
> > On 14 October 2012 15:50, John  wrote:
> >> IPs shouldnt get hit with an edit throttle, (it is really really high)
> >
> > It doesn't seem it! Over the past few months, I've had it triggered
> > four times in an hour in two workshops, and one or two times in
> > perhaps four more. They're not all at the same location or using the
> > same machines, though they were all using institutional networks.
> > These are all new logged-in contributors editing from - presumably -
> > the same IP; I've not had it happen to me in the same sessions, but
> > that might just be chance.
> >
> > These aren't very busy networks, however, and I can't imagine there's
> > a vast flood of active editing coming from them at the same time as
> > the workshop...
> >
> > Is it possible to see where this is configured?
> >
> > --
> > - Andrew Gray
> >   andrew.g...@dunelm.org.uk
> >
> > ___
> > Wikimedia-l mailing list
> > Wikimedia-l@lists.wikimedia.org
> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
>
>
___
Wikimedia-l mailing list
Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l

Re: [Wikimedia-l] Throttling (was: Re: Please can someone put 50p in the meter)

[John]

Next time you get said message can you take a screenshot and let us
know, (it is by default somewhere over 60/edits per minute)

On Sun, Oct 14, 2012 at 11:25 AM, Andrew Gray  wrote:
> On 14 October 2012 15:50, John  wrote:
>> IPs shouldnt get hit with an edit throttle, (it is really really high)
>
> It doesn't seem it! Over the past few months, I've had it triggered
> four times in an hour in two workshops, and one or two times in
> perhaps four more. They're not all at the same location or using the
> same machines, though they were all using institutional networks.
> These are all new logged-in contributors editing from - presumably -
> the same IP; I've not had it happen to me in the same sessions, but
> that might just be chance.
>
> These aren't very busy networks, however, and I can't imagine there's
> a vast flood of active editing coming from them at the same time as
> the workshop...
>
> Is it possible to see where this is configured?
>
> --
> - Andrew Gray
>   andrew.g...@dunelm.org.uk
>
> ___
> Wikimedia-l mailing list
> Wikimedia-l@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l

___
Wikimedia-l mailing list
Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l

Re: [Wikimedia-l] Throttling (was: Re: Please can someone put 50p in the meter)

[Andrew Gray]

On 14 October 2012 15:50, John  wrote:
> IPs shouldnt get hit with an edit throttle, (it is really really high)

It doesn't seem it! Over the past few months, I've had it triggered
four times in an hour in two workshops, and one or two times in
perhaps four more. They're not all at the same location or using the
same machines, though they were all using institutional networks.
These are all new logged-in contributors editing from - presumably -
the same IP; I've not had it happen to me in the same sessions, but
that might just be chance.

These aren't very busy networks, however, and I can't imagine there's
a vast flood of active editing coming from them at the same time as
the workshop...

Is it possible to see where this is configured?

-- 
- Andrew Gray
  andrew.g...@dunelm.org.uk

___
Wikimedia-l mailing list
Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l

Re: [Wikimedia-l] Throttling (was: Re: Please can someone put 50p in the meter)

[John]

IPs shouldnt get hit with an edit throttle, (it is really really high)

On Sun, Oct 14, 2012 at 10:04 AM, Andrew Gray  wrote:
> On 14 October 2012 13:59, John  wrote:
>> If you are planning an event it is fairly easy to get your IP address
>> temporarily whitelisted from the account creation throttle. You just
>> need to know your IP address that will be used.
>
> Is it possible to whitelist IPs from the "edit throttle", though? That
> one's the killer, and it's not really possible to workaround.
>
> --
> - Andrew Gray
>   andrew.g...@dunelm.org.uk
>
> ___
> Wikimedia-l mailing list
> Wikimedia-l@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l

___
Wikimedia-l mailing list
Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l

Re: [Wikimedia-l] Throttling (was: Re: Please can someone put 50p in the meter)

[Andrew Gray]

On 14 October 2012 13:59, John  wrote:
> If you are planning an event it is fairly easy to get your IP address
> temporarily whitelisted from the account creation throttle. You just
> need to know your IP address that will be used.

Is it possible to whitelist IPs from the "edit throttle", though? That
one's the killer, and it's not really possible to workaround.

-- 
- Andrew Gray
  andrew.g...@dunelm.org.uk

___
Wikimedia-l mailing list
Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l

Re: [Wikimedia-l] Throttling (was: Re: Please can someone put 50p in the meter)

[John]

If you are planning an event it is fairly easy to get your IP address
temporarily whitelisted from the account creation throttle. You just
need to know your IP address that will be used.

On Sun, Oct 14, 2012 at 8:55 AM, Craig Franklin  wrote:
> Can I second this one, we've run into it occasionally in WMAU outreach
> sessions as well, and it's always fun explaining why it's said "no" to
> someone without a foundation in computers or internet culture.  A brief
> explanation of why it's happened and what to do in order to not lose your
> edit, made in simple language, would be lovely.
>
> We do find that the best way to get around the account creation throttle is
> to get people to create their accounts beforehand.  In a given class,
> there's usually one or two who don't get the message or are unable to do
> it, but they can usually be dealt with by the instructor without triggering
> anything.
>
> Cheers,
> Craig
>
> Date: Sat, 13 Oct 2012 18:33:54 +0100
>> From: Andrew Gray 
>> To: Wikimedia Mailing List ,
>> Philippe Beaudette 
>> Subject: Re: [Wikimedia-l] Please can someone put 50p in the meter
>> Message-ID:
>> > fljhrgck+9ftttqmhsx1cgd+ob50vxtom0+qcjrih...@mail.gmail.com>
>> Content-Type: text/plain; charset=ISO-8859-1
>>
>> I evade account creation by always making them log in first...
>>
>> Periodically, with a roomful of users, we'll get told that an edit has been
>> "throttled"; no further details, I think. It seems to happen with one or at
>> most two editors at a time out of a dozen, but it can happen to different
>> people later on. This happened several times in a couple of weeks in the
>> summer (I only started workshops in June), and then occasionally since -
>> including yesterday. I originally assumed it was related to external-link
>> additions by new users, but I've seen it for no-link sandbox edits as well.
>>
>> My guess is that this entails something to do with checking for multiple
>> edits from the same IP at once, but I don't know if this is actually the
>> reason, or if it can be disabled/whitelisted.
>>
>> (It's the one I give, though! Corrections gratefully appreciated)
>>
>> - Andrew.
>> On 13 Oct 2012 17:25, "Philippe Beaudette"  wrote:
>>
>> > On Sat, Oct 13, 2012 at 9:07 AM, WereSpielChequers
>> >  wrote:
>> > > As it is this combined with the "throttling feature" made for
>> > > quite a bit of disruption to a session where we had ten people having
>> an
>> > > introduction to editing.
>> >
>> > By "throttling feature", do you mean the account creation
>> > restrictions?  If so, you know there are ways around that, right?
>> > Email me offlist, so as not to clutter the list, and I'll give you a
>> > pointer.
>> >
>> > If you mean something different, disregard :)
>> >
>> > pb
>> > ___
>> > Philippe Beaudette
>> > Director, Community Advocacy
>> > Wikimedia Foundation, Inc.
>> >
>> > 415-839-6885, x 6643
>> >
>> > phili...@wikimedia.org
>> >
>> > ___
>> > Wikimedia-l mailing list
>> > Wikimedia-l@lists.wikimedia.org
>> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
>> >
>>
>>
>>
> ___
> Wikimedia-l mailing list
> Wikimedia-l@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l

___
Wikimedia-l mailing list
Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l

[Wikimedia-l] Throttling (was: Re: Please can someone put 50p in the meter)

[Craig Franklin]

Can I second this one, we've run into it occasionally in WMAU outreach
sessions as well, and it's always fun explaining why it's said "no" to
someone without a foundation in computers or internet culture.  A brief
explanation of why it's happened and what to do in order to not lose your
edit, made in simple language, would be lovely.

We do find that the best way to get around the account creation throttle is
to get people to create their accounts beforehand.  In a given class,
there's usually one or two who don't get the message or are unable to do
it, but they can usually be dealt with by the instructor without triggering
anything.

Cheers,
Craig

Date: Sat, 13 Oct 2012 18:33:54 +0100
> From: Andrew Gray 
> To: Wikimedia Mailing List ,
> Philippe Beaudette 
> Subject: Re: [Wikimedia-l] Please can someone put 50p in the meter
> Message-ID:
>  fljhrgck+9ftttqmhsx1cgd+ob50vxtom0+qcjrih...@mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
>
> I evade account creation by always making them log in first...
>
> Periodically, with a roomful of users, we'll get told that an edit has been
> "throttled"; no further details, I think. It seems to happen with one or at
> most two editors at a time out of a dozen, but it can happen to different
> people later on. This happened several times in a couple of weeks in the
> summer (I only started workshops in June), and then occasionally since -
> including yesterday. I originally assumed it was related to external-link
> additions by new users, but I've seen it for no-link sandbox edits as well.
>
> My guess is that this entails something to do with checking for multiple
> edits from the same IP at once, but I don't know if this is actually the
> reason, or if it can be disabled/whitelisted.
>
> (It's the one I give, though! Corrections gratefully appreciated)
>
> - Andrew.
> On 13 Oct 2012 17:25, "Philippe Beaudette"  wrote:
>
> > On Sat, Oct 13, 2012 at 9:07 AM, WereSpielChequers
> >  wrote:
> > > As it is this combined with the "throttling feature" made for
> > > quite a bit of disruption to a session where we had ten people having
> an
> > > introduction to editing.
> >
> > By "throttling feature", do you mean the account creation
> > restrictions?  If so, you know there are ways around that, right?
> > Email me offlist, so as not to clutter the list, and I'll give you a
> > pointer.
> >
> > If you mean something different, disregard :)
> >
> > pb
> > ___
> > Philippe Beaudette
> > Director, Community Advocacy
> > Wikimedia Foundation, Inc.
> >
> > 415-839-6885, x 6643
> >
> > phili...@wikimedia.org
> >
> > ___
> > Wikimedia-l mailing list
> > Wikimedia-l@lists.wikimedia.org
> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
> >
>
>
>
___
Wikimedia-l mailing list
Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l