Re: [Wikitech-l] Reason for actionthrottledtext API blocking?

[Gergő Tisza]

On Mon, Jan 13, 2020 at 6:57 AM Baskauf, Steven James <
steve.bask...@vanderbilt.edu> wrote:

> The other thing that is different about what I'm doing and what is being
> done by the other user who is not encountering this problem is that I'm
> authenticating directly by establishing a session when the script starts
> (lines 347-349).


As Brad said, you should use OAuth:
https://www.mediawiki.org/wiki/Manual:Pywikibot/OAuth/Wikimedia
It won't help with the throttling, but it's simpler and more secure (and if
you do use PAWS, it should work without any setup).

Eventually I will probably apply for a bot flag, but I doubt that this bot
> will ever be autonomous, so is that really necessary?


While the ultimate authority on thi is always the community of the given
wiki (via its bureaucrats, or a bot approval committee in some cases), IMO
semi-autonomous tools are the ones where a human reviews every edit, which
does not seem to be the case here.
In any case, it is necessary if you want to make several dozen edits a
minute and are not in any other group which has the noratelimit right.


> Would it matter if I used my own account instead of a separate bot account?
>

Depends on the exact limit you've hit, but probably not unless you are an
administrator on that wiki.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Timelines for Parsoid/PHP to replace legacy PHP parser

[Lord_Farin]

Thank you so much for the very extensive reply! That's all I hoped to get,
and so much more. Much appreciated!

On Sun, 12 Jan 2020 at 15:37, Subramanya Sastry 
wrote:

> On 1/12/20 5:33 PM, Lord_Farin wrote:
>
> > Hi Wikitech,
> >
> > I've been catching up on the recent achievements regarding Parsoid/PHP,
> > well done!
>
> Thanks!
>
> The switchover of wikitext engines is going to take some time. I would
> be surprised if we got all the ducks lined up before 18 months from now
> -- we have a bunch of work to do still.
>
> Other details below:
>
> > With WMF sites being migrated, of course non-WMF sites start to creep
> into
> > the picture. As I'm involved in running one of those, I'm curious to know
> > if and how you are going to support this upgrade? I've read about Linter
> > and ParserMigration but I'm not clear on how they fit into the picture.
>
> We built the Linter and ParserMigration extensions to support the
> replacement of HTML4 Tidy with RemexHTML [1]. We anticipate leveraging
> those in our efforts to consolidate behind Parsoid (post-unification
> work) as the default wikitext engine for MediaWiki. We don't quite know
> the specifics yet. My hunch is that this replacement is going to be most
> complex for Wikimedia wikis and expect most 3rd party wikis to have a
> much easier time switching over.
>
> > I'm asking specifically because we are running some custom extensions
> which
> > will probably break with the advent of Parsoid/PHP.
> > At present we are running MW 1.33 on PHP 7.0, but we are not using VE.
> > It would be fine if we as a maintenance team have to invest some (or even
> > considerable) time and effort but I would like to know the size of the
> > endeavour beforehand...
>
> One of the changes that will take some work is how extensions interact
> with the parser (Parsoid in the future). So far, this happens through
> access to the Parser object as well as through parser hooks. However, in
> the Parsoid regime, this model will change. While the details are yet to
> be finalized and we are yet to publish the first draft for review
> (likely in the next couple months), here is how we've been thinking
> about this:
>
> 1. Extensions will no longer have direct access to the parser itself --
> all interaction will be through an API / interface.
>
> 2. Hooks are unlikely to be based on timelines of how wikitext passes
> through the parser, i.e. before something happens, or after something
> happens. We are going to move more towards a pure functional model as
> far as possible. So, as far as extension tags are concerned, they get
> access to the tag source, args, and possibly some other information and
> are expected to return output HTML / a DOM fragment (here they will
> leverage the parser API/interface I mention in 1. above). Most
> extensions that implement custom tags already behave in this manner and
> this simply formalizes that.
>
> 3. Some extensions set parser state and update it across invocations. We
> currently have no intention of supporting that. We are going to look at
> what the underlying need is that is being modeled through side-effects /
> state and will to provide first-class support for that in some manner.
> For example, some (like Cite) use state for enumeration and numbering
> purposes, and this can be done as a post-processing pass on the DOM when
> they get to inspect the "final" DOM. Presumably these global document
> processors are the exception, not the norm. But, statelessness lets us
> process the document in arbitrary order (or even skip processing parts
> of the document by reusing extension/template/media output from previous
> versions of the document), and use the final post-processing step as the
> synchronization step to enforce source-text ordering (like numbering).
>
> We anticipate most extensions are going to need some (hopefully minor)
> changes. If your extension doesn't deal with wikitext itself, the
> changes are probably going to be relatively minor. But, if your
> extension deals with wikitext, then it might need an update in terms of
> how it generates its output (using the ParsoidExtensionAPI interface
> instead of an actual parser object), but once again, this is unlikely to
> be very significant changes. However, if your extension maintains state
> across invocations, then it might need some rethink (as stated in 3.
> above).
>
> As our extension currently does some hacks to get a correct behaviour when
modifying the ToC, I guess that the new model will be a fair bit easier. As
stated, it will no longer be necessary to work with wikitext and instead a
hook at postprocessing level should be sufficient for that. This makes all
the templating also much easier.

In summary I'm looking forward to the exact implementation and while I
expect considerable work, it will be no way near the complexity that has
currently been built; just work migrating from A to B.


> If you want to get a really early look, you can poke around the Parsoid

[Wikitech-l] Wikimedia Technical Talks 2020 -- Open Call

[Sarah R]

2019 was a great year for Wikimedia Technical Talks
! We are looking
forward to a new year and a new line-up of speakers starting in February
2020!

If you have an idea for a Tech Talk (or know someone who does) -- please
propose a talk
!
We'll discuss your idea with you, and if it's a good fit, we'll get you on
this year's schedule.❤📆❤


   - If you are saying to yourself, *"UGH! NO WAY! I HATE PUBLIC
   SPEAKING"* 😐 You are not alone! Many people have a fear of public
   speaking .
   One of the best ways to get past it is through practice!
   - If you are thinking,* "An hour is way too long to talk to a faceless
   audience!!!" 😬 *You are probably right! An hour is a long time to
   listen, too! Let's come up with a length and format that will work for you!
   - If you are thinking, *"I have an idea for a talk, but I don't have the
   time!!!"* ⏰ No problem! We can schedule your talk for a few months from
   now!
   - If you are thinking, *"Ideas? I have no ideas!"* 💡Take a look at some
   past tech talks
   

   for inspiration!


Tech talks are a great way to share your work, refine your ideas for other
professional venues (conferences, papers, etc), and be more visible in the
Wikimedia technical community! 👓 Hope you'll join us this coming year as a
speaker or a viewer!

Please feel free to forward this email to appropriate lists.

Kindly,

Sarah R. Rodlund
Technical Writer, Developer Advocacy

srodl...@wikimedia.org
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Reason for actionthrottledtext API blocking?

[Brad Jorsch (Anomie)]

On Mon, Jan 13, 2020 at 9:57 AM Baskauf, Steven James <
steve.bask...@vanderbilt.edu> wrote:

> 1. The bot account doesn't have a bot flag.
> https://www.wikidata.org/wiki/Wikidata:Requests_for_permissions/Bot/DanmicholoBot
> implies that the lack of a bot flag might be the problem. But the script is
> not an autonomous bot and is still under development, so I didn't think a
> bot flag was necessary under those circumstances.
>

This is relevant only in that the bot group includes the "noratelimit"
right, which bypasses rate limiting in many cases.


> 2. I'm not running the script from paws.wmflabs.org .  Is it a problem
> that the API calls are coming from an unknown IP address? Does that matter?
> 3. I'm not using Pywikibot.  But that shouldn't be necessary.
>

Neither of these should be relevant.

On the other hand, a relevant factor is that only certain actions are
subject to the rate limits, and different actions have different limits.
The other user you discussed it with may have been using different actions
that aren't limited, or that have limits he wasn't hitting.


> The other thing that is different about what I'm doing and what is being
> done by the other user who is not encountering this problem is that I'm
> authenticating directly by establishing a session when the script starts
> (lines 347-349).  The other user apparently somehow doesn't have to
> authenticate when using Pywikibot as long as he's logged in to
> paws.wmflabs.org .  But I haven't dug in to find out how Pywikibot works,
> so I don't really understand how that is possible or whether that's
> important in establishing that the bot is legitimate and not a vandal/spam
> bot.
>

He may be using OAuth. See
https://www.mediawiki.org/wiki/OAuth/For_Developers and
https://www.mediawiki.org/wiki/OAuth/Owner-only_consumers for some details.

-- 
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Reason for actionthrottledtext API blocking?

[Baskauf, Steven James]

I am working on a script to load and update Vanderbilt researcher records in 
Wikidata via the API (see 
https://github.com/HeardLibrary/linked-data/tree/master/publications for 
information and 
https://github.com/HeardLibrary/linked-data/blob/master/publications/process_csv_metadata_full.py
 for the relevant script).

The script is working adequately; however after about 25 writes, I get blocked 
with an actionthrottledtext error with the message " As an anti-abuse measure, 
you are limited from performing this action too many times in a short space of 
time, and you have exceeded this limit.\nPlease try again in a few minutes."  I 
am able to avoid being blocked if I delay three seconds between writes, but a 
delay of only one second results in me still being blocked.

I am observing maxlag (lines 282-313) and am sending a User-Agent header, 
VanderBot/0.1 (steve.bask...@vanderbilt.edu).  I'm making the API calls using 
the bot account https://www.wikidata.org/wiki/User:VanderBot, which does not 
have a bot flag.

I've been talking to another user who is running a script from paws.wmflabs.org 
using Pywikibot without a bot flag and he has not encountered this error.  So I 
am wondering what I am doing wrong that is causing this error and what I can do 
to stop being blocked.  The possible causes I can imagine are:

1. The bot account doesn't have a bot flag.  
https://www.wikidata.org/wiki/Wikidata:Requests_for_permissions/Bot/DanmicholoBot
 implies that the lack of a bot flag might be the problem. But the script is 
not an autonomous bot and is still under development, so I didn't think a bot 
flag was necessary under those circumstances.
2. I'm not running the script from paws.wmflabs.org .  Is it a problem that the 
API calls are coming from an unknown IP address? Does that matter?
3. I'm not using Pywikibot.  But that shouldn't be necessary.

The other thing that is different about what I'm doing and what is being done 
by the other user who is not encountering this problem is that I'm 
authenticating directly by establishing a session when the script starts (lines 
347-349).  The other user apparently somehow doesn't have to authenticate when 
using Pywikibot as long as he's logged in to paws.wmflabs.org .  But I haven't 
dug in to find out how Pywikibot works, so I don't really understand how that 
is possible or whether that's important in establishing that the bot is 
legitimate and not a vandal/spam bot.

Any advice you can give about how to stop being blocked would be appreciated.  
Eventually I will probably apply for a bot flag, but I doubt that this bot will 
ever be autonomous, so is that really necessary?  Would it matter if I used my 
own account instead of a separate bot account?

Steve

--
Steven J. Baskauf, Ph.D.
Data Science and Data Curation Specialist
Jean & Alexander Heard Libraries, Vanderbilt University
Nashville, TN 37235, USA

Office: Eskind Biomedical Library, EMB 111
Phone: (615) 343-4582
https://my.vanderbilt.edu/baskauf/

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] TechCom Radar 2020-01-08

[Kate Chapman]

Hi All,

Here are the minutes from this week's TechCom meeting:

* Has been resolved (was not a RFC, but brought to TechCom’s
attention): Service ownership and maintenance


* expected to go to last call after the next TechCom meeting: Adopt a
modern js framework 

* Reaching out to those that filed the ticket to discuss further:
Standalone regex evaluation service.


* Moved to Last Call ending 22nd January: Hook container with strong
types and DI 

* Discussion on Stable Interface Policy Continues:


You can also find our meeting minutes at


See also the TechCom RFC board
.

If you prefer you can subscribe to our newsletter here


Thanks,
Kate

--
Kate Chapman
Senior Program Manager, Core Platform
Wikimedia Foundation
kchap...@wikimedia.org

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] How to alter the Visual Editor configuration object for tool bar

[Egbe Eugene]

Yes it appears under "insert" -> 'more'.

Isn't there a way to make it visible on the tool bar so it is easy and more
visible to access?

On Mon, Jan 13, 2020 at 12:08 PM Robert Vogel  wrote:

> Hi!
>
> have you tried this?
>
> https://www.mediawiki.org/wiki/VisualEditor/Gadgets/Add_a_tool
>
> --
> Robert
> 
> Von: Wikitech-l  im Auftrag von
> Egbe Eugene 
> Gesendet: Montag, 13. Januar 2020 10:21
> An: Wikimedia developers 
> Betreff: [Wikitech-l] How to alter the Visual Editor configuration object
> for tool bar
>
> Hi All,
>
> How do I access the  configuration object for the main toolbar of
> VisualEditor and add a new option to this to represent a new tool? below is
> the image of the tool bar I am referring to.
>
>  Thanks,
> --
> Eugene233
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] How to alter the Visual Editor configuration object for tool bar

[Robert Vogel]

Hi!

have you tried this?

https://www.mediawiki.org/wiki/VisualEditor/Gadgets/Add_a_tool

--
Robert

Von: Wikitech-l  im Auftrag von Egbe 
Eugene 
Gesendet: Montag, 13. Januar 2020 10:21
An: Wikimedia developers 
Betreff: [Wikitech-l] How to alter the Visual Editor configuration object for 
tool bar

Hi All,

How do I access the  configuration object for the main toolbar of
VisualEditor and add a new option to this to represent a new tool? below is
the image of the tool bar I am referring to.

 Thanks,
--
Eugene233
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] How to alter the Visual Editor configuration object for tool bar

[Egbe Eugene]

Hi All,

How do I access the  configuration object for the main toolbar of
VisualEditor and add a new option to this to represent a new tool? below is
the image of the tool bar I am referring to.

 Thanks,
-- 
Eugene233
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l