Re: [Wikitech-l] Uploading new versions of other people's patches to gerrit

[Isarra Yos]

On 09/04/2019 19:40, Tyler Cipriani wrote:

Hello,

On 19-04-09 17:50:12, Isarra Yos wrote:
To clarify, I get what you're trying to do, but there has got to be a 
better solution besides denying key features to and thus impairing 
long-term contributors, because disabling this for everyone (but 
apparently WMDE (?!)) does exactly that. On the wikis, for instance, 
we have specific groups for this sort of thing (rollback, file 
moving, bypass captcha, bypass rate limits), to let established users 
do the things they need to do, without it being allowed of absolutely 
everyone from the start, and without requiring them to be admins, 
either, to do it. Perhaps actually using one of our more general 
existing groups for this here would make sense?


Bawolff has suggested Trusted-Contributors as a group that might make 
sense to add. That seems sane to me, so I've added that group in 
addition to the list above. Hopefully that unblocks your work.


I agree: having specific groups that are granted specific permissions 
that allow established users to continue their work unobstructed 
should be achievable in the near-term.



Thank you. And apparently I was only even added to that group just last 
night in the hopes that it would get around this, so... now that it 
does, great! (For anyone else who needs to be added to this for now, who 
should they be reaching out to?)


Sorry about reacting so unkindly - it was just particularly frustrating 
when it seemed like I was being ignored here, but other groups were 
being resolved (though as I understand it now, apparently specific 
members with access simply added that themselves, go figure).


-I


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Discovery Weekly Update for the week starting 2019-03-25

[Chris Koerner]

Greetings,

This is the weekly update from the Search Platform team for the week
starting 2019-03-25 and 2019-04-01.

As always, feedback and questions are welcome.

== Discussions ==

=== Search ===
* ElasticSearch upgrade to v6:
** incident [0]
*Trey finished a deep dive into the performance of language
identification for cross-wiki searching [1] (example [2]) and
punctuation-related problems, and discovered things are working pretty
well overall, but the Chinese language model is a bit off.
* Erik noticed that the inlabel / incaption keywords should highlight
the label/caption but were not [3]
* David worked on fixing an error code that Elasticsearch 6
nested_path and nested_filter are deprecated [4] and
_retry_on_conflict was deprecated [5]
* We worked on migrating mjolnir to stdout/syslog/cee logging output [6]
* The team worked on upgrade to elasticsearch 6.5.4 for cirrus / codfw
(specifically) [7] and for eqiad [8]
* Erik worked on the implementation and testing of glent m0
integration with wmf infrastructure [9]
* David did a lot of work to update the mw-config to use the psi
elastic clusters [10]
* David found that the auto_generate_phrase_queries is deprecated and
ineffective [11]
* The team fixed an old bug where we were getting fatal errors -
"cannot perform this operation with arrays" from
CirrusSearch/ElasticaWrite (using JobQueueDB) [12]
* Gehel worked to make spicerack more robust when unfreezing writes to
elasticsearch / cirrus [13] as well as creating a cookbook to reset
frozen write state on elasticsearch / cirrus [14]
* Stas moved WikibaseLexeme search code to WikibaseLexemeCirrusSearch
extension [15]
* We noticed that Elasticsearch indices went read-only, causing a huge lag [16]
* We also saw where search exceptions handling was printing response
information on the screen [17]
* The team fixed an issue where mwgrep was not working [18]
* We also fixed an issue where Elasticsearch 6 needed to silence
deprecation warnings to avoid logspam [19]
* We needed to create an extra elasticsearch clusters in the beta cluster [20]
* We also needed some alerts so we know if mjolnir starts misbehaving [21]
* We also converted check_elasticsearch.py icinga plugin to py3 [22]
* We needed to start using local nginx reverse proxy for connections reuse [23]
* The version of curator that we currently use (5.2.0) isn't
compatible with elasticsearch 6. Which causes issues in a few cron on
logtash servers (see blelow). Version 5.6.0 supports both
elasticsearch 5 and 6.so...we updated it [24]
* We also did some cleanup of the reprepro configuration for
elasticsearch-curator [25]
* Getting a centralized way to inspect the content of the search
profiles might be helpful when investigating search behaviors. In the
same vein as other dump debug APIs (mapping/settings/cirrusdoc) David
suggested that we should add a new simple API to dump the profiles
(cirrus-profiles-dump) [26]
* David also found that a call to a member function toArray() on a
non-object (null) in
vendor/ruflin/elastica/lib/Elastica/Client.php:736 and fixed it [27]

[0] 
https://wikitech.wikimedia.org/wiki/Incident_documentation/20190327-elasticsearch
report
[1] 
https://www.mediawiki.org/wiki/User:TJones_(WMF)/Notes/Review_of_Language_Identification_in_Production,_with_a_Special_Focus_on_Stupid_Identification_Tricks
[2] 
https://en.wikipedia.org/w/index.php?search=%D0%93%D0%B0%D1%80%D1%80%D0%B8+%D0%9F%D0%BE%D1%82%D1%82%D0%B5%D1%80%D0%B5
[3] https://phabricator.wikimedia.org/T217809
[4] https://phabricator.wikimedia.org/T219266
[5] https://phabricator.wikimedia.org/T219265
[6] https://phabricator.wikimedia.org/T218833
[7] https://phabricator.wikimedia.org/T218878
[8] https://phabricator.wikimedia.org/T218879
[9] https://phabricator.wikimedia.org/T218164
[10] https://phabricator.wikimedia.org/T210381
[11] https://phabricator.wikimedia.org/T219267
[12] https://phabricator.wikimedia.org/T124196
[13] https://phabricator.wikimedia.org/T219640
[14] https://phabricator.wikimedia.org/T219638
[15] https://phabricator.wikimedia.org/T216206
[16] https://phabricator.wikimedia.org/T219364
[17] https://phabricator.wikimedia.org/T216959
[18] https://phabricator.wikimedia.org/T219162
[19] https://phabricator.wikimedia.org/T219269
[20] https://phabricator.wikimedia.org/T213940
[21] https://phabricator.wikimedia.org/T214494
[22] https://phabricator.wikimedia.org/T215439
[23] https://phabricator.wikimedia.org/T215491
[24] https://phabricator.wikimedia.org/T218991
[25] https://phabricator.wikimedia.org/T216235
[26] https://phabricator.wikimedia.org/T218682
[27] https://phabricator.wikimedia.org/T217402



Subscribe to receive on-wiki (or opt-in email) notifications of the
Discovery weekly update.

https://www.mediawiki.org/wiki/Newsletter:Discovery_Weekly

The archive of all past updates can be found on MediaWiki.org:

https://www.mediawiki.org/wiki/Discovery/Status_updates

Interested in getting involved? See tasks marked as "Easy" or
"Volunteer needed" in 

Re: [Wikitech-l] Uploading new versions of other people's patches to gerrit

[Tyler Cipriani]

Hello,

On 19-04-09 17:50:12, Isarra Yos wrote:
To clarify, I get what you're trying to do, but there has got to be a 
better solution besides denying key features to and thus impairing 
long-term contributors, because disabling this for everyone (but 
apparently WMDE (?!)) does exactly that. On the wikis, for instance, 
we have specific groups for this sort of thing (rollback, file moving, 
bypass captcha, bypass rate limits), to let established users do the 
things they need to do, without it being allowed of absolutely 
everyone from the start, and without requiring them to be admins, 
either, to do it. Perhaps actually using one of our more general 
existing groups for this here would make sense?


Bawolff has suggested Trusted-Contributors as a group that might make 
sense to add. That seems sane to me, so I've added that group in 
addition to the list above. Hopefully that unblocks your work.


I agree: having specific groups that are granted specific permissions 
that allow established users to continue their work unobstructed should 
be achievable in the near-term.


-- Tyler


signature.asc
Description: PGP signature
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Uploading new versions of other people's patches to gerrit

[Isarra Yos]

On 09/04/2019 17:53, Greg Grossmeier wrote:

PS: I just saw your follow-up reply. Sadly Gerrit is not mediawiki ;)
so it doesn't have all of the other built in anti-abuse features that
comes with the nice user features.


Most of MediaWiki's anti-abuse features consist of only applying certain 
rights to certain groups. This is, likewise, a certain right. Gerrit, 
likewise, has groups. This right can be attached to groups, as evidenced 
by the fact that it has already been attached to certain groups.


Given that we have no other feasible ways to do the thing the right 
allows, all I am asking is that this right be added to something that 
known volunteers and other trusted contributors can be added to. I 
believe there is even a group called 'trusted-contributors' already.


-I


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Save the date! Wikimedia Technical Conference 2019: November 12-15, 2019

[Rachel Farrand]

Hello everyone!

Please save the dates of November 12-15 for the Wikimedia Technical
Conference 2019 which will be held in Atlanta, Georgia, USA.

We are in the initial stages of forming the program committee and clearly
defining the scope and content for the event.

We will update our event’s MediaWiki page
 as
everything develops — announcing when the theme has been defined and again
when the event registration opens. For now, please add the dates to your
calendar and watch our event page if you are interested.

Any comments, suggestions, or discussion that you would like the program
committee to take into consideration, answer, or discuss can be added to
the talk page or are welcome on this phabricator task.


Thank you!


-- 
Rachel Farrand
Events Program Manager
Technical Collaboration Team
Wikimedia Foundation
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Uploading new versions of other people's patches to gerrit

[Greg Grossmeier]

On Tue, Apr 9, 2019 at 10:38 AM Isarra Yos  wrote:
> Hi. I'm trying to think of a way to put this politely, but I can't, so:
> this is insane.

Please try to remain civil and polite, even when things aren't as you want.

We (Release Engineering, the Security team, and SRE) are all trying to
make the situation better for everyone. Right now we can't undo this.
We hope to have a better solution (opening the right up more than it
is now) soon pending other in-progress work (as Tyler said).

I request your patience as we work through this.

PS: I just saw your follow-up reply. Sadly Gerrit is not mediawiki ;)
so it doesn't have all of the other built in anti-abuse features that
comes with the nice user features.

-- 
Greg Grossmeier
Release Team Manager

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Uploading new versions of other people's patches to gerrit

[Isarra Yos]

On 09/04/2019 17:38, Isarra Yos wrote:

On 09/04/2019 16:44, Tyler Cipriani wrote:

On 19-04-09 05:17:17, Isarra Yos wrote:
This seems to be broken, or something. It's causing problems for 
collaboration. Please fix.


This is the addPatchSet permission in Gerrit.

That particular permission was recently abused, and at that time the 
permission was modified. The current status will remain for at least 
the next week or two while we sort out some other Gerrit problems.


The current status is that "Project Owners" can use addPatchSet for 
any projects they own: this does not necessarily map to +2/-2 
permissions for a given project, but should mostly align.


For any projects in the "mediawiki" hierarchy, the "mediawiki" group 
has the ability to addPatchSet.


After our current problems are resolved, I'd like to talk both with 
the folks impacted and with the security team about finding some 
middle ground between the current status and anyone being able to 
modify any patchset at any time.


Hi. I'm trying to think of a way to put this politely, but I can't, 
so: this is insane.


Please reconsider, or I will likely simply need to put off a lot of my 
work entirely until it is resolved, which is particularly problematic 
as this was the week when I was supposed to properly resume my grant 
work in particular. I do not have ownership on many of the 
repositories that I need to work on, despite those also being among 
the ones most likely to require the sort of back and forth amending 
that this blocks, so this is about as bad for me as having gerrit down 
entirely, as that is pretty much where having to fall back to emailing 
patches puts us. (And when that happened I just threw my arms in the 
air and went to the beach.)


To clarify, I get what you're trying to do, but there has got to be a 
better solution besides denying key features to and thus impairing 
long-term contributors, because disabling this for everyone (but 
apparently WMDE (?!)) does exactly that. On the wikis, for instance, we 
have specific groups for this sort of thing (rollback, file moving, 
bypass captcha, bypass rate limits), to let established users do the 
things they need to do, without it being allowed of absolutely everyone 
from the start, and without requiring them to be admins, either, to do 
it. Perhaps actually using one of our more general existing groups for 
this here would make sense?


-I


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Uploading new versions of other people's patches to gerrit

[Isarra Yos]

On 09/04/2019 16:44, Tyler Cipriani wrote:

On 19-04-09 05:17:17, Isarra Yos wrote:
This seems to be broken, or something. It's causing problems for 
collaboration. Please fix.


This is the addPatchSet permission in Gerrit.

That particular permission was recently abused, and at that time the 
permission was modified. The current status will remain for at least 
the next week or two while we sort out some other Gerrit problems.


The current status is that "Project Owners" can use addPatchSet for 
any projects they own: this does not necessarily map to +2/-2 
permissions for a given project, but should mostly align.


For any projects in the "mediawiki" hierarchy, the "mediawiki" group 
has the ability to addPatchSet.


After our current problems are resolved, I'd like to talk both with 
the folks impacted and with the security team about finding some 
middle ground between the current status and anyone being able to 
modify any patchset at any time.


Hi. I'm trying to think of a way to put this politely, but I can't, so: 
this is insane.


Please reconsider, or I will likely simply need to put off a lot of my 
work entirely until it is resolved, which is particularly problematic as 
this was the week when I was supposed to properly resume my grant work 
in particular. I do not have ownership on many of the repositories that 
I need to work on, despite those also being among the ones most likely 
to require the sort of back and forth amending that this blocks, so this 
is about as bad for me as having gerrit down entirely, as that is pretty 
much where having to fall back to emailing patches puts us. (And when 
that happened I just threw my arms in the air and went to the beach.)


-I


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Uploading new versions of other people's patches to gerrit

[Tyler Cipriani]

On 19-04-09 05:17:17, Isarra Yos wrote:
This seems to be broken, or something. It's causing problems for 
collaboration. Please fix.


This is the addPatchSet permission in Gerrit.

That particular permission was recently abused, and at that time the 
permission was modified. The current status will remain for at least the 
next week or two while we sort out some other Gerrit problems.


The current status is that "Project Owners" can use addPatchSet for any 
projects they own: this does not necessarily map to +2/-2 permissions 
for a given project, but should mostly align.


For any projects in the "mediawiki" hierarchy, the "mediawiki" group has 
the ability to addPatchSet.


After our current problems are resolved, I'd like to talk both with the 
folks impacted and with the security team about finding some middle 
ground between the current status and anyone being able to modify any 
patchset at any time.


-- Tyler


signature.asc
Description: PGP signature
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Uploading new versions of other people's patches to gerrit

[Isarra Yos]

On 09/04/2019 09:21, Bartosz Dziewoński wrote:
Hopefully this will not stay disabled forever, for I also miss the 
feature.


In the meantime, you can use the traditional Git way of generating 
patch files and sending them by email (or perhaps, in a modern twist, 
via Phabricator).


To generate a patch file of the most recent commit:

    git format-patch -1

(that's 'dash one', and it's critical, because otherwise Git will 
generate patch files for ALL commits)


Then upload the generated file to Phabricator or something.

To apply such a patch file and turn it into a commit:

    git am foo.patch

There are some other options for dealing with multiple dependent 
commits, if you ever need them.



The problem is I'm going to need to do that for like one in every five 
commits, and often for multiple patches in a set. And... yes, we /do/ 
wind up with multiple dependent commits at times.


To explain why this comes up so much for me: I am a UX designer. I'm not 
much of a dev. Most of what I do is describe generally what I want and 
then throw it at someone else, and they implement the actual logic, but 
then they pass it back to me to fill in the rest of the interaction 
stuff, because not only would it be incredibly tedious for me to try to 
explain every possible combination users might run into, it probably 
wouldn't even work. Without the code in front of me, without testing it 
and playing through each use case to ensure it actually results in the 
expected behaviour, I am going to miss things.


While it does work in some cases to simply have the logic in one patch 
and the interaction in another, much of what we work on is considered 
stable, where we should not be merging half-complete patches unless we 
absolutely must.


(Note too that this is specifically /my/ use case - this isn't even 
getting into issues with minor changes that are easier to apply than to 
explain, especially when helping newcomers through where it'd be 
needlessly frustrating for them to implement a bunch of minor fixes when 
the core of their patch is in fact good; or when just dealing with 
typos; or when restoring someone's five-year-old abandoned patch because 
wait, we actually do need that now; or...)


Should we just, uh, not be using gerrit, or something? Because that's 
just... complicated.


-I

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] mwoauth.errors.OAuthException in Flask

[Brad Jorsch (Anomie)]

On Tue, Apr 9, 2019 at 10:17 AM David Barratt 
wrote:

> Unfortunately (afaik) there is no way to test the workflow without a
> getting a real consumer.
>

That shouldn't be too much of a problem, since the owner can use the
consumer for testing while it's still in the "proposed" state.


-- 
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] mwoauth.errors.OAuthException in Flask

[Brad Jorsch (Anomie)]

On Tue, Apr 9, 2019 at 6:00 AM Egbe Eugene  wrote:

> *MediaWiki response lacks token information: {b'Consumer is owner-only,  class': [b'"external"
> href="https://www.mediawiki.org/wiki/Help:OAuth/Errors#E010
> ">E010']}*
>
> I am not sure why i got no response token.


You got no response token because you got an error response instead. As
stated at the link in the error message, owner-only consumers are
pre-authorized so the client should be configured with the token key and
secret directly instead of using the authorization endpoints.

I don't know how to do that using Flask; if you need help with that you'd
probably do better to ask in a forum specific to Flask.

-- 
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] mwoauth.errors.OAuthException in Flask

[David Barratt]

With owner-only consumers, you cannot use the grant process:
https://www.mediawiki.org/wiki/OAuth/Owner-only_consumers you can use the
resource directly. When you request an owner-only token, you should get an
access token at that point. Unfortunately (afaik) there is no way to test
the workflow without a getting a real consumer.

On Tue, Apr 9, 2019 at 6:00 AM Egbe Eugene  wrote:

> Hello All,
>
> i am not sure this is the right channel to send this. Nevertheless, I am
> trying to login using OAuth from a flask application and I am getting an
> error which says
>
> *MediaWiki response lacks token information: {b'Consumer is owner-only,  class': [b'"external"
> href="https://www.mediawiki.org/wiki/Help:OAuth/Errors#E010
> ">E010']}*
>
> I am not sure why i got no response token. Please i need some help on this.
>
> Thanks.
> 
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Uploading new versions of other people's patches to gerrit

[MGChecker]

> On 09/04/2019 05:17, Isarra Yos wrote:
>> This seems to be broken, or something. It's causing problems for 
>> collaboration. Please fix.
>
> I now hear that this is intentional for spam/vandalism reasons. This seems 
> unwise. Regardless, where is the documentation about this? What is the 
> recommended method to now collaborate on specific patches?
> 
> Halp.

I agree with the others that this is really problematic for collaboration, 
including people without +2 permissions. We shouldn't remove important Gerrit 
functionality as anti-spam measure. Maybe we can allow this only to users with 
a minimal amount of trust, like the #Trusted-Contributors project membership on 
Phabricator implies?

-MGC
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] mwoauth.errors.OAuthException in Flask

[Egbe Eugene]

Hello All,

i am not sure this is the right channel to send this. Nevertheless, I am
trying to login using OAuth from a flask application and I am getting an
error which says

*MediaWiki response lacks token information: {b'Consumer is owner-only, https://www.mediawiki.org/wiki/Help:OAuth/Errors#E010
">E010']}*

I am not sure why i got no response token. Please i need some help on this.

Thanks.

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Uploading new versions of other people's patches to gerrit

[Daniel Kinzler]

Am 09.04.19 um 11:21 schrieb Bartosz Dziewoński:
> In the meantime, you can use the traditional Git way of generating patch files
> and sending them by email (or perhaps, in a modern twist, via Phabricator).

Waa! what? really?

Can we get this back at least for people who have +2?


-- 
Daniel Kinzler
Principal Software Engineer, Core Platform
Wikimedia Foundation

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Uploading new versions of other people's patches to gerrit

[Bartosz Dziewoński]

On 2019-04-09 07:43, Isarra Yos wrote:

On 09/04/2019 05:17, Isarra Yos wrote:
This seems to be broken, or something. It's causing problems for 
collaboration. Please fix. 


I now hear that this is intentional for spam/vandalism reasons. This 
seems unwise. Regardless, where is the documentation about this? What is 
the recommended method to now collaborate on specific patches?


Hopefully this will not stay disabled forever, for I also miss the feature.

In the meantime, you can use the traditional Git way of generating patch 
files and sending them by email (or perhaps, in a modern twist, via 
Phabricator).


To generate a patch file of the most recent commit:

git format-patch -1

(that's 'dash one', and it's critical, because otherwise Git will 
generate patch files for ALL commits)


Then upload the generated file to Phabricator or something.

To apply such a patch file and turn it into a commit:

git am foo.patch

There are some other options for dealing with multiple dependent 
commits, if you ever need them.



--
Bartosz Dziewoński

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l