[Wikitech-l] Scoring Platform Team update

[Aaron Halfaker]

Hey folks!

I just posted a new update to the blog.   This update covers roughly the
last month.

https://phabricator.wikimedia.org/phame/post/view/58/status_update_july_11th_2017/

As of July 1st, we are officially the Scoring Platform team. We're
welcoming Adam Wight to the team officially.  There will be a nice
announcement that we'll post to the Wikimedia Blog in a few days.

The last ~month was very productive, but we had two major production
issues[1,2]. As you will see in the blog post, there's a series of tasks
that address problems that were related to these issues.

Despite dealing with production issues, we've been able to get a very
substantial change to the revscoring library merged. This change will make
accessing information about models (build environment, test statistics,
scoring thresholds, etc.) much easier. This will cause a breaking change in
ORES UI so we'll be making an announcement when we roll it out. Stay tuned.

We've also increased our language and model coverage substantially. We even
built and deployed a totally new type of model to help out French
Wikisource!

See the post for more details :)

1. https://wikitech.wikimedia.org/wiki/Incident_documentation/20170613-ORES
2. https://wikitech.wikimedia.org/wiki/Incident_documentation/20170623-ORES

-Aaron
Principal research scientist
Lead of the Scoring Platform team
Defender of the universe
Eater of toast
Wiki of the media foundation
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] [MediaWiki-announce] MediaWiki 1.29.0-rc.1 now available

[Chad Horohoe]

Hi all,

First, I'd like apologize for the extreme delay in getting 1.29 out the
door. There's been a bunch of rather nasty bugs that have popped right
around the first release candidate. They've mostly been related to database
transactions and job queue support. As both are kind of crucial I've
decided to hold back the release. As of today, all pending blockers to 1.29
have been resolved. But considering the severity, I'd like to get one last
release candidate in. So without further ado

I'd like to announce the immediate availability of MediaWiki 1.29.0-rc.1,
the second release candidate for 1.29.x. Links at the end of the e-mail.

This is not a final release and should not be used for production websites.
As always please do try out the release candidate in a test environment.
It's how we find bugs that didn't surface in initial development :)

Full release notes:
https://phabricator.wikimedia.org/diffusion/MW/browse/REL1_29/RELEASE-NOTES-1.29
https://www.mediawiki.org/wiki/Release_notes/1.29

**
Download:
https://releases.wikimedia.org/mediawiki/1.29/mediawiki-1.29.0-rc.1.tar.gz

Core only (no extensions):
https://releases.wikimedia.org/mediawiki/1.29/mediawiki-core-1.29.0-rc.1.tar.gz.sig

Patch to previous version (1.29.0-rc.0):
https://releases.wikimedia.org/mediawiki/1.29/mediawiki-1.29.0-rc.1.patch.gz

GPG signatures:
https://releases.wikimedia.org/mediawiki/1.29/mediawiki-1.29.0-rc.1.tar.gz.sig
https://releases.wikimedia.org/mediawiki/1.29/mediawiki-core-1.29.0-rc.1.tar.gz.sig
https://releases.wikimedia.org/mediawiki/1.29/mediawiki-1.29.0-rc.1.patch.gz.sig

Public keys:
https://www.mediawiki.org/keys/keys.html

--
Chad Horohoe
___
MediaWiki announcements mailing list
To unsubscribe, go to:
https://lists.wikimedia.org/mailman/listinfo/mediawiki-announce
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Labs and Tool Labs being renamed

[Bryan Davis]

TL;DR:
* "Tool Labs" is being renamed to "Toolforge"
* The name for our OpenStack cluster is changing from "Labs" to "Cloud VPS"
* The prefered term for projects such as Toolforge and Beta Cluster
running on Cloud VPS is "VPS projects"
* "Data Services" is a new collective name for the databases, dumps,
and other curated data sets managed by the Cloud Services team
* "Wiki replicas" is the new name for the private-information-redacted
copies of Wikimedia's production wiki databases
* No domain name changes are scheduled at this time, but we control
wikimediacloud.org, wmcloud.org, and toolforge.org
* The Cloud Services logo will still be the unicorn rampant on a green
field surrounded by the red & blue bars of the Wikimedia Community
logo
* Toolforge and Cloud VPS will have distinct images to represent them
on wikitech and in other web contexts

In February when the formation of the Cloud Services team was
announced [0] there was a foreshadowing of more branding changes to
come:

> This new team will soon begin working on rebranding efforts intended
> to reduce confusion about the products they maintain. This refocus
> and re-branding will take time to execute, but the team is looking
> forward to the challenge.

In May we announced a consultation period on a straw dog proposal [1]
for the rebranding efforts [2][3]. Discussion that followed both on
and off wiki were used to refine the initial proposal [4]. During the
hackathon in Vienna the team started to make changes on Wikitech
reflecting both the new naming and the new way that we are trying to
think about the large suite of services that are offered. Starting
this month, the changes that are planned [5] are becoming more visible
in Phabricator and other locations.

It may come as a surprise to many of you on this list, but many
people, even very active movement participants, do not know what Labs
and Tool Labs are and how they work. The fact that the Wikimedia
Foundation and volunteers collaborate to offer a public cloud
computing service that is available for use by anyone who can show a
reasonable benefit to the movement is a surprise to many. When we made
the internal pitch at the Foundation to form the Cloud Services team,
the core of our arguments were the "Labs labs labs" problem [6] and
this larger lack of awareness for our Labs OpenStack cluster and the
Tool Labs shared hosting/platform as a service product.

The use of the term 'labs' in regards to multiple related-but-distinct
products, and the natural tendency to shorten often used names, leads
to ambiguity and confusion. Additionally the term 'labs' itself
commonly refers to 'experimental projects' when applied to software;
the OpenStack cloud and the tools hosting environments maintained by
WMCS have been viable customer facing projects for a long time. Both
environments host projects with varying levels of maturity, but the
collective group of projects should not be considered experimental or
inconsequential.

[0]: https://lists.wikimedia.org/pipermail/labs-l/2017-February/004918.html
[1]: https://en.wikipedia.org/wiki/Straw_man_proposal
[2]: https://lists.wikimedia.org/pipermail/labs-l/2017-May/005002.html
[3]: https://lists.wikimedia.org/pipermail/wikitech-l/2017-May/088184.html
[4]: 
https://wikitech.wikimedia.org/wiki/User:BryanDavis/Rebranding_Cloud_Services_products
[5]: https://phabricator.wikimedia.org/T168480
[6]: https://wikitech.wikimedia.org/wiki/Labs_labs_labs

Bryan (on behalf of the Wikimedia Cloud Services team)
-- 
Bryan Davis  Wikimedia Foundation
[[m:User:BDavis_(WMF)]] Manager, Cloud Services  Boise, ID USA
irc: bd808v:415.839.6885 x6855

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Discovery Weekly Update for the week starting 2017-07-03

[Chris Koerner]

Hello, This is the Discovery update for last week. Apologies for the
delay in getting it out.

== Discussions ==

=== Search ===
* Created a method for the Kafka consumer to take 'learn to rank'
queries from a queue and run them against ElasticSearch to generate
relevance labels [0]
* Added in the ability to use kafka in our LTRank feature generation
queries and pushing them into ElasticSearch for analysis [1]
* Added ability to extract TF and IDF based features in the
ElasticSearch 'learning to rank' plugin [2]
* A/B test still in progress 'explore similar' links, but we're
running into a few bugs that will be sorted out next week [3]
* Fixed a bug where searching for phrase queries did not highlight
page content [4]

=== Analysis  ===
* Fixed a bug with the sister project snippets and eventlogging [5]
* Finished up analysis for determining what is a reasonable per-IP
ratelimit for maps [6]
* Fixed a minor dashboard bug (splines) [7]

[0] https://phabricator.wikimedia.org/T162059
[1] https://phabricator.wikimedia.org/T162072
[2] https://phabricator.wikimedia.org/T167437
[3] https://phabricator.wikimedia.org/T164856
[4] https://phabricator.wikimedia.org/T167798
[5] https://phabricator.wikimedia.org/T168916
[6] https://phabricator.wikimedia.org/T169175
[7] https://phabricator.wikimedia.org/T169125



Yours,
Chris Koerner
Community Liaison
Wikimedia Foundation

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] 2017-07-12 Scrum of Scrums meeting notes

[Grace Gellerman]

https://www.mediawiki.org/wiki/Scrum_of_scrums/2017-07-12

*= 2017-07-12 =*
contact: https://www.mediawiki.org/wiki/Wikimedia_Engineering

== Callouts ==
* Reading Web found a browser bug that might be interesting to others:
Firefox fires onload twice in certain circumstances
https://phabricator.wikimedia.org/T170018
* RelEng/Performance: 3D thumbnails broken on beta, Multimedia could your
input https://phabricator.wikimedia.org/T170444
** Parsing: Need input about naming of a property in Parsoid's DOM output
for language variants ( see
https://lists.wikimedia.org/pipermail/mediawiki-i18n/2017-July/001129.html )

== Audiences ==
=== Readers ===
 iOS native app 
* Blocked by: none
* Blocking: none
* Updates: 5.5.1 Released. Working on 5.6 (Dark mode, on this day in the
feed) : https://phabricator.wikimedia.org/project/view/2701/

 Android native app 
* Blocked by: n/a
* Blocking: n/a
* Updates: (all ongoing)
** reading list syncing UI work is nearly done; on deck: ZIM support
** Stephen working on cross-platform CSS/JS consolidation
** engineer interviews in progress

 Reading Web 
* Blocked by:
* Blocking:
* Updates:
**Discovered a browser bug that impacts EventLogging (
https://phabricator.wikimedia.org/T170018 ) If your projects are relying on
it you may want to check that.
**Planning to roll out Page previews to all wikis except German and English.
**Minerva and MobileFrontend will be split this week
**Updating Vector's print styles

 Reading Infrastructure 
* Blocked by: N/A
* Blocking: N/A
* Updates:
** Discussions about Summary 2.0 Spec and title properties.
** Measuring payload reduction of MCS transformations with removal of
reference lists.
** working on Reading Lists

 Multimedia 
* Blocked by: N/A
* Blocking: N/A
* Updates
** Beta thumbnails are looking better, but 3D thumbnails are still broken -
could use input from releng/performance about what might be causing this
https://phabricator.wikimedia.org/T170444
** Patch to properly label webm audio files as audio/webm (currently
video/webm)

 Discovery 
* Blocked by: N/A
* Blocking: N/A
* Updates:
** (maps) Ops out this week, ops + developers out next week
** (search) running A/B test on Special:Search "explore similar" feature

=== Community Tech ===
* Blocked by:
* Blocking:
* Updates:
** HTML5 section IDs (https://phabricator.wikimedia.org/T152540) are still
coming
** Preparing CodeMirror and LoginNotify for wider deployment

=== Contributors ===
 Editing 
* Blocked by:
* Blocking:
* Updates:

 Parsing 
* Blocked by:
* Blocking:
* Updates:
** Need input about naming of a property in Parsoid's DOM output for
language variants ( see
https://lists.wikimedia.org/pipermail/mediawiki-i18n/2017-July/001129.html )

 Global Collaboration 
= Collaboration =
* Blocked by:
* Blocking:
* Updates:
** Better UI for Echo notification blacklist feature
** RCFilters
*** Initial implementation of live updates!
*** Bug fixes
** Article Reminder notification
*** Initial implementation behind feature flag

 Contributors Design 
* Blocked by:
* Blocking:
* Updates:


== Technology ==
=== Analytics ===
* Blocked by:
* Blocking:
* Updates:

=== Cloud Services ===
* Blocked by:
* Blocking:
* Updates:

=== Fundraising Tech ===
* Blocked by:
* Blocking:
* Updates:
** Building CentralNotice admin button to clear banner cache for specific
banners / languages: https://phabricator.wikimedia.org/T168673
** Upgrading integration with our main credit card provider:
https://phabricator.wikimedia.org/T163946
** Deployed initial version of pulling 3rd-party mailer data into CiviCRM:
https://phabricator.wikimedia.org/T161758
** Fixing stubborn PayPal Express Checkout / dead session bug:
https://phabricator.wikimedia.org/T167923
** Optimizing thank you email sender job

=== MediaWiki Platform ===
* Blocked by:
* Blocking:
* Updates:

=== Performance ===
* Blocked by:
* Blocking:
* Updates:

=== Release Engineering ===
* Blocked by:
* Blocking:
* Updates:
** 1.29-rc.1 is out, hopefully release out this week (Friday if nothing
comes up)

=== Research ===
* Blocked by:
* Blocking:
* Updates:

=== Scoring Platform ===
* Blocked by:
* Blocking:
* Updates:
** Thank you to the people who helped with the June 23 outage, especially
AKosiaris and SSastry!
***https://wikitech.wikimedia.org/wiki/Incident_documentation/20170623-ORES
***
https://github.com/wiki-ai/revscoring/commit/ab11dc4c30e8b9824949b05b7d12006746d34df7
***
https://github.com/wiki-ai/revscoring/commit/c8d5c3a863059b83647e739fdcb8ce8cc3c2a06a
***
https://github.com/wiki-ai/ores/commit/2657d8a8ab4e8fa517922fcef07a598b01c5db83
***https://phabricator.wikimedia.org/T168965 (Good demo in the last post)
**We're unblocked and unblocking on coordinating the Extension:ORES handoff
with Collaboration, https://phabricator.wikimedia.org/T167911.  Looks like
we're within negotiating range of feature parity.
**Having a conversation about "ethical" AI, 

Re: [Wikitech-l] Tidy will be replaced by RemexHTML on Wikimedia wikis latest by June 2018

[Nicolas Vervelle]

On Wed, Jul 12, 2017 at 4:43 PM, Subramanya Sastry 
wrote:

> On 07/12/2017 01:12 AM, Nicolas Vervelle wrote:
>
> Hi Subbu,
>>
>> Using the localized names, I've found that not all Linter categories are
>> listed in the API result. Is it normal ?
>> For example, on frwiki, Linter reports 3 "mixed-content" errors for "Les
>> Trolls (film)" but this category is not in the API siteinfo call.
>>
>
> Yup.
>
> Parsoid currently has detection for more patterns than are exposed via the
> Linter extension. Mixed content is more informational at this point - it
> will become relevant when we are ready to start nudging markup towards
> being more well-formed / well-balanced than it is now.
>
> This was raised earlier on the Linter Extension talk page as well (
> https://www.mediawiki.org/w/index.php?title=Topic:Tszvb85ccd
> 0thbeo_showPostId=tteddfdly7fin8p6#flow-post-tteddfdly7fin8p6 )
>

Ok, I will only report patterns known by the Linter extension then.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Tidy will be replaced by RemexHTML on Wikimedia wikis latest by June 2018

[Subramanya Sastry]

On 07/12/2017 01:12 AM, Nicolas Vervelle wrote:


Hi Subbu,

Using the localized names, I've found that not all Linter categories are
listed in the API result. Is it normal ?
For example, on frwiki, Linter reports 3 "mixed-content" errors for "Les
Trolls (film)" but this category is not in the API siteinfo call.


Yup.

Parsoid currently has detection for more patterns than are exposed via 
the Linter extension. Mixed content is more informational at this point 
- it will become relevant when we are ready to start nudging markup 
towards being more well-formed / well-balanced than it is now.


This was raised earlier on the Linter Extension talk page as well ( 
https://www.mediawiki.org/w/index.php?title=Topic:Tszvb85ccd0thbeo_showPostId=tteddfdly7fin8p6#flow-post-tteddfdly7fin8p6 
)


Subbu.

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Tidy will be replaced by RemexHTML on Wikimedia wikis latest by June 2018

[Nicolas Vervelle]

Hi Subbu,

Using the localized names, I've found that not all Linter categories are
listed in the API result. Is it normal ?
For example, on frwiki, Linter reports 3 "mixed-content" errors for "Les
Trolls (film)" but this category is not in the API siteinfo call.

Nico

On Wed, Jul 12, 2017 at 8:02 AM, Nicolas Vervelle 
wrote:

>
>
> On Tue, Jul 11, 2017 at 5:05 PM, Subramanya Sastry 
> wrote:
>
>> On 07/11/2017 05:13 AM, Nicolas Vervelle wrote:
>>
>> But I have a few questions / suggestions regarding Linter for the moment:
>>>
>>> - Is is possible to retrieve also the localized names of the Linter
>>> categories and priorities: for example, on frwiki, you can see on the
>>> Linter page [1] that the high priority is translated into "Priorité
>>> haute"
>>> and that self-closed-tag has a user friendly name "Balises
>>> auto-fermantes".
>>> I don't see the localized names in the informations sent by the API
>>> for
>>> siteinfo.
>>>
>>
>> Okay, will file a bug and take a look at this.
>
>
> I used Arlo answer, and I'm getting the localized names from the messages,
> so I can do without the localized names in Linter answers.
>
> Nico
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Tidy will be replaced by RemexHTML on Wikimedia wikis latest by June 2018

[Nicolas Vervelle]

On Tue, Jul 11, 2017 at 5:05 PM, Subramanya Sastry 
wrote:

> On 07/11/2017 05:13 AM, Nicolas Vervelle wrote:
>
> But I have a few questions / suggestions regarding Linter for the moment:
>>
>> - Is is possible to retrieve also the localized names of the Linter
>> categories and priorities: for example, on frwiki, you can see on the
>> Linter page [1] that the high priority is translated into "Priorité
>> haute"
>> and that self-closed-tag has a user friendly name "Balises
>> auto-fermantes".
>> I don't see the localized names in the informations sent by the API
>> for
>> siteinfo.
>>
>
> Okay, will file a bug and take a look at this.


I used Arlo answer, and I'm getting the localized names from the messages,
so I can do without the localized names in Linter answers.

Nico
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l