[Wikitech-l] Group0 on wmf.20, all to wmf.20 tomorrow (was: Re: Upgrade of 1.28.0-wmf.19 to group 1 is on hold)

[Tyler Cipriani]

Group0 wikis (mediawikiwiki, test2wiki, testwiki, testwikidatawiki, and
zerowiki) are running version 1.28.0-wmf.20 of MediaWiki and extensions.
All other wikis are running 1.28.0-wmf.18.

Tomorrow there will be a shortened train schedule in the normal train
deployment window during which 1.28.0-wmf.20 will be pushed to all
wikis.

Any blockers to this plan are tracked on Phabricator[0].

Thank you for all your help and patience while we get the train
schedule[1] back on track!

-- Tyler

[0]. 
[1]. 

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] [Discussion needed] Process for taking over abandoned tools

[Bryan Davis]

In 2015, a phabricator task [0] and RfC discussion on meta [1] were
started to create a process for determining when a tool has been
abandoned by it's original maintainer(s) and how to hand control of
the tool over to interested volunteers. The process stalled out
without resolution.

Our on-wiki communities are still highly dependent on volunteer
developed tools and vulnerable to disruption when the original
developers move on. I have drafted two straw dog [2] policies that
attempt to define fair and workable solutions to the general problem.
The proposals take two different but compatible approaches to solving
the problem of abandonment. The Tool Labs developer community could
choose to adopt either or both policies as protection for the
communities that they serve.

The first policy describes a *right to fork* for all Tool Labs hosted
software. This policy clarifies the existing Tool Labs Open Source and
Open Data requirements and defines a process for requesting access to
code and data that are not already published publicly.

The second policy is a more aggressive *abandoned tool policy* that
describes a process for adding new maintainers to a tool account
(adoption) with a future possibility of removing the original
maintainers (usurpation). This policy is based primarily on the
discussions that happened on Meta in 2015.

Both policies propose creating a new committee of volunteers to
evaluate requests and perform cleanup of sensitive data in the tool
before providing the source code or direct access to the tool account.
This provision is key actually implementing both proposals. Paid
administration and management does not scale any better than paid
editing. To continue to grow and thrive, the Tool Labs developer
community needs to become more active in enforcing and expanding their
own policies. Membership in the committees would require signing the
Wikimedia Foundation's Volunteer NDA [3] to ensure that sensitive data
is handled appropriately. If both polices are adopted the two
committees should be collapsed into a single group with authority to
handle both types of requests.

The straw dog policies are posted on Wikitech:
* https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/Right_to_fork_policy
* https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/Abandoned_tool_policy

Discussion of the particulars of each proposal should happen on their
associated talk pages. As an example it would be appropriate to debate
whether the 14 day non-functional waiting period is too short or too
long on the Abandoned tool policy talk page. Discussion of the process
in general can happen on Meta [1].

I would like discussion to remain open through *2016-10-12* (3 weeks
from date of posting). Following the discussion period I hope to call
for an approval vote of some sort to make the policies official.
Wikitech and Tool Labs do not currently have well defined policies for
establishing consensus, but I'm sure we can collectively come up with
something reasonable.


[0]: https://phabricator.wikimedia.org/T87730
[1]: https://meta.wikimedia.org/wiki/Requests_for_comment/Abandoned_Labs_tools
[2]: https://en.wikipedia.org/wiki/Straw_man_proposal
[3]: https://wikitech.wikimedia.org/wiki/Volunteer_NDA

Bryan
-- 
Bryan Davis  Wikimedia Foundation
[[m:User:BDavis_(WMF)]]  Sr Software EngineerBoise, ID USA
irc: bd808v:415.839.6885 x6855

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] WikiDev17 Collaboration topic (Re: Opening up MediaWiki dev summit in January?)

[C. Scott Ananian]

I edited the topic on wiki (
https://www.mediawiki.org/wiki/WikiDev17/Topic_ideas#Collaboration) a
little bit, to separate out the "Technical Collaboration" and "Editor
Collaboration" pieces a bit.  Both are interesting, although I feel like
the target audiences would diverge greatly.  It would be most interesting
to explore how those subtopics could be profitably combined -- why not
dogfood our own tools to do code review with the same mechanisms we use to
review wikitext diffs? -- but I admit that may well be a step too far.
Perhaps simple juxtaposition at the summit will accomplish the desired
cross-fertilization.

It is perhaps unfortunate that we have an existing team named
"Collaboration".  It creates a whiff of toe-stepping and cookie-licking
danger to have a topic eponymous with an existing team.  The page you cite (
https://www.mediawiki.org/wiki/Collaboration) is actually quite useful in
distinguishing the broad idea of "collaboration" from the actual remit (and
great work!) of the currently-constituted team.

Part of why I suggested this summit topic was that I felt that a proper
response to its challenge would of necessity be *cross team* and *whole
org*.  I think our individual parsing, editing, reading, collaboration, etc
teams do a reasonable job setting their own goals.  But what is missing is
the sort of discussion and decision-making that can occur at a broad
summit, where we reconcile (for example):

* improvements to discussion pages (collaboration team)
* real-time collaborative editing (editing team)
* the introduction of "user groups" backing WikiProjects to the core DB
(mythical "core team")
* broading the notion of "a revision" in core DB (mythical "core team")
* real-time reading (reading team)
* social factors in UX to preserve and strengthen and diversify our
community and stop harassment (community engagement, design teams)
* existing workflow mechanisms and projects (collaboration team, mythical
wikiedu team)
* better diff tools (collaboration team + editing team)
* draft namespace/merge tools (i don't think anyone owns this)

So perhaps naming the topic "Collaboration" is as much a mistake as it
would be to name a topic "Editing", "Discovery", "Reading", etc.---although
I don't have another name to propose---since the goal for a summit topic
should be to identify opportunities to solve problems which have proven
difficult to resolve with only smaller-scale collaboration inside our
existing team structures.  To identify fixed points of consensus upon which
we can, Atlas-like, shift the entire organization. :)
  --scott

On Wed, Sep 21, 2016 at 1:53 AM, Rob Lanphier  wrote:

> Scott suggested the following as one of three suggested topic ideas
> for WikiDev17.  The three ideas:
> 1) Collaboration
> 2) Wikitext Maintenance
> 3) Machine Translation
>
> More inline about "1) Collaboration" below:
>
> On Tue, Sep 20, 2016 at 10:05 AM, C. Scott Ananian
>  wrote:
> > *1. *(A unified vision for) *Collaboration*
> >
> >- Real-time collaboration (not just editing, but chatting, curation,
> >patrolling)
> >- WikiProject enhancements: User groups, finding people to work with,
> >making these first class DB concepts
> >- Civility/diversity/inclusiveness, mechanisms to handle/prevent
> >harassment, vandalism, trolling while working together
> >- Real-time reading -- watching edits occur in real time
> >- Integration with WikiEdu
> >- Broadening notion of "an edit" in DB -- multiple contributors,
> >possibly multiple levels of granularity
> >- Tip-toeing toward "draft"/"merge" models of editing
> >- Better diff tools: refreshed non-wikitext UX, timelines, authorship
> >maps, etc.
>
> I've copied this wholesale into the "Collaboration" area on
> [[WikiDev17/Topic ideas]], and quoted it directly here:
> 
>
> Let's use this thread to focus on this part of Scott's proposal.  A
> lot of these seems in scope for the Wikimedia Collaboration team.
> Does the scope that you're thinking of align with what the team has
> published on their page:
> 
>
> Rob
> (p.s. please feel free to start separate threads with the other parts
> of Scott's proposal)
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l




-- 
(http://cscott.net)
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] 2016-09-21 Scrum of Scrums meeting notes

[Grace Gellerman]

https://www.mediawiki.org/wiki/Scrum_of_scrums/2016-09-21


= 2016-09-21 =

== Product ==
=== Editing ===
 Collaboration 
* Blocking
* Blocked
** Services is working with us on the ReviewStream feed we're developing.
* Updates
** Continued work on MessagePoster.  PageTriage change to use it up, but
not merged yet
** Fixed moderation (e.g. when a page or topic is deleted) of Echo
notifications
** Some other Echo changes, such as to how seen time works, GENDER i18n,
and to the page-linked notification

 Language 
* Blocked: none
* Blocking: none
* Updates:
** Content Translation work continue; templates and other fixes.
** Jessie migration for Apertium is scheduled on October first week.

 Parsing (Subbu not attending because of a conflict) 
* Blocked: Parsoid's native  implementation is blocked on getting
feedback from the VE team.
* Patches in gerrit to parse language variant markup in Parsoid -- need to
resolve a few questions before we can deploy this
* Magic-word based optout of global user pages merged (__NOGLOBAL__)
* Kunal has filed an RFC to remove magic link functionality in the parser:
https://www.mediawiki.org/wiki/Requests_for_comment/Future_of_magic_links
* Evaluation ongoing of a (already merged in core) PHP version of a HTML5
parser (as a potential replacement for Tidy).

=== Community Tech ===
* No blockers
* Copyvio tools for French Wikipedia
https://phabricator.wikimedia.org/T145431
* More performance testing in hopes to get PageAssessments deployed to
English Wikipedia
* Education Programs Dashboard https://phabricator.wikimedia.org/T125546
* Changes to CentralAuth for cross-wiki watchlists
* Investigating work on improving blocking and harassment prevention tools

=== Reading ===

 Reading Web 
* Current sprint: https://phabricator.wikimedia.org/project/view/2186/
* Preparing to enable hovercards ab test in italian and russian
* Preparing to enable related articles in mobile in more wikis
* Lazy images blog post
* Next sprint: https://phabricator.wikimedia.org/project/view/2221/
* Enable related articles in mobile in more wikis

 Mobile Content Service (MCS) 
* Getting the aggregated feed endpoint working on Beta Cluster since feed
aggragation happens in RB now.

 Android native app 
* Current sprint: https://phabricator.wikimedia.org/project/view/2216/
* Next sprint: https://phabricator.wikimedia.org/project/view/2238/
* Released to beta Monday 9/19.  Plan is to get feedback, fix bugs, and
release nav overhaul to the stable app by EOM.

 iOS native app 
* Current release board:
https://phabricator.wikimedia.org/project/view/2188/
** In regression
** Expected to be released Sept 21

* Next release board: https://phabricator.wikimedia.org/project/board/2220/
** In progress
** MIgrating iOS Feed over to the MCS
** Adding In the news and a local notification
** Expected Delivery ~3 weeks

 Reading Infrastructure =
* no

=== Fundraising Tech ===
* Almost done getting rid of ActiveMQ!
* Fixing CentralNotice db calls for strict mode
* More refinements to CiviCRM contact de-duplication

=== RelEng ===
* Blocking
** no
* Blocked
** wmf.18 load-time regression https://phabricator.wikimedia.org/T146099
* Updates
** Skipping wmf.19 entirely due to undiagnosed load-time regression
https://wikitech.wikimedia.org/wiki/Incident_documentation/20160915-MediaWiki
*** Tentative plan is to cut wmf.20 and deploy to group0 today and, if all
goes well, adjust the remainder of this week's schedule accordingly
*** https://phabricator.wikimedia.org/T144644#2654240
** Completed Beta Cluster database migration to jessie/MariaDB 10 yesterday
** Reminder: no deploys week of Sept 26th
** Reminder: no train week of Oct 17th (but SWATs OK)

=== Analytics ===
* Blocking: None
* Blocked: None
* Updates:
** New AQS cluster is up! Massive reduction of latency. We'll increase the
throttling threshold.
** Public Event Streams: Finalized Kasocki prototype. Considering using
server side events / streaming instead of web-sockets.
** Edit history reconstruction: Some improvements on the quality of data,
still a couple issues to fix.
** Pivot (druid based UI): More work on productionizing it using
servicerunner
** A/B testing design - Evaluating feasibility of design ideas.
** Nuria participated in the Technology off-site, very productive.

=== Technical Operations ===
* '''Blocking'''
** None
* '''Blocked'''
** None
* Updates
** TechOps offsite happening week of Sept 25 (last week of quarter), please
work around this for deployments
** Will not be able to attend SoS for the same reason
** Puppet migration complete
** Varnish upload cluster upgraded to version 4. There are some caveats,
but looking good https://phabricator.wikimedia.org/T145661
** Apertium migration to Jessie scheduled for Oct 3
** OpenStackManager in wikitech is being deprecated. There will be
information coming your way in the next weeks.

=== Security ===
* Patch deployed for 

[Wikitech-l] ORES article quality data as a database table

[Amir Ladsgroup]

One of ORES [1] applications is determining article quality. For example,
What would be the best assessment of an article in the given revision.
Users in wikiprojects use ORES data to check if articles need
re-assessment. e.g. if an article is in "Start" level and now good it's
enough to be a "B" article.

As part of Q4 goals, we made a dataset of article quality scores of all
articles in English Wikipedia [2] (Here's the link to download the dataset
[3]) and we are publishing it in figshare as something you can cite [4]
also we are working on publishing monthly data for researchers to track
article quality data change over time. [5]

As a pet project of mine, I always wanted to put these data in a database.
So we can query the database and get much more useful data. For example
quality of articles in category 'History_of_Essex' [6] [7]. The weighed sum
is a measure of quality which is a decimal number between 0 (really stub)
to 5 (a definitely featured article). We have also prediction column which
is a number in this map [8] for example if prediction is 5, it means ORES
thinks it should be a featured article.

I leave more use cases to your imagination :)

I'm looking for a more permanent place to put these data, please tell me if
it's useful for you.
[1] ORES is not a anti-vandalism tool, it's an infrastructure to use AI in
Wikipedia.
[2] https://phabricator.wikimedia.org/T135684
[3] (117 MBs)
https://datasets.wikimedia.org/public-datasets/enwiki/article_quality/wp10-scores-enwiki-20160820.tsv.bz2
[4] https://phabricator.wikimedia.org/T145332
[5] https://phabricator.wikimedia.org/T145655
[6] https://quarry.wmflabs.org/query/12647
[7] https://quarry.wmflabs.org/query/12662
[8]
https://github.com/wiki-ai/wikiclass/blob/3ff2f6c44c52905c7202515c5c8b525fb1ceb291/wikiclass/utilities/extract_scores.py#L37

Have fun!
Amir
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l