[Wikitech-l] Discovery Weekly Update for the week starting 2018-10-01

[Chris Koerner]

Hello again. It's been a while.

This is the weekly update from the Search Platform team for the week
starting 2018-09-17 through 2018-10-01.

As always, feedback and questions welcome.

== Discussions ==

=== Search ===
* Implemented indexing statement values as part of main data in
Wikidata, so that statement values are now searchable without special
syntax [0]
* Reindexed wikidata which also enables qualifier indexing [1]
* Mathew worked on resolving an elasticsearch shard size alert by
doing an in place reindex [2]
* There was a lot of work done to investigate a brief outage of
CirrusSearch (mw exception spike for api.php) [3], but it's resolved
enough for now.
* Gehel and others worked on refactoring puppet to support multiple
elasticsearch instances on same node [4]
* Erik worked on an issue where the text content of wiki page in
search index can merge words making them unfindable [5]
* Stas updated the search engine of Wikidata to enable searching by
author name string [6]
* David and Erik worked together on evaluating adding an image quality
score to media search result ranking [7]
* Stas added X-Search-Id to WikidataCompletionSearchClicks events [8]
* David added a way to configure timeouts of autocomplete queries [9]
* Erik upgraded saneitizer to constantly re-index documents [10]
* David investigated why interwiki cache hit/miss was no longer
reported (since 2017) and decided to drop the support for caching
interwiki queries [11]
* Mathew and Gehel worked on raising the alert level on disk space for
old elasticsearch servers [12]
* Erik worked to correct issues where the Cirrus MLT cache had a 0%
hit rate on switchover [13]

=== WDQS ===

* Added new NTriples RDF dump (which makes it easier to do per-line
processing) [14]
* Internal cluster switched to Kafka events as change source, public
cluster next [15]

== Did you know? ==
* Different languages can have a different number of sounds they use;
the set of sounds used in a particular language is called its
“phonemic inventory”. [16] The numbers of sounds can range from 11 to
over 140! Having more sounds than letters, or different sounds than
the usual sound associated with a letter, can be the source of unusual
orthographies and/or transliteration schemes—including "q" formerly
being used as a vowel in Natqgu (now Natügu), a language of the
Solomon Islands.

[0] https://phabricator.wikimedia.org/T163642
[1] https://phabricator.wikimedia.org/T193407
[2] https://phabricator.wikimedia.org/T204362
[3] https://phabricator.wikimedia.org/T204776
[4] https://phabricator.wikimedia.org/T198351
[5] https://phabricator.wikimedia.org/T195389
[6] https://phabricator.wikimedia.org/T179815
[7] https://phabricator.wikimedia.org/T202339
[8] https://phabricator.wikimedia.org/T205597
[9] https://phabricator.wikimedia.org/T204959
[10] https://phabricator.wikimedia.org/T203622
[11] https://phabricator.wikimedia.org/T191961
[12] https://phabricator.wikimedia.org/T204361
[13] https://phabricator.wikimedia.org/T204148
[14] https://phabricator.wikimedia.org/T144103
[15] https://phabricator.wikimedia.org/T189458
[16] https://en.wikipedia.org/wiki/Phonemic_inventory



Subscribe to receive on-wiki (or opt-in email) notifications of the
Discovery weekly update.

https://www.mediawiki.org/wiki/Newsletter:Discovery_Weekly

The archive of all past updates can be found on MediaWiki.org:

https://www.mediawiki.org/wiki/Discovery/Status_updates

Interested in getting involved? See tasks marked as "Easy" or
"Volunteer needed" in Phabricator.

[1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R
[2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R


Yours,
Chris Koerner
Community Relations Specialist
Wikimedia Foundation

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] [Wikimedia-l] FileLinkTool

[Pine W]

Thanks, this looks like a nice addition that will cumulatively save people
a lot of time if used widely. I'm forwarding to the Commons and Wikitech
lists. I haven't tested it but the concept looks good.

Pine
( https://meta.wikimedia.org/wiki/User:Pine )


On Tue, Oct 9, 2018 at 10:14 AM Ananth Subray 
wrote:

> Dear All,
>
> We are very happy to inform you all that Indic-TechCom has made a new tool
> to give a link to source file after doing the derivative work on commons.
> We all know that how important to mention the source file correctly after
> doing the derivative work. Based on the request of community members
> Indic-TechCom has created. To know more about the tool visit the meta
> page[1]. Indic-TechCom request you all to use it and spread the work to
> your community members. In case if you have suggestions, please let us
> know.
>
>
>
>
> [1]. https://meta.wikimedia.org/wiki/Indic-TechCom/Tools/FileLinkTool
> ___
> Wikimedia-l mailing list, guidelines at:
> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> https://meta.wikimedia.org/wiki/Wikimedia-l
> New messages to: wikimedi...@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> 
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] OOUI v0.29.2 released

[Volker E.]

Hi everyone,
last week we've released OOUI v0.29.0 and identified sequentially two
issues on Beta-Cluster which we've addressed in v0.29.1 next day and
v0.29.2 released yesterday.
Bartosz Dziewoński took the chance and hardened the JS/PHP comparison
test suite with former hotfix.
Thanks to Pablo Grass of Wikimedia Deutschland and Petar Petkovic for
help in identifying latter of those issues and Ed Sanders for instant
fix!

Key highlights of this release:
 - OOUI, similar to MediaWiki, now requires PHP 7
   With this change we've also switched code to PHP 7 syntax in order
to make code reusable between MediaWiki and OOUI (see below)
 - Deprecated CapsuleMultiselectWidget got dropped, which reduces
widgets CSS by 8%  percent after gzipping[0]

Possible breaking changes:
The following changes might cause breaking of your interface, please carefully
consider if they affect your code.

- Drop CapsuleMultiselectWidget, deprecated since v0.27.5. Use
superior TagMultiselectWidget instead.
- TextInputWidget: Drop support for `multiline: true`, deprecated in
0.22.2. Use MultilineTextInputWidget directly.
- Consistently name document listeners[1]
- Formally require PHP 7
-- Use PHP 5.6 variadic function syntax
-- Use PHP 7 "\u{}" Unicode codepoint escapes[2]
-- Use PHP 7 '??' operator instead of '?:' with 'isset()'[3]
- Upgrade jQuery from 3.2.1 to 3.3.1

You can find details on additional new features & icons, code-level
and accessibility
changes, styling and interaction design amendments, and all
improvements since v0.28.0 in the full changelog[4].

If you have any further queries or need help dealing with breaking
changes, please let me know.

As always, library documentation is available on mediawiki.org[5],
there is comprehensive generated code-level documentation and
interactive demos and tutorials hosted on doc.wikimedia.org[6].

OOUI version: 0.29.2
MediaWiki version: 1.32.0-wmf.26
Date of deployment to production: Regular train (due to DC switchover
next week), starting Thursday 18 October

Best,
Volker

[0] - https://phabricator.wikimedia.org/T183299#4637565
[1] - https://gerrit.wikimedia.org/r/#/c/oojs/ui/+/437960/
[2] - https://gerrit.wikimedia.org/r/#/c/oojs/ui/+/456500/
[3] - https://gerrit.wikimedia.org/r/#/c/oojs/ui/+/456499/
[4] - https://phabricator.wikimedia.org/diffusion/GOJU/browse/master/History.md
[5] - https://www.mediawiki.org/wiki/OOUI
[6] - https://doc.wikimedia.org/oojs-ui/master/

--
Senior User Experience Engineer
Wikimedia Foundation

volke...@wikimedia.org | @Volker_E

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] non-obvious uses of in your language

[C. Scott Ananian]

The relevant Parsoid feature request for having VE use linktrails is
https://phabricator.wikimedia.org/T50463 since in general Parsoid just
generates [[Book|books]] when VE gives it `books`.

If VE gives Parsoid `books` it will assume that's what
the author actually meant, and will generate `[[Book]]s` using a
very general mechanism used for a number of other syntax conflicts (like if
you actually want to start a line with the literal character `*`).

I don't think the answer is to invent new syntax for linktrail separation
-- we already have quite enough different ways of escaping and/or
token-breaking already, as partially enumerated in this thread already.
The only one I would be happy to faciliatate would be `-{}-` since it is
already an odd parser corner case -- it is parsed by the wikitext
preprocessor but then spit back out as literal text by the second parsing
phase unless LanguageConverter is enabled for the specific page language.
It would simplify the parse if the LanguageConverter constructs were
"always on" instead of being en/disabled on a page-by-page basis.
  --scott

On Sun, Oct 7, 2018 at 12:23 PM Amir E. Aharoni <
amir.ahar...@mail.huji.ac.il> wrote:

> ... And, more importantly, its form doesn't say "separate the trail from
> the link". Just like , it only *happened* to do it (I tried on
> Wikipedia, and it doesn't do it now).
>
> The point I'm trying to make in this thread is that  happens to do
> certain things other than showing wiki syntax without parsing, and is used
> for them as if it's *intended* for it, but this is a hack. If a certain
> functionality is needed, such as separating the trail from the link, then
> it's worth considering creating a piece of syntax for it.
>
> --
> Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי
> http://aharoni.wordpress.com
> ‪“We're living in pieces,
> I want to live in peace.” – T. Moore‬
>
>
> ‫בתאריך יום א׳, 7 באוק׳ 2018 ב-19:08 מאת ‪bawolff‬‏ <‪bawolff...@gmail.com
> ‬‏>:‬
>
> > Alas, no longer valid in XML or HTML5. (Although HTML5 will still
> > parse it as an empty comment, but with a  "incorrectly-opened-comment"
> > error.
> >
> > --
> > Brian
> >
> >
> > On Sat, Oct 6, 2018 at 6:57 AM Chad  wrote:
> > >
> > > Found it :)
> > >
> > > https://www.w3.org/MarkUp/SGML/sgml-lex/sgml-lex
> > >
> > > Search for "empty comment declaration" :)
> > >
> > > -Chad
> > >
> > > On Fri, Oct 5, 2018, 11:50 PM Chad  wrote:
> > >
> > > > I'm personally a fan of .
> > > >
> > > > I came across it years ago--it's a null comment. Can't find the
> > reference
> > > > at the moment though.
> > > >
> > > > -Chad
> > > >
> > > > On Thu, Oct 4, 2018, 2:25 PM Daniel Kinzler 
> > > > wrote:
> > > >
> > > >> Am 04.10.2018 um 18:58 schrieb Thiemo Kreuz:
> > > >> > The syntax "[[Schnee]]reichtum" is quite common in the
> > > >> > German community. There are not many other ways to achieve the
> same:
> > > >> >  or ­ can be used instead.[1] The later is often the
> > > >> > better alternative, but an auto-replacement is not possible. For
> > > >> > example, "[[Bund]]estag" must become
> "[[Bund]]es­tag".
> > > >>
> > > >> We could introduce new syntax for this, such as &nope; or even
> > &nowiki;.
> > > >>
> > > >> Or how about {{}} for "this is a syntactic element, but it does
> > nothing"?
> > > >> But if
> > > >> that is mixed in with template expansion, it won't work if it
> expands
> > to
> > > >> nothing, since template expansion happens before link parsing,
> right?
> > For
> > > >> better
> > > >> or worse...
> > > >>
> > > >> --
> > > >> Daniel Kinzler
> > > >> Principal Software Engineer, MediaWiki Platform
> > > >> Wikimedia Foundation
> > > >>
> > > >> ___
> > > >> Wikitech-l mailing list
> > > >> Wikitech-l@lists.wikimedia.org
> > > >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > > >
> > > >
> > > On Oct 5, 2018 11:50 PM, "Chad"  wrote:
> > >
> > > I'm personally a fan of .
> > >
> > > I came across it years ago--it's a null comment. Can't find the
> reference
> > > at the moment though.
> > >
> > > -Chad
> > >
> > > On Thu, Oct 4, 2018, 2:25 PM Daniel Kinzler 
> > wrote:
> > >
> > > > Am 04.10.2018 um 18:58 schrieb Thiemo Kreuz:
> > > > > The syntax "[[Schnee]]reichtum" is quite common in the
> > > > > German community. There are not many other ways to achieve the
> same:
> > > > >  or ­ can be used instead.[1] The later is often the
> > > > > better alternative, but an auto-replacement is not possible. For
> > > > > example, "[[Bund]]estag" must become
> "[[Bund]]es­tag".
> > > >
> > > > We could introduce new syntax for this, such as &nope; or even
> > &nowiki;.
> > > >
> > > > Or how about {{}} for "this is a syntactic element, but it does
> > nothing"?
> > > > But if
> > > > that is mixed in with template expansion, it won't work if it expands
> > to
> > > > nothing, since template expansion happens before link parsing, right?
> > For
> > > > better
> > > > or worse...
> > > >
> > > > --
> 

[Wikitech-l] Wednesday 3-4 pm UTC: Technical Advice IRC Meeting

[Birgit Müller]

Hey all,

another round of the Technical Advice IRC meeting is taking place on
**Wednesday, Oct 10, 3-4 pm UTC** on #wikimedia-tech.

This week's meeting will be hosted by @Amir1 and @CFisch_WMDE - Question
are welcome in English, Persian & German :-)

The Technical Advice IRC Meeting is a weekly support event for volunteer
developers. Every Wednesday, two full-time developers are available to help
you with all your questions about Mediawiki, gadgets, tools and more! This
can be anything from "how to get started" over "who would be the best
contact for X" to specific questions on your project.

If you know already what you would like to discuss or ask, please add your
topic to the next meeting:
https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting


Birgit (for the Technical Advice IRC Meeting crew)


-- 
Birgit Müller
Community Communications Manager
Software Development and Engineering




Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Tel. (030) 219 158 26-0
http://wikimedia.de

Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen
Wissens frei teilhaben kann. Helfen Sie uns dabei!
http://spenden.wikimedia.de/

Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/681/51985.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Avatars coming to gerrit

[Daniel Kinzler]

Am 06.10.2018 um 02:43 schrieb Paladox via Wikitech-l:
>  Note that we will only accept png images. Please do not upload any other 
> format.

I note that your avatar is 400x400 pixels. Is thsi the prefered dimension?


-- 
Daniel Kinzler
Principal Software Engineer, Core Platform
Wikimedia Foundation

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Avatars coming to gerrit

[Niklas Laxström]

la 6. lokak. 2018 klo 3.44 Paladox via Wikitech-l
(wikitech-l@lists.wikimedia.org) kirjoitti:

> We also have a license file for you to specify the license of your image at 
> https://gerrit.wikimedia.org/r/plugins/gitiles/All-Avatars/+/refs/heads/master/LICENSE
> If none is specified then it defaults to GPL 2.0+.

What if one is uncomfortable or unwilling to license a photo of
themselves under an open source licence?
  -Niklas

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l