Re: [Wikitech-l] Several primary database masters switchovers scheduled (read-only required)

[Manuel Arostegui]

On Tue, Sep 17, 2019 at 6:00 AM Manuel Arostegui 
wrote:

> Hello,
>
> s2 primary master switchover will start in 1h:
> https://phabricator.wikimedia.org/T230785
> List of affected wikis:
> https://raw.githubusercontent.com/wikimedia/operations-mediawiki-config/master/dblists/s2.dblist
>
>

s2 switchover was done
read-only start: 05:00:44
read-only stop: 05:01:34

Total read-only time: 50 seconds.

Manuel.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Several primary database masters switchovers scheduled (read-only required)

[Manuel Arostegui]

Hello,

s2 primary master switchover will start in 1h:
https://phabricator.wikimedia.org/T230785
List of affected wikis:
https://raw.githubusercontent.com/wikimedia/operations-mediawiki-config/master/dblists/s2.dblist


Thanks,
Manuel.

On Tue, Aug 20, 2019 at 12:49 PM Manuel Arostegui 
wrote:

> Hello,
>
> The following primary database masters will be switched over during the
> next few weeks (more details at https://phabricator.wikimedia.org/T230788
> ):
>
> Impact:
> *Writes will be blocked*
> *Reads will remain unaffected*
>
> These are the concrete days, hours and affected wikis:
>
> * s8: 10th Sept from 05:00-05:30 UTC. The affected wiki is: wikidatawiki -
> tracking task: T230762
>
> * s2: 17th Sept from 05:00-05:30 UTC. The list of affected wikis is at:
> https://raw.githubusercontent.com/wikimedia/operations-mediawiki-config/master/dblists/s2.dblist
>  - tracking task: T230785
>
> * s3: 24th Sept from 05:00-05:30 UTC. The list of affected wikis is at:
> https://raw.githubusercontent.com/wikimedia/operations-mediawiki-config/master/dblists/s3.dblist
> - tracking task: T230783
>
> * s4: 26th Sept from 05:00-05:30 UTC. The affected wiki is: commonswiki -
> tracking task: T230784
>
> If everything goes well, we do not expect to use those 30 minutes of
> read-only and rather just a few minutes.
>
> We will email send an email the day of each failover before and after it
> is done.
>
> Sorry for any inconvenience this might cause.
>
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] TechCom Radar 2019-09-11

[Kate Chapman]

Hi All,

Here are the minutes from this week's TechCom meeting:

* RFC Approved: Define criteria for setting explicit PHP support
target for MediaWiki 

Patch available for RFC: Section headings should have a clickable
anchor  (please comment on
ticket if you are interested in this)

Discussed: Drop IE6 and IE7 basic compatibility and security support.


You can also find our meeting minutes at


See also the TechCom RFC board
.

If you prefer you can subscribe to our newsletter here


Thanks,
Kate

--
Kate Chapman
Senior Program Manager, Core Platform
Wikimedia Foundation
kchap...@wikimedia.org

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Fwd: GSoC 2019 and Outreachy Round 18 are over! Call for projects and mentors for Outreachy Round 19 begins.

[Srishti Sethi]

Hello all,

Just a reminder that the deadline to submit projects on the Outreachy
website is September 24th. If you are considering to mentor a project via
Outreachy, please reach out to me with a project idea before the deadline!

Cheers,
Srishti

*Srishti Sethi*
Developer Advocate
Wikimedia Foundation 



-- Forwarded message -
From: Srishti Sethi 
Date: Wed, Sep 4, 2019 at 2:47 PM
Subject: GSoC 2019 and Outreachy Round 18 are over! Call for projects and
mentors for Outreachy Round 19 begins.
To: Wikimedia developers , Wikimedia
Mailing List 


Hello everyone,

Google Summer of Code 2019 and Outreachy Round 18 are over! See a complete
list of 15 projects interns worked on this past summer:
https://www.mediawiki.org/wiki/Google_Summer_of_Code/Past_projects#2019,
https://www.mediawiki.org/wiki/Outreachy/Past_projects#Round_18

If you find a project relevant to your area of interest, you can explore
more via the resources mentioned in the links above. Help test the tools or
features and share your feedback or any information on bugs you encountered
in a comment on the corresponding Phabricator task.

Next up is Outreachy Round 19! Wikimedia is participating in the winter
edition!

If you are interested in mentoring a coding or non-coding (documentation,
design, research, outreach, translation, etc.) project, and you are not
familiar with the program, reach out to me (via email sse...@wikimedia.org)
to learn more about the next steps.

If you are already familiar with the program and want to share your idea
for a project that you would like to mentor, share in a comment on this
Phabricator task: https://phabricator.wikimedia.org/T232048.

The *deadline to submit* projects on the Outreachy website is *September
24th, 2019*.

Some relevant links:
* A recent blog post highlighting one of our intern's and mentor's
experience participating in Outreachy program with Wikimedia:
https://wikimediafoundation.org/news/2019/06/25/wikimedia-and-outreachy-technical-internships-are-bridging-the-free-and-open-source-inclusion-gap/
* Outreachy website: https://www.outreachy.org/.
* Roles and responsibilities of an Outreachy mentor:
https://www.mediawiki.org/wiki/Outreachy/Mentors

Looking forward to your participation!

Cheers,
Srishti

*Srishti Sethi*
Developer Advocate
Wikimedia Foundation 
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Startup module size and ResourceLoader registry overhead

[Amir Sarabadani]

Hello,
Startup module, is the Javascript code [1] that is being served at almost
every request to Wikimedia as part of  (non-blocking though) to load
other RL modules. It does important things for example it checks if the
requested modules already exist with the given hash in the local storage
cache.

Because of the given reasons, the startup module needs and includes list of
all registered modules with their version hash in the code. As the result
if you register a module, even if you don't load it, it adds a rather small
overhead (around 19 bytes after minification and compression) to every
request to any wiki the code is enabled. So if you register a new module in
core or one of the extensions that is deployed in all wikis (CX,
WikibaseClient, etc.), it's going to be added to every page request,
meaning 30 GBs more traffic every day to our users. Even if you don't use
it anywhere.

If you're adding a feature, 30GB/day is acceptable but sometimes developers
use Resource loader for dependency management or class loading (yours truly
used to do that) and introduce 20 modules instead of one and each one
causing an extra 600 GB/day. The big overhead for our users is bad for
three reasons: 1- RL has to handle the dependency management, making every
page view slightly slower and worse UX for our users 2- The extra network
is bad with places without access to broadband connection, where Wikipedia
is the most likely place that people learn and grow [2] 3- The scale we are
talking is so massive (petabytes a year) that It has environmental impact.

Let me give you an example, a couple of weeks ago we dropped 137 modules
from WikibaseClient. After deployment, it dropped 4TB/day from our network
(= 1.5 PB/year) and synthetic data shows, in an average case we drop 40 ms
from every response time [3].

We now have a dashboard to track size of size of RL registry [4] plus a
weekly metrics for changes [5][6]

If you're looking for ways to help. I wrote a tool [7] to do some graph
analysis and it gives list of extensions that has modules that can be
merged. The extensions that according to the analysis (that can have false
positives) can get better are TimedMediaHandler, PageTriage, Graph,
RevisionSlider, CodeMirror, Citoid, TemplateData, TwoColConflict,
Collection, CentralNotice, AdvancedSearch, 3D, MobileFrontend and many more
including some bits and pieces in core. I put the graphs of modules that
can be merged at [8] and I invite you to have fun with those modules.
Modules can be merged using package modules [9]

Most of the is work done by the performance team [10] and volunteers and
developers in lots of teams. I joined the party later as volunteer/WMDE
staff and I'm sharing mostly the results and writing long emails. Big kudos
to Krinkle, James Forrester, Santhosh Thottingal, Jon Robson, Alaa Sarhaan,
Alexandros Kosiaris, and so many others that helped and I forgot.

[1] An example from English Wikipedia:
https://en.wikipedia.org/w/load.php?lang=en=startup=scripts=1=vector=true
[2] https://arxiv.org/abs/1812.00474
[3] https://phabricator.wikimedia.org/T203696#5387672
[4] https://grafana.wikimedia.org/d/BvWJlaDWk/startup-module-size?orgId=1
[5] https://gist.github.com/Krinkle/f76229f512fead79fb4868824b5bee07
[6]
https://docs.google.com/document/d/1SESOADAH9phJTeLo4lqipAjYUMaLpGsQTAUqdgyZb4U
[7] https://phabricator.wikimedia.org/T232728
[8] https://phabricator.wikimedia.org/T233048
[9] https://www.mediawiki.org/wiki/ResourceLoader/Package_modules
[10] https://phabricator.wikimedia.org/T202154

Best
-- 
Amir (he/him)
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Note: Contents of mw-config's InitialiseSettings has moved to VariantSettings

[James Forrester]

If you don't write or deploy Wikimedia production config patches, you can
ignore this.

This is a note to highlight a change, which will be obvious as soon as you
try to rebase a patch: the parts of configuration that vary per-wiki, which
previously have lived in InitialiseSettings.php, have now moved to a
(mostly) static array in "VariantSettings.php".

This change allows for proper testing of variant configuration, and the
eventual migration to actual static configuration files for increased site
stability and developer confidence. See
https://phabricator.wikimedia.org/T223602 for more details.

Yours,
-- 
*James D. Forrester* (he/him  or they/themself
)
Wikimedia Foundation 
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Gerrit: Internet Explorer no longer supported under PolyGerrit

[Paladox via Wikitech-l]

Hello All!
PolyGerrit no longer supports Internet Explorer. It never really worked so now 
the offical status is "not supported".
You can use alternative browsers like Chrome, Firefox, Opera and Microsft Edge.
GWTUI will support internet explorer. Though GWTUI has been removed in newer 
versions of gerrit.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] [Train] 1.34.0-wmf.22 status update

[Antoine Musso]

On 12/09/2019 23:42, Antoine Musso wrote:

Hello,

Deployment of MediaWiki 1.34.0-wmf.22 on the Wikimedia cluster is 
currently blocked. It is blocked on a php7.2 bug that has yet to be 
tracked and determined.



Immediately after deploying to group1 (which has commons and wikidata), 
I noticed an infamous error in the logs:


  PHP Notice: Undefined index:"

Which sounds similar to known errors with php7.2 (T229433, T231089 among 
others) but this time it hits LBFactoryMulti.php which is for the 
database load balancing. So that sounds scary and I thus decided to 
block the train.


The notice happens at:

  $groupLoads[ILoadBalancer::GROUP_GENERIC]

The GROUP_GENERIC constant is an empty string. Brad "anomie" Jorsch 
quickly found out the code always set that key!


https://phabricator.wikimedia.org/T232613#5485181

So what is left is a edge case issue in php 7.2, maybe in the opcache or 
something even scarier.


Thanks to Tim Starling, we have a solution to issue a core dump of php 
when that issue happens. There is already available on the cluster, but 
it has to be investigated which is not that trivial.


Hello,

Eventually Tim Starling and Giuseppe Lavagetto continued the debugging 
over the week-end (thank you!).  The issue came from the PHP Memcached 
extension:

https://phabricator.wikimedia.org/T232613#5494695

The fix has been deployed on the Wikimedia Cluster at 12:00 UTC and the 
errors are gone.


I am thus processing promoting 1.34.0-wmf.22 to the rest of the wikis.


Thank you Tim and Giuseppe!

--
Antoine "hashar" Musso


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Making a cross-wiki authenticated call to Wikidata with Wikimedia Android data client?

[Josephine Lim]

Hello,

We have started using the wikimedia-android-data-client library(
https://github.com/wikimedia/wikimedia-android-data-client) to make
mediawiki api calls in the Commons Android app, replacing the legacy
library that we were using previously. Most of the authenticated calls
(i.e. login, upload, nomination for deletion, thank, notifications etc)
made to Commons wiki are working with the new library, but we are stuck
with a cross-wiki call to Wikidata. We are trying to call
`Service:wbcreateclaim` to create a claim on Wikidata but the call is
failing.

We have posted relevant http logs at
https://github.com/wikimedia/wikimedia-android-data-client/issues/21#issue-469284270


We would greatly appreciate it if could take a look at the logs and suggest
what we might be doing wrong. Is it because of some issue with cookies?
Because as far as we can see, as expected we are sending the params in POST
request body with application/x-www-form-urlencoded.

Relevant code:
https://github.com/commons-app/apps-android-commons/blob/backend-overhaul/app/src/main/java/fr/free/nrw/commons/wikidata/WikidataClient.java


Relevant method call: service.postCreateClaim(entityId, snaktype, property,
value, "en", csrfTokenClient getTokenBlocking())

Thank you so much!

Best regards,
Josephine
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l