[Wikitech-l] Re: Recreating CI with Quibble

[Antoine Musso]

Le 19/04/2024 à 10:42, Sebastian Berlin a écrit :
I'm trying to recreate a CI build to debug things locally. I've 
followed the instructions under 
https://doc.wikimedia.org/quibble/#reproducing-a-ci-build, but the 
command for listing Docker images doesn't work. There are no images 
with "quibble" in the name. Where can I find the available Quibble images?


Hello,

Quibble is not very user friendly. I started it during the 2017 Vienna 
hackathon as a quick hack to unify the complicated stack of repos, 
shell, python, Jenkins job into a single place. That has served to 
reduce the CI complexity and made the jobs quite simpler.


Overall I can't say it is very pleasant to use as a human, there are a 
few oddities such as:


- cleaning the dependencies and reinstalling everytime (use --skip-deps)
- misnamed options such as --skip-zuul to prevent refetching and doing a 
git-clean of everything
- one need to set ZUUL_PROJECT to trigger the logic that runs 
extension/skin tests

And more :)

I have sent a few patches recently to slightly improve it such as:

- automatically deleting the autogenerated LocalSettings.php instead of 
having the MediaWiki installer to bail out after some minutes.


- Adding --change  which retrieve from Gerrit the 
metadata of the given change and initialize the ZUUL_* environment 
variables (though I think one still has to set ZUUL_PROJECT to trigger 
the proper logic ...)


Back on topic, the documentation about reproducing a CI build deserves 
to be refreshed:


- the /v2/_catalog registry endpoint has evolved and is paginated 
nowadays, see 
https://distribution.github.io/distribution/spec/api/#listing-repositories 
or look at which image is currently defined in CI ( 
https://gerrit.wikimedia.org/g/integration/config/+/refs/heads/master/jjb/mediawiki.yaml 
).


Note that CI does a merge commit of the propose patch or patches against 
the tip of the target branch. That is done by processes named 
zuul-merger which exposes the resulting merges using git-daemon. Those 
repositories are not publicly available, though they should be reachable 
from a WMCS instance.


I am more than happy to assist and chat about Quibble. I will very 
warmly welcome any help on that front :)


Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: Recreating CI with Quibble

[Antoine Musso]

Le 19/04/2024 à 11:03, Arthur Taylor a écrit :

Hi Sebastian,

I've also been playing around with debugging CI images on my local 
machine recently. I've pushed an update to the Quibble docs on 
Mediawiki [1]. Hopefully that's helpful - feedback and changes welcome!


Thanks,

Arthur

[1] https://www.mediawiki.org/wiki/Continuous_integration/Quibble


Hello,

That documentation was merely an helper for people maintaining the CI 
stack linking to the releasing doc which is in the repository, how to 
rebuild the CI and then switch the jobs.


The Quibble manual / user documentation is inside the source repository 
under the doc/source directory 
https://gerrit.wikimedia.org/g/integration/quibble/+/refs/heads/master/doc/source 
.  It ends up being published at https://doc.wikimedia.org/quibble/ 
which is the canonical place.


The wiki edit at 
https://www.mediawiki.org/w/index.php?title=Continuous_integration/Quibble=prev=6476384 
 
is amazing, and it would be really great to have it moved to the source 
repository.  I like the Quabble.py script to pause before running tests, 
though one can also ask Quibble to run bash as a test command: `quibble 
-c bash`.


Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: [Ops] Gerrit upgrade Monday March 25th at 9am UTC

[Antoine Musso]

Le 21/03/2024 à 16:03, Antoine Musso a écrit :
I will be *upgrading Gerrit* from the 3.7 series to the 3.8 series. I 
have scheduled the upgrade for *Monday March 25th at 9am UTC*. It is 
immediately after the UTC morning backport & config window.


Hello,

I have cancelled the upgrade since I did not perform all the preliminary 
tasks required to upgrade us to Gerrit 3.8. I will reschedule it for 
next week.


Antoine "hashar" Musso

___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Gerrit upgrade Monday March 25th at 9am UTC

[Antoine Musso]

Hello,

I will be *upgrading Gerrit* from the 3.7 series to the 3.8 series. I 
have scheduled the upgrade for *Monday March 25th at 9am UTC*. It is 
immediately after the UTC morning backport & config window.


The upgrade requires the Gerrit service to be stopped for the duration 
of the upgrade. Given we do not need to reindex all the changes, the 
downtime should be just a few minutes.


Gerrit 3.8 brings:

 * Rebase on behalf of the uploader, so that the rebaser does not take
   over the change (the original uploader is preserved)
 * Rebase a chain of changes: when working with a series of change, the
   whole series can be rebased atomically which saves a lot of manual
   rebasing actions
 * Browser Notifications: get a notification when a change requires
   your attention
 * And more UI changes
   

The release notes: https://www.gerritcodereview.com/3.8.html
The upgrade task: 
https://phabricator.wikimedia.org/T354886
Deployment calendar entry 



Antoine "hashar" Musso
Wikimedia Release Engineering
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: [Ops] Gerrit upgrade Thursday 01/11 at 9am UTC

[Antoine Musso]

Le 22/01/2024 à 15:14, Zoran Dori a écrit :

Hello,
Gerrit 3.7 looks amazing to me, although 
https://gerrit.wikimedia.org/r/q/status:open+-is:wip  looks a bit weird to me, 
but I'll get used to it.


Hello,

It is hard to tell what you find weird there, but I suspect it might be 
due to some extra columns or missing ones. The columns can be adjusted 
in our user preferences:


https://gerrit.wikimedia.org/r/settings/#ChangeTableColumns

The columns are:

- Number
- Subject
- Owner
- Reviewers
- Repo
- Branch
- Updated
- Size
- Status


I don't have any complaints. I'm looking forward to seeing Gerrit upgraded to 
3.8. 
Eventually it will happen. I have only had a a quick look at the 
upstream change log and filed https://phabricator.wikimedia.org/T354886


--
Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: [Ops] Gerrit upgrade Thursday 01/11 at 9am UTC

[Antoine Musso]

Le 09/01/2024 à 16:06, Antoine Musso a écrit :


Hello,

I will be *upgrading Gerrit* from the 3.5 series (which is end of 
life) to the 3.6 series. I have scheduled the upgrade for *Thursday 
January 11st at 9am UTC*. It is immediately after the UTC morning 
backport & config window.


The upgrade will require the Gerrit service to be stopped for the 
duration of the upgrade. I do not anticipate any issue and the service 
should be back up in half an hour.


Gerrit 3.6 introduces Submit Requirements 
<https://gerrit-documentation.storage.googleapis.com/Documentation/3.6.0/config-submit-requirements.html> 
which would let us tune what is required for a change to be merged by 
Gerrit. I do not plan to make any change to the default rule, but the 
requirements (such as a change having Code-Review +2 and Verified +1) 
would show up in the GUI which will make it easier to understand what 
is missing.


There are lot of other smaller changes in the upstream release notes:

https://www.gerritcodereview.com/3.6.html

The upgrade task: https://phabricator.wikimedia.org/T309870

Deployment calendar entry 
<https://wikitech.wikimedia.org/wiki/Deployments#deploycal-item-20240111T0900>.



Hello,

I have completed the upgrade and Gerrit is back up. If anything feels 
off please reach out on IRC #wikimedia-releng and/or file a task in 
Phabricator against #Gerrit :-)


Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Gerrit upgrade Monday January 22nd at 9am UTC

[Antoine Musso]

Hello,

I will be *upgrading Gerrit* from the 3.6 series (Upgraded on Jan 11) to 
the 3.7 series. I have scheduled the upgrade for *Monday January 22nd at 
9am UTC*. It is immediately after the UTC morning backport & config window.


The upgrade will require the Gerrit service to be stopped for the 
duration of the upgrade. This upgrade requires a full reindexing of the 
nearly million of changes accross the couple thousands of repositories, 
and if I remember well that step alone takes ~ 40 minutes.


Gerrit 3.7 brings:

 * Full markdown support in comments (3.6 already added support for
   |`backticks`| )
 * Mention of users via |@| which is disabled by default but we
   might enable
 * Bulk actions from the change list to add reviewer, topics or hashtags
 * Auto switch between light and dark theme based on OS preference
 * And more UI changes
   

The release notes: https://www.gerritcodereview.com/3.7.html 

The upgrade task: https://phabricator.wikimedia.org/T354885 

Deployment calendar entry 



Antoine "hashar" Musso
Wikimedia Release Engineering___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Gerrit upgrade Thursday 01/11 at 9am UTC

[Antoine Musso]

Hello,

I will be *upgrading Gerrit* from the 3.5 series (which is end of life) 
to the 3.6 series. I have scheduled the upgrade for *Thursday January 
11st at 9am UTC*. It is immediately after the UTC morning backport & 
config window.


The upgrade will require the Gerrit service to be stopped for the 
duration of the upgrade. I do not anticipate any issue and the service 
should be back up in half an hour.


Gerrit 3.6 introduces Submit Requirements 
 
which would let us tune what is required for a change to be merged by 
Gerrit. I do not plan to make any change to the default rule, but the 
requirements (such as a change having Code-Review +2 and Verified +1) 
would show up in the GUI which will make it easier to understand what is 
missing.


There are lot of other smaller changes in the upstream release notes:

https://www.gerritcodereview.com/3.6.html

The upgrade task: https://phabricator.wikimedia.org/T309870

Deployment calendar entry 
.



Antoine "hashar" Musso
Wikimedia Release Engineering

___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: [Ops] [Train] 1.41.0-wmf.30 blocked

[Antoine Musso]

Le 12/10/2023 à 11:17, Antoine Musso a écrit :

Hello,

The 1.41.0-wmf.30 version of MediaWiki is blocked.

The new version deployed to group 1 but can proceed no further until the 
issue is resolved:


Error: Typed property 
GrowthExperiments\NewcomerTasks\AddLink\LinkRecommendationUpdater::$linkRecommendationTaskType must not be accessed before initialization

https://phabricator.wikimedia.org/T348719

Once the issue is resolved, the train can resume.


Hello,

The fix has been done and I am going to deploy it in the next few 
minutes, then I will promote group 2 wikis back to 1.41.0-wmf.30.


https://gerrit.wikimedia.org/r/965217

Thanks to everyone involved!

Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] [Train] 1.41.0-wmf.30 blocked

[Antoine Musso]

Hello,

The 1.41.0-wmf.30 version of MediaWiki is blocked.

The new version deployed to group 1 but can proceed no further until the 
issue is resolved:


Error: Typed property 
GrowthExperiments\NewcomerTasks\AddLink\LinkRecommendationUpdater::$linkRecommendationTaskType 
must not be accessed before initialization

https://phabricator.wikimedia.org/T348719

Once the issue is resolved, the train can resume.

cheers,

--
Antoine Musso & Andre Klapper
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: [Wikitech-ambassadors] [invitation to participate] Listening Sessions for the Technical Decision-Making Retrospective

[Antoine Musso]

Le 05/09/2023 à 12:12, Amir Sarabadani a écrit :

Hi,
I think the email somehow didn't reach wikitech-l and some other 
mailing lists (except ambassadors)


The message did made it to wikitech-l:

https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/thread/7XEMHHZZR4EF2JFKYBGZFKTSPUEPI6TW/

I only got it via [wikibots-l] though. I suspect it is an issue with 
mailman which would prevent delivering the same messages multiple times 
even if it is dispatched to different mailing lists.


There is a bug filed https://gitlab.com/mailman/mailman/-/issues/914 
which looks awkwardly close to the issue:


   /When a message is cross-posted and held on more than one list,
   handling the held post on one list will delete the held message from
   the message store. The held post is still in the held message view
   on the other list(s) and viewing, rejecting or discarding the post
   all behave reasonably, but accepting the post results in
   urllib.error.HTTPError: HTTP Error 500: {"title": "500 Internal
   Server Error"}
   /

I have filed it as https://phabricator.wikimedia.org/T345691 , then I 
don't know anything about mailman, but it might be worth looking at the 
server side logs./

/


Antoine "hashar" Musso
Wikimedia - Release Engineering___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: [Ops] Re: [Train] 1.41.0-wmf.9 status update: blocked at group1 on T336962, T336964

[Antoine Musso]

Le 19/05/2023 à 13:43, Antoine Musso a écrit :


Hello,

The first issue got fixed. For the second I have spend sometime this 
morning debugging it and apparently it is due to an item that got 
redirected twice: https://phabricator.wikimedia.org/T336952


I am seeking confirmation from people that knows more and if that 
specific error rules out the code, I will process with the MediaWiki 
train upgrade this afternoon. Most probably at 13:00 UTC (one hour from 
now).


The issue has been confirmed to be preexistent and due to a double 
redirect in a Wikidata item.


I am deploying 1.40.0-wmf.9 to all wikis right now.

--
Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: [Ops] [Train] 1.41.0-wmf.9 status update: blocked at group1 on T336962, T336964

[Antoine Musso]

Le 18/05/2023 à 21:00, Brennen Bearnes a écrit :

The 1.41.0-wmf.9 version of MediaWiki is blocked[0].

The new version is currently deployed to group1, but can proceed no 
further until these issues are resolved:


* T336962 - UnexpectedValueException: Unknown image suggestions API 
kind: istype-depicts

- https://phabricator.wikimedia.org/T336962

* T336964 - InvalidArgumentException: Data for lt_namespace and lt_title 
must be non-empty

- https://phabricator.wikimedia.org/T336964

Once these issues are resolved train can resume.

Thank you for any help resolving these issues!

-- Your disgruntled train drivers

[0]. https://phabricator.wikimedia.org/T330215
[1]. https://versions.toolforge.org/


Hello,

The first issue got fixed. For the second I have spend sometime this 
morning debugging it and apparently it is due to an item that got 
redirected twice: https://phabricator.wikimedia.org/T336952


I am seeking confirmation from people that knows more and if that 
specific error rules out the code, I will process with the MediaWiki 
train upgrade this afternoon. Most probably at 13:00 UTC (one hour from 
now).


Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Python requests broken by urllib3 version 2.x

[Antoine Musso]

Hello,

This is for python projects.

Today, May 4th, urllib3  has 
released a new major version 2.0.2 which breaks the extremely popular 
requests  library.


The fix is to pin urllib3<2 to prevent the new major version from being 
installed (example 
).


https://phabricator.wikimedia.org/T335977

Upstream issue: https://github.com/psf/requests/issues/6432


Antoine "hashar" Musso
Wikimedia Release Engineering___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: [Ops] [Train] 1.41.0-wmf.3 status update

[Antoine Musso]

Le 04/04/2023 à 17:09, Antoine Musso a écrit :


Roll out of MediaWiki 1.41.0-wmf.3 is blocked (T330209 
<https://phabricator.wikimedia.org/T330209>).


The new version is only deployed on testswiki and will not move 
forward due to:


  * T333966 - message keys shown on beta and test Wikidata
<https://phabricator.wikimedia.org/T333966>

While we had 28056 l10n entries with 1.41.0-wmf.2 we only have 18527 
entries in 1.41.0-wmf.3 or 34% less. Something is off in how we grab 
the l10n messages or how we merge them from the various sources.


Once the issue is resolved the train can resume.


Hello,

The blocker has been addressed and we have all l10n messages again 
(thank you!). I will promote group 0 to 1.41.0-wmf.3 right now.


Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] [Train] 1.41.0-wmf.3 status update

[Antoine Musso]

Hello,

Roll out of MediaWiki 1.41.0-wmf.3 is blocked (T330209 
).


The new version is only deployed on testswiki and will not move forward 
due to:


 * T333966 - message keys shown on beta and test Wikidata
   

While we had 28056 l10n entries with 1.41.0-wmf.2 we only have 18527 
entries in 1.41.0-wmf.3 or 34% less. Something is off in how we grab the 
l10n messages or how we merge them from the various sources.


Once the issue is resolved the train can resume.

Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Zuul CI status now displayed on Gerrit changes

[Antoine Musso]

Hello,

I have deployed a change to Gerrit which makes it display the ongoing 
CI/Zuul build if there is any.


If Jenkins jobs are running, you would see below the commit message some 
gray chipset with the name of the Zuul pipeline (test, gate-and-submit 
..). The Check tab shows the jobs details 
(https://phabricator.wikimedia.org/F36925186).


By exposing the Zuul CI status directly in the Gerrit web UI, people 
will notice a build error earlier. That also saves the hassle of having 
to monitor https://integration.wikimedia.org/zuul/.


There are a few glitches:

 * the way I have implemented it abuses the model proposed by Gerrit
   and in progress jobs are always considered a SUCCESS but will be
   marked as ERROR when they fail.
 * code is entirely running in the client browser. It is unable to send
   notifications or triggers any email when a build has failed. The
   EarlyWarning bot by Kosta Harlan
    does it though)
 * I am not a JavaScript developer per see but learned about TypeScript
   for static analysis and rediscovered QUnit. So at least there are
   some basic guarantees.
 * there are surely a bunch of edge cases that I have not properly handled

The code for those that are curious is at 
https://gerrit.wikimedia.org/g/operations/software/gerrit/+/refs/heads/deploy/wmf/stable-3.5/plugins/wm-zuul-status.js


If you see problems, JavaScript errors etc please paste them on 
https://phabricator.wikimedia.org/T214068 :)


Antoine "hashar" Musso


___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Gerrit upgrade March 6th 9:00 UTC

[Antoine Musso]

Hello,

I will upgrade our Gerrit instances from 3.5.4 to 3.5.5. I have 
scheduled it for Monday March 6th at 9:00 UTC after the backport and 
config window.


The service will be unavailable for a few minutes while the upgrade occurs.

Release notes: https://www.gerritcodereview.com/3.5.html#355
Our task: https://phabricator.wikimedia.org/T330663

Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: Deprecation: directly invoking maintenance scripts

[Antoine Musso]

Le 28/02/2023 à 21:42, MusikAnimal a écrit :
Hello! Where might I find documentation on the new maintenance runner 
system? I can't find any examples in the Phabricator task or the 
linked RFC, and searching for "MaintenanceRunner" or "run.php" yields 
no results on mediawiki.org . Specifically, I 
was expecting Manual:run.php 
 to exist with some 
info on how it works.


If given a good example patch, I'm happy to help write the docs :)


Hello,

run.php is more or less a wrapper so that instead of invoking:

 php maintenance/parse.php

One should use:

 maintenance/run parse

That will be released in MediaWiki 1.40 and all maintenance scripts will 
emit a warning to the console when invoked directly.


Here are some references to assist in writing the documentation:

 * invocation examples in the commit message of
   https://gerrit.wikimedia.org/r/c/mediawiki/core/+/798983
 * the warning being added
   https://gerrit.wikimedia.org/r/c/mediawiki/core/+/874922
 * T99268 - RfC: Create a proper command-line runner for MediaWiki
   maintenance tasks 

And as an extra, I have implemented a back compat layer in Quibble 
 (which is written in Python): 
https://gerrit.wikimedia.org/r/c/integration/quibble/+/875981/6/quibble/mediawiki/maintenance.py



Antoine "hashar" Musso
Wikimedia Release Engineering___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: [Proposal] Disable setting the "Lowest" Priority value in Phabricator

[Antoine Musso]

Le 27/02/2023 à 13:05, David Gerard a écrit :

Can I just note that however you word it, closing volunteers' good
faith bugs because nobody is available from the organisation right now
is an excellent way to get them never to file a bug again.


Hello,

Possibly, though laying them open for years with priority "lowest" is 
not ideal either. At least when the task is closed there is a clear 
intent it is not going to be fixed/implemented.



Note: the issue applies to all users (volunteers and paid staff from the 
various organizations participating on Phabricator)


--
Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: Gerrit, CI, Phabricator restarts

[Antoine Musso]

Le 03/01/2023 à 11:15, Antoine Musso a écrit :

Hello,

This morning (10:00 - 12:00 UTC) we are restarting Gerrit, CI servers 
and Phabricator servers. Each service will be unavailable for a few 
minutes as the servers are being rebooted.


Hello,

We have successfully restarted all the services.

--
Jelto Wodstrcil && Antoine Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Gerrit, CI, Phabricator restarts

[Antoine Musso]

Hello,

This morning (10:00 - 12:00 UTC) we are restarting Gerrit, CI servers 
and Phabricator servers. Each service will be unavailable for a few 
minutes as the servers are being rebooted.


--
Jelto Wodstrcil && Antoine Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: [Ops] [train] 1.40.0-wmf.14 status update

[Antoine Musso]

Le 14/12/2022 à 10:40, Antoine Musso a écrit :
The 1.40.0-wmf.14 version of MediaWiki is [blocked].  The new version is 
currently only deployed on group0 and will not proceed further:


* Parsoid: UnexpectedValueException: Invalid version string ""
   https://phabricator.wikimedia.org/T325137

It is most probably related to sun setting RestBase from our 
infrastructure and I have made the involved people aware of this task on 
our internal communication system ( #restbase-sunset on Slack).


[blocked] https://phabricator.wikimedia.org/T320519


Hello,

The issue has been resolved!

I will promote group 1 wikis to 1.40.0-wmf.14 at 16:30 UTC
(in a few minutes)

--
Antoine Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] [train] 1.40.0-wmf.14 status update

[Antoine Musso]

The 1.40.0-wmf.14 version of MediaWiki is [blocked].  The new version is 
currently only deployed on group0 and will not proceed further:


* Parsoid: UnexpectedValueException: Invalid version string ""
  https://phabricator.wikimedia.org/T325137

It is most probably related to sun setting RestBase from our 
infrastructure and I have made the involved people aware of this task on 
our internal communication system ( #restbase-sunset on Slack).


[blocked] https://phabricator.wikimedia.org/T320519

--
Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: Experiment nicer CI reporting in Gerrit

[Antoine Musso]

Le 05/12/2022 à 14:27, Antoine Musso a écrit :


Hello,

On our Gerrit, CI results are displayed below the commit message as a 
HTML table. It is achieved by a few lines of JavaScript which parse 
the comments. I went to write a replacement based on a system builtin 
Gerrit: Checks API 
<https://gerrit.wikimedia.org/r/Documentation/pg-plugin-checks-api.html> .




Hello,

I have deployed the new UI plugin on our Gerrit. It replaces the HTML 
table shown below the commit message by a few bubbles representing the 
state of tests for the latest patchset.


Details of the CI jobs are available in the "Checks" table next to 
"Files" and "Comments". There is a drop down that lets one browse CI 
runs for previous patchsets.


The job results are still available as comments and are the one 
triggering notifications or Verified +1/-1.  The new UI comes on top of 
that to offer a slightly better representation.


If you find issues or have questions, please either:

- reply to this thread

- reach out to me on IRC (hashar on libera.chat in #wikimedia-releng)

- file a task in Phabricator against #gerrit  ;)

Antoine "hashar" Musso

___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Experiment nicer CI reporting in Gerrit

[Antoine Musso]

Hello,

On our Gerrit, CI results are displayed below the commit message as a 
HTML table. It is achieved by a few lines of JavaScript which parse the 
comments. I went to write a replacement based on a system builtin 
Gerrit: Checks API 
<https://gerrit.wikimedia.org/r/Documentation/pg-plugin-checks-api.html> 
. An early example: https://phabricator.wikimedia.org/F35814298 .


I could use some early adopters to try out the plugin and gather some 
early feedback. If all goes well I will roll it on our Gerrit instance.


If you are curious or want to provide some feedback, below are the 
instructions to run it on your local machine.


In Chrome/Chromium, install the Gerrit Frontend Dev Helper extension 
<https://chrome.google.com/webstore/detail/gerrit-fe-dev-helper/jimgomcnodkialnpmienbomamgomglkd>. 
It is used to inject the JavaScript plugin from a locally running web 
server.


Create a new empty directory

Retrieve the JavaScript Gerrit plugin from Change 859083 
<https://gerrit.wikimedia.org/r/c/operations/software/gerrit/+/859083/>:


   curl -o wm-checks-api.js
   
'https://gerrit.wikimedia.org/r/changes/operations%2Fsoftware%2Fgerrit~859083/revisions/16/files/plugins%2Fwm-checks-api.js/download'

Retrieve a PHP router for the PHP built-in webserver. It would inject 
cross origin headers when serving a response:


   curl -o plugins-router.php
   
'https://gerrit.wikimedia.org/r/changes/operations%2Fsoftware%2Fgerrit~860885/revisions/1/files/plugins-router.php/download'

Start a PHP Webserver to serve the plugin:

   php -S 127.0.0.1:8081 plugins-router.php

Head to https://gerrit.wikimedia.org/ and enable the Gerrit FE dev 
helper plugin. The page will reload.


Click again the browser extension and a configuration popup will appear. 
Using the ADD button add an entry with:


 * Operator: injectJSPlugin
 * Destination: http://127.0.0.1:8081/wm-checks-api.js

Click SAVE. The page reloads and the plugin should have been injected 
(there would be a little red box in the bottom right of the Gerrit 
page). When browsing a change that previously had CI comments, you 
should see a Checks tab which hold the results found by the plugin.


The series of changes is in Gerrit 
https://gerrit.wikimedia.org/r/q/topic:checks-api


--
Antoine Musso
/(a good chunk of the code was written late at night over a week-end, I 
had to rerelearn JavaScript and discovered TypeScript in the process)./


___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Gerrit now forbids plain HTTP

[Antoine Musso]

Hello,

Wikimedia is deprecating and phasing out unencrypted HTTP (T238720 
<https://phabricator.wikimedia.org/T238720>), as part of this effort 
http://gerrit.wikimedia.org now yields a 403 Forbidden error.


It should not cause any issue since we have always serve Gerrit over 
HTTPS and there is a header set to instruct browser to always use that 
even when asking for a http:// URL.  It might be possible though that a 
bot or a local config relied on http:// and those unlikely case will now 
be broken.


The change is https://gerrit.wikimedia.org/r/c/operations/puppet/+/859986

If you see any issue, please poke the Phabricator task above.

Antoine Musso && Valentin Gutierrez
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Gerrit back since 11:45 UTC

[Antoine Musso]

Hello,

The Gerrit 3.5 upgrade did not work as expected. The disk partition 
filed up due to some disk backed caches filing it which broke the service.


Giuseppe and Clément stepped in to create a new disk partition and 
relocate the Gerrit files to the new partition. We restarted Gerrit at 
11:45 UTC and it is running version 3.5.4.


The follow up task for the disk space issue is:
https://phabricator.wikimedia.org/T323262

I have created a place holder incident report:
https://wikitech.wikimedia.org/wiki/Incidents/2022-11-17_Gerrit_3.5_upgrade

--
Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: Gerrit upgrade November 17th 9:00 UTC

[Antoine Musso]

Le 17/11/2022 à 10:57, Clément Goubert a écrit :

Hi,

Please stand by, Antoine is working on it.


Hello,

It is taking way longer than expected. During the upgrade the root 
partition ended up filed which caused multiple issues such as caches not 
being writable anymore, indexing of changes to the Lucene secondary 
indices failing.


I have thus stopped Gerrit entirely.

I have cleared a lot of files to free up disk (old kernels, obsolete 
core dumps etc). I am currently running a full reindexing of all 847549 
changes to ensure they are properly known and up-to-date with the 
version expected by Gerrit 3.5.


An alternatively would have been to rollback to Gerrit 3.4 which would 
have required a full reindexing to downgrade the schema. I have chose to 
pursue the upgrade.


I don't have a good estimate at how long it will take. As I write this 
it has done 740k changes out of 847k (87%).


Antoine Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Gerrit upgrade November 17th 9:00 UTC

[Antoine Musso]

Hello,

I will upgrade our Gerrit from 3.4.8 to 3.5.4. I have scheduled it for 
tomorrow November 17th at 9:00 UTC after the backport window.


The service will be unavailable for a few minutes while the upgrade occurs.

Release notes: https://www.gerritcodereview.com/3.5.html
Our task: https://phabricator.wikimedia.org/T307334

--
Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: [Ops] [Train] 1.40.0-wmf.6 status update

[Antoine Musso]

Le 18/10/2022 à 11:37, Antoine Musso a écrit :


Hello,

The 1.40.0-wmf.6 version of MediaWiki is blocked (T320511 
<https://phabricator.wikimedia.org/T320511>). Upon deployment to 
group0 wikis I found out CheckUser to be broken:


Unknown column 'cuc_private' in 'field list'

I have filed it as T321041 
<https://phabricator.wikimedia.org/T321041>. I am guessing CheckUser 
should be taught to not insert the cuc_private field unless needed and 
long term the schema change should be applied to all databases.


I have thus blocked the train entirely until the issue get resolved.

The issue has been fixed by reverting a change. A follow up task will be 
filed to address the lack of cuc_private field in our wiki databases.


I have promoted group 0 wikis to 1.40.0-wmf.6


Antoine "hashar" Musso
Wikimedia Release Engineering___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] [Train] 1.40.0-wmf.6 status update

[Antoine Musso]

Hello,

The 1.40.0-wmf.6 version of MediaWiki is blocked (T320511 
). Upon deployment to group0 
wikis I found out CheckUser to be broken:


   Unknown column 'cuc_private' in 'field list'

I have filed it as T321041 . 
I am guessing CheckUser should be taught to not insert the cuc_private 
field unless needed and long term the schema change should be applied to 
all databases.


I have thus blocked the train entirely until the issue get resolved.


Antoine "hashar" Musso
Wikimedia Release Engineering

___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: TDF is looking for community representatives

[Antoine Musso]

Le 15/10/2022 à 09:26, Dušan Kreheľ a écrit :

Where is the working space of the technical forum? The someone chat,
wiki discussion, mailist(s) or the the forum somewhere?


Hello Dušan,

As far as I know, the working space is a Google Drive (which still 
allows to share a folder/documents with people outside of the WMF).


There is a small chat channel on the WMF private Slack, which is merely 
to sync up about the process or state a new document got proposed. It is 
not a general work area with lot of discussions.


There is a public email TDFsupport@ (which is a Google group mailing 
list), though it receives low traffic and I don't think its content is 
publicly available.



The original intent in 2020 (see [background]) stated the process would 
be held on Phabricator and there is a tag there:

https://phabricator.wikimedia.org/tag/tech-decision-forum/

It gives a quick overview but the bulk of the work is stored in Google 
Drive and is thus not accessible.


I am guessing if one wants more details they should reach the author of 
the proposal, a forum representative or I maybe the TDFSupport email.



[background] 
https://www.mediawiki.org/wiki/Technical_decision_making/Background


--
Antoine "hashar" Musso
Wikimedia Release Engineering
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Gerrit upgraded from 3.4.5 to 3.4.6

[Antoine Musso]

Hello,

I have upgraded Gerrit from version 3.4.5 to 3.4.6. It addresses a 
security issue and had one breaking change which should not affect us. 
It should not have any user facing change beside slightly better 
performance here and there.



Release notes: https://www.gerritcodereview.com/3.4.html#346
Our task: https://phabricator.wikimedia.org/T319513

--
Antoine "hashar" Musso
Release Engineering
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: How to specify version of MW that Jenkins should run?

[Antoine Musso]

Le 15/09/2022 à 08:55, Robert Vogel via Wikitech-l a écrit :
@Hashar: Just wondering, are there any plans to change CI configs 
after moving development/code-review from gerrit to gitlab [1]? Maybe 
some file living in the repo directly rather than using a centralized 
repo like `integration/config`?


[1] https://www.mediawiki.org/wiki/GitLab/Gerrit_to_GitLab


Hello,


In short it is complicated. Essentially, YES, GitLab provides a way to 
define your CI jobs directly in the code repository (see Gitlab CI quick 
start ).



In longer form it is even more complicated. Given I like wall of texts, 
below is some historical context regarding migrating MediaWiki CI. It is 
rather long form and most can probably stop reading at this point.



*Current CI*


The test infrastructure for MediaWiki is build with:

- Zuul a workflow system (X happens in Gerrit, do A, B, C tasks, report 
to Gerrit)


- Jenkins freestyle jobs, which you can really think as a library of 
shell scripts running shell scripts serially. The jobs are mostly 
invoking commands inside containers (docker run )


- Docker images to provide the execution environment (such as the same 
php version and php extensions Wikimedia uses in production)



The Zuul version we use is from 2017 and has all the CI configuration 
centralized (the infamous integration/config repository) though in 
practice the job/container delegates a lot of the logic to CI entry 
points 
 
such as composer test. Zuul got frozen just before a large rework by 
upstream. It is around that time we:


- moved from a shared build environment to Docker images

- I wrote Quibble  to simplify the 
execution stack (instead of a hundred of scripts scattered in three 
different repositories)



*A digression about GitLab*


My grand idea at the time was to be able to trigger a Kubernetes job 
with a list of parameters (provided by Zuul and consumed by Quibble) and 
an execution environment (a Docker image). Then later on find out a way 
to integrate with the our tool to create container image on the fly: the 
novel Blubber . The upgrade 
to Zuul, phase out of Jenkins and moving the execution from WMCS to a 
Kubernetes cluster,  those did not happen. The foundation had other 
technical priorities and to be fair we did not have the human resources 
to achieve any of it.



A bit later we formed a working group for the future of CI, you can see 
the requirements at 
https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/CI_Futures_WG/Requirements 
, and a feedback request on wikitech-l 
. 
Different solutions got evaluated and I have required Zuul version 3 to 
be included in the evaluation at least for comparison. Defining the CI 
workflow in the code repository is referred to as "*self serve CI*". 
That became popular with Travis CI. Zuul and Jenkins projects 
implemented similar feature around 2017 as well (Zuul version 3 and 
Jenkins with pipeline as code).



Proof of concepts for 3 different CI systems were build in 2019 one got 
picked (Argo) and we had a nice proposal: 
https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Seakeeper_proposal 
. It is a good read as to what we envisioned to push.



Due to internal tensions at the foundation (others wanted to adopt 
GitLab, some *really* hate Gerrit, others would rather use GitHub), our 
Seakeeper proposal got shot at some point late in 2019.  I was not in 
any of this discussions since I had other matters to deal with and went 
briefly out of job due to some Kafkaesque 
 legal situation. Eventually 
Covid has hit early in 2020. In short I don't think I had the mental 
state to be involved or consulted in any of those discussions (nor do I 
think anyone would have thought of reaching out to the /de facto/ 
internal CI expert, but I am exaggerating and ranting at this point. I 
apologize).



GitLab entered the foundation annual plan for July 2020 - June 2021.  
You can read the public announce at 
https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/message/S6GHXFS2GD2XFCDXT47KDY26RFHCK7R3/ 
. There were some backslashes about lack of a public consultation (and 
rightfully so) but in the end the decision had already been made.  To my 
knowledge all people who were actively pushing for GitLab have since 
left the foundation or moved to different departments.



*Self serve CI!*


All of that to say that, in GitLab, one can define the CI workflow 
directly from the repository. I think the documentation is 
https://docs.gitlab.com/ee/ci/pipelines/merge_request_pipelines.html and 
that system is entirely different than our current 

[Wikitech-l] Re: How to specify version of MW that Jenkins should run?

[Antoine Musso]

Le 12/09/2022 à 12:20, Sebastian Berlin a écrit :
I'm having some issues with Jenkins running the latest version of MW 
for an extension (Wikispeech 
). Jenkins run 
with the 1.40 alpha, but we want the extension to support the latest 
LTS (1.35 as of right now).


I tried changing in extension.json to:
"requires": {
 "MediaWiki": "1.35.*"
}
which caused an error for Jenkins:

12:44:09 A dependency error was encountered while installing the
extension "Wikispeech": Wikispeech is not compatible with the
current MediaWiki core (version 1.40.0-alpha), it requires: 1.35.*.

The whole log is here: https://phabricator.wikimedia.org/P34482.

How can I specify what version of MW to run with? I've looked at both 
the MW wiki and Wikitech, but hasn't found any instructions for this.


Hello,

There are a lot of good answers and hints in this thread. I will 
complement the answers since I am the one that has introduced the CI 
system currently used for MediaWiki and I used to be in the Wikimedia 
Foundation team which was responsible for MediaWiki development 
(Platform Engineering, disbanded in 2015).


MediaWiki is primarily developed for the Wikimedia project and the focus 
on CI is to ensure that the master branch of MediaWiki core, extensions, 
skins and their composer dependencies are working well together.  The CI 
jobs prefixed "wmf-quibble" clone a few dozen of extensions and are 
triggered whenever one send a patch to one of those extensions or 
mediawiki/core. Hence we have more or less a guarantee that the master 
branches of the participating repositories are all working together.


For the deployment to the Wikimedia cluster, we cut a new deployment 
branch (wmf/XXX) directly from the master branch. Since a change could 
only have been merged in the master branch if all the CI jobs passed, we 
are pretty sure that the set of repositories with their wmf branches are 
passing tests. There are some caveats still but that is the rough idea.


The MediaWiki releases are generated from the REL* branches and CI 
follows the same logic. If one send a patch to their extensions REL1_35 
branch, CI checkout the REL1_35 branch of mediawiki/core and any other 
repositories participating in the job.


Thus if you send a patch to WikiSpeech REL1_35 you would get 
mediawiki/core and Vector at REL1_35 as well ensuring everything works 
well together (and by everything, I mean what ever is covered by tests). 
But if you send a patch for the master branch, it is the master branch 
of all repositories being checked out. In all case the value(s) of the 
requires field in extension.json does not affect the branch used in CI. 
It is entirely based on the convention that all repos are using the same 
branch.


There are lot of use cases for developers ensuring the master branch is 
compatible with the current development version (master) AND keeping 
back compatibility all the way up to the latest LTS. But that is not 
enforced by CI. I know Translate does it, most probably through manual 
testing.



What we could potentially do is create a job which would inspect the 
extension requires field to get the target branch to test (so if one 
requires 1.35, target the REL1_35 branch) then invoke the testrunner 
with that branch override (--branch REL1_35). Then if you send a patch 
to WikiSpeech master branch, such a job would test its code against 
mediawiki/core / vector using REL1_35. I am not entirely sure how to 
implement it, but the best place would probably be 
https://gerrit.wikimedia.org/g/integration/quibble/ It is the code 
holding all the logic to clone/checkout repositories, setup MediaWiki 
and run the test suite.  I am guessing when a change is targeting an 
extension or skin, Quibble would clone it first, inspect the 
extension.json and then internally set the default branch (the 
`--branch` parameter). And we can have a Jenkins job using that feature 
and running along the rest of the jobs.


It will most probably be fine for extensions that do not have many other 
dependencies.


The relevant task would be https://phabricator.wikimedia.org/T196467 . I 
think the devil is defining the workflow that is needed, which is more 
or less the text above.



Antoine "hashar" Musso




___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: How to specify version of MW that Jenkins should run?

[Antoine Musso]

Hello,


The DonationInterface as its own custom job. It is developed against the 
master branch but is deployed using the "deployment" branch on a FORK of 
MediaWiki 1.35 done under the fundraising/REL1_35 branch. To support 
that unique workflow, Quibble (the MediaWiki test runner I wrote to 
"simplify" CI) is given a branch override to have it checkout REL1_35 by 
default instead of master and to use fundraising/REL1_35 for the project 
having it. The job definition is at 
https://gerrit.wikimedia.org/r/plugins/gitiles/integration/config/+/refs/heads/master/jjb/wm-fundraising.yaml#93 
which when expanded would look something such as:


 quibble --branch REL1_35
    --project-branch 
mediawiki/extensions/DonationInterface=master

    --project-branch "mediawiki/core=fundraising/REL1_35"
    --project-branch "mediawiki/vendor=fundraising/REL1_35"
    mediawiki/vendor
    mediawiki/extensions/DonationInterface
    mediawiki/extensions/FundraisingEmailUnsubscribe
    mediawiki/extensions/ParserFunctions
    mediawiki/extensions/cldr

 Thus if one sends a change to:

- mediawiki/core fundraising/REL1_35 we will checkout DonationInterface 
master branch but cldr at REL1_35


- DonationInterface master, we checkout mediawiki/core 
fundraising/REL1_35 but cldr at REL1_35


It is a bit complicated but luckily the jobs only need to be altered 
when a new LTS is out, and usually it only implies bumping the version 
of the REL branch and ensuring the fundraising/REL* branch has been 
created on the repositories.




Antoine "hashar" Musso


Le 12/09/2022 à 16:54, Robert Vogel via Wikitech-l a écrit :
The "DonationInterface" has some "special handling" in the CI config: 
https://github.com/wikimedia/integration-config/blob/d2ef596ea8f697c0920270939c1bf967e0be/zuul/layout.yaml#L4464-L4480


They apparently use some dedicated Quibble test runner: 
https://github.com/wikimedia/integration-config/blob/d2ef596ea8f697c0920270939c1bf967e0be/jjb/wm-fundraising.yaml#L63-L76


I don't think this pattern can easily be re-used.

*Von:* Ostrzyciel 
*Gesendet:* Montag, 12. September 2022 16:42
*An:* wikitech-l@lists.wikimedia.org 
*Betreff:* [Wikitech-l] Re: How to specify version of MW that Jenkins 
should run?

Hi all,

I've been looking into a similar thing for a different extension, 
where I wanted it to be compatible with MW 1.37+ (see task 
). Basically, I want the 
tests to run both on REL1_37 and master to make sure that whatever I 
merge is compatible with 1.37. The cherry-picking approach is not a 
valid solution, as it's tedious, error-prone, and I can never be sure 
that the patch works with both branches.


I know of one extension that seemingly has something like this: 
DonationInterface. For example, this master patch 
 
had its tests run against REL1_35.


Can something like this be replicated in other extensions? I think 
it's a vital feature for extension developers, because compatibility 
with only master is next to useless. To be honest, it boggles my mind 
that this hasn't become common practice long ago.


--
The Slightly Mind-Boggled Ostrzyciel

On 12/09/2022 16:14, Robert Vogel via Wikitech-l wrote:
The continous integration testing currently uses the same branch fot 
the "dependecies" as it tests. So if you create a change set on 
branch `master`, it will use `master` for all the other repos (like 
mw core, extensions, skins) it clones to build up the environment. I 
am not aware of any way to influence this on a per repo level.


The only way I am currently aware of is to commit against a branch 
called `REL1_35` of your extension and then cherry-pick the changes 
up to `master`, once the tests have passed. This will make the CI 
tests use the `REL1_35` branch of each dependency (including mw core).


--
Robert___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Transient CI issue due to composer packages

[Antoine Musso]

Hello,

For the last few hours we had random CI build issues affecting MediaWiki 
repositories. The symptom looks like:



 [UnexpectedValueException (-1)] 


 '/workspace/src/vendor/./composer/tmp-x'
  is a corrupted zip archive (0 bytes), try again.

After looking at the whole CI stack, I highly suspect it was a transient 
issue on GitHub infrastructure which would serve some Zip assets as zero 
bytes archives.  The dependencies affected seem to have been:


wmde/hamcrest-html-matchers:v1.0.0
phpdocumentor/type-resolver:1.6.1

I don't see build failures anymore I am thus assuming the issue has been 
solved by GitHub.



For any change in Gerrit that was affected, you can comment "recheck" in 
Gerrit to trigger a rebuild.


If you have approved a patch with Code-Review +2, you would need to 
remove your Code-Review +2 vote and apply again.  If someone else 
approved a change, you can Code-Review +2 it to trigger the test and 
merge cycle.


The task:
https://phabricator.wikimedia.org/T309764

--
Antoine "hashar" Musso
Wikimedia Release Engineering
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: [Ops] [train] 1.39.0-wmf.10 status update

[Antoine Musso]

Le 04/05/2022 à 17:09, Antoine Musso a écrit :

Hello,

The 1.39.0-wmf.10 version of MediaWiki is blocked[0].

The new version is deployed to group0 but can proceed no further until 
these issue is resolved:


* wbsearchentities produces no results - 
https://phabricator.wikimedia.org/T307586


It has been reported by Lucas Werkmeister from WMDE and I have 
immediately rolled back group1 wikis.

...

[0]. https://phabricator.wikimedia.org/T305216
[1]. https://versions.toolforge.org/
The issue T305216 has been resolved by Erik Bernhardson. It was a 
problem in CirrusSearch and has been resolved yesterday.


I am going to promote all wikis to 1.39.0-wmf.10.

--
Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] [train] 1.39.0-wmf.10 status update

[Antoine Musso]

Hello,

The 1.39.0-wmf.10 version of MediaWiki is blocked[0].

The new version is deployed to group0 but can proceed no further until 
these issue is resolved:


* wbsearchentities produces no results - 
https://phabricator.wikimedia.org/T307586


It has been reported by Lucas Werkmeister from WMDE and I have 
immediately rolled back group1 wikis.



Once the issue is resolved train can resume. If the issue is resolved on 
Friday the train will resume Monday.


Thank you for your help resolving these issues!

-- Your humble train toiler

[0]. https://phabricator.wikimedia.org/T305216
[1]. https://versions.toolforge.org/
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Gerrit upgrade April 28th - 08:00 UTC

[Antoine Musso]

Hello,

I will upgrade our Gerrit from 3.3.10 to [version 3.4.4] this Thursday 
April 28th at 8:00 UTC.  I will first upgrade the [Gerrit replica] then 
the primary server.  The service will thus be unavailable for a few 
minutes while I restart the service.



Upstream release notes highlights three new features:

1) Checks UI: we do not plan to use this feature

2) Unresolved Comments ported to latest patchset

Unresolved comments that were left on older patchsets will now also be 
shown on newer patchsets.


3) Comment Chips and Context

Comment state (resolved, unresolved, draft) is summarized by chips below 
the commit message. The Comments Tab and the Change Log will not just 
show the comment thread, but also the snippet of code (where the comment 
was made) as context.



And various small UI changes, see [version 3.4.4] for a full list.



[version 3.4.4] https://www.gerritcodereview.com/3.4.html
[Gerrit replica]
https://wikitech.wikimedia.org/wiki/Gerrit-replica.wikimedia.org
Upgrade task: https://phabricator.wikimedia.org/T292759


--
Antoine "hashar" Musso
Wikimedia Release Engineering
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] CI composer update from 2.1.8 to 2.3.3

[Antoine Musso]

Hello,

We are updating composer in the CI images from 2.1.8 to 2.3.3 which 
update most jobs relying on PHP.


The "Quibble" jobs have not been upgraded due to prerequisites tasks 
that have not been completed yet.


If you find something suspiciously related to the composer upgrade, 
please report on the upgrade task or as a subtask:


https://phabricator.wikimedia.org/T303867

Thank you!

--
James Forrester & Antoine Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: Gerrit & CI maintenance April 7th at 7:00 UTC

[Antoine Musso]

Le 06/04/2022 à 17:40, Antoine Musso a écrit :

Hello,

I will restart the Gerrit and CI services tomorrow Thursday April 7th at 
7:00 UTC.  The services will be unavailable for up to half an hour while 
the maintenance is being conducted.


I have shifted the morning UTC backport and config window by half an 
hour. The MediaWiki train for 1.39.0-wmf.36 scheduled for 8:00 UTC might 
be delayed a bit depending on how many config and backports we have to 
process.


Synchronization will be on Libera.chat IRC channel #wikimedia-operations


Hello,

I have successfully restarted Gerrit and CI services.

Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Gerrit & CI maintenance April 7th at 7:00 UTC

[Antoine Musso]

Hello,

I will restart the Gerrit and CI services tomorrow Thursday April 7th at 
7:00 UTC.  The services will be unavailable for up to half an hour while 
the maintenance is being conducted.


I have shifted the morning UTC backport and config window by half an 
hour. The MediaWiki train for 1.39.0-wmf.36 scheduled for 8:00 UTC might 
be delayed a bit depending on how many config and backports we have to 
process.


Synchronization will be on Libera.chat IRC channel #wikimedia-operations

cheers,

--
Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: [Ops] [Train] 1.39.0-wmf.1 status update

[Antoine Musso]

Le 21/03/2022 à 14:18, Jaime Nuche a écrit :

The 1.39.0-wmf.1 version of MediaWiki is blocked[0].

The new version is deployed to testwikis, but can proceed no
further until these issues are resolved:

* Wikimedia\Rdbms\DBTransactionError: Transaction round stage must be 'cursory' 
(not 'within-rollback-callbacks') -https://phabricator.wikimedia.org/T281451
* Wikimedia\Timestamp\TimestampException: 
Wikimedia\Timestamp\ConvertibleTimestamp::getTimestamp: The timestamp cannot be 
represented in the specified format -https://phabricator.wikimedia.org/T304307

Once these issues are resolved train can resume. If these issues are
resolved on a Friday the train will resume Monday.

Thank you for your help resolving these issues!

-- Your humble train toiler

[0].https://phabricator.wikimedia.org/T300203
[1]. 


Hello,

The blocker has been resolved by reverting a change made to Echo. We are 
going to promote group 1 wikis to 1.39.0-wmf.1.


cheers,

Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: [Ops] Re: CI server maintenance Jan 18 16:00 UTC

[Antoine Musso]

On 18/01/2022 17:56, Antoine Musso wrote:

Hello,

The maintenance is complete.

If you have send patches in Gerrit you can have add them back to CI by 
commenting on the change: "recheck".


Some patches would require a second Code-Review +2 in order to have them 
processed since the queue got lost as the system got restarted.


Hello,

Something broke slightly after the restart and build were no more 
processed until I restarted Zuul at 17:37 UTC.


It should be good now.


--
Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: [Ops] CI server maintenance Jan 18 16:00 UTC

[Antoine Musso]

On 17/01/2022 19:21, Antoine Musso wrote:

Hello,

The continuous integration server contint2001 will be restarted for a 
hardware maintenance on Tuesday January 18th at 16:00 UTC. During the 
maintenance, the CI systems will be unavailable:


- Jenkins
- Zuul
- https://integration.wikimedia.org/

The out-of-band management system requires an update to address 
intermittent loss of connectivity.  We have to restart the server.



Time conversions:

PST  8:00
CT  10:00
UTC 16:00
CET 17:00

The task is:
https://phabricator.wikimedia.org/T283582


Hello,

The maintenance is complete.

If you have send patches in Gerrit you can have add them back to CI by 
commenting on the change: "recheck".


Some patches would require a second Code-Review +2 in order to have them 
processed since the queue got lost as the system got restarted.


--
Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] CI server maintenance Jan 18 16:00 UTC

[Antoine Musso]

Hello,

The continuous integration server contint2001 will be restarted for a 
hardware maintenance on Tuesday January 18th at 16:00 UTC. During the 
maintenance, the CI systems will be unavailable:


- Jenkins
- Zuul
- https://integration.wikimedia.org/

The out-of-band management system requires an update to address 
intermittent loss of connectivity.  We have to restart the server.



Time conversions:

PST  8:00
CT  10:00
UTC 16:00
CET 17:00

The task is:
https://phabricator.wikimedia.org/T283582

cheers,

--
Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: [Ops] Re: [Train] 1.38.0-wmf.13 blocked

[Antoine Musso]

On 16/12/2021 18:20, Antoine Musso wrote:

On 16/12/2021 17:59, Antoine Musso wrote:


Those blocking tasks have all been addressed and the backports I have 
been deployed (Thank you everyone).


I am going to promote group 1 wikis to wmf.13 right now and will do 
rest of the wikis during the normal window at 20:00 UTC.



Hello,

We are delaying the group 1 deployment a little due to another hotfix 
going on which need some careful monitoring. Once validated, I will move 
the train forward.


Hello,

The wikis have been deployed to group 1 at 18:00 UTC and the rest of the 
wikis at 20:20 UTC.


There is no new error showing it thus look like it is a success so far.

--
Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: [Ops] Re: [Train] 1.38.0-wmf.13 blocked

[Antoine Musso]

On 16/12/2021 17:59, Antoine Musso wrote:


Those blocking tasks have all been addressed and the backports I have 
been deployed (Thank you everyone).


I am going to promote group 1 wikis to wmf.13 right now and will do 
rest of the wikis during the normal window at 20:00 UTC.



Hello,

We are delaying the group 1 deployment a little due to another hotfix 
going on which need some careful monitoring. Once validated, I will move 
the train forward.


Sync up is held on IRC #wikimedia-operations

Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: [Ops] [Train] 1.38.0-wmf.13 blocked

[Antoine Musso]

On 15/12/2021 22:41, Tyler Cipriani wrote:
Please don't scroll past this. This Wednesday, for the 1st time 
recently, we humbly ask you to defend Wikipedia's train.[0]


The 1.38.0-wmf.13  train is 
blocked.


The new version is rolled back to group0 
 until we resolve:


  * Error: Call to a member function getId() on null

  o tl;dr: users can't login
  o Already being worked on
  * PHP Notice: Array to string conversion

  o Impact unclear, maybe not a blocker


Hello,

Those blocking tasks have all been addressed and the backports I have 
been deployed (Thank you everyone).


I am going to promote group 1 wikis to wmf.13 right now and will do rest 
of the wikis during the normal window at 20:00 UTC.


Antoine "hashar" Musso

___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: [Ops] [Train]  1.38.0-wmf.9 status update

[Antoine Musso]

On 19/11/2021 17:49, Tyler Cipriani wrote:
The 1.38.0-wmf.9 version of MediaWiki is blocked on group0[0] due to a 
memory leak.


The new version is deployed to group0[1], but can proceed no
further until these issues are resolved:

* T296098 - 1.38.0-wmf.9 seems to have introduced a memory leak 



Once these issues are resolved train can resume.

Thank you for your help resolving these issues!


Hello,

It might have been related to T296063 - 4x increase in database queries 
after deploy of 1.38.0-wmf.9 to all wikis 
, though we are not entirely 
sure.  After discussions, I will roll 1.38.0-wmf.9 to all wikis now and 
we will monitor memory usage.


Antoine "hashar" Musso
Your alternate train conductor

___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Faulty CI builds for MediaWiki repositories

[Antoine Musso]

Hello,

One of the CI Jenkins jobs running for MediaWiki repositories had a 
faulty npm cache being restored. That caused some dependencies to not be 
properly installed and lead to a faulty build.


The issue apparently started yesterday Nov 8th around 17:30 UTC and 
should be now resolved as of 8:30 UTC after I have deleted the faulty cache.


The affected job is quibble-vendor-mysql-php72-selenium-docker which 
runs the Selenium browser test. If a change made to an extension or skin 
fails mysteriously, please try rechecking the CI change by commenting 
`recheck` on the change or using:


 recheck due to npm cache corruption - T295341

It has hit us a few times in the recent time:
* https://phabricator.wikimedia.org/T294426
* https://phabricator.wikimedia.org/T293937
* https://phabricator.wikimedia.org/T295341

I have filed a generic task to further investigate the issue:

* https://phabricator.wikimedia.org/T295351


--
Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l]  Summary of 1.38.0-wmf.5 train deployment

[Antoine Musso]

This email is a summary of the Wikimedia production deployment of 
1.38.0-wmf.5 which has been successfully completed last week.


 * Conductor: Antoine "hashar" Musso
 * Backup Conductor: Ahmon Dancy
 * Blocker Task: T281169 
 * Current Status 


    Numbers

Sparklines comparing with the last 5 trains.

 * 351 Patches ██▁▂█
 * 0 Rollbacks ▁▃█▁▁
 * 0 Days of delay ▁▁█▁▁
 * 3 Blockers ▄█▇█▁


   ✨ Traintastic Folks 

Thanks to folks who reported or resolved blockers:

 * Jon Robson, remembered me about JavaScript client side errors to be
   taken of
 * Elena and Joseph Seddon, flagged a blocker for group 1 ahead of time
   and got it fixed with Simone Cuomo.
 * Subramanya Sastry, who has noiticed a missed class in Parsoid
 * C. Scott, Lucas Werkmeister who promptly addressed deprecation errors
 * Ahmon Dancy for being the backup conductor and closely watching
   error logs

Antoine "hashar" Musso

___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] [train] branch got cut again with new heads

[Antoine Musso]

Hello,

The branch cut last night has erroneously been done as 1.37.0-wmf.24 
instead of bumping minor version to ge us 1.38.0-wmf.1


We have thus recut the branch roughly 2 hours ago and will deploy the 
result.  There are a few additional changes that are included and which 
can be found at:


https://www.mediawiki.org/w/index.php?diff=4819281=4818542=source


If there is one potentially not ready for deployment, please refer to it 
on the 1.38.0-wmf.1 blockers task:


https://phabricator.wikimedia.org/T281165


Your train assistant conductor

--
Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: Train status for 1.37.0-wmf.23: blocked at group 0

[Antoine Musso]

On 15/09/2021 22:53, Mukunda Modell wrote:
During today's train deployment for 1.37.0-wmf.23,  we discovered a 
couple of blocking issues which resulted in rolling back to group 0.


Here they are in all of their glory:

1. T291128 Wikimedia\Rdbms\DBQueryError: Error 1146: Table 
'arbcom_dewiki.echo_push_subscription' doesn't exist 

2. T291124 PHP Notice: Undefined index: format 



Hello,

Those issues have been promptly fixed and patches have been deployed. I 
will promote group 1 wikis to 1.37.0-wmf.23 at 13:00 UTC / 15:00 CEST.


Antoine "hashar" Musso


___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: Proposal: new structure for MediaWiki RELEASE-NOTES

[Antoine Musso]

Le 20/07/2021 à 06:06, Brian Wolff a écrit :
> We could write a custom merge driver for RELEASE-NOTES which always 
puts ‘ours’ before ’theirs’,

but I’m not sure the result will justify the investment.

Didnt someone already do that? There used to be a ruby script floating 
around for that.


The tool is: https://github.com/MatmaRex/mediawikireleasenotes-driver 
 created in 
2013. It leads me to some material we wrote at that time when the issue 
got first identified:


* 2013 RFC 
https://www.mediawiki.org/wiki/Requests_for_comment/Release_notes_automation 



* 2018 task (still open) https://phabricator.wikimedia.org/T200392 



If we go with a merge driver, it would needs to be installed on CI 
(zuul-merger does use cgit) and on Gerrit (which uses jGit).


Antoine "hashar" Musso


___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: Proposal: new structure for MediaWiki RELEASE-NOTES

[Antoine Musso]

Le 20/07/2021 à 09:48, Riccardo Coccioli a écrit :
On Tue, Jul 20, 2021 at 3:38 AM Petr Pchelko > wrote:


*Alternative solutions*

We could write a custom merge driver for RELEASE-NOTES which
always puts ‘ours’ before ’theirs’,
but I’m not sure the result will justify the investment.


Probably overkill for MediaWiki but I'd like to mention the way that 
Python developers (CPython) manages their release notes, in case it 
might be useful.
The TL;DR is that each change has a unique valid reStructuredText file 
in a specific directory and then there is a tool to merge all changes 
when a release is made.


The full process from a contributor point of view is described in [1].
The tool used to both generate the change files and merge them into a 
release file is [2].


[1] 
https://devguide.python.org/committing/#updating-news-and-what-s-new-in-python 


[2] https://pypi.org/project/blurb/ 


OpenStack has a similar tool: reno. The doc has an overview of the 
requirements: https://docs.openstack.org/reno/latest/user/design.html 
 and the usage 
doc for quick glance: 
https://docs.openstack.org/reno/latest/user/usage.html 



There is an a 30 mins presentation 
https://www.youtube.com/watch?v=tEOGJ_h0Lx0 



Short of having to introduce a Python tool to the developers, maybe the 
PHP ecosystem has a similar tool?  Or we can reach out to other high 
traffic projects and see how they are managing their changelog and maybe 
forge a common tool.


Antoine "hashar" Musso

___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: [Ops] Gerrit patch upgrade July 19 16:00 UTC

[Antoine Musso]

Le 19/07/2021 à 16:32, Antoine Musso a écrit :
We will upgrade our Gerrit today starting at 16:00 UTC to apply the 
latest bug fixes.  It will be unavailable for a few minutes when we 
restart it to the new version.


The upgrade is for [bug fixes] and will bring us from 3.2.7 to 3.2.11.

We will start with the [replica] and then the main instance.


Task: https://phabricator.wikimedia.org/T278990
[replica] https://wikitech.wikimedia.org/wiki/Gerrit/Replica
[bug fixes] https://www.gerritcodereview.com/3.2.html#3211



Hello,

Both Gerrit have been upgraded to 3.2.11.  There is a configuration 
settings that is off which we manually fixed while doing the update.  A 
patch will follow up to address it.


Meanwhile puppet is still disabled, but the Gerrit service is up and 
running.



--
Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Gerrit patch upgrade July 19 16:00 UTC

[Antoine Musso]

Hello,

We will upgrade our Gerrit today starting at 16:00 UTC to apply the 
latest bug fixes.  It will be unavailable for a few minutes when we 
restart it to the new version.


The upgrade is for [bug fixes] and will bring us from 3.2.7 to 3.2.11.

We will start with the [replica] and then the main instance.


Task: https://phabricator.wikimedia.org/T278990
[replica] https://wikitech.wikimedia.org/wiki/Gerrit/Replica
[bug fixes] https://www.gerritcodereview.com/3.2.html#3211

--
Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Quibble 1.0.0 released

[Antoine Musso]

Hello,

I have released Quibble 1.0.0 a few minutes ago.

I wrote the first line of code during the 2017 Wikimania Hackathon 
(Vienna, Austria). It has since served hundred of thousands of builds on 
our CI and Kosta Harlan rightfully pointed out it was about time to 
endorse as being stable. Hence 1.0.0.


A quick summary:

* Use glob pattern in composer.local.json
* Support for mediawiki/core composer run phpunit:entrypoint which will 
eventually replace our old phpunit.php wrapper.

* Can now connect to an existing MySQL database (--db-is-external)
* Load Parsoid from vendor as a fallback

The changelog with a few more details is at:
https://doc.wikimedia.org/quibble/changelog.html

cheers,

--
Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: Selenium tests with two instances of Mediawiki

[Antoine Musso]

Le 02/07/2021 à 10:39, karima.ra...@gmail.com a écrit :

Hello,

I would like to push my tests in Jenkins/Gerrit. For the moment, I test 
manually and with Selenium IDE my extensions :
https://www.mediawiki.org/wiki/Extension:PushAll
https://www.mediawiki.org/wiki/Extension:NamespaceData
https://www.mediawiki.org/wiki/Extension:LinkedWiki

I need to have two instances of Mediawiki with these extensions because I need 
to test transferring pages from a private wiki to a public wiki.
For the moment in local, I use two docker containers with Mediawiki (and with 
Centos7 because I install/compile also specific databases)

I have the time to do this task properly.
How would you do it if you had to?


Hello Karima,

We unfortunately do not have support to spawn multiple MediaWiki at this 
time. Not that it is impossible but we just don't have the code to be 
able to define those instances and boot them.


On CI the tests are run in a single container. In it, MediaWiki is being 
setup and installed using Quibble: https://doc.wikimedia.org/quibble/ It 
is a home made tool that roughly:


* clone a set of repositories defined in CI and get patches from Gerrit
* install the composer dependencies
* run unit test commands
* Spawn a database and web server
* run the rest of tests such as the Selenium ones

It is all done serially and Quibble only supports setting up and 
spawning a single MediaWiki instance.  In the ideal world we would rely 
on something like docker-composer to spawn a set of containers and then 
run tests in that environment, but that doesn't exist yet.


Maybe some of those tests could be written via PHPUnit? I don't know how 
to simulate two MediaWiki instances in there though :-\


--
Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: Upstream shutdowns

[Antoine Musso]

Le 02/06/2021 à 03:35, K. Peachey a écrit :
There is also one of the search systems migrating to non O/S from 
memory... elastic?


Hello,

Yes that is ElasticSearch. They have recently announced they are now 
licensing code under Server Side Public License (SSPL) 
 it is a viral 
license which in short requires that any service based on such code get 
licensed with the same license.  I am not a lawyer, but I guess that 
would imply that our whole stack switch to it as well.  As such it is 
not considered free by Debian or the Open Source Initiative (OSI).


As I understand it, the move has been made possible since all 
contributions were subject to a Contributor License Agreement (CLA) 
. So that 
although the code was placed under a free license, the CLA effectively 
grants unrestricted publishing right to an organization, they can thus 
relicense the code however they want.   If I get it right, the old code 
is still under a free license but it can also be used under the new non 
free license.    As I get it the move has been done due to frictions 
with Amazon which is providing a search service.   Amazon announced they 
would be behind the free community fork: https://www.opensearch.org/




Antoine "hashar" Musso


___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Gerrit upgrade to Java 11 - June 1st 7:00 UTC

[Antoine Musso]

Hello,

On Tuesday June 1st at 7:00 UTC we will switch Gerrit from Java 8 to 
Java 11. The system will be briefly unavailable has it restarts with the 
JVM.


The window would be roughly half an hour, the service interruption 
should be just a few minutes. If anything is suspicious we will roll 
back to Java 8 and restart which would lead another short interruption.


The task is: https://phabricator.wikimedia.org/T268225

--
Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: [Ops] Re: [train] 1.37.0-wmf.6 status update

[Antoine Musso]

Le 20/05/2021 à 19:13, Antoine Musso a écrit :

Hello,

I have pushed 1.37.0-wmf.6 to group 1 wikis earlier today at 14:00 UTC.

Other wikis will be done at 19:00 UTC (2 hours from this email).

Blocker task:https://phabricator.wikimedia.org/T281147


I have pushed 1.37.0-wmf.6 to the rest of the wikis and it was 
uneventful as far as I could tell. I just claim this week train to be 
success.


Thank you for the reports and quick patches on Wednesday to address the 
few issues we have encountered.


cheers,

--

Antoine "hashar" Musso

___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: [Ops] [train] 1.37.0-wmf.6 status update

[Antoine Musso]

Hello,

I have pushed 1.37.0-wmf.6 to group 1 wikis earlier today at 14:00 UTC.

Other wikis will be done at 19:00 UTC (2 hours from this email).

Blocker task:https://phabricator.wikimedia.org/T281147

--
Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] [train] 1.37.0-wmf.6 status update

[Antoine Musso]

Hello,

The 1.37.0-wmf.6 version of MediaWiki is blocked[0].

I had it deployed to group 1 for a few hours and rolled back after we 
found some blockers.  We can proceed no further until these issues are 
resolved:


* Special:RecentChanges in it.wikiversity dies with an internal error
  https://phabricator.wikimedia.org/T283170

* InvalidArgumentException: Unable to normalize the provided actor name 
x.y.z.v/16

  https://phabricator.wikimedia.org/T283167

They seem related to refactoring that happened in mediawiki/core.


Once these issues are resolved train can resume. If these issues are
resolved on a Friday the train will resume Monday.

Thank you for your help resolving these issues!

[0]. 
[1]. 

--
Antoine "hashar" Musso
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Quibble 0.0.47 released

[Antoine Musso]

Hello,

I have released Quibble 0.0.47 a minute ago:

* Fixes installation under Python 3.5
* Inject MediaWiki related environment variables when running a user script
* Run "composer test-some" for mediawiki/core (which will let us make 
"composer test" to run against all files as it is done on all other 
repositories).

* Test Parsoid as if it were an extension

And that is about it. The complete changelog is at:
https://doc.wikimedia.org/quibble/changelog.html

The CI jobs will eventually be updated to it over the next hours / days.

cheers,

--
Antoine "hashar" Musso


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] [Ops] [train] Rollback of 1.36.0-wmf.37

[Antoine Musso]

Le 02/04/2021 à 12:02, Antoine Musso a écrit :
I have rolled back the deployment of 1.36.0-wmf.37 from all wikis since 
Special:Export was broken:


* Special:Export broken: always generates an empty file
   https://phabricator.wikimedia.org/T278579


Hello,

The Special:Export issue has been fixed by Amir Sarabadani and Sam Reed 
and I have pushed 1.36.0-wmf.37 again to all wikis at 13:42 UTC.


Thank you for the quick action!

--
Antoine "hashar" Musso

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] [train] Rollback of 1.36.0-wmf.37

[Antoine Musso]

Hello,

I have rolled back the deployment of 1.36.0-wmf.37 from all wikis since 
Special:Export was broken:


* Special:Export broken: always generates an empty file
  https://phabricator.wikimedia.org/T278579

The issue is reproducible on the beta cluster infrastructure and got 
spotted there last Friday:

https://en.wikipedia.beta.wmflabs.org/wiki/Special:Export/Main_Page


Once the issue is resolve the train can resume.

If anyone has any lead as to why Special:Export might silently fail, any 
help is appreciated.



wmf.37 blocking task: https://phabricator.wikimedia.org/T278343

cheers,

--
Antoine "hashar" Musso

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] [Train] 1.36.0-wmf.36 rolled back from group 1/2

[Antoine Musso]

Hello,

The 1.36.0-wmf.36 version of MediaWiki is blocked. The few blockers from 
Wednesday got addressed and we rolled to group 1 again.  Today I have 
deployed to group 2 and then finally rolled back BOTH group 1 and 2.


The new blockers:

* FlaggedRevs: PHP Notice: Undefined index: status
  https://phabricator.wikimedia.org/T278478

Impact unknown to me. It might well cause user facing issue on enwiki. 
That is worth rolling back group 2.



* Translate\PageTranslation\ParserOutput::sourcePageTextForRendering() 
must be an instance of Language, instance of StubUserLang given

  https://phabricator.wikimedia.org/T278429

Which I filed earlier today as part of log triage. We did not 
immediately recognized it prevented edits on pages using Translate. 
Notably on metawiki.   That is worth rolling back group 1.



* User not found by actor ID: [id]
  https://phabricator.wikimedia.org/T277795

Which is from last week. It still had at least one occurrence for the 
brief time we had 1.36.0-wmf.36 on all wikis.


Once these issues are resolved the train can resume and we can try 
pushing 1.36.0-wmf.36 to all wikis.



1.36.0-wmf.36 general task:
https://phabricator.wikimedia.org/T274940

Thank you for your assistance.


--
Antoine "hashar" Musso

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] [Ops] [Train] 1.36.0-wmf.36 blocked

[Antoine Musso]

Le 24/03/2021 à 20:41, Antoine Musso a écrit :

Hello,

The 1.36.0-wmf.36 version of MediaWiki is blocked.  Immediately after 
pushing it to group 1 wikis, it encountered three blockers:


* Class 'GlobalUsageHooks' not found
   https://phabricator.wikimedia.org/T278375

* Constructing RevisionRecord for a page that can't exist:
   Special:MyLanguage/Main Page
   [Called from MediaWiki\Revision\MutableRevisionRecord::__construct]
   https://phabricator.wikimedia.org/T278376

* Argument 1 passed to 
ProofreadPage\Index\IndexTemplateStyles::__construct() must be a Title, 
null given.

   https://phabricator.wikimedia.org/T278379


Hello,

The blockers have been addressed, either via a patch, a revert or 
determined to not cause any issue.


I will promote 1.36.0-wmf.36 to group 1 wikis now.

Thank you!

--
Antoine "hashar" Musso

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] [Train] 1.36.0-wmf.36 blocked

[Antoine Musso]

Hello,

The 1.36.0-wmf.36 version of MediaWiki is blocked.  Immediately after 
pushing it to group 1 wikis, it encountered three blockers:


* Class 'GlobalUsageHooks' not found
  https://phabricator.wikimedia.org/T278375

* Constructing RevisionRecord for a page that can't exist:
  Special:MyLanguage/Main Page
  [Called from MediaWiki\Revision\MutableRevisionRecord::__construct]
  https://phabricator.wikimedia.org/T278376

* Argument 1 passed to 
ProofreadPage\Index\IndexTemplateStyles::__construct() must be a Title, 
null given.

  https://phabricator.wikimedia.org/T278379


The later had the most log entries and apparently prevents page from 
being submitted on wikisources wiki.



Once these issues are resolved the train can resume and we can try 
pushing to group 1 wikis again.



1.36.0-wmf.36 general task:
https://phabricator.wikimedia.org/T274940


Thank you for your assistance.

--
Antoine "hashar" Musso

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Upgrade of CI jobs to Quibble 0.0.46

[Antoine Musso]

Hello,

On Monday 02/15 I will start upgrading Quibble on the CI jobs to version 
0.0.46. The notable changes for CI are:


* Run phpunit-unit stage before MediaWiki installation.
  T266441 Kosta Harlan
* Fix regression which made us run linters for repositories besides 
MediaWiki extensions or skins (eg: mediawiki/services/parsoid).

  T263500 Antoine Musso
* Mute zuul.CloneMapper logging when running browser tests.
  Antoine Musso

It opens the way to use Apache instead of the php built-in webserver. 
That would make the Selenium tests slightly faster, but we have to add 
Apache to the images, that will be done later.


The full changelog is:
https://doc.wikimedia.org/quibble/changelog.html

The task tracking the upgrade is:
https://phabricator.wikimedia.org/T274590

cheers,

--
Antoine Musso

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] tox upgraded on CI from 3.10 to 3.21.4

[Antoine Musso]

Hello,

For python projects, tox has been upgraded from 3.10 to 3.21.4.  That 
was notably to address an issue when python3.9 is defined as an 
environment although it is not available in our image, tox would 
fallback to whatever default python 3 instead of failing (T274232).


You can read the full tox changelog at:

https://tox.readthedocs.io/en/latest/changelog.html

If there is any issue, please file a task in Phabricator against 
#continuous-integration-config


--
Antoine "hashar" Musso

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] [Ops] [Train] 1.36.0-wmf.29 to group0/1 at 14:00 UTC

[Antoine Musso]

Le 09/02/2021 à 11:38, Antoine Musso a écrit :

Hello,

The 1.36.0-wmf.29 update had been blocked on a caching issue apparently 
caused by the FeaturedFeeds extension:


* MemcachedPeclBagOStuff: Serialization of 'Closure' is not allowed
- https://phabricator.wikimedia.org/T273242

The fix got reviewed and backported today and I will thus promote 
1.36.0-wmf.29 to group 0 and immediately after to group 1 starting at 
14:00 UTC today (15:00 CET).


If all goes well, I guess I can try to promote all the remaining wiki as 
well or we might do it later today (at the usual 20:00 UTC).


The patch we had for FeaturedFeeds was not working and we reverted it. 
1.36.0-wmf.29 has been promoted to group 1 and so far the issue has not 
surfaced.


I am going to promote the rest of the wikis now.

--
Antoine "hashar" Musso
Wikimedia Release Engineering

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] [Train] 1.36.0-wmf.29 to group0/1 at 14:00 UTC

[Antoine Musso]

Hello,

The 1.36.0-wmf.29 update had been blocked on a caching issue apparently 
caused by the FeaturedFeeds extension:


* MemcachedPeclBagOStuff: Serialization of 'Closure' is not allowed
- https://phabricator.wikimedia.org/T273242

The fix got reviewed and backported today and I will thus promote 
1.36.0-wmf.29 to group 0 and immediately after to group 1 starting at 
14:00 UTC today (15:00 CET).


If all goes well, I guess I can try to promote all the remaining wiki as 
well or we might do it later today (at the usual 20:00 UTC).



--
Antoine "hashar" Musso
Wikimedia Release Engineering


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] [train] 1.36.0-wmf.29 last blocker

[Antoine Musso]

Hello,

The last blocker for 1.36.0-wmf.29 is:

* MemcachedPeclBagOStuff: Serialization of 'Closure' is not allowed
 - https://phabricator.wikimedia.org/T273242


It comes from the extension FeaturedFeeds which attempt to serialize 
some User objects in the cache. An issue we already identified in a 
previous train when we marked User as not being serializable:

 https://phabricator.wikimedia.org/T264391


This time it is unknown what caused the regression (most probably it is 
due to a change in mediawiki/core).  Anyway there is a patch up for 
review to stop serializing those objects but instead drop usage of User 
objets and cache an array of properties instead of the object itself:


https://gerrit.wikimedia.org/r/c/mediawiki/extensions/FeaturedFeeds/+/661357/

Without this fixed, the train can not progress further.

cheers,

--
Antoine "hashar" Musso

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] [Ops] [Train] 1.36.0-wmf.27 - 29 status update (abandoning .28, proceeding with .29 when unblocked)

[Antoine Musso]

Hello,

This is an update for the train blockers.

Le 01/02/2021 à 22:28, Brennen Bearnes a écrit :


We're treating the following outstanding issues as blockers for wmf.29:

* ApiEchoUnreadNotificationPages.php PHP Notice: Undefined index: query
- https://phabricator.wikimedia.org/T273479


That one is due to an API query that propably errors out and thus lack a 
'query' result.  It is probably just about checking 'query' isset and 
ignoring and logging results having an 'error' key.


isset() approach:
https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Echo/+/661046

Adding log when 'query' is not there (should probably squashed in the 
previous change).

https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Echo/+/661075


* MemcachedPeclBagOStuff: Serialization of 'Closure' is not allowed
- https://phabricator.wikimedia.org/T273242


This might be due to FeaturedFeed, some classes are serialized and 
cached however they might contain a Title or an User object which are 
not serializable. Daniel Kinzler pointed at: 
https://phabricator.wikimedia.org/T264391



* Accessing WikiPage that cannot exist as a page: w:Help:Books/Book 
creator text. [Called from WikiPage::exists]

- https://phabricator.wikimedia.org/T273101


Solved!



* MimeAnalyzer::improveTypeFromExtension() must be of the type string, 
null given

- https://phabricator.wikimedia.org/T273249


Due to the introduction of a strict type. There are a few follow up 
changes, but maybe it is easier to revert.



Finally there is a new blocker since yesterday:

* Deprecation warning: Expected RevisionRecord to belong to ...
- https://phabricator.wikimedia.org/T273622

Some changes have been merged already. I guess we have to backport them 
to 1.36.0-wmf.29.



cheers,

--
Antoine "hashar" Musso

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Gerrit upgraded to v3.2.7

[Antoine Musso]

Hello,

I have upgraded our Gerrit instances from v3.2.5 to v3.2.7 today at 
10:00 UTC. There should be barely any user facing changes.



The upgrade addresses a memory leak I have found on Christmas which was 
forcing us to restart our Gerrit on a monthly basis. For reference:

 https://phabricator.wikimedia.org/T263008


If you find a regression please do file a new task in Phabricator 
against #gerrit


Thank you!

--
Antoine "hashar" Musso

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] [Ops] [Train] 1.36.0-wmf.28 completion

[Antoine Musso]

Le 01/02/2021 à 11:33, Antoine Musso a écrit :

Hello,

1.36.0-wmf.28 had to be rolled back entirely on Friday due to a faulty 
hard deprecation of some of MediaWiki interfaces.


Backport patches have/are being merged right now. I will push the train 
to group 0 as soon as that has completed.


I will push to group 1 13:00 UTC / 14:00 CET
(just after the backport window).

The rest of wikis will be done later tonight, at 20:00 UTC which is the 
usual American train slot.


Train blocker task: https://phabricator.wikimedia.org/T271342


Hello,

1.36.0-wmf.28 has reached group 1 wikis.

--
Antoine "hashar" Musso

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] [Train] 1.36.0-wmf.28 completion

[Antoine Musso]

Hello,

1.36.0-wmf.28 had to be rolled back entirely on Friday due to a faulty 
hard deprecation of some of MediaWiki interfaces.


Backport patches have/are being merged right now. I will push the train 
to group 0 as soon as that has completed.


I will push to group 1 13:00 UTC / 14:00 CET
(just after the backport window).

The rest of wikis will be done later tonight, at 20:00 UTC which is the 
usual American train slot.



Train blocker task: https://phabricator.wikimedia.org/T271342

--
Antoine "hashar" Musso

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Test runner Quibble 0.0.46 released

[Antoine Musso]

Hello,

I am pleased to announce the release of Quibble 0.0.46 mainly driven by 
Adam Wight && Kosta Harlan.



The major feature is support for using an external web server such as 
Apache. The php builtin server driven by Quibble serves requests 
serially and does not offer all the customization Apache can do.


The source repository has an example Dockerfile that leverage the use of 
supervisord to spawn Apache and point Quibble to it. We will roll that 
system to the CI jobs progressively over the next few weeks.


The journey started when Kosta benchmarked php vs Apache and by serving 
requests in parallel we have already addressed issues found in MediaWiki 
test suites.


Python 3.8 is officially supported, 3.4 or earlier are no more tested 
and if still using those you should really upgrade.


Running under podman (a daemonless alternative to docker) is now 
recognized as a container environment (thanks Marius Hoch).



Doc: https://doc.wikimedia.org/quibble/
Changelog: https://doc.wikimedia.org/quibble/changelog.html
Source: https://gerrit.wikimedia.org/g/integration/quibble/
Bug/features: #quibble tag in Phabricator

Quibble introduction: https://phabricator.wikimedia.org/J99

cheers,

--
Antoine "hashar" Musso

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Doxygen upgraded on CI

[Antoine Musso]

Hello,

I have upgraded Doxygen on the CI infrastructure this morning to 1.8.19.

Most repositories used 1.8.18.

MediaWiki extensions were still using 1.8.16 due to a crash occurring 
when processing Wikibase [T254465].



Changelog is available at:
https://www.doxygen.nl/manual/changelog.html#log_1_8_19


[T254465] https://phabricator.wikimedia.org/T254465


--
Antoine "hashar" Musso
Release Engineering

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] 1.36.0-wmf.12 cancelled and next week plan

[Antoine Musso]

Hello,

The wikis are still on 1.36.0-wmf.10 as we are still working on a user 
authentication issue from last week that prevented us from rolling 
1.36.0-wmf.11 [T264370].


We are effectively cancelling this week deployment of 1.36.0-wmf.12. We 
start rolling 1.36.0-wmf.11 on Tuesday and reasses from there.


The change scheduled for 1.36.0-wmf.12 would thus be deployed as part of 
the next train: 1.36.0-wmf.13.


Recap:

1.36.0-wmf.10, fully deployed Thursday, Sep 24th
1.36.0-wmf.11, rolled back Thursday, Oct 1st
1.36.0-wmf.12, scheduled this week and canceled hereby

We aim at deploying wmf.11 on Tuesday October 13rd and wmf.13 in the 
days that follow.



[T264370] User authentication security issue (Oct 1)
  https://phabricator.wikimedia.org/T264370

--
Antoine "hashar" Musso & cie
Release Engineering

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Making breaking changes without deprecation?

[Antoine Musso]

On 28/08/2020 18:26, Greg Rundlett (freephile) wrote:
> I like the idea of streamlining deprecation and avoiding the cost of
> maintaining obsolete code. I also **want** to publish my code on Gerrit.
> 
> As a 3rd-party extension developer who doesn't write a lot of code, one of
> the biggest complaints that I have is that it's "hard" to publish your work
> in Gerrit (and benefit from the visibility of being in the MediaWiki
> ecosystem). It's very easy to create a new repo at GitHub. It would be
> wonderful if there was some facility to "import" GitHub repos into Gerrit.


Hello,

Indeed repository creation is restricted to a handful of people since
the repositories are shared among every users, unlike Github in which
you have your own user/org namespace in which you can create any
repositories as you want.


A request for a new repository can be made on the wiki page:
https://www.mediawiki.org/wiki/Gerrit/New_repositories/Requests

And it can be asked to import an existing Github repository, which is
merely about:

 git clone --mirror 
 git push --mirror 

https://www.mediawiki.org/wiki/Git/Creating_new_repositories#Importing_from_an_existing_repository


>From there eventually you will get the benefit of our homegrown CI
system and routine maintenance by various bots (localization updates,
deprecation cleanup, libraries updates, latest code styling etc).


The Gerrit and Github models are not that different. While in the github
model one does:

 1) fork repository
 2) push to a user branch
 3) request pull request
 4) amend OR add commits and push until change is merged

In Gerrit that is:

 1) clone repository
 2) push to the special refs/for/
 3) **amend** and push until change is merged

I am eluding the maintenance of a serie of commits which is arguably
easier in the Github model since each pull request is for a whole branch.

There are some basics at:

https://www.mediawiki.org/wiki/Gerrit/Tutorial/tl;dr


Still, yes the repository creation is a bit annoying and maybe that can
be toolized.

-- 
Antoine "hashar" Musso

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Making breaking changes without deprecation?

[Antoine Musso]

On 31/08/2020 18:58, Bill Pirkle wrote:
> A clear and straightforward policy for getting things "in" sounds great.
> However, this might encourage the addition of extensions that are
> ultimately abandoned and which themselves become a code maintenance burden.
> We should also consider our policy for getting things "out". This is often
> a more difficult issue.

Hello,

Abandoned extensions are definitely a burden. The good news is that we
do archive them eventually.  The bulk of the work is done via the
#cleanup project in Phabricator.

Several people take care of investigating public usage, interest by past
author or maybe they got superseeded by another extension.  The bulk of
the work is done in Phabricator under #cleanup

https://phabricator.wikimedia.org/tag/projects-cleanup/


(the 'Fill an archive request' prepopulate the task form with a check
list of actions to complete in order to have a repository fully archived).

-- 
Antoine "hashar" Musso

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] CI comments filtering in Gerrit

[Antoine Musso]

Hello,

From now on, new CI messages send to Gerrit can be filtered out in the
web interface by selecting the "Only comments" filter.   That is done by
sending a review with '--tag autogenerated:ci' which get caught by the
filter.

That might help reducing the clutter on the change view, and in most
case the tests results you are interesting in are the latest which are
now displayed as a table just below the commit message (was T256575 done
by Paladox and QChris).

An example of the feature is at:
 https://phabricator.wikimedia.org/T48148#6294913

Do note that old CI messages written for the last 9 years or so are not
tagged and can not be filtered out.

If you see any bot for which messages should be filtered, please report
them on https://phabricator.wikimedia.org/T48148  or reach out to the
bot author(s) to use a tag!

-- 
Antoine "hashar" Musso


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Moving from Gerrit to GitLab

[Antoine Musso]

Le 06/07/2020 à 10:36, Federico Leva (Nemo) a écrit :
> 
> (I also note that the page references something called "OKR" which was
> not previously introduced to this list, as far as I know, including a
> specific line which is not found in any public document. Can we link a
> definition of what OKR means in the context of WMF, and ideally also
> some public document containing the referenced line? I suppose it would
> be some kind of annual or quarterly plan or something like that.)

Hello Federico,

OKR stands for "Objectives and Key Results", it is a management
framework to keep track of objectives and their achievements (=
"outcomes").  So that each department, team, individuals plan ambitious
objectives and resulting outcomes which are assessed at the end of a period.

If I remember properly the concept originates from Intel back in the
1980's and has later being adopted by Google which has let them grow
dramatically. It eventually has spread to the technology scene in the US
(LinkedIn, Uber etc).

The first occurrence for us is probably the 2015 article which stated
OKR was mean to be applied for fiscal year 2015 and onward:
https://www.mediawiki.org/wiki/Wikimedia_Engineering/OKR

Folks will correct me if needed, but I think the framework got formally
introduced to the whole foundation for fiscal year 2019-2020.
Previously we had annual and quarterly goals which is essentially the
same thing: write down what is planned and commit to do it.

You can read a short introduction by Google at:
https://rework.withgoogle.com/guides/set-goals-with-okrs/steps/introduction/

:)

-- 
Antoine "hashar" Musso




___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] 1.39.0-wmf.39 train is blocked (Flow/Sidebar issue)

[Antoine Musso]

Le 30/06/2020 à 16:51, Antoine Musso a écrit :
> Hello,
> 
> The 1.39.0-wmf.39 is blocked, immediately after deploying it to group0
> wikis, Flow topics ended up being error pages.
> ...
> /srv/mediawiki/php-1.35.0-wmf.39/extensions/ElectronPdfService/src/ElectronPdfServiceHooks.php:62
>  PHP Notice: Undefined index: print
> ...
> The breaking bug is: https://phabricator.wikimedia.org/T256761

Hello,

The root cause has been promptly fixed and 1.39.0-wmf.39 is on group 0 now.

In no particular order, thank you: Taavi "Majavah" Väänänen, Demian, Jon
Robson and Gergő "tgr" Tisza.

-- 
Antoine "hashar" Musso

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] 1.39.0-wmf.39 train is blocked (Flow/Sidebar issue)

[Antoine Musso]

Hello,

The 1.39.0-wmf.39 is blocked, immediately after deploying it to group0
wikis, Flow topics ended up being error pages.


When browsing a topic page, there are two issues bubbling up in logstash:

From VectorTemplate->buildSidebar()
* /srv/mediawiki/php-1.35.0-wmf.39/includes/language/Message.php:261
$key must be a string or an array

From Skin->buildSidebar / ElectronPdfServiceHooks::onSidebarBeforeOutput

*
/srv/mediawiki/php-1.35.0-wmf.39/extensions/ElectronPdfService/src/ElectronPdfServiceHooks.php:62
 PHP Notice: Undefined index: print

Which as I understand it causes an exception.

The breaking bug is: https://phabricator.wikimedia.org/T256761

-- 
Antoine "hashar" Musso
Release Engineering


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] [Train] 1.35.0-wmf.32 status update

[Antoine Musso]

Le 19/05/2020 à 20:57, Antoine Musso a écrit :
> The 1.35.0-wmf.32 train is deployed on group 0 / 1.
> 
> It has been blocked due to a large increase of writes on the databases
> (T247028). An issue that will need coordination with the database
> administrators for monitoring.
> 
> There is also a regression in the Vector skin which prevents adding
> links in other languages when the page does not have any links yet. T252800.
> 
> * https://phabricator.wikimedia.org/T247028
> * https://phabricator.wikimedia.org/T252800
> 
> We are in a bit odd position since this week had no train deployment
> planned. We might try to push 1.35.0-wmf.32 to the remaining wikis this
> week, but most probably we might end up delaying until Monday May 25th.
> 
> cheers,

Hello,

I have just found out that Monday 25th is a no deploy day due an holiday
in the US. I will thus push 1.35.0-wmf.32 to all wikis on Tuesday 25th at:

* 13:00 UTC
* 06:00 PDT
* 15:00 CEST

The blocker task is:
* https://phabricator.wikimedia.org/T249964

cheers,

-- 
Antoine "hashar" Musso

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] [Train] 1.35.0-wmf.32 status update

[Antoine Musso]

Hello,

The 1.35.0-wmf.32 train is deployed on group 0 / 1.

It has been blocked due to a large increase of writes on the databases
(T247028). An issue that will need coordination with the database
administrators for monitoring.

There is also a regression in the Vector skin which prevents adding
links in other languages when the page does not have any links yet. T252800.

* https://phabricator.wikimedia.org/T247028
* https://phabricator.wikimedia.org/T252800

We are in a bit odd position since this week had no train deployment
planned. We might try to push 1.35.0-wmf.32 to the remaining wikis this
week, but most probably we might end up delaying until Monday May 25th.

cheers,

-- 
Antoine "hashar" Musso

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] [Train] 1.35.0-wmf.32 status update

[Antoine Musso]

Hello,

The 1.35.0-wmf.32 version of MediaWiki is blocked.

The new version is deployed to testwikis, groups 0 and 1 but can not
proceed any further until these issues are resolved:

* https://phabricator.wikimedia.org/T252803
Wikibase AffectedPagesFinder: Call to a member function exists() on null

* https://phabricator.wikimedia.org/T247028
Database 'INSERT' query rate doubled (module_deps regression?)


The last one is a bit scary, it happened during the previous weeks as
well but this time it seems to cause a lot more troubles.

Thank you for your help resolving these issues!

-- 
Antoine "hashar" Musso

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] CI downtime Monday May 11st 12:00 UTC

[Antoine Musso]

Hello,

We tried the upgrade and unfortunately had to rollback.

The operation itself went smoothly (rsync, puppet, dns) but we have hit
an issue with Jenkins no more being able to allocate an instance to run
jobs on.

I suspect it might be related to the java upgrade (8 to 11). I will try
to find the root cause and we will schedule a new maintenance window.

For patches missing a CI result, you can retrigger it by commenting in
Gerrit "recheck"  or for changes that had Code-Review +2, remove your
vote and apply it again to trigger the test and merge.

Sorry for the inconvenience.

-- 
Antoine Musso



Le 07/05/2020 à 17:19, Antoine Musso a écrit :
> Hello,
> 
> We need to upgrade the operating system on the server hosting the CI
> systems (Zuul, Jenkins).  We will switch the services from contint1001
> (eqiad) to contint2001 (codfw) which has been upgraded to Buster.
> 
> The maintenance is scheduled for Monday May 11st 14:00 CEST (12:00 UTC),
> just after the European SWAT window.
> 
> The maintenance window is TWO HOURS, but probably the switch will be
> complete in a shorter time frame.
> 
> 
> During the maintenance, CI will not be available: no tests will be run
> and events (code-review +2, rebase, new patchsets) will not be captured.
> 
> The beta cluster will NOT be automatically updated.
> 
> If a change has to happen on the configuration repositories:
> 
> * operations/puppet :
>   locally run: bundle update && bundle exec rake test
> * operations/mediawiki-config :
>   locally run: composer install && composer test
> 
> Then manually V+2 and Submit the change.
> 
> After the maintenance, Antoine will retrigger most events.
> 
> The related changes are:
> 
> https://gerrit.wikimedia.org/r/c/operations/dns/+/594480
> https://gerrit.wikimedia.org/r/c/operations/puppet/+/594477
> 
> The related ticket is:  https://phabricator.wikimedia.org/T224591
> 
> Synchronization will happen on IRC in #wikimedia-operations
> 
> Later we will switch back from contint2001 to contint1001. The goal is
> to get rid of jessie servers and run fully on buster.

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] CI downtime Monday May 11st 12:00 UTC

[Antoine Musso]

Hello,

We need to upgrade the operating system on the server hosting the CI
systems (Zuul, Jenkins).  We will switch the services from contint1001
(eqiad) to contint2001 (codfw) which has been upgraded to Buster.

The maintenance is scheduled for Monday May 11st 14:00 CEST (12:00 UTC),
just after the European SWAT window.

The maintenance window is TWO HOURS, but probably the switch will be
complete in a shorter time frame.


During the maintenance, CI will not be available: no tests will be run
and events (code-review +2, rebase, new patchsets) will not be captured.

The beta cluster will NOT be automatically updated.

If a change has to happen on the configuration repositories:

* operations/puppet :
  locally run: bundle update && bundle exec rake test
* operations/mediawiki-config :
  locally run: composer install && composer test

Then manually V+2 and Submit the change.

After the maintenance, Antoine will retrigger most events.

The related changes are:

https://gerrit.wikimedia.org/r/c/operations/dns/+/594480
https://gerrit.wikimedia.org/r/c/operations/puppet/+/594477

The related ticket is:  https://phabricator.wikimedia.org/T224591

Synchronization will happen on IRC in #wikimedia-operations

Later we will switch back from contint2001 to contint1001. The goal is
to get rid of jessie servers and run fully on buster.

cheers,

-- 
Antoine Musso && Daniel Zahn


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Archiving/deleting repos

[Antoine Musso]

On 15/04/2020 13:21, Zoran Dori wrote:
> Hello,
> can someone resolve other needed things in these task as close it as
> resolved
> 
>1. https://phabricator.wikimedia.org/T250240
>2. https://phabricator.wikimedia.org/T248760
>3. https://phabricator.wikimedia.org/T248758
>4. https://phabricator.wikimedia.org/T248765
>5. https://phabricator.wikimedia.org/T249656
>6. https://phabricator.wikimedia.org/T248965

Hello Zoran,

I have marked those repositories archived in Gerrit, removed the Github
mirror and marked all those tasks as resolved :)

Thank you for the cleanup!

-- 
Antoine "hashar" Musso


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] [Ops] [Train] 1.35.0-wmf.23 and wmf.24 status update (both blocked)

[Antoine Musso]

On 17/03/2020 20:14, Brennen Bearnes wrote:
> The 1.35.0-wmf.23 version of MediaWiki is still blocked[0].
> 
> Neither this version nor the subsequent 1.35.0-wmf.24 can proceed until
> this issue is resolved:
> 
> * T247562: Warning: Memcached::setMulti(): failed to set key
> global:segment:...
> - https://phabricator.wikimedia.org/T247562
> 
> Thanks to the many folks who have contributed debugging on T247562 so
> far.  If anyone else has any insight, further input would be certainly
> be appreciated.
> 
> -- Your erratic train enabler
> 
> [0]. 

Hello,

Eventually we have found the root cause of the issue. The local cache
was being invalidated when different wikis tried to use it which caused
MediaWiki processes to fetch a large key from the same memcached server
and overloading it.

The hotfix is deployed. I will push 1.35.0-wmf.23 to all wikis this
afternoon at or slightly after 13:00 UTC.


-- 
Antoine "hashar" Musso

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] CI Debian jobs now using Buster

[Antoine Musso]

Hello,

CI has jobs to build Debian packages, I have moved them from Jessie to
Buster instances.

Building the package is made in a build environment matching the target
distribution (using cowbuilder), but some operations are executed on the
host.  There might be side effect as a result of the upgrade.

For the utilities I can think of:

pbuilder 0.215 > 0.230.4
* which notably cause the building user to have a home directory set to
a non existent path /nonexistent

pristine-tar 1.33 > 1.46
* add support for delta version 3

git-buildpackage 0.6.22 > 0.9.14

If you see any oddity in the debian-glue* CI jobs, please fill a task in
Phabricator against #continuous-integration-infrastructure.


The migration task was:

Migrate debian-glue jobs to Buster instances
https://phabricator.wikimedia.org/T224943


-- 
Antoine "hashar" Musso

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Gerrit outdated changes

[Antoine Musso]

On 22/10/2019 23:12, Antoine Musso wrote:
> Hello,
> 
> As part of the Gerrit migration to a new server, some repositories can
> have stalled / outdated changes.  In the Gerrit UI you might find a
> change suddenly open or some recent patchsets to no more be present.
<...>
> 
> The task is: https://phabricator.wikimedia.org/T236114

Hello,

As mentioned by Tyler Cipriani in a different thread, that should be
most fixed now.

The details are on the task:
https://phabricator.wikimedia.org/T236114#5596087
https://phabricator.wikimedia.org/T236114#5597092


-- 
Antoine "hashar" Musso


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l