[Wikitech-l] Re: How to specify version of MW that Jenkins should run?

[Antoine Musso]

Le 15/09/2022 à 08:55, Robert Vogel via Wikitech-l a écrit :
@Hashar: Just wondering, are there any plans to change CI configs 
after moving development/code-review from gerrit to gitlab [1]? Maybe 
some file living in the repo directly rather than using a centralized 
repo like `integration/config`?


[1] https://www.mediawiki.org/wiki/GitLab/Gerrit_to_GitLab


Hello,


In short it is complicated. Essentially, YES, GitLab provides a way to 
define your CI jobs directly in the code repository (see Gitlab CI quick 
start ).



In longer form it is even more complicated. Given I like wall of texts, 
below is some historical context regarding migrating MediaWiki CI. It is 
rather long form and most can probably stop reading at this point.



*Current CI*


The test infrastructure for MediaWiki is build with:

- Zuul a workflow system (X happens in Gerrit, do A, B, C tasks, report 
to Gerrit)


- Jenkins freestyle jobs, which you can really think as a library of 
shell scripts running shell scripts serially. The jobs are mostly 
invoking commands inside containers (docker run )


- Docker images to provide the execution environment (such as the same 
php version and php extensions Wikimedia uses in production)



The Zuul version we use is from 2017 and has all the CI configuration 
centralized (the infamous integration/config repository) though in 
practice the job/container delegates a lot of the logic to CI entry 
points 
 
such as composer test. Zuul got frozen just before a large rework by 
upstream. It is around that time we:


- moved from a shared build environment to Docker images

- I wrote Quibble  to simplify the 
execution stack (instead of a hundred of scripts scattered in three 
different repositories)



*A digression about GitLab*


My grand idea at the time was to be able to trigger a Kubernetes job 
with a list of parameters (provided by Zuul and consumed by Quibble) and 
an execution environment (a Docker image). Then later on find out a way 
to integrate with the our tool to create container image on the fly: the 
novel Blubber . The upgrade 
to Zuul, phase out of Jenkins and moving the execution from WMCS to a 
Kubernetes cluster,  those did not happen. The foundation had other 
technical priorities and to be fair we did not have the human resources 
to achieve any of it.



A bit later we formed a working group for the future of CI, you can see 
the requirements at 
https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/CI_Futures_WG/Requirements 
, and a feedback request on wikitech-l 
. 
Different solutions got evaluated and I have required Zuul version 3 to 
be included in the evaluation at least for comparison. Defining the CI 
workflow in the code repository is referred to as "*self serve CI*". 
That became popular with Travis CI. Zuul and Jenkins projects 
implemented similar feature around 2017 as well (Zuul version 3 and 
Jenkins with pipeline as code).



Proof of concepts for 3 different CI systems were build in 2019 one got 
picked (Argo) and we had a nice proposal: 
https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Seakeeper_proposal 
. It is a good read as to what we envisioned to push.



Due to internal tensions at the foundation (others wanted to adopt 
GitLab, some *really* hate Gerrit, others would rather use GitHub), our 
Seakeeper proposal got shot at some point late in 2019.  I was not in 
any of this discussions since I had other matters to deal with and went 
briefly out of job due to some Kafkaesque 
 legal situation. Eventually 
Covid has hit early in 2020. In short I don't think I had the mental 
state to be involved or consulted in any of those discussions (nor do I 
think anyone would have thought of reaching out to the /de facto/ 
internal CI expert, but I am exaggerating and ranting at this point. I 
apologize).



GitLab entered the foundation annual plan for July 2020 - June 2021.  
You can read the public announce at 
https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/message/S6GHXFS2GD2XFCDXT47KDY26RFHCK7R3/ 
. There were some backslashes about lack of a public consultation (and 
rightfully so) but in the end the decision had already been made.  To my 
knowledge all people who were actively pushing for GitLab have since 
left the foundation or moved to different departments.



*Self serve CI!*


All of that to say that, in GitLab, one can define the CI workflow 
directly from the repository. I think the documentation is 
https://docs.gitlab.com/ee/ci/pipelines/merge_request_pipelines.html and 
that system is entirely different than our current 

[Wikitech-l] Re: How to specify version of MW that Jenkins should run?

[Sebastian Berlin]

Thanks for the explanations, Antoine. I'll have a look and see if I can
figure it out. Also agree with what Robert just said that it would be nice
if this was more easily configurable. Not sure what other extensions are
supporting, but I'm guessing not all of them are targeting latest MW alpha.

*Sebastian Berlin*
Utvecklare/*Developer*
Wikimedia Sverige (WMSE)

E-post/*E-Mail*: sebastian.ber...@wikimedia.se
Telefon/*Phone*: (+46) 0707 - 92 03 84


On Thu, 15 Sept 2022 at 08:56, Robert Vogel via Wikitech-l <
wikitech-l@lists.wikimedia.org> wrote:

> @Hashar: Just wondering, are there any plans to change CI configs after
> moving development/code-review from gerrit to gitlab [1]? Maybe some file
> living in the repo directly rather than using a centralized repo like
> `integration/config`?
>
> [1] https://www.mediawiki.org/wiki/GitLab/Gerrit_to_GitLab
>
> --
> Robert
> --
> *Von:* Antoine Musso 
> *Gesendet:* Dienstag, 13. September 2022 15:42
> *An:* Wikitech-l ; Sebastian Berlin <
> sebastian.ber...@wikimedia.se>
> *Betreff:* [Wikitech-l] Re: How to specify version of MW that Jenkins
> should run?
>
> Le 12/09/2022 à 12:20, Sebastian Berlin a écrit :
>
> I'm having some issues with Jenkins running the latest version of MW for
> an extension (Wikispeech
> ). Jenkins run with
> the 1.40 alpha, but we want the extension to support the latest LTS (1.35
> as of right now).
>
> I tried changing in extension.json to:
> "requires": {
>  "MediaWiki": "1.35.*"
> }
> which caused an error for Jenkins:
>
> 12:44:09 A dependency error was encountered while installing the extension
> "Wikispeech": Wikispeech is not compatible with the current MediaWiki core
> (version 1.40.0-alpha), it requires: 1.35.*.
>
> The whole log is here: https://phabricator.wikimedia.org/P34482.
>
> How can I specify what version of MW to run with? I've looked at both the
> MW wiki and Wikitech, but hasn't found any instructions for this.
>
> Hello,
>
> There are a lot of good answers and hints in this thread. I will
> complement the answers since I am the one that has introduced the CI system
> currently used for MediaWiki and I used to be in the Wikimedia Foundation
> team which was responsible for MediaWiki development (Platform Engineering,
> disbanded in 2015).
>
> MediaWiki is primarily developed for the Wikimedia project and the focus
> on CI is to ensure that the master branch of MediaWiki core, extensions,
> skins and their composer dependencies are working well together.  The CI
> jobs prefixed "wmf-quibble" clone a few dozen of extensions and are
> triggered whenever one send a patch to one of those extensions or
> mediawiki/core. Hence we have more or less a guarantee that the master
> branches of the participating repositories are all working together.
>
> For the deployment to the Wikimedia cluster, we cut a new deployment
> branch (wmf/XXX) directly from the master branch. Since a change could only
> have been merged in the master branch if all the CI jobs passed, we are
> pretty sure that the set of repositories with their wmf branches are
> passing tests. There are some caveats still but that is the rough idea.
>
> The MediaWiki releases are generated from the REL* branches and CI follows
> the same logic. If one send a patch to their extensions REL1_35 branch, CI
> checkout the REL1_35 branch of mediawiki/core and any other repositories
> participating in the job.
>
> Thus if you send a patch to WikiSpeech REL1_35 you would get
> mediawiki/core and Vector at REL1_35 as well ensuring everything works well
> together (and by everything, I mean what ever is covered by tests). But if
> you send a patch for the master branch, it is the master branch of all
> repositories being checked out. In all case the value(s) of the requires
> field in extension.json does not affect the branch used in CI. It is
> entirely based on the convention that all repos are using the same branch.
>
> There are lot of use cases for developers ensuring the master branch is
> compatible with the current development version (master) AND keeping back
> compatibility all the way up to the latest LTS. But that is not enforced by
> CI. I know Translate does it, most probably through manual testing.
>
>
> What we could potentially do is create a job which would inspect the
> extension requires field to get the target branch to test (so if one
> requires 1.35, target the REL1_35 branch) then invoke the testrunner with
> that branch override (--branch REL1_35). Then if you send a patch to
> WikiSpeech master branch, such a job would test its code against
> mediawiki/core / vector using REL1_35. I am not entirely sure how to
> implement it, but the best place would probably be
> https://gerrit.wikimedia.org/g/integration/quibble/  It is the code
> holding all the logic to clone/checkout repositories, setup MediaWiki and
> run the test suite.  I am guessing when a change is 

[Wikitech-l] Re: How to specify version of MW that Jenkins should run?

[Robert Vogel via Wikitech-l]

@Hashar: Just wondering, are there any plans to change CI configs after moving 
development/code-review from gerrit to gitlab [1]? Maybe some file living in 
the repo directly rather than using a centralized repo like 
`integration/config`?

[1] https://www.mediawiki.org/wiki/GitLab/Gerrit_to_GitLab

--
Robert

Von: Antoine Musso 
Gesendet: Dienstag, 13. September 2022 15:42
An: Wikitech-l ; Sebastian Berlin 

Betreff: [Wikitech-l] Re: How to specify version of MW that Jenkins should run?

Le 12/09/2022 à 12:20, Sebastian Berlin a écrit :
I'm having some issues with Jenkins running the latest version of MW for an 
extension (Wikispeech). 
Jenkins run with the 1.40 alpha, but we want the extension to support the 
latest LTS (1.35 as of right now).

I tried changing in extension.json to:
"requires": {
 "MediaWiki": "1.35.*"
}
which caused an error for Jenkins:
12:44:09 A dependency error was encountered while installing the extension 
"Wikispeech": Wikispeech is not compatible with the current MediaWiki core 
(version 1.40.0-alpha), it requires: 1.35.*.
The whole log is here: https://phabricator.wikimedia.org/P34482.

How can I specify what version of MW to run with? I've looked at both the MW 
wiki and Wikitech, but hasn't found any instructions for this.

Hello,

There are a lot of good answers and hints in this thread. I will complement the 
answers since I am the one that has introduced the CI system currently used for 
MediaWiki and I used to be in the Wikimedia Foundation team which was 
responsible for MediaWiki development (Platform Engineering, disbanded in 2015).

MediaWiki is primarily developed for the Wikimedia project and the focus on CI 
is to ensure that the master branch of MediaWiki core, extensions, skins and 
their composer dependencies are working well together.  The CI jobs prefixed 
"wmf-quibble" clone a few dozen of extensions and are triggered whenever one 
send a patch to one of those extensions or mediawiki/core. Hence we have more 
or less a guarantee that the master branches of the participating repositories 
are all working together.

For the deployment to the Wikimedia cluster, we cut a new deployment branch 
(wmf/XXX) directly from the master branch. Since a change could only have been 
merged in the master branch if all the CI jobs passed, we are pretty sure that 
the set of repositories with their wmf branches are passing tests. There are 
some caveats still but that is the rough idea.

The MediaWiki releases are generated from the REL* branches and CI follows the 
same logic. If one send a patch to their extensions REL1_35 branch, CI checkout 
the REL1_35 branch of mediawiki/core and any other repositories participating 
in the job.

Thus if you send a patch to WikiSpeech REL1_35 you would get mediawiki/core and 
Vector at REL1_35 as well ensuring everything works well together (and by 
everything, I mean what ever is covered by tests). But if you send a patch for 
the master branch, it is the master branch of all repositories being checked 
out. In all case the value(s) of the requires field in extension.json does not 
affect the branch used in CI. It is entirely based on the convention that all 
repos are using the same branch.

There are lot of use cases for developers ensuring the master branch is 
compatible with the current development version (master) AND keeping back 
compatibility all the way up to the latest LTS. But that is not enforced by CI. 
I know Translate does it, most probably through manual testing.


What we could potentially do is create a job which would inspect the extension 
requires field to get the target branch to test (so if one requires 1.35, 
target the REL1_35 branch) then invoke the testrunner with that branch override 
(--branch REL1_35). Then if you send a patch to WikiSpeech master branch, such 
a job would test its code against mediawiki/core / vector using REL1_35. I am 
not entirely sure how to implement it, but the best place would probably be 
https://gerrit.wikimedia.org/g/integration/quibble/  It is the code holding all 
the logic to clone/checkout repositories, setup MediaWiki and run the test 
suite.  I am guessing when a change is targeting an extension or skin, Quibble 
would clone it first, inspect the extension.json and then internally set the 
default branch (the `--branch` parameter). And we can have a Jenkins job using 
that feature and running along the rest of the jobs.

It will most probably be fine for extensions that do not have many other 
dependencies.

The relevant task would be https://phabricator.wikimedia.org/T196467 . I think 
the devil is defining the workflow that is needed, which is more or less the 
text above.


Antoine "hashar" Musso




___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org

[Wikitech-l] Re: How to specify version of MW that Jenkins should run?

[Antoine Musso]

Le 12/09/2022 à 12:20, Sebastian Berlin a écrit :
I'm having some issues with Jenkins running the latest version of MW 
for an extension (Wikispeech 
). Jenkins run 
with the 1.40 alpha, but we want the extension to support the latest 
LTS (1.35 as of right now).


I tried changing in extension.json to:
"requires": {
 "MediaWiki": "1.35.*"
}
which caused an error for Jenkins:

12:44:09 A dependency error was encountered while installing the
extension "Wikispeech": Wikispeech is not compatible with the
current MediaWiki core (version 1.40.0-alpha), it requires: 1.35.*.

The whole log is here: https://phabricator.wikimedia.org/P34482.

How can I specify what version of MW to run with? I've looked at both 
the MW wiki and Wikitech, but hasn't found any instructions for this.


Hello,

There are a lot of good answers and hints in this thread. I will 
complement the answers since I am the one that has introduced the CI 
system currently used for MediaWiki and I used to be in the Wikimedia 
Foundation team which was responsible for MediaWiki development 
(Platform Engineering, disbanded in 2015).


MediaWiki is primarily developed for the Wikimedia project and the focus 
on CI is to ensure that the master branch of MediaWiki core, extensions, 
skins and their composer dependencies are working well together.  The CI 
jobs prefixed "wmf-quibble" clone a few dozen of extensions and are 
triggered whenever one send a patch to one of those extensions or 
mediawiki/core. Hence we have more or less a guarantee that the master 
branches of the participating repositories are all working together.


For the deployment to the Wikimedia cluster, we cut a new deployment 
branch (wmf/XXX) directly from the master branch. Since a change could 
only have been merged in the master branch if all the CI jobs passed, we 
are pretty sure that the set of repositories with their wmf branches are 
passing tests. There are some caveats still but that is the rough idea.


The MediaWiki releases are generated from the REL* branches and CI 
follows the same logic. If one send a patch to their extensions REL1_35 
branch, CI checkout the REL1_35 branch of mediawiki/core and any other 
repositories participating in the job.


Thus if you send a patch to WikiSpeech REL1_35 you would get 
mediawiki/core and Vector at REL1_35 as well ensuring everything works 
well together (and by everything, I mean what ever is covered by tests). 
But if you send a patch for the master branch, it is the master branch 
of all repositories being checked out. In all case the value(s) of the 
requires field in extension.json does not affect the branch used in CI. 
It is entirely based on the convention that all repos are using the same 
branch.


There are lot of use cases for developers ensuring the master branch is 
compatible with the current development version (master) AND keeping 
back compatibility all the way up to the latest LTS. But that is not 
enforced by CI. I know Translate does it, most probably through manual 
testing.



What we could potentially do is create a job which would inspect the 
extension requires field to get the target branch to test (so if one 
requires 1.35, target the REL1_35 branch) then invoke the testrunner 
with that branch override (--branch REL1_35). Then if you send a patch 
to WikiSpeech master branch, such a job would test its code against 
mediawiki/core / vector using REL1_35. I am not entirely sure how to 
implement it, but the best place would probably be 
https://gerrit.wikimedia.org/g/integration/quibble/ It is the code 
holding all the logic to clone/checkout repositories, setup MediaWiki 
and run the test suite.  I am guessing when a change is targeting an 
extension or skin, Quibble would clone it first, inspect the 
extension.json and then internally set the default branch (the 
`--branch` parameter). And we can have a Jenkins job using that feature 
and running along the rest of the jobs.


It will most probably be fine for extensions that do not have many other 
dependencies.


The relevant task would be https://phabricator.wikimedia.org/T196467 . I 
think the devil is defining the workflow that is needed, which is more 
or less the text above.



Antoine "hashar" Musso




___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: How to specify version of MW that Jenkins should run?

[Antoine Musso]

Hello,


The DonationInterface as its own custom job. It is developed against the 
master branch but is deployed using the "deployment" branch on a FORK of 
MediaWiki 1.35 done under the fundraising/REL1_35 branch. To support 
that unique workflow, Quibble (the MediaWiki test runner I wrote to 
"simplify" CI) is given a branch override to have it checkout REL1_35 by 
default instead of master and to use fundraising/REL1_35 for the project 
having it. The job definition is at 
https://gerrit.wikimedia.org/r/plugins/gitiles/integration/config/+/refs/heads/master/jjb/wm-fundraising.yaml#93 
which when expanded would look something such as:


 quibble --branch REL1_35
    --project-branch 
mediawiki/extensions/DonationInterface=master

    --project-branch "mediawiki/core=fundraising/REL1_35"
    --project-branch "mediawiki/vendor=fundraising/REL1_35"
    mediawiki/vendor
    mediawiki/extensions/DonationInterface
    mediawiki/extensions/FundraisingEmailUnsubscribe
    mediawiki/extensions/ParserFunctions
    mediawiki/extensions/cldr

 Thus if one sends a change to:

- mediawiki/core fundraising/REL1_35 we will checkout DonationInterface 
master branch but cldr at REL1_35


- DonationInterface master, we checkout mediawiki/core 
fundraising/REL1_35 but cldr at REL1_35


It is a bit complicated but luckily the jobs only need to be altered 
when a new LTS is out, and usually it only implies bumping the version 
of the REL branch and ensuring the fundraising/REL* branch has been 
created on the repositories.




Antoine "hashar" Musso


Le 12/09/2022 à 16:54, Robert Vogel via Wikitech-l a écrit :
The "DonationInterface" has some "special handling" in the CI config: 
https://github.com/wikimedia/integration-config/blob/d2ef596ea8f697c0920270939c1bf967e0be/zuul/layout.yaml#L4464-L4480


They apparently use some dedicated Quibble test runner: 
https://github.com/wikimedia/integration-config/blob/d2ef596ea8f697c0920270939c1bf967e0be/jjb/wm-fundraising.yaml#L63-L76


I don't think this pattern can easily be re-used.

*Von:* Ostrzyciel 
*Gesendet:* Montag, 12. September 2022 16:42
*An:* wikitech-l@lists.wikimedia.org 
*Betreff:* [Wikitech-l] Re: How to specify version of MW that Jenkins 
should run?

Hi all,

I've been looking into a similar thing for a different extension, 
where I wanted it to be compatible with MW 1.37+ (see task 
). Basically, I want the 
tests to run both on REL1_37 and master to make sure that whatever I 
merge is compatible with 1.37. The cherry-picking approach is not a 
valid solution, as it's tedious, error-prone, and I can never be sure 
that the patch works with both branches.


I know of one extension that seemingly has something like this: 
DonationInterface. For example, this master patch 
 
had its tests run against REL1_35.


Can something like this be replicated in other extensions? I think 
it's a vital feature for extension developers, because compatibility 
with only master is next to useless. To be honest, it boggles my mind 
that this hasn't become common practice long ago.


--
The Slightly Mind-Boggled Ostrzyciel

On 12/09/2022 16:14, Robert Vogel via Wikitech-l wrote:
The continous integration testing currently uses the same branch fot 
the "dependecies" as it tests. So if you create a change set on 
branch `master`, it will use `master` for all the other repos (like 
mw core, extensions, skins) it clones to build up the environment. I 
am not aware of any way to influence this on a per repo level.


The only way I am currently aware of is to commit against a branch 
called `REL1_35` of your extension and then cherry-pick the changes 
up to `master`, once the tests have passed. This will make the CI 
tests use the `REL1_35` branch of each dependency (including mw core).


--
Robert___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: How to specify version of MW that Jenkins should run?

[Ostrzyciel]

...and it's not likely that Wikimedia would want to investigate that, 
because helping devs make their extensions compatible and secure across 
MW versions is not within the scope of the Foundation's mission.


It's really sad to me that MW has CI that only suits WMF's use of the 
software, and is inadequate to the needs of third party wikis. It is 
also quite frustrating to me, because it means I am forced to move my 
extension to GitHub or GitLab where I can set up CI that meets the bare 
minimum requirements for professionally maintaining an extension. Ugh :(


--
The Very Disappointed And Rather Grumpy Ostrzyciel

On 12/09/2022 16:54, Robert Vogel via Wikitech-l wrote:
The "DonationInterface" has some "special handling" in the CI config: 
https://github.com/wikimedia/integration-config/blob/d2ef596ea8f697c0920270939c1bf967e0be/zuul/layout.yaml#L4464-L4480


They apparently use some dedicated Quibble test runner: 
https://github.com/wikimedia/integration-config/blob/d2ef596ea8f697c0920270939c1bf967e0be/jjb/wm-fundraising.yaml#L63-L76


I don't think this pattern can easily be re-used.

*Von:* Ostrzyciel 
*Gesendet:* Montag, 12. September 2022 16:42
*An:* wikitech-l@lists.wikimedia.org 
*Betreff:* [Wikitech-l] Re: How to specify version of MW that Jenkins 
should run?

Hi all,

I've been looking into a similar thing for a different extension, 
where I wanted it to be compatible with MW 1.37+ (see task 
). Basically, I want the 
tests to run both on REL1_37 and master to make sure that whatever I 
merge is compatible with 1.37. The cherry-picking approach is not a 
valid solution, as it's tedious, error-prone, and I can never be sure 
that the patch works with both branches.


I know of one extension that seemingly has something like this: 
DonationInterface. For example, this master patch 
 
had its tests run against REL1_35.


Can something like this be replicated in other extensions? I think 
it's a vital feature for extension developers, because compatibility 
with only master is next to useless. To be honest, it boggles my mind 
that this hasn't become common practice long ago.


--
The Slightly Mind-Boggled Ostrzyciel

On 12/09/2022 16:14, Robert Vogel via Wikitech-l wrote:
The continous integration testing currently uses the same branch fot 
the "dependecies" as it tests. So if you create a change set on 
branch `master`, it will use `master` for all the other repos (like 
mw core, extensions, skins) it clones to build up the environment. I 
am not aware of any way to influence this on a per repo level.


The only way I am currently aware of is to commit against a branch 
called `REL1_35` of your extension and then cherry-pick the changes 
up to `master`, once the tests have passed. This will make the CI 
tests use the `REL1_35` branch of each dependency (including mw core).


--
Robert

*Von:* Sebastian Berlin  


*Gesendet:* Montag, 12. September 2022 12:20
*An:* Wikimedia developers  

*Betreff:* [Wikitech-l] How to specify version of MW that Jenkins 
should run?
I'm having some issues with Jenkins running the latest version of MW 
for an extension (Wikispeech 
). Jenkins run 
with the 1.40 alpha, but we want the extension to support the latest 
LTS (1.35 as of right now).


I tried changing in extension.json to:
"requires": {
 "MediaWiki": "1.35.*"
}
which caused an error for Jenkins:

12:44:09 A dependency error was encountered while installing the
extension "Wikispeech": Wikispeech is not compatible with the
current MediaWiki core (version 1.40.0-alpha), it requires: 1.35.*.

The whole log is here: https://phabricator.wikimedia.org/P34482 
.


How can I specify what version of MW to run with? I've looked at both 
the MW wiki and Wikitech, but hasn't found any instructions for this.


Best,

*Sebastian Berlin*
Utvecklare//Developer/
Wikimedia Sverige (WMSE)

E-post//E-Mail/: sebastian.ber...@wikimedia.se 


Telefon//Phone/: (+46) 0707 - 92 03 84

___
Wikitech-l mailing list --wikitech-l@lists.wikimedia.org  

To unsubscribe send an email towikitech-l-le...@lists.wikimedia.org  

https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/  





___
Wikitech-l mailing list --wikitech-l@lists.wikimedia.org
To unsubscribe send an email 

[Wikitech-l] Re: How to specify version of MW that Jenkins should run?

[Robert Vogel via Wikitech-l]

The "DonationInterface" has some "special handling" in the CI config: 
https://github.com/wikimedia/integration-config/blob/d2ef596ea8f697c0920270939c1bf967e0be/zuul/layout.yaml#L4464-L4480

They apparently use some dedicated Quibble test runner: 
https://github.com/wikimedia/integration-config/blob/d2ef596ea8f697c0920270939c1bf967e0be/jjb/wm-fundraising.yaml#L63-L76

I don't think this pattern can easily be re-used.

Von: Ostrzyciel 
Gesendet: Montag, 12. September 2022 16:42
An: wikitech-l@lists.wikimedia.org 
Betreff: [Wikitech-l] Re: How to specify version of MW that Jenkins should run?

Hi all,

I've been looking into a similar thing for a different extension, where I 
wanted it to be compatible with MW 1.37+ (see 
task). Basically, I want the tests 
to run both on REL1_37 and master to make sure that whatever I merge is 
compatible with 1.37. The cherry-picking approach is not a valid solution, as 
it's tedious, error-prone, and I can never be sure that the patch works with 
both branches.

I know of one extension that seemingly has something like this: 
DonationInterface. For example, this master 
patch
 had its tests run against REL1_35.

Can something like this be replicated in other extensions? I think it's a vital 
feature for extension developers, because compatibility with only master is 
next to useless. To be honest, it boggles my mind that this hasn't become 
common practice long ago.

--
The Slightly Mind-Boggled Ostrzyciel

On 12/09/2022 16:14, Robert Vogel via Wikitech-l wrote:
The continous integration testing currently uses the same branch fot the 
"dependecies" as it tests. So if you create a change set on branch `master`, it 
will use `master` for all the other repos (like mw core, extensions, skins) it 
clones to build up the environment. I am not aware of any way to influence this 
on a per repo level.

The only way I am currently aware of is to commit against a branch called 
`REL1_35` of your extension and then cherry-pick the changes up to `master`, 
once the tests have passed. This will make the CI tests use the `REL1_35` 
branch of each dependency (including mw core).

--
Robert

Von: Sebastian Berlin 

Gesendet: Montag, 12. September 2022 12:20
An: Wikimedia developers 

Betreff: [Wikitech-l] How to specify version of MW that Jenkins should run?

I'm having some issues with Jenkins running the latest version of MW for an 
extension (Wikispeech). 
Jenkins run with the 1.40 alpha, but we want the extension to support the 
latest LTS (1.35 as of right now).

I tried changing in extension.json to:
"requires": {
 "MediaWiki": "1.35.*"
}
which caused an error for Jenkins:
12:44:09 A dependency error was encountered while installing the extension 
"Wikispeech": Wikispeech is not compatible with the current MediaWiki core 
(version 1.40.0-alpha), it requires: 1.35.*.
The whole log is here: https://phabricator.wikimedia.org/P34482.

How can I specify what version of MW to run with? I've looked at both the MW 
wiki and Wikitech, but hasn't found any instructions for this.

Best,

Sebastian Berlin
Utvecklare/Developer
Wikimedia Sverige (WMSE)

E-post/E-Mail: 
sebastian.ber...@wikimedia.se
Telefon/Phone: (+46) 0707 - 92 03 84



___
Wikitech-l mailing list -- 
wikitech-l@lists.wikimedia.org
To unsubscribe send an email to 
wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: How to specify version of MW that Jenkins should run?

[Ostrzyciel]

Hi all,

I've been looking into a similar thing for a different extension, where 
I wanted it to be compatible with MW 1.37+ (see task 
). Basically, I want the 
tests to run both on REL1_37 and master to make sure that whatever I 
merge is compatible with 1.37. The cherry-picking approach is not a 
valid solution, as it's tedious, error-prone, and I can never be sure 
that the patch works with both branches.


I know of one extension that seemingly has something like this: 
DonationInterface. For example, this master patch 
 
had its tests run against REL1_35.


Can something like this be replicated in other extensions? I think it's 
a vital feature for extension developers, because compatibility with 
only master is next to useless. To be honest, it boggles my mind that 
this hasn't become common practice long ago.


--
The Slightly Mind-Boggled Ostrzyciel

On 12/09/2022 16:14, Robert Vogel via Wikitech-l wrote:
The continous integration testing currently uses the same branch fot 
the "dependecies" as it tests. So if you create a change set on branch 
`master`, it will use `master` for all the other repos (like mw core, 
extensions, skins) it clones to build up the environment. I am not 
aware of any way to influence this on a per repo level.


The only way I am currently aware of is to commit against a branch 
called `REL1_35` of your extension and then cherry-pick the changes up 
to `master`, once the tests have passed. This will make the CI tests 
use the `REL1_35` branch of each dependency (including mw core).


--
Robert

*Von:* Sebastian Berlin 
*Gesendet:* Montag, 12. September 2022 12:20
*An:* Wikimedia developers 
*Betreff:* [Wikitech-l] How to specify version of MW that Jenkins 
should run?
I'm having some issues with Jenkins running the latest version of MW 
for an extension (Wikispeech 
). Jenkins run 
with the 1.40 alpha, but we want the extension to support the latest 
LTS (1.35 as of right now).


I tried changing in extension.json to:
"requires": {
 "MediaWiki": "1.35.*"
}
which caused an error for Jenkins:

12:44:09 A dependency error was encountered while installing the
extension "Wikispeech": Wikispeech is not compatible with the
current MediaWiki core (version 1.40.0-alpha), it requires: 1.35.*.

The whole log is here: https://phabricator.wikimedia.org/P34482.

How can I specify what version of MW to run with? I've looked at both 
the MW wiki and Wikitech, but hasn't found any instructions for this.


Best,

*Sebastian Berlin*
Utvecklare//Developer/
Wikimedia Sverige (WMSE)

E-post//E-Mail/: sebastian.ber...@wikimedia.se
Telefon//Phone/: (+46) 0707 - 92 03 84

___
Wikitech-l mailing list --wikitech-l@lists.wikimedia.org
To unsubscribe send an email towikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/


___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: How to specify version of MW that Jenkins should run?

[Robert Vogel via Wikitech-l]

The continous integration testing currently uses the same branch fot the 
"dependecies" as it tests. So if you create a change set on branch `master`, it 
will use `master` for all the other repos (like mw core, extensions, skins) it 
clones to build up the environment. I am not aware of any way to influence this 
on a per repo level.

The only way I am currently aware of is to commit against a branch called 
`REL1_35` of your extension and then cherry-pick the changes up to `master`, 
once the tests have passed. This will make the CI tests use the `REL1_35` 
branch of each dependency (including mw core).

--
Robert

Von: Sebastian Berlin 
Gesendet: Montag, 12. September 2022 12:20
An: Wikimedia developers 
Betreff: [Wikitech-l] How to specify version of MW that Jenkins should run?

I'm having some issues with Jenkins running the latest version of MW for an 
extension (Wikispeech). 
Jenkins run with the 1.40 alpha, but we want the extension to support the 
latest LTS (1.35 as of right now).

I tried changing in extension.json to:
"requires": {
 "MediaWiki": "1.35.*"
}
which caused an error for Jenkins:
12:44:09 A dependency error was encountered while installing the extension 
"Wikispeech": Wikispeech is not compatible with the current MediaWiki core 
(version 1.40.0-alpha), it requires: 1.35.*.
The whole log is here: https://phabricator.wikimedia.org/P34482.

How can I specify what version of MW to run with? I've looked at both the MW 
wiki and Wikitech, but hasn't found any instructions for this.

Best,

Sebastian Berlin
Utvecklare/Developer
Wikimedia Sverige (WMSE)

E-post/E-Mail: 
sebastian.ber...@wikimedia.se
Telefon/Phone: (+46) 0707 - 92 03 84
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: How to specify version of MW that Jenkins should run?

[Sebastian Berlin]

Thanks for the replies.

The things is that we *want* Jenkins to run with MediaWiki 1.35, which is
why I changed the version in extension.json. There are changes in the
development version that Jenkins runs that aren't compatible with our code.

*Sebastian Berlin*
Utvecklare/*Developer*
Wikimedia Sverige (WMSE)

E-post/*E-Mail*: sebastian.ber...@wikimedia.se
Telefon/*Phone*: (+46) 0707 - 92 03 84


On Mon, 12 Sept 2022 at 13:04, Derick Alangi  wrote:

> Hi,
>
> Based on the error you posted, I could reproduce it locally and I think
> the issue is that I don't think the registration mechanism currently
> supports regular expression for *{ "requires": { "MediaWiki": "..." } },* so,
> it's advised to pick a (minimum) version of MediaWiki core you want your
> extension to work with and then use either of the logical operators: ">=",
> ">", "<=", "<" to cover a range of versions needed.
>
> The most syntax I've seen in almost all extension files is: *{
> "requires": { "MediaWiki": ">= 1.XX.X" } } *for example: *{ "requires": {
> "MediaWiki": ">= 1.35.0" } }.*
>
> Maybe Legoktm has a few more ideas on this and I would also say that
> pinning to a specific version won't work either like "= 1.35.0" or "1.35.0".
>
> Regards!
>
> *--*
>
> *Derick A.*
>
> *From One Developer to Another!*
>
>
> On Mon, Sep 12, 2022 at 11:53 AM Derick Alangi 
> wrote:
>
>> Hi Sebastian,
>>
>> Can you change:
>> "requires": {
>>  "MediaWiki": "1.35.*"
>> }
>>
>> to:
>> "requires": {
>>  "MediaWiki": ">=1.35.0"
>> }
>>
>> Let me know if that helps.
>>
>> *--*
>>
>> *Derick A.*
>>
>> *From One Developer to Another!*
>>
>>
>> On Mon, Sep 12, 2022 at 11:21 AM Sebastian Berlin <
>> sebastian.ber...@wikimedia.se> wrote:
>>
>>> I'm having some issues with Jenkins running the latest version of MW for
>>> an extension (Wikispeech
>>> ). Jenkins run
>>> with the 1.40 alpha, but we want the extension to support the latest LTS
>>> (1.35 as of right now).
>>>
>>> I tried changing in extension.json to:
>>> "requires": {
>>>  "MediaWiki": "1.35.*"
>>> }
>>> which caused an error for Jenkins:
>>>
 12:44:09 A dependency error was encountered while installing the
 extension "Wikispeech": Wikispeech is not compatible with the current
 MediaWiki core (version 1.40.0-alpha), it requires: 1.35.*.

>>> The whole log is here: https://phabricator.wikimedia.org/P34482.
>>>
>>> How can I specify what version of MW to run with? I've looked at both
>>> the MW wiki and Wikitech, but hasn't found any instructions for this.
>>>
>>> Best,
>>>
>>> *Sebastian Berlin*
>>> Utvecklare/*Developer*
>>> Wikimedia Sverige (WMSE)
>>>
>>> E-post/*E-Mail*: sebastian.ber...@wikimedia.se
>>> Telefon/*Phone*: (+46) 0707 - 92 03 84
>>> ___
>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>> To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
>>>
>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>
>> ___
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: How to specify version of MW that Jenkins should run?

[Derick Alangi]

Hi,

Based on the error you posted, I could reproduce it locally and I think the
issue is that I don't think the registration mechanism currently supports
regular expression for *{ "requires": { "MediaWiki": "..." } },* so, it's
advised to pick a (minimum) version of MediaWiki core you want your
extension to work with and then use either of the logical operators: ">=",
">", "<=", "<" to cover a range of versions needed.

The most syntax I've seen in almost all extension files is: *{ "requires":
{ "MediaWiki": ">= 1.XX.X" } } *for example: *{ "requires": { "MediaWiki":
">= 1.35.0" } }.*

Maybe Legoktm has a few more ideas on this and I would also say that
pinning to a specific version won't work either like "= 1.35.0" or "1.35.0".

Regards!

*--*

*Derick A.*

*From One Developer to Another!*


On Mon, Sep 12, 2022 at 11:53 AM Derick Alangi 
wrote:

> Hi Sebastian,
>
> Can you change:
> "requires": {
>  "MediaWiki": "1.35.*"
> }
>
> to:
> "requires": {
>  "MediaWiki": ">=1.35.0"
> }
>
> Let me know if that helps.
>
> *--*
>
> *Derick A.*
>
> *From One Developer to Another!*
>
>
> On Mon, Sep 12, 2022 at 11:21 AM Sebastian Berlin <
> sebastian.ber...@wikimedia.se> wrote:
>
>> I'm having some issues with Jenkins running the latest version of MW for
>> an extension (Wikispeech
>> ). Jenkins run with
>> the 1.40 alpha, but we want the extension to support the latest LTS (1.35
>> as of right now).
>>
>> I tried changing in extension.json to:
>> "requires": {
>>  "MediaWiki": "1.35.*"
>> }
>> which caused an error for Jenkins:
>>
>>> 12:44:09 A dependency error was encountered while installing the
>>> extension "Wikispeech": Wikispeech is not compatible with the current
>>> MediaWiki core (version 1.40.0-alpha), it requires: 1.35.*.
>>>
>> The whole log is here: https://phabricator.wikimedia.org/P34482.
>>
>> How can I specify what version of MW to run with? I've looked at both the
>> MW wiki and Wikitech, but hasn't found any instructions for this.
>>
>> Best,
>>
>> *Sebastian Berlin*
>> Utvecklare/*Developer*
>> Wikimedia Sverige (WMSE)
>>
>> E-post/*E-Mail*: sebastian.ber...@wikimedia.se
>> Telefon/*Phone*: (+46) 0707 - 92 03 84
>> ___
>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>> To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
>>
>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>
>
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: How to specify version of MW that Jenkins should run?

[Andre Klapper]

On Mon, 2022-09-12 at 12:20 +0200, Sebastian Berlin wrote:
> I tried changing in extension.json to:
>  "requires": {
>  "MediaWiki": "1.35.*"

Wouldn't this be?:
 "MediaWiki": ">= 1.35.0"

Cheers,
andre
-- 
Andre Klapper (he/him) | Bugwrangler / Developer Advocate
https://blogs.gnome.org/aklapper/
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: How to specify version of MW that Jenkins should run?

[Derick Alangi]

Hi Sebastian,

Can you change:
"requires": {
 "MediaWiki": "1.35.*"
}

to:
"requires": {
 "MediaWiki": ">=1.35.0"
}

Let me know if that helps.

*--*

*Derick A.*

*From One Developer to Another!*


On Mon, Sep 12, 2022 at 11:21 AM Sebastian Berlin <
sebastian.ber...@wikimedia.se> wrote:

> I'm having some issues with Jenkins running the latest version of MW for
> an extension (Wikispeech
> ). Jenkins run with
> the 1.40 alpha, but we want the extension to support the latest LTS (1.35
> as of right now).
>
> I tried changing in extension.json to:
> "requires": {
>  "MediaWiki": "1.35.*"
> }
> which caused an error for Jenkins:
>
>> 12:44:09 A dependency error was encountered while installing the
>> extension "Wikispeech": Wikispeech is not compatible with the current
>> MediaWiki core (version 1.40.0-alpha), it requires: 1.35.*.
>>
> The whole log is here: https://phabricator.wikimedia.org/P34482.
>
> How can I specify what version of MW to run with? I've looked at both the
> MW wiki and Wikitech, but hasn't found any instructions for this.
>
> Best,
>
> *Sebastian Berlin*
> Utvecklare/*Developer*
> Wikimedia Sverige (WMSE)
>
> E-post/*E-Mail*: sebastian.ber...@wikimedia.se
> Telefon/*Phone*: (+46) 0707 - 92 03 84
> ___
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/