[Pywikipedia-bugs] [Maniphest] [Unassigned] T132138: Perform full test suite using Wikimedia CI

[Xqt]

Xqt removed Magul as the assignee of this task.
Xqt added a subscriber: Magul.

TASK DETAIL
  https://phabricator.wikimedia.org/T132138

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Xqt
Cc: Magul, brennen, LarsWirzenius, Framawiki, Ricordisamoa, jayvdb, Aklapper, 
JanZerebecki, pywikibot-bugs-list, Redabr4, Zanziii, Sadisticturd, 
NebulousIris, Wenyi, A.S.Kochergin, Tbscho, MayS, Mdupont, JJMC89, Avicennasis, 
Liudvikas, mys_721tx, Dalba, Masti, Alchimista, Rxy, hashar, greg
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs

[Pywikipedia-bugs] [Maniphest] [Changed Project Column] T132138: Perform full test suite using Wikimedia CI

[greg]

greg moved this task from To Triage to Backlog on the Release-Engineering-Team 
board.
greg edited projects, added Release-Engineering-Team (Backlog); removed 
Release-Engineering-Team.

TASK DETAIL
  https://phabricator.wikimedia.org/T132138

WORKBOARD
  https://phabricator.wikimedia.org/project/board/20/

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Magul, greg
Cc: brennen, LarsWirzenius, Framawiki, Ricordisamoa, jayvdb, Aklapper, 
JanZerebecki, pywikibot-bugs-list, Redabr4, Zanziii, Sadisticturd, 
NebulousIris, Wenyi, A.S.Kochergin, Tbscho, MayS, Mdupont, JJMC89, Avicennasis, 
Liudvikas, mys_721tx, Dalba, Masti, Alchimista, Rxy, hashar, greg
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs

[Pywikipedia-bugs] [Maniphest] [Changed Project Column] T100903: Run pywikibot test suite regularly on beta cluster as part of MediaWiki/Wikimedia CI

[greg]

greg moved this task from To Triage to Backlog on the Release-Engineering-Team 
board.
greg edited projects, added Release-Engineering-Team (Backlog); removed 
Release-Engineering-Team.

TASK DETAIL
  https://phabricator.wikimedia.org/T100903

WORKBOARD
  https://phabricator.wikimedia.org/project/board/20/

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: greg
Cc: Dvorapa, Lokal_Profil, Luke081515, Legoktm, gerritbot, Ricordisamoa, 
hashar, mmodell, greg, pywikibot-bugs-list, jayvdb, Aklapper, Redabr4, Zanziii, 
Sadisticturd, NebulousIris, Wenyi, A.S.Kochergin, Bsandipan, Tbscho, MayS, 
Mdupont, JJMC89, Avicennasis, Liudvikas, thcipriani, mys_721tx, Dalba, Masti, 
Alchimista, Rxy, Jay8g, Krenair
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs

[Pywikipedia-bugs] [Maniphest] [Updated] T186208: Magul's quick tests doesn't run anymore

[hashar]

hashar added a comment.


  @Dvorapa thank you so much for your replies. I guess I have all the context 
to jump in and analysis a proper solution :-]
  
  > The easy solution for the top priority Magul's test replacement would be 
something like this: Someone enters a magic keyword in the Gerrit patchset 
comment -> Wikimedia CI registers it and creates a pull request to GitHub with 
the code (perhaps using GH API, also closes it immediately as it wil not be 
needed on GitHub) -> and GitHub-Travis-Appveyor handles the rest of the work.
  
  Our situation with GitHub is already rather sort of a mess. Others previously 
suggested to have a Gerrit -> GitHub -> Travis -> 3rd party tools, I vetoed it 
as not robust enough.  There are too many stacks and it is asking for troubles. 
  That being said, people do take advantage of repositories being synced to 
Github to activate jobs that run once a change is merged. That is fine since it 
does not touch Gerrit / Wikimedia CI and thus #release-engineering-team 
 is not 
involved :-]
  
  > The harder solution would be to mimick Travis/Appveyor in Wikimedia CI as 
you said, then when someone enters a magic keyword in the Gerrit patchset 
comment or someone +2 the patchset -> Wikimedia CI registers it and runs the 
tox on predefined set of environments including several Python versions, three 
major OS platforms and several presets.
  
  The Wikimedia CI uses a Docker container that runs tox. It recently got 
support for python 2.7, 3.4, 3.5, 3.6 and 3.7 T191764 
. We should be able to add them 
right now in `tox.ini` and add CI jobs for them.
  
  ---
  
  32bits, yes I agree we can drop it. I don't think pywikibot would have many 
issues with it anyway.
  
  That leaves us with the .travis.yml 
. 
Seems all of that can be converted to testenv in tox.ini, possibly by invoking 
different testing scripts.  It does not seem too complicated :]

TASK DETAIL
  https://phabricator.wikimedia.org/T186208

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: hashar
Cc: hashar, Liuxinyu970226, Legoktm, Framawiki, Dalba, Dvorapa, Magul, 
Aklapper, Xqt, pywikibot-bugs-list, Redabr4, Zanziii, Sadisticturd, 
NebulousIris, Wenyi, A.S.Kochergin, Tbscho, MayS, Mdupont, JJMC89, Avicennasis, 
Liudvikas, mys_721tx, jayvdb, dduvall, Masti, Alchimista, Rxy, greg
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs

[Pywikipedia-bugs] [Maniphest] [Commented On] T186208: Magul's quick tests doesn't run anymore

[Dvorapa]

Dvorapa added a comment.


  Currently running:
  
  | When| What  
  | Where| 
status  |
  | on every patchset and on +2 | nose, nose32, style and doctest   
  | Jenkins CI   | 
ok  |
  | postmerge   | doctest, generates docs   
  | Jenkins CI   | 
ok  |
  | postmerge   | nose, style, doc, unittest on various wikis, 
py versions, environments, platforms and architectures | Travis, Appveyor | 
ok with some smaller issues |
  | postmerge   | coverage and redundancy   
  | Codecov, Codeclimate | 
ok with some smaller issues |
  |
  
  Wishes and broken:
  
  | When   | What   
| Where 
  | status|
  | on demand for one patchset | nose and unittest on various wikis, py 
versions, environments, platforms and architectures | Magul's tool activating 
Travis CI   | down, high priority   |
  | postmerge  | nose and unittest on various wikis, py 
versions, environments, platforms and architectures | Wikimedia CI instead of 
Travis/Appveyor | wish, lowest priority |
  |
  
  Bypassed partially:
  
  | When   | What   
 | Where| status|
  | before new MediaWiki version comes out | nose and unittest of current 
master on beta-cluster | anywhere | wish, medium priority |
  | postmerge  | nose and unittest on beta-cluster  
 | Travis, Appveyor | ok|

TASK DETAIL
  https://phabricator.wikimedia.org/T186208

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: hashar, Dvorapa
Cc: hashar, Liuxinyu970226, Legoktm, Framawiki, Dalba, Dvorapa, Magul, 
Aklapper, Xqt, pywikibot-bugs-list, Redabr4, Zanziii, Sadisticturd, 
NebulousIris, Wenyi, A.S.Kochergin, Tbscho, MayS, Mdupont, JJMC89, Avicennasis, 
Liudvikas, mys_721tx, jayvdb, dduvall, Masti, Alchimista, Rxy, greg
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs

[Pywikipedia-bugs] [Maniphest] [Commented On] T186208: Magul's quick tests doesn't run anymore

[Dvorapa]

Dvorapa added a comment.


  In T186208#4998959 , 
@hashar wrote:
  
  > Hello, this got raised to #release-engineering-team 
 meeting last 
Monday, thought I was busy with other random duties.
  >  This week will be easier so I gotta review the request but in short 
Wikimedia CI does not support Travis or AppVeyor at all.
  
  
  I understand. Some other devs already told me you were all busy last week and 
you are busy in general.
  
  > For Travis, it is heavily tight to github and we cant reproduce that with 
Gerrit and our CI (Jenkins/Zuul/Docker).  That being said, most of the time it 
is very straightforward to rewrite the travis logic to a Jenkins job.
  
  The basic thing why we are running Travis and Appveyor is to run unittests 
and nosetests and some others on multiple devices, like 32bit, 64bit, Win, 
Linux, Mac, Python 2.7, 3.4, 3.5, 3.6, 3.7 and 3.8. I mean various combinations 
of those and various presets (pwb env variables on/off, several Wikimedia 
wikis). The logic of both services are really close to one another, so it 
should be ok to merge the main points in Wikimedia CI. Also in my opinion there 
is no need to test py2.7, 3.4 and also perhaps 32bit anymore (as these are 
slowly ending their career).
  
  > AppVeyor I don't think I have ever heard of it. But I guess it is a third 
party closed source software and we would probably not be able to reproduce it.
  
  The config and the logs could be examined to get the idea of what these 
services are doing:
  https://phabricator.wikimedia.org/diffusion/PWBC/browse/master/.travis.yml
  https://travis-ci.org/wikimedia/pywikibot/builds
  
  https://phabricator.wikimedia.org/diffusion/PWBC/browse/master/.appveyor.yml
  https://ci.appveyor.com/project/ladsgroup/pywikibot-g4xqx
  
  > Note: we used to have a Jenkins job that ran the pywikibot tests against 
the #beta-cluster-infrastructure 
 
(901f445d1058f55d8ceae23cff4cf7b58537a778 

 (T100903 ).
  >  ) but it was broken/never completed and I have removed it T188256 
. The job was doing:
  > 
  >   export PYWIKIBOT2_DIR=$WORKSPACE
  >   export TOX_TESTENV_PASSENV=PY_COLORS
  >   export PY_COLORS=1
  >   tox -e venv -- pip install --upgrade ndg-httpsclient
  >   tox -e venv -- python -m generate_family_file 
'http://en.wikipedia.beta.wmflabs.org/' 'wpbeta' 'y'
  >   tox -e venv -- python -m generate_user_files -dir:$WORKSPACE 
-family:wpbeta -lang:en -v
  >   echo "console_encoding='utf8'" >> $WORKSPACE/user-config.py
  >   tox -e venv -- pip install -rdev-requirements.txt
  >   tox -e venv -- python setup.py nosetests --tests tests --verbosity=2 -a 
'"family=wpbeta,code=en"'
  >
  > 
  > We can look at moving that as a script directly into `pywikibot/core.git`, 
add a new testenv in tox (eg: `integration-beta`) and then simply have a 
Jenkins job that invokes that testenv. Then move forward and fix it :)
  
  I see, the point was to test the every new MediaWiki version coming up. We 
run some beta-cluster tests on Travis and Appveyor currently, so it should be 
easy to mimick it. It should be ok to add a script or a plaintext list of 
commands into the repo.
  
  The main issue (as you can see by the priority) is this one, the formerly 
known Magul's quick Travis test. Every Gerrit patchset is tested by unittest, 
nosetest, doctest, some other similar stuff, but all on one platform with one 
exact presets. The Travis/Appveyor tests are quite heavy and time-consuming 
(They are running in several environments to compare), so there is no need to 
run them on every patchset. But if we added Magul back then into reviewers, 
he/his tool run the unittests on Travis on demand. This is what we currently 
miss the most I think as Travis and Appveyor after merge are still working for 
us (although we have no access to the Wikimedia Appveyor account and perhaps 
neither Wikimedia has), so there is no need to hurry with moving completely to 
Wikimedia CI. Also we run beta-cluster tests there, so also not a priority I 
think.
  
  The easy solution for the top priority Magul's test replacement would be 
something like this: Someone enters a magic keyword in the Gerrit patchset 
comment -> Wikimedia CI registers it and creates a pull request to GitHub with 
the code (perhaps using GH API, also closes it immediately as it wil not be 
needed on GitHub) -> and GitHub-Travis-Appveyor handles the rest of the work.
  
  The harder solution would be to mimick Travis/Appveyor in Wikimedia CI as you 
said, then when someone enters a magic keyword in the Gerrit patchset comment 
or someone +2 the patchset -> Wikimedia CI registers it and runs the tox on 

[Pywikipedia-bugs] [Maniphest] [Claimed] T186208: Magul's quick tests doesn't run anymore

[hashar]

hashar claimed this task.
hashar added a comment.
Restricted Application edited projects, added Release-Engineering-Team 
(Kanban); removed Release-Engineering-Team.


  Hello, this got raised to #release-engineering-team 
 meeting last 
Monday, thought I was busy with other random duties.
  This week will be easier so I gotta review the request but in short Wikimedia 
CI does not support Travis or AppVeyor at all.
  
  For Travis, it is heavily tight to github and we cant reproduce that with 
Gerrit and our CI (Jenkins/Zuul/Docker).  That being said, most of the time it 
is very straightforward to rewrite the travis logic to a Jenkins job.
  
  AppVeyor I don't think I have ever heard of it. But I guess it is a third 
party closed source software and we would probably not be able to reproduce it.
  
  Note: we used to have a Jenkins job that ran the pywikibot tests against the 
#beta-cluster-infrastructure 
 
(901f445d1058f55d8ceae23cff4cf7b58537a778 

 (T100903 ).
  ) but it was broken/never completed and I have removed it T188256 
. The job was doing:
  
export PYWIKIBOT2_DIR=$WORKSPACE
export TOX_TESTENV_PASSENV=PY_COLORS
export PY_COLORS=1
tox -e venv -- pip install --upgrade ndg-httpsclient
tox -e venv -- python -m generate_family_file 
'http://en.wikipedia.beta.wmflabs.org/' 'wpbeta' 'y'
tox -e venv -- python -m generate_user_files -dir:$WORKSPACE -family:wpbeta 
-lang:en -v
echo "console_encoding='utf8'" >> $WORKSPACE/user-config.py
tox -e venv -- pip install -rdev-requirements.txt
tox -e venv -- python setup.py nosetests --tests tests --verbosity=2 -a 
'"family=wpbeta,code=en"'
  
  We can look at moving that as a script directly into `pywikibot/core.git`, 
add a new testenv in tox (eg: `integration-beta`) and then simply have a 
Jenkins job that invokes that testenv. Then move forward and fix it :)

TASK DETAIL
  https://phabricator.wikimedia.org/T186208

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: hashar
Cc: hashar, Liuxinyu970226, Legoktm, Framawiki, Dalba, Dvorapa, Magul, 
Aklapper, Xqt, pywikibot-bugs-list, Redabr4, Zanziii, Sadisticturd, 
NebulousIris, Wenyi, A.S.Kochergin, Tbscho, MayS, Mdupont, JJMC89, Avicennasis, 
Liudvikas, mys_721tx, jayvdb, dduvall, Masti, Alchimista, Rxy, greg
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs

[Pywikipedia-bugs] [Maniphest] [Edited] T186208: Magul's quick tests doesn't run anymore

[Dvorapa]

Dvorapa updated the task description.

TASK DETAIL
  https://phabricator.wikimedia.org/T186208

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Dvorapa
Cc: Liuxinyu970226, Legoktm, Framawiki, Dalba, Dvorapa, Magul, Aklapper, Xqt, 
pywikibot-bugs-list, Redabr4, Zanziii, Sadisticturd, NebulousIris, Wenyi, 
A.S.Kochergin, Tbscho, MayS, Mdupont, JJMC89, Avicennasis, Liudvikas, 
mys_721tx, jayvdb, Masti, Alchimista, Rxy, hashar, greg
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs