hashar added a comment.
Looking at some random builds, it usually takes 15 / 20 seconds to start the
next command which is a bug by itself:
00:17:39.940 INFO:backend.MySQL:Terminating MySQL
00:17:55.311 [wmf-quibble-selenium-php72-docker] $ /bin/bash -xe
/tmp/jenkins4972097457850427826.sh
00:19:01.909 INFO:backend.MySQL:Terminating MySQL
00:19:20.063 [wmf-quibble-selenium-php72-docker] $ /bin/bash -xe
/tmp/jenkins443040905686928605.sh
00:18:17.942 INFO:backend.MySQL:Terminating MySQL
00:18:34.254 [wmf-quibble-selenium-php72-docker] $ /bin/bash -xe
/tmp/jenkins7269376008034149723.sh
I have looked at that job duration trend on
https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php72-docker/buildTimeTrend
and the job is constantly way too slow whenever it happens to run on
`integration-agent-docker-1020`:
00:30:08.778 INFO:backend.MySQL:Terminating MySQL
00:31:21.208 [wmf-quibble-selenium-php72-docker] $ /bin/bash -xe
/tmp/jenkins8569331040755739778.sh
00:38:54.981 INFO:backend.MySQL:Terminating MySQL
00:42:18.062 [wmf-quibble-selenium-php72-docker@2] $ /bin/bash -xe
/tmp/jenkins6862799909867122709.sh
00:39:41.829 INFO:backend.MySQL:Terminating MySQL
00:44:44.184 [wmf-quibble-selenium-php72-docker] $ /bin/bash -xe
/tmp/jenkins6896978991300300753.sh
00:31:37.581 INFO:backend.MySQL:Terminating MySQL
00:32:57.181 [wmf-quibble-selenium-php72-docker@2] $ /bin/bash -xe
/tmp/jenkins6303269567405582114.sh
00:30:08.106 INFO:backend.MySQL:Terminating MySQL
00:31:22.239 [wmf-quibble-selenium-php72-docker] $ /bin/bash -xe
/tmp/jenkins6647671750866068876.sh
That agent runs on a WMCS host that uses Ceph for file backend (T260916
<https://phabricator.wikimedia.org/T260916>). Which itself does not explain the
problem since other agents are affected though in seconds, not minutes! I will
unpoll it.
TASK DETAIL
https://phabricator.wikimedia.org/T265615
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: hashar
Cc: hashar, awight, kostajh, Lucas_Werkmeister_WMDE, Akuckartz, darthmon_wmde,
Nandana, NebulousIris, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer,
_jensen, rosalieper, Liudvikas, Scott_WUaS, thcipriani, Wikidata-bugs, aude,
Jdforrester-WMF, Mbch331, Jay8g
_______________________________________________
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
