[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

Stashbot added a comment. Mentioned in SAL (#wikimedia-releng) [2020-04-17T21:11:53Z] Zuul: Add Scribunto to the gate after only five years of talking about it T125050 TASK DETAIL https://phabricator.wikimedia.org/T125050 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

gerritbot added a comment. Change 497574 **merged** by jenkins-bot: [integration/config@master] gate: Add Scribunto https://gerrit.wikimedia.org/r/497574 TASK DETAIL https://phabricator.wikimedia.org/T125050 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

gerritbot added a comment. Change 587600 **merged** by jenkins-bot: [mediawiki/extensions/Scribunto@master] tests: Mark non-LuaSandbox tests as @group Standalone https://gerrit.wikimedia.org/r/587600 TASK DETAIL https://phabricator.wikimedia.org/T125050 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

gerritbot added a comment. Change 587600 had a related patch set uploaded (by Jforrester; owner: Jforrester): [mediawiki/extensions/Scribunto@master] tests: Mark LuaStandalone tests as @group Standalone https://gerrit.wikimedia.org/r/587600 TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

Addshore added a comment. @hashar do you think we can try re adding this extension to the gate jobs and see how the performance is? TASK DETAIL https://phabricator.wikimedia.org/T125050 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

hashar added a comment. In T125050#5353427 , @Ladsgroup wrote: > This somehow happened in the last weekend for wikibase: https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-hhvm-docker/60806/consoleFull (on Wikibase patch:

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

Ladsgroup added a comment. This somehow happened in the last weekend for wikibase: https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-hhvm-docker/60806/consoleFull (on Wikibase patch: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/524688) Does anyone know why?

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

WMDE-Fisch added a comment. I created a new task for the creation and implementation of that group, I think it might be better to move the discussion and task around that there. TASK DETAIL https://phabricator.wikimedia.org/T125050 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

Jdforrester-WMF added a comment. If we want a integration test group whose objective is "integration-tests-we-want-to-run-only-in-this-extension", then the name should reflect the intent right from the start to avoid people making assumptions we then have to support. How about

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

Anomie added a comment. In T125050#5233908 , @hashar wrote: > I guess we can come up with a PHPUnit group name, in mediawiki/core add a new testsuite that would exclude the group and then switch the wmf-quibble jobs to use that new

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

hashar added a comment. Similar is FileImporter which has some slow tests ( https://gerrit.wikimedia.org/r/#/c/514065/ ), and IIRC CirrusSearch has a few slow tests we would probably want to skip as well. I guess we can come up with a PHPUnit group name, in mediawiki/core add a new

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

Anomie added a comment. In T125050#5120646 , @hashar wrote: > Scribunto has a set of test to make sure it works fine with some LUA interpreters. That is really nice but takes a few minutes iirc. When one send a patch for another

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

Addshore added a comment. Sounds okay to me! TASK DETAIL https://phabricator.wikimedia.org/T125050 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: Pablo-WMDE, Jakob_WMDE, Tarrow, gerritbot, Addshore, TerraCodes, Liuxinyu970226,

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

hashar added a comment. Since this is causing outages, I guess it is really time to tackle the issue. For the context: We have wmf-quibble-* jobs which clone several extensions and run all their tests. That is also known as the extension-gate. Scribunto has a set of test to

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

greg added a comment. In T125050#5074872 , @WMDE-leszek wrote: > So this task has been Unbreak now for almost two weeks. What could WMDE help with to get this unblocked/resolved? Hi, sorry for the delay. 1. Traditionally

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

WMDE-leszek added a comment. So this task has been Unbreak now for almost two weeks. What could WMDE help with to get this unblocked/resolved? TASK DETAIL https://phabricator.wikimedia.org/T125050 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To:

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

gerritbot added a comment. Change 497574 had a related patch set uploaded (by Jforrester; owner: Jforrester): [integration/config@master] gate: Add Scribunto https://gerrit.wikimedia.org/r/497574 TASK DETAIL https://phabricator.wikimedia.org/T125050 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

hashar added a comment. Seems the Wikibase related jobs that use Scribunto are passing just fine. That does not close that task though which is about adding Scribunto to extension-gate.TASK DETAILhttps://phabricator.wikimedia.org/T125050EMAIL

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

hashar added a comment. The Trusty instances that run the php5 jobs now allow Linux to over commit the memory. So a fork() should work now. I triggered a job and it managed to pass just fine. Guess you can check php5 or CR+2 again patches that were falling.TASK

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

gerritbot added a comment. Change 346634 merged by jenkins-bot: [integration/config@master] dib: allow Linux memory overcommit on Trusty https://gerrit.wikimedia.org/r/346634TASK DETAILhttps://phabricator.wikimedia.org/T125050EMAIL

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

gerritbot added a comment. Change 346634 had a related patch set uploaded (by Hashar): [integration/config@master] dib: allow Linux memory overcommit on Trusty https://gerrit.wikimedia.org/r/346634TASK DETAILhttps://phabricator.wikimedia.org/T125050EMAIL

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

hashar added a comment. Note to self, diskimage-builder wipe out /etc/fstab: elements/debootstrap/install.d/15-cleanup-debootstrapcat << EOF | tee /etc/fstab > /dev/null proc /proc proc nodev,noexec,nosuid 0 0 LABEL=${DIB_ROOT_LABEL} / ${FS_TYPE} errors=remount-ro 0 1 EOF Eg there is no swap

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

hashar added a comment. --log-junit is what fails the build so no. What fork() is doing is that it tries to allocate the whole virtual memory from the caller. In a trace above the PHP process had 1.8G and the system had 562M free + 959 cached. .That is not enough reclaimable memory to fit the

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

hoo added a comment. @hashar If we can't allocate more resources, can we disable --log-junit for just this job for now?TASK DETAILhttps://phabricator.wikimedia.org/T125050EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: Krinkle, Stashbot,

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

hashar added a comment. Ran the whole suite again with the ulimit in place but without --log-junit which is memory heavy. The whole suite pass just fine and there are no fork() issues. debug log reports 912.2M and PHPUnit reports 924.50MB So my best guess is that: our code invokes

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

hashar added a comment. Gave it a try directly on the instance running solely the Scribunto tests after the memory limit got raised to 70M which ends up being ulimit -v 68359. WORKSPACE=/home/jenkins/workspace/mwext-testextension-php55-composer-trusty \ php tests/phpunit/phpunit.php

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

hoo added a comment. In T125050#3152305, @hashar wrote: What this show is that at the point I took the capture, there were several Scribunto/engines/LuaStandalone/binaries/lua5_1_5_linux_64_generic/lua process running. I guess they are on hold somehow. Though the lua_ulimit.sh does not appear.

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

Krinkle added a comment. https://gerrit.wikimedia.org/r/#/c/345628/1 https://integration.wikimedia.org/ci/job/mwext-testextension-php55-composer-trusty/259/console 153) LuaStandalone: LuaWikibaseLibraryTests[38]: ... proc_open(): fork failed - Cannot allocate memory Is failing 5x in a row in

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

hashar added a comment. Next thing. On Jenkins we have a monitoring system named JavaMelody which can be used to inspect an instance, specially the process list. Eg via a link like https://integration.wikimedia.org/ci/monitoring/nodes/ci-trusty-wikimedia-597386?part=processes which is really the

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

hashar added a comment. I manually rebuild mwext-testextension-php55-composer-trusty for Wikibase. Instructed nodepool to not delete the instance (ssh labnodepool1001.eqiad.wmnet nodepool hold ) and connected to it while the test was running. The vmstat at the end of the build. Memory metrics

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

Stashbot added a comment. Mentioned in SAL (#wikimedia-releng) [2017-04-03T20:39:03Z] Nodepool: holding instance ci-trusty-wikimedia-597386 in an attempt debug Wikibase/Scribunto memory usage exploding T125050TASK DETAILhttps://phabricator.wikimedia.org/T125050EMAIL

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

Anomie added a comment. In T125050#3150798, @Paladox wrote: We should try and make scribuntu more performant i.e. performance wise. Or at least skip the tests that cause the tests to take so long. As I said earlier, it seems this Scribunto failure is a symptom of a problem, not the cause. The

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

hoo added a comment. In T125050#3150798, @Paladox wrote: We should try and make scribuntu more performant i.e. performance wise. Or at least skip the tests that cause the tests to take so long. Not the performance is the problem but the fact that the tests are failing is.TASK

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

Paladox added a comment. We should try and make scribuntu more performant i.e. performance wise. Or at least skip the tests that cause the tests to take so long.TASK DETAILhttps://phabricator.wikimedia.org/T125050EMAIL

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

Anomie added a comment. In T125050#3140483, @hashar wrote: For Wikibase tests, the job mwext-testextension-php55-composer-trusty fails. The end of debug log shows a spam of: 1016.5270 1799.0M Scribunto_LuaStandaloneInterpreter::terminate: terminating I note that the CLI log for #149 (which

[Wikidata-bugs] [Maniphest] [Commented On] T125050: [Task] Add Scribunto to extension-gate in CI

aude added a comment. now tests are failing: 03:32:45 53) LuaStandalone: LuaWikibaseEntityLibraryTests[39]: mw.wikibase.entity.getDescription integration 03:32:45 proc_open(): fork failed - Cannot allocate memory 03:32:45 03:32:45