https://bugzilla.wikimedia.org/show_bug.cgi?id=71128
Bug ID: 71128
Summary: Jenkins: lanthanum tmpfs is filling up with stale tmp
files
Product: Wikimedia
Version: wmf-deployment
Hardware: All
OS: All
Status: NEW
Severity: critical
Priority: Unprioritized
Component: Continuous integration
Assignee: [email protected]
Reporter: [email protected]
CC: [email protected], [email protected],
[email protected]
Web browser: ---
Mobile Platform: ---
Every few days it is going critical. Let's two criticals were
* September 17 19:00
* September 22 11:20
$ df -h
..
tmpfs 512M 505M 7.8M 99% /var/lib/jenkins-slave/tmpfs
..
Example contents:
[18:25 UTC] krinkle at lanthanum.eqiad.wmnet in /var/lib/jenkins-slave/tmpfs
$ l
mediawiki-core-extensions-integration/
mediawiki-core-install-sqlite/
mediawiki-core-phpunit-api/
mediawiki-core-phpunit-databaseless/
mediawiki-core-phpunit-misc/
mediawiki-core-regression-REL1_23/
mwext-Flow-qunit/
mwext-WikimediaEvents-testextension/
parsoidsvc-php-parsertests/
mediawiki-core-regression-master:
build7546.sqlite
build7547.sqlite
build7550.sqlite
mediawiki-vendor-integration:
total 21M
MW_PHPUnit_ExifRotationTest_pCtdaJ/
MW_PHPUnit_TextPassDumperTest_Cpe5LO/
MW_PHPUnit_TextPassDumperTest_J1dvmI
MW_PHPUnit_TextPassDumperTest_QmhN3D
MW_PHPUnit_TextPassDumperTest_rYX7QZ/
277K Sep 22 17:26 build1787.sqlite
291K Sep 22 17:35 build1788.sqlite
291K Sep 22 17:42 build1793.sqlite
291K Sep 22 17:54 build1798.sqlite
291K Sep 22 18:02 build1809.sqlite
265K Sep 22 18:03 build1812.sqlite
271K Sep 22 18:05 build1813.sqlite
291K Sep 22 18:12 build1814.sqlite
291K Sep 22 18:25 build1819.sqlite
4.1M Aug 20 15:08 mw-A8c7xJ
4.1M Aug 20 15:12 mw-BxZ3Ez
4.1M Aug 26 13:42 mw-R4MZCN
4.1M Aug 26 13:42 mw-tbdCvd
0 Sep 20 18:21 transform_f2c5b84944be-1.jpg
I've purged a bunch of files for now, possibly broke a few currently running
builds.
Problems:
* The tmpfs partition is way too small (~ 500MB).
* Stuff isn't being purged.
* These are not regular build artefacts (which Jenkins stores separately and we
do have them expire/purge properly).
* These are files only needed for the duration of the test and should be
removed right after a test has run.
--
You are receiving this mail because:
You are the assignee for the bug.
You are on the CC list for the bug.
_______________________________________________
Wikibugs-l mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
