ArthurTaylor added a comment.
A few more updates here. I looked at the memory usage for the above request
(in the failing case), and instrumented the run with repeated calls to
`memory_get_usage()` in the interactive debugger in PHPStorm. What I saw there
is that in `Module.php::getModuleContent` for the `test.DiscussionTools`
module, the memory usage goes up dramatically:
public function getModuleContent( Context $context ) {
$contextHash = $context->getHash(); // <--- MEMORY USAGE 16.76
MB
// Cache this expensive operation. This calls builds the
scripts, styles, and messages
// content which typically involves filesystem and/or database
access.
if ( !array_key_exists( $contextHash, $this->contents ) ) {
$this->contents[$contextHash] = $this->buildContent(
$context );
}
return $this->contents[$contextHash]; // <-- MEMORY USAGE 45.06
MB
}
so we know here that just the content of the files is about 30MB of data that
we need to reformat in memory before streaming back to the client. When we then
get to `XmlJsCode.php::encodeObject`, the situation is already worse and about
to get critical:
public static function encodeObject( $obj, $pretty = false ) {
// $obj in this case is an array with 147 keys
corresponding to different files / modules
$parts = [];
foreach ( $obj as $key => $value ) {
$parts[] =
( $pretty ? ' ' : '' ) .
Xml::encodeJsVar( $key, $pretty ) .
( $pretty ? ': ' : ':' ) .
Xml::encodeJsVar( $value, $pretty );
}
// By the time we get down here -> MEMORY USAGE 65.93 MB
return new self(
'{' .
( $pretty ? "\n" : '' ) .
implode( $pretty ? ",\n" : ',', $parts ) . // The
implode itself then doubles the space required for
( $pretty ? "\n" : '' ) . // this 20MB
string manipulation -> MEMORY USAGE 86.39
'}'
);
}
With `memory_limit` at 128MB, at least in the `php-fpm` environment, the
`implode` pushes us past the limit.
The proposed patch makes better use of memory inside `XmlJsCode.php` so that
we don't immediately explode at that point. Unfortunately, that doesn't
eliminate the issue - instead, we run out of memory in my local testing inside
`JavaScriptMinifier::minifyInternal` - see attached:
F42547488: load-with-patch.php.html
<https://phabricator.wikimedia.org/F42547488>
The logs from the CI run show an error at `ResourceLoader.php#L851`, which is
`echo $response;`
TASK DETAIL
https://phabricator.wikimedia.org/T356896
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: ArthurTaylor
Cc: karapayneWMDE, Lucas_Werkmeister_WMDE, Aklapper, Michael,
Danny_Benjafield_WMDE, Isabelladantes1983, Themindcoder, Adamm71, Jersione,
Hellket777, LisafBia6531, Astuthiodit_1, 786, Biggs657, Invadibot, maantietaja,
Juan90264, Alter-paule, Beast1978, ItamarWMDE, Un1tY, Akuckartz, Hook696,
Kent7301, joker88john, CucyNoiD, Nandana, Gaboe420, Giuliamocci, Cpaulf30,
Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, QZanden, cmadeo, KimKelting,
LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS,
Wikidata-bugs, aude, jayvdb, Ricordisamoa, Mbch331
_______________________________________________
Wikidata-bugs mailing list -- [email protected]
To unsubscribe send an email to [email protected]
