Verdy_p added a comment.
I can not as well a constant growth of computing time in Lua, especially in
recursiveClone <mwInit.lua:41>
which is now the topmost time-spender in pages using many Lua calls (even
without using Wikidata at all).
As well there's a constant need to get the Mediawiki expansion of
"{{int:Lang}}", which is also very costly from Lua, even if the parser expasion
is cached (it should not change across multiple "#invoke" calls from the same
parsed page): this expansions is still needed because Scribunto, unfortunately,
still does NOT expose the "userLanguage", in the "mw.*" properties, even if
that property is exposed in the PHP API (that Scribunto natively has access
to!) And there's no other way to get this (Scribunto only exposes the site's
default language). When will there be a "mw.userLanguage" property in Scribunto
? (possibly other variables exposed in the PHP API), put together into a
read-only table). This would allow us to drop costly calls to the
"frame:expand()" from Lua back to Mediawiki (which is slow simply because it
also performs an internal costly "recursiveClone()" call.
Note that the "recursiveClone()" is called multiple times instead of keeping
the SAME clone for reuse in further calls again from MediaWiki (then to Lua
back then back to the MediaWiki parser) in a cache, preserved as long as the
current page is being fully parsed: that Mediawiki parser clone is just used to
preprocess fragments of pages (e.g. a single template transclusion, or parser
function call, or magic keyword expansion). And because these clones are not
persistant, they don't seen to have their expansion cached (so if you call from
Lua to Mediawiki for such expansion of the same string, the result of that
expansion is visibly never kept, or jsut kept in the cache of the "recursive
clone", which is never reused later.
I don't know what "recursiveClone()" does, but it is by evidence constantly
doing more and more over time (some Wikimedia servers will cope with this, but
their performance varies a lot (for example the "mw1411" parser is MUCH faster
than "mw1269": we cannot predict which server will be selected randomly, but
there's evidence that they are not balanced accordingly even if they haver the
same limit of 10 seconds to parse any page: on "mw1269" now we have systematic
failures, but on "mw1411" this works. But other time, less and less servers are
capable to supprot the same pages, as they need more time to do their job and
there's very little we can do to optimize the page and save processing time, or
memory).
So I am convinced that Scribunto now a severe stability problem and that the
more you update it, to cover corner cases, the more it needs time and resources
to do exactly the same intended things (and I note as well that Scribunto
constantly need more memory). Scribunto requires a stronger analysis of its
performances and to know what it wastes (may be the bug is not in Scribunto
itself but in the PHP engine?).
TASK DETAIL
https://phabricator.wikimedia.org/T189409
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: thiemowmde, Verdy_p
Cc: Verdy_p, hoo, Lucas_Werkmeister_WMDE, gerritbot, Agabi10, Marostegui,
Reedy, thiemowmde, Lydia_Pintscher, MaxSem, RolandUnger, Invadibot,
maantietaja, Akuckartz, Nandana, lucamauri, Lahi, Gq86, GoranSMilovanovic,
Jayprakash12345, QZanden, LawExplorer, Vali.matei, SundanceRaphael, _jensen,
rosalieper, Scott_WUaS, Volker_E, Wong128hk, alex-mashin, Wikidata-bugs, aude,
Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Jay8g
_______________________________________________
Wikidata-bugs mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
