Verdy_p added a comment.
And please add getEntityByLang(lang), which will implement language
fallbacks, so that we can load (and cache) entities only using relevant
languages; and remove all sitelinks from this call: sitelinks can be loaded
selectively on specific entities.
The cache of entities with selective lang could store more entities without
more cost in memory (for now it's limited to 15 entities, not enough for
infoboxes and many Lua modules that perform repeated calls to the wikidata
server for the same entity)
And as I said in Commons, the current JSON model is way to verbose to be
exposed "as is" in Lua:
- compress snaks and datavalues into a single table:
- remove "snaktype" and "datatype" internals, just keep "type"
- remove "datavalue", just use "value"
- remove superfluous language codes and sitecodes in individual labels,
descriptions, and aliases and the subable grouping them with the actual value.
This can become a new more compact JSON model (the old model is just a legacy
that may be kept for compatibility but no longer recommanded).
I've made tests already, and the number of tables for a full entity (even
without filtering any language, or sitelinks) is divided by 3, the total number
of strings is also divided by 4 to 5 sometimes more, and using it in Lua
modules is much faster and simpler (with mich less table lookups, so it saves
CPU time as well)!
As well check that Scribunto uses the 32-bit version and not the 64-bit
version of Lua (given the memory contraints of ~50MB for Lua scripts in
Scribunto, the 64-bit version just wastes memory for lot of pointers inside
tables and for pointing to strings, the 64-bit version would offer no interest
as we will never allow scripts to use more than 3 Gigabytes).
TASK DETAIL
https://phabricator.wikimedia.org/T253485
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Verdy_p
Cc: Verdy_p, Lydia_Pintscher, Aklapper, Mike_Peel, darthmon_wmde, Nandana,
lucamauri, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen,
rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331
_______________________________________________
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
