Smalyshev created this task.
Smalyshev added projects: Wikidata, Commons, SDC General,
Wikidata-Query-Service, WikibaseMediaInfo.
TASK DESCRIPTION
When dumping the entities to RDF, we try to load the data in batches, so that
we do not call the database for each entity separately. However, this does not
work for MediaInfo, since it has this:
public function getTitleForId( EntityId $id ) {
return Title::newFromID( $id->getNumericId() );
}
Which produces database fetch for every entity ID. Backtrace leading to this
is:
frame #0: Title::newFromID(id=13) at /var/www/w/includes/Title.php:471
frame #1:
Wikibase\MediaInfo\Content\MediaInfoHandler->getTitleForId(id=Wikibase\MediaInfo\DataModel\MediaInfoId)
at /var/www/w/extensions/WikibaseMediaInfo/src/Content/MediaInfoHandler.php:187
frame #2:
Wikibase\Repo\Content\EntityContentFactory->getTitleForId(id=Wikibase\MediaInfo\DataModel\MediaInfoId)
at
/var/www/w/extensions/Wikibase/repo/includes/Content/EntityContentFactory.php:149
=> array_map(callback=array(2), array(9)) (internal function)
frame #3:
Wikibase\Dumpers\RdfDumpGenerator->preBatchDump(entities=array(9)) at
/var/www/w/extensions/Wikibase/repo/includes/Dumpers/RdfDumpGenerator.php:118
frame #4: Wikibase\Dumpers\DumpGenerator->dumpEntities(entityIds=array(9),
dumpCount=unknown type: 10) at
/var/www/w/extensions/Wikibase/repo/includes/Dumpers/DumpGenerator.php:295
frame #5:
Wikibase\Dumpers\DumpGenerator->generateDump(idPager=Wikibase\Repo\Store\Sql\SqlEntityIdPager)
at /var/www/w/extensions/Wikibase/repo/includes/Dumpers/DumpGenerator.php:262
Probably we may need some kind of batching strategy do deal with it? It's
different from other entities because other entities can create Title from
EntityId without DB access. Ironically, the code that uses Titles -
PageProps::getProperties - actually needs IDs, so it immediately converts
Titles to IDs. Moreover, I'm not even sure page properties that we dump even
exist for NS_FILE+MediaInfo entities. So we're wasting a lot of DB queries for
nothing.
TASK DETAIL
https://phabricator.wikimedia.org/T222497
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Smalyshev
Cc: daniel, Lucas_Werkmeister_WMDE, Addshore, Ramsey-WMF, Aklapper,
WMDE-leszek, Smalyshev, alaa_wmde, Nandana, JKSTNK, Lahi, PDrouin-WMF, Gq86,
E1presidente, Cparle, Anooprao, SandraF_WMF, GoranSMilovanovic, QZanden,
EBjune, Tramullas, Acer, merbst, LawExplorer, Salgo60, Silverfish, Poyekhali,
_jensen, rosalieper, Jonas, Xmlizer, Susannaanas, Wong128hk, Jane023, jkroll,
Wikidata-bugs, Jdouglas, Base, matthiasmullie, aude, Tobias1984, El_Grafo,
Dinoguy1000, Manybubbles, Ricordisamoa, Wesalius, Lydia_Pintscher,
Fabrice_Florin, Raymond, Jdforrester-WMF, Steinsplitter, Mbch331, Keegan
_______________________________________________
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
