Jarekt created this task. Jarekt added projects: Wikidata-Query-Service, Commons. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata.
TASK DESCRIPTION At the moment Wikimedia Commons Query Service (WCQS) allows SPARQL queries to look up stuff on wikidata using "service <https://query.wikidata.org/sparql>" (see Example <https://commons.wikimedia.org/wiki/Commons:SPARQL_query_service/queries/examples#Map_of_images_found_in_the_medieval_Atlas_De_Wit>) or in Commons categories using "service wikibase:mwapi" (see example <https://commons.wikimedia.org/wiki/Commons:SPARQL_query_service/queries/examples#Wikidata_items_of_files_in_Category:Artworks_with_structured_data_with_redirected_P6243_property>). Unfortunately those 2 services only allow a limited number of records to pass to and from. Limit of service <https://query.wikidata.org/sparql> is in thousands or tens of thousands and limit on service wikibase:mwapi is 10k (see here <https://www.mediawiki.org/wiki/Wikidata_Query_Service/User_Manual/MWAPI#Pagination>) The issue is that we now have 65M files on commons and we should expect similar number of SDC items. I would like to search all the wikidata items in SDC to detect redirects. That would require to send millions of item numbers to https://query.wikidata.org/sparql service. I would also like to find all the files in a category with millions of files that do not have some SDC property. Those 2 use-cases would require much larger limits on number of records sent to those 2 services. TASK DETAIL https://phabricator.wikimedia.org/T261716 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Jarekt Cc: Jarekt, Aklapper, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86, Ramsey-WMF, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Poyekhali, _jensen, rosalieper, Taiwania_Justo, Scott_WUaS, Jonas, Xmlizer, Ixocactus, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, El_Grafo, Dinoguy1000, Manybubbles, Steinsplitter, Mbch331, Keegan
_______________________________________________ Wikidata-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
