Jheald added a comment. I think you underline exactly my point.
The size of the two final solution sets is very similar. Per the query above, there are currently 345,221 category-items with a https://phabricator.wikimedia.org/P373; whereas according to Autolist (CLAIM[31:4167836] AND LINK[commonswiki]), there are currently 340,027 category-items with a Commons sitelink. In fact, the total number of Commons sitelinks (680,000) going into the join is rather less than the total number of https://phabricator.wikimedia.org/P373 statements (1,240,000). But the first query is *much* slower because those 680,000 Commons sitelinks are not easily accessible. Instead to get there the system has to try to do a join with the set of all 46 million sitelinks, then join again with the set of sitelinks in English (7.7 million), which it then has to filter to get the sitelinks to Commons, rather than en-wiki or en-wikisource. So it's not surprising that it is far far slower than it would be if relations were available like ?item wikibase:hasSitelinkTo wd:Q565 Now I know very little about the internals of Wikibase, so I don't know what is specific to Wikidata compared to what is generic to Wikibase (or Wikibase coupled to a generic MediaWiki installation). But at least for Wikidata, and the Wikidata UI, it does seem that the set of available destination sitelinks is very narrowly controlled, so it would not be too hard to keep track of a list of corresponding items in the Wikibase. I think doing that would have significant advantages for us. But if another installation was not keeping such a list, then it would be easy enough to set a flag that it was not available, so no "wikibase:hasSitelinkTo" statements would be being maintained, and that other installation would be no worse off than we are already. However, as I understand it, the list of available projects that can be sitelinked to is pretty fundamental in Wikibase, with special code to handle the addition and removal of such sitelinks and reflect them in external database tables, so it should not be too hard to add a routine to have them reflected in the triplestore too. TASK DETAIL https://phabricator.wikimedia.org/T120198 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Jheald Cc: Smalyshev, Aklapper, Steinsplitter, StudiesWorld, Jheald, jkroll, Wikidata-bugs, Jdouglas, aude, Deskana, Manybubbles, Mbch331 _______________________________________________ Wikidata-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
