[Wikitech-l] Cross-wiki search API?

2015-09-11 Thread Magnus Manske
I seem to remember that all Wikimedia wikis now share a single search index, and per-wiki searches are filtered through a tag for the respective wiki. If that is indeed the case, is there an API to search all wikis, but omitting that tag? ___ Wikitech-l

Re: [Wikitech-l] Cross-wiki search API?

2015-09-11 Thread Eran Rosenthal
See also related bug: https://phabricator.wikimedia.org/T71489 On Fri, Sep 11, 2015 at 12:45 PM, Magnus Manske wrote: > I seem to remember that all Wikimedia wikis now share a single search > index, and per-wiki searches are filtered through a tag for the

Re: [Wikitech-l] Cross-wiki search API?

2015-09-11 Thread Erik Bernhardson
I should add we are currently also looking into https://phabricator.wikimedia.org/T109715 which would make the full data set available for raw queries (directly to elasticsearch) to labs users. When issuing raw queries you can query every index in elasticsearch. its not performant at all, but it

Re: [Wikitech-l] Cross-wiki search API?

2015-09-11 Thread Erik Bernhardson
Unfortunately there is not a single index that is then filtered by tags, in WMF production we currently have 1824 different search indexes. Due to these being all independent search indexes the scores generated by queries in one in index are not directly comparable to the scores generated by