Re: [Wikitech-ambassadors] New search backend live on mediawiki.org
Sorry for not sending an update earlier. This week has been crazy. Anyway, we switched CirrusSearch to the primary search backend on MediaWiki on Wednesday morning San Francisco time. Nothing is on fire yet so the release was successful in that sense but we've filed three new bugs so it certainly wasn't an unmitigated success. We're probably getting to the point where we can start converting wikis volunteered by ambassadors. We'd add CirrusSearch as a secondary, build the index, and then we and the ambassador will do some testing with the special URL parameter mentioned at the beginning of this thread. When we're all confident that CirrusSearch is an improvement over what is in production now for that wiki we'll switch it over to primary. I'd like to start this process for a few wikis soon. Italian Wikctionary has already been volunteered so we'll add CirrusSearch as a secondary for it soon. I'll be back to working full steam on bugs next week and many of the currently open bugs are waiting on the next release of Elasticsearch which is supposed to be real soon so they should fall into place pretty quickly after we upgrade. You can always check the open bugs here: https://bugzilla.wikimedia.org/buglist.cgi?columnlist=priority%2Cbug_status%2Cresolution%2Cassigned_to%2Cshort_desc%2Cchangeddate%2Cstatus_whiteboardcomponent=CirrusSearchlist_id=233075product=MediaWiki%20extensionsquery_format=advancedresolution=---resolution=LATERresolution=DUPLICATEorder=priority%2Cbug_status%2Cbug_severity%2Cassigned_to%2Cbug_idquery_based_on=CirrusSearch Nik On Thu, Sep 12, 2013 at 10:57 PM, billinghurst billinghu...@gmail.com wrote: Is there any update on this for the wider world? If successful, is there a timetable for broader implementation? thanks. Regards billinghurst On Wed, 28 Aug 2013 14:21:32 -0400, Nikolas Everett never...@wikimedia.org wrote: Today we threw the big lever and turned on our new search backend at mediawiki.org. It isn't the default yet but it is just about ready for you to try. Here is what is we think we've improved: 1. Templates are now expanded during search so: 1a. You can search for text included in templates 1b. You can search for categories included in templates 2. The search engine is updated very quickly after articles change. 3. A few funky things around intitle and incategory: 3a. You can combine them with a regular query (incategory:kings peaceful) 3b. You can use prefix searches with them (incategory:norma*) 3c. You can use them everywhere in the query (roger incategory:normans) What we think we've made worse and we're working on fixing: 1. Because we're expanding templates some things that probably shouldn't be searched are being searched. We've fixed a few of these issues but I wouldn't be surprised if more come up. We opened Bug 53426 regarding audio tags. 2. The relative weighting of matches is going to be different. We're still fine tuning this and we'd appreciate any anecdotes describing search results that seem out of order. 3. We don't currently index headings beyond the article title in any special way. We'll be fixing that soon. (Bug 53481) 4. Searching for file names or clusters of punctuation characters doesn't work as well as it used to. It still works reasonably well if you surround your query in quotes but it isn't as good as it was. (Bugs 53013 and 52948) 5. Did you mean suggestions currently aren't highlighted at all and sometimes we'll suggest things that aren't actually better. (Bugs 52286 and 52860) 6. incategory:category with spaces isn't working. (Bug 53415) What we've changed that you probably don't care about: 1. Updating search in bulk is much more slow then before. This is the cost of expanding templates. 2. Search is now backed by a horizontally scalable search backend that is being actively developed (Elasticsearch) so we're in a much better place to expand on the new solution as time goes on. Neat stuff if you run your own MediaWiki: CirrusSearch is much easier to install than our current search infrastructure. So what will you notice? Nothing! That is because while the new search backend (CirrusSearch) is indexing we've left the current search infrastructure as the default while we work on our list of bugs. You can see the results from CirrusSearch by performing your search as normal and adding srbackend=CirrusSearch to the url parameters. If you notice any problems with CirrusSearch please file bugs directly for it: https://bugzilla.wikimedia.org/enter_bug.cgi?product=MediaWiki%20extensionscomponent=CirrusSearch Nik Everett ___ Wikitech-ambassadors mailing list Wikitech-ambassadors@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-ambassadors ___ Wikitech-ambassadors mailing list Wikitech-ambassadors@lists.wikimedia.org
Re: [Wikitech-ambassadors] New search backend live on mediawiki.org
Thanks for the update, and zero need to be sorry. Appreciate that report and the grand work in the new indexing and searching functionality. Without asking the community that I somewhat represent here, I believe that English Wikisource would be a good place to test the Wikisources, as one of the wikis that is upper-medium in size. The Wikisources are looking forward to testing the ability to have transcluded x-ns pages indexed (Page: - main) which is a particular advantage with the CirrusSearch (which I successfully tested at test2wiki). Regards, Billinghurst On Fri, 13 Sep 2013 02:56:15 -0400, Nikolas Everett never...@wikimedia.org wrote: Sorry for not sending an update earlier. This week has been crazy. Anyway, we switched CirrusSearch to the primary search backend on MediaWiki on Wednesday morning San Francisco time. Nothing is on fire yet so the release was successful in that sense but we've filed three new bugs so it certainly wasn't an unmitigated success. We're probably getting to the point where we can start converting wikis volunteered by ambassadors. We'd add CirrusSearch as a secondary, build the index, and then we and the ambassador will do some testing with the special URL parameter mentioned at the beginning of this thread. When we're all confident that CirrusSearch is an improvement over what is in production now for that wiki we'll switch it over to primary. I'd like to start this process for a few wikis soon. Italian Wikctionary has already been volunteered so we'll add CirrusSearch as a secondary for it soon. I'll be back to working full steam on bugs next week and many of the currently open bugs are waiting on the next release of Elasticsearch which is supposed to be real soon so they should fall into place pretty quickly after we upgrade. You can always check the open bugs here: https://bugzilla.wikimedia.org/buglist.cgi?columnlist=priority%2Cbug_status%2Cresolution%2Cassigned_to%2Cshort_desc%2Cchangeddate%2Cstatus_whiteboardcomponent=CirrusSearchlist_id=233075product=MediaWiki%20extensionsquery_format=advancedresolution=---resolution=LATERresolution=DUPLICATEorder=priority%2Cbug_status%2Cbug_severity%2Cassigned_to%2Cbug_idquery_based_on=CirrusSearch Nik On Thu, Sep 12, 2013 at 10:57 PM, billinghurst billinghu...@gmail.com wrote: Is there any update on this for the wider world? If successful, is there a timetable for broader implementation? thanks. Regards billinghurst ___ Wikitech-ambassadors mailing list Wikitech-ambassadors@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-ambassadors
Re: [Wikitech-ambassadors] New search backend live on mediawiki.org
On 2013-08-28 8:24 PM, Sumana Harihareswara suma...@wikimedia.org wrote: On 08/28/2013 02:21 PM, Nikolas Everett wrote: 3. A few funky things around intitle and incategory: 3a. You can combine them with a regular query (incategory:kings peaceful) 3b. You can use prefix searches with them (incategory:norma*) 3c. You can use them everywhere in the query (roger incategory:normans) I especially want to point out here that, with the new search, we can do category intersection searches, which people have wanted FOR YEARS. Just as an example picked at random, you could search for incategory:women and incategory:novelists and incategory:American. Just to pick an example at random. ;-) https://en.wikipedia.org/wiki/Wikipedia:Categories_for_discussion/Log/2013_April_24#Category:American_women_novelists This is really exciting but we want to make sure it works right before we roll it out to our bigger sites! So please do try it ou. On test2.wikipedia.org you can copy templates and articles from your home wiki and then run the search to see if things work okay. -- Sumana Harihareswara Engineering Community Manager Wikimedia Foundation ___ Wikitech-ambassadors mailing list Wikitech-ambassadors@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-ambassadors Its not that that's new, its that it actually works. (We previously had category intersection of this form, it just didn't include categories from templates since template expansion wasn't done) I think the community members who want this really want it with a more discoverable form. Maybe once new search is rolled out we should look into making an advanced search interface. -bawolff ___ Wikitech-ambassadors mailing list Wikitech-ambassadors@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-ambassadors
Re: [Wikitech-ambassadors] New search backend live on mediawiki.org
Brian Wolff, 13/09/2013 13:52: Its not that that's new, its that it actually works. (We previously had category intersection of this form, it just didn't include categories from templates since template expansion wasn't done) I think the community members who want this really want it with a more discoverable form. Maybe once new search is rolled out we should look into making an advanced search interface. That's https://bugzilla.wikimedia.org/show_bug.cgi?id=21988 I suppose. You'd probably want an advanced search interface that can be expanded or modified by extensions, so a set of blocking bugs in core and CirrusSearch. Currently extensions can't even change what profile is the default (https://bugzilla.wikimedia.org/show_bug.cgi?id=38395). Nemo ___ Wikitech-ambassadors mailing list Wikitech-ambassadors@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-ambassadors
[Wikitech-ambassadors] Fwd: Call for tech projects needing contributors
Hello ambassadors! Your help spreading this message in your communities is welcome. Thank you! Original Message Subject: Call for tech projects needing contributors Date: Fri, 13 Sep 2013 07:34:31 -0700 From: Quim Gil q...@wikimedia.org Organization: Wikimedia Foundation To: Wikimedia developers wikitec...@lists.wikimedia.org The current round of Google Summer of Code FLOSS Outreach Program for Women is about to end, and it's time to start a new cycle of mentored projects in Wikimedia tech. Check and contribute to https://www.mediawiki.org/wiki/Mentorship_programs/Possible_projects if you are * a Wikimedia project awaiting a specific software feature * an organization with budget for tech activities looking for a short term goal * a tech contributor with a cool idea for Wikimedia projects or MediaWiki in general Even if software development is a prominent activity, we also encourage proposals focusing on other technical areas: quality assurance, design, sysadmin, promotion... Post your proposal soon, edit it often. By submitting a proposal to the Possible Projects page you get attention and help from the tech community in the form of reality checks and contacts with possible mentors, interested projects and funding sources. 21 projects were selected in our last round, finishing now: https://www.mediawiki.org/wiki/Summer_of_Code_2013 https://www.mediawiki.org/wiki/Outreach_Program_for_Women We keep searching for more opportunities to channels these projects, both within the Wikimedia movement (Individual Engagement Grants, chapters...) and out there (internship programs encouraging free software and diversity in tech). We want to hear your feedback! Use the discussion page or reply here. -- Quim Gil Technical Contributor Coordinator @ Wikimedia Foundation http://www.mediawiki.org/wiki/User:Qgil ___ Wikitech-ambassadors mailing list Wikitech-ambassadors@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-ambassadors
