Re: [Wikitech-l] Disambiguator extension deployed to all WMF wikis (action required)
Thanks a lot, I've updated WPCleaner [1] to use the new disambiguation property, instead of looking for templates or categories. I've been able to reduce the number of calls to the API when analzying a page for links to disambiguation pages with this property. Nico [1] http://en.wikipedia.org/wiki/Wikipedia:WPCleaner On Wed, Jul 10, 2013 at 12:10 AM, Ryan Kaldari rkald...@wikimedia.orgwrote: The Disambiguator extension (http://www.mediawiki.org/** wiki/Extension:Disambiguatorhttp://www.mediawiki.org/wiki/Extension:Disambiguator) is now deployed to all WMF wikis. This will enable us to: 1. Remove disambiguation code from core, including Special:Disambiguations (bug 35981) 2. Stop requiring wikis to maintain template lists at MediaWiki:Disambiguationspage 3. Add features like warning users when they are linking to disambiguation pages (https://gerrit.wikimedia.org/**r/#/c/70564https://gerrit.wikimedia.org/r/#/c/70564 ) 4. Remove disambiguation pages from things like Special:Random and Special:LonelyPages 4. Enable the development of more powerful 3rd party tools for dealing with disambiguation pages There is, however, one action required of each wiki that wants to make use of the Disambiguator extension: Every disambiguation page on the wiki needs to include the __DISAMBIG__ magic word (or an equivalent alias). Typically, this only requires adding the magic word to a single template that is included on all the disambiguation pages. For example, on Commons, this was accomplished with the following change: https://commons.wikimedia.org/**w/index.php?title=Template%** 3ADisambigdiff=99758122**oldid=99728960https://commons.wikimedia.org/w/index.php?title=Template%3ADisambigdiff=99758122oldid=99728960 On English Wikipedia, it was a bit more complicated: https://en.wikipedia.org/w/**index.php?title=Template%** 3ADmboxdiff=560507118oldid=**540384230https://en.wikipedia.org/w/index.php?title=Template%3ADmboxdiff=560507118oldid=540384230 Once you've made this change, you should start seeing pages appear on Special:DisambiguationPages within 3 days. If you have any questions or problems, let me know. Ryan Kaldari Wikimedia Foundation __**_ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/**mailman/listinfo/wikitech-lhttps://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Disambiguator extension deployed to all WMF wikis (action required)
Nicolas Vervelle wrote: Has the API been modified so that we can ask it if a page is a disambiguation page ? Looks like it. Starting point: https://en.wikipedia.org/w/api.php List of available property names: https://en.wikipedia.org/w/api.php?action=querylist=pagepropnamesppnlimit =100 Look up properties of a particular title: https://en.wikipedia.org/w/api.php?action=queryprop=pagepropstitles=Madon na pageprops disambiguation= wikibase_item=q1564372 / https://en.wikipedia.org/wiki/Special:PagesWithProp can look up pages by property name. I'm not sure if there's an equivalent module for the Web API yet. The Web API has disambiguation-related query pages as well (including Special:DisambiguationPageLinks, which I'm only now learning exists). MZMcBride ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Disambiguator extension deployed to all WMF wikis (action required)
Nice extension :) You may use generator to enjoy this new property. For example to check whether there is a disambig link from SOME_TITLE en.wikipedia.org/w/api.php?action=querygenerator=linkstitles=SOME_TITLEprop=pagepropsppprop=disambiguationgpllimit=500 (without this extension it was possible only with external tools such as dablinks: http://toolserver.org/~dispenser/view/Dablinks) On Wed, Jul 10, 2013 at 9:00 AM, MZMcBride z...@mzmcbride.com wrote: Nicolas Vervelle wrote: Has the API been modified so that we can ask it if a page is a disambiguation page ? Looks like it. Starting point: https://en.wikipedia.org/w/api.php List of available property names: https://en.wikipedia.org/w/api.php?action=querylist=pagepropnamesppnlimit =100 Look up properties of a particular title: https://en.wikipedia.org/w/api.php?action=queryprop=pagepropstitles=Madon na pageprops disambiguation= wikibase_item=q1564372 / https://en.wikipedia.org/wiki/Special:PagesWithProp can look up pages by property name. I'm not sure if there's an equivalent module for the Web API yet. The Web API has disambiguation-related query pages as well (including Special:DisambiguationPageLinks, which I'm only now learning exists). MZMcBride ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Disambiguator extension deployed to all WMF wikis (action required)
As all templates requiring it are listed in [[MediaWiki:Disambiguationspage]], why not run a global bot? It's surely easier than trying to find 800 editors. Nemo ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Disambiguator extension deployed to all WMF wikis (action required)
I bet the JobQueue would be loving that. On Wed, Jul 10, 2013 at 4:52 PM, Federico Leva (Nemo) nemow...@gmail.com wrote: As all templates requiring it are listed in [[MediaWiki:Disambiguationspage]], why not run a global bot? It's surely easier than trying to find 800 editors. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Disambiguator extension deployed to all WMF wikis (action required)
Thanks Eran and MZMcBride, I'm going to update WPCleaner to take advantage of this new possibility. It should result in less API requests and so a faster loading of pages for fixing disambiguation links. Great :) Nico On Wed, Jul 10, 2013 at 8:33 AM, Eran Rosenthal eranro...@gmail.com wrote: Nice extension :) You may use generator to enjoy this new property. For example to check whether there is a disambig link from SOME_TITLE en.wikipedia.org/w/api.php?action=querygenerator=linkstitles=SOME_TITLEprop=pagepropsppprop=disambiguationgpllimit=500 (without this extension it was possible only with external tools such as dablinks: http://toolserver.org/~dispenser/view/Dablinks) On Wed, Jul 10, 2013 at 9:00 AM, MZMcBride z...@mzmcbride.com wrote: Nicolas Vervelle wrote: Has the API been modified so that we can ask it if a page is a disambiguation page ? Looks like it. Starting point: https://en.wikipedia.org/w/api.php List of available property names: https://en.wikipedia.org/w/api.php?action=querylist=pagepropnamesppnlimit =100 Look up properties of a particular title: https://en.wikipedia.org/w/api.php?action=queryprop=pagepropstitles=Madon na pageprops disambiguation= wikibase_item=q1564372 / https://en.wikipedia.org/wiki/Special:PagesWithProp can look up pages by property name. I'm not sure if there's an equivalent module for the Web API yet. The Web API has disambiguation-related query pages as well (including Special:DisambiguationPageLinks, which I'm only now learning exists). MZMcBride ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Disambiguator extension deployed to all WMF wikis (action required)
Le 10/07/13 00:10, Ryan Kaldari a écrit : The Disambiguator extension (http://www.mediawiki.org/wiki/Extension:Disambiguator) is now deployed to all WMF wikis. This will enable us to: 1. Remove disambiguation code from core, including Special:Disambiguations (bug 35981) 2. Stop requiring wikis to maintain template lists at MediaWiki:Disambiguationspage 3. Add features like warning users when they are linking to disambiguation pages (https://gerrit.wikimedia.org/r/#/c/70564) 4. Remove disambiguation pages from things like Special:Random and Special:LonelyPages 4. Enable the development of more powerful 3rd party tools for dealing with disambiguation pages That is really wonderful! I love the Disambiguation system but always thought it was not very nicely implemented. Thank you a ton to have taken care of making it more modern. Kudos to everyone involved \O/ -- Antoine hashar Musso ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Disambiguator extension deployed to all WMF wikis (action required)
Good, Thanks to Eran answer, I know how to determine for all links in a page which one are to a disambiguation page. For WPCleaner, I also need to be able to retrieve the complete list of disambiguation pages through the API. I looked at the allpages request [1], but I didn't see a way to get only pages with this property set. Is there an other way ? Nico [1] http://www.mediawiki.org/wiki/API:Allpages On Wed, Jul 10, 2013 at 9:30 AM, Nicolas Vervelle nverve...@gmail.comwrote: Thanks Eran and MZMcBride, I'm going to update WPCleaner to take advantage of this new possibility. It should result in less API requests and so a faster loading of pages for fixing disambiguation links. Great :) Nico On Wed, Jul 10, 2013 at 8:33 AM, Eran Rosenthal eranro...@gmail.comwrote: Nice extension :) You may use generator to enjoy this new property. For example to check whether there is a disambig link from SOME_TITLE en.wikipedia.org/w/api.php?action=querygenerator=linkstitles=SOME_TITLEprop=pagepropsppprop=disambiguationgpllimit=500 (without this extension it was possible only with external tools such as dablinks: http://toolserver.org/~dispenser/view/Dablinks) On Wed, Jul 10, 2013 at 9:00 AM, MZMcBride z...@mzmcbride.com wrote: Nicolas Vervelle wrote: Has the API been modified so that we can ask it if a page is a disambiguation page ? Looks like it. Starting point: https://en.wikipedia.org/w/api.php List of available property names: https://en.wikipedia.org/w/api.php?action=querylist=pagepropnamesppnlimit =100 Look up properties of a particular title: https://en.wikipedia.org/w/api.php?action=queryprop=pagepropstitles=Madon na pageprops disambiguation= wikibase_item=q1564372 / https://en.wikipedia.org/wiki/Special:PagesWithProp can look up pages by property name. I'm not sure if there's an equivalent module for the Web API yet. The Web API has disambiguation-related query pages as well (including Special:DisambiguationPageLinks, which I'm only now learning exists). MZMcBride ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Disambiguator extension deployed to all WMF wikis (action required)
So, a disambiguation page still a article? Or not? Chinese wikipedia: http://zh.wikipedia.org/ My blog: http://shizhao.org twitter: https://twitter.com/shizhao [[zh:User:Shizhao]] 2013/7/10 Nicolas Vervelle nverve...@gmail.com: Good, Thanks to Eran answer, I know how to determine for all links in a page which one are to a disambiguation page. For WPCleaner, I also need to be able to retrieve the complete list of disambiguation pages through the API. I looked at the allpages request [1], but I didn't see a way to get only pages with this property set. Is there an other way ? Nico [1] http://www.mediawiki.org/wiki/API:Allpages On Wed, Jul 10, 2013 at 9:30 AM, Nicolas Vervelle nverve...@gmail.comwrote: Thanks Eran and MZMcBride, I'm going to update WPCleaner to take advantage of this new possibility. It should result in less API requests and so a faster loading of pages for fixing disambiguation links. Great :) Nico On Wed, Jul 10, 2013 at 8:33 AM, Eran Rosenthal eranro...@gmail.comwrote: Nice extension :) You may use generator to enjoy this new property. For example to check whether there is a disambig link from SOME_TITLE en.wikipedia.org/w/api.php?action=querygenerator=linkstitles=SOME_TITLEprop=pagepropsppprop=disambiguationgpllimit=500 (without this extension it was possible only with external tools such as dablinks: http://toolserver.org/~dispenser/view/Dablinks) On Wed, Jul 10, 2013 at 9:00 AM, MZMcBride z...@mzmcbride.com wrote: Nicolas Vervelle wrote: Has the API been modified so that we can ask it if a page is a disambiguation page ? Looks like it. Starting point: https://en.wikipedia.org/w/api.php List of available property names: https://en.wikipedia.org/w/api.php?action=querylist=pagepropnamesppnlimit =100 Look up properties of a particular title: https://en.wikipedia.org/w/api.php?action=queryprop=pagepropstitles=Madon na pageprops disambiguation= wikibase_item=q1564372 / https://en.wikipedia.org/wiki/Special:PagesWithProp can look up pages by property name. I'm not sure if there's an equivalent module for the Web API yet. The Web API has disambiguation-related query pages as well (including Special:DisambiguationPageLinks, which I'm only now learning exists). MZMcBride ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Disambiguator extension deployed to all WMF wikis (action required)
On Wed, Jul 10, 2013 at 2:00 AM, MZMcBride z...@mzmcbride.com wrote: Nicolas Vervelle wrote: Has the API been modified so that we can ask it if a page is a disambiguation page ? https://en.wikipedia.org/wiki/Special:PagesWithProp can look up pages by property name. I'm not sure if there's an equivalent module for the Web API yet. https://en.wikipedia.org/w/api.php?action=querylist=pageswithproppwppropname=disambiguation The Web API has disambiguation-related query pages as well (including Special:DisambiguationPageLinks, which I'm only now learning exists). Special:DisambiguationPageLinks and Special:DisambiguationPages are added by the Disambiguator extension. -- Brad Jorsch (Anomie) Software Engineer Wikimedia Foundation ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Disambiguator extension deployed to all WMF wikis (action required)
On Wed, 10 Jul 2013 10:49:27 -0700, Ryan Kaldari rkald...@wikimedia.org wrote: On 7/10/13 2:46 AM, Nicolas Vervelle wrote: Good, Thanks to Eran answer, I know how to determine for all links in a page which one are to a disambiguation page. For WPCleaner, I also need to be able to retrieve the complete list of disambiguation pages through the API. I looked at the allpages request [1], but I didn't see a way to get only pages with this property set. Is there an other way ? Disambiguator implements an API for retrieving all disambiguation pages: api.php?action=querylist=querypageqppage=DisambiguationPages Unfortunately, since the WMF wikis run in MiserMode, this API will only return a maximum of 1000-5000 results (depending on what wiki you're on). I'm going to do some profiling/optimization on the query though and see if it would be possible to remove the 'expensive' designation, which would allow retrieving all the disambiguation pages. Ryan Kaldari Wasn't the expensive designation due to the use of multiple templatelinks in the query? Now that page_props are being used instead the expensive bit might go away. -- ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/] ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Disambiguator extension deployed to all WMF wikis (action required)
On 7/10/13 11:02 AM, Daniel Friesen wrote: On Wed, 10 Jul 2013 10:49:27 -0700, Ryan Kaldari rkald...@wikimedia.org wrote: Disambiguator implements an API for retrieving all disambiguation pages: api.php?action=querylist=querypageqppage=DisambiguationPages Unfortunately, since the WMF wikis run in MiserMode, this API will only return a maximum of 1000-5000 results (depending on what wiki you're on). I'm going to do some profiling/optimization on the query though and see if it would be possible to remove the 'expensive' designation, which would allow retrieving all the disambiguation pages. Ryan Kaldari Wasn't the expensive designation due to the use of multiple templatelinks in the query? Now that page_props are being used instead the expensive bit might go away I've optimized the query so that it is no longer expensive. Just needs code review: https://gerrit.wikimedia.org/r/#/c/73008 Ryan Kaldari ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Disambiguator extension deployed to all WMF wikis (action required)
On Wed, Jul 10, 2013 at 1:49 PM, Ryan Kaldari rkald...@wikimedia.org wrote: Disambiguator implements an API for retrieving all disambiguation pages: api.php?action=querylist=querypageqppage=DisambiguationPages ... That really shouldn't be allowed by list=querypage, since we have list=pageswithprop that does the same thing. I submitted https://gerrit.wikimedia.org/r/73034 and https://gerrit.wikimedia.org/r/73035 to take care of that. -- Brad Jorsch (Anomie) Software Engineer Wikimedia Foundation ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Disambiguator extension deployed to all WMF wikis (action required)
On 7/10/13 2:15 PM, Brad Jorsch (Anomie) wrote: On Wed, Jul 10, 2013 at 1:49 PM, Ryan Kaldari rkald...@wikimedia.org wrote: Disambiguator implements an API for retrieving all disambiguation pages: api.php?action=querylist=querypageqppage=DisambiguationPages ... That really shouldn't be allowed by list=querypage, since we have list=pageswithprop that does the same thing. Oops, I didn't know about that one. Looks like you can now pull all the disambiguation pages using: https://en.wikipedia.org/w/api.php?action=querygenerator=pageswithpropgpwppropname=disambiguation That should take care of Nico's request. Ryan Kaldari ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Disambiguator extension deployed to all WMF wikis (action required)
Yes, thanks, I will use pageswithprop to retrieve all disambiguation pages. I didn't find it at first because it wasn't listed in http://www.mediawiki.org/wiki/API:Lists Nico On Thu, Jul 11, 2013 at 12:49 AM, Ryan Kaldari rkald...@wikimedia.orgwrote: On 7/10/13 2:15 PM, Brad Jorsch (Anomie) wrote: On Wed, Jul 10, 2013 at 1:49 PM, Ryan Kaldari rkald...@wikimedia.org wrote: Disambiguator implements an API for retrieving all disambiguation pages: api.php?action=querylist=**querypageqppage=**DisambiguationPages ... That really shouldn't be allowed by list=querypage, since we have list=pageswithprop that does the same thing. Oops, I didn't know about that one. Looks like you can now pull all the disambiguation pages using: https://en.wikipedia.org/w/**api.php?action=query** generator=pageswithprop**gpwppropname=disambiguationhttps://en.wikipedia.org/w/api.php?action=querygenerator=pageswithpropgpwppropname=disambiguation That should take care of Nico's request. Ryan Kaldari __**_ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/**mailman/listinfo/wikitech-lhttps://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Disambiguator extension deployed to all WMF wikis (action required)
The Disambiguator extension (http://www.mediawiki.org/wiki/Extension:Disambiguator) is now deployed to all WMF wikis. This will enable us to: 1. Remove disambiguation code from core, including Special:Disambiguations (bug 35981) 2. Stop requiring wikis to maintain template lists at MediaWiki:Disambiguationspage 3. Add features like warning users when they are linking to disambiguation pages (https://gerrit.wikimedia.org/r/#/c/70564) 4. Remove disambiguation pages from things like Special:Random and Special:LonelyPages 4. Enable the development of more powerful 3rd party tools for dealing with disambiguation pages There is, however, one action required of each wiki that wants to make use of the Disambiguator extension: Every disambiguation page on the wiki needs to include the __DISAMBIG__ magic word (or an equivalent alias). Typically, this only requires adding the magic word to a single template that is included on all the disambiguation pages. For example, on Commons, this was accomplished with the following change: https://commons.wikimedia.org/w/index.php?title=Template%3ADisambigdiff=99758122oldid=99728960 On English Wikipedia, it was a bit more complicated: https://en.wikipedia.org/w/index.php?title=Template%3ADmboxdiff=560507118oldid=540384230 Once you've made this change, you should start seeing pages appear on Special:DisambiguationPages within 3 days. If you have any questions or problems, let me know. Ryan Kaldari Wikimedia Foundation ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Disambiguator extension deployed to all WMF wikis (action required)
Great !!! Has the API been modified so that we can ask it if a page is a disambiguation page ? Nico On Wed, Jul 10, 2013 at 12:10 AM, Ryan Kaldari rkald...@wikimedia.orgwrote: The Disambiguator extension (http://www.mediawiki.org/** wiki/Extension:Disambiguatorhttp://www.mediawiki.org/wiki/Extension:Disambiguator) is now deployed to all WMF wikis. This will enable us to: 1. Remove disambiguation code from core, including Special:Disambiguations (bug 35981) 2. Stop requiring wikis to maintain template lists at MediaWiki:Disambiguationspage 3. Add features like warning users when they are linking to disambiguation pages (https://gerrit.wikimedia.org/**r/#/c/70564https://gerrit.wikimedia.org/r/#/c/70564 ) 4. Remove disambiguation pages from things like Special:Random and Special:LonelyPages 4. Enable the development of more powerful 3rd party tools for dealing with disambiguation pages There is, however, one action required of each wiki that wants to make use of the Disambiguator extension: Every disambiguation page on the wiki needs to include the __DISAMBIG__ magic word (or an equivalent alias). Typically, this only requires adding the magic word to a single template that is included on all the disambiguation pages. For example, on Commons, this was accomplished with the following change: https://commons.wikimedia.org/**w/index.php?title=Template%** 3ADisambigdiff=99758122**oldid=99728960https://commons.wikimedia.org/w/index.php?title=Template%3ADisambigdiff=99758122oldid=99728960 On English Wikipedia, it was a bit more complicated: https://en.wikipedia.org/w/**index.php?title=Template%** 3ADmboxdiff=560507118oldid=**540384230https://en.wikipedia.org/w/index.php?title=Template%3ADmboxdiff=560507118oldid=540384230 Once you've made this change, you should start seeing pages appear on Special:DisambiguationPages within 3 days. If you have any questions or problems, let me know. Ryan Kaldari Wikimedia Foundation __**_ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/**mailman/listinfo/wikitech-lhttps://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
