Most definitely! That would be absolutely fantastic. What format of list would be most useful for you to work with? On 9 Sep 2014 15:38, "John" <[email protected]> wrote:
> Its not that big of a deal, once i set the system up. is it possible to > have you post the list in a static location on your webserver? I could then > just have the bot grab and use that list. > > > On Tue, Sep 9, 2014 at 10:35 AM, Navino Evans <[email protected]> > wrote: > >> That's great to know, thank you. >> >> We'll make sure we only use the API within that limit - basically just >> for individual calls when a user adds a new event to our database. >> >> For the bulk processing, we would need to update the backlinks >> information as a monthly maintenance task, so I wouldn't want to trouble >> you with this each time. >> >> Would you rather we stick with data dump processing for the large scale >> stuff? >> >> >> >> On 9 Sep 2014 15:05, "John" <[email protected]> wrote: >> >>> If you want a report on that many pages drop me a list of those titles >>> and and I can write a report for you given that volume of affected pages. >>> >>> I would say 1-2 seconds between quires should be reasonable for a >>> moderate volume of quires. Any large scale request I will do server side >>> and avoid hammering the web-servers for something that is better batched. >>> >>> >>> On Tue, Sep 9, 2014 at 9:58 AM, Navino Evans <[email protected]> >>> wrote: >>> >>>> Once again, a huge thank you for taking the time to do this John - >>>> That's exactly what I was looking for! - the helpfulness of this community >>>> never ceases to amaze me :) >>>> >>>> Hopefully I haven't initiated a journey down the rabbit hole into a >>>> fully fledged muliti-language counting machine ;) >>>> >>>> >>>> Can I just ask what the limit of reasonable use would be for making API >>>> calls to this new tool? (e.g. number of calls per day) >>>> >>>> It would be incredibly useful if we could use it to update the events >>>> in our database once a month (we are using it to rank historical events by >>>> 'importance'), but we are already have approximately 1.5 million events so >>>> am aware this may be way beyond what would be acceptable. >>>> >>>> On Tue, Sep 9, 2014 at 2:56 PM, John <[email protected]> wrote: >>>> >>>>> That's doable, however it will require a little more time as I need to >>>>> unearth some old code to handle multi-projects/languages >>>>> >>>>> >>>>> On Tue, Sep 9, 2014 at 9:51 AM, Jan Ainali <[email protected]> >>>>> wrote: >>>>> >>>>>> Awesome John! >>>>>> >>>>>> Now I only wish that one could specify language code also ;) >>>>>> >>>>>> >>>>>> *Med vänliga hälsningar,Jan Ainali* >>>>>> >>>>>> Verksamhetschef, Wikimedia Sverige >>>>>> <http://se.wikimedia.org/wiki/Huvudsida> >>>>>> 0729 - 67 29 48 >>>>>> >>>>>> >>>>>> *Tänk dig en värld där varje människa har fri tillgång till >>>>>> mänsklighetens samlade kunskap. Det är det vi gör.* >>>>>> Bli medlem. <http://blimedlem.wikimedia.se> >>>>>> >>>>>> >>>>>> 2014-09-09 15:34 GMT+02:00 John <[email protected]>: >>>>>> >>>>>>> Per request, its no frills but what you what you asked for: >>>>>>> http://tools.wmflabs.org/betacommand-dev/cgi-bin/backlinks >>>>>>> >>>>>>> >>>>>>> On Tue, Sep 9, 2014 at 8:32 AM, Navino Evans <[email protected] >>>>>>> > wrote: >>>>>>> >>>>>>>> That is fantastic news... I'm incredibly grateful for the help and >>>>>>>> advice. >>>>>>>> >>>>>>>> On Tue, Sep 9, 2014 at 1:27 PM, John <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Given the overhead of the API and that he only needs a count >>>>>>>>> getting that info should be fairly easy via a python cgi wrapper >>>>>>>>> around an >>>>>>>>> sql query. >>>>>>>>> >>>>>>>>> The only thing that I cannot do is #3 since the software does not >>>>>>>>> differentiate between links in templates and links not in templates. >>>>>>>>> Its a >>>>>>>>> requested feature for years now. >>>>>>>>> >>>>>>>>> Give me a few hours and ill get you the tool you want. This should >>>>>>>>> be less than 30 minutes work >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, Sep 9, 2014 at 7:55 AM, Jan Ainali < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Related tip: In the API you can get a list of backlinks (but you >>>>>>>>>> have to count them yourself) from the main namespace including all >>>>>>>>>> redirects by a query like this: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> https://en.wikipedia.org/w/api.php?action=query&list=backlinks&format=json&bltitle=Example&blnamespace=0&blfilterredir=all&bllimit=250&blredirect= >>>>>>>>>> >>>>>>>>>> More info at: https://www.mediawiki.org/wiki/API:Backlinks >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> *Med vänliga hälsningar,Jan Ainali* >>>>>>>>>> >>>>>>>>>> Verksamhetschef, Wikimedia Sverige >>>>>>>>>> <http://se.wikimedia.org/wiki/Huvudsida> >>>>>>>>>> 0729 - 67 29 48 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> *Tänk dig en värld där varje människa har fri tillgång till >>>>>>>>>> mänsklighetens samlade kunskap. Det är det vi gör.* >>>>>>>>>> Bli medlem. <http://blimedlem.wikimedia.se> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 2014-09-09 13:41 GMT+02:00 Navino Evans <[email protected]>: >>>>>>>>>> >>>>>>>>>>> Wow! That would be awesome :) >>>>>>>>>>> >>>>>>>>>>> The API we are looking for can be as simple as sending a GET >>>>>>>>>>> request to a url ( >>>>>>>>>>> http://www.somewhere.com/api/count?t=wikipedia_title_goes_here), >>>>>>>>>>> returning a number in "text/plain" format. >>>>>>>>>>> >>>>>>>>>>> The actual count that we're interested is for English Wikipedia >>>>>>>>>>> only, and would ideally include the following, all added up into a >>>>>>>>>>> single >>>>>>>>>>> number: >>>>>>>>>>> >>>>>>>>>>> 1) All links from articles in Main Namespace only (for our >>>>>>>>>>> purpose it would be better to not include links from User pages, >>>>>>>>>>> Talk pages >>>>>>>>>>> etc if possible) >>>>>>>>>>> >>>>>>>>>>> 2) Including links from Redirect pages (e.g. counting a link >>>>>>>>>>> from "Michel Jackson" redirect as part of the count from the article >>>>>>>>>>> "Michael Jackson") >>>>>>>>>>> >>>>>>>>>>> 3) Excluding links that are within a template transcluded in an >>>>>>>>>>> article (so we don't need to count the links inside Navboxes within >>>>>>>>>>> an >>>>>>>>>>> article for example) >>>>>>>>>>> >>>>>>>>>>> 4) For our purpose, it doesn't really matter whether >>>>>>>>>>> transclusions of the actual page that is called are included in the >>>>>>>>>>> count >>>>>>>>>>> (we generally won't be using it for checking templates, timeline >>>>>>>>>>> and list >>>>>>>>>>> articles). >>>>>>>>>>> >>>>>>>>>>> Just to give the full picture for this request - my use of this >>>>>>>>>>> tool will be for a company (www.histropedia.com), so I wouldn't >>>>>>>>>>> want to take up your time with this unless it's something you feel >>>>>>>>>>> should >>>>>>>>>>> be available for wider use. My plan was to get the developer >>>>>>>>>>> working on our >>>>>>>>>>> site to make this tool for the community if it didn't exist >>>>>>>>>>> somewhere, but >>>>>>>>>>> we would be reliant on datadumps so could not get live information >>>>>>>>>>> (which >>>>>>>>>>> would be incredibly useful for us, and I hope many others). >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Mon, Sep 8, 2014 at 8:10 PM, John <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> What numbers/data do you want? I can whip up a replacement for >>>>>>>>>>>> it. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Monday, September 8, 2014, Navino Evans < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi All, >>>>>>>>>>>>> >>>>>>>>>>>>> Hi all, does anyone know if there is a tool currently >>>>>>>>>>>>> available for counting backlinks to Wikipedia articles via an >>>>>>>>>>>>> API? I have >>>>>>>>>>>>> been using this tool >>>>>>>>>>>>> http://dispenser.homenet.org/~dispenser/cgi-bin/backlinkscount.py >>>>>>>>>>>>> - but it seems to have finally gone offline completely following >>>>>>>>>>>>> some >>>>>>>>>>>>> recent controversy with user:Dispenser. >>>>>>>>>>>>> >>>>>>>>>>>>> Any advice much appreciated! >>>>>>>>>>>>> >>>>>>>>>>>>> Regards, >>>>>>>>>>>>> >>>>>>>>>>>>> Navino >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> Labs-l mailing list >>>>>>>>>>>> [email protected] >>>>>>>>>>>> https://lists.wikimedia.org/mailman/listinfo/labs-l >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> ___________________________ >>>>>>>>>>> >>>>>>>>>>> Histropedia >>>>>>>>>>> The Timeline for all of History >>>>>>>>>>> www.histropedia.com >>>>>>>>>>> >>>>>>>>>>> Follow us on: >>>>>>>>>>> Twitter <https://twitter.com/Histropedia> Facebo >>>>>>>>>>> <https://www.facebook.com/Histropedia>ok >>>>>>>>>>> <https://www.facebook.com/Histropedia> Google + >>>>>>>>>>> <https://plus.google.com/u/0/b/104484373317792180682/104484373317792180682/posts> >>>>>>>>>>> L <http://www.linkedin.com/company/histropedia-ltd>inke >>>>>>>>>>> <http://www.linkedin.com/company/histropedia-ltd>dIn >>>>>>>>>>> <http://www.linkedin.com/company/histropedia-ltd> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Labs-l mailing list >>>>>>>>>>> [email protected] >>>>>>>>>>> https://lists.wikimedia.org/mailman/listinfo/labs-l >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Labs-l mailing list >>>>>>>>>> [email protected] >>>>>>>>>> https://lists.wikimedia.org/mailman/listinfo/labs-l >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Labs-l mailing list >>>>>>>>> [email protected] >>>>>>>>> https://lists.wikimedia.org/mailman/listinfo/labs-l >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> ___________________________ >>>>>>>> >>>>>>>> Histropedia >>>>>>>> The Timeline for all of History >>>>>>>> www.histropedia.com >>>>>>>> >>>>>>>> Follow us on: >>>>>>>> Twitter <https://twitter.com/Histropedia> Facebo >>>>>>>> <https://www.facebook.com/Histropedia>ok >>>>>>>> <https://www.facebook.com/Histropedia> Google + >>>>>>>> <https://plus.google.com/u/0/b/104484373317792180682/104484373317792180682/posts> >>>>>>>> L <http://www.linkedin.com/company/histropedia-ltd>inke >>>>>>>> <http://www.linkedin.com/company/histropedia-ltd>dIn >>>>>>>> <http://www.linkedin.com/company/histropedia-ltd> >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Labs-l mailing list >>>>>>>> [email protected] >>>>>>>> https://lists.wikimedia.org/mailman/listinfo/labs-l >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Labs-l mailing list >>>>>>> [email protected] >>>>>>> https://lists.wikimedia.org/mailman/listinfo/labs-l >>>>>>> >>>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Labs-l mailing list >>>>>> [email protected] >>>>>> https://lists.wikimedia.org/mailman/listinfo/labs-l >>>>>> >>>>>> >>>>> >>>>> _______________________________________________ >>>>> Labs-l mailing list >>>>> [email protected] >>>>> https://lists.wikimedia.org/mailman/listinfo/labs-l >>>>> >>>>> >>>> >>>> >>>> -- >>>> ___________________________ >>>> >>>> Histropedia >>>> The Timeline for all of History >>>> www.histropedia.com >>>> >>>> Follow us on: >>>> Twitter <https://twitter.com/Histropedia> Facebo >>>> <https://www.facebook.com/Histropedia>ok >>>> <https://www.facebook.com/Histropedia> Google + >>>> <https://plus.google.com/u/0/b/104484373317792180682/104484373317792180682/posts> >>>> L <http://www.linkedin.com/company/histropedia-ltd>inke >>>> <http://www.linkedin.com/company/histropedia-ltd>dIn >>>> <http://www.linkedin.com/company/histropedia-ltd> >>>> >>>> >>>> _______________________________________________ >>>> Labs-l mailing list >>>> [email protected] >>>> https://lists.wikimedia.org/mailman/listinfo/labs-l >>>> >>>> >>> >>> _______________________________________________ >>> Labs-l mailing list >>> [email protected] >>> https://lists.wikimedia.org/mailman/listinfo/labs-l >>> >>> >> _______________________________________________ >> Labs-l mailing list >> [email protected] >> https://lists.wikimedia.org/mailman/listinfo/labs-l >> >> > > _______________________________________________ > Labs-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/labs-l > >
_______________________________________________ Labs-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/labs-l
