That's great to know, thank you. We'll make sure we only use the API within that limit - basically just for individual calls when a user adds a new event to our database.
For the bulk processing, we would need to update the backlinks information as a monthly maintenance task, so I wouldn't want to trouble you with this each time. Would you rather we stick with data dump processing for the large scale stuff? On 9 Sep 2014 15:05, "John" <[email protected]> wrote: > If you want a report on that many pages drop me a list of those titles and > and I can write a report for you given that volume of affected pages. > > I would say 1-2 seconds between quires should be reasonable for a moderate > volume of quires. Any large scale request I will do server side and avoid > hammering the web-servers for something that is better batched. > > > On Tue, Sep 9, 2014 at 9:58 AM, Navino Evans <[email protected]> > wrote: > >> Once again, a huge thank you for taking the time to do this John - That's >> exactly what I was looking for! - the helpfulness of this community never >> ceases to amaze me :) >> >> Hopefully I haven't initiated a journey down the rabbit hole into a fully >> fledged muliti-language counting machine ;) >> >> >> Can I just ask what the limit of reasonable use would be for making API >> calls to this new tool? (e.g. number of calls per day) >> >> It would be incredibly useful if we could use it to update the events in >> our database once a month (we are using it to rank historical events by >> 'importance'), but we are already have approximately 1.5 million events so >> am aware this may be way beyond what would be acceptable. >> >> On Tue, Sep 9, 2014 at 2:56 PM, John <[email protected]> wrote: >> >>> That's doable, however it will require a little more time as I need to >>> unearth some old code to handle multi-projects/languages >>> >>> >>> On Tue, Sep 9, 2014 at 9:51 AM, Jan Ainali <[email protected]> >>> wrote: >>> >>>> Awesome John! >>>> >>>> Now I only wish that one could specify language code also ;) >>>> >>>> >>>> *Med vänliga hälsningar,Jan Ainali* >>>> >>>> Verksamhetschef, Wikimedia Sverige >>>> <http://se.wikimedia.org/wiki/Huvudsida> >>>> 0729 - 67 29 48 >>>> >>>> >>>> *Tänk dig en värld där varje människa har fri tillgång till >>>> mänsklighetens samlade kunskap. Det är det vi gör.* >>>> Bli medlem. <http://blimedlem.wikimedia.se> >>>> >>>> >>>> 2014-09-09 15:34 GMT+02:00 John <[email protected]>: >>>> >>>>> Per request, its no frills but what you what you asked for: >>>>> http://tools.wmflabs.org/betacommand-dev/cgi-bin/backlinks >>>>> >>>>> >>>>> On Tue, Sep 9, 2014 at 8:32 AM, Navino Evans <[email protected]> >>>>> wrote: >>>>> >>>>>> That is fantastic news... I'm incredibly grateful for the help and >>>>>> advice. >>>>>> >>>>>> On Tue, Sep 9, 2014 at 1:27 PM, John <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Given the overhead of the API and that he only needs a count getting >>>>>>> that info should be fairly easy via a python cgi wrapper around an sql >>>>>>> query. >>>>>>> >>>>>>> The only thing that I cannot do is #3 since the software does not >>>>>>> differentiate between links in templates and links not in templates. >>>>>>> Its a >>>>>>> requested feature for years now. >>>>>>> >>>>>>> Give me a few hours and ill get you the tool you want. This should >>>>>>> be less than 30 minutes work >>>>>>> >>>>>>> >>>>>>> On Tue, Sep 9, 2014 at 7:55 AM, Jan Ainali <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Related tip: In the API you can get a list of backlinks (but you >>>>>>>> have to count them yourself) from the main namespace including all >>>>>>>> redirects by a query like this: >>>>>>>> >>>>>>>> >>>>>>>> https://en.wikipedia.org/w/api.php?action=query&list=backlinks&format=json&bltitle=Example&blnamespace=0&blfilterredir=all&bllimit=250&blredirect= >>>>>>>> >>>>>>>> More info at: https://www.mediawiki.org/wiki/API:Backlinks >>>>>>>> >>>>>>>> >>>>>>>> *Med vänliga hälsningar,Jan Ainali* >>>>>>>> >>>>>>>> Verksamhetschef, Wikimedia Sverige >>>>>>>> <http://se.wikimedia.org/wiki/Huvudsida> >>>>>>>> 0729 - 67 29 48 >>>>>>>> >>>>>>>> >>>>>>>> *Tänk dig en värld där varje människa har fri tillgång till >>>>>>>> mänsklighetens samlade kunskap. Det är det vi gör.* >>>>>>>> Bli medlem. <http://blimedlem.wikimedia.se> >>>>>>>> >>>>>>>> >>>>>>>> 2014-09-09 13:41 GMT+02:00 Navino Evans <[email protected]>: >>>>>>>> >>>>>>>>> Wow! That would be awesome :) >>>>>>>>> >>>>>>>>> The API we are looking for can be as simple as sending a GET >>>>>>>>> request to a url ( >>>>>>>>> http://www.somewhere.com/api/count?t=wikipedia_title_goes_here), >>>>>>>>> returning a number in "text/plain" format. >>>>>>>>> >>>>>>>>> The actual count that we're interested is for English Wikipedia >>>>>>>>> only, and would ideally include the following, all added up into a >>>>>>>>> single >>>>>>>>> number: >>>>>>>>> >>>>>>>>> 1) All links from articles in Main Namespace only (for our >>>>>>>>> purpose it would be better to not include links from User pages, Talk >>>>>>>>> pages >>>>>>>>> etc if possible) >>>>>>>>> >>>>>>>>> 2) Including links from Redirect pages (e.g. counting a link from >>>>>>>>> "Michel Jackson" redirect as part of the count from the article >>>>>>>>> "Michael >>>>>>>>> Jackson") >>>>>>>>> >>>>>>>>> 3) Excluding links that are within a template transcluded in an >>>>>>>>> article (so we don't need to count the links inside Navboxes within an >>>>>>>>> article for example) >>>>>>>>> >>>>>>>>> 4) For our purpose, it doesn't really matter whether transclusions >>>>>>>>> of the actual page that is called are included in the count (we >>>>>>>>> generally >>>>>>>>> won't be using it for checking templates, timeline and list articles). >>>>>>>>> >>>>>>>>> Just to give the full picture for this request - my use of this >>>>>>>>> tool will be for a company (www.histropedia.com), so I wouldn't >>>>>>>>> want to take up your time with this unless it's something you feel >>>>>>>>> should >>>>>>>>> be available for wider use. My plan was to get the developer working >>>>>>>>> on our >>>>>>>>> site to make this tool for the community if it didn't exist >>>>>>>>> somewhere, but >>>>>>>>> we would be reliant on datadumps so could not get live information >>>>>>>>> (which >>>>>>>>> would be incredibly useful for us, and I hope many others). >>>>>>>>> >>>>>>>>> >>>>>>>>> On Mon, Sep 8, 2014 at 8:10 PM, John <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> What numbers/data do you want? I can whip up a replacement for it. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Monday, September 8, 2014, Navino Evans < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi All, >>>>>>>>>>> >>>>>>>>>>> Hi all, does anyone know if there is a tool currently available >>>>>>>>>>> for counting backlinks to Wikipedia articles via an API? I have >>>>>>>>>>> been using >>>>>>>>>>> this tool >>>>>>>>>>> http://dispenser.homenet.org/~dispenser/cgi-bin/backlinkscount.py >>>>>>>>>>> - but it seems to have finally gone offline completely following >>>>>>>>>>> some >>>>>>>>>>> recent controversy with user:Dispenser. >>>>>>>>>>> >>>>>>>>>>> Any advice much appreciated! >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> >>>>>>>>>>> Navino >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Labs-l mailing list >>>>>>>>>> [email protected] >>>>>>>>>> https://lists.wikimedia.org/mailman/listinfo/labs-l >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> ___________________________ >>>>>>>>> >>>>>>>>> Histropedia >>>>>>>>> The Timeline for all of History >>>>>>>>> www.histropedia.com >>>>>>>>> >>>>>>>>> Follow us on: >>>>>>>>> Twitter <https://twitter.com/Histropedia> Facebo >>>>>>>>> <https://www.facebook.com/Histropedia>ok >>>>>>>>> <https://www.facebook.com/Histropedia> Google + >>>>>>>>> <https://plus.google.com/u/0/b/104484373317792180682/104484373317792180682/posts> >>>>>>>>> L <http://www.linkedin.com/company/histropedia-ltd>inke >>>>>>>>> <http://www.linkedin.com/company/histropedia-ltd>dIn >>>>>>>>> <http://www.linkedin.com/company/histropedia-ltd> >>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Labs-l mailing list >>>>>>>>> [email protected] >>>>>>>>> https://lists.wikimedia.org/mailman/listinfo/labs-l >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Labs-l mailing list >>>>>>>> [email protected] >>>>>>>> https://lists.wikimedia.org/mailman/listinfo/labs-l >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Labs-l mailing list >>>>>>> [email protected] >>>>>>> https://lists.wikimedia.org/mailman/listinfo/labs-l >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> ___________________________ >>>>>> >>>>>> Histropedia >>>>>> The Timeline for all of History >>>>>> www.histropedia.com >>>>>> >>>>>> Follow us on: >>>>>> Twitter <https://twitter.com/Histropedia> Facebo >>>>>> <https://www.facebook.com/Histropedia>ok >>>>>> <https://www.facebook.com/Histropedia> Google + >>>>>> <https://plus.google.com/u/0/b/104484373317792180682/104484373317792180682/posts> >>>>>> L <http://www.linkedin.com/company/histropedia-ltd>inke >>>>>> <http://www.linkedin.com/company/histropedia-ltd>dIn >>>>>> <http://www.linkedin.com/company/histropedia-ltd> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Labs-l mailing list >>>>>> [email protected] >>>>>> https://lists.wikimedia.org/mailman/listinfo/labs-l >>>>>> >>>>>> >>>>> >>>>> _______________________________________________ >>>>> Labs-l mailing list >>>>> [email protected] >>>>> https://lists.wikimedia.org/mailman/listinfo/labs-l >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> Labs-l mailing list >>>> [email protected] >>>> https://lists.wikimedia.org/mailman/listinfo/labs-l >>>> >>>> >>> >>> _______________________________________________ >>> Labs-l mailing list >>> [email protected] >>> https://lists.wikimedia.org/mailman/listinfo/labs-l >>> >>> >> >> >> -- >> ___________________________ >> >> Histropedia >> The Timeline for all of History >> www.histropedia.com >> >> Follow us on: >> Twitter <https://twitter.com/Histropedia> Facebo >> <https://www.facebook.com/Histropedia>ok >> <https://www.facebook.com/Histropedia> Google + >> <https://plus.google.com/u/0/b/104484373317792180682/104484373317792180682/posts> >> L <http://www.linkedin.com/company/histropedia-ltd>inke >> <http://www.linkedin.com/company/histropedia-ltd>dIn >> <http://www.linkedin.com/company/histropedia-ltd> >> >> >> _______________________________________________ >> Labs-l mailing list >> [email protected] >> https://lists.wikimedia.org/mailman/listinfo/labs-l >> >> > > _______________________________________________ > Labs-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/labs-l > >
_______________________________________________ Labs-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/labs-l
