Thanks.Now, there is no sleep time between writes. It will reduce our deployment time to a large extent. I tried several hacks to remove sleep time without success. I didnt imagine such a simple fix would settle this. I believed i was completely familiar with pywikipedia framework but i guess there is still a lot to know. I noticed several threads to improve pywikipedia framework documentation.I strongly agree now. :)
Thanks, Chinmay On Wed, Aug 21, 2013 at 12:27 AM, Merlijn van Deen <[email protected]>wrote: > The sleep time as implemented in pywikipedia is influenced by three > factors: > > - the minimum time between requests, minthrottle=1 (by default) in > user-config.py > - the minimum time between page saves, put_throttle=10 > - the database lag on the DB servers - pages will not be saved unless the > lag is less than maxlag = 5 > > (all values in seconds). > > In the case of wikidata, the maximum number of saves/edits per minute is > 60 (iirc), so you could reduce put_throttle to 1 by adding > > put_throttle=1 > > in your user-config.py > > Best, > Merlijn > > > On 20 August 2013 20:16, Chinmay Naik <[email protected]> wrote: > >> Hi, >> I am Chinmay Naik, a Gsoc intern and operator of ProteinBoxBot(runs on >> pywikipedia - formerly rewrite branch). When i run the bot, there exists a >> sleep time of around 10 s (7s average) between two successive writes. >> The bot will handle around 40,000 wikidata items corresponding to gene >> wiki articles from http://en.wikipedia.org/wiki/Category:Human_proteins. >> Also , i will be uploading around 30 (claims + sources) for each >> wikidata item. >> >> If u take a look at the recent edits ( >> https://www.wikidata.org/w/index.php?title=Special:Contributions/ProteinBoxBot&offset=&limit=250&target=ProteinBoxBot), >> it takes around 5 mins to completely write all claims+sources to each >> wikidata item. This is a very large time lag and will hugely affect >> deployment. Is there any way to reduce this sleep time? Any pointers on >> this would be helpful. >> >> Thanks, >> Chinmay >> >> _______________________________________________ >> Pywikipedia-l mailing list >> [email protected] >> https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l >> >> > > _______________________________________________ > Pywikipedia-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l > >
_______________________________________________ Pywikipedia-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
