I am starting to think it may be due to my own custom scheduler. If I uploaded the code somewhere could anybody take a look? it's a 150 lines py script.
On Wed, Aug 10, 2016 at 12:10 PM, Francesco Di Mizio < [email protected]> wrote: > Thanks for that Pierre. I have it on on production and the performances do > not degrade indeed. > > buildbot@e2123a50d1fb:~$ cat profile.txt > No samples recorded. > > I need to study statprof a bit now ;) > > On Wed, Aug 10, 2016 at 11:20 AM, Vasily <[email protected]> wrote: > >> One can also use Intel® VTune™, you know... :-) >> >> Thanks, >> Vasily >> 09 авг. 2016 г. 23:56 пользователь "Pierre Tardy" <[email protected]> >> написал: >> >>> You can add following snippet to your master.cfg. It looks statprof does >>> not work on osx, so I cannot test it to the end, but I know similar did >>> work on my prod >>> >>> >>> import statprof >>> from twisted.application.internet import TimerService >>> from buildbot.util.service import BuildbotService >>> class BuildbotTimerService(TimerService, BuildbotService): >>> name = "timer" >>> def __init__(self): >>> BuildbotService.__init__(self) >>> TimerService.__init__(self, 10, self.dump_stats) >>> def dump_stats(self): >>> statprof.stop() >>> with open("profile.txt", "w") as f: >>> statprof.display(f) >>> statprof.start() >>> >>> c['services'] = [ BuildbotTimerService() ] >>> >>> >>> Le mar. 9 août 2016 à 21:44, Francesco Di Mizio < >>> [email protected]> a écrit : >>> >>>> Pierre, >>>> >>>> if you can enlighten on how you intend to use statprof, I will be happy >>>> to give it a try. >>>> This problem here is on top of my list right now as it's making my prod >>>> env unusable. >>>> >>>> On Tue, Aug 9, 2016 at 8:25 PM, Pierre Tardy <[email protected]> wrote: >>>> >>>>> Please note that cprofile is very intrusive and will lead to about 3x >>>>> slower code. >>>>> >>>>> So I would not run that in prod. This is why I prefer statprof which >>>>> is using a non intrusive method to get the stats >>>>> >>>>> Le mar. 9 août 2016 19:15, Francesco Di Mizio < >>>>> [email protected]> a écrit : >>>>> >>>>>> On a side note I have managed to get twistd to dump a blob I can then >>>>>> load and analyze. Got to see if I can run this in production. >>>>>> If anybody is curious the following worked for me: >>>>>> twistd --savestats -n --profiler=cprofile --profile=/crcdata/ >>>>>> profile.stats -y ./buildbot.tac >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Aug 9, 2016 at 5:21 PM, Francesco Di Mizio < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Well that's expected to an extent - p4 poller is running 'p4 >>>>>>> changes' every pollinterval seconds. >>>>>>> >>>>>>> Anyway just as a test I've tried to disable the poller and still at >>>>>>> times I am seeing the CPU spiking up to above 100%. I believe I really >>>>>>> need >>>>>>> to profile this somehow. >>>>>>> >>>>>>> On Tue, Aug 9, 2016 at 5:08 PM, Dan Kegel <[email protected]> wrote: >>>>>>> >>>>>>>> Also watch 'top' and see if poller processes are hogging resources. >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>> >>> _______________________________________________ >>> users mailing list >>> [email protected] >>> https://lists.buildbot.net/mailman/listinfo/users >>> >> >
_______________________________________________ users mailing list [email protected] https://lists.buildbot.net/mailman/listinfo/users
