Thanks for that Pierre. I have it on on production and the performances do not degrade indeed.
buildbot@e2123a50d1fb:~$ cat profile.txt No samples recorded. I need to study statprof a bit now ;) On Wed, Aug 10, 2016 at 11:20 AM, Vasily <[email protected]> wrote: > One can also use Intel® VTune™, you know... :-) > > Thanks, > Vasily > 09 авг. 2016 г. 23:56 пользователь "Pierre Tardy" <[email protected]> > написал: > >> You can add following snippet to your master.cfg. It looks statprof does >> not work on osx, so I cannot test it to the end, but I know similar did >> work on my prod >> >> >> import statprof >> from twisted.application.internet import TimerService >> from buildbot.util.service import BuildbotService >> class BuildbotTimerService(TimerService, BuildbotService): >> name = "timer" >> def __init__(self): >> BuildbotService.__init__(self) >> TimerService.__init__(self, 10, self.dump_stats) >> def dump_stats(self): >> statprof.stop() >> with open("profile.txt", "w") as f: >> statprof.display(f) >> statprof.start() >> >> c['services'] = [ BuildbotTimerService() ] >> >> >> Le mar. 9 août 2016 à 21:44, Francesco Di Mizio < >> [email protected]> a écrit : >> >>> Pierre, >>> >>> if you can enlighten on how you intend to use statprof, I will be happy >>> to give it a try. >>> This problem here is on top of my list right now as it's making my prod >>> env unusable. >>> >>> On Tue, Aug 9, 2016 at 8:25 PM, Pierre Tardy <[email protected]> wrote: >>> >>>> Please note that cprofile is very intrusive and will lead to about 3x >>>> slower code. >>>> >>>> So I would not run that in prod. This is why I prefer statprof which is >>>> using a non intrusive method to get the stats >>>> >>>> Le mar. 9 août 2016 19:15, Francesco Di Mizio < >>>> [email protected]> a écrit : >>>> >>>>> On a side note I have managed to get twistd to dump a blob I can then >>>>> load and analyze. Got to see if I can run this in production. >>>>> If anybody is curious the following worked for me: >>>>> twistd --savestats -n --profiler=cprofile --profile=/crcdata/ >>>>> profile.stats -y ./buildbot.tac >>>>> >>>>> >>>>> >>>>> >>>>> On Tue, Aug 9, 2016 at 5:21 PM, Francesco Di Mizio < >>>>> [email protected]> wrote: >>>>> >>>>>> Well that's expected to an extent - p4 poller is running 'p4 changes' >>>>>> every pollinterval seconds. >>>>>> >>>>>> Anyway just as a test I've tried to disable the poller and still at >>>>>> times I am seeing the CPU spiking up to above 100%. I believe I really >>>>>> need >>>>>> to profile this somehow. >>>>>> >>>>>> On Tue, Aug 9, 2016 at 5:08 PM, Dan Kegel <[email protected]> wrote: >>>>>> >>>>>>> Also watch 'top' and see if poller processes are hogging resources. >>>>>>> >>>>>> >>>>>> >>>>> >>> >> _______________________________________________ >> users mailing list >> [email protected] >> https://lists.buildbot.net/mailman/listinfo/users >> >
_______________________________________________ users mailing list [email protected] https://lists.buildbot.net/mailman/listinfo/users
