The ram looks like sufficient, it might be a good test to try and
increase the number of cpu for that VM.
In your trace, I can see the use of up to 7 threads at the same
time, so you might gain by going 8 CPUs
Also, make sure that your VM host is not overbooked. In my
experience of using VMware VMs provided by IT, overbooking has
been a source of inexplicable performance issues.
12k lines is a lot, but buildbot shall support this kind of load
without issue.
Le ven. 26 août 2016 à 12:05, Francesco Di Mizio
<[email protected] <mailto:[email protected]>>
a écrit :
It's a vmware virtual machine with 4 GIGs RAM and 4
CPUs@3Ghz. It runs, among other marginal things, 2 docker
containers - one for the buildbot and one for the postgres db.
The most beefy logs have around 12K lines. Is it too much?
Also some other logs are read from the worker's filesystem
and added as additional logs.
On Fri, Aug 26, 2016 at 11:51 AM, Pierre Tardy
<[email protected] <mailto:[email protected]>> wrote:
Cool!
I can indeed see 3 spikes.
Looks related to logs and logs compression.
What is the HW spec of your master machine?
How much log does your build generate?
Pierre
Le ven. 26 août 2016 à 11:42, Francesco Di Mizio
<[email protected]
<mailto:[email protected]>> a écrit :
Pierre,
I enabled it, waited 1 min and saw the spike, then
stopped after a few secs. Attached the json.
Awesome tool btw, work wonders!
On Thu, Aug 25, 2016 at 1:14 PM, Pierre Tardy
<[email protected] <mailto:[email protected]>> wrote:
You can try to hit the button in the morning with
a two hours gather period, and hope that you see
the spike during that period..
Le jeu. 25 août 2016 à 12:17, Francesco Di Mizio
<[email protected]
<mailto:[email protected]>> a écrit :
Thanks a lot! Pierre I will def will give it
a shot. I am not sure I'll be able to smash
that 'start recording' button as the UI
isusuallystuck when the CPU spikes. Updates
to come!
On Thu, Aug 25, 2016 at 10:45 AM, Pierre
Tardy <[email protected]
<mailto:[email protected]>> wrote:
Hi Francesco,
I spent some time in order to implement a
profiler plugin for buildbot
You can give it a look, and send your
profile.json file if you need more
analysis from me.
https://github.com/tardyp/buildbot_profiler
Regards,
Pierre
Le mer. 24 août 2016 à 22:43, Francesco
Di Mizio <[email protected]
<mailto:[email protected]>> a
écrit :
I've tried and it's not an easy task
because of my Win into Vagrant into
Docker setup.
I'll try again soon when I get a
Linux box!
On Fri, Aug 19, 2016 at 5:54 PM,
Vasily <[email protected]
<mailto:[email protected]>> wrote:
Umm, no. VTune has Python support
starting 2017 Beta, and, well, it
was my team (at Intel) work
actually :-)
P.S. I'm from Intel, too.
Thanks,
Vasily
19 авг. 2016 г. 18:17
пользователь "Francesco Di Mizio"
<[email protected]
<mailto:[email protected]>>
написал:
I had thought you were making
fun of Intel somehow ;)
On Aug 19, 2016 5:07 PM,
"Pierre Tardy"
<[email protected]
<mailto:[email protected]>> wrote:
ahah
I though this was a taunt
on me being employed by
Intel.
I actually had mitigated
experience with vtune few
years ago, and didn't
know they had python
support until then.
Being an opensource guy,
I usually neglegate to
look at proprietary stuff.
Pierre
Le ven. 19 août 2016
à 12:18, Vasily
<[email protected]
<mailto:[email protected]>>
a écrit :
I'm again suggesting
to look into Python
profiling
capabilities of
Intel® VTune™
Amplifier. It could
run statistical
profiling for a long
time and display CPU
usage over time, so
the developer can
look at specific time
range where CPU usage
was too high and see
which functions were
executed.
Thanks,
Vasily
19 авг. 2016 г. 11:57
пользователь "Pierre
Tardy"
<[email protected]
<mailto:[email protected]>>
написал:
Hi Francesco,
Your described
setup looks sane
to me.
The problems we
are trying to
catch are cpu
spikes, as far as
I understand,
which does not
happen for very
long, but are
very annoying for
users, as it is
blocking the reactor.
This problem is
not easy to see
in the profile
you sent, as this
profile is over
long time, so we
see the average
of each method
during the day
and not the spikes.
What would really
be needed is a
on-demand
profiler which
would detect cpu
spikes and only
log the stack
traces during
those times.
Here is a nice
blog pst
explaining why
statistic
profiling is cool
and easy to
implement in python.
https://nylas.com/blog/performance
For 0.9.1 I want
to concentrate on
scalability, and
write a debugging
ui plugin based
on those ideas
(and probably code)
That would be
great if your
team can help on
that matter.
Regards,
Pierre
_______________________________________________
users mailing list
[email protected] <mailto:[email protected]>
https://lists.buildbot.net/mailman/listinfo/users
