Calling all Proflers! I am running into an issue with CPU usage on remote nova-conductor and I am trying to profile it to see where its consuming the most amount of cpu, so that we can investigate further. The etherpad where we have been working on this issue is located at: https://etherpad.openstack.org/p/remote-conductor-performance
I tired running nova conductor under Cprofiler with 20 workers, however it only saw the main thread. So I started conducotr with a single worker and cProfiler was able to see more requests. I ran the following: * python -m cProfile -o conductor-1worker /usr/bin/nova-conductor * ./gprof2dot/gprof2dot.py -f pstats conductor-1worker | dot -Tsvg -o conductor-1worker.svg * SVG output here: http://tempsend.com/5340867576/F283/conductor1worker.svg The SVG showed that most of the time was spent in multiple different locations but AMQP seemed to be the highest consume of time. However, I am reading that cProfiler can get confused under eventlet so I am wondering if the cProfiler can be trusted. I tired using greenletprofiler however, that started to causes errors to be thrown in the conductor logs around connecting to rabbitmq. So what I am asking is those of you who have experience debugging python with eventlet, what tooling do you use? ___________________________________________________________________ Kris Lindgren Senior Linux Systems Engineer GoDaddy
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
