Re: [Openstack] profiling nova-api

2012-04-13 Thread Yuriy Taraday
Great work! About a year ago I found out than config was in the very top of profiling table. It is there again. That time there was a small misprint that led to constant config reloading. This time i tried to add caching since our config became so dynamic. My efforts can be found here: https://rev

Re: [Openstack] profiling nova-api

2012-04-12 Thread Yun Mao
Hi Jay, I will take a look later when I find a box with multiple cores to take advantage it.. Agreed - cprofiling is not so useful this case. This would be a purely performance benchmark. If implemented correctly, we should see a notable gain. However, It will probably not that linear because we

Re: [Openstack] profiling nova-api

2012-04-12 Thread Sean Dague
On 04/11/2012 04:48 PM, Yun Mao wrote: * Database access: Each "nova list" API call will issue 4 db APIs: 3 instance_get_all_by_filters(), 1 instance_fault_get_by_instance_uuids(), so 1200 db API calls total (note: not necessarily 1200 SQL statements, could be more). The 900 instance_get_all_by_

Re: [Openstack] profiling nova-api

2012-04-12 Thread Pitucha, Stanislaw Izaak
Just a small warning: since cProfile has its own overhead on entering/exiting every method and uses deterministic rather than statistical profiling, it may misrepresent tiny functions. They will take much more wall-clock time simply due to the profiling overhead * number of executions. I'm not s

Re: [Openstack] profiling nova-api

2012-04-12 Thread Jay Pipes
On 04/12/2012 10:39 AM, Justin Santa Barbara wrote: Both the profiling & the multi-process work are great - good stuff! Jay: Won't going multi-process just make it harder to profile? Yep, that's why I said "also note that your code profiling technique is unlikely to be effective since cProfil

Re: [Openstack] profiling nova-api

2012-04-12 Thread Justin Santa Barbara
Both the profiling & the multi-process work are great - good stuff! Jay: Won't going multi-process just make it harder to profile? I think it's actually a good thing to profile without the multi-process patch, find & fix and the bottlenecks in single-request performance, and then use multi-proces

Re: [Openstack] profiling nova-api

2012-04-12 Thread Jay Pipes
Hi Yun! Thanks very much for sharing this information. Can I ask you to pull the code in this branch: https://review.openstack.org/#change,5762 And retry your test? On my test machines, the work that Huang Zhiteng has done to convert Nova's servers to use multiple operating system processes

Re: [Openstack] profiling nova-api

2012-04-11 Thread Joshua Harlow
Awesome, great stuff to know! On 4/11/12 1:48 PM, "Yun Mao" wrote: Hi Stackers, I spent some time looking at nova-api today. Setup: everything-on-one-node devstack, essex trunk. I setup 1 user with 10 tiny VMs. Client: 3 python threads each doing a loop of "nova list" equivalent for 100 times.