[openstack-dev] [Nova] Launching multiple VMs in Nova

2014-10-02 Thread Oleg Bondarev
Hi,

It turns out that there is a 1:1 relationship between rpc_thread_pool_size
messaging config [1] and the number of instances that can be spawned
simultaneously.
Please see bug [2] for more details.
I think this should be at least documented. Thoughts?

Thanks,
Oleg

[1]
https://github.com/openstack/oslo.messaging/blob/master/oslo/messaging/_executors/impl_eventlet.py
[2] https://bugs.launchpad.net/neutron/+bug/1372049
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] Launching multiple VMs in Nova

2014-10-02 Thread Chris Friesen

On 10/02/2014 08:10 AM, Oleg Bondarev wrote:

Hi,

It turns out that there is a 1:1 relationship between
rpc_thread_pool_size messaging config [1] and the number of instances
that can be spawned simultaneously.
Please see bug [2] for more details.
I think this should be at least documented. Thoughts?

Thanks,
Oleg

[1]
https://github.com/openstack/oslo.messaging/blob/master/oslo/messaging/_executors/impl_eventlet.py
[2] https://bugs.launchpad.net/neutron/+bug/1372049


Seems like the fix would be to allow the oslo.messaging thread pool to 
grow as needed.


If we don't fix it, then yes this should probably be documented somewhere.

I'm guessing there are other places in nova where we might get bit by 
the same scenario if the timing is just right.


Chris


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] Launching multiple VMs in Nova

2014-10-02 Thread Jay Pipes

On 10/02/2014 11:23 AM, Chris Friesen wrote:

On 10/02/2014 08:10 AM, Oleg Bondarev wrote:

Hi,

It turns out that there is a 1:1 relationship between
rpc_thread_pool_size messaging config [1] and the number of instances
that can be spawned simultaneously.
Please see bug [2] for more details.
I think this should be at least documented. Thoughts?

Thanks,
Oleg

[1]
https://github.com/openstack/oslo.messaging/blob/master/oslo/messaging/_executors/impl_eventlet.py

[2] https://bugs.launchpad.net/neutron/+bug/1372049


Seems like the fix would be to allow the oslo.messaging thread pool to
grow as needed.

If we don't fix it, then yes this should probably be documented somewhere.

I'm guessing there are other places in nova where we might get bit by
the same scenario if the timing is just right.


Note that this is *per compute node*. So, yes, the rpc_thread_pool_size 
directly limits the number of instances that can be spawned 
simultaneously on a compute node, but it's important to point out that 
this isn't across all of your Nova deployment, but just per compute 
node. If you have 10 compute nodes, you could theoretically spawn ~640 
instances simultaneously given the default configuration settings. 
However, at that point, you will likely run into other sources of 
contention in the nova-scheduler and nova-conductor communication. :)


So, bottom line, Oleg, yes, it should be documented. And just make sure 
the documentation is clear that it refers to the number of instances 
spawned at once on each compute node.


Best,
-jay

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev