Re: [Openstack-operators] [nova] Couple of CellsV2 questions

2018-07-24 Thread Jonathan Mills
Thanks, Matt.  Those are all good suggestions, and we will incorporate
your feedback into our plans.

On 07/23/2018 05:57 PM, Matt Riedemann wrote:
> I'll try to help a bit inline. Also cross-posting to openstack-dev and
> tagging with [nova] to highlight it.
> 
> On 7/23/2018 10:43 AM, Jonathan Mills wrote:
>> I am looking at implementing CellsV2 with multiple cells, and there's
>> a few things I'm seeking clarification on:
>>
>> 1) How does a superconductor know that it is a superconductor?  Is its
>> operation different in any fundamental way?  Is there any explicit
>> configuration or a setting in the database required? Or does it simply
>> not care one way or another?
> 
> It's a topology term, not really anything in config or the database that
> distinguishes the "super" conductor. I assume you've gone over the
> service layout in the docs:
> 
> https://docs.openstack.org/nova/latest/user/cellsv2-layout.html#service-layout
> 
> 
> There are also some summit talks from Dan about the topology linked here:
> 
> https://docs.openstack.org/nova/latest/user/cells.html#cells-v2
> 
> The superconductor is the conductor service at the "top" of the tree
> which interacts with the API and scheduler (controller) services and
> routes operations to the cell. Then once in a cell, the operation should
> ideally be confined there. So, for example, reschedules during a build
> would be confined to the cell. The cell conductor doesn't go back "up"
> to the scheduler to get a new set of hosts for scheduling. This of
> course depends on which release you're using and your configuration, see
> the caveats section in the cellsv2-layout doc.
> 
>>
>> 2) When I ran the command "nova-manage cell_v2 create_cell
>> --name=cell1 --verbose", the entry created for cell1 in the api
>> database includes only one rabbitmq server, but I have three of them
>> as an HA cluster.  Does it only support talking to one rabbitmq server
>> in this configuration? Or can I just update the cell1 transport_url in
>> the database to point to all three? Is that a supported configuration?
> 
> First, don't update stuff directly in the database if you don't have to.
> :) What you set on the transport_url should be whatever oslo.messaging
> can handle:
> 
> https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.transport_url
> 
> 
> There is at least one reported bug for this but I'm not sure I fully
> grok it or what its status is at this point:
> 
> https://bugs.launchpad.net/nova/+bug/1717915
> 
>>
>> 3) Is there anything wrong with having one cell share the amqp bus
>> with your control plane, while having additional cells use their own
>> amqp buses? Certainly I realize that the point of CellsV2 is to shard
>> the amqp bus for greater horizontal scalability.  But in my case, my
>> first cell is on the smaller side, and happens to be colocated with
>> the control plane hardware (whereas other cells will be in other parts
>> of the datacenter, or in other datacenters with high-speed links).  I
>> was thinking of just pointing that first cell back at the same
>> rabbitmq servers used by the control plane, but perhaps directing them
>> at their own rabbitmq vhost. Is that a terrible idea?
> 
> Would need to get input from operators and/or Dan Smith's opinion on
> this one, but I'd say it's no worse than having a flat single cell
> deployment. However, if you're going to do multi-cell long-term anyway,
> then it would be best to get in the mindset and discipline of not
> relying on shared MQ between the controller services and the cells. In
> other words, just do the right thing from the start rather than have to
> worry about maybe changing the deployment / configuration for that one
> cell down the road when it's harder.
> 

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


Re: [Openstack-operators] [nova] Couple of CellsV2 questions

2018-07-23 Thread Matt Riedemann
I'll try to help a bit inline. Also cross-posting to openstack-dev and 
tagging with [nova] to highlight it.


On 7/23/2018 10:43 AM, Jonathan Mills wrote:
I am looking at implementing CellsV2 with multiple cells, and there's a 
few things I'm seeking clarification on:


1) How does a superconductor know that it is a superconductor?  Is its 
operation different in any fundamental way?  Is there any explicit 
configuration or a setting in the database required? Or does it simply 
not care one way or another?


It's a topology term, not really anything in config or the database that 
distinguishes the "super" conductor. I assume you've gone over the 
service layout in the docs:


https://docs.openstack.org/nova/latest/user/cellsv2-layout.html#service-layout

There are also some summit talks from Dan about the topology linked here:

https://docs.openstack.org/nova/latest/user/cells.html#cells-v2

The superconductor is the conductor service at the "top" of the tree 
which interacts with the API and scheduler (controller) services and 
routes operations to the cell. Then once in a cell, the operation should 
ideally be confined there. So, for example, reschedules during a build 
would be confined to the cell. The cell conductor doesn't go back "up" 
to the scheduler to get a new set of hosts for scheduling. This of 
course depends on which release you're using and your configuration, see 
the caveats section in the cellsv2-layout doc.




2) When I ran the command "nova-manage cell_v2 create_cell --name=cell1 
--verbose", the entry created for cell1 in the api database includes 
only one rabbitmq server, but I have three of them as an HA cluster.  
Does it only support talking to one rabbitmq server in this 
configuration? Or can I just update the cell1 transport_url in the 
database to point to all three? Is that a supported configuration?


First, don't update stuff directly in the database if you don't have to. 
:) What you set on the transport_url should be whatever oslo.messaging 
can handle:


https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.transport_url

There is at least one reported bug for this but I'm not sure I fully 
grok it or what its status is at this point:


https://bugs.launchpad.net/nova/+bug/1717915



3) Is there anything wrong with having one cell share the amqp bus with 
your control plane, while having additional cells use their own amqp 
buses? Certainly I realize that the point of CellsV2 is to shard the 
amqp bus for greater horizontal scalability.  But in my case, my first 
cell is on the smaller side, and happens to be colocated with the 
control plane hardware (whereas other cells will be in other parts of 
the datacenter, or in other datacenters with high-speed links).  I was 
thinking of just pointing that first cell back at the same rabbitmq 
servers used by the control plane, but perhaps directing them at their 
own rabbitmq vhost. Is that a terrible idea?


Would need to get input from operators and/or Dan Smith's opinion on 
this one, but I'd say it's no worse than having a flat single cell 
deployment. However, if you're going to do multi-cell long-term anyway, 
then it would be best to get in the mindset and discipline of not 
relying on shared MQ between the controller services and the cells. In 
other words, just do the right thing from the start rather than have to 
worry about maybe changing the deployment / configuration for that one 
cell down the road when it's harder.


--

Thanks,

Matt

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators