Re: [openstack-dev] [nova] Pike PTG recap - cells

[Balazs Gibizer]

On Tue, Feb 28, 2017 at 3:10 AM, Zhenyu Zheng 
 wrote:

Matt,

Thanks for the recap, it's a pity I cannot attend PTG due to personal 
reason, I will be willing to take the work you mentioned, and will 
check the details with you.


And another thing, I don't know whether you guys discussed or not, I 
saw in [1] we are talking about adding tags field(and others of 
cause) to the instance notification
payload, to be able to send during instance.create and 
instance.update. The fact is that currently we cannot add tags during 
boot, nor we send notifications when we
add/update/delete tags latter(it is a direct DB change and no 
instance.update notification has been sent out), so for tags field, 
we have to wait for another instance.update
action to get the latest info. And I have already tried to working on 
the boot thing[2] and already planned to work on the tag notification 
thing[3].


So, are there any plans about those? Maybe it is OK to send out 
instance.update notification in tags actions once [1] got merged?


Hi,

The [1] was shortly discussed and agreed that it is high priority for 
Pike. The basic things in that bp have code up for review [4] so we 
have a good chance to merge it in early Pike. The BDM part of [1] needs 
some data modeling work still.


I did an initial review round on [3] and left some question in the spec.

[4] 
https://review.openstack.org/#/q/project:openstack/nova+topic:bp/additional-notification-fields-for-searchlight


Cheers,
gibi



Thanks,

Kevin Zheng

[1] 
https://blueprints.launchpad.net/nova/+spec/additional-notification-fields-for-searchlight
[2] 
https://blueprints.launchpad.net/nova/+spec/support-tag-instance-when-boot

[3] https://blueprints.launchpad.net/nova/+spec/send-tag-notification


On Tue, Feb 28, 2017 at 6:33 AM, Matt Riedemann  
wrote:
We talked about cells on Wednesday morning at the PTG. The full 
etherpad is here [1].


Searchlight integration
---

We talked a bit about what needs to happen for this, and it starts 
with getting the data into searchlight so that it can serve the REST 
API, which is being worked in this blueprint [2]. We want to get 
that done early in Pike.


We plan on making the use of Searchlight configurable in Nova since 
at first you might not even have anything in it, so listing 
instances wouldn't work. We're also going to attempt to merge-sort 
when listing instances across multiple cells but it's going to be a 
known issue that it will be slow.


For testing Nova with Searchlight, we need to start by enabling the 
Searchlight devstack plugin in the nova-next CI job, which I'll work 
on.


I'm going to talk to Kevin Zheng about seeing if he can spend some 
time on getting Nova to use Searchlight if it's (1) configured for 
use and (2) is available (the endpoint is in the service catalog). 
Kevin is a Searchlight core and familiar with the Nova API code, so 
he's a good candidate for working on this (assuming he's available 
and willing to own it).


Cells-aware gaps in the API
---

Dan Smith has started a blueprint [3] for closing gaps in the API 
which break in a multi-cell deployment. He has a test patch [4] to 
expose the failures and then they can be worked on individually. The 
pattern of the work is in [5]. Help is welcome here, so please 
attend the weekly cells meeting [6] if you want to help out.


Auto-discovery of compute hosts
---

The "discover_hosts_in_cells_interval" config option was introduced 
in Ocata which controls a periodic task in the scheduler to discover 
new unmapped compute hosts but it's not very efficient since it 
queries all cell mappings and then all compute nodes in each cell 
mapping and checks to see if those compute nodes are yet mapped to 
the cell in the nova_api database. Dan Smith has a series of changes 
[7] which should make that discovery process more efficient, it just 
needs to be cleaned up a bit.


Service arrangement
---

Dan Smith is working on a series of changes in both Nova and 
devstack for testing with multiple cells [8]. The general idea is 
that there will still be two nodes and two nova-compute services. 
There will be three nova-conductor services, one per cell, and then 
another top-level "super conductor" which is there for building 
instances and sending the server create down to one of the cells. 
All three conductors are going to be running in the subnode just to 
balance the resources a bit otherwise the primary node is going to 
be starved. The multi-cell job won't be running migration tests 
since we don't currently support instance move operations between 
cells. We're going to work a hack into the scheduler to restrict a 
move operation to the same cell the instance is already in. This 
means the live migration job will still be a single-cell setup where 
both nova-computes are in the same cell.


Getting rid of 

Re: [openstack-dev] [nova] Pike PTG recap - cells

[Zhenyu Zheng]

Matt,

Thanks for the recap, it's a pity I cannot attend PTG due to personal
reason, I will be willing to take the work you mentioned, and will check
the details with you.

And another thing, I don't know whether you guys discussed or not, I saw in
[1] we are talking about adding tags field(and others of cause) to the
instance notification
payload, to be able to send during instance.create and instance.update. The
fact is that currently we cannot add tags during boot, nor we send
notifications when we
add/update/delete tags latter(it is a direct DB change and no
instance.update notification has been sent out), so for tags field, we have
to wait for another instance.update
action to get the latest info. And I have already tried to working on the
boot thing[2] and already planned to work on the tag notification thing[3].

So, are there any plans about those? Maybe it is OK to send out
instance.update notification in tags actions once [1] got merged?

Thanks,

Kevin Zheng

[1] https://blueprints.launchpad.net/nova/+spec/additional-notif
ication-fields-for-searchlight
[2]
https://blueprints.launchpad.net/nova/+spec/support-tag-instance-when-boot
[3] https://blueprints.launchpad.net/nova/+spec/send-tag-notification


On Tue, Feb 28, 2017 at 6:33 AM, Matt Riedemann  wrote:

> We talked about cells on Wednesday morning at the PTG. The full etherpad
> is here [1].
>
> Searchlight integration
> ---
>
> We talked a bit about what needs to happen for this, and it starts with
> getting the data into searchlight so that it can serve the REST API, which
> is being worked in this blueprint [2]. We want to get that done early in
> Pike.
>
> We plan on making the use of Searchlight configurable in Nova since at
> first you might not even have anything in it, so listing instances wouldn't
> work. We're also going to attempt to merge-sort when listing instances
> across multiple cells but it's going to be a known issue that it will be
> slow.
>
> For testing Nova with Searchlight, we need to start by enabling the
> Searchlight devstack plugin in the nova-next CI job, which I'll work on.
>
> I'm going to talk to Kevin Zheng about seeing if he can spend some time on
> getting Nova to use Searchlight if it's (1) configured for use and (2) is
> available (the endpoint is in the service catalog). Kevin is a Searchlight
> core and familiar with the Nova API code, so he's a good candidate for
> working on this (assuming he's available and willing to own it).
>
> Cells-aware gaps in the API
> ---
>
> Dan Smith has started a blueprint [3] for closing gaps in the API which
> break in a multi-cell deployment. He has a test patch [4] to expose the
> failures and then they can be worked on individually. The pattern of the
> work is in [5]. Help is welcome here, so please attend the weekly cells
> meeting [6] if you want to help out.
>
> Auto-discovery of compute hosts
> ---
>
> The "discover_hosts_in_cells_interval" config option was introduced in
> Ocata which controls a periodic task in the scheduler to discover new
> unmapped compute hosts but it's not very efficient since it queries all
> cell mappings and then all compute nodes in each cell mapping and checks to
> see if those compute nodes are yet mapped to the cell in the nova_api
> database. Dan Smith has a series of changes [7] which should make that
> discovery process more efficient, it just needs to be cleaned up a bit.
>
> Service arrangement
> ---
>
> Dan Smith is working on a series of changes in both Nova and devstack for
> testing with multiple cells [8]. The general idea is that there will still
> be two nodes and two nova-compute services. There will be three
> nova-conductor services, one per cell, and then another top-level "super
> conductor" which is there for building instances and sending the server
> create down to one of the cells. All three conductors are going to be
> running in the subnode just to balance the resources a bit otherwise the
> primary node is going to be starved. The multi-cell job won't be running
> migration tests since we don't currently support instance move operations
> between cells. We're going to work a hack into the scheduler to restrict a
> move operation to the same cell the instance is already in. This means the
> live migration job will still be a single-cell setup where both
> nova-computes are in the same cell.
>
> Getting rid of nova-consoleauth
> ---
>
> There is an unfinished blueprint [9] from Paul Murray which melwitt is
> going to pick up for Pike. The idea is to move the tokens into the database
> so we don't care where the consoleauth service lives and then we can also
> kill the service.
>
> [1] https://etherpad.openstack.org/p/nova-ptg-pike-cells
> [2] https://blueprints.launchpad.net/nova/+spec/additional-notif
> ication-fields-for-searchlight
> [3] 

[openstack-dev] [nova] Pike PTG recap - cells

[Matt Riedemann]

We talked about cells on Wednesday morning at the PTG. The full etherpad 
is here [1].


Searchlight integration
---

We talked a bit about what needs to happen for this, and it starts with 
getting the data into searchlight so that it can serve the REST API, 
which is being worked in this blueprint [2]. We want to get that done 
early in Pike.


We plan on making the use of Searchlight configurable in Nova since at 
first you might not even have anything in it, so listing instances 
wouldn't work. We're also going to attempt to merge-sort when listing 
instances across multiple cells but it's going to be a known issue that 
it will be slow.


For testing Nova with Searchlight, we need to start by enabling the 
Searchlight devstack plugin in the nova-next CI job, which I'll work on.


I'm going to talk to Kevin Zheng about seeing if he can spend some time 
on getting Nova to use Searchlight if it's (1) configured for use and 
(2) is available (the endpoint is in the service catalog). Kevin is a 
Searchlight core and familiar with the Nova API code, so he's a good 
candidate for working on this (assuming he's available and willing to 
own it).


Cells-aware gaps in the API
---

Dan Smith has started a blueprint [3] for closing gaps in the API which 
break in a multi-cell deployment. He has a test patch [4] to expose the 
failures and then they can be worked on individually. The pattern of the 
work is in [5]. Help is welcome here, so please attend the weekly cells 
meeting [6] if you want to help out.


Auto-discovery of compute hosts
---

The "discover_hosts_in_cells_interval" config option was introduced in 
Ocata which controls a periodic task in the scheduler to discover new 
unmapped compute hosts but it's not very efficient since it queries all 
cell mappings and then all compute nodes in each cell mapping and checks 
to see if those compute nodes are yet mapped to the cell in the nova_api 
database. Dan Smith has a series of changes [7] which should make that 
discovery process more efficient, it just needs to be cleaned up a bit.


Service arrangement
---

Dan Smith is working on a series of changes in both Nova and devstack 
for testing with multiple cells [8]. The general idea is that there will 
still be two nodes and two nova-compute services. There will be three 
nova-conductor services, one per cell, and then another top-level "super 
conductor" which is there for building instances and sending the server 
create down to one of the cells. All three conductors are going to be 
running in the subnode just to balance the resources a bit otherwise the 
primary node is going to be starved. The multi-cell job won't be running 
migration tests since we don't currently support instance move 
operations between cells. We're going to work a hack into the scheduler 
to restrict a move operation to the same cell the instance is already 
in. This means the live migration job will still be a single-cell setup 
where both nova-computes are in the same cell.


Getting rid of nova-consoleauth
---

There is an unfinished blueprint [9] from Paul Murray which melwitt is 
going to pick up for Pike. The idea is to move the tokens into the 
database so we don't care where the consoleauth service lives and then 
we can also kill the service.


[1] https://etherpad.openstack.org/p/nova-ptg-pike-cells
[2] 
https://blueprints.launchpad.net/nova/+spec/additional-notification-fields-for-searchlight

[3] https://blueprints.launchpad.net/nova/+spec/cells-aware-api
[4] https://review.openstack.org/#/c/433318/
[5] https://review.openstack.org/#/c/433362/
[6] http://eavesdrop.openstack.org/#Nova_Cellsv2_Meeting
[7] https://review.openstack.org/#/c/427901/
[8] https://review.openstack.org/#/c/436094/
[9] https://blueprints.launchpad.net/nova/+spec/convert-consoles-to-objects

--

Thanks,

Matt Riedemann

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev