Re: [openstack-dev] [all][Kingbird]Multi-Region Orchestrator

[Zane Bitter]

On 07/02/18 11:20, Chris Friesen wrote:

One use-case I've seen for this sort of thing is someone that has multiple 
geographically-separate clouds, and maybe they want to run the same heat stack 
in all of them.

Something like "create a keypair in each of the clouds with the same 
public key and same name" could be done by the end user with some 
coding, but it's convenient to have a tool to do it for you.


You can do this inside the Heat stack:

https://docs.openstack.org/heat/latest/template_guide/openstack.html#OS::Nova::KeyPair

- ZB

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][Kingbird]Multi-Region Orchestrator

[Goutham Pratapa]

Hi Chris,

Thanks for writing to us.

Our idea is just the same. and we are working on how to do it :)

Thanks for the use-case :)

On Wed, Feb 7, 2018 at 9:50 PM, Chris Friesen 
wrote:

> On 02/05/2018 06:33 PM, Jay Pipes wrote:
>
> It does seem to me, however, that if the intention is *not* to get into the
>> multi-cloud orchestration game, that a simpler solution to this
>> multi-region
>> OpenStack deployment use case would be to simply have a global Glance and
>> Keystone infrastructure that can seamlessly scale to multiple regions.
>>
>> That way, there'd be no need for replicating anything.
>>
>
> One use-case I've seen for this sort of thing is someone that has multiple
> geographically-separate clouds, and maybe they want to run the same heat
> stack in all of them.
>
> So they can use global glance/keystone, but they need to ensure that they
> have the right flavor(s) available in all the clouds.  This needs to be
> done by the admin user, so it can't be done as part of the normal user's
> heat stack.


> Something like "create a keypair in each of the clouds with the same
> public key and same name" could be done by the end user with some coding,
> but it's convenient to have a tool to do it for you.
>
> Chris
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Cheers !!!
Goutham Pratapa
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][Kingbird]Multi-Region Orchestrator

[Goutham Pratapa]

Hi Zane,

Thanks for writing to us.

We have a doubt and I have mentioned in the inline comments can you please
help us in that regard.

On Tue, Feb 6, 2018 at 11:53 PM, Zane Bitter  wrote:

> On 31/01/18 01:49, Goutham Pratapa wrote:
>
>> *Kingbird (The Multi Region orchestrator):*
>>
>> We are proud to announce kingbird is not only a centralized quota and
>> resource-manager but also a  Multi-region Orchestrator.
>>
>
> I'd invite you to consider coming up with a different short description
> for the project, because this one reads ambiguously. It can be interpreted
> as either an orchestrator that works across multiple regions, or a tool
> that 'orchestrates' multiple regions for some new definition of
> 'orchestration' (and I regret that we already have more than one). I gather
> you mean the latter; the former already exists in OpenStack.

>Yes as you said it can be interpreted as a tool that can orchestrate
> multiple-regions.

Just to be sure does openstack already has project which can replicate the
> resources and orchestrate??? why because In coming cycle our idea is that a
> user just gives a VM-ID or Vm-name and we sync all the resources with which
> the vm is actually created ofcourse we cant have the same network in
> target-region so we may need the network-id or port-id from the target
> region from user so that kingbird will boot up the requested vm in the
> target region(s).
>

> cheers,
> Zane.
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Cheers !!!
Goutham Pratapa
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][Kingbird]Multi-Region Orchestrator

[Goutham Pratapa]

On Tue, Feb 6, 2018 at 6:03 AM, Jay Pipes  wrote:

> Goutham, comments inline...
>
> Also, FYI, using HTML email with different color fonts to indicate
> different people talking is not particularly mailing list-friendly. For
> reasons why, just check out your last post:
>
> http://lists.openstack.org/pipermail/openstack-dev/2018-Janu
> ary/126842.html
>
> You can't tell who is saying what in the mailing list post...
>
> Much better to use non-HTML email and demarcate responses with the
> traditional > marker. :)
>
> OK, comments inline below.
>
> On 01/31/2018 01:17 PM, Goutham Pratapa wrote:
>
>> Hi Jay,
>>
>> Thanks for the questions.. :)
>>
>> What precisely do you mean by "resources" above ??
>>
>> Resources as-in resources required to boot-up a vm (Keypair, Image,
>> Flavors )
>>
>
> Gotcha. Thanks for the answer.
>
> Also, by "syncing", do you mean "replicating"? The reason I ask is because
>> in the case of, say, VM "resources", you can't "sync" a VM across regions.
>> You can replicate its bootable image, but you can't "sync" a VM's state
>> across multiple OpenStack deployments.
>>
>> Yes as you said syncing as-in replicating only.
>>
>
> Gotcha. You could, of course, actually use synchronous (or semi-sync)
> replication for various databases, including Glance and Keystone's
> identity/assignment information, but yes, async replication is just as good.
>
> and yes we cannot sync vm's across regions but our idea is to
>> sync/replicate all the parameters required to boot a vm
>>
>
> OK, sounds good.
>
> (viz. *image, keypair, flavor*) which are originally there in the source
>> region to the target regions in a single-go.
>>
>
> Gotcha.
>
> Some questions on scope that piqued my interest while reading your
> response...
>
> Is Kingbird predominantly designed to be the multi-region orchestrator for
> OpenStack deployments that are all owned/operated by the same deployer? Or
> does Kingbird have intentions of providing glue services between multiple
> fully-independent OpenStack deployments (possibly operated by different
> deployers)?
>
> Further, does Kingbird intend to get into the multi-cloud (as in AWS,
> OpenStack, Azure, etc) orchestration game?
>>
>>  > For now Kingbird is designed for openstack  deployments that are all
>> owned by the same deployer and yes we would like to get into multi-cloud
>> orchestration dont know how ?? But the idea is there. (If you can please
>> guide us then may be we can acheive this :) )
>
> > We have to see how far we can adhere between different
>> multiple-openstack deployments
>
> I'm curious what you mean by "resource management". Could you elaborate a
>> bit on this?
>>
>> Resource management as-in managing the resources i.e say a user has a
>> glance image(*qcow2 or ami format*) or
>> say flavor(*works only if admin*) with some properties or keypair present
>> in one source regionand he wants the same image or
>> same flavor with same properties or the same keypair in another set of
>> regions user may have to recreate them in all target regions.
>>
>> But with the help of kingbird you can do all the operations in a single
>> go.
>>
>> --> If user wants to sync a resource of type keypair he can replicate the
>> keypair into multiple target regions in single go (similarly glance images
>> and flavors )
>> --> If user wants different type of resource( keypair,image and flavor)
>> in a single go then user can  give a yaml file as input and kingbird
>> replicates all resources in all target regions
>>
>
> OK, I understand your use case here, thanks.
>
> It does seem to me, however, that if the intention is *not* to get into
> the multi-cloud orchestration game, that a simpler solution to this
> multi-region OpenStack deployment use case would be to simply have a global
> Glance and Keystone infrastructure that can seamlessly scale to multiple
> regions.

> Frankly we never tried this. we will have to try this.
>
> That way, there'd be no need for replicating anything.
>
> I suppose what I'm recommending it that instead of the concept of a region
> (or availability zone in Nova for that matter) being a mostly-configuration
> option thing, that the OpenStack contributor community actually work to
> make regions (the concept that Keystone labels a region; which is just a
> grouping of service endpoints) the one and only concept of a user-facing
> "partition" throughout OpenStack.
>
> That way we would have OpenStack services like Glance, Nova, Cinder,
> Neutron, etc just *natively* understand which region they are in and how/if
> they can communicate with other regions.
>
> Sometimes it seems we (as a community) go through lots of hoops working
> around fundamental architectural problems in OpenStack instead of just
> fixing those problems to begin with. See: Nova cellsv1 (and some of
> cellsv2), Keystone federation, the lack of a real availability zone concept
> anywhere, Nova shelve/unshelve (partly developed because VMs and IPs were
> 

Re: [openstack-dev] [all][Kingbird]Multi-Region Orchestrator

[Chris Friesen]

On 02/05/2018 06:33 PM, Jay Pipes wrote:


It does seem to me, however, that if the intention is *not* to get into the
multi-cloud orchestration game, that a simpler solution to this multi-region
OpenStack deployment use case would be to simply have a global Glance and
Keystone infrastructure that can seamlessly scale to multiple regions.

That way, there'd be no need for replicating anything.


One use-case I've seen for this sort of thing is someone that has multiple 
geographically-separate clouds, and maybe they want to run the same heat stack 
in all of them.


So they can use global glance/keystone, but they need to ensure that they have 
the right flavor(s) available in all the clouds.  This needs to be done by the 
admin user, so it can't be done as part of the normal user's heat stack.


Something like "create a keypair in each of the clouds with the same public key 
and same name" could be done by the end user with some coding, but it's 
convenient to have a tool to do it for you.


Chris

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][Kingbird]Multi-Region Orchestrator

[Zane Bitter]

On 31/01/18 01:49, Goutham Pratapa wrote:

*Kingbird (The Multi Region orchestrator):*

We are proud to announce kingbird is not only a centralized quota and 
resource-manager but also a  Multi-region Orchestrator.


I'd invite you to consider coming up with a different short description 
for the project, because this one reads ambiguously. It can be 
interpreted as either an orchestrator that works across multiple 
regions, or a tool that 'orchestrates' multiple regions for some new 
definition of 'orchestration' (and I regret that we already have more 
than one). I gather you mean the latter; the former already exists in 
OpenStack.


cheers,
Zane.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][Kingbird]Multi-Region Orchestrator

[Jay Pipes]

Goutham, comments inline...

Also, FYI, using HTML email with different color fonts to indicate 
different people talking is not particularly mailing list-friendly. For 
reasons why, just check out your last post:


http://lists.openstack.org/pipermail/openstack-dev/2018-January/126842.html

You can't tell who is saying what in the mailing list post...

Much better to use non-HTML email and demarcate responses with the 
traditional > marker. :)


OK, comments inline below.

On 01/31/2018 01:17 PM, Goutham Pratapa wrote:

Hi Jay,

Thanks for the questions.. :)

What precisely do you mean by "resources" above ??

Resources as-in resources required to boot-up a vm (Keypair, Image, 
Flavors )


Gotcha. Thanks for the answer.

Also, by "syncing", do you mean "replicating"? The reason I ask is 
because in the case of, say, VM "resources", you can't "sync" a VM 
across regions. You can replicate its bootable image, but you can't 
"sync" a VM's state across multiple OpenStack deployments.


Yes as you said syncing as-in replicating only.


Gotcha. You could, of course, actually use synchronous (or semi-sync) 
replication for various databases, including Glance and Keystone's 
identity/assignment information, but yes, async replication is just as good.


and yes we cannot sync vm's across regions but our idea is to 
sync/replicate all the parameters required to boot a vm


OK, sounds good.

(viz. *image, keypair, flavor*) which are originally there in the source 
region to the target regions in a single-go.


Gotcha.

Some questions on scope that piqued my interest while reading your 
response...


Is Kingbird predominantly designed to be the multi-region orchestrator 
for OpenStack deployments that are all owned/operated by the same 
deployer? Or does Kingbird have intentions of providing glue services 
between multiple fully-independent OpenStack deployments (possibly 
operated by different deployers)?


Further, does Kingbird intend to get into the multi-cloud (as in AWS, 
OpenStack, Azure, etc) orchestration game?


I'm curious what you mean by "resource management". Could you elaborate 
a bit on this?


Resource management as-in managing the resources i.e say a user has a 
glance image(*qcow2 or ami format*) or
say flavor(*works only if admin*) with some properties or keypair 
present in one source regionand he wants the same image or
same flavor with same properties or the same keypair in another set of 
regions user may have to recreate them in all target regions.


But with the help of kingbird you can do all the operations in a single go.

--> If user wants to sync a resource of type keypair he can replicate 
the keypair into multiple target regions in single go (similarly glance 
images and flavors )
--> If user wants different type of resource( keypair,image and flavor) 
in a single go then user can  give a yaml file as input and kingbird 
replicates all resources in all target regions


OK, I understand your use case here, thanks.

It does seem to me, however, that if the intention is *not* to get into 
the multi-cloud orchestration game, that a simpler solution to this 
multi-region OpenStack deployment use case would be to simply have a 
global Glance and Keystone infrastructure that can seamlessly scale to 
multiple regions.


That way, there'd be no need for replicating anything.

I suppose what I'm recommending it that instead of the concept of a 
region (or availability zone in Nova for that matter) being a 
mostly-configuration option thing, that the OpenStack contributor 
community actually work to make regions (the concept that Keystone 
labels a region; which is just a grouping of service endpoints) the one 
and only concept of a user-facing "partition" throughout OpenStack.


That way we would have OpenStack services like Glance, Nova, Cinder, 
Neutron, etc just *natively* understand which region they are in and 
how/if they can communicate with other regions.


Sometimes it seems we (as a community) go through lots of hoops working 
around fundamental architectural problems in OpenStack instead of just 
fixing those problems to begin with. See: Nova cellsv1 (and some of 
cellsv2), Keystone federation, the lack of a real availability zone 
concept anywhere, Nova shelve/unshelve (partly developed because VMs and 
IPs were too closely coupled at the time), the list goes on and on...


Anyway, mostly just rambling/ranting... just food for thought.

Best,
-jay


Thanks
Goutham.

On Wed, Jan 31, 2018 at 9:25 PM, Jay Pipes > wrote:


On 01/31/2018 01:49 AM, Goutham Pratapa wrote:

*Kingbird (The Multi Region orchestrator):*

We are proud to announce kingbird is not only a centralized
quota and resource-manager but also a  Multi-region Orchestrator.

*Use-cases covered:

*- Admin can synchronize and periodically balance quotas across
regions and can have a global view of quotas of all the tenants
   

Re: [openstack-dev] [all][Kingbird]Multi-Region Orchestrator

[Goutham Pratapa]

Hi Jay,

Thanks for the questions.. :)

What precisely do you mean by "resources" above ??

Resources as-in resources required to boot-up a vm (Keypair, Image, Flavors
)

Also, by "syncing", do you mean "replicating"? The reason I ask is because
in the case of, say, VM "resources", you can't "sync" a VM across regions.
You can replicate its bootable image, but you can't "sync" a VM's state
across multiple OpenStack deployments.

>
Yes as you said syncing as-in replicating only.

and yes we cannot sync vm's across regions but our idea is to
sync/replicate all the parameters required to boot a vm

(viz. *image, keypair, flavor*) which are originally there in the source
region to the target regions in a single-go.

I'm curious what you mean by "resource management". Could you elaborate a
bit on this?

Resource management as-in managing the resources i.e say a user has a
glance image(*qcow2 or ami format*) or
say flavor(*works only if admin*) with some properties or keypair present
in one source region and he wants the same image or
same flavor with same properties or the same keypair in another set of
regions user may have to recreate them in all target regions.

But with the help of kingbird you can do all the operations in a single go.

--> If user wants to sync a resource of type keypair he can replicate the
keypair into multiple target regions in single go (similarly glance images
and flavors )
--> If user wants different type of resource( keypair,image and flavor) in
a single go then user can  give a yaml file as input and kingbird
replicates all resources in all target regions


Thanks
Goutham.

On Wed, Jan 31, 2018 at 9:25 PM, Jay Pipes  wrote:

> On 01/31/2018 01:49 AM, Goutham Pratapa wrote:
>
>> *Kingbird (The Multi Region orchestrator):*
>>
>> We are proud to announce kingbird is not only a centralized quota and
>> resource-manager but also a  Multi-region Orchestrator.
>>
>> *Use-cases covered:
>>
>> *- Admin can synchronize and periodically balance quotas across regions
>> and can have a global view of quotas of all the tenants across regions.
>> - A user can sync a resource or a group of resources from one region to
>> other in a single go
>>
>
> What precisely do you mean by "resources" above?
>
> Also, by "syncing", do you mean "replicating"? The reason I ask is because
> in the case of, say, VM "resources", you can't "sync" a VM across regions.
> You can replicate its bootable image, but you can't "sync" a VM's state
> across multiple OpenStack deployments.
>
>   A user can sync multiple key-pairs, images, and flavors from one region
>> to other, ( Flavor can be synced only by admin)
>>
>> - A user must have complete tempest test-coverage for all the
>> scenarios/services rendered by kingbird.
>>
>> - Horizon plugin so that user can access/view global limits.
>>
>> * Our Road-map:*
>>
>> -- Automation scripts for kingbird in
>>  -ansible,
>>  -salt
>>  -puppet.
>> -- Add SSL support to kingbird
>> -- Resource management in Kingbird-dashboard.
>>
>
> I'm curious what you mean by "resource management". Could you elaborate a
> bit on this?
>
> Thanks,
> -jay
>
> -- Kingbird in a docker
>> -- Add Kingbird into Kolla.
>>
>> We are looking out for*_contributors and ideas_* which can enhance
>> Kingbird and make kingbird a one-stop solution for all multi-region problems
>>
>>
>>
>> *_Stable Branches :_
>> *
>> *
>> Kingbird-server: https://github.com/openstack/kingbird/tree/stable/queens
>> 
>> *
>> *Python-Kingbird-client (0.2.1): https://github.com/openstack/p
>> ython-kingbirdclient/tree/0.2.1 > python-kingbirdclient/tree/0.2.1>
>> *
>>
>> I would like to Thank all the people who have helped us in achieving this
>> milestone and guided us all throughout this Journey :)
>>
>> Thanks
>> Goutham Pratapa
>> PTL
>> OpenStack-Kingbird.
>>
>>
>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>


-- 
Cheers !!!
Goutham Pratapa
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][Kingbird]Multi-Region Orchestrator

[Jay Pipes]

On 01/31/2018 01:49 AM, Goutham Pratapa wrote:

*Kingbird (The Multi Region orchestrator):*

We are proud to announce kingbird is not only a centralized quota and 
resource-manager but also a  Multi-region Orchestrator.


*Use-cases covered:

*- Admin can synchronize and periodically balance quotas across regions 
and can have a global view of quotas of all the tenants across regions.
- A user can sync a resource or a group of resources from one region to 
other in a single go


What precisely do you mean by "resources" above?

Also, by "syncing", do you mean "replicating"? The reason I ask is 
because in the case of, say, VM "resources", you can't "sync" a VM 
across regions. You can replicate its bootable image, but you can't 
"sync" a VM's state across multiple OpenStack deployments.


  A user can sync multiple key-pairs, images, and flavors from one 
region to other, ( Flavor can be synced only by admin)


- A user must have complete tempest test-coverage for all the 
scenarios/services rendered by kingbird.


- Horizon plugin so that user can access/view global limits.

* Our Road-map:*

-- Automation scripts for kingbird in
     -ansible,
     -salt
     -puppet.
-- Add SSL support to kingbird
-- Resource management in Kingbird-dashboard.


I'm curious what you mean by "resource management". Could you elaborate 
a bit on this?


Thanks,
-jay


-- Kingbird in a docker
-- Add Kingbird into Kolla.

We are looking out for*_contributors and ideas_* which can enhance 
Kingbird and make kingbird a one-stop solution for all multi-region problems




*_Stable Branches :_
*
*
Kingbird-server: 
https://github.com/openstack/kingbird/tree/stable/queens 


*
*Python-Kingbird-client (0.2.1): 
https://github.com/openstack/python-kingbirdclient/tree/0.2.1 


*

I would like to Thank all the people who have helped us in achieving 
this milestone and guided us all throughout this Journey :)


Thanks
Goutham Pratapa
PTL
OpenStack-Kingbird.



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][Kingbird]Multi-Region Orchestrator

[Goutham Pratapa]

OK, sorry that was a mistake :)

Thank you, Thierry  :)


On Wed, Jan 31, 2018 at 2:51 PM, Thierry Carrez 
wrote:

> Goutham Pratapa wrote:
> > *Kingbird (The Multi Region orchestrator):*
> >
> > We are proud to announce kingbird is not only a centralized quota and
> > resource-manager but also a  Multi-region Orchestrator.
> > [...]
> > Thanks
> > Goutham Pratapa
> > PTL
> > OpenStack-Kingbird.
>
> Quick clarification: Kingbird is not (yet) an official OpenStack
> component, so it should probably not call itself OpenStack Kingbird.
>
> Regards,
>
> --
> Thierry Carrez (ttx)
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Cheers !!!
Goutham Pratapa
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][Kingbird]Multi-Region Orchestrator

[Thierry Carrez]

Goutham Pratapa wrote:
> *Kingbird (The Multi Region orchestrator):*
> 
> We are proud to announce kingbird is not only a centralized quota and
> resource-manager but also a  Multi-region Orchestrator.
> [...]
> Thanks
> Goutham Pratapa
> PTL
> OpenStack-Kingbird.

Quick clarification: Kingbird is not (yet) an official OpenStack
component, so it should probably not call itself OpenStack Kingbird.

Regards,

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev