Re: [openstack-dev] [all] Introducing the Cloud Service Federation project (cross-project design summit proposal)
A purchases IaaS services from reseller B, who sources those services from service provider C, nobody at C (OpenStack admin, root on any box) should be able to identify that A is consuming resources; all that they can see is that the requests are coming from B. It’s unclear if we should defer this requirement to a future version, or whether there’s something we need to (or can) do now. The main focus of Cloud Service Federation is managing the life cycle of virtual regions and service chaining. It builds on the Keystone federated identity work over the last two cycles, but identity is only part of the problem. However, I recognize that we do have an issue with terminology. For a lot of people, “federation” is simply interpreted as “identity federation”. If there’s a better term than “cloud service federation”, I’d love to hear it. (The Cisco term “Intercloud” is accurate, but probably inappropriate!) Geoff On Apr 15, 2015, at 7:07 PM, Adam Young ayo...@redhat.com wrote: On 04/15/2015 04:23 PM, Geoff Arnold wrote: That’s the basic idea. Now, if you’re a reseller of cloud services, you deploy Horizon+Aggregator/Keystone behind your public endpoint, with your branding on Horizon. You then bind each of your Aggregator Regions to a Virtual Region from one of your providers. As a reseller, you don’t actually deploy the rest of OpenStack. As for tokens, there are at least two variations, each with pros and cons: proxy and pass-through. Still working through implications of both. Geoff Read the Keysteon to Keystone (K2K) docs in the Keystone spec repo, as that addresses some of the issues here. On Apr 15, 2015, at 12:49 PM, Fox, Kevin M kevin@pnnl.gov wrote: So, an Aggregator would basically be a stripped down keystone that basically provided a dynamic service catalog that points to the registered other regions? You could then point a horizon, cli, or rest api at the aggregator service? I guess if it was an identity provider too, it can potentially talk to the remote keystone and generate project scoped tokens, though you'd need project+region scoped tokens, which I'm not sure exists today? Thanks, Kevin From: Geoff Arnold [ge...@geoffarnold.com] Sent: Wednesday, April 15, 2015 12:05 PM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [all] Introducing the Cloud Service Federation project (cross-project design summit proposal) tl;dr We want to implement a new system which we’re calling an Aggregator which is based on Horizon and Keystone, and that can provide access to virtual Regions from multiple independent OpenStack providers. We plan on developing this system as a project in Stackforge, but we need help right now in identifying any unexpected dependencies. For the last 6-7 years, there has been great interest in the potential for various business models involving multiple clouds and/or cloud providers. These business models include but are not limited to, federation, reseller, broker, cloud-bursting, hybrid and intercloud. The core concept of this initiative is to go beyond the simple dyadic relationship between a cloud service provider and a cloud service consumer to a more sophisticated “supply chain” of cloud services, dynamically configured, and operated by different business entities. This is an ambitious goal, but there is a general sense that OpenStack is becoming stable and mature enough to support such an undertaking. Until now, OpenStack has focused on the logical abstraction of a Region as the basis for cloud service consumption. A user interacts with Horizon and Keystone instances for a Region, and through them gains access to the services and resources within the specified Region. A recent extension of this model has been to share Horizon and Keystone instances between several Regions. This simplifies the user experience and enables single sign on to a “single pane of glass”. However, in this configuration all of the services, shared or otherwise, are still administered by a single entity, and the configuration of the whole system is essentially static and centralized. We’re proposing that the first step in realizing the Cloud Service Federation use cases is to enable the administrative separation of interface and region: to create a new system which provides the same user interface as today - Horizon, Keystone - but which is administratively separate from the Region(s) which provide the actual IaaS resources. We don’t yet have a good name for this system; we’ve been referring to it as the “Aggregator”. It includes slightly-modified Horizon and Keystone services, together with a subsystem which configures these services to implement the mapping of “Aggregator Regions” to multiple, administratively independent, “Provider Regions”. Just as the User-Provider relationship in OpenStack is “on demand”, we want the Aggregator-Provider mappings to be dynamic, established by APIs
Re: [openstack-dev] [all] Introducing the Cloud Service Federation project (cross-project design summit proposal)
potentially talk to the remote keystone and generate project scoped tokens, though you'd need project+region scoped tokens, which I'm not sure exists today? Thanks, Kevin From: Geoff Arnold [ge...@geoffarnold.com mailto:ge...@geoffarnold.com] Sent: Wednesday, April 15, 2015 12:05 PM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [all] Introducing the Cloud Service Federation project (cross-project design summit proposal) tl;dr We want to implement a new system which we’re calling an Aggregator which is based on Horizon and Keystone, and that can provide access to virtual Regions from multiple independent OpenStack providers. We plan on developing this system as a project in Stackforge, but we need help right now in identifying any unexpected dependencies. For the last 6-7 years, there has been great interest in the potential for various business models involving multiple clouds and/or cloud providers. These business models include but are not limited to, federation, reseller, broker, cloud-bursting, hybrid and intercloud. The core concept of this initiative is to go beyond the simple dyadic relationship between a cloud service provider and a cloud service consumer to a more sophisticated “supply chain” of cloud services, dynamically configured, and operated by different business entities. This is an ambitious goal, but there is a general sense that OpenStack is becoming stable and mature enough to support such an undertaking. Until now, OpenStack has focused on the logical abstraction of a Region as the basis for cloud service consumption. A user interacts with Horizon and Keystone instances for a Region, and through them gains access to the services and resources within the specified Region. A recent extension of this model has been to share Horizon and Keystone instances between several Regions. This simplifies the user experience and enables single sign on to a “single pane of glass”. However, in this configuration all of the services, shared or otherwise, are still administered by a single entity, and the configuration of the whole system is essentially static and centralized. We’re proposing that the first step in realizing the Cloud Service Federation use cases is to enable the administrative separation of interface and region: to create a new system which provides the same user interface as today - Horizon, Keystone - but which is administratively separate from the Region(s) which provide the actual IaaS resources. We don’t yet have a good name for this system; we’ve been referring to it as the “Aggregator”. It includes slightly-modified Horizon and Keystone services, together with a subsystem which configures these services to implement the mapping of “Aggregator Regions” to multiple, administratively independent, “Provider Regions”. Just as the User-Provider relationship in OpenStack is “on demand”, we want the Aggregator-Provider mappings to be dynamic, established by APIs, rather than statically configured. We want to achieve this without substantially changing the user experience, and with no changes to applications or to core OpenStack services. The Aggregator represents an additional way of accessing a cloud; it does not replace the existing Horizon and Keystone. The functionality and workflow is as follows: A user, X, logs into the Horizon interface provided by Aggregator A. The user sees two Regions, V1 and V2, and selects V1. This Region is actually provided by OpenStack service provider P; it’s the Region which P knows as R4. X now creates a new tenant project, T. Leveraging the Hierarchical Multitenancy work in Kilo, T is actually instantiated as a subproject of a Domain in R4, thus providing namespace isolation and quota management. Now X can deploy and operate her project T as she is used to, using Horizon, CLI, or other client-side tools. UI and API requests are forwarded by the Aggregator to P’s Region R4. [I’ll transfer this to the wiki and add diagrams.] As noted, the high-level workflow is relatively straightforward, but we need to understand two important concepts. First, how does P make R4 available for use by A? Are all of the services and resources in R4 available to A, or can P restrict things in some way? What’s the lifecycle of the relationship? Secondly, how do we handle identity? Can we arrange that same identity provider is used in the Aggregator and in the relevant domain within R4? One answer to these issues is to introduce what Mark Shuttleworth called “virtual Regions” at his talk in Paris; add a layer which exposes a Domain within a Region (with associated IDM, quotas, and other policies) as a browsable, consumable resource aggregate. To implement this, P can add a new service to R4, the Virtual Region Manager, with the twin roles
Re: [openstack-dev] [all] Introducing the Cloud Service Federation project (cross-project design summit proposal)
So your building a cloud out of pieces of other clouds. Would that be a VirtualCloud, a MetaCloud, other? :) No matter what you call it, I think its a great idea. Kevin From: Geoff Arnold [ge...@geoffarnold.com] Sent: Wednesday, April 15, 2015 9:24 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [all] Introducing the Cloud Service Federation project (cross-project design summit proposal) Yeah, we’ve taken account of: https://github.com/openstack/keystone-specs/blob/master/specs/juno/keystone-to-keystone-federation.rst http://blog.rodrigods.com/playing-with-keystone-to-keystone-federation/ http://docs.openstack.org/developer/keystone/configure_federation.html One of the use cases we’re wrestling with requires fairly strong anonymization: if user A purchases IaaS services from reseller B, who sources those services from service provider C, nobody at C (OpenStack admin, root on any box) should be able to identify that A is consuming resources; all that they can see is that the requests are coming from B. It’s unclear if we should defer this requirement to a future version, or whether there’s something we need to (or can) do now. The main focus of Cloud Service Federation is managing the life cycle of virtual regions and service chaining. It builds on the Keystone federated identity work over the last two cycles, but identity is only part of the problem. However, I recognize that we do have an issue with terminology. For a lot of people, “federation” is simply interpreted as “identity federation”. If there’s a better term than “cloud service federation”, I’d love to hear it. (The Cisco term “Intercloud” is accurate, but probably inappropriate!) Geoff On Apr 15, 2015, at 7:07 PM, Adam Young ayo...@redhat.commailto:ayo...@redhat.com wrote: On 04/15/2015 04:23 PM, Geoff Arnold wrote: That’s the basic idea. Now, if you’re a reseller of cloud services, you deploy Horizon+Aggregator/Keystone behind your public endpoint, with your branding on Horizon. You then bind each of your Aggregator Regions to a Virtual Region from one of your providers. As a reseller, you don’t actually deploy the rest of OpenStack. As for tokens, there are at least two variations, each with pros and cons: proxy and pass-through. Still working through implications of both. Geoff Read the Keysteon to Keystone (K2K) docs in the Keystone spec repo, as that addresses some of the issues here. On Apr 15, 2015, at 12:49 PM, Fox, Kevin M kevin@pnnl.govmailto:kevin@pnnl.gov wrote: So, an Aggregator would basically be a stripped down keystone that basically provided a dynamic service catalog that points to the registered other regions? You could then point a horizon, cli, or rest api at the aggregator service? I guess if it was an identity provider too, it can potentially talk to the remote keystone and generate project scoped tokens, though you'd need project+region scoped tokens, which I'm not sure exists today? Thanks, Kevin From: Geoff Arnold [ge...@geoffarnold.commailto:ge...@geoffarnold.com] Sent: Wednesday, April 15, 2015 12:05 PM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [all] Introducing the Cloud Service Federation project (cross-project design summit proposal) tl;dr We want to implement a new system which we’re calling an Aggregator which is based on Horizon and Keystone, and that can provide access to virtual Regions from multiple independent OpenStack providers. We plan on developing this system as a project in Stackforge, but we need help right now in identifying any unexpected dependencies. For the last 6-7 years, there has been great interest in the potential for various business models involving multiple clouds and/or cloud providers. These business models include but are not limited to, federation, reseller, broker, cloud-bursting, hybrid and intercloud. The core concept of this initiative is to go beyond the simple dyadic relationship between a cloud service provider and a cloud service consumer to a more sophisticated “supply chain” of cloud services, dynamically configured, and operated by different business entities. This is an ambitious goal, but there is a general sense that OpenStack is becoming stable and mature enough to support such an undertaking. Until now, OpenStack has focused on the logical abstraction of a Region as the basis for cloud service consumption. A user interacts with Horizon and Keystone instances for a Region, and through them gains access to the services and resources within the specified Region. A recent extension of this model has been to share Horizon and Keystone instances between several Regions. This simplifies the user experience and enables single sign on to a “single pane of glass”. However, in this configuration all of the services, shared
Re: [openstack-dev] [all] Introducing the Cloud Service Federation project (cross-project design summit proposal)
branding on Horizon. You then bind each of your Aggregator Regions to a Virtual Region from one of your providers. As a reseller, you don’t actually deploy the rest of OpenStack. As for tokens, there are at least two variations, each with pros and cons: proxy and pass-through. Still working through implications of both. Geoff Read the Keysteon to Keystone (K2K) docs in the Keystone spec repo, as that addresses some of the issues here. On Apr 15, 2015, at 12:49 PM, Fox, Kevin M kevin@pnnl.gov wrote: So, an Aggregator would basically be a stripped down keystone that basically provided a dynamic service catalog that points to the registered other regions? You could then point a horizon, cli, or rest api at the aggregator service? I guess if it was an identity provider too, it can potentially talk to the remote keystone and generate project scoped tokens, though you'd need project+region scoped tokens, which I'm not sure exists today? Thanks, Kevin From: Geoff Arnold [ge...@geoffarnold.com] Sent: Wednesday, April 15, 2015 12:05 PM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [all] Introducing the Cloud Service Federation project (cross-project design summit proposal) tl;dr We want to implement a new system which we’re calling an Aggregator which is based on Horizon and Keystone, and that can provide access to virtual Regions from multiple independent OpenStack providers. We plan on developing this system as a project in Stackforge, but we need help right now in identifying any unexpected dependencies. For the last 6-7 years, there has been great interest in the potential for various business models involving multiple clouds and/or cloud providers. These business models include but are not limited to, federation, reseller, broker, cloud-bursting, hybrid and intercloud. The core concept of this initiative is to go beyond the simple dyadic relationship between a cloud service provider and a cloud service consumer to a more sophisticated “supply chain” of cloud services, dynamically configured, and operated by different business entities. This is an ambitious goal, but there is a general sense that OpenStack is becoming stable and mature enough to support such an undertaking. Until now, OpenStack has focused on the logical abstraction of a Region as the basis for cloud service consumption. A user interacts with Horizon and Keystone instances for a Region, and through them gains access to the services and resources within the specified Region. A recent extension of this model has been to share Horizon and Keystone instances between several Regions. This simplifies the user experience and enables single sign on to a “single pane of glass”. However, in this configuration all of the services, shared or otherwise, are still administered by a single entity, and the configuration of the whole system is essentially static and centralized. We’re proposing that the first step in realizing the Cloud Service Federation use cases is to enable the administrative separation of interface and region: to create a new system which provides the same user interface as today - Horizon, Keystone - but which is administratively separate from the Region(s) which provide the actual IaaS resources. We don’t yet have a good name for this system; we’ve been referring to it as the “Aggregator”. It includes slightly-modified Horizon and Keystone services, together with a subsystem which configures these services to implement the mapping of “Aggregator Regions” to multiple, administratively independent, “Provider Regions”. Just as the User-Provider relationship in OpenStack is “on demand”, we want the Aggregator-Provider mappings to be dynamic, established by APIs, rather than statically configured. We want to achieve this without substantially changing the user experience, and with no changes to applications or to core OpenStack services. The Aggregator represents an additional way of accessing a cloud; it does not replace the existing Horizon and Keystone. The functionality and workflow is as follows: A user, X, logs into the Horizon interface provided by Aggregator A. The user sees two Regions, V1 and V2, and selects V1. This Region is actually provided by OpenStack service provider P; it’s the Region which P knows as R4. X now creates a new tenant project, T. Leveraging the Hierarchical Multitenancy work in Kilo, T is actually instantiated as a subproject of a Domain in R4, thus providing namespace isolation and quota management. Now X can deploy and operate her project T as she is used to, using Horizon, CLI, or other client-side tools. UI and API requests are forwarded by the Aggregator to P’s Region R4. [I’ll transfer this to the wiki and add diagrams.] As noted, the high-level workflow is relatively straightforward, but we need to understand two important
Re: [openstack-dev] [all] Introducing the Cloud Service Federation project (cross-project design summit proposal)
wrestling with requires fairly strong anonymization: if user A purchases IaaS services from reseller B, who sources those services from service provider C, nobody at C (OpenStack admin, root on any box) should be able to identify that A is consuming resources; all that they can see is that the requests are coming from B. It’s unclear if we should defer this requirement to a future version, or whether there’s something we need to (or can) do now. The main focus of Cloud Service Federation is managing the life cycle of virtual regions and service chaining. It builds on the Keystone federated identity work over the last two cycles, but identity is only part of the problem. However, I recognize that we do have an issue with terminology. For a lot of people, “federation” is simply interpreted as “identity federation”. If there’s a better term than “cloud service federation”, I’d love to hear it. (The Cisco term “Intercloud” is accurate, but probably inappropriate!) Geoff On Apr 15, 2015, at 7:07 PM, Adam Young ayo...@redhat.com mailto:ayo...@redhat.com wrote: On 04/15/2015 04:23 PM, Geoff Arnold wrote: That’s the basic idea. Now, if you’re a reseller of cloud services, you deploy Horizon+Aggregator/Keystone behind your public endpoint, with your branding on Horizon. You then bind each of your Aggregator Regions to a Virtual Region from one of your providers. As a reseller, you don’t actually deploy the rest of OpenStack. As for tokens, there are at least two variations, each with pros and cons: proxy and pass-through. Still working through implications of both. Geoff Read the Keysteon to Keystone (K2K) docs in the Keystone spec repo, as that addresses some of the issues here. On Apr 15, 2015, at 12:49 PM, Fox, Kevin M kevin@pnnl.gov mailto:kevin@pnnl.gov wrote: So, an Aggregator would basically be a stripped down keystone that basically provided a dynamic service catalog that points to the registered other regions? You could then point a horizon, cli, or rest api at the aggregator service? I guess if it was an identity provider too, it can potentially talk to the remote keystone and generate project scoped tokens, though you'd need project+region scoped tokens, which I'm not sure exists today? Thanks, Kevin From: Geoff Arnold [ge...@geoffarnold.com mailto:ge...@geoffarnold.com] Sent: Wednesday, April 15, 2015 12:05 PM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [all] Introducing the Cloud Service Federation project (cross-project design summit proposal) tl;dr We want to implement a new system which we’re calling an Aggregator which is based on Horizon and Keystone, and that can provide access to virtual Regions from multiple independent OpenStack providers. We plan on developing this system as a project in Stackforge, but we need help right now in identifying any unexpected dependencies. For the last 6-7 years, there has been great interest in the potential for various business models involving multiple clouds and/or cloud providers. These business models include but are not limited to, federation, reseller, broker, cloud-bursting, hybrid and intercloud. The core concept of this initiative is to go beyond the simple dyadic relationship between a cloud service provider and a cloud service consumer to a more sophisticated “supply chain” of cloud services, dynamically configured, and operated by different business entities. This is an ambitious goal, but there is a general sense that OpenStack is becoming stable and mature enough to support such an undertaking. Until now, OpenStack has focused on the logical abstraction of a Region as the basis for cloud service consumption. A user interacts with Horizon and Keystone instances for a Region, and through them gains access to the services and resources within the specified Region. A recent extension of this model has been to share Horizon and Keystone instances between several Regions. This simplifies the user experience and enables single sign on to a “single pane of glass”. However, in this configuration all of the services, shared or otherwise, are still administered by a single entity, and the configuration of the whole system is essentially static and centralized. We’re proposing that the first step in realizing the Cloud Service Federation use cases is to enable the administrative separation of interface and region: to create a new system which provides the same user interface as today - Horizon, Keystone - but which is administratively separate from the Region(s) which provide the actual IaaS resources. We don’t yet have a good name for this system; we’ve been referring to it as the “Aggregator”. It includes slightly-modified Horizon and Keystone services, together with a subsystem which
Re: [openstack-dev] [all] Introducing the Cloud Service Federation project (cross-project design summit proposal)
That’s the basic idea. Now, if you’re a reseller of cloud services, you deploy Horizon+Aggregator/Keystone behind your public endpoint, with your branding on Horizon. You then bind each of your Aggregator Regions to a Virtual Region from one of your providers. As a reseller, you don’t actually deploy the rest of OpenStack. As for tokens, there are at least two variations, each with pros and cons: proxy and pass-through. Still working through implications of both. Geoff On Apr 15, 2015, at 12:49 PM, Fox, Kevin M kevin@pnnl.gov wrote: So, an Aggregator would basically be a stripped down keystone that basically provided a dynamic service catalog that points to the registered other regions? You could then point a horizon, cli, or rest api at the aggregator service? I guess if it was an identity provider too, it can potentially talk to the remote keystone and generate project scoped tokens, though you'd need project+region scoped tokens, which I'm not sure exists today? Thanks, Kevin From: Geoff Arnold [ge...@geoffarnold.com] Sent: Wednesday, April 15, 2015 12:05 PM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [all] Introducing the Cloud Service Federation project (cross-project design summit proposal) tl;dr We want to implement a new system which we’re calling an Aggregator which is based on Horizon and Keystone, and that can provide access to virtual Regions from multiple independent OpenStack providers. We plan on developing this system as a project in Stackforge, but we need help right now in identifying any unexpected dependencies. For the last 6-7 years, there has been great interest in the potential for various business models involving multiple clouds and/or cloud providers. These business models include but are not limited to, federation, reseller, broker, cloud-bursting, hybrid and intercloud. The core concept of this initiative is to go beyond the simple dyadic relationship between a cloud service provider and a cloud service consumer to a more sophisticated “supply chain” of cloud services, dynamically configured, and operated by different business entities. This is an ambitious goal, but there is a general sense that OpenStack is becoming stable and mature enough to support such an undertaking. Until now, OpenStack has focused on the logical abstraction of a Region as the basis for cloud service consumption. A user interacts with Horizon and Keystone instances for a Region, and through them gains access to the services and resources within the specified Region. A recent extension of this model has been to share Horizon and Keystone instances between several Regions. This simplifies the user experience and enables single sign on to a “single pane of glass”. However, in this configuration all of the services, shared or otherwise, are still administered by a single entity, and the configuration of the whole system is essentially static and centralized. We’re proposing that the first step in realizing the Cloud Service Federation use cases is to enable the administrative separation of interface and region: to create a new system which provides the same user interface as today - Horizon, Keystone - but which is administratively separate from the Region(s) which provide the actual IaaS resources. We don’t yet have a good name for this system; we’ve been referring to it as the “Aggregator”. It includes slightly-modified Horizon and Keystone services, together with a subsystem which configures these services to implement the mapping of “Aggregator Regions” to multiple, administratively independent, “Provider Regions”. Just as the User-Provider relationship in OpenStack is “on demand”, we want the Aggregator-Provider mappings to be dynamic, established by APIs, rather than statically configured. We want to achieve this without substantially changing the user experience, and with no changes to applications or to core OpenStack services. The Aggregator represents an additional way of accessing a cloud; it does not replace the existing Horizon and Keystone. The functionality and workflow is as follows: A user, X, logs into the Horizon interface provided by Aggregator A. The user sees two Regions, V1 and V2, and selects V1. This Region is actually provided by OpenStack service provider P; it’s the Region which P knows as R4. X now creates a new tenant project, T. Leveraging the Hierarchical Multitenancy work in Kilo, T is actually instantiated as a subproject of a Domain in R4, thus providing namespace isolation and quota management. Now X can deploy and operate her project T as she is used to, using Horizon, CLI, or other client-side tools. UI and API requests are forwarded by the Aggregator to P’s Region R4. [I’ll transfer this to the wiki and add diagrams.] As noted
Re: [openstack-dev] [all] Introducing the Cloud Service Federation project (cross-project design summit proposal)
So, an Aggregator would basically be a stripped down keystone that basically provided a dynamic service catalog that points to the registered other regions? You could then point a horizon, cli, or rest api at the aggregator service? I guess if it was an identity provider too, it can potentially talk to the remote keystone and generate project scoped tokens, though you'd need project+region scoped tokens, which I'm not sure exists today? Thanks, Kevin From: Geoff Arnold [ge...@geoffarnold.com] Sent: Wednesday, April 15, 2015 12:05 PM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [all] Introducing the Cloud Service Federation project (cross-project design summit proposal) tl;dr We want to implement a new system which we’re calling an Aggregator which is based on Horizon and Keystone, and that can provide access to virtual Regions from multiple independent OpenStack providers. We plan on developing this system as a project in Stackforge, but we need help right now in identifying any unexpected dependencies. For the last 6-7 years, there has been great interest in the potential for various business models involving multiple clouds and/or cloud providers. These business models include but are not limited to, federation, reseller, broker, cloud-bursting, hybrid and intercloud. The core concept of this initiative is to go beyond the simple dyadic relationship between a cloud service provider and a cloud service consumer to a more sophisticated “supply chain” of cloud services, dynamically configured, and operated by different business entities. This is an ambitious goal, but there is a general sense that OpenStack is becoming stable and mature enough to support such an undertaking. Until now, OpenStack has focused on the logical abstraction of a Region as the basis for cloud service consumption. A user interacts with Horizon and Keystone instances for a Region, and through them gains access to the services and resources within the specified Region. A recent extension of this model has been to share Horizon and Keystone instances between several Regions. This simplifies the user experience and enables single sign on to a “single pane of glass”. However, in this configuration all of the services, shared or otherwise, are still administered by a single entity, and the configuration of the whole system is essentially static and centralized. We’re proposing that the first step in realizing the Cloud Service Federation use cases is to enable the administrative separation of interface and region: to create a new system which provides the same user interface as today - Horizon, Keystone - but which is administratively separate from the Region(s) which provide the actual IaaS resources. We don’t yet have a good name for this system; we’ve been referring to it as the “Aggregator”. It includes slightly-modified Horizon and Keystone services, together with a subsystem which configures these services to implement the mapping of “Aggregator Regions” to multiple, administratively independent, “Provider Regions”. Just as the User-Provider relationship in OpenStack is “on demand”, we want the Aggregator-Provider mappings to be dynamic, established by APIs, rather than statically configured. We want to achieve this without substantially changing the user experience, and with no changes to applications or to core OpenStack services. The Aggregator represents an additional way of accessing a cloud; it does not replace the existing Horizon and Keystone. The functionality and workflow is as follows: A user, X, logs into the Horizon interface provided by Aggregator A. The user sees two Regions, V1 and V2, and selects V1. This Region is actually provided by OpenStack service provider P; it’s the Region which P knows as R4. X now creates a new tenant project, T. Leveraging the Hierarchical Multitenancy work in Kilo, T is actually instantiated as a subproject of a Domain in R4, thus providing namespace isolation and quota management. Now X can deploy and operate her project T as she is used to, using Horizon, CLI, or other client-side tools. UI and API requests are forwarded by the Aggregator to P’s Region R4. [I’ll transfer this to the wiki and add diagrams.] As noted, the high-level workflow is relatively straightforward, but we need to understand two important concepts. First, how does P make R4 available for use by A? Are all of the services and resources in R4 available to A, or can P restrict things in some way? What’s the lifecycle of the relationship? Secondly, how do we handle identity? Can we arrange that same identity provider is used in the Aggregator and in the relevant domain within R4? One answer to these issues is to introduce what Mark Shuttleworth called “virtual Regions” at his talk in Paris; add a layer which exposes a Domain within a Region (with associated IDM
[openstack-dev] [all] Introducing the Cloud Service Federation project (cross-project design summit proposal)
tl;dr We want to implement a new system which we’re calling an Aggregator which is based on Horizon and Keystone, and that can provide access to virtual Regions from multiple independent OpenStack providers. We plan on developing this system as a project in Stackforge, but we need help right now in identifying any unexpected dependencies. For the last 6-7 years, there has been great interest in the potential for various business models involving multiple clouds and/or cloud providers. These business models include but are not limited to, federation, reseller, broker, cloud-bursting, hybrid and intercloud. The core concept of this initiative is to go beyond the simple dyadic relationship between a cloud service provider and a cloud service consumer to a more sophisticated “supply chain” of cloud services, dynamically configured, and operated by different business entities. This is an ambitious goal, but there is a general sense that OpenStack is becoming stable and mature enough to support such an undertaking. Until now, OpenStack has focused on the logical abstraction of a Region as the basis for cloud service consumption. A user interacts with Horizon and Keystone instances for a Region, and through them gains access to the services and resources within the specified Region. A recent extension of this model has been to share Horizon and Keystone instances between several Regions. This simplifies the user experience and enables single sign on to a “single pane of glass”. However, in this configuration all of the services, shared or otherwise, are still administered by a single entity, and the configuration of the whole system is essentially static and centralized. We’re proposing that the first step in realizing the Cloud Service Federation use cases is to enable the administrative separation of interface and region: to create a new system which provides the same user interface as today - Horizon, Keystone - but which is administratively separate from the Region(s) which provide the actual IaaS resources. We don’t yet have a good name for this system; we’ve been referring to it as the “Aggregator”. It includes slightly-modified Horizon and Keystone services, together with a subsystem which configures these services to implement the mapping of “Aggregator Regions” to multiple, administratively independent, “Provider Regions”. Just as the User-Provider relationship in OpenStack is “on demand”, we want the Aggregator-Provider mappings to be dynamic, established by APIs, rather than statically configured. We want to achieve this without substantially changing the user experience, and with no changes to applications or to core OpenStack services. The Aggregator represents an additional way of accessing a cloud; it does not replace the existing Horizon and Keystone. The functionality and workflow is as follows: A user, X, logs into the Horizon interface provided by Aggregator A. The user sees two Regions, V1 and V2, and selects V1. This Region is actually provided by OpenStack service provider P; it’s the Region which P knows as R4. X now creates a new tenant project, T. Leveraging the Hierarchical Multitenancy work in Kilo, T is actually instantiated as a subproject of a Domain in R4, thus providing namespace isolation and quota management. Now X can deploy and operate her project T as she is used to, using Horizon, CLI, or other client-side tools. UI and API requests are forwarded by the Aggregator to P’s Region R4. [I’ll transfer this to the wiki and add diagrams.] As noted, the high-level workflow is relatively straightforward, but we need to understand two important concepts. First, how does P make R4 available for use by A? Are all of the services and resources in R4 available to A, or can P restrict things in some way? What’s the lifecycle of the relationship? Secondly, how do we handle identity? Can we arrange that same identity provider is used in the Aggregator and in the relevant domain within R4? One answer to these issues is to introduce what Mark Shuttleworth called “virtual Regions” at his talk in Paris; add a layer which exposes a Domain within a Region (with associated IDM, quotas, and other policies) as a browsable, consumable resource aggregate. To implement this, P can add a new service to R4, the Virtual Region Manager, with the twin roles of defining Virtual Regions in terms of physical Region resources, and managing the service provider side of the negotiation with the Aggregator when setting up Aggregator-to-provider mappings. The intention is that the Virtual Region Manager will be a non-disruptive add-on to an existing OpenStack deployment. Obviously there are many more issues to be solved, both within OpenStack and outside (especially in the areas of OSS and BSS). However, we have the beginnings of an architecture which seems to address many of the interesting use cases. The immediate question is how to
Re: [openstack-dev] [all] Introducing the Cloud Service Federation project (cross-project design summit proposal)
Yeah, we’ve taken account of: https://github.com/openstack/keystone-specs/blob/master/specs/juno/keystone-to-keystone-federation.rst https://github.com/openstack/keystone-specs/blob/master/specs/juno/keystone-to-keystone-federation.rst http://blog.rodrigods.com/playing-with-keystone-to-keystone-federation/ http://blog.rodrigods.com/playing-with-keystone-to-keystone-federation/ http://docs.openstack.org/developer/keystone/configure_federation.html http://docs.openstack.org/developer/keystone/configure_federation.html One of the use cases we’re wrestling with requires fairly strong anonymization: if user A purchases IaaS services from reseller B, who sources those services from service provider C, nobody at C (OpenStack admin, root on any box) should be able to identify that A is consuming resources; all that they can see is that the requests are coming from B. It’s unclear if we should defer this requirement to a future version, or whether there’s something we need to (or can) do now. The main focus of Cloud Service Federation is managing the life cycle of virtual regions and service chaining. It builds on the Keystone federated identity work over the last two cycles, but identity is only part of the problem. However, I recognize that we do have an issue with terminology. For a lot of people, “federation” is simply interpreted as “identity federation”. If there’s a better term than “cloud service federation”, I’d love to hear it. (The Cisco term “Intercloud” is accurate, but probably inappropriate!) Geoff On Apr 15, 2015, at 7:07 PM, Adam Young ayo...@redhat.com wrote: On 04/15/2015 04:23 PM, Geoff Arnold wrote: That’s the basic idea. Now, if you’re a reseller of cloud services, you deploy Horizon+Aggregator/Keystone behind your public endpoint, with your branding on Horizon. You then bind each of your Aggregator Regions to a Virtual Region from one of your providers. As a reseller, you don’t actually deploy the rest of OpenStack. As for tokens, there are at least two variations, each with pros and cons: proxy and pass-through. Still working through implications of both. Geoff Read the Keysteon to Keystone (K2K) docs in the Keystone spec repo, as that addresses some of the issues here. On Apr 15, 2015, at 12:49 PM, Fox, Kevin M kevin@pnnl.gov wrote: So, an Aggregator would basically be a stripped down keystone that basically provided a dynamic service catalog that points to the registered other regions? You could then point a horizon, cli, or rest api at the aggregator service? I guess if it was an identity provider too, it can potentially talk to the remote keystone and generate project scoped tokens, though you'd need project+region scoped tokens, which I'm not sure exists today? Thanks, Kevin From: Geoff Arnold [ge...@geoffarnold.com] Sent: Wednesday, April 15, 2015 12:05 PM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [all] Introducing the Cloud Service Federation project (cross-project design summit proposal) tl;dr We want to implement a new system which we’re calling an Aggregator which is based on Horizon and Keystone, and that can provide access to virtual Regions from multiple independent OpenStack providers. We plan on developing this system as a project in Stackforge, but we need help right now in identifying any unexpected dependencies. For the last 6-7 years, there has been great interest in the potential for various business models involving multiple clouds and/or cloud providers. These business models include but are not limited to, federation, reseller, broker, cloud-bursting, hybrid and intercloud. The core concept of this initiative is to go beyond the simple dyadic relationship between a cloud service provider and a cloud service consumer to a more sophisticated “supply chain” of cloud services, dynamically configured, and operated by different business entities. This is an ambitious goal, but there is a general sense that OpenStack is becoming stable and mature enough to support such an undertaking. Until now, OpenStack has focused on the logical abstraction of a Region as the basis for cloud service consumption. A user interacts with Horizon and Keystone instances for a Region, and through them gains access to the services and resources within the specified Region. A recent extension of this model has been to share Horizon and Keystone instances between several Regions. This simplifies the user experience and enables single sign on to a “single pane of glass”. However, in this configuration all of the services, shared or otherwise, are still administered by a single entity, and the configuration of the whole system is essentially static and centralized. We’re proposing that the first step in realizing the Cloud Service Federation use cases is to enable
Re: [openstack-dev] [all] Introducing the Cloud Service Federation project (cross-project design summit proposal)
On 04/15/2015 04:23 PM, Geoff Arnold wrote: That’s the basic idea. Now, if you’re a reseller of cloud services, you deploy Horizon+Aggregator/Keystone behind your public endpoint, with your branding on Horizon. You then bind each of your Aggregator Regions to a Virtual Region from one of your providers. As a reseller, you don’t actually deploy the rest of OpenStack. As for tokens, there are at least two variations, each with pros and cons: proxy and pass-through. Still working through implications of both. Geoff Read the Keysteon to Keystone (K2K) docs in the Keystone spec repo, as that addresses some of the issues here. On Apr 15, 2015, at 12:49 PM, Fox, Kevin M kevin@pnnl.gov wrote: So, an Aggregator would basically be a stripped down keystone that basically provided a dynamic service catalog that points to the registered other regions? You could then point a horizon, cli, or rest api at the aggregator service? I guess if it was an identity provider too, it can potentially talk to the remote keystone and generate project scoped tokens, though you'd need project+region scoped tokens, which I'm not sure exists today? Thanks, Kevin From: Geoff Arnold [ge...@geoffarnold.com] Sent: Wednesday, April 15, 2015 12:05 PM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [all] Introducing the Cloud Service Federation project (cross-project design summit proposal) tl;dr We want to implement a new system which we’re calling an Aggregator which is based on Horizon and Keystone, and that can provide access to virtual Regions from multiple independent OpenStack providers. We plan on developing this system as a project in Stackforge, but we need help right now in identifying any unexpected dependencies. For the last 6-7 years, there has been great interest in the potential for various business models involving multiple clouds and/or cloud providers. These business models include but are not limited to, federation, reseller, broker, cloud-bursting, hybrid and intercloud. The core concept of this initiative is to go beyond the simple dyadic relationship between a cloud service provider and a cloud service consumer to a more sophisticated “supply chain” of cloud services, dynamically configured, and operated by different business entities. This is an ambitious goal, but there is a general sense that OpenStack is becoming stable and mature enough to support such an undertaking. Until now, OpenStack has focused on the logical abstraction of a Region as the basis for cloud service consumption. A user interacts with Horizon and Keystone instances for a Region, and through them gains access to the services and resources within the specified Region. A recent extension of this model has been to share Horizon and Keystone instances between several Regions. This simplifies the user experience and enables single sign on to a “single pane of glass”. However, in this configuration all of the services, shared or otherwise, are still administered by a single entity, and the configuration of the whole system is essentially static and centralized. We’re proposing that the first step in realizing the Cloud Service Federation use cases is to enable the administrative separation of interface and region: to create a new system which provides the same user interface as today - Horizon, Keystone - but which is administratively separate from the Region(s) which provide the actual IaaS resources. We don’t yet have a good name for this system; we’ve been referring to it as the “Aggregator”. It includes slightly-modified Horizon and Keystone services, together with a subsystem which configures these services to implement the mapping of “Aggregator Regions” to multiple, administratively independent, “Provider Regions”. Just as the User-Provider relationship in OpenStack is “on demand”, we want the Aggregator-Provider mappings to be dynamic, established by APIs, rather than statically configured. We want to achieve this without substantially changing the user experience, and with no changes to applications or to core OpenStack services. The Aggregator represents an additional way of accessing a cloud; it does not replace the existing Horizon and Keystone. The functionality and workflow is as follows: A user, X, logs into the Horizon interface provided by Aggregator A. The user sees two Regions, V1 and V2, and selects V1. This Region is actually provided by OpenStack service provider P; it’s the Region which P knows as R4. X now creates a new tenant project, T. Leveraging the Hierarchical Multitenancy work in Kilo, T is actually instantiated as a subproject of a Domain in R4, thus providing namespace isolation and quota management. Now X can deploy and operate her project T as she is used to, using Horizon, CLI, or other client-side tools. UI and API requests are forwarded by the Aggregator
Re: [openstack-dev] [all] Introducing the Cloud Service Federation project (cross-project design summit proposal)
Sounds like a very interesting idea. Have you talked to the keystone folks?, I would do this work into the keystone project itself (just a separate daemon). This still looks like identity management (federated, but identity) I know the burden of working with a mainstream project could be higher, but benefits are also higher: it becomes more useful (it becomes automatically available for everyone) and also passes through the review process of the general keystone contributors, thus getting a more robust code. Best regards, Miguel On 16/4/2015, at 6:24, Geoff Arnold ge...@geoffarnold.com wrote: Yeah, we’ve taken account of: https://github.com/openstack/keystone-specs/blob/master/specs/juno/keystone-to-keystone-federation.rst https://github.com/openstack/keystone-specs/blob/master/specs/juno/keystone-to-keystone-federation.rst http://blog.rodrigods.com/playing-with-keystone-to-keystone-federation/ http://blog.rodrigods.com/playing-with-keystone-to-keystone-federation/ http://docs.openstack.org/developer/keystone/configure_federation.html http://docs.openstack.org/developer/keystone/configure_federation.html One of the use cases we’re wrestling with requires fairly strong anonymization: if user A purchases IaaS services from reseller B, who sources those services from service provider C, nobody at C (OpenStack admin, root on any box) should be able to identify that A is consuming resources; all that they can see is that the requests are coming from B. It’s unclear if we should defer this requirement to a future version, or whether there’s something we need to (or can) do now. The main focus of Cloud Service Federation is managing the life cycle of virtual regions and service chaining. It builds on the Keystone federated identity work over the last two cycles, but identity is only part of the problem. However, I recognize that we do have an issue with terminology. For a lot of people, “federation” is simply interpreted as “identity federation”. If there’s a better term than “cloud service federation”, I’d love to hear it. (The Cisco term “Intercloud” is accurate, but probably inappropriate!) Geoff On Apr 15, 2015, at 7:07 PM, Adam Young ayo...@redhat.com mailto:ayo...@redhat.com wrote: On 04/15/2015 04:23 PM, Geoff Arnold wrote: That’s the basic idea. Now, if you’re a reseller of cloud services, you deploy Horizon+Aggregator/Keystone behind your public endpoint, with your branding on Horizon. You then bind each of your Aggregator Regions to a Virtual Region from one of your providers. As a reseller, you don’t actually deploy the rest of OpenStack. As for tokens, there are at least two variations, each with pros and cons: proxy and pass-through. Still working through implications of both. Geoff Read the Keysteon to Keystone (K2K) docs in the Keystone spec repo, as that addresses some of the issues here. On Apr 15, 2015, at 12:49 PM, Fox, Kevin M kevin@pnnl.gov mailto:kevin@pnnl.gov wrote: So, an Aggregator would basically be a stripped down keystone that basically provided a dynamic service catalog that points to the registered other regions? You could then point a horizon, cli, or rest api at the aggregator service? I guess if it was an identity provider too, it can potentially talk to the remote keystone and generate project scoped tokens, though you'd need project+region scoped tokens, which I'm not sure exists today? Thanks, Kevin From: Geoff Arnold [ge...@geoffarnold.com mailto:ge...@geoffarnold.com] Sent: Wednesday, April 15, 2015 12:05 PM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [all] Introducing the Cloud Service Federation project (cross-project design summit proposal) tl;dr We want to implement a new system which we’re calling an Aggregator which is based on Horizon and Keystone, and that can provide access to virtual Regions from multiple independent OpenStack providers. We plan on developing this system as a project in Stackforge, but we need help right now in identifying any unexpected dependencies. For the last 6-7 years, there has been great interest in the potential for various business models involving multiple clouds and/or cloud providers. These business models include but are not limited to, federation, reseller, broker, cloud-bursting, hybrid and intercloud. The core concept of this initiative is to go beyond the simple dyadic relationship between a cloud service provider and a cloud service consumer to a more sophisticated “supply chain” of cloud services, dynamically configured, and operated by different business entities. This is an ambitious goal, but there is a general sense that OpenStack is becoming stable and mature enough to support such an undertaking. Until now, OpenStack has focused on the logical abstraction
