Re: [openstack-dev] [all][sdk] Integrating OpenStack and k8s with a service broker
Now that the project is set up, let's tag future messages on this topic with [service-broker]. Here's one to start with that will help you find everything: http://lists.openstack.org/pipermail/openstack-dev/2018-June/131923.html cheers, Zane. On 05/06/18 12:19, Zane Bitter wrote: I've been doing some investigation into the Service Catalog in Kubernetes and how we can get OpenStack resources to show up in the catalog for use by applications running in Kubernetes. (The Big 3 public clouds already support this.) The short answer is via an implementation of something called the Open Service Broker API, but there are shortcuts available to make it easier to do. I'm convinced that this is readily achievable and something we ought to do as a community. I've put together a (long-winded) FAQ below to answer all of your questions about it. Would you be interested in working on a new project to implement this integration? Reply to this thread and let's collect a list of volunteers to form the initial core review team. cheers, Zane. What is the Open Service Broker API? The Open Service Broker API[1] is a standard way to expose external resources to applications running in a PaaS. It was originally developed in the context of CloudFoundry, but the same standard was adopted by Kubernetes (and hence OpenShift) in the form of the Service Catalog extension[2]. (The Service Catalog in Kubernetes is the component that calls out to a service broker.) So a single implementation can cover the most popular open-source PaaS offerings. In many cases, the services take the form of simply a pre-packaged application that also runs inside the PaaS. But they don't have to be - services can be anything. Provisioning via the service broker ensures that the services requested are tied in to the PaaS's orchestration of the application's lifecycle. (This is certainly not the be-all and end-all of integration between OpenStack and containers - we also need ways to tie PaaS-based applications into the OpenStack's orchestration of a larger group of resources. Some applications may even use both. But it's an important part of the story.) What sorts of services would OpenStack expose? -- Some example use cases might be: * The application needs a reliable message queue. Rather than spinning up multiple storage-backed containers with anti-affinity policies and dealing with the overhead of managing e.g. RabbitMQ, the application requests a Zaqar queue from an OpenStack cloud. The overhead of running the queueing service is amortised across all of the applications in the cloud. The queue gets cleaned up correctly when the application is removed, since it is tied into the application definition. * The application needs a database. Rather than spinning one up in a storage-backed container and dealing with the overhead of managing it, the application requests a Trove DB from an OpenStack cloud. * The application includes a service that needs to run on bare metal for performance reasons (e.g. could also be a database). The application requests a bare-metal server from Nova w/ Ironic for the purpose. (The same applies to requesting a VM, but there are alternatives like KubeVirt - which also operates through the Service Catalog - available for getting a VM in Kubernetes. There are no non-proprietary alternatives for getting a bare-metal server.) AWS[3], Azure[4], and GCP[5] all have service brokers available that support these and many more services that they provide. I don't know of any reason in principle not to expose every type of resource that OpenStack provides via a service broker. How is this different from cloud-provider-openstack? The Cloud Controller[6] interface in Kubernetes allows Kubernetes itself to access features of the cloud to provide its service. For example, if k8s needs persistent storage for a container then it can request that from Cinder through cloud-provider-openstack[7]. It can also request a load balancer from Octavia instead of having to start a container running HAProxy to load balance between multiple instances of an application container (thus enabling use of hardware load balancers via the cloud's abstraction for them). In contrast, the Service Catalog interface allows the *application* running on Kubernetes to access features of the cloud. What does a service broker look like? - A service broker provides an HTTP API with 5 actions: * List the services provided by the broker * Create an instance of a resource * Bind the resource into an instance of the application * Unbind the resource from an instance of the application * Delete the resource The binding step is used for things like providing a set of DB credentials to a container. You can rotate credentials when
Re: [openstack-dev] [all][sdk] Integrating OpenStack and k8s with a service broker
Thanks, Zane for putting this up. It's a great service to expose infrastructure to application, and a potential cross-community works as well. > > Would you be interested in working on a new project to implement this > integration? Reply to this thread and let's collect a list of volunteers > to form the initial core review team. > Glad to help > I'd prefer to go with the pure-Ansible autogenerated way so we can have > support for everything, but looking at the GCP[5]/Azure[4]/AWS[3] > brokers they have 10, 11 and 17 services respectively, so arguably we > could get a comparable number of features exposed without investing > crazy amounts of time if we had to write templates explicitly. > If we going to generate another project to provide this service, I believe to use pure-Ansible will be a better option indeed. Once service gets stable, it's actually quite easy(at first glance) for Heat to adopt this (just create a service broker with our new service while creating a resource I believe?). Sounds like the use case of service broker might be when application request for a single resource exposed with Broker. And the resource dependency will be relatively simple. And we should just keep it simple and don't start thinking about who and how that application was created and keep the application out of dependency (I mean if the user likes to manage the total dependency, they can consider using heat with service broker once we integrated). -- May The Force of OpenStack Be With You, Rico Lin __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][sdk] Integrating OpenStack and k8s with a service broker
"do you think this is an area that OpenLab could help out with?" << YES!! please ping mrhillsman and RuiChen over on #askopenlab -- Dims On Wed, Jun 6, 2018 at 11:44 AM, Zane Bitter wrote: > On 06/06/18 11:18, Chris Hoge wrote: >> >> Hi Zane, >> >> Do you think this effort would make sense as a subproject within the Cloud >> Provider OpenStack repository hosted within the Kubernetes org? We have >> a solid group of people working on the cloud provider, and while it’s not >> the same code, it’s a collection of the same expertise and test resources. > > > TBH, I think it makes more sense as part of the OpenStack community. If you > look at how the components interact, it goes: > > Kubernetes Service Catalog -> Automation Broker -> [this] -> OpenStack > > So the interfaces with k8s are already well-defined and owned by other > teams. It's the interface with OpenStack that requires the closest > co-ordination. (Particularly if we end up autogenerating the playbooks from > introspection on shade.) If you look at where the other clouds host their > service brokers or Ansible Playbook Bundles, they're not part of the > equivalent Kubernetes Cloud Providers either. > > We'll definitely want testing though. Given that this is effectively another > user interface to OpenStack, do you think this is an area that OpenLab could > help out with? > >> Even if it's hosted as an OpenStack project, we should still make sure >> we have documentation and pointers from the >> kubernetes/cloud-provider-openstack >> to guide users in the right direction. > > > Sure, that makes sense to cross-advertise it to people we know are using k8s > on top of OpenStack already. (Although note that k8s does not have to be > running on top of OpenStack for the service broker to be useful, unlike the > cloud provider.) > >> While I'm not in a position to directly contribute, I'm happy to offer >> any support I can through the SIG-OpenStack and SIG-Cloud-Provider >> roles I have in the K8s community. > > > Thanks! > > cheers, > Zane. > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Davanum Srinivas :: https://twitter.com/dims __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][sdk] Integrating OpenStack and k8s with a service broker
On 06/06/18 11:18, Chris Hoge wrote: Hi Zane, Do you think this effort would make sense as a subproject within the Cloud Provider OpenStack repository hosted within the Kubernetes org? We have a solid group of people working on the cloud provider, and while it’s not the same code, it’s a collection of the same expertise and test resources. TBH, I think it makes more sense as part of the OpenStack community. If you look at how the components interact, it goes: Kubernetes Service Catalog -> Automation Broker -> [this] -> OpenStack So the interfaces with k8s are already well-defined and owned by other teams. It's the interface with OpenStack that requires the closest co-ordination. (Particularly if we end up autogenerating the playbooks from introspection on shade.) If you look at where the other clouds host their service brokers or Ansible Playbook Bundles, they're not part of the equivalent Kubernetes Cloud Providers either. We'll definitely want testing though. Given that this is effectively another user interface to OpenStack, do you think this is an area that OpenLab could help out with? Even if it's hosted as an OpenStack project, we should still make sure we have documentation and pointers from the kubernetes/cloud-provider-openstack to guide users in the right direction. Sure, that makes sense to cross-advertise it to people we know are using k8s on top of OpenStack already. (Although note that k8s does not have to be running on top of OpenStack for the service broker to be useful, unlike the cloud provider.) While I'm not in a position to directly contribute, I'm happy to offer any support I can through the SIG-OpenStack and SIG-Cloud-Provider roles I have in the K8s community. Thanks! cheers, Zane. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][sdk] Integrating OpenStack and k8s with a service broker
Hi Zane, Do you think this effort would make sense as a subproject within the Cloud Provider OpenStack repository hosted within the Kubernetes org? We have a solid group of people working on the cloud provider, and while it’s not the same code, it’s a collection of the same expertise and test resources. Even if it's hosted as an OpenStack project, we should still make sure we have documentation and pointers from the kubernetes/cloud-provider-openstack to guide users in the right direction. While I'm not in a position to directly contribute, I'm happy to offer any support I can through the SIG-OpenStack and SIG-Cloud-Provider roles I have in the K8s community. -Chris > On Jun 5, 2018, at 9:19 AM, Zane Bitter wrote: > > I've been doing some investigation into the Service Catalog in Kubernetes and > how we can get OpenStack resources to show up in the catalog for use by > applications running in Kubernetes. (The Big 3 public clouds already support > this.) The short answer is via an implementation of something called the Open > Service Broker API, but there are shortcuts available to make it easier to do. > > I'm convinced that this is readily achievable and something we ought to do as > a community. > > I've put together a (long-winded) FAQ below to answer all of your questions > about it. > > Would you be interested in working on a new project to implement this > integration? Reply to this thread and let's collect a list of volunteers to > form the initial core review team. > > cheers, > Zane. > > > What is the Open Service Broker API? > > > The Open Service Broker API[1] is a standard way to expose external resources > to applications running in a PaaS. It was originally developed in the context > of CloudFoundry, but the same standard was adopted by Kubernetes (and hence > OpenShift) in the form of the Service Catalog extension[2]. (The Service > Catalog in Kubernetes is the component that calls out to a service broker.) > So a single implementation can cover the most popular open-source PaaS > offerings. > > In many cases, the services take the form of simply a pre-packaged > application that also runs inside the PaaS. But they don't have to be - > services can be anything. Provisioning via the service broker ensures that > the services requested are tied in to the PaaS's orchestration of the > application's lifecycle. > > (This is certainly not the be-all and end-all of integration between > OpenStack and containers - we also need ways to tie PaaS-based applications > into the OpenStack's orchestration of a larger group of resources. Some > applications may even use both. But it's an important part of the story.) > > What sorts of services would OpenStack expose? > -- > > Some example use cases might be: > > * The application needs a reliable message queue. Rather than spinning up > multiple storage-backed containers with anti-affinity policies and dealing > with the overhead of managing e.g. RabbitMQ, the application requests a Zaqar > queue from an OpenStack cloud. The overhead of running the queueing service > is amortised across all of the applications in the cloud. The queue gets > cleaned up correctly when the application is removed, since it is tied into > the application definition. > > * The application needs a database. Rather than spinning one up in a > storage-backed container and dealing with the overhead of managing it, the > application requests a Trove DB from an OpenStack cloud. > > * The application includes a service that needs to run on bare metal for > performance reasons (e.g. could also be a database). The application requests > a bare-metal server from Nova w/ Ironic for the purpose. (The same applies to > requesting a VM, but there are alternatives like KubeVirt - which also > operates through the Service Catalog - available for getting a VM in > Kubernetes. There are no non-proprietary alternatives for getting a > bare-metal server.) > > AWS[3], Azure[4], and GCP[5] all have service brokers available that support > these and many more services that they provide. I don't know of any reason in > principle not to expose every type of resource that OpenStack provides via a > service broker. > > How is this different from cloud-provider-openstack? > > > The Cloud Controller[6] interface in Kubernetes allows Kubernetes itself to > access features of the cloud to provide its service. For example, if k8s > needs persistent storage for a container then it can request that from Cinder > through cloud-provider-openstack[7]. It can also request a load balancer from > Octavia instead of having to start a container running HAProxy to load > balance between multiple instances of an application container (thus enabling > use of hardware load balancers via the cloud's abstraction for them). > >
Re: [openstack-dev] [all][sdk] Integrating OpenStack and k8s with a service broker
Hi Zane, please count me in :-) Cheers, Lingxian Kong On Wed, Jun 6, 2018 at 4:52 AM, Remo Mattei wrote: > I will be happy to check it out. > > Remo > > > On Jun 5, 2018, at 9:19 AM, Zane Bitter wrote: > > I've been doing some investigation into the Service Catalog in Kubernetes > and how we can get OpenStack resources to show up in the catalog for use by > applications running in Kubernetes. (The Big 3 public clouds already > support this.) The short answer is via an implementation of something > called the Open Service Broker API, but there are shortcuts available to > make it easier to do. > > I'm convinced that this is readily achievable and something we ought to do > as a community. > > I've put together a (long-winded) FAQ below to answer all of your > questions about it. > > Would you be interested in working on a new project to implement this > integration? Reply to this thread and let's collect a list of volunteers to > form the initial core review team. > > cheers, > Zane. > > > What is the Open Service Broker API? > > > The Open Service Broker API[1] is a standard way to expose external > resources to applications running in a PaaS. It was originally developed in > the context of CloudFoundry, but the same standard was adopted by > Kubernetes (and hence OpenShift) in the form of the Service Catalog > extension[2]. (The Service Catalog in Kubernetes is the component that > calls out to a service broker.) So a single implementation can cover the > most popular open-source PaaS offerings. > > In many cases, the services take the form of simply a pre-packaged > application that also runs inside the PaaS. But they don't have to be - > services can be anything. Provisioning via the service broker ensures that > the services requested are tied in to the PaaS's orchestration of the > application's lifecycle. > > (This is certainly not the be-all and end-all of integration between > OpenStack and containers - we also need ways to tie PaaS-based applications > into the OpenStack's orchestration of a larger group of resources. Some > applications may even use both. But it's an important part of the story.) > > What sorts of services would OpenStack expose? > -- > > Some example use cases might be: > > * The application needs a reliable message queue. Rather than spinning up > multiple storage-backed containers with anti-affinity policies and dealing > with the overhead of managing e.g. RabbitMQ, the application requests a > Zaqar queue from an OpenStack cloud. The overhead of running the queueing > service is amortised across all of the applications in the cloud. The queue > gets cleaned up correctly when the application is removed, since it is tied > into the application definition. > > * The application needs a database. Rather than spinning one up in a > storage-backed container and dealing with the overhead of managing it, the > application requests a Trove DB from an OpenStack cloud. > > * The application includes a service that needs to run on bare metal for > performance reasons (e.g. could also be a database). The application > requests a bare-metal server from Nova w/ Ironic for the purpose. (The same > applies to requesting a VM, but there are alternatives like KubeVirt - > which also operates through the Service Catalog - available for getting a > VM in Kubernetes. There are no non-proprietary alternatives for getting a > bare-metal server.) > > AWS[3], Azure[4], and GCP[5] all have service brokers available that > support these and many more services that they provide. I don't know of any > reason in principle not to expose every type of resource that OpenStack > provides via a service broker. > > How is this different from cloud-provider-openstack? > > > The Cloud Controller[6] interface in Kubernetes allows Kubernetes itself > to access features of the cloud to provide its service. For example, if k8s > needs persistent storage for a container then it can request that from > Cinder through cloud-provider-openstack[7]. It can also request a load > balancer from Octavia instead of having to start a container running > HAProxy to load balance between multiple instances of an application > container (thus enabling use of hardware load balancers via the cloud's > abstraction for them). > > In contrast, the Service Catalog interface allows the *application* > running on Kubernetes to access features of the cloud. > > What does a service broker look like? > - > > A service broker provides an HTTP API with 5 actions: > > * List the services provided by the broker > * Create an instance of a resource > * Bind the resource into an instance of the application > * Unbind the resource from an instance of the application > * Delete the resource > > The binding step is used for things like providing a set of DB credentials > to a
[openstack-dev] [all][sdk] Integrating OpenStack and k8s with a service broker
Hi Zane, > Would you be interested in working on a new project to implement this > integration? Reply to this thread and let's collect a list of volunteers > to form the initial core review team. Yes, I would also like to join. That's exactly what I am looking at in my company as part of K8 over OpenStack offering. Regards, Artem __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][sdk] Integrating OpenStack and k8s with a service broker
I will be happy to check it out. Remo > On Jun 5, 2018, at 9:19 AM, Zane Bitter wrote: > > I've been doing some investigation into the Service Catalog in Kubernetes and > how we can get OpenStack resources to show up in the catalog for use by > applications running in Kubernetes. (The Big 3 public clouds already support > this.) The short answer is via an implementation of something called the Open > Service Broker API, but there are shortcuts available to make it easier to do. > > I'm convinced that this is readily achievable and something we ought to do as > a community. > > I've put together a (long-winded) FAQ below to answer all of your questions > about it. > > Would you be interested in working on a new project to implement this > integration? Reply to this thread and let's collect a list of volunteers to > form the initial core review team. > > cheers, > Zane. > > > What is the Open Service Broker API? > > > The Open Service Broker API[1] is a standard way to expose external resources > to applications running in a PaaS. It was originally developed in the context > of CloudFoundry, but the same standard was adopted by Kubernetes (and hence > OpenShift) in the form of the Service Catalog extension[2]. (The Service > Catalog in Kubernetes is the component that calls out to a service broker.) > So a single implementation can cover the most popular open-source PaaS > offerings. > > In many cases, the services take the form of simply a pre-packaged > application that also runs inside the PaaS. But they don't have to be - > services can be anything. Provisioning via the service broker ensures that > the services requested are tied in to the PaaS's orchestration of the > application's lifecycle. > > (This is certainly not the be-all and end-all of integration between > OpenStack and containers - we also need ways to tie PaaS-based applications > into the OpenStack's orchestration of a larger group of resources. Some > applications may even use both. But it's an important part of the story.) > > What sorts of services would OpenStack expose? > -- > > Some example use cases might be: > > * The application needs a reliable message queue. Rather than spinning up > multiple storage-backed containers with anti-affinity policies and dealing > with the overhead of managing e.g. RabbitMQ, the application requests a Zaqar > queue from an OpenStack cloud. The overhead of running the queueing service > is amortised across all of the applications in the cloud. The queue gets > cleaned up correctly when the application is removed, since it is tied into > the application definition. > > * The application needs a database. Rather than spinning one up in a > storage-backed container and dealing with the overhead of managing it, the > application requests a Trove DB from an OpenStack cloud. > > * The application includes a service that needs to run on bare metal for > performance reasons (e.g. could also be a database). The application requests > a bare-metal server from Nova w/ Ironic for the purpose. (The same applies to > requesting a VM, but there are alternatives like KubeVirt - which also > operates through the Service Catalog - available for getting a VM in > Kubernetes. There are no non-proprietary alternatives for getting a > bare-metal server.) > > AWS[3], Azure[4], and GCP[5] all have service brokers available that support > these and many more services that they provide. I don't know of any reason in > principle not to expose every type of resource that OpenStack provides via a > service broker. > > How is this different from cloud-provider-openstack? > > > The Cloud Controller[6] interface in Kubernetes allows Kubernetes itself to > access features of the cloud to provide its service. For example, if k8s > needs persistent storage for a container then it can request that from Cinder > through cloud-provider-openstack[7]. It can also request a load balancer from > Octavia instead of having to start a container running HAProxy to load > balance between multiple instances of an application container (thus enabling > use of hardware load balancers via the cloud's abstraction for them). > > In contrast, the Service Catalog interface allows the *application* running > on Kubernetes to access features of the cloud. > > What does a service broker look like? > - > > A service broker provides an HTTP API with 5 actions: > > * List the services provided by the broker > * Create an instance of a resource > * Bind the resource into an instance of the application > * Unbind the resource from an instance of the application > * Delete the resource > > The binding step is used for things like providing a set of DB credentials to > a container. You can rotate credentials when replacing a
[openstack-dev] [all][sdk] Integrating OpenStack and k8s with a service broker
I've been doing some investigation into the Service Catalog in Kubernetes and how we can get OpenStack resources to show up in the catalog for use by applications running in Kubernetes. (The Big 3 public clouds already support this.) The short answer is via an implementation of something called the Open Service Broker API, but there are shortcuts available to make it easier to do. I'm convinced that this is readily achievable and something we ought to do as a community. I've put together a (long-winded) FAQ below to answer all of your questions about it. Would you be interested in working on a new project to implement this integration? Reply to this thread and let's collect a list of volunteers to form the initial core review team. cheers, Zane. What is the Open Service Broker API? The Open Service Broker API[1] is a standard way to expose external resources to applications running in a PaaS. It was originally developed in the context of CloudFoundry, but the same standard was adopted by Kubernetes (and hence OpenShift) in the form of the Service Catalog extension[2]. (The Service Catalog in Kubernetes is the component that calls out to a service broker.) So a single implementation can cover the most popular open-source PaaS offerings. In many cases, the services take the form of simply a pre-packaged application that also runs inside the PaaS. But they don't have to be - services can be anything. Provisioning via the service broker ensures that the services requested are tied in to the PaaS's orchestration of the application's lifecycle. (This is certainly not the be-all and end-all of integration between OpenStack and containers - we also need ways to tie PaaS-based applications into the OpenStack's orchestration of a larger group of resources. Some applications may even use both. But it's an important part of the story.) What sorts of services would OpenStack expose? -- Some example use cases might be: * The application needs a reliable message queue. Rather than spinning up multiple storage-backed containers with anti-affinity policies and dealing with the overhead of managing e.g. RabbitMQ, the application requests a Zaqar queue from an OpenStack cloud. The overhead of running the queueing service is amortised across all of the applications in the cloud. The queue gets cleaned up correctly when the application is removed, since it is tied into the application definition. * The application needs a database. Rather than spinning one up in a storage-backed container and dealing with the overhead of managing it, the application requests a Trove DB from an OpenStack cloud. * The application includes a service that needs to run on bare metal for performance reasons (e.g. could also be a database). The application requests a bare-metal server from Nova w/ Ironic for the purpose. (The same applies to requesting a VM, but there are alternatives like KubeVirt - which also operates through the Service Catalog - available for getting a VM in Kubernetes. There are no non-proprietary alternatives for getting a bare-metal server.) AWS[3], Azure[4], and GCP[5] all have service brokers available that support these and many more services that they provide. I don't know of any reason in principle not to expose every type of resource that OpenStack provides via a service broker. How is this different from cloud-provider-openstack? The Cloud Controller[6] interface in Kubernetes allows Kubernetes itself to access features of the cloud to provide its service. For example, if k8s needs persistent storage for a container then it can request that from Cinder through cloud-provider-openstack[7]. It can also request a load balancer from Octavia instead of having to start a container running HAProxy to load balance between multiple instances of an application container (thus enabling use of hardware load balancers via the cloud's abstraction for them). In contrast, the Service Catalog interface allows the *application* running on Kubernetes to access features of the cloud. What does a service broker look like? - A service broker provides an HTTP API with 5 actions: * List the services provided by the broker * Create an instance of a resource * Bind the resource into an instance of the application * Unbind the resource from an instance of the application * Delete the resource The binding step is used for things like providing a set of DB credentials to a container. You can rotate credentials when replacing a container by revoking the existing credentials on unbind and creating a new set on bind, without replacing the entire resource. Is there an easier way? --- Yes! Folks from OpenShift came up with a project called the Automation Broker[8]. To add
