Re: [openstack-dev] [magnum] Nesting /containers resource under /bays
The requirements that running a fully containerized application optimally & effectively requires the usage of a dedicated COE tool such as Swarm, Kubernetes or Marathon+Mesos. OpenStack is better suited for managing the underlying infrastructure. Mike Metral Product Architect – Private Cloud R&D email: mike.met...@rackspace.com<mailto:mike.met...@rackspace.com> cell: +1-305-282-7606 From: Hongbin Lu Reply-To: "OpenStack Development Mailing List (not for usage questions)" Date: Friday, January 15, 2016 at 8:02 PM To: "OpenStack Development Mailing List (not for usage questions)" Subject: Re: [openstack-dev] [magnum] Nesting /containers resource under /bays A reason is the container abstraction brings containers to OpenStack: Keystone for authentication, Heat for orchestration, Horizon for UI, etc. From: Kyle Kelley [mailto:rgb...@gmail.com] Sent: January-15-16 10:42 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [magnum] Nesting /containers resource under /bays What are the reasons for keeping /containers? On Fri, Jan 15, 2016 at 9:14 PM, Hongbin Lu mailto:hongbin...@huawei.com>> wrote: Disagree. If the container managing part is removed, Magnum is just a COE deployment tool. This is really a scope-mismatch IMO. The middle ground I can see is to have a flag that allows operators to turned off the container managing part. If it is turned off, COEs are not managed by Magnum and requests sent to the /container endpoint will return a reasonable error code. Thoughts? Best regards, Hongbin From: Mike Metral [mailto:mike.met...@rackspace.com<mailto:mike.met...@rackspace.com>] Sent: January-15-16 6:24 PM To: openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org> Subject: Re: [openstack-dev] [magnum] Nesting /containers resource under /bays I too believe that the /containers endpoint is obstructive to the overall goal of Magnum. IMO, Magnum’s scope should only be concerned with: 1. Provisioning the underlying infrastructure required by the Container Orchestration Engine (COE) and 2. Instantiating the COE itself on top of said infrastructure from step #1. Anything further regarding Magnum interfacing or interacting with containers starts to get into a gray area that could easily evolve into: * Potential race conditions between Magnum and the designated COE and * Would create design & implementation overhead and debt that could bite us in the long run seeing how all COE’s operate & are based off various different paradigms in terms of describing & managing containers, and this divergence will only continue to grow with time. * Not to mention, the recreation of functionality around managing containers in Magnum seems redundant in nature as this is the very reason to want to use a COE in the first place – because it’s a more suited tool for the task If there is low-hanging fruit in terms of common functionality across all COE’s, then those generic capabilities could be abstracted and integrated into Magnum, but these have to be carefully examined beforehand to ensure true parity exists for the capability across all COE’s. However, I still worry that going down this route toes the line that Magnum should and could be a part of the managing container story to some degree – which again should be the sole responsibility of the COE, not Magnum. I’m in favor of doing away with the /containers endpoint – continuing with it just looks like a snowball of scope-mismatch and management issues just waiting to happen. Mike Metral Product Architect – Private Cloud R&D - Rackspace From: Hongbin Lu mailto:hongbin...@huawei.com>> Sent: Thursday, January 14, 2016 1:59 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [magnum] Nesting /containers resource under /bays In short, the container IDs assigned by Magnum are independent of the container IDs assigned by Docker daemon. Magnum do the IDs mapping before doing a native API call. In particular, here is how it works. If users create a container through Magnum endpoint, Magnum will do the followings: 1. Generate a uuid (if not provided). 2. Call Docker Swarm API to create a container, with its hostname equal to the generated uuid. 3. Persist container to DB with the generated uuid. If users perform an operation on an existing container, they must provide the uuid (or the name) of the container (if name is provided, it will be used to lookup the uuid). Magnum will do the followings: 1. Call Docker Swarm API to list all containers. 2. Find the container whose hostname is equal to the provided uuid, record its “docker_id” that is the ID assigned by native tool. 3. Call Docker Swarm API with “docker_id” to perform the operation. Magnum doesn’t assume all o
Re: [openstack-dev] [magnum] Nesting /containers resource under /bays
I too believe that the /containers endpoint is obstructive to the overall goal of Magnum. IMO, Magnum’s scope should only be concerned with: 1. Provisioning the underlying infrastructure required by the Container Orchestration Engine (COE) and 2. Instantiating the COE itself on top of said infrastructure from step #1. Anything further regarding Magnum interfacing or interacting with containers starts to get into a gray area that could easily evolve into: * Potential race conditions between Magnum and the designated COE and * Would create design & implementation overhead and debt that could bite us in the long run seeing how all COE’s operate & are based off various different paradigms in terms of describing & managing containers, and this divergence will only continue to grow with time. * Not to mention, the recreation of functionality around managing containers in Magnum seems redundant in nature as this is the very reason to want to use a COE in the first place – because it’s a more suited tool for the task If there is low-hanging fruit in terms of common functionality across all COE’s, then those generic capabilities could be abstracted and integrated into Magnum, but these have to be carefully examined beforehand to ensure true parity exists for the capability across all COE’s. However, I still worry that going down this route toes the line that Magnum should and could be a part of the managing container story to some degree – which again should be the sole responsibility of the COE, not Magnum. I’m in favor of doing away with the /containers endpoint – continuing with it just looks like a snowball of scope-mismatch and management issues just waiting to happen. Mike Metral Product Architect – Private Cloud R&D - Rackspace From: Hongbin Lu mailto:hongbin...@huawei.com>> Sent: Thursday, January 14, 2016 1:59 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [magnum] Nesting /containers resource under /bays In short, the container IDs assigned by Magnum are independent of the container IDs assigned by Docker daemon. Magnum do the IDs mapping before doing a native API call. In particular, here is how it works. If users create a container through Magnum endpoint, Magnum will do the followings: 1. Generate a uuid (if not provided). 2. Call Docker Swarm API to create a container, with its hostname equal to the generated uuid. 3. Persist container to DB with the generated uuid. If users perform an operation on an existing container, they must provide the uuid (or the name) of the container (if name is provided, it will be used to lookup the uuid). Magnum will do the followings: 1. Call Docker Swarm API to list all containers. 2. Find the container whose hostname is equal to the provided uuid, record its “docker_id” that is the ID assigned by native tool. 3. Call Docker Swarm API with “docker_id” to perform the operation. Magnum doesn’t assume all operations to be routed through Magnum endpoints. Alternatively, users can directly call the native APIs. In this case, the created resources are not managed by Magnum and won’t be accessible through Magnum’s endpoints. Hope it is clear. Best regards, Hongbin From: Kyle Kelley [mailto:kyle.kel...@rackspace.com] Sent: January-14-16 11:39 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [magnum] Nesting /containers resource under /bays This presumes a model where Magnum is in complete control of the IDs of individual containers. How does this work with the Docker daemon? > In Rest API, you can set the “uuid” field in the json request body (this is > not supported in CLI, but it is an easy add). In the Rest API for Magnum or Docker? Has Magnum completely broken away from exposing native tooling - are all container operations assumed to be routed through Magnum endpoints? > For the idea of nesting container resource, I prefer not to do that if there > are alternatives or it can be work around. IMO, it sets a limitation that a > container must have a bay, which might not be the case in future. For > example, we might add a feature that creating a container will automatically > create a bay. If a container must have a bay on creation, such feature is > impossible. If that's *really* a feature you need and are fully involved in designing for, this seems like a case where creating a container via these endpoints would create a bay and return the full resource+subresource. Personally, I think these COE endpoints need to not be in the main spec, to reduce the surface area until these are put into further use. From: Hongbin Lu mailto:hongbin...@huawei.com>> Sent: Wednesday, January 13, 2016 5:00 PM To: OpenStack Development Mailing L
