[openstack-dev] [neutron][kuryr][magnum] Design Specification for Kuryr
On Fri, Aug 14, 2015 at 5:36 PM, Adrian Otto adrian.o...@rackspace.com wrote: That's clear, thanks Gal. The feature benchmark should be parity with how the majority of other (complete) remote drivers for libnetwork behave. From a Magnum perspective we value consistency from an end user perspective when you use containers on OpenStack compared to when you run them outside of an OpenStack cloud environment. When I previously evaluated the port mapping question, the most valid reasons I could see of why an operator that uses Neutron to provide container networking would want port mapping was: - density and availability of floating IPs, i.e. how many endpoints I could expose, which instead of being determined by the amount of free FIPs it would be #FIPs times ports. - current restrictions in the container orchestration software to use. If we offer extra features that's okay, but we do need to be careful not to to leave feature gaps where things hat work elsewhere don't work on OpenStack. Do we know if there are other libnetwork remote drivers that support port mapping, or have a statement of intent to implement it? That should help guide us here. Is it too early to know? I would imagine that those that have a network model in which the container networking is accessible/routable from the host networking namespace will find a way to accommodate the port mapping use case and will always use the host IP as the address to reach the containers it contains. The last status I saw for Docker's own overlay driver was that they did not have a way to provide port mapping and that one option that they had considered for software that depended on it was to have one veth on the overlay and one veth for port mapping. Under the Neutron networking implementations the networks in which the instances running in a compute node take part are not necessarily routable/reachable from the default host namespace. This is something that would break some assumptions made by container orchestration that was developed when there was only the typical docker0 bridge scenario. In Kubernetes, for example one would normally target the minions with external load balancers and then the port mapping would get the data from the host to the correct pod. In Neutron, to serve this case, the usual thing would be to use Neutron's LBaaS and assign a FIP to the LBaaS. Of course, that means that you get a whole FIP for a single service and that may be expensive. We have to work with the current bay model providers and see how many assumptions need to be accommodated (either by Neutron API usage patterns or extensions) and which could be smoothened out by means of plugins in the orchestrator. -- Adrian On Aug 14, 2015, at 8:09 AM, Gal Sagie gal.sa...@gmail.com wrote: Hi Adrian, Thanks for the explanation, i agree with you that we shouldn't break anything useful in docker, but from what i understand (and please correct me if i am wrong) you are describing an implementation detail of docker networking (at its default current state). Kuryr is not an implementation of containers networking, it is meant to allow docker networking to be constructed using Neutron plugins and solutions. For the point i was trying to make, lets take the simple case of connecting containers in a host (not the nested VM case), assuming we use with Kuryr the L2 OVS agent and L3 agent (Neutron reference implementation) and we are able to plug containers to a neutron network and define a floating ip to it, why would we need port mapping? (the iptables translation is already happening as we have NAT) Hope that make sense, and please correct me if i am saying nonsense or i didn't grasp the full use case of port mapping. But none the less, we will need to allow anything that docker allows and keep compatibility with all the available tooling that depends on it as you mention (and of course be flexible to use Kuryr in the same environment with other docker remote drivers) Gal On Fri, Aug 14, 2015 at 5:26 PM, Adrian Otto adrian.o...@rackspace.com wrote: The port mapping feature is the -p flag on the docker run command. It determines which ports in the network namespace of the container are exposed to the root namespace. It configures iptables rules and docker proxy capabilities to achieve the desired result. This feature is essential, so we must not break it. In other words, this feature is what allows a network port within a container to be externally accessed, and on what IP address(es) and port number(s) on the host. Example: docker run -d -p 12.34.56.7:8000:80 nginx:latest This runs the nginx container and exposes top port 80 from inside the container to tcp port 8000 on 12.34.56.7 on the host. Without this feature docker is rather useless for running network services unless you use -net host or an equivalent workaround. This could break a lot of tooling that depends on -p. -- Adrian On Aug 14,
Re: [openstack-dev] [neutron][kuryr][magnum] Design Specification for Kuryr
Hi feisky, I think thats a great question, not because of port-mapping in particular :) but because we need to think on a feature by feature basis and map all the features the dockers API allow which we cannot support directly with Neutron API or its services sub-projects API. (apuimedo, maybe we need to set this as a future task for us) But we also need to understand the use cases for supporting these API's so we can address them and give them priorities (and this is something we as a community need to decide how to handle). For your question, given that we have network isolation and security from neutron API's and given we have NAT support (by Neutron API and the plugins implementing the network) , what do you see as a use case to use the port-mapping ? I welcome you and everyone else to raise and describe these use cases so the Neutron/Kuryr community can think how to solve and help, and if needed also adjust or add extensions for support. Thanks Gal. On Fri, Aug 14, 2015 at 7:28 AM, feisky feisk...@gmail.com wrote: Will Kuryr supports docker's port-mapping? -- View this message in context: http://openstack.10931.n7.nabble.com/neutron-kuryr-magnum-Design-Specification-for-Kuryr-tp82256p82299.html Sent from the Developer mailing list archive at Nabble.com. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best Regards , The G. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][kuryr][magnum] Design Specification for Kuryr
The port mapping feature is the -p flag on the docker run command. It determines which ports in the network namespace of the container are exposed to the root namespace. It configures iptables rules and docker proxy capabilities to achieve the desired result. This feature is essential, so we must not break it. In other words, this feature is what allows a network port within a container to be externally accessed, and on what IP address(es) and port number(s) on the host. Example: docker run -d -p 12.34.56.7:8000:80 nginx:latest This runs the nginx container and exposes top port 80 from inside the container to tcp port 8000 on 12.34.56.7 on the host. Without this feature docker is rather useless for running network services unless you use -net host or an equivalent workaround. This could break a lot of tooling that depends on -p. -- Adrian On Aug 14, 2015, at 6:57 AM, Gal Sagie gal.sa...@gmail.commailto:gal.sa...@gmail.com wrote: Hi feisky, I think thats a great question, not because of port-mapping in particular :) but because we need to think on a feature by feature basis and map all the features the dockers API allow which we cannot support directly with Neutron API or its services sub-projects API. (apuimedo, maybe we need to set this as a future task for us) But we also need to understand the use cases for supporting these API's so we can address them and give them priorities (and this is something we as a community need to decide how to handle). For your question, given that we have network isolation and security from neutron API's and given we have NAT support (by Neutron API and the plugins implementing the network) , what do you see as a use case to use the port-mapping ? I welcome you and everyone else to raise and describe these use cases so the Neutron/Kuryr community can think how to solve and help, and if needed also adjust or add extensions for support. Thanks Gal. On Fri, Aug 14, 2015 at 7:28 AM, feisky feisk...@gmail.commailto:feisk...@gmail.com wrote: Will Kuryr supports docker's port-mapping? -- View this message in context: http://openstack.10931.n7.nabble.com/neutron-kuryr-magnum-Design-Specification-for-Kuryr-tp82256p82299.html Sent from the Developer mailing list archive at Nabble.comhttp://Nabble.com. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best Regards , The G. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.orgmailto:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][kuryr][magnum] Design Specification for Kuryr
Hi Adrian, Thanks for the explanation, i agree with you that we shouldn't break anything useful in docker, but from what i understand (and please correct me if i am wrong) you are describing an implementation detail of docker networking (at its default current state). Kuryr is not an implementation of containers networking, it is meant to allow docker networking to be constructed using Neutron plugins and solutions. For the point i was trying to make, lets take the simple case of connecting containers in a host (not the nested VM case), assuming we use with Kuryr the L2 OVS agent and L3 agent (Neutron reference implementation) and we are able to plug containers to a neutron network and define a floating ip to it, why would we need port mapping? (the iptables translation is already happening as we have NAT) Hope that make sense, and please correct me if i am saying nonsense or i didn't grasp the full use case of port mapping. But none the less, we will need to allow anything that docker allows and keep compatibility with all the available tooling that depends on it as you mention (and of course be flexible to use Kuryr in the same environment with other docker remote drivers) Gal On Fri, Aug 14, 2015 at 5:26 PM, Adrian Otto adrian.o...@rackspace.com wrote: The port mapping feature is the -p flag on the docker run command. It determines which ports in the network namespace of the container are exposed to the root namespace. It configures iptables rules and docker proxy capabilities to achieve the desired result. This feature is essential, so we must not break it. In other words, this feature is what allows a network port within a container to be externally accessed, and on what IP address(es) and port number(s) on the host. Example: docker run -d -p 12.34.56.7:8000:80 nginx:latest This runs the nginx container and exposes top port 80 from inside the container to tcp port 8000 on 12.34.56.7 on the host. Without this feature docker is rather useless for running network services unless you use -net host or an equivalent workaround. This could break a lot of tooling that depends on -p. -- Adrian On Aug 14, 2015, at 6:57 AM, Gal Sagie gal.sa...@gmail.com wrote: Hi feisky, I think thats a great question, not because of port-mapping in particular :) but because we need to think on a feature by feature basis and map all the features the dockers API allow which we cannot support directly with Neutron API or its services sub-projects API. (apuimedo, maybe we need to set this as a future task for us) But we also need to understand the use cases for supporting these API's so we can address them and give them priorities (and this is something we as a community need to decide how to handle). For your question, given that we have network isolation and security from neutron API's and given we have NAT support (by Neutron API and the plugins implementing the network) , what do you see as a use case to use the port-mapping ? I welcome you and everyone else to raise and describe these use cases so the Neutron/Kuryr community can think how to solve and help, and if needed also adjust or add extensions for support. Thanks Gal. On Fri, Aug 14, 2015 at 7:28 AM, feisky feisk...@gmail.com wrote: Will Kuryr supports docker's port-mapping? -- View this message in context: http://openstack.10931.n7.nabble.com/neutron-kuryr-magnum-Design-Specification-for-Kuryr-tp82256p82299.html Sent from the Developer mailing list archive at Nabble.com. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best Regards , The G. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best Regards , The G. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][kuryr][magnum] Design Specification for Kuryr
That's clear, thanks Gal. The feature benchmark should be parity with how the majority of other (complete) remote drivers for libnetwork behave. From a Magnum perspective we value consistency from an end user perspective when you use containers on OpenStack compared to when you run them outside of an OpenStack cloud environment. If we offer extra features that's okay, but we do need to be careful not to to leave feature gaps where things hat work elsewhere don't work on OpenStack. Do we know if there are other libnetwork remote drivers that support port mapping, or have a statement of intent to implement it? That should help guide us here. Is it too early to know? -- Adrian On Aug 14, 2015, at 8:09 AM, Gal Sagie gal.sa...@gmail.commailto:gal.sa...@gmail.com wrote: Hi Adrian, Thanks for the explanation, i agree with you that we shouldn't break anything useful in docker, but from what i understand (and please correct me if i am wrong) you are describing an implementation detail of docker networking (at its default current state). Kuryr is not an implementation of containers networking, it is meant to allow docker networking to be constructed using Neutron plugins and solutions. For the point i was trying to make, lets take the simple case of connecting containers in a host (not the nested VM case), assuming we use with Kuryr the L2 OVS agent and L3 agent (Neutron reference implementation) and we are able to plug containers to a neutron network and define a floating ip to it, why would we need port mapping? (the iptables translation is already happening as we have NAT) Hope that make sense, and please correct me if i am saying nonsense or i didn't grasp the full use case of port mapping. But none the less, we will need to allow anything that docker allows and keep compatibility with all the available tooling that depends on it as you mention (and of course be flexible to use Kuryr in the same environment with other docker remote drivers) Gal On Fri, Aug 14, 2015 at 5:26 PM, Adrian Otto adrian.o...@rackspace.commailto:adrian.o...@rackspace.com wrote: The port mapping feature is the -p flag on the docker run command. It determines which ports in the network namespace of the container are exposed to the root namespace. It configures iptables rules and docker proxy capabilities to achieve the desired result. This feature is essential, so we must not break it. In other words, this feature is what allows a network port within a container to be externally accessed, and on what IP address(es) and port number(s) on the host. Example: docker run -d -p 12.34.56.7:8000:80 nginx:latest This runs the nginx container and exposes top port 80 from inside the container to tcp port 8000 on 12.34.56.7 on the host. Without this feature docker is rather useless for running network services unless you use -net host or an equivalent workaround. This could break a lot of tooling that depends on -p. -- Adrian On Aug 14, 2015, at 6:57 AM, Gal Sagie gal.sa...@gmail.commailto:gal.sa...@gmail.com wrote: Hi feisky, I think thats a great question, not because of port-mapping in particular :) but because we need to think on a feature by feature basis and map all the features the dockers API allow which we cannot support directly with Neutron API or its services sub-projects API. (apuimedo, maybe we need to set this as a future task for us) But we also need to understand the use cases for supporting these API's so we can address them and give them priorities (and this is something we as a community need to decide how to handle). For your question, given that we have network isolation and security from neutron API's and given we have NAT support (by Neutron API and the plugins implementing the network) , what do you see as a use case to use the port-mapping ? I welcome you and everyone else to raise and describe these use cases so the Neutron/Kuryr community can think how to solve and help, and if needed also adjust or add extensions for support. Thanks Gal. On Fri, Aug 14, 2015 at 7:28 AM, feisky feisk...@gmail.commailto:feisk...@gmail.com wrote: Will Kuryr supports docker's port-mapping? -- View this message in context: http://openstack.10931.n7.nabble.com/neutron-kuryr-magnum-Design-Specification-for-Kuryr-tp82256p82299.html Sent from the Developer mailing list archive at Nabble.comhttp://Nabble.com. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best Regards , The G. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe:
Re: [openstack-dev] [neutron][kuryr][magnum] Design Specification for Kuryr
I would suggest considering docker port mapping as a creation of an apropriate security group's rule and also creation of something which was never exist in Neutron before which will be responsible for port forwarding. This feature could be useful for VMs also instead of/jointly with using floating IPs. What do you think? On Aug 14, 2015 6:12 PM, Gal Sagie gal.sa...@gmail.com wrote: Hi Adrian, Thanks for the explanation, i agree with you that we shouldn't break anything useful in docker, but from what i understand (and please correct me if i am wrong) you are describing an implementation detail of docker networking (at its default current state). Kuryr is not an implementation of containers networking, it is meant to allow docker networking to be constructed using Neutron plugins and solutions. For the point i was trying to make, lets take the simple case of connecting containers in a host (not the nested VM case), assuming we use with Kuryr the L2 OVS agent and L3 agent (Neutron reference implementation) and we are able to plug containers to a neutron network and define a floating ip to it, why would we need port mapping? (the iptables translation is already happening as we have NAT) Hope that make sense, and please correct me if i am saying nonsense or i didn't grasp the full use case of port mapping. But none the less, we will need to allow anything that docker allows and keep compatibility with all the available tooling that depends on it as you mention (and of course be flexible to use Kuryr in the same environment with other docker remote drivers) Gal On Fri, Aug 14, 2015 at 5:26 PM, Adrian Otto adrian.o...@rackspace.com wrote: The port mapping feature is the -p flag on the docker run command. It determines which ports in the network namespace of the container are exposed to the root namespace. It configures iptables rules and docker proxy capabilities to achieve the desired result. This feature is essential, so we must not break it. In other words, this feature is what allows a network port within a container to be externally accessed, and on what IP address(es) and port number(s) on the host. Example: docker run -d -p 12.34.56.7:8000:80 nginx:latest This runs the nginx container and exposes top port 80 from inside the container to tcp port 8000 on 12.34.56.7 on the host. Without this feature docker is rather useless for running network services unless you use -net host or an equivalent workaround. This could break a lot of tooling that depends on -p. -- Adrian On Aug 14, 2015, at 6:57 AM, Gal Sagie gal.sa...@gmail.com wrote: Hi feisky, I think thats a great question, not because of port-mapping in particular :) but because we need to think on a feature by feature basis and map all the features the dockers API allow which we cannot support directly with Neutron API or its services sub-projects API. (apuimedo, maybe we need to set this as a future task for us) But we also need to understand the use cases for supporting these API's so we can address them and give them priorities (and this is something we as a community need to decide how to handle). For your question, given that we have network isolation and security from neutron API's and given we have NAT support (by Neutron API and the plugins implementing the network) , what do you see as a use case to use the port-mapping ? I welcome you and everyone else to raise and describe these use cases so the Neutron/Kuryr community can think how to solve and help, and if needed also adjust or add extensions for support. Thanks Gal. On Fri, Aug 14, 2015 at 7:28 AM, feisky feisk...@gmail.com wrote: Will Kuryr supports docker's port-mapping? -- View this message in context: http://openstack.10931.n7.nabble.com/neutron-kuryr-magnum-Design-Specification-for-Kuryr-tp82256p82299.html Sent from the Developer mailing list archive at Nabble.com. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best Regards , The G. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org ?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best Regards , The G. __ OpenStack Development Mailing List (not for
[openstack-dev] [neutron][kuryr][magnum] Design Specification for Kuryr
Kyle, Can we please arrange for an official design specification for Kuyyr so members of the Magnum team can relay input to be sure that our mutual interests in this work are addressed? Thanks, Adrian __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][kuryr][magnum] Design Specification for Kuryr
On Thu, Aug 13, 2015 at 1:56 PM, Adrian Otto adrian.o...@rackspace.com wrote: Kyle, Can we please arrange for an official design specification for Kuyyr so members of the Magnum team can relay input to be sure that our mutual interests in this work are addressed? I agree 100%. See Gals email here [1] for a status update. At this point the team is collecting ideas in an etherpad. Gal, can I request that you formulate those into a spec of some sort, it makes sense to file that in the Neutron spec repo at this point. We can all then review it there. Thanks! Kyle [1] http://lists.openstack.org/pipermail/openstack-dev/2015-August/071940.html Thanks, Adrian __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][kuryr][magnum] Design Specification for Kuryr
Will Kuryr supports docker's port-mapping? -- View this message in context: http://openstack.10931.n7.nabble.com/neutron-kuryr-magnum-Design-Specification-for-Kuryr-tp82256p82299.html Sent from the Developer mailing list archive at Nabble.com. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][kuryr][magnum] Design Specification for Kuryr
Sure, will have it ready for review next week and put the link here for everyone. On Thu, Aug 13, 2015 at 10:17 PM, Kyle Mestery mest...@mestery.com wrote: On Thu, Aug 13, 2015 at 1:56 PM, Adrian Otto adrian.o...@rackspace.com wrote: Kyle, Can we please arrange for an official design specification for Kuyyr so members of the Magnum team can relay input to be sure that our mutual interests in this work are addressed? I agree 100%. See Gals email here [1] for a status update. At this point the team is collecting ideas in an etherpad. Gal, can I request that you formulate those into a spec of some sort, it makes sense to file that in the Neutron spec repo at this point. We can all then review it there. Thanks! Kyle [1] http://lists.openstack.org/pipermail/openstack-dev/2015-August/071940.html Thanks, Adrian __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best Regards , The G. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
