Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
- Original Message - > From: "Kai Qiang Wu" <wk...@cn.ibm.com> > To: "OpenStack Development Mailing List (not for usage questions)" > <openstack-dev@lists.openstack.org> > Sent: Tuesday, March 15, 2016 3:20:46 PM > Subject: Re: [openstack-dev] [magnum] Discussion of supporting > single/multiple OS distro > > Hi Stdake, > > There is a patch about Atomic 23 support in Magnum. And atomic 23 uses > kubernetes 1.0.6, and docker 1.9.1. > From Steve Gordon, I learnt they did have a two-weekly release. To me it > seems each atomic 23 release not much difference, (minor change) > The major rebases/updates may still have to wait for e.g. Fedora Atomic 24. Well, the emphasis here is on *may*. As was pointed out in that same thread [1] rebases certainly can occur although those builds need to get karma in the fedora build system to be pushed into updates and subsequently included in the next rebuild (e.g. see [2] for a newer K8S build). The main point is that if a rebase involves introducing some element of backwards incompatibility then that would have to wait to the next major (F24) - outside of that there is some flexibility. > So maybe we not need to test every Atomic 23 two-weekly. > Pick one or update old, when we find it is integrated with new kubernetes > or docker, etcd etc. If other small changes(not include security), seems > not need to update so frequently, it can save some efforts. A question I have posed before and that I think will need to be answered if Magnum is indeed to move towards the model for handling drivers proposed in this thread is what are the expectations Magnum has for each image/coe combination in terms of versions of key components for a given Magnum release, and what are the expectations Magnum has for same when looking forwards to say Newton. Based on our discussion it seemed like there were some issues that mean kubernetes-1.1.0 would be preferable for example (although that it wasn't there was in fact itself a bug it would seem, but regardless it's a valid example), but is that expectation documented somewhere? It seems like based on feature roadmap it should be possible to at least put forward minimum required versions for key components (e.g. docker, k8s, flanel, etcd for the K8S COE)? This would make it easier to guide the relevant upstreams to ensure their images support the Magnum team's needs and at least minimize the need to do custom builds if not eliminate it. -Steve [1] https://lists.fedoraproject.org/archives/list/cl...@lists.fedoraproject.org/thread/ZJARDKSB3KGMKLACCZSQALZHV54PAJUB/ [2] https://bodhi.fedoraproject.org/updates/FEDORA-2016-a89f5ce5f4 > From: "Steven Dake (stdake)" <std...@cisco.com> > To: "OpenStack Development Mailing List (not for usage questions)" > <openstack-dev@lists.openstack.org> > Date: 16/03/2016 03:23 am > Subject: Re: [openstack-dev] [magnum] Discussion of supporting > single/multiple OS distro > > > > WFM as long as we stick to the spirit of the proposal and don't end up in a > situation where there is only one distribution. Others in the thread had > indicated there would be only one distribution in tree, which I'd find > disturbing for reasons already described on this thread. > > While we are about it, we should move to the latest version of atomic and > chase atomic every two weeks on their release. Thoughts? > > Regards > -steve > > > From: Hongbin Lu <hongbin...@huawei.com> > Reply-To: "OpenStack Development Mailing List (not for usage questions)" < > openstack-dev@lists.openstack.org> > Date: Monday, March 14, 2016 at 8:10 PM > To: "OpenStack Development Mailing List (not for usage questions)" < > openstack-dev@lists.openstack.org> > Subject: Re: [openstack-dev] [magnum] Discussion of supporting > single/multiple OS distro > > > > From: Adrian Otto [mailto:adrian.o...@rackspace.com] > Sent: March-14-16 4:49 PM > To: OpenStack Development Mailing List (not for usage > questions) > Subject: Re: [openstack-dev] [magnum] Discussion of supporting > single/multiple OS distro > > Steve, > > I think you may have misunderstood our intent here. We are not > seeking to lock in to a single OS vendor. Each COE driver can > have a different OS. We can have multiple drivers per COE. The > point is that drivers should be simple, and therefore should > support one Bay node OS each. That would mean taking what we > have today in our Kubernetes Bay type implementation and > breaking it down into two driver
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
HI Steve, Some points to highlight here: 1> There are some work discussion about COE dynamic supports across different OS distro. 2> For atomic, we did have many requirements before, it was an old story, seem some not meet our needs (which once asked in atomic IRC channel or community) So we built some images by ourselves. But if atomic community could provide related support, it would more beneficial for both( as we use it, it would be tested by us daily jenkins and developers) Maybe for the requirements, need some clear channel, like: 1> What's the official channel to open requirements to Atomic community ? Is it github or something else which can easily track ? 2> What's the normal process to deal with such requirements, and coordinate ways. 3> Others Thanks Best Wishes, Kai Qiang Wu (吴开强 Kennan) IBM China System and Technology Lab, Beijing E-mail: wk...@cn.ibm.com Tel: 86-10-82451647 Address: Building 28(Ring Building), ZhongGuanCun Software Park, No.8 Dong Bei Wang West Road, Haidian District Beijing P.R.China 100193 Follow your heart. You are miracle! From: Steve Gordon <sgor...@redhat.com> To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org> Date: 17/03/2016 09:24 pm Subject:Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro - Original Message - > From: "Kai Qiang Wu" <wk...@cn.ibm.com> > To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org> > Sent: Tuesday, March 15, 2016 3:20:46 PM > Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro > > Hi Stdake, > > There is a patch about Atomic 23 support in Magnum. And atomic 23 uses > kubernetes 1.0.6, and docker 1.9.1. > From Steve Gordon, I learnt they did have a two-weekly release. To me it > seems each atomic 23 release not much difference, (minor change) > The major rebases/updates may still have to wait for e.g. Fedora Atomic 24. Well, the emphasis here is on *may*. As was pointed out in that same thread [1] rebases certainly can occur although those builds need to get karma in the fedora build system to be pushed into updates and subsequently included in the next rebuild (e.g. see [2] for a newer K8S build). The main point is that if a rebase involves introducing some element of backwards incompatibility then that would have to wait to the next major (F24) - outside of that there is some flexibility. > So maybe we not need to test every Atomic 23 two-weekly. > Pick one or update old, when we find it is integrated with new kubernetes > or docker, etcd etc. If other small changes(not include security), seems > not need to update so frequently, it can save some efforts. A question I have posed before and that I think will need to be answered if Magnum is indeed to move towards the model for handling drivers proposed in this thread is what are the expectations Magnum has for each image/coe combination in terms of versions of key components for a given Magnum release, and what are the expectations Magnum has for same when looking forwards to say Newton. Based on our discussion it seemed like there were some issues that mean kubernetes-1.1.0 would be preferable for example (although that it wasn't there was in fact itself a bug it would seem, but regardless it's a valid example), but is that expectation documented somewhere? It seems like based on feature roadmap it should be possible to at least put forward minimum required versions for key components (e.g. docker, k8s, flanel, etcd for the K8S COE)? This would make it easier to guide the relevant upstreams to ensure their images support the Magnum team's needs and at least minimize the need to do custom builds if not eliminate it. -Steve [1] https://lists.fedoraproject.org/archives/list/cl...@lists.fedoraproject.org/thread/ZJARDKSB3KGMKLACCZSQALZHV54PAJUB/ [2] https://bodhi.fedoraproject.org/updates/FEDORA-2016-a89f5ce5f4 > From: "Steven Dake (stdake)" <std...@cisco.com> > To:"OpenStack Development Mailing List (not for usage questions)" > <openstack-dev@lists.openstack.org> > Date: 16/03/2016 03:23 am > Subject: Re: [openstack-dev] [magnum] Discussion of supporting > single/multiple OS distro > > > > WFM as long as we stick to the spirit of the proposal and don't end up in a > situation where there is only one distribution. Others in the thread had > indicated there would be only one distribution in tree, which I'd find > distu
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
Hi Stdake, There is a patch about Atomic 23 support in Magnum. And atomic 23 uses kubernetes 1.0.6, and docker 1.9.1. >From Steve Gordon, I learnt they did have a two-weekly release. To me it seems each atomic 23 release not much difference, (minor change) The major rebases/updates may still have to wait for e.g. Fedora Atomic 24. So maybe we not need to test every Atomic 23 two-weekly. Pick one or update old, when we find it is integrated with new kubernetes or docker, etcd etc. If other small changes(not include security), seems not need to update so frequently, it can save some efforts. What do you think ? Thanks Best Wishes, Kai Qiang Wu (吴开强 Kennan) IBM China System and Technology Lab, Beijing E-mail: wk...@cn.ibm.com Tel: 86-10-82451647 Address: Building 28(Ring Building), ZhongGuanCun Software Park, No.8 Dong Bei Wang West Road, Haidian District Beijing P.R.China 100193 Follow your heart. You are miracle! From: "Steven Dake (stdake)" <std...@cisco.com> To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org> Date: 16/03/2016 03:23 am Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro WFM as long as we stick to the spirit of the proposal and don't end up in a situation where there is only one distribution. Others in the thread had indicated there would be only one distribution in tree, which I'd find disturbing for reasons already described on this thread. While we are about it, we should move to the latest version of atomic and chase atomic every two weeks on their release. Thoughts? Regards -steve From: Hongbin Lu <hongbin...@huawei.com> Reply-To: "OpenStack Development Mailing List (not for usage questions)" < openstack-dev@lists.openstack.org> Date: Monday, March 14, 2016 at 8:10 PM To: "OpenStack Development Mailing List (not for usage questions)" < openstack-dev@lists.openstack.org> Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro From: Adrian Otto [mailto:adrian.o...@rackspace.com] Sent: March-14-16 4:49 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro Steve, I think you may have misunderstood our intent here. We are not seeking to lock in to a single OS vendor. Each COE driver can have a different OS. We can have multiple drivers per COE. The point is that drivers should be simple, and therefore should support one Bay node OS each. That would mean taking what we have today in our Kubernetes Bay type implementation and breaking it down into two drivers: one for CoreOS and another for Fedora/Atomic. New drivers would start out in a contrib directory where complete functional testing would not be required. In order to graduate one out of contrib and into the realm of support of the Magnum dev team, it would need to have a full set of tests, and someone actively maintaining it. OK. It sounds like the proposal allows more than one OS to be in-tree, as long as the second OS goes through an incubation process. If that is what you mean, it sounds reasonable to me. Multi-personality driers would be relatively complex. That approach would slow down COE specific feature development, and complicate maintenance that is needed as new versions of the dependency chain are bundled in (docker, k8s, etcd, etc.). We have all agreed that having integration points that allow for alternate OS selection is still our direction. This follows the pattern that we set previously when deciding what networking options to support. We will have one that’s included as a default, and a way to plug in alternates. Here is what I expect to see when COE drivers are implemented: Docker Swarm: Default driver Fedora/Atomic Alternate driver: TBD Kubernetes: Default driver Fedora/Atomic Alternate driver: CoreOS Apache Mesos/Marathon: Default driver: Ubuntu Alternate driver: TBD We can allow an arbitrary number of alternates. Those TBD items can be initially added to the contrib directory, and with the right level of community support can be advanced to defaults if shown to wor
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
WFM as long as we stick to the spirit of the proposal and don't end up in a situation where there is only one distribution. Others in the thread had indicated there would be only one distribution in tree, which I'd find disturbing for reasons already described on this thread. While we are about it, we should move to the latest version of atomic and chase atomic every two weeks on their release. Thoughts? Regards -steve From: Hongbin Lu <hongbin...@huawei.com<mailto:hongbin...@huawei.com>> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Date: Monday, March 14, 2016 at 8:10 PM To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro From: Adrian Otto [mailto:adrian.o...@rackspace.com] Sent: March-14-16 4:49 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro Steve, I think you may have misunderstood our intent here. We are not seeking to lock in to a single OS vendor. Each COE driver can have a different OS. We can have multiple drivers per COE. The point is that drivers should be simple, and therefore should support one Bay node OS each. That would mean taking what we have today in our Kubernetes Bay type implementation and breaking it down into two drivers: one for CoreOS and another for Fedora/Atomic. New drivers would start out in a contrib directory where complete functional testing would not be required. In order to graduate one out of contrib and into the realm of support of the Magnum dev team, it would need to have a full set of tests, and someone actively maintaining it. OK. It sounds like the proposal allows more than one OS to be in-tree, as long as the second OS goes through an incubation process. If that is what you mean, it sounds reasonable to me. Multi-personality driers would be relatively complex. That approach would slow down COE specific feature development, and complicate maintenance that is needed as new versions of the dependency chain are bundled in (docker, k8s, etcd, etc.). We have all agreed that having integration points that allow for alternate OS selection is still our direction. This follows the pattern that we set previously when deciding what networking options to support. We will have one that’s included as a default, and a way to plug in alternates. Here is what I expect to see when COE drivers are implemented: Docker Swarm: Default driver Fedora/Atomic Alternate driver: TBD Kubernetes: Default driver Fedora/Atomic Alternate driver: CoreOS Apache Mesos/Marathon: Default driver: Ubuntu Alternate driver: TBD We can allow an arbitrary number of alternates. Those TBD items can be initially added to the contrib directory, and with the right level of community support can be advanced to defaults if shown to work better, be more straightforward to maintain, be more secure, or whatever criteria is important to us when presented with the choice. Such criteria will be subject to community consensus. This should allow for free experimentation with alternates to allow for innovation. See how this is not locking in a single OS vendor? Adrian On Mar 14, 2016, at 12:41 PM, Steven Dake (stdake) <std...@cisco.com<mailto:std...@cisco.com>> wrote: Hongbin, When we are at a disagreement in the Kolla core team, we have the Kolla core reviewers vote on the matter. This is typical standard OpenStack best practice. I think the vote would be something like "Implement one OS/COE/network/storage prototype, or implement many." I don't have a horse in this race, but I think it would be seriously damaging to Magnum to lock in to a single vendor. Regards -steve From: Hongbin Lu <hongbin...@huawei.com<mailto:hongbin...@huawei.com>> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Date: Monday, March 7, 2016 at 10:06 AM To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro From: Corey O'Brien [mailto:coreypobr...@gmail.com] Sent: March-07-16 8:11 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro Hongbin, I think the offer to support different OS options is a perfect example both of what we want and what we don't want. We definitely
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
From: Adrian Otto [mailto:adrian.o...@rackspace.com] Sent: March-14-16 4:49 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro Steve, I think you may have misunderstood our intent here. We are not seeking to lock in to a single OS vendor. Each COE driver can have a different OS. We can have multiple drivers per COE. The point is that drivers should be simple, and therefore should support one Bay node OS each. That would mean taking what we have today in our Kubernetes Bay type implementation and breaking it down into two drivers: one for CoreOS and another for Fedora/Atomic. New drivers would start out in a contrib directory where complete functional testing would not be required. In order to graduate one out of contrib and into the realm of support of the Magnum dev team, it would need to have a full set of tests, and someone actively maintaining it. OK. It sounds like the proposal allows more than one OS to be in-tree, as long as the second OS goes through an incubation process. If that is what you mean, it sounds reasonable to me. Multi-personality driers would be relatively complex. That approach would slow down COE specific feature development, and complicate maintenance that is needed as new versions of the dependency chain are bundled in (docker, k8s, etcd, etc.). We have all agreed that having integration points that allow for alternate OS selection is still our direction. This follows the pattern that we set previously when deciding what networking options to support. We will have one that's included as a default, and a way to plug in alternates. Here is what I expect to see when COE drivers are implemented: Docker Swarm: Default driver Fedora/Atomic Alternate driver: TBD Kubernetes: Default driver Fedora/Atomic Alternate driver: CoreOS Apache Mesos/Marathon: Default driver: Ubuntu Alternate driver: TBD We can allow an arbitrary number of alternates. Those TBD items can be initially added to the contrib directory, and with the right level of community support can be advanced to defaults if shown to work better, be more straightforward to maintain, be more secure, or whatever criteria is important to us when presented with the choice. Such criteria will be subject to community consensus. This should allow for free experimentation with alternates to allow for innovation. See how this is not locking in a single OS vendor? Adrian On Mar 14, 2016, at 12:41 PM, Steven Dake (stdake) <std...@cisco.com<mailto:std...@cisco.com>> wrote: Hongbin, When we are at a disagreement in the Kolla core team, we have the Kolla core reviewers vote on the matter. This is typical standard OpenStack best practice. I think the vote would be something like "Implement one OS/COE/network/storage prototype, or implement many." I don't have a horse in this race, but I think it would be seriously damaging to Magnum to lock in to a single vendor. Regards -steve From: Hongbin Lu <hongbin...@huawei.com<mailto:hongbin...@huawei.com>> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Date: Monday, March 7, 2016 at 10:06 AM To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro From: Corey O'Brien [mailto:coreypobr...@gmail.com] Sent: March-07-16 8:11 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro Hongbin, I think the offer to support different OS options is a perfect example both of what we want and what we don't want. We definitely want to allow for someone like yourself to maintain templates for whatever OS they want and to have that option be easily integrated in to a Magnum deployment. However, when developing features or bug fixes, we can't wait for you to have time to add it for whatever OS you are promising to maintain. It might be true that supporting additional OS could slow down the development speed, but the key question is how much the impact will be. Does it outweigh the benefits? IMO, the impact doesn't seem to be significant, given the fact that most features and bug fixes are OS agnostic. Also, keep in mind that every features we introduced (variety of COEs, variety of Nova virt-driver, variety of network driver, variety of volume driver, variety of ...) incurs a maintenance overhead. If you want an optimal development speed, we will be limited to support a single COE/virt driver/network driver/volume driver. I guess that is not the direction we like to be? Instead, we would all be force
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
Steve, I think you may have misunderstood our intent here. We are not seeking to lock in to a single OS vendor. Each COE driver can have a different OS. We can have multiple drivers per COE. The point is that drivers should be simple, and therefore should support one Bay node OS each. That would mean taking what we have today in our Kubernetes Bay type implementation and breaking it down into two drivers: one for CoreOS and another for Fedora/Atomic. New drivers would start out in a contrib directory where complete functional testing would not be required. In order to graduate one out of contrib and into the realm of support of the Magnum dev team, it would need to have a full set of tests, and someone actively maintaining it. Multi-personality driers would be relatively complex. That approach would slow down COE specific feature development, and complicate maintenance that is needed as new versions of the dependency chain are bundled in (docker, k8s, etcd, etc.). We have all agreed that having integration points that allow for alternate OS selection is still our direction. This follows the pattern that we set previously when deciding what networking options to support. We will have one that’s included as a default, and a way to plug in alternates. Here is what I expect to see when COE drivers are implemented: Docker Swarm: Default driver Fedora/Atomic Alternate driver: TBD Kubernetes: Default driver Fedora/Atomic Alternate driver: CoreOS Apache Mesos/Marathon: Default driver: Ubuntu Alternate driver: TBD We can allow an arbitrary number of alternates. Those TBD items can be initially added to the contrib directory, and with the right level of community support can be advanced to defaults if shown to work better, be more straightforward to maintain, be more secure, or whatever criteria is important to us when presented with the choice. Such criteria will be subject to community consensus. This should allow for free experimentation with alternates to allow for innovation. See how this is not locking in a single OS vendor? Adrian On Mar 14, 2016, at 12:41 PM, Steven Dake (stdake) <std...@cisco.com<mailto:std...@cisco.com>> wrote: Hongbin, When we are at a disagreement in the Kolla core team, we have the Kolla core reviewers vote on the matter. This is typical standard OpenStack best practice. I think the vote would be something like "Implement one OS/COE/network/storage prototype, or implement many." I don't have a horse in this race, but I think it would be seriously damaging to Magnum to lock in to a single vendor. Regards -steve From: Hongbin Lu <hongbin...@huawei.com<mailto:hongbin...@huawei.com>> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Date: Monday, March 7, 2016 at 10:06 AM To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro From: Corey O'Brien [mailto:coreypobr...@gmail.com] Sent: March-07-16 8:11 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro Hongbin, I think the offer to support different OS options is a perfect example both of what we want and what we don't want. We definitely want to allow for someone like yourself to maintain templates for whatever OS they want and to have that option be easily integrated in to a Magnum deployment. However, when developing features or bug fixes, we can't wait for you to have time to add it for whatever OS you are promising to maintain. It might be true that supporting additional OS could slow down the development speed, but the key question is how much the impact will be. Does it outweigh the benefits? IMO, the impact doesn’t seem to be significant, given the fact that most features and bug fixes are OS agnostic. Also, keep in mind that every features we introduced (variety of COEs, variety of Nova virt-driver, variety of network driver, variety of volume driver, variety of …) incurs a maintenance overhead. If you want an optimal development speed, we will be limited to support a single COE/virt driver/network driver/volume driver. I guess that is not the direction we like to be? Instead, we would all be forced to develop the feature for that OS as well. If every member of the team had a special OS like that we'd all have to maintain all of them. To be clear, I don’t have a special OS, I guess neither do others who disagreed in this thread. Alternatively, what was agreed on by most at the midcycle was that if someone like yourself wanted to support a specific OS option, we would have an easy place for those contributions
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
Hongbin, When we are at a disagreement in the Kolla core team, we have the Kolla core reviewers vote on the matter. This is typical standard OpenStack best practice. I think the vote would be something like "Implement one OS/COE/network/storage prototype, or implement many." I don't have a horse in this race, but I think it would be seriously damaging to Magnum to lock in to a single vendor. Regards -steve From: Hongbin Lu <hongbin...@huawei.com<mailto:hongbin...@huawei.com>> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Date: Monday, March 7, 2016 at 10:06 AM To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro From: Corey O'Brien [mailto:coreypobr...@gmail.com] Sent: March-07-16 8:11 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro Hongbin, I think the offer to support different OS options is a perfect example both of what we want and what we don't want. We definitely want to allow for someone like yourself to maintain templates for whatever OS they want and to have that option be easily integrated in to a Magnum deployment. However, when developing features or bug fixes, we can't wait for you to have time to add it for whatever OS you are promising to maintain. It might be true that supporting additional OS could slow down the development speed, but the key question is how much the impact will be. Does it outweigh the benefits? IMO, the impact doesn’t seem to be significant, given the fact that most features and bug fixes are OS agnostic. Also, keep in mind that every features we introduced (variety of COEs, variety of Nova virt-driver, variety of network driver, variety of volume driver, variety of …) incurs a maintenance overhead. If you want an optimal development speed, we will be limited to support a single COE/virt driver/network driver/volume driver. I guess that is not the direction we like to be? Instead, we would all be forced to develop the feature for that OS as well. If every member of the team had a special OS like that we'd all have to maintain all of them. To be clear, I don’t have a special OS, I guess neither do others who disagreed in this thread. Alternatively, what was agreed on by most at the midcycle was that if someone like yourself wanted to support a specific OS option, we would have an easy place for those contributions to go without impacting the rest of the team. The team as a whole would agree to develop all features for at least the reference OS. Could we re-confirm that this is a team agreement? There is no harm to re-confirm it in the design summit/ML/team meeting. Frankly, it doesn’t seem to be. Then individuals or companies who are passionate about an alternative OS can develop the features for that OS. Corey On Sat, Mar 5, 2016 at 12:30 AM Hongbin Lu <hongbin...@huawei.com<mailto:hongbin...@huawei.com>> wrote: From: Adrian Otto [mailto:adrian.o...@rackspace.com<mailto:adrian.o...@rackspace.com>] Sent: March-04-16 6:31 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro Steve, On Mar 4, 2016, at 2:41 PM, Steven Dake (stdake) <std...@cisco.com<mailto:std...@cisco.com>> wrote: From: Adrian Otto <adrian.o...@rackspace.com<mailto:adrian.o...@rackspace.com>> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Date: Friday, March 4, 2016 at 12:48 PM To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro Hongbin, To be clear, this pursuit is not about what OS options cloud operators can select. We will be offering a method of choice. It has to do with what we plan to build comprehensive testing for, This is easy. Once we build comprehensive tests for the first OS, just re-run it for other OS(s). and the implications that has on our pace of feature development. My guidance here is that we resist the temptation to create a system with more permutations than we can possibly support. The relation between bay node OS, Heat Template, Heat Template parameters, COE, and COE dependencies (could-init, docker, flannel, etcd, etc.) are multiplicative in nature. From the mid cycle, it was clear to me that: 1
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
From: Corey O'Brien [mailto:coreypobr...@gmail.com] Sent: March-07-16 8:11 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro Hongbin, I think the offer to support different OS options is a perfect example both of what we want and what we don't want. We definitely want to allow for someone like yourself to maintain templates for whatever OS they want and to have that option be easily integrated in to a Magnum deployment. However, when developing features or bug fixes, we can't wait for you to have time to add it for whatever OS you are promising to maintain. It might be true that supporting additional OS could slow down the development speed, but the key question is how much the impact will be. Does it outweigh the benefits? IMO, the impact doesn’t seem to be significant, given the fact that most features and bug fixes are OS agnostic. Also, keep in mind that every features we introduced (variety of COEs, variety of Nova virt-driver, variety of network driver, variety of volume driver, variety of …) incurs a maintenance overhead. If you want an optimal development speed, we will be limited to support a single COE/virt driver/network driver/volume driver. I guess that is not the direction we like to be? Instead, we would all be forced to develop the feature for that OS as well. If every member of the team had a special OS like that we'd all have to maintain all of them. To be clear, I don’t have a special OS, I guess neither do others who disagreed in this thread. Alternatively, what was agreed on by most at the midcycle was that if someone like yourself wanted to support a specific OS option, we would have an easy place for those contributions to go without impacting the rest of the team. The team as a whole would agree to develop all features for at least the reference OS. Could we re-confirm that this is a team agreement? There is no harm to re-confirm it in the design summit/ML/team meeting. Frankly, it doesn’t seem to be. Then individuals or companies who are passionate about an alternative OS can develop the features for that OS. Corey On Sat, Mar 5, 2016 at 12:30 AM Hongbin Lu <hongbin...@huawei.com<mailto:hongbin...@huawei.com>> wrote: From: Adrian Otto [mailto:adrian.o...@rackspace.com<mailto:adrian.o...@rackspace.com>] Sent: March-04-16 6:31 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro Steve, On Mar 4, 2016, at 2:41 PM, Steven Dake (stdake) <std...@cisco.com<mailto:std...@cisco.com>> wrote: From: Adrian Otto <adrian.o...@rackspace.com<mailto:adrian.o...@rackspace.com>> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Date: Friday, March 4, 2016 at 12:48 PM To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro Hongbin, To be clear, this pursuit is not about what OS options cloud operators can select. We will be offering a method of choice. It has to do with what we plan to build comprehensive testing for, This is easy. Once we build comprehensive tests for the first OS, just re-run it for other OS(s). and the implications that has on our pace of feature development. My guidance here is that we resist the temptation to create a system with more permutations than we can possibly support. The relation between bay node OS, Heat Template, Heat Template parameters, COE, and COE dependencies (could-init, docker, flannel, etcd, etc.) are multiplicative in nature. From the mid cycle, it was clear to me that: 1) We want to test at least one OS per COE from end-to-end with comprehensive functional tests. 2) We want to offer clear and precise integration points to allow cloud operators to substitute their own OS in place of whatever one is the default for the given COE. A COE shouldn’t have a default necessarily that locks out other defaults. Magnum devs are the experts in how these systems operate, and as such need to take on the responsibility of the implementation for multi-os support. 3) We want to control the total number of configuration permutations to simplify our efforts as a project. We agreed that gate testing all possible permutations is intractable. I disagree with this point, but I don't have the bandwidth available to prove it ;) That’s exactly my point. It takes a chunk of human bandwidth to carry that responsibility. If we had a system engineer assigned from each of the various upstream OS distros working with Magnum, this would not
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
Hongbin, I think the offer to support different OS options is a perfect example both of what we want and what we don't want. We definitely want to allow for someone like yourself to maintain templates for whatever OS they want and to have that option be easily integrated in to a Magnum deployment. However, when developing features or bug fixes, we can't wait for you to have time to add it for whatever OS you are promising to maintain. Instead, we would all be forced to develop the feature for that OS as well. If every member of the team had a special OS like that we'd all have to maintain all of them. Alternatively, what was agreed on by most at the midcycle was that if someone like yourself wanted to support a specific OS option, we would have an easy place for those contributions to go without impacting the rest of the team. The team as a whole would agree to develop all features for at least the reference OS. Then individuals or companies who are passionate about an alternative OS can develop the features for that OS. Corey On Sat, Mar 5, 2016 at 12:30 AM Hongbin Lu <hongbin...@huawei.com> wrote: > > > > > *From:* Adrian Otto [mailto:adrian.o...@rackspace.com] > *Sent:* March-04-16 6:31 PM > > > *To:* OpenStack Development Mailing List (not for usage questions) > > *Subject:* Re: [openstack-dev] [magnum] Discussion of supporting > single/multiple OS distro > > > > Steve, > > > > On Mar 4, 2016, at 2:41 PM, Steven Dake (stdake) <std...@cisco.com> wrote: > > > > *From: *Adrian Otto <adrian.o...@rackspace.com> > *Reply-To: *"OpenStack Development Mailing List (not for usage > questions)" <openstack-dev@lists.openstack.org> > *Date: *Friday, March 4, 2016 at 12:48 PM > *To: *"OpenStack Development Mailing List (not for usage questions)" < > openstack-dev@lists.openstack.org> > *Subject: *Re: [openstack-dev] [magnum] Discussion of supporting > single/multiple OS distro > > > > Hongbin, > > > > To be clear, this pursuit is not about what OS options cloud operators can > select. We will be offering a method of choice. It has to do with what we > plan to build comprehensive testing for, > > This is easy. Once we build comprehensive tests for the first OS, just > re-run it for other OS(s). > > > > and the implications that has on our pace of feature development. My > guidance here is that we resist the temptation to create a system with more > permutations than we can possibly support. The relation between bay node > OS, Heat Template, Heat Template parameters, COE, and COE dependencies > (could-init, docker, flannel, etcd, etc.) are multiplicative in nature. > From the mid cycle, it was clear to me that: > > > > 1) We want to test at least one OS per COE from end-to-end with > comprehensive functional tests. > > 2) We want to offer clear and precise integration points to allow cloud > operators to substitute their own OS in place of whatever one is the > default for the given COE. > > > > A COE shouldn’t have a default necessarily that locks out other defaults. > Magnum devs are the experts in how these systems operate, and as such need > to take on the responsibility of the implementation for multi-os support. > > > > 3) We want to control the total number of configuration permutations to > simplify our efforts as a project. We agreed that gate testing all possible > permutations is intractable. > > > > I disagree with this point, but I don't have the bandwidth available to > prove it ;) > > > > That’s exactly my point. It takes a chunk of human bandwidth to carry that > responsibility. If we had a system engineer assigned from each of the > various upstream OS distros working with Magnum, this would not be a big > deal. Expecting our current contributors to support a variety of OS > variants is not realistic. > > You have my promise to support an additional OS for 1 or 2 popular COEs. > > > > Change velocity among all the components we rely on has been very high. We > see some of our best contributors frequently sidetracked in the details of > the distros releasing versions of code that won’t work with ours. We want > to upgrade a component to add a new feature, but struggle to because the > new release of the distro that offers that component is otherwise > incompatible. Multiply this by more distros, and we expect a real problem. > > At Magnum upstream, the overhead doesn’t seem to come from the OS. > Perhaps, that is specific to your downstream? > > > > There is no harm if you have 30 gates running the various combinations. > Infrastructure can handle the load. Whether devs have the cycles to make a > fully bulletproof gate is the q
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
From: Adrian Otto [mailto:adrian.o...@rackspace.com] Sent: March-04-16 6:31 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro Steve, On Mar 4, 2016, at 2:41 PM, Steven Dake (stdake) <std...@cisco.com<mailto:std...@cisco.com>> wrote: From: Adrian Otto <adrian.o...@rackspace.com<mailto:adrian.o...@rackspace.com>> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Date: Friday, March 4, 2016 at 12:48 PM To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro Hongbin, To be clear, this pursuit is not about what OS options cloud operators can select. We will be offering a method of choice. It has to do with what we plan to build comprehensive testing for, This is easy. Once we build comprehensive tests for the first OS, just re-run it for other OS(s). and the implications that has on our pace of feature development. My guidance here is that we resist the temptation to create a system with more permutations than we can possibly support. The relation between bay node OS, Heat Template, Heat Template parameters, COE, and COE dependencies (could-init, docker, flannel, etcd, etc.) are multiplicative in nature. From the mid cycle, it was clear to me that: 1) We want to test at least one OS per COE from end-to-end with comprehensive functional tests. 2) We want to offer clear and precise integration points to allow cloud operators to substitute their own OS in place of whatever one is the default for the given COE. A COE shouldn't have a default necessarily that locks out other defaults. Magnum devs are the experts in how these systems operate, and as such need to take on the responsibility of the implementation for multi-os support. 3) We want to control the total number of configuration permutations to simplify our efforts as a project. We agreed that gate testing all possible permutations is intractable. I disagree with this point, but I don't have the bandwidth available to prove it ;) That's exactly my point. It takes a chunk of human bandwidth to carry that responsibility. If we had a system engineer assigned from each of the various upstream OS distros working with Magnum, this would not be a big deal. Expecting our current contributors to support a variety of OS variants is not realistic. You have my promise to support an additional OS for 1 or 2 popular COEs. Change velocity among all the components we rely on has been very high. We see some of our best contributors frequently sidetracked in the details of the distros releasing versions of code that won't work with ours. We want to upgrade a component to add a new feature, but struggle to because the new release of the distro that offers that component is otherwise incompatible. Multiply this by more distros, and we expect a real problem. At Magnum upstream, the overhead doesn't seem to come from the OS. Perhaps, that is specific to your downstream? There is no harm if you have 30 gates running the various combinations. Infrastructure can handle the load. Whether devs have the cycles to make a fully bulletproof gate is the question I think you answered with the word intractable. Actually, our existing gate tests are really stressing out our CI infra. At least one of the new infrastructure providers that replaced HP have equipment that runs considerably slower. For example, our swam functional gate now frequently fails because it can't finish before the allowed time limit of 2 hours where it could finish substantially faster before. If we expanded the workload considerably, we might quickly work to the detriment of other projects by perpetually clogging the CI pipelines. We want to be a good citizen of the openstack CI community. Testing configuration of third party software should be done with third party CI setups. That's one of the reasons those exist. Ideally, each would be maintained by those who have a strategic (commercial?) interest in support for that particular OS. I can tell you in Kolla we spend a lot of cycles just getting basic gating going of building containers and then deploying them. We have even made inroads into testing the deployment. We do CentOS, Ubuntu, and soon Oracle Linux, for both source and binary and build and deploy. Lots of gates and if they aren't green we know the patch is wrong. Remember that COE's are tested on nova instances within heat stacks. Starting lots of nova instances within devstack in the gates is problematic. We are looking into using a libvirt-lxc instance type from nova instead of a libvirt
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
Steve, On Mar 4, 2016, at 2:41 PM, Steven Dake (stdake) <std...@cisco.com<mailto:std...@cisco.com>> wrote: From: Adrian Otto <adrian.o...@rackspace.com<mailto:adrian.o...@rackspace.com>> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Date: Friday, March 4, 2016 at 12:48 PM To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro Hongbin, To be clear, this pursuit is not about what OS options cloud operators can select. We will be offering a method of choice. It has to do with what we plan to build comprehensive testing for, and the implications that has on our pace of feature development. My guidance here is that we resist the temptation to create a system with more permutations than we can possibly support. The relation between bay node OS, Heat Template, Heat Template parameters, COE, and COE dependencies (could-init, docker, flannel, etcd, etc.) are multiplicative in nature. From the mid cycle, it was clear to me that: 1) We want to test at least one OS per COE from end-to-end with comprehensive functional tests. 2) We want to offer clear and precise integration points to allow cloud operators to substitute their own OS in place of whatever one is the default for the given COE. A COE shouldn’t have a default necessarily that locks out other defaults. Magnum devs are the experts in how these systems operate, and as such need to take on the responsibility of the implementation for multi-os support. 3) We want to control the total number of configuration permutations to simplify our efforts as a project. We agreed that gate testing all possible permutations is intractable. I disagree with this point, but I don't have the bandwidth available to prove it ;) That’s exactly my point. It takes a chunk of human bandwidth to carry that responsibility. If we had a system engineer assigned from each of the various upstream OS distros working with Magnum, this would not be a big deal. Expecting our current contributors to support a variety of OS variants is not realistic. Change velocity among all the components we rely on has been very high. We see some of our best contributors frequently sidetracked in the details of the distros releasing versions of code that won’t work with ours. We want to upgrade a component to add a new feature, but struggle to because the new release of the distro that offers that component is otherwise incompatible. Multiply this by more distros, and we expect a real problem. There is no harm if you have 30 gates running the various combinations. Infrastructure can handle the load. Whether devs have the cycles to make a fully bulletproof gate is the question I think you answered with the word intractable. Actually, our existing gate tests are really stressing out our CI infra. At least one of the new infrastructure providers that replaced HP have equipment that runs considerably slower. For example, our swam functional gate now frequently fails because it can’t finish before the allowed time limit of 2 hours where it could finish substantially faster before. If we expanded the workload considerably, we might quickly work to the detriment of other projects by perpetually clogging the CI pipelines. We want to be a good citizen of the openstack CI community. Testing configuration of third party software should be done with third party CI setups. That’s one of the reasons those exist. Ideally, each would be maintained by those who have a strategic (commercial?) interest in support for that particular OS. I can tell you in Kolla we spend a lot of cycles just getting basic gating going of building containers and then deploying them. We have even made inroads into testing the deployment. We do CentOS, Ubuntu, and soon Oracle Linux, for both source and binary and build and deploy. Lots of gates and if they aren't green we know the patch is wrong. Remember that COE’s are tested on nova instances within heat stacks. Starting lots of nova instances within devstack in the gates is problematic. We are looking into using a libvirt-lxc instance type from nova instead of a libvirt-kvm instance to help alleviate this. Until then, limiting the scope of our gate tests is appropriate. We will continue our efforts to make them reasonably efficient. Thanks, Adrian Regards -steve Note that it will take a thoughtful approach (subject to discussion) to balance these interests. Please take a moment to review the interest above. Do you or others disagree with these? If so, why? Adrian On Mar 4, 2016, at 9:09 AM, Hongbin Lu <hongbin...@huawei.com<mailto:hongbin...@huawei.com&
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
From: Adrian Otto <adrian.o...@rackspace.com<mailto:adrian.o...@rackspace.com>> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Date: Friday, March 4, 2016 at 12:48 PM To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro Hongbin, To be clear, this pursuit is not about what OS options cloud operators can select. We will be offering a method of choice. It has to do with what we plan to build comprehensive testing for, and the implications that has on our pace of feature development. My guidance here is that we resist the temptation to create a system with more permutations than we can possibly support. The relation between bay node OS, Heat Template, Heat Template parameters, COE, and COE dependencies (could-init, docker, flannel, etcd, etc.) are multiplicative in nature. From the mid cycle, it was clear to me that: 1) We want to test at least one OS per COE from end-to-end with comprehensive functional tests. 2) We want to offer clear and precise integration points to allow cloud operators to substitute their own OS in place of whatever one is the default for the given COE. A COE shouldn’t have a default necessarily that locks out other defaults. Magnum devs are the experts in how these systems operate, and as such need to take on the responsibility of the implementation for multi-os support. 3) We want to control the total number of configuration permutations to simplify our efforts as a project. We agreed that gate testing all possible permutations is intractable. I disagree with this point, but I don't have the bandwidth available to prove it ;) There is no harm if you have 30 gates running the various combinations. Infrastructure can handle the load. Whether devs have the cycles to make a fully bulletproof gate is the question I think you answered with the word intractable. I can tell you in Kolla we spend a lot of cycles just getting basic gating going of building containers and then deploying them. We have even made inroads into testing the deployment. We do CentOS, Ubuntu, and soon Oracle Linux, for both source and binary and build and deploy. Lots of gates and if they aren't green we know the patch is wrong. Regards -steve Note that it will take a thoughtful approach (subject to discussion) to balance these interests. Please take a moment to review the interest above. Do you or others disagree with these? If so, why? Adrian On Mar 4, 2016, at 9:09 AM, Hongbin Lu <hongbin...@huawei.com<mailto:hongbin...@huawei.com>> wrote: I don’t think there is any consensus on supporting single distro. There are multiple disagreements on this thread, including several senior team members and a project co-founder. This topic should be re-discussed (possibly at the design summit). Best regards, Hongbin From: Corey O'Brien [mailto:coreypobr...@gmail.com] Sent: March-04-16 11:37 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro I don't think anyone is saying that code should somehow block support for multiple distros. The discussion at midcycle was about what the we should gate on and ensure feature parity for as a team. Ideally, we'd like to get support for every distro, I think, but no one wants to have that many gates. Instead, the consensus at the midcycle was to have 1 reference distro for each COE, gate on those and develop features there, and then have any other distros be maintained by those in the community that are passionate about them. The issue also isn't about how difficult or not it is. The problem we want to avoid is spending precious time guaranteeing that new features and bug fixes make it through multiple distros. Corey On Fri, Mar 4, 2016 at 11:18 AM Steven Dake (stdake) <std...@cisco.com<mailto:std...@cisco.com>> wrote: My position on this is simple. Operators are used to using specific distros because that is what they used in the 90s,and the 00s, and the 10s. Yes, 25 years of using a distro, and you learn it inside and out. This means you don't want to relearn a new distro, especially if your an RPM user going to DEB or a DEB user going to RPM. These are non-starter options for operators, and as a result, mean that distro choice is a must. Since CoreOS is a new OS in the marketplace, it may make sense to consider placing it in "third" position in terms of support. Besides that problem, various distribution companies will only support distros running in Vms if it matches the host kernel, which makes total sense to me. This means on an Ubun
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
+1 on the points Adrian makes below. On Mar 4, 2016, at 12:52 PM, Adrian Otto <adrian.o...@rackspace.com<mailto:adrian.o...@rackspace.com>> wrote: Hongbin, To be clear, this pursuit is not about what OS options cloud operators can select. We will be offering a method of choice. It has to do with what we plan to build comprehensive testing for, and the implications that has on our pace of feature development. My guidance here is that we resist the temptation to create a system with more permutations than we can possibly support. The relation between bay node OS, Heat Template, Heat Template parameters, COE, and COE dependencies (could-init, docker, flannel, etcd, etc.) are multiplicative in nature. From the mid cycle, it was clear to me that: 1) We want to test at least one OS per COE from end-to-end with comprehensive functional tests. 2) We want to offer clear and precise integration points to allow cloud operators to substitute their own OS in place of whatever one is the default for the given COE. 3) We want to control the total number of configuration permutations to simplify our efforts as a project. We agreed that gate testing all possible permutations is intractable. Note that it will take a thoughtful approach (subject to discussion) to balance these interests. Please take a moment to review the interest above. Do you or others disagree with these? If so, why? Adrian On Mar 4, 2016, at 9:09 AM, Hongbin Lu <hongbin...@huawei.com<mailto:hongbin...@huawei.com>> wrote: I don’t think there is any consensus on supporting single distro. There are multiple disagreements on this thread, including several senior team members and a project co-founder. This topic should be re-discussed (possibly at the design summit). Best regards, Hongbin From: Corey O'Brien [mailto:coreypobr...@gmail.com] Sent: March-04-16 11:37 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro I don't think anyone is saying that code should somehow block support for multiple distros. The discussion at midcycle was about what the we should gate on and ensure feature parity for as a team. Ideally, we'd like to get support for every distro, I think, but no one wants to have that many gates. Instead, the consensus at the midcycle was to have 1 reference distro for each COE, gate on those and develop features there, and then have any other distros be maintained by those in the community that are passionate about them. The issue also isn't about how difficult or not it is. The problem we want to avoid is spending precious time guaranteeing that new features and bug fixes make it through multiple distros. Corey On Fri, Mar 4, 2016 at 11:18 AM Steven Dake (stdake) <std...@cisco.com<mailto:std...@cisco.com>> wrote: My position on this is simple. Operators are used to using specific distros because that is what they used in the 90s,and the 00s, and the 10s. Yes, 25 years of using a distro, and you learn it inside and out. This means you don't want to relearn a new distro, especially if your an RPM user going to DEB or a DEB user going to RPM. These are non-starter options for operators, and as a result, mean that distro choice is a must. Since CoreOS is a new OS in the marketplace, it may make sense to consider placing it in "third" position in terms of support. Besides that problem, various distribution companies will only support distros running in Vms if it matches the host kernel, which makes total sense to me. This means on an Ubuntu host if I want support I need to run Ubuntu vms, on a RHEL host I want to run RHEL vms, because, hey, I want my issues supported. For these reasons and these reasons alone, there is no good rationale to remove multi-distro support from Magnum. All I've heard in this thread so far is "its too hard". Its not too hard, especially with Heat conditionals making their way into Mitaka. Regards -steve From: Hongbin Lu <hongbin...@huawei.com<mailto:hongbin...@huawei.com>> Reply-To: "openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Date: Monday, February 29, 2016 at 9:40 AM To: "openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Subject: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro Hi team, This is a continued discussion from a review [1]. Corey O'Brien suggested to have Magnum support a single OS distro (Atomic). I disagreed. I think we should bring the discussion to here to get broader set of inputs. Corey O'Brien >From the midcycle, we decided we weren't going to contin
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
Hongbin, To be clear, this pursuit is not about what OS options cloud operators can select. We will be offering a method of choice. It has to do with what we plan to build comprehensive testing for, and the implications that has on our pace of feature development. My guidance here is that we resist the temptation to create a system with more permutations than we can possibly support. The relation between bay node OS, Heat Template, Heat Template parameters, COE, and COE dependencies (could-init, docker, flannel, etcd, etc.) are multiplicative in nature. From the mid cycle, it was clear to me that: 1) We want to test at least one OS per COE from end-to-end with comprehensive functional tests. 2) We want to offer clear and precise integration points to allow cloud operators to substitute their own OS in place of whatever one is the default for the given COE. 3) We want to control the total number of configuration permutations to simplify our efforts as a project. We agreed that gate testing all possible permutations is intractable. Note that it will take a thoughtful approach (subject to discussion) to balance these interests. Please take a moment to review the interest above. Do you or others disagree with these? If so, why? Adrian On Mar 4, 2016, at 9:09 AM, Hongbin Lu <hongbin...@huawei.com<mailto:hongbin...@huawei.com>> wrote: I don’t think there is any consensus on supporting single distro. There are multiple disagreements on this thread, including several senior team members and a project co-founder. This topic should be re-discussed (possibly at the design summit). Best regards, Hongbin From: Corey O'Brien [mailto:coreypobr...@gmail.com] Sent: March-04-16 11:37 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro I don't think anyone is saying that code should somehow block support for multiple distros. The discussion at midcycle was about what the we should gate on and ensure feature parity for as a team. Ideally, we'd like to get support for every distro, I think, but no one wants to have that many gates. Instead, the consensus at the midcycle was to have 1 reference distro for each COE, gate on those and develop features there, and then have any other distros be maintained by those in the community that are passionate about them. The issue also isn't about how difficult or not it is. The problem we want to avoid is spending precious time guaranteeing that new features and bug fixes make it through multiple distros. Corey On Fri, Mar 4, 2016 at 11:18 AM Steven Dake (stdake) <std...@cisco.com<mailto:std...@cisco.com>> wrote: My position on this is simple. Operators are used to using specific distros because that is what they used in the 90s,and the 00s, and the 10s. Yes, 25 years of using a distro, and you learn it inside and out. This means you don't want to relearn a new distro, especially if your an RPM user going to DEB or a DEB user going to RPM. These are non-starter options for operators, and as a result, mean that distro choice is a must. Since CoreOS is a new OS in the marketplace, it may make sense to consider placing it in "third" position in terms of support. Besides that problem, various distribution companies will only support distros running in Vms if it matches the host kernel, which makes total sense to me. This means on an Ubuntu host if I want support I need to run Ubuntu vms, on a RHEL host I want to run RHEL vms, because, hey, I want my issues supported. For these reasons and these reasons alone, there is no good rationale to remove multi-distro support from Magnum. All I've heard in this thread so far is "its too hard". Its not too hard, especially with Heat conditionals making their way into Mitaka. Regards -steve From: Hongbin Lu <hongbin...@huawei.com<mailto:hongbin...@huawei.com>> Reply-To: "openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Date: Monday, February 29, 2016 at 9:40 AM To: "openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Subject: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro Hi team, This is a continued discussion from a review [1]. Corey O'Brien suggested to have Magnum support a single OS distro (Atomic). I disagreed. I think we should bring the discussion to here to get broader set of inputs. Corey O'Brien From the midcycle, we decided we weren't going to continue to support 2 different versions of the k8s template. Instead, we were going to maintain the Fedora Atomic version of k8s and remove the coreos templates from the tree. I
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
I don’t think there is any consensus on supporting single distro. There are multiple disagreements on this thread, including several senior team members and a project co-founder. This topic should be re-discussed (possibly at the design summit). Best regards, Hongbin From: Corey O'Brien [mailto:coreypobr...@gmail.com] Sent: March-04-16 11:37 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro I don't think anyone is saying that code should somehow block support for multiple distros. The discussion at midcycle was about what the we should gate on and ensure feature parity for as a team. Ideally, we'd like to get support for every distro, I think, but no one wants to have that many gates. Instead, the consensus at the midcycle was to have 1 reference distro for each COE, gate on those and develop features there, and then have any other distros be maintained by those in the community that are passionate about them. The issue also isn't about how difficult or not it is. The problem we want to avoid is spending precious time guaranteeing that new features and bug fixes make it through multiple distros. Corey On Fri, Mar 4, 2016 at 11:18 AM Steven Dake (stdake) <std...@cisco.com<mailto:std...@cisco.com>> wrote: My position on this is simple. Operators are used to using specific distros because that is what they used in the 90s,and the 00s, and the 10s. Yes, 25 years of using a distro, and you learn it inside and out. This means you don't want to relearn a new distro, especially if your an RPM user going to DEB or a DEB user going to RPM. These are non-starter options for operators, and as a result, mean that distro choice is a must. Since CoreOS is a new OS in the marketplace, it may make sense to consider placing it in "third" position in terms of support. Besides that problem, various distribution companies will only support distros running in Vms if it matches the host kernel, which makes total sense to me. This means on an Ubuntu host if I want support I need to run Ubuntu vms, on a RHEL host I want to run RHEL vms, because, hey, I want my issues supported. For these reasons and these reasons alone, there is no good rationale to remove multi-distro support from Magnum. All I've heard in this thread so far is "its too hard". Its not too hard, especially with Heat conditionals making their way into Mitaka. Regards -steve From: Hongbin Lu <hongbin...@huawei.com<mailto:hongbin...@huawei.com>> Reply-To: "openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Date: Monday, February 29, 2016 at 9:40 AM To: "openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Subject: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro Hi team, This is a continued discussion from a review [1]. Corey O'Brien suggested to have Magnum support a single OS distro (Atomic). I disagreed. I think we should bring the discussion to here to get broader set of inputs. Corey O'Brien From the midcycle, we decided we weren't going to continue to support 2 different versions of the k8s template. Instead, we were going to maintain the Fedora Atomic version of k8s and remove the coreos templates from the tree. I don't think we should continue to develop features for coreos k8s if that is true. In addition, I don't think we should break the coreos template by adding the trust token as a heat parameter. Hongbin Lu I was on the midcycle and I don't remember any decision to remove CoreOS support. Why you want to remove CoreOS templates from the tree. Please note that this is a very big decision and please discuss it with the team thoughtfully and make sure everyone agree. Corey O'Brien Removing the coreos templates was a part of the COE drivers decision. Since each COE driver will only support 1 distro+version+coe we discussed which ones to support in tree. The decision was that instead of trying to support every distro and every version for every coe, the magnum tree would only have support for 1 version of 1 distro for each of the 3 COEs (swarm/docker/mesos). Since we already are going to support Atomic for swarm, removing coreos and keeping Atomic for kubernetes was the favored choice. Hongbin Lu Strongly disagree. It is a huge risk to support a single distro. The selected distro could die in the future. Who knows. Why make Magnum take this huge risk? Again, the decision of supporting single distro is a very big decision. Please bring it up to the team and have it discuss thoughtfully before making any decision. Also, Magnum doesn't have to supp
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
I don't think anyone is saying that code should somehow block support for multiple distros. The discussion at midcycle was about what the we should gate on and ensure feature parity for as a team. Ideally, we'd like to get support for every distro, I think, but no one wants to have that many gates. Instead, the consensus at the midcycle was to have 1 reference distro for each COE, gate on those and develop features there, and then have any other distros be maintained by those in the community that are passionate about them. The issue also isn't about how difficult or not it is. The problem we want to avoid is spending precious time guaranteeing that new features and bug fixes make it through multiple distros. Corey On Fri, Mar 4, 2016 at 11:18 AM Steven Dake (stdake) <std...@cisco.com> wrote: > My position on this is simple. > > Operators are used to using specific distros because that is what they > used in the 90s,and the 00s, and the 10s. Yes, 25 years of using a distro, > and you learn it inside and out. This means you don't want to relearn a > new distro, especially if your an RPM user going to DEB or a DEB user going > to RPM. These are non-starter options for operators, and as a result, mean > that distro choice is a must. Since CoreOS is a new OS in the marketplace, > it may make sense to consider placing it in "third" position in terms of > support. > > Besides that problem, various distribution companies will only support > distros running in Vms if it matches the host kernel, which makes total > sense to me. This means on an Ubuntu host if I want support I need to run > Ubuntu vms, on a RHEL host I want to run RHEL vms, because, hey, I want my > issues supported. > > For these reasons and these reasons alone, there is no good rationale to > remove multi-distro support from Magnum. All I've heard in this thread so > far is "its too hard". Its not too hard, especially with Heat conditionals > making their way into Mitaka. > > Regards > -steve > > From: Hongbin Lu <hongbin...@huawei.com> > Reply-To: "openstack-dev@lists.openstack.org" < > openstack-dev@lists.openstack.org> > Date: Monday, February 29, 2016 at 9:40 AM > To: "openstack-dev@lists.openstack.org" <openstack-dev@lists.openstack.org > > > Subject: [openstack-dev] [magnum] Discussion of supporting > single/multiple OS distro > > Hi team, > > > > This is a continued discussion from a review [1]. Corey O'Brien suggested > to have Magnum support a single OS distro (Atomic). I disagreed. I think we > should bring the discussion to here to get broader set of inputs. > > > > *Corey O'Brien* > > *From the midcycle, we decided we weren't going to continue to support 2 > different versions of the k8s template. Instead, we were going to maintain > the Fedora Atomic version of k8s and remove the coreos templates from the > tree. I don't think we should continue to develop features for coreos k8s > if that is true.* > > *In addition, I don't think we should break the coreos template by adding > the trust token as a heat parameter.* > > > > *Hongbin Lu* > > *I was on the midcycle and I don't remember any decision to remove CoreOS > support. Why you want to remove CoreOS templates from the tree. Please note > that this is a very big decision and please discuss it with the team > thoughtfully and make sure everyone agree.* > > > > *Corey O'Brien* > > *Removing the coreos templates was a part of the COE drivers decision. > Since each COE driver will only support 1 distro+version+coe we discussed > which ones to support in tree. The decision was that instead of trying to > support every distro and every version for every coe, the magnum tree would > only have support for 1 version of 1 distro for each of the 3 COEs > (swarm/docker/mesos). Since we already are going to support Atomic for > swarm, removing coreos and keeping Atomic for kubernetes was the favored > choice.* > > > > *Hongbin Lu* > > *Strongly disagree. It is a huge risk to support a single distro. The > selected distro could die in the future. Who knows. Why make Magnum take > this huge risk? Again, the decision of supporting single distro is a very > big decision. Please bring it up to the team and have it discuss > thoughtfully before making any decision. Also, Magnum doesn't have to > support every distro and every version for every coe, but should support > *more than one* popular distro for some COEs (especially for the popular > COEs).* > > > > *Corey O'Brien* > > *The discussion at the midcycle started from the idea of adding support > for RHEL and CentOS. We all discussed and decided that we wouldn't try to > support every
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
My position on this is simple. Operators are used to using specific distros because that is what they used in the 90s,and the 00s, and the 10s. Yes, 25 years of using a distro, and you learn it inside and out. This means you don't want to relearn a new distro, especially if your an RPM user going to DEB or a DEB user going to RPM. These are non-starter options for operators, and as a result, mean that distro choice is a must. Since CoreOS is a new OS in the marketplace, it may make sense to consider placing it in "third" position in terms of support. Besides that problem, various distribution companies will only support distros running in Vms if it matches the host kernel, which makes total sense to me. This means on an Ubuntu host if I want support I need to run Ubuntu vms, on a RHEL host I want to run RHEL vms, because, hey, I want my issues supported. For these reasons and these reasons alone, there is no good rationale to remove multi-distro support from Magnum. All I've heard in this thread so far is "its too hard". Its not too hard, especially with Heat conditionals making their way into Mitaka. Regards -steve From: Hongbin Lu <hongbin...@huawei.com<mailto:hongbin...@huawei.com>> Reply-To: "openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Date: Monday, February 29, 2016 at 9:40 AM To: "openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Subject: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro Hi team, This is a continued discussion from a review [1]. Corey O'Brien suggested to have Magnum support a single OS distro (Atomic). I disagreed. I think we should bring the discussion to here to get broader set of inputs. Corey O'Brien >From the midcycle, we decided we weren't going to continue to support 2 >different versions of the k8s template. Instead, we were going to maintain the >Fedora Atomic version of k8s and remove the coreos templates from the tree. I >don't think we should continue to develop features for coreos k8s if that is >true. In addition, I don't think we should break the coreos template by adding the trust token as a heat parameter. Hongbin Lu I was on the midcycle and I don't remember any decision to remove CoreOS support. Why you want to remove CoreOS templates from the tree. Please note that this is a very big decision and please discuss it with the team thoughtfully and make sure everyone agree. Corey O'Brien Removing the coreos templates was a part of the COE drivers decision. Since each COE driver will only support 1 distro+version+coe we discussed which ones to support in tree. The decision was that instead of trying to support every distro and every version for every coe, the magnum tree would only have support for 1 version of 1 distro for each of the 3 COEs (swarm/docker/mesos). Since we already are going to support Atomic for swarm, removing coreos and keeping Atomic for kubernetes was the favored choice. Hongbin Lu Strongly disagree. It is a huge risk to support a single distro. The selected distro could die in the future. Who knows. Why make Magnum take this huge risk? Again, the decision of supporting single distro is a very big decision. Please bring it up to the team and have it discuss thoughtfully before making any decision. Also, Magnum doesn't have to support every distro and every version for every coe, but should support *more than one* popular distro for some COEs (especially for the popular COEs). Corey O'Brien The discussion at the midcycle started from the idea of adding support for RHEL and CentOS. We all discussed and decided that we wouldn't try to support everything in tree. Magnum would provide support in-tree for 1 per COE and the COE driver interface would allow others to add support for their preferred distro out of tree. Hongbin Lu I agreed the part that "we wouldn't try to support everything in tree". That doesn't imply the decision to support single distro. Again, support single distro is a huge risk. Why make Magnum take this huge risk? [1] https://review.openstack.org/#/c/277284/ Best regards, Hongbin __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
From: Adrian Otto [mailto:adrian.o...@rackspace.com] Sent: March-01-16 9:54 AM To: Guz Egor Cc: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro This issue involves what I refer to as "OS religion" operators have this WRT bay nodes, but users don't. I suppose this is a key reason why OpenStack does not have any concept of supported OS images today. Where I can see the value in offering various choices in Magnum, maintaining a reference implementation of an OS image has shown that it requires non-trivial resources, and It is definitely non-trivial to create the first reference implementation, since we create it from scratch. However, I don't think it is hard to maintain an additional reference implementation. From technical point of view, most of the work in the first implementation can be reused and consolidated in some ways. Perhaps, what I failed to see is the claimed difficulties to maintain an additional OS. To discuss it further, I would suggest to work on an etherpad to list the overheads and benefits so that we can do a reasonable tradeoff. Thoughts? expanding that to several will certainly require more. The question really comes down to the importance of this particular choice as a development team focus. Is it more important than a compelling network or storage integration with OpenStack services? I doubt it. We all agree there should be a way to use an alternate OS image with Magnum. That has been our intent from the start. We are not discussing removing that option. However, rather than having multiple OS images the Magnum team maintains, maybe we could clearly articulate how to plug in to Magnum, and set up a third party CI, and allow various OS vendors to participate to make their options work with those requirements. If this approach works, then it may even reduce the need for a reference implementation at all if multiple upstream options result. -- Adrian On Mar 1, 2016, at 12:28 AM, Guz Egor <guz_e...@yahoo.com<mailto:guz_e...@yahoo.com>> wrote: Adrian, I disagree, host OS is very important for operators because of integration with all internal tools/repos/etc. I think it make sense to limit OS support in Magnum main source. But not sure that Fedora Atomic is right choice, first of all there is no documentation about it and I don't think it's used/tested a lot by Docker/Kub/Mesos community. It make sense to go with Ubuntu (I believe it's still most adopted platform in all three COEs and OpenStack deployments) and CoreOS (is highly adopted/tested in Kub community and Mesosphere DCOS uses it as well). We can implement CoreOS support as driver and users can use it as reference implementation. --- Egor From: Adrian Otto <adrian.o...@rackspace.com<mailto:adrian.o...@rackspace.com>> To: OpenStack Development Mailing List (not for usage questions) <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Sent: Monday, February 29, 2016 10:36 AM Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro Consider this: Which OS runs on the bay nodes is not important to end users. What matters to users is the environments their containers execute in, which has only one thing in common with the bay node OS: the kernel. The linux syscall interface is stable enough that the various linux distributions can all run concurrently in neighboring containers sharing same kernel. There is really no material reason why the bay OS choice must match what distro the container is based on. Although I'm persuaded by Hongbin's concern to mitigate risk of future changes WRT whatever OS distro is the prevailing one for bay nodes, there are a few items of concern about duality I'd like to zero in on: 1) Participation from Magnum contributors to support the CoreOS specific template features has been weak in recent months. By comparison, participation relating to Fedora/Atomic have been much stronger. 2) Properly testing multiple bay node OS distros (would) significantly increase the run time and complexity of our functional tests. 3) Having support for multiple bay node OS choices requires more extensive documentation, and more comprehensive troubleshooting details. If we proceed with just one supported disto for bay nodes, and offer extensibility points to allow alternates to be used in place of it, we should be able to address the risk concern of the chosen distro by selecting an alternate when that change is needed, by using those extensibility points. These include the ability to specify your own bay image, and the ability to use your own associated Heat template. I see value in risk mitigation, it may make sense to simplify in the short term and address that need when it becomes necessary. My point of view might be diff
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
This issue involves what I refer to as "OS religion" operators have this WRT bay nodes, but users don't. I suppose this is a key reason why OpenStack does not have any concept of supported OS images today. Where I can see the value in offering various choices in Magnum, maintaining a reference implementation of an OS image has shown that it requires non-trivial resources, and expanding that to several will certainly require more. The question really comes down to the importance of this particular choice as a development team focus. Is it more important than a compelling network or storage integration with OpenStack services? I doubt it. We all agree there should be a way to use an alternate OS image with Magnum. That has been our intent from the start. We are not discussing removing that option. However, rather than having multiple OS images the Magnum team maintains, maybe we could clearly articulate how to plug in to Magnum, and set up a third party CI, and allow various OS vendors to participate to make their options work with those requirements. If this approach works, then it may even reduce the need for a reference implementation at all if multiple upstream options result. -- Adrian On Mar 1, 2016, at 12:28 AM, Guz Egor <guz_e...@yahoo.com<mailto:guz_e...@yahoo.com>> wrote: Adrian, I disagree, host OS is very important for operators because of integration with all internal tools/repos/etc. I think it make sense to limit OS support in Magnum main source. But not sure that Fedora Atomic is right choice, first of all there is no documentation about it and I don't think it's used/tested a lot by Docker/Kub/Mesos community. It make sense to go with Ubuntu (I believe it's still most adopted platform in all three COEs and OpenStack deployments) and CoreOS (is highly adopted/tested in Kub community and Mesosphere DCOS uses it as well). We can implement CoreOS support as driver and users can use it as reference implementation. --- Egor From: Adrian Otto <adrian.o...@rackspace.com<mailto:adrian.o...@rackspace.com>> To: OpenStack Development Mailing List (not for usage questions) <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Sent: Monday, February 29, 2016 10:36 AM Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro Consider this: Which OS runs on the bay nodes is not important to end users. What matters to users is the environments their containers execute in, which has only one thing in common with the bay node OS: the kernel. The linux syscall interface is stable enough that the various linux distributions can all run concurrently in neighboring containers sharing same kernel. There is really no material reason why the bay OS choice must match what distro the container is based on. Although I’m persuaded by Hongbin’s concern to mitigate risk of future changes WRT whatever OS distro is the prevailing one for bay nodes, there are a few items of concern about duality I’d like to zero in on: 1) Participation from Magnum contributors to support the CoreOS specific template features has been weak in recent months. By comparison, participation relating to Fedora/Atomic have been much stronger. 2) Properly testing multiple bay node OS distros (would) significantly increase the run time and complexity of our functional tests. 3) Having support for multiple bay node OS choices requires more extensive documentation, and more comprehensive troubleshooting details. If we proceed with just one supported disto for bay nodes, and offer extensibility points to allow alternates to be used in place of it, we should be able to address the risk concern of the chosen distro by selecting an alternate when that change is needed, by using those extensibility points. These include the ability to specify your own bay image, and the ability to use your own associated Heat template. I see value in risk mitigation, it may make sense to simplify in the short term and address that need when it becomes necessary. My point of view might be different if we had contributors willing and ready to address the variety of drawbacks that accompany the strategy of supporting multiple bay node OS choices. In absence of such a community interest, my preference is to simplify to increase our velocity. This seems to me to be a relatively easy way to reduce complexity around heat template versioning. What do you think? Thanks, Adrian On Feb 29, 2016, at 8:40 AM, Hongbin Lu <hongbin...@huawei.com<mailto:hongbin...@huawei.com>> wrote: Hi team, This is a continued discussion from a review [1]. Corey O'Brien suggested to have Magnum support a single OS distro (Atomic). I disagreed. I think we should bring the discussion to here to get broader set of inputs. Corey O'Brien >From the midcycle, we decided we weren't going to c
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
- Original Message - > From: "Kai Qiang Wu" <wk...@cn.ibm.com> > To: "OpenStack Development Mailing List (not for usage questions)" > <openstack-dev@lists.openstack.org> > Cc: "Josh Berkus" <jber...@redhat.com> > > We found some issue about atomic host run some docker volume plugin, while > atomic and docker volume plugin both not sure what's the root cause of > that. > > here is the link > https://github.com/docker/docker/issues/18005#issuecomment-190215862 Thanks for highlighting this PR, I'll add it to my list. > Also I did not find atomic image update quickly, ( but k8s and docker both > release quickly, which can lacks of new feature applied in our > development), I think atomic have a gap for that. This is definitely likely to continue to be a real issue particularly w.r.t. docker itself - containerization of k8s, flannel, and etcd will alleviate at least some of the pain though as does the fact that updates are now in fact being pushed out every couple of weeks. As it stands the official Fedora Atomic images appear to actually contain equivalent or newer components than the custom builds at https://fedorapeople.org/groups/magnum/ (?), e.g.: Fedora-Cloud-Atomic-23-20160223.x86_64.qcow2: docker-1.9.1 flannel-0.5.4 kubernetes-1.1.0 etcd-2.2.1 fedora-21-7.qcow2: docker-1.9.1 flannel-0.5.0 kubernetes-1.1.0 etcd-2.0.13 I digress though, as I said in the follow up either way it doesn't seem to me like only having support for one image would be a win for users - it does make sense though to expect more of the work to support each image to come from the folks interested in maintaining that support though than being spread across the entire magnum team. Thanks, Steve > > Kai Qiang Wu (吴开强 Kennan) > IBM China System and Technology Lab, Beijing > > E-mail: wk...@cn.ibm.com > Tel: 86-10-82451647 > Address: Building 28(Ring Building), ZhongGuanCun Software Park, > No.8 Dong Bei Wang West Road, Haidian District Beijing P.R.China > 100193 > > Follow your heart. You are miracle! > > > > From: Steve Gordon <sgor...@redhat.com> > To: Guz Egor <guz_e...@yahoo.com>, "OpenStack Development Mailing > List (not for usage questions)" > <openstack-dev@lists.openstack.org> > Cc: Josh Berkus <jber...@redhat.com> > Date: 01/03/2016 08:19 pm > Subject: Re: [openstack-dev] [magnum] Discussion of supporting > single/multiple OS distro > > > > - Original Message - > > From: "Guz Egor" <guz_e...@yahoo.com> > > To: "OpenStack Development Mailing List (not for usage questions)" > <openstack-dev@lists.openstack.org> > > > > Adrian, > > I disagree, host OS is very important for operators because of > integration > > with all internal tools/repos/etc. > > I think it make sense to limit OS support in Magnum main source. But not > sure > > that Fedora Atomic is right choice,first of all there is no documentation > > about it and I don't think it's used/tested a lot by Docker/Kub/Mesos > > community. > > Project Atomic documentation for the most part lives here: > > http://www.projectatomic.io/docs/ > > To help us improve it, it would be useful to know what you think is > missing. E.g. I saw recently in the IRC channel it was discussed that there > is no documentation on (re)building the image but this is the first hit in > a Google search for same and it seems to largely match what has been copied > into Magnum's docs for same: > > > http://www.projectatomic.io/blog/2014/08/build-your-own-atomic-centos-or-fedora/ > > > I have no doubt that there are areas where the documentation is lacking, > but it's difficult to resolve a claim that there is no documentation at > all. I recently kicked off a thread over on the atomic list to try and > relay some of the concerns that were raised on this list and in the IRC > channel recently, it would be great if Magnum folks could chime in with > more specifics: > > > https://lists.projectatomic.io/projectatomic-archives/atomic/2016-February/thread.html#9 > > > Separately I had asked about containerization of kubernetes/etcd/flannel > which remains outstanding: > > > https://lists.fedoraproject.org/archives/list/cl...@lists.fedoraproject.org/thread/XICO4NJCTPI43AWG332EIM2HNFYPZ6ON/ > > > Fedora Atomic builds do seem to be hitting their planned two weekly update > cadence now though which may alleviate this conce
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
I tend to agree that multiple os support is OK (we can limited to popular ones first, like redhat os, ubuntu os) But we not tend to cover all OS, it would much burden for maintain, and extra small requirements should be maintain by 3-rd party if possible(through drivers). Thanks Best Wishes, Kai Qiang Wu (吴开强 Kennan) IBM China System and Technology Lab, Beijing E-mail: wk...@cn.ibm.com Tel: 86-10-82451647 Address: Building 28(Ring Building), ZhongGuanCun Software Park, No.8 Dong Bei Wang West Road, Haidian District Beijing P.R.China 100193 Follow your heart. You are miracle! From: Steve Gordon <sgor...@redhat.com> To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org> Cc: Martin Andre <maan...@redhat.com>, Josh Berkus <jber...@redhat.com> Date: 01/03/2016 08:25 pm Subject: Re: [openstack-dev] [magnum]Discussion of supporting single/multiple OS distro - Original Message - > From: "Steve Gordon" <sgor...@redhat.com> > To: "Guz Egor" <guz_e...@yahoo.com>, "OpenStack Development Mailing List (not for usage questions)" > <openstack-dev@lists.openstack.org> > > - Original Message - > > From: "Guz Egor" <guz_e...@yahoo.com> > > To: "OpenStack Development Mailing List (not for usage questions)" > > <openstack-dev@lists.openstack.org> > > > > Adrian, > > I disagree, host OS is very important for operators because of integration > > with all internal tools/repos/etc. > > I think it make sense to limit OS support in Magnum main source. But not > > sure > > that Fedora Atomic is right choice,first of all there is no documentation > > about it and I don't think it's used/tested a lot by Docker/Kub/Mesos > > community. > > Project Atomic documentation for the most part lives here: > > http://www.projectatomic.io/docs/ > > To help us improve it, it would be useful to know what you think is missing. > E.g. I saw recently in the IRC channel it was discussed that there is no > documentation on (re)building the image but this is the first hit in a > Google search for same and it seems to largely match what has been copied > into Magnum's docs for same: > > http://www.projectatomic.io/blog/2014/08/build-your-own-atomic-centos-or-fedora/ > > I have no doubt that there are areas where the documentation is lacking, but > it's difficult to resolve a claim that there is no documentation at all. I > recently kicked off a thread over on the atomic list to try and relay some > of the concerns that were raised on this list and in the IRC channel > recently, it would be great if Magnum folks could chime in with more > specifics: > > https://lists.projectatomic.io/projectatomic-archives/atomic/2016-February/thread.html#9 > > Separately I had asked about containerization of kubernetes/etcd/flannel > which remains outstanding: > > https://lists.fedoraproject.org/archives/list/cl...@lists.fedoraproject.org/thread/XICO4NJCTPI43AWG332EIM2HNFYPZ6ON/ > > Fedora Atomic builds do seem to be hitting their planned two weekly update > cadence now though which may alleviate this concern at least somewhat in the > interim: > > https://lists.fedoraproject.org/archives/list/cl...@lists.fedoraproject.org/thread/CW5BQS3ODAVYJGAJGAZ6UA3XQMKEISVJ/ > https://fedorahosted.org/cloud/ticket/139 > > Thanks, > > Steve I meant to add, I don't believe choosing a single operating system image to support - regardless of which it is - is the right move for users and largely agree with what Ton Ngo put forward in his most recent post in the thread. I'm simply highlighting that there are folks willing/able to work on improving things from the Atomic side and we are endeavoring to provide them actionable feedback from the Magnum community to do so. Thanks, Steve > > It make sense to go with Ubuntu (I believe it's still most adopted > > platform in all three COEs and OpenStack deployments) and CoreOS (is > > highly adopted/tested in Kub community and Mesosphere DCOS uses it as > > well). > > We can implement CoreOS support as driver and users can use it as > > reference > > implementation. > > > > --- Egor > > From: Adrian Otto <adrian.o...@rackspace.com> > > To: OpenStack Development Mailing List (not for usage questions) > > <openstack-dev@lists.openstack.org> > > Sent: Monday, February 29, 2016 10:36 AM > >
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
We found some issue about atomic host run some docker volume plugin, while atomic and docker volume plugin both not sure what's the root cause of that. here is the link https://github.com/docker/docker/issues/18005#issuecomment-190215862 Also I did not find atomic image update quickly, ( but k8s and docker both release quickly, which can lacks of new feature applied in our development), I think atomic have a gap for that. Thanks Best Wishes, Kai Qiang Wu (吴开强 Kennan) IBM China System and Technology Lab, Beijing E-mail: wk...@cn.ibm.com Tel: 86-10-82451647 Address: Building 28(Ring Building), ZhongGuanCun Software Park, No.8 Dong Bei Wang West Road, Haidian District Beijing P.R.China 100193 Follow your heart. You are miracle! From: Steve Gordon <sgor...@redhat.com> To: Guz Egor <guz_e...@yahoo.com>, "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org> Cc: Josh Berkus <jber...@redhat.com> Date: 01/03/2016 08:19 pm Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro - Original Message - > From: "Guz Egor" <guz_e...@yahoo.com> > To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org> > > Adrian, > I disagree, host OS is very important for operators because of integration > with all internal tools/repos/etc. > I think it make sense to limit OS support in Magnum main source. But not sure > that Fedora Atomic is right choice,first of all there is no documentation > about it and I don't think it's used/tested a lot by Docker/Kub/Mesos > community. Project Atomic documentation for the most part lives here: http://www.projectatomic.io/docs/ To help us improve it, it would be useful to know what you think is missing. E.g. I saw recently in the IRC channel it was discussed that there is no documentation on (re)building the image but this is the first hit in a Google search for same and it seems to largely match what has been copied into Magnum's docs for same: http://www.projectatomic.io/blog/2014/08/build-your-own-atomic-centos-or-fedora/ I have no doubt that there are areas where the documentation is lacking, but it's difficult to resolve a claim that there is no documentation at all. I recently kicked off a thread over on the atomic list to try and relay some of the concerns that were raised on this list and in the IRC channel recently, it would be great if Magnum folks could chime in with more specifics: https://lists.projectatomic.io/projectatomic-archives/atomic/2016-February/thread.html#9 Separately I had asked about containerization of kubernetes/etcd/flannel which remains outstanding: https://lists.fedoraproject.org/archives/list/cl...@lists.fedoraproject.org/thread/XICO4NJCTPI43AWG332EIM2HNFYPZ6ON/ Fedora Atomic builds do seem to be hitting their planned two weekly update cadence now though which may alleviate this concern at least somewhat in the interim: https://lists.fedoraproject.org/archives/list/cl...@lists.fedoraproject.org/thread/CW5BQS3ODAVYJGAJGAZ6UA3XQMKEISVJ/ https://fedorahosted.org/cloud/ticket/139 Thanks, Steve > It make sense to go with Ubuntu (I believe it's still most adopted > platform in all three COEs and OpenStack deployments) and CoreOS (is > highly adopted/tested in Kub community and Mesosphere DCOS uses it as well). > We can implement CoreOS support as driver and users can use it as reference > implementation. > --- Egor > From: Adrian Otto <adrian.o...@rackspace.com> > To: OpenStack Development Mailing List (not for usage questions) > <openstack-dev@lists.openstack.org> > Sent: Monday, February 29, 2016 10:36 AM > Subject: Re: [openstack-dev] [magnum] Discussion of supporting > single/multiple OS distro > > Consider this: Which OS runs on the bay nodes is not important to end users. > What matters to users is the environments their containers execute in, which > has only one thing in common with the bay node OS: the kernel. The linux > syscall interface is stable enough that the various linux distributions can > all run concurrently in neighboring containers sharing same kernel. There is > really no material reason why the bay OS choice must match what distro the > container is based on. Although I’m persuaded by Hongbin’s concern to > mitigate risk of future changes WRT whatever OS distro is the prevailing one > for bay nodes, there are a few items of concern about duality I’d like to > zero in on: > 1) Participation from Magnum contributors to support the CoreOS speci
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
- Original Message - > From: "Steve Gordon" <sgor...@redhat.com> > To: "Guz Egor" <guz_e...@yahoo.com>, "OpenStack Development Mailing List (not > for usage questions)" > <openstack-dev@lists.openstack.org> > > - Original Message - > > From: "Guz Egor" <guz_e...@yahoo.com> > > To: "OpenStack Development Mailing List (not for usage questions)" > > <openstack-dev@lists.openstack.org> > > > > Adrian, > > I disagree, host OS is very important for operators because of integration > > with all internal tools/repos/etc. > > I think it make sense to limit OS support in Magnum main source. But not > > sure > > that Fedora Atomic is right choice,first of all there is no documentation > > about it and I don't think it's used/tested a lot by Docker/Kub/Mesos > > community. > > Project Atomic documentation for the most part lives here: > > http://www.projectatomic.io/docs/ > > To help us improve it, it would be useful to know what you think is missing. > E.g. I saw recently in the IRC channel it was discussed that there is no > documentation on (re)building the image but this is the first hit in a > Google search for same and it seems to largely match what has been copied > into Magnum's docs for same: > > > http://www.projectatomic.io/blog/2014/08/build-your-own-atomic-centos-or-fedora/ > > I have no doubt that there are areas where the documentation is lacking, but > it's difficult to resolve a claim that there is no documentation at all. I > recently kicked off a thread over on the atomic list to try and relay some > of the concerns that were raised on this list and in the IRC channel > recently, it would be great if Magnum folks could chime in with more > specifics: > > > https://lists.projectatomic.io/projectatomic-archives/atomic/2016-February/thread.html#9 > > Separately I had asked about containerization of kubernetes/etcd/flannel > which remains outstanding: > > > https://lists.fedoraproject.org/archives/list/cl...@lists.fedoraproject.org/thread/XICO4NJCTPI43AWG332EIM2HNFYPZ6ON/ > > Fedora Atomic builds do seem to be hitting their planned two weekly update > cadence now though which may alleviate this concern at least somewhat in the > interim: > > > https://lists.fedoraproject.org/archives/list/cl...@lists.fedoraproject.org/thread/CW5BQS3ODAVYJGAJGAZ6UA3XQMKEISVJ/ > https://fedorahosted.org/cloud/ticket/139 > > Thanks, > > Steve I meant to add, I don't believe choosing a single operating system image to support - regardless of which it is - is the right move for users and largely agree with what Ton Ngo put forward in his most recent post in the thread. I'm simply highlighting that there are folks willing/able to work on improving things from the Atomic side and we are endeavoring to provide them actionable feedback from the Magnum community to do so. Thanks, Steve > > It make sense to go with Ubuntu (I believe it's still most adopted > > platform in all three COEs and OpenStack deployments) and CoreOS (is > > highly adopted/tested in Kub community and Mesosphere DCOS uses it as > > well). > > We can implement CoreOS support as driver and users can use it as > > reference > > implementation. > > > > --- Egor > > From: Adrian Otto <adrian.o...@rackspace.com> > > To: OpenStack Development Mailing List (not for usage questions) > > <openstack-dev@lists.openstack.org> > > Sent: Monday, February 29, 2016 10:36 AM > > Subject: Re: [openstack-dev] [magnum] Discussion of supporting > > single/multiple OS distro > > > > Consider this: Which OS runs on the bay nodes is not important to end > > users. > > What matters to users is the environments their containers execute in, > > which > > has only one thing in common with the bay node OS: the kernel. The linux > > syscall interface is stable enough that the various linux distributions can > > all run concurrently in neighboring containers sharing same kernel. There > > is > > really no material reason why the bay OS choice must match what distro the > > container is based on. Although I’m persuaded by Hongbin’s concern to > > mitigate risk of future changes WRT whatever OS distro is the prevailing > > one > > for bay nodes, there are a few items of concern about duality I’d like to > > zero in on: > > 1) Participation from Magnum contributors to support the CoreOS specific > > template features has been weak in recent months. By comparison, > > participation relating to Fedora/At
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
- Original Message - > From: "Guz Egor" <guz_e...@yahoo.com> > To: "OpenStack Development Mailing List (not for usage questions)" > <openstack-dev@lists.openstack.org> > > Adrian, > I disagree, host OS is very important for operators because of integration > with all internal tools/repos/etc. > I think it make sense to limit OS support in Magnum main source. But not sure > that Fedora Atomic is right choice,first of all there is no documentation > about it and I don't think it's used/tested a lot by Docker/Kub/Mesos > community. Project Atomic documentation for the most part lives here: http://www.projectatomic.io/docs/ To help us improve it, it would be useful to know what you think is missing. E.g. I saw recently in the IRC channel it was discussed that there is no documentation on (re)building the image but this is the first hit in a Google search for same and it seems to largely match what has been copied into Magnum's docs for same: http://www.projectatomic.io/blog/2014/08/build-your-own-atomic-centos-or-fedora/ I have no doubt that there are areas where the documentation is lacking, but it's difficult to resolve a claim that there is no documentation at all. I recently kicked off a thread over on the atomic list to try and relay some of the concerns that were raised on this list and in the IRC channel recently, it would be great if Magnum folks could chime in with more specifics: https://lists.projectatomic.io/projectatomic-archives/atomic/2016-February/thread.html#9 Separately I had asked about containerization of kubernetes/etcd/flannel which remains outstanding: https://lists.fedoraproject.org/archives/list/cl...@lists.fedoraproject.org/thread/XICO4NJCTPI43AWG332EIM2HNFYPZ6ON/ Fedora Atomic builds do seem to be hitting their planned two weekly update cadence now though which may alleviate this concern at least somewhat in the interim: https://lists.fedoraproject.org/archives/list/cl...@lists.fedoraproject.org/thread/CW5BQS3ODAVYJGAJGAZ6UA3XQMKEISVJ/ https://fedorahosted.org/cloud/ticket/139 Thanks, Steve > It make sense to go with Ubuntu (I believe it's still most adopted > platform in all three COEs and OpenStack deployments) and CoreOS (is > highly adopted/tested in Kub community and Mesosphere DCOS uses it as well). > We can implement CoreOS support as driver and users can use it as reference > implementation. > --- Egor > From: Adrian Otto <adrian.o...@rackspace.com> > To: OpenStack Development Mailing List (not for usage questions) > <openstack-dev@lists.openstack.org> > Sent: Monday, February 29, 2016 10:36 AM > Subject: Re: [openstack-dev] [magnum] Discussion of supporting > single/multiple OS distro > > Consider this: Which OS runs on the bay nodes is not important to end users. > What matters to users is the environments their containers execute in, which > has only one thing in common with the bay node OS: the kernel. The linux > syscall interface is stable enough that the various linux distributions can > all run concurrently in neighboring containers sharing same kernel. There is > really no material reason why the bay OS choice must match what distro the > container is based on. Although I’m persuaded by Hongbin’s concern to > mitigate risk of future changes WRT whatever OS distro is the prevailing one > for bay nodes, there are a few items of concern about duality I’d like to > zero in on: > 1) Participation from Magnum contributors to support the CoreOS specific > template features has been weak in recent months. By comparison, > participation relating to Fedora/Atomic have been much stronger. > 2) Properly testing multiple bay node OS distros (would) significantly > increase the run time and complexity of our functional tests. > 3) Having support for multiple bay node OS choices requires more extensive > documentation, and more comprehensive troubleshooting details. > If we proceed with just one supported disto for bay nodes, and offer > extensibility points to allow alternates to be used in place of it, we > should be able to address the risk concern of the chosen distro by selecting > an alternate when that change is needed, by using those extensibility > points. These include the ability to specify your own bay image, and the > ability to use your own associated Heat template. > I see value in risk mitigation, it may make sense to simplify in the short > term and address that need when it becomes necessary. My point of view might > be different if we had contributors willing and ready to address the variety > of drawbacks that accompany the strategy of supporting multiple bay node OS > choices. In absence of such a community interest, my preference is to > simplify to increase our
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
Adrian, I disagree, host OS is very important for operators because of integration with all internal tools/repos/etc. I think it make sense to limit OS support in Magnum main source. But not sure that Fedora Atomic is right choice,first of all there is no documentation about it and I don't think it's used/tested a lot by Docker/Kub/Mesos community.It make sense to go with Ubuntu (I believe it's still most adopted platform in all three COEs and OpenStack deployments) and CoreOS (is highly adopted/tested in Kub community and Mesosphere DCOS uses it as well). We can implement CoreOS support as driver and users can use it as reference implementation. --- Egor From: Adrian Otto <adrian.o...@rackspace.com> To: OpenStack Development Mailing List (not for usage questions) <openstack-dev@lists.openstack.org> Sent: Monday, February 29, 2016 10:36 AM Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro Consider this: Which OS runs on the bay nodes is not important to end users. What matters to users is the environments their containers execute in, which has only one thing in common with the bay node OS: the kernel. The linux syscall interface is stable enough that the various linux distributions can all run concurrently in neighboring containers sharing same kernel. There is really no material reason why the bay OS choice must match what distro the container is based on. Although I’m persuaded by Hongbin’s concern to mitigate risk of future changes WRT whatever OS distro is the prevailing one for bay nodes, there are a few items of concern about duality I’d like to zero in on: 1) Participation from Magnum contributors to support the CoreOS specific template features has been weak in recent months. By comparison, participation relating to Fedora/Atomic have been much stronger. 2) Properly testing multiple bay node OS distros (would) significantly increase the run time and complexity of our functional tests. 3) Having support for multiple bay node OS choices requires more extensive documentation, and more comprehensive troubleshooting details. If we proceed with just one supported disto for bay nodes, and offer extensibility points to allow alternates to be used in place of it, we should be able to address the risk concern of the chosen distro by selecting an alternate when that change is needed, by using those extensibility points. These include the ability to specify your own bay image, and the ability to use your own associated Heat template. I see value in risk mitigation, it may make sense to simplify in the short term and address that need when it becomes necessary. My point of view might be different if we had contributors willing and ready to address the variety of drawbacks that accompany the strategy of supporting multiple bay node OS choices. In absence of such a community interest, my preference is to simplify to increase our velocity. This seems to me to be a relatively easy way to reduce complexity around heat template versioning. What do you think? Thanks, Adrian On Feb 29, 2016, at 8:40 AM, Hongbin Lu <hongbin...@huawei.com> wrote: Hi team, This is a continued discussion from a review [1]. Corey O'Brien suggested to have Magnum support a single OS distro (Atomic). I disagreed. I think we should bring the discussion to here to get broader set of inputs. Corey O'Brien From the midcycle, we decided we weren't going to continue to support 2 different versions of the k8s template. Instead, we were going to maintain the Fedora Atomic version of k8s and remove the coreos templates from the tree. I don't think we should continue to develop features for coreos k8s if that is true. In addition, I don't think we should break the coreos template by adding the trust token as a heat parameter. Hongbin Lu I was on the midcycle and I don't remember any decision to remove CoreOS support. Why you want to remove CoreOS templates from the tree. Please note that this is a very big decision and please discuss it with the team thoughtfully and make sure everyone agree. Corey O'Brien Removing the coreos templates was a part of the COE drivers decision. Since each COE driver will only support 1 distro+version+coe we discussed which ones to support in tree. The decision was that instead of trying to support every distro and every version for every coe, the magnum tree would only have support for 1 version of 1 distro for each of the 3 COEs (swarm/docker/mesos). Since we already are going to support Atomic for swarm, removing coreos and keeping Atomic for kubernetes was the favored choice. Hongbin Lu Strongly disagree. It is a huge risk to support a single distro. The selected distro could die in the future. Who knows. Why make Magnum take this huge risk? Again, the decision of supporting single distro is a very big decision. Please bring it up to the team and have it discuss thoughtfully
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
There seems to be different concerns, so it would be helpful to consider them separately: Ability to accommodate different distros How to support given limited resources: gate tests, developers From the discussion at the midcycle, I think we do have agreement that Magnum must be able to accommodate multiple distros in the form of "drivers". What a driver looks like is yet to be designed but it would likely include images, templates, scripts, template definition, etc. An example was articulated for the use case: financial institutions have strict policy about the specific distros that have been certified for their use, and will insist on using their own images. Having worked with banking customers before, I can attest to this kind of hard restriction, and we would expect these users to develop their own drivers. Magnum current support for the various distros is somewhat ad hoc, but going forward, I am sure we will develop a well designed structure for a distro driver and refactor the current code to fit this structure. Then looking at the current code base, we will probably have drivers for Fedora, Fedora Atomic, CoreOs, Ubuntu. RHEL would be a good exercise to test drive creating a new driver. This leads to the concern of how to properly support all these drivers given the limited resources for the project. For a community based project, support is driven by interest in the community, so the level of support will inevitably be uneven. Over time, something that attracts no interest will eventually be removed, but we should be careful not to remove things prematurely. We can think of several ways to cope with the varied level of support. One way is to have an indication of the maturity of each drivers (similarly to how OpenStack projects are rated). This can be based on some metrics like number of tests, or could also be just a qualitative description like "high, medium, low". This would help users to choose what to use, or to invest in development if there is interest. Another way is to select a few key drivers as fully supported, and put the remaining in a contrib directory to indicate that support for these is on a best effort basis. Drivers can be moved around as interest changes over time. Ton, From: 王华 <wanghua.hum...@gmail.com> To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org> Date: 02/29/2016 06:30 PM Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro I think users need the support for multiple OS choices. Users may want to modify the OS by themselves to meet the requirement of their business. If Magnum only supports a single OS distro, we should have a convenient way to change one OS distro to another. But the OSes are so different, the work is difficult. So if Magnum only supports a single OS distro, the users are locked into one OS distro. Best Regards, Wanghua __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
I think users need the support for multiple OS choices. Users may want to modify the OS by themselves to meet the requirement of their business. If Magnum only supports a single OS distro, we should have a convenient way to change one OS distro to another. But the OSes are so different, the work is difficult. So if Magnum only supports a single OS distro, the users are locked into one OS distro. Best Regards, Wanghua __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
From: Adrian Otto [mailto:adrian.o...@rackspace.com] Sent: February-29-16 1:36 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro Consider this: Which OS runs on the bay nodes is not important to end users. What matters to users is the environments their containers execute in, which has only one thing in common with the bay The bay nodes are under user’s tenant. That means end users can to SSH to the nodes and play with the containers. Therefore, the choice of OS is important to end users. node OS: the kernel. The linux syscall interface is stable enough that the various linux distributions can all run concurrently in neighboring containers sharing same kernel. There is really no material reason why the bay OS choice must match what distro the container is based on. Although I’m persuaded by Hongbin’s concern to mitigate risk of future changes WRT whatever OS distro is the prevailing one for bay nodes, there are a few items of concern about duality I’d like to zero in on: 1) Participation from Magnum contributors to support the CoreOS specific template features has been weak in recent months. By comparison, participation relating to Fedora/Atomic have been much stronger. I have been fixing the CoreOS templates recently. If other contributors are willing to work with me on this efforts, it is reasonable to expect the CoreOS contribution to be stronger. 2) Properly testing multiple bay node OS distros (would) significantly increase the run time and complexity of our functional tests. This is not true technically. We can re-run the Atomic tests on CoreOS by changing a single field (which is the image). What needs to be done is moving common modules into a base class and let OS-specific modules inherit from them. 3) Having support for multiple bay node OS choices requires more extensive documentation, and more comprehensive troubleshooting details. This might be true, but we could point to the troubleshooting document of specific OS. If the selected OS delivered a comprehensive troubleshooting document, this problem is resolved. If we proceed with just one supported disto for bay nodes, and offer extensibility points to allow alternates to be used in place of it, we should be able to address the risk concern of the chosen distro by selecting an alternate when that change is needed, by using those extensibility points. These include the ability to specify your own bay image, and the ability to use your own associated Heat template. I see value in risk mitigation, it may make sense to simplify in the short term and address that need when it becomes necessary. My point of view might be different if we had contributors willing I think it becomes necessary now. I have been working on Magnum starting from the early stage of the project. Probably, I am the most senior active contributor. Based on my experiences, there are a lot of problems of locking in a single OS. Basically, all the issues from OS upstream are populated to Magnum (e.g. we experienced various known/unknown bugs, pain on image building, lack of documentation, lack of upstream support etc.). All these experiences remind me not relying on a single OS, because you never know what will be the next obstacle. and ready to address the variety of drawbacks that accompany the strategy of supporting multiple bay node OS choices. In absence of such a community interest, my preference is to simplify to increase our velocity. This seems to me to be a relatively easy way to reduce complexity around heat template versioning. What do you think? Thanks, Adrian On Feb 29, 2016, at 8:40 AM, Hongbin Lu <hongbin...@huawei.com<mailto:hongbin...@huawei.com>> wrote: Hi team, This is a continued discussion from a review [1]. Corey O'Brien suggested to have Magnum support a single OS distro (Atomic). I disagreed. I think we should bring the discussion to here to get broader set of inputs. Corey O'Brien From the midcycle, we decided we weren't going to continue to support 2 different versions of the k8s template. Instead, we were going to maintain the Fedora Atomic version of k8s and remove the coreos templates from the tree. I don't think we should continue to develop features for coreos k8s if that is true. In addition, I don't think we should break the coreos template by adding the trust token as a heat parameter. Hongbin Lu I was on the midcycle and I don't remember any decision to remove CoreOS support. Why you want to remove CoreOS templates from the tree. Please note that this is a very big decision and please discuss it with the team thoughtfully and make sure everyone agree. Corey O'Brien Removing the coreos templates was a part of the COE drivers decision. Since each COE driver will only support 1 distro+version+coe we discussed which ones to support in tree. The decision was
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
From: Adrian Otto Reply-To: "OpenStack Development Mailing List (not for usage questions)" Date: Monday 29 February 2016 at 19:36 To: "OpenStack Development Mailing List (not for usage questions)" Subject: Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro Consider this: Which OS runs on the bay nodes is not important to end users. What matters to users is the environments their containers execute in, which has only one thing in common with the bay node OS: the kernel. The linux syscall interface is stable enough that the various linux distributions can all run concurrently in neighboring containers sharing same kernel. There is really no material reason why the bay OS choice must match what distro the container is based on. Although I’m persuaded by Hongbin’s concern to mitigate risk of future changes WRT whatever OS distro is the prevailing one for bay nodes, there are a few items of concern about duality I’d like to zero in on: 1) Participation from Magnum contributors to support the CoreOS specific template features has been weak in recent months. By comparison, participation relating to Fedora/Atomic have been much stronger. 2) Properly testing multiple bay node OS distros (would) significantly increase the run time and complexity of our functional tests. 3) Having support for multiple bay node OS choices requires more extensive documentation, and more comprehensive troubleshooting details. If we proceed with just one supported disto for bay nodes, and offer extensibility points to allow alternates to be used in place of it, we should be able to address the risk concern of the chosen distro by selecting an alternate when that change is needed, by using those extensibility points. These include the ability to specify your own bay image, and the ability to use your own associated Heat template. There are potential operator implications if we limited the selection of supported distros for bay nodes too far. There is much that will go on to integrate with the centre’s automation tools to ensure the capacity in these nodes can be managed consistently (such as monitoring, alarming, QA cycles,...). I feel that supporting a Ubuntu flavor and a RHEL/CentOS/SL flavor would cover most scenarios. In any case, a plug in architecture is needed to support innovation but the baseline tests should cover the common use cases such as a RHEL and Ubuntu flavor. I see value in risk mitigation, it may make sense to simplify in the short term and address that need when it becomes necessary. My point of view might be different if we had contributors willing and ready to address the variety of drawbacks that accompany the strategy of supporting multiple bay node OS choices. In absence of such a community interest, my preference is to simplify to increase our velocity. This seems to me to be a relatively easy way to reduce complexity around heat template versioning. What do you think? Thanks, Adrian On Feb 29, 2016, at 8:40 AM, Hongbin Lu <hongbin...@huawei.com> wrote: Hi team, This is a continued discussion from a review [1]. Corey O'Brien suggested to have Magnum support a single OS distro (Atomic). I disagreed. I think we should bring the discussion to here to get broader set of inputs. Corey O'Brien >From the midcycle, we decided we weren't going to continue to support 2 >different versions of the k8s template. Instead, we were going to maintain the >Fedora Atomic version of k8s and remove the coreos templates from the tree. I >don't think we should continue to develop features for coreos k8s if that is >true. In addition, I don't think we should break the coreos template by adding the trust token as a heat parameter. Hongbin Lu I was on the midcycle and I don't remember any decision to remove CoreOS support. Why you want to remove CoreOS templates from the tree. Please note that this is a very big decision and please discuss it with the team thoughtfully and make sure everyone agree. Corey O'Brien Removing the coreos templates was a part of the COE drivers decision. Since each COE driver will only support 1 distro+version+coe we discussed which ones to support in tree. The decision was that instead of trying to support every distro and every version for every coe, the magnum tree would only have support for 1 version of 1 distro for each of the 3 COEs (swarm/docker/mesos). Since we already are going to support Atomic for swarm, removing coreos and keeping Atomic for kubernetes was the favored choice. Hongbin Lu Strongly disagree. It is a huge risk to support a single distro. The selected distro could die in the future. Who knows. Why make Magnum take this huge risk? Again, the decision of supporting single distro is a very big decision. Please bring it up to the team and have it discuss thoughtfully before making any decision. Also, Magnum doesn't have to support ever
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
Consider this: Which OS runs on the bay nodes is not important to end users. What matters to users is the environments their containers execute in, which has only one thing in common with the bay node OS: the kernel. The linux syscall interface is stable enough that the various linux distributions can all run concurrently in neighboring containers sharing same kernel. There is really no material reason why the bay OS choice must match what distro the container is based on. Although I’m persuaded by Hongbin’s concern to mitigate risk of future changes WRT whatever OS distro is the prevailing one for bay nodes, there are a few items of concern about duality I’d like to zero in on: 1) Participation from Magnum contributors to support the CoreOS specific template features has been weak in recent months. By comparison, participation relating to Fedora/Atomic have been much stronger. 2) Properly testing multiple bay node OS distros (would) significantly increase the run time and complexity of our functional tests. 3) Having support for multiple bay node OS choices requires more extensive documentation, and more comprehensive troubleshooting details. If we proceed with just one supported disto for bay nodes, and offer extensibility points to allow alternates to be used in place of it, we should be able to address the risk concern of the chosen distro by selecting an alternate when that change is needed, by using those extensibility points. These include the ability to specify your own bay image, and the ability to use your own associated Heat template. I see value in risk mitigation, it may make sense to simplify in the short term and address that need when it becomes necessary. My point of view might be different if we had contributors willing and ready to address the variety of drawbacks that accompany the strategy of supporting multiple bay node OS choices. In absence of such a community interest, my preference is to simplify to increase our velocity. This seems to me to be a relatively easy way to reduce complexity around heat template versioning. What do you think? Thanks, Adrian On Feb 29, 2016, at 8:40 AM, Hongbin Lu> wrote: Hi team, This is a continued discussion from a review [1]. Corey O'Brien suggested to have Magnum support a single OS distro (Atomic). I disagreed. I think we should bring the discussion to here to get broader set of inputs. Corey O'Brien From the midcycle, we decided we weren't going to continue to support 2 different versions of the k8s template. Instead, we were going to maintain the Fedora Atomic version of k8s and remove the coreos templates from the tree. I don't think we should continue to develop features for coreos k8s if that is true. In addition, I don't think we should break the coreos template by adding the trust token as a heat parameter. Hongbin Lu I was on the midcycle and I don't remember any decision to remove CoreOS support. Why you want to remove CoreOS templates from the tree. Please note that this is a very big decision and please discuss it with the team thoughtfully and make sure everyone agree. Corey O'Brien Removing the coreos templates was a part of the COE drivers decision. Since each COE driver will only support 1 distro+version+coe we discussed which ones to support in tree. The decision was that instead of trying to support every distro and every version for every coe, the magnum tree would only have support for 1 version of 1 distro for each of the 3 COEs (swarm/docker/mesos). Since we already are going to support Atomic for swarm, removing coreos and keeping Atomic for kubernetes was the favored choice. Hongbin Lu Strongly disagree. It is a huge risk to support a single distro. The selected distro could die in the future. Who knows. Why make Magnum take this huge risk? Again, the decision of supporting single distro is a very big decision. Please bring it up to the team and have it discuss thoughtfully before making any decision. Also, Magnum doesn't have to support every distro and every version for every coe, but should support *more than one* popular distro for some COEs (especially for the popular COEs). Corey O'Brien The discussion at the midcycle started from the idea of adding support for RHEL and CentOS. We all discussed and decided that we wouldn't try to support everything in tree. Magnum would provide support in-tree for 1 per COE and the COE driver interface would allow others to add support for their preferred distro out of tree. Hongbin Lu I agreed the part that "we wouldn't try to support everything in tree". That doesn't imply the decision to support single distro. Again, support single distro is a huge risk. Why make Magnum take this huge risk? [1] https://review.openstack.org/#/c/277284/ Best regards, Hongbin __ OpenStack Development Mailing
Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
One of the main reasons for moving to a "bay driver” model was to allow us to focus our efforts. We talked about focusing our support to the distros with a religion around them, e.g. Ubuntu and a Red Hat derivative. Being frank, I do not see much benefit in supporting a small distro if we have support for a big one. We have seen the templates for these various distros languish in the past and become quickly outdated. I would much rather have a concerted effort around a single distro. The way we could support multiple distros in Magnum would be to create a new “bay driver” for that distro+template+template_definition. This set of items would be self contained and would not interact with another bay driver that used that same COE. This will allow that bay driver to move at the pace of the team working on it and explicitly list out the features supported by this bay driver. As it currently stands it is difficult to understand the parity between two distros using the same COE. I raised the concern that this could lead to a duplication of code, but we felt that this refactor had more benefits and we could easily work around this duplication. Tom From: Hongbin Lu <hongbin...@huawei.com<mailto:hongbin...@huawei.com>> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Date: Monday, 29 February 2016 at 16:40 To: "openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> Subject: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro Hi team, This is a continued discussion from a review [1]. Corey O'Brien suggested to have Magnum support a single OS distro (Atomic). I disagreed. I think we should bring the discussion to here to get broader set of inputs. Corey O'Brien From the midcycle, we decided we weren't going to continue to support 2 different versions of the k8s template. Instead, we were going to maintain the Fedora Atomic version of k8s and remove the coreos templates from the tree. I don't think we should continue to develop features for coreos k8s if that is true. In addition, I don't think we should break the coreos template by adding the trust token as a heat parameter. Hongbin Lu I was on the midcycle and I don't remember any decision to remove CoreOS support. Why you want to remove CoreOS templates from the tree. Please note that this is a very big decision and please discuss it with the team thoughtfully and make sure everyone agree. Corey O'Brien Removing the coreos templates was a part of the COE drivers decision. Since each COE driver will only support 1 distro+version+coe we discussed which ones to support in tree. The decision was that instead of trying to support every distro and every version for every coe, the magnum tree would only have support for 1 version of 1 distro for each of the 3 COEs (swarm/docker/mesos). Since we already are going to support Atomic for swarm, removing coreos and keeping Atomic for kubernetes was the favored choice. Hongbin Lu Strongly disagree. It is a huge risk to support a single distro. The selected distro could die in the future. Who knows. Why make Magnum take this huge risk? Again, the decision of supporting single distro is a very big decision. Please bring it up to the team and have it discuss thoughtfully before making any decision. Also, Magnum doesn't have to support every distro and every version for every coe, but should support *more than one* popular distro for some COEs (especially for the popular COEs). Corey O'Brien The discussion at the midcycle started from the idea of adding support for RHEL and CentOS. We all discussed and decided that we wouldn't try to support everything in tree. Magnum would provide support in-tree for 1 per COE and the COE driver interface would allow others to add support for their preferred distro out of tree. Hongbin Lu I agreed the part that "we wouldn't try to support everything in tree". That doesn't imply the decision to support single distro. Again, support single distro is a huge risk. Why make Magnum take this huge risk? [1] https://review.openstack.org/#/c/277284/ Best regards, Hongbin __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [magnum] Discussion of supporting single/multiple OS distro
Hi team, This is a continued discussion from a review [1]. Corey O'Brien suggested to have Magnum support a single OS distro (Atomic). I disagreed. I think we should bring the discussion to here to get broader set of inputs. Corey O'Brien >From the midcycle, we decided we weren't going to continue to support 2 >different versions of the k8s template. Instead, we were going to maintain the >Fedora Atomic version of k8s and remove the coreos templates from the tree. I >don't think we should continue to develop features for coreos k8s if that is >true. In addition, I don't think we should break the coreos template by adding the trust token as a heat parameter. Hongbin Lu I was on the midcycle and I don't remember any decision to remove CoreOS support. Why you want to remove CoreOS templates from the tree. Please note that this is a very big decision and please discuss it with the team thoughtfully and make sure everyone agree. Corey O'Brien Removing the coreos templates was a part of the COE drivers decision. Since each COE driver will only support 1 distro+version+coe we discussed which ones to support in tree. The decision was that instead of trying to support every distro and every version for every coe, the magnum tree would only have support for 1 version of 1 distro for each of the 3 COEs (swarm/docker/mesos). Since we already are going to support Atomic for swarm, removing coreos and keeping Atomic for kubernetes was the favored choice. Hongbin Lu Strongly disagree. It is a huge risk to support a single distro. The selected distro could die in the future. Who knows. Why make Magnum take this huge risk? Again, the decision of supporting single distro is a very big decision. Please bring it up to the team and have it discuss thoughtfully before making any decision. Also, Magnum doesn't have to support every distro and every version for every coe, but should support *more than one* popular distro for some COEs (especially for the popular COEs). Corey O'Brien The discussion at the midcycle started from the idea of adding support for RHEL and CentOS. We all discussed and decided that we wouldn't try to support everything in tree. Magnum would provide support in-tree for 1 per COE and the COE driver interface would allow others to add support for their preferred distro out of tree. Hongbin Lu I agreed the part that "we wouldn't try to support everything in tree". That doesn't imply the decision to support single distro. Again, support single distro is a huge risk. Why make Magnum take this huge risk? [1] https://review.openstack.org/#/c/277284/ Best regards, Hongbin __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev