Re: [openstack-dev] [tripleo] POC patch for using tripleo-repos for repo setup
On Wed, Nov 23, 2016 at 11:07 AM, Ben Nemec wrote: > > > On 11/22/2016 08:18 PM, Emilien Macchi wrote: >> >> Even if I was part of those who approved this feature, I still have >> some comments, inline: >> >> On Tue, Nov 22, 2016 at 1:30 PM, Alex Schultz wrote: >>> >>> On Tue, Nov 22, 2016 at 11:06 AM, Ben Nemec >>> wrote: On 11/21/2016 05:26 PM, Alex Schultz wrote: > > > On Mon, Nov 21, 2016 at 2:57 PM, Ben Nemec > wrote: >> >> >> Hi, >> >> I have a POC patch[1] up to demonstrate the use of the tripleo-repos >> tool >> [2] as a replacement for most of tripleo.sh --repo-setup. It has now >> passed >> all of the CI jobs so I wanted to solicit some feedback. >> >> There are a few changes related to repo naming because the tool names >> repo >> files based on the repo name rather than always calling them something >> generic like delorean.repo. I think it's less confusing to have the >> delorean-newton repo file named delorean-newton.repo, but I'm open to >> discussion on that. >> >> If no one has any major objections to how the tool looks/works right >> now, >> I'll proceed with the work to get it imported into the openstack >> namespace >> as part of TripleO. We can always iterate on it after import too, of >> course, so this isn't a speak now or forever hold your peace >> situation. >> :-) >> > > Why a python standalone application for the management of specifically > (and only?) tripleo repositories. It seems we should be trying to > leverage existing tooling and not hiding the problem behind a python > app. It's not that I enjoy the current method described in the spec > (3 curls, 1 sed, 1 bash thing, and a yum install) but it seems like to > write 586 lines of python and tests might be the wrong approach. > Would it be better to just devote some time to rpm generation for > these and deliver it in discrete RPMs? 'yum install > http://tripleo.org/repos-current.rpm' seems way more straight forward. That's essentially trading "curl ..." for "yum install ..." in the docs. The repo rpm would have to be part of the delorean build so you'd still have to be pointing people at a delorean repo. It would also still require code changes somewhere to handle the mixed current/current-tripleo setup that we run for development and test. Given how specific to tripleo that is I'm not sure how much sense it makes to implement it elsewhere. >>> >>> I'm asking because essentially we're delivering basically static repo >>> files. Which should really be done via RPM. Upgrades and cleanups are >>> already well established practices between RPMs. I'm not seeing the >>> reasoning why a python app. I thought about this further and I'm not >>> sure why this should be done on the client side via an app rather than >>> at repository build/promotion time. As long as we're including the >>> repo rpm, we can always create simple 302 redirects from a webserver >>> to get the latest version. I don't see why we should introduce a >>> client tool for this when the action is really on the repository >>> packaging side. This seems odd doing system configuration via a >>> python script rather than a configuration management tool like >>> ansible, puppet or even just packaging. >>> There are also optional ceph and opstool repos and at least ceph needs to match the version of OpenStack being installed. Sure, you could just document all that logic, but then the logic has to be reimplemented in CI anyway so you end up with code to maintain either way. At least with one tool the logic is shared between the user and dev/test paths, which is one of the primary motivations behind it. Pretty much every place that we have divergence between what users do and what developers do becomes a pain point for users because developers only fix the things they actually use. >>> >>> Yes we should not have a different path for dev/test vs operational >>> deployments, but I'm not convinced we need to add a custom python tool >>> to handle this only for tripleo. There are already established >>> patterns of repository rpm delivery and installation via existing >>> tools. What are we getting from this tool that can't already be done? >> >> >> That is true, here are some of them: >> - centos-release-ceph-(hammer|jewel) rpm that deploys repos. >> - since we are moving TripleO CI to use tripleo-quickstart, we could >> handle repository with Ansible, directly in the roles. > > > This is exactly what I'm trying to avoid here. I want us to be using the > same thing for repo management in CI and dev and real user environments. > Unless we're making quickstart the new API for TripleO this basically > defeats the whole purpose of red
Re: [openstack-dev] [tripleo] POC patch for using tripleo-repos for repo setup
On 11/22/2016 08:18 PM, Emilien Macchi wrote: Even if I was part of those who approved this feature, I still have some comments, inline: On Tue, Nov 22, 2016 at 1:30 PM, Alex Schultz wrote: On Tue, Nov 22, 2016 at 11:06 AM, Ben Nemec wrote: On 11/21/2016 05:26 PM, Alex Schultz wrote: On Mon, Nov 21, 2016 at 2:57 PM, Ben Nemec wrote: Hi, I have a POC patch[1] up to demonstrate the use of the tripleo-repos tool [2] as a replacement for most of tripleo.sh --repo-setup. It has now passed all of the CI jobs so I wanted to solicit some feedback. There are a few changes related to repo naming because the tool names repo files based on the repo name rather than always calling them something generic like delorean.repo. I think it's less confusing to have the delorean-newton repo file named delorean-newton.repo, but I'm open to discussion on that. If no one has any major objections to how the tool looks/works right now, I'll proceed with the work to get it imported into the openstack namespace as part of TripleO. We can always iterate on it after import too, of course, so this isn't a speak now or forever hold your peace situation. :-) Why a python standalone application for the management of specifically (and only?) tripleo repositories. It seems we should be trying to leverage existing tooling and not hiding the problem behind a python app. It's not that I enjoy the current method described in the spec (3 curls, 1 sed, 1 bash thing, and a yum install) but it seems like to write 586 lines of python and tests might be the wrong approach. Would it be better to just devote some time to rpm generation for these and deliver it in discrete RPMs? 'yum install http://tripleo.org/repos-current.rpm' seems way more straight forward. That's essentially trading "curl ..." for "yum install ..." in the docs. The repo rpm would have to be part of the delorean build so you'd still have to be pointing people at a delorean repo. It would also still require code changes somewhere to handle the mixed current/current-tripleo setup that we run for development and test. Given how specific to tripleo that is I'm not sure how much sense it makes to implement it elsewhere. I'm asking because essentially we're delivering basically static repo files. Which should really be done via RPM. Upgrades and cleanups are already well established practices between RPMs. I'm not seeing the reasoning why a python app. I thought about this further and I'm not sure why this should be done on the client side via an app rather than at repository build/promotion time. As long as we're including the repo rpm, we can always create simple 302 redirects from a webserver to get the latest version. I don't see why we should introduce a client tool for this when the action is really on the repository packaging side. This seems odd doing system configuration via a python script rather than a configuration management tool like ansible, puppet or even just packaging. There are also optional ceph and opstool repos and at least ceph needs to match the version of OpenStack being installed. Sure, you could just document all that logic, but then the logic has to be reimplemented in CI anyway so you end up with code to maintain either way. At least with one tool the logic is shared between the user and dev/test paths, which is one of the primary motivations behind it. Pretty much every place that we have divergence between what users do and what developers do becomes a pain point for users because developers only fix the things they actually use. Yes we should not have a different path for dev/test vs operational deployments, but I'm not convinced we need to add a custom python tool to handle this only for tripleo. There are already established patterns of repository rpm delivery and installation via existing tools. What are we getting from this tool that can't already be done? That is true, here are some of them: - centos-release-ceph-(hammer|jewel) rpm that deploys repos. - since we are moving TripleO CI to use tripleo-quickstart, we could handle repository with Ansible, directly in the roles. This is exactly what I'm trying to avoid here. I want us to be using the same thing for repo management in CI and dev and real user environments. Unless we're making quickstart the new API for TripleO this basically defeats the whole purpose of redoing the repo setup IMHO. There are other benefits too - the tool cleans up old repos so you don't have to worry about accidentally being left with a stray repo file that could cause problems. You can also install the repos to a non-system path so you can make use of them without actually installing them on the host system. I'm not entirely clear on the use case for that, but it's something that comes up from time to time. If it's an rpm, the upgrades should cleanup after themselves. Do we have specific and documented use cases where dropping the repo information into an
Re: [openstack-dev] [tripleo] POC patch for using tripleo-repos for repo setup
Even if I was part of those who approved this feature, I still have some comments, inline: On Tue, Nov 22, 2016 at 1:30 PM, Alex Schultz wrote: > On Tue, Nov 22, 2016 at 11:06 AM, Ben Nemec wrote: >> >> >> On 11/21/2016 05:26 PM, Alex Schultz wrote: >>> >>> On Mon, Nov 21, 2016 at 2:57 PM, Ben Nemec wrote: Hi, I have a POC patch[1] up to demonstrate the use of the tripleo-repos tool [2] as a replacement for most of tripleo.sh --repo-setup. It has now passed all of the CI jobs so I wanted to solicit some feedback. There are a few changes related to repo naming because the tool names repo files based on the repo name rather than always calling them something generic like delorean.repo. I think it's less confusing to have the delorean-newton repo file named delorean-newton.repo, but I'm open to discussion on that. If no one has any major objections to how the tool looks/works right now, I'll proceed with the work to get it imported into the openstack namespace as part of TripleO. We can always iterate on it after import too, of course, so this isn't a speak now or forever hold your peace situation. :-) >>> >>> Why a python standalone application for the management of specifically >>> (and only?) tripleo repositories. It seems we should be trying to >>> leverage existing tooling and not hiding the problem behind a python >>> app. It's not that I enjoy the current method described in the spec >>> (3 curls, 1 sed, 1 bash thing, and a yum install) but it seems like to >>> write 586 lines of python and tests might be the wrong approach. >>> Would it be better to just devote some time to rpm generation for >>> these and deliver it in discrete RPMs? 'yum install >>> http://tripleo.org/repos-current.rpm' seems way more straight forward. >> >> >> That's essentially trading "curl ..." for "yum install ..." in the docs. >> The repo rpm would have to be part of the delorean build so you'd still have >> to be pointing people at a delorean repo. It would also still require code >> changes somewhere to handle the mixed current/current-tripleo setup that we >> run for development and test. Given how specific to tripleo that is I'm not >> sure how much sense it makes to implement it elsewhere. >> > > I'm asking because essentially we're delivering basically static repo > files. Which should really be done via RPM. Upgrades and cleanups are > already well established practices between RPMs. I'm not seeing the > reasoning why a python app. I thought about this further and I'm not > sure why this should be done on the client side via an app rather than > at repository build/promotion time. As long as we're including the > repo rpm, we can always create simple 302 redirects from a webserver > to get the latest version. I don't see why we should introduce a > client tool for this when the action is really on the repository > packaging side. This seems odd doing system configuration via a > python script rather than a configuration management tool like > ansible, puppet or even just packaging. > >> There are also optional ceph and opstool repos and at least ceph needs to >> match the version of OpenStack being installed. Sure, you could just >> document all that logic, but then the logic has to be reimplemented in CI >> anyway so you end up with code to maintain either way. At least with one >> tool the logic is shared between the user and dev/test paths, which is one >> of the primary motivations behind it. Pretty much every place that we have >> divergence between what users do and what developers do becomes a pain point >> for users because developers only fix the things they actually use. >> > > Yes we should not have a different path for dev/test vs operational > deployments, but I'm not convinced we need to add a custom python tool > to handle this only for tripleo. There are already established > patterns of repository rpm delivery and installation via existing > tools. What are we getting from this tool that can't already be done? That is true, here are some of them: - centos-release-ceph-(hammer|jewel) rpm that deploys repos. - since we are moving TripleO CI to use tripleo-quickstart, we could handle repository with Ansible, directly in the roles. >> There are other benefits too - the tool cleans up old repos so you don't >> have to worry about accidentally being left with a stray repo file that >> could cause problems. You can also install the repos to a non-system path >> so you can make use of them without actually installing them on the host >> system. I'm not entirely clear on the use case for that, but it's something >> that comes up from time to time. >> > > If it's an rpm, the upgrades should cleanup after themselves. Do we > have specific and documented use cases where dropping the repo > information into an alternative location is required? Even with rpm, > can't we just rpm -
Re: [openstack-dev] [tripleo] POC patch for using tripleo-repos for repo setup
On Tue, Nov 22, 2016 at 11:06 AM, Ben Nemec wrote: > > > On 11/21/2016 05:26 PM, Alex Schultz wrote: >> >> On Mon, Nov 21, 2016 at 2:57 PM, Ben Nemec wrote: >>> >>> Hi, >>> >>> I have a POC patch[1] up to demonstrate the use of the tripleo-repos tool >>> [2] as a replacement for most of tripleo.sh --repo-setup. It has now >>> passed >>> all of the CI jobs so I wanted to solicit some feedback. >>> >>> There are a few changes related to repo naming because the tool names >>> repo >>> files based on the repo name rather than always calling them something >>> generic like delorean.repo. I think it's less confusing to have the >>> delorean-newton repo file named delorean-newton.repo, but I'm open to >>> discussion on that. >>> >>> If no one has any major objections to how the tool looks/works right now, >>> I'll proceed with the work to get it imported into the openstack >>> namespace >>> as part of TripleO. We can always iterate on it after import too, of >>> course, so this isn't a speak now or forever hold your peace situation. >>> :-) >>> >> >> Why a python standalone application for the management of specifically >> (and only?) tripleo repositories. It seems we should be trying to >> leverage existing tooling and not hiding the problem behind a python >> app. It's not that I enjoy the current method described in the spec >> (3 curls, 1 sed, 1 bash thing, and a yum install) but it seems like to >> write 586 lines of python and tests might be the wrong approach. >> Would it be better to just devote some time to rpm generation for >> these and deliver it in discrete RPMs? 'yum install >> http://tripleo.org/repos-current.rpm' seems way more straight forward. > > > That's essentially trading "curl ..." for "yum install ..." in the docs. > The repo rpm would have to be part of the delorean build so you'd still have > to be pointing people at a delorean repo. It would also still require code > changes somewhere to handle the mixed current/current-tripleo setup that we > run for development and test. Given how specific to tripleo that is I'm not > sure how much sense it makes to implement it elsewhere. > I'm asking because essentially we're delivering basically static repo files. Which should really be done via RPM. Upgrades and cleanups are already well established practices between RPMs. I'm not seeing the reasoning why a python app. I thought about this further and I'm not sure why this should be done on the client side via an app rather than at repository build/promotion time. As long as we're including the repo rpm, we can always create simple 302 redirects from a webserver to get the latest version. I don't see why we should introduce a client tool for this when the action is really on the repository packaging side. This seems odd doing system configuration via a python script rather than a configuration management tool like ansible, puppet or even just packaging. > There are also optional ceph and opstool repos and at least ceph needs to > match the version of OpenStack being installed. Sure, you could just > document all that logic, but then the logic has to be reimplemented in CI > anyway so you end up with code to maintain either way. At least with one > tool the logic is shared between the user and dev/test paths, which is one > of the primary motivations behind it. Pretty much every place that we have > divergence between what users do and what developers do becomes a pain point > for users because developers only fix the things they actually use. > Yes we should not have a different path for dev/test vs operational deployments, but I'm not convinced we need to add a custom python tool to handle this only for tripleo. There are already established patterns of repository rpm delivery and installation via existing tools. What are we getting from this tool that can't already be done? > There are other benefits too - the tool cleans up old repos so you don't > have to worry about accidentally being left with a stray repo file that > could cause problems. You can also install the repos to a non-system path > so you can make use of them without actually installing them on the host > system. I'm not entirely clear on the use case for that, but it's something > that comes up from time to time. > If it's an rpm, the upgrades should cleanup after themselves. Do we have specific and documented use cases where dropping the repo information into an alternative location is required? Even with rpm, can't we just rpm --prefix=/some/other/location? Thanks, -Alex > >> >> Thanks, >> -Alex >> >>> 1: https://review.openstack.org/#/c/395813 >>> 2: >>> >>> https://specs.openstack.org/openstack/tripleo-specs/specs/policy/tripleo-repos.html >>> (note that this spec was mistakenly submitted as a policy, it will be >>> moving >>> to the ocata directory soon) >>> >>> Thanks. >>> >>> -Ben >>> >>> >>> __ >>> OpenStack Developmen
Re: [openstack-dev] [tripleo] POC patch for using tripleo-repos for repo setup
On 11/21/2016 05:26 PM, Alex Schultz wrote: On Mon, Nov 21, 2016 at 2:57 PM, Ben Nemec wrote: Hi, I have a POC patch[1] up to demonstrate the use of the tripleo-repos tool [2] as a replacement for most of tripleo.sh --repo-setup. It has now passed all of the CI jobs so I wanted to solicit some feedback. There are a few changes related to repo naming because the tool names repo files based on the repo name rather than always calling them something generic like delorean.repo. I think it's less confusing to have the delorean-newton repo file named delorean-newton.repo, but I'm open to discussion on that. If no one has any major objections to how the tool looks/works right now, I'll proceed with the work to get it imported into the openstack namespace as part of TripleO. We can always iterate on it after import too, of course, so this isn't a speak now or forever hold your peace situation. :-) Why a python standalone application for the management of specifically (and only?) tripleo repositories. It seems we should be trying to leverage existing tooling and not hiding the problem behind a python app. It's not that I enjoy the current method described in the spec (3 curls, 1 sed, 1 bash thing, and a yum install) but it seems like to write 586 lines of python and tests might be the wrong approach. Would it be better to just devote some time to rpm generation for these and deliver it in discrete RPMs? 'yum install http://tripleo.org/repos-current.rpm' seems way more straight forward. That's essentially trading "curl ..." for "yum install ..." in the docs. The repo rpm would have to be part of the delorean build so you'd still have to be pointing people at a delorean repo. It would also still require code changes somewhere to handle the mixed current/current-tripleo setup that we run for development and test. Given how specific to tripleo that is I'm not sure how much sense it makes to implement it elsewhere. There are also optional ceph and opstool repos and at least ceph needs to match the version of OpenStack being installed. Sure, you could just document all that logic, but then the logic has to be reimplemented in CI anyway so you end up with code to maintain either way. At least with one tool the logic is shared between the user and dev/test paths, which is one of the primary motivations behind it. Pretty much every place that we have divergence between what users do and what developers do becomes a pain point for users because developers only fix the things they actually use. There are other benefits too - the tool cleans up old repos so you don't have to worry about accidentally being left with a stray repo file that could cause problems. You can also install the repos to a non-system path so you can make use of them without actually installing them on the host system. I'm not entirely clear on the use case for that, but it's something that comes up from time to time. Thanks, -Alex 1: https://review.openstack.org/#/c/395813 2: https://specs.openstack.org/openstack/tripleo-specs/specs/policy/tripleo-repos.html (note that this spec was mistakenly submitted as a policy, it will be moving to the ocata directory soon) Thanks. -Ben __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tripleo] POC patch for using tripleo-repos for repo setup
On Mon, Nov 21, 2016 at 2:57 PM, Ben Nemec wrote: > Hi, > > I have a POC patch[1] up to demonstrate the use of the tripleo-repos tool > [2] as a replacement for most of tripleo.sh --repo-setup. It has now passed > all of the CI jobs so I wanted to solicit some feedback. > > There are a few changes related to repo naming because the tool names repo > files based on the repo name rather than always calling them something > generic like delorean.repo. I think it's less confusing to have the > delorean-newton repo file named delorean-newton.repo, but I'm open to > discussion on that. > > If no one has any major objections to how the tool looks/works right now, > I'll proceed with the work to get it imported into the openstack namespace > as part of TripleO. We can always iterate on it after import too, of > course, so this isn't a speak now or forever hold your peace situation. :-) > Why a python standalone application for the management of specifically (and only?) tripleo repositories. It seems we should be trying to leverage existing tooling and not hiding the problem behind a python app. It's not that I enjoy the current method described in the spec (3 curls, 1 sed, 1 bash thing, and a yum install) but it seems like to write 586 lines of python and tests might be the wrong approach. Would it be better to just devote some time to rpm generation for these and deliver it in discrete RPMs? 'yum install http://tripleo.org/repos-current.rpm' seems way more straight forward. Thanks, -Alex > 1: https://review.openstack.org/#/c/395813 > 2: > https://specs.openstack.org/openstack/tripleo-specs/specs/policy/tripleo-repos.html > (note that this spec was mistakenly submitted as a policy, it will be moving > to the ocata directory soon) > > Thanks. > > -Ben > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [tripleo] POC patch for using tripleo-repos for repo setup
Hi, I have a POC patch[1] up to demonstrate the use of the tripleo-repos tool [2] as a replacement for most of tripleo.sh --repo-setup. It has now passed all of the CI jobs so I wanted to solicit some feedback. There are a few changes related to repo naming because the tool names repo files based on the repo name rather than always calling them something generic like delorean.repo. I think it's less confusing to have the delorean-newton repo file named delorean-newton.repo, but I'm open to discussion on that. If no one has any major objections to how the tool looks/works right now, I'll proceed with the work to get it imported into the openstack namespace as part of TripleO. We can always iterate on it after import too, of course, so this isn't a speak now or forever hold your peace situation. :-) 1: https://review.openstack.org/#/c/395813 2: https://specs.openstack.org/openstack/tripleo-specs/specs/policy/tripleo-repos.html (note that this spec was mistakenly submitted as a policy, it will be moving to the ocata directory soon) Thanks. -Ben __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
