Re: [openstack-dev] [TripleO] Proposal for a new tool: dlrn-repo
I'm just going to take this opportunity to point out that when RDO moved the dlrn repos this week the tool did not break because requests is smarter than curl. Also, it would have been super nice to have exactly one place to update the repo locations instead of at least 3. On 06/13/2016 03:29 PM, Ben Nemec wrote: > So our documented repo setup steps are three curls, a sed, and a > multi-line bash command. And the best part? That's not even what we > test. The commands we actually use in tripleo.sh --repo-setup consist > of the following: three curls, four seds, and (maybe) the same > multi-line bash command. Although whether that big list of packages in > includepkgs is actually up to date with what we're testing is anybody's > guess because without actually plugging both into a diff tool you > probably can't visually find any differences. > > What is my point? That this whole process is overly complicated and > error-prone. If you miss one of those half dozen plus commands you're > going to end up with a broken repo setup. As one of the first things > that a new user has to do in TripleO, this is a pretty poor introduction > to the project. > > My proposal is an rdo-release-esque project that will handle the repo > setup for you, except that since dlrn doesn't really deal in releases I > think the -repo name makes more sense. Here's a first pass at such a > tool: https://github.com/cybertron/dlrn-repo > > This would reduce the existing commands in tripleo.sh from: > sudo sed -i -e 's%priority=.*%priority=30%' $REPO_PREFIX/delorean-deps.repo > sudo curl -o $REPO_PREFIX/delorean.repo > $DELOREAN_REPO_URL/$DELOREAN_REPO_FILE > sudo sed -i -e 's%priority=.*%priority=20%' $REPO_PREFIX/delorean.repo > sudo curl -o $REPO_PREFIX/delorean-current.repo > http://trunk.rdoproject.org/centos7/current/delorean.repo > sudo sed -i -e 's%priority=.*%priority=10%' > $REPO_PREFIX/delorean-current.repo > sudo sed -i 's/\[delorean\]/\[delorean-current\]/' > $REPO_PREFIX/delorean-current.repo > sudo /bin/bash -c "cat <<-EOF>>$REPO_PREFIX/delorean-current.repo > includepkgs=diskimage-builder,instack,instack-undercloud,os-apply-config,os-cloud-config,os-collect-config,os-net-config,os-refresh-config,python-tripleoclient,tripleo-common,openstack-tripleo-heat-templates,openstack-tripleo-image-elements,openstack-tripleo,openstack-tripleo-puppet-elements > EOF" > sudo yum -y install yum-plugin-priorities > > to: > sudo yum install -y http://tripleo.org/dlrn-repo.rpm # or wherever > sudo dlrn-repo tripleo-current > > As you can see in the readme it also supports the stable branch repos or > running against latest master of everything. > > Overall I think this is clearly a better user experience, and as an > added bonus it would allow us to use the exact same code for repo > management on the user side and in CI, which we can't have with a > developer-specific tool like tripleo.sh. > > There's plenty left to do before this would be fully integrated (import > to TripleO, package, update docs, update CI), so I wanted to solicit > some broader input before pursuing it further. > > Thanks. > > -Ben > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] Proposal for a new tool: dlrn-repo
On 06/13/2016 05:58 PM, Derek Higgins wrote: > On 13 June 2016 at 21:29, Ben Nemec wrote: >> So our documented repo setup steps are three curls, a sed, and a >> multi-line bash command. And the best part? That's not even what we >> test. The commands we actually use in tripleo.sh --repo-setup consist >> of the following: three curls, four seds, and (maybe) the same >> multi-line bash command. Although whether that big list of packages in >> includepkgs is actually up to date with what we're testing is anybody's >> guess because without actually plugging both into a diff tool you >> probably can't visually find any differences. > > Looking at the docs I think we should remove the list of packages > altogether, what we document for people trying to use tripleo should > only include the current-tripleo and deps repositories, as we know > this has passed a periodic CI job. This would reduce the documented > process too just 2 curls. The only place we need to worry about > pulling certain packages from /current is in CI and for devs who need > the absolute most up to date tripleo packages in these two cases > tripleo.sh should be used. This actually touches on another point that I forgot to mention in my initial email, which is that I want to reduce the divergence between the developer workflow and the user one. As long as we have developers using completely different tools to deploy their environments it's going to be hard to get them to care about the user tools. tripleo.sh should be a thin wrapper around the documented deploy process, not something that is required. IMNSHO, of course. > > >> What is my point? That this whole process is overly complicated and >> error-prone. If you miss one of those half dozen plus commands you're >> going to end up with a broken repo setup. As one of the first things >> that a new user has to do in TripleO, this is a pretty poor introduction >> to the project. > > Yup, couldn't agree more here, the simpler we can make things for a > new user the better > >> >> My proposal is an rdo-release-esque project that will handle the repo >> setup for you, except that since dlrn doesn't really deal in releases I >> think the -repo name makes more sense. Here's a first pass at such a >> tool: https://github.com/cybertron/dlrn-repo >> >> This would reduce the existing commands in tripleo.sh from: >> sudo sed -i -e 's%priority=.*%priority=30%' $REPO_PREFIX/delorean-deps.repo >> sudo curl -o $REPO_PREFIX/delorean.repo >> $DELOREAN_REPO_URL/$DELOREAN_REPO_FILE >> sudo sed -i -e 's%priority=.*%priority=20%' $REPO_PREFIX/delorean.repo >> sudo curl -o $REPO_PREFIX/delorean-current.repo >> http://trunk.rdoproject.org/centos7/current/delorean.repo >> sudo sed -i -e 's%priority=.*%priority=10%' >> $REPO_PREFIX/delorean-current.repo >> sudo sed -i 's/\[delorean\]/\[delorean-current\]/' >> $REPO_PREFIX/delorean-current.repo >> sudo /bin/bash -c "cat <<-EOF>>$REPO_PREFIX/delorean-current.repo >> includepkgs=diskimage-builder,instack,instack-undercloud,os-apply-config,os-cloud-config,os-collect-config,os-net-config,os-refresh-config,python-tripleoclient,tripleo-common,openstack-tripleo-heat-templates,openstack-tripleo-image-elements,openstack-tripleo,openstack-tripleo-puppet-elements >> EOF" >> sudo yum -y install yum-plugin-priorities >> >> to: >> sudo yum install -y http://tripleo.org/dlrn-repo.rpm # or wherever >> sudo dlrn-repo tripleo-current >> >> As you can see in the readme it also supports the stable branch repos or >> running against latest master of everything. >> >> Overall I think this is clearly a better user experience, and as an >> added bonus it would allow us to use the exact same code for repo >> management on the user side and in CI, which we can't have with a >> developer-specific tool like tripleo.sh. >> >> There's plenty left to do before this would be fully integrated (import >> to TripleO, package, update docs, update CI), so I wanted to solicit >> some broader input before pursuing it further. > > I'm a little on the fence about this, I think the main problem you > bring up is the duplication of the includepkgs list, which I think we > can just remove from the docs, so whats left is the ugly blurb of > script in tripleo.sh --repo-setup, using a tool to do this certainly > improves the code, but does the creation of a new project complicate > things in its own way? > > If we do go ahead with this the one suggestion I would have is > s/dlrn/trunk/g > delorean is the tool used to create trunk repositories, we shouldn't > care, it may even change some day, we are just dealing with trunk > repositories Hmm, trunk-repo would be extremely generic. Maybe os-trunk-repo? Are there plans to use dlrn for non-OpenStack things in the long term? > >> >> Thanks. >> >> -Ben >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.opens
Re: [openstack-dev] [TripleO] Proposal for a new tool: dlrn-repo
On 13 June 2016 at 21:29, Ben Nemec wrote: > So our documented repo setup steps are three curls, a sed, and a > multi-line bash command. And the best part? That's not even what we > test. The commands we actually use in tripleo.sh --repo-setup consist > of the following: three curls, four seds, and (maybe) the same > multi-line bash command. Although whether that big list of packages in > includepkgs is actually up to date with what we're testing is anybody's > guess because without actually plugging both into a diff tool you > probably can't visually find any differences. Looking at the docs I think we should remove the list of packages altogether, what we document for people trying to use tripleo should only include the current-tripleo and deps repositories, as we know this has passed a periodic CI job. This would reduce the documented process too just 2 curls. The only place we need to worry about pulling certain packages from /current is in CI and for devs who need the absolute most up to date tripleo packages in these two cases tripleo.sh should be used. > What is my point? That this whole process is overly complicated and > error-prone. If you miss one of those half dozen plus commands you're > going to end up with a broken repo setup. As one of the first things > that a new user has to do in TripleO, this is a pretty poor introduction > to the project. Yup, couldn't agree more here, the simpler we can make things for a new user the better > > My proposal is an rdo-release-esque project that will handle the repo > setup for you, except that since dlrn doesn't really deal in releases I > think the -repo name makes more sense. Here's a first pass at such a > tool: https://github.com/cybertron/dlrn-repo > > This would reduce the existing commands in tripleo.sh from: > sudo sed -i -e 's%priority=.*%priority=30%' $REPO_PREFIX/delorean-deps.repo > sudo curl -o $REPO_PREFIX/delorean.repo > $DELOREAN_REPO_URL/$DELOREAN_REPO_FILE > sudo sed -i -e 's%priority=.*%priority=20%' $REPO_PREFIX/delorean.repo > sudo curl -o $REPO_PREFIX/delorean-current.repo > http://trunk.rdoproject.org/centos7/current/delorean.repo > sudo sed -i -e 's%priority=.*%priority=10%' > $REPO_PREFIX/delorean-current.repo > sudo sed -i 's/\[delorean\]/\[delorean-current\]/' > $REPO_PREFIX/delorean-current.repo > sudo /bin/bash -c "cat <<-EOF>>$REPO_PREFIX/delorean-current.repo > includepkgs=diskimage-builder,instack,instack-undercloud,os-apply-config,os-cloud-config,os-collect-config,os-net-config,os-refresh-config,python-tripleoclient,tripleo-common,openstack-tripleo-heat-templates,openstack-tripleo-image-elements,openstack-tripleo,openstack-tripleo-puppet-elements > EOF" > sudo yum -y install yum-plugin-priorities > > to: > sudo yum install -y http://tripleo.org/dlrn-repo.rpm # or wherever > sudo dlrn-repo tripleo-current > > As you can see in the readme it also supports the stable branch repos or > running against latest master of everything. > > Overall I think this is clearly a better user experience, and as an > added bonus it would allow us to use the exact same code for repo > management on the user side and in CI, which we can't have with a > developer-specific tool like tripleo.sh. > > There's plenty left to do before this would be fully integrated (import > to TripleO, package, update docs, update CI), so I wanted to solicit > some broader input before pursuing it further. I'm a little on the fence about this, I think the main problem you bring up is the duplication of the includepkgs list, which I think we can just remove from the docs, so whats left is the ugly blurb of script in tripleo.sh --repo-setup, using a tool to do this certainly improves the code, but does the creation of a new project complicate things in its own way? If we do go ahead with this the one suggestion I would have is s/dlrn/trunk/g delorean is the tool used to create trunk repositories, we shouldn't care, it may even change some day, we are just dealing with trunk repositories > > Thanks. > > -Ben > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [TripleO] Proposal for a new tool: dlrn-repo
So our documented repo setup steps are three curls, a sed, and a multi-line bash command. And the best part? That's not even what we test. The commands we actually use in tripleo.sh --repo-setup consist of the following: three curls, four seds, and (maybe) the same multi-line bash command. Although whether that big list of packages in includepkgs is actually up to date with what we're testing is anybody's guess because without actually plugging both into a diff tool you probably can't visually find any differences. What is my point? That this whole process is overly complicated and error-prone. If you miss one of those half dozen plus commands you're going to end up with a broken repo setup. As one of the first things that a new user has to do in TripleO, this is a pretty poor introduction to the project. My proposal is an rdo-release-esque project that will handle the repo setup for you, except that since dlrn doesn't really deal in releases I think the -repo name makes more sense. Here's a first pass at such a tool: https://github.com/cybertron/dlrn-repo This would reduce the existing commands in tripleo.sh from: sudo sed -i -e 's%priority=.*%priority=30%' $REPO_PREFIX/delorean-deps.repo sudo curl -o $REPO_PREFIX/delorean.repo $DELOREAN_REPO_URL/$DELOREAN_REPO_FILE sudo sed -i -e 's%priority=.*%priority=20%' $REPO_PREFIX/delorean.repo sudo curl -o $REPO_PREFIX/delorean-current.repo http://trunk.rdoproject.org/centos7/current/delorean.repo sudo sed -i -e 's%priority=.*%priority=10%' $REPO_PREFIX/delorean-current.repo sudo sed -i 's/\[delorean\]/\[delorean-current\]/' $REPO_PREFIX/delorean-current.repo sudo /bin/bash -c "cat <<-EOF>>$REPO_PREFIX/delorean-current.repo includepkgs=diskimage-builder,instack,instack-undercloud,os-apply-config,os-cloud-config,os-collect-config,os-net-config,os-refresh-config,python-tripleoclient,tripleo-common,openstack-tripleo-heat-templates,openstack-tripleo-image-elements,openstack-tripleo,openstack-tripleo-puppet-elements EOF" sudo yum -y install yum-plugin-priorities to: sudo yum install -y http://tripleo.org/dlrn-repo.rpm # or wherever sudo dlrn-repo tripleo-current As you can see in the readme it also supports the stable branch repos or running against latest master of everything. Overall I think this is clearly a better user experience, and as an added bonus it would allow us to use the exact same code for repo management on the user side and in CI, which we can't have with a developer-specific tool like tripleo.sh. There's plenty left to do before this would be fully integrated (import to TripleO, package, update docs, update CI), so I wanted to solicit some broader input before pursuing it further. Thanks. -Ben __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
