Re: [openstack-dev] [requirements][stable] EOL tags and upper-constraints.txt in tox.ini
On 17-09-22 15:07:14, Doug Hellmann wrote:
> Excerpts from Tony Breeds's message of 2017-09-21 20:20:22 -0500:
> > On Thu, Sep 21, 2017 at 12:13:23PM -0400, Doug Hellmann wrote:
> > > Excerpts from Tony Breeds's message of 2017-09-21 08:36:39 -0400:
> > > > On Wed, Sep 20, 2017 at 06:08:22PM -0400, Doug Hellmann wrote:
> > > >
> > > > > I like the idea. I'm not sure why, if the constraints file is only
> > > > > used
> > > > > for the dependency installation step, we still need tox_install.sh?
> > > >
> > > > Right now that isn't true, when we get something like my idea
> > > > implemented we'd still need the tox_install.sh in projects that need
> > > > services (not published on pypi) like horizon plugins or neutron stadium
> > > > projects. Fixing that issue is a totally different discussion, one I
> > > > started at the PTG but I need to let those conversations settle and do
> > > > research on wasy to fix that.
> > > >
> > > > > If
> > > > > that's just to avoid updating the URL when we create branches, I can
> > > > > live with continuing to do that step if we figure out some other way
> > > > > to
> > > > > minimize the open race window.
> > > >
> > > > So lets check we're on the same page with the race window point. At the
> > > > moment the process looks like:
> > > > 1. projects tag RC1 and when generate a stable/series branch.
> > > > 2. We generate a reviews that updates .gitreview
> > > > 3. We generate a reviews that updates .tox.ini
> > > > 4. time passes
> > > > 5. requirements creates a stable/series branch
> > > > 6. requirements thaws
> > > >
> > > > Now the race is that if projects merge the patch from step 3 before step
> > > > 5 they're broken (on stable/series) because there isn't a
> > > > 'stable/series' in the requirements repo. There are some additional
> > > > issues
> > > > for cycle-trailing projects but nothing radically different.
> > > >
> > > > Correct?
> > > >
> > > > Assuming I have that right In the new world:
> > > >
> > > > 0. requirements publish master.txt and series.txt
> > > > 1. projects tag RC1 and when generate a stable/series branch.
> > > > 2. We generate a reviews that updates .gitreview
> > > > 3. We generate a reviews that updates .tox.ini
> > > > 4. time passes
> > > > 5. requirements creates a stable/series branch
> > > > 6. requiremenst now publish series.txt, new_series.txt and master.txt
> > > > 6. requirements thaws
> > >
> > > Where in that sequence do we make the change so that we're not
> > > publishing to series.txt from the new stable branch in requirements and
> > > from master in requirements? Between step 4 and 5? Or is the job smart
> > > enough to not do that?
> >
> > Right now the job is dumb, but yes we'd teach the job to detect that's
> > it's a stable branch and only publish it's series branch. We also teach
> > the job to not publish to $series.txt if that stable branch exists.
> >
> > So I think the publish job looks like:
> >
> > ---
> > # preamble
> > # typed directly into email so be warned ;P
> > # We probably want to check if ZUUL_BRANCH is the correct variable to
> > # use.
> > case "$ZUUL_BRANCH" in
> > stable/*)
> > series=$(basename "$ZUUL_BRANCH")
> > git show origin/$ZUUL_BRANCH:upper-constraints.txt >
> > publish/constraints/upper/${series}.txt
> > ;;
> > master)
> > for series in queens master ; do
> > if ! git rev-parse origin/stable/${series} ; then
> > git show origin/master:upper-constraints.txt >
> > publish/constraints/upper/${series}.txt
> > fi
> > done
> > ;;
> > esac
> > # postample
> > ---
> >
> > So using the queens release as an example:
> >
> > Jan 22 - Jan 26R-5 Requirements freeze
> > NOTES: openstack/requirements (master) publishes
> > {queens,master}.txt
> > Jan 29 - Feb 02R-4
> > Feb 05 - Feb 09R-3RC1 target week
> > ACTIONS: Projects create stable/queens branches
> > ACTIONS: Generate .gitreview and UPPER_CONSTRAINTS
> > changes
> > ACTIONS: Projects merge .gitreview and
> > UPPER_CONSTRAINTS changes
> > Feb 12 - Feb 16R-2
> > ACTIONS: Projects create stable/queens branch for
> > openstack/requirements
> > ACTIONS: Generate .gitreview and UPPER_CONSTRAINTS
> > changes
> > ACTIONS: Projects merge .gitreview and
> > UPPER_CONSTRAINTS changes
> > ACTIONS: Make sure master publishes {rocky,master}.txt
> > (optionally add the S release at this point,
> > it doesn't hurt)
>
> We could add new "future" series names as soon as we know them, since we
> would just be publishing to a file that nothing uses.
>
> > Feb 19 - Feb 23R-1
> > Feb 26 - Mar 02R+0Queens release
> > Mar 05 - Mar 09R+1
> > Mar 12 - Mar 16R+2Queens cycle-trailing release deadline
> >
> > There's a whole other side
Re: [openstack-dev] [requirements][stable] EOL tags and upper-constraints.txt in tox.ini
Excerpts from Tony Breeds's message of 2017-09-21 20:20:22 -0500:
> On Thu, Sep 21, 2017 at 12:13:23PM -0400, Doug Hellmann wrote:
> > Excerpts from Tony Breeds's message of 2017-09-21 08:36:39 -0400:
> > > On Wed, Sep 20, 2017 at 06:08:22PM -0400, Doug Hellmann wrote:
> > >
> > > > I like the idea. I'm not sure why, if the constraints file is only used
> > > > for the dependency installation step, we still need tox_install.sh?
> > >
> > > Right now that isn't true, when we get something like my idea
> > > implemented we'd still need the tox_install.sh in projects that need
> > > services (not published on pypi) like horizon plugins or neutron stadium
> > > projects. Fixing that issue is a totally different discussion, one I
> > > started at the PTG but I need to let those conversations settle and do
> > > research on wasy to fix that.
> > >
> > > > If
> > > > that's just to avoid updating the URL when we create branches, I can
> > > > live with continuing to do that step if we figure out some other way to
> > > > minimize the open race window.
> > >
> > > So lets check we're on the same page with the race window point. At the
> > > moment the process looks like:
> > > 1. projects tag RC1 and when generate a stable/series branch.
> > > 2. We generate a reviews that updates .gitreview
> > > 3. We generate a reviews that updates .tox.ini
> > > 4. time passes
> > > 5. requirements creates a stable/series branch
> > > 6. requirements thaws
> > >
> > > Now the race is that if projects merge the patch from step 3 before step
> > > 5 they're broken (on stable/series) because there isn't a
> > > 'stable/series' in the requirements repo. There are some additional
> > > issues
> > > for cycle-trailing projects but nothing radically different.
> > >
> > > Correct?
> > >
> > > Assuming I have that right In the new world:
> > >
> > > 0. requirements publish master.txt and series.txt
> > > 1. projects tag RC1 and when generate a stable/series branch.
> > > 2. We generate a reviews that updates .gitreview
> > > 3. We generate a reviews that updates .tox.ini
> > > 4. time passes
> > > 5. requirements creates a stable/series branch
> > > 6. requiremenst now publish series.txt, new_series.txt and master.txt
> > > 6. requirements thaws
> >
> > Where in that sequence do we make the change so that we're not
> > publishing to series.txt from the new stable branch in requirements and
> > from master in requirements? Between step 4 and 5? Or is the job smart
> > enough to not do that?
>
> Right now the job is dumb, but yes we'd teach the job to detect that's
> it's a stable branch and only publish it's series branch. We also teach
> the job to not publish to $series.txt if that stable branch exists.
>
> So I think the publish job looks like:
>
> ---
> # preamble
> # typed directly into email so be warned ;P
> # We probably want to check if ZUUL_BRANCH is the correct variable to
> # use.
> case "$ZUUL_BRANCH" in
> stable/*)
> series=$(basename "$ZUUL_BRANCH")
> git show origin/$ZUUL_BRANCH:upper-constraints.txt >
> publish/constraints/upper/${series}.txt
> ;;
> master)
> for series in queens master ; do
> if ! git rev-parse origin/stable/${series} ; then
> git show origin/master:upper-constraints.txt >
> publish/constraints/upper/${series}.txt
> fi
> done
> ;;
> esac
> # postample
> ---
>
> So using the queens release as an example:
>
> Jan 22 - Jan 26R-5 Requirements freeze
> NOTES: openstack/requirements (master) publishes
> {queens,master}.txt
> Jan 29 - Feb 02R-4
> Feb 05 - Feb 09R-3RC1 target week
> ACTIONS: Projects create stable/queens branches
> ACTIONS: Generate .gitreview and UPPER_CONSTRAINTS changes
> ACTIONS: Projects merge .gitreview and UPPER_CONSTRAINTS
> changes
> Feb 12 - Feb 16R-2
> ACTIONS: Projects create stable/queens branch for
> openstack/requirements
> ACTIONS: Generate .gitreview and UPPER_CONSTRAINTS changes
> ACTIONS: Projects merge .gitreview and UPPER_CONSTRAINTS
> changes
> ACTIONS: Make sure master publishes {rocky,master}.txt
> (optionally add the S release at this point, it
> doesn't hurt)
We could add new "future" series names as soon as we know them, since we
would just be publishing to a file that nothing uses.
> Feb 19 - Feb 23R-1
> Feb 26 - Mar 02R+0Queens release
> Mar 05 - Mar 09R+1
> Mar 12 - Mar 16R+2Queens cycle-trailing release deadline
>
> There's a whole other side issue about how long requirements is frozen
> for. Ignoring that do you think the above process will remove the race,
> and mean that EOL branches can possibly continue to run tests?
>
>
> Yours Tony.
Yes, that does look like a sound approach.
Doug
Re: [openstack-dev] [requirements][stable] EOL tags and upper-constraints.txt in tox.ini
On Thu, Sep 21, 2017 at 12:13:23PM -0400, Doug Hellmann wrote:
> Excerpts from Tony Breeds's message of 2017-09-21 08:36:39 -0400:
> > On Wed, Sep 20, 2017 at 06:08:22PM -0400, Doug Hellmann wrote:
> >
> > > I like the idea. I'm not sure why, if the constraints file is only used
> > > for the dependency installation step, we still need tox_install.sh?
> >
> > Right now that isn't true, when we get something like my idea
> > implemented we'd still need the tox_install.sh in projects that need
> > services (not published on pypi) like horizon plugins or neutron stadium
> > projects. Fixing that issue is a totally different discussion, one I
> > started at the PTG but I need to let those conversations settle and do
> > research on wasy to fix that.
> >
> > > If
> > > that's just to avoid updating the URL when we create branches, I can
> > > live with continuing to do that step if we figure out some other way to
> > > minimize the open race window.
> >
> > So lets check we're on the same page with the race window point. At the
> > moment the process looks like:
> > 1. projects tag RC1 and when generate a stable/series branch.
> > 2. We generate a reviews that updates .gitreview
> > 3. We generate a reviews that updates .tox.ini
> > 4. time passes
> > 5. requirements creates a stable/series branch
> > 6. requirements thaws
> >
> > Now the race is that if projects merge the patch from step 3 before step
> > 5 they're broken (on stable/series) because there isn't a
> > 'stable/series' in the requirements repo. There are some additional issues
> > for cycle-trailing projects but nothing radically different.
> >
> > Correct?
> >
> > Assuming I have that right In the new world:
> >
> > 0. requirements publish master.txt and series.txt
> > 1. projects tag RC1 and when generate a stable/series branch.
> > 2. We generate a reviews that updates .gitreview
> > 3. We generate a reviews that updates .tox.ini
> > 4. time passes
> > 5. requirements creates a stable/series branch
> > 6. requiremenst now publish series.txt, new_series.txt and master.txt
> > 6. requirements thaws
>
> Where in that sequence do we make the change so that we're not
> publishing to series.txt from the new stable branch in requirements and
> from master in requirements? Between step 4 and 5? Or is the job smart
> enough to not do that?
Right now the job is dumb, but yes we'd teach the job to detect that's
it's a stable branch and only publish it's series branch. We also teach
the job to not publish to $series.txt if that stable branch exists.
So I think the publish job looks like:
---
# preamble
# typed directly into email so be warned ;P
# We probably want to check if ZUUL_BRANCH is the correct variable to
# use.
case "$ZUUL_BRANCH" in
stable/*)
series=$(basename "$ZUUL_BRANCH")
git show origin/$ZUUL_BRANCH:upper-constraints.txt >
publish/constraints/upper/${series}.txt
;;
master)
for series in queens master ; do
if ! git rev-parse origin/stable/${series} ; then
git show origin/master:upper-constraints.txt >
publish/constraints/upper/${series}.txt
fi
done
;;
esac
# postample
---
So using the queens release as an example:
Jan 22 - Jan 26 R-5 Requirements freeze
NOTES: openstack/requirements (master) publishes
{queens,master}.txt
Jan 29 - Feb 02 R-4
Feb 05 - Feb 09 R-3 RC1 target week
ACTIONS: Projects create stable/queens branches
ACTIONS: Generate .gitreview and UPPER_CONSTRAINTS changes
ACTIONS: Projects merge .gitreview and UPPER_CONSTRAINTS
changes
Feb 12 - Feb 16 R-2
ACTIONS: Projects create stable/queens branch for
openstack/requirements
ACTIONS: Generate .gitreview and UPPER_CONSTRAINTS changes
ACTIONS: Projects merge .gitreview and UPPER_CONSTRAINTS
changes
ACTIONS: Make sure master publishes {rocky,master}.txt
(optionally add the S release at this point, it
doesn't hurt)
Feb 19 - Feb 23 R-1
Feb 26 - Mar 02 R+0 Queens release
Mar 05 - Mar 09 R+1
Mar 12 - Mar 16 R+2 Queens cycle-trailing release deadline
There's a whole other side issue about how long requirements is frozen
for. Ignoring that do you think the above process will remove the race,
and mean that EOL branches can possibly continue to run tests?
Yours Tony.
signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements][stable] EOL tags and upper-constraints.txt in tox.ini
On 17-09-21 12:13:23, Doug Hellmann wrote: > Excerpts from Tony Breeds's message of 2017-09-21 08:36:39 -0400: > > On Wed, Sep 20, 2017 at 06:08:22PM -0400, Doug Hellmann wrote: > > > > > I like the idea. I'm not sure why, if the constraints file is only used > > > for the dependency installation step, we still need tox_install.sh? > > > > Right now that isn't true, when we get something like my idea > > implemented we'd still need the tox_install.sh in projects that need > > services (not published on pypi) like horizon plugins or neutron stadium > > projects. Fixing that issue is a totally different discussion, one I > > started at the PTG but I need to let those conversations settle and do > > research on wasy to fix that. > > > > > If > > > that's just to avoid updating the URL when we create branches, I can > > > live with continuing to do that step if we figure out some other way to > > > minimize the open race window. > > > > So lets check we're on the same page with the race window point. At the > > moment the process looks like: > > 1. projects tag RC1 and when generate a stable/series branch. > > 2. We generate a reviews that updates .gitreview > > 3. We generate a reviews that updates .tox.ini > > 4. time passes > > 5. requirements creates a stable/series branch > > 6. requirements thaws > > > > Now the race is that if projects merge the patch from step 3 before step > > 5 they're broken (on stable/series) because there isn't a > > 'stable/series' in the requirements repo. There are some additional issues > > for cycle-trailing projects but nothing radically different. > > > > Correct? > > > > Assuming I have that right In the new world: > > > > 0. requirements publish master.txt and series.txt > > 1. projects tag RC1 and when generate a stable/series branch. > > 2. We generate a reviews that updates .gitreview > > 3. We generate a reviews that updates .tox.ini > > 4. time passes > > 5. requirements creates a stable/series branch > > 6. requiremenst now publish series.txt, new_series.txt and master.txt > > 6. requirements thaws > > Where in that sequence do we make the change so that we're not > publishing to series.txt from the new stable branch in requirements and > from master in requirements? Between step 4 and 5? Or is the job smart > enough to not do that? > > Where in the sequence do we add new_series.txt? Also between 4 and 5? > That step of switching from publishing series.txt to new-series.txt happens in step 6. Step 6 should be dependant on step 5's patchset/review. The requirements freeze itself hapens some time during step 4 I think. > > In this scenario We've removed the race as there is a series.txt file > > available befoer the project and requirements branch. > > > > Also[1] if, right now, projects used queens.txt we wouldn't need to > > update tox.ini when we branch stable/queens, but we would need to update > > master. This is a point of confusion that we'll need to document and > > possible check for somewhere in our tools. > > > > Yours Tony. > > > > [1] This just occurred to me > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Matthew Thode (prometheanfire) signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements][stable] EOL tags and upper-constraints.txt in tox.ini
Excerpts from Tony Breeds's message of 2017-09-21 08:36:39 -0400: > On Wed, Sep 20, 2017 at 06:08:22PM -0400, Doug Hellmann wrote: > > > I like the idea. I'm not sure why, if the constraints file is only used > > for the dependency installation step, we still need tox_install.sh? > > Right now that isn't true, when we get something like my idea > implemented we'd still need the tox_install.sh in projects that need > services (not published on pypi) like horizon plugins or neutron stadium > projects. Fixing that issue is a totally different discussion, one I > started at the PTG but I need to let those conversations settle and do > research on wasy to fix that. > > > If > > that's just to avoid updating the URL when we create branches, I can > > live with continuing to do that step if we figure out some other way to > > minimize the open race window. > > So lets check we're on the same page with the race window point. At the > moment the process looks like: > 1. projects tag RC1 and when generate a stable/series branch. > 2. We generate a reviews that updates .gitreview > 3. We generate a reviews that updates .tox.ini > 4. time passes > 5. requirements creates a stable/series branch > 6. requirements thaws > > Now the race is that if projects merge the patch from step 3 before step > 5 they're broken (on stable/series) because there isn't a > 'stable/series' in the requirements repo. There are some additional issues > for cycle-trailing projects but nothing radically different. > > Correct? > > Assuming I have that right In the new world: > > 0. requirements publish master.txt and series.txt > 1. projects tag RC1 and when generate a stable/series branch. > 2. We generate a reviews that updates .gitreview > 3. We generate a reviews that updates .tox.ini > 4. time passes > 5. requirements creates a stable/series branch > 6. requiremenst now publish series.txt, new_series.txt and master.txt > 6. requirements thaws Where in that sequence do we make the change so that we're not publishing to series.txt from the new stable branch in requirements and from master in requirements? Between step 4 and 5? Or is the job smart enough to not do that? Where in the sequence do we add new_series.txt? Also between 4 and 5? > In this scenario We've removed the race as there is a series.txt file > available befoer the project and requirements branch. > > Also[1] if, right now, projects used queens.txt we wouldn't need to > update tox.ini when we branch stable/queens, but we would need to update > master. This is a point of confusion that we'll need to document and > possible check for somewhere in our tools. > > Yours Tony. > > [1] This just occurred to me __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements][stable] EOL tags and upper-constraints.txt in tox.ini
On Wed, Sep 20, 2017 at 06:08:22PM -0400, Doug Hellmann wrote: > I like the idea. I'm not sure why, if the constraints file is only used > for the dependency installation step, we still need tox_install.sh? Right now that isn't true, when we get something like my idea implemented we'd still need the tox_install.sh in projects that need services (not published on pypi) like horizon plugins or neutron stadium projects. Fixing that issue is a totally different discussion, one I started at the PTG but I need to let those conversations settle and do research on wasy to fix that. > If > that's just to avoid updating the URL when we create branches, I can > live with continuing to do that step if we figure out some other way to > minimize the open race window. So lets check we're on the same page with the race window point. At the moment the process looks like: 1. projects tag RC1 and when generate a stable/series branch. 2. We generate a reviews that updates .gitreview 3. We generate a reviews that updates .tox.ini 4. time passes 5. requirements creates a stable/series branch 6. requirements thaws Now the race is that if projects merge the patch from step 3 before step 5 they're broken (on stable/series) because there isn't a 'stable/series' in the requirements repo. There are some additional issues for cycle-trailing projects but nothing radically different. Correct? Assuming I have that right In the new world: 0. requirements publish master.txt and series.txt 1. projects tag RC1 and when generate a stable/series branch. 2. We generate a reviews that updates .gitreview 3. We generate a reviews that updates .tox.ini 4. time passes 5. requirements creates a stable/series branch 6. requiremenst now publish series.txt, new_series.txt and master.txt 6. requirements thaws In this scenario We've removed the race as there is a series.txt file available befoer the project and requirements branch. Also[1] if, right now, projects used queens.txt we wouldn't need to update tox.ini when we branch stable/queens, but we would need to update master. This is a point of confusion that we'll need to document and possible check for somewhere in our tools. Yours Tony. [1] This just occurred to me signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements][stable] EOL tags and upper-constraints.txt in tox.ini
Excerpts from Tony Breeds's message of 2017-09-20 17:18:51 -0400:
> On Wed, Sep 20, 2017 at 04:51:21PM -0400, Doug Hellmann wrote:
> > Excerpts from Jeremy Stanley's message of 2017-09-20 18:59:50 +:
> > > On 2017-09-20 14:46:32 -0400 (-0400), Tony Breeds wrote:
> > > [...]
> > > > I'd like to find a solution that doesn't need a tox_install.sh
> > > [...]
> > >
> > > This wart came up when discussing Zuul v3 job translations... what
> > > would be ideal is if someone has the bandwidth to work upstream in
> > > tox to add a constraints file option agreeable to its maintainer so
> > > that we don't have to override install-command to add it in all our
> > > configs.
> >
> > One of the reasons we need a script is to modify the constraint list to
> > remove the current library from the list if it is present. I'm not sure
> > that special logic would be something the tox maintainers would want.
>
> My proposal is that if 'constraints' is enabled then it's only used for
> the deps install. Since that should get everything that $project needs
> if it's omitted during that phase it wont hurt us *and* removes the
> needs for that aspect of the tox_install.sh
>
> In my head it looks like:
> ---
> [testenv]
> usedevelop = True
> # constraints =
> {env:UPPER_CONSTRAINTS_FILE:https://tarballs.openstack.org/constraints/master.txt}
> constraints =
> {env:UPPER_CONSTRAINTS_FILE:https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt}
> install_command = pip install {constraints} {opts} {packages}
> deps = -r{toxinidir}/requirements.txt
>-r{toxinidir}/test-requirements.txt
> ---
>
> It wont remove tox_install.sh but it will reduce the number of projects
> that needs it. For example oslso.* and os-* shouldn't need a
> tox_install.sh it we can make the above happen.
>
>
> Yours Tony.
I like the idea. I'm not sure why, if the constraints file is only used
for the dependency installation step, we still need tox_install.sh? If
that's just to avoid updating the URL when we create branches, I can
live with continuing to do that step if we figure out some other way to
minimize the open race window.
Doug
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements][stable] EOL tags and upper-constraints.txt in tox.ini
On Wed, Sep 20, 2017 at 04:51:21PM -0400, Doug Hellmann wrote:
> Excerpts from Jeremy Stanley's message of 2017-09-20 18:59:50 +:
> > On 2017-09-20 14:46:32 -0400 (-0400), Tony Breeds wrote:
> > [...]
> > > I'd like to find a solution that doesn't need a tox_install.sh
> > [...]
> >
> > This wart came up when discussing Zuul v3 job translations... what
> > would be ideal is if someone has the bandwidth to work upstream in
> > tox to add a constraints file option agreeable to its maintainer so
> > that we don't have to override install-command to add it in all our
> > configs.
>
> One of the reasons we need a script is to modify the constraint list to
> remove the current library from the list if it is present. I'm not sure
> that special logic would be something the tox maintainers would want.
My proposal is that if 'constraints' is enabled then it's only used for
the deps install. Since that should get everything that $project needs
if it's omitted during that phase it wont hurt us *and* removes the
needs for that aspect of the tox_install.sh
In my head it looks like:
---
[testenv]
usedevelop = True
# constraints =
{env:UPPER_CONSTRAINTS_FILE:https://tarballs.openstack.org/constraints/master.txt}
constraints =
{env:UPPER_CONSTRAINTS_FILE:https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt}
install_command = pip install {constraints} {opts} {packages}
deps = -r{toxinidir}/requirements.txt
-r{toxinidir}/test-requirements.txt
---
It wont remove tox_install.sh but it will reduce the number of projects
that needs it. For example oslso.* and os-* shouldn't need a
tox_install.sh it we can make the above happen.
Yours Tony.
signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements][stable] EOL tags and upper-constraints.txt in tox.ini
Excerpts from Jeremy Stanley's message of 2017-09-20 18:59:50 +: > On 2017-09-20 14:46:32 -0400 (-0400), Tony Breeds wrote: > [...] > > I'd like to find a solution that doesn't need a tox_install.sh > [...] > > This wart came up when discussing Zuul v3 job translations... what > would be ideal is if someone has the bandwidth to work upstream in > tox to add a constraints file option agreeable to its maintainer so > that we don't have to override install-command to add it in all our > configs. One of the reasons we need a script is to modify the constraint list to remove the current library from the list if it is present. I'm not sure that special logic would be something the tox maintainers would want. Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements][stable] EOL tags and upper-constraints.txt in tox.ini
On 2017-09-20 15:19:47 -0400 (-0400), Tony Breeds wrote: > On Wed, Sep 20, 2017 at 06:59:50PM +, Jeremy Stanley wrote: > > On 2017-09-20 14:46:32 -0400 (-0400), Tony Breeds wrote: > > [...] > > > I'd like to find a solution that doesn't need a tox_install.sh > > [...] > > > > This wart came up when discussing Zuul v3 job translations... what > > would be ideal is if someone has the bandwidth to work upstream in > > tox to add a constraints file option agreeable to its maintainer so > > that we don't have to override install-command to add it in all our > > configs. > > I'm happy to work on it in tox but I don't really have an idea of how to > handle Doug's ideal to avoid having to update the URL for each git > branch. [...] Yes, it would help us get rid of ugly wrapper scripts in many places, but unfortunately doesn't address needing to generate patches to adjust URLs/filenames. -- Jeremy Stanley signature.asc Description: Digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements][stable] EOL tags and upper-constraints.txt in tox.ini
On Wed, Sep 20, 2017 at 06:59:50PM +, Jeremy Stanley wrote: > On 2017-09-20 14:46:32 -0400 (-0400), Tony Breeds wrote: > [...] > > I'd like to find a solution that doesn't need a tox_install.sh > [...] > > This wart came up when discussing Zuul v3 job translations... what > would be ideal is if someone has the bandwidth to work upstream in > tox to add a constraints file option agreeable to its maintainer so > that we don't have to override install-command to add it in all our > configs. I'm happy to work on it in tox but I don't really have an idea of how to handle Doug's ideal to avoid having to update the URL for each git branch. That was my original motivator for doing something 'dynamic'[1] but I can't really think of a nice generic solution. We *could* have tox take a template and expand it from the package data effectively doing my suggestion from [1] without the shell. That feels like over engineering and doesn't accommodate OpenStack Hosted projects very well. [1] http://lists.openstack.org/pipermail/openstack-dev/2017-August/120626.html > -- > Jeremy Stanley > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Yours Tony. signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements][stable] EOL tags and upper-constraints.txt in tox.ini
On 2017-09-20 14:46:32 -0400 (-0400), Tony Breeds wrote: [...] > I'd like to find a solution that doesn't need a tox_install.sh [...] This wart came up when discussing Zuul v3 job translations... what would be ideal is if someone has the bandwidth to work upstream in tox to add a constraints file option agreeable to its maintainer so that we don't have to override install-command to add it in all our configs. -- Jeremy Stanley signature.asc Description: Digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements][stable] EOL tags and upper-constraints.txt in tox.ini
On Wed, Sep 20, 2017 at 02:24:32PM -0400, Doug Hellmann wrote: > That solves the problem of having the constraints file disappear after > the EOL, but it doesn't really solve the problem of having to update the > branches every time we open one. Having tox_install.sh figure out the > URL from the .gitreview file addresses that, but it may be too magic for > some folks. I don't know if we have generally agreed to let anything > other than git-review look at that file. Right, I really like the idea of working this out at runtime but I'd like to find a solution that doesn't need a tox_install.sh or similar Also using .gitreview doesn't help people that are using sdists or (probably) packages. I don't know if we want that to stop us but it's something we need to discuss [tony@thor tmp]$ wget -o /dev/null http://tarballs.openstack.org/nova/nova-master.tar.gz [tony@thor tmp]$ tar tvf nova-master.tar.gz | grep git -rw-rw-r-- jenkins/jenkins 2 2017-09-20 05:30 nova-16.0.0.0rc2.dev393/nova/CA/projects/.gitignore -rw-rw-r-- jenkins/jenkins 2 2017-09-20 05:30 nova-16.0.0.0rc2.dev393/nova/CA/reqs/.gitignore -rw-rw-r-- jenkins/jenkins121 2017-09-20 05:30 nova-16.0.0.0rc2.dev393/nova/CA/.gitignore Yours Tony. signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements][stable] EOL tags and upper-constraints.txt in tox.ini
On 2017-09-20 14:37:27 -0400 (-0400), Doug Hellmann wrote: > Excerpts from Jeremy Stanley's message of 2017-09-20 18:35:23 +: > > On 2017-09-20 14:24:32 -0400 (-0400), Doug Hellmann wrote: > > [...] > > > it doesn't really solve the problem of having to update the > > > branches every time we open one. Having tox_install.sh figure out > > > the URL from the .gitreview file addresses that, but it may be too > > > magic for some folks. I don't know if we have generally agreed to > > > let anything other than git-review look at that file. > > > > Further, git-review has grown additional magic to allow it to figure > > out sane values for defaultbranch when omitted from the .gitreview > > file, so having other tools depend on that implementation detail > > could come back to bite us down the road. > > It sounds like we want git-review to have a mode that prints the > branch to which the change would be proposed, without actually > proposing anything. Our scripts could then use that. "Our scripts" maybe... but calling it from tox.ini to guess branches adds an implicit dependency on git-review for every project doing that, at least for developers who want to be able to make use of the constraints URL fallback. -- Jeremy Stanley signature.asc Description: Digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements][stable] EOL tags and upper-constraints.txt in tox.ini
On Wed, Sep 20, 2017 at 02:09:15PM -0400, Tony Breeds wrote: > On Wed, Sep 20, 2017 at 01:43:51PM -0400, Tony Breeds wrote: > > > The solution I thought we decide on at the PTG is: > > * Add a post job to all branches that publish a constraints/$series.txt > >to $server (I don't mind if it's releases.o.o or tarballs.o.o). > > Actually we might be better to do this daily from the periodic pipeline. > In our CI we always gate with what is in git so that wouldn't be > impacted. The question is do we need external consumers to be "up to > the minute" or is a days lag acceptable? > > I kinda feel like it's okay to be a little laggy. https://review.openstack.org/#/q/topic:tools/publish-constraints Comments welcome. Once those (or similar) land I'll happily update the release tools and optionally $project repos. Yours Tony. signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements][stable] EOL tags and upper-constraints.txt in tox.ini
Excerpts from Jeremy Stanley's message of 2017-09-20 18:35:23 +: > On 2017-09-20 14:24:32 -0400 (-0400), Doug Hellmann wrote: > [...] > > it doesn't really solve the problem of having to update the > > branches every time we open one. Having tox_install.sh figure out > > the URL from the .gitreview file addresses that, but it may be too > > magic for some folks. I don't know if we have generally agreed to > > let anything other than git-review look at that file. > > Further, git-review has grown additional magic to allow it to figure > out sane values for defaultbranch when omitted from the .gitreview > file, so having other tools depend on that implementation detail > could come back to bite us down the road. It sounds like we want git-review to have a mode that prints the branch to which the change would be proposed, without actually proposing anything. Our scripts could then use that. Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements][stable] EOL tags and upper-constraints.txt in tox.ini
On 2017-09-20 14:24:32 -0400 (-0400), Doug Hellmann wrote: [...] > it doesn't really solve the problem of having to update the > branches every time we open one. Having tox_install.sh figure out > the URL from the .gitreview file addresses that, but it may be too > magic for some folks. I don't know if we have generally agreed to > let anything other than git-review look at that file. Further, git-review has grown additional magic to allow it to figure out sane values for defaultbranch when omitted from the .gitreview file, so having other tools depend on that implementation detail could come back to bite us down the road. -- Jeremy Stanley signature.asc Description: Digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements][stable] EOL tags and upper-constraints.txt in tox.ini
On 17-09-20 13:43:51, Tony Breeds wrote: > On Wed, Sep 20, 2017 at 01:08:45PM -0400, Doug Hellmann wrote: > > Excerpts from Jeremy Stanley's message of 2017-09-20 13:36:38 +: > > > On 2017-09-20 08:41:14 -0400 (-0400), Doug Hellmann wrote: > > > [...] > > > > Is there any reason not to use the published files for all regular > > > > builds, instead of having tox.ini point to a git URL? Maybe only for > > > > regular builds off of stable branches? > > > > > > I'm not sure what you mean by "regular builds" but the plan as I > > > > s/regular/non-CI/ > > > > > understood it was to switch from git.o.o URLs to releases.o.o URLs > > > in the tox.ini files in all branches of projects already consuming > > > constraints files that way. > > > > > > As early as now (if we already had the publication job in place) we > > > could update them all in master branches to retrieve something like > > > https://releases.openstack.org/constraints/queens-upper-constraints.txt > > > and then once stable/queens is branched in all repos (including the > > > requirements repo), switch the job to begin publishing to a URL for > > > rocky and push tox.ini updates to master branches switching the URL > > > to that as early in the cycle as possible. Alternatively, we could > > > publish master and queens copies (identical initially) and expect > > > the master branch tox.ini files to refer to master but then switch > > > them to queens on the stable/queens branch during RC. It just comes > > > down to which the stable/requirements/release teams think makes the > > > most sense from a procedural perspective. > > > > We should think through which of those approaches is going to result in > > the least amount of synchronization. We do have a window in which to > > make changes in a given branch for consuming projects, but making the > > relevant changes in the requirements repository seems like it could be > > error prone, especially because we try to branch it after the other > > repositories. > > The solution I thought we decide on at the PTG is: > * Add a post job to all branches that publish a constraints/$series.txt >to $server (I don't mind if it's releases.o.o or tarballs.o.o). > * On the master branch we publish both master.txt and $series.txt >(currently queens.txt) when we fork rocky from master we update the >publish job to publish master and rocky.txt.As Doug points out >there is a timing race here but it;s much smaller than the one we >have now. > * We update the projects to use the static (non-git) URL for the >constraints files. > * We update the release tools to generate the new style URL. > > Optionally, and this requires discussion, we use a custom tox_install.sh > to extract the branch from .gitreview and generate the URL which would > remove the need for the last step. There are clearly pros and cons to > that so I'm not advocating for it now. > > Those constraints files would never be removed but they'd stop getting > updated when we EOL the requirements branch. > > How does that sound? > > Yours Tony. That's correct, I haven't had time this week to create the jobs quite yet, but hope to do so either this weekend or next week. It's #3 in our list of items to generate tasks/bugs from. https://etherpad.openstack.org/p/queens-PTG-requirements -- Matthew Thode (prometheanfire) signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements][stable] EOL tags and upper-constraints.txt in tox.ini
Excerpts from Tony Breeds's message of 2017-09-20 13:43:51 -0400: > On Wed, Sep 20, 2017 at 01:08:45PM -0400, Doug Hellmann wrote: > > Excerpts from Jeremy Stanley's message of 2017-09-20 13:36:38 +: > > > On 2017-09-20 08:41:14 -0400 (-0400), Doug Hellmann wrote: > > > [...] > > > > Is there any reason not to use the published files for all regular > > > > builds, instead of having tox.ini point to a git URL? Maybe only for > > > > regular builds off of stable branches? > > > > > > I'm not sure what you mean by "regular builds" but the plan as I > > > > s/regular/non-CI/ > > > > > understood it was to switch from git.o.o URLs to releases.o.o URLs > > > in the tox.ini files in all branches of projects already consuming > > > constraints files that way. > > > > > > As early as now (if we already had the publication job in place) we > > > could update them all in master branches to retrieve something like > > > https://releases.openstack.org/constraints/queens-upper-constraints.txt > > > and then once stable/queens is branched in all repos (including the > > > requirements repo), switch the job to begin publishing to a URL for > > > rocky and push tox.ini updates to master branches switching the URL > > > to that as early in the cycle as possible. Alternatively, we could > > > publish master and queens copies (identical initially) and expect > > > the master branch tox.ini files to refer to master but then switch > > > them to queens on the stable/queens branch during RC. It just comes > > > down to which the stable/requirements/release teams think makes the > > > most sense from a procedural perspective. > > > > We should think through which of those approaches is going to result in > > the least amount of synchronization. We do have a window in which to > > make changes in a given branch for consuming projects, but making the > > relevant changes in the requirements repository seems like it could be > > error prone, especially because we try to branch it after the other > > repositories. > > The solution I thought we decide on at the PTG is: > * Add a post job to all branches that publish a constraints/$series.txt >to $server (I don't mind if it's releases.o.o or tarballs.o.o). > * On the master branch we publish both master.txt and $series.txt >(currently queens.txt) when we fork rocky from master we update the >publish job to publish master and rocky.txt.As Doug points out >there is a timing race here but it;s much smaller than the one we >have now. Yes, I think that would work. It adds another manual step to the release process (to update the job) but that can technically be done as soon as we know the next release name because we can publish from master to as many different future names as we want. We're starting to have a fair amount of automation that relies on knowing the names of the release series and their statuses. I wonder if we can produce a central library of some sort to give us that information in a consumable format so we only have to update it in one place. > * We update the projects to use the static (non-git) URL for the >constraints files. > * We update the release tools to generate the new style URL. > > Optionally, and this requires discussion, we use a custom tox_install.sh > to extract the branch from .gitreview and generate the URL which would > remove the need for the last step. There are clearly pros and cons to > that so I'm not advocating for it now. > > Those constraints files would never be removed but they'd stop getting > updated when we EOL the requirements branch. > > How does that sound? That solves the problem of having the constraints file disappear after the EOL, but it doesn't really solve the problem of having to update the branches every time we open one. Having tox_install.sh figure out the URL from the .gitreview file addresses that, but it may be too magic for some folks. I don't know if we have generally agreed to let anything other than git-review look at that file. Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements][stable] EOL tags and upper-constraints.txt in tox.ini
On Wed, Sep 20, 2017 at 01:43:51PM -0400, Tony Breeds wrote: > The solution I thought we decide on at the PTG is: > * Add a post job to all branches that publish a constraints/$series.txt >to $server (I don't mind if it's releases.o.o or tarballs.o.o). Actually we might be better to do this daily from the periodic pipeline. In our CI we always gate with what is in git so that wouldn't be impacted. The question is do we need external consumers to be "up to the minute" or is a days lag acceptable? I kinda feel like it's okay to be a little laggy. Yours Tony. signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements][stable] EOL tags and upper-constraints.txt in tox.ini
On Wed, Sep 20, 2017 at 01:08:45PM -0400, Doug Hellmann wrote: > Excerpts from Jeremy Stanley's message of 2017-09-20 13:36:38 +: > > On 2017-09-20 08:41:14 -0400 (-0400), Doug Hellmann wrote: > > [...] > > > Is there any reason not to use the published files for all regular > > > builds, instead of having tox.ini point to a git URL? Maybe only for > > > regular builds off of stable branches? > > > > I'm not sure what you mean by "regular builds" but the plan as I > > s/regular/non-CI/ > > > understood it was to switch from git.o.o URLs to releases.o.o URLs > > in the tox.ini files in all branches of projects already consuming > > constraints files that way. > > > > As early as now (if we already had the publication job in place) we > > could update them all in master branches to retrieve something like > > https://releases.openstack.org/constraints/queens-upper-constraints.txt > > and then once stable/queens is branched in all repos (including the > > requirements repo), switch the job to begin publishing to a URL for > > rocky and push tox.ini updates to master branches switching the URL > > to that as early in the cycle as possible. Alternatively, we could > > publish master and queens copies (identical initially) and expect > > the master branch tox.ini files to refer to master but then switch > > them to queens on the stable/queens branch during RC. It just comes > > down to which the stable/requirements/release teams think makes the > > most sense from a procedural perspective. > > We should think through which of those approaches is going to result in > the least amount of synchronization. We do have a window in which to > make changes in a given branch for consuming projects, but making the > relevant changes in the requirements repository seems like it could be > error prone, especially because we try to branch it after the other > repositories. The solution I thought we decide on at the PTG is: * Add a post job to all branches that publish a constraints/$series.txt to $server (I don't mind if it's releases.o.o or tarballs.o.o). * On the master branch we publish both master.txt and $series.txt (currently queens.txt) when we fork rocky from master we update the publish job to publish master and rocky.txt.As Doug points out there is a timing race here but it;s much smaller than the one we have now. * We update the projects to use the static (non-git) URL for the constraints files. * We update the release tools to generate the new style URL. Optionally, and this requires discussion, we use a custom tox_install.sh to extract the branch from .gitreview and generate the URL which would remove the need for the last step. There are clearly pros and cons to that so I'm not advocating for it now. Those constraints files would never be removed but they'd stop getting updated when we EOL the requirements branch. How does that sound? Yours Tony. signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements][stable] EOL tags and upper-constraints.txt in tox.ini
Excerpts from Jeremy Stanley's message of 2017-09-20 13:36:38 +: > On 2017-09-20 08:41:14 -0400 (-0400), Doug Hellmann wrote: > [...] > > Is there any reason not to use the published files for all regular > > builds, instead of having tox.ini point to a git URL? Maybe only for > > regular builds off of stable branches? > > I'm not sure what you mean by "regular builds" but the plan as I s/regular/non-CI/ > understood it was to switch from git.o.o URLs to releases.o.o URLs > in the tox.ini files in all branches of projects already consuming > constraints files that way. > > As early as now (if we already had the publication job in place) we > could update them all in master branches to retrieve something like > https://releases.openstack.org/constraints/queens-upper-constraints.txt > and then once stable/queens is branched in all repos (including the > requirements repo), switch the job to begin publishing to a URL for > rocky and push tox.ini updates to master branches switching the URL > to that as early in the cycle as possible. Alternatively, we could > publish master and queens copies (identical initially) and expect > the master branch tox.ini files to refer to master but then switch > them to queens on the stable/queens branch during RC. It just comes > down to which the stable/requirements/release teams think makes the > most sense from a procedural perspective. We should think through which of those approaches is going to result in the least amount of synchronization. We do have a window in which to make changes in a given branch for consuming projects, but making the relevant changes in the requirements repository seems like it could be error prone, especially because we try to branch it after the other repositories. In any case, I like whichever approach can be made to work and will leave bikeshedding on URLs paths to the reviews that implement the one we pick. If it would be useful, I would be happy to help advise anyone who wants to modify the release automation scripts to handle this new case. Doug > Remember also that the timing on this doesn't require extreme > precision and there are no chicken-and-egg/catch-22 problems > associated with updating because the URLs in question are merely a > fallback method for when the constraints file is not already > provided locally. In the CI system, we directly provide constraints > files so that we can respect depends-on to requirements repo changes > and the like, so in practice this fallback is primarily for the > convenience of developers running tox locally. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements][stable] EOL tags and upper-constraints.txt in tox.ini
On 2017-09-20 08:41:14 -0400 (-0400), Doug Hellmann wrote: [...] > Is there any reason not to use the published files for all regular > builds, instead of having tox.ini point to a git URL? Maybe only for > regular builds off of stable branches? I'm not sure what you mean by "regular builds" but the plan as I understood it was to switch from git.o.o URLs to releases.o.o URLs in the tox.ini files in all branches of projects already consuming constraints files that way. As early as now (if we already had the publication job in place) we could update them all in master branches to retrieve something like https://releases.openstack.org/constraints/queens-upper-constraints.txt and then once stable/queens is branched in all repos (including the requirements repo), switch the job to begin publishing to a URL for rocky and push tox.ini updates to master branches switching the URL to that as early in the cycle as possible. Alternatively, we could publish master and queens copies (identical initially) and expect the master branch tox.ini files to refer to master but then switch them to queens on the stable/queens branch during RC. It just comes down to which the stable/requirements/release teams think makes the most sense from a procedural perspective. Remember also that the timing on this doesn't require extreme precision and there are no chicken-and-egg/catch-22 problems associated with updating because the URLs in question are merely a fallback method for when the constraints file is not already provided locally. In the CI system, we directly provide constraints files so that we can respect depends-on to requirements repo changes and the like, so in practice this fallback is primarily for the convenience of developers running tox locally. -- Jeremy Stanley signature.asc Description: Digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements][stable] EOL tags and upper-constraints.txt in tox.ini
Excerpts from Jeremy Stanley's message of 2017-09-20 12:30:59 +: > On 2017-09-20 07:11:51 -0500 (-0500), Matthew Thode wrote: > > On 17-09-20 13:35:44, Michal Pryc wrote: > > > EOL releases for the nova component (checked neutron as well, > > > possibly many other components) have wrong pointers to the > > > upper-constraints.txt files as they are referencing > > > stable/branch rather then branch-eol > [...] > > I think this is an error in our process that should be fixed > > (newton is going EOL soon). > [...] > > As you may recall, at the PTG we also discussed a solution to this > problem (under the auspices of solving the reverse scenario during > the RC period for upcoming releases): specifically, publishing > branch series constraints files to the releases site. EOL'd projects > can refer to those at a static URL indefinitely rather than being at > the mercy of branch/tag changes in the Git repository. Is there any reason not to use the published files for all regular builds, instead of having tox.ini point to a git URL? Maybe only for regular builds off of stable branches? Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements][stable] EOL tags and upper-constraints.txt in tox.ini
On 2017-09-20 07:11:51 -0500 (-0500), Matthew Thode wrote: > On 17-09-20 13:35:44, Michal Pryc wrote: > > EOL releases for the nova component (checked neutron as well, > > possibly many other components) have wrong pointers to the > > upper-constraints.txt files as they are referencing > > stable/branch rather then branch-eol [...] > I think this is an error in our process that should be fixed > (newton is going EOL soon). [...] As you may recall, at the PTG we also discussed a solution to this problem (under the auspices of solving the reverse scenario during the RC period for upcoming releases): specifically, publishing branch series constraints files to the releases site. EOL'd projects can refer to those at a static URL indefinitely rather than being at the mercy of branch/tag changes in the Git repository. -- Jeremy Stanley signature.asc Description: Digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements][stable] EOL tags and upper-constraints.txt in tox.ini
On 17-09-20 13:35:44, Michal Pryc wrote: > Hi, > > EOL releases for the nova component (checked neutron as well, possibly many > other components) have wrong pointers to the upper-constraints.txt files as > they are referencing stable/branch rather then branch-eol, see example: > > https://github.com/openstack/nova/blob/liberty-eol/tox.ini#L12 > > Line: > https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt?h=stable/liberty > > Should be: > https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt?h=liberty-eol > > EOL means there should be no more changes to the release, but in this case > process of EOL'ing made regression as now it's impossible to run tests > against those tags (this applies to liberty/mitaka and in the future newer > releases). > > Should this be fixed somehow or EOL isn't really meant to be touched and > fixed and those will be left broken ? > > -- > best > Michal Pryc > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev I think this is an error in our process that should be fixed (newton is going EOL soon). While it'd be nice to fix I don't think it's worth the effort as an EOL'd and now unbranched project. We could create a temporary branch from the commit and retag, but as a rule we should not use the same tag twice (removing the pointer that existed and reusing it to point to a diferent commit). -- Matthew Thode (prometheanfire) signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
