Re: [openstack-dev] [packaging] liberty doesn't have caps on deps

[Matt Riedemann]

On 10/16/2015 7:16 AM, Sean Dague wrote:

On 10/16/2015 04:23 AM, Thierry Carrez wrote:

Robert Collins wrote:

[...]
BUT: we haven't (ever!) tested that the lowest versions we specify
work. When folk know they are adding a hard dependency on a version we
do raise the lower versions, but thats adhoc and best effort today.
I'd like to see a lower-constraints.txt reflecting the oldest version
that works across all of OpenStack (as a good boundary case to test) -
but we need to fix pip first to teach it to choose lower versions over
higher versions (https://github.com/pypa/pip/issues/3188 - I thought
I'd filed it previously but couldn't find it...)

More generally, we don't [yet] have the testing setup to test multiple
versions on an ongoing basis, so we can't actually make any statement
other than 'upper-constraints.txt is known to work'. Note: before
constraints we couldn't even make *that* statement. The statement we
could make then was 'if you look up the change in gerrit and from that
the CI dvsm test run which got through the gate, then you can
figureout *a* version of the dependencies that worked.


And that is the critical bit. The system we had in kilo and before may
appear to be more practical to interpret downstream, but the assertions
it was making were mostly untested. So the capping was a convenient
illusion: things beyond the cap may be working, and things below the cap
could actually be broken. At least the upper-constraints expresses
clearly the combination that works and was tested. Combined with the
uncapped requirements (which express what *should* be working, to the
best of our knowledge), they make a more accurate, albeit admittedly
more complex, set of information for downstream packagers.


And equally important is that pip only really reacts well to version
capping / pinning if you do it all at once across all your things.

When we had a cap, and raised it, we had to:

A) raise it in low level oslo libs (like oslo.config)
B) release those libraries with new caps
C) raise the cap in all the things that used that library
D) release those libraries with new caps
E) ... repeat
...
M) raise the cap in all top level openstack server projects

So a dozen library releases could easily be triggered by fixing one cap
or pin in one low level library, that were no functional changes, they
were just requirements changes. The only reason M could use the old
version of A is because pip wouldn't let you install the 2 together. Not
for any functional reasons.

-Sean



This thread is now directly related to the upgrade impact being 
discussed in liberty here:


https://review.openstack.org/#/c/255245/

--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [packaging] liberty doesn't have caps on deps

[Thierry Carrez]

Robert Collins wrote:
> [...]
> BUT: we haven't (ever!) tested that the lowest versions we specify
> work. When folk know they are adding a hard dependency on a version we
> do raise the lower versions, but thats adhoc and best effort today.
> I'd like to see a lower-constraints.txt reflecting the oldest version
> that works across all of OpenStack (as a good boundary case to test) -
> but we need to fix pip first to teach it to choose lower versions over
> higher versions (https://github.com/pypa/pip/issues/3188 - I thought
> I'd filed it previously but couldn't find it...)
> 
> More generally, we don't [yet] have the testing setup to test multiple
> versions on an ongoing basis, so we can't actually make any statement
> other than 'upper-constraints.txt is known to work'. Note: before
> constraints we couldn't even make *that* statement. The statement we
> could make then was 'if you look up the change in gerrit and from that
> the CI dvsm test run which got through the gate, then you can
> figureout *a* version of the dependencies that worked.

And that is the critical bit. The system we had in kilo and before may
appear to be more practical to interpret downstream, but the assertions
it was making were mostly untested. So the capping was a convenient
illusion: things beyond the cap may be working, and things below the cap
could actually be broken. At least the upper-constraints expresses
clearly the combination that works and was tested. Combined with the
uncapped requirements (which express what *should* be working, to the
best of our knowledge), they make a more accurate, albeit admittedly
more complex, set of information for downstream packagers.

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [packaging] liberty doesn't have caps on deps

[Doug Hellmann]

Excerpts from Robert Collins's message of 2015-10-16 09:29:21 +1300:
> On 16 October 2015 at 08:57, Doug Hellmann  wrote:
> > Excerpts from Matthew Thode's message of 2015-10-15 14:35:08 -0500:
> >> On 10/15/2015 02:17 PM, Matt Riedemann wrote:
> >> >
> 
> >
> > We're going to try to cover it in
> > http://mitakadesignsummit.sched.org/event/27c17a3c35d72997b372ddf4759fe1be#.ViAE67SO820
> > but we have a lot of other things to fit into that session so I want to
> > put this one last so we don't derail the other discussions.
> 
> I thought it was
> http://mitakadesignsummit.sched.org/event/0a2307779b4ab81892ba24de379e9dcc#.ViAMfd94u00
> that it was slated for, where its squarely on topic :)
> 
> -Rob
> 

It may fit there, too, but I don't know if Sean expects the existing
outline to fill the time slot.

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [packaging] liberty doesn't have caps on deps

[Sean Dague]

On 10/16/2015 04:23 AM, Thierry Carrez wrote:
> Robert Collins wrote:
>> [...]
>> BUT: we haven't (ever!) tested that the lowest versions we specify
>> work. When folk know they are adding a hard dependency on a version we
>> do raise the lower versions, but thats adhoc and best effort today.
>> I'd like to see a lower-constraints.txt reflecting the oldest version
>> that works across all of OpenStack (as a good boundary case to test) -
>> but we need to fix pip first to teach it to choose lower versions over
>> higher versions (https://github.com/pypa/pip/issues/3188 - I thought
>> I'd filed it previously but couldn't find it...)
>>
>> More generally, we don't [yet] have the testing setup to test multiple
>> versions on an ongoing basis, so we can't actually make any statement
>> other than 'upper-constraints.txt is known to work'. Note: before
>> constraints we couldn't even make *that* statement. The statement we
>> could make then was 'if you look up the change in gerrit and from that
>> the CI dvsm test run which got through the gate, then you can
>> figureout *a* version of the dependencies that worked.
> 
> And that is the critical bit. The system we had in kilo and before may
> appear to be more practical to interpret downstream, but the assertions
> it was making were mostly untested. So the capping was a convenient
> illusion: things beyond the cap may be working, and things below the cap
> could actually be broken. At least the upper-constraints expresses
> clearly the combination that works and was tested. Combined with the
> uncapped requirements (which express what *should* be working, to the
> best of our knowledge), they make a more accurate, albeit admittedly
> more complex, set of information for downstream packagers.

And equally important is that pip only really reacts well to version
capping / pinning if you do it all at once across all your things.

When we had a cap, and raised it, we had to:

A) raise it in low level oslo libs (like oslo.config)
B) release those libraries with new caps
C) raise the cap in all the things that used that library
D) release those libraries with new caps
E) ... repeat
...
M) raise the cap in all top level openstack server projects

So a dozen library releases could easily be triggered by fixing one cap
or pin in one low level library, that were no functional changes, they
were just requirements changes. The only reason M could use the old
version of A is because pip wouldn't let you install the 2 together. Not
for any functional reasons.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [packaging] liberty doesn't have caps on deps

[Doug Hellmann]

Excerpts from Matthew Thode's message of 2015-10-15 17:40:27 -0500:
> On 10/15/2015 05:29 PM, Robert Collins wrote:
> > On 16 October 2015 at 11:21, Matthew Thode  
> > wrote:
> >> On 10/15/2015 05:12 PM, Robert Collins wrote:
> >>> On 16 October 2015 at 08:10, Matthew Thode  
> >>> wrote:
>  On 10/15/2015 02:04 PM, Robert Collins wrote:
> >>> ...
> >> Where are my caps?
> >
> > The known good versions of dependencies for liberty are
> > http://git.openstack.org/cgit/openstack/requirements/tree/upper-constraints.txt?h=stable/liberty
> >
>  That's good, does that represent an upper constraint for the lower
>  constraint imposed by by the package?  Why is this kept separate?
>  Keeping it separate means that it's not trivial to merge them with
>  what's in each package's requirements.
> >>>
> >>> It represents *one* known good combination of packages. We know that
> >>> that combination passed CI, and we then do all our tests with it. To
> >>> change it, we run it past CI and only move onto using the new set when
> >>> its passed CI.
> >>>
> >>> If we merged it to each packages requirements, we'd reintroduce the
> >>> deadlock that caused so much grief only 6 months back - I don't see
> >>> any reason, desire, or tolerance for doing that upstream.
> >>>
> >>> Its kept separate because it requires 2N commits to shift known-good
> >>> caps around for N repos using per-repo rules.
> >>>
> >>> With hundreds of repositories, it takes us hundreds of commits in two
> >>> batches - and a round trip time of 2 hours per batch (check + gate) to
> >>> shift *a single* requirement. With hundreds of dependencies, thats an
> >>> intolerable amount of overhead.
> >>>
> >>
> >> ya, that is annoying, unfortunately packages don't need to know *ONE*
> >> good combination, they need to know them all, or at least the cap.  What
> >> you are basically doing is shifting all of this extra work onto the
> >> packagers, in fact I wouldn't be surprised if they needed to start
> >> vendoring all of openstack in a virtualenv instead of doing actual 
> >> packages.
> > 
> > Here's an example of the havoc caps cause:
> >  https://review.openstack.org/#/c/234862/
> > 
> > I don't understand the statement about shifting work. Previously the
> > situation was that you had to guess whether a given release of a
> > dependency (both internal and external) worked with e.g. liberty.
> > 
> > Now there is an explicit list of what works.
> > 
> > Isn't that *better* ?
> > 
> Yes, it's good to know what works, does that show multiple versions of
> the same package when multiple are known to work?  If so I can build a
> range from that.  It's not better as it is because I still don't know
> where liberty ends and mitaka begins.  Is there any place I can find that?

In terms of deliverables we're calling part of liberty vs. mitaka,
you can find them in the openstack/releases repository in the YAML
files in the deliverables directory. The results are published to
http://docs.openstack.org/releases/ for human consumption.

Doug

> 
> >> My question remains though, if someone pip installs nova liberty, will
> >> it pull in deps from mitaka?  As it is now, without caps I cannot
> >> reliably package openstack, do you have a solution? I should probably
> >> start removing the liberty packages I did package since upstream seems
> >> so hostile...
> > 
> > If you don't use the constraints file (which is pip consumable and
> > published on the web so it can be used with pip install trivially)
> > then yes, it will install the latest versions of all packages which
> > are presumed to be doing backwards compatible changes. Things that we
> > *know* are going to be broken still get caps - and we accept the cost
> > of the 2-step dance needed to raise them.
> > 
> 
> What about a daily or weekly check without the constraints file so you
> know when something breaks?  This would allow you to at least know when
> you need to place caps and I could consume that.  I'm fine with leaving
> caps off.  If I can consume mitaka deps in liberty then that's great :D
> 
> > The big question here is, I think, 'should we assume OpenStack
> > originated dependencies are better or worse than third party
> > dependencies?' Third party dependencies we presume are backwards
> > compatible (and will thus only break by mistake, which constraints
> > guards against) unless we have reason not to - and we have open ended
> > deps on them. This is where the spec about clients libraries is aimed.
> > 
> > It makes me very sad to know that you consider what we've done as
> > hostile. We did it for a bunch of good reasons:
> > 
> >  - safer release process
> >  - smoother updates for [apt, rpm at least] redistributors
> >  - faster turnaround of dependency usage in CI
> >  - step on the path to testing lower bounds
> >  - step on the path to alignment with upstream packaging tooling
> > 
> > -Rob
> 

Re: [openstack-dev] [packaging] liberty doesn't have caps on deps

[Matthew Thode]

On 10/15/2015 03:29 PM, Robert Collins wrote:
> On 16 October 2015 at 08:57, Doug Hellmann  wrote:
>> Excerpts from Matthew Thode's message of 2015-10-15 14:35:08 -0500:
>>> On 10/15/2015 02:17 PM, Matt Riedemann wrote:

> 
>>
>> We're going to try to cover it in
>> http://mitakadesignsummit.sched.org/event/27c17a3c35d72997b372ddf4759fe1be#.ViAE67SO820
>> but we have a lot of other things to fit into that session so I want to
>> put this one last so we don't derail the other discussions.
> 
> I thought it was
> http://mitakadesignsummit.sched.org/event/0a2307779b4ab81892ba24de379e9dcc#.ViAMfd94u00
> that it was slated for, where its squarely on topic :)
> 
> -Rob
> 
I'd be nice if we can get this sorted, personally I can't make it to
Friday afternoon but I would like to go to wherever this happens.

-- 
-- Matthew Thode (prometheanfire)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [packaging] liberty doesn't have caps on deps

[Robert Collins]

...
>> Now there is an explicit list of what works.
>>
>> Isn't that *better* ?
>>
> Yes, it's good to know what works, does that show multiple versions of
> the same package when multiple are known to work?

It shows the highest version we know to work. So you could in
principle take any version less than that that is permitted by the
package-local requirements. E.g. consider oslo.messaging and Nova.

Nova's dep on oslo.messaging is:
oslo.messaging>=1.8.0 # Apache-2.0

upper-constraints is:
oslo.messaging===2.5.0

So any version >= 1.8.0 and <= 2.5.0

We haven't successfully tested 2.6.0 (or it would be in
upper-constraints), and we didn't have any failures with versions in
between (or they would be listed in Nova's requirements as a != line.

BUT: we haven't (ever!) tested that the lowest versions we specify
work. When folk know they are adding a hard dependency on a version we
do raise the lower versions, but thats adhoc and best effort today.
I'd like to see a lower-constraints.txt reflecting the oldest version
that works across all of OpenStack (as a good boundary case to test) -
but we need to fix pip first to teach it to choose lower versions over
higher versions (https://github.com/pypa/pip/issues/3188 - I thought
I'd filed it previously but couldn't find it...)

More generally, we don't [yet] have the testing setup to test multiple
versions on an ongoing basis, so we can't actually make any statement
other than 'upper-constraints.txt is known to work'. Note: before
constraints we couldn't even make *that* statement. The statement we
could make then was 'if you look up the change in gerrit and from that
the CI dvsm test run which got through the gate, then you can
figureout *a* version of the dependencies that worked.

>  If so I can build a
> range from that.  It's not better as it is because I still don't know
> where liberty ends and mitaka begins.  Is there any place I can find that?

We don't always release every library in every release. See for
instance this warning from Juno's global-requirements.txt:

# NOTE(tonyb): the oslo.utils acceptable versions overlap in juno and kilo
# please ensure that any new releases are handled carefully
oslo.utils>=1.4.0,<1.5.0 # Apache-2.0

So - there may not in fact *be* a line where liberty ends and mitaka begins.

And if we can sort out the backwards compat stuff, we should be able
to actually get rid of that distinction, and instead say the much
simpler thing 'we're compatible with everything released while this
branch is supported'.

But in the interim, we're using semver everywhere, so at least for the
internal dependencies, you could just set a cap on the next major
version within Gentoo. It may be too restrictive, but it should work
(and if it doesn't then we've failed at semver and we need to figure
out how not to do that).

> What about a daily or weekly check without the constraints file so you
> know when something breaks?

We generate a new list daily with the latest releases on PyPI:
http://logs.openstack.org/periodic/propose-requirements-constraints-master/5b4e488/console.html
is an example log from the generation of such an update. That gets
proposed to the requirements project and can be reviewed here -
https://review.openstack.org/#/q/status:open+project:openstack/requirements+topic:openstack/requirements/constraints,n,z

So we see things that break, and then we can propose != exclusions for
them (or if its better, fixes to our projects - depends on the nature
of the interaction and so on).

>  This would allow you to at least know when
> you need to place caps and I could consume that.

Ish - it tells us where things break, and from that we can decide on
exclusion (single bad release) or cap (deliberate API break they won't
roll back). Caps are massively more disruptive, so we try to avoid
them.

> I'm fine with leaving
> caps off.  If I can consume mitaka deps in liberty then that's great :D
...
> It's a step backwards from my perspective, before I had a clear
> delineation where support of something stops.  Now I don't know when it
> stops and any given update could break the system.  I'm not sure how it
> could be smoother for other systems.

The problem is, you didn't actually have such a line (as should be
evident by the fact we've broken juno again and again and again since
th release).

> So, does this mean that I can just leave the packages uncapped and know
> that they will work?  Are there tests being run for this scenario?

No, though I *want* us to get to that place, and thats the debate
we're having about backwards compat in clients and libraries.

{Note that we've never systematically capped third party libraries and
so they are always a potential surprise too - but its again the
balance between presume-bad and presume-good behaviour from the
upstreams}.

-Rob

-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

__

Re: [openstack-dev] [packaging] liberty doesn't have caps on deps

[Robert Collins]

On 16 October 2015 at 12:09, Matthew Thode  wrote:
> On 10/15/2015 06:00 PM, Robert Collins wrote:

> Thanks for the replies, I think I have a way forward (using upper-reqs
> as a cap).  I might need to make a tool that munges the (test-)reqs.txt
> to generate one as an easier reference.  having to open up two files to
> reference is kinda annoying.

The parser in openstack_requirements for requirements files may be of
use to you in doing that. We use it to e.g. validate that patches to
requirements are compatible with the current upper-constraints.

-Rob

-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [packaging] liberty doesn't have caps on deps

[Doug Hellmann]

Excerpts from Matthew Thode's message of 2015-10-15 14:35:08 -0500:
> On 10/15/2015 02:17 PM, Matt Riedemann wrote:
> > 
> > 
> > On 10/15/2015 2:10 PM, Matthew Thode wrote:
> >> On 10/15/2015 02:04 PM, Robert Collins wrote:
> >>> On 16 October 2015 at 08:01, Matthew Thode
> >>>  wrote:
>  So, this is my perspective in packing liberty for Gentoo.
> 
>  We can have multiple versions of a package available to install,
>  because
>  of this we generally directly translate the valid dependency versions
>  from requirements.
> >>>
> >>> Cool.
> >>>
>  this
>   oslo.concurrency>=2.3.0 # Apache-2.0
>  becomes this
>   >=dev-python/oslo-concurrency-2.3.0[${PYTHON_USEDEP}]
> 
>  Now what happens when I package something from mitaka (2.7.0 in this
>  case would be mitaka).  It's undefined behaviour as far as I know.
> 
>  Basically, I can see no reason why the policy of caps changed from kilo
>  to liberty, it was actually nice to package for liberty, I can see this
>  going very bad very quick.
> >>>
> >>> They changed because it was causing huge trauma and multiple day long
> >>> gate wedges around release times. Covered in detail here -
> >>> http://specs.openstack.org/openstack/openstack-specs/specs/requirements-management.html
> >>>
> >>>
>  Where are my caps?
> >>>
> >>> The known good versions of dependencies for liberty are
> >>> http://git.openstack.org/cgit/openstack/requirements/tree/upper-constraints.txt?h=stable/liberty
> >>>
> >>>
> >> That's good, does that represent an upper constraint for the lower
> >> constraint imposed by by the package?  Why is this kept separate?
> >> Keeping it separate means that it's not trivial to merge them with
> >> what's in each package's requirements.
> > 
> > I'm not sure I understand the first question but I believe that u-c is
> > automatically adjusted and if there is a conflict with the minimum
> > version required of a dependency then the cap is adjusted (or vice-versa).
> > 
> > It's separate, at least in part, because the changes to
> > global-requirements are synced to all projects in the projects.txt file
> > in the requirements repo, which causes a bunch of churn to get those
> > changes approved in the registered projects and then released
> > appropriately.
> > 
> > The global sync to the ecosystem isn't fun, I'll agree, but I do think
> > that thinks have been better since Juno/Kilo because (1) we don't allow
> > <= caps in g-r anymore (we were not allowing patch version updates which
> > wedged us several times) and (2) we're better about releasing things
> > with minor version updates per semver - whereas in the past the cats
> > were releasing on their own volition and picking the version they
> > thought was best, which creeped into having multiple versions that could
> > be acceptable across branches, and we'd have wedges those ways too. I
> > think a lot of that has been fixed by the openstack/releases repo so
> > that the cats now have to line up for the release of their library with
> > a centralized team.
> > 
> 
> I'd agree with this, I still don't know what to cap things to.  I need
> to figure out what the caps should be...  It could be hard to sync
> across projects but like you say, most of that's gotten better since then.
> 
> >>
> >>> You should be able to trivially pull those versions out and into your
> >>> liberty set of packages.
> >>>
> >>> Theres another iteration on this in discussion now, which has to do
> >>> with backwards compat *and testing of cap changes*, we'll be in the
> >>> backwards compat fishbowl session in Tokyo if you're interested.
> >>>
> >>> -Rob
> >>>
> >>
> >> I'll be at the fishbowl :D
> >>
> > 
> 
> Anyone have a sched link to that fishbowl, I can't find it :(
> 

We're going to try to cover it in
http://mitakadesignsummit.sched.org/event/27c17a3c35d72997b372ddf4759fe1be#.ViAE67SO820
but we have a lot of other things to fit into that session so I want to
put this one last so we don't derail the other discussions.

Doug


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [packaging] liberty doesn't have caps on deps

[Doug Hellmann]

Excerpts from Doug Hellmann's message of 2015-10-15 15:57:20 -0400:
> Excerpts from Matthew Thode's message of 2015-10-15 14:35:08 -0500:
> > On 10/15/2015 02:17 PM, Matt Riedemann wrote:
> > > 
> > > 
> > > On 10/15/2015 2:10 PM, Matthew Thode wrote:
> > >> On 10/15/2015 02:04 PM, Robert Collins wrote:
> > >>> On 16 October 2015 at 08:01, Matthew Thode
> > >>>  wrote:
> >  So, this is my perspective in packing liberty for Gentoo.
> > 
> >  We can have multiple versions of a package available to install,
> >  because
> >  of this we generally directly translate the valid dependency versions
> >  from requirements.
> > >>>
> > >>> Cool.
> > >>>
> >  this
> >   oslo.concurrency>=2.3.0 # Apache-2.0
> >  becomes this
> >   >=dev-python/oslo-concurrency-2.3.0[${PYTHON_USEDEP}]
> > 
> >  Now what happens when I package something from mitaka (2.7.0 in this
> >  case would be mitaka).  It's undefined behaviour as far as I know.
> > 
> >  Basically, I can see no reason why the policy of caps changed from kilo
> >  to liberty, it was actually nice to package for liberty, I can see this
> >  going very bad very quick.
> > >>>
> > >>> They changed because it was causing huge trauma and multiple day long
> > >>> gate wedges around release times. Covered in detail here -
> > >>> http://specs.openstack.org/openstack/openstack-specs/specs/requirements-management.html
> > >>>
> > >>>
> >  Where are my caps?
> > >>>
> > >>> The known good versions of dependencies for liberty are
> > >>> http://git.openstack.org/cgit/openstack/requirements/tree/upper-constraints.txt?h=stable/liberty
> > >>>
> > >>>
> > >> That's good, does that represent an upper constraint for the lower
> > >> constraint imposed by by the package?  Why is this kept separate?
> > >> Keeping it separate means that it's not trivial to merge them with
> > >> what's in each package's requirements.
> > > 
> > > I'm not sure I understand the first question but I believe that u-c is
> > > automatically adjusted and if there is a conflict with the minimum
> > > version required of a dependency then the cap is adjusted (or vice-versa).
> > > 
> > > It's separate, at least in part, because the changes to
> > > global-requirements are synced to all projects in the projects.txt file
> > > in the requirements repo, which causes a bunch of churn to get those
> > > changes approved in the registered projects and then released
> > > appropriately.
> > > 
> > > The global sync to the ecosystem isn't fun, I'll agree, but I do think
> > > that thinks have been better since Juno/Kilo because (1) we don't allow
> > > <= caps in g-r anymore (we were not allowing patch version updates which
> > > wedged us several times) and (2) we're better about releasing things
> > > with minor version updates per semver - whereas in the past the cats
> > > were releasing on their own volition and picking the version they
> > > thought was best, which creeped into having multiple versions that could
> > > be acceptable across branches, and we'd have wedges those ways too. I
> > > think a lot of that has been fixed by the openstack/releases repo so
> > > that the cats now have to line up for the release of their library with
> > > a centralized team.
> > > 
> > 
> > I'd agree with this, I still don't know what to cap things to.  I need
> > to figure out what the caps should be...  It could be hard to sync
> > across projects but like you say, most of that's gotten better since then.
> > 
> > >>
> > >>> You should be able to trivially pull those versions out and into your
> > >>> liberty set of packages.
> > >>>
> > >>> Theres another iteration on this in discussion now, which has to do
> > >>> with backwards compat *and testing of cap changes*, we'll be in the
> > >>> backwards compat fishbowl session in Tokyo if you're interested.
> > >>>
> > >>> -Rob
> > >>>
> > >>
> > >> I'll be at the fishbowl :D
> > >>
> > > 
> > 
> > Anyone have a sched link to that fishbowl, I can't find it :(
> > 
> 
> We're going to try to cover it in
> http://mitakadesignsummit.sched.org/event/27c17a3c35d72997b372ddf4759fe1be#.ViAE67SO820
> but we have a lot of other things to fit into that session so I want to
> put this one last so we don't derail the other discussions.

Looking back at the outline for that session, it's much more likely that
we'll postpone the discussion until Friday's meetup time.

http://mitakadesignsummit.sched.org/event/65564a6d98bb83d881b712fda033aaed#.ViAI0rSO820

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [packaging] liberty doesn't have caps on deps

[Matthew Thode]

On 10/15/2015 05:29 PM, Robert Collins wrote:
> On 16 October 2015 at 11:21, Matthew Thode  wrote:
>> On 10/15/2015 05:12 PM, Robert Collins wrote:
>>> On 16 October 2015 at 08:10, Matthew Thode  
>>> wrote:
 On 10/15/2015 02:04 PM, Robert Collins wrote:
>>> ...
>> Where are my caps?
>
> The known good versions of dependencies for liberty are
> http://git.openstack.org/cgit/openstack/requirements/tree/upper-constraints.txt?h=stable/liberty
>
 That's good, does that represent an upper constraint for the lower
 constraint imposed by by the package?  Why is this kept separate?
 Keeping it separate means that it's not trivial to merge them with
 what's in each package's requirements.
>>>
>>> It represents *one* known good combination of packages. We know that
>>> that combination passed CI, and we then do all our tests with it. To
>>> change it, we run it past CI and only move onto using the new set when
>>> its passed CI.
>>>
>>> If we merged it to each packages requirements, we'd reintroduce the
>>> deadlock that caused so much grief only 6 months back - I don't see
>>> any reason, desire, or tolerance for doing that upstream.
>>>
>>> Its kept separate because it requires 2N commits to shift known-good
>>> caps around for N repos using per-repo rules.
>>>
>>> With hundreds of repositories, it takes us hundreds of commits in two
>>> batches - and a round trip time of 2 hours per batch (check + gate) to
>>> shift *a single* requirement. With hundreds of dependencies, thats an
>>> intolerable amount of overhead.
>>>
>>
>> ya, that is annoying, unfortunately packages don't need to know *ONE*
>> good combination, they need to know them all, or at least the cap.  What
>> you are basically doing is shifting all of this extra work onto the
>> packagers, in fact I wouldn't be surprised if they needed to start
>> vendoring all of openstack in a virtualenv instead of doing actual packages.
> 
> Here's an example of the havoc caps cause:
>  https://review.openstack.org/#/c/234862/
> 
> I don't understand the statement about shifting work. Previously the
> situation was that you had to guess whether a given release of a
> dependency (both internal and external) worked with e.g. liberty.
> 
> Now there is an explicit list of what works.
> 
> Isn't that *better* ?
> 
Yes, it's good to know what works, does that show multiple versions of
the same package when multiple are known to work?  If so I can build a
range from that.  It's not better as it is because I still don't know
where liberty ends and mitaka begins.  Is there any place I can find that?

>> My question remains though, if someone pip installs nova liberty, will
>> it pull in deps from mitaka?  As it is now, without caps I cannot
>> reliably package openstack, do you have a solution? I should probably
>> start removing the liberty packages I did package since upstream seems
>> so hostile...
> 
> If you don't use the constraints file (which is pip consumable and
> published on the web so it can be used with pip install trivially)
> then yes, it will install the latest versions of all packages which
> are presumed to be doing backwards compatible changes. Things that we
> *know* are going to be broken still get caps - and we accept the cost
> of the 2-step dance needed to raise them.
> 

What about a daily or weekly check without the constraints file so you
know when something breaks?  This would allow you to at least know when
you need to place caps and I could consume that.  I'm fine with leaving
caps off.  If I can consume mitaka deps in liberty then that's great :D

> The big question here is, I think, 'should we assume OpenStack
> originated dependencies are better or worse than third party
> dependencies?' Third party dependencies we presume are backwards
> compatible (and will thus only break by mistake, which constraints
> guards against) unless we have reason not to - and we have open ended
> deps on them. This is where the spec about clients libraries is aimed.
> 
> It makes me very sad to know that you consider what we've done as
> hostile. We did it for a bunch of good reasons:
> 
>  - safer release process
>  - smoother updates for [apt, rpm at least] redistributors
>  - faster turnaround of dependency usage in CI
>  - step on the path to testing lower bounds
>  - step on the path to alignment with upstream packaging tooling
> 
> -Rob
> 
> 
It's a step backwards from my perspective, before I had a clear
delineation where support of something stops.  Now I don't know when it
stops and any given update could break the system.  I'm not sure how it
could be smoother for other systems.

So, does this mean that I can just leave the packages uncapped and know
that they will work?  Are there tests being run for this scenario?

-- 
Matthew Thode (prometheanfire)

__
OpenStack Development Mailing 

Re: [openstack-dev] [packaging] liberty doesn't have caps on deps

[Robert Collins]

On 16 October 2015 at 08:10, Matthew Thode  wrote:
> On 10/15/2015 02:04 PM, Robert Collins wrote:
...
>>> Where are my caps?
>>
>> The known good versions of dependencies for liberty are
>> http://git.openstack.org/cgit/openstack/requirements/tree/upper-constraints.txt?h=stable/liberty
>>
> That's good, does that represent an upper constraint for the lower
> constraint imposed by by the package?  Why is this kept separate?
> Keeping it separate means that it's not trivial to merge them with
> what's in each package's requirements.

It represents *one* known good combination of packages. We know that
that combination passed CI, and we then do all our tests with it. To
change it, we run it past CI and only move onto using the new set when
its passed CI.

If we merged it to each packages requirements, we'd reintroduce the
deadlock that caused so much grief only 6 months back - I don't see
any reason, desire, or tolerance for doing that upstream.

Its kept separate because it requires 2N commits to shift known-good
caps around for N repos using per-repo rules.

With hundreds of repositories, it takes us hundreds of commits in two
batches - and a round trip time of 2 hours per batch (check + gate) to
shift *a single* requirement. With hundreds of dependencies, thats an
intolerable amount of overhead.

>> You should be able to trivially pull those versions out and into your
>> liberty set of packages.
>>
>> Theres another iteration on this in discussion now, which has to do
>> with backwards compat *and testing of cap changes*, we'll be in the
>> backwards compat fishbowl session in Tokyo if you're interested.
>>
>> -Rob
>>
>
> I'll be at the fishbowl :D

Great!

-Rob


-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [packaging] liberty doesn't have caps on deps

[Robert Collins]

On 16 October 2015 at 11:21, Matthew Thode  wrote:
> On 10/15/2015 05:12 PM, Robert Collins wrote:
>> On 16 October 2015 at 08:10, Matthew Thode  wrote:
>>> On 10/15/2015 02:04 PM, Robert Collins wrote:
>> ...
> Where are my caps?

 The known good versions of dependencies for liberty are
 http://git.openstack.org/cgit/openstack/requirements/tree/upper-constraints.txt?h=stable/liberty

>>> That's good, does that represent an upper constraint for the lower
>>> constraint imposed by by the package?  Why is this kept separate?
>>> Keeping it separate means that it's not trivial to merge them with
>>> what's in each package's requirements.
>>
>> It represents *one* known good combination of packages. We know that
>> that combination passed CI, and we then do all our tests with it. To
>> change it, we run it past CI and only move onto using the new set when
>> its passed CI.
>>
>> If we merged it to each packages requirements, we'd reintroduce the
>> deadlock that caused so much grief only 6 months back - I don't see
>> any reason, desire, or tolerance for doing that upstream.
>>
>> Its kept separate because it requires 2N commits to shift known-good
>> caps around for N repos using per-repo rules.
>>
>> With hundreds of repositories, it takes us hundreds of commits in two
>> batches - and a round trip time of 2 hours per batch (check + gate) to
>> shift *a single* requirement. With hundreds of dependencies, thats an
>> intolerable amount of overhead.
>>
>
> ya, that is annoying, unfortunately packages don't need to know *ONE*
> good combination, they need to know them all, or at least the cap.  What
> you are basically doing is shifting all of this extra work onto the
> packagers, in fact I wouldn't be surprised if they needed to start
> vendoring all of openstack in a virtualenv instead of doing actual packages.

Here's an example of the havoc caps cause:
 https://review.openstack.org/#/c/234862/

I don't understand the statement about shifting work. Previously the
situation was that you had to guess whether a given release of a
dependency (both internal and external) worked with e.g. liberty.

Now there is an explicit list of what works.

Isn't that *better* ?

> My question remains though, if someone pip installs nova liberty, will
> it pull in deps from mitaka?  As it is now, without caps I cannot
> reliably package openstack, do you have a solution? I should probably
> start removing the liberty packages I did package since upstream seems
> so hostile...

If you don't use the constraints file (which is pip consumable and
published on the web so it can be used with pip install trivially)
then yes, it will install the latest versions of all packages which
are presumed to be doing backwards compatible changes. Things that we
*know* are going to be broken still get caps - and we accept the cost
of the 2-step dance needed to raise them.

The big question here is, I think, 'should we assume OpenStack
originated dependencies are better or worse than third party
dependencies?' Third party dependencies we presume are backwards
compatible (and will thus only break by mistake, which constraints
guards against) unless we have reason not to - and we have open ended
deps on them. This is where the spec about clients libraries is aimed.

It makes me very sad to know that you consider what we've done as
hostile. We did it for a bunch of good reasons:

 - safer release process
 - smoother updates for [apt, rpm at least] redistributors
 - faster turnaround of dependency usage in CI
 - step on the path to testing lower bounds
 - step on the path to alignment with upstream packaging tooling

-Rob


-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [packaging] liberty doesn't have caps on deps

[Matthew Thode]

On 10/15/2015 05:12 PM, Robert Collins wrote:
> On 16 October 2015 at 08:10, Matthew Thode  wrote:
>> On 10/15/2015 02:04 PM, Robert Collins wrote:
> ...
 Where are my caps?
>>>
>>> The known good versions of dependencies for liberty are
>>> http://git.openstack.org/cgit/openstack/requirements/tree/upper-constraints.txt?h=stable/liberty
>>>
>> That's good, does that represent an upper constraint for the lower
>> constraint imposed by by the package?  Why is this kept separate?
>> Keeping it separate means that it's not trivial to merge them with
>> what's in each package's requirements.
> 
> It represents *one* known good combination of packages. We know that
> that combination passed CI, and we then do all our tests with it. To
> change it, we run it past CI and only move onto using the new set when
> its passed CI.
> 
> If we merged it to each packages requirements, we'd reintroduce the
> deadlock that caused so much grief only 6 months back - I don't see
> any reason, desire, or tolerance for doing that upstream.
> 
> Its kept separate because it requires 2N commits to shift known-good
> caps around for N repos using per-repo rules.
> 
> With hundreds of repositories, it takes us hundreds of commits in two
> batches - and a round trip time of 2 hours per batch (check + gate) to
> shift *a single* requirement. With hundreds of dependencies, thats an
> intolerable amount of overhead.
> 

ya, that is annoying, unfortunately packages don't need to know *ONE*
good combination, they need to know them all, or at least the cap.  What
you are basically doing is shifting all of this extra work onto the
packagers, in fact I wouldn't be surprised if they needed to start
vendoring all of openstack in a virtualenv instead of doing actual packages.

My question remains though, if someone pip installs nova liberty, will
it pull in deps from mitaka?  As it is now, without caps I cannot
reliably package openstack, do you have a solution? I should probably
start removing the liberty packages I did package since upstream seems
so hostile...

>>> You should be able to trivially pull those versions out and into your
>>> liberty set of packages.
>>>
>>> Theres another iteration on this in discussion now, which has to do
>>> with backwards compat *and testing of cap changes*, we'll be in the
>>> backwards compat fishbowl session in Tokyo if you're interested.
>>>
>>> -Rob
>>>
>>
>> I'll be at the fishbowl :D
> 
> Great!
> 
> -Rob
> 
> 


-- 
-- Matthew Thode (prometheanfire)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [packaging] liberty doesn't have caps on deps

[Robert Collins]

On 16 October 2015 at 08:57, Doug Hellmann  wrote:
> Excerpts from Matthew Thode's message of 2015-10-15 14:35:08 -0500:
>> On 10/15/2015 02:17 PM, Matt Riedemann wrote:
>> >

>
> We're going to try to cover it in
> http://mitakadesignsummit.sched.org/event/27c17a3c35d72997b372ddf4759fe1be#.ViAE67SO820
> but we have a lot of other things to fit into that session so I want to
> put this one last so we don't derail the other discussions.

I thought it was
http://mitakadesignsummit.sched.org/event/0a2307779b4ab81892ba24de379e9dcc#.ViAMfd94u00
that it was slated for, where its squarely on topic :)

-Rob

-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [packaging] liberty doesn't have caps on deps

[Matthew Thode]

On 10/15/2015 02:17 PM, Matt Riedemann wrote:
> 
> 
> On 10/15/2015 2:10 PM, Matthew Thode wrote:
>> On 10/15/2015 02:04 PM, Robert Collins wrote:
>>> On 16 October 2015 at 08:01, Matthew Thode
>>>  wrote:
 So, this is my perspective in packing liberty for Gentoo.

 We can have multiple versions of a package available to install,
 because
 of this we generally directly translate the valid dependency versions
 from requirements.
>>>
>>> Cool.
>>>
 this
  oslo.concurrency>=2.3.0 # Apache-2.0
 becomes this
  >=dev-python/oslo-concurrency-2.3.0[${PYTHON_USEDEP}]

 Now what happens when I package something from mitaka (2.7.0 in this
 case would be mitaka).  It's undefined behaviour as far as I know.

 Basically, I can see no reason why the policy of caps changed from kilo
 to liberty, it was actually nice to package for liberty, I can see this
 going very bad very quick.
>>>
>>> They changed because it was causing huge trauma and multiple day long
>>> gate wedges around release times. Covered in detail here -
>>> http://specs.openstack.org/openstack/openstack-specs/specs/requirements-management.html
>>>
>>>
 Where are my caps?
>>>
>>> The known good versions of dependencies for liberty are
>>> http://git.openstack.org/cgit/openstack/requirements/tree/upper-constraints.txt?h=stable/liberty
>>>
>>>
>> That's good, does that represent an upper constraint for the lower
>> constraint imposed by by the package?  Why is this kept separate?
>> Keeping it separate means that it's not trivial to merge them with
>> what's in each package's requirements.
> 
> I'm not sure I understand the first question but I believe that u-c is
> automatically adjusted and if there is a conflict with the minimum
> version required of a dependency then the cap is adjusted (or vice-versa).
> 
> It's separate, at least in part, because the changes to
> global-requirements are synced to all projects in the projects.txt file
> in the requirements repo, which causes a bunch of churn to get those
> changes approved in the registered projects and then released
> appropriately.
> 
> The global sync to the ecosystem isn't fun, I'll agree, but I do think
> that thinks have been better since Juno/Kilo because (1) we don't allow
> <= caps in g-r anymore (we were not allowing patch version updates which
> wedged us several times) and (2) we're better about releasing things
> with minor version updates per semver - whereas in the past the cats
> were releasing on their own volition and picking the version they
> thought was best, which creeped into having multiple versions that could
> be acceptable across branches, and we'd have wedges those ways too. I
> think a lot of that has been fixed by the openstack/releases repo so
> that the cats now have to line up for the release of their library with
> a centralized team.
> 

I'd agree with this, I still don't know what to cap things to.  I need
to figure out what the caps should be...  It could be hard to sync
across projects but like you say, most of that's gotten better since then.

>>
>>> You should be able to trivially pull those versions out and into your
>>> liberty set of packages.
>>>
>>> Theres another iteration on this in discussion now, which has to do
>>> with backwards compat *and testing of cap changes*, we'll be in the
>>> backwards compat fishbowl session in Tokyo if you're interested.
>>>
>>> -Rob
>>>
>>
>> I'll be at the fishbowl :D
>>
> 

Anyone have a sched link to that fishbowl, I can't find it :(

-- 
Matthew Thode

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [packaging] liberty doesn't have caps on deps

[Robert Collins]

On 16 October 2015 at 08:01, Matthew Thode  wrote:
> So, this is my perspective in packing liberty for Gentoo.
>
> We can have multiple versions of a package available to install, because
> of this we generally directly translate the valid dependency versions
> from requirements.

Cool.

> this
> oslo.concurrency>=2.3.0 # Apache-2.0
> becomes this
> >=dev-python/oslo-concurrency-2.3.0[${PYTHON_USEDEP}]
>
> Now what happens when I package something from mitaka (2.7.0 in this
> case would be mitaka).  It's undefined behaviour as far as I know.
>
> Basically, I can see no reason why the policy of caps changed from kilo
> to liberty, it was actually nice to package for liberty, I can see this
> going very bad very quick.

They changed because it was causing huge trauma and multiple day long
gate wedges around release times. Covered in detail here -
http://specs.openstack.org/openstack/openstack-specs/specs/requirements-management.html

> Where are my caps?

The known good versions of dependencies for liberty are
http://git.openstack.org/cgit/openstack/requirements/tree/upper-constraints.txt?h=stable/liberty

You should be able to trivially pull those versions out and into your
liberty set of packages.

Theres another iteration on this in discussion now, which has to do
with backwards compat *and testing of cap changes*, we'll be in the
backwards compat fishbowl session in Tokyo if you're interested.

-Rob

-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [packaging] liberty doesn't have caps on deps

[Matthew Thode]

On 10/15/2015 02:04 PM, Robert Collins wrote:
> On 16 October 2015 at 08:01, Matthew Thode  wrote:
>> So, this is my perspective in packing liberty for Gentoo.
>>
>> We can have multiple versions of a package available to install, because
>> of this we generally directly translate the valid dependency versions
>> from requirements.
> 
> Cool.
> 
>> this
>> oslo.concurrency>=2.3.0 # Apache-2.0
>> becomes this
>> >=dev-python/oslo-concurrency-2.3.0[${PYTHON_USEDEP}]
>>
>> Now what happens when I package something from mitaka (2.7.0 in this
>> case would be mitaka).  It's undefined behaviour as far as I know.
>>
>> Basically, I can see no reason why the policy of caps changed from kilo
>> to liberty, it was actually nice to package for liberty, I can see this
>> going very bad very quick.
> 
> They changed because it was causing huge trauma and multiple day long
> gate wedges around release times. Covered in detail here -
> http://specs.openstack.org/openstack/openstack-specs/specs/requirements-management.html
> 
>> Where are my caps?
> 
> The known good versions of dependencies for liberty are
> http://git.openstack.org/cgit/openstack/requirements/tree/upper-constraints.txt?h=stable/liberty
> 
That's good, does that represent an upper constraint for the lower
constraint imposed by by the package?  Why is this kept separate?
Keeping it separate means that it's not trivial to merge them with
what's in each package's requirements.

> You should be able to trivially pull those versions out and into your
> liberty set of packages.
> 
> Theres another iteration on this in discussion now, which has to do
> with backwards compat *and testing of cap changes*, we'll be in the
> backwards compat fishbowl session in Tokyo if you're interested.
> 
> -Rob
> 

I'll be at the fishbowl :D

-- 
Matthew Thode (prometheanfire)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [packaging] liberty doesn't have caps on deps

[Matt Riedemann]

On 10/15/2015 2:10 PM, Matthew Thode wrote:

On 10/15/2015 02:04 PM, Robert Collins wrote:

On 16 October 2015 at 08:01, Matthew Thode  wrote:

So, this is my perspective in packing liberty for Gentoo.

We can have multiple versions of a package available to install, because
of this we generally directly translate the valid dependency versions
from requirements.


Cool.


this
 oslo.concurrency>=2.3.0 # Apache-2.0
becomes this
 >=dev-python/oslo-concurrency-2.3.0[${PYTHON_USEDEP}]

Now what happens when I package something from mitaka (2.7.0 in this
case would be mitaka).  It's undefined behaviour as far as I know.

Basically, I can see no reason why the policy of caps changed from kilo
to liberty, it was actually nice to package for liberty, I can see this
going very bad very quick.


They changed because it was causing huge trauma and multiple day long
gate wedges around release times. Covered in detail here -
http://specs.openstack.org/openstack/openstack-specs/specs/requirements-management.html


Where are my caps?


The known good versions of dependencies for liberty are
http://git.openstack.org/cgit/openstack/requirements/tree/upper-constraints.txt?h=stable/liberty


That's good, does that represent an upper constraint for the lower
constraint imposed by by the package?  Why is this kept separate?
Keeping it separate means that it's not trivial to merge them with
what's in each package's requirements.


I'm not sure I understand the first question but I believe that u-c is 
automatically adjusted and if there is a conflict with the minimum 
version required of a dependency then the cap is adjusted (or vice-versa).


It's separate, at least in part, because the changes to 
global-requirements are synced to all projects in the projects.txt file 
in the requirements repo, which causes a bunch of churn to get those 
changes approved in the registered projects and then released appropriately.


The global sync to the ecosystem isn't fun, I'll agree, but I do think 
that thinks have been better since Juno/Kilo because (1) we don't allow 
<= caps in g-r anymore (we were not allowing patch version updates which 
wedged us several times) and (2) we're better about releasing things 
with minor version updates per semver - whereas in the past the cats 
were releasing on their own volition and picking the version they 
thought was best, which creeped into having multiple versions that could 
be acceptable across branches, and we'd have wedges those ways too. I 
think a lot of that has been fixed by the openstack/releases repo so 
that the cats now have to line up for the release of their library with 
a centralized team.





You should be able to trivially pull those versions out and into your
liberty set of packages.

Theres another iteration on this in discussion now, which has to do
with backwards compat *and testing of cap changes*, we'll be in the
backwards compat fishbowl session in Tokyo if you're interested.

-Rob



I'll be at the fishbowl :D



--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev