Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Morgan Fainberg]

 On Jun 16, 2015, at 14:56, Thomas Goirand z...@debian.org wrote:
 
 On 06/16/2015 12:06 PM, Thierry Carrez wrote:
 It also removes the stupid encouragement to use all components from the
 same date. With everything tagged at the same date, you kinda send the
 message that those various things should be used together. With
 everything tagged separately, you send te message that you can mix and
 match components from stable/* as you see fit. I mean, it's totally
 valid to use stable branch components from various points in time
 together, since they are all supposed to work.
 
 Though there's now zero guidance at what should be the speed of
 releasing server packages to our users.
 
 I really think it should be a distribution decision. You could release
 all commits, release every 2 months, release after each CVE, release
 as-needed when a bug in Debian BTS is fixed. I don't see what guidance
 upstream should give, apart from enabling all models. Currently we make
 most models more difficult than they should be, to promote an arbitrary
 time-based model. With plan D, we enable all models.
 
 Let me put this in another way: with the plan D, I'll be lost, and wont
 ever know when to release a new stable version in Debian. I don't know
 better than anyone else. If we had each upstream project saying
 individually: Ok, now we gathered enough bugfixes so that it's
 important to get it in downstream distributions, I'd happily follow
 this kind of guidance. But the plan is to just commit bugfixes, and hope
 that downstream distros (ie: me in this case) just catch when a new
 release worse the effort.
 
 As pointed elsewhere, plan D assumes we move to generating release notes
 for each commit. So you won't lose track of what is fixed in each
 version. If anything, that will give you proper release notes for
 CVE-fix commits, something you didn't have before, since we wouldn't cut
 a proper point release after a CVE fix but on a pre-determined
 time-based schedule.
 
 Overall, I think even your process stands to benefit from the proposed
 evolution.
 
 I just hope so. If any core / PTL is reading me in this thread, I would
 strongly encourage you guys to get in touch and ping me when you think
 some commits in the stable release should be uploaded to Debian. A quick
 message on IRC can be enough.
 

For what it is worth, I trust the downstream distributions to make this call. I 
expect that any/all stable patches accepted in a model where release notes are 
generated per-commit (Plan D) this ends up looking like the Redhat model where 
patches are bundled in. 

In general we should have care in accepting a stable patch. But as the PTL (and 
more generally as a core) asking me to determine when there is enough patches 
to generate a release is far too distribution/packager specific and 
subjective. I do not expect to be reaching out to each and every one to say 
hey did you release? This, in my opinion is not a reasonable ask. I do not 
know what justifies time for a release for Debian or Redhat or Suse or Gentoo 
... Etc. Please do not expect the PTLs and Cores to become experts on each and 
every one of these. I expect the packager to be making the subjective call in 
the non-time-based model outlined. 

--Morgan
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thierry Carrez]

Ihar Hrachyshka wrote:
 On 06/17/2015 05:40 PM, Douglas Mendizábal wrote:
 I tend to agree with Thomas that plan D is not ideal.  For one, it 
 prevents changes to the stable branch that span multiple CRs, since
 a two patch change would generate two tags and there would be no
 clear indication that the first patch should not be released on its
 own.
 
 If we will end up with a half-broken product due to merging a patch
 without another one, then those patches should be squashed. Also, I
 wonder how they will pass gate if something is broken. Do you suggest
 that test coverage is incomplete?

Also remember we are talking about *stable branches* here. The only
acceptable things there are bugfix backports and CVE fixes. So:

- there is no case of partially-merged features that span multiple CRs
- in the rare case of a bugfix needing multiple commits, those should be
squashed in the stable branch
- if the fix is so huge that it can't be merged as a single patch, it's
probably not desirable in the stable branch

The stable branch should always be usable and releaseable. If it's not,
you're doing it wrong.

-- 
Thierry Carrez (ttx)



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Dave Walker]

On 17 June 2015 at 17:19, Thierry Carrez thie...@openstack.org wrote:
 Ihar Hrachyshka wrote:
 On 06/17/2015 05:40 PM, Douglas Mendizábal wrote:
 I tend to agree with Thomas that plan D is not ideal.  For one, it
 prevents changes to the stable branch that span multiple CRs, since
 a two patch change would generate two tags and there would be no
 clear indication that the first patch should not be released on its
 own.

 If we will end up with a half-broken product due to merging a patch
 without another one, then those patches should be squashed. Also, I
 wonder how they will pass gate if something is broken. Do you suggest
 that test coverage is incomplete?

 Also remember we are talking about *stable branches* here. The only
 acceptable things there are bugfix backports and CVE fixes. So:

 - there is no case of partially-merged features that span multiple CRs
 - in the rare case of a bugfix needing multiple commits, those should be
 squashed in the stable branch
 - if the fix is so huge that it can't be merged as a single patch, it's
 probably not desirable in the stable branch

 The stable branch should always be usable and releaseable. If it's not,
 you're doing it wrong.


Just to add to this, we have a decent demonstrable history of
squashing commits together that have been required to land
concurrently.  We need to do this when the Gate is wedged, and
requires two or more concurrent backports to unwedge.

Such as: https://review.openstack.org/#/c/163378/

As Ihar points out, the test coverage mandates that we do this.. which
nicely helps us provide some assurance that the branch is constantly
releasable.

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Douglas Mendizábal]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

I tend to agree with Thomas that plan D is not ideal.  For one, it
prevents changes to the stable branch that span multiple CRs, since a
two patch change would generate two tags and there would be no clear
indication that the first patch should not be released on its own.

My preference would be for Plan C where the PTL would push a new
2015.1.XXX tag when an important fix (or fixes) lands in a stable branch
.

I also disagree with Morgan that packagers are better informed to make
the determination of when to release.  PTLs should be aware of all
changes landing in the stable branches, and should be able to push a
tag immediately after an important fix lands.  Asking the packagers to
make the determination means that they would have to be aware of every
patch landing in every project, which I think is a lot to ask.

- - Douglas Mendizábal

On 6/17/15 9:55 AM, Morgan Fainberg wrote:
 
 On Jun 16, 2015, at 14:56, Thomas Goirand z...@debian.org
 wrote:
 
 On 06/16/2015 12:06 PM, Thierry Carrez wrote:
 It also removes the stupid encouragement to use all
 components from the same date. With everything tagged at
 the same date, you kinda send the message that those
 various things should be used together. With everything
 tagged separately, you send te message that you can mix
 and match components from stable/* as you see fit. I mean,
 it's totally valid to use stable branch components from
 various points in time together, since they are all
 supposed to work.
 
 Though there's now zero guidance at what should be the speed
 of releasing server packages to our users.
 
 I really think it should be a distribution decision. You could
 release all commits, release every 2 months, release after each
 CVE, release as-needed when a bug in Debian BTS is fixed. I
 don't see what guidance upstream should give, apart from
 enabling all models. Currently we make most models more
 difficult than they should be, to promote an arbitrary 
 time-based model. With plan D, we enable all models.
 
 Let me put this in another way: with the plan D, I'll be lost,
 and wont ever know when to release a new stable version in
 Debian. I don't know better than anyone else. If we had each
 upstream project saying individually: Ok, now we gathered enough
 bugfixes so that it's important to get it in downstream
 distributions, I'd happily follow this kind of guidance. But the
 plan is to just commit bugfixes, and hope that downstream distros
 (ie: me in this case) just catch when a new release worse the
 effort.
 
 As pointed elsewhere, plan D assumes we move to generating
 release notes for each commit. So you won't lose track of what
 is fixed in each version. If anything, that will give you
 proper release notes for CVE-fix commits, something you didn't
 have before, since we wouldn't cut a proper point release after
 a CVE fix but on a pre-determined time-based schedule.
 
 Overall, I think even your process stands to benefit from the
 proposed evolution.
 
 I just hope so. If any core / PTL is reading me in this thread, I
 would strongly encourage you guys to get in touch and ping me
 when you think some commits in the stable release should be
 uploaded to Debian. A quick message on IRC can be enough.
 
 
 For what it is worth, I trust the downstream distributions to make
 this call. I expect that any/all stable patches accepted in a model
 where release notes are generated per-commit (Plan D) this ends up
 looking like the Redhat model where patches are bundled in.
 
 In general we should have care in accepting a stable patch. But as
 the PTL (and more generally as a core) asking me to determine when
 there is enough patches to generate a release is far too
 distribution/packager specific and subjective. I do not expect to
 be reaching out to each and every one to say hey did you release?
 This, in my opinion is not a reasonable ask. I do not know what
 justifies time for a release for Debian or Redhat or Suse or
 Gentoo ... Etc. Please do not expect the PTLs and Cores to become
 experts on each and every one of these. I expect the packager to be
 making the subjective call in the non-time-based model outlined.
 
 --Morgan 
 __


 
OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe 
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
-BEGIN PGP SIGNATURE-
Comment: GPGTools - https://gpgtools.org

iQIcBAEBCgAGBQJVgZTgAAoJEB7Z2EQgmLX7qhkP/RwB4sJU2nXeAe4M27VXT/9t
tPD3j0njOMuL6T5g41D/aXIcEvvk1RqjJOg+VdXq7Os3t0AAHNkKng85qGKp+Our
xMOSGBtFjqQycDDVEFrltYCkhmDdmsVw4SWd0zmJ2MHzn+R014lZL5SdsHjhWmRr
c3NfZH0Cxm9ujfMbdjsZoF3i6noZsotUZLy5Tu5iW/Kp46syNDfA47Fxya15T54a
LyA3Qqw/C/1Nov89IyS0AA4nv5BzmWGh8+t6rZnC2NQUPVGVnvnpr5KOIadbwqEh
BtLQWaeXDlbJPovCpHnZ9CUwtIZRdYUXoXgdYJVFcBMmDM0NrIWX4+YlJ5rCgdeC

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Ihar Hrachyshka]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 06/17/2015 05:40 PM, Douglas Mendizábal wrote:
 I tend to agree with Thomas that plan D is not ideal.  For one, it 
 prevents changes to the stable branch that span multiple CRs, since
 a two patch change would generate two tags and there would be no
 clear indication that the first patch should not be released on its
 own.

If we will end up with a half-broken product due to merging a patch
without another one, then those patches should be squashed. Also, I
wonder how they will pass gate if something is broken. Do you suggest
that test coverage is incomplete?

 
 My preference would be for Plan C where the PTL would push a new 
 2015.1.XXX tag when an important fix (or fixes) lands in a stable
 branch .
 
 I also disagree with Morgan that packagers are better informed to
 make the determination of when to release.  PTLs should be aware of
 all changes landing in the stable branches, and should be able to
 push a tag immediately after an important fix lands.  Asking the
 packagers to make the determination means that they would have to
 be aware of every patch landing in every project, which I think is
 a lot to ask.

I wouldn't expect e.g. neutron PTL to know every since patch in stable
branches. If anyone, you better put the burden onto corresponding
- -stable-maint team.

 
 - Douglas Mendizábal
 
 On 6/17/15 9:55 AM, Morgan Fainberg wrote:
 
 On Jun 16, 2015, at 14:56, Thomas Goirand z...@debian.org 
 wrote:
 
 On 06/16/2015 12:06 PM, Thierry Carrez wrote:
 It also removes the stupid encouragement to use all 
 components from the same date. With everything tagged at 
 the same date, you kinda send the message that those 
 various things should be used together. With everything 
 tagged separately, you send te message that you can mix 
 and match components from stable/* as you see fit. I
 mean, it's totally valid to use stable branch components
 from various points in time together, since they are all 
 supposed to work.
 
 Though there's now zero guidance at what should be the
 speed of releasing server packages to our users.
 
 I really think it should be a distribution decision. You
 could release all commits, release every 2 months, release
 after each CVE, release as-needed when a bug in Debian BTS is
 fixed. I don't see what guidance upstream should give,
 apart from enabling all models. Currently we make most models
 more difficult than they should be, to promote an arbitrary 
 time-based model. With plan D, we enable all models.
 
 Let me put this in another way: with the plan D, I'll be lost, 
 and wont ever know when to release a new stable version in 
 Debian. I don't know better than anyone else. If we had each 
 upstream project saying individually: Ok, now we gathered
 enough bugfixes so that it's important to get it in downstream 
 distributions, I'd happily follow this kind of guidance. But
 the plan is to just commit bugfixes, and hope that downstream
 distros (ie: me in this case) just catch when a new release
 worse the effort.
 
 As pointed elsewhere, plan D assumes we move to generating 
 release notes for each commit. So you won't lose track of
 what is fixed in each version. If anything, that will give
 you proper release notes for CVE-fix commits, something you
 didn't have before, since we wouldn't cut a proper point
 release after a CVE fix but on a pre-determined time-based
 schedule.
 
 Overall, I think even your process stands to benefit from
 the proposed evolution.
 
 I just hope so. If any core / PTL is reading me in this thread,
 I would strongly encourage you guys to get in touch and ping
 me when you think some commits in the stable release should be 
 uploaded to Debian. A quick message on IRC can be enough.
 
 
 For what it is worth, I trust the downstream distributions to
 make this call. I expect that any/all stable patches accepted in
 a model where release notes are generated per-commit (Plan D)
 this ends up looking like the Redhat model where patches are
 bundled in.
 
 In general we should have care in accepting a stable patch. But
 as the PTL (and more generally as a core) asking me to determine
 when there is enough patches to generate a release is far too 
 distribution/packager specific and subjective. I do not expect
 to be reaching out to each and every one to say hey did you
 release? This, in my opinion is not a reasonable ask. I do not
 know what justifies time for a release for Debian or Redhat or
 Suse or Gentoo ... Etc. Please do not expect the PTLs and Cores
 to become experts on each and every one of these. I expect the
 packager to be making the subjective call in the non-time-based
 model outlined.
 
 --Morgan 
 _
_

 

 
 
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: 
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe 
 

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/16/2015 12:06 PM, Thierry Carrez wrote:
 It also removes the stupid encouragement to use all components from the
 same date. With everything tagged at the same date, you kinda send the
 message that those various things should be used together. With
 everything tagged separately, you send te message that you can mix and
 match components from stable/* as you see fit. I mean, it's totally
 valid to use stable branch components from various points in time
 together, since they are all supposed to work.

 Though there's now zero guidance at what should be the speed of
 releasing server packages to our users.
 
 I really think it should be a distribution decision. You could release
 all commits, release every 2 months, release after each CVE, release
 as-needed when a bug in Debian BTS is fixed. I don't see what guidance
 upstream should give, apart from enabling all models. Currently we make
 most models more difficult than they should be, to promote an arbitrary
 time-based model. With plan D, we enable all models.

Let me put this in another way: with the plan D, I'll be lost, and wont
ever know when to release a new stable version in Debian. I don't know
better than anyone else. If we had each upstream project saying
individually: Ok, now we gathered enough bugfixes so that it's
important to get it in downstream distributions, I'd happily follow
this kind of guidance. But the plan is to just commit bugfixes, and hope
that downstream distros (ie: me in this case) just catch when a new
release worse the effort.

 As pointed elsewhere, plan D assumes we move to generating release notes
 for each commit. So you won't lose track of what is fixed in each
 version. If anything, that will give you proper release notes for
 CVE-fix commits, something you didn't have before, since we wouldn't cut
 a proper point release after a CVE fix but on a pre-determined
 time-based schedule.
 
 Overall, I think even your process stands to benefit from the proposed
 evolution.

I just hope so. If any core / PTL is reading me in this thread, I would
strongly encourage you guys to get in touch and ping me when you think
some commits in the stable release should be uploaded to Debian. A quick
message on IRC can be enough.

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Ihar Hrachyshka]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 06/15/2015 11:24 PM, Thomas Goirand wrote:
 On 06/15/2015 05:19 PM, Ian Cordasco wrote:
 On 6/15/15, 09:24, Thomas Goirand z...@debian.org wrote:
 
 On 06/08/2015 01:55 PM, Kuvaja, Erno wrote:
 One thing I like about plan D is that it would give also
 indicator how much the stable branch has moved in each
 individual project.
 
 The only indication you will get is how many patches it has. I
 fail to see how this is valuable information. No info on how
 important they are or anything of this kind, which is a way
 more important.
 
 Thomas
 
 Are you implying that stable point releases as they exist today
 provide importance? How is that case?
 
 I'm just replying to Kuvaja, Erno, who pretends that plan D will
 give indicator how much the stable branch has moved in. My answer
 is: not it wont. Maybe there was lots of cosmetic patches (like
 typos, man page, or who knows...). The number of patch is an
 indication of ... nothing, really!
 
 The only way to have indications is release notes. Since it looks
 like there's not enough resources to write them, we're giving up on
 them. It's really a shame, because I'm sure it was valuable for our
 users.
 

Proper estimation of available resources is not a shame, it's realistic.

If you have resources to spare on actually maintaining stable branches
and release process around them, you're always welcome to join the
effort. For now, I see you are comfortable in your position of a
consumer of deliverables. That's fine, but also keep in mind you get
what you pay.

Ihar
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQEcBAEBCAAGBQJVf+VtAAoJEC5aWaUY1u57/AYIAOquvtP3wxhrXuZfWquqWedq
oPCkrJVSMlKpW3WUJr7/3xqoDzVen4UT/0Du0T+uE6cTBTlt3Fqt1cEh4iQ1WOcA
qBfFJlX7HHpwkJnjF5MJBiTCiDjegrQFTRPOb/O46hSE+a1JUNPH9ob1OiKpyk39
oSGSSCI2YmS5IIpiUNDsYzRK6twVZsCut+18AXp89Hx8oK8t7Lu3jfrEq8Z5R5Hv
xnea0AV+/WZLchUPqGss5htOhqVkXH6A3W9Wvg10427tPCVin/5SIcfCvD/2zZ0a
V9/ikUphtUq0xV2zh2Uc17Tkkub8hKD21u+v7XRiy8RijPr4ouct77gOLpP5JuU=
=F0Vb
-END PGP SIGNATURE-

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thierry Carrez]

Thomas Goirand wrote:
 On 06/10/2015 03:46 PM, Thierry Carrez wrote:
 So we could do what you're asking (option B) for Kilo stable releases,
 but I think it's not really a viable option for stable/liberty onward.
 
 I fail to understand how version numbers are related to doing
 synchronized point releases. Sure, we have a problem with the version
 numbers, but we can still do a point release of all the server projects
 at once.

Agreed. We could still do a coordinated point release without using
the same version numbers. We'd call it the kilo.1 release and it would
translate into different point release version numbers for each project.
That could even be done on top of plan D by a group of people caring
enough to create a wiki page to reference the corresponding version numbers.

 FYI, after a CVE, I have uploaded cinder with this version number:
 2015.1.0+2015.06.06.git24.493b6c7f12-1
 
 IMO, it looks ugly, and not comprehensive to Debian users. If I really
 have to go this way, I will, but I would have  not to.

Like Alan said, with plan D this actually looks a lot better. You'd be
able to package/distribute 2015.1.25 instead of
2015.1.0+2015.06.06.git24.493b6c7f12-1

 It also removes the stupid encouragement to use all components from the
 same date. With everything tagged at the same date, you kinda send the
 message that those various things should be used together. With
 everything tagged separately, you send te message that you can mix and
 match components from stable/* as you see fit. I mean, it's totally
 valid to use stable branch components from various points in time
 together, since they are all supposed to work.
 
 Though there's now zero guidance at what should be the speed of
 releasing server packages to our users.

I really think it should be a distribution decision. You could release
all commits, release every 2 months, release after each CVE, release
as-needed when a bug in Debian BTS is fixed. I don't see what guidance
upstream should give, apart from enabling all models. Currently we make
most models more difficult than they should be, to promote an arbitrary
time-based model. With plan D, we enable all models.

 So I totally get that we should still have reference points to be able
 to tell this is fixed in openstack/nova stable/liberty starting with
 12.1.134post34 (or whatever we settle with). I totally get that any of
 those should ship with relevant release notes. But that's about all I
 think we need ?
 
 That's not a little thing that you're pointing at: having reference
 points is very important. But yeah, it may be the only thing... Like
 we will only loose track of what is fixed in which version. :(

As pointed elsewhere, plan D assumes we move to generating release notes
for each commit. So you won't lose track of what is fixed in each
version. If anything, that will give you proper release notes for
CVE-fix commits, something you didn't have before, since we wouldn't cut
a proper point release after a CVE fix but on a pre-determined
time-based schedule.

Overall, I think even your process stands to benefit from the proposed
evolution.

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/08/2015 01:55 PM, Kuvaja, Erno wrote:
 One thing I like about plan D
 is that it would give also indicator how much the stable branch has moved in
 each individual project. 

The only indication you will get is how many patches it has. I fail to
see how this is valuable information. No info on how important they are
or anything of this kind, which is a way more important.

Thomas


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/10/2015 03:46 PM, Thierry Carrez wrote:
 The main issue with B is that it doesn't work well once server component
 versions start to diverge, which will be the case starting with Liberty.

Review that policy then.

 We already couldn't (with Swift using separate versioning), but we worked
 around that by ignoring Swift from stable releases. As more projects opt
 into that, that will no longer be a possible workaround.

The only issue is having a correct version number. Swift could well
iterate after 20151, for example doing 20151.1 then 20151.2, etc.

 So we could do what you're asking (option B) for Kilo stable releases,
 but I think it's not really a viable option for stable/liberty onward.

I fail to understand how version numbers are related to doing
synchronized point releases. Sure, we have a problem with the version
numbers, but we can still do a point release of all the server projects
at once.

 This is really one of the things I think we want to get away from...
 If *every* stable commit is treated with the seriousness of it
 creating a release, lets make every commit a release.

 This means that Debian may be using a (micro)patch release newer or
 older than a different distro, but the key is that it empowers the
 vendors and/or users to select a release cadence that best fits them,
 rather than being tied to an arbitrary upstream community wide date.
 
 +1

FYI, after a CVE, I have uploaded cinder with this version number:
2015.1.0+2015.06.06.git24.493b6c7f12-1

IMO, it looks ugly, and not comprehensive to Debian users. If I really
have to go this way, I will, but I would have  not to.

 It also removes the stupid encouragement to use all components from the
 same date. With everything tagged at the same date, you kinda send the
 message that those various things should be used together. With
 everything tagged separately, you send te message that you can mix and
 match components from stable/* as you see fit. I mean, it's totally
 valid to use stable branch components from various points in time
 together, since they are all supposed to work.

Though there's now zero guidance at what should be the speed of
releasing server packages to our users.

 So I totally get that we should still have reference points to be able
 to tell this is fixed in openstack/nova stable/liberty starting with
 12.1.134post34 (or whatever we settle with). I totally get that any of
 those should ship with relevant release notes. But that's about all I
 think we need ?

That's not a little thing that you're pointing at: having reference
points is very important. But yeah, it may be the only thing... Like
we will only loose track of what is fixed in which version. :(

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/10/2015 11:27 AM, Dave Walker wrote:
 On 10 June 2015 at 09:53, Thomas Goirand z...@debian.org wrote:
 On 06/05/2015 02:46 PM, Thierry Carrez wrote:
 So.. summarizing the various options again:

 Plan A
 Just drop stable point releases.
 (-) No more release notes
 (-) Lack of reference points to compare installations

 Plan B
 Push date-based tags across supported projects from time to time.
 (-) Encourages to continue using same version across the board
 (-) Almost as much work as making proper releases

 Plan C
 Let projects randomly tag point releases whenever
 (-) Still a bit costly in terms of herding cats

 Plan D
 Drop stable point releases, publish per-commit tarballs
 (-) Requires some infra changes, takes some storage space

 Plans B, C and D also require some release note / changelog generation
 from data maintained *within* the repository.

 Personally I think the objections raised against plan A are valid. I
 like plan D, since it's more like releasing every commit than not
 releasing anymore. I think it's the most honest trade-off. I could go
 with plan C, but I think it's added work for no additional value to the
 user.

 What would be your preferred option ?

 I see no point of doing D. I already don't use tarballs, and those who
 do could as well switch to generating them (how hard is it to run
 python setup.py sdist or git archive?).

 What counts is having a schedule date, where all distros are releasing a
 point release, so we have a common reference point. If that is a fully
 automated process, then great, less work for everyone, and it wont
 change anything from what we had in the past (we can even collectively
 decide for point release dates...).

 Cheers,

 Thomas Goirand (zigo)
 
 This is really one of the things I think we want to get away from...
 If *every* stable commit is treated with the seriousness of it
 creating a release, lets make every commit a release.
 
 This means that Debian may be using a (micro)patch release newer or
 older than a different distro, but the key is that it empowers the
 vendors and/or users to select a release cadence that best fits them,
 rather than being tied to an arbitrary upstream community wide date.

What you don't get here is that downstream distributions *DO* want the
arbitrary upstream community wide date, and don't want to just use any
commit.

 Yes, this might mean that your cadence might be more or less regular
 than an alternative vendor / distribution, but the key is that it
 empowers the vendor to meet the needs of their users/customers.

When did a distribution vendors express this as an issue? Having
community-wide release dates doesn't prevent any vendor to do a patch
release if he wants to.

 For example, you could select a cadence of rebasing to a release every
 6 months - where as another consumer could choose to do one every 6
 weeks.

Which is what would be nice to avoid, so we have the same code base.
Otherwise we may be affected differently by a CVE.

 The difference is how much of a jump, at which intervals..
 Alternatively, a vendor might choose just to go with stock release +
 their own select cherry picked patches from stable/*, which is also a
 model that works.

This already happens, we don't need to remove point releases to do that.

 The issue around producing tarballs, is really about having forwards
 and backwards verification by means of sha/md5 sums, which is hard to
 do when generating your own orig tarball.

Opposite way. When generating my tarball, I do it with a GPG signed tag.
This is verifiable very easily. By the way, sha  md5 are in no way
tools to sign a release, for that, we have PGP (and cross-signing of keys).

 Debian, Ubuntu and I
 believe Arch have made varying use of 'pristine-tar' - which was an
 effort to re-producible tarballs using xdelta to make the sum match.
 However, Maintainers seem to be moving away from this now.

As much as I know, I'm the only one using git archive ... | xz ... to
generate my own tarballs, but maybe this will change some day.

 When I perform source NEW reviews for Ubuntu Archive, I always check
 that getting the source orig tarball can be done with either
 get-orig-source (inspecting the generation method) or uscan and then
 diff the tarballs with the one included on the upload and the one
 generated.  Timestamps (or even shasums) haven't been an important
 issue for me, but the actual content and verifiable source is what has
 mattered more.

Correct. Though for me, a signed git tag is a way better than any md5 in
the world.

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Ian Cordasco]

On 6/15/15, 09:24, Thomas Goirand z...@debian.org wrote:

On 06/08/2015 01:55 PM, Kuvaja, Erno wrote:
 One thing I like about plan D
 is that it would give also indicator how much the stable branch has
moved in
 each individual project.

The only indication you will get is how many patches it has. I fail to
see how this is valuable information. No info on how important they are
or anything of this kind, which is a way more important.

Thomas

Are you implying that stable point releases as they exist today provide
importance? How is that case? They're coordinated to happen at (nearly)
the same time and that's about all. Perhaps the most important changes are
CVE fixes. Let's look at a two cases for a stable point release now:

1. A point release without a CVE fix
2. A point release with a CVE fix (or more than one)

In the first case, how does a tagged version provide information about
importance? A release would have been tagged whether the latest commit (or
N commits) had been merged or not. In the second case, downstream
redistributors (or at least Debian) has already shipped a new version with
the fix. The importance of that CVE fix being included in a tag that was
created arbitrarily is then different than the importance it might have if
Debian didn't patch the existing versions. (Note, I'm not advocating you
change this practice.) I don't see how tags detail the importance of the
included commits any more than their existence on a stable branch.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/15/2015 05:19 PM, Ian Cordasco wrote:
 On 6/15/15, 09:24, Thomas Goirand z...@debian.org wrote:
 
 On 06/08/2015 01:55 PM, Kuvaja, Erno wrote:
 One thing I like about plan D
 is that it would give also indicator how much the stable branch has
 moved in
 each individual project.

 The only indication you will get is how many patches it has. I fail to
 see how this is valuable information. No info on how important they are
 or anything of this kind, which is a way more important.

 Thomas
 
 Are you implying that stable point releases as they exist today provide
 importance? How is that case?

I'm just replying to Kuvaja, Erno, who pretends that plan D will give
indicator how much the stable branch has moved in. My answer is: not
it wont. Maybe there was lots of cosmetic patches (like typos, man page,
or who knows...). The number of patch is an indication of ... nothing,
really!

The only way to have indications is release notes. Since it looks like
there's not enough resources to write them, we're giving up on them.
It's really a shame, because I'm sure it was valuable for our users.

Thomas


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Ian Cordasco]

On 6/15/15, 16:24, Thomas Goirand z...@debian.org wrote:

On 06/15/2015 05:19 PM, Ian Cordasco wrote:
 On 6/15/15, 09:24, Thomas Goirand z...@debian.org wrote:
 
 On 06/08/2015 01:55 PM, Kuvaja, Erno wrote:
 One thing I like about plan D
 is that it would give also indicator how much the stable branch has
 moved in
 each individual project.

 The only indication you will get is how many patches it has. I fail to
 see how this is valuable information. No info on how important they are
 or anything of this kind, which is a way more important.

 Thomas
 
 Are you implying that stable point releases as they exist today provide
 importance? How is that case?

I'm just replying to Kuvaja, Erno, who pretends that plan D will give
indicator how much the stable branch has moved in. My answer is: not
it wont. Maybe there was lots of cosmetic patches (like typos, man page,
or who knows...). The number of patch is an indication of ... nothing,
really!

The only way to have indications is release notes. Since it looks like
there's not enough resources to write them, we're giving up on them.
It's really a shame, because I'm sure it was valuable for our users.

Thomas

I would suggest you read the section of the wiki that describes what is an
appropriate commit to land on a stable branch:
https://wiki.openstack.org/wiki/StableBranch#Appropriate_Fixes

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Alan Pevec]

 The only issue is having a correct version number. Swift could well
 iterate after 20151, for example doing 20151.1 then 20151.2, etc.

Swift never had coordinated YEAR.N versions,
https://wiki.openstack.org/wiki/Swift/version_map
and they never released a version from their stable/* branches.
e.g. 2.2.1 and 2.2.2 are in Launchpad Series kilo
https://launchpad.net/swift/kilo
and were created from master branch, not stable/juno as one would expect.
I heard Swift team call them Juno version released during Kilo cycle
which confused me a lot but if you take into account that Swift master
is stable and always backward compatible, it does make sense.
In short, Swift is different enough so best not to force them into
coordinated point releases which is what we did.
And when you think of it more, Swift model should be the goal for all
OpenStack project: stable master, always backward compatbile, no
upgrade issues = no need for stable/* branches :)

 So we could do what you're asking (option B) for Kilo stable releases,
 but I think it's not really a viable option for stable/liberty onward.

 I fail to understand how version numbers are related to doing
 synchronized point releases. Sure, we have a problem with the version
 numbers, but we can still do a point release of all the server projects
 at once.

I don't see what value does the same tag across certain set of projects bring?
Taking it to the extreme, why don't you put the same version across
all packages in a distro?
Reality is that each openstack is independent and compatibility among
them is ensured by CI not by using the coordinated version tags.

 FYI, after a CVE, I have uploaded cinder with this version number:
 2015.1.0+2015.06.06.git24.493b6c7f12-1

And with modified[*] plan D it would be 2015.1.25

 IMO, it looks ugly, and not comprehensive to Debian users. If I really
 have to go this way, I will, but I would have  not to.

You don't, if you'd support planD :)

 Though there's now zero guidance at what should be the speed of
 releasing server packages to our users.

Why there should be any guidance from upstream? Let your users tell
you and with planD you can update whenever requested fix is merged on
stable!

 So I totally get that we should still have reference points to be able
 to tell this is fixed in openstack/nova stable/liberty starting with
 12.1.134post34 (or whatever we settle with). I totally get that any of
 those should ship with relevant release notes. But that's about all I
 think we need ?

 That's not a little thing that you're pointing at: having reference
 points is very important. But yeah, it may be the only thing... Like
 we will only loose track of what is fixed in which version. :(

On the contrary, with planD you'd have reference points and lots of them :)
And release notes would be generated from commit messages - this
requires better commit messages on stable branches but that's always
good. Here's how it would look like for current Cinder stable/kilo:
CHANGES
===

2015.1.25
-

* Disallow backing files when uploading volumes to image

2015.1.24
-

* Dispose DB connections between backend proc starts

2015.1.23
-

* Allow provisioning to reach max oversubscription
...

Cheers,
Alan


[*] modified plan D: automatically sign and push git tag after each
commit on stable branch. We get both, tags and tarballs in one go.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/09/2015 04:54 PM, Jeremy Stanley wrote:
 On 2015-06-09 10:55:55 +0200 (+0200), Thomas Goirand wrote:
 [...]
 That's far from being in place. Also, while we are removing point
 releases, and support for Icehouse, we still don't have a common private
 Gerrit for security, which we've been told about 2 years ago.

 Removing collaboration tools before new ones are in place is
 definitively not the way to go.
 [...]
 
 In the stable branch management session at the Liberty summit, the
 Infrastructure Team position was confirmed that we can't maintain
 non-supported branches in a second Gerrit any more than we can in
 the primary one. The idea that there might be some branch of an
 OpenStack project which we just give up on testing but keep
 available for new changes is distasteful from a quality perspective.
 If there is sufficient interest from distro representatives in
 collaborating upstream on backports to, for example, stable/icehouse
 then that interest needs to include the resources necessary to
 maintain sufficient testing upstream (as defined by the upstream
 community).

Jeremy,

I have made a round table in Paris, and I have names of interested
people who would work on this. But without a collaboration tool (and at
least a common Git repository to work on), it is going to be harder to
work on together.

 So, I'm sorry, but don't look to an OpenStack-maintained non-public
 code review system as a workaround for continuing to collaborate in
 private on branches unsupported upstream in public. Most of the time
 it ends up being non-distro upstream developers (quality assurance,
 release management, infrastructure, vulnerability management,
 individual projects) who bear the majority of the responsibility for
 keeping testing working on those branches and we're clearly at our
 breaking point now trying to maintain three besides our master
 branch.

I understand, and I haven't asked the VMT or anyone upstream to work on
this.

 If the interested distributions come back with teams of people to
 dedicate to this work, which means getting deeply embedded in the
 groups who currently maintain the existing testing and release
 management for stable branches and are currently understaffed for
 that effort, then we can certainly revisit the number of branches
 we're able to maintain in both the public and (future) embargoed
 security review systems.

As we discussed, the goal isn't to maintain upstream CI, but to have a
place to share our work between distributions. We would do the testing
downstream, not using OpenStack infra.

Please get this security gerrit up, so that we can work on a patch
during the embargo period.

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Dave Walker]

On 10 June 2015 at 09:53, Thomas Goirand z...@debian.org wrote:
 On 06/05/2015 02:46 PM, Thierry Carrez wrote:
 So.. summarizing the various options again:

 Plan A
 Just drop stable point releases.
 (-) No more release notes
 (-) Lack of reference points to compare installations

 Plan B
 Push date-based tags across supported projects from time to time.
 (-) Encourages to continue using same version across the board
 (-) Almost as much work as making proper releases

 Plan C
 Let projects randomly tag point releases whenever
 (-) Still a bit costly in terms of herding cats

 Plan D
 Drop stable point releases, publish per-commit tarballs
 (-) Requires some infra changes, takes some storage space

 Plans B, C and D also require some release note / changelog generation
 from data maintained *within* the repository.

 Personally I think the objections raised against plan A are valid. I
 like plan D, since it's more like releasing every commit than not
 releasing anymore. I think it's the most honest trade-off. I could go
 with plan C, but I think it's added work for no additional value to the
 user.

 What would be your preferred option ?

 I see no point of doing D. I already don't use tarballs, and those who
 do could as well switch to generating them (how hard is it to run
 python setup.py sdist or git archive?).

 What counts is having a schedule date, where all distros are releasing a
 point release, so we have a common reference point. If that is a fully
 automated process, then great, less work for everyone, and it wont
 change anything from what we had in the past (we can even collectively
 decide for point release dates...).

 Cheers,

 Thomas Goirand (zigo)

This is really one of the things I think we want to get away from...
If *every* stable commit is treated with the seriousness of it
creating a release, lets make every commit a release.

This means that Debian may be using a (micro)patch release newer or
older than a different distro, but the key is that it empowers the
vendors and/or users to select a release cadence that best fits them,
rather than being tied to an arbitrary upstream community wide date.

Yes, this might mean that your cadence might be more or less regular
than an alternative vendor / distribution, but the key is that it
empowers the vendor to meet the needs of their users/customers.

For example, you could select a cadence of rebasing to a release every
6 months - where as another consumer could choose to do one every 6
weeks.  The difference is how much of a jump, at which intervals..
Alternatively, a vendor might choose just to go with stock release +
their own select cherry picked patches from stable/*, which is also a
model that works.

The issue around producing tarballs, is really about having forwards
and backwards verification by means of sha/md5 sums, which is hard to
do when generating your own orig tarball.  Debian, Ubuntu and I
believe Arch have made varying use of 'pristine-tar' - which was an
effort to re-producible tarballs using xdelta to make the sum match.
However, Maintainers seem to be moving away from this now.

When I perform source NEW reviews for Ubuntu Archive, I always check
that getting the source orig tarball can be done with either
get-orig-source (inspecting the generation method) or uscan and then
diff the tarballs with the one included on the upload and the one
generated.  Timestamps (or even shasums) haven't been an important
issue for me, but the actual content and verifiable source is what has
mattered more.

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/05/2015 02:46 PM, Thierry Carrez wrote:
 So.. summarizing the various options again:
 
 Plan A
 Just drop stable point releases.
 (-) No more release notes
 (-) Lack of reference points to compare installations
 
 Plan B
 Push date-based tags across supported projects from time to time.
 (-) Encourages to continue using same version across the board
 (-) Almost as much work as making proper releases
 
 Plan C
 Let projects randomly tag point releases whenever
 (-) Still a bit costly in terms of herding cats
 
 Plan D
 Drop stable point releases, publish per-commit tarballs
 (-) Requires some infra changes, takes some storage space
 
 Plans B, C and D also require some release note / changelog generation
 from data maintained *within* the repository.
 
 Personally I think the objections raised against plan A are valid. I
 like plan D, since it's more like releasing every commit than not
 releasing anymore. I think it's the most honest trade-off. I could go
 with plan C, but I think it's added work for no additional value to the
 user.
 
 What would be your preferred option ?

I see no point of doing D. I already don't use tarballs, and those who
do could as well switch to generating them (how hard is it to run
python setup.py sdist or git archive?).

What counts is having a schedule date, where all distros are releasing a
point release, so we have a common reference point. If that is a fully
automated process, then great, less work for everyone, and it wont
change anything from what we had in the past (we can even collectively
decide for point release dates...).

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thierry Carrez]

Dave Walker wrote:
 On 10 June 2015 at 09:53, Thomas Goirand z...@debian.org wrote:
 What would be your preferred option ?

 I see no point of doing D. I already don't use tarballs, and those who
 do could as well switch to generating them (how hard is it to run
 python setup.py sdist or git archive?).

 What counts is having a schedule date, where all distros are releasing a
 point release, so we have a common reference point. If that is a fully
 automated process, then great, less work for everyone, and it wont
 change anything from what we had in the past (we can even collectively
 decide for point release dates...).

That would be option B.

The main issue with B is that it doesn't work well once server component
versions start to diverge, which will be the case starting with Liberty.
You can't tag everything with the same version number anymore. We
already couldn't (with Swift using separate versioning), but we worked
around that by ignoring Swift from stable releases. As more projects opt
into that, that will no longer be a possible workaround.

So we could do what you're asking (option B) for Kilo stable releases,
but I think it's not really a viable option for stable/liberty onward.

 This is really one of the things I think we want to get away from...
 If *every* stable commit is treated with the seriousness of it
 creating a release, lets make every commit a release.
 
 This means that Debian may be using a (micro)patch release newer or
 older than a different distro, but the key is that it empowers the
 vendors and/or users to select a release cadence that best fits them,
 rather than being tied to an arbitrary upstream community wide date.

+1

It also removes the stupid encouragement to use all components from the
same date. With everything tagged at the same date, you kinda send the
message that those various things should be used together. With
everything tagged separately, you send te message that you can mix and
match components from stable/* as you see fit. I mean, it's totally
valid to use stable branch components from various points in time
together, since they are all supposed to work.

So I totally get that we should still have reference points to be able
to tell this is fixed in openstack/nova stable/liberty starting with
12.1.134post34 (or whatever we settle with). I totally get that any of
those should ship with relevant release notes. But that's about all I
think we need ?

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Dave Walker]

On 10 June 2015 at 17:06, Fox, Kevin M kevin@pnnl.gov wrote:
 So... as an op, without release notes, how am I supposed to figure out the 
 proper upgrade procedure's when you often have to lockstep, in the right 
 order, nova+neutron upgrades (or other project combinations)?

 Thanks,
 Kevin

Hi Kevin,

I suspect there is some confusion here between stable point releases
and major releases.  The major releases are still continuing as
expected with rich release notes.  This is talking about stable patch
releases such as:

https://wiki.openstack.org/wiki/ReleaseNotes/2014.2.3

As you can see there is only notices in Known Issues and Limitations
section of low impact.  I do not believe we've ever required ordering
in the updates of these, as they are supposed to be pretty
conservative changes that shouldn't enforce limitations like that.

HTH

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Fox, Kevin M]

So... as an op, without release notes, how am I supposed to figure out the 
proper upgrade procedure's when you often have to lockstep, in the right order, 
nova+neutron upgrades (or other project combinations)?

Thanks,
Kevin

From: Thomas Goirand [z...@debian.org]
Sent: Wednesday, June 10, 2015 1:53 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [all] [stable] No longer doing stable point
releases

On 06/05/2015 02:46 PM, Thierry Carrez wrote:
 So.. summarizing the various options again:

 Plan A
 Just drop stable point releases.
 (-) No more release notes
 (-) Lack of reference points to compare installations

 Plan B
 Push date-based tags across supported projects from time to time.
 (-) Encourages to continue using same version across the board
 (-) Almost as much work as making proper releases

 Plan C
 Let projects randomly tag point releases whenever
 (-) Still a bit costly in terms of herding cats

 Plan D
 Drop stable point releases, publish per-commit tarballs
 (-) Requires some infra changes, takes some storage space

 Plans B, C and D also require some release note / changelog generation
 from data maintained *within* the repository.

 Personally I think the objections raised against plan A are valid. I
 like plan D, since it's more like releasing every commit than not
 releasing anymore. I think it's the most honest trade-off. I could go
 with plan C, but I think it's added work for no additional value to the
 user.

 What would be your preferred option ?

I see no point of doing D. I already don't use tarballs, and those who
do could as well switch to generating them (how hard is it to run
python setup.py sdist or git archive?).

What counts is having a schedule date, where all distros are releasing a
point release, so we have a common reference point. If that is a fully
automated process, then great, less work for everyone, and it wont
change anything from what we had in the past (we can even collectively
decide for point release dates...).

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Fox, Kevin M]

Hi Dave,

Ah, I did misunderstand. Thanks for clarifying.

Kevin

From: Dave Walker [em...@daviey.com]
Sent: Wednesday, June 10, 2015 9:32 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [all] [stable] No longer doing stable point
releases

On 10 June 2015 at 17:06, Fox, Kevin M kevin@pnnl.gov wrote:
 So... as an op, without release notes, how am I supposed to figure out the 
 proper upgrade procedure's when you often have to lockstep, in the right 
 order, nova+neutron upgrades (or other project combinations)?

 Thanks,
 Kevin

Hi Kevin,

I suspect there is some confusion here between stable point releases
and major releases.  The major releases are still continuing as
expected with rich release notes.  This is talking about stable patch
releases such as:

https://wiki.openstack.org/wiki/ReleaseNotes/2014.2.3

As you can see there is only notices in Known Issues and Limitations
section of low impact.  I do not believe we've ever required ordering
in the updates of these, as they are supposed to be pretty
conservative changes that shouldn't enforce limitations like that.

HTH

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/08/2015 12:46 AM, Alan Pevec wrote:
 How do you check if project X in version n works with project Y in
 version m, using this non-scheduled point release free for all model?
 
 That was an illusion of point releases, as Thierry said there wasn't
 significantly more testing and I don't remember any testing reports
 during stable freeze periods.

There is: the point releases end up in all distributions, and we all do
our own tests.

 All we had was upstream CI testing, but that's what we get for every commit.

The stable release gate is famously always broken, and now that we're
proposing to stop doing the point releases, it's going to be even worse.

 Why every 2 weeks? Why every month?
 
 Sure, no problem, every distro can update whenever it works for them,
 what is important for me is that we have a common reference points.
 With plan D that would be automatically generated maj.min.N where N is
 the number of commits since maj.min tag which doesn't depend on
 anyone's manually pushing git tag.

As long as we have a common reference point, manually or not, I'm satisfied.

 Not synchronizing brings a bunch of problems. The only problem raised by
 synchronous point releases is we don't have enough resources, which I
 fail to understand, given how cheap it is to tag a release. If the
 release managers don't have enough time to do so, it is my view that
 it's ok to give this power to individual projects. But that's orthogonal
 to having synchronous point releases.
 
 Tag is indeed cheap, this is more about reflecting reality and not
 keeping this synchronized illusion alive.

If distros are releasing packages based on the same tag, I fail to see
where we have an illusion.

 BTW point release process is more[1] than just pushing signed git tag,
 the most time consuming work is cats herding (i.e. pushing for reviews
 before the freeze) and Launchpad pruning.
 The former was hopeless and will be even more with big-tent and the
 latter we should avoid by automatically generated changelog.

So, if I get you right, you are saying that our QA work isn't working as
well as we want, and therefore, we should accept that fact, and get rid
of that QA work. While it's important to realize what our problems are,
I don't think this is going to the right direction still... :(

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/07/2015 06:13 PM, Ian Cordasco wrote:
 If you consider *every* commit to be a release, then your life becomes
 easier.

 What does this mean? Am I supposed to upload a patched release to Debian
 every day? I suppose I didn't understand you correctly here.
 
 This discussion is about stable branches. Maybe early in a stable branch's
 life there are lots of commits, but as it grows older, the number of
 commits made to a stable branch certainly isn't 1 per day.

If you consider all the projects within the big-tent, that's a lot of
commits still.

 If we were to do this in downstream distros, we wouldn't have an
 upstream number matching each commit. This would be a problem because we
 would loose track of what version we're at between distros.
 
 It seems from other discussions that there is little coordination between
 distros in the first place, and that is only beginning to improve now, but
 only between close relatives (Ubuntu  Debian, RHEL  CentOS  Fedora,
 etc.). If that's the case, the work to improve coordination, including
 sharing repositories, etc. would seem to alleviate this concern regardless
 of coordinated release tagging.

That's far from being in place. Also, while we are removing point
releases, and support for Icehouse, we still don't have a common private
Gerrit for security, which we've been told about 2 years ago.

Removing collaboration tools before new ones are in place is
definitively not the way to go.

 That said, I know you've said in the past that you do most of your
 packaging in your free-time because it's not part of your job
 responsibilities.

I'm a full time Mirantis employee, and it's my full-time job to do
packaging of OpenStack in Debian (that, plus helping to do MOS).

 I know the same is true for a bunch of OpenStack's
 packagers (including the Gentoo packager).

I believe Gentoo is the only case.

 As someone
 who spends the majority of their free time supporting other software
 beyond OpenStack, I understand how insulting it is to be told you have to
 do more work in the time that you're volunteering.

It being a paid-for job IMO doesn't change the fact we should work
efficiently and the correct way. :)

 I've only been around for a little less than a
 year though, so I have no memory of a backport in one service breaking
 another.

It happened that requirements changed from one version to the next, and
2 projects needed to be updated at once. I don't see this happening
smoothly if we don't have coordinated point releases. And yes, I know,
requirements are *supposed* to be frozen, but between the theory and
what really happens, there's a world...

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/01/2015 01:20 PM, Dimitri John Ledkov wrote:
 On 29 May 2015 at 14:41, Thierry Carrez thie...@openstack.org wrote:
 Hi everyone,

 TL;DR:
 - We propose to stop tagging coordinated point releases (like 2015.1.1)
 - We continue maintaining stable branches as a trusted source of stable
 updates for all projects though

 
 Ideally I would still want to see tarballs published, even if they are
 generated with a $ git describe --tags name in it.
 
 E.g. keystone-2015.1.0-3-g1373bfa.tar.gz

Please, no dashes in version numbers. While Debian *can* cope with them,
it's recommended to avoid it (as the dash is the separator between the
upstream version and the Debian release number).

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/08/2015 12:25 AM, Alan Pevec wrote:
 and *Plan D* would be to start doing automatic per-project
 micro-versions on each commit: e.g. 2015.1.N where N is increased on
 each commit.

 How do you gpg sign these tags? I hope the solution isn't to store a key
 in infra without a passphrase.
 
 Plan D doesn't include git tags, 2015.1.N would be generated by PBR
 automatically.

Then please don't do Plan D. I don't use tarballs, and I don't have
the intention to use them. It's not an efficient way for me to work.

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/07/2015 05:57 PM, Ian Cordasco wrote:
 On 6/7/15, 03:41, Thomas Goirand z...@debian.org wrote:
 
 On 05/29/2015 09:23 PM, Ian Cordasco wrote:
 Could you explain this as well? Do you mean fragmentation between what
 distros are offering? In other words, Ubuntu is packaging Kilo @ SHA1
 and
 RHEL is at SHA2. I'm not entirely certain that's a bad thing. That seems
 to give the packagers more freedom.

 What happens when there's a security patch? Will upstream publish
 patches for each and every distro? I don't believe so.
 
 Does upstream do that now? I don't think so.

The point it: they don't need to do it, because all distro are using the
same reference point (ie: the point releases).

 On 05/29/2015 09:23 PM, Ian Cordasco wrote:
 The point of the embargo is to give time for testing patches and prepare
 a new patched version. Sometimes, we discover problems with the provided
 patch during the embargo period. Yes, we use the embargo to sometimes
 adapt the patch to the version we have in our distributions, but we
 would prefer if that work wasn't needed.
 
 But there aren't point releases for every CVE fix. There are point
 releases that are coordinated at the moment. So if you're waiting for
 those point releases to publish a new version of that package in your
 package repositories, that's news to me. I've seen packagers take patches
 and apply them and merely change the build metadata. Is this only done for
 severe CVEs at the moment?

I try to do a security fix on every OSSA the way you describe above. I
suppose other distros are doing the same (but I didn't take the time to
check).

 If every commit were a release, then you could all synchronize on that, if
 you all packaged each commit or at least, generate a new package each time
 a CVE is publicly patched through gerrit.

Adding a bunch of unrelated commits for a CVE fix may be acceptable for
Debian Sid, but it wouldn't for the stable distribution.

But anyway, the discussion about point releases is only barely related
to CVE fixing. The point is that we would like to have a common
reference point between distribution, were we would all be able to say:
version X.Y.Z of this server has a bug, it broke the CI of N and M
distribution, we need a fix release. Without such a coordination, we
wouldn't have as much attention from upstream to produce clean point
releases.

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/08/2015 05:42 PM, Jeremy Stanley wrote:
 On 2015-06-07 10:55:29 +0200 (+0200), Thomas Goirand wrote:
 How do you gpg sign these tags? I hope the solution isn't to store
 a key in infra without a passphrase.
 
 How does, e.g., Debian sign its Release file for
 jessie-proposed-updates? I hope the solution isn't to store the
 ftp-master automatic archive signing key in infra without a
 passphrase. (This is a rhetorical question... I see from comments at
 https://wiki.debian.org/SecureApt that it is indeed the case.) In
 fact, I don't really mind this. It's at least an attestation that
 the machine where the signature was generated had access to the
 automatic signing key, which is in turn signed by and revocable by
 the systems administrators entrusted to protect that machine.

Fair enough. And I'll trust you will safeguard everything correctly.

:)

Thomas


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/08/2015 06:10 PM, Jeremy Stanley wrote:
 On 2015-06-08 16:53:21 +0100 (+0100), Dave Walker wrote:
 This breaks the desire of wanting to have a shared version scheme if
 consumers add their own local patches via git.  This works fine for
 consumers that do not use git for handling their local patches, but
 does not support the model of allowing the user to rebase using git.

 Perhaps tags ARE superior for this?
 
 Agreed, even the .devN (or earlier .postN) versioning has the same
 issue. If you rely on PBR autoversioning for non-tagged commits then
 your local commits _will_ conflict with upstream autoversions.

The problem is that we use the 2 first numbers as major version. If
instead of:

2015.1.N

we switch to something like:

20151.X.Y

then we restore sanity and a real semver system.

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Jeremy Stanley]

On 2015-06-09 10:55:55 +0200 (+0200), Thomas Goirand wrote:
[...]
 That's far from being in place. Also, while we are removing point
 releases, and support for Icehouse, we still don't have a common private
 Gerrit for security, which we've been told about 2 years ago.
 
 Removing collaboration tools before new ones are in place is
 definitively not the way to go.
[...]

In the stable branch management session at the Liberty summit, the
Infrastructure Team position was confirmed that we can't maintain
non-supported branches in a second Gerrit any more than we can in
the primary one. The idea that there might be some branch of an
OpenStack project which we just give up on testing but keep
available for new changes is distasteful from a quality perspective.
If there is sufficient interest from distro representatives in
collaborating upstream on backports to, for example, stable/icehouse
then that interest needs to include the resources necessary to
maintain sufficient testing upstream (as defined by the upstream
community).

So, I'm sorry, but don't look to an OpenStack-maintained non-public
code review system as a workaround for continuing to collaborate in
private on branches unsupported upstream in public. Most of the time
it ends up being non-distro upstream developers (quality assurance,
release management, infrastructure, vulnerability management,
individual projects) who bear the majority of the responsibility for
keeping testing working on those branches and we're clearly at our
breaking point now trying to maintain three besides our master
branch.

If the interested distributions come back with teams of people to
dedicate to this work, which means getting deeply embedded in the
groups who currently maintain the existing testing and release
management for stable branches and are currently understaffed for
that effort, then we can certainly revisit the number of branches
we're able to maintain in both the public and (future) embargoed
security review systems.
-- 
Jeremy Stanley


signature.asc
Description: Digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Robert Collins]

On 9 June 2015 at 03:48, Jeremy Stanley fu...@yuggoth.org wrote:
 On 2015-06-08 10:54:32 +1200 (+1200), Robert Collins wrote:
 On 8 June 2015 at 10:14, Alan Pevec ape...@gmail.com wrote:
  2015-06-06 19:08 GMT+02:00 Ian Cordasco ian.corda...@rackspace.com:
  Not exactly. PBR/OpenStack follow SemVer and 2015.1.0.38 isn't valid
  SemVer (http://semver.org/)
 
  Right, so semver compatible versioning on stable/kilo would be 2015.1.N
  but PBR doesn't support that.

 PBR supports it fine. Whats the issue?

 Having PBR automatically increment the N in 2015.1.N for each patch
 applied on the branch rather than needing it to be tagged.

I don't think pbr needs to do that to have the proposal work. We could
tag from cron and use the actual tag pipeline to do release related
stuff.

E.g. its layer conflation to have pbr start assuming that untagged
code isn't a release.

-Rob

-- 
Robert Collins rbtcoll...@hp.com
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Kuvaja, Erno]

 -Original Message-
 From: Thierry Carrez [mailto:thie...@openstack.org]
 Sent: Friday, June 05, 2015 1:46 PM
 To: openstack-dev@lists.openstack.org
 Subject: Re: [openstack-dev] [all] [stable] No longer doing stable point
 releases
 
 So.. summarizing the various options again:
 
 Plan A
 Just drop stable point releases.
 (-) No more release notes
 (-) Lack of reference points to compare installations
 
 Plan B
 Push date-based tags across supported projects from time to time.
 (-) Encourages to continue using same version across the board
 (-) Almost as much work as making proper releases
 
 Plan C
 Let projects randomly tag point releases whenever
 (-) Still a bit costly in terms of herding cats
 
 Plan D
 Drop stable point releases, publish per-commit tarballs
 (-) Requires some infra changes, takes some storage space
 
 Plans B, C and D also require some release note / changelog generation from
 data maintained *within* the repository.
 
 Personally I think the objections raised against plan A are valid. I like 
 plan D,
 since it's more like releasing every commit than not releasing anymore. I
 think it's the most honest trade-off. I could go with plan C, but I think it's
 added work for no additional value to the user.
 
 What would be your preferred option ?
 
 --
 Thierry Carrez (ttx)
 

I don't think plans B  C are any benefit for  anyone based on the statements 
discussed earlier so won't be repeating those. One thing I like about plan D is 
that it would give also indicator how much the stable branch has moved in each 
individual project. 

Yes this can be seen from git, but also having just running number on the 
stable release for each commit would be really quick glimpse opposed to SHAs or 
counting  the changes from git  log.

- Erno

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Jeremy Stanley]

On 2015-06-07 10:55:29 +0200 (+0200), Thomas Goirand wrote:
 How do you gpg sign these tags? I hope the solution isn't to store
 a key in infra without a passphrase.

How does, e.g., Debian sign its Release file for
jessie-proposed-updates? I hope the solution isn't to store the
ftp-master automatic archive signing key in infra without a
passphrase. (This is a rhetorical question... I see from comments at
https://wiki.debian.org/SecureApt that it is indeed the case.) In
fact, I don't really mind this. It's at least an attestation that
the machine where the signature was generated had access to the
automatic signing key, which is in turn signed by and revocable by
the systems administrators entrusted to protect that machine.
-- 
Jeremy Stanley


signature.asc
Description: Digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Jeremy Stanley]

On 2015-06-08 00:25:47 +0200 (+0200), Alan Pevec wrote:
 BTW there's an issue re. verification that
 https://tarballs.openstack.org/ is using cert for
 security.openstack.org but should be easily fixed by infra.

Uh, nope. Try again. You're redirected to security.openstack.org if
you accept that cert. It's true that tarballs.openstack.org is a
vhost on the same system, but it's not intended as a secure source
of release artifacts at the moment and so is only served via HTTP.
We may at some point add HTTPS, but more likely we'll add detached
signatures along with the tarballs/wheels we upload there through
some sort of automation rather than relying on HTTPS alone.
-- 
Jeremy Stanley


signature.asc
Description: Digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Jeremy Stanley]

On 2015-06-08 10:54:32 +1200 (+1200), Robert Collins wrote:
 On 8 June 2015 at 10:14, Alan Pevec ape...@gmail.com wrote:
  2015-06-06 19:08 GMT+02:00 Ian Cordasco ian.corda...@rackspace.com:
  Not exactly. PBR/OpenStack follow SemVer and 2015.1.0.38 isn't valid
  SemVer (http://semver.org/)
 
  Right, so semver compatible versioning on stable/kilo would be 2015.1.N
  but PBR doesn't support that.
 
 PBR supports it fine. Whats the issue?

Having PBR automatically increment the N in 2015.1.N for each patch
applied on the branch rather than needing it to be tagged.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Jeremy Stanley]

On 2015-06-08 16:53:21 +0100 (+0100), Dave Walker wrote:
 This breaks the desire of wanting to have a shared version scheme if
 consumers add their own local patches via git.  This works fine for
 consumers that do not use git for handling their local patches, but
 does not support the model of allowing the user to rebase using git.
 
 Perhaps tags ARE superior for this?

Agreed, even the .devN (or earlier .postN) versioning has the same
issue. If you rely on PBR autoversioning for non-tagged commits then
your local commits _will_ conflict with upstream autoversions.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Dave Walker]

On 8 June 2015 at 16:48, Jeremy Stanley fu...@yuggoth.org wrote:
 On 2015-06-08 10:54:32 +1200 (+1200), Robert Collins wrote:
 On 8 June 2015 at 10:14, Alan Pevec ape...@gmail.com wrote:
  2015-06-06 19:08 GMT+02:00 Ian Cordasco ian.corda...@rackspace.com:
  Not exactly. PBR/OpenStack follow SemVer and 2015.1.0.38 isn't valid
  SemVer (http://semver.org/)
 
  Right, so semver compatible versioning on stable/kilo would be 2015.1.N
  but PBR doesn't support that.

 PBR supports it fine. Whats the issue?

 Having PBR automatically increment the N in 2015.1.N for each patch
 applied on the branch rather than needing it to be tagged.

This breaks the desire of wanting to have a shared version scheme if
consumers add their own local patches via git.  This works fine for
consumers that do not use git for handling their local patches, but
does not support the model of allowing the user to rebase using git.

Perhaps tags ARE superior for this?

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Ian Cordasco]

On 6/7/15, 03:41, Thomas Goirand z...@debian.org wrote:

On 05/29/2015 09:23 PM, Ian Cordasco wrote:
 Could you explain this as well? Do you mean fragmentation between what
 distros are offering? In other words, Ubuntu is packaging Kilo @ SHA1
and
 RHEL is at SHA2. I'm not entirely certain that's a bad thing. That seems
 to give the packagers more freedom.

What happens when there's a security patch? Will upstream publish
patches for each and every distro? I don't believe so.

Does upstream do that now? I don't think so. They send out proposed
patches for each version that's affected (if I understand the process
documents correctly).

On 05/29/2015 09:23 PM, Ian Cordasco wrote:
 Perhaps I'm wrong, but when a CVE is released, don't the downstream
 packagers usually patch whatever version they have and push that out?

We would like to have a single patch to share between distros.
Fragmenting the work helps nobody.

You already have this if the documentation is correct.

 Isn't that the point of them being on an private list to receive
 embargoed notifications with the patches?

The point of the embargo is to give time for testing patches and prepare
a new patched version. Sometimes, we discover problems with the provided
patch during the embargo period. Yes, we use the embargo to sometimes
adapt the patch to the version we have in our distributions, but we
would prefer if that work wasn't needed.

But there aren't point releases for every CVE fix. There are point
releases that are coordinated at the moment. So if you're waiting for
those point releases to publish a new version of that package in your
package repositories, that's news to me. I've seen packagers take patches
and apply them and merely change the build metadata. Is this only done for
severe CVEs at the moment?

If every commit were a release, then you could all synchronize on that, if
you all packaged each commit or at least, generate a new package each time
a CVE is publicly patched through gerrit.

I think we're probably talking past each other. I have had one set of
experiences with downstream redistributors and CVEs in packages, and
you're describing an entirely different process. I can't think of a time I
haven't published a CVE where older versions of the package weren't
patched by downstream redistributors as well as the current one,
especially if they don't have the free time to publish the new package
version.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Ian Cordasco]

On 6/7/15, 03:47, Thomas Goirand z...@debian.org wrote:

On 05/29/2015 09:36 PM, Dave Walker wrote:
 Responses inline.
 
 On 29 May 2015 6:15 pm, Haïkel hgue...@fedoraproject.org
 mailto:hgue...@fedoraproject.org wrote:

 2015-05-29 15:41 GMT+02:00 Thierry Carrez thie...@openstack.org
 mailto:thie...@openstack.org:
  Hi everyone,
 
  TL;DR:
  - We propose to stop tagging coordinated point releases (like
2015.1.1)
  - We continue maintaining stable branches as a trusted source of
stable
  updates for all projects though
 

 Hi,

 I'm one of the main maintainer of the packages for Fedora/RHEL/CentOS.
 We try to stick as much as possible to upstream (almost zero
 downstream patches),
 and without intermediate releases, it will get difficult.
 
 If you consider *every* commit to be a release, then your life becomes
 easier.

What does this mean? Am I supposed to upload a patched release to Debian
every day? I suppose I didn't understand you correctly here.

This discussion is about stable branches. Maybe early in a stable branch's
life there are lots of commits, but as it grows older, the number of
commits made to a stable branch certainly isn't 1 per day.

If we were to do this in downstream distros, we wouldn't have an
upstream number matching each commit. This would be a problem because we
would loose track of what version we're at between distros.

It seems from other discussions that there is little coordination between
distros in the first place, and that is only beginning to improve now, but
only between close relatives (Ubuntu  Debian, RHEL  CentOS  Fedora,
etc.). If that's the case, the work to improve coordination, including
sharing repositories, etc. would seem to alleviate this concern regardless
of coordinated release tagging.

 I'm personally not fond of this as it will lead to more fragmentation.
 It may encourage
 bad behaviors like shipping downstream patches for bug fixes and CVE
 instead
 of collaborating upstream to differentiate themselves.
 For instance, if we had no point-based release, for issues tracking
 purposes, we would
 have to maintain our sets of tags somewhere.
 
 I disagree, each distro already does security patching and whilst I
 expect this to still happens, it actually *encourages* upstream first
 workflow as you can select a release on your own cadence that includes
 commits you need, for your users.

We are discussing point releases. This is only far related to security
fixes. Point releases are including bug fixes which, most of the time,
aren't security fixes which are by the way always backported to the
previous point release, and uploaded as hotfixes to distributions.

Not every bug fix gets backported during the life of a stable branch. At
least in Glance, the majority of backports are security fixes and the rest
are rather serious bugs that significantly break the service or prevent
the use of the service. Last I checked, bug fixes needed to meet a certain
set of criteria before a backport was created. I'm fairly certain this
hasn't changed, but I admit that I haven't checked for changes in the last
several months.

That said, I know you've said in the past that you do most of your
packaging in your free-time because it's not part of your job
responsibilities. I know the same is true for a bunch of OpenStack's
packagers (including the Gentoo packager). This is what makes me torn on
whether or not we should totally drop stable point releases. As someone
who spends the majority of their free time supporting other software
beyond OpenStack, I understand how insulting it is to be told you have to
do more work in the time that you're volunteering.

This makes me more in favor of letting projects cut their own point
releases for stable branches. That said, I don't think it will alleviate
your concerns of guaranteeing that 2015.2.1 of X works with 2015.2.1 of Y,
but if these truly are only bug fixes, then I think the guarantee is
implicit in the branch. I've only been around for a little less than a
year though, so I have no memory of a backport in one service breaking
another.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Robert Collins]

On 8 June 2015 at 10:14, Alan Pevec ape...@gmail.com wrote:
 2015-06-06 19:08 GMT+02:00 Ian Cordasco ian.corda...@rackspace.com:
 Not exactly. PBR/OpenStack follow SemVer and 2015.1.0.38 isn't valid
 SemVer (http://semver.org/)

 Right, so semver compatible versioning on stable/kilo would be 2015.1.N
 but PBR doesn't support that.

PBR supports it fine. Whats the issue?

-Rob


-- 
Robert Collins rbtcoll...@hp.com
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Alan Pevec]

2015-06-06 19:08 GMT+02:00 Ian Cordasco ian.corda...@rackspace.com:
 Not exactly. PBR/OpenStack follow SemVer and 2015.1.0.38 isn't valid
 SemVer (http://semver.org/)

Right, so semver compatible versioning on stable/kilo would be 2015.1.N
but PBR doesn't support that.

 If we want to be pedantic about the version numbers generated by PBR (at the 
 gate, in tox, etc.), it should be next version number.devN

That's what PBR does now, even for projects without version in
setup.cfg, it assumes MAJOR.MINOR.PATCH++ from the last tag on the
branch.

Cheers,
Alan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Joshua Harlow]

Robert Collins wrote:

On 6 June 2015 at 08:53, Joshua Harlowharlo...@outlook.com  wrote:

Hopefully it's somewhat obvious to folks that altering the PBR version
schema (yet again), breaks *all the people* from using it in a reliable
manner, and if the goal of PBR is to bring reasonableness, well changing it
in a way that breaks things seems like the-anti-pattern of PBR.

I know from building with anvil and building RPMs that we only used tags in
https://github.com/stackforge/anvil/tree/master/conf/origins because the PBR
version number would change format/style/other (and also partially because
nothing existed that converted to a *sane* rpm version string, although this
supposedly exists now via a new PBR function).


Well is has the thing you collaborated on. Whether thats sane, an RPM
person needs to say :)


Sanity is in the eye of the beholder so +1 from me, hopefully not to 
insane, ha :-P




-Rob




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Alan Pevec]

 and *Plan D* would be to start doing automatic per-project
 micro-versions on each commit: e.g. 2015.1.N where N is increased on
 each commit.

 How do you gpg sign these tags? I hope the solution isn't to store a key
 in infra without a passphrase.

Plan D doesn't include git tags, 2015.1.N would be generated by PBR
automatically.

 FYI, I don't use tarballs (just git), and generate my own orig.tar.xz
 out of a signed git tag, so I am not affected by this.

We could generate it too but upstream SourceURL is preferred[1] so it
can be easily verified.
BTW there's an issue re. verification that
https://tarballs.openstack.org/ is using cert for
security.openstack.org but should be easily fixed by infra.


Cheers,
Alan

[1] https://fedoraproject.org/wiki/Packaging:SourceURL

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Robert Collins]

On 8 June 2015 at 10:45, Robert Collins robe...@robertcollins.net wrote:


 You'll also note that according to PEP 440, (as Jeremy pointed out) .postN
 is meant for non-code changes. If we want to be pedantic about the version
 numbers generated by PBR (at the gate, in tox, etc.), it should be next
 version number.devN but that's shaving an entirely different yak, and one
 that I don't think is especially concerning or a serious problem.

 Its a very concerning problem for continuous deployers, and thats why
 pbr 0.11 is now the minimum in global requirements, because we solved
 it. It can cause gate issues with dependencies not upgrading for
 instance (or running the latest beta rather than the local tree meant
 to be tested) [even with latest pip].

I missed the reference - sorry -
http://docs.openstack.org/developer/pbr/#version - documents the
algorithm pbr uses to generate versions. The reference to git sha's is
incorrect, I'll push up a doc fix for that shortly (we store that in a
pbr metadata file in the sdist and can report on that via the pbr CLI,
but its not in the version string anymore.

-Rob

-- 
Robert Collins rbtcoll...@hp.com
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 05/29/2015 07:14 PM, Haïkel wrote:
 2015-05-29 15:41 GMT+02:00 Thierry Carrez thie...@openstack.org:
 Hi everyone,

 TL;DR:
 - We propose to stop tagging coordinated point releases (like 2015.1.1)
 - We continue maintaining stable branches as a trusted source of stable
 updates for all projects though

 
 Hi,
 
 I'm one of the main maintainer of the packages for Fedora/RHEL/CentOS.
 We try to stick as much as possible to upstream (almost zero
 downstream patches),
 and without intermediate releases, it will get difficult.
 
 I'm personally not fond of this as it will lead to more fragmentation.
 It may encourage
 bad behaviors like shipping downstream patches for bug fixes and CVE instead
 of collaborating upstream to differentiate themselves.
 For instance, if we had no point-based release, for issues tracking
 purposes, we would
 have to maintain our sets of tags somewhere.
 
 There's also the release notes issue that has already been mentioned.
 Still continuous release notes won't solve the problem, as you wouldn't
 be able to map these to the actual packages. Will we require operators
 to find from which git commit, the packages were built and then try to figure
 out which fixes are and are not included?

 (1) Projects do not all follow the same versioning, so some projects
 (like Swift) were not part of the stable point releases. More and more
 projects are considering issuing intermediary releases (like Swift
 does), like Ironic. That would result in a variety of version numbers,
 and ultimately less and less projects being able to have a common
 2015.1.1-like version.

 
 And that's actually a pain point to track for these releases in which
 OpenStack branch belong. And this is probably something that needs to
 be resolved.
 
 (2) Producing those costs a non-trivial amount of effort on a very small
 team of volunteers, especially with projects caring about stable
 branches in various amounts. We were constantly missing the
 pre-announced dates on those ones. Looks like that effort could be
 better spent improving the stable branches themselves and keeping them
 working.
 
 Agreed, but why not switching to a time-based release?
 Regularly, we tag/generate/upload tarballs, this could even be automated.
 As far as I'm concerned, I would be more happy to have more frequent releases.
 
 Now we realize that the cross-section of our community which was present
 in that session might not fully represent the consumers of those
 artifacts, which is why we expand the discussion on this mailing-list
 (and soon on the operators ML).
 
 Thanks, I was not able to join this discussion, and that was the kind
 of proposal that I was fearing to see happen.
 
 If you were a consumer of those and will miss them, please explain why.
 In particular, please let us know how consuming that version (which was
 only made available every n months) is significantly better than picking
 your preferred time and get all the current stable branch HEADs at that
 time.
 
 We provide both type of builds
 * git continuous builds = for testing/CI and early feedback on potential 
 issues
 * point-release based builds = for GA, and production
 
 Anyway, I won't force anyone to do something they don't want to do but I'm
 willing to step in to keep point releases in one form or another.
 
 Regards,
 H.

I fully agree with what Haikel wrote (and therefore wont repeat).

The same way, I didn't agree with the change of version scheme either.

Why not using something like: 201504.0.0, and allow projects to do
intermediate releases like 201506, 201507, etc? Or 20151 / 20152, and
then allow the middle number to increase as it pleases projects?
Basically, anything better than just random numbers?

Also, besides Ironic, which are the other projects that are willing to
release more often than 6 months?

Cheers,

Thomas


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 05/29/2015 09:23 PM, Ian Cordasco wrote:
 Could you explain this as well? Do you mean fragmentation between what
 distros are offering? In other words, Ubuntu is packaging Kilo @ SHA1 and
 RHEL is at SHA2. I'm not entirely certain that's a bad thing. That seems
 to give the packagers more freedom.

What happens when there's a security patch? Will upstream publish
patches for each and every distro? I don't believe so.

On 05/29/2015 09:23 PM, Ian Cordasco wrote:
 Perhaps I'm wrong, but when a CVE is released, don't the downstream
 packagers usually patch whatever version they have and push that out?

We would like to have a single patch to share between distros.
Fragmenting the work helps nobody.

 Isn't that the point of them being on an private list to receive
 embargoed notifications with the patches?

The point of the embargo is to give time for testing patches and prepare
a new patched version. Sometimes, we discover problems with the provided
patch during the embargo period. Yes, we use the embargo to sometimes
adapt the patch to the version we have in our distributions, but we
would prefer if that work wasn't needed.

Thomas


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 05/29/2015 09:36 PM, Dave Walker wrote:
 Responses inline.
 
 On 29 May 2015 6:15 pm, Haïkel hgue...@fedoraproject.org
 mailto:hgue...@fedoraproject.org wrote:

 2015-05-29 15:41 GMT+02:00 Thierry Carrez thie...@openstack.org
 mailto:thie...@openstack.org:
  Hi everyone,
 
  TL;DR:
  - We propose to stop tagging coordinated point releases (like 2015.1.1)
  - We continue maintaining stable branches as a trusted source of stable
  updates for all projects though
 

 Hi,

 I'm one of the main maintainer of the packages for Fedora/RHEL/CentOS.
 We try to stick as much as possible to upstream (almost zero
 downstream patches),
 and without intermediate releases, it will get difficult.
 
 If you consider *every* commit to be a release, then your life becomes
 easier.

What does this mean? Am I supposed to upload a patched release to Debian
every day? I suppose I didn't understand you correctly here.

If we were to do this in downstream distros, we wouldn't have an
upstream number matching each commit. This would be a problem because we
would loose track of what version we're at between distros.

 I'm personally not fond of this as it will lead to more fragmentation.
 It may encourage
 bad behaviors like shipping downstream patches for bug fixes and CVE
 instead
 of collaborating upstream to differentiate themselves.
 For instance, if we had no point-based release, for issues tracking
 purposes, we would
 have to maintain our sets of tags somewhere.
 
 I disagree, each distro already does security patching and whilst I
 expect this to still happens, it actually *encourages* upstream first
 workflow as you can select a release on your own cadence that includes
 commits you need, for your users.

We are discussing point releases. This is only far related to security
fixes. Point releases are including bug fixes which, most of the time,
aren't security fixes which are by the way always backported to the
previous point release, and uploaded as hotfixes to distributions.

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/01/2015 10:40 AM, Ihar Hrachyshka wrote:
 No time synchronization between projects, no freeze dates, nothing, just
 SemVer compatible version bumps e.g. once per two weeks or month.
 Would it fix your problem?

How do you check if project X in version n works with project Y in
version m, using this non-scheduled point release free for all model?

Why every 2 weeks? Why every month?

Not synchronizing brings a bunch of problems. The only problem raised by
synchronous point releases is we don't have enough resources, which I
fail to understand, given how cheap it is to tag a release. If the
release managers don't have enough time to do so, it is my view that
it's ok to give this power to individual projects. But that's orthogonal
to having synchronous point releases.

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/01/2015 05:32 PM, Alan Pevec wrote:
 *Plan C* would be to just let projects tag stable point releases from
 time to time. That would solve all the original stated problems. And
 that would solve objections 2 and 3, which I think are the most valid ones.
 
 and *Plan D* would be to start doing automatic per-project
 micro-versions on each commit: e.g. 2015.1.N where N is increased on
 each commit.

How do you gpg sign these tags? I hope the solution isn't to store a key
in infra without a passphrase.

 There's just TBD item how to provide source tarballs for this.

FYI, I don't use tarballs (just git), and generate my own orig.tar.xz
out of a signed git tag, so I am not affected by this.

Cheers,

Thomas


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Robert Collins]

On 6 June 2015 at 08:53, Joshua Harlow harlo...@outlook.com wrote:
 Hopefully it's somewhat obvious to folks that altering the PBR version
 schema (yet again), breaks *all the people* from using it in a reliable
 manner, and if the goal of PBR is to bring reasonableness, well changing it
 in a way that breaks things seems like the-anti-pattern of PBR.

 I know from building with anvil and building RPMs that we only used tags in
 https://github.com/stackforge/anvil/tree/master/conf/origins because the PBR
 version number would change format/style/other (and also partially because
 nothing existed that converted to a *sane* rpm version string, although this
 supposedly exists now via a new PBR function).

Well is has the thing you collaborated on. Whether thats sane, an RPM
person needs to say :)

-Rob


-- 
Robert Collins rbtcoll...@hp.com
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Robert Collins]

On 7 June 2015 at 05:08, Ian Cordasco ian.corda...@rackspace.com wrote:


 On 6/6/15, 02:03, Alan Pevec ape...@gmail.com wrote:

2015-06-05 15:16 GMT+02:00 Jeremy Stanley fu...@yuggoth.org:
 On 2015-06-05 14:56:30 +0200 (+0200), Thierry Carrez wrote:
 [...]
 I was wondering if we could switch to post-versioning on stable
 branches, and basically generate:

 2015.1.0.post38
 [...]

 I think the recommendation from the PyPI maintainers is to not use
 .postN suffixes since they are intended to indicate non-code
 changes.

So that's PBR bug then and it should generate 2015.1.0.38 ?

 Not exactly. PBR/OpenStack follow SemVer

We follow a tweaked semver
http://docs.openstack.org/developer/pbr/semver.html but also honour
the date based server versions.

 and 2015.1.0.38 isn't valid
 SemVer (http://semver.org/) Specifically note that version generated above
 is also not valid SemVer but is valid under PEP 440 (which is what
 setuptools and pip use; https://www.python.org/dev/peps/pep-0440/). So we
 could ditch sticking strictly to SemVer for release versioning (meaning we
 could use 2015.1.0.38) or we need a new way to indicate all of this. Note
 that while PEP 440 and SemVer are not fully compatible
 (https://www.python.org/dev/peps/pep-0440/#semantic-versioning) both
 SemVer and PEP 440 allow for extra information. PEP 440 calls this Local
 Version Segments

We haven't implemented local version segments as such in pbr yet, and
I'm very open to us doing so - but only in concert with consolidating
the local-vendor stuff from Nova/Cinder etc into the same spec,
because we've only really got one shot to bring those things together
sanely, and I don't want to use the field up but not converge on those
things.

 (https://www.python.org/dev/peps/pep-0440/#local-version-segments) while
 SemVer calls it Build Metadata. That means that the version string would
 mean different things for different consumers with different mental models
 and would probably conflict with downstream redistributors who add their
 own versioning information to a version string.

Well - there's a defined mechanism in e.g. Nova for this already, we
just need to make sure we deliver the same use cases and tie it all
together.

 You'll also note that according to PEP 440, (as Jeremy pointed out) .postN
 is meant for non-code changes. If we want to be pedantic about the version
 numbers generated by PBR (at the gate, in tox, etc.), it should be next
 version number.devN but that's shaving an entirely different yak, and one
 that I don't think is especially concerning or a serious problem.

Its a very concerning problem for continuous deployers, and thats why
pbr 0.11 is now the minimum in global requirements, because we solved
it. It can cause gate issues with dependencies not upgrading for
instance (or running the latest beta rather than the local tree meant
to be tested) [even with latest pip].

pbr  0.11 generated .postN as a response to the OMG PEP-426 broke
everything wedge, and as a result 0.11 and above interpret .postN as
.devN for backwards compatibility reasons - so we can't use .postN
trivially, even if it was semantically appropriate - and as you note,
its not).

We can alter pbr to do more digits if that will solve a problem, but I
haven't seen a problem put forward so far that doesn't map reasonably
well to semver.

-Rob


-- 
Robert Collins rbtcoll...@hp.com
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Alan Pevec]

 How do you check if project X in version n works with project Y in
 version m, using this non-scheduled point release free for all model?

That was an illusion of point releases, as Thierry said there wasn't
significantly more testing and I don't remember any testing reports
during stable freeze periods.
All we had was upstream CI testing, but that's what we get for every commit.

 Why every 2 weeks? Why every month?

Sure, no problem, every distro can update whenever it works for them,
what is important for me is that we have a common reference points.
With plan D that would be automatically generated maj.min.N where N is
the number of commits since maj.min tag which doesn't depend on
anyone's manually pushing git tag.

 Not synchronizing brings a bunch of problems. The only problem raised by
 synchronous point releases is we don't have enough resources, which I
 fail to understand, given how cheap it is to tag a release. If the
 release managers don't have enough time to do so, it is my view that
 it's ok to give this power to individual projects. But that's orthogonal
 to having synchronous point releases.

Tag is indeed cheap, this is more about reflecting reality and not
keeping this synchronized illusion alive.
BTW point release process is more[1] than just pushing signed git tag,
the most time consuming work is cats herding (i.e. pushing for reviews
before the freeze) and Launchpad pruning.
The former was hopeless and will be even more with big-tent and the
latter we should avoid by automatically generated changelog.

Cheers,
Alan

[1] https://wiki.openstack.org/wiki/StableBranchRelease

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Jeremy Stanley]

On 2015-06-06 09:03:13 +0200 (+0200), Alan Pevec wrote:
 2015-06-05 15:16 GMT+02:00 Jeremy Stanley fu...@yuggoth.org:
[...]
  I think the recommendation from the PyPI maintainers is to not use
  .postN suffixes since they are intended to indicate non-code
  changes.
 
 So that's PBR bug then and it should generate 2015.1.0.38 ?

PBR no longer generates versions like 2015.1.0.post38 but rather
2015.1.1.dev38 instead, specifically because the pip/PyPI
maintainers recommended against the former pattern.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Ian Cordasco]

On 6/6/15, 02:03, Alan Pevec ape...@gmail.com wrote:

2015-06-05 15:16 GMT+02:00 Jeremy Stanley fu...@yuggoth.org:
 On 2015-06-05 14:56:30 +0200 (+0200), Thierry Carrez wrote:
 [...]
 I was wondering if we could switch to post-versioning on stable
 branches, and basically generate:

 2015.1.0.post38
 [...]

 I think the recommendation from the PyPI maintainers is to not use
 .postN suffixes since they are intended to indicate non-code
 changes.

So that's PBR bug then and it should generate 2015.1.0.38 ?

Not exactly. PBR/OpenStack follow SemVer and 2015.1.0.38 isn't valid
SemVer (http://semver.org/) Specifically note that version generated above
is also not valid SemVer but is valid under PEP 440 (which is what
setuptools and pip use; https://www.python.org/dev/peps/pep-0440/). So we
could ditch sticking strictly to SemVer for release versioning (meaning we
could use 2015.1.0.38) or we need a new way to indicate all of this. Note
that while PEP 440 and SemVer are not fully compatible
(https://www.python.org/dev/peps/pep-0440/#semantic-versioning) both
SemVer and PEP 440 allow for extra information. PEP 440 calls this Local
Version Segments 
(https://www.python.org/dev/peps/pep-0440/#local-version-segments) while
SemVer calls it Build Metadata. That means that the version string would
mean different things for different consumers with different mental models
and would probably conflict with downstream redistributors who add their
own versioning information to a version string.

You'll also note that according to PEP 440, (as Jeremy pointed out) .postN
is meant for non-code changes. If we want to be pedantic about the version
numbers generated by PBR (at the gate, in tox, etc.), it should be next
version number.devN but that's shaving an entirely different yak, and one
that I don't think is especially concerning or a serious problem.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thierry Carrez]

So.. summarizing the various options again:

Plan A
Just drop stable point releases.
(-) No more release notes
(-) Lack of reference points to compare installations

Plan B
Push date-based tags across supported projects from time to time.
(-) Encourages to continue using same version across the board
(-) Almost as much work as making proper releases

Plan C
Let projects randomly tag point releases whenever
(-) Still a bit costly in terms of herding cats

Plan D
Drop stable point releases, publish per-commit tarballs
(-) Requires some infra changes, takes some storage space

Plans B, C and D also require some release note / changelog generation
from data maintained *within* the repository.

Personally I think the objections raised against plan A are valid. I
like plan D, since it's more like releasing every commit than not
releasing anymore. I think it's the most honest trade-off. I could go
with plan C, but I think it's added work for no additional value to the
user.

What would be your preferred option ?

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thierry Carrez]

Jeremy Stanley wrote:
 On 2015-06-01 15:57:17 + (+), Jeremy Stanley wrote:
 [...]
 The biggest hurdle is that we'd need a separate upload job name
 for those since the current version of Zuul lacks a way to run a
 particular job for different branches in different pipelines (we'd
 want to do versioned uploads for all pre-release and release
 pipeline refs, but also for post pipeline refs only when the
 branch name is like stable/.*).
 
 Actually, scratch that. It's a bit more complicated since the post
 pipeline isn't actually branch-relevant. We'd need to tweak the
 tarball and wheel creation scripts to check the containing branch,
 like we do for some proposal jobs. Still, I think it wouldn't be too
 hard.

Exploring plan D, I was looking at the versions we currently generate on
stable branches and I think they would not convey the right message:

2015.1.1.dev38

- but there won't be a 2015.1.1 !
- but this is not under development !

I was wondering if we could switch to post-versioning on stable
branches, and basically generate:

2015.1.0.post38

... which would convey the right message.

I /think/ all it would take would be, as the first post-release commit
to the stable branch, to remove the preversion from setup.cfg (rather
than bump it to the next .1). I think pbr would switch to postversioning
in that case and generate postX versions from the last tag in the branch.

Not sure we would do that for stable/kilo, though, since we already
pushed 2015.1.1.devX versions in the wild.

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Daniel P. Berrange]

On Fri, Jun 05, 2015 at 02:46:07PM +0200, Thierry Carrez wrote:
 So.. summarizing the various options again:
 
 Plan A
 Just drop stable point releases.
 (-) No more release notes
 (-) Lack of reference points to compare installations
 
 Plan B
 Push date-based tags across supported projects from time to time.
 (-) Encourages to continue using same version across the board
 (-) Almost as much work as making proper releases
 
 Plan C
 Let projects randomly tag point releases whenever
 (-) Still a bit costly in terms of herding cats
 
 Plan D
 Drop stable point releases, publish per-commit tarballs
 (-) Requires some infra changes, takes some storage space
 
 Plans B, C and D also require some release note / changelog generation
 from data maintained *within* the repository.
 
 Personally I think the objections raised against plan A are valid. I
 like plan D, since it's more like releasing every commit than not
 releasing anymore. I think it's the most honest trade-off. I could go
 with plan C, but I think it's added work for no additional value to the
 user.

I don't see a whole lot of difference between plan A and D.
Publishing per-commit tarballs is merely saving the downstream
users the need to run a 'git archive' command, and providing
some auto-generated changelog that's already available from
'git log'.

If the downsteam consumer has their own extra patches ontop of the
stable branch, then it seems D is even less useful than A.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Jeremy Stanley]

On 2015-06-05 14:56:30 +0200 (+0200), Thierry Carrez wrote:
[...]
 I was wondering if we could switch to post-versioning on stable
 branches, and basically generate:
 
 2015.1.0.post38
[...]

I think the recommendation from the PyPI maintainers is to not use
.postN suffixes since they are intended to indicate non-code
changes.

We could look at a mode for PBR which just adds a fourth unqualified
integer component that increments per patchset from the last known
tag (though that's technically not SemVer any longer)? Or maybe we
could have a PBR mode that just increments the third integer
component of the version for every commit without needing a new tag?
Or we could just push a corresponding tag for each merged commit
(though that essentially brings us back to the tag arbitrarily
option)?
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thierry Carrez]

Daniel P. Berrange wrote:
 On Fri, Jun 05, 2015 at 02:46:07PM +0200, Thierry Carrez wrote:
 Plan A
 Just drop stable point releases.
 (-) No more release notes
 (-) Lack of reference points to compare installations

 Plan B
 Push date-based tags across supported projects from time to time.
 (-) Encourages to continue using same version across the board
 (-) Almost as much work as making proper releases

 Plan C
 Let projects randomly tag point releases whenever
 (-) Still a bit costly in terms of herding cats

 Plan D
 Drop stable point releases, publish per-commit tarballs
 (-) Requires some infra changes, takes some storage space

 Plans B, C and D also require some release note / changelog generation
 from data maintained *within* the repository.

 [...]
 
 I don't see a whole lot of difference between plan A and D.
 Publishing per-commit tarballs is merely saving the downstream
 users the need to run a 'git archive' command, and providing
 some auto-generated changelog that's already available from
 'git log'.

I guess the main difference is that we would still release something
(i.e. generate, version, publish and archive a tarball) in case of D,
which makes it easier to reuse a known point. It would also exhibit
version numbers more visibly, which makes referencing them slightly
easier (currently you have to look into the generated, transient
$PROJECT-stable-$SERIES.tar.gz tarball to see that version number).

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Matthew Thode]

On 06/05/2015 07:46 AM, Thierry Carrez wrote:
 So.. summarizing the various options again:
 
 Plan A
 Just drop stable point releases.
 (-) No more release notes
 (-) Lack of reference points to compare installations
 
 Plan B
 Push date-based tags across supported projects from time to time.
 (-) Encourages to continue using same version across the board
 (-) Almost as much work as making proper releases
 
 Plan C
 Let projects randomly tag point releases whenever
 (-) Still a bit costly in terms of herding cats
 
 Plan D
 Drop stable point releases, publish per-commit tarballs
 (-) Requires some infra changes, takes some storage space
 
 Plans B, C and D also require some release note / changelog generation
 from data maintained *within* the repository.
 
 Personally I think the objections raised against plan A are valid. I
 like plan D, since it's more like releasing every commit than not
 releasing anymore. I think it's the most honest trade-off. I could go
 with plan C, but I think it's added work for no additional value to the
 user.
 
 What would be your preferred option ?
 
Either A or D are what I will likely do.  Also, I'll likely build infra
on my end to generate ebuilds like B proposes.

-- 
-- Matthew Thode (prometheanfire)



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Alan Pevec]

 If the downsteam consumer has their own extra patches ontop of the
 stable branch, then it seems D is even less useful than A.

It is not - downstream (speaking for RDO) I would keep Version:
2015.1.N where N is stable patch# so that we have a common reference
point with other distros.
Our patches, if any, then come on top this, so it's clear which
patches are added by distro.

Cheers,
Alan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Tristan Cacqueray]

On 06/05/2015 05:46 AM, Thierry Carrez wrote:
 So.. summarizing the various options again:
 
 Plan A
 Just drop stable point releases.
 (-) No more release notes
 (-) Lack of reference points to compare installations
 
 Plan B
 Push date-based tags across supported projects from time to time.
 (-) Encourages to continue using same version across the board
 (-) Almost as much work as making proper releases
 
 Plan C
 Let projects randomly tag point releases whenever
 (-) Still a bit costly in terms of herding cats
 
 Plan D
 Drop stable point releases, publish per-commit tarballs
 (-) Requires some infra changes, takes some storage space
 
 Plans B, C and D also require some release note / changelog generation
 from data maintained *within* the repository.
 
 Personally I think the objections raised against plan A are valid. I
 like plan D, since it's more like releasing every commit than not
 releasing anymore. I think it's the most honest trade-off. I could go
 with plan C, but I think it's added work for no additional value to the
 user.
 
 What would be your preferred option ?
 

Apologize if I'm off-track here, but Plan A and D seems like a steep
change for users. Imo having stable release (at least between two
releases) is a valid use-case.

I guess Plan C is the preferred option, along with a stable-release
tag, projects that opt-in would have the responsibility to create stable
branches and maintain them.

Regards,
Tristan



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Clint Byrum]

Excerpts from Thierry Carrez's message of 2015-06-05 05:46:07 -0700:
 So.. summarizing the various options again:
 

Thanks for the excellent summary Thierry
 Plan C
 Let projects randomly tag point releases whenever
 (-) Still a bit costly in terms of herding cats
 

I feel like plan C is the smallest change one can make while still moving
toward lowering the project-wide maintenance costs of stable branches.

I don't know if I saw this highlighted enough during the discussion phase,
but the reason this might be a good idea is that stable releases, unlike
trunk, have a very different purpose and mode.

Unlike with the coordinated release of many projects all at once, we
are not trying to make sure the pieces still fit together. We've been
careful, in stable release policy, to be sure that we are minimizing
changes, so I think coordinating them is unnecessary overhead.

However, releasing all the commits every time does also mean that there's
no signal to downstream that now is a good time to pull this stable
branch and incorporate it into your systems. Even if each project ends up
releasing weekly right after their meeting in a semi-automatic fashion,
this means that users and packagers can attend the meeting, read the
minutes, and have a passive communication stream that doesn't require
them to read and grok every single commit whenever it might be approved.

Finally, I don't think this is still costly in terms of herding cats,
because we have very few cats actually grazing in the stable branch
pastures. I suggest that stable branch maintainers simply get used to
releasing on a weekly basis if there are new commits, and use that as
an opportunity to highlight the excellent work their community has done.
If they can't do that now, lets empower them to tag and release and send
release announcements.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Joshua Harlow]

Daniel P. Berrange wrote:

On Fri, Jun 05, 2015 at 02:46:07PM +0200, Thierry Carrez wrote:

So.. summarizing the various options again:

Plan A
Just drop stable point releases.
(-) No more release notes
(-) Lack of reference points to compare installations

Plan B
Push date-based tags across supported projects from time to time.
(-) Encourages to continue using same version across the board
(-) Almost as much work as making proper releases

Plan C
Let projects randomly tag point releases whenever
(-) Still a bit costly in terms of herding cats

Plan D
Drop stable point releases, publish per-commit tarballs
(-) Requires some infra changes, takes some storage space

Plans B, C and D also require some release note / changelog generation
from data maintained *within* the repository.

Personally I think the objections raised against plan A are valid. I
like plan D, since it's more like releasing every commit than not
releasing anymore. I think it's the most honest trade-off. I could go
with plan C, but I think it's added work for no additional value to the
user.


I don't see a whole lot of difference between plan A and D.
Publishing per-commit tarballs is merely saving the downstream
users the need to run a 'git archive' command, and providing
some auto-generated changelog that's already available from
'git log'.

If the downsteam consumer has their own extra patches ontop of the
stable branch, then it seems D is even less useful than A.


+1 to this, and I'm 99% sure every big/medium/all(?) cloud has extra 
patches on-top of stable branches; so publishing a tarball, meh, it will 
have to be regenerated anyway (with said patches) from the git commit it 
came from...




Regards,
Daniel


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Matthew Thode]

On 06/05/2015 09:34 AM, Alan Pevec wrote:
 If the downsteam consumer has their own extra patches ontop of the
 stable branch, then it seems D is even less useful than A.
 
 It is not - downstream (speaking for RDO) I would keep Version:
 2015.1.N where N is stable patch# so that we have a common reference
 point with other distros.
 Our patches, if any, then come on top this, so it's clear which
 patches are added by distro.
 
 Cheers,
 Alan
 
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
Yes, we use a revbump strategy here for patches and changing
2015.0.0-r10 or something.  I think this is the standard way most
distros signify changes from the base package.

-- 
Matthew Thode



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Maish Saidel-Keesing]

On 06/01/15 03:20, Steve Gordon wrote:

- Original Message -

From: Maish Saidel-Keesing mais...@maishsk.com
To: openstack-dev@lists.openstack.org

On 05/29/15 18:25, Matthew Thode wrote:

On 05/29/2015 10:18 AM, Ihar Hrachyshka wrote:

What about release notes? How can we now communicate some changes that
require operator consideration or action?

Ihar

On 05/29/2015 03:41 PM, Thierry Carrez wrote:

Hi everyone,
TL;DR: - We propose to stop tagging coordinated point releases
(like 2015.1.1) - We continue maintaining stable branches as a
trusted source of stable updates for all projects though
Long version:
At the stable branch session in Vancouver we discussed recent
evolutions in the stable team processes and how to further adapt
the work of the team in a big tent world.
One of the key questions there was whether we should continue
doing stable point releases. Those were basically tags with the
same version number (2015.1.1) that we would periodically push to
the stable branches for all projects.
Those create three problems.
(1) Projects do not all follow the same versioning, so some
projects (like Swift) were not part of the stable point releases.
More and more projects are considering issuing intermediary
releases (like Swift does), like Ironic. That would result in a
variety of version numbers, and ultimately less and less projects
being able to have a common 2015.1.1-like version.
(2) Producing those costs a non-trivial amount of effort on a very
small team of volunteers, especially with projects caring about
stable branches in various amounts. We were constantly missing the
pre-announced dates on those ones. Looks like that effort could be
better spent improving the stable branches themselves and keeping
them working.
(3) The resulting stable point releases are mostly useless.
Stable branches are supposed to be always usable, and the
released version did not undergo significantly more testing.
Issuing them actually discourages people from taking whatever point
in stable branches makes the most sense for them, testing and
deploying that.
The suggestion we made during that session (and which was approved
by the session participants) is therefore to just get rid of the
stable point release concept altogether for non-libraries. That
said:
- we'd still do individual point releases for libraries (for
critical bugs and security issues), so that you can still depend on
a specific version there
- we'd still very much maintain stable branches (and actually focus
our efforts on that work) to ensure they are a continuous source of
safe upgrades for users of a given series
Now we realize that the cross-section of our community which was
present in that session might not fully represent the consumers of
those artifacts, which is why we expand the discussion on this
mailing-list (and soon on the operators ML).
If you were a consumer of those and will miss them, please explain
why. In particular, please let us know how consuming that version
(which was only made available every n months) is significantly
better than picking your preferred time and get all the current
stable branch HEADs at that time.
Thanks in advance for your feedback,
[1] https://etherpad.openstack.org/p/YVR-relmgt-stable-branch

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


for release notes just do git log between commit hashes?

Do you really think that is what an Operator will do? I do not think is
a realistic expectation or something that will work.
--
Best Regards,
Maish Saidel-Keesing

While I agree most operators probably don't want to necessarily dig this out of git 
themselves and it would need to be collated/exposed in a nicer way it seems like it would 
actually be more accurate than the current release notes for all the 
non-security bug fixes in stable which basically amount to a list of launchpad bug 
queries per project. You then have to sift through each bug to work out if the 
description reflects what was actually done etc:

 https://wiki.openstack.org/wiki/ReleaseNotes/2014.2.3

-Steve
I agree - and if this could be automated in some way to make is 
presentable - that would be ideal

--
Best Regards,
Maish Saidel-Keesing

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Alan Pevec]

 You will get different checksums with tar and/or gzip, you can check the
 extracted files and they should be the same.

Yes, content is the same, it's just difference in timestamps of
folders and generated files (ChangeLog, egg-info etc).

 I would like to see signed commits in the 'official' repos (at
 git.openstack.org), if only because relying on sha alone doesn't seem
 enough for some.

Yeah, maybe RPM and I are just too old and should drop reproducibility
requirement :)
It might be nitpicking to expect exact same bits but it seems that
only missing part is an option to tell sdist to set specific timestamp
on generated files and folders.


Cheers,
Alan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Dimitri John Ledkov]

On 29 May 2015 at 14:41, Thierry Carrez thie...@openstack.org wrote:
 Hi everyone,

 TL;DR:
 - We propose to stop tagging coordinated point releases (like 2015.1.1)
 - We continue maintaining stable branches as a trusted source of stable
 updates for all projects though


Ideally I would still want to see tarballs published, even if they are
generated with a $ git describe --tags name in it.

E.g. keystone-2015.1.0-3-g1373bfa.tar.gz

keystone-stable-kilo.tar.gz - is really bad name imho, as it's ever
changing tarball. It would be nice if that was e.g. a symlink to a
keystone-2015.1.0-3-g1373bfa.tar.gz if said snapshot is latest.

If the stable tarball snapshots had stable, ever increasing names, I
would switch to using those straight away. (Out of which $ git
describe --tags satisfies said requirement)

-- 
Regards,

Dimitri.
Pura Vida!

https://clearlinux.org
Open Source Technology Center
Intel Corporation (UK) Ltd. - Co. Reg. #1134945 - Pipers Way, Swindon SN3 1RJ.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Matthias Runge]

On 01/06/15 12:10, Flavio Percoco wrote:


Is this a real problem? What are *tarball timestamps* used for in the
packaging world?

I'm sure there's a way we can workaround this issue.


timestamps just give you a hint, how old the source actually is, not 
when a packager downloaded the tarball somewhere. It just gives you a 
more realistic idea, how ancient the ancient code release is.




And: you probably want some hashes to verify, your downloaded tarball
is actually, what you wanted.


These can be generated as well. You can generate a tarball hash for
each commit and keep it around. The hash shouldn't change if the
tarball is generated on-the-fly. You could actually generate it
on-the-fly as well.
Sure, you can. You still need to provide that info. Ideally you'd 
prepare a signed file containing your hash.


I mean, something comparable to:

http://centos.bio.lmu.de/7/isos/x86_64/sha256sum.txt.asc

(for CentOS 7 iso files).


Matthias


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Ihar Hrachyshka]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 05/29/2015 06:24 PM, Jeremy Stanley wrote:
 On 2015-05-29 17:50:01 +0200 (+0200), Ihar Hrachyshka wrote: [...]
 if we attempt to fix a security issue that has a backwards 
 incompatible fix, then we are forced in introducing a new 
 configuration option to opt-in the new secure world.
 [...]
 
 To my knowledge that's how we've handled these in the past anyway, 
 accompanied by publication of a security note (not advisory) 
 suggesting the steps necessary to enable the breaking change when 
 opting into the bug fix.
 

AFAIK at least for keep alive option for wsgi TCP connections we
haven't done it, at least specifically for stable branches (the fixes
were propagated into stable branches during several minor stable
releases).

Ihar
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQEcBAEBCAAGBQJVbBGWAAoJEC5aWaUY1u577wEH/iqzGLGlc+Rb9bYrWeGRcQxt
bHwbU6kIVPAiJkT9eJGzoRonte/2u/+a6xvIiCXF6kcxIzYF3H25htbDsLe+DWmL
A8jPiSUPYksA/R/9FBoRBZBGr3Qs11imJcQ4PuSMNkM9KiZmlr/yle4kYpAtX9bY
VSsSVasgfplXX+1yR2mm6e0cSQlYWy5y8P2eKRhwYLAzc9LYDOpS2wPnI3Pz+gOQ
Muf3ogYbcC5dlXjZ0skf1QiGdj0Tenm9oVrSot+XhvMqE/sym9SfaV6OvP3193kq
N1xryagRdtsX7DkUfDuolziPMC2x20Q0JZZAcL13hQEa2O8QAjdW747JCb5+lUg=
=YW79
-END PGP SIGNATURE-

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Robert Collins]

On 1 Jun 2015 10:50 am, Alan Pevec ape...@gmail.com wrote:

 2015-05-29 18:30 GMT+02:00 Jeremy Stanley fu...@yuggoth.org:
  On 2015-05-29 16:30:12 +0100 (+0100), Dave Walker wrote:
  This is generally my opinion as-well, I always hoped that *every*
  commit would be considered a release rather than an arbitrary
  tagged date.
  [...]
 
  If we switch away from lockstep major/minor release versioning
  anyway (again separate discussion underway but seems a distinct
  possibility) then I think the confusion over why stable point
  releases are mismatched becomes less of an issue. At that point we
  may want to reconsider and actually tag each of them with a
  sequential micro (patch in semver terminology) version bump. Could
  help in communication around security fixes in particular.

 Yes, if dropping stable point releases, sub-version schema is still
 needed for clear communication in OSSAs and proposed continuous
 releases notes.
 One issue is how would we provide source tarballs, statically hosting
 tarballs for each and every micro version is not realistic, also those
 wouldn't be signed.

Sorry, but why isn't it realistic, and why wouldn't they be signed?

Rob
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Ihar Hrachyshka]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 06/01/2015 02:20 AM, Steve Gordon wrote:
 - Original Message -
 From: Maish Saidel-Keesing mais...@maishsk.com To:
 openstack-dev@lists.openstack.org
 
 On 05/29/15 18:25, Matthew Thode wrote:
 On 05/29/2015 10:18 AM, Ihar Hrachyshka wrote:
 What about release notes? How can we now communicate some
 changes that require operator consideration or action?
 
 Ihar
 
 On 05/29/2015 03:41 PM, Thierry Carrez wrote:
 Hi everyone, TL;DR: - We propose to stop tagging
 coordinated point releases (like 2015.1.1) - We continue
 maintaining stable branches as a trusted source of stable
 updates for all projects though Long version: At the
 stable branch session in Vancouver we discussed recent 
 evolutions in the stable team processes and how to further
 adapt the work of the team in a big tent world. One of
 the key questions there was whether we should continue 
 doing stable point releases. Those were basically tags with
 the same version number (2015.1.1) that we would
 periodically push to the stable branches for all projects. 
 Those create three problems. (1) Projects do not all follow
 the same versioning, so some projects (like Swift) were not
 part of the stable point releases. More and more projects
 are considering issuing intermediary releases (like Swift
 does), like Ironic. That would result in a variety of
 version numbers, and ultimately less and less projects 
 being able to have a common 2015.1.1-like version. (2)
 Producing those costs a non-trivial amount of effort on a
 very small team of volunteers, especially with projects
 caring about stable branches in various amounts. We were
 constantly missing the pre-announced dates on those ones.
 Looks like that effort could be better spent improving the
 stable branches themselves and keeping them working. (3)
 The resulting stable point releases are mostly useless. 
 Stable branches are supposed to be always usable, and the 
 released version did not undergo significantly more
 testing. Issuing them actually discourages people from
 taking whatever point in stable branches makes the most
 sense for them, testing and deploying that. The suggestion
 we made during that session (and which was approved by the
 session participants) is therefore to just get rid of the 
 stable point release concept altogether for
 non-libraries. That said: - we'd still do individual point
 releases for libraries (for critical bugs and security
 issues), so that you can still depend on a specific version
 there - we'd still very much maintain stable branches (and
 actually focus our efforts on that work) to ensure they are
 a continuous source of safe upgrades for users of a given
 series Now we realize that the cross-section of our
 community which was present in that session might not fully
 represent the consumers of those artifacts, which is why we
 expand the discussion on this mailing-list (and soon on the
 operators ML). If you were a consumer of those and will
 miss them, please explain why. In particular, please let us
 know how consuming that version (which was only made
 available every n months) is significantly better than
 picking your preferred time and get all the current stable
 branch HEADs at that time. Thanks in advance for your
 feedback, [1]
 https://etherpad.openstack.org/p/YVR-relmgt-stable-branch
 
 ___
___

 
OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

 
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 for release notes just do git log between commit hashes?
 Do you really think that is what an Operator will do? I do not
 think is a realistic expectation or something that will work. -- 
 Best Regards, Maish Saidel-Keesing
 
 While I agree most operators probably don't want to necessarily dig
 this out of git themselves and it would need to be collated/exposed
 in a nicer way it seems like it would actually be more accurate
 than the current release notes for all the non-security bug fixes
 in stable which basically amount to a list of launchpad bug queries
 per project. You then have to sift through each bug to work out if
 the description reflects what was actually done etc:
 
 https://wiki.openstack.org/wiki/ReleaseNotes/2014.2.3

I think the section that is really useful is Known Issues and
Limitations. Sometimes it suggests operators to apply some
configuration changes to their nodes:

https://wiki.openstack.org/wiki/ReleaseNotes/2014.2.2

Ihar
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQEcBAEBCAAGBQJVbBEvAAoJEC5aWaUY1u57PZgIAONnV7PKTF9kCyLZHhC/iOfJ
ao1xUrbwK7g6Li5/jha35BijJ0zsR4bFkNknDj8hGf81w0dpUs8A/mndYKsZB9K6
Awv5eho3b7qulO93nkI/JcWlpctGjN7kEAroSFEjDlhbbeiJAvKgTHsmEQy5ilBr
2ymU59LpG/YFWVBsRR1Odm6qBY8+x+YH4vX/GIK1vNy6cvBEqPKHo8dndrvgnHFd

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Alan Pevec]

 One issue is how would we provide source tarballs, statically hosting
 tarballs for each and every micro version is not realistic, also those
 wouldn't be signed.

 Sorry, but why isn't it realistic, and why wouldn't they be signed?

I thought storage requirements to generate tarball for each commit for
each project is unrealistic but infra could tell.
Re. signing - I was judging from current stable release procedure[*]
where tarballs are manually signed by the release manager before
upload to Launchpad.


Cheers,
Alan

[*] https://wiki.openstack.org/wiki/StableBranchRelease#Upload_The_Release

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Matthias Runge]

On 01/06/15 10:29, Flavio Percoco wrote:


AFAIK, these tarballs are generated on the flight in GH. Couldn't we
do the same ?
The issue with on-the-fly generation is: timestamps are actually new, 
and not the same as when the release was tagged.


And: you probably want some hashes to verify, your downloaded tarball is 
actually, what you wanted.


Matthias

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Flavio Percoco]

On 01/06/15 11:27 +0200, Matthias Runge wrote:

On 01/06/15 10:29, Flavio Percoco wrote:


AFAIK, these tarballs are generated on the flight in GH. Couldn't we
do the same ?


s/flight/fly+(facepalm)/ LOL, on the *flight* would be really hard.

The issue with on-the-fly generation is: timestamps are actually new, 
and not the same as when the release was tagged.


Is this a real problem? What are *tarball timestamps* used for in the
packaging world?

I'm sure there's a way we can workaround this issue.



And: you probably want some hashes to verify, your downloaded tarball 
is actually, what you wanted.


These can be generated as well. You can generate a tarball hash for
each commit and keep it around. The hash shouldn't change if the
tarball is generated on-the-fly. You could actually generate it
on-the-fly as well.

Cheers,
Flavio

--
@flaper87
Flavio Percoco


pgpJ147KU7g8S.pgp
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Flavio Percoco]

On 01/06/15 10:07 +0200, Alan Pevec wrote:

One issue is how would we provide source tarballs, statically hosting
tarballs for each and every micro version is not realistic, also those
wouldn't be signed.


Sorry, but why isn't it realistic, and why wouldn't they be signed?


I thought storage requirements to generate tarball for each commit for
each project is unrealistic but infra could tell.
Re. signing - I was judging from current stable release procedure[*]
where tarballs are manually signed by the release manager before
upload to Launchpad.


AFAIK, these tarballs are generated on the flight in GH. Couldn't we
do the same ?

Flavio

--
@flaper87
Flavio Percoco


pgpt5wIdnMRGi.pgp
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Ihar Hrachyshka]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 05/29/2015 11:46 PM, Haïkel wrote:
 A release version makes it easy to know what fixes are shipped in a
 package. If you rebase on stable branches, then you can just put
 the git sha1sum (though, it's not very friendly) in the version,
 and leverage git branch --contains to find out if you fix is
 included. Some distributors may choose to use their own release
 scheme, adding complexity to this simple but common problem. Other
 may choose to cherry-pick which adds more complexity than the 
 previous scenario.
 
 Let's say you're an operator and you want to check if a CVE is
 shipped in all your nodes, if you can't check with just the release
 version, it will be complicated. It could be a barrier for
 heterogeneous systems

Is it possible to give tagging permissions to projects themselves? No
time synchronization between projects, no freeze dates, nothing, just
SemVer compatible version bumps e.g. once per two weeks or month.
Would it fix your problem?

Ihar
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQEcBAEBCAAGBQJVbBp7AAoJEC5aWaUY1u57GugH/3JL9Izi7zULUHMCkPwsjE+w
qmkd25f3tMj9GxHpv8CbmLyz3H+5MtZfJHgX+AJYhkX/nF0SRkVvURdbYEDD4mrB
8w8W8nieD02KgWtglmIvulJs3atYmrXpLf38s7aWkciIp3MuUI0qEQ2LzZSV8JJq
50igFTle6IRR6NVbf391+nxYO33Eqv8Ckw7wVknJX22V2HueJuDzIgqGZeIOzfu4
drqzqSua2ZBhAJVR3bscdV+hSVieW2c2C9BLPaL3gG8/SD/7Qt/Ilj2lWgcTyAiy
LIHZt4sq7KPbm2AULItO+HooGHlNJLVL1yoBK4yFO+KvPNIZvtaF91YJ2+hn1UQ=
=Apa3
-END PGP SIGNATURE-

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thierry Carrez]

Thierry Carrez wrote:
 TL;DR:
 - We propose to stop tagging coordinated point releases (like 2015.1.1)
 - We continue maintaining stable branches as a trusted source of stable
 updates for all projects though

Thanks everyone for the constructive discussion on this. Lots of good
stuff. Let me summarize the objections so far:

1. we'd lose the focus rhythm

Point releases triggered some attention to stable branches: the
necessity to fix the CI there when it's broken, and a surge in backports
activity.

2. it would be difficult to get proper release notes

If we don't have point releases anymore, we don't have release notes
anymore. Release notes contain various types of information: the list of
security fixes, the occasional upgrade warning, and the list of bugfixes.

3. we'd lose reference points

Tagged versions made great common reference points to share information
between various distros/deployments. If a distro packaged 2014.2.2, you
could take the same tag and package it and assume it was the same thing.
They also served as fixed in version X information on OSSAs.

4. we'd lose the works well together piece of information

The various pieces tagged with the same version number could be assumed
to work well together. Recovering that would have to be pieced back
together from commit dates and/or openstack/openstack stable branches.

Missed anything ?

Please see my following email as I try to address each of those.

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Flavio Percoco]

On 29/05/15 09:44 -0500, Matt Riedemann wrote:



On 5/29/2015 8:41 AM, Thierry Carrez wrote:

Hi everyone,

TL;DR:
- We propose to stop tagging coordinated point releases (like 2015.1.1)
- We continue maintaining stable branches as a trusted source of stable
updates for all projects though

Long version:

At the stable branch session in Vancouver we discussed recent
evolutions in the stable team processes and how to further adapt the
work of the team in a big tent world.

One of the key questions there was whether we should continue doing
stable point releases. Those were basically tags with the same version
number (2015.1.1) that we would periodically push to the stable
branches for all projects.

Those create three problems.

(1) Projects do not all follow the same versioning, so some projects
(like Swift) were not part of the stable point releases. More and more
projects are considering issuing intermediary releases (like Swift
does), like Ironic. That would result in a variety of version numbers,
and ultimately less and less projects being able to have a common
2015.1.1-like version.

(2) Producing those costs a non-trivial amount of effort on a very small
team of volunteers, especially with projects caring about stable
branches in various amounts. We were constantly missing the
pre-announced dates on those ones. Looks like that effort could be
better spent improving the stable branches themselves and keeping them
working.

(3) The resulting stable point releases are mostly useless. Stable
branches are supposed to be always usable, and the released version
did not undergo significantly more testing. Issuing them actually
discourages people from taking whatever point in stable branches makes
the most sense for them, testing and deploying that.

The suggestion we made during that session (and which was approved by
the session participants) is therefore to just get rid of the stable
point release concept altogether for non-libraries. That said:

- we'd still do individual point releases for libraries (for critical
bugs and security issues), so that you can still depend on a specific
version there

- we'd still very much maintain stable branches (and actually focus our
efforts on that work) to ensure they are a continuous source of safe
upgrades for users of a given series

Now we realize that the cross-section of our community which was present
in that session might not fully represent the consumers of those
artifacts, which is why we expand the discussion on this mailing-list
(and soon on the operators ML).

If you were a consumer of those and will miss them, please explain why.
In particular, please let us know how consuming that version (which was
only made available every n months) is significantly better than picking
your preferred time and get all the current stable branch HEADs at that
time.

Thanks in advance for your feedback,

[1] https://etherpad.openstack.org/p/YVR-relmgt-stable-branch



To reiterate what I said in the session, for my team personally (IBM), 
we don't align with the point release schedules on stable anyway, we 
release our own stable release fix packs as needed on our own 
schedules, so in that regard I don't see a point in the stable point 
releases - especially since most of the time I don't know when those 
are going to be anyway so we can't plan for them accurately.


Having said that, what I mentioned in IRC the other day is the one 
upside I see to the point releases is it is a milestone that requires 
focus from the stable maintainers, which means if stable has been 
broken for a few weeks and no one has really noticed, converging on a 
stable point release at least forces attention there.


I don't think that is a very good argument for keeping stable point 
releases though, since as you said we don't do any additional testing 
above and beyond what normally happens in the Jenkins runs.  Some of 
the distributions might have extra regression testing scenarios, I'm 
not sure, but no one really spoke to that in the session from the 
distros that were present - I assume they do, but they can do that on 
their own schedule anyway IMO.


I agree it's not a good point in favor of point releases. If anything,
that's just showing some issues with the stable workflow that probably
require more attention than the milestone itself.


I am a bit cynical about thinking that dropping point releases will 
make people spend more time on caring about the health of the stable 
branches (persistent gate failures) or stale changes out for review.  
I combed through a lot of open stable/icehouse changes yesterday and 
there were many that should have been abandoned 6 months ago but were 
just sitting there, and others that were good fixes to have and should 
have been merged by now.


More than making people focus more on the health of stable branches
the whole goal is to remove something that is(?) not necessary.
Basically, this proposal removes unnecessary(?) work from
stable/release 

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Robert Collins]

On 1 Jun 2015 8:07 pm, Alan Pevec ape...@gmail.com wrote:

  One issue is how would we provide source tarballs, statically hosting
  tarballs for each and every micro version is not realistic, also those
  wouldn't be signed.
 
  Sorry, but why isn't it realistic, and why wouldn't they be signed?

 I thought storage requirements to generate tarball for each commit for
 each project is unrealistic but infra could tell.

There are only a small number of commits to stake branches... I suspect we
could easily archive an sdist per commit there.

 Re. signing - I was judging from current stable release procedure[*]
 where tarballs are manually signed by the release manager before
 upload to Launchpad

That seems like something we could do differently if we were to archive an
sdist power commit on stable.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thierry Carrez]

Thierry Carrez wrote:
 Thanks everyone for the constructive discussion on this. Lots of good
 stuff. Let me summarize the objections so far:
 
 1. we'd lose the focus rhythm
 
 Point releases triggered some attention to stable branches: the
 necessity to fix the CI there when it's broken, and a surge in backports
 activity.

The focus rhythm was (a) less effective recently as more projects just
ignored the stable release windows and (b) less necessary since we
switched to per-project stable maintenance teams that regularly backport
stuff. I actually tend to think that getting rid of the let's care
about stable branches only every 2 months mentality would be a plus.

 2. it would be difficult to get proper release notes
 
 If we don't have point releases anymore, we don't have release notes
 anymore. Release notes contain various types of information: the list of
 security fixes, the occasional upgrade warning, and the list of bugfixes.

We'd probably have to find a way to provide the same information in the
tarball itself, so that if you picked any of them you could still get a
list of the fixes in there.

 3. we'd lose reference points
 
 Tagged versions made great common reference points to share information
 between various distros/deployments. If a distro packaged 2014.2.2, you
 could take the same tag and package it and assume it was the same thing.
 They also served as fixed in version X information on OSSAs.

That one is a valid concern. It is true that the proposed change will
result in making it more difficult to compare post-release deployments
and google for fixed in 2015.1.1 information. Please note that this
reference point was rather fuzzy, since distros would end up applying
their own extra patches on top of our tag, so you are actually not
comparing the exact same thing anyway. But I'd agree that it provided
marginal value.

 4. we'd lose the works well together piece of information
 
 The various pieces tagged with the same version number could be assumed
 to work well together. Recovering that would have to be pieced back
 together from commit dates and/or openstack/openstack stable branches.

I think this one is an illusion. Stable branch policy enforces that
there is no change in API or behavior in stable branches. So you could
technically take any commit from any branch and it should work as well
together as the release one. For those who really want a CI-tested
combination, I think they can go through the extra hassle of
determining one through openstack/openstack stable branches.

So.. what now ? What would be an alternate solution ?

Plan B
==

We could continue to push random synchronization tags to solve (3). We'd
get rid of most of the release process and just tag things tested
together on a given date (solving (4)). We should find a way to
outsource release notes maintenance to each project, possibly by
maintaining those in stable branches directly (solving (2)).

For kilo we can still call those 2015.1.X and apply them on the Kilo
integrated release set of projects.

For liberty we will no longer have an inner sanctum of integrated
projects. So we could tag all server OpenStack projects which maintain
a stable branch. Since versioning will continue to diverge, the version
tagged will be different for every project, which will be confusing. So
we could also additionally tag with something like liberty SP1 across
the board to facilitate finding the various pieces tagged at the same
moment.

Coming back to our original list of problems we wanted to solve:

 (1) Projects do not all follow the same versioning, so some projects
 (like Swift) were not part of the stable point releases. More and more
 projects are considering issuing intermediary releases (like Swift
 does), like Ironic. That would result in a variety of version numbers,
 and ultimately less and less projects being able to have a common
 2015.1.1-like version.

I think the alternate proposal is a bit confusing (since over the long
run projects will not have the same version numbers, but people will be
used to assume that anything tagged with the same version number was
issued around the same time).

 (2) Producing those costs a non-trivial amount of effort on a very small
 team of volunteers, especially with projects caring about stable
 branches in various amounts. We were constantly missing the
 pre-announced dates on those ones. Looks like that effort could be
 better spent improving the stable branches themselves and keeping them
 working.

The alternate proposal is slightly less costly than the current
situation (no stable release prep period, just a very large set of tags
to push), but obviously more costly than just getting rid of the whole idea.

 (3) The resulting stable point releases are mostly useless. Stable
 branches are supposed to be always usable, and the released version
 did not undergo significantly more testing. Issuing them actually
 discourages people from taking whatever point in stable 

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Jeremy Stanley]

On 2015-06-01 12:04:34 +0200 (+0200), Thierry Carrez wrote:
 Thierry Carrez wrote:
[...]
  2. it would be difficult to get proper release notes
  
  If we don't have point releases anymore, we don't have release notes
  anymore. Release notes contain various types of information: the list of
  security fixes, the occasional upgrade warning, and the list of bugfixes.
 
 We'd probably have to find a way to provide the same information in the
 tarball itself, so that if you picked any of them you could still get a
 list of the fixes in there.
[...]

PBR has the ability to generate a ChangeLog as part of the sdist
tarball generation. If the ChangeLog itself isn't suitable, we could
do something akin to what Doug described about scraping commit
messages for particular specially-formatted header lines and then
maybe have a separate release notes sdist hook in PBR. Or we could
just expect projects to update a release notes file in that repo on
any stable branch change which implies some action on the part of
the downstream consumers.

As to the separate concern raised by a couple of respondents for a
lack of signatures on stable branch tarballs (be they per-commit or
arbitrarily tagged by their respective project teams), I've been
itching to implement some mechanical generation of detached
signatures to publish along with artifacts uploaded to
tarballs.openstack.org and PyPI anyway. We could presumably leverage
that. It wouldn't be a human signature, just a signature made by a
trusted host in the infrastructure build chain whose key was signed
by some of the infrastructure root admins, but would at least attest
to the integrity of the file once uploaded to the point of
publication.
-- 
Jeremy Stanley


signature.asc
Description: Digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Jeremy Stanley]

On 2015-06-01 17:32:03 +0200 (+0200), Alan Pevec wrote:
[...]
 and *Plan D* would be to start doing automatic per-project
 micro-versions on each commit: e.g. 2015.1.N where N is increased on
 each commit. There's just TBD item how to provide source tarballs for
 this.
[...]

That would happen automatically (they're already generated and
uploaded to tarballs.openstack.org each time a tag is pushed).
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Jeremy Stanley]

On 2015-06-01 12:20:22 +0100 (+0100), Dimitri John Ledkov wrote:
[...]
 If the stable tarball snapshots had stable, ever increasing names, I
 would switch to using those straight away. (Out of which $ git
 describe --tags satisfies said requirement)

Current PBR releases already create monotonically-increasing
SemVer-like PEP-440-compliant versions for any commit, and name the
resulting sdist tarballs and wheels accordingly. It would be fairly
easy right now to switch stable branch tarball uploads on
tarballs.openstack.org to do that rather than constantly replace a
branch tip tarball. The biggest hurdle is that we'd need a separate
upload job name for those since the current version of Zuul lacks a
way to run a particular job for different branches in different
pipelines (we'd want to do versioned uploads for all pre-release and
release pipeline refs, but also for post pipeline refs only when the
branch name is like stable/.*).
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Alan Pevec]

 *Plan C* would be to just let projects tag stable point releases from
 time to time. That would solve all the original stated problems. And
 that would solve objections 2 and 3, which I think are the most valid ones.

and *Plan D* would be to start doing automatic per-project
micro-versions on each commit: e.g. 2015.1.N where N is increased on
each commit. There's just TBD item how to provide source tarballs for
this.
This would solve 3. reference points for OSSAs and relnotes.
Solution for 2. could be to add doc/source/release-notes.rst for each
project wishing to maintain it, docs are already published for each
branch e.g. http://docs.openstack.org/developer/keystone/kilo/

Cheers,
Alan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Haïkel]

2015-06-01 17:32 GMT+02:00 Alan Pevec ape...@gmail.com:
 *Plan C* would be to just let projects tag stable point releases from
 time to time. That would solve all the original stated problems. And
 that would solve objections 2 and 3, which I think are the most valid ones.

 and *Plan D* would be to start doing automatic per-project
 micro-versions on each commit: e.g. 2015.1.N where N is increased on
 each commit. There's just TBD item how to provide source tarballs for
 this.

+1 for micro-versions rather than raw git checksums.

 This would solve 3. reference points for OSSAs and relnotes.
 Solution for 2. could be to add doc/source/release-notes.rst for each
 project wishing to maintain it, docs are already published for each
 branch e.g. http://docs.openstack.org/developer/keystone/kilo/

 Cheers,
 Alan

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Ian Cordasco]

On 6/1/15, 06:20, Dimitri John Ledkov dimitri.j.led...@intel.com wrote:

On 29 May 2015 at 14:41, Thierry Carrez thie...@openstack.org wrote:
 Hi everyone,

 TL;DR:
 - We propose to stop tagging coordinated point releases (like 2015.1.1)
 - We continue maintaining stable branches as a trusted source of stable
 updates for all projects though


Ideally I would still want to see tarballs published, even if they are
generated with a $ git describe --tags name in it.

E.g. keystone-2015.1.0-3-g1373bfa.tar.gz

keystone-stable-kilo.tar.gz - is really bad name imho, as it's ever
changing tarball. It would be nice if that was e.g. a symlink to a
keystone-2015.1.0-3-g1373bfa.tar.gz if said snapshot is latest.

If the stable tarball snapshots had stable, ever increasing names, I
would switch to using those straight away. (Out of which $ git
describe --tags satisfies said requirement)

Except this won't help anyone deploying with pip (which I know people
don't believe actually happens). That's not a valid pip version number
(neither 2015.1.0-3-g1373bfa nor stable-kilo are valid). If someone were
to use this archive with something like

pip install 'keystone=2015.1.0' --find-links
https://stable-archive.openstack.org/kilo/

Then they wouldn't find anything new. If the releases are to be
auto-versioned, they should be done in a different way.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Jeremy Stanley]

On 2015-06-01 15:57:17 + (+), Jeremy Stanley wrote:
[...]
 The biggest hurdle is that we'd need a separate upload job name
 for those since the current version of Zuul lacks a way to run a
 particular job for different branches in different pipelines (we'd
 want to do versioned uploads for all pre-release and release
 pipeline refs, but also for post pipeline refs only when the
 branch name is like stable/.*).

Actually, scratch that. It's a bit more complicated since the post
pipeline isn't actually branch-relevant. We'd need to tweak the
tarball and wheel creation scripts to check the containing branch,
like we do for some proposal jobs. Still, I think it wouldn't be too
hard.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Alan Pevec]

2015-05-29 18:30 GMT+02:00 Jeremy Stanley fu...@yuggoth.org:
 On 2015-05-29 16:30:12 +0100 (+0100), Dave Walker wrote:
 This is generally my opinion as-well, I always hoped that *every*
 commit would be considered a release rather than an arbitrary
 tagged date.
 [...]

 If we switch away from lockstep major/minor release versioning
 anyway (again separate discussion underway but seems a distinct
 possibility) then I think the confusion over why stable point
 releases are mismatched becomes less of an issue. At that point we
 may want to reconsider and actually tag each of them with a
 sequential micro (patch in semver terminology) version bump. Could
 help in communication around security fixes in particular.

Yes, if dropping stable point releases, sub-version schema is still
needed for clear communication in OSSAs and proposed continuous
releases notes.
One issue is how would we provide source tarballs, statically hosting
tarballs for each and every micro version is not realistic, also those
wouldn't be signed.
RPM packages traditionally expect pristine upstream tarballs which can
be verified and generating them from git is not reproducible e.g.
right now in nova stable/kilo branch:
python ./setup.py sdist
mv dist/nova-2015.1.1.dev20.tar.gz dist/nova-2015.1.1.dev20.tar.gz-TAKE1
python ./setup.py sdist
diff dist/nova-2015.1.1.dev20.tar.gz-TAKE1 dist/nova-2015.1.1.dev20.tar.gz
Binary files dist/nova-2015.1.1.dev20.tar.gz-TAKE1 and
dist/nova-2015.1.1.dev20.tar.gz differ

Before dropping point releases, I would like to have:
* idempotent sdist on the same SHA
* dynamic tarball generation service like github archive
* switch to micro-version i.e. current nova stable/kilo would be 2015.1.20


Cheers,
Alan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Matthew Thode]

On 05/31/2015 05:50 PM, Alan Pevec wrote:
 2015-05-29 18:30 GMT+02:00 Jeremy Stanley fu...@yuggoth.org:
 On 2015-05-29 16:30:12 +0100 (+0100), Dave Walker wrote:
 This is generally my opinion as-well, I always hoped that *every*
 commit would be considered a release rather than an arbitrary
 tagged date.
 [...]

 If we switch away from lockstep major/minor release versioning
 anyway (again separate discussion underway but seems a distinct
 possibility) then I think the confusion over why stable point
 releases are mismatched becomes less of an issue. At that point we
 may want to reconsider and actually tag each of them with a
 sequential micro (patch in semver terminology) version bump. Could
 help in communication around security fixes in particular.
 
 Yes, if dropping stable point releases, sub-version schema is still
 needed for clear communication in OSSAs and proposed continuous
 releases notes.
 One issue is how would we provide source tarballs, statically hosting
 tarballs for each and every micro version is not realistic, also those
 wouldn't be signed.
 RPM packages traditionally expect pristine upstream tarballs which can
 be verified and generating them from git is not reproducible e.g.
 right now in nova stable/kilo branch:
 python ./setup.py sdist
 mv dist/nova-2015.1.1.dev20.tar.gz dist/nova-2015.1.1.dev20.tar.gz-TAKE1
 python ./setup.py sdist
 diff dist/nova-2015.1.1.dev20.tar.gz-TAKE1 dist/nova-2015.1.1.dev20.tar.gz
 Binary files dist/nova-2015.1.1.dev20.tar.gz-TAKE1 and
 dist/nova-2015.1.1.dev20.tar.gz differ
 
 Before dropping point releases, I would like to have:
 * idempotent sdist on the same SHA
 * dynamic tarball generation service like github archive
 * switch to micro-version i.e. current nova stable/kilo would be 2015.1.20
 
 
 Cheers,
 Alan
 
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
Generating tarballs from commit sha's isn't enough?

I'm personally thinking of installing a file somewhere that references
what commit hash the package was sourced from.  I'm thinking of doing
weekly releases.

Tarball generation would be nice.

You will get different checksums with tar and/or gzip, you can check the
extracted files and they should be the same.

I would like to see signed commits in the 'official' repos (at
git.openstack.org), if only because relying on sha alone doesn't seem
enough for some.

-- 
Matthew Thode



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Steve Gordon]

- Original Message -
 From: Maish Saidel-Keesing mais...@maishsk.com
 To: openstack-dev@lists.openstack.org
 
 On 05/29/15 18:25, Matthew Thode wrote:
  On 05/29/2015 10:18 AM, Ihar Hrachyshka wrote:
  What about release notes? How can we now communicate some changes that
  require operator consideration or action?
 
  Ihar
 
  On 05/29/2015 03:41 PM, Thierry Carrez wrote:
  Hi everyone,
  TL;DR: - We propose to stop tagging coordinated point releases
  (like 2015.1.1) - We continue maintaining stable branches as a
  trusted source of stable updates for all projects though
  Long version:
  At the stable branch session in Vancouver we discussed recent
  evolutions in the stable team processes and how to further adapt
  the work of the team in a big tent world.
  One of the key questions there was whether we should continue
  doing stable point releases. Those were basically tags with the
  same version number (2015.1.1) that we would periodically push to
  the stable branches for all projects.
  Those create three problems.
  (1) Projects do not all follow the same versioning, so some
  projects (like Swift) were not part of the stable point releases.
  More and more projects are considering issuing intermediary
  releases (like Swift does), like Ironic. That would result in a
  variety of version numbers, and ultimately less and less projects
  being able to have a common 2015.1.1-like version.
  (2) Producing those costs a non-trivial amount of effort on a very
  small team of volunteers, especially with projects caring about
  stable branches in various amounts. We were constantly missing the
  pre-announced dates on those ones. Looks like that effort could be
  better spent improving the stable branches themselves and keeping
  them working.
  (3) The resulting stable point releases are mostly useless.
  Stable branches are supposed to be always usable, and the
  released version did not undergo significantly more testing.
  Issuing them actually discourages people from taking whatever point
  in stable branches makes the most sense for them, testing and
  deploying that.
  The suggestion we made during that session (and which was approved
  by the session participants) is therefore to just get rid of the
  stable point release concept altogether for non-libraries. That
  said:
  - we'd still do individual point releases for libraries (for
  critical bugs and security issues), so that you can still depend on
  a specific version there
  - we'd still very much maintain stable branches (and actually focus
  our efforts on that work) to ensure they are a continuous source of
  safe upgrades for users of a given series
  Now we realize that the cross-section of our community which was
  present in that session might not fully represent the consumers of
  those artifacts, which is why we expand the discussion on this
  mailing-list (and soon on the operators ML).
  If you were a consumer of those and will miss them, please explain
  why. In particular, please let us know how consuming that version
  (which was only made available every n months) is significantly
  better than picking your preferred time and get all the current
  stable branch HEADs at that time.
  Thanks in advance for your feedback,
  [1] https://etherpad.openstack.org/p/YVR-relmgt-stable-branch
 
  __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
  for release notes just do git log between commit hashes?
 Do you really think that is what an Operator will do? I do not think is
 a realistic expectation or something that will work.
 --
 Best Regards,
 Maish Saidel-Keesing

While I agree most operators probably don't want to necessarily dig this out of 
git themselves and it would need to be collated/exposed in a nicer way it seems 
like it would actually be more accurate than the current release notes for 
all the non-security bug fixes in stable which basically amount to a list of 
launchpad bug queries per project. You then have to sift through each bug to 
work out if the description reflects what was actually done etc:

https://wiki.openstack.org/wiki/ReleaseNotes/2014.2.3

-Steve

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Maish Saidel-Keesing]

On 05/29/15 18:25, Matthew Thode wrote:

On 05/29/2015 10:18 AM, Ihar Hrachyshka wrote:

What about release notes? How can we now communicate some changes that
require operator consideration or action?

Ihar

On 05/29/2015 03:41 PM, Thierry Carrez wrote:

Hi everyone,
TL;DR: - We propose to stop tagging coordinated point releases
(like 2015.1.1) - We continue maintaining stable branches as a
trusted source of stable updates for all projects though
Long version:
At the stable branch session in Vancouver we discussed recent
evolutions in the stable team processes and how to further adapt
the work of the team in a big tent world.
One of the key questions there was whether we should continue
doing stable point releases. Those were basically tags with the
same version number (2015.1.1) that we would periodically push to
the stable branches for all projects.
Those create three problems.
(1) Projects do not all follow the same versioning, so some
projects (like Swift) were not part of the stable point releases.
More and more projects are considering issuing intermediary
releases (like Swift does), like Ironic. That would result in a
variety of version numbers, and ultimately less and less projects
being able to have a common 2015.1.1-like version.
(2) Producing those costs a non-trivial amount of effort on a very
small team of volunteers, especially with projects caring about
stable branches in various amounts. We were constantly missing the
pre-announced dates on those ones. Looks like that effort could be
better spent improving the stable branches themselves and keeping
them working.
(3) The resulting stable point releases are mostly useless.
Stable branches are supposed to be always usable, and the
released version did not undergo significantly more testing.
Issuing them actually discourages people from taking whatever point
in stable branches makes the most sense for them, testing and
deploying that.
The suggestion we made during that session (and which was approved
by the session participants) is therefore to just get rid of the
stable point release concept altogether for non-libraries. That
said:
- we'd still do individual point releases for libraries (for
critical bugs and security issues), so that you can still depend on
a specific version there
- we'd still very much maintain stable branches (and actually focus
our efforts on that work) to ensure they are a continuous source of
safe upgrades for users of a given series
Now we realize that the cross-section of our community which was
present in that session might not fully represent the consumers of
those artifacts, which is why we expand the discussion on this
mailing-list (and soon on the operators ML).
If you were a consumer of those and will miss them, please explain
why. In particular, please let us know how consuming that version
(which was only made available every n months) is significantly
better than picking your preferred time and get all the current
stable branch HEADs at that time.
Thanks in advance for your feedback,
[1] https://etherpad.openstack.org/p/YVR-relmgt-stable-branch


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


for release notes just do git log between commit hashes?
Do you really think that is what an Operator will do? I do not think is 
a realistic expectation or something that will work.

--
Best Regards,
Maish Saidel-Keesing

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [all] [stable] No longer doing stable point releases

[Thierry Carrez]

Hi everyone,

TL;DR:
- We propose to stop tagging coordinated point releases (like 2015.1.1)
- We continue maintaining stable branches as a trusted source of stable
updates for all projects though

Long version:

At the stable branch session in Vancouver we discussed recent
evolutions in the stable team processes and how to further adapt the
work of the team in a big tent world.

One of the key questions there was whether we should continue doing
stable point releases. Those were basically tags with the same version
number (2015.1.1) that we would periodically push to the stable
branches for all projects.

Those create three problems.

(1) Projects do not all follow the same versioning, so some projects
(like Swift) were not part of the stable point releases. More and more
projects are considering issuing intermediary releases (like Swift
does), like Ironic. That would result in a variety of version numbers,
and ultimately less and less projects being able to have a common
2015.1.1-like version.

(2) Producing those costs a non-trivial amount of effort on a very small
team of volunteers, especially with projects caring about stable
branches in various amounts. We were constantly missing the
pre-announced dates on those ones. Looks like that effort could be
better spent improving the stable branches themselves and keeping them
working.

(3) The resulting stable point releases are mostly useless. Stable
branches are supposed to be always usable, and the released version
did not undergo significantly more testing. Issuing them actually
discourages people from taking whatever point in stable branches makes
the most sense for them, testing and deploying that.

The suggestion we made during that session (and which was approved by
the session participants) is therefore to just get rid of the stable
point release concept altogether for non-libraries. That said:

- we'd still do individual point releases for libraries (for critical
bugs and security issues), so that you can still depend on a specific
version there

- we'd still very much maintain stable branches (and actually focus our
efforts on that work) to ensure they are a continuous source of safe
upgrades for users of a given series

Now we realize that the cross-section of our community which was present
in that session might not fully represent the consumers of those
artifacts, which is why we expand the discussion on this mailing-list
(and soon on the operators ML).

If you were a consumer of those and will miss them, please explain why.
In particular, please let us know how consuming that version (which was
only made available every n months) is significantly better than picking
your preferred time and get all the current stable branch HEADs at that
time.

Thanks in advance for your feedback,

[1] https://etherpad.openstack.org/p/YVR-relmgt-stable-branch

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Ihar Hrachyshka]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

What about release notes? How can we now communicate some changes that
require operator consideration or action?

Ihar

On 05/29/2015 03:41 PM, Thierry Carrez wrote:
 Hi everyone,
 
 TL;DR: - We propose to stop tagging coordinated point releases
 (like 2015.1.1) - We continue maintaining stable branches as a
 trusted source of stable updates for all projects though
 
 Long version:
 
 At the stable branch session in Vancouver we discussed recent 
 evolutions in the stable team processes and how to further adapt
 the work of the team in a big tent world.
 
 One of the key questions there was whether we should continue
 doing stable point releases. Those were basically tags with the
 same version number (2015.1.1) that we would periodically push to
 the stable branches for all projects.
 
 Those create three problems.
 
 (1) Projects do not all follow the same versioning, so some
 projects (like Swift) were not part of the stable point releases.
 More and more projects are considering issuing intermediary
 releases (like Swift does), like Ironic. That would result in a
 variety of version numbers, and ultimately less and less projects
 being able to have a common 2015.1.1-like version.
 
 (2) Producing those costs a non-trivial amount of effort on a very
 small team of volunteers, especially with projects caring about
 stable branches in various amounts. We were constantly missing the 
 pre-announced dates on those ones. Looks like that effort could be 
 better spent improving the stable branches themselves and keeping
 them working.
 
 (3) The resulting stable point releases are mostly useless.
 Stable branches are supposed to be always usable, and the
 released version did not undergo significantly more testing.
 Issuing them actually discourages people from taking whatever point
 in stable branches makes the most sense for them, testing and
 deploying that.
 
 The suggestion we made during that session (and which was approved
 by the session participants) is therefore to just get rid of the
 stable point release concept altogether for non-libraries. That
 said:
 
 - we'd still do individual point releases for libraries (for
 critical bugs and security issues), so that you can still depend on
 a specific version there
 
 - we'd still very much maintain stable branches (and actually focus
 our efforts on that work) to ensure they are a continuous source of
 safe upgrades for users of a given series
 
 Now we realize that the cross-section of our community which was
 present in that session might not fully represent the consumers of
 those artifacts, which is why we expand the discussion on this
 mailing-list (and soon on the operators ML).
 
 If you were a consumer of those and will miss them, please explain
 why. In particular, please let us know how consuming that version
 (which was only made available every n months) is significantly
 better than picking your preferred time and get all the current
 stable branch HEADs at that time.
 
 Thanks in advance for your feedback,
 
 [1] https://etherpad.openstack.org/p/YVR-relmgt-stable-branch
 
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQEcBAEBCAAGBQJVaIMzAAoJEC5aWaUY1u57/iIIANPync0FRB0Pe77fBWXEuCqX
jUdoRmhQR3kzGar5UYUXAh5wa2RD/xT1JIlrwjaz4I0jKKGs7nQJ2RoTG68OtgXf
Dk3WzVg7GgmJXrzFim1cACIzDmrX704gegfCSmf8qd0WC03AT3gI/uSR0NYmiW75
xBWSltPPt1T0PaKgV9lLQ6kyCjzdUySve4815jtGPf1hZUpyQlw1+7NE9LEUfbw8
P+wLCB4UyIQ00Hjf90pnjVOT6bhMQ2/2ldoPZsDTCI5PWB52Mqk9ZteuC/1QLBkS
OSRD4fZSMeGDXkSTpIKStZm3J36cM9BGsFzC2OcZ+C52yE3vS4g7cBFaPlLub6k=
=tP/b
-END PGP SIGNATURE-

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Dave Walker]

On 29 May 2015 at 14:41, Thierry Carrez thie...@openstack.org wrote:
 Hi everyone,

 TL;DR:
 - We propose to stop tagging coordinated point releases (like 2015.1.1)
 - We continue maintaining stable branches as a trusted source of stable
 updates for all projects though

 Long version:

 At the stable branch session in Vancouver we discussed recent
 evolutions in the stable team processes and how to further adapt the
 work of the team in a big tent world.

 One of the key questions there was whether we should continue doing
 stable point releases. Those were basically tags with the same version
 number (2015.1.1) that we would periodically push to the stable
 branches for all projects.

 Those create three problems.

 (1) Projects do not all follow the same versioning, so some projects
 (like Swift) were not part of the stable point releases. More and more
 projects are considering issuing intermediary releases (like Swift
 does), like Ironic. That would result in a variety of version numbers,
 and ultimately less and less projects being able to have a common
 2015.1.1-like version.

 (2) Producing those costs a non-trivial amount of effort on a very small
 team of volunteers, especially with projects caring about stable
 branches in various amounts. We were constantly missing the
 pre-announced dates on those ones. Looks like that effort could be
 better spent improving the stable branches themselves and keeping them
 working.

 (3) The resulting stable point releases are mostly useless. Stable
 branches are supposed to be always usable, and the released version
 did not undergo significantly more testing. Issuing them actually
 discourages people from taking whatever point in stable branches makes
 the most sense for them, testing and deploying that.

 The suggestion we made during that session (and which was approved by
 the session participants) is therefore to just get rid of the stable
 point release concept altogether for non-libraries. That said:

 - we'd still do individual point releases for libraries (for critical
 bugs and security issues), so that you can still depend on a specific
 version there

 - we'd still very much maintain stable branches (and actually focus our
 efforts on that work) to ensure they are a continuous source of safe
 upgrades for users of a given series

 Now we realize that the cross-section of our community which was present
 in that session might not fully represent the consumers of those
 artifacts, which is why we expand the discussion on this mailing-list
 (and soon on the operators ML).

 If you were a consumer of those and will miss them, please explain why.
 In particular, please let us know how consuming that version (which was
 only made available every n months) is significantly better than picking
 your preferred time and get all the current stable branch HEADs at that
 time.

 Thanks in advance for your feedback,

 [1] https://etherpad.openstack.org/p/YVR-relmgt-stable-branch

 --
 Thierry Carrez (ttx)

This is generally my opinion as-well, I always hoped that *every*
commit would be considered a release rather than an arbitrary tagged
date.  This empowers vendors and distributors to create their own
service pack style update on a cadence that suits their requirements
and users, rather than feeling tied to cross-vendor schedule or
feeling bad picking interim commits.

The primary push back on this when we started the stable branches was
a vendor wanting to have known release versions for their customers,
and I don't think we have had comment from that (or all) vendors.  I
hope this is seen as a positive thing, as it really is IMO.

I have a question about still having library releases you mentioned,
as generally - everything in python is a library.  I don't think we
have a definition of what in OpenStack is considered a mere library,
compared to a project that wouldn't have a release.

I also wondered if it might make sense for us to do a better job of
storing metadata of what the shasums of projects used to pass gate for
a given commit - as this might be both useful as a known good state
but also, slightly unrelated, might be helpful in debugging gate
blockages in the future.

Thanks

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Matt Riedemann]

On 5/29/2015 10:30 AM, Dave Walker wrote:

On 29 May 2015 at 14:41, Thierry Carrez thie...@openstack.org wrote:

Hi everyone,

TL;DR:
- We propose to stop tagging coordinated point releases (like 2015.1.1)
- We continue maintaining stable branches as a trusted source of stable
updates for all projects though

Long version:

At the stable branch session in Vancouver we discussed recent
evolutions in the stable team processes and how to further adapt the
work of the team in a big tent world.

One of the key questions there was whether we should continue doing
stable point releases. Those were basically tags with the same version
number (2015.1.1) that we would periodically push to the stable
branches for all projects.

Those create three problems.

(1) Projects do not all follow the same versioning, so some projects
(like Swift) were not part of the stable point releases. More and more
projects are considering issuing intermediary releases (like Swift
does), like Ironic. That would result in a variety of version numbers,
and ultimately less and less projects being able to have a common
2015.1.1-like version.

(2) Producing those costs a non-trivial amount of effort on a very small
team of volunteers, especially with projects caring about stable
branches in various amounts. We were constantly missing the
pre-announced dates on those ones. Looks like that effort could be
better spent improving the stable branches themselves and keeping them
working.

(3) The resulting stable point releases are mostly useless. Stable
branches are supposed to be always usable, and the released version
did not undergo significantly more testing. Issuing them actually
discourages people from taking whatever point in stable branches makes
the most sense for them, testing and deploying that.

The suggestion we made during that session (and which was approved by
the session participants) is therefore to just get rid of the stable
point release concept altogether for non-libraries. That said:

- we'd still do individual point releases for libraries (for critical
bugs and security issues), so that you can still depend on a specific
version there

- we'd still very much maintain stable branches (and actually focus our
efforts on that work) to ensure they are a continuous source of safe
upgrades for users of a given series

Now we realize that the cross-section of our community which was present
in that session might not fully represent the consumers of those
artifacts, which is why we expand the discussion on this mailing-list
(and soon on the operators ML).

If you were a consumer of those and will miss them, please explain why.
In particular, please let us know how consuming that version (which was
only made available every n months) is significantly better than picking
your preferred time and get all the current stable branch HEADs at that
time.

Thanks in advance for your feedback,

[1] https://etherpad.openstack.org/p/YVR-relmgt-stable-branch

--
Thierry Carrez (ttx)


This is generally my opinion as-well, I always hoped that *every*
commit would be considered a release rather than an arbitrary tagged
date.  This empowers vendors and distributors to create their own
service pack style update on a cadence that suits their requirements
and users, rather than feeling tied to cross-vendor schedule or
feeling bad picking interim commits.

The primary push back on this when we started the stable branches was
a vendor wanting to have known release versions for their customers,
and I don't think we have had comment from that (or all) vendors.  I
hope this is seen as a positive thing, as it really is IMO.

I have a question about still having library releases you mentioned,
as generally - everything in python is a library.  I don't think we
have a definition of what in OpenStack is considered a mere library,
compared to a project that wouldn't have a release.


A library from an OpenStack POV, from my POV :), is anything that the 
'server' projects, e.g. nova, cinder, keystone, glance, etc, depend on. 
 Primarily the oslo libraries, the clients, and everything they depend on.


It's probably easier to think of it as anything in the 
global-requirements list:


https://github.com/openstack/requirements/blob/master/global-requirements.txt

Note that nova, keystone, glance, cinder, etc aren't in that list.



I also wondered if it might make sense for us to do a better job of
storing metadata of what the shasums of projects used to pass gate for
a given commit - as this might be both useful as a known good state
but also, slightly unrelated, might be helpful in debugging gate
blockages in the future.

Thanks

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



--

Thanks,

Matt 

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Jeremy Stanley]

On 2015-05-29 16:30:12 +0100 (+0100), Dave Walker wrote:
 This is generally my opinion as-well, I always hoped that *every*
 commit would be considered a release rather than an arbitrary
 tagged date.
[...]

If we switch away from lockstep major/minor release versioning
anyway (again separate discussion underway but seems a distinct
possibility) then I think the confusion over why stable point
releases are mismatched becomes less of an issue. At that point we
may want to reconsider and actually tag each of them with a
sequential micro (patch in semver terminology) version bump. Could
help in communication around security fixes in particular.

 I also wondered if it might make sense for us to do a better job of
 storing metadata of what the shasums of projects used to pass gate for
 a given commit - as this might be both useful as a known good state
 but also, slightly unrelated, might be helpful in debugging gate
 blockages in the future.

I think if we get stable branches back into the openstack/openstack
pseudo-repo it might help in this regard. Also Robert's plan for
requirements revamp should make it easier for us to keep track of
what versions of which dependencies were used when testing these.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thierry Carrez]

Neil Jerram wrote:
 
 
 On 29/05/15 14:41, Thierry Carrez wrote:
 Hi everyone,

 TL;DR:
 - We propose to stop tagging coordinated point releases (like 2015.1.1)
 - We continue maintaining stable branches as a trusted source of stable
 updates for all projects though
 [...]
 
 I initially misunderstood your email as saying there will no longer be
 releases at all... but I see now that you are only talking about
 .X.N with N = 1, and that we will continue to have .X.0.  Right?

Yes.

 Will the main releases continue to be tagged as .X.0 ?

That is an orthogonal topic, which warrants its own thread, but we also
discussed that in Vancouver. The general idea there is that since we'll
more and more Swift-like things with their own version, the value of
common release versioning for the rest of them becomes questionable.
What we need is a convenient way of telling which version number is the
kilo release for each project that aligns with the 6-month cycle.

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Matthew Thode]

On 05/29/2015 09:44 AM, Matt Riedemann wrote:
 
 
 On 5/29/2015 8:41 AM, Thierry Carrez wrote:
 Hi everyone,

 TL;DR:
 - We propose to stop tagging coordinated point releases (like 2015.1.1)
 - We continue maintaining stable branches as a trusted source of stable
 updates for all projects though

 Long version:

 At the stable branch session in Vancouver we discussed recent
 evolutions in the stable team processes and how to further adapt the
 work of the team in a big tent world.

 One of the key questions there was whether we should continue doing
 stable point releases. Those were basically tags with the same version
 number (2015.1.1) that we would periodically push to the stable
 branches for all projects.

 Those create three problems.

 (1) Projects do not all follow the same versioning, so some projects
 (like Swift) were not part of the stable point releases. More and more
 projects are considering issuing intermediary releases (like Swift
 does), like Ironic. That would result in a variety of version numbers,
 and ultimately less and less projects being able to have a common
 2015.1.1-like version.

 (2) Producing those costs a non-trivial amount of effort on a very small
 team of volunteers, especially with projects caring about stable
 branches in various amounts. We were constantly missing the
 pre-announced dates on those ones. Looks like that effort could be
 better spent improving the stable branches themselves and keeping them
 working.

 (3) The resulting stable point releases are mostly useless. Stable
 branches are supposed to be always usable, and the released version
 did not undergo significantly more testing. Issuing them actually
 discourages people from taking whatever point in stable branches makes
 the most sense for them, testing and deploying that.

 The suggestion we made during that session (and which was approved by
 the session participants) is therefore to just get rid of the stable
 point release concept altogether for non-libraries. That said:

 - we'd still do individual point releases for libraries (for critical
 bugs and security issues), so that you can still depend on a specific
 version there

 - we'd still very much maintain stable branches (and actually focus our
 efforts on that work) to ensure they are a continuous source of safe
 upgrades for users of a given series

 Now we realize that the cross-section of our community which was present
 in that session might not fully represent the consumers of those
 artifacts, which is why we expand the discussion on this mailing-list
 (and soon on the operators ML).

 If you were a consumer of those and will miss them, please explain why.
 In particular, please let us know how consuming that version (which was
 only made available every n months) is significantly better than picking
 your preferred time and get all the current stable branch HEADs at that
 time.

 Thanks in advance for your feedback,

 [1] https://etherpad.openstack.org/p/YVR-relmgt-stable-branch

 
 To reiterate what I said in the session, for my team personally (IBM),
 we don't align with the point release schedules on stable anyway, we
 release our own stable release fix packs as needed on our own schedules,
 so in that regard I don't see a point in the stable point releases -
 especially since most of the time I don't know when those are going to
 be anyway so we can't plan for them accurately.
 
 Having said that, what I mentioned in IRC the other day is the one
 upside I see to the point releases is it is a milestone that requires
 focus from the stable maintainers, which means if stable has been broken
 for a few weeks and no one has really noticed, converging on a stable
 point release at least forces attention there.
 
 I don't think that is a very good argument for keeping stable point
 releases though, since as you said we don't do any additional testing
 above and beyond what normally happens in the Jenkins runs.  Some of the
 distributions might have extra regression testing scenarios, I'm not
 sure, but no one really spoke to that in the session from the distros
 that were present - I assume they do, but they can do that on their own
 schedule anyway IMO.
 
 I am a bit cynical about thinking that dropping point releases will make
 people spend more time on caring about the health of the stable branches
 (persistent gate failures) or stale changes out for review.  I combed
 through a lot of open stable/icehouse changes yesterday and there were
 many that should have been abandoned 6 months ago but were just sitting
 there, and others that were good fixes to have and should have been
 merged by now.
 
 Personally I've been trying to point out some of these in the
 #openstack-stable IRC channel when I see them so that we don't wait so
 long on these that they fall into a stable support phase where we don't
 think they are appropriate for merging anymore, but if we had acted
 sooner they'd be in.
 
 But I'm also the new guy on the team so 

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Jeremy Stanley]

On 2015-05-29 17:50:01 +0200 (+0200), Ihar Hrachyshka wrote:
[...]
 if we attempt to fix a security issue that has a backwards
 incompatible fix, then we are forced in introducing a new
 configuration option to opt-in the new secure world.
[...]

To my knowledge that's how we've handled these in the past anyway,
accompanied by publication of a security note (not advisory)
suggesting the steps necessary to enable the breaking change when
opting into the bug fix.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev