Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Dave Walker]

On 17 June 2015 at 17:19, Thierry Carrez  wrote:
> Ihar Hrachyshka wrote:
>> On 06/17/2015 05:40 PM, Douglas Mendizábal wrote:
>>> I tend to agree with Thomas that plan D is not ideal.  For one, it
>>> prevents changes to the stable branch that span multiple CRs, since
>>> a two patch change would generate two tags and there would be no
>>> clear indication that the first patch should not be released on its
>>> own.
>>
>> If we will end up with a half-broken product due to merging a patch
>> without another one, then those patches should be squashed. Also, I
>> wonder how they will pass gate if something is broken. Do you suggest
>> that test coverage is incomplete?
>
> Also remember we are talking about *stable branches* here. The only
> acceptable things there are bugfix backports and CVE fixes. So:
>
> - there is no case of partially-merged features that span multiple CRs
> - in the rare case of a bugfix needing multiple commits, those should be
> squashed in the stable branch
> - if the fix is so huge that it can't be merged as a single patch, it's
> probably not desirable in the stable branch
>
> The stable branch should always be usable and releaseable. If it's not,
> you're doing it wrong.
>

Just to add to this, we have a decent demonstrable history of
squashing commits together that have been required to land
concurrently.  We need to do this when the Gate is wedged, and
requires two or more concurrent backports to unwedge.

Such as: https://review.openstack.org/#/c/163378/

As Ihar points out, the test coverage mandates that we do this.. which
nicely helps us provide some assurance that the branch is constantly
releasable.

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thierry Carrez]

Ihar Hrachyshka wrote:
> On 06/17/2015 05:40 PM, Douglas Mendizábal wrote:
>> I tend to agree with Thomas that plan D is not ideal.  For one, it 
>> prevents changes to the stable branch that span multiple CRs, since
>> a two patch change would generate two tags and there would be no
>> clear indication that the first patch should not be released on its
>> own.
> 
> If we will end up with a half-broken product due to merging a patch
> without another one, then those patches should be squashed. Also, I
> wonder how they will pass gate if something is broken. Do you suggest
> that test coverage is incomplete?

Also remember we are talking about *stable branches* here. The only
acceptable things there are bugfix backports and CVE fixes. So:

- there is no case of partially-merged features that span multiple CRs
- in the rare case of a bugfix needing multiple commits, those should be
squashed in the stable branch
- if the fix is so huge that it can't be merged as a single patch, it's
probably not desirable in the stable branch

The stable branch should always be usable and releaseable. If it's not,
you're doing it wrong.

-- 
Thierry Carrez (ttx)



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Ihar Hrachyshka]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 06/17/2015 05:40 PM, Douglas Mendizábal wrote:
> I tend to agree with Thomas that plan D is not ideal.  For one, it 
> prevents changes to the stable branch that span multiple CRs, since
> a two patch change would generate two tags and there would be no
> clear indication that the first patch should not be released on its
> own.

If we will end up with a half-broken product due to merging a patch
without another one, then those patches should be squashed. Also, I
wonder how they will pass gate if something is broken. Do you suggest
that test coverage is incomplete?

> 
> My preference would be for "Plan C" where the PTL would push a new 
> 2015.1.XXX tag when an important fix (or fixes) lands in a stable
> branch .
> 
> I also disagree with Morgan that packagers are better informed to
> make the determination of when to release.  PTLs should be aware of
> all changes landing in the stable branches, and should be able to
> push a tag immediately after an important fix lands.  Asking the
> packagers to make the determination means that they would have to
> be aware of every patch landing in every project, which I think is
> a lot to ask.

I wouldn't expect e.g. neutron PTL to know every since patch in stable
branches. If anyone, you better put the burden onto corresponding
- -stable-maint team.

> 
> - Douglas Mendizábal
> 
> On 6/17/15 9:55 AM, Morgan Fainberg wrote:
> 
>>> On Jun 16, 2015, at 14:56, Thomas Goirand  
>>> wrote:
>>> 
>>> On 06/16/2015 12:06 PM, Thierry Carrez wrote:
>> It also removes the stupid encouragement to use all 
>> components from the same date. With everything tagged at 
>> the same date, you kinda send the message that those 
>> various things should be used together. With everything 
>> tagged separately, you send te message that you can mix 
>> and match components from stable/* as you see fit. I
>> mean, it's totally valid to use stable branch components
>> from various points in time together, since they are all 
>> supposed to work.
> 
> Though there's now zero guidance at what should be the
> speed of releasing server packages to our users.
 
 I really think it should be a distribution decision. You
 could release all commits, release every 2 months, release
 after each CVE, release as-needed when a bug in Debian BTS is
 fixed. I don't see what "guidance" upstream should give,
 apart from enabling all models. Currently we make most models
 more difficult than they should be, to promote an arbitrary 
 time-based model. With plan D, we enable all models.
>>> 
>>> Let me put this in another way: with the plan D, I'll be lost, 
>>> and wont ever know when to release a new stable version in 
>>> Debian. I don't know better than anyone else. If we had each 
>>> upstream project saying individually: "Ok, now we gathered
>>> enough bugfixes so that it's important to get it in downstream 
>>> distributions", I'd happily follow this kind of guidance. But
>>> the plan is to just commit bugfixes, and hope that downstream
>>> distros (ie: me in this case) just catch when a new release
>>> worse the effort.
>>> 
 As pointed elsewhere, plan D assumes we move to generating 
 release notes for each commit. So you won't lose track of
 what is fixed in each version. If anything, that will give
 you proper release notes for CVE-fix commits, something you
 didn't have before, since we wouldn't cut a proper point
 release after a CVE fix but on a pre-determined time-based
 schedule.
 
 Overall, I think even your process stands to benefit from
 the proposed evolution.
>>> 
>>> I just hope so. If any core / PTL is reading me in this thread,
>>> I would strongly encourage you guys to get in touch and ping
>>> me when you think some commits in the stable release should be 
>>> uploaded to Debian. A quick message on IRC can be enough.
>>> 
> 
>> For what it is worth, I trust the downstream distributions to
>> make this call. I expect that any/all stable patches accepted in
>> a model where release notes are generated per-commit (Plan D)
>> this ends up looking like the Redhat model where patches are
>> bundled in.
> 
>> In general we should have care in accepting a stable patch. But
>> as the PTL (and more generally as a core) asking me to determine
>> when there is "enough patches to generate a release" is far too 
>> distribution/packager specific and subjective. I do not expect
>> to be reaching out to each and every one to say "hey did you
>> release?" This, in my opinion is not a reasonable ask. I do not
>> know what justifies "time for a release" for Debian or Redhat or
>> Suse or Gentoo ... Etc. Please do not expect the PTLs and Cores
>> to become experts on each and every one of these. I expect the
>> packager to be making the subjective call in the non-time-based
>> model outlined.
> 
>> --Morgan 
>> ___

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Douglas Mendizábal]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

I tend to agree with Thomas that plan D is not ideal.  For one, it
prevents changes to the stable branch that span multiple CRs, since a
two patch change would generate two tags and there would be no clear
indication that the first patch should not be released on its own.

My preference would be for "Plan C" where the PTL would push a new
2015.1.XXX tag when an important fix (or fixes) lands in a stable branch
.

I also disagree with Morgan that packagers are better informed to make
the determination of when to release.  PTLs should be aware of all
changes landing in the stable branches, and should be able to push a
tag immediately after an important fix lands.  Asking the packagers to
make the determination means that they would have to be aware of every
patch landing in every project, which I think is a lot to ask.

- - Douglas Mendizábal

On 6/17/15 9:55 AM, Morgan Fainberg wrote:
> 
>> On Jun 16, 2015, at 14:56, Thomas Goirand 
>> wrote:
>> 
>> On 06/16/2015 12:06 PM, Thierry Carrez wrote:
> It also removes the stupid encouragement to use all
> components from the same date. With everything tagged at
> the same date, you kinda send the message that those
> various things should be used together. With everything
> tagged separately, you send te message that you can mix
> and match components from stable/* as you see fit. I mean,
> it's totally valid to use stable branch components from
> various points in time together, since they are all
> supposed to work.
 
 Though there's now zero guidance at what should be the speed
 of releasing server packages to our users.
>>> 
>>> I really think it should be a distribution decision. You could
>>> release all commits, release every 2 months, release after each
>>> CVE, release as-needed when a bug in Debian BTS is fixed. I
>>> don't see what "guidance" upstream should give, apart from
>>> enabling all models. Currently we make most models more
>>> difficult than they should be, to promote an arbitrary 
>>> time-based model. With plan D, we enable all models.
>> 
>> Let me put this in another way: with the plan D, I'll be lost,
>> and wont ever know when to release a new stable version in
>> Debian. I don't know better than anyone else. If we had each
>> upstream project saying individually: "Ok, now we gathered enough
>> bugfixes so that it's important to get it in downstream
>> distributions", I'd happily follow this kind of guidance. But the
>> plan is to just commit bugfixes, and hope that downstream distros
>> (ie: me in this case) just catch when a new release worse the
>> effort.
>> 
>>> As pointed elsewhere, plan D assumes we move to generating
>>> release notes for each commit. So you won't lose track of what
>>> is fixed in each version. If anything, that will give you
>>> proper release notes for CVE-fix commits, something you didn't
>>> have before, since we wouldn't cut a proper point release after
>>> a CVE fix but on a pre-determined time-based schedule.
>>> 
>>> Overall, I think even your process stands to benefit from the
>>> proposed evolution.
>> 
>> I just hope so. If any core / PTL is reading me in this thread, I
>> would strongly encourage you guys to get in touch and ping me
>> when you think some commits in the stable release should be
>> uploaded to Debian. A quick message on IRC can be enough.
>> 
> 
> For what it is worth, I trust the downstream distributions to make
> this call. I expect that any/all stable patches accepted in a model
> where release notes are generated per-commit (Plan D) this ends up
> looking like the Redhat model where patches are bundled in.
> 
> In general we should have care in accepting a stable patch. But as
> the PTL (and more generally as a core) asking me to determine when
> there is "enough patches to generate a release" is far too
> distribution/packager specific and subjective. I do not expect to
> be reaching out to each and every one to say "hey did you release?"
> This, in my opinion is not a reasonable ask. I do not know what
> justifies "time for a release" for Debian or Redhat or Suse or
> Gentoo ... Etc. Please do not expect the PTLs and Cores to become
> experts on each and every one of these. I expect the packager to be
> making the subjective call in the non-time-based model outlined.
> 
> --Morgan 
> __

>
> 
OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe 
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
-BEGIN PGP SIGNATURE-
Comment: GPGTools - https://gpgtools.org

iQIcBAEBCgAGBQJVgZTgAAoJEB7Z2EQgmLX7qhkP/RwB4sJU2nXeAe4M27VXT/9t
tPD3j0njOMuL6T5g41D/aXIcEvvk1RqjJOg+VdXq7Os3t0AAHNkKng85qGKp+Our
xMOSGBtFjqQycDDVEFrltYCkhmDdmsVw4SWd0zmJ2MHzn+R014lZL5SdsHjhWmRr
c3NfZH0Cxm9ujfMbdjsZoF3i6noZsotUZLy5Tu5iW/Kp46syN

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Morgan Fainberg]

> On Jun 16, 2015, at 14:56, Thomas Goirand  wrote:
> 
> On 06/16/2015 12:06 PM, Thierry Carrez wrote:
 It also removes the stupid encouragement to use all components from the
 same date. With everything tagged at the same date, you kinda send the
 message that those various things should be used together. With
 everything tagged separately, you send te message that you can mix and
 match components from stable/* as you see fit. I mean, it's totally
 valid to use stable branch components from various points in time
 together, since they are all supposed to work.
>>> 
>>> Though there's now zero guidance at what should be the speed of
>>> releasing server packages to our users.
>> 
>> I really think it should be a distribution decision. You could release
>> all commits, release every 2 months, release after each CVE, release
>> as-needed when a bug in Debian BTS is fixed. I don't see what "guidance"
>> upstream should give, apart from enabling all models. Currently we make
>> most models more difficult than they should be, to promote an arbitrary
>> time-based model. With plan D, we enable all models.
> 
> Let me put this in another way: with the plan D, I'll be lost, and wont
> ever know when to release a new stable version in Debian. I don't know
> better than anyone else. If we had each upstream project saying
> individually: "Ok, now we gathered enough bugfixes so that it's
> important to get it in downstream distributions", I'd happily follow
> this kind of guidance. But the plan is to just commit bugfixes, and hope
> that downstream distros (ie: me in this case) just catch when a new
> release worse the effort.
> 
>> As pointed elsewhere, plan D assumes we move to generating release notes
>> for each commit. So you won't lose track of what is fixed in each
>> version. If anything, that will give you proper release notes for
>> CVE-fix commits, something you didn't have before, since we wouldn't cut
>> a proper point release after a CVE fix but on a pre-determined
>> time-based schedule.
>> 
>> Overall, I think even your process stands to benefit from the proposed
>> evolution.
> 
> I just hope so. If any core / PTL is reading me in this thread, I would
> strongly encourage you guys to get in touch and ping me when you think
> some commits in the stable release should be uploaded to Debian. A quick
> message on IRC can be enough.
> 

For what it is worth, I trust the downstream distributions to make this call. I 
expect that any/all stable patches accepted in a model where release notes are 
generated per-commit (Plan D) this ends up looking like the Redhat model where 
patches are bundled in. 

In general we should have care in accepting a stable patch. But as the PTL (and 
more generally as a core) asking me to determine when there is "enough patches 
to generate a release" is far too distribution/packager specific and 
subjective. I do not expect to be reaching out to each and every one to say 
"hey did you release?" This, in my opinion is not a reasonable ask. I do not 
know what justifies "time for a release" for Debian or Redhat or Suse or Gentoo 
... Etc. Please do not expect the PTLs and Cores to become experts on each and 
every one of these. I expect the packager to be making the subjective call in 
the non-time-based model outlined. 

--Morgan
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/16/2015 12:06 PM, Thierry Carrez wrote:
>>> It also removes the stupid encouragement to use all components from the
>>> same date. With everything tagged at the same date, you kinda send the
>>> message that those various things should be used together. With
>>> everything tagged separately, you send te message that you can mix and
>>> match components from stable/* as you see fit. I mean, it's totally
>>> valid to use stable branch components from various points in time
>>> together, since they are all supposed to work.
>>
>> Though there's now zero guidance at what should be the speed of
>> releasing server packages to our users.
> 
> I really think it should be a distribution decision. You could release
> all commits, release every 2 months, release after each CVE, release
> as-needed when a bug in Debian BTS is fixed. I don't see what "guidance"
> upstream should give, apart from enabling all models. Currently we make
> most models more difficult than they should be, to promote an arbitrary
> time-based model. With plan D, we enable all models.

Let me put this in another way: with the plan D, I'll be lost, and wont
ever know when to release a new stable version in Debian. I don't know
better than anyone else. If we had each upstream project saying
individually: "Ok, now we gathered enough bugfixes so that it's
important to get it in downstream distributions", I'd happily follow
this kind of guidance. But the plan is to just commit bugfixes, and hope
that downstream distros (ie: me in this case) just catch when a new
release worse the effort.

> As pointed elsewhere, plan D assumes we move to generating release notes
> for each commit. So you won't lose track of what is fixed in each
> version. If anything, that will give you proper release notes for
> CVE-fix commits, something you didn't have before, since we wouldn't cut
> a proper point release after a CVE fix but on a pre-determined
> time-based schedule.
> 
> Overall, I think even your process stands to benefit from the proposed
> evolution.

I just hope so. If any core / PTL is reading me in this thread, I would
strongly encourage you guys to get in touch and ping me when you think
some commits in the stable release should be uploaded to Debian. A quick
message on IRC can be enough.

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thierry Carrez]

Thomas Goirand wrote:
> On 06/10/2015 03:46 PM, Thierry Carrez wrote:
>> So we could do what you're asking (option B) for Kilo stable releases,
>> but I think it's not really a viable option for stable/liberty onward.
> 
> I fail to understand how version numbers are related to doing
> synchronized point releases. Sure, we have a problem with the version
> numbers, but we can still do a point release of all the server projects
> at once.

Agreed. We could still do a "coordinated point release" without using
the same version numbers. We'd call it the kilo.1 release and it would
translate into different point release version numbers for each project.
That could even be done on top of plan D by a group of people caring
enough to create a wiki page to reference the corresponding version numbers.

> FYI, after a CVE, I have uploaded cinder with this version number:
> 2015.1.0+2015.06.06.git24.493b6c7f12-1
> 
> IMO, it looks ugly, and not comprehensive to Debian users. If I really
> have to go this way, I will, but I would have  not to.

Like Alan said, with plan D this actually looks a lot better. You'd be
able to package/distribute 2015.1.25 instead of
2015.1.0+2015.06.06.git24.493b6c7f12-1

>> It also removes the stupid encouragement to use all components from the
>> same date. With everything tagged at the same date, you kinda send the
>> message that those various things should be used together. With
>> everything tagged separately, you send te message that you can mix and
>> match components from stable/* as you see fit. I mean, it's totally
>> valid to use stable branch components from various points in time
>> together, since they are all supposed to work.
> 
> Though there's now zero guidance at what should be the speed of
> releasing server packages to our users.

I really think it should be a distribution decision. You could release
all commits, release every 2 months, release after each CVE, release
as-needed when a bug in Debian BTS is fixed. I don't see what "guidance"
upstream should give, apart from enabling all models. Currently we make
most models more difficult than they should be, to promote an arbitrary
time-based model. With plan D, we enable all models.

>> So I totally get that we should still have reference points to be able
>> to tell "this is fixed in openstack/nova stable/liberty starting with
>> 12.1.134post34" (or whatever we settle with). I totally get that any of
>> those should ship with relevant release notes. But that's about all I
>> think we need ?
> 
> That's not a little thing that you're pointing at: having reference
> points is very important. But yeah, it may be the "only" thing... Like
> we will "only" loose track of what is fixed in which version. :(

As pointed elsewhere, plan D assumes we move to generating release notes
for each commit. So you won't lose track of what is fixed in each
version. If anything, that will give you proper release notes for
CVE-fix commits, something you didn't have before, since we wouldn't cut
a proper point release after a CVE fix but on a pre-determined
time-based schedule.

Overall, I think even your process stands to benefit from the proposed
evolution.

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Ihar Hrachyshka]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 06/15/2015 11:24 PM, Thomas Goirand wrote:
> On 06/15/2015 05:19 PM, Ian Cordasco wrote:
>> On 6/15/15, 09:24, "Thomas Goirand"  wrote:
>> 
>>> On 06/08/2015 01:55 PM, Kuvaja, Erno wrote:
 One thing I like about plan D is that it would give also
 indicator how much the stable branch has moved in each
 individual project.
>>> 
>>> The only indication you will get is how many patches it has. I
>>> fail to see how this is valuable information. No info on how
>>> important they are or anything of this kind, which is a way
>>> more important.
>>> 
>>> Thomas
>> 
>> Are you implying that stable point releases as they exist today
>> provide importance? How is that case?
> 
> I'm just replying to Kuvaja, Erno, who pretends that plan D will
> give "indicator how much the stable branch has moved in". My answer
> is: not it wont. Maybe there was lots of cosmetic patches (like
> typos, man page, or who knows...). The number of patch is an
> indication of ... nothing, really!
> 
> The only way to have indications is release notes. Since it looks
> like there's not enough resources to write them, we're giving up on
> them. It's really a shame, because I'm sure it was valuable for our
> users.
> 

Proper estimation of available resources is not a shame, it's realistic.

If you have resources to spare on actually maintaining stable branches
and release process around them, you're always welcome to join the
effort. For now, I see you are comfortable in your position of a
consumer of deliverables. That's fine, but also keep in mind you get
what you pay.

Ihar
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQEcBAEBCAAGBQJVf+VtAAoJEC5aWaUY1u57/AYIAOquvtP3wxhrXuZfWquqWedq
oPCkrJVSMlKpW3WUJr7/3xqoDzVen4UT/0Du0T+uE6cTBTlt3Fqt1cEh4iQ1WOcA
qBfFJlX7HHpwkJnjF5MJBiTCiDjegrQFTRPOb/O46hSE+a1JUNPH9ob1OiKpyk39
oSGSSCI2YmS5IIpiUNDsYzRK6twVZsCut+18AXp89Hx8oK8t7Lu3jfrEq8Z5R5Hv
xnea0AV+/WZLchUPqGss5htOhqVkXH6A3W9Wvg10427tPCVin/5SIcfCvD/2zZ0a
V9/ikUphtUq0xV2zh2Uc17Tkkub8hKD21u+v7XRiy8RijPr4ouct77gOLpP5JuU=
=F0Vb
-END PGP SIGNATURE-

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Alan Pevec]

> The only issue is having a correct version number. Swift could well
> iterate after "20151", for example doing "20151.1" then "20151.2", etc.

Swift never had "coordinated" YEAR.N versions,
https://wiki.openstack.org/wiki/Swift/version_map
and they never released a version from their stable/* branches.
e.g. 2.2.1 and 2.2.2 are in Launchpad Series kilo
https://launchpad.net/swift/kilo
and were created from master branch, not stable/juno as one would expect.
I heard Swift team call them "Juno version released during Kilo cycle"
which confused me a lot but if you take into account that Swift master
is stable and always backward compatible, it does make sense.
In short, Swift is different enough so best not to force them into
coordinated point releases which is what we did.
And when you think of it more, Swift model should be the goal for all
OpenStack project: stable master, always backward compatbile, no
upgrade issues => no need for stable/* branches :)

>> So we could do what you're asking (option B) for Kilo stable releases,
>> but I think it's not really a viable option for stable/liberty onward.
>
> I fail to understand how version numbers are related to doing
> synchronized point releases. Sure, we have a problem with the version
> numbers, but we can still do a point release of all the server projects
> at once.

I don't see what value does the same tag across certain set of projects bring?
Taking it to the extreme, why don't you put the same version across
all packages in a distro?
Reality is that each openstack is independent and compatibility among
them is ensured by CI not by using the coordinated version tags.

> FYI, after a CVE, I have uploaded cinder with this version number:
> 2015.1.0+2015.06.06.git24.493b6c7f12-1

And with modified[*] plan D it would be 2015.1.25

> IMO, it looks ugly, and not comprehensive to Debian users. If I really
> have to go this way, I will, but I would have  not to.

You don't, if you'd support planD :)

> Though there's now zero guidance at what should be the speed of
> releasing server packages to our users.

Why there should be any guidance from upstream? Let your users tell
you and with planD you can update whenever requested fix is merged on
stable!

>> So I totally get that we should still have reference points to be able
>> to tell "this is fixed in openstack/nova stable/liberty starting with
>> 12.1.134post34" (or whatever we settle with). I totally get that any of
>> those should ship with relevant release notes. But that's about all I
>> think we need ?
>
> That's not a little thing that you're pointing at: having reference
> points is very important. But yeah, it may be the "only" thing... Like
> we will "only" loose track of what is fixed in which version. :(

On the contrary, with planD you'd have reference points and lots of them :)
And release notes would be generated from commit messages - this
requires better commit messages on stable branches but that's always
good. Here's how it would look like for current Cinder stable/kilo:
CHANGES
===

2015.1.25
-

* Disallow backing files when uploading volumes to image

2015.1.24
-

* Dispose DB connections between backend proc starts

2015.1.23
-

* Allow provisioning to reach max oversubscription
...

Cheers,
Alan


[*] modified plan D: automatically sign and push git tag after each
commit on stable branch. We get both, tags and tarballs in one go.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Ian Cordasco]

On 6/15/15, 16:24, "Thomas Goirand"  wrote:

>On 06/15/2015 05:19 PM, Ian Cordasco wrote:
>> On 6/15/15, 09:24, "Thomas Goirand"  wrote:
>> 
>>> On 06/08/2015 01:55 PM, Kuvaja, Erno wrote:
 One thing I like about plan D
 is that it would give also indicator how much the stable branch has
 moved in
 each individual project.
>>>
>>> The only indication you will get is how many patches it has. I fail to
>>> see how this is valuable information. No info on how important they are
>>> or anything of this kind, which is a way more important.
>>>
>>> Thomas
>> 
>> Are you implying that stable point releases as they exist today provide
>> importance? How is that case?
>
>I'm just replying to Kuvaja, Erno, who pretends that plan D will give
>"indicator how much the stable branch has moved in". My answer is: not
>it wont. Maybe there was lots of cosmetic patches (like typos, man page,
>or who knows...). The number of patch is an indication of ... nothing,
>really!
>
>The only way to have indications is release notes. Since it looks like
>there's not enough resources to write them, we're giving up on them.
>It's really a shame, because I'm sure it was valuable for our users.
>
>Thomas

I would suggest you read the section of the wiki that describes what is an
appropriate commit to land on a stable branch:
https://wiki.openstack.org/wiki/StableBranch#Appropriate_Fixes

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/15/2015 05:19 PM, Ian Cordasco wrote:
> On 6/15/15, 09:24, "Thomas Goirand"  wrote:
> 
>> On 06/08/2015 01:55 PM, Kuvaja, Erno wrote:
>>> One thing I like about plan D
>>> is that it would give also indicator how much the stable branch has
>>> moved in
>>> each individual project.
>>
>> The only indication you will get is how many patches it has. I fail to
>> see how this is valuable information. No info on how important they are
>> or anything of this kind, which is a way more important.
>>
>> Thomas
> 
> Are you implying that stable point releases as they exist today provide
> importance? How is that case?

I'm just replying to Kuvaja, Erno, who pretends that plan D will give
"indicator how much the stable branch has moved in". My answer is: not
it wont. Maybe there was lots of cosmetic patches (like typos, man page,
or who knows...). The number of patch is an indication of ... nothing,
really!

The only way to have indications is release notes. Since it looks like
there's not enough resources to write them, we're giving up on them.
It's really a shame, because I'm sure it was valuable for our users.

Thomas


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Ian Cordasco]

On 6/15/15, 09:24, "Thomas Goirand"  wrote:

>On 06/08/2015 01:55 PM, Kuvaja, Erno wrote:
>> One thing I like about plan D
>> is that it would give also indicator how much the stable branch has
>>moved in
>> each individual project.
>
>The only indication you will get is how many patches it has. I fail to
>see how this is valuable information. No info on how important they are
>or anything of this kind, which is a way more important.
>
>Thomas

Are you implying that stable point releases as they exist today provide
importance? How is that case? They're coordinated to happen at (nearly)
the same time and that's about all. Perhaps the most important changes are
CVE fixes. Let's look at a two cases for a stable point release now:

1. A point release without a CVE fix
2. A point release with a CVE fix (or more than one)

In the first case, how does a tagged version provide information about
importance? A release would have been tagged whether the latest commit (or
N commits) had been merged or not. In the second case, downstream
redistributors (or at least Debian) has already shipped a new version with
the fix. The importance of that CVE fix being included in a tag that was
created arbitrarily is then different than the importance it might have if
Debian didn't patch the existing versions. (Note, I'm not advocating you
change this practice.) I don't see how tags detail the importance of the
included commits any more than their existence on a stable branch.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/08/2015 01:55 PM, Kuvaja, Erno wrote:
> One thing I like about plan D
> is that it would give also indicator how much the stable branch has moved in
> each individual project. 

The only indication you will get is how many patches it has. I fail to
see how this is valuable information. No info on how important they are
or anything of this kind, which is a way more important.

Thomas


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/10/2015 03:46 PM, Thierry Carrez wrote:
> The main issue with B is that it doesn't work well once server component
> versions start to diverge, which will be the case starting with Liberty.

Review that policy then.

> We already couldn't (with Swift using separate versioning), but we worked
> around that by ignoring Swift from stable releases. As more projects opt
> into that, that will no longer be a possible workaround.

The only issue is having a correct version number. Swift could well
iterate after "20151", for example doing "20151.1" then "20151.2", etc.

> So we could do what you're asking (option B) for Kilo stable releases,
> but I think it's not really a viable option for stable/liberty onward.

I fail to understand how version numbers are related to doing
synchronized point releases. Sure, we have a problem with the version
numbers, but we can still do a point release of all the server projects
at once.

>> This is really one of the things I think we want to get away from...
>> If *every* stable commit is treated with the seriousness of it
>> creating a release, lets make every commit a release.
>>
>> This means that Debian may be using a (micro)patch release newer or
>> older than a different distro, but the key is that it empowers the
>> vendors and/or users to select a release cadence that best fits them,
>> rather than being tied to an arbitrary upstream community wide date.
> 
> +1

FYI, after a CVE, I have uploaded cinder with this version number:
2015.1.0+2015.06.06.git24.493b6c7f12-1

IMO, it looks ugly, and not comprehensive to Debian users. If I really
have to go this way, I will, but I would have  not to.

> It also removes the stupid encouragement to use all components from the
> same date. With everything tagged at the same date, you kinda send the
> message that those various things should be used together. With
> everything tagged separately, you send te message that you can mix and
> match components from stable/* as you see fit. I mean, it's totally
> valid to use stable branch components from various points in time
> together, since they are all supposed to work.

Though there's now zero guidance at what should be the speed of
releasing server packages to our users.

> So I totally get that we should still have reference points to be able
> to tell "this is fixed in openstack/nova stable/liberty starting with
> 12.1.134post34" (or whatever we settle with). I totally get that any of
> those should ship with relevant release notes. But that's about all I
> think we need ?

That's not a little thing that you're pointing at: having reference
points is very important. But yeah, it may be the "only" thing... Like
we will "only" loose track of what is fixed in which version. :(

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/10/2015 11:27 AM, Dave Walker wrote:
> On 10 June 2015 at 09:53, Thomas Goirand  wrote:
>> On 06/05/2015 02:46 PM, Thierry Carrez wrote:
>>> So.. summarizing the various options again:
>>>
>>> Plan A
>>> Just drop stable point releases.
>>> (-) No more release notes
>>> (-) Lack of reference points to compare installations
>>>
>>> Plan B
>>> Push date-based tags across supported projects from time to time.
>>> (-) Encourages to continue using same version across the board
>>> (-) Almost as much work as making proper releases
>>>
>>> Plan C
>>> Let projects randomly tag point releases whenever
>>> (-) Still a bit costly in terms of herding cats
>>>
>>> Plan D
>>> Drop stable point releases, publish per-commit tarballs
>>> (-) Requires some infra changes, takes some storage space
>>>
>>> Plans B, C and D also require some release note / changelog generation
>>> from data maintained *within* the repository.
>>>
>>> Personally I think the objections raised against plan A are valid. I
>>> like plan D, since it's more like releasing every commit than "not
>>> releasing anymore". I think it's the most honest trade-off. I could go
>>> with plan C, but I think it's added work for no additional value to the
>>> user.
>>>
>>> What would be your preferred option ?
>>
>> I see no point of doing D. I already don't use tarballs, and those who
>> do could as well switch to generating them (how hard is it to run
>> "python setup.py sdist" or "git archive"?).
>>
>> What counts is having a schedule date, where all distros are releasing a
>> point release, so we have a common reference point. If that is a fully
>> automated process, then great, less work for everyone, and it wont
>> change anything from what we had in the past (we can even collectively
>> decide for point release dates...).
>>
>> Cheers,
>>
>> Thomas Goirand (zigo)
> 
> This is really one of the things I think we want to get away from...
> If *every* stable commit is treated with the seriousness of it
> creating a release, lets make every commit a release.
> 
> This means that Debian may be using a (micro)patch release newer or
> older than a different distro, but the key is that it empowers the
> vendors and/or users to select a release cadence that best fits them,
> rather than being tied to an arbitrary upstream community wide date.

What you don't get here is that downstream distributions *DO* want the
"arbitrary upstream community wide date", and don't want to just use any
commit.

> Yes, this might mean that your cadence might be more or less regular
> than an alternative vendor / distribution, but the key is that it
> empowers the vendor to meet the needs of their users/customers.

When did a distribution vendors express this as an issue? Having
community-wide release dates doesn't prevent any vendor to do a patch
release if he wants to.

> For example, you could select a cadence of rebasing to a release every
> 6 months - where as another consumer could choose to do one every 6
> weeks.

Which is what would be nice to avoid, so we have the same code base.
Otherwise we may be affected differently by a CVE.

> The difference is how much of a jump, at which intervals..
> Alternatively, a vendor might choose just to go with stock release +
> their own select cherry picked patches from stable/*, which is also a
> model that works.

This already happens, we don't need to remove point releases to do that.

> The issue around producing tarballs, is really about having forwards
> and backwards verification by means of sha/md5 sums, which is hard to
> do when generating your own orig tarball.

Opposite way. When generating my tarball, I do it with a GPG signed tag.
This is verifiable very easily. By the way, sha & md5 are in no way
tools to sign a release, for that, we have PGP (and cross-signing of keys).

> Debian, Ubuntu and I
> believe Arch have made varying use of 'pristine-tar' - which was an
> effort to re-producible tarballs using xdelta to make the sum match.
> However, Maintainers seem to be moving away from this now.

As much as I know, I'm the only one using "git archive ... | xz ..." to
generate my own tarballs, but maybe this will change some day.

> When I perform source NEW reviews for Ubuntu Archive, I always check
> that getting the source orig tarball can be done with either
> get-orig-source (inspecting the generation method) or uscan and then
> diff the tarballs with the one included on the upload and the one
> generated.  Timestamps (or even shasums) haven't been an important
> issue for me, but the actual content and verifiable source is what has
> mattered more.

Correct. Though for me, a signed git tag is a way better than any md5 in
the world.

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/ope

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Fox, Kevin M]

Hi Dave,

Ah, I did misunderstand. Thanks for clarifying.

Kevin

From: Dave Walker [em...@daviey.com]
Sent: Wednesday, June 10, 2015 9:32 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [all] [stable] No longer doing stable point
releases

On 10 June 2015 at 17:06, Fox, Kevin M  wrote:
> So... as an op, without release notes, how am I supposed to figure out the 
> proper upgrade procedure's when you often have to lockstep, in the right 
> order, nova+neutron upgrades (or other project combinations)?
>
> Thanks,
> Kevin

Hi Kevin,

I suspect there is some confusion here between stable point releases
and major releases.  The major releases are still continuing as
expected with rich release notes.  This is talking about stable patch
releases such as:

https://wiki.openstack.org/wiki/ReleaseNotes/2014.2.3

As you can see there is only notices in "Known Issues and Limitations"
section of low impact.  I do not believe we've ever required ordering
in the updates of these, as they are supposed to be pretty
conservative changes that shouldn't enforce limitations like that.

HTH

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Dave Walker]

On 10 June 2015 at 17:06, Fox, Kevin M  wrote:
> So... as an op, without release notes, how am I supposed to figure out the 
> proper upgrade procedure's when you often have to lockstep, in the right 
> order, nova+neutron upgrades (or other project combinations)?
>
> Thanks,
> Kevin

Hi Kevin,

I suspect there is some confusion here between stable point releases
and major releases.  The major releases are still continuing as
expected with rich release notes.  This is talking about stable patch
releases such as:

https://wiki.openstack.org/wiki/ReleaseNotes/2014.2.3

As you can see there is only notices in "Known Issues and Limitations"
section of low impact.  I do not believe we've ever required ordering
in the updates of these, as they are supposed to be pretty
conservative changes that shouldn't enforce limitations like that.

HTH

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Fox, Kevin M]

So... as an op, without release notes, how am I supposed to figure out the 
proper upgrade procedure's when you often have to lockstep, in the right order, 
nova+neutron upgrades (or other project combinations)?

Thanks,
Kevin

From: Thomas Goirand [z...@debian.org]
Sent: Wednesday, June 10, 2015 1:53 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [all] [stable] No longer doing stable point
releases

On 06/05/2015 02:46 PM, Thierry Carrez wrote:
> So.. summarizing the various options again:
>
> Plan A
> Just drop stable point releases.
> (-) No more release notes
> (-) Lack of reference points to compare installations
>
> Plan B
> Push date-based tags across supported projects from time to time.
> (-) Encourages to continue using same version across the board
> (-) Almost as much work as making proper releases
>
> Plan C
> Let projects randomly tag point releases whenever
> (-) Still a bit costly in terms of herding cats
>
> Plan D
> Drop stable point releases, publish per-commit tarballs
> (-) Requires some infra changes, takes some storage space
>
> Plans B, C and D also require some release note / changelog generation
> from data maintained *within* the repository.
>
> Personally I think the objections raised against plan A are valid. I
> like plan D, since it's more like releasing every commit than "not
> releasing anymore". I think it's the most honest trade-off. I could go
> with plan C, but I think it's added work for no additional value to the
> user.
>
> What would be your preferred option ?

I see no point of doing D. I already don't use tarballs, and those who
do could as well switch to generating them (how hard is it to run
"python setup.py sdist" or "git archive"?).

What counts is having a schedule date, where all distros are releasing a
point release, so we have a common reference point. If that is a fully
automated process, then great, less work for everyone, and it wont
change anything from what we had in the past (we can even collectively
decide for point release dates...).

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thierry Carrez]

Dave Walker wrote:
> On 10 June 2015 at 09:53, Thomas Goirand  wrote:
>>> What would be your preferred option ?
>>
>> I see no point of doing D. I already don't use tarballs, and those who
>> do could as well switch to generating them (how hard is it to run
>> "python setup.py sdist" or "git archive"?).
>>
>> What counts is having a schedule date, where all distros are releasing a
>> point release, so we have a common reference point. If that is a fully
>> automated process, then great, less work for everyone, and it wont
>> change anything from what we had in the past (we can even collectively
>> decide for point release dates...).

That would be option B.

The main issue with B is that it doesn't work well once server component
versions start to diverge, which will be the case starting with Liberty.
You can't tag everything with the same version number anymore. We
already couldn't (with Swift using separate versioning), but we worked
around that by ignoring Swift from stable releases. As more projects opt
into that, that will no longer be a possible workaround.

So we could do what you're asking (option B) for Kilo stable releases,
but I think it's not really a viable option for stable/liberty onward.

> This is really one of the things I think we want to get away from...
> If *every* stable commit is treated with the seriousness of it
> creating a release, lets make every commit a release.
> 
> This means that Debian may be using a (micro)patch release newer or
> older than a different distro, but the key is that it empowers the
> vendors and/or users to select a release cadence that best fits them,
> rather than being tied to an arbitrary upstream community wide date.

+1

It also removes the stupid encouragement to use all components from the
same date. With everything tagged at the same date, you kinda send the
message that those various things should be used together. With
everything tagged separately, you send te message that you can mix and
match components from stable/* as you see fit. I mean, it's totally
valid to use stable branch components from various points in time
together, since they are all supposed to work.

So I totally get that we should still have reference points to be able
to tell "this is fixed in openstack/nova stable/liberty starting with
12.1.134post34" (or whatever we settle with). I totally get that any of
those should ship with relevant release notes. But that's about all I
think we need ?

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Dave Walker]

On 10 June 2015 at 09:53, Thomas Goirand  wrote:
> On 06/05/2015 02:46 PM, Thierry Carrez wrote:
>> So.. summarizing the various options again:
>>
>> Plan A
>> Just drop stable point releases.
>> (-) No more release notes
>> (-) Lack of reference points to compare installations
>>
>> Plan B
>> Push date-based tags across supported projects from time to time.
>> (-) Encourages to continue using same version across the board
>> (-) Almost as much work as making proper releases
>>
>> Plan C
>> Let projects randomly tag point releases whenever
>> (-) Still a bit costly in terms of herding cats
>>
>> Plan D
>> Drop stable point releases, publish per-commit tarballs
>> (-) Requires some infra changes, takes some storage space
>>
>> Plans B, C and D also require some release note / changelog generation
>> from data maintained *within* the repository.
>>
>> Personally I think the objections raised against plan A are valid. I
>> like plan D, since it's more like releasing every commit than "not
>> releasing anymore". I think it's the most honest trade-off. I could go
>> with plan C, but I think it's added work for no additional value to the
>> user.
>>
>> What would be your preferred option ?
>
> I see no point of doing D. I already don't use tarballs, and those who
> do could as well switch to generating them (how hard is it to run
> "python setup.py sdist" or "git archive"?).
>
> What counts is having a schedule date, where all distros are releasing a
> point release, so we have a common reference point. If that is a fully
> automated process, then great, less work for everyone, and it wont
> change anything from what we had in the past (we can even collectively
> decide for point release dates...).
>
> Cheers,
>
> Thomas Goirand (zigo)

This is really one of the things I think we want to get away from...
If *every* stable commit is treated with the seriousness of it
creating a release, lets make every commit a release.

This means that Debian may be using a (micro)patch release newer or
older than a different distro, but the key is that it empowers the
vendors and/or users to select a release cadence that best fits them,
rather than being tied to an arbitrary upstream community wide date.

Yes, this might mean that your cadence might be more or less regular
than an alternative vendor / distribution, but the key is that it
empowers the vendor to meet the needs of their users/customers.

For example, you could select a cadence of rebasing to a release every
6 months - where as another consumer could choose to do one every 6
weeks.  The difference is how much of a jump, at which intervals..
Alternatively, a vendor might choose just to go with stock release +
their own select cherry picked patches from stable/*, which is also a
model that works.

The issue around producing tarballs, is really about having forwards
and backwards verification by means of sha/md5 sums, which is hard to
do when generating your own orig tarball.  Debian, Ubuntu and I
believe Arch have made varying use of 'pristine-tar' - which was an
effort to re-producible tarballs using xdelta to make the sum match.
However, Maintainers seem to be moving away from this now.

When I perform source NEW reviews for Ubuntu Archive, I always check
that getting the source orig tarball can be done with either
get-orig-source (inspecting the generation method) or uscan and then
diff the tarballs with the one included on the upload and the one
generated.  Timestamps (or even shasums) haven't been an important
issue for me, but the actual content and verifiable source is what has
mattered more.

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/05/2015 02:46 PM, Thierry Carrez wrote:
> So.. summarizing the various options again:
> 
> Plan A
> Just drop stable point releases.
> (-) No more release notes
> (-) Lack of reference points to compare installations
> 
> Plan B
> Push date-based tags across supported projects from time to time.
> (-) Encourages to continue using same version across the board
> (-) Almost as much work as making proper releases
> 
> Plan C
> Let projects randomly tag point releases whenever
> (-) Still a bit costly in terms of herding cats
> 
> Plan D
> Drop stable point releases, publish per-commit tarballs
> (-) Requires some infra changes, takes some storage space
> 
> Plans B, C and D also require some release note / changelog generation
> from data maintained *within* the repository.
> 
> Personally I think the objections raised against plan A are valid. I
> like plan D, since it's more like releasing every commit than "not
> releasing anymore". I think it's the most honest trade-off. I could go
> with plan C, but I think it's added work for no additional value to the
> user.
> 
> What would be your preferred option ?

I see no point of doing D. I already don't use tarballs, and those who
do could as well switch to generating them (how hard is it to run
"python setup.py sdist" or "git archive"?).

What counts is having a schedule date, where all distros are releasing a
point release, so we have a common reference point. If that is a fully
automated process, then great, less work for everyone, and it wont
change anything from what we had in the past (we can even collectively
decide for point release dates...).

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/09/2015 04:54 PM, Jeremy Stanley wrote:
> On 2015-06-09 10:55:55 +0200 (+0200), Thomas Goirand wrote:
> [...]
>> That's far from being in place. Also, while we are removing point
>> releases, and support for Icehouse, we still don't have a common private
>> Gerrit for security, which we've been told about 2 years ago.
>>
>> Removing collaboration tools before new ones are in place is
>> definitively not the way to go.
> [...]
> 
> In the stable branch management session at the Liberty summit, the
> Infrastructure Team position was confirmed that we can't maintain
> non-supported branches in a second Gerrit any more than we can in
> the primary one. The idea that there might be some branch of an
> OpenStack project which we just give up on testing but keep
> available for new changes is distasteful from a quality perspective.
> If there is sufficient interest from distro representatives in
> collaborating upstream on backports to, for example, stable/icehouse
> then that interest needs to include the resources necessary to
> maintain sufficient testing upstream (as defined by the upstream
> community).

Jeremy,

I have made a round table in Paris, and I have names of interested
people who would work on this. But without a collaboration tool (and at
least a common Git repository to work on), it is going to be harder to
work on together.

> So, I'm sorry, but don't look to an OpenStack-maintained non-public
> code review system as a workaround for continuing to collaborate in
> private on branches unsupported upstream in public. Most of the time
> it ends up being non-distro upstream developers (quality assurance,
> release management, infrastructure, vulnerability management,
> individual projects) who bear the majority of the responsibility for
> keeping testing working on those branches and we're clearly at our
> breaking point now trying to maintain three besides our master
> branch.

I understand, and I haven't asked the VMT or anyone upstream to work on
this.

> If the interested distributions come back with teams of people to
> dedicate to this work, which means getting deeply embedded in the
> groups who currently maintain the existing testing and release
> management for stable branches and are currently understaffed for
> that effort, then we can certainly revisit the number of branches
> we're able to maintain in both the public and (future) embargoed
> security review systems.

As we discussed, the goal isn't to maintain upstream CI, but to have a
place to share our work between distributions. We would do the testing
downstream, not using OpenStack infra.

Please get this security gerrit up, so that we can work on a patch
during the embargo period.

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Jeremy Stanley]

On 2015-06-09 10:55:55 +0200 (+0200), Thomas Goirand wrote:
[...]
> That's far from being in place. Also, while we are removing point
> releases, and support for Icehouse, we still don't have a common private
> Gerrit for security, which we've been told about 2 years ago.
> 
> Removing collaboration tools before new ones are in place is
> definitively not the way to go.
[...]

In the stable branch management session at the Liberty summit, the
Infrastructure Team position was confirmed that we can't maintain
non-supported branches in a second Gerrit any more than we can in
the primary one. The idea that there might be some branch of an
OpenStack project which we just give up on testing but keep
available for new changes is distasteful from a quality perspective.
If there is sufficient interest from distro representatives in
collaborating upstream on backports to, for example, stable/icehouse
then that interest needs to include the resources necessary to
maintain sufficient testing upstream (as defined by the upstream
community).

So, I'm sorry, but don't look to an OpenStack-maintained non-public
code review system as a workaround for continuing to collaborate in
private on branches unsupported upstream in public. Most of the time
it ends up being non-distro upstream developers (quality assurance,
release management, infrastructure, vulnerability management,
individual projects) who bear the majority of the responsibility for
keeping testing working on those branches and we're clearly at our
breaking point now trying to maintain three besides our master
branch.

If the interested distributions come back with teams of people to
dedicate to this work, which means getting deeply embedded in the
groups who currently maintain the existing testing and release
management for stable branches and are currently understaffed for
that effort, then we can certainly revisit the number of branches
we're able to maintain in both the public and (future) embargoed
security review systems.
-- 
Jeremy Stanley


signature.asc
Description: Digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/08/2015 06:10 PM, Jeremy Stanley wrote:
> On 2015-06-08 16:53:21 +0100 (+0100), Dave Walker wrote:
>> This breaks the desire of wanting to have a shared version scheme if
>> consumers add their own local patches via git.  This works fine for
>> consumers that do not use git for handling their local patches, but
>> does not support the model of allowing the user to rebase using git.
>>
>> Perhaps tags ARE superior for this?
> 
> Agreed, even the .devN (or earlier .postN) versioning has the same
> issue. If you rely on PBR autoversioning for non-tagged commits then
> your local commits _will_ conflict with upstream autoversions.

The problem is that we use the 2 first numbers as major version. If
instead of:

2015.1.N

we switch to something like:

20151.X.Y

then we restore sanity and a real semver system.

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/01/2015 01:20 PM, Dimitri John Ledkov wrote:
> On 29 May 2015 at 14:41, Thierry Carrez  wrote:
>> Hi everyone,
>>
>> TL;DR:
>> - We propose to stop tagging coordinated point releases (like 2015.1.1)
>> - We continue maintaining stable branches as a trusted source of stable
>> updates for all projects though
>>
> 
> Ideally I would still want to see tarballs published, even if they are
> generated with a $ git describe --tags name in it.
> 
> E.g. keystone-2015.1.0-3-g1373bfa.tar.gz

Please, no dashes in version numbers. While Debian *can* cope with them,
it's recommended to avoid it (as the dash is the separator between the
upstream version and the Debian release number).

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/08/2015 12:25 AM, Alan Pevec wrote:
>>> and *Plan D* would be to start doing automatic per-project
>>> micro-versions on each commit: e.g. 2015.1.N where N is increased on
>>> each commit.
>>
>> How do you gpg sign these tags? I hope the solution isn't to store a key
>> in infra without a passphrase.
> 
> Plan D doesn't include git tags, 2015.1.N would be generated by PBR
> automatically.

Then please don't do "Plan D". I don't use tarballs, and I don't have
the intention to use them. It's not an efficient way for me to work.

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/08/2015 05:42 PM, Jeremy Stanley wrote:
> On 2015-06-07 10:55:29 +0200 (+0200), Thomas Goirand wrote:
>> How do you gpg sign these tags? I hope the solution isn't to store
>> a key in infra without a passphrase.
> 
> How does, e.g., Debian sign its Release file for
> jessie-proposed-updates? I hope the solution isn't to store the
> ftp-master automatic archive signing key in infra without a
> passphrase. (This is a rhetorical question... I see from comments at
> https://wiki.debian.org/SecureApt that it is indeed the case.) In
> fact, I don't really mind this. It's at least an attestation that
> the machine where the signature was generated had access to the
> automatic signing key, which is in turn signed by and revocable by
> the systems administrators entrusted to protect that machine.

Fair enough. And I'll trust you will safeguard everything correctly.

:)

Thomas


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/07/2015 06:13 PM, Ian Cordasco wrote:
>>> If you consider *every* commit to be a release, then your life becomes
>>> easier.
>>
>> What does this mean? Am I supposed to upload a patched release to Debian
>> every day? I suppose I didn't understand you correctly here.
> 
> This discussion is about stable branches. Maybe early in a stable branch's
> life there are lots of commits, but as it grows older, the number of
> commits made to a stable branch certainly isn't 1 per day.

If you consider all the projects within the big-tent, that's a lot of
commits still.

>> If we were to do this in downstream distros, we wouldn't have an
>> upstream number matching each commit. This would be a problem because we
>> would loose track of what version we're at between distros.
> 
> It seems from other discussions that there is little coordination between
> distros in the first place, and that is only beginning to improve now, but
> only between close relatives (Ubuntu & Debian, RHEL & CentOS & Fedora,
> etc.). If that's the case, the work to improve coordination, including
> sharing repositories, etc. would seem to alleviate this concern regardless
> of coordinated release tagging.

That's far from being in place. Also, while we are removing point
releases, and support for Icehouse, we still don't have a common private
Gerrit for security, which we've been told about 2 years ago.

Removing collaboration tools before new ones are in place is
definitively not the way to go.

> That said, I know you've said in the past that you do most of your
> packaging in your free-time because it's not part of your job
> responsibilities.

I'm a full time Mirantis employee, and it's my full-time job to do
packaging of OpenStack in Debian (that, plus helping to do MOS).

> I know the same is true for a bunch of OpenStack's
> packagers (including the Gentoo packager).

I believe Gentoo is the only case.

> As someone
> who spends the majority of their free time supporting other software
> beyond OpenStack, I understand how insulting it is to be told you have to
> do more work in the time that you're volunteering.

It being a paid-for job IMO doesn't change the fact we should work
efficiently and the correct way. :)

> I've only been around for a little less than a
> year though, so I have no memory of a backport in one service breaking
> another.

It happened that requirements changed from one version to the next, and
2 projects needed to be updated at once. I don't see this happening
smoothly if we don't have coordinated point releases. And yes, I know,
requirements are *supposed* to be frozen, but between the theory and
what really happens, there's a world...

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/08/2015 12:46 AM, Alan Pevec wrote:
>> How do you check if project X in version n works with project Y in
>> version m, using this non-scheduled point release "free for all" model?
> 
> That was an illusion of point releases, as Thierry said there wasn't
> significantly more testing and I don't remember any testing reports
> during stable freeze periods.

There is: the point releases end up in all distributions, and we all do
our own tests.

> All we had was upstream CI testing, but that's what we get for every commit.

The stable release gate is famously always broken, and now that we're
proposing to stop doing the point releases, it's going to be even worse.

>> Why every 2 weeks? Why every month?
> 
> Sure, no problem, every distro can update whenever it works for them,
> what is important for me is that we have a common reference points.
> With plan D that would be automatically generated maj.min.N where N is
> the number of commits since maj.min tag which doesn't depend on
> anyone's manually pushing git tag.

As long as we have a common reference point, manually or not, I'm satisfied.

>> Not synchronizing brings a bunch of problems. The only problem raised by
>> synchronous point releases is "we don't have enough resources", which I
>> fail to understand, given how cheap it is to tag a release. If the
>> release managers don't have enough time to do so, it is my view that
>> it's ok to give this power to individual projects. But that's orthogonal
>> to having synchronous point releases.
> 
> Tag is indeed cheap, this is more about reflecting reality and not
> keeping this "synchronized" illusion alive.

If distros are releasing packages based on the same tag, I fail to see
where we have an illusion.

> BTW point release process is more[1] than just pushing signed git tag,
> the most time consuming work is cats herding (i.e. pushing for reviews
> before the freeze) and Launchpad pruning.
> The former was hopeless and will be even more with big-tent and the
> latter we should avoid by automatically generated changelog.

So, if I get you right, you are saying that our QA work isn't working as
well as we want, and therefore, we should accept that fact, and get rid
of that QA work. While it's important to realize what our problems are,
I don't think this is going to the right direction still... :(

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/07/2015 05:57 PM, Ian Cordasco wrote:
> On 6/7/15, 03:41, "Thomas Goirand"  wrote:
> 
>> On 05/29/2015 09:23 PM, Ian Cordasco wrote:
>>> Could you explain this as well? Do you mean fragmentation between what
>>> distros are offering? In other words, Ubuntu is packaging Kilo @ SHA1
>>> and
>>> RHEL is at SHA2. I'm not entirely certain that's a bad thing. That seems
>>> to give the packagers more freedom.
>>
>> What happens when there's a security patch? Will upstream publish
>> patches for each and every distro? I don't believe so.
> 
> Does upstream do that now? I don't think so.

The point it: they don't need to do it, because all distro are using the
same reference point (ie: the point releases).

>> On 05/29/2015 09:23 PM, Ian Cordasco wrote:
>> The point of the embargo is to give time for testing patches and prepare
>> a new patched version. Sometimes, we discover problems with the provided
>> patch during the embargo period. Yes, we use the embargo to sometimes
>> adapt the patch to the version we have in our distributions, but we
>> would prefer if that work wasn't needed.
> 
> But there aren't point releases for every CVE fix. There are point
> releases that are coordinated at the moment. So if you're waiting for
> those point releases to publish a new version of that package in your
> package repositories, that's news to me. I've seen packagers take patches
> and apply them and merely change the build metadata. Is this only done for
> "severe" CVEs at the moment?

I try to do a security fix on every OSSA the way you describe above. I
suppose other distros are doing the same (but I didn't take the time to
check).

> If every commit were a release, then you could all synchronize on that, if
> you all packaged each commit or at least, generate a new package each time
> a CVE is publicly patched through gerrit.

Adding a bunch of unrelated commits for a CVE fix may be acceptable for
Debian Sid, but it wouldn't for the stable distribution.

But anyway, the discussion about point releases is only barely related
to CVE fixing. The point is that we would like to have a common
reference point between distribution, were we would all be able to say:
version X.Y.Z of this server has a bug, it broke the CI of N and M
distribution, we need a fix release. Without such a coordination, we
wouldn't have as much attention from upstream to produce clean point
releases.

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Robert Collins]

On 9 June 2015 at 03:48, Jeremy Stanley  wrote:
> On 2015-06-08 10:54:32 +1200 (+1200), Robert Collins wrote:
>> On 8 June 2015 at 10:14, Alan Pevec  wrote:
>> > 2015-06-06 19:08 GMT+02:00 Ian Cordasco :
>> >> Not exactly. PBR/OpenStack follow SemVer and 2015.1.0.38 isn't valid
>> >> SemVer (http://semver.org/)
>> >
>> > Right, so semver compatible versioning on stable/kilo would be 2015.1.N
>> > but PBR doesn't support that.
>>
>> PBR supports it fine. Whats the issue?
>
> Having PBR automatically increment the N in 2015.1.N for each patch
> applied on the branch rather than needing it to be tagged.

I don't think pbr needs to do that to have the proposal work. We could
tag from cron and use the actual tag pipeline to do release related
stuff.

E.g. its layer conflation to have pbr start assuming that untagged
code isn't a release.

-Rob

-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Jeremy Stanley]

On 2015-06-08 16:53:21 +0100 (+0100), Dave Walker wrote:
> This breaks the desire of wanting to have a shared version scheme if
> consumers add their own local patches via git.  This works fine for
> consumers that do not use git for handling their local patches, but
> does not support the model of allowing the user to rebase using git.
> 
> Perhaps tags ARE superior for this?

Agreed, even the .devN (or earlier .postN) versioning has the same
issue. If you rely on PBR autoversioning for non-tagged commits then
your local commits _will_ conflict with upstream autoversions.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Dave Walker]

On 8 June 2015 at 16:48, Jeremy Stanley  wrote:
> On 2015-06-08 10:54:32 +1200 (+1200), Robert Collins wrote:
>> On 8 June 2015 at 10:14, Alan Pevec  wrote:
>> > 2015-06-06 19:08 GMT+02:00 Ian Cordasco :
>> >> Not exactly. PBR/OpenStack follow SemVer and 2015.1.0.38 isn't valid
>> >> SemVer (http://semver.org/)
>> >
>> > Right, so semver compatible versioning on stable/kilo would be 2015.1.N
>> > but PBR doesn't support that.
>>
>> PBR supports it fine. Whats the issue?
>
> Having PBR automatically increment the N in 2015.1.N for each patch
> applied on the branch rather than needing it to be tagged.

This breaks the desire of wanting to have a shared version scheme if
consumers add their own local patches via git.  This works fine for
consumers that do not use git for handling their local patches, but
does not support the model of allowing the user to rebase using git.

Perhaps tags ARE superior for this?

--
Kind Regards,
Dave Walker

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Jeremy Stanley]

On 2015-06-08 10:54:32 +1200 (+1200), Robert Collins wrote:
> On 8 June 2015 at 10:14, Alan Pevec  wrote:
> > 2015-06-06 19:08 GMT+02:00 Ian Cordasco :
> >> Not exactly. PBR/OpenStack follow SemVer and 2015.1.0.38 isn't valid
> >> SemVer (http://semver.org/)
> >
> > Right, so semver compatible versioning on stable/kilo would be 2015.1.N
> > but PBR doesn't support that.
> 
> PBR supports it fine. Whats the issue?

Having PBR automatically increment the N in 2015.1.N for each patch
applied on the branch rather than needing it to be tagged.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Jeremy Stanley]

On 2015-06-08 00:25:47 +0200 (+0200), Alan Pevec wrote:
> BTW there's an issue re. verification that
> https://tarballs.openstack.org/ is using cert for
> security.openstack.org but should be easily fixed by infra.

Uh, nope. Try again. You're redirected to security.openstack.org if
you accept that cert. It's true that tarballs.openstack.org is a
vhost on the same system, but it's not intended as a secure source
of release artifacts at the moment and so is only served via HTTP.
We may at some point add HTTPS, but more likely we'll add detached
signatures along with the tarballs/wheels we upload there through
some sort of automation rather than relying on HTTPS alone.
-- 
Jeremy Stanley


signature.asc
Description: Digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Jeremy Stanley]

On 2015-06-07 10:55:29 +0200 (+0200), Thomas Goirand wrote:
> How do you gpg sign these tags? I hope the solution isn't to store
> a key in infra without a passphrase.

How does, e.g., Debian sign its Release file for
jessie-proposed-updates? I hope the solution isn't to store the
ftp-master automatic archive signing key in infra without a
passphrase. (This is a rhetorical question... I see from comments at
https://wiki.debian.org/SecureApt that it is indeed the case.) In
fact, I don't really mind this. It's at least an attestation that
the machine where the signature was generated had access to the
automatic signing key, which is in turn signed by and revocable by
the systems administrators entrusted to protect that machine.
-- 
Jeremy Stanley


signature.asc
Description: Digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Kuvaja, Erno]

> -Original Message-
> From: Thierry Carrez [mailto:thie...@openstack.org]
> Sent: Friday, June 05, 2015 1:46 PM
> To: openstack-dev@lists.openstack.org
> Subject: Re: [openstack-dev] [all] [stable] No longer doing stable point
> releases
> 
> So.. summarizing the various options again:
> 
> Plan A
> Just drop stable point releases.
> (-) No more release notes
> (-) Lack of reference points to compare installations
> 
> Plan B
> Push date-based tags across supported projects from time to time.
> (-) Encourages to continue using same version across the board
> (-) Almost as much work as making proper releases
> 
> Plan C
> Let projects randomly tag point releases whenever
> (-) Still a bit costly in terms of herding cats
> 
> Plan D
> Drop stable point releases, publish per-commit tarballs
> (-) Requires some infra changes, takes some storage space
> 
> Plans B, C and D also require some release note / changelog generation from
> data maintained *within* the repository.
> 
> Personally I think the objections raised against plan A are valid. I like 
> plan D,
> since it's more like releasing every commit than "not releasing anymore". I
> think it's the most honest trade-off. I could go with plan C, but I think it's
> added work for no additional value to the user.
> 
> What would be your preferred option ?
> 
> --
> Thierry Carrez (ttx)
> 

I don't think plans B & C are any benefit for  anyone based on the statements 
discussed earlier so won't be repeating those. One thing I like about plan D is 
that it would give also indicator how much the stable branch has moved in each 
individual project. 

Yes this can be seen from git, but also having just running number on the 
stable release for each commit would be really quick glimpse opposed to SHAs or 
counting  the changes from git  log.

- Erno

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Joshua Harlow]

Robert Collins wrote:

On 6 June 2015 at 08:53, Joshua Harlow  wrote:

Hopefully it's somewhat obvious to folks that altering the PBR version
schema (yet again), breaks *all the people* from using it in a reliable
manner, and if the goal of PBR is to bring reasonableness, well changing it
in a way that breaks things seems like the-anti-pattern of PBR.

I know from building with anvil and building RPMs that we only used tags in
https://github.com/stackforge/anvil/tree/master/conf/origins because the PBR
version number would change format/style/other (and also partially because
nothing existed that converted to a *sane* rpm version string, although this
supposedly exists now via a new PBR function).


Well is has the thing you collaborated on. Whether thats sane, an RPM
person needs to say :)


Sanity is in the eye of the beholder so +1 from me, hopefully not to 
insane, ha :-P




-Rob




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Robert Collins]

On 6 June 2015 at 08:53, Joshua Harlow  wrote:
> Hopefully it's somewhat obvious to folks that altering the PBR version
> schema (yet again), breaks *all the people* from using it in a reliable
> manner, and if the goal of PBR is to bring reasonableness, well changing it
> in a way that breaks things seems like the-anti-pattern of PBR.
>
> I know from building with anvil and building RPMs that we only used tags in
> https://github.com/stackforge/anvil/tree/master/conf/origins because the PBR
> version number would change format/style/other (and also partially because
> nothing existed that converted to a *sane* rpm version string, although this
> supposedly exists now via a new PBR function).

Well is has the thing you collaborated on. Whether thats sane, an RPM
person needs to say :)

-Rob


-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Robert Collins]

On 8 June 2015 at 10:14, Alan Pevec  wrote:
> 2015-06-06 19:08 GMT+02:00 Ian Cordasco :
>> Not exactly. PBR/OpenStack follow SemVer and 2015.1.0.38 isn't valid
>> SemVer (http://semver.org/)
>
> Right, so semver compatible versioning on stable/kilo would be 2015.1.N
> but PBR doesn't support that.

PBR supports it fine. Whats the issue?

-Rob


-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Robert Collins]

On 8 June 2015 at 10:45, Robert Collins  wrote:

>
>> You'll also note that according to PEP 440, (as Jeremy pointed out) .postN
>> is meant for non-code changes. If we want to be pedantic about the version
>> numbers generated by PBR (at the gate, in tox, etc.), it should be > version number>.devN but that's shaving an entirely different yak, and one
>> that I don't think is especially concerning or a serious problem.
>
> Its a very concerning problem for continuous deployers, and thats why
> pbr 0.11 is now the minimum in global requirements, because we solved
> it. It can cause gate issues with dependencies not upgrading for
> instance (or running the latest beta rather than the local tree meant
> to be tested) [even with latest pip].

I missed the reference - sorry -
http://docs.openstack.org/developer/pbr/#version - documents the
algorithm pbr uses to generate versions. The reference to git sha's is
incorrect, I'll push up a doc fix for that shortly (we store that in a
pbr metadata file in the sdist and can report on that via the pbr CLI,
but its not in the version string anymore.

-Rob

-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Robert Collins]

On 7 June 2015 at 05:08, Ian Cordasco  wrote:
>
>
> On 6/6/15, 02:03, "Alan Pevec"  wrote:
>
>>2015-06-05 15:16 GMT+02:00 Jeremy Stanley :
>>> On 2015-06-05 14:56:30 +0200 (+0200), Thierry Carrez wrote:
>>> [...]
 I was wondering if we could switch to post-versioning on stable
 branches, and basically generate:

 "2015.1.0.post38"
>>> [...]
>>>
>>> I think the recommendation from the PyPI maintainers is to not use
>>> .postN suffixes since they are intended to indicate non-code
>>> changes.
>>
>>So that's PBR bug then and it should generate 2015.1.0.38 ?
>
> Not exactly. PBR/OpenStack follow SemVer

We follow a tweaked semver
http://docs.openstack.org/developer/pbr/semver.html but also honour
the date based server versions.

> and 2015.1.0.38 isn't valid
> SemVer (http://semver.org/) Specifically note that version generated above
> is also not valid SemVer but is valid under PEP 440 (which is what
> setuptools and pip use; https://www.python.org/dev/peps/pep-0440/). So we
> could ditch sticking strictly to SemVer for release versioning (meaning we
> could use 2015.1.0.38) or we need a new way to indicate all of this. Note
> that while PEP 440 and SemVer are not fully compatible
> (https://www.python.org/dev/peps/pep-0440/#semantic-versioning) both
> SemVer and PEP 440 allow for extra information. PEP 440 calls this "Local
> Version Segments"

We haven't implemented local version segments as such in pbr yet, and
I'm very open to us doing so - but only in concert with consolidating
the local-vendor stuff from Nova/Cinder etc into the same spec,
because we've only really got one shot to bring those things together
sanely, and I don't want to use the field up but not converge on those
things.

> (https://www.python.org/dev/peps/pep-0440/#local-version-segments) while
> SemVer calls it "Build Metadata". That means that the version string would
> mean different things for different consumers with different mental models
> and would probably conflict with downstream redistributors who add their
> own versioning information to a version string.

Well - there's a defined mechanism in e.g. Nova for this already, we
just need to make sure we deliver the same use cases and tie it all
together.

> You'll also note that according to PEP 440, (as Jeremy pointed out) .postN
> is meant for non-code changes. If we want to be pedantic about the version
> numbers generated by PBR (at the gate, in tox, etc.), it should be  version number>.devN but that's shaving an entirely different yak, and one
> that I don't think is especially concerning or a serious problem.

Its a very concerning problem for continuous deployers, and thats why
pbr 0.11 is now the minimum in global requirements, because we solved
it. It can cause gate issues with dependencies not upgrading for
instance (or running the latest beta rather than the local tree meant
to be tested) [even with latest pip].

pbr < 0.11 generated .postN as a response to the OMG PEP-426 broke
everything wedge, and as a result 0.11 and above interpret .postN as
.devN for backwards compatibility reasons - so we can't use .postN
trivially, even if it was semantically appropriate - and as you note,
its not).

We can alter pbr to do more digits if that will solve a problem, but I
haven't seen a problem put forward so far that doesn't map reasonably
well to semver.

-Rob


-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Alan Pevec]

> How do you check if project X in version n works with project Y in
> version m, using this non-scheduled point release "free for all" model?

That was an illusion of point releases, as Thierry said there wasn't
significantly more testing and I don't remember any testing reports
during stable freeze periods.
All we had was upstream CI testing, but that's what we get for every commit.

> Why every 2 weeks? Why every month?

Sure, no problem, every distro can update whenever it works for them,
what is important for me is that we have a common reference points.
With plan D that would be automatically generated maj.min.N where N is
the number of commits since maj.min tag which doesn't depend on
anyone's manually pushing git tag.

> Not synchronizing brings a bunch of problems. The only problem raised by
> synchronous point releases is "we don't have enough resources", which I
> fail to understand, given how cheap it is to tag a release. If the
> release managers don't have enough time to do so, it is my view that
> it's ok to give this power to individual projects. But that's orthogonal
> to having synchronous point releases.

Tag is indeed cheap, this is more about reflecting reality and not
keeping this "synchronized" illusion alive.
BTW point release process is more[1] than just pushing signed git tag,
the most time consuming work is cats herding (i.e. pushing for reviews
before the freeze) and Launchpad pruning.
The former was hopeless and will be even more with big-tent and the
latter we should avoid by automatically generated changelog.

Cheers,
Alan

[1] https://wiki.openstack.org/wiki/StableBranchRelease

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Alan Pevec]

>> and *Plan D* would be to start doing automatic per-project
>> micro-versions on each commit: e.g. 2015.1.N where N is increased on
>> each commit.
>
> How do you gpg sign these tags? I hope the solution isn't to store a key
> in infra without a passphrase.

Plan D doesn't include git tags, 2015.1.N would be generated by PBR
automatically.

> FYI, I don't use tarballs (just git), and generate my own orig.tar.xz
> out of a signed git tag, so I am not affected by this.

We could generate it too but upstream SourceURL is preferred[1] so it
can be easily verified.
BTW there's an issue re. verification that
https://tarballs.openstack.org/ is using cert for
security.openstack.org but should be easily fixed by infra.


Cheers,
Alan

[1] https://fedoraproject.org/wiki/Packaging:SourceURL

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Alan Pevec]

2015-06-06 19:08 GMT+02:00 Ian Cordasco :
> Not exactly. PBR/OpenStack follow SemVer and 2015.1.0.38 isn't valid
> SemVer (http://semver.org/)

Right, so semver compatible versioning on stable/kilo would be 2015.1.N
but PBR doesn't support that.

> If we want to be pedantic about the version numbers generated by PBR (at the 
> gate, in tox, etc.), it should be .devN

That's what PBR does now, even for projects without version in
setup.cfg, it assumes MAJOR.MINOR.PATCH++ from the last tag on the
branch.

Cheers,
Alan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Ian Cordasco]

On 6/7/15, 03:47, "Thomas Goirand"  wrote:

>On 05/29/2015 09:36 PM, Dave Walker wrote:
>> Responses inline.
>> 
>> On 29 May 2015 6:15 pm, "Haïkel" > > wrote:
>>>
>>> 2015-05-29 15:41 GMT+02:00 Thierry Carrez > >:
>>> > Hi everyone,
>>> >
>>> > TL;DR:
>>> > - We propose to stop tagging coordinated point releases (like
>>>2015.1.1)
>>> > - We continue maintaining stable branches as a trusted source of
>>>stable
>>> > updates for all projects though
>>> >
>>>
>>> Hi,
>>>
>>> I'm one of the main maintainer of the packages for Fedora/RHEL/CentOS.
>>> We try to stick as much as possible to upstream (almost zero
>>> downstream patches),
>>> and without intermediate releases, it will get difficult.
>> 
>> If you consider *every* commit to be a release, then your life becomes
>> easier.
>
>What does this mean? Am I supposed to upload a patched release to Debian
>every day? I suppose I didn't understand you correctly here.

This discussion is about stable branches. Maybe early in a stable branch's
life there are lots of commits, but as it grows older, the number of
commits made to a stable branch certainly isn't 1 per day.

>If we were to do this in downstream distros, we wouldn't have an
>upstream number matching each commit. This would be a problem because we
>would loose track of what version we're at between distros.

It seems from other discussions that there is little coordination between
distros in the first place, and that is only beginning to improve now, but
only between close relatives (Ubuntu & Debian, RHEL & CentOS & Fedora,
etc.). If that's the case, the work to improve coordination, including
sharing repositories, etc. would seem to alleviate this concern regardless
of coordinated release tagging.

>>> I'm personally not fond of this as it will lead to more fragmentation.
>>> It may encourage
>>> bad behaviors like shipping downstream patches for bug fixes and CVE
>> instead
>>> of collaborating upstream to differentiate themselves.
>>> For instance, if we had no point-based release, for issues tracking
>>> purposes, we would
>>> have to maintain our sets of tags somewhere.
>> 
>> I disagree, each distro already does security patching and whilst I
>> expect this to still happens, it actually *encourages* upstream first
>> workflow as you can select a release on your own cadence that includes
>> commits you need, for your users.
>
>We are discussing point releases. This is only far related to security
>fixes. Point releases are including bug fixes which, most of the time,
>aren't security fixes which are by the way always backported to the
>previous point release, and uploaded as hotfixes to distributions.

Not every bug fix gets backported during the life of a stable branch. At
least in Glance, the majority of backports are security fixes and the rest
are rather serious bugs that significantly break the service or prevent
the use of the service. Last I checked, bug fixes needed to meet a certain
set of criteria before a backport was created. I'm fairly certain this
hasn't changed, but I admit that I haven't checked for changes in the last
several months.

That said, I know you've said in the past that you do most of your
packaging in your free-time because it's not part of your job
responsibilities. I know the same is true for a bunch of OpenStack's
packagers (including the Gentoo packager). This is what makes me torn on
whether or not we should totally drop stable point releases. As someone
who spends the majority of their free time supporting other software
beyond OpenStack, I understand how insulting it is to be told you have to
do more work in the time that you're volunteering.

This makes me more in favor of letting projects cut their own point
releases for stable branches. That said, I don't think it will alleviate
your concerns of guaranteeing that 2015.2.1 of X works with 2015.2.1 of Y,
but if these truly are only bug fixes, then I think the guarantee is
implicit in the branch. I've only been around for a little less than a
year though, so I have no memory of a backport in one service breaking
another.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Ian Cordasco]

On 6/7/15, 03:41, "Thomas Goirand"  wrote:

>On 05/29/2015 09:23 PM, Ian Cordasco wrote:
>> Could you explain this as well? Do you mean fragmentation between what
>> distros are offering? In other words, Ubuntu is packaging Kilo @ SHA1
>>and
>> RHEL is at SHA2. I'm not entirely certain that's a bad thing. That seems
>> to give the packagers more freedom.
>
>What happens when there's a security patch? Will upstream publish
>patches for each and every distro? I don't believe so.

Does upstream do that now? I don't think so. They send out proposed
patches for each version that's affected (if I understand the process
documents correctly).

>On 05/29/2015 09:23 PM, Ian Cordasco wrote:
>> Perhaps I'm wrong, but when a CVE is released, don't the downstream
>> packagers usually patch whatever version they have and push that out?
>
>We would like to have a single patch to share between distros.
>Fragmenting the work helps nobody.

You already have this if the documentation is correct.

>> Isn't that the point of them being on an private list to receive
>> embargoed notifications with the patches?
>
>The point of the embargo is to give time for testing patches and prepare
>a new patched version. Sometimes, we discover problems with the provided
>patch during the embargo period. Yes, we use the embargo to sometimes
>adapt the patch to the version we have in our distributions, but we
>would prefer if that work wasn't needed.

But there aren't point releases for every CVE fix. There are point
releases that are coordinated at the moment. So if you're waiting for
those point releases to publish a new version of that package in your
package repositories, that's news to me. I've seen packagers take patches
and apply them and merely change the build metadata. Is this only done for
"severe" CVEs at the moment?

If every commit were a release, then you could all synchronize on that, if
you all packaged each commit or at least, generate a new package each time
a CVE is publicly patched through gerrit.

I think we're probably talking past each other. I have had one set of
experiences with downstream redistributors and CVEs in packages, and
you're describing an entirely different process. I can't think of a time I
haven't published a CVE where older versions of the package weren't
patched by downstream redistributors as well as the current one,
especially if they don't have the free time to publish the new package
version.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/01/2015 05:32 PM, Alan Pevec wrote:
>> *Plan C* would be to just let projects tag stable point releases from
>> time to time. That would solve all the original stated problems. And
>> that would solve objections 2 and 3, which I think are the most valid ones.
> 
> and *Plan D* would be to start doing automatic per-project
> micro-versions on each commit: e.g. 2015.1.N where N is increased on
> each commit.

How do you gpg sign these tags? I hope the solution isn't to store a key
in infra without a passphrase.

> There's just TBD item how to provide source tarballs for this.

FYI, I don't use tarballs (just git), and generate my own orig.tar.xz
out of a signed git tag, so I am not affected by this.

Cheers,

Thomas


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 06/01/2015 10:40 AM, Ihar Hrachyshka wrote:
> No time synchronization between projects, no freeze dates, nothing, just
> SemVer compatible version bumps e.g. once per two weeks or month.
> Would it fix your problem?

How do you check if project X in version n works with project Y in
version m, using this non-scheduled point release "free for all" model?

Why every 2 weeks? Why every month?

Not synchronizing brings a bunch of problems. The only problem raised by
synchronous point releases is "we don't have enough resources", which I
fail to understand, given how cheap it is to tag a release. If the
release managers don't have enough time to do so, it is my view that
it's ok to give this power to individual projects. But that's orthogonal
to having synchronous point releases.

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 05/29/2015 09:36 PM, Dave Walker wrote:
> Responses inline.
> 
> On 29 May 2015 6:15 pm, "Haïkel"  > wrote:
>>
>> 2015-05-29 15:41 GMT+02:00 Thierry Carrez  >:
>> > Hi everyone,
>> >
>> > TL;DR:
>> > - We propose to stop tagging coordinated point releases (like 2015.1.1)
>> > - We continue maintaining stable branches as a trusted source of stable
>> > updates for all projects though
>> >
>>
>> Hi,
>>
>> I'm one of the main maintainer of the packages for Fedora/RHEL/CentOS.
>> We try to stick as much as possible to upstream (almost zero
>> downstream patches),
>> and without intermediate releases, it will get difficult.
> 
> If you consider *every* commit to be a release, then your life becomes
> easier.

What does this mean? Am I supposed to upload a patched release to Debian
every day? I suppose I didn't understand you correctly here.

If we were to do this in downstream distros, we wouldn't have an
upstream number matching each commit. This would be a problem because we
would loose track of what version we're at between distros.

>> I'm personally not fond of this as it will lead to more fragmentation.
>> It may encourage
>> bad behaviors like shipping downstream patches for bug fixes and CVE
> instead
>> of collaborating upstream to differentiate themselves.
>> For instance, if we had no point-based release, for issues tracking
>> purposes, we would
>> have to maintain our sets of tags somewhere.
> 
> I disagree, each distro already does security patching and whilst I
> expect this to still happens, it actually *encourages* upstream first
> workflow as you can select a release on your own cadence that includes
> commits you need, for your users.

We are discussing point releases. This is only far related to security
fixes. Point releases are including bug fixes which, most of the time,
aren't security fixes which are by the way always backported to the
previous point release, and uploaded as hotfixes to distributions.

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 05/29/2015 09:23 PM, Ian Cordasco wrote:
> Could you explain this as well? Do you mean fragmentation between what
> distros are offering? In other words, Ubuntu is packaging Kilo @ SHA1 and
> RHEL is at SHA2. I'm not entirely certain that's a bad thing. That seems
> to give the packagers more freedom.

What happens when there's a security patch? Will upstream publish
patches for each and every distro? I don't believe so.

On 05/29/2015 09:23 PM, Ian Cordasco wrote:
> Perhaps I'm wrong, but when a CVE is released, don't the downstream
> packagers usually patch whatever version they have and push that out?

We would like to have a single patch to share between distros.
Fragmenting the work helps nobody.

> Isn't that the point of them being on an private list to receive
> embargoed notifications with the patches?

The point of the embargo is to give time for testing patches and prepare
a new patched version. Sometimes, we discover problems with the provided
patch during the embargo period. Yes, we use the embargo to sometimes
adapt the patch to the version we have in our distributions, but we
would prefer if that work wasn't needed.

Thomas


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thomas Goirand]

On 05/29/2015 07:14 PM, Haïkel wrote:
> 2015-05-29 15:41 GMT+02:00 Thierry Carrez :
>> Hi everyone,
>>
>> TL;DR:
>> - We propose to stop tagging coordinated point releases (like 2015.1.1)
>> - We continue maintaining stable branches as a trusted source of stable
>> updates for all projects though
>>
> 
> Hi,
> 
> I'm one of the main maintainer of the packages for Fedora/RHEL/CentOS.
> We try to stick as much as possible to upstream (almost zero
> downstream patches),
> and without intermediate releases, it will get difficult.
> 
> I'm personally not fond of this as it will lead to more fragmentation.
> It may encourage
> bad behaviors like shipping downstream patches for bug fixes and CVE instead
> of collaborating upstream to differentiate themselves.
> For instance, if we had no point-based release, for issues tracking
> purposes, we would
> have to maintain our sets of tags somewhere.
> 
> There's also the release notes issue that has already been mentioned.
> Still continuous release notes won't solve the problem, as you wouldn't
> be able to map these to the actual packages. Will we require operators
> to find from which git commit, the packages were built and then try to figure
> out which fixes are and are not included?
>
>> (1) Projects do not all follow the same versioning, so some projects
>> (like Swift) were not part of the "stable point releases". More and more
>> projects are considering issuing intermediary releases (like Swift
>> does), like Ironic. That would result in a variety of version numbers,
>> and ultimately less and less projects being able to have a common
>> "2015.1.1"-like version.
>>
> 
> And that's actually a pain point to track for these releases in which
> OpenStack branch belong. And this is probably something that needs to
> be resolved.
> 
>> (2) Producing those costs a non-trivial amount of effort on a very small
>> team of volunteers, especially with projects caring about stable
>> branches in various amounts. We were constantly missing the
>> pre-announced dates on those ones. Looks like that effort could be
>> better spent improving the stable branches themselves and keeping them
>> working.
> 
> Agreed, but why not switching to a time-based release?
> Regularly, we tag/generate/upload tarballs, this could even be automated.
> As far as I'm concerned, I would be more happy to have more frequent releases.
> 
>> Now we realize that the cross-section of our community which was present
>> in that session might not fully represent the consumers of those
>> artifacts, which is why we expand the discussion on this mailing-list
>> (and soon on the operators ML).
> 
> Thanks, I was not able to join this discussion, and that was the kind
> of proposal that I was fearing to see happen.
> 
>> If you were a consumer of those and will miss them, please explain why.
>> In particular, please let us know how consuming that version (which was
>> only made available every n months) is significantly better than picking
>> your preferred time and get all the current stable branch HEADs at that
>> time.
> 
> We provide both type of builds
> * git continuous builds => for testing/CI and early feedback on potential 
> issues
> * point-release based builds => for GA, and production
> 
> Anyway, I won't force anyone to do something they don't want to do but I'm
> willing to step in to keep point releases in one form or another.
> 
> Regards,
> H.

I fully agree with what Haikel wrote (and therefore wont repeat).

The same way, I didn't agree with the change of version scheme either.

Why not using something like: 201504.0.0, and allow projects to do
intermediate releases like 201506, 201507, etc? Or 20151 / 20152, and
then allow the middle number to increase as it pleases projects?
Basically, anything better than just random numbers?

Also, besides Ironic, which are the other projects that are willing to
release more often than 6 months?

Cheers,

Thomas


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Ian Cordasco]

On 6/6/15, 02:03, "Alan Pevec"  wrote:

>2015-06-05 15:16 GMT+02:00 Jeremy Stanley :
>> On 2015-06-05 14:56:30 +0200 (+0200), Thierry Carrez wrote:
>> [...]
>>> I was wondering if we could switch to post-versioning on stable
>>> branches, and basically generate:
>>>
>>> "2015.1.0.post38"
>> [...]
>>
>> I think the recommendation from the PyPI maintainers is to not use
>> .postN suffixes since they are intended to indicate non-code
>> changes.
>
>So that's PBR bug then and it should generate 2015.1.0.38 ?

Not exactly. PBR/OpenStack follow SemVer and 2015.1.0.38 isn't valid
SemVer (http://semver.org/) Specifically note that version generated above
is also not valid SemVer but is valid under PEP 440 (which is what
setuptools and pip use; https://www.python.org/dev/peps/pep-0440/). So we
could ditch sticking strictly to SemVer for release versioning (meaning we
could use 2015.1.0.38) or we need a new way to indicate all of this. Note
that while PEP 440 and SemVer are not fully compatible
(https://www.python.org/dev/peps/pep-0440/#semantic-versioning) both
SemVer and PEP 440 allow for extra information. PEP 440 calls this "Local
Version Segments" 
(https://www.python.org/dev/peps/pep-0440/#local-version-segments) while
SemVer calls it "Build Metadata". That means that the version string would
mean different things for different consumers with different mental models
and would probably conflict with downstream redistributors who add their
own versioning information to a version string.

You'll also note that according to PEP 440, (as Jeremy pointed out) .postN
is meant for non-code changes. If we want to be pedantic about the version
numbers generated by PBR (at the gate, in tox, etc.), it should be .devN but that's shaving an entirely different yak, and one
that I don't think is especially concerning or a serious problem.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Jeremy Stanley]

On 2015-06-06 09:03:13 +0200 (+0200), Alan Pevec wrote:
> 2015-06-05 15:16 GMT+02:00 Jeremy Stanley :
[...]
> > I think the recommendation from the PyPI maintainers is to not use
> > .postN suffixes since they are intended to indicate non-code
> > changes.
> 
> So that's PBR bug then and it should generate 2015.1.0.38 ?

PBR no longer generates versions like 2015.1.0.post38 but rather
2015.1.1.dev38 instead, specifically because the pip/PyPI
maintainers recommended against the former pattern.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Alan Pevec]

2015-06-05 15:16 GMT+02:00 Jeremy Stanley :
> On 2015-06-05 14:56:30 +0200 (+0200), Thierry Carrez wrote:
> [...]
>> I was wondering if we could switch to post-versioning on stable
>> branches, and basically generate:
>>
>> "2015.1.0.post38"
> [...]
>
> I think the recommendation from the PyPI maintainers is to not use
> .postN suffixes since they are intended to indicate non-code
> changes.

So that's PBR bug then and it should generate 2015.1.0.38 ?

Cheers,
Alan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Alan Pevec]

> Finally, I don't think this is still costly in terms of herding cats,
> because we have very few cats actually grazing in the stable branch
> pastures. I suggest that stable branch maintainers simply get used to
> releasing on a weekly basis if there are new commits, and use that as
> an opportunity to highlight the excellent work their community has done.

If you mean $PROJECT-stable-maint then I agree, let's give C option to them.
But also option D as a default for projects not wanting to take this
release responsibility and still want a stable branch.


Cheers,
Alan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Joshua Harlow]

Hopefully it's somewhat obvious to folks that altering the PBR version 
schema (yet again), breaks *all the people* from using it in a reliable 
manner, and if the goal of PBR is to bring reasonableness, well changing 
it in a way that breaks things seems like the-anti-pattern of PBR.


I know from building with anvil and building RPMs that we only used tags 
in https://github.com/stackforge/anvil/tree/master/conf/origins because 
the PBR version number would change format/style/other (and also 
partially because nothing existed that converted to a *sane* rpm version 
string, although this supposedly exists now via a new PBR function).


Thierry Carrez wrote:

Jeremy Stanley wrote:

On 2015-06-01 15:57:17 + (+), Jeremy Stanley wrote:
[...]

The biggest hurdle is that we'd need a separate upload job name
for those since the current version of Zuul lacks a way to run a
particular job for different branches in different pipelines (we'd
want to do versioned uploads for all pre-release and release
pipeline refs, but also for post pipeline refs only when the
branch name is like stable/.*).

Actually, scratch that. It's a bit more complicated since the post
pipeline isn't actually branch-relevant. We'd need to tweak the
tarball and wheel creation scripts to check the containing branch,
like we do for some proposal jobs. Still, I think it wouldn't be too
hard.


Exploring plan D, I was looking at the versions we currently generate on
stable branches and I think they would not convey the right message:

"2015.1.1.dev38"

- but there won't be a 2015.1.1 !
- but this is not "under development" !

I was wondering if we could switch to post-versioning on stable
branches, and basically generate:

"2015.1.0.post38"

... which would convey the right message.

I /think/ all it would take would be, as the first post-release commit
to the stable branch, to remove the preversion from setup.cfg (rather
than bump it to the next .1). I think pbr would switch to postversioning
in that case and generate postX versions from the last tag in the branch.

Not sure we would do that for stable/kilo, though, since we already
pushed 2015.1.1.devX versions in the wild.



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Clint Byrum]

Excerpts from Thierry Carrez's message of 2015-06-05 05:46:07 -0700:
> So.. summarizing the various options again:
> 

Thanks for the excellent summary Thierry
> Plan C
> Let projects randomly tag point releases whenever
> (-) Still a bit costly in terms of herding cats
> 

I feel like plan C is the smallest change one can make while still moving
toward lowering the project-wide maintenance costs of stable branches.

I don't know if I saw this highlighted enough during the discussion phase,
but the reason this might be a good idea is that stable releases, unlike
"trunk", have a very different purpose and mode.

Unlike with the coordinated release of many projects all at once, we
are not trying to make sure the pieces still fit together. We've been
careful, in stable release policy, to be sure that we are minimizing
changes, so I think coordinating them is unnecessary overhead.

However, releasing all the commits every time does also mean that there's
no signal to downstream that now is a good time to pull this stable
branch and incorporate it into your systems. Even if each project ends up
releasing weekly right after their meeting in a semi-automatic fashion,
this means that users and packagers can attend the meeting, read the
minutes, and have a passive communication stream that doesn't require
them to read and grok every single commit whenever it might be approved.

Finally, I don't think this is still costly in terms of herding cats,
because we have very few cats actually grazing in the stable branch
pastures. I suggest that stable branch maintainers simply get used to
releasing on a weekly basis if there are new commits, and use that as
an opportunity to highlight the excellent work their community has done.
If they can't do that now, lets empower them to tag and release and send
release announcements.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Joshua Harlow]

Daniel P. Berrange wrote:

On Fri, Jun 05, 2015 at 02:46:07PM +0200, Thierry Carrez wrote:

So.. summarizing the various options again:

Plan A
Just drop stable point releases.
(-) No more release notes
(-) Lack of reference points to compare installations

Plan B
Push date-based tags across supported projects from time to time.
(-) Encourages to continue using same version across the board
(-) Almost as much work as making proper releases

Plan C
Let projects randomly tag point releases whenever
(-) Still a bit costly in terms of herding cats

Plan D
Drop stable point releases, publish per-commit tarballs
(-) Requires some infra changes, takes some storage space

Plans B, C and D also require some release note / changelog generation
from data maintained *within* the repository.

Personally I think the objections raised against plan A are valid. I
like plan D, since it's more like releasing every commit than "not
releasing anymore". I think it's the most honest trade-off. I could go
with plan C, but I think it's added work for no additional value to the
user.


I don't see a whole lot of difference between plan A and D.
Publishing per-commit tarballs is merely saving the downstream
users the need to run a 'git archive' command, and providing
some auto-generated changelog that's already available from
'git log'.

If the downsteam consumer has their own extra patches ontop of the
stable branch, then it seems D is even less useful than A.


+1 to this, and I'm 99% sure every big/medium/all(?) cloud has extra 
patches on-top of stable branches; so publishing a tarball, meh, it will 
have to be regenerated anyway (with said patches) from the git commit it 
came from...




Regards,
Daniel


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Matthew Thode]

On 06/05/2015 09:34 AM, Alan Pevec wrote:
>> If the downsteam consumer has their own extra patches ontop of the
>> stable branch, then it seems D is even less useful than A.
> 
> It is not - downstream (speaking for RDO) I would keep Version:
> 2015.1.N where N is stable patch# so that we have a common reference
> point with other distros.
> Our patches, if any, then come on top this, so it's clear which
> patches are added by distro.
> 
> Cheers,
> Alan
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
Yes, we use a revbump strategy here for patches and changing
2015.0.0-r10 or something.  I think this is the standard way most
distros signify changes from the base package.

-- 
Matthew Thode



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Tristan Cacqueray]

On 06/05/2015 05:46 AM, Thierry Carrez wrote:
> So.. summarizing the various options again:
> 
> Plan A
> Just drop stable point releases.
> (-) No more release notes
> (-) Lack of reference points to compare installations
> 
> Plan B
> Push date-based tags across supported projects from time to time.
> (-) Encourages to continue using same version across the board
> (-) Almost as much work as making proper releases
> 
> Plan C
> Let projects randomly tag point releases whenever
> (-) Still a bit costly in terms of herding cats
> 
> Plan D
> Drop stable point releases, publish per-commit tarballs
> (-) Requires some infra changes, takes some storage space
> 
> Plans B, C and D also require some release note / changelog generation
> from data maintained *within* the repository.
> 
> Personally I think the objections raised against plan A are valid. I
> like plan D, since it's more like releasing every commit than "not
> releasing anymore". I think it's the most honest trade-off. I could go
> with plan C, but I think it's added work for no additional value to the
> user.
> 
> What would be your preferred option ?
> 

Apologize if I'm off-track here, but Plan A and D seems like a steep
change for users. Imo having stable release (at least between two
releases) is a valid use-case.

I guess Plan C is the preferred option, along with a "stable-release"
tag, projects that opt-in would have the responsibility to create stable
branches and maintain them.

Regards,
Tristan



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Alan Pevec]

> If the downsteam consumer has their own extra patches ontop of the
> stable branch, then it seems D is even less useful than A.

It is not - downstream (speaking for RDO) I would keep Version:
2015.1.N where N is stable patch# so that we have a common reference
point with other distros.
Our patches, if any, then come on top this, so it's clear which
patches are added by distro.

Cheers,
Alan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Matthew Thode]

On 06/05/2015 07:46 AM, Thierry Carrez wrote:
> So.. summarizing the various options again:
> 
> Plan A
> Just drop stable point releases.
> (-) No more release notes
> (-) Lack of reference points to compare installations
> 
> Plan B
> Push date-based tags across supported projects from time to time.
> (-) Encourages to continue using same version across the board
> (-) Almost as much work as making proper releases
> 
> Plan C
> Let projects randomly tag point releases whenever
> (-) Still a bit costly in terms of herding cats
> 
> Plan D
> Drop stable point releases, publish per-commit tarballs
> (-) Requires some infra changes, takes some storage space
> 
> Plans B, C and D also require some release note / changelog generation
> from data maintained *within* the repository.
> 
> Personally I think the objections raised against plan A are valid. I
> like plan D, since it's more like releasing every commit than "not
> releasing anymore". I think it's the most honest trade-off. I could go
> with plan C, but I think it's added work for no additional value to the
> user.
> 
> What would be your preferred option ?
> 
Either A or D are what I will likely do.  Also, I'll likely build infra
on my end to generate ebuilds like B proposes.

-- 
-- Matthew Thode (prometheanfire)



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thierry Carrez]

Daniel P. Berrange wrote:
> On Fri, Jun 05, 2015 at 02:46:07PM +0200, Thierry Carrez wrote:
>> Plan A
>> Just drop stable point releases.
>> (-) No more release notes
>> (-) Lack of reference points to compare installations
>>
>> Plan B
>> Push date-based tags across supported projects from time to time.
>> (-) Encourages to continue using same version across the board
>> (-) Almost as much work as making proper releases
>>
>> Plan C
>> Let projects randomly tag point releases whenever
>> (-) Still a bit costly in terms of herding cats
>>
>> Plan D
>> Drop stable point releases, publish per-commit tarballs
>> (-) Requires some infra changes, takes some storage space
>>
>> Plans B, C and D also require some release note / changelog generation
>> from data maintained *within* the repository.
>>
>> [...]
> 
> I don't see a whole lot of difference between plan A and D.
> Publishing per-commit tarballs is merely saving the downstream
> users the need to run a 'git archive' command, and providing
> some auto-generated changelog that's already available from
> 'git log'.

I guess the main difference is that we would still "release" something
(i.e. generate, version, publish and archive a tarball) in case of D,
which makes it easier to reuse a known point. It would also exhibit
version numbers more visibly, which makes referencing them slightly
easier (currently you have to look into the generated, transient
$PROJECT-stable-$SERIES.tar.gz tarball to see that version number).

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Jeremy Stanley]

On 2015-06-05 14:56:30 +0200 (+0200), Thierry Carrez wrote:
[...]
> I was wondering if we could switch to post-versioning on stable
> branches, and basically generate:
> 
> "2015.1.0.post38"
[...]

I think the recommendation from the PyPI maintainers is to not use
.postN suffixes since they are intended to indicate non-code
changes.

We could look at a mode for PBR which just adds a fourth unqualified
integer component that increments per patchset from the last known
tag (though that's technically not SemVer any longer)? Or maybe we
could have a PBR mode that just increments the third integer
component of the version for every commit without needing a new tag?
Or we could just push a corresponding tag for each merged commit
(though that essentially brings us back to the "tag arbitrarily"
option)?
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Daniel P. Berrange]

On Fri, Jun 05, 2015 at 02:46:07PM +0200, Thierry Carrez wrote:
> So.. summarizing the various options again:
> 
> Plan A
> Just drop stable point releases.
> (-) No more release notes
> (-) Lack of reference points to compare installations
> 
> Plan B
> Push date-based tags across supported projects from time to time.
> (-) Encourages to continue using same version across the board
> (-) Almost as much work as making proper releases
> 
> Plan C
> Let projects randomly tag point releases whenever
> (-) Still a bit costly in terms of herding cats
> 
> Plan D
> Drop stable point releases, publish per-commit tarballs
> (-) Requires some infra changes, takes some storage space
> 
> Plans B, C and D also require some release note / changelog generation
> from data maintained *within* the repository.
> 
> Personally I think the objections raised against plan A are valid. I
> like plan D, since it's more like releasing every commit than "not
> releasing anymore". I think it's the most honest trade-off. I could go
> with plan C, but I think it's added work for no additional value to the
> user.

I don't see a whole lot of difference between plan A and D.
Publishing per-commit tarballs is merely saving the downstream
users the need to run a 'git archive' command, and providing
some auto-generated changelog that's already available from
'git log'.

If the downsteam consumer has their own extra patches ontop of the
stable branch, then it seems D is even less useful than A.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thierry Carrez]

Jeremy Stanley wrote:
> On 2015-06-01 15:57:17 + (+), Jeremy Stanley wrote:
> [...]
>> The biggest hurdle is that we'd need a separate upload job name
>> for those since the current version of Zuul lacks a way to run a
>> particular job for different branches in different pipelines (we'd
>> want to do versioned uploads for all pre-release and release
>> pipeline refs, but also for post pipeline refs only when the
>> branch name is like stable/.*).
> 
> Actually, scratch that. It's a bit more complicated since the post
> pipeline isn't actually branch-relevant. We'd need to tweak the
> tarball and wheel creation scripts to check the containing branch,
> like we do for some proposal jobs. Still, I think it wouldn't be too
> hard.

Exploring plan D, I was looking at the versions we currently generate on
stable branches and I think they would not convey the right message:

"2015.1.1.dev38"

- but there won't be a 2015.1.1 !
- but this is not "under development" !

I was wondering if we could switch to post-versioning on stable
branches, and basically generate:

"2015.1.0.post38"

... which would convey the right message.

I /think/ all it would take would be, as the first post-release commit
to the stable branch, to remove the preversion from setup.cfg (rather
than bump it to the next .1). I think pbr would switch to postversioning
in that case and generate postX versions from the last tag in the branch.

Not sure we would do that for stable/kilo, though, since we already
pushed 2015.1.1.devX versions in the wild.

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thierry Carrez]

So.. summarizing the various options again:

Plan A
Just drop stable point releases.
(-) No more release notes
(-) Lack of reference points to compare installations

Plan B
Push date-based tags across supported projects from time to time.
(-) Encourages to continue using same version across the board
(-) Almost as much work as making proper releases

Plan C
Let projects randomly tag point releases whenever
(-) Still a bit costly in terms of herding cats

Plan D
Drop stable point releases, publish per-commit tarballs
(-) Requires some infra changes, takes some storage space

Plans B, C and D also require some release note / changelog generation
from data maintained *within* the repository.

Personally I think the objections raised against plan A are valid. I
like plan D, since it's more like releasing every commit than "not
releasing anymore". I think it's the most honest trade-off. I could go
with plan C, but I think it's added work for no additional value to the
user.

What would be your preferred option ?

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Jeremy Stanley]

On 2015-06-01 15:57:17 + (+), Jeremy Stanley wrote:
[...]
> The biggest hurdle is that we'd need a separate upload job name
> for those since the current version of Zuul lacks a way to run a
> particular job for different branches in different pipelines (we'd
> want to do versioned uploads for all pre-release and release
> pipeline refs, but also for post pipeline refs only when the
> branch name is like stable/.*).

Actually, scratch that. It's a bit more complicated since the post
pipeline isn't actually branch-relevant. We'd need to tweak the
tarball and wheel creation scripts to check the containing branch,
like we do for some proposal jobs. Still, I think it wouldn't be too
hard.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Jeremy Stanley]

On 2015-06-01 12:20:22 +0100 (+0100), Dimitri John Ledkov wrote:
[...]
> If the stable tarball snapshots had stable, ever increasing names, I
> would switch to using those straight away. (Out of which $ git
> describe --tags satisfies said requirement)

Current PBR releases already create monotonically-increasing
SemVer-like PEP-440-compliant versions for any commit, and name the
resulting sdist tarballs and wheels accordingly. It would be fairly
easy right now to switch stable branch tarball uploads on
tarballs.openstack.org to do that rather than constantly replace a
branch tip tarball. The biggest hurdle is that we'd need a separate
upload job name for those since the current version of Zuul lacks a
way to run a particular job for different branches in different
pipelines (we'd want to do versioned uploads for all pre-release and
release pipeline refs, but also for post pipeline refs only when the
branch name is like stable/.*).
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Jeremy Stanley]

On 2015-06-01 17:32:03 +0200 (+0200), Alan Pevec wrote:
[...]
> and *Plan D* would be to start doing automatic per-project
> micro-versions on each commit: e.g. 2015.1.N where N is increased on
> each commit. There's just TBD item how to provide source tarballs for
> this.
[...]

That would happen automatically (they're already generated and
uploaded to tarballs.openstack.org each time a tag is pushed).
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Jeremy Stanley]

On 2015-06-01 12:04:34 +0200 (+0200), Thierry Carrez wrote:
> Thierry Carrez wrote:
[...]
> > 2. it would be difficult to get proper release notes
> > 
> > If we don't have point releases anymore, we don't have release notes
> > anymore. Release notes contain various types of information: the list of
> > security fixes, the occasional upgrade warning, and the list of bugfixes.
> 
> We'd probably have to find a way to provide the same information in the
> tarball itself, so that if you picked any of them you could still get a
> list of the fixes in there.
[...]

PBR has the ability to generate a ChangeLog as part of the sdist
tarball generation. If the ChangeLog itself isn't suitable, we could
do something akin to what Doug described about scraping commit
messages for particular specially-formatted header lines and then
maybe have a separate release notes sdist hook in PBR. Or we could
just expect projects to update a release notes file in that repo on
any stable branch change which implies some action on the part of
the downstream consumers.

As to the separate concern raised by a couple of respondents for a
lack of signatures on stable branch tarballs (be they per-commit or
arbitrarily tagged by their respective project teams), I've been
itching to implement some mechanical generation of detached
signatures to publish along with artifacts uploaded to
tarballs.openstack.org and PyPI anyway. We could presumably leverage
that. It wouldn't be a human signature, just a signature made by a
trusted host in the infrastructure build chain whose key was signed
by some of the infrastructure root admins, but would at least attest
to the integrity of the file once uploaded to the point of
publication.
-- 
Jeremy Stanley


signature.asc
Description: Digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Haïkel]

2015-06-01 17:32 GMT+02:00 Alan Pevec :
>> *Plan C* would be to just let projects tag stable point releases from
>> time to time. That would solve all the original stated problems. And
>> that would solve objections 2 and 3, which I think are the most valid ones.
>
> and *Plan D* would be to start doing automatic per-project
> micro-versions on each commit: e.g. 2015.1.N where N is increased on
> each commit. There's just TBD item how to provide source tarballs for
> this.

+1 for micro-versions rather than raw git checksums.

> This would solve 3. reference points for OSSAs and relnotes.
> Solution for 2. could be to add doc/source/release-notes.rst for each
> project wishing to maintain it, docs are already published for each
> branch e.g. http://docs.openstack.org/developer/keystone/kilo/
>
> Cheers,
> Alan
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Ian Cordasco]

On 6/1/15, 06:20, "Dimitri John Ledkov"  wrote:

>On 29 May 2015 at 14:41, Thierry Carrez  wrote:
>> Hi everyone,
>>
>> TL;DR:
>> - We propose to stop tagging coordinated point releases (like 2015.1.1)
>> - We continue maintaining stable branches as a trusted source of stable
>> updates for all projects though
>>
>
>Ideally I would still want to see tarballs published, even if they are
>generated with a $ git describe --tags name in it.
>
>E.g. keystone-2015.1.0-3-g1373bfa.tar.gz
>
>keystone-stable-kilo.tar.gz -> is really bad name imho, as it's ever
>changing tarball. It would be nice if that was e.g. a symlink to a
>keystone-2015.1.0-3-g1373bfa.tar.gz if said snapshot is latest.
>
>If the stable tarball snapshots had stable, ever increasing names, I
>would switch to using those straight away. (Out of which $ git
>describe --tags satisfies said requirement)

Except this won't help anyone deploying with pip (which I know people
don't believe actually happens). That's not a valid pip version number
(neither 2015.1.0-3-g1373bfa nor stable-kilo are valid). If someone were
to use this archive with something like

pip install 'keystone>=2015.1.0' --find-links
https://stable-archive.openstack.org/kilo/

Then they wouldn't find anything new. If the releases are to be
auto-versioned, they should be done in a different way.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Alan Pevec]

> *Plan C* would be to just let projects tag stable point releases from
> time to time. That would solve all the original stated problems. And
> that would solve objections 2 and 3, which I think are the most valid ones.

and *Plan D* would be to start doing automatic per-project
micro-versions on each commit: e.g. 2015.1.N where N is increased on
each commit. There's just TBD item how to provide source tarballs for
this.
This would solve 3. reference points for OSSAs and relnotes.
Solution for 2. could be to add doc/source/release-notes.rst for each
project wishing to maintain it, docs are already published for each
branch e.g. http://docs.openstack.org/developer/keystone/kilo/

Cheers,
Alan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Dimitri John Ledkov]

On 29 May 2015 at 14:41, Thierry Carrez  wrote:
> Hi everyone,
>
> TL;DR:
> - We propose to stop tagging coordinated point releases (like 2015.1.1)
> - We continue maintaining stable branches as a trusted source of stable
> updates for all projects though
>

Ideally I would still want to see tarballs published, even if they are
generated with a $ git describe --tags name in it.

E.g. keystone-2015.1.0-3-g1373bfa.tar.gz

keystone-stable-kilo.tar.gz -> is really bad name imho, as it's ever
changing tarball. It would be nice if that was e.g. a symlink to a
keystone-2015.1.0-3-g1373bfa.tar.gz if said snapshot is latest.

If the stable tarball snapshots had stable, ever increasing names, I
would switch to using those straight away. (Out of which $ git
describe --tags satisfies said requirement)

-- 
Regards,

Dimitri.
Pura Vida!

https://clearlinux.org
Open Source Technology Center
Intel Corporation (UK) Ltd. - Co. Reg. #1134945 - Pipers Way, Swindon SN3 1RJ.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Matthias Runge]

On 01/06/15 12:10, Flavio Percoco wrote:


Is this a real problem? What are *tarball timestamps* used for in the
packaging world?

I'm sure there's a way we can workaround this issue.


timestamps just give you a hint, how old the source actually is, not 
when a packager downloaded the tarball somewhere. It just gives you a 
more realistic idea, how ancient the ancient code release is.




And: you probably want some hashes to verify, your downloaded tarball
is actually, what you wanted.


These can be generated as well. You can generate a tarball hash for
each commit and keep it around. The hash shouldn't change if the
tarball is generated on-the-fly. You could actually generate it
on-the-fly as well.
Sure, you can. You still need to provide that info. Ideally you'd 
prepare a signed file containing your hash.


I mean, something comparable to:

http://centos.bio.lmu.de/7/isos/x86_64/sha256sum.txt.asc

(for CentOS 7 iso files).


Matthias


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Flavio Percoco]

On 01/06/15 11:27 +0200, Matthias Runge wrote:

On 01/06/15 10:29, Flavio Percoco wrote:


AFAIK, these tarballs are generated on the flight in GH. Couldn't we
do the same ?


s/flight/fly+(facepalm)/ LOL, on the *flight* would be really hard.

The issue with on-the-fly generation is: timestamps are actually new, 
and not the same as when the release was tagged.


Is this a real problem? What are *tarball timestamps* used for in the
packaging world?

I'm sure there's a way we can workaround this issue.



And: you probably want some hashes to verify, your downloaded tarball 
is actually, what you wanted.


These can be generated as well. You can generate a tarball hash for
each commit and keep it around. The hash shouldn't change if the
tarball is generated on-the-fly. You could actually generate it
on-the-fly as well.

Cheers,
Flavio

--
@flaper87
Flavio Percoco


pgpJ147KU7g8S.pgp
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thierry Carrez]

Thierry Carrez wrote:
> Thanks everyone for the constructive discussion on this. Lots of good
> stuff. Let me summarize the objections so far:
> 
> 1. we'd lose the focus rhythm
> 
> Point releases triggered some attention to stable branches: the
> necessity to fix the CI there when it's broken, and a surge in backports
> activity.

The focus rhythm was (a) less effective recently as more projects just
ignored the stable release windows and (b) less necessary since we
switched to per-project stable maintenance teams that regularly backport
stuff. I actually tend to think that getting rid of the "let's care
about stable branches only every 2 months" mentality would be a plus.

> 2. it would be difficult to get proper release notes
> 
> If we don't have point releases anymore, we don't have release notes
> anymore. Release notes contain various types of information: the list of
> security fixes, the occasional upgrade warning, and the list of bugfixes.

We'd probably have to find a way to provide the same information in the
tarball itself, so that if you picked any of them you could still get a
list of the fixes in there.

> 3. we'd lose reference points
> 
> Tagged versions made great common reference points to share information
> between various distros/deployments. If a distro packaged 2014.2.2, you
> could take the same tag and package it and assume it was the same thing.
> They also served as "fixed in version X" information on OSSAs.

That one is a valid concern. It is true that the proposed change will
result in making it more difficult to compare post-release deployments
and google for "fixed in 2015.1.1" information. Please note that this
"reference point" was rather fuzzy, since distros would end up applying
their own extra patches on top of our tag, so you are actually not
comparing the exact same thing anyway. But I'd agree that it provided
marginal value.

> 4. we'd lose the "works well together" piece of information
> 
> The various pieces tagged with the same version number could be assumed
> to work well together. Recovering that would have to be pieced back
> together from commit dates and/or openstack/openstack stable branches.

I think this one is an illusion. Stable branch policy enforces that
there is no change in API or behavior in stable branches. So you could
technically take any commit from any branch and it should work as well
together as the "release" one. For those who really want a "CI-tested
combination", I think they can go through the extra hassle of
determining one through openstack/openstack stable branches.

So.. what now ? What would be an alternate solution ?

Plan B
==

We could continue to push random synchronization tags to solve (3). We'd
get rid of most of the release process and just tag things tested
together on a given date (solving (4)). We should find a way to
outsource release notes maintenance to each project, possibly by
maintaining those in stable branches directly (solving (2)).

For kilo we can still call those 2015.1.X and apply them on the Kilo
"integrated release" set of projects.

For liberty we will no longer have an inner sanctum of "integrated"
projects. So we could tag all "server" OpenStack projects which maintain
a stable branch. Since versioning will continue to diverge, the version
tagged will be different for every project, which will be confusing. So
we could also additionally tag with something like "liberty SP1" across
the board to facilitate finding the various pieces tagged at the same
moment.

Coming back to our original list of problems we wanted to solve:

> (1) Projects do not all follow the same versioning, so some projects
> (like Swift) were not part of the "stable point releases". More and more
> projects are considering issuing intermediary releases (like Swift
> does), like Ironic. That would result in a variety of version numbers,
> and ultimately less and less projects being able to have a common
> "2015.1.1"-like version.

I think the alternate proposal is a bit confusing (since over the long
run projects will not have the same version numbers, but people will be
used to assume that anything tagged with the same version number was
issued around the same time).

> (2) Producing those costs a non-trivial amount of effort on a very small
> team of volunteers, especially with projects caring about stable
> branches in various amounts. We were constantly missing the
> pre-announced dates on those ones. Looks like that effort could be
> better spent improving the stable branches themselves and keeping them
> working.

The alternate proposal is slightly less costly than the current
situation (no stable release prep period, just a very large set of tags
to push), but obviously more costly than just getting rid of the whole idea.

> (3) The resulting "stable point releases" are mostly useless. Stable
> branches are supposed to be always usable, and the "released" version
> did not undergo significantly more testing. Issuing them a

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Matthias Runge]

On 01/06/15 10:29, Flavio Percoco wrote:


AFAIK, these tarballs are generated on the flight in GH. Couldn't we
do the same ?
The issue with on-the-fly generation is: timestamps are actually new, 
and not the same as when the release was tagged.


And: you probably want some hashes to verify, your downloaded tarball is 
actually, what you wanted.


Matthias

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Robert Collins]

On 1 Jun 2015 8:07 pm, "Alan Pevec"  wrote:
>
> >> One issue is how would we provide source tarballs, statically hosting
> >> tarballs for each and every micro version is not realistic, also those
> >> wouldn't be signed.
> >
> > Sorry, but why isn't it realistic, and why wouldn't they be signed?
>
> I thought storage requirements to generate tarball for each commit for
> each project is unrealistic but infra could tell.

There are only a small number of commits to stake branches... I suspect we
could easily archive an sdist per commit there.

> Re. signing - I was judging from current stable release procedure[*]
> where tarballs are manually signed by the release manager before
> upload to Launchpad

That seems like something we could do differently if we were to archive an
sdist power commit on stable.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Thierry Carrez]

Thierry Carrez wrote:
> TL;DR:
> - We propose to stop tagging coordinated point releases (like 2015.1.1)
> - We continue maintaining stable branches as a trusted source of stable
> updates for all projects though

Thanks everyone for the constructive discussion on this. Lots of good
stuff. Let me summarize the objections so far:

1. we'd lose the focus rhythm

Point releases triggered some attention to stable branches: the
necessity to fix the CI there when it's broken, and a surge in backports
activity.

2. it would be difficult to get proper release notes

If we don't have point releases anymore, we don't have release notes
anymore. Release notes contain various types of information: the list of
security fixes, the occasional upgrade warning, and the list of bugfixes.

3. we'd lose reference points

Tagged versions made great common reference points to share information
between various distros/deployments. If a distro packaged 2014.2.2, you
could take the same tag and package it and assume it was the same thing.
They also served as "fixed in version X" information on OSSAs.

4. we'd lose the "works well together" piece of information

The various pieces tagged with the same version number could be assumed
to work well together. Recovering that would have to be pieced back
together from commit dates and/or openstack/openstack stable branches.

Missed anything ?

Please see my following email as I try to address each of those.

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Ihar Hrachyshka]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 05/29/2015 11:46 PM, Haïkel wrote:
> A release version makes it easy to know what fixes are shipped in a
> package. If you rebase on stable branches, then you can just put
> the git sha1sum (though, it's not very friendly) in the version,
> and leverage git branch --contains to find out if you fix is
> included. Some distributors may choose to use their own release
> scheme, adding complexity to this simple but common problem. Other
> may choose to cherry-pick which adds more complexity than the 
> previous scenario.
> 
> Let's say you're an operator and you want to check if a CVE is
> shipped in all your nodes, if you can't check with just the release
> version, it will be complicated. It could be a barrier for
> heterogeneous systems

Is it possible to give tagging permissions to projects themselves? No
time synchronization between projects, no freeze dates, nothing, just
SemVer compatible version bumps e.g. once per two weeks or month.
Would it fix your problem?

Ihar
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQEcBAEBCAAGBQJVbBp7AAoJEC5aWaUY1u57GugH/3JL9Izi7zULUHMCkPwsjE+w
qmkd25f3tMj9GxHpv8CbmLyz3H+5MtZfJHgX+AJYhkX/nF0SRkVvURdbYEDD4mrB
8w8W8nieD02KgWtglmIvulJs3atYmrXpLf38s7aWkciIp3MuUI0qEQ2LzZSV8JJq
50igFTle6IRR6NVbf391+nxYO33Eqv8Ckw7wVknJX22V2HueJuDzIgqGZeIOzfu4
drqzqSua2ZBhAJVR3bscdV+hSVieW2c2C9BLPaL3gG8/SD/7Qt/Ilj2lWgcTyAiy
LIHZt4sq7KPbm2AULItO+HooGHlNJLVL1yoBK4yFO+KvPNIZvtaF91YJ2+hn1UQ=
=Apa3
-END PGP SIGNATURE-

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Flavio Percoco]

On 01/06/15 10:07 +0200, Alan Pevec wrote:

One issue is how would we provide source tarballs, statically hosting
tarballs for each and every micro version is not realistic, also those
wouldn't be signed.


Sorry, but why isn't it realistic, and why wouldn't they be signed?


I thought storage requirements to generate tarball for each commit for
each project is unrealistic but infra could tell.
Re. signing - I was judging from current stable release procedure[*]
where tarballs are manually signed by the release manager before
upload to Launchpad.


AFAIK, these tarballs are generated on the flight in GH. Couldn't we
do the same ?

Flavio

--
@flaper87
Flavio Percoco


pgpt5wIdnMRGi.pgp
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Flavio Percoco]

On 29/05/15 09:44 -0500, Matt Riedemann wrote:



On 5/29/2015 8:41 AM, Thierry Carrez wrote:

Hi everyone,

TL;DR:
- We propose to stop tagging coordinated point releases (like 2015.1.1)
- We continue maintaining stable branches as a trusted source of stable
updates for all projects though

Long version:

At the "stable branch" session in Vancouver we discussed recent
evolutions in the stable team processes and how to further adapt the
work of the team in a "big tent" world.

One of the key questions there was whether we should continue doing
stable point releases. Those were basically tags with the same version
number ("2015.1.1") that we would periodically push to the stable
branches for all projects.

Those create three problems.

(1) Projects do not all follow the same versioning, so some projects
(like Swift) were not part of the "stable point releases". More and more
projects are considering issuing intermediary releases (like Swift
does), like Ironic. That would result in a variety of version numbers,
and ultimately less and less projects being able to have a common
"2015.1.1"-like version.

(2) Producing those costs a non-trivial amount of effort on a very small
team of volunteers, especially with projects caring about stable
branches in various amounts. We were constantly missing the
pre-announced dates on those ones. Looks like that effort could be
better spent improving the stable branches themselves and keeping them
working.

(3) The resulting "stable point releases" are mostly useless. Stable
branches are supposed to be always usable, and the "released" version
did not undergo significantly more testing. Issuing them actually
discourages people from taking whatever point in stable branches makes
the most sense for them, testing and deploying that.

The suggestion we made during that session (and which was approved by
the session participants) is therefore to just get rid of the "stable
point release" concept altogether for non-libraries. That said:

- we'd still do individual point releases for libraries (for critical
bugs and security issues), so that you can still depend on a specific
version there

- we'd still very much maintain stable branches (and actually focus our
efforts on that work) to ensure they are a continuous source of safe
upgrades for users of a given series

Now we realize that the cross-section of our community which was present
in that session might not fully represent the consumers of those
artifacts, which is why we expand the discussion on this mailing-list
(and soon on the operators ML).

If you were a consumer of those and will miss them, please explain why.
In particular, please let us know how consuming that version (which was
only made available every n months) is significantly better than picking
your preferred time and get all the current stable branch HEADs at that
time.

Thanks in advance for your feedback,

[1] https://etherpad.openstack.org/p/YVR-relmgt-stable-branch



To reiterate what I said in the session, for my team personally (IBM), 
we don't align with the point release schedules on stable anyway, we 
release our own stable release fix packs as needed on our own 
schedules, so in that regard I don't see a point in the stable point 
releases - especially since most of the time I don't know when those 
are going to be anyway so we can't plan for them accurately.


Having said that, what I mentioned in IRC the other day is the one 
upside I see to the point releases is it is a milestone that requires 
focus from the stable maintainers, which means if stable has been 
broken for a few weeks and no one has really noticed, converging on a 
stable point release at least forces attention there.


I don't think that is a very good argument for keeping stable point 
releases though, since as you said we don't do any additional testing 
above and beyond what normally happens in the Jenkins runs.  Some of 
the distributions might have extra regression testing scenarios, I'm 
not sure, but no one really spoke to that in the session from the 
distros that were present - I assume they do, but they can do that on 
their own schedule anyway IMO.


I agree it's not a good point in favor of point releases. If anything,
that's just showing some issues with the stable workflow that probably
require more attention than the milestone itself.


I am a bit cynical about thinking that dropping point releases will 
make people spend more time on caring about the health of the stable 
branches (persistent gate failures) or stale changes out for review.  
I combed through a lot of open stable/icehouse changes yesterday and 
there were many that should have been abandoned 6 months ago but were 
just sitting there, and others that were good fixes to have and should 
have been merged by now.


More than making people focus more on the health of stable branches
the whole goal is to remove something that is(?) not necessary.
Basically, this proposal removes unnecessary(?) work from

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Alan Pevec]

>> One issue is how would we provide source tarballs, statically hosting
>> tarballs for each and every micro version is not realistic, also those
>> wouldn't be signed.
>
> Sorry, but why isn't it realistic, and why wouldn't they be signed?

I thought storage requirements to generate tarball for each commit for
each project is unrealistic but infra could tell.
Re. signing - I was judging from current stable release procedure[*]
where tarballs are manually signed by the release manager before
upload to Launchpad.


Cheers,
Alan

[*] https://wiki.openstack.org/wiki/StableBranchRelease#Upload_The_Release

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Ihar Hrachyshka]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 05/29/2015 06:24 PM, Jeremy Stanley wrote:
> On 2015-05-29 17:50:01 +0200 (+0200), Ihar Hrachyshka wrote: [...]
>> if we attempt to fix a security issue that has a backwards 
>> incompatible fix, then we are forced in introducing a new 
>> configuration option to opt-in the new secure world.
> [...]
> 
> To my knowledge that's how we've handled these in the past anyway, 
> accompanied by publication of a security note (not advisory) 
> suggesting the steps necessary to enable the breaking change when 
> opting into the bug fix.
> 

AFAIK at least for keep alive option for wsgi TCP connections we
haven't done it, at least specifically for stable branches (the fixes
were propagated into stable branches during several minor stable
releases).

Ihar
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQEcBAEBCAAGBQJVbBGWAAoJEC5aWaUY1u577wEH/iqzGLGlc+Rb9bYrWeGRcQxt
bHwbU6kIVPAiJkT9eJGzoRonte/2u/+a6xvIiCXF6kcxIzYF3H25htbDsLe+DWmL
A8jPiSUPYksA/R/9FBoRBZBGr3Qs11imJcQ4PuSMNkM9KiZmlr/yle4kYpAtX9bY
VSsSVasgfplXX+1yR2mm6e0cSQlYWy5y8P2eKRhwYLAzc9LYDOpS2wPnI3Pz+gOQ
Muf3ogYbcC5dlXjZ0skf1QiGdj0Tenm9oVrSot+XhvMqE/sym9SfaV6OvP3193kq
N1xryagRdtsX7DkUfDuolziPMC2x20Q0JZZAcL13hQEa2O8QAjdW747JCb5+lUg=
=YW79
-END PGP SIGNATURE-

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Ihar Hrachyshka]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 06/01/2015 02:20 AM, Steve Gordon wrote:
> - Original Message -
>> From: "Maish Saidel-Keesing"  To:
>> openstack-dev@lists.openstack.org
>> 
>> On 05/29/15 18:25, Matthew Thode wrote:
>>> On 05/29/2015 10:18 AM, Ihar Hrachyshka wrote:
 What about release notes? How can we now communicate some
 changes that require operator consideration or action?
 
 Ihar
 
 On 05/29/2015 03:41 PM, Thierry Carrez wrote:
> Hi everyone, TL;DR: - We propose to stop tagging
> coordinated point releases (like 2015.1.1) - We continue
> maintaining stable branches as a trusted source of stable
> updates for all projects though Long version: At the
> "stable branch" session in Vancouver we discussed recent 
> evolutions in the stable team processes and how to further
> adapt the work of the team in a "big tent" world. One of
> the key questions there was whether we should continue 
> doing stable point releases. Those were basically tags with
> the same version number ("2015.1.1") that we would
> periodically push to the stable branches for all projects. 
> Those create three problems. (1) Projects do not all follow
> the same versioning, so some projects (like Swift) were not
> part of the "stable point releases". More and more projects
> are considering issuing intermediary releases (like Swift
> does), like Ironic. That would result in a variety of
> version numbers, and ultimately less and less projects 
> being able to have a common "2015.1.1"-like version. (2)
> Producing those costs a non-trivial amount of effort on a
> very small team of volunteers, especially with projects
> caring about stable branches in various amounts. We were
> constantly missing the pre-announced dates on those ones.
> Looks like that effort could be better spent improving the
> stable branches themselves and keeping them working. (3)
> The resulting "stable point releases" are mostly useless. 
> Stable branches are supposed to be always usable, and the 
> "released" version did not undergo significantly more
> testing. Issuing them actually discourages people from
> taking whatever point in stable branches makes the most
> sense for them, testing and deploying that. The suggestion
> we made during that session (and which was approved by the
> session participants) is therefore to just get rid of the 
> "stable point release" concept altogether for
> non-libraries. That said: - we'd still do individual point
> releases for libraries (for critical bugs and security
> issues), so that you can still depend on a specific version
> there - we'd still very much maintain stable branches (and
> actually focus our efforts on that work) to ensure they are
> a continuous source of safe upgrades for users of a given
> series Now we realize that the cross-section of our
> community which was present in that session might not fully
> represent the consumers of those artifacts, which is why we
> expand the discussion on this mailing-list (and soon on the
> operators ML). If you were a consumer of those and will
> miss them, please explain why. In particular, please let us
> know how consuming that version (which was only made
> available every n months) is significantly better than
> picking your preferred time and get all the current stable
> branch HEADs at that time. Thanks in advance for your
> feedback, [1]
> https://etherpad.openstack.org/p/YVR-relmgt-stable-branch
 
 ___
___

 
OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

 
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
>>> for release notes just do git log between commit hashes?
>> Do you really think that is what an Operator will do? I do not
>> think is a realistic expectation or something that will work. -- 
>> Best Regards, Maish Saidel-Keesing
> 
> While I agree most operators probably don't want to necessarily dig
> this out of git themselves and it would need to be collated/exposed
> in a nicer way it seems like it would actually be more accurate
> than the current "release notes" for all the non-security bug fixes
> in stable which basically amount to a list of launchpad bug queries
> per project. You then have to sift through each bug to work out if
> the description reflects what was actually done etc:
> 
> https://wiki.openstack.org/wiki/ReleaseNotes/2014.2.3

I think the section that is really useful is "Known Issues and
Limitations". Sometimes it suggests operators to apply some
configuration changes to their nodes:

https://wiki.openstack.org/wiki/ReleaseNotes/2014.2.2

Ihar
-BEGIN PGP SIGN

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Robert Collins]

On 1 Jun 2015 10:50 am, "Alan Pevec"  wrote:
>
> 2015-05-29 18:30 GMT+02:00 Jeremy Stanley :
> > On 2015-05-29 16:30:12 +0100 (+0100), Dave Walker wrote:
> >> This is generally my opinion as-well, I always hoped that *every*
> >> commit would be considered a release rather than an arbitrary
> >> tagged date.
> > [...]
> >
> > If we switch away from lockstep major/minor release versioning
> > anyway (again separate discussion underway but seems a distinct
> > possibility) then I think the confusion over why stable point
> > releases are mismatched becomes less of an issue. At that point we
> > may want to reconsider and actually tag each of them with a
> > sequential micro (patch in semver terminology) version bump. Could
> > help in communication around security fixes in particular.
>
> Yes, if dropping stable point releases, sub-version schema is still
> needed for clear communication in OSSAs and proposed continuous
> releases notes.
> One issue is how would we provide source tarballs, statically hosting
> tarballs for each and every micro version is not realistic, also those
> wouldn't be signed.

Sorry, but why isn't it realistic, and why wouldn't they be signed?

Rob
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Alan Pevec]

> You will get different checksums with tar and/or gzip, you can check the
> extracted files and they should be the same.

Yes, content is the same, it's just difference in timestamps of
folders and generated files (ChangeLog, egg-info etc).

> I would like to see signed commits in the 'official' repos (at
> git.openstack.org), if only because relying on sha alone doesn't seem
> enough for some.

Yeah, maybe RPM and I are just too old and should drop reproducibility
requirement :)
It might be nitpicking to expect exact same bits but it seems that
only missing part is an option to tell sdist to set specific timestamp
on generated files and folders.


Cheers,
Alan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Maish Saidel-Keesing]

On 06/01/15 03:20, Steve Gordon wrote:

- Original Message -

From: "Maish Saidel-Keesing" 
To: openstack-dev@lists.openstack.org

On 05/29/15 18:25, Matthew Thode wrote:

On 05/29/2015 10:18 AM, Ihar Hrachyshka wrote:

What about release notes? How can we now communicate some changes that
require operator consideration or action?

Ihar

On 05/29/2015 03:41 PM, Thierry Carrez wrote:

Hi everyone,
TL;DR: - We propose to stop tagging coordinated point releases
(like 2015.1.1) - We continue maintaining stable branches as a
trusted source of stable updates for all projects though
Long version:
At the "stable branch" session in Vancouver we discussed recent
evolutions in the stable team processes and how to further adapt
the work of the team in a "big tent" world.
One of the key questions there was whether we should continue
doing stable point releases. Those were basically tags with the
same version number ("2015.1.1") that we would periodically push to
the stable branches for all projects.
Those create three problems.
(1) Projects do not all follow the same versioning, so some
projects (like Swift) were not part of the "stable point releases".
More and more projects are considering issuing intermediary
releases (like Swift does), like Ironic. That would result in a
variety of version numbers, and ultimately less and less projects
being able to have a common "2015.1.1"-like version.
(2) Producing those costs a non-trivial amount of effort on a very
small team of volunteers, especially with projects caring about
stable branches in various amounts. We were constantly missing the
pre-announced dates on those ones. Looks like that effort could be
better spent improving the stable branches themselves and keeping
them working.
(3) The resulting "stable point releases" are mostly useless.
Stable branches are supposed to be always usable, and the
"released" version did not undergo significantly more testing.
Issuing them actually discourages people from taking whatever point
in stable branches makes the most sense for them, testing and
deploying that.
The suggestion we made during that session (and which was approved
by the session participants) is therefore to just get rid of the
"stable point release" concept altogether for non-libraries. That
said:
- we'd still do individual point releases for libraries (for
critical bugs and security issues), so that you can still depend on
a specific version there
- we'd still very much maintain stable branches (and actually focus
our efforts on that work) to ensure they are a continuous source of
safe upgrades for users of a given series
Now we realize that the cross-section of our community which was
present in that session might not fully represent the consumers of
those artifacts, which is why we expand the discussion on this
mailing-list (and soon on the operators ML).
If you were a consumer of those and will miss them, please explain
why. In particular, please let us know how consuming that version
(which was only made available every n months) is significantly
better than picking your preferred time and get all the current
stable branch HEADs at that time.
Thanks in advance for your feedback,
[1] https://etherpad.openstack.org/p/YVR-relmgt-stable-branch

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


for release notes just do git log between commit hashes?

Do you really think that is what an Operator will do? I do not think is
a realistic expectation or something that will work.
--
Best Regards,
Maish Saidel-Keesing

While I agree most operators probably don't want to necessarily dig this out of git 
themselves and it would need to be collated/exposed in a nicer way it seems like it would 
actually be more accurate than the current "release notes" for all the 
non-security bug fixes in stable which basically amount to a list of launchpad bug 
queries per project. You then have to sift through each bug to work out if the 
description reflects what was actually done etc:

 https://wiki.openstack.org/wiki/ReleaseNotes/2014.2.3

-Steve
I agree - and if this could be automated in some way to make is 
presentable - that would be ideal

--
Best Regards,
Maish Saidel-Keesing

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Steve Gordon]

- Original Message -
> From: "Maish Saidel-Keesing" 
> To: openstack-dev@lists.openstack.org
> 
> On 05/29/15 18:25, Matthew Thode wrote:
> > On 05/29/2015 10:18 AM, Ihar Hrachyshka wrote:
> >> What about release notes? How can we now communicate some changes that
> >> require operator consideration or action?
> >>
> >> Ihar
> >>
> >> On 05/29/2015 03:41 PM, Thierry Carrez wrote:
> >>> Hi everyone,
> >>> TL;DR: - We propose to stop tagging coordinated point releases
> >>> (like 2015.1.1) - We continue maintaining stable branches as a
> >>> trusted source of stable updates for all projects though
> >>> Long version:
> >>> At the "stable branch" session in Vancouver we discussed recent
> >>> evolutions in the stable team processes and how to further adapt
> >>> the work of the team in a "big tent" world.
> >>> One of the key questions there was whether we should continue
> >>> doing stable point releases. Those were basically tags with the
> >>> same version number ("2015.1.1") that we would periodically push to
> >>> the stable branches for all projects.
> >>> Those create three problems.
> >>> (1) Projects do not all follow the same versioning, so some
> >>> projects (like Swift) were not part of the "stable point releases".
> >>> More and more projects are considering issuing intermediary
> >>> releases (like Swift does), like Ironic. That would result in a
> >>> variety of version numbers, and ultimately less and less projects
> >>> being able to have a common "2015.1.1"-like version.
> >>> (2) Producing those costs a non-trivial amount of effort on a very
> >>> small team of volunteers, especially with projects caring about
> >>> stable branches in various amounts. We were constantly missing the
> >>> pre-announced dates on those ones. Looks like that effort could be
> >>> better spent improving the stable branches themselves and keeping
> >>> them working.
> >>> (3) The resulting "stable point releases" are mostly useless.
> >>> Stable branches are supposed to be always usable, and the
> >>> "released" version did not undergo significantly more testing.
> >>> Issuing them actually discourages people from taking whatever point
> >>> in stable branches makes the most sense for them, testing and
> >>> deploying that.
> >>> The suggestion we made during that session (and which was approved
> >>> by the session participants) is therefore to just get rid of the
> >>> "stable point release" concept altogether for non-libraries. That
> >>> said:
> >>> - we'd still do individual point releases for libraries (for
> >>> critical bugs and security issues), so that you can still depend on
> >>> a specific version there
> >>> - we'd still very much maintain stable branches (and actually focus
> >>> our efforts on that work) to ensure they are a continuous source of
> >>> safe upgrades for users of a given series
> >>> Now we realize that the cross-section of our community which was
> >>> present in that session might not fully represent the consumers of
> >>> those artifacts, which is why we expand the discussion on this
> >>> mailing-list (and soon on the operators ML).
> >>> If you were a consumer of those and will miss them, please explain
> >>> why. In particular, please let us know how consuming that version
> >>> (which was only made available every n months) is significantly
> >>> better than picking your preferred time and get all the current
> >>> stable branch HEADs at that time.
> >>> Thanks in advance for your feedback,
> >>> [1] https://etherpad.openstack.org/p/YVR-relmgt-stable-branch
> >>
> >> __
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> > for release notes just do git log between commit hashes?
> Do you really think that is what an Operator will do? I do not think is
> a realistic expectation or something that will work.
> --
> Best Regards,
> Maish Saidel-Keesing

While I agree most operators probably don't want to necessarily dig this out of 
git themselves and it would need to be collated/exposed in a nicer way it seems 
like it would actually be more accurate than the current "release notes" for 
all the non-security bug fixes in stable which basically amount to a list of 
launchpad bug queries per project. You then have to sift through each bug to 
work out if the description reflects what was actually done etc:

https://wiki.openstack.org/wiki/ReleaseNotes/2014.2.3

-Steve

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Matthew Thode]

On 05/31/2015 05:50 PM, Alan Pevec wrote:
> 2015-05-29 18:30 GMT+02:00 Jeremy Stanley :
>> On 2015-05-29 16:30:12 +0100 (+0100), Dave Walker wrote:
>>> This is generally my opinion as-well, I always hoped that *every*
>>> commit would be considered a release rather than an arbitrary
>>> tagged date.
>> [...]
>>
>> If we switch away from lockstep major/minor release versioning
>> anyway (again separate discussion underway but seems a distinct
>> possibility) then I think the confusion over why stable point
>> releases are mismatched becomes less of an issue. At that point we
>> may want to reconsider and actually tag each of them with a
>> sequential micro (patch in semver terminology) version bump. Could
>> help in communication around security fixes in particular.
> 
> Yes, if dropping stable point releases, sub-version schema is still
> needed for clear communication in OSSAs and proposed continuous
> releases notes.
> One issue is how would we provide source tarballs, statically hosting
> tarballs for each and every micro version is not realistic, also those
> wouldn't be signed.
> RPM packages traditionally expect pristine upstream tarballs which can
> be verified and generating them from git is not reproducible e.g.
> right now in nova stable/kilo branch:
> python ./setup.py sdist
> mv dist/nova-2015.1.1.dev20.tar.gz dist/nova-2015.1.1.dev20.tar.gz-TAKE1
> python ./setup.py sdist
> diff dist/nova-2015.1.1.dev20.tar.gz-TAKE1 dist/nova-2015.1.1.dev20.tar.gz
> Binary files dist/nova-2015.1.1.dev20.tar.gz-TAKE1 and
> dist/nova-2015.1.1.dev20.tar.gz differ
> 
> Before dropping point releases, I would like to have:
> * idempotent sdist on the same SHA
> * dynamic tarball generation service like github archive
> * switch to micro-version i.e. current nova stable/kilo would be 2015.1.20
> 
> 
> Cheers,
> Alan
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
Generating tarballs from commit sha's isn't enough?

I'm personally thinking of installing a file somewhere that references
what commit hash the package was sourced from.  I'm thinking of doing
weekly releases.

Tarball generation would be nice.

You will get different checksums with tar and/or gzip, you can check the
extracted files and they should be the same.

I would like to see signed commits in the 'official' repos (at
git.openstack.org), if only because relying on sha alone doesn't seem
enough for some.

-- 
Matthew Thode



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Alan Pevec]

2015-05-29 18:30 GMT+02:00 Jeremy Stanley :
> On 2015-05-29 16:30:12 +0100 (+0100), Dave Walker wrote:
>> This is generally my opinion as-well, I always hoped that *every*
>> commit would be considered a release rather than an arbitrary
>> tagged date.
> [...]
>
> If we switch away from lockstep major/minor release versioning
> anyway (again separate discussion underway but seems a distinct
> possibility) then I think the confusion over why stable point
> releases are mismatched becomes less of an issue. At that point we
> may want to reconsider and actually tag each of them with a
> sequential micro (patch in semver terminology) version bump. Could
> help in communication around security fixes in particular.

Yes, if dropping stable point releases, sub-version schema is still
needed for clear communication in OSSAs and proposed continuous
releases notes.
One issue is how would we provide source tarballs, statically hosting
tarballs for each and every micro version is not realistic, also those
wouldn't be signed.
RPM packages traditionally expect pristine upstream tarballs which can
be verified and generating them from git is not reproducible e.g.
right now in nova stable/kilo branch:
python ./setup.py sdist
mv dist/nova-2015.1.1.dev20.tar.gz dist/nova-2015.1.1.dev20.tar.gz-TAKE1
python ./setup.py sdist
diff dist/nova-2015.1.1.dev20.tar.gz-TAKE1 dist/nova-2015.1.1.dev20.tar.gz
Binary files dist/nova-2015.1.1.dev20.tar.gz-TAKE1 and
dist/nova-2015.1.1.dev20.tar.gz differ

Before dropping point releases, I would like to have:
* idempotent sdist on the same SHA
* dynamic tarball generation service like github archive
* switch to micro-version i.e. current nova stable/kilo would be 2015.1.20


Cheers,
Alan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Maish Saidel-Keesing]

On 05/29/15 18:25, Matthew Thode wrote:

On 05/29/2015 10:18 AM, Ihar Hrachyshka wrote:

What about release notes? How can we now communicate some changes that
require operator consideration or action?

Ihar

On 05/29/2015 03:41 PM, Thierry Carrez wrote:

Hi everyone,
TL;DR: - We propose to stop tagging coordinated point releases
(like 2015.1.1) - We continue maintaining stable branches as a
trusted source of stable updates for all projects though
Long version:
At the "stable branch" session in Vancouver we discussed recent
evolutions in the stable team processes and how to further adapt
the work of the team in a "big tent" world.
One of the key questions there was whether we should continue
doing stable point releases. Those were basically tags with the
same version number ("2015.1.1") that we would periodically push to
the stable branches for all projects.
Those create three problems.
(1) Projects do not all follow the same versioning, so some
projects (like Swift) were not part of the "stable point releases".
More and more projects are considering issuing intermediary
releases (like Swift does), like Ironic. That would result in a
variety of version numbers, and ultimately less and less projects
being able to have a common "2015.1.1"-like version.
(2) Producing those costs a non-trivial amount of effort on a very
small team of volunteers, especially with projects caring about
stable branches in various amounts. We were constantly missing the
pre-announced dates on those ones. Looks like that effort could be
better spent improving the stable branches themselves and keeping
them working.
(3) The resulting "stable point releases" are mostly useless.
Stable branches are supposed to be always usable, and the
"released" version did not undergo significantly more testing.
Issuing them actually discourages people from taking whatever point
in stable branches makes the most sense for them, testing and
deploying that.
The suggestion we made during that session (and which was approved
by the session participants) is therefore to just get rid of the
"stable point release" concept altogether for non-libraries. That
said:
- we'd still do individual point releases for libraries (for
critical bugs and security issues), so that you can still depend on
a specific version there
- we'd still very much maintain stable branches (and actually focus
our efforts on that work) to ensure they are a continuous source of
safe upgrades for users of a given series
Now we realize that the cross-section of our community which was
present in that session might not fully represent the consumers of
those artifacts, which is why we expand the discussion on this
mailing-list (and soon on the operators ML).
If you were a consumer of those and will miss them, please explain
why. In particular, please let us know how consuming that version
(which was only made available every n months) is significantly
better than picking your preferred time and get all the current
stable branch HEADs at that time.
Thanks in advance for your feedback,
[1] https://etherpad.openstack.org/p/YVR-relmgt-stable-branch


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


for release notes just do git log between commit hashes?
Do you really think that is what an Operator will do? I do not think is 
a realistic expectation or something that will work.

--
Best Regards,
Maish Saidel-Keesing

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Haïkel]

2015-05-29 21:36 GMT+02:00 Dave Walker :
> Responses inline.
>
> On 29 May 2015 6:15 pm, "Haïkel"  wrote:
>>
>> 2015-05-29 15:41 GMT+02:00 Thierry Carrez :
>> > Hi everyone,
>> >
>> > TL;DR:
>> > - We propose to stop tagging coordinated point releases (like 2015.1.1)
>> > - We continue maintaining stable branches as a trusted source of stable
>> > updates for all projects though
>> >
>>
>> Hi,
>>
>> I'm one of the main maintainer of the packages for Fedora/RHEL/CentOS.
>> We try to stick as much as possible to upstream (almost zero
>> downstream patches),
>> and without intermediate releases, it will get difficult.
>
> If you consider *every* commit to be a release, then your life becomes
> easier. This is just a case of bumping the SemVer patch version per commit
> (as eloquently put by Jeremy).  We even have tooling to automate the version
> generation via pbr..
>
> Therefore, you might want to jump from X.X.100 to X.X.200 which would mean
> 100 commits since the last update.
>

We have continuous builds for every commit master for a while now, and
it's been a
great tool with CI to have early feedback (missing deps, integration
issues etc.).
We could easily reuse that platform to track stable branches.

The problem is that downstream QA/CI cycle of a package could be much
longer than between
two commits. So we'd end up jamming updates.
I'd rather not drop downstream QA as it testes integration bits, and
it's unlikely
to be something that could be upstream.


>> I'm personally not fond of this as it will lead to more fragmentation.
>> It may encourage
>> bad behaviors like shipping downstream patches for bug fixes and CVE
>> instead
>> of collaborating upstream to differentiate themselves.
>> For instance, if we had no point-based release, for issues tracking
>> purposes, we would
>> have to maintain our sets of tags somewhere.
>
> I disagree, each distro already does security patching and whilst I expect
> this to still happens, it actually *encourages* upstream first workflow as
> you can select a release on your own cadence that includes commits you need,
> for your users.
>

If they choose to rebase upon stable branches, you could also cherry-pick.

>> There's also the release notes issue that has already been mentioned.
>> Still continuous release notes won't solve the problem, as you wouldn't
>> be able to map these to the actual packages. Will we require operators
>> to find from which git commit, the packages were built and then try to
>> figure
>> out which fixes are and are not included?
>
> Can you provide more detail? I'm not understanding the problem.
>

A release version makes it easy to know what fixes are shipped in a package.
If you rebase on stable branches, then you can just put the git sha1sum (though,
it's not very friendly) in the version, and leverage git branch
--contains to find out
if you fix is included.
Some distributors may choose to use their own release scheme, adding
complexity to
this simple but common problem.
Other may choose to cherry-pick which adds more complexity than the
previous scenario.

Let's say you're an operator and you want to check if a CVE is shipped
in all your nodes,
if you can't check with just the release version, it will be complicated.
It could be a barrier for heterogeneous systems

>> > Long version:
>> >
>> > At the "stable branch" session in Vancouver we discussed recent
>> > evolutions in the stable team processes and how to further adapt the
>> > work of the team in a "big tent" world.
>> >
>> > One of the key questions there was whether we should continue doing
>> > stable point releases. Those were basically tags with the same version
>> > number ("2015.1.1") that we would periodically push to the stable
>> > branches for all projects.
>> >
>> > Those create three problems.
>> >
>> > (1) Projects do not all follow the same versioning, so some projects
>> > (like Swift) were not part of the "stable point releases". More and more
>> > projects are considering issuing intermediary releases (like Swift
>> > does), like Ironic. That would result in a variety of version numbers,
>> > and ultimately less and less projects being able to have a common
>> > "2015.1.1"-like version.
>> >
>>
>> And that's actually a pain point to track for these releases in which
>> OpenStack branch belong. And this is probably something that needs to
>> be resolved.
>>
>> > (2) Producing those costs a non-trivial amount of effort on a very small
>> > team of volunteers, especially with projects caring about stable
>> > branches in various amounts. We were constantly missing the
>> > pre-announced dates on those ones. Looks like that effort could be
>> > better spent improving the stable branches themselves and keeping them
>> > working.
>> >
>>
>> Agreed, but why not switching to a time-based release?
>> Regularly, we tag/generate/upload tarballs, this could even be automated.
>> As far as I'm concerned, I would be more happy to have more frequent
>> releases.
>>
>> > (

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Haïkel]

2015-05-29 21:23 GMT+02:00 Ian Cordasco :
>
>
> On 5/29/15, 12:14, "Haïkel"  wrote:
>
>>2015-05-29 15:41 GMT+02:00 Thierry Carrez :
>>> Hi everyone,
>>>
>>> TL;DR:
>>> - We propose to stop tagging coordinated point releases (like 2015.1.1)
>>> - We continue maintaining stable branches as a trusted source of stable
>>> updates for all projects though
>>>
>>
>>Hi,
>>
>>I'm one of the main maintainer of the packages for Fedora/RHEL/CentOS.
>>We try to stick as much as possible to upstream (almost zero
>>downstream patches),
>>and without intermediate releases, it will get difficult.
>
> Can you expound on why this is difficult? I believe you, but I want to
> understand it better.
>

It's impossible to ship a package for every commit, so that means we'll
end up to ship.
1. random commit => bad for tracking issues
2. time-based release (ie: rebuilding every 2 or 4 weeks)
3. cherry-picking commits from stable branches => leads to practical forks

>>I'm personally not fond of this as it will lead to more fragmentation.
>
> Could you explain this as well? Do you mean fragmentation between what
> distros are offering? In other words, Ubuntu is packaging Kilo @ SHA1 and
> RHEL is at SHA2. I'm not entirely certain that's a bad thing. That seems
> to give the packagers more freedom.
>

Freedom leads to fragmentation, it's not bad though I prefer collaborating
in stabilizing the same releases.
That's a personal preference, not an argument :)

>>It may encourage
>>bad behaviors like shipping downstream patches for bug fixes and CVE
>>instead
>>of collaborating upstream to differentiate themselves.
>
> Perhaps I'm wrong, but when a CVE is released, don't the downstream
> packagers usually patch whatever version they have and push that out?
> Isn't that the point of them being on an private list to receive embargoed
> notifications with the patches?
>

Yes, but everyone will have different strategies, when releasing a package as I
explained

>>For instance, if we had no point-based release, for issues tracking
>>purposes, we would
>>have to maintain our sets of tags somewhere.
>
> But, if I understand correct, downstream sometimes has patches they apply
> (or develop) to ensure the package is rock solid on their distribution.
> Those aren't always relevant upstream so you maintain them. How is this
> different?
>

Pure downstream patches are not a problem, though we aim to drop them
for our packages.
We'd need the aforementioned tags to track what we currently shipping in our
packages. Having a proper release version/changelog makes it easy to check
if you're shipping a fix or not either by release on release version
or git commit.

Usually, downstream patches are rebased against latest release.

>>There's also the release notes issue that has already been mentioned.
>>Still continuous release notes won't solve the problem, as you wouldn't
>>be able to map these to the actual packages. Will we require operators
>>to find from which git commit, the packages were built and then try to
>>figure
>>out which fixes are and are not included?
>
> I think this is wrong. If it's a continuously updated set of notes, then
> whatever SHA the head of stable/X is at will be the correct set of notes
> for that branch. If you decide to package a SHA earlier than that, then
> you would need to do this, but I'm not sure why you would want to package
> a SHA that isn't at the HEAD of that branch.
>

We were speaking about leveraging wiki, if we talking about Changelog shipped
in git, we agree then.
But that requires ensuring that changelogs are properly updated with
every commit,
then (which is not the case actually).

>>
>>> Long version:
>>>
>>> At the "stable branch" session in Vancouver we discussed recent
>>> evolutions in the stable team processes and how to further adapt the
>>> work of the team in a "big tent" world.
>>>
>>> One of the key questions there was whether we should continue doing
>>> stable point releases. Those were basically tags with the same version
>>> number ("2015.1.1") that we would periodically push to the stable
>>> branches for all projects.
>>>
>>> Those create three problems.
>>>
>>> (1) Projects do not all follow the same versioning, so some projects
>>> (like Swift) were not part of the "stable point releases". More and more
>>> projects are considering issuing intermediary releases (like Swift
>>> does), like Ironic. That would result in a variety of version numbers,
>>> and ultimately less and less projects being able to have a common
>>> "2015.1.1"-like version.
>>>
>>
>>And that's actually a pain point to track for these releases in which
>>OpenStack branch belong. And this is probably something that needs to
>>be resolved.
>
> Well there's been a lot of discussion around not integrating releases at
> all. That said, I'm not sure I disagree. Coordinating release numbers is
> fine. Coordinating release dates seems less so, especially since they
> prevent the project from delivering what it's promised so 

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Dave Walker]

On 29 May 2015 7:41 pm, "Matt Riedemann"  wrote:

>
> This, IMO, is about the only time right now that I see doing point
releases on stable as worthwhile.  In other words, things have been very
touchy in stable for at least the last 6 months, so in the rare moments of
stability with the gate on stable is when I'd cut a release before the next
gate breaker.  You can get some examples of why here:


I disagree this would help things, as every commit that lands the gate was
perfectly functional at that time by definition.

It is usually retrospection that the gate falls to pass, with the usual
case being changing of bound direct or indirect dependencies.

If we grab a recent point release tarball, and put it back through the gate
with the same declared requirement bounding - it will still fail (even
though it passed when the release was cut).

--
Kind Regards,
Dave Walker
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Dave Walker]

Responses inline.

On 29 May 2015 6:15 pm, "Haïkel"  wrote:
>
> 2015-05-29 15:41 GMT+02:00 Thierry Carrez :
> > Hi everyone,
> >
> > TL;DR:
> > - We propose to stop tagging coordinated point releases (like 2015.1.1)
> > - We continue maintaining stable branches as a trusted source of stable
> > updates for all projects though
> >
>
> Hi,
>
> I'm one of the main maintainer of the packages for Fedora/RHEL/CentOS.
> We try to stick as much as possible to upstream (almost zero
> downstream patches),
> and without intermediate releases, it will get difficult.

If you consider *every* commit to be a release, then your life becomes
easier. This is just a case of bumping the SemVer patch version per commit
(as eloquently put by Jeremy).  We even have tooling to automate the
version generation via pbr..

Therefore, you might want to jump from X.X.100 to X.X.200 which would mean
100 commits since the last update.

> I'm personally not fond of this as it will lead to more fragmentation.
> It may encourage
> bad behaviors like shipping downstream patches for bug fixes and CVE
instead
> of collaborating upstream to differentiate themselves.
> For instance, if we had no point-based release, for issues tracking
> purposes, we would
> have to maintain our sets of tags somewhere.

I disagree, each distro already does security patching and whilst I expect
this to still happens, it actually *encourages* upstream first workflow as
you can select a release on your own cadence that includes commits you
need, for your users.

> There's also the release notes issue that has already been mentioned.
> Still continuous release notes won't solve the problem, as you wouldn't
> be able to map these to the actual packages. Will we require operators
> to find from which git commit, the packages were built and then try to
figure
> out which fixes are and are not included?

Can you provide more detail? I'm not understanding the problem.

> > Long version:
> >
> > At the "stable branch" session in Vancouver we discussed recent
> > evolutions in the stable team processes and how to further adapt the
> > work of the team in a "big tent" world.
> >
> > One of the key questions there was whether we should continue doing
> > stable point releases. Those were basically tags with the same version
> > number ("2015.1.1") that we would periodically push to the stable
> > branches for all projects.
> >
> > Those create three problems.
> >
> > (1) Projects do not all follow the same versioning, so some projects
> > (like Swift) were not part of the "stable point releases". More and more
> > projects are considering issuing intermediary releases (like Swift
> > does), like Ironic. That would result in a variety of version numbers,
> > and ultimately less and less projects being able to have a common
> > "2015.1.1"-like version.
> >
>
> And that's actually a pain point to track for these releases in which
> OpenStack branch belong. And this is probably something that needs to
> be resolved.
>
> > (2) Producing those costs a non-trivial amount of effort on a very small
> > team of volunteers, especially with projects caring about stable
> > branches in various amounts. We were constantly missing the
> > pre-announced dates on those ones. Looks like that effort could be
> > better spent improving the stable branches themselves and keeping them
> > working.
> >
>
> Agreed, but why not switching to a time-based release?
> Regularly, we tag/generate/upload tarballs, this could even be automated.
> As far as I'm concerned, I would be more happy to have more frequent
releases.
>
> > (3) The resulting "stable point releases" are mostly useless. Stable
> > branches are supposed to be always usable, and the "released" version
> > did not undergo significantly more testing. Issuing them actually
> > discourages people from taking whatever point in stable branches makes
> > the most sense for them, testing and deploying that.
> >
> > The suggestion we made during that session (and which was approved by
> > the session participants) is therefore to just get rid of the "stable
> > point release" concept altogether for non-libraries. That said:
> >
> > - we'd still do individual point releases for libraries (for critical
> > bugs and security issues), so that you can still depend on a specific
> > version there
> >
> > - we'd still very much maintain stable branches (and actually focus our
> > efforts on that work) to ensure they are a continuous source of safe
> > upgrades for users of a given series
> >
> > Now we realize that the cross-section of our community which was present
> > in that session might not fully represent the consumers of those
> > artifacts, which is why we expand the discussion on this mailing-list
> > (and soon on the operators ML).
> >
>
> Thanks, I was not able to join this discussion, and that was the kind
> of proposal
> that I was fearing to see happen.
>
> > If you were a consumer of those and will miss them, please explain why.
> > In pa

Re: [openstack-dev] [all] [stable] No longer doing stable point releases

[Ian Cordasco]

On 5/29/15, 12:14, "Haïkel"  wrote:

>2015-05-29 15:41 GMT+02:00 Thierry Carrez :
>> Hi everyone,
>>
>> TL;DR:
>> - We propose to stop tagging coordinated point releases (like 2015.1.1)
>> - We continue maintaining stable branches as a trusted source of stable
>> updates for all projects though
>>
>
>Hi,
>
>I'm one of the main maintainer of the packages for Fedora/RHEL/CentOS.
>We try to stick as much as possible to upstream (almost zero
>downstream patches),
>and without intermediate releases, it will get difficult.

Can you expound on why this is difficult? I believe you, but I want to
understand it better.

>I'm personally not fond of this as it will lead to more fragmentation.

Could you explain this as well? Do you mean fragmentation between what
distros are offering? In other words, Ubuntu is packaging Kilo @ SHA1 and
RHEL is at SHA2. I'm not entirely certain that's a bad thing. That seems
to give the packagers more freedom.

>It may encourage
>bad behaviors like shipping downstream patches for bug fixes and CVE
>instead
>of collaborating upstream to differentiate themselves.

Perhaps I'm wrong, but when a CVE is released, don't the downstream
packagers usually patch whatever version they have and push that out?
Isn't that the point of them being on an private list to receive embargoed
notifications with the patches?

>For instance, if we had no point-based release, for issues tracking
>purposes, we would
>have to maintain our sets of tags somewhere.

But, if I understand correct, downstream sometimes has patches they apply
(or develop) to ensure the package is rock solid on their distribution.
Those aren't always relevant upstream so you maintain them. How is this
different?

>There's also the release notes issue that has already been mentioned.
>Still continuous release notes won't solve the problem, as you wouldn't
>be able to map these to the actual packages. Will we require operators
>to find from which git commit, the packages were built and then try to
>figure
>out which fixes are and are not included?

I think this is wrong. If it's a continuously updated set of notes, then
whatever SHA the head of stable/X is at will be the correct set of notes
for that branch. If you decide to package a SHA earlier than that, then
you would need to do this, but I'm not sure why you would want to package
a SHA that isn't at the HEAD of that branch.

>
>> Long version:
>>
>> At the "stable branch" session in Vancouver we discussed recent
>> evolutions in the stable team processes and how to further adapt the
>> work of the team in a "big tent" world.
>>
>> One of the key questions there was whether we should continue doing
>> stable point releases. Those were basically tags with the same version
>> number ("2015.1.1") that we would periodically push to the stable
>> branches for all projects.
>>
>> Those create three problems.
>>
>> (1) Projects do not all follow the same versioning, so some projects
>> (like Swift) were not part of the "stable point releases". More and more
>> projects are considering issuing intermediary releases (like Swift
>> does), like Ironic. That would result in a variety of version numbers,
>> and ultimately less and less projects being able to have a common
>> "2015.1.1"-like version.
>>
>
>And that's actually a pain point to track for these releases in which
>OpenStack branch belong. And this is probably something that needs to
>be resolved.

Well there's been a lot of discussion around not integrating releases at
all. That said, I'm not sure I disagree. Coordinating release numbers is
fine. Coordinating release dates seems less so, especially since they
prevent the project from delivering what it's promised so that it can
manage to get something that's super stable by an arbitrary date.

>
>> (2) Producing those costs a non-trivial amount of effort on a very small
>> team of volunteers, especially with projects caring about stable
>> branches in various amounts. We were constantly missing the
>> pre-announced dates on those ones. Looks like that effort could be
>> better spent improving the stable branches themselves and keeping them
>> working.
>>
>
>Agreed, but why not switching to a time-based release?
>Regularly, we tag/generate/upload tarballs, this could even be automated.
>As far as I'm concerned, I would be more happy to have more frequent
>releases.

A time based release would probably consist of just tagging whatever SHA
is the HEAD of that branch, especially if it's automated. And if it is
automated, the release notes will mostly just be the one-line log info
between the last release and HEAD (kind of like the oslo release notes).
That seems to me, to be something that y'all could do just as easily as we
could.

>
>> (3) The resulting "stable point releases" are mostly useless. Stable
>> branches are supposed to be always usable, and the "released" version
>> did not undergo significantly more testing. Issuing them actually
>> discourages people from taking whatever p