On Mon, Mar 2, 2015 at 3:38 PM, Joe Gordon <joe.gord...@gmail.com> wrote:
> > > On Mon, Mar 2, 2015 at 9:59 AM, Kyle Mestery <mest...@mestery.com> wrote: > >> On Mon, Mar 2, 2015 at 9:57 AM, Ihar Hrachyshka <ihrac...@redhat.com> >> wrote: >> >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>> >>> Hi Daniel, >>> >>> thanks for a clear write-up of the matter and food for thought. >>> >>> I think the idea of having more smooth development mode that would not >>> make people to wait for 6+ months to release a new feature is great. >>> >>> ++ >> >> >>> It's insane to expect that feature priorities won't ever slightly >>> shift in the next 6 months. Having a limited list of features targeted >>> for the next 6 months is prone to mistakes, since people behind some >>> >> > * Sure, we have had a few things that popped up, nova EC2 split for > example. But this is fairly rare. > > >> of approved features may need to postpone the effort for any type of >>> reasons (personal, job rush, bad resource allocation, ...), and we >> >> then end up with approved specs with no actual code drops, using >>> review 'slots' that would better be spent for other features that were >>> not that lucky to get a rubber stamp during spec phase. Prior resource >>> >> > * As stated below specs approval is very much not rubber stamping > * As stated below this doesn't even make sense, we are *not* using review > slots. > > >> allocation would probably work somehow if we were working for the same >>> company that would define priorities to us, but it's not the case. >> >> >>> It should be noted that even though Nova is using slots for reviews, >> Neutron is not. I've found that it's hard to try and "slot" people in to >> review specific things. During Juno I tried this for Neutron, and it failed >> miserably. For Kilo in Neutron, we're not using slots but instead I've >> tried to highlight the approved specs of "Essential" and "High" priority >> for review by all reviews, core and non-core included. It's gone ok, but >> the reality is you can't force people to review things. There are steps >> submitters can take to try and get timely review (lots of small, easy to >> digest patches, quick turnaround of comments, engagement in IRC and ML, >> etc.). >> > > So this is a big fat lie, one that others believe as well. Nova is *not* > using slots for reviews. We discussed using slots for reviews but did not > adopt them. > > "But I read it on the internet, it must be true." As I said in a prior email, I'm sorry for that. I recalled reading about nova's use of slots. > >> >>> Anecdotally, in neutron, we have two Essential blueprints for Kilo, >>> and there are no code drops or patches in review for any of those, so >>> I would expect them to fail to merge. At the same time, I will need to >>> wait for the end of Kilo to consider adding support for guru reports >>> to the project. Or in oslo world, I will need to wait for Liberty to >>> introduce features in oslo.policy that are needed by neutron to switch >>> to it, etc. >>> >>> To be fair, there are many reasons those to "Essential" BPs do not have >> code. I still expect the Pecan focused to have code, but I already moved >> the Plugin one out of Kilo at this point because there was no chance the >> code would land. >> >> But I get your point here. I think this thread has highlighted the fact >> that the BP/spec process worked to some extent, but for small things, the >> core reviewer team should have the ability to say "Yes, we can easily merge >> that, lets approve that spec" even if it's late in the cycle. >> >> >>> Another problem is that currently milestones are used merely for >>> targeting bugs and features, but no one really cares about whether the >>> actual milestone shipment actually works. Again, a story from neutron >>> world: Kilo-1 was shipped in the middle of advanced services split, >>> with some essential patches around distutils setup missing (no proper >>> migration plan applied, conflicting config files in neutron and *aas >>> repos, etc.) >>> >>> This is true, the milestone release matter but are not given enough >> focus and they release (for the most part) irregardless of items in them, >> given they are not long-lived, etc. >> >> So I'm all for reforms around processes we apply. >>> >>> "If there's one thing OpenStack is good at, it's change." >> >> >>> That said, I don't believe the real problem here is that we don't >>> generate project tarballs frequently enough. >>> >>> Major problems I see as critical to tackle in our dev process are: >>> >>> - - enforced spec/dev mode. Solution: allow to propose (and approve) a >>> reasonable spec at any moment in the cycle; allow code drops for >>> approved specs at any moment in the cycle (except pre-release >>> stabilization time); stop targeting specs: if it's sane, it's probably >>> sane N+2 cycle from now too. >>> >>> I'd say this is fine for specs that are small and people agree can >> easily be merged. I'd say this is not the case for large features near the >> end of the release which are unlikely to gather enough review momentum to >> actually merge. >> >> >>> - - core team rubber stamping a random set of specs and putting -2 on >>> all other specs due to "project priorities". Solution: stop pretending >> >> core team (or core drivers) can reasonably estimate review and coding >>> resources for the next cycle. Instead allows community to decide >>> what's worth the effort by approving all technically reasonable specs >>> and allowing everyone to invest time and effort in specs (s)he seems >>> worth it. >>> >>> If you're referring to Neutron here, I think you fail to estimate the >> amount of time the neutron-drivers team (along with a handful of other >> folks) spent reviewing specs and trying to allocate them for this release. >> We're not just rubber stamping things, we're reviewing, providing comment, >> and ensuring things fit in a consistent roadmap for the next release. In >> the past, we've had this sort of "wild west" where all specs are approved, >> everyone focuses on their own stuff, and a large chunk of things fail to >> merge. Juno was a classic example of that. Now, we'll likely see a similar >> amount of that happening in Kilo despite the changes to the process. But >> I'd at least argue the process here around spec approvals is trying to >> maintain some coherency release to release. >> > > Same thing is true in nova as well. > > >> >> >> - - no due stabilization process before dev milestones. Solution: >>> introduce one in your team workflow, be more strict in what goes in >>> during pre milestone stabilization time. >>> >>> If all above is properly applied, we would get into position similar >>> to your proposal. The difference though is that upstream project would >>> not call milestone tarballs 'Releases'. Still, if there are brave >>> vendors to ship milestones, fine, they would have the same freedom as >>> in your proposal. >>> >>> > While I don't agree with some of your points above, I think what you state > above is a good goal IMHO (In fact I proposed the same thing in a previous > thread) > > >> Note: all the steps mentioned above can be applied on *per team* basis >>> without breaking existing release workflow. >>> >>> I think the ideas above are worth exploring further, and I look forward >> to some discussions on this in Vancouver, as well as continued discussion >> here on the ML before then. >> > > >> Thanks, >> Kyle >> >> >>> Some more comments from stable-maint/distribution side below. >>> >>> On 02/24/2015 10:53 AM, Daniel P. Berrange wrote: >>> [...skip...] >>> > The modest proposal =================== >>> [...skip...] >>> > >>> > Stable branches --------------- >>> > >>> > The consequences of a 2 month release cycle appear fairly severe >>> > for the stable branch maint teams at first sight. This is not, >>> > however, an insurmountable problem. The linux kernel shows an easy >>> > way forward with their approach of only maintaining stable branches >>> > for a subset of major releases, based around user / vendor demand. >>> > So it is still entirely conceivable that the stable team only >>> > provide stable branch releases for 2 out of the 6 yearly releases. >>> > ie no additional burden over what they face today. Of course they >>> > might decide they want to do more stable branches, but maintain >>> > each for a shorter time. So I could equally see them choosing todo >>> > 3 or 4 stable branches a year. Whatever is most effective for those >>> > involved and those consuming them is fine. >>> > >>> >>> Since it's not only stable branches that are affected (translators, >>> documentation writers, VMT were already mentioned), those affected >>> will probably need to come up with some synchronized decision. >>> >>> Let's say we still decide to support two out of six releases (same >>> scheme as is now). In that case no process that we usually attach to >>> releases will be running after release dates. This makes me wonder how >>> it's different from milestones we already have. >>> >>> Do you think any downstream vendors will actually ship and support >>> upstream releases that upstream drops any guarantees for (no VMT, no >>> stable branches, no gate runs, ...) right after the release date? I >>> doubt there will be significant interest in it. >>> >>> > Downstream vendors ------------------ >>> > >>> > Downstream product vendors would face a similar decision about how >>> > they wished to produce. Some OS vendors may choose to ship all >>> > stable releases in their distros as they become available, others >>> > may choose to focus on a subset of the releases. In some cases more >>> > frequent release may actually simplify life for the vendors. For >>> > example, if there is a feature they need which doesn't make the cut >>> > for a release today, they currently have the choice of waiting 6 >>> > months for the next release, or carrying non-upstream patches, or >>> > backporting patches from master. With a 2 month release cycle, it >>> > is more practical for vendors to simply wait 2 months for the next >>> > release to ship. So this would probably reduce the amount of patch >>> > backporting vendors have todo in general, giving them more time to >>> > focus on useful upstream work. >>> > >>> > Design summit ------------- >>> > >>> > Another consequence of 2 month release cycles would be that the >>> > design summits become entirely detached from the release cycles, >>> > as they'd be on different cycle lengths. So instead of having a >>> > focus on release planning, design summits would likely become a >>> > forum for general ongoing collaboration and discussions. Probably a >>> > half & half merge between the current design summit format and a >>> > more traditional open source conference. This also ought to result >>> > in less pressure for all contributors to attend all summits, and so >>> > be time efficient travelwise and beneficial in cost reduction for >>> > vendors. >>> > >>> > More interestingly, is that by detaching them from release cycles, >>> > the summit organizers could have more flexibility with scheduling >>> > the summits. They could be placed on the dates that are most >>> > suitable for desired travel locations, rather than dates suited to >>> > release schedules. Likewise there is more freedom if there was ever >>> > a wish to have design summits and the main track conference >>> > separated. >>> > >>> > Election process ---------------- >>> > >>> > I wouldn't suggest any changes to project elections or governance. >>> > It is fine to have elections every 6 months. There is no need for >>> > that to be tied to release cycle length in any way, particularly as >>> > people often serve for more than one release already. >>> > >>> > Release naming -------------- >>> > >>> > Last but not least, there would be some questions around what this >>> > might mean for release naming. There are many options and I don't >>> > intend to suggest a particular one is best, just highlight some >>> > possibilities that could be considered. >>> > >>> > The easiest is to just drop the idea of release names entirely and >>> > have everyone refer to the release versions instead. Another >>> > option is to give each 2 month release its own name, so we'd have >>> > to come up with 3x as many names during the year. The person >>> > running the release naming elections will probably have a heart >>> > attack at this idea :-) Another option is to have names that span a >>> > couple of releases, perhaps all releases for an entire year get a >>> > given name. >>> > >>> > Upgrades & deprecation ---------------------- >>> > >>> > It is common right now for projects to say upgrades are only >>> > supported between releases N-1 and N. ie to go from Icehouse to >>> > Kilo, you need to first deploy Juno. This is passable when you're >>> > talking 6 month gaps between cycles, but when there are 2 month >>> > gaps it is not reasonable to expect everyone to be moving fast >>> > enough to keep up with every release. If an organization's >>> > beurocracy means they can't deploy more often than every 12 months, >>> > forcing them to deploy the 5 intermediate releases to run upgrade >>> > scripts is quite unpleasant. We would likely have to look at >>> > officially allowing upgrades between any (N-3, N-2, N-1) to N. From >>> > a database POV this should not be hard, since the DB migration >>> > scripts don't have any built in assumptions about this. Similarly >>> > the versioned objects used by Nova are quite flexible in this >>> > regard too, as long as the compat code isn't deleted too soon. >>> >>> Let's say horizontal teams decided to stick to 2 supported releases >>> per year scheme (the current one). What would grenade jobs for stable >>> branches (and master) test in this case? >>> >>> > >>> > Deprecation warnings would need similar consideration. It would not >>> > be sufficient to deprecate in one release and delete in the next. >>> > We'd likely want to say that depecations last for a given time >>> > period rather than number of releases, eg 6 months. This could be >>> > easily handled by simply including the date of initial deprecation >>> > in the deprecation message. It would thus be clear when the feature >>> > will be removed to all involved. >>> > >>> > Why do releases at all ? ------------------------ >>> > >>> > Some people might wonder why we should do releases at all. We >>> > already aim to support a continuous deployment model for users as >>> > an option. Continuous deployment, however, is quite a conceptual >>> > jump for many users and organizations in particular. Their internal >>> > software approval processes and bureaucracy can make this >>> > impractical to even consider in the forseeable future. So if >>> > openstack did not do releases, then many downstream vendors would >>> > still end up taking git snapshots and forming periodic releases >>> > around them. The unsatisfactory result would be a balkanization of >>> > OpenStack where each vendor duplicated the others work with their >>> > own release cycles in slightly different ways. To some extent this >>> > happens already as vendors backport features, due to inability to >>> > wait 6 months for next major release. A two month release cycle >>> > though would likely reduce the amount of difference between what >>> > each vendor ships which would be a good thing in general. Releases >>> > also serve as a useful hook for marketing activities by the project >>> > and vendors alike, giving a good opportunity to promote the new >>> > body of features now available to users. It might be difficult to >>> > drum up media enthusiasm to promote arbitrary git snapshots, but >>> > then I'm no marketing expert. >>> > >>> >>> I think it's still easier for people to work in release/stability mode >>> for some predefined time than "forever". >>> >>> > >>> > >>> > >>> > Regards, Daniel >>> > >>> > [1] >>> > >>> http://lists.openstack.org/pipermail/openstack-dev/2014-September/044872.html >>> > >>> > >>> [2] http://www.openstack.org/projects/openstack-faq/ >>> > [3] https://wiki.openstack.org/wiki/Kilo_Release_Schedule [4] >>> > >>> http://lists.openstack.org/pipermail/openstack-dev/2014-October/047922.html >>> > >>> -----BEGIN PGP SIGNATURE----- >>> Version: GnuPG v1 >>> >>> iQEbBAEBAgAGBQJU9IheAAoJEC5aWaUY1u57/M4H912jtcA8IP424DZnkDVm/6dF >>> uXl02JqeUeFvne+M9/JoRIO+GwuY05kfmwFAXY5RNX5PmW1MA+eLpirKI5waMjzJ >>> 6zAwViir2sLtrXjpHNKkUrQxzURWN72qvnU+idCJlExU4ZXCkKcTuGo0YC6j46WH >>> mLuRheb3wRqmkMrZ7nlShaAiSkLBYpslP+NuKH4JuWzyAZqrQsTwcu9duOD9QiTR >>> 294ojzqihCth7npMeXcdMtdkl+GoxP494ojouWqVTAYCpWDfTiZw3/Slb0VTYVIe >>> uIS1gN2Hmur1vTbkM4kgvU3D9Uf+uLS9vH9GDg+zyDsjQQm/6RToCOTFiAYp8w== >>> =nhsu >>> -----END PGP SIGNATURE----- >>> >>> >>> __________________________________________________________________________ >>> OpenStack Development Mailing List (not for usage questions) >>> Unsubscribe: >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >> >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
