As an aside, it would be nice to admit we basically revisit everything each time it becomes relevant again, and for policy decisions like this (that don’t need to be agreed in advance) we should try to legislate the minimum necessary policy to proceed today, and leave future refinements for later when the relevant context arises.
> On 10 Dec 2024, at 13:00, Benedict Elliott Smith <bened...@apache.org> wrote: > > I agree with Aleksey that if we think something is broken, we shouldn’t use > euphemisms, and for this reason I don’t like unstable (this could for > instance simply mean API unstable). If we intend to never need this > descriptor, we should avoid bike-shedding and insert a “placeholder” for now > to be refined as and when we need it when we have the necessary future > context. > > i.e. > > preview -> beta -> [“has problems that will take time to resolve placeholder” > -> beta] -> GA > > > >> On 10 Dec 2024, at 12:39, Josh McKenzie <jmcken...@apache.org> wrote: >> >> +1 to this classification with one addition. I think we need to augment this >> with formalization on what we do with features we don't recommend people use >> (i.e. MV in their current incarnation). For something retroactively found to >> be unstable, we could add an "Unstable" qualification for it, leaving us >> with: >> >> Unstable: Warnings on use, clearly communicated as to why, either on-track >> to be fixed or removed from the codebase. No lingering for years in a fugue >> state. We should target never needing this classification. >> Preview: Ready to be tried by end users but has caveats and most likely is >> not api stable. Developer only documentation acceptable. >> Beta: Feature complete/API stable but has not had enough testing to be >> considered rock solid. Developer and User documentation required. >> GA: Ready for use, no known issue, PMC is satisfied with the testing that >> has been done >> >> To walk through how some of the flow might look to test the above: >> >> Simple case: >> - Preview -> Beta -> GA >> >> Late discovered defect case: >> - Preview -> Beta -> Unstable -> Beta -> GA >> >> Pathological worst-case (i.e. MV): >> - Preview -> Beta -> GA -> Unstable -> [Preview|Removed] >> >> On Tue, Dec 10, 2024, at 12:29 PM, Jeremiah Jordan wrote: >>> I agree with Aleksey and Patrick. We should define terminology and then >>> stick to it. My preferred list would be: >>> >>> Preview - Ready to be tried by end users but has caveats and most likely is >>> not api stable. >>> Beta - Feature complete/API stable but has not had enough testing to be >>> considered rock solid. >>> GA - Ready for use, no known issue, PMC is satisfied with the testing that >>> has been done >>> >>> Whether or not something is enabled by default or the default >>> implementation is a separate access from the readiness. Though if we are >>> replacing an existing thing with a new default I would hope we apply extra >>> rigor to allowing that to happen. >>> >>> -Jeremiah >>> >>> On Dec 10, 2024 at 11:15:37 AM, Patrick McFadin <pmcfa...@gmail.com >>> <mailto:pmcfa...@gmail.com>> wrote: >>>> I'm going to try to pull this back from the inevitable bikeshedding >>>> and airing of grievances that happen. Rewind all the way back to >>>> Josh's original point, which is a defined process. Why I really love >>>> this being brought up is our maturing process of communicating to the >>>> larger user base. The dev list has very few participants. Less than >>>> 1000 last I looked. Most users I talk to just want to know what they >>>> are getting. Well-formed, clear communication is how the PMC can let >>>> end users know that a new feature is one of three states: >>>> >>>> 1. Beta >>>> 2. Generally Available >>>> 3. Default (where appropriate) >>>> >>>> Yes! The work is just sorting out what each level means and then >>>> codifying that in confluence. Then, we look at any features that are >>>> under question, assign a level, and determine what it takes to go from >>>> one state to another. >>>> >>>> The CEPs need to reflect this change. What makes a Beta, GA, Default >>>> for new feature X. It makes it clear for implementers and end users, >>>> which is an important feature of project maturity. >>>> >>>> Patrick >>> >>> >>> On Dec 10, 2024 at 5:46:38 AM, Aleksey Yeshchenko <alek...@apple.com >>> <mailto:alek...@apple.com>> wrote: >>>> What we’ve done is we’ve overloaded the term ‘experimental’ to mean too >>>> many related but different ideas. We need additional, more specific >>>> terminology to disambiguate. >>>> >>>> 1. Labelling released features that were known to be unstable at release >>>> as ‘experimental’ retroactively shouldn’t happen and AFAIK only happened >>>> once, with MVs, and ‘experimental’ there was just a euphemism for >>>> ‘broken’. Our practices are more mature now, I like to think, that a >>>> situation like this would not arise in the future - the bar for releasing >>>> a completed marketable feature is higher. So the label ‘experimental’ >>>> should not be applied retroactively to anything. >>>> >>>> 2. It’s possible that a released, once considered production-ready >>>> feature, might be discovered to be deeply flawed after being released >>>> already. We need to temporarily mark such a feature as ‘broken' or >>>> ‘flawed'. Not experimental, and not even ‘unstable’. Make sure we emit a >>>> warning on its use everywhere, and, if possible, make it opt-in in the >>>> next major, at the very least, to prevent new uses of it. Announce on dev, >>>> add a note in NEWS.txt, etc. If the flaws are later addressed, remove the >>>> label. Removing the feature itself might not be possible, but should be >>>> considered, with heavy advanced telegraphing to the community. >>>> >>>> 3. There is probably room for genuine use of ‘experimental’ as a feature >>>> label. For opt-in features that we commit with an understanding that they >>>> might not make it at all. Unstable API is implied here, but a feature can >>>> also have an unstable API without being experimental - so ‘experimental' >>>> doesn’t equal to ‘api-unstable’. These should not be relied on by any >>>> production code, they would be heavily gated by unambiguous configuration >>>> flags, disabled by default, allowed to be removed or changed in any >>>> version including a minor one. >>>> >>>> 4. New features without known flaws, intended to be production-ready and >>>> marketable eventually, that we may want to gain some real-world confidence >>>> with before we are happy to market or make default. UCS, for example, >>>> which seems to be in heavy use in Astra and doesn’t have any known open >>>> issues (AFAIK). It’s not experimental, it’s not unstable, it’s not ‘alpha’ >>>> or ‘beta’, it just hasn't been widely enough used to have gained a lot of >>>> confidence. It’s just new. I’m not sure what label even applies here. It’s >>>> just a regular feature that happens to be new, doesn’t need a label, just >>>> needs to see some widespread use before we can make it a default. No other >>>> limitation on its use. >>>> >>>> 5. Early-integrated, not-yet fully-completed features that are NOT >>>> experimental in nature. Isolated, gated behind deep configuration flags. >>>> Have a CEP behind them, we trust that they will be eventually completed, >>>> but for pragmatic reasons it just made sense to commit them at an earlier >>>> stage. ‘Preview’, ‘alpha’, ‘beta’ are labels that could apply here >>>> depending on current feature readiness status. API-instability is implied. >>>> Once finished they just become a regular new feature, no flag needed, no >>>> heavy config gating needed. >>>> >>>> I might be missing some scenarios here. >
