Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Patrick McFadin]

We already have an established "alpha," IMO, and it's called branch
and trunk. For example, the CEP-15 branch is the alpha for Accord and
TCM, and then it will be merged into trunk. The next stop is beta and
on to the regular release train.

I'm just optimizing to keep it simple and clean for end users. Less options.

Patrick

On Thu, Dec 12, 2024 at 7:57 AM Josh McKenzie  wrote:
>
> But MVs are not alpha or preview, as they are not actively being worked on. 
> They are currently broken. Calling them ‘alpha’ makes ‘alpha’ overloaded and 
> less useful.
>
> I'm asserting they should either be marked 'alpha/Preview' and actively being 
> worked on, or be deprecated and removed.
>
> To Paulo's point, Jaydeep's working on base<->view reconciliation which, to 
> me, would indicate they're alpha. And until we have a solution for that 
> reconciliation in the face of lost data in the base table, no global index or 
> MV implementation will be ready for production.
>
> On Thu, Dec 12, 2024, at 10:40 AM, Brandon Williams wrote:
>
> I think it would be better to avoid 'alpha' since we do beta releases,
> and I agree with Aleksey that we'd be overloading 'alpha' and perhaps
> causing confusion.
>
> Kind Regards,
> Brandon
>
> On Thu, Dec 12, 2024 at 8:58 AM Benedict  wrote:
> >
> > I think alpha is fine. It communicates fairly well that there’s no near 
> > term expectation they will be production capable.
> >
> > There is (I think) still an intention to improve them, but they are janky. 
> > If we don’t intend to begin fixing the feature within the next year or so 
> > we should deprecate it entirely.
> >
> >
> > On 12 Dec 2024, at 14:46, Aleksey Yeshchenko  wrote:
> >
> > But MVs are not alpha or preview, as they are not actively being worked 
> > on. They are currently broken. Calling them ‘alpha’ makes ‘alpha’ 
> > overloaded and less useful.
> >
> > On 12 Dec 2024, at 14:00, Josh McKenzie  wrote:
> >
> > But we also need an approved non-euphemism for features like MVs (I suggest 
> > ‘broken’) and possibly a softer version of it ('dangerous') for our 
> > existing features that work fine in some narrow well-defined circumstances 
> > but will blow in your face if you don’t know exactly what you are doing.
> >
> > Feels like the real answer is:
> >
> > Endeavor to never get ourselves into this state
> > Take immediate action if we discover we're there (fix feature if possible, 
> > deprecate and remove if not). Not "leave to fester for years"
> >
> > I like the introduction of 'alpha' as an alias for 'Preview'; not sure why 
> > that wasn't what we immediately came up with collectively given how 
> > widespread its usage is. :)
> >
> > What would demoting MV's to 'alpha' right now look like? We'd warn on their 
> > usage w/some different structure and verbiage, and it'd be pretty 
> > implicitly clear to people they shouldn't use it in production right?
> >
> > It seems to me that the 3 categories would be sufficient even to handle our 
> > current scenario where we have some things in the system that are a Bad 
> > Idea to use in production.
> >
> > On Thu, Dec 12, 2024, at 6:06 AM, Aleksey Yeshchenko wrote:
> >
> > I don’t like ‘unstable’ either, albeit for a different reason, but I don’t 
> > think three is enough and fits, as we already have some features that don’t 
> > fit into either of (preview,beta,ga) - released but broken, released but 
> > dangerous, deprecated, removed.
> >
> > For new features going forward, alpha (preview) -> beta -> GA works well 
> > enough.
> >
> > But we also need an approved non-euphemism for features like MVs (I suggest 
> > ‘broken’) and possibly a softer version of it ('dangerous') for our 
> > existing features that work fine in some narrow well-defined circumstances 
> > but will blow in your face if you don’t know exactly what you are doing.
> >
> > These classifications are largely orthogonal.
> >
> > Alpha(preview)->Beta->GA communicates readiness of a feature under 
> > development, with GA being the default final state for most features.
> >
> > From there a feature can transition into ‘broken’ or ‘dangerous’ territory. 
> > Serious issues get uncovered (very) late sometimes. It is what it is.
> > And we do deprecate and remove functionality when it’s superseded.
> >
> >
> > -1 on unstable. It's way too many words than are needed. Three is a
> > magic number and fits:
> >
> > Preview
> > Beta
> > GA
> >
> >
> > On 11 Dec 2024, at 18:50, Josh McKenzie  wrote:
> >
> > A structured, disciplined approach to graduating something from [Optional] 
> > -> [Default] makes sense to me, similar to how we're talking about a 
> > structured flow of [Preview] -> [Beta] -> [GA]. Having those clear stages 
> > gives us a framework to define what requirements of stage transitions would 
> > be which'll ideally lead to us producing higher quality, more predictable, 
> > more consistent results for our end users.
> >
> > For instance, requirements from [Optional] -> [Default] could be

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Josh McKenzie]

> But MVs are not alpha or preview, as they are not actively being worked on. 
> They are currently broken. Calling them ‘alpha’ makes ‘alpha’ overloaded and 
> less useful.
I'm asserting they should either be marked 'alpha/Preview' and actively being 
worked on, or be deprecated and removed.

To Paulo's point, Jaydeep's working on base<->view reconciliation which, to me, 
would indicate they're alpha. And until we have a solution for that 
reconciliation in the face of lost data in the base table, no global index or 
MV implementation will be ready for production.

On Thu, Dec 12, 2024, at 10:40 AM, Brandon Williams wrote:
> I think it would be better to avoid 'alpha' since we do beta releases,
> and I agree with Aleksey that we'd be overloading 'alpha' and perhaps
> causing confusion.
> 
> Kind Regards,
> Brandon
> 
> On Thu, Dec 12, 2024 at 8:58 AM Benedict  wrote:
> >
> > I think alpha is fine. It communicates fairly well that there’s no near 
> > term expectation they will be production capable.
> >
> > There is (I think) still an intention to improve them, but they are janky. 
> > If we don’t intend to begin fixing the feature within the next year or so 
> > we should deprecate it entirely.
> >
> >
> > On 12 Dec 2024, at 14:46, Aleksey Yeshchenko  wrote:
> >
> > But MVs are not alpha or preview, as they are not actively being worked 
> > on. They are currently broken. Calling them ‘alpha’ makes ‘alpha’ 
> > overloaded and less useful.
> >
> > On 12 Dec 2024, at 14:00, Josh McKenzie  wrote:
> >
> > But we also need an approved non-euphemism for features like MVs (I suggest 
> > ‘broken’) and possibly a softer version of it ('dangerous') for our 
> > existing features that work fine in some narrow well-defined circumstances 
> > but will blow in your face if you don’t know exactly what you are doing.
> >
> > Feels like the real answer is:
> >
> > Endeavor to never get ourselves into this state
> > Take immediate action if we discover we're there (fix feature if possible, 
> > deprecate and remove if not). Not "leave to fester for years"
> >
> > I like the introduction of 'alpha' as an alias for 'Preview'; not sure why 
> > that wasn't what we immediately came up with collectively given how 
> > widespread its usage is. :)
> >
> > What would demoting MV's to 'alpha' right now look like? We'd warn on their 
> > usage w/some different structure and verbiage, and it'd be pretty 
> > implicitly clear to people they shouldn't use it in production right?
> >
> > It seems to me that the 3 categories would be sufficient even to handle our 
> > current scenario where we have some things in the system that are a Bad 
> > Idea to use in production.
> >
> > On Thu, Dec 12, 2024, at 6:06 AM, Aleksey Yeshchenko wrote:
> >
> > I don’t like ‘unstable’ either, albeit for a different reason, but I don’t 
> > think three is enough and fits, as we already have some features that don’t 
> > fit into either of (preview,beta,ga) - released but broken, released but 
> > dangerous, deprecated, removed.
> >
> > For new features going forward, alpha (preview) -> beta -> GA works well 
> > enough.
> >
> > But we also need an approved non-euphemism for features like MVs (I suggest 
> > ‘broken’) and possibly a softer version of it ('dangerous') for our 
> > existing features that work fine in some narrow well-defined circumstances 
> > but will blow in your face if you don’t know exactly what you are doing.
> >
> > These classifications are largely orthogonal.
> >
> > Alpha(preview)->Beta->GA communicates readiness of a feature under 
> > development, with GA being the default final state for most features.
> >
> > From there a feature can transition into ‘broken’ or ‘dangerous’ territory. 
> > Serious issues get uncovered (very) late sometimes. It is what it is.
> > And we do deprecate and remove functionality when it’s superseded.
> >
> >
> > -1 on unstable. It's way too many words than are needed. Three is a
> > magic number and fits:
> >
> > Preview
> > Beta
> > GA
> >
> >
> > On 11 Dec 2024, at 18:50, Josh McKenzie  wrote:
> >
> > A structured, disciplined approach to graduating something from [Optional] 
> > -> [Default] makes sense to me, similar to how we're talking about a 
> > structured flow of [Preview] -> [Beta] -> [GA]. Having those clear stages 
> > gives us a framework to define what requirements of stage transitions would 
> > be which'll ideally lead to us producing higher quality, more predictable, 
> > more consistent results for our end users.
> >
> > For instance, requirements from [Optional] -> [Default] could be higher 
> > level abstractions like:
> >
> > Confidence in stability
> > Strong evidence to indicate superiority in majority of workloads (by count 
> > or importance or size, etc)
> >
> > These are all things we kind of do implicitly and ad-hoc on the mailing 
> > list, and I'm not looking to tie us down to any granular structure or 
> > specificity. More thinking it could be useful for someone tha

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Brandon Williams]

I think it would be better to avoid 'alpha' since we do beta releases,
and I agree with Aleksey that we'd be overloading 'alpha' and perhaps
causing confusion.

Kind Regards,
Brandon

On Thu, Dec 12, 2024 at 8:58 AM Benedict  wrote:
>
> I think alpha is fine. It communicates fairly well that there’s no near term 
> expectation they will be production capable.
>
> There is (I think) still an intention to improve them, but they are janky. If 
> we don’t intend to begin fixing the feature within the next year or so we 
> should deprecate it entirely.
>
>
> On 12 Dec 2024, at 14:46, Aleksey Yeshchenko  wrote:
>
> But MVs are not alpha or preview, as they are not actively being worked on. 
> They are currently broken. Calling them ‘alpha’ makes ‘alpha’ overloaded and 
> less useful.
>
> On 12 Dec 2024, at 14:00, Josh McKenzie  wrote:
>
> But we also need an approved non-euphemism for features like MVs (I suggest 
> ‘broken’) and possibly a softer version of it ('dangerous') for our existing 
> features that work fine in some narrow well-defined circumstances but will 
> blow in your face if you don’t know exactly what you are doing.
>
> Feels like the real answer is:
>
> Endeavor to never get ourselves into this state
> Take immediate action if we discover we're there (fix feature if possible, 
> deprecate and remove if not). Not "leave to fester for years"
>
> I like the introduction of 'alpha' as an alias for 'Preview'; not sure why 
> that wasn't what we immediately came up with collectively given how 
> widespread its usage is. :)
>
> What would demoting MV's to 'alpha' right now look like? We'd warn on their 
> usage w/some different structure and verbiage, and it'd be pretty implicitly 
> clear to people they shouldn't use it in production right?
>
> It seems to me that the 3 categories would be sufficient even to handle our 
> current scenario where we have some things in the system that are a Bad Idea 
> to use in production.
>
> On Thu, Dec 12, 2024, at 6:06 AM, Aleksey Yeshchenko wrote:
>
> I don’t like ‘unstable’ either, albeit for a different reason, but I don’t 
> think three is enough and fits, as we already have some features that don’t 
> fit into either of (preview,beta,ga) - released but broken, released but 
> dangerous, deprecated, removed.
>
> For new features going forward, alpha (preview) -> beta -> GA works well 
> enough.
>
> But we also need an approved non-euphemism for features like MVs (I suggest 
> ‘broken’) and possibly a softer version of it ('dangerous') for our existing 
> features that work fine in some narrow well-defined circumstances but will 
> blow in your face if you don’t know exactly what you are doing.
>
> These classifications are largely orthogonal.
>
> Alpha(preview)->Beta->GA communicates readiness of a feature under 
> development, with GA being the default final state for most features.
>
> From there a feature can transition into ‘broken’ or ‘dangerous’ territory. 
> Serious issues get uncovered (very) late sometimes. It is what it is.
> And we do deprecate and remove functionality when it’s superseded.
>
>
> -1 on unstable. It's way too many words than are needed. Three is a
> magic number and fits:
>
> Preview
> Beta
> GA
>
>
> On 11 Dec 2024, at 18:50, Josh McKenzie  wrote:
>
> A structured, disciplined approach to graduating something from [Optional] -> 
> [Default] makes sense to me, similar to how we're talking about a structured 
> flow of [Preview] -> [Beta] -> [GA]. Having those clear stages gives us a 
> framework to define what requirements of stage transitions would be which'll 
> ideally lead to us producing higher quality, more predictable, more 
> consistent results for our end users.
>
> For instance, requirements from [Optional] -> [Default] could be higher level 
> abstractions like:
>
> Confidence in stability
> Strong evidence to indicate superiority in majority of workloads (by count or 
> importance or size, etc)
>
> These are all things we kind of do implicitly and ad-hoc on the mailing list, 
> and I'm not looking to tie us down to any granular structure or specificity. 
> More thinking it could be useful for someone that's worked on something who 
> wonders "Huh. How do I take this from being optional to the default?" and 
> having an answer better than "reinvent the wheel every time and fling 
> spaghetti at the dev list and pray".
>
> :)
>
>
> On Wed, Dec 11, 2024, at 1:04 PM, Paulo Motta wrote:
>
> Thanks for bringing up this topic, Josh.
>
> Outside of the major features (ie. MV/SAI/TCM/Accord), one related discussion 
> in this topic is: how can we "promote" small improvements in existing 
> features from optional to default ?
>
> It makes sense to have optimizations launched behind a feature flag initially 
> (beta phase) while the improvement gets real world exposure, but I think we 
> need a better way to promote these optimizations to default behavior on a 
> regular cadence.
>
> Take for example optimized repairs from CASSAND

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Paulo Motta]

Thanks for clarifying your view and sorry for the diversion on the thread
topic, let’s get back to it. It looks like this warrants its own discussion
on future of MVs in the era of accord (whether we still want to provide
eventually consistent MVs in the current format, or remove it in favor of
Accord-backed global indexes).

On Thu, 12 Dec 2024 at 10:10 Benedict  wrote:

> I don’t think they should be called or treated as the same feature. Or at
> least we should rebrand. They also have quite different properties.
>
> I would prefer to introduce “Global Indexes” backed by accord, since this
> is also a clearer name, and gives us a clean break from the mess of MVs. We
> can decide if we want to also retain eventually consistent views on the
> data, and what we want to call them, and if so whether we want to introduce
> an isolation parameter at the same time or keep them well demarcated.
>
> On 12 Dec 2024, at 15:04, Paulo Motta  wrote:
>
> 
>
> > If we don’t intend to begin fixing the feature within the next year or
> so we should deprecate it entirely.
>
> +1 - this is probably topic for another thread but isn’t MVs fundamentally
> solved with Accord? In my ignorance this is “just” a matter of adding an
> Accord backend to MV syntax to fix it reliably.
>
> On Thu, 12 Dec 2024 at 09:58 Benedict  wrote:
>
>> I think alpha is fine. It communicates fairly well that there’s no near
>> term expectation they will be production capable.
>>
>> There is (I think) still an intention to improve them, but they are
>> janky. If we don’t intend to begin fixing the feature within the next year
>> or so we should deprecate it entirely.
>>
>>
>> On 12 Dec 2024, at 14:46, Aleksey Yeshchenko  wrote:
>>
>> But MVs are not alpha or preview, as they are not actively being worked
>> on. They are currently broken. Calling them ‘alpha’ makes ‘alpha’
>> overloaded and less useful.
>>
>>
>>
>> On 12 Dec 2024, at 14:00, Josh McKenzie  wrote:
>>
>> But we also need an approved non-euphemism for features like MVs (I
>> suggest ‘broken’) and possibly a softer version of it ('dangerous') for
>> our existing features that work fine in some narrow well-defined
>> circumstances but will blow in your face if you don’t know exactly what you
>> are doing.
>>
>> Feels like the real answer is:
>>
>>1. Endeavor to never get ourselves into this state
>>2. Take immediate action if we discover we're there (fix feature if
>>possible, deprecate and remove if not). Not "leave to fester for years"
>>
>> I like the introduction of 'alpha' as an alias for 'Preview'; not sure
>> why that wasn't what we immediately came up with collectively given how
>> widespread its usage is. :)
>>
>> What would demoting MV's to 'alpha' right now look like? We'd warn on
>> their usage w/some different structure and verbiage, and it'd be pretty
>> implicitly clear to people they shouldn't use it in production right?
>>
>> It seems to me that the 3 categories would be sufficient even to handle
>> our current scenario where we have some things in the system that are a Bad
>> Idea to use in production.
>>
>> On Thu, Dec 12, 2024, at 6:06 AM, Aleksey Yeshchenko wrote:
>>
>> I don’t like ‘unstable’ either, albeit for a different reason, but I
>> don’t think three is enough and fits, as we already have some features that
>> don’t fit into either of (preview,beta,ga) - released but broken, released
>> but dangerous, deprecated, removed.
>>
>> For new features going forward, alpha (preview) -> beta -> GA works well
>> enough.
>>
>> But we also need an approved non-euphemism for features like MVs (I
>> suggest ‘broken’) and possibly a softer version of it ('dangerous') for
>> our existing features that work fine in some narrow well-defined
>> circumstances but will blow in your face if you don’t know exactly what you
>> are doing.
>>
>> These classifications are largely orthogonal.
>>
>> Alpha(preview)->Beta->GA communicates readiness of a feature under
>> development, with GA being the default final state for most features.
>>
>> From there a feature can transition into ‘broken’ or ‘dangerous’
>> territory. Serious issues get uncovered (very) late sometimes. It is what
>> it is.
>> And we do deprecate and remove functionality when it’s superseded.
>>
>>
>> -1 on unstable. It's way too many words than are needed. Three is a
>> magic number and fits:
>>
>> Preview
>> Beta
>> GA
>>
>>
>> On 11 Dec 2024, at 18:50, Josh McKenzie  wrote:
>>
>> A structured, disciplined approach to graduating something from
>> [Optional] -> [Default] makes sense to me, similar to how we're talking
>> about a structured flow of [Preview] -> [Beta] -> [GA]. Having those clear
>> stages gives us a framework to define what requirements of stage
>> transitions would be which'll ideally lead to us producing higher quality,
>> more predictable, more consistent results for our end users.
>>
>> For instance, requirements from [Optional] -> [Default] could be higher
>> level abstractions like:
>>
>>

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Benedict]

I don’t think they should be called or treated as the same feature. Or at least we should rebrand. They also have quite different properties.I would prefer to introduce “Global Indexes” backed by accord, since this is also a clearer name, and gives us a clean break from the mess of MVs. We can decide if we want to also retain eventually consistent views on the data, and what we want to call them, and if so whether we want to introduce an isolation parameter at the same time or keep them well demarcated.On 12 Dec 2024, at 15:04, Paulo Motta  wrote:> If we don’t intend to begin fixing the feature within the next year or so we should deprecate it entirely.+1 - this is probably topic for another thread but isn’t MVs fundamentally solved with Accord? In my ignorance this is “just” a matter of adding an Accord backend to MV syntax to fix it reliably.On Thu, 12 Dec 2024 at 09:58 Benedict  wrote:I think alpha is fine. It communicates fairly well that there’s no near term expectation they will be production capable. There is (I think) still an intention to improve them, but they are janky. If we don’t intend to begin fixing the feature within the next year or so we should deprecate it entirely.On 12 Dec 2024, at 14:46, Aleksey Yeshchenko  wrote:But MVs are not alpha or preview, as they are not actively being worked on. They are currently broken. Calling them ‘alpha’ makes ‘alpha’ overloaded and less useful.On 12 Dec 2024, at 14:00, Josh McKenzie  wrote:But we also need an approved non-euphemism for features like MVs (I suggest ‘broken’) and possibly a softer version of it ('dangerous') for our existing features that work fine in some narrow well-defined circumstances but will blow in your face if you don’t know exactly what you are doing.Feels like the real answer is:Endeavor to never get ourselves into this stateTake immediate action if we discover we're there (fix feature if possible, deprecate and remove if not). Not "leave to fester for years"I like the introduction of 'alpha' as an alias for 'Preview'; not sure why that wasn't what we immediately came up with collectively given how widespread its usage is. :)What would demoting MV's to 'alpha' right now look like? We'd warn on their usage w/some different structure and verbiage, and it'd be pretty implicitly clear to people they shouldn't use it in production right?It seems to me that the 3 categories would be sufficient even to handle our current scenario where we have some things in the system that are a Bad Idea to use in production.On Thu, Dec 12, 2024, at 6:06 AM, Aleksey Yeshchenko wrote:I don’t like ‘unstable’ either, albeit for a different reason, but I don’t think three is enough and fits, as we already have some features that don’t fit into either of (preview,beta,ga) - released but broken, released but dangerous, deprecated, removed.For new features going forward, alpha (preview) -> beta -> GA works well enough.But we also need an approved non-euphemism for features like MVs (I suggest ‘broken’) and possibly a softer version of it ('dangerous') for our existing features that work fine in some narrow well-defined circumstances but will blow in your face if you don’t know exactly what you are doing.These classifications are largely orthogonal.Alpha(preview)->Beta->GA communicates readiness of a feature under development, with GA being the default final state for most features.From there a feature can transition into ‘broken’ or ‘dangerous’ territory. Serious issues get uncovered (very) late sometimes. It is what it is.And we do deprecate and remove functionality when it’s superseded.-1 on unstable. It's way too many words than are needed. Three is amagic number and fits:PreviewBetaGAOn 11 Dec 2024, at 18:50, Josh McKenzie  wrote:A structured, disciplined approach to graduating something from [Optional] -> [Default] makes sense to me, similar to how we're talking about a structured flow of [Preview] -> [Beta] -> [GA]. Having those clear stages gives us a framework to define what requirements of stage transitions would be which'll ideally lead to us producing higher quality, more predictable, more consistent results for our end users.For instance, requirements from [Optional] -> [Default] could be higher level abstractions like:Confidence in stabilityStrong evidence to indicate superiority in majority of workloads (by count or importance or size, etc)These are all things we kind of do implicitly and ad-hoc on the mailing list, and I'm not looking to tie us down to any granular structure or specificity. More thinking it could be useful for someone that's worked on something who wonders "Huh. How do I take this from being optional to the default?" and having an answer better than "reinvent the wheel every time and fling spaghetti at the dev list and pray".:)On Wed, Dec 11, 2024, at 1:04 PM, Paulo Motta wrote:Thanks for bringing up this topic, Josh. Outside of the major f

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Paulo Motta]

> If we don’t intend to begin fixing the feature within the next year or so
we should deprecate it entirely.

+1 - this is probably topic for another thread but isn’t MVs fundamentally
solved with Accord? In my ignorance this is “just” a matter of adding an
Accord backend to MV syntax to fix it reliably.

On Thu, 12 Dec 2024 at 09:58 Benedict  wrote:

> I think alpha is fine. It communicates fairly well that there’s no near
> term expectation they will be production capable.
>
> There is (I think) still an intention to improve them, but they are janky.
> If we don’t intend to begin fixing the feature within the next year or so
> we should deprecate it entirely.
>
>
> On 12 Dec 2024, at 14:46, Aleksey Yeshchenko  wrote:
>
> But MVs are not alpha or preview, as they are not actively being worked
> on. They are currently broken. Calling them ‘alpha’ makes ‘alpha’
> overloaded and less useful.
>
>
>
> On 12 Dec 2024, at 14:00, Josh McKenzie  wrote:
>
> But we also need an approved non-euphemism for features like MVs (I
> suggest ‘broken’) and possibly a softer version of it ('dangerous') for
> our existing features that work fine in some narrow well-defined
> circumstances but will blow in your face if you don’t know exactly what you
> are doing.
>
> Feels like the real answer is:
>
>1. Endeavor to never get ourselves into this state
>2. Take immediate action if we discover we're there (fix feature if
>possible, deprecate and remove if not). Not "leave to fester for years"
>
> I like the introduction of 'alpha' as an alias for 'Preview'; not sure why
> that wasn't what we immediately came up with collectively given how
> widespread its usage is. :)
>
> What would demoting MV's to 'alpha' right now look like? We'd warn on
> their usage w/some different structure and verbiage, and it'd be pretty
> implicitly clear to people they shouldn't use it in production right?
>
> It seems to me that the 3 categories would be sufficient even to handle
> our current scenario where we have some things in the system that are a Bad
> Idea to use in production.
>
> On Thu, Dec 12, 2024, at 6:06 AM, Aleksey Yeshchenko wrote:
>
> I don’t like ‘unstable’ either, albeit for a different reason, but I don’t
> think three is enough and fits, as we already have some features that don’t
> fit into either of (preview,beta,ga) - released but broken, released but
> dangerous, deprecated, removed.
>
> For new features going forward, alpha (preview) -> beta -> GA works well
> enough.
>
> But we also need an approved non-euphemism for features like MVs (I
> suggest ‘broken’) and possibly a softer version of it ('dangerous') for
> our existing features that work fine in some narrow well-defined
> circumstances but will blow in your face if you don’t know exactly what you
> are doing.
>
> These classifications are largely orthogonal.
>
> Alpha(preview)->Beta->GA communicates readiness of a feature under
> development, with GA being the default final state for most features.
>
> From there a feature can transition into ‘broken’ or ‘dangerous’
> territory. Serious issues get uncovered (very) late sometimes. It is what
> it is.
> And we do deprecate and remove functionality when it’s superseded.
>
>
> -1 on unstable. It's way too many words than are needed. Three is a
> magic number and fits:
>
> Preview
> Beta
> GA
>
>
> On 11 Dec 2024, at 18:50, Josh McKenzie  wrote:
>
> A structured, disciplined approach to graduating something from [Optional]
> -> [Default] makes sense to me, similar to how we're talking about a
> structured flow of [Preview] -> [Beta] -> [GA]. Having those clear stages
> gives us a framework to define what requirements of stage transitions would
> be which'll ideally lead to us producing higher quality, more predictable,
> more consistent results for our end users.
>
> For instance, requirements from [Optional] -> [Default] could be higher
> level abstractions like:
>
>- Confidence in stability
>- Strong evidence to indicate superiority in majority of workloads (by
>count or importance or size, etc)
>
> These are all things we kind of do implicitly and ad-hoc on the mailing
> list, and I'm not looking to tie us down to any granular structure or
> specificity. More thinking it could be useful for someone that's worked on
> something who wonders "Huh. How do I take this from being optional to the
> default?" and having an answer better than "reinvent the wheel every time
> and fling spaghetti at the dev list and pray".
>
> :)
>
>
> On Wed, Dec 11, 2024, at 1:04 PM, Paulo Motta wrote:
>
> Thanks for bringing up this topic, Josh.
>
> Outside of the major features (ie. MV/SAI/TCM/Accord), one related
> discussion in this topic is: how can we "promote" small improvements in
> existing features from optional to default ?
>
> It makes sense to have optimizations launched behind a feature flag
> initially (beta phase) while the improvement gets real world exposure, but
> I think we need a better way to promote

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Benedict]

I think alpha is fine. It communicates fairly well that there’s no near term expectation they will be production capable. There is (I think) still an intention to improve them, but they are janky. If we don’t intend to begin fixing the feature within the next year or so we should deprecate it entirely.On 12 Dec 2024, at 14:46, Aleksey Yeshchenko  wrote:But MVs are not alpha or preview, as they are not actively being worked on. They are currently broken. Calling them ‘alpha’ makes ‘alpha’ overloaded and less useful.On 12 Dec 2024, at 14:00, Josh McKenzie  wrote:But we also need an approved non-euphemism for features like MVs (I suggest ‘broken’) and possibly a softer version of it ('dangerous') for our existing features that work fine in some narrow well-defined circumstances but will blow in your face if you don’t know exactly what you are doing.Feels like the real answer is:Endeavor to never get ourselves into this stateTake immediate action if we discover we're there (fix feature if possible, deprecate and remove if not). Not "leave to fester for years"I like the introduction of 'alpha' as an alias for 'Preview'; not sure why that wasn't what we immediately came up with collectively given how widespread its usage is. :)What would demoting MV's to 'alpha' right now look like? We'd warn on their usage w/some different structure and verbiage, and it'd be pretty implicitly clear to people they shouldn't use it in production right?It seems to me that the 3 categories would be sufficient even to handle our current scenario where we have some things in the system that are a Bad Idea to use in production.On Thu, Dec 12, 2024, at 6:06 AM, Aleksey Yeshchenko wrote:I don’t like ‘unstable’ either, albeit for a different reason, but I don’t think three is enough and fits, as we already have some features that don’t fit into either of (preview,beta,ga) - released but broken, released but dangerous, deprecated, removed.For new features going forward, alpha (preview) -> beta -> GA works well enough.But we also need an approved non-euphemism for features like MVs (I suggest ‘broken’) and possibly a softer version of it ('dangerous') for our existing features that work fine in some narrow well-defined circumstances but will blow in your face if you don’t know exactly what you are doing.These classifications are largely orthogonal.Alpha(preview)->Beta->GA communicates readiness of a feature under development, with GA being the default final state for most features.From there a feature can transition into ‘broken’ or ‘dangerous’ territory. Serious issues get uncovered (very) late sometimes. It is what it is.And we do deprecate and remove functionality when it’s superseded.-1 on unstable. It's way too many words than are needed. Three is amagic number and fits:PreviewBetaGAOn 11 Dec 2024, at 18:50, Josh McKenzie  wrote:A structured, disciplined approach to graduating something from [Optional] -> [Default] makes sense to me, similar to how we're talking about a structured flow of [Preview] -> [Beta] -> [GA]. Having those clear stages gives us a framework to define what requirements of stage transitions would be which'll ideally lead to us producing higher quality, more predictable, more consistent results for our end users.For instance, requirements from [Optional] -> [Default] could be higher level abstractions like:Confidence in stabilityStrong evidence to indicate superiority in majority of workloads (by count or importance or size, etc)These are all things we kind of do implicitly and ad-hoc on the mailing list, and I'm not looking to tie us down to any granular structure or specificity. More thinking it could be useful for someone that's worked on something who wonders "Huh. How do I take this from being optional to the default?" and having an answer better than "reinvent the wheel every time and fling spaghetti at the dev list and pray".:)On Wed, Dec 11, 2024, at 1:04 PM, Paulo Motta wrote:Thanks for bringing up this topic, Josh. Outside of the major features (ie. MV/SAI/TCM/Accord), one related discussion in this topic is: how can we "promote" small improvements in existing features from optional to default ?It makes sense to have optimizations launched behind a feature flag initially (beta phase) while the improvement gets real world exposure, but I think we need a better way to promote these optimizations to default behavior on a regular cadence.Take for example optimized repairs from CASSANDRA-16274. It was launched in 4.x as an optional feature gated behind a flag, ie. auto_optimise_full_repair_streams: false. I could be easily missing something, but is there a world where non-optimized repairs make sense once this optimization is proven to work ? I agree this is fine while the feature is maturing, but at some point we need to rip the bandaid and make the optimization default (and clearly communicate that). This would allow cleanup code toil of default behavior that is no longer being used, because everyone is ena

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Paulo Motta]

> But MVs are not alpha or preview, as they are not actively being worked
on.

fwiw I think Jaydeep and Runtian are looking into improving MV status quo
according to
https://lists.apache.org/thread/d3qo3vjxn4116htf175yzcg94s6jq07d

On Thu, 12 Dec 2024 at 09:45 Aleksey Yeshchenko  wrote:

> But MVs are not alpha or preview, as they are not actively being worked
> on. They are currently broken. Calling them ‘alpha’ makes ‘alpha’
> overloaded and less useful.
>
>
> On 12 Dec 2024, at 14:00, Josh McKenzie  wrote:
>
> But we also need an approved non-euphemism for features like MVs (I
> suggest ‘broken’) and possibly a softer version of it ('dangerous') for
> our existing features that work fine in some narrow well-defined
> circumstances but will blow in your face if you don’t know exactly what you
> are doing.
>
> Feels like the real answer is:
>
>1. Endeavor to never get ourselves into this state
>2. Take immediate action if we discover we're there (fix feature if
>possible, deprecate and remove if not). Not "leave to fester for years"
>
> I like the introduction of 'alpha' as an alias for 'Preview'; not sure why
> that wasn't what we immediately came up with collectively given how
> widespread its usage is. :)
>
> What would demoting MV's to 'alpha' right now look like? We'd warn on
> their usage w/some different structure and verbiage, and it'd be pretty
> implicitly clear to people they shouldn't use it in production right?
>
> It seems to me that the 3 categories would be sufficient even to handle
> our current scenario where we have some things in the system that are a Bad
> Idea to use in production.
>
> On Thu, Dec 12, 2024, at 6:06 AM, Aleksey Yeshchenko wrote:
>
> I don’t like ‘unstable’ either, albeit for a different reason, but I don’t
> think three is enough and fits, as we already have some features that don’t
> fit into either of (preview,beta,ga) - released but broken, released but
> dangerous, deprecated, removed.
>
> For new features going forward, alpha (preview) -> beta -> GA works well
> enough.
>
> But we also need an approved non-euphemism for features like MVs (I
> suggest ‘broken’) and possibly a softer version of it ('dangerous') for
> our existing features that work fine in some narrow well-defined
> circumstances but will blow in your face if you don’t know exactly what you
> are doing.
>
> These classifications are largely orthogonal.
>
> Alpha(preview)->Beta->GA communicates readiness of a feature under
> development, with GA being the default final state for most features.
>
> From there a feature can transition into ‘broken’ or ‘dangerous’
> territory. Serious issues get uncovered (very) late sometimes. It is what
> it is.
> And we do deprecate and remove functionality when it’s superseded.
>
>
> -1 on unstable. It's way too many words than are needed. Three is a
> magic number and fits:
>
> Preview
> Beta
> GA
>
>
> On 11 Dec 2024, at 18:50, Josh McKenzie  wrote:
>
> A structured, disciplined approach to graduating something from [Optional]
> -> [Default] makes sense to me, similar to how we're talking about a
> structured flow of [Preview] -> [Beta] -> [GA]. Having those clear stages
> gives us a framework to define what requirements of stage transitions would
> be which'll ideally lead to us producing higher quality, more predictable,
> more consistent results for our end users.
>
> For instance, requirements from [Optional] -> [Default] could be higher
> level abstractions like:
>
>- Confidence in stability
>- Strong evidence to indicate superiority in majority of workloads (by
>count or importance or size, etc)
>
> These are all things we kind of do implicitly and ad-hoc on the mailing
> list, and I'm not looking to tie us down to any granular structure or
> specificity. More thinking it could be useful for someone that's worked on
> something who wonders "Huh. How do I take this from being optional to the
> default?" and having an answer better than "reinvent the wheel every time
> and fling spaghetti at the dev list and pray".
>
> :)
>
>
> On Wed, Dec 11, 2024, at 1:04 PM, Paulo Motta wrote:
>
> Thanks for bringing up this topic, Josh.
>
> Outside of the major features (ie. MV/SAI/TCM/Accord), one related
> discussion in this topic is: how can we "promote" small improvements in
> existing features from optional to default ?
>
> It makes sense to have optimizations launched behind a feature flag
> initially (beta phase) while the improvement gets real world exposure, but
> I think we need a better way to promote these optimizations to default
> behavior on a regular cadence.
>
> Take for example optimized repairs from CASSANDRA-16274. It was launched
> in 4.x as an optional feature gated behind a flag,
> ie. auto_optimise_full_repair_streams: false.
>
> I could be easily missing something, but is there a world where
> non-optimized repairs make sense once this optimization is proven to work ?
> I agree this is fine while the feature is maturing, bu

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Aleksey Yeshchenko]

But MVs are not alpha or preview, as they are not actively being worked on. 
They are currently broken. Calling them ‘alpha’ makes ‘alpha’ overloaded and 
less useful.

> On 12 Dec 2024, at 14:00, Josh McKenzie  wrote:
> 
>> But we also need an approved non-euphemism for features like MVs (I suggest 
>> ‘broken’) and possibly a softer version of it ('dangerous') for our existing 
>> features that work fine in some narrow well-defined circumstances but will 
>> blow in your face if you don’t know exactly what you are doing.
> Feels like the real answer is:
> Endeavor to never get ourselves into this state
> Take immediate action if we discover we're there (fix feature if possible, 
> deprecate and remove if not). Not "leave to fester for years"
> I like the introduction of 'alpha' as an alias for 'Preview'; not sure why 
> that wasn't what we immediately came up with collectively given how 
> widespread its usage is. :)
> 
> What would demoting MV's to 'alpha' right now look like? We'd warn on their 
> usage w/some different structure and verbiage, and it'd be pretty implicitly 
> clear to people they shouldn't use it in production right?
> 
> It seems to me that the 3 categories would be sufficient even to handle our 
> current scenario where we have some things in the system that are a Bad Idea 
> to use in production.
> 
> On Thu, Dec 12, 2024, at 6:06 AM, Aleksey Yeshchenko wrote:
>> I don’t like ‘unstable’ either, albeit for a different reason, but I don’t 
>> think three is enough and fits, as we already have some features that don’t 
>> fit into either of (preview,beta,ga) - released but broken, released but 
>> dangerous, deprecated, removed.
>> 
>> For new features going forward, alpha (preview) -> beta -> GA works well 
>> enough.
>> 
>> But we also need an approved non-euphemism for features like MVs (I suggest 
>> ‘broken’) and possibly a softer version of it ('dangerous') for our existing 
>> features that work fine in some narrow well-defined circumstances but will 
>> blow in your face if you don’t know exactly what you are doing.
>> 
>> These classifications are largely orthogonal.
>> 
>> Alpha(preview)->Beta->GA communicates readiness of a feature under 
>> development, with GA being the default final state for most features.
>> 
>> From there a feature can transition into ‘broken’ or ‘dangerous’ territory. 
>> Serious issues get uncovered (very) late sometimes. It is what it is.
>> And we do deprecate and remove functionality when it’s superseded.
>> 
>> 
>>> -1 on unstable. It's way too many words than are needed. Three is a
>>> magic number and fits:
>>> 
>>> Preview
>>> Beta
>>> GA
>> 
>>> On 11 Dec 2024, at 18:50, Josh McKenzie  wrote:
>>> 
>>> A structured, disciplined approach to graduating something from [Optional] 
>>> -> [Default] makes sense to me, similar to how we're talking about a 
>>> structured flow of [Preview] -> [Beta] -> [GA]. Having those clear stages 
>>> gives us a framework to define what requirements of stage transitions would 
>>> be which'll ideally lead to us producing higher quality, more predictable, 
>>> more consistent results for our end users.
>>> 
>>> For instance, requirements from [Optional] -> [Default] could be higher 
>>> level abstractions like:
>>> Confidence in stability
>>> Strong evidence to indicate superiority in majority of workloads (by count 
>>> or importance or size, etc)
>>> These are all things we kind of do implicitly and ad-hoc on the mailing 
>>> list, and I'm not looking to tie us down to any granular structure or 
>>> specificity. More thinking it could be useful for someone that's worked on 
>>> something who wonders "Huh. How do I take this from being optional to the 
>>> default?" and having an answer better than "reinvent the wheel every time 
>>> and fling spaghetti at the dev list and pray".
>>> 
>>> :)
>>> 
>>> 
>>> On Wed, Dec 11, 2024, at 1:04 PM, Paulo Motta wrote:
 Thanks for bringing up this topic, Josh. 
 
 Outside of the major features (ie. MV/SAI/TCM/Accord), one related 
 discussion in this topic is: how can we "promote" small improvements in 
 existing features from optional to default ?
 
 It makes sense to have optimizations launched behind a feature flag 
 initially (beta phase) while the improvement gets real world exposure, but 
 I think we need a better way to promote these optimizations to default 
 behavior on a regular cadence.
 
 Take for example optimized repairs from CASSANDRA-16274. It was launched 
 in 4.x as an optional feature gated behind a flag, ie. 
 auto_optimise_full_repair_streams: false. 
 
 I could be easily missing something, but is there a world where 
 non-optimized repairs make sense once this optimization is proven to work 
 ? I agree this is fine while the feature is maturing, but at some point we 
 need to rip the bandaid and make the optimization default (and clearly 
 communicate that). This would allow 

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Josh McKenzie]

> But we also need an approved non-euphemism for features like MVs (I suggest 
> ‘broken’) and possibly a softer version of it ('dangerous') for our existing 
> features that work fine in some narrow well-defined circumstances but will 
> blow in your face if you don’t know exactly what you are doing.
Feels like the real answer is:
 1. Endeavor to never get ourselves into this state
 2. Take immediate action if we discover we're there (fix feature if possible, 
deprecate and remove if not). Not "leave to fester for years"
I like the introduction of 'alpha' as an alias for 'Preview'; not sure why that 
wasn't what we immediately came up with collectively given how widespread its 
usage is. :)

What would demoting MV's to 'alpha' right now look like? We'd warn on their 
usage w/some different structure and verbiage, and it'd be pretty implicitly 
clear to people they shouldn't use it in production right?

It seems to me that the 3 categories would be sufficient even to handle our 
current scenario where we have some things in the system that are a Bad Idea to 
use in production.

On Thu, Dec 12, 2024, at 6:06 AM, Aleksey Yeshchenko wrote:
> I don’t like ‘unstable’ either, albeit for a different reason, but I don’t 
> think three is enough and fits, as we already have some features that don’t 
> fit into either of (preview,beta,ga) - released but broken, released but 
> dangerous, deprecated, removed.
> 
> For new features going forward, alpha (preview) -> beta -> GA works well 
> enough.
> 
> But we also need an approved non-euphemism for features like MVs (I suggest 
> ‘broken’) and possibly a softer version of it ('dangerous') for our existing 
> features that work fine in some narrow well-defined circumstances but will 
> blow in your face if you don’t know exactly what you are doing.
> 
> These classifications are largely orthogonal.
> 
> Alpha(preview)->Beta->GA communicates readiness of a feature under 
> development, with GA being the default final state for most features.
> 
> From there a feature can transition into ‘broken’ or ‘dangerous’ territory. 
> Serious issues get uncovered (very) late sometimes. It is what it is.
> And we do deprecate and remove functionality when it’s superseded.
> 
> 
>> -1 on unstable. It's way too many words than are needed. Three is a
>> magic number and fits:
>> 
>> Preview
>> Beta
>> GA
> 
>> On 11 Dec 2024, at 18:50, Josh McKenzie  wrote:
>> 
>> A structured, disciplined approach to graduating something from [Optional] 
>> -> [Default] makes sense to me, similar to how we're talking about a 
>> structured flow of [Preview] -> [Beta] -> [GA]. Having those clear stages 
>> gives us a framework to define what requirements of stage transitions would 
>> be which'll ideally lead to us producing higher quality, more predictable, 
>> more consistent results for our end users.
>> 
>> For instance, requirements from [Optional] -> [Default] could be higher 
>> level abstractions like:
>>  • Confidence in stability
>>  • Strong evidence to indicate superiority in majority of workloads (by 
>> count or importance or size, etc)
>> These are all things we kind of do implicitly and ad-hoc on the mailing 
>> list, and I'm not looking to tie us down to any granular structure or 
>> specificity. More thinking it could be useful for someone that's worked on 
>> something who wonders "Huh. How do I take this from being optional to the 
>> default?" and having an answer better than "reinvent the wheel every time 
>> and fling spaghetti at the dev list and pray".
>> 
>> :)
>> 
>> 
>> On Wed, Dec 11, 2024, at 1:04 PM, Paulo Motta wrote:
>>> Thanks for bringing up this topic, Josh. 
>>> 
>>> Outside of the major features (ie. MV/SAI/TCM/Accord), one related 
>>> discussion in this topic is: how can we "promote" small improvements in 
>>> existing features from optional to default ?
>>> 
>>> It makes sense to have optimizations launched behind a feature flag 
>>> initially (beta phase) while the improvement gets real world exposure, but 
>>> I think we need a better way to promote these optimizations to default 
>>> behavior on a regular cadence.
>>> 
>>> Take for example optimized repairs from CASSANDRA-16274. It was launched in 
>>> 4.x as an optional feature gated behind a flag, ie. 
>>> auto_optimise_full_repair_streams: false. 
>>> 
>>> I could be easily missing something, but is there a world where 
>>> non-optimized repairs make sense once this optimization is proven to work ? 
>>> I agree this is fine while the feature is maturing, but at some point we 
>>> need to rip the bandaid and make the optimization default (and clearly 
>>> communicate that). This would allow cleanup code toil of default behavior 
>>> that is no longer being used, because everyone is enabling the improvement 
>>> during deployment.
>>> 
>>> This is just one example to demonstrate the issue and I don't want this 
>>> discussion to focus on this particular case, but I can think of other 
>>> improvements launc

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Aleksey Yeshchenko]

I don’t like ‘unstable’ either, albeit for a different reason, but I don’t 
think three is enough and fits, as we already have some features that don’t fit 
into either of (preview,beta,ga) - released but broken, released but dangerous, 
deprecated, removed.

For new features going forward, alpha (preview) -> beta -> GA works well enough.

But we also need an approved non-euphemism for features like MVs (I suggest 
‘broken’) and possibly a softer version of it ('dangerous') for our existing 
features that work fine in some narrow well-defined circumstances but will blow 
in your face if you don’t know exactly what you are doing.

These classifications are largely orthogonal.

Alpha(preview)->Beta->GA communicates readiness of a feature under development, 
with GA being the default final state for most features.

From there a feature can transition into ‘broken’ or ‘dangerous’ territory. 
Serious issues get uncovered (very) late sometimes. It is what it is.
And we do deprecate and remove functionality when it’s superseded.


> -1 on unstable. It's way too many words than are needed. Three is a
> magic number and fits:
> 
> Preview
> Beta
> GA

> On 11 Dec 2024, at 18:50, Josh McKenzie  wrote:
> 
> A structured, disciplined approach to graduating something from [Optional] -> 
> [Default] makes sense to me, similar to how we're talking about a structured 
> flow of [Preview] -> [Beta] -> [GA]. Having those clear stages gives us a 
> framework to define what requirements of stage transitions would be which'll 
> ideally lead to us producing higher quality, more predictable, more 
> consistent results for our end users.
> 
> For instance, requirements from [Optional] -> [Default] could be higher level 
> abstractions like:
> Confidence in stability
> Strong evidence to indicate superiority in majority of workloads (by count or 
> importance or size, etc)
> These are all things we kind of do implicitly and ad-hoc on the mailing list, 
> and I'm not looking to tie us down to any granular structure or specificity. 
> More thinking it could be useful for someone that's worked on something who 
> wonders "Huh. How do I take this from being optional to the default?" and 
> having an answer better than "reinvent the wheel every time and fling 
> spaghetti at the dev list and pray".
> 
> :)
> 
> 
> On Wed, Dec 11, 2024, at 1:04 PM, Paulo Motta wrote:
>> Thanks for bringing up this topic, Josh. 
>> 
>> Outside of the major features (ie. MV/SAI/TCM/Accord), one related 
>> discussion in this topic is: how can we "promote" small improvements in 
>> existing features from optional to default ?
>> 
>> It makes sense to have optimizations launched behind a feature flag 
>> initially (beta phase) while the improvement gets real world exposure, but I 
>> think we need a better way to promote these optimizations to default 
>> behavior on a regular cadence.
>> 
>> Take for example optimized repairs from CASSANDRA-16274. It was launched in 
>> 4.x as an optional feature gated behind a flag, ie. 
>> auto_optimise_full_repair_streams: false. 
>> 
>> I could be easily missing something, but is there a world where 
>> non-optimized repairs make sense once this optimization is proven to work ? 
>> I agree this is fine while the feature is maturing, but at some point we 
>> need to rip the bandaid and make the optimization default (and clearly 
>> communicate that). This would allow cleanup code toil of default behavior 
>> that is no longer being used, because everyone is enabling the improvement 
>> during deployment.
>> 
>> This is just one example to demonstrate the issue and I don't want this 
>> discussion to focus on this particular case, but I can think of other 
>> improvements launched as optional that are never made default.
>> 
>> I don't know if this should be continued to be addressed on a 
>> improvement-by-improvement basis or if we could have a more streamlined 
>> process to review and communicate these changes more consciously at every 
>> major release.
>> 
>> In the same way we open a loop when adding an optimized behavior behind a 
>> feature flag, I think we should have a process to close these loops by 
>> promoting these optimizations to default when it makes sense.
>> 
>> On Tue, Dec 10, 2024 at 2:10 PM Josh McKenzie > > wrote:
>> 
>> So some questions to test a world w/3 classifications (Preview, Beta, GA):
>> - What would we do with the current experimental features (MV's, JDK17, 
>> witnesses, etc)? Flag them as preview or beta as appropriate on a 
>> case-by-case basis and add runtime warnings / documentation where missing?
>> 
>> - What would we do in the future if a feature's GA and we discover a Very 
>> Big Problem with it that'll take some time to fix? Keep it GA but cut a 
>> hotfix release w/a bunch of warnings? Bounce it back to Preview? Leave it be 
>> and just feverishly try and fix it?
>> 
>>> for policy decisions like this (that don’t need to be agreed in advance) 

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Josh McKenzie]

A structured, disciplined approach to graduating something from [Optional] -> 
[Default] makes sense to me, similar to how we're talking about a structured 
flow of [Preview] -> [Beta] -> [GA]. Having those clear stages gives us a 
framework to define what requirements of stage transitions would be which'll 
ideally lead to us producing higher quality, more predictable, more consistent 
results for our end users.

For instance, requirements from [Optional] -> [Default] could be higher level 
abstractions like:
 • Confidence in stability
 • Strong evidence to indicate superiority in majority of workloads (by count 
or importance or size, etc)
These are all things we kind of do implicitly and ad-hoc on the mailing list, 
and I'm not looking to tie us down to any granular structure or specificity. 
More thinking it could be useful for someone that's worked on something who 
wonders "Huh. How do I take this from being optional to the default?" and 
having an answer better than "reinvent the wheel every time and fling spaghetti 
at the dev list and pray".

:)


On Wed, Dec 11, 2024, at 1:04 PM, Paulo Motta wrote:
> Thanks for bringing up this topic, Josh. 
> 
> Outside of the major features (ie. MV/SAI/TCM/Accord), one related discussion 
> in this topic is: how can we "promote" small improvements in existing 
> features from optional to default ?
> 
> It makes sense to have optimizations launched behind a feature flag initially 
> (beta phase) while the improvement gets real world exposure, but I think we 
> need a better way to promote these optimizations to default behavior on a 
> regular cadence.
> 
> Take for example optimized repairs from CASSANDRA-16274. It was launched in 
> 4.x as an optional feature gated behind a flag, ie. 
> auto_optimise_full_repair_streams: false. 
> 
> I could be easily missing something, but is there a world where non-optimized 
> repairs make sense once this optimization is proven to work ? I agree this is 
> fine while the feature is maturing, but at some point we need to rip the 
> bandaid and make the optimization default (and clearly communicate that). 
> This would allow cleanup code toil of default behavior that is no longer 
> being used, because everyone is enabling the improvement during deployment.
> 
> This is just one example to demonstrate the issue and I don't want this 
> discussion to focus on this particular case, but I can think of other 
> improvements launched as optional that are never made default.
> 
> I don't know if this should be continued to be addressed on a 
> improvement-by-improvement basis or if we could have a more streamlined 
> process to review and communicate these changes more consciously at every 
> major release.
> 
> In the same way we open a loop when adding an optimized behavior behind a 
> feature flag, I think we should have a process to close these loops by 
> promoting these optimizations to default when it makes sense.
> 
> On Tue, Dec 10, 2024 at 2:10 PM Josh McKenzie  wrote:
>> __
>> So some questions to test a world w/3 classifications (Preview, Beta, GA):
>> - What would we do with the current experimental features (MV's, JDK17, 
>> witnesses, etc)? Flag them as preview or beta as appropriate on a 
>> case-by-case basis and add runtime warnings / documentation where missing?
>> 
>> - What would we do in the future if a feature's GA and we discover a Very 
>> Big Problem with it that'll take some time to fix? Keep it GA but cut a 
>> hotfix release w/a bunch of warnings? Bounce it back to Preview? Leave it be 
>> and just feverishly try and fix it?
>> 
>>> for policy decisions like this (that don’t need to be agreed in advance) we 
>>> should try to legislate the minimum necessary policy to proceed today
>> Definitely agree; MV's being in limbo for years strains the "3-step 
>> classification" structure for me. If we want to avoid having a solution for 
>> the MV-shaped case on the grounds we won't allow ourselves to reach this 
>> state again in the future, that seems reasonable. With the caveat that we 
>> *might* be in a similar situation with vector search right now, etc.
>> 
>> 
>> On Tue, Dec 10, 2024, at 1:48 PM, Benedict Elliott Smith wrote:
>>> Yep, I agree with this - we can revisit if we ever absolutely feel the need 
>>> to add additional states for exceptional circumstances.
>>> 
>>> > On 10 Dec 2024, at 13:24, Patrick McFadin  wrote:
>>> > 
>>> > -1 on unstable. It's way too many words than are needed. Three is a
>>> > magic number and fits:
>>> > 
>>> > Preview
>>> > Beta
>>> > GA
>>> > 
>>> > As a matter of testing the process, any pending CEP should go though
>>> > this exercise so we can see how it will work.
>>> > 
>>> > PS
>>> > Got the actual numbers from Whimsy.
>>> > DEV - 1425 users
>>> > USER - 2650
>>> > 
>>> > This means that when features experience a state change, finding more
>>> > avenues to get the word out will be important.
>>> > 
>>> > On Tue, Dec 10, 2024 at 10:04 AM Benedict Elliott Smith
>

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Paulo Motta]

Thanks for bringing up this topic, Josh.

Outside of the major features (ie. MV/SAI/TCM/Accord), one related
discussion in this topic is: how can we "promote" small improvements in
existing features from optional to default ?

It makes sense to have optimizations launched behind a feature flag
initially (beta phase) while the improvement gets real world exposure, but
I think we need a better way to promote these optimizations to default
behavior on a regular cadence.

Take for example optimized repairs from CASSANDRA-16274. It was launched in
4.x as an optional feature gated behind a flag,
ie. auto_optimise_full_repair_streams: false.

I could be easily missing something, but is there a world where
non-optimized repairs make sense once this optimization is proven to work ?
I agree this is fine while the feature is maturing, but at some point we
need to rip the bandaid and make the optimization default (and clearly
communicate that). This would allow cleanup code toil of default behavior
that is no longer being used, because everyone is enabling the improvement
during deployment.

This is just one example to demonstrate the issue and I don't want this
discussion to focus on this particular case, but I can think of other
improvements launched as optional that are never made default.

I don't know if this should be continued to be addressed on a
improvement-by-improvement basis or if we could have a more streamlined
process to review and communicate these changes more consciously at every
major release.

In the same way we open a loop when adding an optimized behavior behind a
feature flag, I think we should have a process to close these loops by
promoting these optimizations to default when it makes sense.

On Tue, Dec 10, 2024 at 2:10 PM Josh McKenzie  wrote:

> So some questions to test a world w/3 classifications (Preview, Beta, GA):
> - What would we do with the current experimental features (MV's, JDK17,
> witnesses, etc)? Flag them as preview or beta as appropriate on a
> case-by-case basis and add runtime warnings / documentation where missing?
>
> - What would we do in the future if a feature's GA and we discover a Very
> Big Problem with it that'll take some time to fix? Keep it GA but cut a
> hotfix release w/a bunch of warnings? Bounce it back to Preview? Leave it
> be and just feverishly try and fix it?
>
> for policy decisions like this (that don’t need to be agreed in advance)
> we should try to legislate the minimum necessary policy to proceed today
>
> Definitely agree; MV's being in limbo for years strains the "3-step
> classification" structure for me. If we want to avoid having a solution for
> the MV-shaped case on the grounds we won't allow ourselves to reach this
> state again in the future, that seems reasonable. With the caveat that we
> *might* be in a similar situation with vector search right now, etc.
>
>
> On Tue, Dec 10, 2024, at 1:48 PM, Benedict Elliott Smith wrote:
>
> Yep, I agree with this - we can revisit if we ever absolutely feel the
> need to add additional states for exceptional circumstances.
>
> > On 10 Dec 2024, at 13:24, Patrick McFadin  wrote:
> >
> > -1 on unstable. It's way too many words than are needed. Three is a
> > magic number and fits:
> >
> > Preview
> > Beta
> > GA
> >
> > As a matter of testing the process, any pending CEP should go though
> > this exercise so we can see how it will work.
> >
> > PS
> > Got the actual numbers from Whimsy.
> > DEV - 1425 users
> > USER - 2650
> >
> > This means that when features experience a state change, finding more
> > avenues to get the word out will be important.
> >
> > On Tue, Dec 10, 2024 at 10:04 AM Benedict Elliott Smith
> >  wrote:
> >>
> >> As an aside, it would be nice to admit we basically revisit everything
> each time it becomes relevant again, and for policy decisions like this
> (that don’t need to be agreed in advance) we should try to legislate the
> minimum necessary policy to proceed today, and leave future refinements for
> later when the relevant context arises.
> >>
> >> On 10 Dec 2024, at 13:00, Benedict Elliott Smith 
> wrote:
> >>
> >> I agree with Aleksey that if we think something is broken, we shouldn’t
> use euphemisms, and for this reason I don’t like unstable (this could for
> instance simply mean API unstable). If we intend to never need this
> descriptor, we should avoid bike-shedding and insert a “placeholder” for
> now to be refined as and when we need it when we have the necessary future
> context.
> >>
> >> i.e.
> >>
> >> preview -> beta -> [“has problems that will take time to resolve
> placeholder” -> beta] -> GA
> >>
> >>
> >>
> >> On 10 Dec 2024, at 12:39, Josh McKenzie  wrote:
> >>
> >> +1 to this classification with one addition. I think we need to augment
> this with formalization on what we do with features we don't recommend
> people use (i.e. MV in their current incarnation). For something
> retroactively found to be unstable, we could add an "Unstable"
> qualification 

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Josh McKenzie]

So some questions to test a world w/3 classifications (Preview, Beta, GA):
- What would we do with the current experimental features (MV's, JDK17, 
witnesses, etc)? Flag them as preview or beta as appropriate on a case-by-case 
basis and add runtime warnings / documentation where missing?

- What would we do in the future if a feature's GA and we discover a Very Big 
Problem with it that'll take some time to fix? Keep it GA but cut a hotfix 
release w/a bunch of warnings? Bounce it back to Preview? Leave it be and just 
feverishly try and fix it?

> for policy decisions like this (that don’t need to be agreed in advance) we 
> should try to legislate the minimum necessary policy to proceed today
Definitely agree; MV's being in limbo for years strains the "3-step 
classification" structure for me. If we want to avoid having a solution for the 
MV-shaped case on the grounds we won't allow ourselves to reach this state 
again in the future, that seems reasonable. With the caveat that we *might* be 
in a similar situation with vector search right now, etc.


On Tue, Dec 10, 2024, at 1:48 PM, Benedict Elliott Smith wrote:
> Yep, I agree with this - we can revisit if we ever absolutely feel the need 
> to add additional states for exceptional circumstances.
> 
> > On 10 Dec 2024, at 13:24, Patrick McFadin  wrote:
> > 
> > -1 on unstable. It's way too many words than are needed. Three is a
> > magic number and fits:
> > 
> > Preview
> > Beta
> > GA
> > 
> > As a matter of testing the process, any pending CEP should go though
> > this exercise so we can see how it will work.
> > 
> > PS
> > Got the actual numbers from Whimsy.
> > DEV - 1425 users
> > USER - 2650
> > 
> > This means that when features experience a state change, finding more
> > avenues to get the word out will be important.
> > 
> > On Tue, Dec 10, 2024 at 10:04 AM Benedict Elliott Smith
> >  wrote:
> >> 
> >> As an aside, it would be nice to admit we basically revisit everything 
> >> each time it becomes relevant again, and for policy decisions like this 
> >> (that don’t need to be agreed in advance) we should try to legislate the 
> >> minimum necessary policy to proceed today, and leave future refinements 
> >> for later when the relevant context arises.
> >> 
> >> On 10 Dec 2024, at 13:00, Benedict Elliott Smith  
> >> wrote:
> >> 
> >> I agree with Aleksey that if we think something is broken, we shouldn’t 
> >> use euphemisms, and for this reason I don’t like unstable (this could for 
> >> instance simply mean API unstable). If we intend to never need this 
> >> descriptor, we should avoid bike-shedding and insert a “placeholder” for 
> >> now to be refined as and when we need it when we have the necessary future 
> >> context.
> >> 
> >> i.e.
> >> 
> >> preview -> beta -> [“has problems that will take time to resolve 
> >> placeholder” -> beta] -> GA
> >> 
> >> 
> >> 
> >> On 10 Dec 2024, at 12:39, Josh McKenzie  wrote:
> >> 
> >> +1 to this classification with one addition. I think we need to augment 
> >> this with formalization on what we do with features we don't recommend 
> >> people use (i.e. MV in their current incarnation). For something 
> >> retroactively found to be unstable, we could add an "Unstable" 
> >> qualification for it, leaving us with:
> >> 
> >> Unstable: Warnings on use, clearly communicated as to why, either on-track 
> >> to be fixed or removed from the codebase. No lingering for years in a 
> >> fugue state. We should target never needing this classification.
> >> Preview: Ready to be tried by end users but has caveats and most likely is 
> >> not api stable. Developer only documentation acceptable.
> >> Beta: Feature complete/API stable but has not had enough testing to be 
> >> considered rock solid. Developer and User documentation required.
> >> GA: Ready for use, no known issue, PMC is satisfied with the testing that 
> >> has been done
> >> 
> >> 
> >> To walk through how some of the flow might look to test the above:
> >> 
> >> Simple case:
> >> - Preview -> Beta -> GA
> >> 
> >> Late discovered defect case:
> >> - Preview -> Beta -> Unstable -> Beta -> GA
> >> 
> >> Pathological worst-case (i.e. MV):
> >> - Preview -> Beta -> GA -> Unstable -> [Preview|Removed]
> >> 
> >> On Tue, Dec 10, 2024, at 12:29 PM, Jeremiah Jordan wrote:
> >> 
> >> I agree with Aleksey and Patrick.  We should define terminology and then 
> >> stick to it.  My preferred list would be:
> >> 
> >> Preview - Ready to be tried by end users but has caveats and most likely 
> >> is not api stable.
> >> Beta - Feature complete/API stable but has not had enough testing to be 
> >> considered rock solid.
> >> GA - Ready for use, no known issue, PMC is satisfied with the testing that 
> >> has been done
> >> 
> >> 
> >> Whether or not something is enabled by default or the default 
> >> implementation is a separate access from the readiness.  Though if we are 
> >> replacing an existing thing with a new default I would hope we apply extra 

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Benedict Elliott Smith]

Yep, I agree with this - we can revisit if we ever absolutely feel the need to 
add additional states for exceptional circumstances.

> On 10 Dec 2024, at 13:24, Patrick McFadin  wrote:
> 
> -1 on unstable. It's way too many words than are needed. Three is a
> magic number and fits:
> 
> Preview
> Beta
> GA
> 
> As a matter of testing the process, any pending CEP should go though
> this exercise so we can see how it will work.
> 
> PS
> Got the actual numbers from Whimsy.
> DEV - 1425 users
> USER - 2650
> 
> This means that when features experience a state change, finding more
> avenues to get the word out will be important.
> 
> On Tue, Dec 10, 2024 at 10:04 AM Benedict Elliott Smith
>  wrote:
>> 
>> As an aside, it would be nice to admit we basically revisit everything each 
>> time it becomes relevant again, and for policy decisions like this (that 
>> don’t need to be agreed in advance) we should try to legislate the minimum 
>> necessary policy to proceed today, and leave future refinements for later 
>> when the relevant context arises.
>> 
>> On 10 Dec 2024, at 13:00, Benedict Elliott Smith  wrote:
>> 
>> I agree with Aleksey that if we think something is broken, we shouldn’t use 
>> euphemisms, and for this reason I don’t like unstable (this could for 
>> instance simply mean API unstable). If we intend to never need this 
>> descriptor, we should avoid bike-shedding and insert a “placeholder” for now 
>> to be refined as and when we need it when we have the necessary future 
>> context.
>> 
>> i.e.
>> 
>> preview -> beta -> [“has problems that will take time to resolve 
>> placeholder” -> beta] -> GA
>> 
>> 
>> 
>> On 10 Dec 2024, at 12:39, Josh McKenzie  wrote:
>> 
>> +1 to this classification with one addition. I think we need to augment this 
>> with formalization on what we do with features we don't recommend people use 
>> (i.e. MV in their current incarnation). For something retroactively found to 
>> be unstable, we could add an "Unstable" qualification for it, leaving us 
>> with:
>> 
>> Unstable: Warnings on use, clearly communicated as to why, either on-track 
>> to be fixed or removed from the codebase. No lingering for years in a fugue 
>> state. We should target never needing this classification.
>> Preview: Ready to be tried by end users but has caveats and most likely is 
>> not api stable. Developer only documentation acceptable.
>> Beta: Feature complete/API stable but has not had enough testing to be 
>> considered rock solid. Developer and User documentation required.
>> GA: Ready for use, no known issue, PMC is satisfied with the testing that 
>> has been done
>> 
>> 
>> To walk through how some of the flow might look to test the above:
>> 
>> Simple case:
>> - Preview -> Beta -> GA
>> 
>> Late discovered defect case:
>> - Preview -> Beta -> Unstable -> Beta -> GA
>> 
>> Pathological worst-case (i.e. MV):
>> - Preview -> Beta -> GA -> Unstable -> [Preview|Removed]
>> 
>> On Tue, Dec 10, 2024, at 12:29 PM, Jeremiah Jordan wrote:
>> 
>> I agree with Aleksey and Patrick.  We should define terminology and then 
>> stick to it.  My preferred list would be:
>> 
>> Preview - Ready to be tried by end users but has caveats and most likely is 
>> not api stable.
>> Beta - Feature complete/API stable but has not had enough testing to be 
>> considered rock solid.
>> GA - Ready for use, no known issue, PMC is satisfied with the testing that 
>> has been done
>> 
>> 
>> Whether or not something is enabled by default or the default implementation 
>> is a separate access from the readiness.  Though if we are replacing an 
>> existing thing with a new default I would hope we apply extra rigor to 
>> allowing that to happen.
>> 
>> -Jeremiah
>> 
>> On Dec 10, 2024 at 11:15:37 AM, Patrick McFadin  wrote:
>> 
>> I'm going to try to pull this back from the inevitable bikeshedding
>> and airing of grievances that happen. Rewind all the way back to
>> Josh's  original point, which is a defined process. Why I really love
>> this being brought up is our maturing process of communicating to the
>> larger user base. The dev list has very few participants. Less than
>> 1000 last I looked. Most users I talk to just want to know what they
>> are getting. Well-formed, clear communication is how the PMC can let
>> end users know that a new feature is one of three states:
>> 
>> 1. Beta
>> 2. Generally Available
>> 3. Default (where appropriate)
>> 
>> Yes! The work is just sorting out what each level means and then
>> codifying that in confluence. Then, we look at any features that are
>> under question, assign a level, and determine what it takes to go from
>> one state to another.
>> 
>> The CEPs need to reflect this change. What makes a Beta, GA, Default
>> for new feature X. It makes it clear for implementers and end users,
>> which is an important feature of project maturity.
>> 
>> Patrick
>> 
>> 
>> 
>> On Dec 10, 2024 at 5:46:38 AM, Aleksey Yeshchenko  wrote:
>> 
>> What we’ve done is we’ve overloa

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Patrick McFadin]

-1 on unstable. It's way too many words than are needed. Three is a
magic number and fits:

Preview
Beta
GA

As a matter of testing the process, any pending CEP should go though
this exercise so we can see how it will work.

PS
Got the actual numbers from Whimsy.
DEV - 1425 users
USER - 2650

This means that when features experience a state change, finding more
avenues to get the word out will be important.

On Tue, Dec 10, 2024 at 10:04 AM Benedict Elliott Smith
 wrote:
>
> As an aside, it would be nice to admit we basically revisit everything each 
> time it becomes relevant again, and for policy decisions like this (that 
> don’t need to be agreed in advance) we should try to legislate the minimum 
> necessary policy to proceed today, and leave future refinements for later 
> when the relevant context arises.
>
> On 10 Dec 2024, at 13:00, Benedict Elliott Smith  wrote:
>
> I agree with Aleksey that if we think something is broken, we shouldn’t use 
> euphemisms, and for this reason I don’t like unstable (this could for 
> instance simply mean API unstable). If we intend to never need this 
> descriptor, we should avoid bike-shedding and insert a “placeholder” for now 
> to be refined as and when we need it when we have the necessary future 
> context.
>
> i.e.
>
> preview -> beta -> [“has problems that will take time to resolve placeholder” 
> -> beta] -> GA
>
>
>
> On 10 Dec 2024, at 12:39, Josh McKenzie  wrote:
>
> +1 to this classification with one addition. I think we need to augment this 
> with formalization on what we do with features we don't recommend people use 
> (i.e. MV in their current incarnation). For something retroactively found to 
> be unstable, we could add an "Unstable" qualification for it, leaving us with:
>
> Unstable: Warnings on use, clearly communicated as to why, either on-track to 
> be fixed or removed from the codebase. No lingering for years in a fugue 
> state. We should target never needing this classification.
> Preview: Ready to be tried by end users but has caveats and most likely is 
> not api stable. Developer only documentation acceptable.
> Beta: Feature complete/API stable but has not had enough testing to be 
> considered rock solid. Developer and User documentation required.
> GA: Ready for use, no known issue, PMC is satisfied with the testing that has 
> been done
>
>
> To walk through how some of the flow might look to test the above:
>
> Simple case:
> - Preview -> Beta -> GA
>
> Late discovered defect case:
> - Preview -> Beta -> Unstable -> Beta -> GA
>
> Pathological worst-case (i.e. MV):
> - Preview -> Beta -> GA -> Unstable -> [Preview|Removed]
>
> On Tue, Dec 10, 2024, at 12:29 PM, Jeremiah Jordan wrote:
>
> I agree with Aleksey and Patrick.  We should define terminology and then 
> stick to it.  My preferred list would be:
>
> Preview - Ready to be tried by end users but has caveats and most likely is 
> not api stable.
> Beta - Feature complete/API stable but has not had enough testing to be 
> considered rock solid.
> GA - Ready for use, no known issue, PMC is satisfied with the testing that 
> has been done
>
>
> Whether or not something is enabled by default or the default implementation 
> is a separate access from the readiness.  Though if we are replacing an 
> existing thing with a new default I would hope we apply extra rigor to 
> allowing that to happen.
>
> -Jeremiah
>
> On Dec 10, 2024 at 11:15:37 AM, Patrick McFadin  wrote:
>
> I'm going to try to pull this back from the inevitable bikeshedding
> and airing of grievances that happen. Rewind all the way back to
> Josh's  original point, which is a defined process. Why I really love
> this being brought up is our maturing process of communicating to the
> larger user base. The dev list has very few participants. Less than
> 1000 last I looked. Most users I talk to just want to know what they
> are getting. Well-formed, clear communication is how the PMC can let
> end users know that a new feature is one of three states:
>
> 1. Beta
> 2. Generally Available
> 3. Default (where appropriate)
>
> Yes! The work is just sorting out what each level means and then
> codifying that in confluence. Then, we look at any features that are
> under question, assign a level, and determine what it takes to go from
> one state to another.
>
> The CEPs need to reflect this change. What makes a Beta, GA, Default
> for new feature X. It makes it clear for implementers and end users,
> which is an important feature of project maturity.
>
> Patrick
>
>
>
> On Dec 10, 2024 at 5:46:38 AM, Aleksey Yeshchenko  wrote:
>
> What we’ve done is we’ve overloaded the term ‘experimental’ to mean too many 
> related but different ideas. We need additional, more specific terminology to 
> disambiguate.
>
> 1. Labelling released features that were known to be unstable at release as 
> ‘experimental’  retroactively shouldn’t happen and AFAIK only happened once, 
> with MVs, and ‘experimental’ there was just a euphemism for ‘

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Benedict Elliott Smith]

As an aside, it would be nice to admit we basically revisit everything each 
time it becomes relevant again, and for policy decisions like this (that don’t 
need to be agreed in advance) we should try to legislate the minimum necessary 
policy to proceed today, and leave future refinements for later when the 
relevant context arises.

> On 10 Dec 2024, at 13:00, Benedict Elliott Smith  wrote:
> 
> I agree with Aleksey that if we think something is broken, we shouldn’t use 
> euphemisms, and for this reason I don’t like unstable (this could for 
> instance simply mean API unstable). If we intend to never need this 
> descriptor, we should avoid bike-shedding and insert a “placeholder” for now 
> to be refined as and when we need it when we have the necessary future 
> context.
> 
> i.e.
> 
> preview -> beta -> [“has problems that will take time to resolve placeholder” 
> -> beta] -> GA
> 
> 
> 
>> On 10 Dec 2024, at 12:39, Josh McKenzie  wrote:
>> 
>> +1 to this classification with one addition. I think we need to augment this 
>> with formalization on what we do with features we don't recommend people use 
>> (i.e. MV in their current incarnation). For something retroactively found to 
>> be unstable, we could add an "Unstable" qualification for it, leaving us 
>> with:
>> 
>> Unstable: Warnings on use, clearly communicated as to why, either on-track 
>> to be fixed or removed from the codebase. No lingering for years in a fugue 
>> state. We should target never needing this classification.
>> Preview: Ready to be tried by end users but has caveats and most likely is 
>> not api stable. Developer only documentation acceptable.
>> Beta: Feature complete/API stable but has not had enough testing to be 
>> considered rock solid. Developer and User documentation required.
>> GA: Ready for use, no known issue, PMC is satisfied with the testing that 
>> has been done
>> 
>> To walk through how some of the flow might look to test the above:
>> 
>> Simple case:
>> - Preview -> Beta -> GA
>> 
>> Late discovered defect case:
>> - Preview -> Beta -> Unstable -> Beta -> GA
>> 
>> Pathological worst-case (i.e. MV):
>> - Preview -> Beta -> GA -> Unstable -> [Preview|Removed]
>> 
>> On Tue, Dec 10, 2024, at 12:29 PM, Jeremiah Jordan wrote:
>>> I agree with Aleksey and Patrick.  We should define terminology and then 
>>> stick to it.  My preferred list would be:
>>> 
>>> Preview - Ready to be tried by end users but has caveats and most likely is 
>>> not api stable.
>>> Beta - Feature complete/API stable but has not had enough testing to be 
>>> considered rock solid.
>>> GA - Ready for use, no known issue, PMC is satisfied with the testing that 
>>> has been done
>>> 
>>> Whether or not something is enabled by default or the default 
>>> implementation is a separate access from the readiness.  Though if we are 
>>> replacing an existing thing with a new default I would hope we apply extra 
>>> rigor to allowing that to happen.
>>> 
>>> -Jeremiah
>>> 
>>> On Dec 10, 2024 at 11:15:37 AM, Patrick McFadin >> > wrote:
 I'm going to try to pull this back from the inevitable bikeshedding
 and airing of grievances that happen. Rewind all the way back to
 Josh's  original point, which is a defined process. Why I really love
 this being brought up is our maturing process of communicating to the
 larger user base. The dev list has very few participants. Less than
 1000 last I looked. Most users I talk to just want to know what they
 are getting. Well-formed, clear communication is how the PMC can let
 end users know that a new feature is one of three states:
 
 1. Beta
 2. Generally Available
 3. Default (where appropriate)
 
 Yes! The work is just sorting out what each level means and then
 codifying that in confluence. Then, we look at any features that are
 under question, assign a level, and determine what it takes to go from
 one state to another.
 
 The CEPs need to reflect this change. What makes a Beta, GA, Default
 for new feature X. It makes it clear for implementers and end users,
 which is an important feature of project maturity.
 
 Patrick
>>> 
>>> 
>>> On Dec 10, 2024 at 5:46:38 AM, Aleksey Yeshchenko >> > wrote:
 What we’ve done is we’ve overloaded the term ‘experimental’ to mean too 
 many related but different ideas. We need additional, more specific 
 terminology to disambiguate.
 
 1. Labelling released features that were known to be unstable at release 
 as ‘experimental’  retroactively shouldn’t happen and AFAIK only happened 
 once, with MVs, and ‘experimental’ there was just a euphemism for 
 ‘broken’. Our practices are more mature now, I like to think, that a 
 situation like this would not arise in the future - the bar for releasing 
 a completed marketable feature is higher. So the label ‘experimental’ 
 should not be a

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Benedict Elliott Smith]

I agree with Aleksey that if we think something is broken, we shouldn’t use 
euphemisms, and for this reason I don’t like unstable (this could for instance 
simply mean API unstable). If we intend to never need this descriptor, we 
should avoid bike-shedding and insert a “placeholder” for now to be refined as 
and when we need it when we have the necessary future context.

i.e.

preview -> beta -> [“has problems that will take time to resolve placeholder” 
-> beta] -> GA



> On 10 Dec 2024, at 12:39, Josh McKenzie  wrote:
> 
> +1 to this classification with one addition. I think we need to augment this 
> with formalization on what we do with features we don't recommend people use 
> (i.e. MV in their current incarnation). For something retroactively found to 
> be unstable, we could add an "Unstable" qualification for it, leaving us with:
> 
> Unstable: Warnings on use, clearly communicated as to why, either on-track to 
> be fixed or removed from the codebase. No lingering for years in a fugue 
> state. We should target never needing this classification.
> Preview: Ready to be tried by end users but has caveats and most likely is 
> not api stable. Developer only documentation acceptable.
> Beta: Feature complete/API stable but has not had enough testing to be 
> considered rock solid. Developer and User documentation required.
> GA: Ready for use, no known issue, PMC is satisfied with the testing that has 
> been done
> 
> To walk through how some of the flow might look to test the above:
> 
> Simple case:
> - Preview -> Beta -> GA
> 
> Late discovered defect case:
> - Preview -> Beta -> Unstable -> Beta -> GA
> 
> Pathological worst-case (i.e. MV):
> - Preview -> Beta -> GA -> Unstable -> [Preview|Removed]
> 
> On Tue, Dec 10, 2024, at 12:29 PM, Jeremiah Jordan wrote:
>> I agree with Aleksey and Patrick.  We should define terminology and then 
>> stick to it.  My preferred list would be:
>> 
>> Preview - Ready to be tried by end users but has caveats and most likely is 
>> not api stable.
>> Beta - Feature complete/API stable but has not had enough testing to be 
>> considered rock solid.
>> GA - Ready for use, no known issue, PMC is satisfied with the testing that 
>> has been done
>> 
>> Whether or not something is enabled by default or the default implementation 
>> is a separate access from the readiness.  Though if we are replacing an 
>> existing thing with a new default I would hope we apply extra rigor to 
>> allowing that to happen.
>> 
>> -Jeremiah
>> 
>> On Dec 10, 2024 at 11:15:37 AM, Patrick McFadin > > wrote:
>>> I'm going to try to pull this back from the inevitable bikeshedding
>>> and airing of grievances that happen. Rewind all the way back to
>>> Josh's  original point, which is a defined process. Why I really love
>>> this being brought up is our maturing process of communicating to the
>>> larger user base. The dev list has very few participants. Less than
>>> 1000 last I looked. Most users I talk to just want to know what they
>>> are getting. Well-formed, clear communication is how the PMC can let
>>> end users know that a new feature is one of three states:
>>> 
>>> 1. Beta
>>> 2. Generally Available
>>> 3. Default (where appropriate)
>>> 
>>> Yes! The work is just sorting out what each level means and then
>>> codifying that in confluence. Then, we look at any features that are
>>> under question, assign a level, and determine what it takes to go from
>>> one state to another.
>>> 
>>> The CEPs need to reflect this change. What makes a Beta, GA, Default
>>> for new feature X. It makes it clear for implementers and end users,
>>> which is an important feature of project maturity.
>>> 
>>> Patrick
>> 
>> 
>> On Dec 10, 2024 at 5:46:38 AM, Aleksey Yeshchenko > > wrote:
>>> What we’ve done is we’ve overloaded the term ‘experimental’ to mean too 
>>> many related but different ideas. We need additional, more specific 
>>> terminology to disambiguate.
>>> 
>>> 1. Labelling released features that were known to be unstable at release as 
>>> ‘experimental’  retroactively shouldn’t happen and AFAIK only happened 
>>> once, with MVs, and ‘experimental’ there was just a euphemism for ‘broken’. 
>>> Our practices are more mature now, I like to think, that a situation like 
>>> this would not arise in the future - the bar for releasing a completed 
>>> marketable feature is higher. So the label ‘experimental’ should not be 
>>> applied retroactively to anything.
>>> 
>>> 2. It’s possible that a released, once considered production-ready feature, 
>>> might be discovered to be deeply flawed after being released already. We 
>>> need to temporarily mark such a feature as ‘broken' or ‘flawed'. Not 
>>> experimental, and not even ‘unstable’. Make sure we emit a warning on its 
>>> use everywhere, and, if possible, make it opt-in in the next major, at the 
>>> very least, to prevent new uses of it. Announce on dev, add a note in 
>>> NEWS.txt

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Ekaterina Dimitrova]

What JD and Josh said resonates also with the thoughts I had.

On Tue, 10 Dec 2024 at 12:46, Caleb Rackliffe 
wrote:

> +1 to Josh's refinement of JD's proposal
>
> On Tue, Dec 10, 2024 at 11:42 AM Josh McKenzie 
> wrote:
>
>> +1 to this classification with one addition. I think we need to augment
>> this with formalization on what we do with features we don't recommend
>> people use (i.e. MV in their current incarnation). For something
>> retroactively found to be unstable, we could add an "Unstable"
>> qualification for it, leaving us with:
>>
>>
>>1. *Unstable: *Warnings on use, clearly communicated as to why,
>>either on-track to be fixed or removed from the codebase. No lingering for
>>years in a fugue state. *We should target never needing this
>>classification.*
>>2. *Preview: *Ready to be tried by end users but has caveats and most
>>likely is not api stable. Developer only documentation acceptable.
>>3. *Beta: *Feature complete/API stable but has not had enough testing
>>to be considered rock solid. Developer and User documentation required.
>>4. *GA: *Ready for use, no known issue, PMC is satisfied with the
>>testing that has been done
>>
>>
>> To walk through how some of the flow might look to test the above:
>>
>> *Simple case:*
>> *- *Preview -> Beta -> GA
>>
>> *Late discovered defect case:*
>> *- *Preview -> Beta -> Unstable -> Beta -> GA
>>
>> *Pathological worst-case (i.e. MV):*
>> - Preview -> Beta -> GA -> Unstable -> [Preview|Removed]
>>
>> On Tue, Dec 10, 2024, at 12:29 PM, Jeremiah Jordan wrote:
>>
>> I agree with Aleksey and Patrick.  We should define terminology and then
>> stick to it.  My preferred list would be:
>>
>>
>>1. Preview - Ready to be tried by end users but has caveats and most
>>likely is not api stable.
>>2. Beta - Feature complete/API stable but has not had enough testing
>>to be considered rock solid.
>>3. GA - Ready for use, no known issue, PMC is satisfied with the
>>testing that has been done
>>
>>
>> Whether or not something is enabled by default or the default
>> implementation is a separate access from the readiness.  Though if we are
>> replacing an existing thing with a new default I would hope we apply extra
>> rigor to allowing that to happen.
>>
>> -Jeremiah
>>
>> On Dec 10, 2024 at 11:15:37 AM, Patrick McFadin 
>> wrote:
>>
>> I'm going to try to pull this back from the inevitable bikeshedding
>> and airing of grievances that happen. Rewind all the way back to
>> Josh's  original point, which is a defined process. Why I really love
>> this being brought up is our maturing process of communicating to the
>> larger user base. The dev list has very few participants. Less than
>> 1000 last I looked. Most users I talk to just want to know what they
>> are getting. Well-formed, clear communication is how the PMC can let
>> end users know that a new feature is one of three states:
>>
>> 1. Beta
>> 2. Generally Available
>> 3. Default (where appropriate)
>>
>> Yes! The work is just sorting out what each level means and then
>> codifying that in confluence. Then, we look at any features that are
>> under question, assign a level, and determine what it takes to go from
>> one state to another.
>>
>> The CEPs need to reflect this change. What makes a Beta, GA, Default
>> for new feature X. It makes it clear for implementers and end users,
>> which is an important feature of project maturity.
>>
>> Patrick
>>
>>
>>
>> On Dec 10, 2024 at 5:46:38 AM, Aleksey Yeshchenko 
>> wrote:
>>
>> What we’ve done is we’ve overloaded the term ‘experimental’ to mean too
>> many related but different ideas. We need additional, more specific
>> terminology to disambiguate.
>>
>> 1. Labelling released features that were known to be unstable at release
>> as ‘experimental’  retroactively shouldn’t happen and AFAIK only happened
>> once, with MVs, and ‘experimental’ there was just a euphemism for ‘broken’.
>> Our practices are more mature now, I like to think, that a situation like
>> this would not arise in the future - the bar for releasing a completed
>> marketable feature is higher. So the label ‘experimental’ should not be
>> applied retroactively to anything.
>>
>> 2. It’s possible that a released, once considered production-ready
>> feature, might be discovered to be deeply flawed after being released
>> already. We need to temporarily mark such a feature as ‘broken' or
>> ‘flawed'. Not experimental, and not even ‘unstable’. Make sure we emit a
>> warning on its use everywhere, and, if possible, make it opt-in in the next
>> major, at the very least, to prevent new uses of it. Announce on dev, add a
>> note in NEWS.txt, etc. If the flaws are later addressed, remove the label.
>> Removing the feature itself might not be possible, but should be
>> considered, with heavy advanced telegraphing to the community.
>>
>> 3. There is probably room for genuine use of ‘experimental’ as a feature
>> label. For opt-in features th

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Caleb Rackliffe]

+1 to Josh's refinement of JD's proposal

On Tue, Dec 10, 2024 at 11:42 AM Josh McKenzie  wrote:

> +1 to this classification with one addition. I think we need to augment
> this with formalization on what we do with features we don't recommend
> people use (i.e. MV in their current incarnation). For something
> retroactively found to be unstable, we could add an "Unstable"
> qualification for it, leaving us with:
>
>
>1. *Unstable: *Warnings on use, clearly communicated as to why, either
>on-track to be fixed or removed from the codebase. No lingering for years
>in a fugue state. *We should target never needing this classification.*
>2. *Preview: *Ready to be tried by end users but has caveats and most
>likely is not api stable. Developer only documentation acceptable.
>3. *Beta: *Feature complete/API stable but has not had enough testing
>to be considered rock solid. Developer and User documentation required.
>4. *GA: *Ready for use, no known issue, PMC is satisfied with the
>testing that has been done
>
>
> To walk through how some of the flow might look to test the above:
>
> *Simple case:*
> *- *Preview -> Beta -> GA
>
> *Late discovered defect case:*
> *- *Preview -> Beta -> Unstable -> Beta -> GA
>
> *Pathological worst-case (i.e. MV):*
> - Preview -> Beta -> GA -> Unstable -> [Preview|Removed]
>
> On Tue, Dec 10, 2024, at 12:29 PM, Jeremiah Jordan wrote:
>
> I agree with Aleksey and Patrick.  We should define terminology and then
> stick to it.  My preferred list would be:
>
>
>1. Preview - Ready to be tried by end users but has caveats and most
>likely is not api stable.
>2. Beta - Feature complete/API stable but has not had enough testing
>to be considered rock solid.
>3. GA - Ready for use, no known issue, PMC is satisfied with the
>testing that has been done
>
>
> Whether or not something is enabled by default or the default
> implementation is a separate access from the readiness.  Though if we are
> replacing an existing thing with a new default I would hope we apply extra
> rigor to allowing that to happen.
>
> -Jeremiah
>
> On Dec 10, 2024 at 11:15:37 AM, Patrick McFadin 
> wrote:
>
> I'm going to try to pull this back from the inevitable bikeshedding
> and airing of grievances that happen. Rewind all the way back to
> Josh's  original point, which is a defined process. Why I really love
> this being brought up is our maturing process of communicating to the
> larger user base. The dev list has very few participants. Less than
> 1000 last I looked. Most users I talk to just want to know what they
> are getting. Well-formed, clear communication is how the PMC can let
> end users know that a new feature is one of three states:
>
> 1. Beta
> 2. Generally Available
> 3. Default (where appropriate)
>
> Yes! The work is just sorting out what each level means and then
> codifying that in confluence. Then, we look at any features that are
> under question, assign a level, and determine what it takes to go from
> one state to another.
>
> The CEPs need to reflect this change. What makes a Beta, GA, Default
> for new feature X. It makes it clear for implementers and end users,
> which is an important feature of project maturity.
>
> Patrick
>
>
>
> On Dec 10, 2024 at 5:46:38 AM, Aleksey Yeshchenko 
> wrote:
>
> What we’ve done is we’ve overloaded the term ‘experimental’ to mean too
> many related but different ideas. We need additional, more specific
> terminology to disambiguate.
>
> 1. Labelling released features that were known to be unstable at release
> as ‘experimental’  retroactively shouldn’t happen and AFAIK only happened
> once, with MVs, and ‘experimental’ there was just a euphemism for ‘broken’.
> Our practices are more mature now, I like to think, that a situation like
> this would not arise in the future - the bar for releasing a completed
> marketable feature is higher. So the label ‘experimental’ should not be
> applied retroactively to anything.
>
> 2. It’s possible that a released, once considered production-ready
> feature, might be discovered to be deeply flawed after being released
> already. We need to temporarily mark such a feature as ‘broken' or
> ‘flawed'. Not experimental, and not even ‘unstable’. Make sure we emit a
> warning on its use everywhere, and, if possible, make it opt-in in the next
> major, at the very least, to prevent new uses of it. Announce on dev, add a
> note in NEWS.txt, etc. If the flaws are later addressed, remove the label.
> Removing the feature itself might not be possible, but should be
> considered, with heavy advanced telegraphing to the community.
>
> 3. There is probably room for genuine use of ‘experimental’ as a feature
> label. For opt-in features that we commit with an understanding that they
> might not make it at all. Unstable API is implied here, but a feature can
> also have an unstable API without being experimental - so ‘experimental'
> doesn’t equal to ‘api-unstable’. These s

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Josh McKenzie]

+1 to this classification with one addition. I think we need to augment this 
with formalization on what we do with features we don't recommend people use 
(i.e. MV in their current incarnation). For something retroactively found to be 
unstable, we could add an "Unstable" qualification for it, leaving us with:

 1. *Unstable: *Warnings on use, clearly communicated as to why, either 
on-track to be fixed or removed from the codebase. No lingering for years in a 
fugue state. *We should target never needing this classification.*
 2. *Preview: *Ready to be tried by end users but has caveats and most likely 
is not api stable. Developer only documentation acceptable.
 3. *Beta: *Feature complete/API stable but has not had enough testing to be 
considered rock solid. Developer and User documentation required.
 4. *GA: *Ready for use, no known issue, PMC is satisfied with the testing that 
has been done

To walk through how some of the flow might look to test the above:

*Simple case:*
*- *Preview -> Beta -> GA

*Late discovered defect case:*
*- *Preview -> Beta -> Unstable -> Beta -> GA

*Pathological worst-case (i.e. MV):*
- Preview -> Beta -> GA -> Unstable -> [Preview|Removed]

On Tue, Dec 10, 2024, at 12:29 PM, Jeremiah Jordan wrote:
> I agree with Aleksey and Patrick.  We should define terminology and then 
> stick to it.  My preferred list would be:
> 
>  1. Preview - Ready to be tried by end users but has caveats and most likely 
> is not api stable.
>  2. Beta - Feature complete/API stable but has not had enough testing to be 
> considered rock solid.
>  3. GA - Ready for use, no known issue, PMC is satisfied with the testing 
> that has been done
> 
> Whether or not something is enabled by default or the default implementation 
> is a separate access from the readiness.  Though if we are replacing an 
> existing thing with a new default I would hope we apply extra rigor to 
> allowing that to happen.
> 
> -Jeremiah
> 
> On Dec 10, 2024 at 11:15:37 AM, Patrick McFadin  wrote:
>> I'm going to try to pull this back from the inevitable bikeshedding
>> and airing of grievances that happen. Rewind all the way back to
>> Josh's  original point, which is a defined process. Why I really love
>> this being brought up is our maturing process of communicating to the
>> larger user base. The dev list has very few participants. Less than
>> 1000 last I looked. Most users I talk to just want to know what they
>> are getting. Well-formed, clear communication is how the PMC can let
>> end users know that a new feature is one of three states:
>> 
>> 1. Beta
>> 2. Generally Available
>> 3. Default (where appropriate)
>> 
>> Yes! The work is just sorting out what each level means and then
>> codifying that in confluence. Then, we look at any features that are
>> under question, assign a level, and determine what it takes to go from
>> one state to another.
>> 
>> The CEPs need to reflect this change. What makes a Beta, GA, Default
>> for new feature X. It makes it clear for implementers and end users,
>> which is an important feature of project maturity.
>> 
>> Patrick
> 
> 
> On Dec 10, 2024 at 5:46:38 AM, Aleksey Yeshchenko  wrote:
>> What we’ve done is we’ve overloaded the term ‘experimental’ to mean too many 
>> related but different ideas. We need additional, more specific terminology 
>> to disambiguate.
>> 
>> 1. Labelling released features that were known to be unstable at release as 
>> ‘experimental’  retroactively shouldn’t happen and AFAIK only happened once, 
>> with MVs, and ‘experimental’ there was just a euphemism for ‘broken’. Our 
>> practices are more mature now, I like to think, that a situation like this 
>> would not arise in the future - the bar for releasing a completed marketable 
>> feature is higher. So the label ‘experimental’ should not be applied 
>> retroactively to anything.
>> 
>> 2. It’s possible that a released, once considered production-ready feature, 
>> might be discovered to be deeply flawed after being released already. We 
>> need to temporarily mark such a feature as ‘broken' or ‘flawed'. Not 
>> experimental, and not even ‘unstable’. Make sure we emit a warning on its 
>> use everywhere, and, if possible, make it opt-in in the next major, at the 
>> very least, to prevent new uses of it. Announce on dev, add a note in 
>> NEWS.txt, etc. If the flaws are later addressed, remove the label. Removing 
>> the feature itself might not be possible, but should be considered, with 
>> heavy advanced telegraphing to the community.
>> 
>> 3. There is probably room for genuine use of ‘experimental’ as a feature 
>> label. For opt-in features that we commit with an understanding that they 
>> might not make it at all. Unstable API is implied here, but a feature can 
>> also have an unstable API without being experimental - so ‘experimental' 
>> doesn’t equal to ‘api-unstable’. These should not be relied on by any 
>> production code, they would be heavily gated by unambiguous configuration 

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Benedict Elliott Smith]

Yep, I agree with all of this line of discussion. +1 any reasonable variation 
of Aleksey, Patrick and Jeremiah’s proposals.

> On 10 Dec 2024, at 12:29, Jeremiah Jordan  wrote:
> 
> I agree with Aleksey and Patrick.  We should define terminology and then 
> stick to it.  My preferred list would be:
> 
> Preview - Ready to be tried by end users but has caveats and most likely is 
> not api stable.
> Beta - Feature complete/API stable but has not had enough testing to be 
> considered rock solid.
> GA - Ready for use, no known issue, PMC is satisfied with the testing that 
> has been done
> 
> Whether or not something is enabled by default or the default implementation 
> is a separate access from the readiness.  Though if we are replacing an 
> existing thing with a new default I would hope we apply extra rigor to 
> allowing that to happen.
> 
> -Jeremiah
> 
> On Dec 10, 2024 at 11:15:37 AM, Patrick McFadin  > wrote:
>> I'm going to try to pull this back from the inevitable bikeshedding
>> and airing of grievances that happen. Rewind all the way back to
>> Josh's  original point, which is a defined process. Why I really love
>> this being brought up is our maturing process of communicating to the
>> larger user base. The dev list has very few participants. Less than
>> 1000 last I looked. Most users I talk to just want to know what they
>> are getting. Well-formed, clear communication is how the PMC can let
>> end users know that a new feature is one of three states:
>> 
>> 1. Beta
>> 2. Generally Available
>> 3. Default (where appropriate)
>> 
>> Yes! The work is just sorting out what each level means and then
>> codifying that in confluence. Then, we look at any features that are
>> under question, assign a level, and determine what it takes to go from
>> one state to another.
>> 
>> The CEPs need to reflect this change. What makes a Beta, GA, Default
>> for new feature X. It makes it clear for implementers and end users,
>> which is an important feature of project maturity.
>> 
>> Patrick
> 
> 
> On Dec 10, 2024 at 5:46:38 AM, Aleksey Yeshchenko  > wrote:
>> What we’ve done is we’ve overloaded the term ‘experimental’ to mean too many 
>> related but different ideas. We need additional, more specific terminology 
>> to disambiguate.
>> 
>> 1. Labelling released features that were known to be unstable at release as 
>> ‘experimental’  retroactively shouldn’t happen and AFAIK only happened once, 
>> with MVs, and ‘experimental’ there was just a euphemism for ‘broken’. Our 
>> practices are more mature now, I like to think, that a situation like this 
>> would not arise in the future - the bar for releasing a completed marketable 
>> feature is higher. So the label ‘experimental’ should not be applied 
>> retroactively to anything.
>> 
>> 2. It’s possible that a released, once considered production-ready feature, 
>> might be discovered to be deeply flawed after being released already. We 
>> need to temporarily mark such a feature as ‘broken' or ‘flawed'. Not 
>> experimental, and not even ‘unstable’. Make sure we emit a warning on its 
>> use everywhere, and, if possible, make it opt-in in the next major, at the 
>> very least, to prevent new uses of it. Announce on dev, add a note in 
>> NEWS.txt, etc. If the flaws are later addressed, remove the label. Removing 
>> the feature itself might not be possible, but should be considered, with 
>> heavy advanced telegraphing to the community.
>> 
>> 3. There is probably room for genuine use of ‘experimental’ as a feature 
>> label. For opt-in features that we commit with an understanding that they 
>> might not make it at all. Unstable API is implied here, but a feature can 
>> also have an unstable API without being experimental - so ‘experimental' 
>> doesn’t equal to ‘api-unstable’. These should not be relied on by any 
>> production code, they would be heavily gated by unambiguous configuration 
>> flags, disabled by default, allowed to be removed or changed in any version 
>> including a minor one.
>> 
>> 4. New features without known flaws, intended to be production-ready and 
>> marketable eventually, that we may want to gain some real-world confidence 
>> with before we are happy to market or make default. UCS, for example, which 
>> seems to be in heavy use in Astra and doesn’t have any known open issues 
>> (AFAIK). It’s not experimental, it’s not unstable, it’s not ‘alpha’ or 
>> ‘beta’, it just hasn't been widely enough used to have gained a lot of 
>> confidence. It’s just new. I’m not sure what label even applies here. It’s 
>> just a regular feature that happens to be new, doesn’t need a label, just 
>> needs to see some widespread use before we can make it a default. No other 
>> limitation on its use.
>> 
>> 5. Early-integrated, not-yet fully-completed features that are NOT 
>> experimental in nature. Isolated, gated behind deep configuration flags. 
>> Have a CEP behind them, we trust that

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Caleb Rackliffe]

Let's say we went with the preview -> beta -> GA option. Does something
like SASI stay in "experimental" while MV, transient replication, etc. move
to "preview"?

On Tue, Dec 10, 2024 at 11:30 AM Jeremiah Jordan 
wrote:

> I agree with Aleksey and Patrick.  We should define terminology and then
> stick to it.  My preferred list would be:
>
>
>1. Preview - Ready to be tried by end users but has caveats and most
>likely is not api stable.
>2. Beta - Feature complete/API stable but has not had enough testing
>to be considered rock solid.
>3. GA - Ready for use, no known issue, PMC is satisfied with the
>testing that has been done
>
>
> Whether or not something is enabled by default or the default
> implementation is a separate access from the readiness.  Though if we are
> replacing an existing thing with a new default I would hope we apply extra
> rigor to allowing that to happen.
>
> -Jeremiah
>
> On Dec 10, 2024 at 11:15:37 AM, Patrick McFadin 
> wrote:
>
>> I'm going to try to pull this back from the inevitable bikeshedding
>> and airing of grievances that happen. Rewind all the way back to
>> Josh's  original point, which is a defined process. Why I really love
>> this being brought up is our maturing process of communicating to the
>> larger user base. The dev list has very few participants. Less than
>> 1000 last I looked. Most users I talk to just want to know what they
>> are getting. Well-formed, clear communication is how the PMC can let
>> end users know that a new feature is one of three states:
>>
>> 1. Beta
>> 2. Generally Available
>> 3. Default (where appropriate)
>>
>> Yes! The work is just sorting out what each level means and then
>> codifying that in confluence. Then, we look at any features that are
>> under question, assign a level, and determine what it takes to go from
>> one state to another.
>>
>> The CEPs need to reflect this change. What makes a Beta, GA, Default
>> for new feature X. It makes it clear for implementers and end users,
>> which is an important feature of project maturity.
>>
>> Patrick
>>
>
>
> On Dec 10, 2024 at 5:46:38 AM, Aleksey Yeshchenko 
> wrote:
>
>> What we’ve done is we’ve overloaded the term ‘experimental’ to mean too
>> many related but different ideas. We need additional, more specific
>> terminology to disambiguate.
>>
>> 1. Labelling released features that were known to be unstable at release
>> as ‘experimental’  retroactively shouldn’t happen and AFAIK only happened
>> once, with MVs, and ‘experimental’ there was just a euphemism for ‘broken’.
>> Our practices are more mature now, I like to think, that a situation like
>> this would not arise in the future - the bar for releasing a completed
>> marketable feature is higher. So the label ‘experimental’ should not be
>> applied retroactively to anything.
>>
>> 2. It’s possible that a released, once considered production-ready
>> feature, might be discovered to be deeply flawed after being released
>> already. We need to temporarily mark such a feature as ‘broken' or
>> ‘flawed'. Not experimental, and not even ‘unstable’. Make sure we emit a
>> warning on its use everywhere, and, if possible, make it opt-in in the next
>> major, at the very least, to prevent new uses of it. Announce on dev, add a
>> note in NEWS.txt, etc. If the flaws are later addressed, remove the label.
>> Removing the feature itself might not be possible, but should be
>> considered, with heavy advanced telegraphing to the community.
>>
>> 3. There is probably room for genuine use of ‘experimental’ as a feature
>> label. For opt-in features that we commit with an understanding that they
>> might not make it at all. Unstable API is implied here, but a feature can
>> also have an unstable API without being experimental - so ‘experimental'
>> doesn’t equal to ‘api-unstable’. These should not be relied on by any
>> production code, they would be heavily gated by unambiguous configuration
>> flags, disabled by default, allowed to be removed or changed in any version
>> including a minor one.
>>
>> 4. New features without known flaws, intended to be production-ready and
>> marketable eventually, that we may want to gain some real-world confidence
>> with before we are happy to market or make default. UCS, for example, which
>> seems to be in heavy use in Astra and doesn’t have any known open issues
>> (AFAIK). It’s not experimental, it’s not unstable, it’s not ‘alpha’ or
>> ‘beta’, it just hasn't been widely enough used to have gained a lot of
>> confidence. It’s just new. I’m not sure what label even applies here. It’s
>> just a regular feature that happens to be new, doesn’t need a label, just
>> needs to see some widespread use before we can make it a default. No other
>> limitation on its use.
>>
>> 5. Early-integrated, not-yet fully-completed features that are NOT
>> experimental in nature. Isolated, gated behind deep configuration flags.
>> Have a CEP behind them, we trust that they will be eventually comple

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Jeremiah Jordan]

 I agree with Aleksey and Patrick.  We should define terminology and then
stick to it.  My preferred list would be:


   1. Preview - Ready to be tried by end users but has caveats and most
   likely is not api stable.
   2. Beta - Feature complete/API stable but has not had enough testing to
   be considered rock solid.
   3. GA - Ready for use, no known issue, PMC is satisfied with the testing
   that has been done


Whether or not something is enabled by default or the default
implementation is a separate access from the readiness.  Though if we are
replacing an existing thing with a new default I would hope we apply extra
rigor to allowing that to happen.

-Jeremiah

On Dec 10, 2024 at 11:15:37 AM, Patrick McFadin  wrote:

> I'm going to try to pull this back from the inevitable bikeshedding
> and airing of grievances that happen. Rewind all the way back to
> Josh's  original point, which is a defined process. Why I really love
> this being brought up is our maturing process of communicating to the
> larger user base. The dev list has very few participants. Less than
> 1000 last I looked. Most users I talk to just want to know what they
> are getting. Well-formed, clear communication is how the PMC can let
> end users know that a new feature is one of three states:
>
> 1. Beta
> 2. Generally Available
> 3. Default (where appropriate)
>
> Yes! The work is just sorting out what each level means and then
> codifying that in confluence. Then, we look at any features that are
> under question, assign a level, and determine what it takes to go from
> one state to another.
>
> The CEPs need to reflect this change. What makes a Beta, GA, Default
> for new feature X. It makes it clear for implementers and end users,
> which is an important feature of project maturity.
>
> Patrick
>


On Dec 10, 2024 at 5:46:38 AM, Aleksey Yeshchenko  wrote:

> What we’ve done is we’ve overloaded the term ‘experimental’ to mean too
> many related but different ideas. We need additional, more specific
> terminology to disambiguate.
>
> 1. Labelling released features that were known to be unstable at release
> as ‘experimental’  retroactively shouldn’t happen and AFAIK only happened
> once, with MVs, and ‘experimental’ there was just a euphemism for ‘broken’.
> Our practices are more mature now, I like to think, that a situation like
> this would not arise in the future - the bar for releasing a completed
> marketable feature is higher. So the label ‘experimental’ should not be
> applied retroactively to anything.
>
> 2. It’s possible that a released, once considered production-ready
> feature, might be discovered to be deeply flawed after being released
> already. We need to temporarily mark such a feature as ‘broken' or
> ‘flawed'. Not experimental, and not even ‘unstable’. Make sure we emit a
> warning on its use everywhere, and, if possible, make it opt-in in the next
> major, at the very least, to prevent new uses of it. Announce on dev, add a
> note in NEWS.txt, etc. If the flaws are later addressed, remove the label.
> Removing the feature itself might not be possible, but should be
> considered, with heavy advanced telegraphing to the community.
>
> 3. There is probably room for genuine use of ‘experimental’ as a feature
> label. For opt-in features that we commit with an understanding that they
> might not make it at all. Unstable API is implied here, but a feature can
> also have an unstable API without being experimental - so ‘experimental'
> doesn’t equal to ‘api-unstable’. These should not be relied on by any
> production code, they would be heavily gated by unambiguous configuration
> flags, disabled by default, allowed to be removed or changed in any version
> including a minor one.
>
> 4. New features without known flaws, intended to be production-ready and
> marketable eventually, that we may want to gain some real-world confidence
> with before we are happy to market or make default. UCS, for example, which
> seems to be in heavy use in Astra and doesn’t have any known open issues
> (AFAIK). It’s not experimental, it’s not unstable, it’s not ‘alpha’ or
> ‘beta’, it just hasn't been widely enough used to have gained a lot of
> confidence. It’s just new. I’m not sure what label even applies here. It’s
> just a regular feature that happens to be new, doesn’t need a label, just
> needs to see some widespread use before we can make it a default. No other
> limitation on its use.
>
> 5. Early-integrated, not-yet fully-completed features that are NOT
> experimental in nature. Isolated, gated behind deep configuration flags.
> Have a CEP behind them, we trust that they will be eventually completed,
> but for pragmatic reasons it just made sense to commit them at an earlier
> stage. ‘Preview’, ‘alpha’, ‘beta’ are labels that could apply here
> depending on current feature readiness status. API-instability is implied.
> Once finished they just become a regular new feature, no flag needed, no
> heavy config gating needed.
>

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Patrick McFadin]

I'm going to try to pull this back from the inevitable bikeshedding
and airing of grievances that happen. Rewind all the way back to
Josh's  original point, which is a defined process. Why I really love
this being brought up is our maturing process of communicating to the
larger user base. The dev list has very few participants. Less than
1000 last I looked. Most users I talk to just want to know what they
are getting. Well-formed, clear communication is how the PMC can let
end users know that a new feature is one of three states:

1. Beta
2. Generally Available
3. Default (where appropriate)

Yes! The work is just sorting out what each level means and then
codifying that in confluence. Then, we look at any features that are
under question, assign a level, and determine what it takes to go from
one state to another.

The CEPs need to reflect this change. What makes a Beta, GA, Default
for new feature X. It makes it clear for implementers and end users,
which is an important feature of project maturity.

Patrick

On Tue, Dec 10, 2024 at 7:40 AM Caleb Rackliffe
 wrote:
>
> I misspoke earlier about the feature gap. SAI supports queries legacy 2i does 
> not, like numeric range queries.
>
> On Dec 10, 2024, at 9:29 AM, Caleb Rackliffe  wrote:
>
> 
> I think my point here is that the hidden table 2i implementation has known 
> correctness/availability/operational/resource usage issues whether it has a 
> theoretical niche use-case or not from a query performance perspective.
>
> To Štefan’s question, yes, more or less. I’d like to at least see some 
> success in production for the cases it was primarily designed for. That might 
> not be enough to make it the default if it needs to perform better than the 
> (broken) legacy 2i in global query situations. SAI is currently bad by design 
> for global queries across 1000s of SSTables (LCS), so it would either need to 
> be used in conjunction with a compaction strategy that aggressively limits 
> the number of live SSTables, otherwise modified to handle that case better, 
> or simply made the default w/ the guardrails it already has around these 
> things becuase there simply isn’t a usable alternative.
>
> On Dec 10, 2024, at 9:13 AM, Benedict Elliott Smith  
> wrote:
>
> 
>
> There is no reason it should ever be more capable than SAI for any 
> partition/token-restricted query use-case, and I don't really see how there's 
> any short-term path for any local 2i implementation in C* to be efficient for 
> anything else
>
>
> While I am not personally aware of much evidence presented that SAI performs 
> better than 2i for the partition-restricted case, I do believe it is 
> theoretically likely to. But any deprecation discussion should include 
> evidence of this as a preamble.
>
> However, there are users that want queries not restricted by partition or 
> token, and SAI is unlikely to serve these use cases as well. Yes, neither 
> perform this use case well, but I cannot support deprecating a feature when 
> its replacement is very likely inferior for some workloads. Since it is hard 
> to prove that nobody is using 2i this way (and I recall from the distant past 
> that such users were known to exist), we need instead to prove SAI can serve 
> these workloads acceptably before we declare it a suitable replacement.
>
> I think there exists a near future world where we can offer proper global 
> secondary indexes, at which point it would be acceptable to deprecate 2i and 
> recommend users switch to either global secondary indexes or SAI. Until then, 
> I cannot see a good argument for it if we want to be considered a stable and 
> mature product.
>
>
> On 10 Dec 2024, at 09:28, Caleb Rackliffe  wrote:
>
> > I’m not convinced SAI has demonstrated a practical or theoretical 
> > capability to fully replace secondary indexes anyway. So it would be very 
> > premature to mark them deprecated.
>
> > If 2i indexes are to be marked as deprecated and SAI is beta, then what is 
> > actually the index implementation we stand behind in the production? It is 
> > like we are "abandoning" the former but the latter is not bullet-proof yet.
>
> The table-based 2i implementation has never been safe to use, and I don't 
> think it ever will be, however we label it. (ex. CASSANDRA-18656, it's 
> on-disk bloat, post-streaming rebuilds, etc.) There is no reason it should 
> ever be more capable than SAI for any partition/token-restricted query 
> use-case, and I don't really see how there's any short-term path for any 
> local 2i implementation in C* to be efficient for anything else. There are 
> presently no feature gaps on the query side.
>
> Anyway, there are still a lot of things we can improve about SAI (and things 
> that already exist and are just waiting in the DS public fork)...I'm just not 
> sure what reasonable use case the old 2i will be able to serve better.
>
> On Tue, Dec 10, 2024 at 5:41 AM Benedict  wrote:
>>
>> I’m not convinced SAI has demonstrated a practica

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Caleb Rackliffe]

It might be a fun experiment to retrofit the Harry tests we're currently
using (once Harry 2.0 lands in trunk from cep-15-accord) to fuzz SAI and
point them at legacy 2i (i.e. the subset of query types legacy 2i supports)
and see if we find anything interesting, but I don't even know if that
rises above something like CASSANDRA-19007
 on the backlog of
things around indexes/filtering I would fix...

On Tue, Dec 10, 2024 at 10:35 AM Benedict Elliott Smith 
wrote:

> I agree with Aleksey on how we should approach feature flags, and if we
> think 2i simply *don’t work* we should make that determination and mark
> them *broken* not *deprecated*.
>
> The only bug mentioned so far is 18656, which doesn’t clearly argue that
> the behaviour is *incorrect* rather than just undesired. The only
> breaking scenario I can think of is if we complete a bootstrap before the
> index build is complete. I am not sure if this is possible, but if it is we
> should probably fix that, and in the meantime perhaps document the flaw and
> describe work arounds (such as repairing after stopping a replica to be
> replaced). This isn’t a “remove the feature” level bug though, given my
> current understanding of it. If anything, it would be much more work than
> just fixing the bug.
>
> If there’s a longer litany of breaking behaviours, let’s enumerate them
> and consider marking the feature as unsafe.
>
> On 10 Dec 2024, at 10:29, Caleb Rackliffe 
> wrote:
>
> I think my point here is that the hidden table 2i implementation has known
> correctness/availability/operational/resource usage issues whether it has a
> theoretical niche use-case or not from a query performance perspective.
>
> To Štefan’s question, yes, more or less. I’d like to at least see some
> success in production for the cases it was primarily designed for. That
> might not be enough to make it the default if it needs to perform better
> than the (broken) legacy 2i in global query situations. SAI is currently
> bad by design for global queries across 1000s of SSTables (LCS), so it
> would either need to be used in conjunction with a compaction strategy that
> aggressively limits the number of live SSTables, otherwise modified to
> handle that case better, or simply made the default w/ the guardrails it
> already has around these things becuase there simply isn’t a usable
> alternative.
>
> On Dec 10, 2024, at 9:13 AM, Benedict Elliott Smith 
> wrote:
>
> 
>
> There is no reason it should ever be more capable than SAI for any
> partition/token-restricted query use-case, and I don't really see how
> there's any short-term path for any local 2i implementation in C* to be
> efficient for anything else
>
>
> While I am not personally aware of much evidence presented that SAI
> performs better than 2i for the partition-restricted case, I do believe it
> is theoretically likely to. But any deprecation discussion should include
> evidence of this as a preamble.
>
> However, there are users that want queries *not* restricted by partition
> or token, and SAI is unlikely to serve these use cases as well. Yes,
> neither perform this use case *well*, but I cannot support deprecating a
> feature when its replacement is very likely inferior for some workloads.
> Since it is hard to prove that nobody is using 2i this way (and I recall
> from the distant past that such users were known to exist), we need instead
> to prove SAI can serve these workloads acceptably before we declare it a
> suitable replacement.
>
> I think there exists a near future world where we can offer proper
> *global* secondary indexes, at which point it would be acceptable to
> deprecate 2i and recommend users switch to either global secondary indexes
> or SAI. Until then, I cannot see a good argument for it if we want to be
> considered a stable and mature product.
>
>
> On 10 Dec 2024, at 09:28, Caleb Rackliffe 
> wrote:
>
> > I’m not convinced SAI has demonstrated a practical or theoretical
> capability to fully replace secondary indexes anyway. So it would be very
> premature to mark them deprecated.
>
> > If 2i indexes are to be marked as deprecated and SAI is beta, then what
> is actually the index implementation we stand behind in the production? It
> is like we are "abandoning" the former but the latter is not bullet-proof
> yet.
>
> The table-based 2i implementation has never been safe to use, and I don't
> think it ever will be, however we label it. (ex. CASSANDRA-18656, it's
> on-disk bloat, post-streaming rebuilds, etc.) There is no reason it should
> ever be more capable than SAI for any partition/token-restricted query
> use-case, and I don't really see how there's any short-term path for any
> local 2i implementation in C* to be efficient for anything else. There are
> presently no feature gaps on the query side.
>
> Anyway, there are still a lot of things we can improve about SAI (and
> things that already exist and are just waiting in the D

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Benedict Elliott Smith]

I agree with Aleksey on how we should approach feature flags, and if we think 
2i simply don’t work we should make that determination and mark them broken not 
deprecated.

The only bug mentioned so far is 18656, which doesn’t clearly argue that the 
behaviour is incorrect rather than just undesired. The only breaking scenario I 
can think of is if we complete a bootstrap before the index build is complete. 
I am not sure if this is possible, but if it is we should probably fix that, 
and in the meantime perhaps document the flaw and describe work arounds (such 
as repairing after stopping a replica to be replaced). This isn’t a “remove the 
feature” level bug though, given my current understanding of it. If anything, 
it would be much more work than just fixing the bug.

If there’s a longer litany of breaking behaviours, let’s enumerate them and 
consider marking the feature as unsafe.

> On 10 Dec 2024, at 10:29, Caleb Rackliffe  wrote:
> 
> I think my point here is that the hidden table 2i implementation has known 
> correctness/availability/operational/resource usage issues whether it has a 
> theoretical niche use-case or not from a query performance perspective.
> 
> To Štefan’s question, yes, more or less. I’d like to at least see some 
> success in production for the cases it was primarily designed for. That might 
> not be enough to make it the default if it needs to perform better than the 
> (broken) legacy 2i in global query situations. SAI is currently bad by design 
> for global queries across 1000s of SSTables (LCS), so it would either need to 
> be used in conjunction with a compaction strategy that aggressively limits 
> the number of live SSTables, otherwise modified to handle that case better, 
> or simply made the default w/ the guardrails it already has around these 
> things becuase there simply isn’t a usable alternative.
> 
>> On Dec 10, 2024, at 9:13 AM, Benedict Elliott Smith  
>> wrote:
>> 
>> 
>>> There is no reason it should ever be more capable than SAI for any 
>>> partition/token-restricted query use-case, and I don't really see how 
>>> there's any short-term path for any local 2i implementation in C* to be 
>>> efficient for anything else
>> 
>> While I am not personally aware of much evidence presented that SAI performs 
>> better than 2i for the partition-restricted case, I do believe it is 
>> theoretically likely to. But any deprecation discussion should include 
>> evidence of this as a preamble.
>> 
>> However, there are users that want queries not restricted by partition or 
>> token, and SAI is unlikely to serve these use cases as well. Yes, neither 
>> perform this use case well, but I cannot support deprecating a feature when 
>> its replacement is very likely inferior for some workloads. Since it is hard 
>> to prove that nobody is using 2i this way (and I recall from the distant 
>> past that such users were known to exist), we need instead to prove SAI can 
>> serve these workloads acceptably before we declare it a suitable replacement.
>> 
>> I think there exists a near future world where we can offer proper global 
>> secondary indexes, at which point it would be acceptable to deprecate 2i and 
>> recommend users switch to either global secondary indexes or SAI. Until 
>> then, I cannot see a good argument for it if we want to be considered a 
>> stable and mature product.
>> 
>> 
>>> On 10 Dec 2024, at 09:28, Caleb Rackliffe  wrote:
>>> 
>>> > I’m not convinced SAI has demonstrated a practical or theoretical 
>>> > capability to fully replace secondary indexes anyway. So it would be very 
>>> > premature to mark them deprecated.
>>> 
>>> > If 2i indexes are to be marked as deprecated and SAI is beta, then what 
>>> > is actually the index implementation we stand behind in the production? 
>>> > It is like we are "abandoning" the former but the latter is not 
>>> > bullet-proof yet.
>>> 
>>> The table-based 2i implementation has never been safe to use, and I don't 
>>> think it ever will be, however we label it. (ex. CASSANDRA-18656, it's 
>>> on-disk bloat, post-streaming rebuilds, etc.) There is no reason it should 
>>> ever be more capable than SAI for any partition/token-restricted query 
>>> use-case, and I don't really see how there's any short-term path for any 
>>> local 2i implementation in C* to be efficient for anything else. There are 
>>> presently no feature gaps on the query side.
>>> 
>>> Anyway, there are still a lot of things we can improve about SAI (and 
>>> things that already exist and are just waiting in the DS public fork)...I'm 
>>> just not sure what reasonable use case the old 2i will be able to serve 
>>> better.
>>> 
>>> On Tue, Dec 10, 2024 at 5:41 AM Benedict >> > wrote:
 I’m not convinced SAI has demonstrated a practical or theoretical 
 capability to fully replace secondary indexes anyway. So it would be very 
 premature to mark them deprecated.
 
> On 10 Dec 2024, at 06:29, Št

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Caleb Rackliffe]

I misspoke earlier about the feature gap. SAI supports queries legacy 2i does not, like numeric range queries.On Dec 10, 2024, at 9:29 AM, Caleb Rackliffe  wrote:I think my point here is that the hidden table 2i implementation has known correctness/availability/operational/resource usage issues whether it has a theoretical niche use-case or not from a query performance perspective.To Štefan’s question, yes, more or less. I’d like to at least see some success in production for the cases it was primarily designed for. That might not be enough to make it the default if it needs to perform better than the (broken) legacy 2i in global query situations. SAI is currently bad by design for global queries across 1000s of SSTables (LCS), so it would either need to be used in conjunction with a compaction strategy that aggressively limits the number of live SSTables, otherwise modified to handle that case better, or simply made the default w/ the guardrails it already has around these things becuase there simply isn’t a usable alternative.On Dec 10, 2024, at 9:13 AM, Benedict Elliott Smith  wrote:There is no reason it should ever be more capable than SAI for any partition/token-restricted query use-case, and I don't really see how there's any short-term path for any local 2i implementation in C* to be efficient for anything elseWhile I am not personally aware of much evidence presented that SAI performs better than 2i for the partition-restricted case, I do believe it is theoretically likely to. But any deprecation discussion should include evidence of this as a preamble.However, there are users that want queries not restricted by partition or token, and SAI is unlikely to serve these use cases as well. Yes, neither perform this use case well, but I cannot support deprecating a feature when its replacement is very likely inferior for some workloads. Since it is hard to prove that nobody is using 2i this way (and I recall from the distant past that such users were known to exist), we need instead to prove SAI can serve these workloads acceptably before we declare it a suitable replacement.I think there exists a near future world where we can offer proper global secondary indexes, at which point it would be acceptable to deprecate 2i and recommend users switch to either global secondary indexes or SAI. Until then, I cannot see a good argument for it if we want to be considered a stable and mature product.On 10 Dec 2024, at 09:28, Caleb Rackliffe  wrote:> I’m not convinced SAI has demonstrated a practical or theoretical capability to fully replace secondary indexes anyway. So it would be very premature to mark them deprecated.> If 2i indexes are to be marked as deprecated and SAI is beta, then what is actually the index implementation we stand behind in the production? It is like we are "abandoning" the former but the latter is not bullet-proof yet.The table-based 2i implementation has never been safe to use, and I don't think it ever will be, however we label it. (ex. CASSANDRA-18656, it's on-disk bloat, post-streaming rebuilds, etc.) There is no reason it should ever be more capable than SAI for any partition/token-restricted query use-case, and I don't really see how there's any short-term path for any local 2i implementation in C* to be efficient for anything else. There are presently no feature gaps on the query side.Anyway, there are still a lot of things we can improve about SAI (and things that already exist and are just waiting in the DS public fork)...I'm just not sure what reasonable use case the old 2i will be able to serve better.On Tue, Dec 10, 2024 at 5:41 AM Benedict  wrote:I’m not convinced SAI has demonstrated a practical or theoretical capability to fully replace secondary indexes anyway. So it would be very premature to mark them deprecated.On 10 Dec 2024, at 06:29, Štefan Miklošovič  wrote: ... then we should NOT mark it to be deprecated. On Tue, Dec 10, 2024 at 12:27 PM Štefan Miklošovič  wrote:I have a hard time getting used to the "terminology" here. If 2i indexes are to be marked as deprecated and SAI is beta, then what is actually the index implementation we stand behind in the production? It is like we are "abandoning" the former but the latter is not bullet-proof yet. The signal it sends is that we don't have a non-deprecated bullet-proof index impl.Maybe it is just about the wording and people are just fine running deprecated things knowing they are production-ready, what I am used to is that if something is deprecated, then there is always a replacement which is recommended. If there isn't a recommended replacement which can fully superseed the current implementation then we should mark it to be deprecated. I understand that you are trying to find some "common ground" / expressing that we are moving towards SAI but I am not sure the wording is entirely correct or we should be careful how we frame it. On Tue, Dec 10, 202

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Caleb Rackliffe]

I think my point here is that the hidden table 2i implementation has known correctness/availability/operational/resource usage issues whether it has a theoretical niche use-case or not from a query performance perspective.To Štefan’s question, yes, more or less. I’d like to at least see some success in production for the cases it was primarily designed for. That might not be enough to make it the default if it needs to perform better than the (broken) legacy 2i in global query situations. SAI is currently bad by design for global queries across 1000s of SSTables (LCS), so it would either need to be used in conjunction with a compaction strategy that aggressively limits the number of live SSTables, otherwise modified to handle that case better, or simply made the default w/ the guardrails it already has around these things becuase there simply isn’t a usable alternative.On Dec 10, 2024, at 9:13 AM, Benedict Elliott Smith  wrote:There is no reason it should ever be more capable than SAI for any partition/token-restricted query use-case, and I don't really see how there's any short-term path for any local 2i implementation in C* to be efficient for anything elseWhile I am not personally aware of much evidence presented that SAI performs better than 2i for the partition-restricted case, I do believe it is theoretically likely to. But any deprecation discussion should include evidence of this as a preamble.However, there are users that want queries not restricted by partition or token, and SAI is unlikely to serve these use cases as well. Yes, neither perform this use case well, but I cannot support deprecating a feature when its replacement is very likely inferior for some workloads. Since it is hard to prove that nobody is using 2i this way (and I recall from the distant past that such users were known to exist), we need instead to prove SAI can serve these workloads acceptably before we declare it a suitable replacement.I think there exists a near future world where we can offer proper global secondary indexes, at which point it would be acceptable to deprecate 2i and recommend users switch to either global secondary indexes or SAI. Until then, I cannot see a good argument for it if we want to be considered a stable and mature product.On 10 Dec 2024, at 09:28, Caleb Rackliffe  wrote:> I’m not convinced SAI has demonstrated a practical or theoretical capability to fully replace secondary indexes anyway. So it would be very premature to mark them deprecated.> If 2i indexes are to be marked as deprecated and SAI is beta, then what is actually the index implementation we stand behind in the production? It is like we are "abandoning" the former but the latter is not bullet-proof yet.The table-based 2i implementation has never been safe to use, and I don't think it ever will be, however we label it. (ex. CASSANDRA-18656, it's on-disk bloat, post-streaming rebuilds, etc.) There is no reason it should ever be more capable than SAI for any partition/token-restricted query use-case, and I don't really see how there's any short-term path for any local 2i implementation in C* to be efficient for anything else. There are presently no feature gaps on the query side.Anyway, there are still a lot of things we can improve about SAI (and things that already exist and are just waiting in the DS public fork)...I'm just not sure what reasonable use case the old 2i will be able to serve better.On Tue, Dec 10, 2024 at 5:41 AM Benedict  wrote:I’m not convinced SAI has demonstrated a practical or theoretical capability to fully replace secondary indexes anyway. So it would be very premature to mark them deprecated.On 10 Dec 2024, at 06:29, Štefan Miklošovič  wrote: ... then we should NOT mark it to be deprecated. On Tue, Dec 10, 2024 at 12:27 PM Štefan Miklošovič  wrote:I have a hard time getting used to the "terminology" here. If 2i indexes are to be marked as deprecated and SAI is beta, then what is actually the index implementation we stand behind in the production? It is like we are "abandoning" the former but the latter is not bullet-proof yet. The signal it sends is that we don't have a non-deprecated bullet-proof index impl.Maybe it is just about the wording and people are just fine running deprecated things knowing they are production-ready, what I am used to is that if something is deprecated, then there is always a replacement which is recommended. If there isn't a recommended replacement which can fully superseed the current implementation then we should mark it to be deprecated. I understand that you are trying to find some "common ground" / expressing that we are moving towards SAI but I am not sure the wording is entirely correct or we should be careful how we frame it. On Tue, Dec 10, 2024 at 12:01 PM Mick Semb Wever  wrote:> A possibility with SAI is to mark it beta while also marking 2i as
> deprecated (and leaving SASI as marked

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Jon Haddad]

I agree with Benedict.

Some of the issues with 2i should actually be significantly improved as
improvements to the storage engine continue. Trie memtables and indexes
should reduce the garbage overhead. Improving compaction throughout should
help with the bootstrap problem. I suspect theres some additional easy
performance wins that can be made that are common with SAI.

I bet theres a lot of room for improvement with existing 2i if anyone
looked at it closely.

Jon


On Tue, Dec 10, 2024 at 7:13 AM Benedict Elliott Smith 
wrote:

> There is no reason it should ever be more capable than SAI for any
> partition/token-restricted query use-case, and I don't really see how
> there's any short-term path for any local 2i implementation in C* to be
> efficient for anything else
>
>
> While I am not personally aware of much evidence presented that SAI
> performs better than 2i for the partition-restricted case, I do believe it
> is theoretically likely to. But any deprecation discussion should include
> evidence of this as a preamble.
>
> However, there are users that want queries *not* restricted by partition
> or token, and SAI is unlikely to serve these use cases as well. Yes,
> neither perform this use case *well*, but I cannot support deprecating a
> feature when its replacement is very likely inferior for some workloads.
> Since it is hard to prove that nobody is using 2i this way (and I recall
> from the distant past that such users were known to exist), we need instead
> to prove SAI can serve these workloads acceptably before we declare it a
> suitable replacement.
>
> I think there exists a near future world where we can offer proper
> *global* secondary indexes, at which point it would be acceptable to
> deprecate 2i and recommend users switch to either global secondary indexes
> or SAI. Until then, I cannot see a good argument for it if we want to be
> considered a stable and mature product.
>
>
> On 10 Dec 2024, at 09:28, Caleb Rackliffe 
> wrote:
>
> > I’m not convinced SAI has demonstrated a practical or theoretical
> capability to fully replace secondary indexes anyway. So it would be very
> premature to mark them deprecated.
>
> > If 2i indexes are to be marked as deprecated and SAI is beta, then what
> is actually the index implementation we stand behind in the production? It
> is like we are "abandoning" the former but the latter is not bullet-proof
> yet.
>
> The table-based 2i implementation has never been safe to use, and I don't
> think it ever will be, however we label it. (ex. CASSANDRA-18656, it's
> on-disk bloat, post-streaming rebuilds, etc.) There is no reason it should
> ever be more capable than SAI for any partition/token-restricted query
> use-case, and I don't really see how there's any short-term path for any
> local 2i implementation in C* to be efficient for anything else. There are
> presently no feature gaps on the query side.
>
> Anyway, there are still a lot of things we can improve about SAI (and
> things that already exist and are just waiting in the DS public fork)...I'm
> just not sure what reasonable use case the old 2i will be able to serve
> better.
>
> On Tue, Dec 10, 2024 at 5:41 AM Benedict  wrote:
>
>> I’m not convinced SAI has demonstrated a practical or theoretical
>> capability to fully replace secondary indexes anyway. So it would be very
>> premature to mark them deprecated.
>>
>> On 10 Dec 2024, at 06:29, Štefan Miklošovič 
>> wrote:
>>
>> 
>>  ... then we should NOT mark it to be deprecated.
>>
>> On Tue, Dec 10, 2024 at 12:27 PM Štefan Miklošovič <
>> smikloso...@apache.org> wrote:
>>
>>> I have a hard time getting used to the "terminology" here. If 2i indexes
>>> are to be marked as deprecated and SAI is beta, then what is actually the
>>> index implementation we stand behind in the production? It is like we are
>>> "abandoning" the former but the latter is not bullet-proof yet. The signal
>>> it sends is that we don't have a non-deprecated bullet-proof index impl.
>>>
>>> Maybe it is just about the wording and people are just fine running
>>> deprecated things knowing they are production-ready, what I am used to is
>>> that if something is deprecated, then there is always a replacement which
>>> is recommended. If there isn't a recommended replacement which can fully
>>> superseed the current implementation then we should mark it to be
>>> deprecated.
>>>
>>> I understand that you are trying to find some "common ground" /
>>> expressing that we are moving towards SAI but I am not sure the wording is
>>> entirely correct or we should be careful how we frame it.
>>>
>>> On Tue, Dec 10, 2024 at 12:01 PM Mick Semb Wever  wrote:
>>>
 > A possibility with SAI is to mark it beta while also marking 2i as
 > deprecated (and leaving SASI as marked).  This sends a clear signal
 > (imho) that SAI is the recommended solution forward but also being
 > honest about its maturity and QA.


  (and leaving SASI as marked *experimental*)

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Benedict Elliott Smith]

> There is no reason it should ever be more capable than SAI for any 
> partition/token-restricted query use-case, and I don't really see how there's 
> any short-term path for any local 2i implementation in C* to be efficient for 
> anything else

While I am not personally aware of much evidence presented that SAI performs 
better than 2i for the partition-restricted case, I do believe it is 
theoretically likely to. But any deprecation discussion should include evidence 
of this as a preamble.

However, there are users that want queries not restricted by partition or 
token, and SAI is unlikely to serve these use cases as well. Yes, neither 
perform this use case well, but I cannot support deprecating a feature when its 
replacement is very likely inferior for some workloads. Since it is hard to 
prove that nobody is using 2i this way (and I recall from the distant past that 
such users were known to exist), we need instead to prove SAI can serve these 
workloads acceptably before we declare it a suitable replacement.

I think there exists a near future world where we can offer proper global 
secondary indexes, at which point it would be acceptable to deprecate 2i and 
recommend users switch to either global secondary indexes or SAI. Until then, I 
cannot see a good argument for it if we want to be considered a stable and 
mature product.


> On 10 Dec 2024, at 09:28, Caleb Rackliffe  wrote:
> 
> > I’m not convinced SAI has demonstrated a practical or theoretical 
> > capability to fully replace secondary indexes anyway. So it would be very 
> > premature to mark them deprecated.
> 
> > If 2i indexes are to be marked as deprecated and SAI is beta, then what is 
> > actually the index implementation we stand behind in the production? It is 
> > like we are "abandoning" the former but the latter is not bullet-proof yet.
> 
> The table-based 2i implementation has never been safe to use, and I don't 
> think it ever will be, however we label it. (ex. CASSANDRA-18656, it's 
> on-disk bloat, post-streaming rebuilds, etc.) There is no reason it should 
> ever be more capable than SAI for any partition/token-restricted query 
> use-case, and I don't really see how there's any short-term path for any 
> local 2i implementation in C* to be efficient for anything else. There are 
> presently no feature gaps on the query side.
> 
> Anyway, there are still a lot of things we can improve about SAI (and things 
> that already exist and are just waiting in the DS public fork)...I'm just not 
> sure what reasonable use case the old 2i will be able to serve better.
> 
> On Tue, Dec 10, 2024 at 5:41 AM Benedict  > wrote:
>> I’m not convinced SAI has demonstrated a practical or theoretical capability 
>> to fully replace secondary indexes anyway. So it would be very premature to 
>> mark them deprecated.
>> 
>>> On 10 Dec 2024, at 06:29, Štefan Miklošovič >> > wrote:
>>> 
>>> 
>>>  ... then we should NOT mark it to be deprecated. 
>>> 
>>> On Tue, Dec 10, 2024 at 12:27 PM Štefan Miklošovič >> > wrote:
 I have a hard time getting used to the "terminology" here. If 2i indexes 
 are to be marked as deprecated and SAI is beta, then what is actually the 
 index implementation we stand behind in the production? It is like we are 
 "abandoning" the former but the latter is not bullet-proof yet. The signal 
 it sends is that we don't have a non-deprecated bullet-proof index impl.
 
 Maybe it is just about the wording and people are just fine running 
 deprecated things knowing they are production-ready, what I am used to is 
 that if something is deprecated, then there is always a replacement which 
 is recommended. If there isn't a recommended replacement which can fully 
 superseed the current implementation then we should mark it to be 
 deprecated. 
 
 I understand that you are trying to find some "common ground" / expressing 
 that we are moving towards SAI but I am not sure the wording is entirely 
 correct or we should be careful how we frame it. 
 
 On Tue, Dec 10, 2024 at 12:01 PM Mick Semb Wever >>> > wrote:
> > A possibility with SAI is to mark it beta while also marking 2i as
> > deprecated (and leaving SASI as marked).  This sends a clear signal
> > (imho) that SAI is the recommended solution forward but also being
> > honest about its maturity and QA.
> 
> 
>  (and leaving SASI as marked *experimental*)

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Štefan Miklošovič]

Ok, so, when SAI has no feature gaps and you consider 2i to not be safe
then what is preventing us from deprecating 2i as suggested? Just not
enough production workloads / experience running that?

Jon mentioned the performance earlier (vector search) so when that is
addressed / improved then we will deprecate 2i? That is how I understand
that because if perf is good and feature gap is none then what else is
missing to deprecate 2i?

I am trying to somehow quantify what needs to be done / what needs to
happen to say 2i is deprecated so we can do that and say that SAI is SAI
without any beta status or similar.

On Tue, Dec 10, 2024 at 3:31 PM Caleb Rackliffe 
wrote:

> > I’m not convinced SAI has demonstrated a practical or theoretical
> capability to fully replace secondary indexes anyway. So it would be very
> premature to mark them deprecated.
>
> > If 2i indexes are to be marked as deprecated and SAI is beta, then what
> is actually the index implementation we stand behind in the production? It
> is like we are "abandoning" the former but the latter is not bullet-proof
> yet.
>
> The table-based 2i implementation has never been safe to use, and I don't
> think it ever will be, however we label it. (ex. CASSANDRA-18656, it's
> on-disk bloat, post-streaming rebuilds, etc.) There is no reason it should
> ever be more capable than SAI for any partition/token-restricted query
> use-case, and I don't really see how there's any short-term path for any
> local 2i implementation in C* to be efficient for anything else. There are
> presently no feature gaps on the query side.
>
> Anyway, there are still a lot of things we can improve about SAI (and
> things that already exist and are just waiting in the DS public fork)...I'm
> just not sure what reasonable use case the old 2i will be able to serve
> better.
>
> On Tue, Dec 10, 2024 at 5:41 AM Benedict  wrote:
>
>> I’m not convinced SAI has demonstrated a practical or theoretical
>> capability to fully replace secondary indexes anyway. So it would be very
>> premature to mark them deprecated.
>>
>> On 10 Dec 2024, at 06:29, Štefan Miklošovič 
>> wrote:
>>
>> 
>>  ... then we should NOT mark it to be deprecated.
>>
>> On Tue, Dec 10, 2024 at 12:27 PM Štefan Miklošovič <
>> smikloso...@apache.org> wrote:
>>
>>> I have a hard time getting used to the "terminology" here. If 2i indexes
>>> are to be marked as deprecated and SAI is beta, then what is actually the
>>> index implementation we stand behind in the production? It is like we are
>>> "abandoning" the former but the latter is not bullet-proof yet. The signal
>>> it sends is that we don't have a non-deprecated bullet-proof index impl.
>>>
>>> Maybe it is just about the wording and people are just fine running
>>> deprecated things knowing they are production-ready, what I am used to is
>>> that if something is deprecated, then there is always a replacement which
>>> is recommended. If there isn't a recommended replacement which can fully
>>> superseed the current implementation then we should mark it to be
>>> deprecated.
>>>
>>> I understand that you are trying to find some "common ground" /
>>> expressing that we are moving towards SAI but I am not sure the wording is
>>> entirely correct or we should be careful how we frame it.
>>>
>>> On Tue, Dec 10, 2024 at 12:01 PM Mick Semb Wever  wrote:
>>>
 > A possibility with SAI is to mark it beta while also marking 2i as
 > deprecated (and leaving SASI as marked).  This sends a clear signal
 > (imho) that SAI is the recommended solution forward but also being
 > honest about its maturity and QA.


  (and leaving SASI as marked *experimental*)

>>>

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Caleb Rackliffe]

> I’m not convinced SAI has demonstrated a practical or theoretical
capability to fully replace secondary indexes anyway. So it would be very
premature to mark them deprecated.

> If 2i indexes are to be marked as deprecated and SAI is beta, then what
is actually the index implementation we stand behind in the production? It
is like we are "abandoning" the former but the latter is not bullet-proof
yet.

The table-based 2i implementation has never been safe to use, and I don't
think it ever will be, however we label it. (ex. CASSANDRA-18656, it's
on-disk bloat, post-streaming rebuilds, etc.) There is no reason it should
ever be more capable than SAI for any partition/token-restricted query
use-case, and I don't really see how there's any short-term path for any
local 2i implementation in C* to be efficient for anything else. There are
presently no feature gaps on the query side.

Anyway, there are still a lot of things we can improve about SAI (and
things that already exist and are just waiting in the DS public fork)...I'm
just not sure what reasonable use case the old 2i will be able to serve
better.

On Tue, Dec 10, 2024 at 5:41 AM Benedict  wrote:

> I’m not convinced SAI has demonstrated a practical or theoretical
> capability to fully replace secondary indexes anyway. So it would be very
> premature to mark them deprecated.
>
> On 10 Dec 2024, at 06:29, Štefan Miklošovič 
> wrote:
>
> 
>  ... then we should NOT mark it to be deprecated.
>
> On Tue, Dec 10, 2024 at 12:27 PM Štefan Miklošovič 
> wrote:
>
>> I have a hard time getting used to the "terminology" here. If 2i indexes
>> are to be marked as deprecated and SAI is beta, then what is actually the
>> index implementation we stand behind in the production? It is like we are
>> "abandoning" the former but the latter is not bullet-proof yet. The signal
>> it sends is that we don't have a non-deprecated bullet-proof index impl.
>>
>> Maybe it is just about the wording and people are just fine running
>> deprecated things knowing they are production-ready, what I am used to is
>> that if something is deprecated, then there is always a replacement which
>> is recommended. If there isn't a recommended replacement which can fully
>> superseed the current implementation then we should mark it to be
>> deprecated.
>>
>> I understand that you are trying to find some "common ground" /
>> expressing that we are moving towards SAI but I am not sure the wording is
>> entirely correct or we should be careful how we frame it.
>>
>> On Tue, Dec 10, 2024 at 12:01 PM Mick Semb Wever  wrote:
>>
>>> > A possibility with SAI is to mark it beta while also marking 2i as
>>> > deprecated (and leaving SASI as marked).  This sends a clear signal
>>> > (imho) that SAI is the recommended solution forward but also being
>>> > honest about its maturity and QA.
>>>
>>>
>>>  (and leaving SASI as marked *experimental*)
>>>
>>

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Aleksey Yeshchenko]

There is room for a few different grades of ‘flawed’. From broken (you should 
really not use this anymore) to “generally usable but be careful and aware of 
these caveats” - with different degrees of gating/warning applied.

> On 10 Dec 2024, at 11:46, Aleksey Yeshchenko  wrote:
> 
> What we’ve done is we’ve overloaded the term ‘experimental’ to mean too many 
> related but different ideas. We need additional, more specific terminology to 
> disambiguate.
> 
> 1. Labelling released features that were known to be unstable at release as 
> ‘experimental’  retroactively shouldn’t happen and AFAIK only happened once, 
> with MVs, and ‘experimental’ there was just a euphemism for ‘broken’. Our 
> practices are more mature now, I like to think, that a situation like this 
> would not arise in the future - the bar for releasing a completed marketable 
> feature is higher. So the label ‘experimental’ should not be applied 
> retroactively to anything.
> 
> 2. It’s possible that a released, once considered production-ready feature, 
> might be discovered to be deeply flawed after being released already. We need 
> to temporarily mark such a feature as ‘broken' or ‘flawed'. Not experimental, 
> and not even ‘unstable’. Make sure we emit a warning on its use everywhere, 
> and, if possible, make it opt-in in the next major, at the very least, to 
> prevent new uses of it. Announce on dev, add a note in NEWS.txt, etc. If the 
> flaws are later addressed, remove the label. Removing the feature itself 
> might not be possible, but should be considered, with heavy advanced 
> telegraphing to the community.
> 
> 3. There is probably room for genuine use of ‘experimental’ as a feature 
> label. For opt-in features that we commit with an understanding that they 
> might not make it at all. Unstable API is implied here, but a feature can 
> also have an unstable API without being experimental - so ‘experimental' 
> doesn’t equal to ‘api-unstable’. These should not be relied on by any 
> production code, they would be heavily gated by unambiguous configuration 
> flags, disabled by default, allowed to be removed or changed in any version 
> including a minor one.
> 
> 4. New features without known flaws, intended to be production-ready and 
> marketable eventually, that we may want to gain some real-world confidence 
> with before we are happy to market or make default. UCS, for example, which 
> seems to be in heavy use in Astra and doesn’t have any known open issues 
> (AFAIK). It’s not experimental, it’s not unstable, it’s not ‘alpha’ or 
> ‘beta’, it just hasn't been widely enough used to have gained a lot of 
> confidence. It’s just new. I’m not sure what label even applies here. It’s 
> just a regular feature that happens to be new, doesn’t need a label, just 
> needs to see some widespread use before we can make it a default. No other 
> limitation on its use.
> 
> 5. Early-integrated, not-yet fully-completed features that are NOT 
> experimental in nature. Isolated, gated behind deep configuration flags. Have 
> a CEP behind them, we trust that they will be eventually completed, but for 
> pragmatic reasons it just made sense to commit them at an earlier stage. 
> ‘Preview’, ‘alpha’, ‘beta’ are labels that could apply here depending on 
> current feature readiness status. API-instability is implied. Once finished 
> they just become a regular new feature, no flag needed, no heavy config 
> gating needed.
> 
> I might be missing some scenarios here.
> 
>> On 10 Dec 2024, at 09:12, Mick Semb Wever  wrote:
>> 
>> I see value in using a beta flag in addition to an experimental flag,
>> and that such a beta flag should see a lot more use than experimental.
>> 
>> Java 17 definitely  falls in the beta category.  I/We definitely
>> recommend its usage in production, but as has been said data is needed
>> over trust and the community hasn't the resources to provide such data
>> – we're just waiting for any user to give us the feedback "we're using
>> it prod".  (My expectations were that we'd hear this by 5.0.3.)
>> 
>> Early integration is valuable sometimes, and anything marked
>> experimental (once we have a beta flag in use) should be able to later
>> become deprecated and removed.  So I agree with Dinesh's point, that
>> also emphasises a high bar for merging – totally agree that we've seen
>> a number of things merged that missed basic testing requirements.
>> 
>> A possibility with SAI is to mark it beta while also marking 2i as
>> deprecated (and leaving SASI as marked).  This sends a clear signal
>> (imho) that SAI is the recommended solution forward but also being
>> honest about its maturity and QA.
>> 
>> 
>> On Tue, 10 Dec 2024 at 09:42, Jon Haddad  wrote:
>>> 
>>> I am strongly against early integration, because we can't / don't remove 
>>> things when we should.  MVs are the prime example here, as is the current 
>>> iteration of Vector search.
>>> 
>>> Early integration works fine when it's internal software 

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Aleksey Yeshchenko]

What we’ve done is we’ve overloaded the term ‘experimental’ to mean too many 
related but different ideas. We need additional, more specific terminology to 
disambiguate.

1. Labelling released features that were known to be unstable at release as 
‘experimental’  retroactively shouldn’t happen and AFAIK only happened once, 
with MVs, and ‘experimental’ there was just a euphemism for ‘broken’. Our 
practices are more mature now, I like to think, that a situation like this 
would not arise in the future - the bar for releasing a completed marketable 
feature is higher. So the label ‘experimental’ should not be applied 
retroactively to anything.

2. It’s possible that a released, once considered production-ready feature, 
might be discovered to be deeply flawed after being released already. We need 
to temporarily mark such a feature as ‘broken' or ‘flawed'. Not experimental, 
and not even ‘unstable’. Make sure we emit a warning on its use everywhere, 
and, if possible, make it opt-in in the next major, at the very least, to 
prevent new uses of it. Announce on dev, add a note in NEWS.txt, etc. If the 
flaws are later addressed, remove the label. Removing the feature itself might 
not be possible, but should be considered, with heavy advanced telegraphing to 
the community.

3. There is probably room for genuine use of ‘experimental’ as a feature label. 
For opt-in features that we commit with an understanding that they might not 
make it at all. Unstable API is implied here, but a feature can also have an 
unstable API without being experimental - so ‘experimental' doesn’t equal to 
‘api-unstable’. These should not be relied on by any production code, they 
would be heavily gated by unambiguous configuration flags, disabled by default, 
allowed to be removed or changed in any version including a minor one.

4. New features without known flaws, intended to be production-ready and 
marketable eventually, that we may want to gain some real-world confidence with 
before we are happy to market or make default. UCS, for example, which seems to 
be in heavy use in Astra and doesn’t have any known open issues (AFAIK). It’s 
not experimental, it’s not unstable, it’s not ‘alpha’ or ‘beta’, it just hasn't 
been widely enough used to have gained a lot of confidence. It’s just new. I’m 
not sure what label even applies here. It’s just a regular feature that happens 
to be new, doesn’t need a label, just needs to see some widespread use before 
we can make it a default. No other limitation on its use.

5. Early-integrated, not-yet fully-completed features that are NOT experimental 
in nature. Isolated, gated behind deep configuration flags. Have a CEP behind 
them, we trust that they will be eventually completed, but for pragmatic 
reasons it just made sense to commit them at an earlier stage. ‘Preview’, 
‘alpha’, ‘beta’ are labels that could apply here depending on current feature 
readiness status. API-instability is implied. Once finished they just become a 
regular new feature, no flag needed, no heavy config gating needed.

I might be missing some scenarios here.

> On 10 Dec 2024, at 09:12, Mick Semb Wever  wrote:
> 
> I see value in using a beta flag in addition to an experimental flag,
> and that such a beta flag should see a lot more use than experimental.
> 
> Java 17 definitely  falls in the beta category.  I/We definitely
> recommend its usage in production, but as has been said data is needed
> over trust and the community hasn't the resources to provide such data
> – we're just waiting for any user to give us the feedback "we're using
> it prod".  (My expectations were that we'd hear this by 5.0.3.)
> 
> Early integration is valuable sometimes, and anything marked
> experimental (once we have a beta flag in use) should be able to later
> become deprecated and removed.  So I agree with Dinesh's point, that
> also emphasises a high bar for merging – totally agree that we've seen
> a number of things merged that missed basic testing requirements.
> 
> A possibility with SAI is to mark it beta while also marking 2i as
> deprecated (and leaving SASI as marked).  This sends a clear signal
> (imho) that SAI is the recommended solution forward but also being
> honest about its maturity and QA.
> 
> 
> On Tue, 10 Dec 2024 at 09:42, Jon Haddad  wrote:
>> 
>> I am strongly against early integration, because we can't / don't remove 
>> things when we should.  MVs are the prime example here, as is the current 
>> iteration of Vector search.
>> 
>> Early integration works fine when it's internal software that you have 
>> control over, it doesn't work well for software that gets deployed and 
>> relied on outside your org.
>> 
>> 
>> 
>> On Mon, Dec 9, 2024 at 2:02 PM Dinesh Joshi  wrote:
>>> 
>>> On Mon, Dec 9, 2024 at 12:26 PM Jon Haddad  wrote:
 
 I hope I've made my point.  The bar for merging in new functionality 
 should be higher.  Features should work with 1TB of data on 3 nodes, 
 that's a low b

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Štefan Miklošovič]

Yes, I agree. I see it the same.

On Tue, Dec 10, 2024 at 12:41 PM Benedict  wrote:

> I’m not convinced SAI has demonstrated a practical or theoretical
> capability to fully replace secondary indexes anyway. So it would be very
> premature to mark them deprecated.
>
> On 10 Dec 2024, at 06:29, Štefan Miklošovič 
> wrote:
>
> 
>  ... then we should NOT mark it to be deprecated.
>
> On Tue, Dec 10, 2024 at 12:27 PM Štefan Miklošovič 
> wrote:
>
>> I have a hard time getting used to the "terminology" here. If 2i indexes
>> are to be marked as deprecated and SAI is beta, then what is actually the
>> index implementation we stand behind in the production? It is like we are
>> "abandoning" the former but the latter is not bullet-proof yet. The signal
>> it sends is that we don't have a non-deprecated bullet-proof index impl.
>>
>> Maybe it is just about the wording and people are just fine running
>> deprecated things knowing they are production-ready, what I am used to is
>> that if something is deprecated, then there is always a replacement which
>> is recommended. If there isn't a recommended replacement which can fully
>> superseed the current implementation then we should mark it to be
>> deprecated.
>>
>> I understand that you are trying to find some "common ground" /
>> expressing that we are moving towards SAI but I am not sure the wording is
>> entirely correct or we should be careful how we frame it.
>>
>> On Tue, Dec 10, 2024 at 12:01 PM Mick Semb Wever  wrote:
>>
>>> > A possibility with SAI is to mark it beta while also marking 2i as
>>> > deprecated (and leaving SASI as marked).  This sends a clear signal
>>> > (imho) that SAI is the recommended solution forward but also being
>>> > honest about its maturity and QA.
>>>
>>>
>>>  (and leaving SASI as marked *experimental*)
>>>
>>

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Benedict]

I’m not convinced SAI has demonstrated a practical or theoretical capability to fully replace secondary indexes anyway. So it would be very premature to mark them deprecated.On 10 Dec 2024, at 06:29, Štefan Miklošovič  wrote: ... then we should NOT mark it to be deprecated. On Tue, Dec 10, 2024 at 12:27 PM Štefan Miklošovič  wrote:I have a hard time getting used to the "terminology" here. If 2i indexes are to be marked as deprecated and SAI is beta, then what is actually the index implementation we stand behind in the production? It is like we are "abandoning" the former but the latter is not bullet-proof yet. The signal it sends is that we don't have a non-deprecated bullet-proof index impl.Maybe it is just about the wording and people are just fine running deprecated things knowing they are production-ready, what I am used to is that if something is deprecated, then there is always a replacement which is recommended. If there isn't a recommended replacement which can fully superseed the current implementation then we should mark it to be deprecated. I understand that you are trying to find some "common ground" / expressing that we are moving towards SAI but I am not sure the wording is entirely correct or we should be careful how we frame it. On Tue, Dec 10, 2024 at 12:01 PM Mick Semb Wever  wrote:> A possibility with SAI is to mark it beta while also marking 2i as
> deprecated (and leaving SASI as marked).  This sends a clear signal
> (imho) that SAI is the recommended solution forward but also being
> honest about its maturity and QA.


 (and leaving SASI as marked *experimental*)

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Štefan Miklošovič]

 ... then we should NOT mark it to be deprecated.

On Tue, Dec 10, 2024 at 12:27 PM Štefan Miklošovič 
wrote:

> I have a hard time getting used to the "terminology" here. If 2i indexes
> are to be marked as deprecated and SAI is beta, then what is actually the
> index implementation we stand behind in the production? It is like we are
> "abandoning" the former but the latter is not bullet-proof yet. The signal
> it sends is that we don't have a non-deprecated bullet-proof index impl.
>
> Maybe it is just about the wording and people are just fine running
> deprecated things knowing they are production-ready, what I am used to is
> that if something is deprecated, then there is always a replacement which
> is recommended. If there isn't a recommended replacement which can fully
> superseed the current implementation then we should mark it to be
> deprecated.
>
> I understand that you are trying to find some "common ground" / expressing
> that we are moving towards SAI but I am not sure the wording is entirely
> correct or we should be careful how we frame it.
>
> On Tue, Dec 10, 2024 at 12:01 PM Mick Semb Wever  wrote:
>
>> > A possibility with SAI is to mark it beta while also marking 2i as
>> > deprecated (and leaving SASI as marked).  This sends a clear signal
>> > (imho) that SAI is the recommended solution forward but also being
>> > honest about its maturity and QA.
>>
>>
>>  (and leaving SASI as marked *experimental*)
>>
>

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Štefan Miklošovič]

I have a hard time getting used to the "terminology" here. If 2i indexes
are to be marked as deprecated and SAI is beta, then what is actually the
index implementation we stand behind in the production? It is like we are
"abandoning" the former but the latter is not bullet-proof yet. The signal
it sends is that we don't have a non-deprecated bullet-proof index impl.

Maybe it is just about the wording and people are just fine running
deprecated things knowing they are production-ready, what I am used to is
that if something is deprecated, then there is always a replacement which
is recommended. If there isn't a recommended replacement which can fully
superseed the current implementation then we should mark it to be
deprecated.

I understand that you are trying to find some "common ground" / expressing
that we are moving towards SAI but I am not sure the wording is entirely
correct or we should be careful how we frame it.

On Tue, Dec 10, 2024 at 12:01 PM Mick Semb Wever  wrote:

> > A possibility with SAI is to mark it beta while also marking 2i as
> > deprecated (and leaving SASI as marked).  This sends a clear signal
> > (imho) that SAI is the recommended solution forward but also being
> > honest about its maturity and QA.
>
>
>  (and leaving SASI as marked *experimental*)
>

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Mick Semb Wever]

> A possibility with SAI is to mark it beta while also marking 2i as
> deprecated (and leaving SASI as marked).  This sends a clear signal
> (imho) that SAI is the recommended solution forward but also being
> honest about its maturity and QA.


 (and leaving SASI as marked *experimental*)

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Mick Semb Wever]

I see value in using a beta flag in addition to an experimental flag,
and that such a beta flag should see a lot more use than experimental.

Java 17 definitely  falls in the beta category.  I/We definitely
recommend its usage in production, but as has been said data is needed
over trust and the community hasn't the resources to provide such data
– we're just waiting for any user to give us the feedback "we're using
it prod".  (My expectations were that we'd hear this by 5.0.3.)

Early integration is valuable sometimes, and anything marked
experimental (once we have a beta flag in use) should be able to later
become deprecated and removed.  So I agree with Dinesh's point, that
also emphasises a high bar for merging – totally agree that we've seen
a number of things merged that missed basic testing requirements.

A possibility with SAI is to mark it beta while also marking 2i as
deprecated (and leaving SASI as marked).  This sends a clear signal
(imho) that SAI is the recommended solution forward but also being
honest about its maturity and QA.


On Tue, 10 Dec 2024 at 09:42, Jon Haddad  wrote:
>
> I am strongly against early integration, because we can't / don't remove 
> things when we should.  MVs are the prime example here, as is the current 
> iteration of Vector search.
>
> Early integration works fine when it's internal software that you have 
> control over, it doesn't work well for software that gets deployed and relied 
> on outside your org.
>
>
>
> On Mon, Dec 9, 2024 at 2:02 PM Dinesh Joshi  wrote:
>>
>> On Mon, Dec 9, 2024 at 12:26 PM Jon Haddad  wrote:
>>>
>>> I hope I've made my point.  The bar for merging in new functionality should 
>>> be higher.  Features should work with 1TB of data on 3 nodes, that's a low 
>>> bar.  I've spent at least a thousand hours over the last 5 years developing 
>>> the tooling to do these tests, there's no reason to not do them, and when 
>>> we know things are broken, we shouldn't ship them.
>>
>>
>> I am a big fan of early integration. I agree that the bar for merging should 
>> be high but at the same time we should lean more heavily on feature flagging 
>> which is also a very common software industry practice. This would allow an 
>> operator to enable features that are deemed risky for production use. It 
>> creates a faster feedback loop and will reveal issues earlier in the 
>> development cycle. It might actually avoid big patches but that is a topic 
>> for a different thread.
>>
>> Dinesh

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Jon Haddad]

I am strongly against early integration, because we can't / don't remove
things when we should.  MVs are the prime example here, as is the current
iteration of Vector search.

Early integration works fine when it's internal software that you have
control over, it doesn't work well for software that gets deployed and
relied on outside your org.



On Mon, Dec 9, 2024 at 2:02 PM Dinesh Joshi  wrote:

> On Mon, Dec 9, 2024 at 12:26 PM Jon Haddad 
> wrote:
>
>> I hope I've made my point.  The bar for merging in new functionality
>> should be higher.  Features should work with 1TB of data on 3 nodes, that's
>> a low bar.  I've spent at least a thousand hours over the last 5 years
>> developing the tooling to do these tests, there's no reason to not do them,
>> and when we know things are broken, we shouldn't ship them.
>>
>
> I am a big fan of early integration. I agree that the bar for merging
> should be high but at the same time we should lean more heavily on feature
> flagging which is also a very common software industry practice. This would
> allow an operator to enable features that are deemed risky for production
> use. It creates a faster feedback loop and will reveal issues earlier in
> the development cycle. It might actually avoid big patches but that is a
> topic for a different thread.
>
> Dinesh
>

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Dinesh Joshi]

On Mon, Dec 9, 2024 at 12:26 PM Jon Haddad  wrote:

> I hope I've made my point.  The bar for merging in new functionality
> should be higher.  Features should work with 1TB of data on 3 nodes, that's
> a low bar.  I've spent at least a thousand hours over the last 5 years
> developing the tooling to do these tests, there's no reason to not do them,
> and when we know things are broken, we shouldn't ship them.
>

I am a big fan of early integration. I agree that the bar for merging
should be high but at the same time we should lean more heavily on feature
flagging which is also a very common software industry practice. This would
allow an operator to enable features that are deemed risky for production
use. It creates a faster feedback loop and will reveal issues earlier in
the development cycle. It might actually avoid big patches but that is a
topic for a different thread.

Dinesh

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Dinesh Joshi]

On Mon, Dec 9, 2024 at 10:52 AM Jeremy Hanna 
wrote:

> In the case of UCS, is it in a beta state until we resolve the discussions
> around UX/documentation?  From a functional and production usage
> perspective, it has been the only compaction strategy available for users
> in DataStax's Astra managed database service in both traditional node based
> deployments and serverless for 5 years across thousands of databases.
>

I am sorry but I have seen instances where this argument was used and it is
frankly puzzling to me. In my view, UCS' implementation inside Astra has
little to do with the stability and production readiness of the code that
is in Apache Cassandra. While its usage inside Astra is definitely a good
data point, the code that is committed to Apache Cassandra and the way it
integrates inside the open source code base could have different bugs /
edge cases that do not appear in Astra. So I would really not be pointing
at Astra as evidence of its robustness and maturity inside Apache
Cassandra's codebase.


> While I agree that it's premature to call that a new default for tables,
> it seems overly cautious to me that it needs to pass through a beta
> period.  Let people opt into it and try it out and if something breaks,
> let's fix it.  I'm not sure of the value of using "beta" with it.
>

I am all for letting users try out new features in Cassandra but we're a
database and we need to weigh risks carefully. Graduating a feature through
different maturity and production readiness levels is very common and the
rate at which they can move through different stages is directly related to
the quantity and quality of testing they have been through.

We can debate about the 'experimental' or 'beta' moniker but the goal is
for the project to accurately inform the operator about the feature's
production readiness so they can weigh the risk of enabling the feature for
themselves.

Thanks,

Dinesh

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Jon Haddad]

ugh I believe this conversation is healthy to
> have and shows the maturity of our project. Thank you, Josh!
>
>
> Best regards,
>
> Ekaterina
>
>
> On Mon, 9 Dec 2024 at 13:21, Jon Haddad  wrote:
>
> The tough thing here is that MVs are marked experimental retroactively,
> because by the time the problems were known, there wasn't much anyone could
> do.  Experimental was our way of saying "oops, we screwed up, let's put a
> label on it" and the same label got applied to a bunch of new stuff
> including Java 17.  They're not even close to being in the same category,
> but they're labeled the same and people treat them as equivalent.
>
> If we knew MVs were so broken before they were merged, they would have
> been -1'ed.  Same with incremental repair (till 4.0), and vector search
> today.  I would have -1'ed all three of these if it was known how poorly
> they actually performed at the time they were committed.
>
> Side note, vector search isn't marked as experimental today, but it's not
> even usable for non-trivial datasets out of the box, so it should be marked
> as such at this point.
>
> I really wish this stuff was tested at a reasonable scale across various
> failure modes before merging, because the harm it does to the community is
> real.  We really shouldn't be put in a position where stuff gets released,
> hyped up, then we find it it's obviously not ready for real world use.  I
> built my tooling (tlp-cluster, now easy-cass-lab, and tlp-stress, now
> easy-cass-stress), with this in mind, but sadly I haven't seen much use of
> it it to verify patches.  The only reason I found a memory leak in
> CASSANDRA-15452 was because I used these tools on multi-TB datasets over
> several days.
>
>
> Jon
>
>
> On Mon, Dec 9, 2024 at 9:55 AM Slater, Ben via dev <
> dev@cassandra.apache.org> wrote:
>
> I'm a little worried by the idea of grouping in MVs with things like a
> Java version under the same "beta" label (acknowledging that they are
> currently grouped under the same "experimental" label).
>
> To me, "beta" implies it's pretty close to production ready and there is
> an intention to get it to production ready in the near future. I don't
> think this really describes MVs as I don't see anyone looking like they are
> trying to get them to really production ready (although I could easily be
> wrong on that).
>
> Maybe there is an argument for "experimental"=this is here to get feedback
> but there's no commitment it will make it to production ready and "beta"=we
> think this is done but we'd like to see some production use before
> declaring it stable. For beta, we'll treat bugs with the same priority as
> "stable" (or at least close to)?
>
> Cheers
> Ben
>
>
>
>
> --
>
> *From:* Jon Haddad 
> *Sent:* 09 December 2024 09:43
> *To:* dev@cassandra.apache.org 
> *Subject:* Re: [DISCUSS] Experimental flagging (fork from Re-evaluate
> compaction defaults in 5.1/trunk)
>
> *EXTERNAL EMAIL - USE CAUTION when clicking links or attachments*
>
>
>
> I like this.  There's a few things marked as experimental today, so I'll
> take a stab at making this more concrete, and I think we should be open to
> graduating certain things out of beta to GA at a faster cycle than a major
> release.
>
> Java versions, for example, should really move out of "beta" quickly.  We
> test against it, and we're not going to drop new versions.  So if we're
> looking at C* 5.0, we should move Java 17 out of experimental / beta
> immediately and call it GA.
>
> SAI and UCS should probably graduate no later than 5.1.
>
> On the other hand, MVs have enough warts I actively recommend against
> using them and should be in beta till we can actually repair them.
>
> I don't know if anyone's actually used transient replication and if it's
> even beta quality... that might actually warrant being called experimental
> still.
>
> 'ALTER ... DROP COMPACT STORAGE' is flagged as experimental.  I'm not sure
> what to do with this.  I advise people migrate their data for any Thrift ->
> CQL cases, mostly because the edge cases are so hard to know in advance,
> especially since by now these codebases are ancient and the original
> developers are long gone.
>
> Thoughts?
>
> Jon
>
>
>
>
> On Mon, Dec 9, 2024 at 6:28 AM Josh McKenzie  wrote:
>
>
> Jon stated:
>
> Side note: I think experimental has been over-used and has lost all
> meaning.  How is Java 17 experimental

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Jeff Jirsa]

> On Dec 9, 2024, at 10:52 AM, Jeremy Hanna  wrote:
> 
> 
> 
> In the case of UCS, is it in a beta state until we resolve the discussions 
> around UX/documentation?  From a functional and production usage perspective, 
> it has been the only compaction strategy available for users in DataStax's 
> Astra managed database service in both traditional node based deployments and 
> serverless for 5 years across thousands of databases.  While I agree that 
> it's premature to call that a new default for tables, it seems overly 
> cautious to me that it needs to pass through a beta period.  Let people opt 
> into it and try it out and if something breaks, let's fix it.  I'm not sure 
> of the value of using "beta" with it.

Devil’s advocate: how does anyone know?

Neither CEP-26 nor Cassandra-18397 contain a single graph or explanation or 
comparison of any prod use case. CEP-26 describes a load test but I don’t see 
the results posted on the ticket or the wiki.

There IS a talk from a conference with a video on YouTube but I missed the 
conference and haven’t had time to watch the video, but it’s not like the 
official project run docs make it easy to infer state.

Look at Cassandra-9666 and see how many people posted production side by side 
comparisons before that got merged. 

(I buy UCS isn’t beta. I don’t buy it’s default ready yet. Only because there’s 
zero tangible evidence readily available to me right now). 

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Jon Haddad]

Huh, I didn't notice it when I grepped the code base.  I stand corrected.

Jon

On Mon, Dec 9, 2024 at 10:57 AM Ekaterina Dimitrova 
wrote:

> Hey Jon,
> The following quick test shows me that vector search is marked as
> experimental (it is just not in cassandra.yaml as materialized views, etc)
>
> cqlsh:k> CREATE TABLE t (pk int, str_val text, val vector,
> PRIMARY KEY(pk));
>
> cqlsh:k> CREATE CUSTOM INDEX ON t(val) USING 'StorageAttachedIndex';
>
>
> Warnings :
>
> SAI ANN indexes on vector columns are experimental and are not recommended
> for production use.
>
> They don't yet support SELECT queries with:
>
>  * Consistency level higher than ONE/LOCAL_ONE.
>
>  * Paging.
>
>  * No LIMIT clauses.
>
>  * PER PARTITION LIMIT clauses.
>
>  * GROUP BY clauses.
>
>  * Aggregation functions.
>
>  * Filters on columns without a SAI index.
>
>
> I do agree that there is differentiation also between experimental and
> beta. But I need to think more before expressing concrete
> opinion/suggestions here. Though I believe this conversation is healthy to
> have and shows the maturity of our project. Thank you, Josh!
>
>
> Best regards,
>
> Ekaterina
>
>
> On Mon, 9 Dec 2024 at 13:21, Jon Haddad  wrote:
>
>> The tough thing here is that MVs are marked experimental retroactively,
>> because by the time the problems were known, there wasn't much anyone could
>> do.  Experimental was our way of saying "oops, we screwed up, let's put a
>> label on it" and the same label got applied to a bunch of new stuff
>> including Java 17.  They're not even close to being in the same category,
>> but they're labeled the same and people treat them as equivalent.
>>
>> If we knew MVs were so broken before they were merged, they would have
>> been -1'ed.  Same with incremental repair (till 4.0), and vector search
>> today.  I would have -1'ed all three of these if it was known how poorly
>> they actually performed at the time they were committed.
>>
>> Side note, vector search isn't marked as experimental today, but it's not
>> even usable for non-trivial datasets out of the box, so it should be marked
>> as such at this point.
>>
>> I really wish this stuff was tested at a reasonable scale across various
>> failure modes before merging, because the harm it does to the community is
>> real.  We really shouldn't be put in a position where stuff gets released,
>> hyped up, then we find it it's obviously not ready for real world use.  I
>> built my tooling (tlp-cluster, now easy-cass-lab, and tlp-stress, now
>> easy-cass-stress), with this in mind, but sadly I haven't seen much use of
>> it it to verify patches.  The only reason I found a memory leak in
>> CASSANDRA-15452 was because I used these tools on multi-TB datasets over
>> several days.
>>
>>
>> Jon
>>
>>
>> On Mon, Dec 9, 2024 at 9:55 AM Slater, Ben via dev <
>> dev@cassandra.apache.org> wrote:
>>
>>> I'm a little worried by the idea of grouping in MVs with things like a
>>> Java version under the same "beta" label (acknowledging that they are
>>> currently grouped under the same "experimental" label).
>>>
>>> To me, "beta" implies it's pretty close to production ready and there is
>>> an intention to get it to production ready in the near future. I don't
>>> think this really describes MVs as I don't see anyone looking like they are
>>> trying to get them to really production ready (although I could easily be
>>> wrong on that).
>>>
>>> Maybe there is an argument for "experimental"=this is here to get
>>> feedback but there's no commitment it will make it to production ready and
>>> "beta"=we think this is done but we'd like to see some production use
>>> before declaring it stable. For beta, we'll treat bugs with the same
>>> priority as "stable" (or at least close to)?
>>>
>>> Cheers
>>> Ben
>>>
>>>
>>>
>>> --
>>> *From:* Jon Haddad 
>>> *Sent:* 09 December 2024 09:43
>>> *To:* dev@cassandra.apache.org 
>>> *Subject:* Re: [DISCUSS] Experimental flagging (fork from Re-evaluate
>>> compaction defaults in 5.1/trunk)
>>>
>>> *EXTERNAL EMAIL - USE CAUTION when clicking links or attachments *
>>>
>>>
>>> I like this.  There's a few things marked as ex

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Josh McKenzie]

> 
>  * Filters on columns without a SAI index.
> 
> 
> 
> I do agree that there is differentiation also between experimental and beta. 
> But I need to think more before expressing concrete opinion/suggestions here. 
> Though I believe this conversation is healthy to have and shows the maturity 
> of our project. Thank you, Josh!
> 
> 
> 
> Best regards,
> 
> Ekaterina
> 
> 
> 
> On Mon, 9 Dec 2024 at 13:21, Jon Haddad  wrote:
>> The tough thing here is that MVs are marked experimental retroactively, 
>> because by the time the problems were known, there wasn't much anyone could 
>> do.  Experimental was our way of saying "oops, we screwed up, let's put a 
>> label on it" and the same label got applied to a bunch of new stuff 
>> including Java 17.  They're not even close to being in the same category, 
>> but they're labeled the same and people treat them as equivalent. 
>> 
>> If we knew MVs were so broken before they were merged, they would have been 
>> -1'ed.  Same with incremental repair (till 4.0), and vector search today.  I 
>> would have -1'ed all three of these if it was known how poorly they actually 
>> performed at the time they were committed.
>> 
>> Side note, vector search isn't marked as experimental today, but it's not 
>> even usable for non-trivial datasets out of the box, so it should be marked 
>> as such at this point.
>> 
>> I really wish this stuff was tested at a reasonable scale across various 
>> failure modes before merging, because the harm it does to the community is 
>> real.  We really shouldn't be put in a position where stuff gets released, 
>> hyped up, then we find it it's obviously not ready for real world use.  I 
>> built my tooling (tlp-cluster, now easy-cass-lab, and tlp-stress, now 
>> easy-cass-stress), with this in mind, but sadly I haven't seen much use of 
>> it it to verify patches.  The only reason I found a memory leak in 
>> CASSANDRA-15452 was because I used these tools on multi-TB datasets over 
>> several days.
>> 
>> 
>> Jon
>> 
>> 
>> On Mon, Dec 9, 2024 at 9:55 AM Slater, Ben via dev 
>>  wrote:
>>> I'm a little worried by the idea of grouping in MVs with things like a Java 
>>> version under the same "beta" label (acknowledging that they are currently 
>>> grouped under the same "experimental" label).
>>> 
>>> To me, "beta" implies it's pretty close to production ready and there is an 
>>> intention to get it to production ready in the near future. I don't think 
>>> this really describes MVs as I don't see anyone looking like they are 
>>> trying to get them to really production ready (although I could easily be 
>>> wrong on that).
>>> 
>>> Maybe there is an argument for "experimental"=this is here to get feedback 
>>> but there's no commitment it will make it to production ready and "beta"=we 
>>> think this is done but we'd like to see some production use before 
>>> declaring it stable. For beta, we'll treat bugs with the same priority as 
>>> "stable" (or at least close to)?
>>> 
>>> Cheers
>>> Ben
>>> 
>>> 
>>> 
>>> 
>>> 
>>> *From:* Jon Haddad 
>>> *Sent:* 09 December 2024 09:43
>>> *To:* dev@cassandra.apache.org 
>>> *Subject:* Re: [DISCUSS] Experimental flagging (fork from Re-evaluate 
>>> compaction defaults in 5.1/trunk)
>>>  
>>> *EXTERNAL EMAIL - USE CAUTION when clicking links or attachments*
>>> 
>>> 
>>> 
>>> I like this.  There's a few things marked as experimental today, so I'll 
>>> take a stab at making this more concrete, and I think we should be open to 
>>> graduating certain things out of beta to GA at a faster cycle than a major 
>>> release.
>>> 
>>> Java versions, for example, should really move out of "beta" quickly.  We 
>>> test against it, and we're not going to drop new versions.  So if we're 
>>> looking at C* 5.0, we should move Java 17 out of experimental / beta 
>>> immediately and call it GA.
>>> 
>>> SAI and UCS should probably graduate no later than 5.1.
>>> 
>>> On the other hand, MVs have enough warts I actively recommend against using 
>>> them and should be in beta till we can actually repair them.  
>>> 
>>> I don't know if anyone's actually

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Jeremy Hanna]

I think "experimental" isn't very helpful or useful except in the case of MVs.  
It's like when we said "unsafe assassinate endpoint" - it pretty much meant - 
don't use this unless you know what you're doing.  I don't think new features 
fall into that category.

As an example, the Java driver docs said that Java 17 support was experimental 
for a time (nb not the DB, the driver).  Some users had requirements to go to 
Java 17 at their companies but it was labeled experimental.  There were no 
known issues.  So there was no planned "once we have this fixed" or "once we 
have this tested."  So the flag is now gone and we are going to fix any 
problems that come up.

However I don't think "beta" is much better.  I don't think it should be the 
path for all new features - I think it should be a case-by-case basis.  Going 
to the next major/minor version in and of itself obviously doesn't make the 
feature more stable or more production ready.  We would need a plan to go from 
beta to GA presumably with "work out X deficiency before calling it GA." 

Should SAI be labeled beta?  Perhaps, but that actively discourages people from 
using it in production.  And if we say it will be out of beta for 5.1 or 
5.1.next, what do we concretely mean will be different?

In the case of UCS, is it in a beta state until we resolve the discussions 
around UX/documentation?  From a functional and production usage perspective, 
it has been the only compaction strategy available for users in DataStax's 
Astra managed database service in both traditional node based deployments and 
serverless for 5 years across thousands of databases.  While I agree that it's 
premature to call that a new default for tables, it seems overly cautious to me 
that it needs to pass through a beta period.  Let people opt into it and try it 
out and if something breaks, let's fix it.  I'm not sure of the value of using 
"beta" with it.

I want Apache Cassandra to be a stable database system with all of what that 
implies.  I just think with general rule like this for all new features we're 
swinging the pendulum a bit too far in the direction of caution.


> On Dec 9, 2024, at 11:54 AM, Slater, Ben via dev  
> wrote:
> 
> I'm a little worried by the idea of grouping in MVs with things like a Java 
> version under the same "beta" label (acknowledging that they are currently 
> grouped under the same "experimental" label).
> 
> To me, "beta" implies it's pretty close to production ready and there is an 
> intention to get it to production ready in the near future. I don't think 
> this really describes MVs as I don't see anyone looking like they are trying 
> to get them to really production ready (although I could easily be wrong on 
> that).
> 
> Maybe there is an argument for "experimental"=this is here to get feedback 
> but there's no commitment it will make it to production ready and "beta"=we 
> think this is done but we'd like to see some production use before declaring 
> it stable. For beta, we'll treat bugs with the same priority as "stable" (or 
> at least close to)?
> 
> Cheers
> Ben
> 
> 
> From: Jon Haddad mailto:j...@rustyrazorblade.com>>
> Sent: 09 December 2024 09:43
> To: dev@cassandra.apache.org <mailto:dev@cassandra.apache.org> 
> mailto:dev@cassandra.apache.org>>
> Subject: Re: [DISCUSS] Experimental flagging (fork from Re-evaluate 
> compaction defaults in 5.1/trunk)
>  
> EXTERNAL EMAIL - USE CAUTION when clicking links or attachments
> 
> 
> 
> I like this.  There's a few things marked as experimental today, so I'll take 
> a stab at making this more concrete, and I think we should be open to 
> graduating certain things out of beta to GA at a faster cycle than a major 
> release.
> 
> Java versions, for example, should really move out of "beta" quickly.  We 
> test against it, and we're not going to drop new versions.  So if we're 
> looking at C* 5.0, we should move Java 17 out of experimental / beta 
> immediately and call it GA.
> 
> SAI and UCS should probably graduate no later than 5.1.
> 
> On the other hand, MVs have enough warts I actively recommend against using 
> them and should be in beta till we can actually repair them.  
> 
> I don't know if anyone's actually used transient replication and if it's even 
> beta quality... that might actually warrant being called experimental still.
> 
> 'ALTER ... DROP COMPACT STORAGE' is flagged as experimental.  I'm not sure 
> what to do with this.  I advise people migrate their data for any Thrift -> 
> CQL

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Ekaterina Dimitrova]

Hey Jon,
The following quick test shows me that vector search is marked as
experimental (it is just not in cassandra.yaml as materialized views, etc)

cqlsh:k> CREATE TABLE t (pk int, str_val text, val vector,
PRIMARY KEY(pk));

cqlsh:k> CREATE CUSTOM INDEX ON t(val) USING 'StorageAttachedIndex';


Warnings :

SAI ANN indexes on vector columns are experimental and are not recommended
for production use.

They don't yet support SELECT queries with:

 * Consistency level higher than ONE/LOCAL_ONE.

 * Paging.

 * No LIMIT clauses.

 * PER PARTITION LIMIT clauses.

 * GROUP BY clauses.

 * Aggregation functions.

 * Filters on columns without a SAI index.


I do agree that there is differentiation also between experimental and
beta. But I need to think more before expressing concrete
opinion/suggestions here. Though I believe this conversation is healthy to
have and shows the maturity of our project. Thank you, Josh!


Best regards,

Ekaterina


On Mon, 9 Dec 2024 at 13:21, Jon Haddad  wrote:

> The tough thing here is that MVs are marked experimental retroactively,
> because by the time the problems were known, there wasn't much anyone could
> do.  Experimental was our way of saying "oops, we screwed up, let's put a
> label on it" and the same label got applied to a bunch of new stuff
> including Java 17.  They're not even close to being in the same category,
> but they're labeled the same and people treat them as equivalent.
>
> If we knew MVs were so broken before they were merged, they would have
> been -1'ed.  Same with incremental repair (till 4.0), and vector search
> today.  I would have -1'ed all three of these if it was known how poorly
> they actually performed at the time they were committed.
>
> Side note, vector search isn't marked as experimental today, but it's not
> even usable for non-trivial datasets out of the box, so it should be marked
> as such at this point.
>
> I really wish this stuff was tested at a reasonable scale across various
> failure modes before merging, because the harm it does to the community is
> real.  We really shouldn't be put in a position where stuff gets released,
> hyped up, then we find it it's obviously not ready for real world use.  I
> built my tooling (tlp-cluster, now easy-cass-lab, and tlp-stress, now
> easy-cass-stress), with this in mind, but sadly I haven't seen much use of
> it it to verify patches.  The only reason I found a memory leak in
> CASSANDRA-15452 was because I used these tools on multi-TB datasets over
> several days.
>
>
> Jon
>
>
> On Mon, Dec 9, 2024 at 9:55 AM Slater, Ben via dev <
> dev@cassandra.apache.org> wrote:
>
>> I'm a little worried by the idea of grouping in MVs with things like a
>> Java version under the same "beta" label (acknowledging that they are
>> currently grouped under the same "experimental" label).
>>
>> To me, "beta" implies it's pretty close to production ready and there is
>> an intention to get it to production ready in the near future. I don't
>> think this really describes MVs as I don't see anyone looking like they are
>> trying to get them to really production ready (although I could easily be
>> wrong on that).
>>
>> Maybe there is an argument for "experimental"=this is here to get
>> feedback but there's no commitment it will make it to production ready and
>> "beta"=we think this is done but we'd like to see some production use
>> before declaring it stable. For beta, we'll treat bugs with the same
>> priority as "stable" (or at least close to)?
>>
>> Cheers
>> Ben
>>
>>
>>
>> --
>> *From:* Jon Haddad 
>> *Sent:* 09 December 2024 09:43
>> *To:* dev@cassandra.apache.org 
>> *Subject:* Re: [DISCUSS] Experimental flagging (fork from Re-evaluate
>> compaction defaults in 5.1/trunk)
>>
>> *EXTERNAL EMAIL - USE CAUTION when clicking links or attachments *
>>
>>
>> I like this.  There's a few things marked as experimental today, so I'll
>> take a stab at making this more concrete, and I think we should be open to
>> graduating certain things out of beta to GA at a faster cycle than a major
>> release.
>>
>> Java versions, for example, should really move out of "beta" quickly.  We
>> test against it, and we're not going to drop new versions.  So if we're
>> looking at C* 5.0, we should move Java 17 out of experimental / beta
>> immediately and call it GA.
>>
>> SAI and UCS should probably graduate no later than 5.1.
>>

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Jon Haddad]

The tough thing here is that MVs are marked experimental retroactively,
because by the time the problems were known, there wasn't much anyone could
do.  Experimental was our way of saying "oops, we screwed up, let's put a
label on it" and the same label got applied to a bunch of new stuff
including Java 17.  They're not even close to being in the same category,
but they're labeled the same and people treat them as equivalent.

If we knew MVs were so broken before they were merged, they would have been
-1'ed.  Same with incremental repair (till 4.0), and vector search today.
I would have -1'ed all three of these if it was known how poorly they
actually performed at the time they were committed.

Side note, vector search isn't marked as experimental today, but it's not
even usable for non-trivial datasets out of the box, so it should be marked
as such at this point.

I really wish this stuff was tested at a reasonable scale across various
failure modes before merging, because the harm it does to the community is
real.  We really shouldn't be put in a position where stuff gets released,
hyped up, then we find it it's obviously not ready for real world use.  I
built my tooling (tlp-cluster, now easy-cass-lab, and tlp-stress, now
easy-cass-stress), with this in mind, but sadly I haven't seen much use of
it it to verify patches.  The only reason I found a memory leak in
CASSANDRA-15452 was because I used these tools on multi-TB datasets over
several days.

Jon


On Mon, Dec 9, 2024 at 9:55 AM Slater, Ben via dev 
wrote:

> I'm a little worried by the idea of grouping in MVs with things like a
> Java version under the same "beta" label (acknowledging that they are
> currently grouped under the same "experimental" label).
>
> To me, "beta" implies it's pretty close to production ready and there is
> an intention to get it to production ready in the near future. I don't
> think this really describes MVs as I don't see anyone looking like they are
> trying to get them to really production ready (although I could easily be
> wrong on that).
>
> Maybe there is an argument for "experimental"=this is here to get feedback
> but there's no commitment it will make it to production ready and "beta"=we
> think this is done but we'd like to see some production use before
> declaring it stable. For beta, we'll treat bugs with the same priority as
> "stable" (or at least close to)?
>
> Cheers
> Ben
>
>
>
> --
> *From:* Jon Haddad 
> *Sent:* 09 December 2024 09:43
> *To:* dev@cassandra.apache.org 
> *Subject:* Re: [DISCUSS] Experimental flagging (fork from Re-evaluate
> compaction defaults in 5.1/trunk)
>
> *EXTERNAL EMAIL - USE CAUTION when clicking links or attachments *
>
>
> I like this.  There's a few things marked as experimental today, so I'll
> take a stab at making this more concrete, and I think we should be open to
> graduating certain things out of beta to GA at a faster cycle than a major
> release.
>
> Java versions, for example, should really move out of "beta" quickly.  We
> test against it, and we're not going to drop new versions.  So if we're
> looking at C* 5.0, we should move Java 17 out of experimental / beta
> immediately and call it GA.
>
> SAI and UCS should probably graduate no later than 5.1.
>
> On the other hand, MVs have enough warts I actively recommend against
> using them and should be in beta till we can actually repair them.
>
> I don't know if anyone's actually used transient replication and if it's
> even beta quality... that might actually warrant being called experimental
> still.
>
> 'ALTER ... DROP COMPACT STORAGE' is flagged as experimental.  I'm not sure
> what to do with this.  I advise people migrate their data for any Thrift ->
> CQL cases, mostly because the edge cases are so hard to know in advance,
> especially since by now these codebases are ancient and the original
> developers are long gone.
>
> Thoughts?
>
> Jon
>
>
>
>
> On Mon, Dec 9, 2024 at 6:28 AM Josh McKenzie  wrote:
>
> Jon stated:
>
> Side note: I think experimental has been over-used and has lost all
> meaning.  How is Java 17 experimental?  Very confusing for the community.
>
>
> Dinesh followed with:
>
> Philosophically, as a project, we should wait until critical features like
> these reach a certain level of maturity prior to recommending it as a
> default. For me maturity is a function of adoption by diverse use-cases in
> production and scale.
>
>
> I'd like to discuss 2 ideas related to the above:
>
>1. We renam

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Slater, Ben via dev]

I'm a little worried by the idea of grouping in MVs with things like a Java 
version under the same "beta" label (acknowledging that they are currently 
grouped under the same "experimental" label).

To me, "beta" implies it's pretty close to production ready and there is an 
intention to get it to production ready in the near future. I don't think this 
really describes MVs as I don't see anyone looking like they are trying to get 
them to really production ready (although I could easily be wrong on that).

Maybe there is an argument for "experimental"=this is here to get feedback but 
there's no commitment it will make it to production ready and "beta"=we think 
this is done but we'd like to see some production use before declaring it 
stable. For beta, we'll treat bugs with the same priority as "stable" (or at 
least close to)?

Cheers
Ben


________
From: Jon Haddad 
Sent: 09 December 2024 09:43
To: dev@cassandra.apache.org 
Subject: Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction 
defaults in 5.1/trunk)

EXTERNAL EMAIL - USE CAUTION when clicking links or attachments



I like this.  There's a few things marked as experimental today, so I'll take a 
stab at making this more concrete, and I think we should be open to graduating 
certain things out of beta to GA at a faster cycle than a major release.

Java versions, for example, should really move out of "beta" quickly.  We test 
against it, and we're not going to drop new versions.  So if we're looking at 
C* 5.0, we should move Java 17 out of experimental / beta immediately and call 
it GA.

SAI and UCS should probably graduate no later than 5.1.

On the other hand, MVs have enough warts I actively recommend against using 
them and should be in beta till we can actually repair them.

I don't know if anyone's actually used transient replication and if it's even 
beta quality... that might actually warrant being called experimental still.

'ALTER ... DROP COMPACT STORAGE' is flagged as experimental.  I'm not sure what 
to do with this.  I advise people migrate their data for any Thrift -> CQL 
cases, mostly because the edge cases are so hard to know in advance, especially 
since by now these codebases are ancient and the original developers are long 
gone.

Thoughts?

Jon




On Mon, Dec 9, 2024 at 6:28 AM Josh McKenzie 
mailto:jmcken...@apache.org>> wrote:
Jon stated:
Side note: I think experimental has been over-used and has lost all meaning.  
How is Java 17 experimental?  Very confusing for the community.

Dinesh followed with:
Philosophically, as a project, we should wait until critical features like 
these reach a certain level of maturity prior to recommending it as a default. 
For me maturity is a function of adoption by diverse use-cases in production 
and scale.

I'd like to discuss 2 ideas related to the above:

  1.  We rename / alias "experimental" to "beta". It's a word that's ubiquitous 
in our field and communicates the correct level of expectation to our users 
(API stable, may have bugs)
  2.  All new features go through one major (either semver MAJOR or MINOR) as 
"beta"

To Jon's point, "experimental" was really a kludge to work around Materialized 
Views having some very sharp edges that users had to be very aware of. We 
haven't really used the flagging much (at all?) since then, and we don't have a 
formalized way to shepherd a new feature through a "soak" period where it can 
"reach a certain level of maturity". We're caught in a chicken-or-egg scenario 
with our current need to get a feature released more broadly to have confidence 
in its stability (to Dinesh's point).

In my mind, the following feature evolution would be healthy for us and good 
for our users:

  1.  Beta
  2.  Generally Available
  3.  Default (where appropriate)

To graduate from Beta -> GA, good UX, user facing documentation, a [DISCUSS] 
thread where we have a clear consensus of readiness, all seem like healthy and 
good steps. From GA -> Default, [DISCUSS] like we're having re: compaction 
strategies, unearthing shortcomings, edge-cases, documentation needs, etc.

Curious what others think.

~Josh

Re: [DISCUSS] Experimental flagging (fork from Re-evaluate compaction defaults in 5.1/trunk)

[Jon Haddad]

I like this.  There's a few things marked as experimental today, so I'll
take a stab at making this more concrete, and I think we should be open to
graduating certain things out of beta to GA at a faster cycle than a major
release.

Java versions, for example, should really move out of "beta" quickly.  We
test against it, and we're not going to drop new versions.  So if we're
looking at C* 5.0, we should move Java 17 out of experimental / beta
immediately and call it GA.

SAI and UCS should probably graduate no later than 5.1.

On the other hand, MVs have enough warts I actively recommend against using
them and should be in beta till we can actually repair them.

I don't know if anyone's actually used transient replication and if it's
even beta quality... that might actually warrant being called experimental
still.

'ALTER ... DROP COMPACT STORAGE' is flagged as experimental.  I'm not sure
what to do with this.  I advise people migrate their data for any Thrift ->
CQL cases, mostly because the edge cases are so hard to know in advance,
especially since by now these codebases are ancient and the original
developers are long gone.

Thoughts?

Jon




On Mon, Dec 9, 2024 at 6:28 AM Josh McKenzie  wrote:

> Jon stated:
>
> Side note: I think experimental has been over-used and has lost all
> meaning.  How is Java 17 experimental?  Very confusing for the community.
>
>
> Dinesh followed with:
>
> Philosophically, as a project, we should wait until critical features like
> these reach a certain level of maturity prior to recommending it as a
> default. For me maturity is a function of adoption by diverse use-cases in
> production and scale.
>
>
> I'd like to discuss 2 ideas related to the above:
>
>1. We rename / alias "experimental" to "beta". It's a word that's
>ubiquitous in our field and communicates the correct level of expectation
>to our users (API stable, may have bugs)
>2. *All new features* go through one major (either semver MAJOR or
>MINOR) as "beta"
>
>
> To Jon's point, "experimental" was really a kludge to work around
> Materialized Views having some very sharp edges that users had to be very
> aware of. We haven't really used the flagging much (at all?) since then,
> and we don't have a formalized way to shepherd a new feature through a
> "soak" period where it can "reach a certain level of maturity". We're
> caught in a chicken-or-egg scenario with our current need to get a feature
> released more broadly to have confidence in its stability (to Dinesh's
> point).
>
> In my mind, the following feature evolution would be healthy for us and
> good for our users:
>
>1. Beta
>2. Generally Available
>3. Default (where appropriate)
>
> To graduate from Beta -> GA, good UX, user facing documentation, a
> [DISCUSS] thread where we have a clear consensus of readiness, all seem
> like healthy and good steps. From GA -> Default, [DISCUSS] like we're
> having re: compaction strategies, unearthing shortcomings, edge-cases,
> documentation needs, etc.
>
> Curious what others think.
>
> ~Josh
>