I humbly suggest these are the wrong questions to ask. Instead, two sides of
just one question matter: how did we miss these problems, and what would we
have needed to do procedurally to have not missed it. Whatever it is, we need
to do it now to have confidence other things were not missed, as well as for
all future features.
We should start by producing a list of what we think is necessary for deploying
successful features. We can then determine what items are missing that would
have been needed to catch a problem. Obvious things are:
* integration tests at scale
* integration tests with a variety of extreme workloads
* integration tests with various cluster topologies
* data integrity tests as part of the above
* all of the above as reproducible tests incorporated into the source tree
We can then ensure Jira accurately represents all of the known issues with MVs
(and other features). This includes those that are poorly defined (such as
"doesn't scale").
Then we can look at all issues and ask: would this approach have caught it, and
if not what do we need to add to the guidelines to prevent a recurrence - and
also ensure this problem is unique? In future we can ask, for bugs found in
features built to these guidelines: why didn't it catch this bug? Do the
guidelines need additional items, or greater specificity about how to meet
given criteria?
I do not think that data from deployments - even if reliably obtained - can
tell us much besides which problems we prioritise.
On 01/07/2020, 01:58, "joshua.mcken...@gmail.com" <joshua.mcken...@gmail.com>
wrote:
It would be incredibly helpful for us to have some empirical data and
agreed upon terms and benchmarks to help us navigate discussions like this:
* How widely used is a feature in C* deployments worldwide?
* What are the primary issues users face when deploying them? Scaling
them? During failure scenarios?
* What does the engineering effort to bridge these gaps look like? Who
will do that? On what time horizon?
* What does our current test coverage for this feature look like?
* What shape of defects are arising with the feature? In a specific
subsection of the module or usage?
* Do we have an agreed upon set of standards for labeling a feature
stable? As experimental? If not, how do we get there?
* What effort will it take to bridge from where we are to where we agree
we need to be? On what timeline is this acceptable?
I believe these are not only answerable questions, but fundamentally the
underlying themes our discussion alludes to. They’re also questions that apply
to a lot more than just MV’s and tie into what you’re speaking to above
Benedict.
> On Jun 30, 2020, at 8:32 PM, sankalp kohli <kohlisank...@gmail.com> wrote:
>
> I see this discussion as several decisions which can be made in small
> increments.
>
> 1. In release cycles, when can we propose a feature to be deprecated or
> marked experimental. Ideally a new feature should come out experimental if
> required but we have several who are candidates now. We can work on
> integrating this in the release lifecycle doc we already have.
> 2. What is the process of making an existing feature experimental? How
does
> it affect major releases around testing.
> 3. What is the process of deprecating/removing an experimental feature.
> (Assuming experimental features should be deprecated/removed)
>
> Coming to MV, I think we need more data before we can say we
> should deprecate MV. Here are some of them which should be part of
> deprecation process
> 1.Talk to customers who use them and understand what is the impact. Give
> them a forum to talk about it.
> 2. Do we have enough resources to bring this feature out of the
> experimental feature list in next 1 or 2 major releases. We cannot have
too
> many experimental features in the database. Marking a feature experimental
> should not be a parking place for a non functioning feature but a place
> while we stabilize it.
>
>
>
>
>> On Tue, Jun 30, 2020 at 4:52 PM <joshua.mcken...@gmail.com> wrote:
>>
>> I followed up with the clarification about unit and dtests for that
reason
>> Dinesh. We test experimental features now.
>>
>> If we’re talking about adding experimental features to the 40 quality
>> testing effort, how does that differ from just saying “we won’t release
>> until we’ve tested and stabilized these features and they’re no longer
>> experimental”?
>>
>> Maybe I’m just misunderstanding something here?
>>
>>>> On Jun 30, 2020, at 7:12 PM, Dinesh Joshi <djo...@apache.org> wrote:
>>>
>>>
>>>>
>>>> On Jun 30, 2020, at 4:05 PM, Brandon Williams <dri...@gmail.com> wrote:
>>>>
>>>> Instead of ripping it out, we could instead disable them in the yaml
>>>> with big fat warning comments around it. That way people already
>>>> using them can just enable them again, but it will raise the bar for
>>>> new users who ignore/miss the warnings in the logs and just use them.
>>>
>>> Not a bad idea. Although, the real issue is that users enable MV on a 3
>> node cluster with a few megs of data and conclude that MVs will
>> horizontally scale with the size of data. This is what causes issues for
>> users who naively roll it out in production and discover that MVs do not
>> scale with their data growth. So whatever we do, the big fat warning
should
>> educate the unsuspecting operator.
>>>
>>> Dinesh
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>>> For additional commands, e-mail: dev-h...@cassandra.apache.org
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>> For additional commands, e-mail: dev-h...@cassandra.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org
