Re: Stability of MaterializedView in 3.11.x | 4.0

[Dor Laor]

On Wed, Aug 28, 2019 at 5:43 PM Jon Haddad  wrote:

> >  Arguably, the other alternative to server-side denormalization is to do
> the denormalization client-side which comes with the same axes of costs and
> complexity, just with more of each.
>
> That's not completely true.  You can write to any number of tables without
> doing a read, and the cost of reading data off disk is significantly
> greater than an insert alone.  You can crush a cluster with a write heavy
> workload and MVs that would otherwise be completely fine to do all writes.
>
> The other issue with MVs is that you still need to understand fundamentals
> of data modeling, that don't magically solve the problem of enormous
> partitions.  One of the reasons I've had to un-MV a lot of clusters is
> because people have put an MV on a table with a low-cardinality field and
> found themselves with a 10GB partition nightmare, so they need to go back
> and remodel the view as something more complex anyways.  In this case, the
> MV was extremely high cost since now they've not only pushed out a poor
> implementation to begin with but now have the cost of a migration as well
> as a rewrite.
>

+1

Moreover, the hard part is that an update for the base table means that
the original data needs to be read and the database (or the poor developer
who implements the denormalized model) needs to delete the data in the view
and then to write the new ones. All need to be of course resilient to all
types of
errors and failures. Had it been simple, there was no need for a database
MV..


>
>
>
> On Wed, Aug 28, 2019 at 9:58 AM Joshua McKenzie 
> wrote:
>
> > >
> > > so we need to start migration from MVs to manual query base table ?
> >
> >  Arguably, the other alternative to server-side denormalization is to do
> > the denormalization client-side which comes with the same axes of costs
> and
> > complexity, just with more of each.
> >
> > Jeff's spot on when he discusses the risk appetite vs. mitigation aspect
> of
> > it. There's a reason banks do end-of-day close-out validation analysis
> and
> > have redundant systems for things like this.
> >
> > On Wed, Aug 28, 2019 at 11:49 AM Jon Haddad  wrote:
> >
> > > I've helped a lot of teams (a dozen to two dozen maybe) migrate away
> from
> > > MVs due to inconsistencies, issues with streaming (have you added or
> > > removed nodes yet?), and massive performance issues to the point of
> > cluster
> > > failure under (what I consider) trivial load.  I haven't gone too deep
> > into
> > > analyzing their issues, folks are usually fine with "move off them", vs
> > > having me do a ton of analysis.
> > >
> > > tlp-stress has a materialized view workload built in, and you can add
> > > arbitrary CQL via the --cql flag to add a MV to any existing workload
> > such
> > > as KeyValue or BasicTimeSeries.
> > >
> > > On Wed, Aug 28, 2019 at 8:11 AM Jeff Jirsa  wrote:
> > >
> > > > There have been people who have had operational issues related to MVs
> > > (many
> > > > of them around running repair), but the biggest concern is
> correctness.
> > > >
> > > > It probably ultimately depends on what type of database you're
> running.
> > > If
> > > > you're running some sort of IOT / analytics workload and you just
> want
> > > > another way to SELECT the data, but you won't notice one of a billion
> > > > records going missing, using MVs may be fine. If you're a bank, and
> one
> > > of
> > > > a billion records going missing means you lose someone's bank
> account,
> > I
> > > > would avoid using MVs.
> > > >
> > > > It's all just risk management.
> > > >
> > > > On Wed, Aug 28, 2019 at 7:18 AM Pankaj Gajjar <
> > > > pankaj.gaj...@contentserv.com>
> > > > wrote:
> > > >
> > > > > Hi Michael,
> > > > >
> > > > > Thanks for putting very clever information " Users of MVs *must*
> > > > determine
> > > > > for themselves, through
> > > > > thorough testing and understanding, if they wish to use them."
> > And
> > > > > this concluded that if there is any issue occur in future then only
> > > > > solution is to rebuild the MVs since Cassandra does not able to
> make
> > > > > consistent synch well.
> > > > >
> > > > > Also, we practically using the 10+ MVs and as of now, we have not
> > faced
> > > > > any issue, so my question to all community member, does anyone face
> > any
> > > > > critical issues ? so we need to start migration from MVs to manual
> > > query
> > > > > base table ?
> > > > >
> > > > > Also, I can understand now, it's experimental and not ready for
> > > > > production, so if possible, please ignore it only right ?
> > > > >
> > > > > Thanks
> > > > > Pankaj
> > > > >
> > > > > On 27/08/19, 19:03, "Michael Shuler"  > > behalf
> > > > > of mich...@pbandjelly.org> wrote:
> > > > >
> > > > > It appears that you found the first message of the chain. I
> > suggest
> > > > > reading the linked JIRA and the complete dev@ thread that
> > arrived
> > > at
> > > > > this conclusion; there are loads 

Re: 4.0 alpha before apachecon?

[Michael Shuler]

Thanks for the reminder :) I have a few days of availability to prep a 
4.0 alpha release. It's an alpha, so I don't have a problem with known 
issues needing work.


I will have an internet-less period of time starting roughly Tuesday 9/3 
through about Friday 9/13. I might get lucky and have a little network 
access in the middle of that time, but I'm not counting on it.


--
Michael

On 8/28/19 10:51 AM, Jon Haddad wrote:

Hey folks,

I think it's time we cut a 4.0 alpha release.  Before I put up a vote
thread, is there a reason not to have a 4.0 alpha before ApacheCon /
Cassandra Summit?

There's a handful of small issues that I should be done for 4.0 (client
list in virtual tables, dynamic snitch improvements, fixing token counts),
I'm not trying to suggest we don't include them, but they're small enough I
think it's OK to merge them in following the first alpha.

Jon



-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: 4.0 alpha before apachecon?

[Jon Haddad]

Yes we do.  It's one of the reasons I've spent about a lot of (thousands?)
hours working on tlp-stress and tlp-cluster in the last 2 years.  I shared
some progress on this a little ways back.  I'll send out a separate email
soon with updates, since we just merged in a *lot* of features that will
help with testing.

On Wed, Aug 28, 2019 at 10:52 AM Dinesh Joshi  wrote:

> +1 on cutting an alpha and having a clear, documented test plan[1] for
> alpha. We need volunteers to drive the test plan, though. :)
>
> Thanks,
>
> Dinesh
>
> [1]
> https://cwiki.apache.org/confluence/display/CASSANDRA/4.0+Quality%3A+Components+and+Test+Plans
>
> > On Aug 28, 2019, at 10:27 AM, Jon Haddad  wrote:
> >
> > Regarding the dynamic snitch improvements, it's gone through several
> rounds
> > of review already and there's been significant testing of it.  Regarding
> > the token change, switching a number from 256 -> 16 isn't so invasive
> that
> > we shouldn't do it.  There's a little extra work that needs to be done
> > there ideally to ensure safety, but it's again small enough where it
> > shouldn't be too big of a problem imo.  Both current implementations (256
> > tokens + our insanely over memory allocating dynamic snitch) limit the
> > ability of people to run large clusters, harming both availability and
> > performance.  It's been extremely harmful for Cassandra's reputation and
> > I'd really like it if we could ship something where I don't have to
> > constantly apologize to people I'm trying to help for the land mine
> > defaults we put out there.
> >
> > To your point, I agree as a community we're lacking in an open, well
> > documented and up to date plan, and it needs to be addressed.  I think
> the
> > virtual meetings idea held at a regular might help a bit with that, I
> > intend on participating there.
> >
> >
> > On Wed, Aug 28, 2019 at 9:52 AM Joshua McKenzie 
> > wrote:
> >
> >>>
> >>> dynamic snitch improvements, fixing token counts
> >>
> >>
> >>
> >>> they're small enough
> >>
> >>
> >> By what axis of measurement out of curiosity? Risk to re-test and
> validate
> >> a final artifact? Do we have a more clear understanding of what testing
> has
> >> taken place across the community?
> >>
> >> The last I saw, our documented test plan
> >> <
> >>
> https://cwiki.apache.org/confluence/display/CASSANDRA/4.0+Quality%3A+Components+and+Test+Plans
> >>>
> >> hasn't
> >> been maintained or kept up to date
> >> <
> >>
> https://issues.apache.org/jira/browse/CASSANDRA-14862?jql=project%20%3D%20CASSANDRA%20AND%20%20labels%20%3D%204.0-QA
> >>> .
> >> Is there another artifact reflecting what testing people have in flight
> to
> >> better reflect what risk of needing to re-test we have from these (and
> >> other) post-freeze changes?
> >>
> >>
> >>
> >> On Wed, Aug 28, 2019 at 11:52 AM Jon Haddad  wrote:
> >>
> >>> Hey folks,
> >>>
> >>> I think it's time we cut a 4.0 alpha release.  Before I put up a vote
> >>> thread, is there a reason not to have a 4.0 alpha before ApacheCon /
> >>> Cassandra Summit?
> >>>
> >>> There's a handful of small issues that I should be done for 4.0 (client
> >>> list in virtual tables, dynamic snitch improvements, fixing token
> >> counts),
> >>> I'm not trying to suggest we don't include them, but they're small
> >> enough I
> >>> think it's OK to merge them in following the first alpha.
> >>>
> >>> Jon
> >>>
> >>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: 4.0 alpha before apachecon?

[Dinesh Joshi]

+1 on cutting an alpha and having a clear, documented test plan[1] for alpha. 
We need volunteers to drive the test plan, though. :)

Thanks,

Dinesh

[1] 
https://cwiki.apache.org/confluence/display/CASSANDRA/4.0+Quality%3A+Components+and+Test+Plans

> On Aug 28, 2019, at 10:27 AM, Jon Haddad  wrote:
> 
> Regarding the dynamic snitch improvements, it's gone through several rounds
> of review already and there's been significant testing of it.  Regarding
> the token change, switching a number from 256 -> 16 isn't so invasive that
> we shouldn't do it.  There's a little extra work that needs to be done
> there ideally to ensure safety, but it's again small enough where it
> shouldn't be too big of a problem imo.  Both current implementations (256
> tokens + our insanely over memory allocating dynamic snitch) limit the
> ability of people to run large clusters, harming both availability and
> performance.  It's been extremely harmful for Cassandra's reputation and
> I'd really like it if we could ship something where I don't have to
> constantly apologize to people I'm trying to help for the land mine
> defaults we put out there.
> 
> To your point, I agree as a community we're lacking in an open, well
> documented and up to date plan, and it needs to be addressed.  I think the
> virtual meetings idea held at a regular might help a bit with that, I
> intend on participating there.
> 
> 
> On Wed, Aug 28, 2019 at 9:52 AM Joshua McKenzie 
> wrote:
> 
>>> 
>>> dynamic snitch improvements, fixing token counts
>> 
>> 
>> 
>>> they're small enough
>> 
>> 
>> By what axis of measurement out of curiosity? Risk to re-test and validate
>> a final artifact? Do we have a more clear understanding of what testing has
>> taken place across the community?
>> 
>> The last I saw, our documented test plan
>> <
>> https://cwiki.apache.org/confluence/display/CASSANDRA/4.0+Quality%3A+Components+and+Test+Plans
>>> 
>> hasn't
>> been maintained or kept up to date
>> <
>> https://issues.apache.org/jira/browse/CASSANDRA-14862?jql=project%20%3D%20CASSANDRA%20AND%20%20labels%20%3D%204.0-QA
>>> .
>> Is there another artifact reflecting what testing people have in flight to
>> better reflect what risk of needing to re-test we have from these (and
>> other) post-freeze changes?
>> 
>> 
>> 
>> On Wed, Aug 28, 2019 at 11:52 AM Jon Haddad  wrote:
>> 
>>> Hey folks,
>>> 
>>> I think it's time we cut a 4.0 alpha release.  Before I put up a vote
>>> thread, is there a reason not to have a 4.0 alpha before ApacheCon /
>>> Cassandra Summit?
>>> 
>>> There's a handful of small issues that I should be done for 4.0 (client
>>> list in virtual tables, dynamic snitch improvements, fixing token
>> counts),
>>> I'm not trying to suggest we don't include them, but they're small
>> enough I
>>> think it's OK to merge them in following the first alpha.
>>> 
>>> Jon
>>> 
>> 


-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Stability of MaterializedView in 3.11.x | 4.0

[Jon Haddad]

>  Arguably, the other alternative to server-side denormalization is to do
the denormalization client-side which comes with the same axes of costs and
complexity, just with more of each.

That's not completely true.  You can write to any number of tables without
doing a read, and the cost of reading data off disk is significantly
greater than an insert alone.  You can crush a cluster with a write heavy
workload and MVs that would otherwise be completely fine to do all writes.

The other issue with MVs is that you still need to understand fundamentals
of data modeling, that don't magically solve the problem of enormous
partitions.  One of the reasons I've had to un-MV a lot of clusters is
because people have put an MV on a table with a low-cardinality field and
found themselves with a 10GB partition nightmare, so they need to go back
and remodel the view as something more complex anyways.  In this case, the
MV was extremely high cost since now they've not only pushed out a poor
implementation to begin with but now have the cost of a migration as well
as a rewrite.



On Wed, Aug 28, 2019 at 9:58 AM Joshua McKenzie 
wrote:

> >
> > so we need to start migration from MVs to manual query base table ?
>
>  Arguably, the other alternative to server-side denormalization is to do
> the denormalization client-side which comes with the same axes of costs and
> complexity, just with more of each.
>
> Jeff's spot on when he discusses the risk appetite vs. mitigation aspect of
> it. There's a reason banks do end-of-day close-out validation analysis and
> have redundant systems for things like this.
>
> On Wed, Aug 28, 2019 at 11:49 AM Jon Haddad  wrote:
>
> > I've helped a lot of teams (a dozen to two dozen maybe) migrate away from
> > MVs due to inconsistencies, issues with streaming (have you added or
> > removed nodes yet?), and massive performance issues to the point of
> cluster
> > failure under (what I consider) trivial load.  I haven't gone too deep
> into
> > analyzing their issues, folks are usually fine with "move off them", vs
> > having me do a ton of analysis.
> >
> > tlp-stress has a materialized view workload built in, and you can add
> > arbitrary CQL via the --cql flag to add a MV to any existing workload
> such
> > as KeyValue or BasicTimeSeries.
> >
> > On Wed, Aug 28, 2019 at 8:11 AM Jeff Jirsa  wrote:
> >
> > > There have been people who have had operational issues related to MVs
> > (many
> > > of them around running repair), but the biggest concern is correctness.
> > >
> > > It probably ultimately depends on what type of database you're running.
> > If
> > > you're running some sort of IOT / analytics workload and you just want
> > > another way to SELECT the data, but you won't notice one of a billion
> > > records going missing, using MVs may be fine. If you're a bank, and one
> > of
> > > a billion records going missing means you lose someone's bank account,
> I
> > > would avoid using MVs.
> > >
> > > It's all just risk management.
> > >
> > > On Wed, Aug 28, 2019 at 7:18 AM Pankaj Gajjar <
> > > pankaj.gaj...@contentserv.com>
> > > wrote:
> > >
> > > > Hi Michael,
> > > >
> > > > Thanks for putting very clever information " Users of MVs *must*
> > > determine
> > > > for themselves, through
> > > > thorough testing and understanding, if they wish to use them."
> And
> > > > this concluded that if there is any issue occur in future then only
> > > > solution is to rebuild the MVs since Cassandra does not able to make
> > > > consistent synch well.
> > > >
> > > > Also, we practically using the 10+ MVs and as of now, we have not
> faced
> > > > any issue, so my question to all community member, does anyone face
> any
> > > > critical issues ? so we need to start migration from MVs to manual
> > query
> > > > base table ?
> > > >
> > > > Also, I can understand now, it's experimental and not ready for
> > > > production, so if possible, please ignore it only right ?
> > > >
> > > > Thanks
> > > > Pankaj
> > > >
> > > > On 27/08/19, 19:03, "Michael Shuler"  > behalf
> > > > of mich...@pbandjelly.org> wrote:
> > > >
> > > > It appears that you found the first message of the chain. I
> suggest
> > > > reading the linked JIRA and the complete dev@ thread that
> arrived
> > at
> > > > this conclusion; there are loads of well formed opinions and
> > > > information. Users of MVs *must* determine for themselves,
> through
> > > > thorough testing and understanding, if they wish to use them.
> > > >
> > > > Linkage:
> > > > https://issues.apache.org/jira/browse/CASSANDRA-13959
> > > >   (sub-linkage..)
> > > >   https://issues.apache.org/jira/browse/CASSANDRA-13595
> > > >   https://issues.apache.org/jira/browse/CASSANDRA-13911
> > > >   https://issues.apache.org/jira/browse/CASSANDRA-13880
> > > >   https://issues.apache.org/jira/browse/CASSANDRA-12872
> > > >   https://issues.apache.org/jira/browse/CASSANDRA-13747
> > > >
> > > > Very 

Re: 4.0 alpha before apachecon?

[Jon Haddad]

Regarding the dynamic snitch improvements, it's gone through several rounds
of review already and there's been significant testing of it.  Regarding
the token change, switching a number from 256 -> 16 isn't so invasive that
we shouldn't do it.  There's a little extra work that needs to be done
there ideally to ensure safety, but it's again small enough where it
shouldn't be too big of a problem imo.  Both current implementations (256
tokens + our insanely over memory allocating dynamic snitch) limit the
ability of people to run large clusters, harming both availability and
performance.  It's been extremely harmful for Cassandra's reputation and
I'd really like it if we could ship something where I don't have to
constantly apologize to people I'm trying to help for the land mine
defaults we put out there.

To your point, I agree as a community we're lacking in an open, well
documented and up to date plan, and it needs to be addressed.  I think the
virtual meetings idea held at a regular might help a bit with that, I
intend on participating there.


On Wed, Aug 28, 2019 at 9:52 AM Joshua McKenzie 
wrote:

> >
> > dynamic snitch improvements, fixing token counts
>
>
>
> > they're small enough
>
>
> By what axis of measurement out of curiosity? Risk to re-test and validate
> a final artifact? Do we have a more clear understanding of what testing has
> taken place across the community?
>
> The last I saw, our documented test plan
> <
> https://cwiki.apache.org/confluence/display/CASSANDRA/4.0+Quality%3A+Components+and+Test+Plans
> >
> hasn't
> been maintained or kept up to date
> <
> https://issues.apache.org/jira/browse/CASSANDRA-14862?jql=project%20%3D%20CASSANDRA%20AND%20%20labels%20%3D%204.0-QA
> >.
> Is there another artifact reflecting what testing people have in flight to
> better reflect what risk of needing to re-test we have from these (and
> other) post-freeze changes?
>
>
>
> On Wed, Aug 28, 2019 at 11:52 AM Jon Haddad  wrote:
>
> > Hey folks,
> >
> > I think it's time we cut a 4.0 alpha release.  Before I put up a vote
> > thread, is there a reason not to have a 4.0 alpha before ApacheCon /
> > Cassandra Summit?
> >
> > There's a handful of small issues that I should be done for 4.0 (client
> > list in virtual tables, dynamic snitch improvements, fixing token
> counts),
> > I'm not trying to suggest we don't include them, but they're small
> enough I
> > think it's OK to merge them in following the first alpha.
> >
> > Jon
> >
>

Re: Stability of MaterializedView in 3.11.x | 4.0

[Joshua McKenzie]

>
> so we need to start migration from MVs to manual query base table ?

 Arguably, the other alternative to server-side denormalization is to do
the denormalization client-side which comes with the same axes of costs and
complexity, just with more of each.

Jeff's spot on when he discusses the risk appetite vs. mitigation aspect of
it. There's a reason banks do end-of-day close-out validation analysis and
have redundant systems for things like this.

On Wed, Aug 28, 2019 at 11:49 AM Jon Haddad  wrote:

> I've helped a lot of teams (a dozen to two dozen maybe) migrate away from
> MVs due to inconsistencies, issues with streaming (have you added or
> removed nodes yet?), and massive performance issues to the point of cluster
> failure under (what I consider) trivial load.  I haven't gone too deep into
> analyzing their issues, folks are usually fine with "move off them", vs
> having me do a ton of analysis.
>
> tlp-stress has a materialized view workload built in, and you can add
> arbitrary CQL via the --cql flag to add a MV to any existing workload such
> as KeyValue or BasicTimeSeries.
>
> On Wed, Aug 28, 2019 at 8:11 AM Jeff Jirsa  wrote:
>
> > There have been people who have had operational issues related to MVs
> (many
> > of them around running repair), but the biggest concern is correctness.
> >
> > It probably ultimately depends on what type of database you're running.
> If
> > you're running some sort of IOT / analytics workload and you just want
> > another way to SELECT the data, but you won't notice one of a billion
> > records going missing, using MVs may be fine. If you're a bank, and one
> of
> > a billion records going missing means you lose someone's bank account, I
> > would avoid using MVs.
> >
> > It's all just risk management.
> >
> > On Wed, Aug 28, 2019 at 7:18 AM Pankaj Gajjar <
> > pankaj.gaj...@contentserv.com>
> > wrote:
> >
> > > Hi Michael,
> > >
> > > Thanks for putting very clever information " Users of MVs *must*
> > determine
> > > for themselves, through
> > > thorough testing and understanding, if they wish to use them." And
> > > this concluded that if there is any issue occur in future then only
> > > solution is to rebuild the MVs since Cassandra does not able to make
> > > consistent synch well.
> > >
> > > Also, we practically using the 10+ MVs and as of now, we have not faced
> > > any issue, so my question to all community member, does anyone face any
> > > critical issues ? so we need to start migration from MVs to manual
> query
> > > base table ?
> > >
> > > Also, I can understand now, it's experimental and not ready for
> > > production, so if possible, please ignore it only right ?
> > >
> > > Thanks
> > > Pankaj
> > >
> > > On 27/08/19, 19:03, "Michael Shuler"  behalf
> > > of mich...@pbandjelly.org> wrote:
> > >
> > > It appears that you found the first message of the chain. I suggest
> > > reading the linked JIRA and the complete dev@ thread that arrived
> at
> > > this conclusion; there are loads of well formed opinions and
> > > information. Users of MVs *must* determine for themselves, through
> > > thorough testing and understanding, if they wish to use them.
> > >
> > > Linkage:
> > > https://issues.apache.org/jira/browse/CASSANDRA-13959
> > >   (sub-linkage..)
> > >   https://issues.apache.org/jira/browse/CASSANDRA-13595
> > >   https://issues.apache.org/jira/browse/CASSANDRA-13911
> > >   https://issues.apache.org/jira/browse/CASSANDRA-13880
> > >   https://issues.apache.org/jira/browse/CASSANDRA-12872
> > >   https://issues.apache.org/jira/browse/CASSANDRA-13747
> > >
> > > Very much worth reading the complete thread:
> > > part1:
> > >
> > >
> >
> https://lists.apache.org/thread.html/d81a61da48e1b872d7599df4edfa8e244d34cbd591a18539f724796f@
> > > 
> > > part2:
> > >
> > >
> >
> https://lists.apache.org/thread.html/19b7fcfd3b47f1526d6e993b3bb97f6c43e5ce204bc976ec0701cdd3@
> > > 
> > >
> > > Quick JQL for open tickets with "mv":
> > >
> > >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20CASSANDRA%20AND%20text%20~%20mv%20AND%20status%20!%3D%20Resolved
> > >
> > > --
> > > Michael
> > >
> > > On 8/27/19 5:01 AM, pankaj gajjar wrote:
> > > > Hello,
> > > >
> > > >
> > > >
> > > > concern about Materialized Views (MVs) in Cassandra.
> Unfortunately
> > > starting
> > > > with version 3.11, MVs are officially considered experimental and
> > > not ready
> > > > for production use, as you can read here:
> > > >
> > > >
> > > >
> > > >
> > >
> >
> http://mail-archives.apache.org/mod_mbox/cassandra-user/201710.mbox/%3cetpan.59f24f38.438f4e99.7...@apple.com%3E
> > > >
> > > >
> > > >
> > > > Can you please someone give some productive feedback on this ? it
> > > would
> > > > help us to further implementation around the MVs in Cassandra.
> > > >
> > > >
> > > >
> > > > Does 

Re: 4.0 alpha before apachecon?

[Joshua McKenzie]

>
> dynamic snitch improvements, fixing token counts



> they're small enough


By what axis of measurement out of curiosity? Risk to re-test and validate
a final artifact? Do we have a more clear understanding of what testing has
taken place across the community?

The last I saw, our documented test plan

hasn't
been maintained or kept up to date
.
Is there another artifact reflecting what testing people have in flight to
better reflect what risk of needing to re-test we have from these (and
other) post-freeze changes?



On Wed, Aug 28, 2019 at 11:52 AM Jon Haddad  wrote:

> Hey folks,
>
> I think it's time we cut a 4.0 alpha release.  Before I put up a vote
> thread, is there a reason not to have a 4.0 alpha before ApacheCon /
> Cassandra Summit?
>
> There's a handful of small issues that I should be done for 4.0 (client
> list in virtual tables, dynamic snitch improvements, fixing token counts),
> I'm not trying to suggest we don't include them, but they're small enough I
> think it's OK to merge them in following the first alpha.
>
> Jon
>

4.0 alpha before apachecon?

[Jon Haddad]

Hey folks,

I think it's time we cut a 4.0 alpha release.  Before I put up a vote
thread, is there a reason not to have a 4.0 alpha before ApacheCon /
Cassandra Summit?

There's a handful of small issues that I should be done for 4.0 (client
list in virtual tables, dynamic snitch improvements, fixing token counts),
I'm not trying to suggest we don't include them, but they're small enough I
think it's OK to merge them in following the first alpha.

Jon

Re: Stability of MaterializedView in 3.11.x | 4.0

[Jon Haddad]

I've helped a lot of teams (a dozen to two dozen maybe) migrate away from
MVs due to inconsistencies, issues with streaming (have you added or
removed nodes yet?), and massive performance issues to the point of cluster
failure under (what I consider) trivial load.  I haven't gone too deep into
analyzing their issues, folks are usually fine with "move off them", vs
having me do a ton of analysis.

tlp-stress has a materialized view workload built in, and you can add
arbitrary CQL via the --cql flag to add a MV to any existing workload such
as KeyValue or BasicTimeSeries.

On Wed, Aug 28, 2019 at 8:11 AM Jeff Jirsa  wrote:

> There have been people who have had operational issues related to MVs (many
> of them around running repair), but the biggest concern is correctness.
>
> It probably ultimately depends on what type of database you're running. If
> you're running some sort of IOT / analytics workload and you just want
> another way to SELECT the data, but you won't notice one of a billion
> records going missing, using MVs may be fine. If you're a bank, and one of
> a billion records going missing means you lose someone's bank account, I
> would avoid using MVs.
>
> It's all just risk management.
>
> On Wed, Aug 28, 2019 at 7:18 AM Pankaj Gajjar <
> pankaj.gaj...@contentserv.com>
> wrote:
>
> > Hi Michael,
> >
> > Thanks for putting very clever information " Users of MVs *must*
> determine
> > for themselves, through
> > thorough testing and understanding, if they wish to use them." And
> > this concluded that if there is any issue occur in future then only
> > solution is to rebuild the MVs since Cassandra does not able to make
> > consistent synch well.
> >
> > Also, we practically using the 10+ MVs and as of now, we have not faced
> > any issue, so my question to all community member, does anyone face any
> > critical issues ? so we need to start migration from MVs to manual query
> > base table ?
> >
> > Also, I can understand now, it's experimental and not ready for
> > production, so if possible, please ignore it only right ?
> >
> > Thanks
> > Pankaj
> >
> > On 27/08/19, 19:03, "Michael Shuler"  > of mich...@pbandjelly.org> wrote:
> >
> > It appears that you found the first message of the chain. I suggest
> > reading the linked JIRA and the complete dev@ thread that arrived at
> > this conclusion; there are loads of well formed opinions and
> > information. Users of MVs *must* determine for themselves, through
> > thorough testing and understanding, if they wish to use them.
> >
> > Linkage:
> > https://issues.apache.org/jira/browse/CASSANDRA-13959
> >   (sub-linkage..)
> >   https://issues.apache.org/jira/browse/CASSANDRA-13595
> >   https://issues.apache.org/jira/browse/CASSANDRA-13911
> >   https://issues.apache.org/jira/browse/CASSANDRA-13880
> >   https://issues.apache.org/jira/browse/CASSANDRA-12872
> >   https://issues.apache.org/jira/browse/CASSANDRA-13747
> >
> > Very much worth reading the complete thread:
> > part1:
> >
> >
> https://lists.apache.org/thread.html/d81a61da48e1b872d7599df4edfa8e244d34cbd591a18539f724796f@
> > 
> > part2:
> >
> >
> https://lists.apache.org/thread.html/19b7fcfd3b47f1526d6e993b3bb97f6c43e5ce204bc976ec0701cdd3@
> > 
> >
> > Quick JQL for open tickets with "mv":
> >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20CASSANDRA%20AND%20text%20~%20mv%20AND%20status%20!%3D%20Resolved
> >
> > --
> > Michael
> >
> > On 8/27/19 5:01 AM, pankaj gajjar wrote:
> > > Hello,
> > >
> > >
> > >
> > > concern about Materialized Views (MVs) in Cassandra. Unfortunately
> > starting
> > > with version 3.11, MVs are officially considered experimental and
> > not ready
> > > for production use, as you can read here:
> > >
> > >
> > >
> > >
> >
> http://mail-archives.apache.org/mod_mbox/cassandra-user/201710.mbox/%3cetpan.59f24f38.438f4e99.7...@apple.com%3E
> > >
> > >
> > >
> > > Can you please someone give some productive feedback on this ? it
> > would
> > > help us to further implementation around the MVs in Cassandra.
> > >
> > >
> > >
> > > Does anyone facing some critical issue or data lose or
> > synchronization
> > > issue ?
> > >
> > >
> > >
> > > Regards
> > >
> > > Pankaj.
> > >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
> >
> >
>

Re: Stability of MaterializedView in 3.11.x | 4.0

[Jeff Jirsa]

There have been people who have had operational issues related to MVs (many
of them around running repair), but the biggest concern is correctness.

It probably ultimately depends on what type of database you're running. If
you're running some sort of IOT / analytics workload and you just want
another way to SELECT the data, but you won't notice one of a billion
records going missing, using MVs may be fine. If you're a bank, and one of
a billion records going missing means you lose someone's bank account, I
would avoid using MVs.

It's all just risk management.

On Wed, Aug 28, 2019 at 7:18 AM Pankaj Gajjar 
wrote:

> Hi Michael,
>
> Thanks for putting very clever information " Users of MVs *must* determine
> for themselves, through
> thorough testing and understanding, if they wish to use them." And
> this concluded that if there is any issue occur in future then only
> solution is to rebuild the MVs since Cassandra does not able to make
> consistent synch well.
>
> Also, we practically using the 10+ MVs and as of now, we have not faced
> any issue, so my question to all community member, does anyone face any
> critical issues ? so we need to start migration from MVs to manual query
> base table ?
>
> Also, I can understand now, it's experimental and not ready for
> production, so if possible, please ignore it only right ?
>
> Thanks
> Pankaj
>
> On 27/08/19, 19:03, "Michael Shuler"  of mich...@pbandjelly.org> wrote:
>
> It appears that you found the first message of the chain. I suggest
> reading the linked JIRA and the complete dev@ thread that arrived at
> this conclusion; there are loads of well formed opinions and
> information. Users of MVs *must* determine for themselves, through
> thorough testing and understanding, if they wish to use them.
>
> Linkage:
> https://issues.apache.org/jira/browse/CASSANDRA-13959
>   (sub-linkage..)
>   https://issues.apache.org/jira/browse/CASSANDRA-13595
>   https://issues.apache.org/jira/browse/CASSANDRA-13911
>   https://issues.apache.org/jira/browse/CASSANDRA-13880
>   https://issues.apache.org/jira/browse/CASSANDRA-12872
>   https://issues.apache.org/jira/browse/CASSANDRA-13747
>
> Very much worth reading the complete thread:
> part1:
>
> https://lists.apache.org/thread.html/d81a61da48e1b872d7599df4edfa8e244d34cbd591a18539f724796f@
> 
> part2:
>
> https://lists.apache.org/thread.html/19b7fcfd3b47f1526d6e993b3bb97f6c43e5ce204bc976ec0701cdd3@
> 
>
> Quick JQL for open tickets with "mv":
>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20CASSANDRA%20AND%20text%20~%20mv%20AND%20status%20!%3D%20Resolved
>
> --
> Michael
>
> On 8/27/19 5:01 AM, pankaj gajjar wrote:
> > Hello,
> >
> >
> >
> > concern about Materialized Views (MVs) in Cassandra. Unfortunately
> starting
> > with version 3.11, MVs are officially considered experimental and
> not ready
> > for production use, as you can read here:
> >
> >
> >
> >
> http://mail-archives.apache.org/mod_mbox/cassandra-user/201710.mbox/%3cetpan.59f24f38.438f4e99.7...@apple.com%3E
> >
> >
> >
> > Can you please someone give some productive feedback on this ? it
> would
> > help us to further implementation around the MVs in Cassandra.
> >
> >
> >
> > Does anyone facing some critical issue or data lose or
> synchronization
> > issue ?
> >
> >
> >
> > Regards
> >
> > Pankaj.
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>
>
>

Re: Stability of MaterializedView in 3.11.x | 4.0

[Pankaj Gajjar]

Hi Michael,

Thanks for putting very clever information " Users of MVs *must* determine for 
themselves, through 
thorough testing and understanding, if they wish to use them." And this 
concluded that if there is any issue occur in future then only solution is to 
rebuild the MVs since Cassandra does not able to make consistent synch well.

Also, we practically using the 10+ MVs and as of now, we have not faced any 
issue, so my question to all community member, does anyone face any critical 
issues ? so we need to start migration from MVs to manual query base table ? 

Also, I can understand now, it's experimental and not ready for production, so 
if possible, please ignore it only right ?

Thanks 
Pankaj 

On 27/08/19, 19:03, "Michael Shuler"  wrote:

It appears that you found the first message of the chain. I suggest 
reading the linked JIRA and the complete dev@ thread that arrived at 
this conclusion; there are loads of well formed opinions and 
information. Users of MVs *must* determine for themselves, through 
thorough testing and understanding, if they wish to use them.

Linkage:
https://issues.apache.org/jira/browse/CASSANDRA-13959
  (sub-linkage..)
  https://issues.apache.org/jira/browse/CASSANDRA-13595
  https://issues.apache.org/jira/browse/CASSANDRA-13911
  https://issues.apache.org/jira/browse/CASSANDRA-13880
  https://issues.apache.org/jira/browse/CASSANDRA-12872
  https://issues.apache.org/jira/browse/CASSANDRA-13747

Very much worth reading the complete thread:
part1: 

https://lists.apache.org/thread.html/d81a61da48e1b872d7599df4edfa8e244d34cbd591a18539f724796f@
part2: 

https://lists.apache.org/thread.html/19b7fcfd3b47f1526d6e993b3bb97f6c43e5ce204bc976ec0701cdd3@

Quick JQL for open tickets with "mv":

https://issues.apache.org/jira/issues/?jql=project%20%3D%20CASSANDRA%20AND%20text%20~%20mv%20AND%20status%20!%3D%20Resolved

-- 
Michael

On 8/27/19 5:01 AM, pankaj gajjar wrote:
> Hello,
> 
> 
> 
> concern about Materialized Views (MVs) in Cassandra. Unfortunately 
starting
> with version 3.11, MVs are officially considered experimental and not 
ready
> for production use, as you can read here:
> 
> 
> 
> 
http://mail-archives.apache.org/mod_mbox/cassandra-user/201710.mbox/%3cetpan.59f24f38.438f4e99.7...@apple.com%3E
> 
> 
> 
> Can you please someone give some productive feedback on this ? it would
> help us to further implementation around the MVs in Cassandra.
> 
> 
> 
> Does anyone facing some critical issue or data lose or synchronization
> issue ?
> 
> 
> 
> Regards
> 
> Pankaj.
> 

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org