Re: Proposal to retroactively mark materialized views experimental

> > So you’d rather continue to lie to users about the stability of the > feature rather than admitting it was merged in prematurely? It was merged prematurely, but a lot has changed since then and a lot of fixes have been made, and now it's really no more awful than any other component of

Re: Cassandra pluggable storage engine (update)

Hi DuyHai, Good point! At this moment, I do not see anything really prevent us from having one storage engine type per table, we are using one RocksDB instance per table anyway. However, we want to do simple things first, and it's easier for us to have storage engine per keyspace, for both

Re: Cassandra pluggable storage engine (update)

Excellent docs, thanks for the update Dikang. A question about a design choice, is there any technical reason to specify the storage engine at keyspace level rather than table level ? It's not overly complicated to move all tables sharing the same storage engine into the same keyspace but then

Re: Cassandra pluggable storage engine (update)

Hi Blake, Great questions! 1. Yeah, we implement the encoding algorithms, which could encode C* data types into byte array, and keep the same sorting order. Our implementation is based on the orderly lib used in HBase, https://github.com/ndimiduk/orderly . 2. Repair is not supported yet, we are

Re: Cassandra pluggable storage engine (update)

Hi Dikang, Cool stuff. 2 questions. Based on your presentation at ngcc, it seems like rocks db stores things in byte order. Does this mean that you have code that makes each of the existing types byte comparable, or is clustering order implementation dependent? Also, I don't see anything in

Cassandra pluggable storage engine (update)

Hello C* developers: In my previous email ( https://www.mail-archive.com/dev@cassandra.apache.org/msg11024.html), I presented that Instagram was kicking off a project to make C*'s storage engine to be pluggable, as other modern databases, like mysql, mongoDB etc, so that users will be able to

Re: CREATE INDEX without IF NOT EXISTS when snapshoting

Kurt, Thanks for your response. Created this ticket . Feel free to add anything to it that seems legit. Downloading Cassandra code right now. Fix seems quite simple. Expect a pull-request soon xD 2017-10-03 20:19 GMT-03:00 kurt greaves

Re: Proposal to retroactively mark materialized views experimental

On Wed, Oct 4, 2017 at 12:23 PM, Jon Haddad wrote: > The default part I was referring to incremental repair. > > SASI still has a pretty fatal issue where nodes OOM: > https://issues.apache.org/jira/browse/CASSANDRA-12662 < >

Re: Proposal to retroactively mark materialized views experimental

The default part I was referring to incremental repair. SASI still has a pretty fatal issue where nodes OOM: https://issues.apache.org/jira/browse/CASSANDRA-12662 > On Oct 4, 2017, at 12:21 PM, Pavel Yaskevich

Re: Proposal to retroactively mark materialized views experimental

On Wed, Oct 4, 2017 at 12:09 PM, Jon Haddad wrote: > MVs work fine for *some use cases*, not the general use case. That’s why > there should be a flag. To opt into the feature when the behavior is only > known to be correct under a certain set of circumstances. Nobody is

Re: Proposal to retroactively mark materialized views experimental

MVs work fine for *some use cases*, not the general use case. That’s why there should be a flag. To opt into the feature when the behavior is only known to be correct under a certain set of circumstances. Nobody is saying the flag should be

Re: Proposal to retroactively mark materialized views experimental

> > So you’d rather continue to lie to users about the stability of the > feature rather than admitting it was merged in prematurely? Much like w/SASI, this is something that's in the code-base that for > certain use-cases apparently works just fine. I don't know of any outstanding issues with

Re: Proposal to retroactively mark materialized views experimental

Strongly disagree with MV’s being isolated part. You can feel the touch of the MVs in the read path, write path, metadata handling, whether you use them or not. And comparing any of those before/after MVs were introduced makes me sad every time I face any of it. It made our codebase

Re: Proposal to retroactively mark materialized views experimental

So you’d rather continue to lie to users about the stability of the feature rather than admitting it was merged in prematurely? I’d rather come clean and avoid future problems, and give people the opportunity to stop using MVs rather than let them keep taking risks they’re unaware of. This is

Re: Proposal to retroactively mark materialized views experimental

> > Oh, come on. You're being disingenuous. Not my intent. MV's (and SASI, for example) are fairly well isolated; we have a history of other changes that are much more broadly and higher impact risk-wise across the code-base. If I were an operator and built a critical part of my business on a

Re: Proposal to retroactively mark materialized views experimental

Yep. Almost! Changing the default in a minor version might break some scripts/tooling that manipulate schema and potentially create new MVs (as dangerous as it currently is - manipulating schema in that way), and that would still not be very nice. Introducing a flag and leaving it at false in

Re: Proposal to retroactively mark materialized views experimental

I don't agree at face value that early re-open is in sum a lot simpler than MV, or that adding CQL and deprecating Thrift was a lot simpler, or the 8099 refactor, etc. Different types of complexity, certainly, and MV's are arguably harder to prove correct due to surface area of exposure to failure

Re: Proposal to retroactively mark materialized views experimental

Introducing feature flags for enabling or disabling different code paths is not sustainable in the long run. It's hard enough to keep up with integration testing with the couple of Jenkins jobs that we have. Running jobs for all permutations of flags that we keep around, would turn out

Re: Proposal to retroactively mark materialized views experimental

> and providing a feature we don't fully understand, have not fully documented the caveats of, let alone discovered all the problems with nor had that knowledge percolate fully into the wider community. There appear to be varying levels of understanding of the implementation details of MV's (that

Re: Proposal to retroactively mark materialized views experimental

> > The flag name `cdc_enabled` is simple and, without adjectives, does not > imply "experimental" or "beta" or anything like that. > It does make life easier for both operators and the C* developers. I would be all for a mv_enabled option, assuming it's enabled by default for all existing

Re: Proposal to retroactively mark materialized views experimental

> > CDC sounds like it is in the same basket, but it already has the > > `cdc_enabled` yaml flag which defaults false. > > I went this route because I was incredibly wary of changing the CL > code and wanted to shield non-CDC users from any and all risk I > reasonably could. This approach so far

Re: Proposal to retroactively mark materialized views experimental

For the record, in case I was unclear, it was never my intention to suggest that we shouldn't warn about MVs: I would agree that we still should and I'm happy that we do. I would also agree that the remaining caveats and limitations should be more clearly documented. But, I kind of got the