To your point about Guardrails vs. Constraints, I do think the distinct roles of “cluster operator” and “application developer” help show how these two frameworks are both valuable. I don’t think I’d expect a cluster operator to be involved in every table design decision, but being able to set warning and error-level guardrails allows an operator to set absolute limits on what the database itself accepts. Table-level constraints allow application developers (hopefully in concert with operators, where they are two distinct people/groups) to add additional, application-layer constraints that are likely to be app specific. To restate what I think you were getting at, your example of a production issue caused by the development team missing a key verbal agreement probably helps illustrate why both table-level constraints and guardrails are valuable.
Imagine that, as an operator, you are generally comfortable with individual values in rows being, say, 256k, but because of the way in which this particular use case works, 64k chunks needed to be enforced. Your cluster-level guardrails could be set at 256k, but the table-level constraints could have enforced this 64k chunk size rule. Doug > On Jun 23, 2024, at 5:38 PM, Jordan West <jorda...@gmail.com> wrote: > > I am generally for this CEP, particularly the sizeOf guardrail. For example, > we recently had an incident caused by a client who wrote outside of the > contract we had verbally established. The constraint would have let us encode > that contract into the database. In this case, clients are writing large > blobs at the application layer and internally the client performs chunking. > We had established a chunk size of 64k, for example. However, the application > team wanted to use a different programming language than the ones we provide > clients for so they wrote their own. The new client had a bug that did not > honor the agreed upon chunk size and wrote chunks that were MBs in size. This > eventually led to a production incident and the issue was discovered as a > result of a bunch of analysis (dumping sstables, etc). Had we had the sizeOf > guardrail it would have turned a production incident with hours of > investigation into a bug found immediately during development. Could this be > done with a node-level guardrail? Likely. But config has the issues described > above and its possible to have two tables with different constraints around > similar fields (for example, two different chunk size configs due to data > shape). Could it be done at the client layer? Yes that's what we are doing > now, but this incident highlights the weakness with that approach (having to > implement the contract everywhere and having disjoint features across > clients). > > I also think there is benefit to application owners. Encoding constraints in > the database ensures continuity as ownership and contributors change and > reduces the need for comments or documentation as the means to enforce or > share this knowledge. > > I think enforcing them at write time makes sense. Thinking about it in the > scope of compaction for example reminds me of a data loss incident where > someone ran a validation in an older version (like 2.0 or 2.1) and a bunch of > 4 byte ints were thrown away because the field expected an 8 byte long. > > My primary concern would be ensuring that we don't implement constraints that > require a read before right (not inList comes to mind as an example of one > that could imply reading before writing and could confuse a user if it > doesn't). > > Regarding the conflict with existing guardrails, I do think that is tougher. > On one hand I find this feature to be more evolved than those guardrails and > would be fine to see them be replaced by it. On the other, the guardrails > provide sole control to the operator which is nice but adds some complexity > that has been rightly called out. But I don't see that as a reason not to go > forward with this feature. We should pick a path and accept the tradeoffs. > > Jordan > > > On Thu, Jun 13, 2024 at 2:39 PM Bernardo Botella > <conta...@bernardobotella.com <mailto:conta...@bernardobotella.com>> wrote: >> Thanks a lot for your comments Abe! >> >> I do agree that the Constraint clause should be as simple as possible. I >> will add a note on the CEP along with some specifics about the proposed >> constraints (removing the ones that are contentious, and adding them to a >> possible future additions section). And yeah, I also think that these >> constraints will help different Cassandra operating paradigms (multi-tenant >> clusters and diverse workflows). >> >> Besides that, I hope that I’ve addressed all the potential concerns and >> feedback on the thread. Let’s let a bit more time for others to chime in >> (any further feedback will be more than welcome), but I’d like to move >> forward with a voting soon if no other concerns are pointed out. >> >> All and all, thanks a lot to everyone that participated in the thread and >> added to the discussion! >> Bernardo >> >> >> >> > On Jun 12, 2024, at 2:37 PM, Abe Ratnofsky <a...@aber.io >> > <mailto:a...@aber.io>> wrote: >> > >> > I've thought about this some more. It would be useful for Cassandra to >> > support user-defined "guardrails" (or constraints, whatever you want to >> > call them), that could be applied per keyspace or table. Whether a user or >> > an operator is considered the owner of a table depends on the organization >> > deploying Cassandra, so allowing both parties to protect their tables >> > against mis-use seems good to me, especially for large multi-tenant >> > clusters with diverse workloads. >> > >> > For example, it would be really useful if a user could set the >> > Guardrails.{read,write}ConsistencyLevels for their tables, or declare >> > whether all operations should be over LWTs to avoid mixing regular and LWT >> > workloads. >> > >> > I'm hesitant about adding lots of expression syntax to the CONSTRAINT >> > clause. I think I'd prefer a function calling syntax that represents: >> > 1. Whether the constraint is system / keyspace / table scoped >> > 2. Where in query processing the constraint is checked >> > 3. What is executed by the check >>
