Josh McKenzie created CASSANDRA-18424:
-----------------------------------------
Summary: Implement graceful paging across tombstones with
short-circuit on paging rather than throwing TombstoneOverwhelmingExceptions
Key: CASSANDRA-18424
URL: https://issues.apache.org/jira/browse/CASSANDRA-18424
Project: Cassandra
Issue Type: New Feature
Components: Messaging/Client, Messaging/Internode
Reporter: Josh McKenzie
Assignee: Josh McKenzie
We implemented the hard stop with a {{TombstoneOverwhelmingException}} almost a
decade ago since paging across many tombstones was the most common way for
nodes to OOM as they iterated across all this data during queries and paging.
With our current implementations and architecture / codebase, we should be able
to combine the {{StoppingTransformation}} and existing {{clustering}} blob we
pass back to clients to allow clients to optionally page across tombstones when
using the async api via the driver and short-circuit a page when they hit the
tombstone failure threshold rather than throwing a
{{{}TombstoneOverwhelmingException{}}}. This would allow for more flexible data
modeling on users' side as well as removing one of the fairly rough edges of
our API's we're currently constrained by.
Making sure this is correct will require extensive fuzz-testing of pagination;
this should likely happen in the Harry project but we could also have a bespoke
model / implementation in the C* codebase we rely on in the interim.
Client warnings at the current default levels would remain; the gap between
warn and "short-circuit pages" (100x ratio currently, 1000 vs. 100000) should
be sufficient for clients to take action on their data models well before they
hit this limit.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
