I think this is a regression introduced as part of CASSANDRA-17248
<https://issues.apache.org/jira/browse/CASSANDRA-17248>, the following code
got introduced in QueryProcessor.java
since C* 3.0.26.

// Make sure the missing one is going to be eventually re-prepared

The above-mentioned code was not present in C* 3.0.25

Alex, could you please take a look at it?


On Fri, Jan 19, 2024 at 7:17 PM Jaydeep Chovatia <chovatia.jayd...@gmail.com>

> Hi,
> Today, in our production, we came across the following scenario:
>    1. We have 100 nodes of the Cassandra cluster on 4.0.6, and our client
>    uses PreparedStatement, say, "*SELECT * FROM T1 WHERE PK=?*"
>    2. We applied a schema change to add a *regular* column, "*ALTER TABLE
>    T1 ADD COLUMN c1 UUID"*
>    3. Around 20 (out of 100) of the Cassandra nodes started throwing the
>    error: "*Successfully prepared, but could not find prepared statement
>    for* "  (Code path: QueryEvents.java
> <https://github.com/apache/cassandra/blob/cassandra-4.0/src/java/org/apache/cassandra/cql3/QueryEvents.java#L225C80-L225C145>
>    )
>    4. The error continued for around 2 hours, and only restarting those
>    20 nodes resolved the issue.
> Our current hypothesis is that there is a race condition in the
> QueryProcessor::prepare
> <https://github.com/apache/cassandra/blob/cassandra-4.0/src/java/org/apache/cassandra/cql3/QueryProcessor.java#L575>
> API in that one thread is evicting the prepared statements while the other
> is adding, which is never-ending. A similar hypothesis has been mentioned
> in a ticket in 2022: https://issues.apache.org/jira/browse/CASSANDRA-17401
> Has anyone ever experienced this? Are there any quick pointers on what
> could have gone wrong?
> Thanks in advance!
> Jaydeep

Reply via email to