[
https://issues.apache.org/jira/browse/CASSANDRA-19662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17850142#comment-17850142
]
Bret McGuire commented on CASSANDRA-19662:
------------------------------------------
Just to clarify a few things here [~kumarbharath]:
# You mentioned "select *" statements were in use for reads. Am I correct in
saying those "select *" statements are implemented using prepared statements?
# Do we know the protocol version in use for this client + server? The
behaviour described above sounds a lot like what you'd expect to see before
CASSANDRA-10786 came into play but the changes implemented there should be in
place for Cassandra 4.0.12 and Java driver 4.17.0. That said, they also still
rely on protocol v5 features to function correctly... so if a different
protocol is specified you won't get the error correcting behaviour in 10786.
> Data Corruption and OOM Issues During Schema Alterations
> ---------------------------------------------------------
>
> Key: CASSANDRA-19662
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19662
> Project: Cassandra
> Issue Type: Bug
> Components: Client/java-driver
> Reporter: BHARATH KUMAR
> Priority: Urgent
> Attachments: BufferUnderflow_plus_error
>
>
> h2. Description
>
> *Overview:* The primary issue is data corruption occurring during schema
> alterations (ADD/DROP column) on large tables(300+ columns and 6TB size ) in
> the production cluster. This is accompanied by out-of-memory (OOM) errors and
> other exceptions, specifically during batch reads. This problem has been
> replicated on multiple clusters, running Apache Cassandra version 4.0.12 and
> Datastax Java Driver Version: 4.17
> *Details:*
> *Main Issue:*
> * *Data Corruption:* When dynamically adding a column to a table, the data
> intended for the new column is shifted, causing misalignment in the data.
> * *Symptoms:* The object implementing
> {{com.datastax.oss.driver.api.core.cql.Row}} returns values shifted against
> the column names returned by {{{}row.getColumnDefinitions(){}}}. The driver
> returns a corrupted row, leading to incorrect data insertion.
> *Additional Issues:*
> *Exceptions:*
> * {{java.nio.BufferUnderflowException}} during batch reads when ALTER TABLE
> ADD/DROP column statements are issued.
> * {{java.lang.ArrayIndexOutOfBoundsException}} in some cases.
> * Buffer underflow exceptions with messages like "Invalid 32-bits integer
> value, expecting 4 bytes but got 292".
> * OOM errors mostly occur during ADD column operations, while other
> exceptions occur during DELETE column operations.
> * *Method Specific:* Errors occur specifically with
> {{{}row.getList(columnName, Float.class){}}}, returning incorrect values.
> *Reproducibility:*
> * The issue is reproducible on larger tables (300 columns, 6 TB size) but
> not on smaller tables.
> * SELECT * statements are used during reads
> * *Method Specific:* Errors occur specifically with
> {{{}row.getList(columnName, Float.class){}}}, returning incorrect values.
> However, the code registers a driver exception when calling the method
> {{{}row.getList(columnName, Float.class){}}}. We pass the exact column name
> obtained from {{{}row.getColumnDefinition{}}}, but it returns the wrong value
> for a column with this name. This suggests that the issue lies with the
> driver returning an object with incorrect properties, rather than with the
> SQL query itself.
> *Debugging Efforts:*
> * *Metadata Refresh:* Enabling metadata refresh did not resolve the issue.
> * *Schema Agreement:* {{session.getCqlSession().checkSchemaAgreement()}} did
> not detect inconsistencies during test execution.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
