[
https://issues.apache.org/jira/browse/CASSANDRA-20118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17903408#comment-17903408
]
Brandon Williams edited comment on CASSANDRA-20118 at 12/5/24 3:30 PM:
-----------------------------------------------------------------------
I'm not sure that's the problem. If I start a 4.1 cluster, insert some data
and then upgrade one node, the new node does issue a schema change but the 4.1
nodes converge on it:
{noformat}
INFO [MigrationStage:1] 2024-12-05 09:21:27,449
DefaultSchemaUpdateHandler.java:222 - Applying schema change due to received
mutations: SchemaTransformationResult{f092dcb1-4fc6-3166-b279-a12bd3eb8d47 -->
dac85318-316f-36ec-93e5-1c1341a52fa3, diff=KeyspacesDiff{created=[],
dropped=[], altered=[KeyspaceDiff{before=KeyspaceMetadata{name=system_auth,
kind=REGULAR, params=KeyspaceParams{durable_writes=true,
replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy,
replication_factor=1}}, tables=[system_auth.roles, system_auth.role_members,
system_auth.role_permissions, system_auth.resource_role_permissons_index,
system_auth.network_permissions], views=[], functions=[], types=[]},
after=KeyspaceMetadata{name=system_auth, kind=REGULAR,
params=KeyspaceParams{durable_writes=true,
replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy,
replication_factor=1}}, tables=[system_auth.cidr_groups,
system_auth.cidr_permissions, system_auth.identity_to_role,
system_auth.network_permissions, system_auth.resource_role_permissons_index,
system_auth.role_members, system_auth.role_permissions, system_auth.roles],
views=[], functions=[], types=[]},
tables=Diff{created=[system_auth.cidr_groups, system_auth.cidr_permissions,
system_auth.identity_to_role], dropped=[], altered=[]}, views=Diff{created=[],
dropped=[], altered=[]}, types=Diff{created=[], dropped=[], altered=[]},
udfs=Diff{created=[], dropped=[], altered=[]}, udas=Diff{created=[],
dropped=[], altered=[]}}]}} }}
{noformat}
{noformat}
$ bin/nodetool describering keyspace1 | grep Schema
Schema Version:dac85318-316f-36ec-93e5-1c1341a52fa3
{noformat}
[~paulchandler] can you attach a 4.1 log too?
was (Author: brandon.williams):
I'm not sure that's the problem. If I start a 4.1 cluster, insert some data
and then upgrade one node, the new node does issue a schema change but the 4.1
nodes converge on it:
{noformat}
INFO [MigrationStage:1] 2024-12-05 09:21:27,449
DefaultSchemaUpdateHandler.java:222 - Applying schema change due to received
mutations: SchemaTransformationResult{f092dcb1-4fc6-3166-b279-a12bd3eb8d47 -->
dac85318-316f-36ec-93e5-1c1341a52fa3, diff=KeyspacesDiff{created=[],
dropped=[], altered=[KeyspaceDiff{before=KeyspaceMetadata{name=system_auth,
kind=REGULAR, params=KeyspaceParams{durable_writes=true,
replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy,
replication_factor=1}}, tables=[system_auth.roles, system_auth.role_members,
system_auth.role_permissions, system_auth.resource_role_permissons_index,
system_auth.network_permissions], views=[], functions=[], types=[]},
after=KeyspaceMetadata{name=system_auth, kind=REGULAR,
params=KeyspaceParams{durable_writes=true,
replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy,
replication_factor=1}}, tables=[system_auth.cidr_groups,
system_auth.cidr_permissions, system_auth.identity_to_role,
system_auth.network_permissions, system_auth.resource_role_permissons_index,
system_auth.role_members, system_auth.role_permissions, system_auth.roles],
views=[], functions=[], types=[]},
tables=Diff{created=[system_auth.cidr_groups, system_auth.cidr_permissions,
system_auth.identity_to_role], dropped=[], altered=[]}, views=Diff{created=[],
dropped=[], altered=[]}, types=Diff{created=[], dropped=[], altered=[]},
udfs=Diff{created=[], dropped=[], altered=[]}, udas=Diff{created=[],
dropped=[], altered=[]}}]}} }}
{noformat}
{noformat}
$ bin/nodetool describering keyspace1 | grep Schema
Schema Version:dac85318-316f-36ec-93e5-1c1341a52fa3
{noformat}
> Hints ignored during Upgrade from C*4 to C*5
> --------------------------------------------
>
> Key: CASSANDRA-20118
> URL: https://issues.apache.org/jira/browse/CASSANDRA-20118
> Project: Apache Cassandra
> Issue Type: Bug
> Components: Consistency/Hints
> Reporter: Paul Chandler
> Priority: Normal
> Fix For: 5.0.x
>
> Attachments: 5-0system.log
>
>
> I have discovered that some hints were not being processed after nodes come
> back up when a cluster in in a mixed mode with some cassandra 4 nodes and
> some cassdandra 5 nodes ( these with a storage compatibility mode CASSANDRA_4
> )
>
> When in this mode there is a schema mismatch after the first node has been
> upgraded, which continues until the last node has been upgraded.
> It seems that the hints are blocked from being sent if there is a schema
> mismatch between the 2 nodes, that can be seen at this line.
> [cassandra/src/java/org/apache/cassandra/hints/HintsDispatchTrigger.java at
> cassandra-5.0 ·
> apache/cassandra|https://github.com/apache/cassandra/blob/cassandra-5.0/src/java/org/apache/cassandra/hints/HintsDispatchTrigger.java#L65]
> I have tested removing this line, and that then does allow the hint to be
> transferred normally. However I am not sure of the implications for doing
> that if the hint is for part of the schema where the actual mismatch occurs.
>
> This creates the problem when a node is being upgraded and is currently down,
> hint files will be created for it on the new cassandra 5 nodes and the old
> cassandra 4 nodes, but the hint files on the old cassandra 4 nodes will not
> be processed, due to the schema mismatch. Leading to potential data loss.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
