[
https://issues.apache.org/jira/browse/CASSANDRA-19017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17787344#comment-17787344
]
Caleb Rackliffe commented on CASSANDRA-19017:
---------------------------------------------
+1 on both patches
The only thing I'd do is get rid of the new {{RepairWithEmptyColumnIndexTest}}
and just modify {{IndexStreamingTest}} to handle the empty case. Something like
this will repro, and then I think we just have to fix the expected file count
for the empties...
{noformat}
Subject: [PATCH] expand IndexStreamingTest to handle empty column indexes
---
Index:
test/distributed/org/apache/cassandra/distributed/test/sai/IndexStreamingTest.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git
a/test/distributed/org/apache/cassandra/distributed/test/sai/IndexStreamingTest.java
b/test/distributed/org/apache/cassandra/distributed/test/sai/IndexStreamingTest.java
---
a/test/distributed/org/apache/cassandra/distributed/test/sai/IndexStreamingTest.java
(revision 6100d26a592b357333ef9ddf7b6853feda54812f)
+++
b/test/distributed/org/apache/cassandra/distributed/test/sai/IndexStreamingTest.java
(date 1700245647831)
@@ -99,14 +99,8 @@
int numIndexComponents = isLiteral ?
V1OnDiskFormat.LITERAL_COMPONENTS.size() :
V1OnDiskFormat.NUMERIC_COMPONENTS.size();
int numComponents = sstableStreamingComponentsCount() +
numSSTableComponents + numIndexComponents;
- if (isLiteral)
- cluster.schemaChange(withKeyspace(
- "CREATE CUSTOM INDEX ON %s.test(literal) USING
'StorageAttachedIndex';"
- ));
- else
- cluster.schemaChange(withKeyspace(
- "CREATE CUSTOM INDEX ON %s.test(numeric) USING
'StorageAttachedIndex';"
- ));
+ cluster.schemaChange(withKeyspace("CREATE INDEX ON
%s.test(literal) USING 'sai';"));
+ cluster.schemaChange(withKeyspace("CREATE INDEX ON
%s.test(numeric) USING 'sai';"));
cluster.stream().forEach(i ->
i.nodetoolResult("disableautocompaction",
KEYSPACE).asserts().success()
@@ -115,12 +109,19 @@
IInvokableInstance second = cluster.get(2);
long sstableCount = 10;
long expectedFiles = isZeroCopyStreaming ? sstableCount *
numComponents : sstableCount;
+
for (int i = 0; i < sstableCount; i++)
{
if (isWide)
- first.executeInternal(withKeyspace("insert into
%s.test(pk, ck, literal, numeric, b) values (?, ?, ?, ?, ?)"), i, i, "v" + i,
i, BLOB);
+ {
+ String insertTemplate = "INSERT INTO %s.test(pk, ck, " +
(isLiteral ? "literal" : "numeric") + ", b) VALUES (?, ?, ?, ?)";
+ first.executeInternal(withKeyspace(insertTemplate), i, i,
isLiteral ? "v" + i : Integer.valueOf(i), BLOB);
+ }
else
- first.executeInternal(withKeyspace("insert into
%s.test(pk, literal, numeric, b) values (?, ?, ?, ?)"), i, "v" + i, i, BLOB);
+ {
+ String insertTemplate = "INSERT INTO %s.test(pk, " +
(isLiteral ? "literal" : "numeric") + ", b) VALUES (?, ?, ?)";
+ first.executeInternal(withKeyspace(insertTemplate), i,
isLiteral ? "v" + i : Integer.valueOf(i), BLOB);
+ }
first.flush(KEYSPACE);
}
{noformat}
> Ensure that empty SAI column indexes do not fail on validation after
> full-SSTable streaming
> -------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-19017
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19017
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Streaming, Feature/SAI
> Reporter: Caleb Rackliffe
> Assignee: Andres de la Peña
> Priority: Normal
> Fix For: 5.0-beta, 5.x
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> I actually discovered this in some exploratory testing for CASSANDRA-19007...
> {noformat}
> @Test
> public void testRepairWithEmptyColumnIndex() throws IOException
> {
> try (Cluster cluster = init(Cluster.build(2).withConfig(config ->
> config.with(GOSSIP).with(NETWORK)).start()))
> {
> cluster.schemaChange(withKeyspace("CREATE TABLE %s.t (k int PRIMARY
> KEY, a int, b int)"));
> cluster.schemaChange(withKeyspace("CREATE INDEX ON %s.t(a) USING
> 'sai'"));
> cluster.schemaChange(withKeyspace("CREATE INDEX ON %s.t(b) USING
> 'sai'"));
> // insert a split row
> cluster.get(1).executeInternal(withKeyspace("INSERT INTO %s.t(k, a)
> VALUES (0, 1)"));
> cluster.get(2).executeInternal(withKeyspace("INSERT INTO %s.t(k, b)
> VALUES (0, 2)"));
> cluster.get(1).flush(KEYSPACE);
> cluster.get(2).flush(KEYSPACE);
> cluster.get(1).nodetoolResult("repair", KEYSPACE).asserts().success();
> // fails w/ no data for the "a" index on node 2!
> }
> }
> {noformat}
> {noformat}
> java.lang.RuntimeException: Repair job has failed with the error message:
> Repair command #1 failed with error Repair session
> 55a34e20-7f24-11ee-b345-d158c708a34e for range [(-1,9223372036854775805],
> (9223372036854775805,-1]] failed with error Stream failed:
> Session peer /127.0.0.2:7012 Failed because of an unknown exception
> java.io.UncheckedIOException: java.nio.file.NoSuchFileException:
> /private/var/folders/4d/zfjs7m7s6x5_l93k33r5k6680000gn/T/dtests2185335212676803469/node1/data0/distributed_test_keyspace/t-3459c6aa21753083afcc5b0c9f77c3f7/distributed_test_keyspace-t-nc-4-big-SAI+aa+t_a_idx+Meta.db
>
> org.apache.cassandra.index.sai.disk.v1.V1OnDiskFormat.rethrowIOException(V1OnDiskFormat.java:241)
>
> org.apache.cassandra.index.sai.disk.v1.V1OnDiskFormat.validatePerColumnIndexComponents(V1OnDiskFormat.java:230)
> java.nio.file.NoSuchFileException:
> /private/var/folders/4d/zfjs7m7s6x5_l93k33r5k6680000gn/T/dtests2185335212676803469/node1/data0/distributed_test_keyspace/t-3459c6aa21753083afcc5b0c9f77c3f7/distributed_test_keyspace-t-nc-4-big-SAI+aa+t_a_idx+Meta.db
>
> java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
>
> java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
> {noformat}
> This is an untested side-effect of a change in CASSANDRA-18670. The fix
> should be fairly simple, essentially making sure we don't assume the column
> index components are present during validation when the index completion
> marker is present.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
