This is an automated email from the ASF dual-hosted git repository.
adelapena pushed a commit to branch cassandra-4.1
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 6c96e2fd41368323844a080d806cbbb7b7fc9790
Merge: 5086b7d0a7 9a0af4112e
Author: Andrés de la Peña <a.penya.gar...@gmail.com>
AuthorDate: Mon Jan 23 11:42:24 2023 +0000
Merge branch 'cassandra-4.0' into cassandra-4.1
CHANGES.txt | 1 +
NEWS.txt | 9 +-
.../cassandra/service/pager/PagingState.java | 8 +
.../CompactStoragePagingWithProtocolTester.java | 179 +++++++++++++++++++++
.../CompactStoragePagingWithProtocolV30Test.java | 33 ++++
.../CompactStoragePagingWithProtocolV3XTest.java | 33 ++++
.../CompactStoragePagingWithProtocolV40Test.java | 33 ++++
7 files changed, 295 insertions(+), 1 deletion(-)
diff --cc CHANGES.txt
index ed0e5dc731,f032fc390e..340ba11ea5
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,8 -1,5 +1,9 @@@
-4.0.8
+4.1.1
+ * Fix PAXOS2_COMMIT_AND_PREPARE_RSP serialisation AssertionError
(CASSANDRA-18164)
+ * Streaming progress virtual table lock contention can trigger
TCP_USER_TIMEOUT and fail streaming (CASSANDRA-18110)
+ * Fix perpetual load of denylist on read in cases where denylist can never
be loaded (CASSANDRA-18116)
+Merged from 4.0:
+ * Fix legacy clustering serialization for paging with compact storage
(CASSANDRA-17507)
* Add support for python 3.11 (CASSANDRA-18088)
* Fix formatting of duration in cqlsh (CASSANDRA-18141)
* Fix sstable loading of keyspaces named snapshots or backups
(CASSANDRA-14013)
diff --cc NEWS.txt
index c1dfbd8127,4915652351..6dea4a07a0
--- a/NEWS.txt
+++ b/NEWS.txt
@@@ -51,20 -51,57 +51,27 @@@ restore snapshots created with the prev
'sstableloader' tool. You can upgrade the file format of your snapshots
using the provided 'sstableupgrade' tool.
-4.0.8
-=====
-
-New features
-------------
-
-Upgrading
----------
- - All previous versions of 4.x contained a mistake on the implementation
of the old CQL native protocol v3. That
- mistake produced issues when paging over tables with compact storage and
a single clustering column during rolling
- upgrades involving 3.x and 4.x nodes. The fix for that issue makes it can
now appear during rolling upgrades from
- 4.0.0-4.0.7. If that is your case, please use protocol v4 or higher in
your driver. See CASSANDRA-17507 for further
- details.
-
-4.0.6
-=====
-
-New features
-------------
-
-
-Upgrading
----------
- - If you haven't set the compaction_thoroughput_mb_per_sec in your 4.0
cassandra.yaml file but you relied on the internal
- default value,then compaction_throughput_mb_per_sec was equal to an old
default value of 16MiB/s in Cassandra 4.0.
- After CASSANDRA-17790 this is changed to 64MiB/s to match the default
value in cassandra.yaml. If you prefer the old
- one of 16MiB/s, you need to set it explicitly in your cassandra.yaml
file.
-
-Deprecation
----------
-4.0.3
+4.1.1
- ===
+ =====
-New features
-------------
-
+G1GC Recommended
+----------------
+ - The G1 settings in jvm8-server.options and jvm11-server.options are
updated according to broad feedback
+ and testing. The G1 settings remain commented out by default in 4.1.x.
It is recommended to switch
+ to G1 for performance and for simpler GC tuning. CMS is already
deprecated in JDK9, and the next major
+ release of Cassandra makes G1 the default configuration.
+ Upgrading
+ ---------
- - otc_coalescing_strategy, otc_coalescing_window_us,
otc_coalescing_enough_coalesced_messages,
- otc_backlog_expiration_interval_ms are deprecated and will be removed
at earliest with next major release.
- otc_coalescing_strategy is disabled since 3.11.
-
-Deprecation
----------
- - JavaScript user-defined functions have been deprecated. They are
planned for removal
- in the next major release. (CASSANDRA-17280)
++ - All previous versions of 4.x contained a mistake on the implementation
of the old CQL native protocol v3. That
++ mistake produced issues when paging over tables with compact storage and
a single clustering column during rolling
++ upgrades involving 3.x and 4.x nodes. The fix for that issue makes it
can now appear during rolling upgrades from
++ 4.1.0 or 4.0.0-4.0.7. If that is your case, please use protocol v4 or
higher in your driver. See CASSANDRA-17507
++ for further details.
-4.0.2
-=====
+4.1
+===
New features
------------
diff --cc
test/distributed/org/apache/cassandra/distributed/upgrade/CompactStoragePagingWithProtocolTester.java
index 0000000000,2683e7d4d3..6dd3bd9f64
mode 000000,100644..100644
---
a/test/distributed/org/apache/cassandra/distributed/upgrade/CompactStoragePagingWithProtocolTester.java
+++
b/test/distributed/org/apache/cassandra/distributed/upgrade/CompactStoragePagingWithProtocolTester.java
@@@ -1,0 -1,179 +1,179 @@@
+ /*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+ package org.apache.cassandra.distributed.upgrade;
+
+ import java.nio.ByteBuffer;
+ import java.util.List;
+
+ import org.junit.Test;
+
+ import com.datastax.driver.core.Cluster;
+ import com.datastax.driver.core.ColumnDefinitions;
+ import com.datastax.driver.core.DataType;
+ import com.datastax.driver.core.ProtocolVersion;
+ import com.datastax.driver.core.ResultSet;
+ import com.datastax.driver.core.Row;
+ import com.datastax.driver.core.Session;
+ import com.datastax.driver.core.SimpleStatement;
+ import com.datastax.driver.core.Statement;
+ import com.vdurmont.semver4j.Semver;
+ import org.apache.cassandra.distributed.api.ConsistencyLevel;
+
+ import static org.apache.cassandra.distributed.api.Feature.GOSSIP;
+ import static org.apache.cassandra.distributed.api.Feature.NATIVE_PROTOCOL;
+ import static org.apache.cassandra.distributed.api.Feature.NETWORK;
+ import static org.junit.Assert.assertEquals;
+
+ /**
+ * Tests paging over a table with {@code COMPACT STORAGE} in a mixed version
cluster using different protocol versions.
+ */
+ public abstract class CompactStoragePagingWithProtocolTester extends
UpgradeTestBase
+ {
+ /**
+ * The initial version from which we are upgrading.
+ */
+ protected abstract Semver initialVersion();
+
+ @Test
+ public void testPagingWithCompactStorageSingleClustering() throws
Throwable
+ {
+ Object[] row1 = new Object[]{ "0", "01", "v" };
+ Object[] row2 = new Object[]{ "0", "02", "v" };
+ Object[] row3 = new Object[]{ "1", "01", "v" };
+ Object[] row4 = new Object[]{ "1", "02", "v" };
+
+ new TestCase()
+ .nodes(2)
+ .nodesToUpgrade(1)
- .singleUpgrade(initialVersion(), CURRENT)
++ .singleUpgrade(initialVersion())
+ .withConfig(config -> config.with(GOSSIP, NETWORK, NATIVE_PROTOCOL))
+ .setup(c -> {
+ c.schemaChange(withKeyspace("CREATE TABLE %s.t (pk text, ck text,
v text, " +
+ "PRIMARY KEY (pk, ck)) WITH COMPACT
STORAGE"));
+ String insert = withKeyspace("INSERT INTO %s.t (pk, ck, v) VALUES
(?, ?, ?)");
+ c.coordinator(1).execute(insert, ConsistencyLevel.ALL, row1);
+ c.coordinator(1).execute(insert, ConsistencyLevel.ALL, row2);
+ c.coordinator(1).execute(insert, ConsistencyLevel.ALL, row3);
+ c.coordinator(1).execute(insert, ConsistencyLevel.ALL, row4);
+ })
+ .runAfterNodeUpgrade((cluster, node) ->
assertRowsWithAllProtocolVersions(row1, row2, row3, row4))
+ .run();
+ }
+
+ @Test
+ public void testPagingWithCompactStorageMultipleClusterings() throws
Throwable
+ {
+ Object[] row1 = new Object[]{ "0", "01", "10", "v" };
+ Object[] row2 = new Object[]{ "0", "01", "20", "v" };
+ Object[] row3 = new Object[]{ "0", "02", "10", "v" };
+ Object[] row4 = new Object[]{ "0", "02", "20", "v" };
+ Object[] row5 = new Object[]{ "1", "01", "10", "v" };
+
+ new TestCase()
+ .nodes(2)
+ .nodesToUpgrade(1)
- .singleUpgrade(initialVersion(), CURRENT)
++ .singleUpgrade(initialVersion())
+ .withConfig(config -> config.with(GOSSIP, NETWORK, NATIVE_PROTOCOL))
+ .setup(c -> {
+ c.schemaChange(withKeyspace("CREATE TABLE %s.t (pk text, ck1
text, ck2 text, v text, " +
+ "PRIMARY KEY (pk, ck1, ck2)) WITH
COMPACT STORAGE"));
+ String insert = withKeyspace("INSERT INTO %s.t (pk, ck1, ck2, v)
VALUES (?, ?, ?, ?)");
+ c.coordinator(1).execute(insert, ConsistencyLevel.ALL, row1);
+ c.coordinator(1).execute(insert, ConsistencyLevel.ALL, row2);
+ c.coordinator(1).execute(insert, ConsistencyLevel.ALL, row3);
+ c.coordinator(1).execute(insert, ConsistencyLevel.ALL, row4);
+ c.coordinator(1).execute(insert, ConsistencyLevel.ALL, row5);
+ })
+ .runAfterNodeUpgrade((cluster, node) ->
assertRowsWithAllProtocolVersions(row1, row2, row3, row4, row5))
+ .run();
+ }
+
+ @Test
+ public void testPagingWithCompactStorageWithoutClustering() throws
Throwable
+ {
+ Object[] row1 = new Object[]{ "1", "v1", "v2" };
+ Object[] row2 = new Object[]{ "2", "v1", "v2" };
+ Object[] row3 = new Object[]{ "3", "v1", "v2" };
+
+ new TestCase()
+ .nodes(2)
+ .nodesToUpgrade(1)
- .singleUpgrade(initialVersion(), CURRENT)
++ .singleUpgrade(initialVersion())
+ .withConfig(config -> config.with(GOSSIP, NETWORK, NATIVE_PROTOCOL))
+ .setup(c -> {
+ c.schemaChange(withKeyspace("CREATE TABLE %s.t (pk text PRIMARY
KEY, v1 text, v2 text) WITH COMPACT STORAGE"));
+ String insert = withKeyspace("INSERT INTO %s.t (pk, v1, v2)
VALUES (?, ?, ?)");
+ c.coordinator(1).execute(insert, ConsistencyLevel.ALL, row1);
+ c.coordinator(1).execute(insert, ConsistencyLevel.ALL, row2);
+ c.coordinator(1).execute(insert, ConsistencyLevel.ALL, row3);
+ })
+ .runAfterNodeUpgrade((cluster, node) ->
assertRowsWithAllProtocolVersions(row3, row2, row1))
+ .run();
+ }
+
+ private void assertRowsWithAllProtocolVersions(Object[]... rows)
+ {
+ String query = withKeyspace("SELECT * FROM %s.t");
+ assertRows(query, ProtocolVersion.V3, rows);
+ assertRows(query, ProtocolVersion.V4, rows);
+ if (initialVersion().isGreaterThanOrEqualTo(v3X))
+ assertRows(query, ProtocolVersion.V5, rows);
+ }
+
+ private static void assertRows(String query, ProtocolVersion
protocolVersion, Object[]... expectedRows)
+ {
+ Cluster.Builder builder = com.datastax.driver.core.Cluster.builder()
+
.addContactPoint("127.0.0.1")
+
.withProtocolVersion(protocolVersion);
+ try (com.datastax.driver.core.Cluster cluster = builder.build();
+ Session session = cluster.connect())
+ {
+ Statement stmt = new SimpleStatement(query);
+
stmt.setConsistencyLevel(com.datastax.driver.core.ConsistencyLevel.ALL);
+ stmt.setFetchSize(1);
+
+ ResultSet result = session.execute(stmt);
+ List<Row> actualRows = result.all();
+ assertEquals(expectedRows.length, actualRows.size());
+
+ ColumnDefinitions columnDefs = result.getColumnDefinitions();
+ com.datastax.driver.core.ProtocolVersion driverProtocolVersion =
+
com.datastax.driver.core.ProtocolVersion.fromInt(protocolVersion.toInt());
+
+ for (int rowIndex = 0; rowIndex < expectedRows.length; rowIndex++)
+ {
+ Object[] expectedRow = expectedRows[rowIndex];
+ Row actualRow = actualRows.get(rowIndex);
+
+ assertEquals(expectedRow.length,
actualRow.getColumnDefinitions().size());
+
+ for (int columnIndex = 0; columnIndex < columnDefs.size();
columnIndex++)
+ {
+ DataType type = columnDefs.getType(columnIndex);
+ ByteBuffer expectedByteValue = cluster.getConfiguration()
+ .getCodecRegistry()
+ .codecFor(type)
+
.serialize(expectedRow[columnIndex], driverProtocolVersion);
+ ByteBuffer actualValue =
actualRow.getBytesUnsafe(columnDefs.getName(columnIndex));
+ assertEquals(expectedByteValue, actualValue);
+ }
+ }
+ }
+ }
+ }
diff --cc
test/distributed/org/apache/cassandra/distributed/upgrade/CompactStoragePagingWithProtocolV40Test.java
index 0000000000,0000000000..d245d9fdb7
new file mode 100644
--- /dev/null
+++
b/test/distributed/org/apache/cassandra/distributed/upgrade/CompactStoragePagingWithProtocolV40Test.java
@@@ -1,0 -1,0 +1,33 @@@
++/*
++ * Licensed to the Apache Software Foundation (ASF) under one
++ * or more contributor license agreements. See the NOTICE file
++ * distributed with this work for additional information
++ * regarding copyright ownership. The ASF licenses this file
++ * to you under the Apache License, Version 2.0 (the
++ * "License"); you may not use this file except in compliance
++ * with the License. You may obtain a copy of the License at
++ *
++ * http://www.apache.org/licenses/LICENSE-2.0
++ *
++ * Unless required by applicable law or agreed to in writing, software
++ * distributed under the License is distributed on an "AS IS" BASIS,
++ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
++ * See the License for the specific language governing permissions and
++ * limitations under the License.
++ */
++
++package org.apache.cassandra.distributed.upgrade;
++
++import com.vdurmont.semver4j.Semver;
++
++/**
++ * {@link CompactStoragePagingWithProtocolTester} for v40 -> CURRENT upgrade
path.
++ */
++public class CompactStoragePagingWithProtocolV40Test extends
CompactStoragePagingWithProtocolTester
++{
++ @Override
++ protected Semver initialVersion()
++ {
++ return v40;
++ }
++}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
