[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository.
jolynch pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 1fce84f9833bd62227dbf8f5d063935457dbc18e
Merge: cd73c14 1911a88
Author: Joseph Lynch
AuthorDate: Thu Dec 9 10:24:19 2021 -0500
Merge branch 'cassandra-3.0' into cassandra-3.11
CHANGES.txt| 1 +
conf/cassandra.yaml| 7 +-
.../apache/cassandra/cache/AutoSavingCache.java| 8 +-
src/java/org/apache/cassandra/config/Config.java | 2 +
.../cassandra/config/DatabaseDescriptor.java | 11 ++
.../org/apache/cassandra/db/lifecycle/View.java| 2 +-
.../org/apache/cassandra/service/CacheService.java | 40 --
.../test/microbench/CacheLoaderBench.java | 137
.../unit/org/apache/cassandra/db/KeyCacheTest.java | 138 -
9 files changed, 332 insertions(+), 14 deletions(-)
diff --cc CHANGES.txt
index 6bda91e,085e1ff..6c70235
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,16 -1,5 +1,17 @@@
-3.0.26:
+3.11.12
+ * Upgrade snakeyaml to 1.26 in 3.11 (CASSANDRA=17028)
+ * Add key validation to ssstablescrub (CASSANDRA-16969)
+ * Update Jackson from 2.9.10 to 2.12.5 (CASSANDRA-16851)
+ * Include SASI components to snapshots (CASSANDRA-15134)
+ * Make assassinate more resilient to missing tokens (CASSANDRA-16847)
+ * Exclude Jackson 1.x transitive dependency of hadoop* provided dependencies
(CASSANDRA-16854)
+ * Validate SASI tokenizer options before adding index to schema
(CASSANDRA-15135)
+ * Fixup scrub output when no data post-scrub and clear up old use of row,
which really means partition (CASSANDRA-16835)
+ * Fix ant-junit dependency issue (CASSANDRA-16827)
+ * Reduce thread contention in CommitLogSegment and HintsBuffer
(CASSANDRA-16072)
+ * Avoid sending CDC column if not enabled (CASSANDRA-16770)
+Merged from 3.0:
+ * Fix slow keycache load which blocks startup for tables with many sstables
(CASSANDRA-14898)
* Fix rare NPE caused by batchlog replay / node decomission races
(CASSANDRA-17049)
* Allow users to view permissions of the roles they created (CASSANDRA-16902)
* Fix failure handling in inter-node communication (CASSANDRA-16334)
diff --cc test/unit/org/apache/cassandra/db/KeyCacheTest.java
index ada6b5b,9cb06b9..f31df18
--- a/test/unit/org/apache/cassandra/db/KeyCacheTest.java
+++ b/test/unit/org/apache/cassandra/db/KeyCacheTest.java
@@@ -17,33 -17,49 +17,49 @@@
*/
package org.apache.cassandra.db;
+import java.io.IOException;
+ import java.util.ArrayList;
import java.util.Collection;
+ import java.util.Collections;
import java.util.HashMap;
import java.util.Iterator;
+ import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.ExecutionException;
+ import java.util.concurrent.TimeUnit;
import com.google.common.collect.ImmutableList;
-import com.google.common.util.concurrent.Uninterruptibles;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;
import org.apache.cassandra.SchemaLoader;
import org.apache.cassandra.Util;
+ import org.apache.cassandra.cache.AutoSavingCache;
+ import org.apache.cassandra.cache.ICache;
import org.apache.cassandra.cache.KeyCacheKey;
-import org.apache.cassandra.config.CFMetaData;
import org.apache.cassandra.config.DatabaseDescriptor;
-import org.apache.cassandra.config.Schema;
import org.apache.cassandra.db.compaction.OperationType;
import org.apache.cassandra.db.compaction.CompactionManager;
import org.apache.cassandra.db.lifecycle.LifecycleTransaction;
import org.apache.cassandra.exceptions.ConfigurationException;
import org.apache.cassandra.io.sstable.format.SSTableReader;
+ import org.apache.cassandra.io.util.DataInputPlus;
import org.apache.cassandra.schema.KeyspaceParams;
import org.apache.cassandra.service.CacheService;
+ import org.apache.cassandra.utils.Pair;
import org.apache.cassandra.utils.concurrent.Refs;
++import org.hamcrest.Matchers;
+ import org.mockito.Mockito;
+ import org.mockito.internal.stubbing.answers.AnswersWithDelay;
import static org.junit.Assert.assertEquals;
-import static org.junit.Assert.assertTrue;
++import static org.junit.Assert.assertNotEquals;
++import static org.junit.Assert.assertThat;
+ import static org.mockito.ArgumentMatchers.any;
+ import static org.mockito.Mockito.doAnswer;
+ import static org.mockito.Mockito.mock;
public class KeyCacheTest
{
@@@ -51,9 -68,11 +68,14 @@@
private static final String COLUMN_FAMILY1 = "Standard1";
private static final String COLUMN_FAMILY2 = "Standard2";
private static final String COLUMN_FAMILY3 = "Standard3";
+private static final String COLUMN_FAMILY4 = "Standard4";
+private static final String COLUMN_FAMILY5 = "Standard5"
[cassandra] branch cassandra-3.0 updated: Fix slow keycache load which blocks startup for tables with many sstables.
This is an automated email from the ASF dual-hosted git repository.
jolynch pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-3.0 by this push:
new 1911a88 Fix slow keycache load which blocks startup for tables with
many sstables.
1911a88 is described below
commit 1911a887e8316d343c9bfe3aca3f9d143e9f4a61
Author: Venkata Harikrishna Nukala
AuthorDate: Sat Oct 23 00:03:45 2021 +0530
Fix slow keycache load which blocks startup for tables with many sstables.
Patch by Venkata Harikrishna Nukala; reviewed by Marcus Eriksson and Joseph
Lynch for CASSANDRA-14898
---
CHANGES.txt| 1 +
build.xml | 4 +-
conf/cassandra.yaml| 7 +-
.../apache/cassandra/cache/AutoSavingCache.java| 8 +-
src/java/org/apache/cassandra/config/Config.java | 2 +
.../cassandra/config/DatabaseDescriptor.java | 11 ++
.../org/apache/cassandra/db/lifecycle/View.java| 2 +-
.../org/apache/cassandra/service/CacheService.java | 38 --
.../test/microbench/CacheLoaderBench.java | 137 +
.../unit/org/apache/cassandra/db/KeyCacheTest.java | 135 +++-
10 files changed, 330 insertions(+), 15 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index cf6a956..085e1ff 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.26:
+ * Fix slow keycache load which blocks startup for tables with many sstables
(CASSANDRA-14898)
* Fix rare NPE caused by batchlog replay / node decomission races
(CASSANDRA-17049)
* Allow users to view permissions of the roles they created (CASSANDRA-16902)
* Fix failure handling in inter-node communication (CASSANDRA-16334)
diff --git a/build.xml b/build.xml
index b42c8ea..41f530d 100644
--- a/build.xml
+++ b/build.xml
@@ -375,8 +375,8 @@
-
-
+
+
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index cc2a369..ec2157b 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -286,7 +286,12 @@ counter_cache_save_period: 7200
# If not set, the default directory is $CASSANDRA_HOME/data/saved_caches.
# saved_caches_directory: /var/lib/cassandra/saved_caches
-# commitlog_sync may be either "periodic" or "batch."
+# Number of seconds the server will wait for each cache (row, key, etc ...) to
load while starting
+# the Cassandra process. Setting this to a negative value is equivalent to
disabling all cache loading on startup
+# while still having the cache during runtime.
+# cache_load_timeout_seconds: 30
+
+# commitlog_sync may be either "periodic" or "batch."
#
# When in batch mode, Cassandra won't ack writes until the commit log
# has been fsynced to disk. It will wait
diff --git a/src/java/org/apache/cassandra/cache/AutoSavingCache.java
b/src/java/org/apache/cassandra/cache/AutoSavingCache.java
index 3da6352..3b7a6b8 100644
--- a/src/java/org/apache/cassandra/cache/AutoSavingCache.java
+++ b/src/java/org/apache/cassandra/cache/AutoSavingCache.java
@@ -198,8 +198,9 @@ public class AutoSavingCache extends
InstrumentingCache>> futures = new
ArrayDeque>>();
-while (in.available() > 0)
+ArrayDeque>> futures = new ArrayDeque<>();
+long loadByNanos = start +
TimeUnit.SECONDS.toNanos(DatabaseDescriptor.getCacheLoadTimeout());
+while (System.nanoTime() < loadByNanos && in.available() > 0)
{
//ksname and cfname are serialized by the serializers in
CacheService
//That is delegated there because there are serializer
specific conditions
@@ -257,6 +258,7 @@ public class AutoSavingCache extends
InstrumentingCache extends
InstrumentingCache> deserialize(DataInputPlus in, ColumnFamilyStore
cfs) throws IOException;
+
+default void cleanupAfterDeserialize() { }
}
}
diff --git a/src/java/org/apache/cassandra/config/Config.java
b/src/java/org/apache/cassandra/config/Config.java
index a835e5a..9e30306 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -254,6 +254,8 @@ public class Config
public volatile int counter_cache_save_period = 7200;
public volatile int counter_cache_keys_to_save = Integer.MAX_VALUE;
+public int cache_load_timeout_seconds = 30;
+
private static boolean isClientMode = false;
private static Supplier overrideLoadConfig = null;
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index b7708cd..1795c98 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescripto
[cassandra] branch trunk updated (507c6f7 -> 97b47c3)
This is an automated email from the ASF dual-hosted git repository. jolynch pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 507c6f7 Merge branch 'cassandra-4.0' into trunk new 1911a88 Fix slow keycache load which blocks startup for tables with many sstables. new 1fce84f Merge branch 'cassandra-3.0' into cassandra-3.11 new e73d05b Merge branch 'cassandra-3.11' into cassandra-4.0 new 97b47c3 Merge branch 'cassandra-4.0' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + conf/cassandra.yaml| 5 + .../apache/cassandra/cache/AutoSavingCache.java| 8 +- src/java/org/apache/cassandra/config/Config.java | 2 + .../cassandra/config/DatabaseDescriptor.java | 11 ++ .../org/apache/cassandra/db/lifecycle/View.java| 2 +- .../org/apache/cassandra/service/CacheService.java | 41 -- .../test/microbench/CacheLoaderBench.java | 137 .../unit/org/apache/cassandra/db/KeyCacheTest.java | 138 - 9 files changed, 332 insertions(+), 13 deletions(-) create mode 100644 test/microbench/org/apache/cassandra/test/microbench/CacheLoaderBench.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into cassandra-4.0
This is an automated email from the ASF dual-hosted git repository. jolynch pushed a commit to branch cassandra-4.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit e73d05bf858fa195ac2f2778027ef0d2ebcd3abd Merge: 8cef32a 1fce84f Author: Joseph Lynch AuthorDate: Thu Dec 9 10:27:29 2021 -0500 Merge branch 'cassandra-3.11' into cassandra-4.0 CHANGES.txt| 1 + conf/cassandra.yaml| 5 + .../apache/cassandra/cache/AutoSavingCache.java| 8 +- src/java/org/apache/cassandra/config/Config.java | 2 + .../cassandra/config/DatabaseDescriptor.java | 11 ++ .../org/apache/cassandra/db/lifecycle/View.java| 2 +- .../org/apache/cassandra/service/CacheService.java | 41 -- .../test/microbench/CacheLoaderBench.java | 137 .../unit/org/apache/cassandra/db/KeyCacheTest.java | 138 - 9 files changed, 332 insertions(+), 13 deletions(-) diff --cc CHANGES.txt index fb9dfaf,6c70235..055a46e --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,31 -1,17 +1,32 @@@ -3.11.12 - * Upgrade snakeyaml to 1.26 in 3.11 (CASSANDRA=17028) +4.0.2 + * Fixed broken classpath when multiple jars in build directory (CASSANDRA-17129) + * DebuggableThreadPoolExecutor does not propagate client warnings (CASSANDRA-17072) + * internode_send_buff_size_in_bytes and internode_recv_buff_size_in_bytes have new names. Backward compatibility with the old names added (CASSANDRA-17141) + * Remove unused configuration parameters from cassandra.yaml (CASSANDRA-17132) + * Queries performed with NODE_LOCAL consistency level do not update request metrics (CASSANDRA-17052) + * Fix multiple full sources can be select unexpectedly for bootstrap streaming (CASSANDRA-16945) + * Fix cassandra.yaml formatting of parameters (CASSANDRA-17131) + * Add backward compatibility for CQLSSTableWriter Date fields (CASSANDRA-17117) + * Push initial client connection messages to trace (CASSANDRA-17038) + * Correct the internode message timestamp if sending node has wrapped (CASSANDRA-16997) + * Avoid race causing us to return null in RangesAtEndpoint (CASSANDRA-16965) + * Avoid rewriting all sstables during cleanup when transient replication is enabled (CASSANDRA-16966) + * Prevent CQLSH from failure on Python 3.10 (CASSANDRA-16987) + * Avoid trying to acquire 0 permits from the rate limiter when taking snapshot (CASSANDRA-16872) + * Upgrade Caffeine to 2.5.6 (CASSANDRA-15153) + * Include SASI components to snapshots (CASSANDRA-15134) + * Fix missed wait latencies in the output of `nodetool tpstats -F` (CASSANDRA-16938) + * Remove all the state pollution between tests in SSTableReaderTest (CASSANDRA-16888) + * Delay auth setup until after gossip has settled to avoid unavailables on startup (CASSANDRA-16783) + * Fix clustering order logic in CREATE MATERIALIZED VIEW (CASSANDRA-16898) + * org.apache.cassandra.db.rows.ArrayCell#unsharedHeapSizeExcludingData includes data twice (CASSANDRA-16900) + * Exclude Jackson 1.x transitive dependency of hadoop* provided dependencies (CASSANDRA-16854) +Merged from 3.11: * Add key validation to ssstablescrub (CASSANDRA-16969) * Update Jackson from 2.9.10 to 2.12.5 (CASSANDRA-16851) - * Include SASI components to snapshots (CASSANDRA-15134) * Make assassinate more resilient to missing tokens (CASSANDRA-16847) - * Exclude Jackson 1.x transitive dependency of hadoop* provided dependencies (CASSANDRA-16854) - * Validate SASI tokenizer options before adding index to schema (CASSANDRA-15135) - * Fixup scrub output when no data post-scrub and clear up old use of row, which really means partition (CASSANDRA-16835) - * Fix ant-junit dependency issue (CASSANDRA-16827) - * Reduce thread contention in CommitLogSegment and HintsBuffer (CASSANDRA-16072) - * Avoid sending CDC column if not enabled (CASSANDRA-16770) Merged from 3.0: + * Fix slow keycache load which blocks startup for tables with many sstables (CASSANDRA-14898) * Fix rare NPE caused by batchlog replay / node decomission races (CASSANDRA-17049) * Allow users to view permissions of the roles they created (CASSANDRA-16902) * Fix failure handling in inter-node communication (CASSANDRA-16334) diff --cc conf/cassandra.yaml index 451f9df,d00c3d9..1b0871f --- a/conf/cassandra.yaml +++ b/conf/cassandra.yaml @@@ -382,22 -368,24 +382,27 @@@ counter_cache_save_period: 720 # If not set, the default directory is $CASSANDRA_HOME/data/saved_caches. # saved_caches_directory: /var/lib/cassandra/saved_caches + # Number of seconds the server will wait for each cache (row, key, etc ...) to load while starting + # the Cassandra process. Setting this to a negative value is equivalent to disabling all cache loading on startup + # while still having the cache during runtime. + # cache_load_timeout_seconds: 30 + -# commitlog_sync may be either "periodic"
[cassandra] 01/01: Merge branch 'cassandra-4.0' into trunk
This is an automated email from the ASF dual-hosted git repository.
jolynch pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 97b47c3b5f845097181130125752bd6efc1e1e47
Merge: 507c6f7 e73d05b
Author: Joseph Lynch
AuthorDate: Thu Dec 9 10:30:15 2021 -0500
Merge branch 'cassandra-4.0' into trunk
CHANGES.txt| 1 +
conf/cassandra.yaml| 5 +
.../apache/cassandra/cache/AutoSavingCache.java| 8 +-
src/java/org/apache/cassandra/config/Config.java | 2 +
.../cassandra/config/DatabaseDescriptor.java | 11 ++
.../org/apache/cassandra/db/lifecycle/View.java| 2 +-
.../org/apache/cassandra/service/CacheService.java | 41 --
.../test/microbench/CacheLoaderBench.java | 137
.../unit/org/apache/cassandra/db/KeyCacheTest.java | 138 -
9 files changed, 332 insertions(+), 13 deletions(-)
diff --cc src/java/org/apache/cassandra/cache/AutoSavingCache.java
index 0e022d4,430161f..d0b897e
--- a/src/java/org/apache/cassandra/cache/AutoSavingCache.java
+++ b/src/java/org/apache/cassandra/cache/AutoSavingCache.java
@@@ -200,8 -209,9 +200,9 @@@ public class AutoSavingCache>> futures = new
ArrayDeque>>();
- while (in.available() > 0)
+ ArrayDeque>> futures = new ArrayDeque<>();
+ long loadByNanos = start +
TimeUnit.SECONDS.toNanos(DatabaseDescriptor.getCacheLoadTimeout());
-while (System.nanoTime() < loadByNanos && in.available() > 0)
++while (nanoTime() < loadByNanos && in.available() > 0)
{
//tableId and indexName are serialized by the serializers
in CacheService
//That is delegated there because there are serializer
specific conditions
diff --cc src/java/org/apache/cassandra/config/Config.java
index 9fb57797,c03bd96..f08da7a
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@@ -332,8 -311,8 +332,10 @@@ public class Confi
public volatile int counter_cache_save_period = 7200;
public volatile int counter_cache_keys_to_save = Integer.MAX_VALUE;
+ public int cache_load_timeout_seconds = 30;
+
+public Long paxos_cache_size_in_mb = null;
+
private static boolean isClientMode = false;
private static Supplier overrideLoadConfig = null;
diff --cc src/java/org/apache/cassandra/service/CacheService.java
index 3636c13,0a281ad..1725d5f
--- a/src/java/org/apache/cassandra/service/CacheService.java
+++ b/src/java/org/apache/cassandra/service/CacheService.java
@@@ -20,10 -20,16 +20,13 @@@ package org.apache.cassandra.service
import java.io.IOException;
import java.nio.ByteBuffer;
import java.util.ArrayList;
+ import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
+ import java.util.Map;
import java.util.concurrent.Callable;
+ import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ExecutionException;
-import java.util.concurrent.Future;
-
-import com.google.common.util.concurrent.Futures;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@@ -456,17 -485,12 +484,12 @@@ public class CacheService implements Ca
reader.descriptor.version,
reader.header);
RowIndexEntry entry =
indexSerializer.deserializeForCache(input);
-return Futures.immediateFuture(Pair.create(new
KeyCacheKey(cfs.metadata(), reader.descriptor, key), entry));
+return ImmediateFuture.success(Pair.create(new
KeyCacheKey(cfs.metadata(), reader.descriptor, key), entry));
}
- private SSTableReader findDesc(int generation,
Iterable collection)
+ public void cleanupAfterDeserialize()
{
- for (SSTableReader sstable : collection)
- {
- if (sstable.descriptor.generation == generation)
- return sstable;
- }
- return null;
+ cachedSSTableReaders.clear();
}
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-4.0 updated (8cef32a -> e73d05b)
This is an automated email from the ASF dual-hosted git repository. jolynch pushed a change to branch cassandra-4.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 8cef32a Flaky CompactionsBytemanTest new 1911a88 Fix slow keycache load which blocks startup for tables with many sstables. new 1fce84f Merge branch 'cassandra-3.0' into cassandra-3.11 new e73d05b Merge branch 'cassandra-3.11' into cassandra-4.0 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + conf/cassandra.yaml| 5 + .../apache/cassandra/cache/AutoSavingCache.java| 8 +- src/java/org/apache/cassandra/config/Config.java | 2 + .../cassandra/config/DatabaseDescriptor.java | 11 ++ .../org/apache/cassandra/db/lifecycle/View.java| 2 +- .../org/apache/cassandra/service/CacheService.java | 41 -- .../test/microbench/CacheLoaderBench.java | 137 .../unit/org/apache/cassandra/db/KeyCacheTest.java | 138 - 9 files changed, 332 insertions(+), 13 deletions(-) create mode 100644 test/microbench/org/apache/cassandra/test/microbench/CacheLoaderBench.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (cd73c14 -> 1fce84f)
This is an automated email from the ASF dual-hosted git repository. jolynch pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from cd73c14 Merge branch 'cassandra-3.0' into cassandra-3.11 new 1911a88 Fix slow keycache load which blocks startup for tables with many sstables. new 1fce84f Merge branch 'cassandra-3.0' into cassandra-3.11 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + conf/cassandra.yaml| 7 +- .../apache/cassandra/cache/AutoSavingCache.java| 8 +- src/java/org/apache/cassandra/config/Config.java | 2 + .../cassandra/config/DatabaseDescriptor.java | 11 ++ .../org/apache/cassandra/db/lifecycle/View.java| 2 +- .../org/apache/cassandra/service/CacheService.java | 40 -- .../test/microbench/CacheLoaderBench.java | 137 .../unit/org/apache/cassandra/db/KeyCacheTest.java | 138 - 9 files changed, 332 insertions(+), 14 deletions(-) create mode 100644 test/microbench/org/apache/cassandra/test/microbench/CacheLoaderBench.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Update defaults for server and client TLS settings
This is an automated email from the ASF dual-hosted git repository. jolynch pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new 674b6cc Update defaults for server and client TLS settings 674b6cc is described below commit 674b6cc1a5e905a9c234c649adaad2de79cfa560 Author: Joseph Lynch AuthorDate: Fri Aug 16 18:15:55 2019 -0700 Update defaults for server and client TLS settings This fixes backwards compatibility with the 3.11 server_encryption_options while exposing the correct defaults after the 4.0 Netty refactor. Patch by Joseph Lynch; Reviewed by Ekaterina Dimitrova for CASSANDRA-15262 --- CHANGES.txt| 1 + conf/cassandra.yaml| 90 ++-- doc/source/operating/security.rst | 2 +- .../cassandra/config/DatabaseDescriptor.java | 10 +-- .../apache/cassandra/config/EncryptionOptions.java | 95 +++--- .../apache/cassandra/db/virtual/SettingsTable.java | 2 +- .../cassandra/net/InboundConnectionInitiator.java | 24 +++--- .../org/apache/cassandra/net/InboundSockets.java | 2 +- .../org/apache/cassandra/security/SSLFactory.java | 8 +- .../cassandra/service/NativeTransportService.java | 2 +- .../org/apache/cassandra/tools/BulkLoader.java | 2 +- .../org/apache/cassandra/tools/LoaderOptions.java | 2 +- .../apache/cassandra/transport/SimpleClient.java | 2 +- .../cassandra/db/virtual/SettingsTableTest.java| 7 +- .../org/apache/cassandra/net/ConnectionTest.java | 1 - .../apache/cassandra/net/MessagingServiceTest.java | 19 ++--- .../apache/cassandra/security/SSLFactoryTest.java | 19 ++--- .../cassandra/stress/util/JavaDriverClient.java| 2 +- 18 files changed, 174 insertions(+), 116 deletions(-) diff --git a/CHANGES.txt b/CHANGES.txt index 2d66d05..569d74b 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 4.0-alpha5 + * Update defaults for server and client TLS settings (CASSANDRA-15262) * Differentiate follower/initator in StreamMessageHeader (CASSANDRA-15665) * Add a startup check to detect if LZ4 uses java rather than native implementation (CASSANDRA-15884) * Fix missing topology events when running multiple nodes on the same network interface (CASSANDRA-15677) diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml index 79f6434..0778628 100644 --- a/conf/cassandra.yaml +++ b/conf/cassandra.yaml @@ -1042,61 +1042,99 @@ dynamic_snitch_reset_interval_in_ms: 60 # until the pinned host was 20% worse than the fastest. dynamic_snitch_badness_threshold: 0.1 -# Enable or disable inter-node encryption +# Configure server-to-server internode encryption +# # JVM and netty defaults for supported SSL socket protocols and cipher suites can # be replaced using custom encryption options. This is not recommended # unless you have policies in place that dictate certain settings, or # need to disable vulnerable ciphers or protocols in case the JVM cannot # be updated. +# # FIPS compliant settings can be configured at JVM level and should not # involve changing encryption settings here: # https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/FIPS.html # -# *NOTE* No custom encryption options are enabled at the moment -# The available internode options are : all, none, dc, rack -# If set to dc cassandra will encrypt the traffic between the DCs -# If set to rack cassandra will encrypt the traffic between the racks -# -# The passwords used in these options must match the passwords used when generating -# the keystore and truststore. For instructions on generating these files, see: +# **NOTE** this default configuration is an insecure configuration. If you need to +# enable server-to-server encryption generate server keystores (and truststores for mutual +# authentication) per: # http://download.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html#CreateKeystore +# Then perform the following configuration changes: # +# Step 1: Set internode_encryption= and explicitly set optional=true. Restart all nodes +# +# Step 2: Set optional=false (or remove it) and if you generated truststores and want to use mutual +# auth set require_client_auth=true. Restart all nodes server_encryption_options: -# set to true for allowing secure incoming connections -enabled: false -# If enabled and optional are both set to true, encrypted and unencrypted connections are handled on the storage_port -optional: false -# if enabled, will open up an encrypted listening socket on ssl_storage_port. Should be used +# On outbound connections, determine which type of peers to securely connect to. +# The available options are : +# none : Do not encrypt outgoing connections +# dc : Encrypt connections to peers in other
[cassandra-dtest] branch master updated: Update dtests for optional server TLS changes
This is an automated email from the ASF dual-hosted git repository.
jolynch pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git
The following commit(s) were added to refs/heads/master by this push:
new a0f5523 Update dtests for optional server TLS changes
a0f5523 is described below
commit a0f5523910dcc96f204220e3c5ba24f7be15f37b
Author: Joseph Lynch
AuthorDate: Tue May 5 23:29:16 2020 -0700
Update dtests for optional server TLS changes
Patch by Joseph Lynch; Reviewed by Ekaterina Dimitrova for CASSANDRA-15262
---
native_transport_ssl_test.py | 27 ---
sslnodetonode_test.py| 34 +++---
2 files changed, 23 insertions(+), 38 deletions(-)
diff --git a/native_transport_ssl_test.py b/native_transport_ssl_test.py
index 50e5e82..6901ef2 100644
--- a/native_transport_ssl_test.py
+++ b/native_transport_ssl_test.py
@@ -102,25 +102,14 @@ class TestNativeTransportSSL(Tester):
if enableSSL:
generate_ssl_stores(self.fixture_dtest_setup.test_path)
-# C* versions before 3.0 (CASSANDRA-10559) do not know about
-# 'client_encryption_options.optional' - so we must not add that
parameter
-if sslOptional:
-cluster.set_configuration_options({
-'client_encryption_options': {
-'enabled': True,
-'optional': sslOptional,
-'keystore':
os.path.join(self.fixture_dtest_setup.test_path, 'keystore.jks'),
-'keystore_password': 'cassandra'
-}
-})
-else:
-cluster.set_configuration_options({
-'client_encryption_options': {
-'enabled': True,
-'keystore':
os.path.join(self.fixture_dtest_setup.test_path, 'keystore.jks'),
-'keystore_password': 'cassandra'
-}
-})
+cluster.set_configuration_options({
+'client_encryption_options': {
+'enabled': True,
+'optional': sslOptional,
+'keystore':
os.path.join(self.fixture_dtest_setup.test_path, 'keystore.jks'),
+'keystore_password': 'cassandra'
+}
+})
if nativePort:
cluster.set_configuration_options({
diff --git a/sslnodetonode_test.py b/sslnodetonode_test.py
index c34fa11..7a43854 100644
--- a/sslnodetonode_test.py
+++ b/sslnodetonode_test.py
@@ -123,27 +123,24 @@ class TestNodeToNodeSSLEncryption(Tester):
credNode1 = sslkeygen.generate_credentials("127.0.0.1")
credNode2 = sslkeygen.generate_credentials("127.0.0.2",
credNode1.cakeystore, credNode1.cacert)
-# first, start cluster without TLS (either listening or connecting
-self.setup_nodes(credNode1, credNode2, internode_encryption='none',
encryption_enabled=False)
+# first, start cluster without TLS (either listening or connecting)
+# Optional should be true by default in 4.0 thanks to CASSANDRA-15262
+self.setup_nodes(credNode1, credNode2, internode_encryption='none')
self.cluster.start()
self.cql_connection(self.node1)
-# next bounce the cluster to listen on both plain/secure sockets (do
not connect secure port, yet, though)
-self.bounce_node_with_updated_config(credNode1, self.node1, 'none',
True, True)
-self.bounce_node_with_updated_config(credNode2, self.node2, 'none',
True, True)
-
# next connect with TLS for the outbound connections
-self.bounce_node_with_updated_config(credNode1, self.node1, 'all',
True, True)
-self.bounce_node_with_updated_config(credNode2, self.node2, 'all',
True, True)
+self.bounce_node_with_updated_config(credNode1, self.node1, 'all',
encryption_optional=True)
+self.bounce_node_with_updated_config(credNode2, self.node2, 'all',
encryption_optional=True)
# now shutdown the plaintext port
-self.bounce_node_with_updated_config(credNode1, self.node1, 'all',
True, False)
-self.bounce_node_with_updated_config(credNode2, self.node2, 'all',
True, False)
+self.bounce_node_with_updated_config(credNode1, self.node1, 'all',
encryption_optional=False)
+self.bounce_node_with_updated_config(credNode2, self.node2, 'all',
encryption_optional=False)
self.cluster.stop()
-def bounce_node_with_updated_config(self, credentials, node,
internode_encryption, encryption_enabled, encryption_optional):
+def bounce_node_with_updated_config(self, credentials, node,
internode_encryption, encryption_optional):
node.stop()
-self.copy_cred(credentials, node, internode_encryption,
encryption_enabled, encryption_optional)
+
[cassandra-dtest] branch master updated: Fix compression_test.TestCompression regression
This is an automated email from the ASF dual-hosted git repository.
jolynch pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git
The following commit(s) were added to refs/heads/master by this push:
new da7fcef Fix compression_test.TestCompression regression
da7fcef is described below
commit da7fcefb16d16af8924cda35c0a6a63ad553694f
Author: Joseph Lynch
AuthorDate: Mon May 4 10:59:35 2020 -0700
Fix compression_test.TestCompression regression
CASSANDRA-15379 changed behavior of how we flush for deflate, namely we
flush LZ4 and then compact to deflate off the hot path. Updated this
test to take this into account.
Patch by Joseph Lynch; Reviewed by Berenguer Blasi for CASSANDRA-15782
---
compression_test.py | 6 +-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/compression_test.py b/compression_test.py
index d865ba2..4688b72 100644
--- a/compression_test.py
+++ b/compression_test.py
@@ -103,7 +103,11 @@ class TestCompression(TestHelper):
for n in range(0, 100):
session.execute("insert into compression_opts_table (id) values
(uuid());")
-sstables = self.flush('compression_opts_table')
+self.flush('compression_opts_table')
+# Due to CASSANDRA-15379 we have to compact to get the actual table
+# compression to take effect since deflate is a slow compressor
+self.perform_node_tool_cmd(cmd='compact',
table='compression_opts_table', indexes=list())
+sstables = self.get_sstables(table='compression_opts_table',
indexes=list())
sstable_paths = self.get_table_paths('compression_opts_table')
found = False
for sstable_path in sstable_paths:
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Flush with fast compressors by default
This is an automated email from the ASF dual-hosted git repository. jolynch pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new 9c1bbf3 Flush with fast compressors by default 9c1bbf3 is described below commit 9c1bbf3ac913f9bdf7a0e0922106804af42d2c1e Author: Joseph Lynch AuthorDate: Sun Nov 3 16:37:18 2019 -0800 Flush with fast compressors by default Previously Zstd, Deflate and LZ4HC could hold up the flush thread while flushing due to their slow compression rates. With this patch we always default to flush with a fast compressor (default to LZ4) and then compress into the high compression strategy during normal compactions that follow. Note that if the existing table compressor is fast (e.g. LZ4 or Snappy) we use that. Patch by Joey Lynch; reviewed by Dinesh Joshi for CASSANDRA-15379 --- CHANGES.txt| 1 + conf/cassandra.yaml| 15 ++ doc/source/operating/compression.rst | 95 -- pylib/cqlshlib/cqlhandling.py | 1 + src/java/org/apache/cassandra/config/Config.java | 9 + .../cassandra/config/DatabaseDescriptor.java | 10 ++ .../cassandra/io/compress/DeflateCompressor.java | 10 ++ .../apache/cassandra/io/compress/ICompressor.java | 26 +++ .../cassandra/io/compress/LZ4Compressor.java | 16 ++ .../cassandra/io/compress/NoopCompressor.java | 80 + .../cassandra/io/compress/ZstdCompressor.java | 9 + .../io/sstable/format/big/BigTableWriter.java | 46 - .../apache/cassandra/schema/CompressionParams.java | 15 ++ .../config/DatabaseDescriptorRefTest.java | 4 +- .../cassandra/io/compress/CQLCompressionTest.java | 194 - .../compress/CompressedSequentialWriterTest.java | 7 + .../cassandra/io/compress/CompressorTest.java | 10 +- 17 files changed, 530 insertions(+), 18 deletions(-) diff --git a/CHANGES.txt b/CHANGES.txt index ab8a7eb..b9c8f8d 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 4.0-alpha4 + * Flush with fast compressors by default (CASSANDRA-15379) * Fix CqlInputFormat regression from the switch to system.size_estimates (CASSANDRA-15637) * Allow sending Entire SSTables over SSL (CASSANDRA-15740) * Fix CQLSH UTF-8 encoding issue for Python 2/3 compatibility (CASSANDRA-15739) diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml index f1e5864..e1d2d2d 100644 --- a/conf/cassandra.yaml +++ b/conf/cassandra.yaml @@ -425,6 +425,21 @@ commitlog_segment_size_in_mb: 32 # parameters: # - +# Compression to apply to SSTables as they flush for compressed tables. +# Note that tables without compression enabled do not respect this flag. +# +# As high ratio compressors like LZ4HC, Zstd, and Deflate can potentially +# block flushes for too long, the default is to flush with a known fast +# compressor in those cases. Options are: +# +# none : Flush without compressing blocks but while still doing checksums. +# fast : Flush with a fast compressor. If the table is already using a +#fast compressor that compressor is used. +# table: Always flush with the same compressor that the table uses. This +#was the pre 4.0 behavior. +# +# flush_compression: fast + # any class that implements the SeedProvider interface and has a # constructor that takes a Map of parameters will do. seed_provider: diff --git a/doc/source/operating/compression.rst b/doc/source/operating/compression.rst index b4308b3..74c992f 100644 --- a/doc/source/operating/compression.rst +++ b/doc/source/operating/compression.rst @@ -20,27 +20,88 @@ Compression --- Cassandra offers operators the ability to configure compression on a per-table basis. Compression reduces the size of -data on disk by compressing the SSTable in user-configurable compression ``chunk_length_in_kb``. Because Cassandra -SSTables are immutable, the CPU cost of compressing is only necessary when the SSTable is written - subsequent updates +data on disk by compressing the SSTable in user-configurable compression ``chunk_length_in_kb``. As Cassandra SSTables +are immutable, the CPU cost of compressing is only necessary when the SSTable is written - subsequent updates to data will land in different SSTables, so Cassandra will not need to decompress, overwrite, and recompress data when UPDATE commands are issued. On reads, Cassandra will locate the relevant compressed chunks on disk, decompress the full chunk, and then proceed with the remainder of the read path (merging data from disks and memtables, read repair, and so on). +Compression algorithms typically trade off between the following three areas: + +- **Compression speed**: How fast does the compression algorithm compress data. This is critical in the flush
[cassandra] branch trunk updated: Add documentation of dynamo
This is an automated email from the ASF dual-hosted git repository. jolynch pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new 0a675d5 Add documentation of dynamo 0a675d5 is described below commit 0a675d5a769b5f27c5b0dfa1fd355db69fb4ef90 Author: dvohra AuthorDate: Mon Jan 6 18:19:39 2020 -0800 Add documentation of dynamo Patch by Deepak Vohra; Reviewed by Joseph Lynch for CASSANDRA-15486 --- CHANGES.txt | 1 + doc/Dockerfile| 2 +- doc/source/architecture/dynamo.rst| 535 +- doc/source/architecture/images/ring.svg | 11 + doc/source/architecture/images/vnodes.svg | 11 + doc/source/architecture/overview.rst | 96 +- doc/source/operating/hints.rst| 12 +- doc/source/operating/read_repair.rst | 2 +- doc/source/operating/snitch.rst | 2 + 9 files changed, 574 insertions(+), 98 deletions(-) diff --git a/CHANGES.txt b/CHANGES.txt index 1c626f2..0730bff 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 4.0-alpha4 + * Add documentation of dynamo (CASSANDRA-15486) * Added documentation for Guarantees (CASSANDRA-15482) * Added documentation for audit logging (CASSANDRA-15474) * Unset GREP_OPTIONS (CASSANDRA-14487) diff --git a/doc/Dockerfile b/doc/Dockerfile index 0f109b7..8999464 100644 --- a/doc/Dockerfile +++ b/doc/Dockerfile @@ -18,5 +18,5 @@ RUN wget -qO - https://adoptopenjdk.jfrog.io/adoptopenjdk/api/gpg/key/public | a RUN apt-get clean -CMD ant gen-doc \ +CMD CASSANDRA_USE_JDK11=true ant gen-doc \ && echo "The locally built documentation can be found here:\n\n build/html/index.html\n\n" diff --git a/doc/source/architecture/dynamo.rst b/doc/source/architecture/dynamo.rst index 380abc2..5b17d9a 100644 --- a/doc/source/architecture/dynamo.rst +++ b/doc/source/architecture/dynamo.rst @@ -15,102 +15,322 @@ .. limitations under the License. Dynamo --- +== + +Apache Cassandra relies on a number of techniques from Amazon's `Dynamo +<http://courses.cse.tamu.edu/caverlee/csce438/readings/dynamo-paper.pdf>`_ +distributed storage key-value system. Each node in the Dynamo system has three +main components: + +- Request coordination over a partitioned dataset +- Ring membership and failure detection +- A local persistence (storage) engine + +Cassandra primarily draws from the first two clustering components, +while using a storage engine based on a Log Structured Merge Tree +(`LSM <http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.44.2782=rep1=pdf>`_). +In particular, Cassandra relies on Dynamo style: + +- Dataset partitioning using consistent hashing +- Multi-master replication using versioned data and tunable consistency +- Distributed cluster membership and failure detection via a gossip protocol +- Incremental scale-out on commodity hardware + +Cassandra was designed this way to meet large-scale (PiB+) business-critical +storage requirements. In particular, as applications demanded full global +replication of petabyte scale datasets along with always available low-latency +reads and writes, it became imperative to design a new kind of database model +as the relational database systems of the time struggled to meet the new +requirements of global scale applications. + +Dataset Partitioning: Consistent Hashing + + +Cassandra achieves horizontal scalability by +`partitioning <https://en.wikipedia.org/wiki/Partition_(database)>`_ +all data stored in the system using a hash function. Each partition is replicated +to multiple physical nodes, often across failure domains such as racks and even +datacenters. As every replica can independently accept mutations to every key +that it owns, every key must be versioned. Unlike in the original Dynamo paper +where deterministic versions and vector clocks were used to reconcile concurrent +updates to a key, Cassandra uses a simpler last write wins model where every +mutation is timestamped (including deletes) and then the latest version of data +is the "winning" value. Formally speaking, Cassandra uses a Last-Write-Wins Element-Set +conflict-free replicated data type for each CQL row (a.k.a `LWW-Element-Set CRDT +<https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type#LWW-Element-Set_(Last-Write-Wins-Element-Set)>`_) +to resolve conflicting mutations on replica sets. + + .. _consistent-hashing-token-ring: + +Consistent Hashing using a Token Ring +^ + +Cassandra partitions data over storage nodes using a special form of hashing +called `consistent hashing <https://en.wikipedia.org/wiki/Consistent_hashing>`_. +In naive data hashing, you typically allocate keys to buckets by taking a hash +of the
[cassandra] branch trunk updated: Add documentation of hints
This is an automated email from the ASF dual-hosted git repository. jolynch pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new 8402d1f Add documentation of hints 8402d1f is described below commit 8402d1f1456dc4da279f53dbd02f5ce7a1b2dffc Author: dvohra AuthorDate: Mon Jan 6 19:37:12 2020 -0800 Add documentation of hints Patch by Deepak Vohra; Reviewed by Joseph Lynch for CASSANDRA-15491 --- CHANGES.txt | 1 + doc/source/architecture/dynamo.rst| 2 + doc/source/operating/hints.rst| 259 +- doc/source/operating/images/hints.svg | 9 ++ doc/source/operating/metrics.rst | 4 + 5 files changed, 273 insertions(+), 2 deletions(-) diff --git a/CHANGES.txt b/CHANGES.txt index 5fe958c..f1da0b7 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 4.0-alpha4 + * Add documentation of hints (CASSANDRA-15491) * updateCoordinatorWriteLatencyTableMetric can produce misleading metrics (CASSANDRA-15569) * Added documentation for read repair and an example of full repair (CASSANDRA-15485) * Make cqlsh and cqlshlib Python 2 & 3 compatible (CASSANDRA-10190) diff --git a/doc/source/architecture/dynamo.rst b/doc/source/architecture/dynamo.rst index 12c586e..380abc2 100644 --- a/doc/source/architecture/dynamo.rst +++ b/doc/source/architecture/dynamo.rst @@ -29,6 +29,8 @@ Failure Detection .. todo:: todo +.. _token-range: + Token Ring/Ranges ^ diff --git a/doc/source/operating/hints.rst b/doc/source/operating/hints.rst index f79f18a..94ff16f 100644 --- a/doc/source/operating/hints.rst +++ b/doc/source/operating/hints.rst @@ -17,6 +17,261 @@ .. highlight:: none Hints -- += -.. todo:: todo +Hinting is a data repair technique applied during write operations. When +replica nodes are unavailable to accept a mutation, either due to failure or +more commonly routine maintenance, coordinators attempting to write to those +replicas store temporary hints on their local filesystem for later application +to the unavailable replica. Hints are an important way to help reduce the +duration of data inconsistency. Coordinators replay hints quickly after +unavailable replica nodes return to the ring. Hints are best effort, however, +and do not guarantee eventual consistency like :ref:`anti-entropy repair +` does. + +Hints are useful because of how Apache Cassandra replicates data to provide +fault tolerance, high availability and durability. Cassandra :ref:`partitions +data across the cluster ` using consistent hashing, and then +replicates keys to multiple nodes along the hash ring. To guarantee +availability, all replicas of a key can accept mutations without consensus, but +this means it is possible for some replicas to accept a mutation while others +do not. When this happens an inconsistency is introduced. + +Hints are one of the three ways, in addition to read-repair and +full/incremental anti-entropy repair, that Cassandra implements the eventual +consistency guarantee that all updates are eventually received by all replicas. +Hints, like read-repair, are best effort and not an alternative to performing +full repair, but they do help reduce the duration of inconsistency between +replicas in practice. + +Hinted Handoff +-- + +Hinted handoff is the process by which Cassandra applies hints to unavailable +nodes. + +For example, consider a mutation is to be made at ``Consistency Level`` +``LOCAL_QUORUM`` against a keyspace with ``Replication Factor`` of ``3``. +Normally the client sends the mutation to a single coordinator, who then sends +the mutation to all three replicas, and when two of the three replicas +acknowledge the mutation the coordinator responds successfully to the client. +If a replica node is unavailable, however, the coordinator stores a hint +locally to the filesystem for later application. New hints will be retained for +up to ``max_hint_window_in_ms`` of downtime (defaults to ``3 hours``). If the +unavailable replica does return to the cluster before the window expires, the +coordinator applies any pending hinted mutations against the replica to ensure +that eventual consistency is maintained. + +.. figure:: images/hints.svg +:alt: Hinted Handoff Example + +Hinted Handoff in Action + +* (``t0``): The write is sent by the client, and the coordinator sends it + to the three replicas. Unfortunately ``replica_2`` is restarting and cannot + receive the mutation. +* (``t1``): The client receives a quorum acknowledgement from the coordinator. + At this point the client believe the write to be durable and visible to reads + (which it is). +* (``t2``): After the write timeout (default ``2s``), the coordinator decides + that ``replica_2`` is unavailable and stores a hint to its local disk. +* (``t3``): Later, when ``repli
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. jolynch pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 50027f52322b7c0006c0b3c51cc6517c252e7e59 Merge: 5b660c8 036fff3 Author: Joseph Lynch AuthorDate: Tue Mar 5 16:06:36 2019 -0800 Merge branch 'cassandra-3.11' into trunk CHANGES.txt | 1 - 1 file changed, 1 deletion(-) diff --cc CHANGES.txt index a264236,2e1234b..a24eeea --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -359,7 -7,7 +359,6 @@@ Merged from 3.0 * Anti-compaction temporarily corrupts sstable state for readers (CASSANDRA-15004) Merged from 2.2: * Fixing invalid CQL in security documentation (CASSANDRA-15020) - * Make tools/bin/token-generator py2/3 compatible (CASSANDRA-15012) - * Multi-version in-JVM dtests (CASSANDRA-14937) 3.11.4 - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated: ninja fix: Remove unnecessary CHANGES.txt entry from CASSANDRA-15012
This is an automated email from the ASF dual-hosted git repository. jolynch pushed a commit to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/cassandra-3.0 by this push: new 67d6132 ninja fix: Remove unnecessary CHANGES.txt entry from CASSANDRA-15012 67d6132 is described below commit 67d613204fa4fb9584f11ec9886a0e7a0d622e92 Author: Joseph Lynch AuthorDate: Tue Mar 5 16:05:13 2019 -0800 ninja fix: Remove unnecessary CHANGES.txt entry from CASSANDRA-15012 --- CHANGES.txt | 1 - 1 file changed, 1 deletion(-) diff --git a/CHANGES.txt b/CHANGES.txt index 868aa21..027beb2 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -5,7 +5,6 @@ * Anti-compaction temporarily corrupts sstable state for readers (CASSANDRA-15004) Merged from 2.2: * Fixing invalid CQL in security documentation (CASSANDRA-15020) - * Make tools/bin/token-generator py2/3 compatible (CASSANDRA-15012) * Multi-version in-JVM dtests (CASSANDRA-14937) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (555c6d2 -> 036fff3)
This is an automated email from the ASF dual-hosted git repository. jolynch pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 555c6d2 Merge branch 'cassandra-3.0' into cassandra-3.11 new 67d6132 ninja fix: Remove unnecessary CHANGES.txt entry from CASSANDRA-15012 new 036fff3 Merge branch 'cassandra-3.0' into cassandra-3.11 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 1 - 1 file changed, 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (5b660c8 -> 50027f5)
This is an automated email from the ASF dual-hosted git repository. jolynch pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 5b660c8 Merge branch 'cassandra-3.11' into trunk new 67d6132 ninja fix: Remove unnecessary CHANGES.txt entry from CASSANDRA-15012 new 036fff3 Merge branch 'cassandra-3.0' into cassandra-3.11 new 50027f5 Merge branch 'cassandra-3.11' into trunk The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 1 - 1 file changed, 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository. jolynch pushed a commit to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 036fff393771240521f24f5837ac16040a872ecd Merge: 555c6d2 67d6132 Author: Joseph Lynch AuthorDate: Tue Mar 5 16:05:53 2019 -0800 Merge branch 'cassandra-3.0' into cassandra-3.11 CHANGES.txt | 1 - 1 file changed, 1 deletion(-) diff --cc CHANGES.txt index 4fe82e1,027beb2..2e1234b --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,13 -1,10 +1,12 @@@ -3.0.19 +3.11.5 + * Add flag to disable SASI indexes, and warnings on creation (CASSANDRA-14866) +Merged from 3.0: * Improve `nodetool status -r` speed (CASSANDRA-14847) * Improve merkle tree size and time on heap (CASSANDRA-14096) - * Add missing commands to nodetool-completion (CASSANDRA-14916) + * Add missing commands to nodetool_completion (CASSANDRA-14916) * Anti-compaction temporarily corrupts sstable state for readers (CASSANDRA-15004) - Merged from 2.2: +Merged from 2.2: * Fixing invalid CQL in security documentation (CASSANDRA-15020) - * Make tools/bin/token-generator py2/3 compatible (CASSANDRA-15012) * Multi-version in-JVM dtests (CASSANDRA-14937) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-dtest] branch master updated: Fix the token_generator_test for Cassandra 2.2
This is an automated email from the ASF dual-hosted git repository.
jolynch pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git
The following commit(s) were added to refs/heads/master by this push:
new 4df308d Fix the token_generator_test for Cassandra 2.2
4df308d is described below
commit 4df308dfe7bbc5fcf5b649fc1e4a3d6bf6daa7ee
Author: Joseph Lynch
AuthorDate: Tue Feb 5 12:11:53 2019 -0800
Fix the token_generator_test for Cassandra 2.2
Ref: Make tools/bin/token-generator py2/3 compatible
Patch by Joseph Lynch; reviewed by Stefan Podkowinski for CASSANDRA-15012
---
token_generator_test.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/token_generator_test.py b/token_generator_test.py
index b6e9025..4c0ce05 100644
--- a/token_generator_test.py
+++ b/token_generator_test.py
@@ -40,7 +40,7 @@ class TestTokenGenerator(Tester):
args.append(str(n))
logger.debug('Invoking {}'.format(args))
-token_gen_output = subprocess.check_output(args)
+token_gen_output = subprocess.check_output(args).decode()
lines = token_gen_output.split("\n")
dc_tokens = None
generated_tokens = []
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (093fe7d -> b780f62)
This is an automated email from the ASF dual-hosted git repository. jolynch pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 093fe7d Merge branch 'cassandra-3.0' into cassandra-3.11 new 595da9c Make tools/bin/token-generator py2/3 compatible new 64ebd88 Merge branch 'cassandra-2.2' into cassandra-3.0 new b780f62 Merge branch 'cassandra-3.0' into cassandra-3.11 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 1 + 1 file changed, 1 insertion(+) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-2.2 updated: Make tools/bin/token-generator py2/3 compatible
This is an automated email from the ASF dual-hosted git repository.
jolynch pushed a commit to branch cassandra-2.2
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-2.2 by this push:
new 595da9c Make tools/bin/token-generator py2/3 compatible
595da9c is described below
commit 595da9c12ed2be0ed6df791daac4f04fc7643a3f
Author: Joseph Lynch
AuthorDate: Tue Feb 5 10:51:49 2019 -0800
Make tools/bin/token-generator py2/3 compatible
Patch by Joseph Lynch; reviewed by Stefan Podkowinski for CASSANDRA-15012
---
CHANGES.txt | 1 +
tools/bin/token-generator | 47 +--
2 files changed, 30 insertions(+), 18 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 400f9d0..e875c41 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.15
+ * Make tools/bin/token-generator py2/3 compatible (CASSANDRA-15012)
* Multi-version in-JVM dtests (CASSANDRA-14937)
diff --git a/tools/bin/token-generator b/tools/bin/token-generator
index b70866d..9ebef54 100755
--- a/tools/bin/token-generator
+++ b/tools/bin/token-generator
@@ -16,14 +16,25 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+from __future__ import print_function
from __future__ import with_statement
+# Py2/3 input compatibility
+try:
+user_input = raw_input
+except NameError:
+user_input = input
import os
import sys
import math
import optparse
import webbrowser
-import urllib
+# Py2/3 quote compat
+try:
+from urllib import quote
+except ImportError:
+from urllib.parse import quote
+
from time import sleep
from itertools import cycle
from tempfile import NamedTemporaryFile
@@ -139,7 +150,7 @@ class Ring:
for (dcnum, dccount) in enumerate(self.dc_counts):
offset = dcnum * dc_offset
arcsize = self.ringrange // (dccount or 1)
-dcs.append(sorted([self.bound_token((n * arcsize + offset) -
self.ringoffset % self.ringrange) for n in xrange(dccount)]))
+dcs.append(sorted([self.bound_token((n * arcsize + offset) -
self.ringoffset % self.ringrange) for n in range(dccount)]))
return dcs
def calc_offset_tokens_onts(self):
@@ -157,10 +168,10 @@ class Ring:
def print_tokens(tokens, tokenwidth, indent=0):
indentstr = ' ' * indent
for dcnum, toklist in enumerate(tokens):
-print "%sDC #%d:" % (indentstr, dcnum + 1)
+print("%sDC #%d:" % (indentstr, dcnum + 1))
nwidth = len(str(len(toklist)))
for tnum, tok in enumerate(toklist):
-print "%s Node #%0*d: % *d" % (indentstr, nwidth, tnum + 1,
tokenwidth, tok)
+print("%s Node #%0*d: % *d" % (indentstr, nwidth, tnum + 1,
tokenwidth, tok))
def calculate_ideal_tokens(datacenters, ringoffset, ringrange, strategy):
return Ring(datacenters, ringoffset, ringrange,
strategy).calculate_offset_tokens()
@@ -171,7 +182,7 @@ def file_to_url(path):
host, path = os.path.splitunc(path)
drive, path = os.path.splitdrive(path)
path = (host or (drive + '|')) + path.replace(os.sep, '/')
-return 'file://' + urllib.quote(path, safe='/')
+return 'file://' + quote(path, safe='/')
html_template = """
@@ -270,7 +281,7 @@ def run_tests(opts):
tokensets = []
for test in tests:
-print "Test %r" % (test,)
+print("Test %r" % (test,))
tokens = calculate_ideal_tokens(test, opts.ringoffset, opts.ringrange,
opts.strat)
print_tokens(tokens, len(str(opts.ringrange)) + 1, indent=2)
tokensets.append(tokens)
@@ -279,7 +290,7 @@ def run_tests(opts):
# ===
def display_html(html, wait_time):
-with NamedTemporaryFile(suffix='.html') as f:
+with NamedTemporaryFile('wt', suffix='.html') as f:
f.write(html)
f.flush()
webbrowser.open(file_to_url(f.name), new=2)
@@ -298,28 +309,28 @@ def write_output(html, opts):
def readnum(prompt, min=None, max=None):
while True:
-x = raw_input(prompt + ' ')
+x = input(prompt + ' ')
try:
val = int(x)
except ValueError:
-print "Oops, %r is not an integer. Try again.\n" % (x,)
+print("Oops, %r is not an integer. Try again.\n" % (x,))
continue
if min is not None and val < min:
-print "Oops, the answer must be at least %d. Try again.\n" % (min,)
+print("Oops, the answer must be at least %d. Try again.\n" %
(min,))
elif max is not None and val > max:
-print "Oops, the answer must be at most %d. Try again.\n" % (max,)
+print("Oops, the answer must be at most %d. Try again.\n"
[cassandra] branch trunk updated (6b3ea1e -> b66fb62)
This is an automated email from the ASF dual-hosted git repository. jolynch pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 6b3ea1e Avoid CME when cancelling compactions for anticompaction new 595da9c Make tools/bin/token-generator py2/3 compatible new 64ebd88 Merge branch 'cassandra-2.2' into cassandra-3.0 new b780f62 Merge branch 'cassandra-3.0' into cassandra-3.11 new b66fb62 Merge branch 'cassandra-3.11' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 2 ++ 1 file changed, 2 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated (127cf8f -> 64ebd88)
This is an automated email from the ASF dual-hosted git repository. jolynch pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 127cf8f Merge branch 'cassandra-2.2' into cassandra-3.0 new 595da9c Make tools/bin/token-generator py2/3 compatible new 64ebd88 Merge branch 'cassandra-2.2' into cassandra-3.0 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 1 + 1 file changed, 1 insertion(+) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository. jolynch pushed a commit to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit b780f624862dc3a4a14589775a155eee8c68aa8a Merge: 093fe7d 64ebd88 Author: Joseph Lynch AuthorDate: Tue Mar 5 13:40:07 2019 -0800 Merge branch 'cassandra-3.0' into cassandra-3.11 CHANGES.txt | 1 + 1 file changed, 1 insertion(+) diff --cc CHANGES.txt index f520aed,51e624f..5a98a76 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,11 -1,10 +1,12 @@@ -3.0.19 +3.11.5 + * Add flag to disable SASI indexes, and warnings on creation (CASSANDRA-14866) +Merged from 3.0: * Improve `nodetool status -r` speed (CASSANDRA-14847) * Improve merkle tree size and time on heap (CASSANDRA-14096) - * Add missing commands to nodetool-completion (CASSANDRA-14916) + * Add missing commands to nodetool_completion (CASSANDRA-14916) * Anti-compaction temporarily corrupts sstable state for readers (CASSANDRA-15004) - Merged from 2.2: +Merged from 2.2: + * Make tools/bin/token-generator py2/3 compatible (CASSANDRA-15012) * Multi-version in-JVM dtests (CASSANDRA-14937) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-2.2' into cassandra-3.0
This is an automated email from the ASF dual-hosted git repository. jolynch pushed a commit to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 64ebd882139c19305e11497085b163383d33c6a3 Merge: 127cf8f 595da9c Author: Joseph Lynch AuthorDate: Tue Mar 5 13:36:15 2019 -0800 Merge branch 'cassandra-2.2' into cassandra-3.0 CHANGES.txt | 1 + 1 file changed, 1 insertion(+) diff --cc CHANGES.txt index fb71bf5,e875c41..51e624f --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,9 -1,5 +1,10 @@@ -2.2.15 +3.0.19 + * Improve `nodetool status -r` speed (CASSANDRA-14847) + * Improve merkle tree size and time on heap (CASSANDRA-14096) + * Add missing commands to nodetool-completion (CASSANDRA-14916) + * Anti-compaction temporarily corrupts sstable state for readers (CASSANDRA-15004) + Merged from 2.2: + * Make tools/bin/token-generator py2/3 compatible (CASSANDRA-15012) * Multi-version in-JVM dtests (CASSANDRA-14937) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. jolynch pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit b66fb62829ea38701d39504019f511ca41a06be6 Merge: 6b3ea1e b780f62 Author: Joseph Lynch AuthorDate: Tue Mar 5 13:41:16 2019 -0800 Merge branch 'cassandra-3.11' into trunk CHANGES.txt | 2 ++ 1 file changed, 2 insertions(+) diff --cc CHANGES.txt index 0283be6,5a98a76..207da38 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -357,6 -5,9 +357,8 @@@ Merged from 3.0 * Improve merkle tree size and time on heap (CASSANDRA-14096) * Add missing commands to nodetool_completion (CASSANDRA-14916) * Anti-compaction temporarily corrupts sstable state for readers (CASSANDRA-15004) + Merged from 2.2: + * Make tools/bin/token-generator py2/3 compatible (CASSANDRA-15012) - * Multi-version in-JVM dtests (CASSANDRA-14937) 3.11.4 - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
