[jira] [Updated] (CASSANDRA-15097) Avoid updating unchanged gossip state
[ https://issues.apache.org/jira/browse/CASSANDRA-15097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Zhuang updated CASSANDRA-15097: --- Fix Version/s: 4.0 3.11.5 3.0.19 Source Control Link: [3f70e7c72c703bc323b169a28e8754ce67d4e479|https://github.com/apache/cassandra/commit/3f70e7c72c703bc323b169a28e8754ce67d4e479] Since Version: 3.0.0 Status: Resolved (was: Ready to Commit) Resolution: Fixed Thanks [~samt] for the review. Committed as [3f70e7c|https://github.com/apache/cassandra/commit/3f70e7c72c703bc323b169a28e8754ce67d4e479]. > Avoid updating unchanged gossip state > - > > Key: CASSANDRA-15097 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15097 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip >Reporter: Jay Zhuang >Assignee: Jay Zhuang >Priority: Normal > Fix For: 3.0.19, 3.11.5, 4.0 > > > The node might get unchanged gossip states, the state might be just updated > after sending a GOSSIP_SYN, then it will get the state that is already up to > date. If the heartbeat in the GOSSIP_ACK message is updated, it will > unnecessary re-apply the same state again, which could be costly like > updating token change. > It's very likely to happen for large cluster when a node startup, as the > first gossip message will sync all endpoints tokens, it could take some time > (in our case about 200 seconds), during that time, it keeps gossip with other > node and get the full token states. Which causes lots of pending gossip tasks. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (26a134a -> ba6821b)
This is an automated email from the ASF dual-hosted git repository. jzhuang pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 26a134a Merge branch 'cassandra-3.11' into trunk add 3f70e7c Avoid updating unchanged gossip states add 71cb061 Merge branch 'cassandra-3.0' into cassandra-3.11 new ba6821b Merge branch 'cassandra-3.11' into trunk The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + src/java/org/apache/cassandra/gms/Gossiper.java| 51 ++--- .../org/apache/cassandra/gms/GossiperTest.java | 123 ++--- 3 files changed, 140 insertions(+), 35 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository.
jzhuang pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit ba6821be4f952781d573cd6ed705250de6aeb5f7
Merge: 26a134a 71cb061
Author: Jay Zhuang
AuthorDate: Thu Jul 18 21:25:55 2019 -0700
Merge branch 'cassandra-3.11' into trunk
CHANGES.txt| 1 +
src/java/org/apache/cassandra/gms/Gossiper.java| 51 ++---
.../org/apache/cassandra/gms/GossiperTest.java | 123 ++---
3 files changed, 140 insertions(+), 35 deletions(-)
diff --cc src/java/org/apache/cassandra/gms/Gossiper.java
index a6c9be7,6a862e5..062abe0
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@@ -22,14 -24,10 +22,15 @@@ import java.util.*
import java.util.Map.Entry;
import java.util.concurrent.*;
import java.util.concurrent.locks.ReentrantLock;
+import java.util.function.BooleanSupplier;
+import java.util.function.Supplier;
+import java.util.stream.Collectors;
+
import javax.annotation.Nullable;
+ import java.util.stream.Collectors;
import com.google.common.annotations.VisibleForTesting;
+import com.google.common.base.Suppliers;
import com.google.common.base.Throwables;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
@@@ -1377,25 -1289,24 +1378,41 @@@ public class Gossiper implements IFailu
Set> remoteStates =
remoteState.states();
assert remoteState.getHeartBeatState().getGeneration() ==
localState.getHeartBeatState().getGeneration();
- localState.addApplicationStates(remoteStates);
-
- //Filter out pre-4.0 versions of data for more complete 4.0 versions
- Set> filtered =
remoteStates.stream().filter(entry -> {
-switch (entry.getKey())
-{
-case INTERNAL_IP:
- return
remoteState.getApplicationState(ApplicationState.INTERNAL_ADDRESS_AND_PORT) ==
null;
-case STATUS:
-return
remoteState.getApplicationState(ApplicationState.STATUS_WITH_PORT) == null;
-case RPC_ADDRESS:
-return
remoteState.getApplicationState(ApplicationState.NATIVE_ADDRESS_AND_PORT) ==
null;
-default:
-return true;
-}
+
-// filter out the states that are already up to date (has the same or
higher version)
++
+ Set> updatedStates =
remoteStates.stream().filter(entry -> {
++// Filter out pre-4.0 versions of data for more complete 4.0
versions
++switch (entry.getKey())
++{
++case INTERNAL_IP:
++if
(remoteState.getApplicationState(ApplicationState.INTERNAL_ADDRESS_AND_PORT) !=
null) return false;
++break;
++case STATUS:
++if
(remoteState.getApplicationState(ApplicationState.STATUS_WITH_PORT) != null)
return false;
++break;
++case RPC_ADDRESS:
++if
(remoteState.getApplicationState(ApplicationState.NATIVE_ADDRESS_AND_PORT) !=
null) return false;
++break;
++default:
++break;
++}
++
++// filter out the states that are already up to date (has the
same or higher version)
+ VersionedValue local =
localState.getApplicationState(entry.getKey());
+ return (local == null || local.version <
entry.getValue().version);
-}).collect(Collectors.toSet());
+}).collect(Collectors.toSet());
- for (Entry remoteEntry : filtered)
- doOnChangeNotifications(addr, remoteEntry.getKey(),
remoteEntry.getValue());
+ if (logger.isTraceEnabled() && updatedStates.size() > 0)
+ {
+ for (Entry entry :
updatedStates)
+ {
+ logger.trace("Updating {} state version to {} for {}",
entry.getKey().toString(), entry.getValue().version, addr);
+ }
+ }
+ localState.addApplicationStates(updatedStates);
+
+ for (Entry updatedEntry :
updatedStates)
+ doOnChangeNotifications(addr, updatedEntry.getKey(),
updatedEntry.getValue());
}
// notify that a local application state is going to change (doesn't get
triggered for remote changes)
diff --cc test/unit/org/apache/cassandra/gms/GossiperTest.java
index 9c25b86,b6b3ffb..97c577c
--- a/test/unit/org/apache/cassandra/gms/GossiperTest.java
+++ b/test/unit/org/apache/cassandra/gms/GossiperTest.java
@@@ -114,158 -74,171 +115,244 @@@ public class GossiperTes
}
@Test
-public void testLargeGenerationJump() throws UnknownHostException
+public void testLargeGenerationJump() throws UnknownHostException,
InterruptedException
{
[cassandra] branch cassandra-3.0 updated: Avoid updating unchanged gossip states
This is an automated email from the ASF dual-hosted git repository.
jzhuang pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-3.0 by this push:
new 3f70e7c Avoid updating unchanged gossip states
3f70e7c is described below
commit 3f70e7c72c703bc323b169a28e8754ce67d4e479
Author: Jay Zhuang
AuthorDate: Thu Apr 18 14:59:39 2019 -0700
Avoid updating unchanged gossip states
patch by Jay Zhuang; reviewed by Sam Tunnicliffe for CASSANDRA-15097
---
CHANGES.txt| 1 +
src/java/org/apache/cassandra/gms/Gossiper.java| 21 +++-
.../org/apache/cassandra/gms/GossiperTest.java | 128 +
3 files changed, 124 insertions(+), 26 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index e0494f5..f04b489 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.19
+ * Avoid updating unchanged gossip states (CASSANDRA-15097)
* Prevent recreation of previously dropped columns with a different kind
(CASSANDRA-14948)
* Prevent client requests from blocking on executor task queue
(CASSANDRA-15013)
* Toughen up column drop/recreate type validations (CASSANDRA-15204)
diff --git a/src/java/org/apache/cassandra/gms/Gossiper.java
b/src/java/org/apache/cassandra/gms/Gossiper.java
index 4ea0a4a..c39f45a 100644
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@ -23,6 +23,7 @@ import java.util.*;
import java.util.Map.Entry;
import java.util.concurrent.*;
import java.util.concurrent.locks.ReentrantLock;
+import java.util.stream.Collectors;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Throwables;
@@ -1252,10 +1253,24 @@ public class Gossiper implements
IFailureDetectionEventListener, GossiperMBean
Set> remoteStates =
remoteState.states();
assert remoteState.getHeartBeatState().getGeneration() ==
localState.getHeartBeatState().getGeneration();
-localState.addApplicationStates(remoteStates);
-for (Entry remoteEntry :
remoteStates)
-doOnChangeNotifications(addr, remoteEntry.getKey(),
remoteEntry.getValue());
+// filter out the states that are already up to date (has the same or
higher version)
+Set> updatedStates =
remoteStates.stream().filter(entry -> {
+VersionedValue local =
localState.getApplicationState(entry.getKey());
+return (local == null || local.version < entry.getValue().version);
+}).collect(Collectors.toSet());
+
+if (logger.isTraceEnabled() && updatedStates.size() > 0)
+{
+for (Entry entry : updatedStates)
+{
+logger.trace("Updating {} state version to {} for {}",
entry.getKey().toString(), entry.getValue().version, addr);
+}
+}
+localState.addApplicationStates(updatedStates);
+
+for (Entry updatedEntry :
updatedStates)
+doOnChangeNotifications(addr, updatedEntry.getKey(),
updatedEntry.getValue());
}
// notify that a local application state is going to change (doesn't get
triggered for remote changes)
diff --git a/test/unit/org/apache/cassandra/gms/GossiperTest.java
b/test/unit/org/apache/cassandra/gms/GossiperTest.java
index f23c016..42e4483 100644
--- a/test/unit/org/apache/cassandra/gms/GossiperTest.java
+++ b/test/unit/org/apache/cassandra/gms/GossiperTest.java
@@ -21,23 +21,19 @@ package org.apache.cassandra.gms;
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.ArrayList;
+import java.util.Collections;
import java.util.List;
import java.util.UUID;
import com.google.common.collect.ImmutableMap;
-import org.junit.After;
-import org.junit.AfterClass;
import org.junit.Before;
-import org.junit.BeforeClass;
import org.junit.Test;
-import org.apache.cassandra.SchemaLoader;
import org.apache.cassandra.Util;
import org.apache.cassandra.config.DatabaseDescriptor;
import org.apache.cassandra.dht.IPartitioner;
import org.apache.cassandra.dht.RandomPartitioner;
import org.apache.cassandra.dht.Token;
-import org.apache.cassandra.exceptions.ConfigurationException;
import org.apache.cassandra.locator.TokenMetadata;
import org.apache.cassandra.service.StorageService;
@@ -68,32 +64,118 @@ public class GossiperTest
public void testLargeGenerationJump() throws UnknownHostException,
InterruptedException
{
Util.createInitialRing(ss, partitioner, endpointTokens, keyTokens,
hosts, hostIds, 2);
-InetAddress remoteHostAddress = hosts.get(1);
+try
+{
+InetAddress remoteHostAddress = hosts.get(1);
-EndpointState initialRemoteState =
Gossiper.instance.getEndpointStateForEndpoint(remoteHostAddress);
-HeartBeatState initialRemoteHeartBeat =
[cassandra] branch cassandra-3.11 updated (670dde9 -> 71cb061)
This is an automated email from the ASF dual-hosted git repository. jzhuang pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 670dde9 Merge branch 'cassandra-3.0' into cassandra-3.11 add 3f70e7c Avoid updating unchanged gossip states new 71cb061 Merge branch 'cassandra-3.0' into cassandra-3.11 The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + src/java/org/apache/cassandra/gms/Gossiper.java| 21 +++- .../org/apache/cassandra/gms/GossiperTest.java | 123 ++--- 3 files changed, 124 insertions(+), 21 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository.
jzhuang pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 71cb0616b7710366a8cd364348c864d656dc5542
Merge: 670dde9 3f70e7c
Author: Jay Zhuang
AuthorDate: Thu Jul 18 21:23:17 2019 -0700
Merge branch 'cassandra-3.0' into cassandra-3.11
CHANGES.txt| 1 +
src/java/org/apache/cassandra/gms/Gossiper.java| 21 +++-
.../org/apache/cassandra/gms/GossiperTest.java | 123 ++---
3 files changed, 124 insertions(+), 21 deletions(-)
diff --cc CHANGES.txt
index f055068,f04b489..0233c0f
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,8 -1,5 +1,9 @@@
-3.0.19
+3.11.5
+ * Fix cassandra-env.sh to use $CASSANDRA_CONF to find cassandra-jaas.config
(CASSANDRA-14305)
+ * Fixed nodetool cfstats printing index name twice (CASSANDRA-14903)
+ * Add flag to disable SASI indexes, and warnings on creation
(CASSANDRA-14866)
+Merged from 3.0:
+ * Avoid updating unchanged gossip states (CASSANDRA-15097)
* Prevent recreation of previously dropped columns with a different kind
(CASSANDRA-14948)
* Prevent client requests from blocking on executor task queue
(CASSANDRA-15013)
* Toughen up column drop/recreate type validations (CASSANDRA-15204)
diff --cc src/java/org/apache/cassandra/gms/Gossiper.java
index 5d2e997,c39f45a..6a862e5
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@@ -24,7 -23,7 +24,8 @@@ import java.util.*
import java.util.Map.Entry;
import java.util.concurrent.*;
import java.util.concurrent.locks.ReentrantLock;
+import javax.annotation.Nullable;
+ import java.util.stream.Collectors;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Throwables;
@@@ -1288,12 -1253,26 +1289,26 @@@ public class Gossiper implements IFailu
Set> remoteStates =
remoteState.states();
assert remoteState.getHeartBeatState().getGeneration() ==
localState.getHeartBeatState().getGeneration();
- localState.addApplicationStates(remoteStates);
- for (Entry remoteEntry :
remoteStates)
- doOnChangeNotifications(addr, remoteEntry.getKey(),
remoteEntry.getValue());
+ // filter out the states that are already up to date (has the same or
higher version)
+ Set> updatedStates =
remoteStates.stream().filter(entry -> {
+ VersionedValue local =
localState.getApplicationState(entry.getKey());
+ return (local == null || local.version <
entry.getValue().version);
+ }).collect(Collectors.toSet());
+
+ if (logger.isTraceEnabled() && updatedStates.size() > 0)
+ {
+ for (Entry entry :
updatedStates)
+ {
+ logger.trace("Updating {} state version to {} for {}",
entry.getKey().toString(), entry.getValue().version, addr);
+ }
+ }
+ localState.addApplicationStates(updatedStates);
+
+ for (Entry updatedEntry :
updatedStates)
+ doOnChangeNotifications(addr, updatedEntry.getKey(),
updatedEntry.getValue());
}
-
+
// notify that a local application state is going to change (doesn't get
triggered for remote changes)
private void doBeforeChangeNotifications(InetAddress addr, EndpointState
epState, ApplicationState apState, VersionedValue newValue)
{
diff --cc test/unit/org/apache/cassandra/gms/GossiperTest.java
index 448620a,42e4483..b6b3ffb
--- a/test/unit/org/apache/cassandra/gms/GossiperTest.java
+++ b/test/unit/org/apache/cassandra/gms/GossiperTest.java
@@@ -70,88 -58,124 +71,174 @@@ public class GossiperTes
public void setup()
{
tmd.clearUnsafe();
-};
+}
@Test
-public void testLargeGenerationJump() throws UnknownHostException,
InterruptedException
+public void testLargeGenerationJump() throws UnknownHostException
{
Util.createInitialRing(ss, partitioner, endpointTokens, keyTokens,
hosts, hostIds, 2);
- InetAddress remoteHostAddress = hosts.get(1);
+ try
+ {
+ InetAddress remoteHostAddress = hosts.get(1);
- EndpointState initialRemoteState =
Gossiper.instance.getEndpointStateForEndpoint(remoteHostAddress);
- HeartBeatState initialRemoteHeartBeat =
initialRemoteState.getHeartBeatState();
+ EndpointState initialRemoteState =
Gossiper.instance.getEndpointStateForEndpoint(remoteHostAddress);
+ HeartBeatState initialRemoteHeartBeat =
initialRemoteState.getHeartBeatState();
- //Util.createInitialRing should have initialized remoteHost's
HeartBeatState's generation to 1
- assertEquals(initialRemoteHeartBeat.getGeneration(), 1);
+ //Util.createInitialRing should have initialized remoteHost's
HeartBeatState's generation to 1
+
[jira] [Commented] (CASSANDRA-15225) FileUtils.close() does not handle non-IOException
[
https://issues.apache.org/jira/browse/CASSANDRA-15225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16888298#comment-16888298
]
Venkata Harikrishna Nukala commented on CASSANDRA-15225:
[~Override] Instead of delivering last exception as IOException, can we use
{{Throwable.addSuppressed()}} ?
cc: [~benedict]
> FileUtils.close() does not handle non-IOException
> -
>
> Key: CASSANDRA-15225
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15225
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Benedict
>Assignee: Liudmila Kornilova
>Priority: Normal
> Labels: pull-request-available
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This can lead to {{close}} not being invoked on remaining items
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15170) Reduce the time needed to release in-JVM dtest cluster resources after close
[ https://issues.apache.org/jira/browse/CASSANDRA-15170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16888194#comment-16888194 ] Jon Meredith commented on CASSANDRA-15170: -- Still searching for issues on trunk. The current issue I'm hitting is that the native libraries used by Netty for epoll support on Linux, and SSL in netty-tcnative hold JNI global gcroots to the instance class loader and prevent it from being garbage collected. Continuing to investigate. > Reduce the time needed to release in-JVM dtest cluster resources after close > > > Key: CASSANDRA-15170 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15170 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Normal > > There are a few issues that slow the in-JVM dtests from reclaiming metaspace > once the cluster is closed. > IsolatedExecutor issues the shutdown on a SingleExecutorThreadPool, sometimes > this thread was still running 10s after the dtest cluster was closed. > Instead, switch to a ThreadPoolExecutor with a core pool size of 0 so that > the thread executing the class loader close executes sooner. > If an OutboundTcpConnection is waiting to connect() and the endpoint is not > answering, it has to wait for a timeout before it exits. Instead it should > check the isShutdown flag and terminate early if shutdown has been requested. > In 3.0 and above, HintsCatalog.load uses java.nio.Files.list outside of a > try-with-resources construct and leaks a file handle for the directory. This > doesn't matter for normal usage, it leaks a file handle for each dtest > Instance created. > On trunk, Netty global event executor threads are still running and delay GC > for the instance class loader. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14476) ShortType and ByteType are incorrectly considered variable-length types
[
https://issues.apache.org/jira/browse/CASSANDRA-14476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16888169#comment-16888169
]
Terrence Cooke commented on CASSANDRA-14476:
I noticed in FastByteOperations.java, inside the UnsafeOperations nested class,
line 146 has the following statement:
Field f = Unsafe.class.getDeclaredField("theUnsafe");
I think this was meant to be:
Field f = UnsafeOperations.class.getDeclaredField("theUnsafe");
since the class sun.misc.Unsafe has no such field but class UnsafeOperations
does.
> ShortType and ByteType are incorrectly considered variable-length types
> ---
>
> Key: CASSANDRA-14476
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14476
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core
>Reporter: Vladimir Krivopalov
>Assignee: Jearvon Dharrie
>Priority: Low
> Labels: lhf
>
> The AbstractType class has a method valueLengthIfFixed() that returns -1 for
> data types with a variable length and a positive value for types with a fixed
> length. This is primarily used for efficient serialization and
> deserialization.
>
> It turns out that there is an inconsistency in types ShortType and ByteType
> as those are in fact fixed-length types (2 bytes and 1 byte, respectively)
> but they don't have the valueLengthIfFixed() method overloaded and it returns
> -1 as if they were of variable length.
>
> It would be good to fix that at some appropriate point, for example, when
> introducing a new version of SSTables format, to keep the meaning of the
> function consistent across data types. Saving some bytes in serialized format
> is a minor but pleasant bonus.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13096) Snapshots slow down jmx scraping
[
https://issues.apache.org/jira/browse/CASSANDRA-13096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16888067#comment-16888067
]
Serban Teodorescu commented on CASSANDRA-13096:
---
See
[https://github.com/criteo/cassandra_exporter#why-cache-metrics-results-this-is-not-the-prometheus-way-]
Snapshot scraping is expensive, and it will be triggered frequently by
Prometheus. That would explain why it gets to normal once you clear the
snapshots.
> Snapshots slow down jmx scraping
>
>
> Key: CASSANDRA-13096
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13096
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Metrics
>Reporter: Maxime Fouilleul
>Priority: Normal
> Attachments: CPU Load.png, Clear Snapshots.png, JMX Scrape
> Duration.png
>
>
> Hello,
> We are scraping the jmx metrics through a prometheus exporter and we noticed
> that some nodes became really long to answer (more than 20 seconds). After
> some investigations we do not find any hardware problem or overload issues on
> there "slow" nodes. It happens on different clusters, some with only few giga
> bytes of dataset and it does not seams to be related to a specific version
> neither as it happens on 2.1, 2.2 and 3.0 nodes.
> After some unsuccessful actions, one of our ideas was to clean the snapshots
> staying on one problematic node:
> {code}
> nodetool clearsnapshot
> {code}
> And the magic happens... as you can see in the attached diagrams, the second
> we cleared the snapshots, the CPU activity dropped immediatly and the
> duration to scrape the jmx metrics goes from +20 secs to instantaneous...
> Can you enlighten us on this issue? Once again, it appears on our three 2.1,
> 2.2 and 3.0 versions, on different volumetry and it is not systematically
> linked to the snapshots as we have some nodes with the same snapshots volume
> which are going pretty well.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15237) Professionally Support Link is Broken
Rahul Singh created CASSANDRA-15237: --- Summary: Professionally Support Link is Broken Key: CASSANDRA-15237 URL: https://issues.apache.org/jira/browse/CASSANDRA-15237 Project: Cassandra Issue Type: Improvement Components: Documentation/Website Reporter: Rahul Singh Assignee: Rahul Singh Attachments: image-2019-07-18-11-05-30-440.png, image-2019-07-18-11-06-03-030.png Link is broken. Either it should go to the new confluence page we are working on or the link be taken down. !image-2019-07-18-11-06-03-030.png! !image-2019-07-18-11-05-30-440.png! -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15172) AbstractLocalAwareExecutorService Exception During Upgrade to 3.11.4
[
https://issues.apache.org/jira/browse/CASSANDRA-15172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16888040#comment-16888040
]
Shalom commented on CASSANDRA-15172:
Trying to push this up a bit because I still want to upgrade to 3.11.4 but I
fear that this issue may recur.
If someone has any idea on what happened here or how to mitigate it, that'd be
awesome!
Thanks!
> AbstractLocalAwareExecutorService Exception During Upgrade to 3.11.4
>
>
> Key: CASSANDRA-15172
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15172
> Project: Cassandra
> Issue Type: Bug
>Reporter: Shalom
>Priority: Normal
>
> Hi All,
> This is the first time I open an issue, so apologies if I'm not following the
> rules properly.
>
> After upgrading a node from version 2.1.21 to 3.11.4, we've started seeing a
> lot of AbstractLocalAwareExecutorService exceptions. This happened right
> after the node successfully started up with the new 3.11.4 binaries.
> INFO [main] 2019-06-05 04:41:37,730 Gossiper.java:1715 - No gossip backlog;
> proceeding
> INFO [main] 2019-06-05 04:41:38,036 NativeTransportService.java:70 - Netty
> using native Epoll event loop
> INFO [main] 2019-06-05 04:41:38,117 Server.java:155 - Using Netty Version:
> [netty-buffer=netty-buffer-4.0.44.Final.452812a,
> netty-codec=netty-codec-4.0.44.Final.452812a,
> netty-codec-haproxy=netty-codec-haproxy-4.0.44.Final.452812a,
> netty-codec-http=netty-codec-http-4.0.44.Final.452812a,
> netty-codec-socks=netty-codec-socks-4.0.44.Final.452812a,
> netty-common=netty-common-4.0.44.Final.452812a,
> netty-handler=netty-handler-4.0.44.Final.452812a,
> netty-tcnative=netty-tcnative-1.1.33.Fork26.142ecbb,
> netty-transport=netty-transport-4.0.44.Final.452812a,
> netty-transport-native-epoll=netty-transport-native-epoll-4.0.44.Final.452812a,
> netty-transport-rxtx=netty-transport-rxtx-4.0.44.Final.452812a,
> netty-transport-sctp=netty-transport-sctp-4.0.44.Final.452812a,
> netty-transport-udt=netty-transport-udt-4.0.44.Final.452812a]
> INFO [main] 2019-06-05 04:41:38,118 Server.java:156 - Starting listening for
> CQL clients on /0.0.0.0:9042 (unencrypted)...
> INFO [main] 2019-06-05 04:41:38,179 CassandraDaemon.java:556 - Not starting
> RPC server as requested. Use JMX (StorageService->startRPCServer()) or
> nodetool (enablethrift) to start it
> INFO [Native-Transport-Requests-21] 2019-06-05 04:41:39,145
> AuthCache.java:161 - (Re)initializing PermissionsCache (validity
> period/update interval/max entries) (2000/2000/1000)
> INFO [OptionalTasks:1] 2019-06-05 04:41:39,729 CassandraAuthorizer.java:409
> - Converting legacy permissions data
> INFO [HANDSHAKE-/10.10.10.8] 2019-06-05 04:41:39,808
> OutboundTcpConnection.java:561 - Handshaking version with /10.10.10.8
> INFO [HANDSHAKE-/10.10.10.9] 2019-06-05 04:41:39,808
> OutboundTcpConnection.java:561 - Handshaking version with /10.10.10.9
> INFO [HANDSHAKE-dc1_02/10.10.10.6] 2019-06-05 04:41:39,809
> OutboundTcpConnection.java:561 - Handshaking version with dc1_02/10.10.10.6
> WARN [ReadStage-2] 2019-06-05 04:41:39,857
> AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread
> Thread[ReadStage-2,5,main]: {}
> java.lang.ArrayIndexOutOfBoundsException: 1
> at
> org.apache.cassandra.db.AbstractBufferClusteringPrefix.get(AbstractBufferClusteringPrefix.java:55)
> at
> org.apache.cassandra.db.LegacyLayout$LegacyRangeTombstoneList.serializedSizeCompound(LegacyLayout.java:2545)
> at
> org.apache.cassandra.db.LegacyLayout$LegacyRangeTombstoneList.serializedSize(LegacyLayout.java:2522)
> at
> org.apache.cassandra.db.LegacyLayout.serializedSizeAsLegacyPartition(LegacyLayout.java:565)
> at
> org.apache.cassandra.db.ReadResponse$Serializer.serializedSize(ReadResponse.java:446)
> at
> org.apache.cassandra.db.ReadResponse$Serializer.serializedSize(ReadResponse.java:352)
> at
> org.apache.cassandra.net.MessageOut.payloadSize(MessageOut.java:171)
> at
> org.apache.cassandra.net.OutboundTcpConnectionPool.getConnection(OutboundTcpConnectionPool.java:77)
> at
> org.apache.cassandra.net.MessagingService.getConnection(MessagingService.java:802)
> at
> org.apache.cassandra.net.MessagingService.sendOneWay(MessagingService.java:953)
> at
> org.apache.cassandra.net.MessagingService.sendReply(MessagingService.java:929)
> at
> org.apache.cassandra.db.ReadCommandVerbHandler.doVerb(ReadCommandVerbHandler.java:62)
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at
>
[jira] [Assigned] (CASSANDRA-14740) BlockingReadRepair does not maintain monotonicity during range movements
[ https://issues.apache.org/jira/browse/CASSANDRA-14740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict reassigned CASSANDRA-14740: Assignee: Benedict > BlockingReadRepair does not maintain monotonicity during range movements > > > Key: CASSANDRA-14740 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14740 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Coordination >Reporter: Benedict >Assignee: Benedict >Priority: Urgent > Labels: correctness > Fix For: 4.0 > > > The BlockingReadRepair code introduced by CASSANDRA-10726 requires that each > of the queried nodes are written to, but pending nodes are not considered. > If there is a pending range movement, one of these writes may be ‘lost’ when > the range movement completes. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-14012) Document gossip protocol
[ https://issues.apache.org/jira/browse/CASSANDRA-14012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanaka reassigned CASSANDRA-14012: -- Assignee: Tanaka > Document gossip protocol > > > Key: CASSANDRA-14012 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14012 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Documentation and Website >Reporter: Jörn Heissler >Assignee: Tanaka >Priority: Low > Labels: Documentation > > I had an issue today with two nodes communicating with each other; there's a > flaw in my configuration (wrong broadcast address). > I saw a little bit of traffic on port 7000, but I couldn't understand it for > lack of documentation. > With documentation I would have understood my issue very quickly (7f 00 01 01 > is a bad broadcast address!). But I didn't recognize those 4 bytes as the bc > address. > Could you please document the gossip protocol? -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-10248) Document compatibilities between native specs and Cassandra versions
[ https://issues.apache.org/jira/browse/CASSANDRA-10248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanaka reassigned CASSANDRA-10248: -- Assignee: Tanaka > Document compatibilities between native specs and Cassandra versions > > > Key: CASSANDRA-10248 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10248 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Documentation and Website >Reporter: Thibault Charbonnier >Assignee: Tanaka >Priority: Low > Labels: documentation > > Nowhere in the native specs is specified for which Cassandra version it is > compatible with. This has been confusing to me when implementing a given > protocol in a Lua driver, and has apparently been confusing other people [1]. > I remember seeing a table specifying which specs were compatible with which > Cassandra version somewhere in the Python driver documentation but I am > currently unable to find it. > Proposed solution: maybe include a small table in each specification file > describing the compatibilities between Cassandra and the current (and > eventually older) specs. > [1] > http://mail-archives.apache.org/mod_mbox/cassandra-dev/201504.mbox/%3ca87729c9-fa6a-4b34-bb7b-b324e154c...@datastax.com%3E -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-8557) Default install from tarball is a bit puzzling
[
https://issues.apache.org/jira/browse/CASSANDRA-8557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tanaka reassigned CASSANDRA-8557:
-
Assignee: Tanaka
> Default install from tarball is a bit puzzling
> --
>
> Key: CASSANDRA-8557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8557
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Tools, Local/Config
> Environment: Tested with Crunchbang Waldorf built on Debian 7 Wheezy.
> Java version:
> {noformat}
> $ java -version
> java version "1.8.0_25"
> Java(TM) SE Runtime Environment (build 1.8.0_25-b17)
> Java HotSpot(TM) 64-Bit Server VM (build 25.25-b02, mixed mode)
> {noformat}
> Python version:
> {noformat}
> $ python --version
> Python 2.7.3
> {noformat}
> I suspect this applies to all *nix tarball installs.
>Reporter: Chris E
>Assignee: Tanaka
>Priority: Low
> Labels: documentation, easyfix
>
> The default tarball installation of Apache Cassandra is kind of confusing for
> a newcomer.
> There are several different points of confusion:
> 1. The default installation doesn't allow remote connections.
> 2. The config file (cassandra.yaml) specifies the use of the
> AllowAllAuthenticator. This is almost immmediately replaced during setup by
> the PasswordAuthenticator. Why not start there?
> 3. The config file (cassandra.yaml) specifies the use of the
> AllowAllAuthorizer. This is almost immediately replaced during setup by the
> CassandraAuthorizer. Why not start there?
> 4. Why does cassandra-cli exist? It even tells the user "This is being
> deprecated." It's confusing figuring out whether to use cqlsh or
> cassandra-cli.
> 5. Running the cassandra script creates a background job that keeps
> running--if you control-c the script, the process continues running.
> 6. The config file (cassandra.yaml) has rpc_interface and rpc_address, and
> the docs there don't spell out that that address is *also* required for using
> remote logins from cqlsh.
> 7. On a freshly-created VM, the localhost flag to rpc_address doesn't appear
> to work (though this may be due to a misconfiguration of the VM). It seems to
> need the assigned IP address of the VM to get it accepting external
> connections.
> 8. The config file (cassandra.yaml) has the request_scheduler using
> NoScheduler--which is fine, but the docs are unclear as to whether or not
> this means that client requests won't be scheduled (at all).
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14227) Extend maximum expiration date
[
https://issues.apache.org/jira/browse/CASSANDRA-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16887860#comment-16887860
]
Laxmikant Upadhyay commented on CASSANDRA-14227:
Resuming the {{localDeletionTime}} value after update on a row with TTL does
not look feasible. Please see the below test case:
*Step 1:*
insert into tab3(id,key,value) VALUES('id2', 'key1', 2) USING TTL 63072;
(20 years).
{code:java}
sstabletojson:
{
"partition" : {
"key" : [ "id2" ],
"position" : 33
},
"rows" : [
{
"type" : "row",
"position" : 75,
"clustering" : [ "key1" ],
"liveness_info" : { "tstamp" : "2019-07-18T07:03:22.198Z", "ttl" : 63072,
"expires_at" : "2038-01-19T03:14:06Z", "expired" : false },
"cells" : [
{ "name" : "value", "value" : 2 }
]
}
]
}{code}
*Step 2:*
1. select ttl(value) from tab3 where id='id2';
This results 584043668 ! and then updating the row with fetched ttl loses the
original ttl value (20 years) so resuming {{localDeletionTime}} based on ttl
the value will not be feasible if someone chooses CAP or CAP_NOWARN option.
2. insert into tab3(id,key,value) VALUES('id2', 'key1', 3) USING TTL 584043668;
{code:java}
sstabletojson:
{
"partition" : {
"key" : [ "id2" ],
"position" : 33
},
"rows" : [
{
"type" : "row",
"position" : 75,
"clustering" : [ "key1" ],
"liveness_info" : { "tstamp" : "2019-07-18T08:54:27.921Z", "ttl" : 584043668,
"expires_at" : "2038-01-19T03:14:06Z", "expired" : false },
"cells" : [
{ "name" : "value", "value" : 3 }
]
}
]
}{code}
> Extend maximum expiration date
> --
>
> Key: CASSANDRA-14227
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14227
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Paulo Motta
>Priority: Normal
>
> The maximum expiration timestamp that can be represented by the storage
> engine is
> 2038-01-19T03:14:06+00:00 due to the encoding of {{localExpirationTime}} as
> an int32.
> On CASSANDRA-14092 we added an overflow policy which rejects requests with
> expiration above the maximum date as a temporary measure, but we should
> remove this limitation by updating the storage engine to support at least the
> maximum allowed TTL of 20 years.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15225) FileUtils.close() does not handle non-IOException
[
https://issues.apache.org/jira/browse/CASSANDRA-15225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CASSANDRA-15225:
---
Labels: pull-request-available (was: )
> FileUtils.close() does not handle non-IOException
> -
>
> Key: CASSANDRA-15225
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15225
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Benedict
>Assignee: Liudmila Kornilova
>Priority: Normal
> Labels: pull-request-available
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> This can lead to {{close}} not being invoked on remaining items
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15225) FileUtils.close() does not handle non-IOException
[
https://issues.apache.org/jira/browse/CASSANDRA-15225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liudmila Kornilova reassigned CASSANDRA-15225:
--
Assignee: Liudmila Kornilova
> FileUtils.close() does not handle non-IOException
> -
>
> Key: CASSANDRA-15225
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15225
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Benedict
>Assignee: Liudmila Kornilova
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> This can lead to {{close}} not being invoked on remaining items
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
