[jira] [Commented] (CASSANDRA-14934) Cassandra Too many open files .
[ https://issues.apache.org/jira/browse/CASSANDRA-14934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16720960#comment-16720960 ] C. Scott Andreas commented on CASSANDRA-14934: -- Hi [~Krishna29], This bug tracker is primarily used by contributors of the Apache Cassandra project toward development of the database itself. For operational concerns / questions, can you reach out to the user's list or public IRC channel for support? A member of the community may be able to help. Here's a page with information on the best channels for support: [http://cassandra.apache.org/community/] > Cassandra Too many open files . > --- > > Key: CASSANDRA-14934 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14934 > Project: Cassandra > Issue Type: Bug > Components: Configuration > Environment: This is in the Production environment . > We have observed 70k open files for the process in non-prd . > > Attached the entire logs of the open files . >Reporter: Umashankar >Priority: Major > Labels: performance > Fix For: 3.10 > > > We have 4 Node with 2 DC . > We are seeing > Caused by: java.nio.file.FileSystemException: > /apps/cassandra/data/commitlog/CommitLog-6-1544453835569.log: Too many open > files . > We have increased the ulimit to 24 k. > lsof | grep > .3.3 million records . > > Found ~ .2.5 millions of records are of TCP connection with either Socket > connection status as ESTABLISHED/LISTEN . Found that connections are of > datasource on 9042 . > > Thanks > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14934) Cassandra Too many open files .
[ https://issues.apache.org/jira/browse/CASSANDRA-14934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas updated CASSANDRA-14934: - Status: Awaiting Feedback (was: Open) > Cassandra Too many open files . > --- > > Key: CASSANDRA-14934 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14934 > Project: Cassandra > Issue Type: Bug > Components: Configuration > Environment: This is in the Production environment . > We have observed 70k open files for the process in non-prd . > > Attached the entire logs of the open files . >Reporter: Umashankar >Assignee: Umashankar >Priority: Major > Labels: performance > Fix For: 3.10 > > > We have 4 Node with 2 DC . > We are seeing > Caused by: java.nio.file.FileSystemException: > /apps/cassandra/data/commitlog/CommitLog-6-1544453835569.log: Too many open > files . > We have increased the ulimit to 24 k. > lsof | grep > .3.3 million records . > > Found ~ .2.5 millions of records are of TCP connection with either Socket > connection status as ESTABLISHED/LISTEN . Found that connections are of > datasource on 9042 . > > Thanks > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-14934) Cassandra Too many open files .
[ https://issues.apache.org/jira/browse/CASSANDRA-14934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas reassigned CASSANDRA-14934: Assignee: Umashankar > Cassandra Too many open files . > --- > > Key: CASSANDRA-14934 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14934 > Project: Cassandra > Issue Type: Bug > Components: Configuration > Environment: This is in the Production environment . > We have observed 70k open files for the process in non-prd . > > Attached the entire logs of the open files . >Reporter: Umashankar >Assignee: Umashankar >Priority: Major > Labels: performance > Fix For: 3.10 > > > We have 4 Node with 2 DC . > We are seeing > Caused by: java.nio.file.FileSystemException: > /apps/cassandra/data/commitlog/CommitLog-6-1544453835569.log: Too many open > files . > We have increased the ulimit to 24 k. > lsof | grep > .3.3 million records . > > Found ~ .2.5 millions of records are of TCP connection with either Socket > connection status as ESTABLISHED/LISTEN . Found that connections are of > datasource on 9042 . > > Thanks > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14934) Cassandra Too many open files .
Umashankar created CASSANDRA-14934: -- Summary: Cassandra Too many open files . Key: CASSANDRA-14934 URL: https://issues.apache.org/jira/browse/CASSANDRA-14934 Project: Cassandra Issue Type: Bug Components: Configuration Environment: This is in the Production environment . We have observed 70k open files for the process in non-prd . Attached the entire logs of the open files . Reporter: Umashankar Fix For: 3.10 We have 4 Node with 2 DC . We are seeing Caused by: java.nio.file.FileSystemException: /apps/cassandra/data/commitlog/CommitLog-6-1544453835569.log: Too many open files . We have increased the ulimit to 24 k. lsof | grep .3.3 million records . Found ~ .2.5 millions of records are of TCP connection with either Socket connection status as ESTABLISHED/LISTEN . Found that connections are of datasource on 9042 . Thanks -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14933) allocate_tokens_for_local_replication_factor
venky created CASSANDRA-14933:
-
Summary: allocate_tokens_for_local_replication_factor
Key: CASSANDRA-14933
URL: https://issues.apache.org/jira/browse/CASSANDRA-14933
Project: Cassandra
Issue Type: Improvement
Reporter: venky
Is Apache Cassandra supports
{color:#FF}*allocate_tokens_for_local_replication_factor* {color}**?
Currently, We are using DSE with allocate_tokens_for_local_replication_factor
and We are in the plan to move from DSE to Apache.
Is it available in Apache Cassandra or is there any plans to implement in
Apache Cassandra?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14932) removenode coordinator, and its hints data will be lost
[
https://issues.apache.org/jira/browse/CASSANDRA-14932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16720829#comment-16720829
]
C. Scott Andreas commented on CASSANDRA-14932:
--
Hi [~sunhaihong],
This bug tracker is primarily used by contributors of the Apache Cassandra
project toward development of the database itself. For operational concerns /
questions, can you reach out to the user's list or public IRC channel for
support? A member of the community may be able to help.
Here's a page with information on the best channels for support:
[http://cassandra.apache.org/community/]
> removenode coordinator, and its hints data will be lost
> ---
>
> Key: CASSANDRA-14932
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14932
> Project: Cassandra
> Issue Type: Bug
> Environment: cassandra version 3.11.3
>Reporter: sunhaihong
>Priority: Major
>
> There are four nodes in cluster. assume them are node A, B, C, D. enabled
> hinted handoff.
> 1) create a keyspace with RF=2, and create a table.
> 2) make node B, C down(nodetool stopdaemon),
> 3) login in node A with cqlsh，set CONSISTENCY ANY, insert into a row(assume
> the row will be stored in node B and C). The row was successfully inserted
> even though the node B,C was down, because the consistency level is ANY. the
> coordinator(node A) wrote hints.
> 4) make node A down(nodetool stopdaemon), then remove node A(nodetool
> removenode ${nodeA_hostId})
> 5) make node B, C come back(nodetool start)
> 6) login in any node of B, C, D. and execute select statement with partition
> key of inserted row. But there is no any data that inserted row on step 3.
>
> These steps lead to data(on step 3 was inserted row) lost.
> Is there any problem with the steps I performed above?
> If yes, How to deal with this situation？
> look forward to your reply, thanks.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-14932) removenode coordinator, and its hints data will be lost
[
https://issues.apache.org/jira/browse/CASSANDRA-14932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
C. Scott Andreas reassigned CASSANDRA-14932:
Assignee: sunhaihong
> removenode coordinator, and its hints data will be lost
> ---
>
> Key: CASSANDRA-14932
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14932
> Project: Cassandra
> Issue Type: Bug
> Environment: cassandra version 3.11.3
>Reporter: sunhaihong
>Assignee: sunhaihong
>Priority: Major
>
> There are four nodes in cluster. assume them are node A, B, C, D. enabled
> hinted handoff.
> 1) create a keyspace with RF=2, and create a table.
> 2) make node B, C down(nodetool stopdaemon),
> 3) login in node A with cqlsh，set CONSISTENCY ANY, insert into a row(assume
> the row will be stored in node B and C). The row was successfully inserted
> even though the node B,C was down, because the consistency level is ANY. the
> coordinator(node A) wrote hints.
> 4) make node A down(nodetool stopdaemon), then remove node A(nodetool
> removenode ${nodeA_hostId})
> 5) make node B, C come back(nodetool start)
> 6) login in any node of B, C, D. and execute select statement with partition
> key of inserted row. But there is no any data that inserted row on step 3.
>
> These steps lead to data(on step 3 was inserted row) lost.
> Is there any problem with the steps I performed above?
> If yes, How to deal with this situation？
> look forward to your reply, thanks.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14849) some empty/invalid bounds aren't caught by SelectStatement
[ https://issues.apache.org/jira/browse/CASSANDRA-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Blake Eggleston updated CASSANDRA-14849: Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed to trunk as [a41b861fa4d4acfbcce13dd62b1e8f48be22f8ed|https://github.com/apache/cassandra/commit/a41b861fa4d4acfbcce13dd62b1e8f48be22f8ed], thanks > some empty/invalid bounds aren't caught by SelectStatement > -- > > Key: CASSANDRA-14849 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14849 > Project: Cassandra > Issue Type: Bug > Components: CQL, Local Write-Read Paths >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Major > Fix For: 4.0 > > > Nonsensical clustering bounds like "c >= 100 AND c < 100" aren't converted to > Slices.NONE like they should be. Although this seems to be completely benign, > it is technically incorrect and complicates some testing since it can cause > memtables and sstables to return different results for the same data for > these bounds in some cases. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
cassandra git commit: Catch empty/invalid bounds in SelectStatement
Repository: cassandra
Updated Branches:
refs/heads/trunk d414c1cd8 -> a41b861fa
Catch empty/invalid bounds in SelectStatement
Patch by Blake Eggleston; Reviewed by Aleksey Yeschenko for CASSANDRA-14849
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a41b861f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a41b861f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a41b861f
Branch: refs/heads/trunk
Commit: a41b861fa4d4acfbcce13dd62b1e8f48be22f8ed
Parents: d414c1c
Author: Blake Eggleston
Authored: Thu Oct 25 13:12:02 2018 -0700
Committer: Blake Eggleston
Committed: Thu Dec 13 14:20:45 2018 -0800
--
CHANGES.txt | 1 +
.../cql3/statements/SelectStatement.java| 8 ++-
src/java/org/apache/cassandra/db/Slice.java | 10 +++-
src/java/org/apache/cassandra/db/Slices.java| 5 +-
.../cql3/statements/SelectStatementTest.java| 63
.../apache/cassandra/db/filter/SliceTest.java | 33 --
6 files changed, 108 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a41b861f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 9b4ab59..bde5b52 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0
+ * Catch empty/invalid bounds in SelectStatement (CASSANDRA-14849)
* Auto-expand replication_factor for NetworkTopologyStrategy (CASSANDRA-14303)
* Transient Replication: support EACH_QUORUM (CASSANDRA-14727)
* BufferPool: allocating thread for new chunks should acquire directly
(CASSANDRA-14832)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a41b861f/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index f847a6e..6e52ab1 100644
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@ -20,6 +20,7 @@ package org.apache.cassandra.cql3.statements;
import java.nio.ByteBuffer;
import java.util.*;
+import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.MoreObjects;
import org.slf4j.Logger;
@@ -625,7 +626,8 @@ public class SelectStatement implements CQLStatement
return new ClusteringIndexNamesFilter(clusterings, isReversed);
}
-private Slices makeSlices(QueryOptions options)
+@VisibleForTesting
+public Slices makeSlices(QueryOptions options)
throws InvalidRequestException
{
SortedSet startBounds =
restrictions.getClusteringColumnsBounds(Bound.START, options);
@@ -637,7 +639,7 @@ public class SelectStatement implements CQLStatement
{
ClusteringBound start = startBounds.first();
ClusteringBound end = endBounds.first();
-return table.comparator.compare(start, end) > 0
+return Slice.isEmpty(table.comparator, start, end)
? Slices.NONE
: Slices.with(table.comparator, Slice.make(start, end));
}
@@ -651,7 +653,7 @@ public class SelectStatement implements CQLStatement
ClusteringBound end = endIter.next();
// Ignore slices that are nonsensical
-if (table.comparator.compare(start, end) > 0)
+if (Slice.isEmpty(table.comparator, start, end))
continue;
builder.add(start, end);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a41b861f/src/java/org/apache/cassandra/db/Slice.java
--
diff --git a/src/java/org/apache/cassandra/db/Slice.java
b/src/java/org/apache/cassandra/db/Slice.java
index 2517186..44f7ac3 100644
--- a/src/java/org/apache/cassandra/db/Slice.java
+++ b/src/java/org/apache/cassandra/db/Slice.java
@@ -159,7 +159,15 @@ public class Slice
public static boolean isEmpty(ClusteringComparator comparator,
ClusteringBound start, ClusteringBound end)
{
assert start.isStart() && end.isEnd();
-return comparator.compare(end, start) <= 0;
+
+int cmp = comparator.compare(start, end);
+
+if (cmp < 0)
+return false;
+else if (cmp > 0)
+return true;
+else
+return start.isExclusive() || end.isExclusive();
}
/**
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a41b861f/src/java/org/apache/cassandra/db/Slices.java
--
diff --git a/src/java/org/apache/cas
[jira] [Commented] (CASSANDRA-14916) Add missing commands to nodetool_completion
[ https://issues.apache.org/jira/browse/CASSANDRA-14916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16720585#comment-16720585 ] jean carlo rivera ura commented on CASSANDRA-14916: --- Yes sure > Add missing commands to nodetool_completion > --- > > Key: CASSANDRA-14916 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14916 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: jean carlo rivera ura >Assignee: jean carlo rivera ura >Priority: Trivial > Labels: pull-request-available > Fix For: 4.0 > > Attachments: > 0001-adding-missing-nodetool-s-commands-to-the-file-nodet.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Since [CASSANDRA-6421|https://issues.apache.org/jira/browse/CASSANDRA-6421], > the file nodetool_completion haven't been modified in order to add the new > features of nodetool command. > I propose this patch to add those missing features. > I tried to follow the logic of the code, I hope I did not miss anything. > [~cscetbon] , I would be happy if you have a look to the patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14916) Add missing commands to nodetool_completion
[ https://issues.apache.org/jira/browse/CASSANDRA-14916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16720520#comment-16720520 ] Dinesh Joshi commented on CASSANDRA-14916: -- [~carlo_4002] thanks for your help. Could you send a PR for 3.0 and 3.11 as well? > Add missing commands to nodetool_completion > --- > > Key: CASSANDRA-14916 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14916 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: jean carlo rivera ura >Assignee: jean carlo rivera ura >Priority: Trivial > Labels: pull-request-available > Fix For: 4.0 > > Attachments: > 0001-adding-missing-nodetool-s-commands-to-the-file-nodet.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Since [CASSANDRA-6421|https://issues.apache.org/jira/browse/CASSANDRA-6421], > the file nodetool_completion haven't been modified in order to add the new > features of nodetool command. > I propose this patch to add those missing features. > I tried to follow the logic of the code, I hope I did not miss anything. > [~cscetbon] , I would be happy if you have a look to the patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14928) MigrationManager attempts to pull schema from different major version nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-14928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-14928: --- Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed as [505a03c77764351e1b649e8c7d73d0421e7bcc13|https://github.com/apache/cassandra/commit/505a03c77764351e1b649e8c7d73d0421e7bcc13]. Thanks! > MigrationManager attempts to pull schema from different major version nodes > --- > > Key: CASSANDRA-14928 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14928 > Project: Cassandra > Issue Type: Bug > Components: 4.0, Distributed Metadata >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg >Priority: Blocker > Labels: pull-request-available > Fix For: 4.0, 2.2.x, 3.0.x, 3.11.x > > > MigrationManager will do the version check against nodes it hasn't connected > to yet so it doesn't know their messaging service version. We should also > check the version in gossip as an additional layer of protection. > This causes many of the upgrade tests to fail. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[08/10] cassandra git commit: Merge branch '14928-3.0' into 14928-3.11
Merge branch '14928-3.0' into 14928-3.11
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/27c53b52
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/27c53b52
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/27c53b52
Branch: refs/heads/cassandra-3.11
Commit: 27c53b526d56bcce41e3b83b854bb4a4b0e8bf4d
Parents: 2edd6fd 1816520
Author: Ariel Weisberg
Authored: Thu Dec 13 12:34:13 2018 -0500
Committer: Ariel Weisberg
Committed: Thu Dec 13 12:34:13 2018 -0500
--
CHANGES.txt | 1 +
.../apache/cassandra/service/MigrationManager.java| 10 --
src/java/org/apache/cassandra/utils/FBUtilities.java | 14 +-
3 files changed, 22 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/27c53b52/CHANGES.txt
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/27c53b52/src/java/org/apache/cassandra/service/MigrationManager.java
--
diff --cc src/java/org/apache/cassandra/service/MigrationManager.java
index a1b3597,92d3dd5..2f71004
--- a/src/java/org/apache/cassandra/service/MigrationManager.java
+++ b/src/java/org/apache/cassandra/service/MigrationManager.java
@@@ -77,34 -77,26 +77,40 @@@ public class MigrationManage
public static void scheduleSchemaPull(InetAddress endpoint, EndpointState
state)
{
-VersionedValue value =
state.getApplicationState(ApplicationState.SCHEMA);
-
-if (!endpoint.equals(FBUtilities.getBroadcastAddress()) && value !=
null)
-maybeScheduleSchemaPull(UUID.fromString(value.value), endpoint,
state.getApplicationState(ApplicationState.RELEASE_VERSION).value);
+UUID schemaVersion = state.getSchemaVersion();
+if (!endpoint.equals(FBUtilities.getBroadcastAddress()) &&
schemaVersion != null)
- maybeScheduleSchemaPull(schemaVersion, endpoint);
++maybeScheduleSchemaPull(schemaVersion, endpoint,
state.getApplicationState(ApplicationState.RELEASE_VERSION).value);
}
/**
* If versions differ this node sends request with local migration list
to the endpoint
* and expecting to receive a list of migrations to apply locally.
*/
- private static void maybeScheduleSchemaPull(final UUID theirVersion,
final InetAddress endpoint)
+ private static void maybeScheduleSchemaPull(final UUID theirVersion,
final InetAddress endpoint, String releaseVersion)
{
+ String ourMajorVersion = FBUtilities.getReleaseVersionMajor();
+ if (!releaseVersion.startsWith(ourMajorVersion))
+ {
+ logger.debug("Not pulling schema because release version in
Gossip is not major version {}, it is {}", ourMajorVersion, releaseVersion);
+ return;
+ }
-
-if ((Schema.instance.getVersion() != null &&
Schema.instance.getVersion().equals(theirVersion)) ||
!shouldPullSchemaFrom(endpoint))
+if (Schema.instance.getVersion() == null)
+{
+logger.debug("Not pulling schema from {}, because local schama
version is not known yet",
+ endpoint);
+return;
+}
+if (Schema.instance.isSameVersion(theirVersion))
+{
+logger.debug("Not pulling schema from {}, because schema versions
match: " +
+ "local/real={}, local/compatible={}, remote={}",
+ endpoint,
+
Schema.schemaVersionToString(Schema.instance.getRealVersion()),
+
Schema.schemaVersionToString(Schema.instance.getAltVersion()),
+ Schema.schemaVersionToString(theirVersion));
+return;
+}
+if (!shouldPullSchemaFrom(endpoint))
{
logger.debug("Not pulling schema because versions match or
shouldPullSchemaFrom returned false");
return;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/27c53b52/src/java/org/apache/cassandra/utils/FBUtilities.java
--
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[01/10] cassandra git commit: MigrationManager attempts to pull schema from different major version nodes
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.2 afa456386 -> 505a03c77
refs/heads/cassandra-3.0 4f3a6feb2 -> 1816520d6
refs/heads/cassandra-3.11 2edd6fd2e -> 27c53b526
refs/heads/trunk b871decba -> d414c1cd8
MigrationManager attempts to pull schema from different major version nodes
Patch by Ariel Weisberg; Reviewed by Dinesh Joshi for CASSANDRA-14928
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/505a03c7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/505a03c7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/505a03c7
Branch: refs/heads/cassandra-2.2
Commit: 505a03c77764351e1b649e8c7d73d0421e7bcc13
Parents: afa4563
Author: Ariel Weisberg
Authored: Mon Dec 10 15:50:19 2018 -0500
Committer: Ariel Weisberg
Committed: Thu Dec 13 12:27:26 2018 -0500
--
CHANGES.txt | 1 +
.../apache/cassandra/service/MigrationManager.java| 11 +--
src/java/org/apache/cassandra/utils/FBUtilities.java | 14 +-
3 files changed, 23 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/505a03c7/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b989f3c..58309fa 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.14
+ * MigrationManager attempts to pull schema from different major version nodes
(CASSANDRA-14928)
* Don't skip entire sstables when reading backwards with mixed clustering
column order
(CASSANDRA-14910)
* Cannot perform slice reads in reverse direction against tables with
clustering columns
http://git-wip-us.apache.org/repos/asf/cassandra/blob/505a03c7/src/java/org/apache/cassandra/service/MigrationManager.java
--
diff --git a/src/java/org/apache/cassandra/service/MigrationManager.java
b/src/java/org/apache/cassandra/service/MigrationManager.java
index 4cb5ed7..d025196 100644
--- a/src/java/org/apache/cassandra/service/MigrationManager.java
+++ b/src/java/org/apache/cassandra/service/MigrationManager.java
@@ -82,15 +82,22 @@ public class MigrationManager
VersionedValue value =
state.getApplicationState(ApplicationState.SCHEMA);
if (!endpoint.equals(FBUtilities.getBroadcastAddress()) && value !=
null)
-maybeScheduleSchemaPull(UUID.fromString(value.value), endpoint);
+maybeScheduleSchemaPull(UUID.fromString(value.value), endpoint,
state.getApplicationState(ApplicationState.RELEASE_VERSION).value);
}
/**
* If versions differ this node sends request with local migration list to
the endpoint
* and expecting to receive a list of migrations to apply locally.
*/
-private static void maybeScheduleSchemaPull(final UUID theirVersion, final
InetAddress endpoint)
+private static void maybeScheduleSchemaPull(final UUID theirVersion, final
InetAddress endpoint, String releaseVersion)
{
+String ourMajorVersion = FBUtilities.getReleaseVersionMajor();
+if (!releaseVersion.startsWith(ourMajorVersion))
+{
+logger.debug("Not pulling schema because release version in Gossip
is not major version {}, it is {}", ourMajorVersion, releaseVersion);
+return;
+}
+
if ((Schema.instance.getVersion() != null &&
Schema.instance.getVersion().equals(theirVersion)) ||
!shouldPullSchemaFrom(endpoint))
{
logger.debug("Not pulling schema because versions match or
shouldPullSchemaFrom returned false");
http://git-wip-us.apache.org/repos/asf/cassandra/blob/505a03c7/src/java/org/apache/cassandra/utils/FBUtilities.java
--
diff --git a/src/java/org/apache/cassandra/utils/FBUtilities.java
b/src/java/org/apache/cassandra/utils/FBUtilities.java
index 23a2c2e..8492fed 100644
--- a/src/java/org/apache/cassandra/utils/FBUtilities.java
+++ b/src/java/org/apache/cassandra/utils/FBUtilities.java
@@ -66,6 +66,8 @@ public class FBUtilities
private static final ObjectMapper jsonMapper = new ObjectMapper(new
JsonFactory());
+public static final String UNKNOWN_RELEASE_VERSION = "Unknown";
+
public static final BigInteger TWO = new BigInteger("2");
private static final String DEFAULT_TRIGGER_DIR = "triggers";
@@ -343,7 +345,7 @@ public class FBUtilities
{
if (in == null)
{
-return System.getProperty("cassandra.releaseVersion",
"Unknown");
+return System.getProperty("cassandra.releaseVersion",
UNKNOWN_RELEASE_VERSION);
}
Properties props = new Properties();
props.
[02/10] cassandra git commit: MigrationManager attempts to pull schema from different major version nodes
MigrationManager attempts to pull schema from different major version nodes
Patch by Ariel Weisberg; Reviewed by Dinesh Joshi for CASSANDRA-14928
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/505a03c7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/505a03c7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/505a03c7
Branch: refs/heads/cassandra-3.0
Commit: 505a03c77764351e1b649e8c7d73d0421e7bcc13
Parents: afa4563
Author: Ariel Weisberg
Authored: Mon Dec 10 15:50:19 2018 -0500
Committer: Ariel Weisberg
Committed: Thu Dec 13 12:27:26 2018 -0500
--
CHANGES.txt | 1 +
.../apache/cassandra/service/MigrationManager.java| 11 +--
src/java/org/apache/cassandra/utils/FBUtilities.java | 14 +-
3 files changed, 23 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/505a03c7/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b989f3c..58309fa 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.14
+ * MigrationManager attempts to pull schema from different major version nodes
(CASSANDRA-14928)
* Don't skip entire sstables when reading backwards with mixed clustering
column order
(CASSANDRA-14910)
* Cannot perform slice reads in reverse direction against tables with
clustering columns
http://git-wip-us.apache.org/repos/asf/cassandra/blob/505a03c7/src/java/org/apache/cassandra/service/MigrationManager.java
--
diff --git a/src/java/org/apache/cassandra/service/MigrationManager.java
b/src/java/org/apache/cassandra/service/MigrationManager.java
index 4cb5ed7..d025196 100644
--- a/src/java/org/apache/cassandra/service/MigrationManager.java
+++ b/src/java/org/apache/cassandra/service/MigrationManager.java
@@ -82,15 +82,22 @@ public class MigrationManager
VersionedValue value =
state.getApplicationState(ApplicationState.SCHEMA);
if (!endpoint.equals(FBUtilities.getBroadcastAddress()) && value !=
null)
-maybeScheduleSchemaPull(UUID.fromString(value.value), endpoint);
+maybeScheduleSchemaPull(UUID.fromString(value.value), endpoint,
state.getApplicationState(ApplicationState.RELEASE_VERSION).value);
}
/**
* If versions differ this node sends request with local migration list to
the endpoint
* and expecting to receive a list of migrations to apply locally.
*/
-private static void maybeScheduleSchemaPull(final UUID theirVersion, final
InetAddress endpoint)
+private static void maybeScheduleSchemaPull(final UUID theirVersion, final
InetAddress endpoint, String releaseVersion)
{
+String ourMajorVersion = FBUtilities.getReleaseVersionMajor();
+if (!releaseVersion.startsWith(ourMajorVersion))
+{
+logger.debug("Not pulling schema because release version in Gossip
is not major version {}, it is {}", ourMajorVersion, releaseVersion);
+return;
+}
+
if ((Schema.instance.getVersion() != null &&
Schema.instance.getVersion().equals(theirVersion)) ||
!shouldPullSchemaFrom(endpoint))
{
logger.debug("Not pulling schema because versions match or
shouldPullSchemaFrom returned false");
http://git-wip-us.apache.org/repos/asf/cassandra/blob/505a03c7/src/java/org/apache/cassandra/utils/FBUtilities.java
--
diff --git a/src/java/org/apache/cassandra/utils/FBUtilities.java
b/src/java/org/apache/cassandra/utils/FBUtilities.java
index 23a2c2e..8492fed 100644
--- a/src/java/org/apache/cassandra/utils/FBUtilities.java
+++ b/src/java/org/apache/cassandra/utils/FBUtilities.java
@@ -66,6 +66,8 @@ public class FBUtilities
private static final ObjectMapper jsonMapper = new ObjectMapper(new
JsonFactory());
+public static final String UNKNOWN_RELEASE_VERSION = "Unknown";
+
public static final BigInteger TWO = new BigInteger("2");
private static final String DEFAULT_TRIGGER_DIR = "triggers";
@@ -343,7 +345,7 @@ public class FBUtilities
{
if (in == null)
{
-return System.getProperty("cassandra.releaseVersion",
"Unknown");
+return System.getProperty("cassandra.releaseVersion",
UNKNOWN_RELEASE_VERSION);
}
Properties props = new Properties();
props.load(in);
@@ -357,6 +359,16 @@ public class FBUtilities
}
}
+public static String getReleaseVersionMajor()
+{
+String releaseVersion = FBUtilities.getReleaseVersionString();
+if (FBUtilities.UNKN
[09/10] cassandra git commit: Merge branch '14928-3.0' into 14928-3.11
Merge branch '14928-3.0' into 14928-3.11
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/27c53b52
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/27c53b52
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/27c53b52
Branch: refs/heads/trunk
Commit: 27c53b526d56bcce41e3b83b854bb4a4b0e8bf4d
Parents: 2edd6fd 1816520
Author: Ariel Weisberg
Authored: Thu Dec 13 12:34:13 2018 -0500
Committer: Ariel Weisberg
Committed: Thu Dec 13 12:34:13 2018 -0500
--
CHANGES.txt | 1 +
.../apache/cassandra/service/MigrationManager.java| 10 --
src/java/org/apache/cassandra/utils/FBUtilities.java | 14 +-
3 files changed, 22 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/27c53b52/CHANGES.txt
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/27c53b52/src/java/org/apache/cassandra/service/MigrationManager.java
--
diff --cc src/java/org/apache/cassandra/service/MigrationManager.java
index a1b3597,92d3dd5..2f71004
--- a/src/java/org/apache/cassandra/service/MigrationManager.java
+++ b/src/java/org/apache/cassandra/service/MigrationManager.java
@@@ -77,34 -77,26 +77,40 @@@ public class MigrationManage
public static void scheduleSchemaPull(InetAddress endpoint, EndpointState
state)
{
-VersionedValue value =
state.getApplicationState(ApplicationState.SCHEMA);
-
-if (!endpoint.equals(FBUtilities.getBroadcastAddress()) && value !=
null)
-maybeScheduleSchemaPull(UUID.fromString(value.value), endpoint,
state.getApplicationState(ApplicationState.RELEASE_VERSION).value);
+UUID schemaVersion = state.getSchemaVersion();
+if (!endpoint.equals(FBUtilities.getBroadcastAddress()) &&
schemaVersion != null)
- maybeScheduleSchemaPull(schemaVersion, endpoint);
++maybeScheduleSchemaPull(schemaVersion, endpoint,
state.getApplicationState(ApplicationState.RELEASE_VERSION).value);
}
/**
* If versions differ this node sends request with local migration list
to the endpoint
* and expecting to receive a list of migrations to apply locally.
*/
- private static void maybeScheduleSchemaPull(final UUID theirVersion,
final InetAddress endpoint)
+ private static void maybeScheduleSchemaPull(final UUID theirVersion,
final InetAddress endpoint, String releaseVersion)
{
+ String ourMajorVersion = FBUtilities.getReleaseVersionMajor();
+ if (!releaseVersion.startsWith(ourMajorVersion))
+ {
+ logger.debug("Not pulling schema because release version in
Gossip is not major version {}, it is {}", ourMajorVersion, releaseVersion);
+ return;
+ }
-
-if ((Schema.instance.getVersion() != null &&
Schema.instance.getVersion().equals(theirVersion)) ||
!shouldPullSchemaFrom(endpoint))
+if (Schema.instance.getVersion() == null)
+{
+logger.debug("Not pulling schema from {}, because local schama
version is not known yet",
+ endpoint);
+return;
+}
+if (Schema.instance.isSameVersion(theirVersion))
+{
+logger.debug("Not pulling schema from {}, because schema versions
match: " +
+ "local/real={}, local/compatible={}, remote={}",
+ endpoint,
+
Schema.schemaVersionToString(Schema.instance.getRealVersion()),
+
Schema.schemaVersionToString(Schema.instance.getAltVersion()),
+ Schema.schemaVersionToString(theirVersion));
+return;
+}
+if (!shouldPullSchemaFrom(endpoint))
{
logger.debug("Not pulling schema because versions match or
shouldPullSchemaFrom returned false");
return;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/27c53b52/src/java/org/apache/cassandra/utils/FBUtilities.java
--
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[04/10] cassandra git commit: MigrationManager attempts to pull schema from different major version nodes
MigrationManager attempts to pull schema from different major version nodes
Patch by Ariel Weisberg; Reviewed by Dinesh Joshi for CASSANDRA-14928
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/505a03c7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/505a03c7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/505a03c7
Branch: refs/heads/trunk
Commit: 505a03c77764351e1b649e8c7d73d0421e7bcc13
Parents: afa4563
Author: Ariel Weisberg
Authored: Mon Dec 10 15:50:19 2018 -0500
Committer: Ariel Weisberg
Committed: Thu Dec 13 12:27:26 2018 -0500
--
CHANGES.txt | 1 +
.../apache/cassandra/service/MigrationManager.java| 11 +--
src/java/org/apache/cassandra/utils/FBUtilities.java | 14 +-
3 files changed, 23 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/505a03c7/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b989f3c..58309fa 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.14
+ * MigrationManager attempts to pull schema from different major version nodes
(CASSANDRA-14928)
* Don't skip entire sstables when reading backwards with mixed clustering
column order
(CASSANDRA-14910)
* Cannot perform slice reads in reverse direction against tables with
clustering columns
http://git-wip-us.apache.org/repos/asf/cassandra/blob/505a03c7/src/java/org/apache/cassandra/service/MigrationManager.java
--
diff --git a/src/java/org/apache/cassandra/service/MigrationManager.java
b/src/java/org/apache/cassandra/service/MigrationManager.java
index 4cb5ed7..d025196 100644
--- a/src/java/org/apache/cassandra/service/MigrationManager.java
+++ b/src/java/org/apache/cassandra/service/MigrationManager.java
@@ -82,15 +82,22 @@ public class MigrationManager
VersionedValue value =
state.getApplicationState(ApplicationState.SCHEMA);
if (!endpoint.equals(FBUtilities.getBroadcastAddress()) && value !=
null)
-maybeScheduleSchemaPull(UUID.fromString(value.value), endpoint);
+maybeScheduleSchemaPull(UUID.fromString(value.value), endpoint,
state.getApplicationState(ApplicationState.RELEASE_VERSION).value);
}
/**
* If versions differ this node sends request with local migration list to
the endpoint
* and expecting to receive a list of migrations to apply locally.
*/
-private static void maybeScheduleSchemaPull(final UUID theirVersion, final
InetAddress endpoint)
+private static void maybeScheduleSchemaPull(final UUID theirVersion, final
InetAddress endpoint, String releaseVersion)
{
+String ourMajorVersion = FBUtilities.getReleaseVersionMajor();
+if (!releaseVersion.startsWith(ourMajorVersion))
+{
+logger.debug("Not pulling schema because release version in Gossip
is not major version {}, it is {}", ourMajorVersion, releaseVersion);
+return;
+}
+
if ((Schema.instance.getVersion() != null &&
Schema.instance.getVersion().equals(theirVersion)) ||
!shouldPullSchemaFrom(endpoint))
{
logger.debug("Not pulling schema because versions match or
shouldPullSchemaFrom returned false");
http://git-wip-us.apache.org/repos/asf/cassandra/blob/505a03c7/src/java/org/apache/cassandra/utils/FBUtilities.java
--
diff --git a/src/java/org/apache/cassandra/utils/FBUtilities.java
b/src/java/org/apache/cassandra/utils/FBUtilities.java
index 23a2c2e..8492fed 100644
--- a/src/java/org/apache/cassandra/utils/FBUtilities.java
+++ b/src/java/org/apache/cassandra/utils/FBUtilities.java
@@ -66,6 +66,8 @@ public class FBUtilities
private static final ObjectMapper jsonMapper = new ObjectMapper(new
JsonFactory());
+public static final String UNKNOWN_RELEASE_VERSION = "Unknown";
+
public static final BigInteger TWO = new BigInteger("2");
private static final String DEFAULT_TRIGGER_DIR = "triggers";
@@ -343,7 +345,7 @@ public class FBUtilities
{
if (in == null)
{
-return System.getProperty("cassandra.releaseVersion",
"Unknown");
+return System.getProperty("cassandra.releaseVersion",
UNKNOWN_RELEASE_VERSION);
}
Properties props = new Properties();
props.load(in);
@@ -357,6 +359,16 @@ public class FBUtilities
}
}
+public static String getReleaseVersionMajor()
+{
+String releaseVersion = FBUtilities.getReleaseVersionString();
+if (FBUtilities.UNKNOWN_RELE
[07/10] cassandra git commit: Merge branch '14928-2.2' into 14928-3.0
Merge branch '14928-2.2' into 14928-3.0
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1816520d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1816520d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1816520d
Branch: refs/heads/trunk
Commit: 1816520d6c59cece5ef8346c95e4f12e7c285751
Parents: 4f3a6fe 505a03c
Author: Ariel Weisberg
Authored: Thu Dec 13 12:30:36 2018 -0500
Committer: Ariel Weisberg
Committed: Thu Dec 13 12:30:36 2018 -0500
--
CHANGES.txt | 1 +
.../apache/cassandra/service/MigrationManager.java| 11 +--
src/java/org/apache/cassandra/utils/FBUtilities.java | 14 +-
3 files changed, 23 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1816520d/CHANGES.txt
--
diff --cc CHANGES.txt
index 01d4789,58309fa..8fd0200
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,33 -1,9 +1,34 @@@
-2.2.14
+3.0.18
+ * Streaming needs to synchronise access to LifecycleTransaction
(CASSANDRA-14554)
+ * Fix cassandra-stress write hang with default options (CASSANDRA-14616)
+ * Differentiate between slices and RTs when decoding legacy bounds
(CASSANDRA-14919)
+ * CommitLogReplayer.handleReplayError should print stack traces
(CASSANDRA-14589)
+ * Netty epoll IOExceptions caused by unclean client disconnects being logged
at INFO (CASSANDRA-14909)
+ * Unfiltered.isEmpty conflicts with Row extends AbstractCollection.isEmpty
(CASSANDRA-14588)
+ * RangeTombstoneList doesn't properly clean up mergeable or superseded rts
in some cases (CASSANDRA-14894)
+ * Fix handling of collection tombstones for dropped columns from legacy
sstables (CASSANDRA-14912)
+ * Throw exception if Columns serialized subset encode more columns than
possible (CASSANDRA-14591)
+ * Drop/add column name with different Kind can result in corruption
(CASSANDRA-14843)
+ * Fix missing rows when reading 2.1 SSTables with static columns in 3.0
(CASSANDRA-14873)
+ * Move TWCS message 'No compaction necessary for bucket size' to Trace level
(CASSANDRA-14884)
+ * Sstable min/max metadata can cause data loss (CASSANDRA-14861)
+ * Dropped columns can cause reverse sstable iteration to return prematurely
(CASSANDRA-14838)
+ * Legacy sstables with multi block range tombstones create invalid bound
sequences (CASSANDRA-14823)
+ * Expand range tombstone validation checks to multiple interim request
stages (CASSANDRA-14824)
+ * Reverse order reads can return incomplete results (CASSANDRA-14803)
+ * Avoid calling iter.next() in a loop when notifying indexers about range
tombstones (CASSANDRA-14794)
+ * Fix purging semi-expired RT boundaries in reversed iterators
(CASSANDRA-14672)
+ * DESC order reads can fail to return the last Unfiltered in the partition
(CASSANDRA-14766)
+ * Fix corrupted collection deletions for dropped columns in 3.0 <-> 2.{1,2}
messages (CASSANDRA-14568)
+ * Fix corrupted static collection deletions in 3.0 <-> 2.{1,2} messages
(CASSANDRA-14568)
+ * Handle failures in parallelAllSSTableOperation
(cleanup/upgradesstables/etc) (CASSANDRA-14657)
+ * Improve TokenMetaData cache populating performance avoid long locking
(CASSANDRA-14660)
+ * Backport: Flush netty client messages immediately (not by default)
(CASSANDRA-13651)
+ * Fix static column order for SELECT * wildcard queries (CASSANDRA-14638)
+ * sstableloader should use discovered broadcast address to connect
intra-cluster (CASSANDRA-14522)
+ * Fix reading columns with non-UTF names from schema (CASSANDRA-14468)
+ Merged from 2.2:
+ * MigrationManager attempts to pull schema from different major version
nodes (CASSANDRA-14928)
- * Don't skip entire sstables when reading backwards with mixed clustering
column order
- (CASSANDRA-14910)
- * Cannot perform slice reads in reverse direction against tables with
clustering columns
- in mixed order (CASSANDRA-14899)
* Fix incorrect cqlsh results when selecting same columns multiple times
(CASSANDRA-13262)
* Returns null instead of NaN or Infinity in JSON strings (CASSANDRA-14377)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1816520d/src/java/org/apache/cassandra/service/MigrationManager.java
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1816520d/src/java/org/apache/cassandra/utils/FBUtilities.java
--
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[03/10] cassandra git commit: MigrationManager attempts to pull schema from different major version nodes
MigrationManager attempts to pull schema from different major version nodes
Patch by Ariel Weisberg; Reviewed by Dinesh Joshi for CASSANDRA-14928
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/505a03c7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/505a03c7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/505a03c7
Branch: refs/heads/cassandra-3.11
Commit: 505a03c77764351e1b649e8c7d73d0421e7bcc13
Parents: afa4563
Author: Ariel Weisberg
Authored: Mon Dec 10 15:50:19 2018 -0500
Committer: Ariel Weisberg
Committed: Thu Dec 13 12:27:26 2018 -0500
--
CHANGES.txt | 1 +
.../apache/cassandra/service/MigrationManager.java| 11 +--
src/java/org/apache/cassandra/utils/FBUtilities.java | 14 +-
3 files changed, 23 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/505a03c7/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b989f3c..58309fa 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.14
+ * MigrationManager attempts to pull schema from different major version nodes
(CASSANDRA-14928)
* Don't skip entire sstables when reading backwards with mixed clustering
column order
(CASSANDRA-14910)
* Cannot perform slice reads in reverse direction against tables with
clustering columns
http://git-wip-us.apache.org/repos/asf/cassandra/blob/505a03c7/src/java/org/apache/cassandra/service/MigrationManager.java
--
diff --git a/src/java/org/apache/cassandra/service/MigrationManager.java
b/src/java/org/apache/cassandra/service/MigrationManager.java
index 4cb5ed7..d025196 100644
--- a/src/java/org/apache/cassandra/service/MigrationManager.java
+++ b/src/java/org/apache/cassandra/service/MigrationManager.java
@@ -82,15 +82,22 @@ public class MigrationManager
VersionedValue value =
state.getApplicationState(ApplicationState.SCHEMA);
if (!endpoint.equals(FBUtilities.getBroadcastAddress()) && value !=
null)
-maybeScheduleSchemaPull(UUID.fromString(value.value), endpoint);
+maybeScheduleSchemaPull(UUID.fromString(value.value), endpoint,
state.getApplicationState(ApplicationState.RELEASE_VERSION).value);
}
/**
* If versions differ this node sends request with local migration list to
the endpoint
* and expecting to receive a list of migrations to apply locally.
*/
-private static void maybeScheduleSchemaPull(final UUID theirVersion, final
InetAddress endpoint)
+private static void maybeScheduleSchemaPull(final UUID theirVersion, final
InetAddress endpoint, String releaseVersion)
{
+String ourMajorVersion = FBUtilities.getReleaseVersionMajor();
+if (!releaseVersion.startsWith(ourMajorVersion))
+{
+logger.debug("Not pulling schema because release version in Gossip
is not major version {}, it is {}", ourMajorVersion, releaseVersion);
+return;
+}
+
if ((Schema.instance.getVersion() != null &&
Schema.instance.getVersion().equals(theirVersion)) ||
!shouldPullSchemaFrom(endpoint))
{
logger.debug("Not pulling schema because versions match or
shouldPullSchemaFrom returned false");
http://git-wip-us.apache.org/repos/asf/cassandra/blob/505a03c7/src/java/org/apache/cassandra/utils/FBUtilities.java
--
diff --git a/src/java/org/apache/cassandra/utils/FBUtilities.java
b/src/java/org/apache/cassandra/utils/FBUtilities.java
index 23a2c2e..8492fed 100644
--- a/src/java/org/apache/cassandra/utils/FBUtilities.java
+++ b/src/java/org/apache/cassandra/utils/FBUtilities.java
@@ -66,6 +66,8 @@ public class FBUtilities
private static final ObjectMapper jsonMapper = new ObjectMapper(new
JsonFactory());
+public static final String UNKNOWN_RELEASE_VERSION = "Unknown";
+
public static final BigInteger TWO = new BigInteger("2");
private static final String DEFAULT_TRIGGER_DIR = "triggers";
@@ -343,7 +345,7 @@ public class FBUtilities
{
if (in == null)
{
-return System.getProperty("cassandra.releaseVersion",
"Unknown");
+return System.getProperty("cassandra.releaseVersion",
UNKNOWN_RELEASE_VERSION);
}
Properties props = new Properties();
props.load(in);
@@ -357,6 +359,16 @@ public class FBUtilities
}
}
+public static String getReleaseVersionMajor()
+{
+String releaseVersion = FBUtilities.getReleaseVersionString();
+if (FBUtilities.UNK
[10/10] cassandra git commit: Merge branch '14928-3.11' into 14928-trunk
Merge branch '14928-3.11' into 14928-trunk
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d414c1cd
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d414c1cd
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d414c1cd
Branch: refs/heads/trunk
Commit: d414c1cd85739db6e4ec40a8d32e6e8a7045b72d
Parents: b871dec 27c53b5
Author: Ariel Weisberg
Authored: Thu Dec 13 12:56:23 2018 -0500
Committer: Ariel Weisberg
Committed: Thu Dec 13 12:56:23 2018 -0500
--
CHANGES.txt | 1 +
.../org/apache/cassandra/schema/MigrationManager.java | 10 --
src/java/org/apache/cassandra/utils/FBUtilities.java | 14 +-
3 files changed, 22 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d414c1cd/CHANGES.txt
--
diff --cc CHANGES.txt
index b8410b8,0d97d3c..9b4ab59
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -365,6 -30,8 +365,7 @@@ Merged from 3.0
* sstableloader should use discovered broadcast address to connect
intra-cluster (CASSANDRA-14522)
* Fix reading columns with non-UTF names from schema (CASSANDRA-14468)
Merged from 2.2:
+ * MigrationManager attempts to pull schema from different major version
nodes (CASSANDRA-14928)
- * Fix incorrect cqlsh results when selecting same columns multiple times
(CASSANDRA-13262)
* Returns null instead of NaN or Infinity in JSON strings (CASSANDRA-14377)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d414c1cd/src/java/org/apache/cassandra/schema/MigrationManager.java
--
diff --cc src/java/org/apache/cassandra/schema/MigrationManager.java
index a439e2e,000..32a6cf1
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/schema/MigrationManager.java
+++ b/src/java/org/apache/cassandra/schema/MigrationManager.java
@@@ -1,444 -1,0 +1,450 @@@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.schema;
+
+import java.io.IOException;
+import java.util.*;
+import java.util.concurrent.*;
+import java.lang.management.ManagementFactory;
+import java.lang.management.RuntimeMXBean;
+
+import com.google.common.util.concurrent.Futures;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.cassandra.concurrent.ScheduledExecutors;
+import org.apache.cassandra.concurrent.Stage;
+import org.apache.cassandra.concurrent.StageManager;
+import org.apache.cassandra.db.*;
+import org.apache.cassandra.exceptions.AlreadyExistsException;
+import org.apache.cassandra.exceptions.ConfigurationException;
+import org.apache.cassandra.gms.*;
+import org.apache.cassandra.io.IVersionedSerializer;
+import org.apache.cassandra.io.util.DataInputPlus;
+import org.apache.cassandra.io.util.DataOutputPlus;
+import org.apache.cassandra.locator.InetAddressAndPort;
+import org.apache.cassandra.net.MessageOut;
+import org.apache.cassandra.net.MessagingService;
+import org.apache.cassandra.schema.Keyspaces.KeyspacesDiff;
+import org.apache.cassandra.utils.FBUtilities;
+
+public class MigrationManager
+{
+private static final Logger logger =
LoggerFactory.getLogger(MigrationManager.class);
+
+public static final MigrationManager instance = new MigrationManager();
+
+private static final RuntimeMXBean runtimeMXBean =
ManagementFactory.getRuntimeMXBean();
+
+private static final int MIGRATION_DELAY_IN_MS = 6;
+
+private static final int MIGRATION_TASK_WAIT_IN_SECONDS =
Integer.parseInt(System.getProperty("cassandra.migration_task_wait_in_seconds",
"1"));
+
+private MigrationManager() {}
+
+public static void scheduleSchemaPull(InetAddressAndPort endpoint,
EndpointState state)
+{
+UUID schemaVersion = state.getSchemaVersion();
+if (!endpoint.equals(FBUtilities.getBroadcastAddressAndPort()) &&
schemaVersion != nul
[05/10] cassandra git commit: Merge branch '14928-2.2' into 14928-3.0
Merge branch '14928-2.2' into 14928-3.0
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1816520d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1816520d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1816520d
Branch: refs/heads/cassandra-3.11
Commit: 1816520d6c59cece5ef8346c95e4f12e7c285751
Parents: 4f3a6fe 505a03c
Author: Ariel Weisberg
Authored: Thu Dec 13 12:30:36 2018 -0500
Committer: Ariel Weisberg
Committed: Thu Dec 13 12:30:36 2018 -0500
--
CHANGES.txt | 1 +
.../apache/cassandra/service/MigrationManager.java| 11 +--
src/java/org/apache/cassandra/utils/FBUtilities.java | 14 +-
3 files changed, 23 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1816520d/CHANGES.txt
--
diff --cc CHANGES.txt
index 01d4789,58309fa..8fd0200
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,33 -1,9 +1,34 @@@
-2.2.14
+3.0.18
+ * Streaming needs to synchronise access to LifecycleTransaction
(CASSANDRA-14554)
+ * Fix cassandra-stress write hang with default options (CASSANDRA-14616)
+ * Differentiate between slices and RTs when decoding legacy bounds
(CASSANDRA-14919)
+ * CommitLogReplayer.handleReplayError should print stack traces
(CASSANDRA-14589)
+ * Netty epoll IOExceptions caused by unclean client disconnects being logged
at INFO (CASSANDRA-14909)
+ * Unfiltered.isEmpty conflicts with Row extends AbstractCollection.isEmpty
(CASSANDRA-14588)
+ * RangeTombstoneList doesn't properly clean up mergeable or superseded rts
in some cases (CASSANDRA-14894)
+ * Fix handling of collection tombstones for dropped columns from legacy
sstables (CASSANDRA-14912)
+ * Throw exception if Columns serialized subset encode more columns than
possible (CASSANDRA-14591)
+ * Drop/add column name with different Kind can result in corruption
(CASSANDRA-14843)
+ * Fix missing rows when reading 2.1 SSTables with static columns in 3.0
(CASSANDRA-14873)
+ * Move TWCS message 'No compaction necessary for bucket size' to Trace level
(CASSANDRA-14884)
+ * Sstable min/max metadata can cause data loss (CASSANDRA-14861)
+ * Dropped columns can cause reverse sstable iteration to return prematurely
(CASSANDRA-14838)
+ * Legacy sstables with multi block range tombstones create invalid bound
sequences (CASSANDRA-14823)
+ * Expand range tombstone validation checks to multiple interim request
stages (CASSANDRA-14824)
+ * Reverse order reads can return incomplete results (CASSANDRA-14803)
+ * Avoid calling iter.next() in a loop when notifying indexers about range
tombstones (CASSANDRA-14794)
+ * Fix purging semi-expired RT boundaries in reversed iterators
(CASSANDRA-14672)
+ * DESC order reads can fail to return the last Unfiltered in the partition
(CASSANDRA-14766)
+ * Fix corrupted collection deletions for dropped columns in 3.0 <-> 2.{1,2}
messages (CASSANDRA-14568)
+ * Fix corrupted static collection deletions in 3.0 <-> 2.{1,2} messages
(CASSANDRA-14568)
+ * Handle failures in parallelAllSSTableOperation
(cleanup/upgradesstables/etc) (CASSANDRA-14657)
+ * Improve TokenMetaData cache populating performance avoid long locking
(CASSANDRA-14660)
+ * Backport: Flush netty client messages immediately (not by default)
(CASSANDRA-13651)
+ * Fix static column order for SELECT * wildcard queries (CASSANDRA-14638)
+ * sstableloader should use discovered broadcast address to connect
intra-cluster (CASSANDRA-14522)
+ * Fix reading columns with non-UTF names from schema (CASSANDRA-14468)
+ Merged from 2.2:
+ * MigrationManager attempts to pull schema from different major version
nodes (CASSANDRA-14928)
- * Don't skip entire sstables when reading backwards with mixed clustering
column order
- (CASSANDRA-14910)
- * Cannot perform slice reads in reverse direction against tables with
clustering columns
- in mixed order (CASSANDRA-14899)
* Fix incorrect cqlsh results when selecting same columns multiple times
(CASSANDRA-13262)
* Returns null instead of NaN or Infinity in JSON strings (CASSANDRA-14377)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1816520d/src/java/org/apache/cassandra/service/MigrationManager.java
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1816520d/src/java/org/apache/cassandra/utils/FBUtilities.java
--
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[06/10] cassandra git commit: Merge branch '14928-2.2' into 14928-3.0
Merge branch '14928-2.2' into 14928-3.0
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1816520d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1816520d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1816520d
Branch: refs/heads/cassandra-3.0
Commit: 1816520d6c59cece5ef8346c95e4f12e7c285751
Parents: 4f3a6fe 505a03c
Author: Ariel Weisberg
Authored: Thu Dec 13 12:30:36 2018 -0500
Committer: Ariel Weisberg
Committed: Thu Dec 13 12:30:36 2018 -0500
--
CHANGES.txt | 1 +
.../apache/cassandra/service/MigrationManager.java| 11 +--
src/java/org/apache/cassandra/utils/FBUtilities.java | 14 +-
3 files changed, 23 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1816520d/CHANGES.txt
--
diff --cc CHANGES.txt
index 01d4789,58309fa..8fd0200
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,33 -1,9 +1,34 @@@
-2.2.14
+3.0.18
+ * Streaming needs to synchronise access to LifecycleTransaction
(CASSANDRA-14554)
+ * Fix cassandra-stress write hang with default options (CASSANDRA-14616)
+ * Differentiate between slices and RTs when decoding legacy bounds
(CASSANDRA-14919)
+ * CommitLogReplayer.handleReplayError should print stack traces
(CASSANDRA-14589)
+ * Netty epoll IOExceptions caused by unclean client disconnects being logged
at INFO (CASSANDRA-14909)
+ * Unfiltered.isEmpty conflicts with Row extends AbstractCollection.isEmpty
(CASSANDRA-14588)
+ * RangeTombstoneList doesn't properly clean up mergeable or superseded rts
in some cases (CASSANDRA-14894)
+ * Fix handling of collection tombstones for dropped columns from legacy
sstables (CASSANDRA-14912)
+ * Throw exception if Columns serialized subset encode more columns than
possible (CASSANDRA-14591)
+ * Drop/add column name with different Kind can result in corruption
(CASSANDRA-14843)
+ * Fix missing rows when reading 2.1 SSTables with static columns in 3.0
(CASSANDRA-14873)
+ * Move TWCS message 'No compaction necessary for bucket size' to Trace level
(CASSANDRA-14884)
+ * Sstable min/max metadata can cause data loss (CASSANDRA-14861)
+ * Dropped columns can cause reverse sstable iteration to return prematurely
(CASSANDRA-14838)
+ * Legacy sstables with multi block range tombstones create invalid bound
sequences (CASSANDRA-14823)
+ * Expand range tombstone validation checks to multiple interim request
stages (CASSANDRA-14824)
+ * Reverse order reads can return incomplete results (CASSANDRA-14803)
+ * Avoid calling iter.next() in a loop when notifying indexers about range
tombstones (CASSANDRA-14794)
+ * Fix purging semi-expired RT boundaries in reversed iterators
(CASSANDRA-14672)
+ * DESC order reads can fail to return the last Unfiltered in the partition
(CASSANDRA-14766)
+ * Fix corrupted collection deletions for dropped columns in 3.0 <-> 2.{1,2}
messages (CASSANDRA-14568)
+ * Fix corrupted static collection deletions in 3.0 <-> 2.{1,2} messages
(CASSANDRA-14568)
+ * Handle failures in parallelAllSSTableOperation
(cleanup/upgradesstables/etc) (CASSANDRA-14657)
+ * Improve TokenMetaData cache populating performance avoid long locking
(CASSANDRA-14660)
+ * Backport: Flush netty client messages immediately (not by default)
(CASSANDRA-13651)
+ * Fix static column order for SELECT * wildcard queries (CASSANDRA-14638)
+ * sstableloader should use discovered broadcast address to connect
intra-cluster (CASSANDRA-14522)
+ * Fix reading columns with non-UTF names from schema (CASSANDRA-14468)
+ Merged from 2.2:
+ * MigrationManager attempts to pull schema from different major version
nodes (CASSANDRA-14928)
- * Don't skip entire sstables when reading backwards with mixed clustering
column order
- (CASSANDRA-14910)
- * Cannot perform slice reads in reverse direction against tables with
clustering columns
- in mixed order (CASSANDRA-14899)
* Fix incorrect cqlsh results when selecting same columns multiple times
(CASSANDRA-13262)
* Returns null instead of NaN or Infinity in JSON strings (CASSANDRA-14377)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1816520d/src/java/org/apache/cassandra/service/MigrationManager.java
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1816520d/src/java/org/apache/cassandra/utils/FBUtilities.java
--
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14916) Add missing commands to nodetool_completion
[ https://issues.apache.org/jira/browse/CASSANDRA-14916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16720415#comment-16720415 ] Ariel Weisberg commented on CASSANDRA-14916: It looks like this needs to be fixed in earlier versions as well? I think starting at 3.0? > Add missing commands to nodetool_completion > --- > > Key: CASSANDRA-14916 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14916 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: jean carlo rivera ura >Assignee: jean carlo rivera ura >Priority: Trivial > Labels: pull-request-available > Fix For: 4.0 > > Attachments: > 0001-adding-missing-nodetool-s-commands-to-the-file-nodet.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Since [CASSANDRA-6421|https://issues.apache.org/jira/browse/CASSANDRA-6421], > the file nodetool_completion haven't been modified in order to add the new > features of nodetool command. > I propose this patch to add those missing features. > I tried to follow the logic of the code, I hope I did not miss anything. > [~cscetbon] , I would be happy if you have a look to the patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14922) In JVM dtests need to clean up after instance shutdown
[
https://issues.apache.org/jira/browse/CASSANDRA-14922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16720109#comment-16720109
]
Benedict commented on CASSANDRA-14922:
--
Do you know where the soft references originate? I wonder if there's anything
we can do to simply eliminate them.
Short of that, I'm comfortable setting the SoftRefLRUPolicyMSPerMB - the
default seems pretty strange (of 1s per MB), and probably does not seem to play
well with Metaspace, since this probably doesn't contribute to the input to the
SoftRefLRUPolicyMSPerMB. So we could have Gigabytes of free space, and judge
that we need the referent to have been unused for hours before we collect it.
It's been a while since I sperlunked in the JDK source, but it might be worth
taking a look to find out exactly what number if provides, but honestly I don't
really see a problem with setting the value to zero. The worst impact of soft
references being freed too-eagrerly is performance, it should never be
functional. We don't depend on them directly in C*, so it would only be via
the JDK. Worst case scenario, we need to investigate again at some later date
an issue with the tests.
Nice catch on the Native API also.
Overall this looks like excellent work (Though I'll leave a full review to
Alex, since he's marked himself reviewer)
> In JVM dtests need to clean up after instance shutdown
> --
>
> Key: CASSANDRA-14922
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14922
> Project: Cassandra
> Issue Type: Bug
> Components: Testing
>Reporter: Joseph Lynch
>Assignee: Joseph Lynch
>Priority: Minor
> Attachments: AllThreadsStopped.png, ClassLoadersRetaining.png,
> Leaking_Metrics_On_Shutdown.png, MainClassRetaining.png,
> Metaspace_Actually_Collected.png, OnlyThreeRootsLeft.png,
> no_more_references.png
>
>
> Currently the unit tests are failing on circleci ([example
> one|https://circleci.com/gh/jolynch/cassandra/300#tests/containers/1],
> [example
> two|https://circleci.com/gh/rustyrazorblade/cassandra/44#tests/containers/1])
> because we use a small container (medium) for unit tests by default and the
> in JVM dtests are leaking a few hundred megabytes of memory per test right
> now. This is not a big deal because the dtest runs with the larger containers
> continue to function fine as well as local testing as the number of in JVM
> dtests is not yet high enough to cause a problem with more than 2GB of
> available heap. However we should fix the memory leak so that going forwards
> we can add more in JVM dtests without worry.
> I've been working with [~ifesdjeen] to debug, and the issue appears to be
> unreleased Table/Keyspace metrics (screenshot showing the leak attached). I
> believe that we have a few potential issues that are leading to the leaks:
> 1. The
> [{{Instance::shutdown}}|https://github.com/apache/cassandra/blob/f22fec927de7ac29120c2f34de5b8cc1c695/test/distributed/org/apache/cassandra/distributed/Instance.java#L328-L354]
> method is not successfully cleaning up all the metrics created by the
> {{CassandraMetricsRegistry}}
> 2. The
> [{{TestCluster::close}}|https://github.com/apache/cassandra/blob/f22fec927de7ac29120c2f34de5b8cc1c695/test/distributed/org/apache/cassandra/distributed/TestCluster.java#L283]
> method is not waiting for all the instances to finish shutting down and
> cleaning up before continuing on
> 3. I'm not sure if this is an issue assuming we clear all metrics, but
> [{{TableMetrics::release}}|https://github.com/apache/cassandra/blob/4ae229f5cd270c2b43475b3f752a7b228de260ea/src/java/org/apache/cassandra/metrics/TableMetrics.java#L951]
> does not release all the metric references (which could leak them)
> I am working on a patch which shuts down everything and assures that we do
> not leak memory.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14932) removenode coordinator, and its hints data will be lost
[
https://issues.apache.org/jira/browse/CASSANDRA-14932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sunhaihong updated CASSANDRA-14932:
---
Priority: Major (was: Minor)
> removenode coordinator, and its hints data will be lost
> ---
>
> Key: CASSANDRA-14932
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14932
> Project: Cassandra
> Issue Type: Bug
> Environment: cassandra version 3.11.3
>Reporter: sunhaihong
>Priority: Major
>
> There are four nodes in cluster. assume them are node A, B, C, D. enabled
> hinted handoff.
> 1) create a keyspace with RF=2, and create a table.
> 2) make node B, C down(nodetool stopdaemon),
> 3) login in node A with cqlsh，set CONSISTENCY ANY, insert into a row(assume
> the row will be stored in node B and C). The row was successfully inserted
> even though the node B,C was down, because the consistency level is ANY. the
> coordinator(node A) wrote hints.
> 4) make node A down(nodetool stopdaemon), then remove node A(nodetool
> removenode ${nodeA_hostId})
> 5) make node B, C come back(nodetool start)
> 6) login in any node of B, C, D. and execute select statement with partition
> key of inserted row. But there is no any data that inserted row on step 3.
>
> These steps lead to data(on step 3 was inserted row) lost.
> Is there any problem with the steps I performed above?
> If yes, How to deal with this situation？
> look forward to your reply, thanks.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14932) removenode coordinator, and its hints data will be lost
sunhaihong created CASSANDRA-14932:
--
Summary: removenode coordinator, and its hints data will be lost
Key: CASSANDRA-14932
URL: https://issues.apache.org/jira/browse/CASSANDRA-14932
Project: Cassandra
Issue Type: Bug
Environment: cassandra version 3.11.3
Reporter: sunhaihong
There are four nodes in cluster. assume them are node A, B, C, D. enabled
hinted handoff.
1) create a keyspace with RF=2, and create a table.
2) make node B, C down(nodetool stopdaemon),
3) login in node A with cqlsh，set CONSISTENCY ANY, insert into a row(assume the
row will be stored in node B and C). The row was successfully inserted even
though the node B,C was down, because the consistency level is ANY. the
coordinator(node A) wrote hints.
4) make node A down(nodetool stopdaemon), then remove node A(nodetool
removenode ${nodeA_hostId})
5) make node B, C come back(nodetool start)
6) login in any node of B, C, D. and execute select statement with partition
key of inserted row. But there is no any data that inserted row on step 3.
These steps lead to data(on step 3 was inserted row) lost.
Is there any problem with the steps I performed above?
If yes, How to deal with this situation？
look forward to your reply, thanks.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
