[jira] [Created] (CASSANDRA-5683) Write timeout in multi-dc environment
Boole Guo created CASSANDRA-5683: Summary: Write timeout in multi-dc environment Key: CASSANDRA-5683 URL: https://issues.apache.org/jira/browse/CASSANDRA-5683 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.9 Environment: apache cassandra 1.1.9 Reporter: Boole Guo When writing in mutil-dc environment, there are many timeout exception. As I know, this version have do good to reduce network bandwidth. Can we assign rpc timeout when writing like consistenylevel? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[1/2] Return current values on failed CAS
Updated Branches:
refs/heads/trunk 56d2296ad - 0ba5cf12b
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0ba5cf12/interface/thrift/gen-java/org/apache/cassandra/thrift/KsDef.java
--
diff --git a/interface/thrift/gen-java/org/apache/cassandra/thrift/KsDef.java
b/interface/thrift/gen-java/org/apache/cassandra/thrift/KsDef.java
index 3f2ee29..826e259 100644
--- a/interface/thrift/gen-java/org/apache/cassandra/thrift/KsDef.java
+++ b/interface/thrift/gen-java/org/apache/cassandra/thrift/KsDef.java
@@ -869,7 +869,7 @@ public class KsDef implements
org.apache.thrift.TBaseKsDef, KsDef._Fields, jav
struct.cf_defs = new ArrayListCfDef(_list144.size);
for (int _i145 = 0; _i145 _list144.size; ++_i145)
{
- CfDef _elem146; // optional
+ CfDef _elem146; // required
_elem146 = new CfDef();
_elem146.read(iprot);
struct.cf_defs.add(_elem146);
@@ -1018,7 +1018,7 @@ public class KsDef implements
org.apache.thrift.TBaseKsDef, KsDef._Fields, jav
struct.cf_defs = new ArrayListCfDef(_list151.size);
for (int _i152 = 0; _i152 _list151.size; ++_i152)
{
- CfDef _elem153; // optional
+ CfDef _elem153; // required
_elem153 = new CfDef();
_elem153.read(iprot);
struct.cf_defs.add(_elem153);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0ba5cf12/interface/thrift/gen-java/org/apache/cassandra/thrift/SlicePredicate.java
--
diff --git
a/interface/thrift/gen-java/org/apache/cassandra/thrift/SlicePredicate.java
b/interface/thrift/gen-java/org/apache/cassandra/thrift/SlicePredicate.java
index 1a3dfd5..0624e93 100644
--- a/interface/thrift/gen-java/org/apache/cassandra/thrift/SlicePredicate.java
+++ b/interface/thrift/gen-java/org/apache/cassandra/thrift/SlicePredicate.java
@@ -467,7 +467,7 @@ public class SlicePredicate implements
org.apache.thrift.TBaseSlicePredicate, S
struct.column_names = new ArrayListByteBuffer(_list16.size);
for (int _i17 = 0; _i17 _list16.size; ++_i17)
{
- ByteBuffer _elem18; // optional
+ ByteBuffer _elem18; // required
_elem18 = iprot.readBinary();
struct.column_names.add(_elem18);
}
@@ -572,7 +572,7 @@ public class SlicePredicate implements
org.apache.thrift.TBaseSlicePredicate, S
struct.column_names = new ArrayListByteBuffer(_list21.size);
for (int _i22 = 0; _i22 _list21.size; ++_i22)
{
-ByteBuffer _elem23; // optional
+ByteBuffer _elem23; // required
_elem23 = iprot.readBinary();
struct.column_names.add(_elem23);
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0ba5cf12/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
--
diff --git
a/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
b/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
index aa14815..3a30a21 100644
--- a/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
+++ b/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java
@@ -485,7 +485,7 @@ public class SuperColumn implements
org.apache.thrift.TBaseSuperColumn, SuperCo
struct.columns = new ArrayListColumn(_list0.size);
for (int _i1 = 0; _i1 _list0.size; ++_i1)
{
- Column _elem2; // optional
+ Column _elem2; // required
_elem2 = new Column();
_elem2.read(iprot);
struct.columns.add(_elem2);
@@ -566,7 +566,7 @@ public class SuperColumn implements
org.apache.thrift.TBaseSuperColumn, SuperCo
struct.columns = new ArrayListColumn(_list5.size);
for (int _i6 = 0; _i6 _list5.size; ++_i6)
{
- Column _elem7; // optional
+ Column _elem7; // required
_elem7 = new Column();
_elem7.read(iprot);
struct.columns.add(_elem7);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0ba5cf12/interface/thrift/gen-java/org/apache/cassandra/thrift/TokenRange.java
--
diff --git
a/interface/thrift/gen-java/org/apache/cassandra/thrift/TokenRange.java
b/interface/thrift/gen-java/org/apache/cassandra/thrift/TokenRange.java
index fc32d02..1393fd5 100644
--- a/interface/thrift/gen-java/org/apache/cassandra/thrift/TokenRange.java
+++ b/interface/thrift/gen-java/org/apache/cassandra/thrift/TokenRange.java
@@ -774,7 +774,7 @@ public class TokenRange implements
[jira] [Commented] (CASSANDRA-5625) Cleanup Tracing class
[ https://issues.apache.org/jira/browse/CASSANDRA-5625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690115#comment-13690115 ] Mikhail Mazursky commented on CASSANDRA-5625: - Can someone merge this? Cleanup Tracing class - Key: CASSANDRA-5625 URL: https://issues.apache.org/jira/browse/CASSANDRA-5625 Project: Cassandra Issue Type: Improvement Reporter: Mikhail Mazursky Priority: Trivial Attachments: trunk-5625.patch 1. Tracing instance cannot be null because it's initialized at classloading time and never changes. Made it final to prove that. Removed all redundant null checks. 2. StorageService has a Tracing.instance() call - it should trace something or can be removed? 3. Made sessions a ConcurrentMap to stress that it should be thread-safe. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690138#comment-13690138
]
Sylvain Lebresne commented on CASSANDRA-5668:
-
I don't think this patch can be committed as is: in both TracingExecutorService
and ExpiredTracingState, the package declaration is before the license header,
and I can't let that slide.
But with that fixed, +1.
NPE in net.OutputTcpConnection when tracing is enabled
--
Key: CASSANDRA-5668
URL: https://issues.apache.org/jira/browse/CASSANDRA-5668
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Ryan McGuire
Assignee: Jonathan Ellis
Fix For: 1.2.6
Attachments: 5668-assert-2.txt, 5668-assert.txt, 5668-logs.tar.gz,
5668_npe_ddl.cql, 5668_npe_insert.cql, 5668.txt, system.log
I get multiple NullPointerException when trying to trace INSERT statements.
To reproduce:
{code}
$ ccm create -v git:trunk
$ ccm populate -n 3
$ ccm start
$ ccm node1 cqlsh 5668_npe_ddl.cql
$ ccm node1 cqlsh 5668_npe_insert.cql
{code}
And see many exceptions like this in the logs of node1:
{code}
ERROR [WRITE-/127.0.0.3] 2013-06-19 14:54:35,885 OutboundTcpConnection.java
(line 197) error writing to /127.0.0.3
java.lang.NullPointerException
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:182)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:144)
{code}
This is similar to CASSANDRA-5658 and is the reason that npe_ddl and
npe_insert are separate files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
git commit: Conditional create/drop ks/table/index statements
Updated Branches:
refs/heads/trunk 0ba5cf12b - b73f9d423
Conditional create/drop ks/table/index statements
patch by michalm; reviewed by slebresne for CASSANDRA-2737
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b73f9d42
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b73f9d42
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b73f9d42
Branch: refs/heads/trunk
Commit: b73f9d42315311e3573018749aeba34f77f1419e
Parents: 0ba5cf1
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Fri Jun 21 11:21:54 2013 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Fri Jun 21 12:02:25 2013 +0200
--
CHANGES.txt | 1 +
doc/cql3/CQL.textile| 26 ---
pylib/cqlshlib/cql3handling.py | 16 +++
src/java/org/apache/cassandra/cql3/Cql.g| 45
.../statements/CreateColumnFamilyStatement.java | 22 --
.../cql3/statements/CreateIndexStatement.java | 14 +-
.../statements/CreateKeyspaceStatement.java | 15 ++-
.../statements/DropColumnFamilyStatement.java | 15 ++-
.../cql3/statements/DropIndexStatement.java | 22 --
.../cql3/statements/DropKeyspaceStatement.java | 14 +-
10 files changed, 141 insertions(+), 49 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/b73f9d42/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index ca27791..044add4 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -61,6 +61,7 @@
* Include a timestamp with all read commands to determine column expiration
(CASSANDRA-5149)
* Streaming 2.0 (CASSANDRA-5286)
+ * Conditional create/drop ks/table/index statements in CQL3 (CASSANDRA-2737)
1.2.6
* Fix cross-DC mutation forwarding (CASSANDRA-5632)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/b73f9d42/doc/cql3/CQL.textile
--
diff --git a/doc/cql3/CQL.textile b/doc/cql3/CQL.textile
index 287a480..34a0c12 100644
--- a/doc/cql3/CQL.textile
+++ b/doc/cql3/CQL.textile
@@ -134,7 +134,7 @@ h3(#createKeyspaceStmt). CREATE KEYSPACE
__Syntax:__
bc(syntax)..
-create-keyspace-stmt ::= CREATE KEYSPACE identifier WITH properties
+create-keyspace-stmt ::= CREATE KEYSPACE (IF NOT EXISTS)? identifier WITH
properties
p.
__Sample:__
@@ -159,6 +159,8 @@ The @replication@ @property@ is mandatory. It must at
least contains the @'cla
* @'NetworkTopologyStrategy'@: A replication strategy that allows to set the
replication factor independently for each data-center. The rest of the
sub-options are key-value pairs where each time the key is the name of a
datacenter and the value the replication factor for that data-center.
* @'OldNetworkTopologyStrategy'@: A legacy replication strategy. You should
avoid this strategy for new keyspaces and prefer @'NetworkTopologyStrategy'@.
+Attempting to create an already existing keyspace will return an error unless
the @IF NOT EXISTS@ option is used. If it is used, the statement will be a
no-op if the keyspace already exists.
+
h3(#useStmt). USE
__Syntax:__
@@ -192,7 +194,7 @@ h3(#dropKeyspaceStmt). DROP KEYSPACE
__Syntax:__
-bc(syntax). drop-keyspace-stmt ::= DROP KEYSPACE identifier
+bc(syntax). drop-keyspace-stmt ::= DROP KEYSPACE ( IF EXISTS )? identifier
__Sample:__
@@ -200,13 +202,15 @@ bc(sample). DROP KEYSPACE myApp;
A @DROP KEYSPACE@ statement results in the immediate, irreversible removal of
an existing keyspace, including all column families in it, and all data
contained in those column families.
+If the keyspace does not exists, the statement will return an error, unless
@IF EXISTS@ is used in which case the operation is a no-op.
+
h3(#createTableStmt). CREATE TABLE
__Syntax:__
bc(syntax)..
-create-table-stmt ::= CREATE (TABLE | COLUMNFAMILY) tablename
+create-table-stmt ::= CREATE ( TABLE | COLUMNFAMILY ) ( IF NOT EXISTS )?
tablename
'(' definition ( ',' definition )* ')'
( WITH option ( AND option)* )?
@@ -245,6 +249,8 @@ CREATE TABLE timeline (
p.
The @CREATE TABLE@ statement creates a new table. Each such table is a set of
_rows_ (usually representing related entities) for which it defines a number of
properties. A table is defined by a name:#createTableName, it defines the a
href=#createTableColumnitcolumns/it/a composing rows of the table and
have a number of options:#createTableOptions. Note that the @CREATE
COLUMNFAMILY@ syntax is supported as an alias for @CREATE TABLE@ (for
historical reasons).
+Attempting to create an already existing table will return an error
[jira] [Created] (CASSANDRA-5684) Multi-DC not working between 1.1.10 and 1.2.4 version
Ananth Gundabattula created CASSANDRA-5684: -- Summary: Multi-DC not working between 1.1.10 and 1.2.4 version Key: CASSANDRA-5684 URL: https://issues.apache.org/jira/browse/CASSANDRA-5684 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.2.4 Environment: Linux Reporter: Ananth Gundabattula We wanted to upgrade to a higher version of cassandra ( a custom build of 1.2.4 ) from current cassandra version 1.1.10. The way we wanted to upgrade is to build a new cluster and enable multi-dc replication; rebuild and switch the application layer to the new cluster. This enabled us to have the least downtime. However, I was not able to get a multi-dc going between these two versions. I followed pretty much the instructions here ( for the 1.2.4 version ) http://www.datastax.com/documentation/cassandra/1.2/index.html#cassandra/operations/ops_add_dc_to_cluster_t.html. I tried enabling multi-dc using these instructions for the 1.1.10 cluster http://www.datastax.com/docs/1.0/initialize/cluster_init_multi_dc Mixing both the approaches did not work either. Specifying a 1.2.4 node as one of the 1.1.10 cluster seed nodes caused errors in the startup of 1.2.4 node. The error was not an exception but a message looking like this in the logs : ClusterName mismatch from /10.xxx.xx.xx DC1!=DC2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-5685) time based compaction strategy
Edward Capriolo created CASSANDRA-5685: -- Summary: time based compaction strategy Key: CASSANDRA-5685 URL: https://issues.apache.org/jira/browse/CASSANDRA-5685 Project: Cassandra Issue Type: New Feature Reporter: Edward Capriolo We have a column family with all sites ttl. The idea is that the odd data files would roll off after they are passed retention. What ends happening is they get compacted with new file and don't roll off. Leveled might help but still not perfect. Suggesting a compaction strategy that will not compact sstables together if the oldest and youngesr column are too far apart. This wilk 3nsure old data does not compact with new xata and help run closer to capacity. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5683) Write timeout in multi-dc environment
[ https://issues.apache.org/jira/browse/CASSANDRA-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690313#comment-13690313 ] Jonathan Ellis commented on CASSANDRA-5683: --- When would per-request control be more useful than a global setting? Write timeout in multi-dc environment -- Key: CASSANDRA-5683 URL: https://issues.apache.org/jira/browse/CASSANDRA-5683 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.9 Environment: apache cassandra 1.1.9 Reporter: Boole Guo Labels: write When writing in mutil-dc environment, there are many timeout exception. As I know, this version have do good to reduce network bandwidth. Can we assign rpc timeout when writing like consistenylevel? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5625) Cleanup Tracing class
[ https://issues.apache.org/jira/browse/CASSANDRA-5625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690314#comment-13690314 ] Jonathan Ellis commented on CASSANDRA-5625: --- I thought you were going to update it after my comments. :P Cleanup Tracing class - Key: CASSANDRA-5625 URL: https://issues.apache.org/jira/browse/CASSANDRA-5625 Project: Cassandra Issue Type: Improvement Reporter: Mikhail Mazursky Priority: Trivial Attachments: trunk-5625.patch 1. Tracing instance cannot be null because it's initialized at classloading time and never changes. Made it final to prove that. Removed all redundant null checks. 2. StorageService has a Tracing.instance() call - it should trace something or can be removed? 3. Made sessions a ConcurrentMap to stress that it should be thread-safe. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5684) Multi-DC not working between 1.1.10 and 1.2.4 version
[ https://issues.apache.org/jira/browse/CASSANDRA-5684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690317#comment-13690317 ] Jonathan Ellis commented on CASSANDRA-5684: --- Why on earth would you do this instead of just upgrading the 1.1 nodes to 1.2 one at a time? Multi-DC not working between 1.1.10 and 1.2.4 version - Key: CASSANDRA-5684 URL: https://issues.apache.org/jira/browse/CASSANDRA-5684 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.2.4 Environment: Linux Reporter: Ananth Gundabattula We wanted to upgrade to a higher version of cassandra ( a custom build of 1.2.4 ) from current cassandra version 1.1.10. The way we wanted to upgrade is to build a new cluster and enable multi-dc replication; rebuild and switch the application layer to the new cluster. This enabled us to have the least downtime. However, I was not able to get a multi-dc going between these two versions. I followed pretty much the instructions here ( for the 1.2.4 version ) http://www.datastax.com/documentation/cassandra/1.2/index.html#cassandra/operations/ops_add_dc_to_cluster_t.html. I tried enabling multi-dc using these instructions for the 1.1.10 cluster http://www.datastax.com/docs/1.0/initialize/cluster_init_multi_dc Mixing both the approaches did not work either. Specifying a 1.2.4 node as one of the 1.1.10 cluster seed nodes caused errors in the startup of 1.2.4 node. The error was not an exception but a message looking like this in the logs : ClusterName mismatch from /10.xxx.xx.xx DC1!=DC2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[2/3] git commit: Fix tracing when operation completesbefore all responses arrive patch by jbellis; reviewed by slebresne for CASSANDRA-5668
Fix tracing when operation completesbefore all responses arrive
patch by jbellis; reviewed by slebresne for CASSANDRA-5668
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fbe8a6eb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fbe8a6eb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fbe8a6eb
Branch: refs/heads/trunk
Commit: fbe8a6eb213ee3558be701c73c31a0da79446657
Parents: 7dc2eb9
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Jun 20 10:19:13 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Fri Jun 21 09:23:39 2013 -0500
--
CHANGES.txt | 1 +
.../DebuggableThreadPoolExecutor.java | 22 ---
.../cassandra/concurrent/StageManager.java | 29 +--
.../concurrent/TracingAwareExecutorService.java | 33 +
.../apache/cassandra/net/MessagingService.java | 14 ---
.../cassandra/net/OutboundTcpConnection.java| 13 ++-
.../cassandra/service/MigrationManager.java | 7 +---
.../cassandra/tracing/ExpiredTraceState.java| 39
.../apache/cassandra/tracing/TraceState.java| 12 --
.../org/apache/cassandra/tracing/Tracing.java | 26 +++--
.../service/AntiEntropyServiceTestAbstract.java | 10 ++---
11 files changed, 156 insertions(+), 50 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/fbe8a6eb/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3847d6a..593bf7c 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
1.2.6
+ * Fix tracing when operation completes before all responses arrive
(CASSANDRA-5668)
* Fix cross-DC mutation forwarding (CASSANDRA-5632)
* Scale hinted_handoff_throttle_in_kb to cluster size (CASSANDRA-5272)
* (Hadoop) Add CQL3 input/output formats (CASSANDRA-4421, 5622)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/fbe8a6eb/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
--
diff --git
a/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
b/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
index 25f15ee..26441ec 100644
--- a/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
+++ b/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
@@ -44,7 +44,7 @@ import static org.apache.cassandra.tracing.Tracing.isTracing;
* threads and the queue is full, we want the enqueuer to block. But to
allow the number of threads to drop if a
* stage is less busy, core thread timeout is enabled.
*/
-public class DebuggableThreadPoolExecutor extends ThreadPoolExecutor
+public class DebuggableThreadPoolExecutor extends ThreadPoolExecutor
implements TracingAwareExecutorService
{
protected static final Logger logger =
LoggerFactory.getLogger(DebuggableThreadPoolExecutor.class);
public static final RejectedExecutionHandler blockingExecutionHandler =
new RejectedExecutionHandler()
@@ -131,12 +131,17 @@ public class DebuggableThreadPoolExecutor extends
ThreadPoolExecutor
protected void onFinalAccept(Runnable task) {}
protected void onFinalRejection(Runnable task) {}
+public void execute(Runnable command, TraceState state)
+{
+super.execute(state == null ? command : new
TraceSessionWrapperObject(command, state));
+}
+
// execute does not call newTaskFor
@Override
public void execute(Runnable command)
{
super.execute(isTracing() !(command instanceof TraceSessionWrapper)
- ? new TraceSessionWrapperObject(command, null)
+ ? new TraceSessionWrapperObject(command)
: command);
}
@@ -145,7 +150,7 @@ public class DebuggableThreadPoolExecutor extends
ThreadPoolExecutor
{
if (isTracing() !(runnable instanceof TraceSessionWrapper))
{
-return new TraceSessionWrapperT(runnable, result);
+return new TraceSessionWrapperT(Executors.callable(runnable,
result));
}
return super.newTaskFor(runnable, result);
}
@@ -256,10 +261,9 @@ public class DebuggableThreadPoolExecutor extends
ThreadPoolExecutor
{
private final TraceState state;
-public TraceSessionWrapper(Runnable runnable, T result)
+public TraceSessionWrapper(Runnable command)
{
-super(runnable, result);
-state = Tracing.instance().get();
+this(command, null);
}
public TraceSessionWrapper(CallableT callable)
@@ -268,6 +272,12 @@ public class
[3/3] git commit: merge from 1.2
merge from 1.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/140b0311 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/140b0311 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/140b0311 Branch: refs/heads/trunk Commit: 140b0311df890d2258f19f3df98f1a996b6f2a6e Parents: b73f9d4 fbe8a6e Author: Jonathan Ellis jbel...@apache.org Authored: Fri Jun 21 09:27:21 2013 -0500 Committer: Jonathan Ellis jbel...@apache.org Committed: Fri Jun 21 09:27:21 2013 -0500 -- CHANGES.txt | 1 + .../DebuggableThreadPoolExecutor.java | 22 --- .../cassandra/concurrent/StageManager.java | 29 +-- .../concurrent/TracingAwareExecutorService.java | 33 + .../apache/cassandra/net/MessagingService.java | 14 --- .../cassandra/net/OutboundTcpConnection.java| 13 ++- .../cassandra/service/MigrationManager.java | 3 +- .../cassandra/tracing/ExpiredTraceState.java| 39 .../apache/cassandra/tracing/TraceState.java| 12 -- .../org/apache/cassandra/tracing/Tracing.java | 26 +++-- .../service/AntiEntropyServiceTestAbstract.java | 9 +++-- 11 files changed, 156 insertions(+), 45 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/140b0311/CHANGES.txt -- diff --cc CHANGES.txt index 044add4,593bf7c..29c6fd2 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,69 -1,5 +1,70 @@@ +2.0 + * Removed on-heap row cache (CASSANDRA-5348) + * use nanotime consistently for node-local timeouts (CASSANDRA-5581) + * Avoid unnecessary second pass on name-based queries (CASSANDRA-5577) + * Experimental triggers (CASSANDRA-1311) + * JEMalloc support for off-heap allocation (CASSANDRA-3997) + * Single-pass compaction (CASSANDRA-4180) + * Removed token range bisection (CASSANDRA-5518) + * Removed compatibility with pre-1.2.5 sstables and network messages + (CASSANDRA-5511) + * removed PBSPredictor (CASSANDRA-5455) + * CAS support (CASSANDRA-5062, 5441, 5442, 5443, 5619) + * Leveled compaction performs size-tiered compactions in L0 + (CASSANDRA-5371, 5439) + * Add yaml network topology snitch for mixed ec2/other envs (CASSANDRA-5339) + * Log when a node is down longer than the hint window (CASSANDRA-4554) + * Optimize tombstone creation for ExpiringColumns (CASSANDRA-4917) + * Improve LeveledScanner work estimation (CASSANDRA-5250, 5407) + * Replace compaction lock with runWithCompactionsDisabled (CASSANDRA-3430) + * Change Message IDs to ints (CASSANDRA-5307) + * Move sstable level information into the Stats component, removing the + need for a separate Manifest file (CASSANDRA-4872) + * avoid serializing to byte[] on commitlog append (CASSANDRA-5199) + * make index_interval configurable per columnfamily (CASSANDRA-3961, CASSANDRA-5650) + * add default_time_to_live (CASSANDRA-3974) + * add memtable_flush_period_in_ms (CASSANDRA-4237) + * replace supercolumns internally by composites (CASSANDRA-3237, 5123) + * upgrade thrift to 0.9.0 (CASSANDRA-3719) + * drop unnecessary keyspace parameter from user-defined compaction API + (CASSANDRA-5139) + * more robust solution to incomplete compactions + counters (CASSANDRA-5151) + * Change order of directory searching for c*.in.sh (CASSANDRA-3983) + * Add tool to reset SSTable compaction level for LCS (CASSANDRA-5271) + * Allow custom configuration loader (CASSANDRA-5045) + * Remove memory emergency pressure valve logic (CASSANDRA-3534) + * Reduce request latency with eager retry (CASSANDRA-4705) + * cqlsh: Remove ASSUME command (CASSANDRA-5331) + * Rebuild BF when loading sstables if bloom_filter_fp_chance + has changed since compaction (CASSANDRA-5015) + * remove row-level bloom filters (CASSANDRA-4885) + * Change Kernel Page Cache skipping into row preheating (disabled by default) + (CASSANDRA-4937) + * Improve repair by deciding on a gcBefore before sending + out TreeRequests (CASSANDRA-4932) + * Add an official way to disable compactions (CASSANDRA-5074) + * Reenable ALTER TABLE DROP with new semantics (CASSANDRA-3919) + * Add binary protocol versioning (CASSANDRA-5436) + * Swap THshaServer for TThreadedSelectorServer (CASSANDRA-5530) + * Add alias support to SELECT statement (CASSANDRA-5075) + * Don't create empty RowMutations in CommitLogReplayer (CASSANDRA-5541) + * Use range tombstones when dropping cfs/columns from schema (CASSANDRA-5579) + * cqlsh: drop CQL2/CQL3-beta support (CASSANDRA-5585) + * Track max/min column names in sstables to be able to optimize slice + queries (CASSANDRA-5514, CASSANDRA-5595, CASSANDRA-5600) + * Binary protocol: allow batching already prepared
[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690328#comment-13690328
]
Jonathan Ellis commented on CASSANDRA-5668:
---
Fixed and committed. :)
NPE in net.OutputTcpConnection when tracing is enabled
--
Key: CASSANDRA-5668
URL: https://issues.apache.org/jira/browse/CASSANDRA-5668
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Ryan McGuire
Assignee: Jonathan Ellis
Fix For: 1.2.6
Attachments: 5668-assert-2.txt, 5668-assert.txt, 5668-logs.tar.gz,
5668_npe_ddl.cql, 5668_npe_insert.cql, 5668.txt, system.log
I get multiple NullPointerException when trying to trace INSERT statements.
To reproduce:
{code}
$ ccm create -v git:trunk
$ ccm populate -n 3
$ ccm start
$ ccm node1 cqlsh 5668_npe_ddl.cql
$ ccm node1 cqlsh 5668_npe_insert.cql
{code}
And see many exceptions like this in the logs of node1:
{code}
ERROR [WRITE-/127.0.0.3] 2013-06-19 14:54:35,885 OutboundTcpConnection.java
(line 197) error writing to /127.0.0.3
java.lang.NullPointerException
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:182)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:144)
{code}
This is similar to CASSANDRA-5658 and is the reason that npe_ddl and
npe_insert are separate files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[1/3] git commit: Fix tracing when operation completesbefore all responses arrive patch by jbellis; reviewed by slebresne for CASSANDRA-5668
Updated Branches:
refs/heads/cassandra-1.2 7dc2eb95c - fbe8a6eb2
refs/heads/trunk b73f9d423 - 140b0311d
Fix tracing when operation completesbefore all responses arrive
patch by jbellis; reviewed by slebresne for CASSANDRA-5668
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fbe8a6eb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fbe8a6eb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fbe8a6eb
Branch: refs/heads/cassandra-1.2
Commit: fbe8a6eb213ee3558be701c73c31a0da79446657
Parents: 7dc2eb9
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Jun 20 10:19:13 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Fri Jun 21 09:23:39 2013 -0500
--
CHANGES.txt | 1 +
.../DebuggableThreadPoolExecutor.java | 22 ---
.../cassandra/concurrent/StageManager.java | 29 +--
.../concurrent/TracingAwareExecutorService.java | 33 +
.../apache/cassandra/net/MessagingService.java | 14 ---
.../cassandra/net/OutboundTcpConnection.java| 13 ++-
.../cassandra/service/MigrationManager.java | 7 +---
.../cassandra/tracing/ExpiredTraceState.java| 39
.../apache/cassandra/tracing/TraceState.java| 12 --
.../org/apache/cassandra/tracing/Tracing.java | 26 +++--
.../service/AntiEntropyServiceTestAbstract.java | 10 ++---
11 files changed, 156 insertions(+), 50 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/fbe8a6eb/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3847d6a..593bf7c 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
1.2.6
+ * Fix tracing when operation completes before all responses arrive
(CASSANDRA-5668)
* Fix cross-DC mutation forwarding (CASSANDRA-5632)
* Scale hinted_handoff_throttle_in_kb to cluster size (CASSANDRA-5272)
* (Hadoop) Add CQL3 input/output formats (CASSANDRA-4421, 5622)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/fbe8a6eb/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
--
diff --git
a/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
b/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
index 25f15ee..26441ec 100644
--- a/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
+++ b/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
@@ -44,7 +44,7 @@ import static org.apache.cassandra.tracing.Tracing.isTracing;
* threads and the queue is full, we want the enqueuer to block. But to
allow the number of threads to drop if a
* stage is less busy, core thread timeout is enabled.
*/
-public class DebuggableThreadPoolExecutor extends ThreadPoolExecutor
+public class DebuggableThreadPoolExecutor extends ThreadPoolExecutor
implements TracingAwareExecutorService
{
protected static final Logger logger =
LoggerFactory.getLogger(DebuggableThreadPoolExecutor.class);
public static final RejectedExecutionHandler blockingExecutionHandler =
new RejectedExecutionHandler()
@@ -131,12 +131,17 @@ public class DebuggableThreadPoolExecutor extends
ThreadPoolExecutor
protected void onFinalAccept(Runnable task) {}
protected void onFinalRejection(Runnable task) {}
+public void execute(Runnable command, TraceState state)
+{
+super.execute(state == null ? command : new
TraceSessionWrapperObject(command, state));
+}
+
// execute does not call newTaskFor
@Override
public void execute(Runnable command)
{
super.execute(isTracing() !(command instanceof TraceSessionWrapper)
- ? new TraceSessionWrapperObject(command, null)
+ ? new TraceSessionWrapperObject(command)
: command);
}
@@ -145,7 +150,7 @@ public class DebuggableThreadPoolExecutor extends
ThreadPoolExecutor
{
if (isTracing() !(runnable instanceof TraceSessionWrapper))
{
-return new TraceSessionWrapperT(runnable, result);
+return new TraceSessionWrapperT(Executors.callable(runnable,
result));
}
return super.newTaskFor(runnable, result);
}
@@ -256,10 +261,9 @@ public class DebuggableThreadPoolExecutor extends
ThreadPoolExecutor
{
private final TraceState state;
-public TraceSessionWrapper(Runnable runnable, T result)
+public TraceSessionWrapper(Runnable command)
{
-super(runnable, result);
-state = Tracing.instance().get();
+this(command, null);
[jira] [Created] (CASSANDRA-5686) During node repair operation, Cassandra throws exception on one of our nodes: failed to uncompress the chunk: FAILED_TO_UNCOMPRESS
Rao created CASSANDRA-5686: -- Summary: During node repair operation, Cassandra throws exception on one of our nodes: failed to uncompress the chunk: FAILED_TO_UNCOMPRESS Key: CASSANDRA-5686 URL: https://issues.apache.org/jira/browse/CASSANDRA-5686 Project: Cassandra Issue Type: Bug Affects Versions: 1.2.5 Reporter: Rao INFO [AntiEntropyStage:1] 2013-06-20 03:08:03,876 AntiEntropyService.java (line 213) [repair #4ae2a970-d991-11e2-bd05-094c1105e54d] Received merkle tree for tags from /135.163.214.175 ERROR [Thread-40] 2013-06-20 03:08:03,903 CassandraDaemon.java (line 175) Exception in thread Thread[Thread-40,5,main] java.lang.RuntimeException: java.io.IOException: failed to uncompress the chunk: FAILED_TO_UNCOMPRESS(5) at org.apache.cassandra.service.AntiEntropyService$Validator$ValidatorSerializer.deserialize(AntiEntropyService.java:438) at org.apache.cassandra.service.AntiEntropyService$Validator$ValidatorSerializer.deserialize(AntiEntropyService.java:421) at org.apache.cassandra.net.MessageIn.read(MessageIn.java:94) at org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:203) at org.apache.cassandra.net.IncomingTcpConnection.handleModernVersion(IncomingTcpConnection.java:135) at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:82) Caused by: java.io.IOException: failed to uncompress the chunk: FAILED_TO_UNCOMPRESS(5) at org.xerial.snappy.SnappyInputStream.hasNextChunk(SnappyInputStream.java:361) at org.xerial.snappy.SnappyInputStream.read(SnappyInputStream.java:383) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.cassandra.utils.MerkleTree$Leaf$LeafSerializer.deserialize(MerkleTree.java:793) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:920) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:713) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:713) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at
[jira] [Updated] (CASSANDRA-5686) During node repair operation, Cassandra throws exception on one of our nodes: failed to uncompress the chunk: FAILED_TO_UNCOMPRESS
[ https://issues.apache.org/jira/browse/CASSANDRA-5686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rao updated CASSANDRA-5686: --- Description: INFO [AntiEntropyStage:1] 2013-06-20 03:08:03,876 AntiEntropyService.java (line 213) [repair #4ae2a970-d991-11e2-bd05-094c1105e54d] Received merkle tree for tags from /135.163.214.175 ERROR [Thread-40] 2013-06-20 03:08:03,903 CassandraDaemon.java (line 175) Exception in thread Thread[Thread-40,5,main] java.lang.RuntimeException: java.io.IOException: failed to uncompress the chunk: FAILED_TO_UNCOMPRESS(5) at org.apache.cassandra.service.AntiEntropyService$Validator$ValidatorSerializer.deserialize(AntiEntropyService.java:438) at org.apache.cassandra.service.AntiEntropyService$Validator$ValidatorSerializer.deserialize(AntiEntropyService.java:421) at org.apache.cassandra.net.MessageIn.read(MessageIn.java:94) at org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:203) at org.apache.cassandra.net.IncomingTcpConnection.handleModernVersion(IncomingTcpConnection.java:135) at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:82) Caused by: java.io.IOException: failed to uncompress the chunk: FAILED_TO_UNCOMPRESS(5) at org.xerial.snappy.SnappyInputStream.hasNextChunk(SnappyInputStream.java:361) at org.xerial.snappy.SnappyInputStream.read(SnappyInputStream.java:383) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.cassandra.utils.MerkleTree$Leaf$LeafSerializer.deserialize(MerkleTree.java:793) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:920) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:713) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:713) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:713) at
[jira] [Updated] (CASSANDRA-5686) During node repair operation, Cassandra throws exception on one of our nodes: failed to uncompress the chunk: FAILED_TO_UNCOMPRESS
[ https://issues.apache.org/jira/browse/CASSANDRA-5686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rao updated CASSANDRA-5686: --- Description: During node repair operation, Cassandra throws exception on one of our nodes: failed to uncompress the chunk: FAILED_TO_UNCOMPRESS INFO [AntiEntropyStage:1] 2013-06-20 03:08:03,876 AntiEntropyService.java (line 213) [repair #4ae2a970-d991-11e2-bd05-094c1105e54d] Received merkle tree for tags from /135.163.214.175 ERROR [Thread-40] 2013-06-20 03:08:03,903 CassandraDaemon.java (line 175) Exception in thread Thread[Thread-40,5,main] java.lang.RuntimeException: java.io.IOException: failed to uncompress the chunk: FAILED_TO_UNCOMPRESS(5) at org.apache.cassandra.service.AntiEntropyService$Validator$ValidatorSerializer.deserialize(AntiEntropyService.java:438) at org.apache.cassandra.service.AntiEntropyService$Validator$ValidatorSerializer.deserialize(AntiEntropyService.java:421) at org.apache.cassandra.net.MessageIn.read(MessageIn.java:94) at org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:203) at org.apache.cassandra.net.IncomingTcpConnection.handleModernVersion(IncomingTcpConnection.java:135) at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:82) Caused by: java.io.IOException: failed to uncompress the chunk: FAILED_TO_UNCOMPRESS(5) at org.xerial.snappy.SnappyInputStream.hasNextChunk(SnappyInputStream.java:361) at org.xerial.snappy.SnappyInputStream.read(SnappyInputStream.java:383) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.cassandra.utils.MerkleTree$Leaf$LeafSerializer.deserialize(MerkleTree.java:793) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:920) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:713) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:713) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896) at org.apache.cassandra.utils.MerkleTree$Inner$InnerSerializer.deserialize(MerkleTree.java:714) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:918) at org.apache.cassandra.utils.MerkleTree$Hashable$HashableSerializer.deserialize(MerkleTree.java:896)
[jira] [Updated] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-5668:
--
Attachment: 5668-followup.txt
followup patch to make session cleanup check for REQUEST_RESPONSE instead of
local-ness, to keep from destroying session prematurely in RMVH forwarding case.
NPE in net.OutputTcpConnection when tracing is enabled
--
Key: CASSANDRA-5668
URL: https://issues.apache.org/jira/browse/CASSANDRA-5668
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Ryan McGuire
Assignee: Jonathan Ellis
Fix For: 1.2.6
Attachments: 5668-assert-2.txt, 5668-assert.txt, 5668-followup.txt,
5668-logs.tar.gz, 5668_npe_ddl.cql, 5668_npe_insert.cql, 5668.txt, system.log
I get multiple NullPointerException when trying to trace INSERT statements.
To reproduce:
{code}
$ ccm create -v git:trunk
$ ccm populate -n 3
$ ccm start
$ ccm node1 cqlsh 5668_npe_ddl.cql
$ ccm node1 cqlsh 5668_npe_insert.cql
{code}
And see many exceptions like this in the logs of node1:
{code}
ERROR [WRITE-/127.0.0.3] 2013-06-19 14:54:35,885 OutboundTcpConnection.java
(line 197) error writing to /127.0.0.3
java.lang.NullPointerException
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:182)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:144)
{code}
This is similar to CASSANDRA-5658 and is the reason that npe_ddl and
npe_insert are separate files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5681) Refactor IESCS in Snitches
[ https://issues.apache.org/jira/browse/CASSANDRA-5681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690385#comment-13690385 ] Jonathan Ellis commented on CASSANDRA-5681: --- +1 I have no better suggestions for the Helper name :) Refactor IESCS in Snitches -- Key: CASSANDRA-5681 URL: https://issues.apache.org/jira/browse/CASSANDRA-5681 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.2.5 Reporter: Jason Brown Assignee: Jason Brown Priority: Minor Labels: snitch Fix For: 1.2.6, 2.0 beta 1 Attachments: 5681-v1.diff Reduce/refactor duplicated IESCS implementations in Ec2MRS, GPFS, and YPNTS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690391#comment-13690391
]
Sylvain Lebresne commented on CASSANDRA-5668:
-
+1 on that follow up.
NPE in net.OutputTcpConnection when tracing is enabled
--
Key: CASSANDRA-5668
URL: https://issues.apache.org/jira/browse/CASSANDRA-5668
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Ryan McGuire
Assignee: Jonathan Ellis
Fix For: 1.2.6
Attachments: 5668-assert-2.txt, 5668-assert.txt, 5668-followup.txt,
5668-logs.tar.gz, 5668_npe_ddl.cql, 5668_npe_insert.cql, 5668.txt, system.log
I get multiple NullPointerException when trying to trace INSERT statements.
To reproduce:
{code}
$ ccm create -v git:trunk
$ ccm populate -n 3
$ ccm start
$ ccm node1 cqlsh 5668_npe_ddl.cql
$ ccm node1 cqlsh 5668_npe_insert.cql
{code}
And see many exceptions like this in the logs of node1:
{code}
ERROR [WRITE-/127.0.0.3] 2013-06-19 14:54:35,885 OutboundTcpConnection.java
(line 197) error writing to /127.0.0.3
java.lang.NullPointerException
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:182)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:144)
{code}
This is similar to CASSANDRA-5658 and is the reason that npe_ddl and
npe_insert are separate files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[3/4] git commit: Merge branch 'cassandra-1.2' into trunk
Merge branch 'cassandra-1.2' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/36aae612 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/36aae612 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/36aae612 Branch: refs/heads/trunk Commit: 36aae612a0b248c2650ef979b9230e50410b7415 Parents: 140b031 18f3a79 Author: Jonathan Ellis jbel...@apache.org Authored: Fri Jun 21 10:39:36 2013 -0500 Committer: Jonathan Ellis jbel...@apache.org Committed: Fri Jun 21 10:39:36 2013 -0500 -- .../org/apache/cassandra/net/OutboundTcpConnection.java | 3 ++- .../org/apache/cassandra/tracing/ExpiredTraceState.java | 2 +- src/java/org/apache/cassandra/tracing/TraceState.java | 4 +--- src/java/org/apache/cassandra/tracing/Tracing.java | 9 - 4 files changed, 8 insertions(+), 10 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/36aae612/src/java/org/apache/cassandra/net/OutboundTcpConnection.java -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/36aae612/src/java/org/apache/cassandra/tracing/TraceState.java -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/36aae612/src/java/org/apache/cassandra/tracing/Tracing.java --
[2/4] git commit: use REQUEST_RESPONSE to determine whether session is local to prevent false positives in RMVH forwarding case patch by jbellis; reviewed by slebresne for CASSANDRA-5668
use REQUEST_RESPONSE to determine whether session is local to prevent false
positives in RMVH forwarding case
patch by jbellis; reviewed by slebresne for CASSANDRA-5668
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/18f3a79f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/18f3a79f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/18f3a79f
Branch: refs/heads/trunk
Commit: 18f3a79fdd2414698817aa4c35d0f9b44d5a7e55
Parents: fbe8a6e
Author: Jonathan Ellis jbel...@apache.org
Authored: Fri Jun 21 10:39:27 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Fri Jun 21 10:39:27 2013 -0500
--
.../org/apache/cassandra/net/OutboundTcpConnection.java | 3 ++-
.../org/apache/cassandra/tracing/ExpiredTraceState.java | 2 +-
src/java/org/apache/cassandra/tracing/TraceState.java | 4 +---
src/java/org/apache/cassandra/tracing/Tracing.java | 9 -
4 files changed, 8 insertions(+), 10 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/18f3a79f/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
--
diff --git a/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
b/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
index ee30d36..7077922 100644
--- a/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
+++ b/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
@@ -188,7 +188,8 @@ public class OutboundTcpConnection extends Thread
else
{
state.trace(message);
-Tracing.instance().stopIfNonLocal(state);
+if (qm.message.verb ==
MessagingService.Verb.REQUEST_RESPONSE)
+Tracing.instance().stopNonLocal(state);
}
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/18f3a79f/src/java/org/apache/cassandra/tracing/ExpiredTraceState.java
--
diff --git a/src/java/org/apache/cassandra/tracing/ExpiredTraceState.java
b/src/java/org/apache/cassandra/tracing/ExpiredTraceState.java
index 6b4f90b..37a013b 100644
--- a/src/java/org/apache/cassandra/tracing/ExpiredTraceState.java
+++ b/src/java/org/apache/cassandra/tracing/ExpiredTraceState.java
@@ -29,7 +29,7 @@ public class ExpiredTraceState extends TraceState
{
public ExpiredTraceState(UUID sessionId)
{
-super(FBUtilities.getBroadcastAddress(), sessionId, true);
+super(FBUtilities.getBroadcastAddress(), sessionId);
}
public int elapsed()
http://git-wip-us.apache.org/repos/asf/cassandra/blob/18f3a79f/src/java/org/apache/cassandra/tracing/TraceState.java
--
diff --git a/src/java/org/apache/cassandra/tracing/TraceState.java
b/src/java/org/apache/cassandra/tracing/TraceState.java
index 25599c4..4d52f8f 100644
--- a/src/java/org/apache/cassandra/tracing/TraceState.java
+++ b/src/java/org/apache/cassandra/tracing/TraceState.java
@@ -45,16 +45,14 @@ public class TraceState
public final InetAddress coordinator;
public final Stopwatch watch;
public final ByteBuffer sessionIdBytes;
-public final boolean isLocallyOwned;
-public TraceState(InetAddress coordinator, UUID sessionId, boolean
locallyOwned)
+public TraceState(InetAddress coordinator, UUID sessionId)
{
assert coordinator != null;
assert sessionId != null;
this.coordinator = coordinator;
this.sessionId = sessionId;
-this.isLocallyOwned = locallyOwned;
sessionIdBytes = ByteBufferUtil.bytes(sessionId);
watch = new Stopwatch();
watch.start();
http://git-wip-us.apache.org/repos/asf/cassandra/blob/18f3a79f/src/java/org/apache/cassandra/tracing/Tracing.java
--
diff --git a/src/java/org/apache/cassandra/tracing/Tracing.java
b/src/java/org/apache/cassandra/tracing/Tracing.java
index eb5bad9..c692436 100644
--- a/src/java/org/apache/cassandra/tracing/Tracing.java
+++ b/src/java/org/apache/cassandra/tracing/Tracing.java
@@ -144,17 +144,16 @@ public class Tracing
{
assert state.get() == null;
-TraceState ts = new TraceState(localAddress, sessionId, true);
+TraceState ts = new TraceState(localAddress, sessionId);
state.set(ts);
sessions.put(sessionId, ts);
return sessionId;
}
-public void stopIfNonLocal(TraceState state)
+public void stopNonLocal(TraceState state)
{
-if (!state.isLocallyOwned)
-sessions.remove(state.sessionId);
+
[4/4] git commit: refactor reconnecting snitches patch by jasobrown; reviewed by jbellis for CASSANDRA-5681
refactor reconnecting snitches
patch by jasobrown; reviewed by jbellis for CASSANDRA-5681
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e75e33fa
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e75e33fa
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e75e33fa
Branch: refs/heads/cassandra-1.2
Commit: e75e33fa6dc5e2a3fe061d747cc98679a65ef960
Parents: 18f3a79
Author: Jonathan Ellis jbel...@apache.org
Authored: Fri Jun 21 10:40:31 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Fri Jun 21 10:40:31 2013 -0500
--
.../cassandra/locator/Ec2MultiRegionSnitch.java | 71 +---
.../locator/GossipingPropertyFileSnitch.java| 63 +-
.../locator/ReconnectableSnitchHelper.java | 88
3 files changed, 94 insertions(+), 128 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e75e33fa/src/java/org/apache/cassandra/locator/Ec2MultiRegionSnitch.java
--
diff --git a/src/java/org/apache/cassandra/locator/Ec2MultiRegionSnitch.java
b/src/java/org/apache/cassandra/locator/Ec2MultiRegionSnitch.java
index 9317941..bd5e091 100644
--- a/src/java/org/apache/cassandra/locator/Ec2MultiRegionSnitch.java
+++ b/src/java/org/apache/cassandra/locator/Ec2MultiRegionSnitch.java
@@ -19,16 +19,11 @@ package org.apache.cassandra.locator;
import java.io.IOException;
import java.net.InetAddress;
-import java.net.UnknownHostException;
import org.apache.cassandra.exceptions.ConfigurationException;
import org.apache.cassandra.config.DatabaseDescriptor;
import org.apache.cassandra.gms.ApplicationState;
-import org.apache.cassandra.gms.EndpointState;
import org.apache.cassandra.gms.Gossiper;
-import org.apache.cassandra.gms.IEndpointStateChangeSubscriber;
-import org.apache.cassandra.gms.VersionedValue;
-import org.apache.cassandra.net.MessagingService;
import org.apache.cassandra.service.StorageService;
/**
@@ -36,16 +31,13 @@ import org.apache.cassandra.service.StorageService;
*
* 2) Snitch will set the private IP as a Gossip application state.
*
- * 3) Snitch implements IESCS and will reset the connection if it is within the
+ * 3) Uses a helper class that implements IESCS and will reset the public IP
connection if it is within the
* same region to communicate via private IP.
*
- * Implements Ec2Snitch to inherit its functionality and extend it for
- * Multi-Region.
- *
* Operational: All the nodes in this cluster needs to be able to (modify the
* Security group settings in AWS) communicate via Public IP's.
*/
-public class Ec2MultiRegionSnitch extends Ec2Snitch implements
IEndpointStateChangeSubscriber
+public class Ec2MultiRegionSnitch extends Ec2Snitch
{
private static final String PUBLIC_IP_QUERY_URL =
http://169.254.169.254/latest/meta-data/public-ipv4;;
private static final String PRIVATE_IP_QUERY_URL =
http://169.254.169.254/latest/meta-data/local-ipv4;;
@@ -62,67 +54,10 @@ public class Ec2MultiRegionSnitch extends Ec2Snitch
implements IEndpointStateCha
DatabaseDescriptor.setBroadcastAddress(localPublicAddress);
}
-public void onJoin(InetAddress endpoint, EndpointState epState)
-{
-if (epState.getApplicationState(ApplicationState.INTERNAL_IP) != null)
-reconnect(endpoint,
epState.getApplicationState(ApplicationState.INTERNAL_IP));
-}
-
-public void onChange(InetAddress endpoint, ApplicationState state,
VersionedValue value)
-{
-if (state == ApplicationState.INTERNAL_IP)
-reconnect(endpoint, value);
-}
-
-public void onAlive(InetAddress endpoint, EndpointState state)
-{
-if (state.getApplicationState(ApplicationState.INTERNAL_IP) != null)
-reconnect(endpoint,
state.getApplicationState(ApplicationState.INTERNAL_IP));
-}
-
-public void onDead(InetAddress endpoint, EndpointState state)
-{
-// do nothing
-}
-
-public void onRestart(InetAddress endpoint, EndpointState state)
-{
-// do nothing
-}
-
-public void onRemove(InetAddress endpoint)
-{
-// do nothing.
-}
-
-private void reconnect(InetAddress publicAddress, VersionedValue
localAddressValue)
-{
-try
-{
-reconnect(publicAddress,
InetAddress.getByName(localAddressValue.value));
-}
-catch (UnknownHostException e)
-{
-logger.error(Error in getting the IP address resolved: , e);
-}
-}
-
-private void reconnect(InetAddress publicAddress, InetAddress localAddress)
-{
-if
(getDatacenter(publicAddress).equals(getDatacenter(localPublicAddress))
-
[1/4] git commit: use REQUEST_RESPONSE to determine whether session is local to prevent false positives in RMVH forwarding case patch by jbellis; reviewed by slebresne for CASSANDRA-5668
Updated Branches:
refs/heads/cassandra-1.2 fbe8a6eb2 - e75e33fa6
refs/heads/trunk 140b0311d - 36aae612a
use REQUEST_RESPONSE to determine whether session is local to prevent false
positives in RMVH forwarding case
patch by jbellis; reviewed by slebresne for CASSANDRA-5668
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/18f3a79f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/18f3a79f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/18f3a79f
Branch: refs/heads/cassandra-1.2
Commit: 18f3a79fdd2414698817aa4c35d0f9b44d5a7e55
Parents: fbe8a6e
Author: Jonathan Ellis jbel...@apache.org
Authored: Fri Jun 21 10:39:27 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Fri Jun 21 10:39:27 2013 -0500
--
.../org/apache/cassandra/net/OutboundTcpConnection.java | 3 ++-
.../org/apache/cassandra/tracing/ExpiredTraceState.java | 2 +-
src/java/org/apache/cassandra/tracing/TraceState.java | 4 +---
src/java/org/apache/cassandra/tracing/Tracing.java | 9 -
4 files changed, 8 insertions(+), 10 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/18f3a79f/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
--
diff --git a/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
b/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
index ee30d36..7077922 100644
--- a/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
+++ b/src/java/org/apache/cassandra/net/OutboundTcpConnection.java
@@ -188,7 +188,8 @@ public class OutboundTcpConnection extends Thread
else
{
state.trace(message);
-Tracing.instance().stopIfNonLocal(state);
+if (qm.message.verb ==
MessagingService.Verb.REQUEST_RESPONSE)
+Tracing.instance().stopNonLocal(state);
}
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/18f3a79f/src/java/org/apache/cassandra/tracing/ExpiredTraceState.java
--
diff --git a/src/java/org/apache/cassandra/tracing/ExpiredTraceState.java
b/src/java/org/apache/cassandra/tracing/ExpiredTraceState.java
index 6b4f90b..37a013b 100644
--- a/src/java/org/apache/cassandra/tracing/ExpiredTraceState.java
+++ b/src/java/org/apache/cassandra/tracing/ExpiredTraceState.java
@@ -29,7 +29,7 @@ public class ExpiredTraceState extends TraceState
{
public ExpiredTraceState(UUID sessionId)
{
-super(FBUtilities.getBroadcastAddress(), sessionId, true);
+super(FBUtilities.getBroadcastAddress(), sessionId);
}
public int elapsed()
http://git-wip-us.apache.org/repos/asf/cassandra/blob/18f3a79f/src/java/org/apache/cassandra/tracing/TraceState.java
--
diff --git a/src/java/org/apache/cassandra/tracing/TraceState.java
b/src/java/org/apache/cassandra/tracing/TraceState.java
index 25599c4..4d52f8f 100644
--- a/src/java/org/apache/cassandra/tracing/TraceState.java
+++ b/src/java/org/apache/cassandra/tracing/TraceState.java
@@ -45,16 +45,14 @@ public class TraceState
public final InetAddress coordinator;
public final Stopwatch watch;
public final ByteBuffer sessionIdBytes;
-public final boolean isLocallyOwned;
-public TraceState(InetAddress coordinator, UUID sessionId, boolean
locallyOwned)
+public TraceState(InetAddress coordinator, UUID sessionId)
{
assert coordinator != null;
assert sessionId != null;
this.coordinator = coordinator;
this.sessionId = sessionId;
-this.isLocallyOwned = locallyOwned;
sessionIdBytes = ByteBufferUtil.bytes(sessionId);
watch = new Stopwatch();
watch.start();
http://git-wip-us.apache.org/repos/asf/cassandra/blob/18f3a79f/src/java/org/apache/cassandra/tracing/Tracing.java
--
diff --git a/src/java/org/apache/cassandra/tracing/Tracing.java
b/src/java/org/apache/cassandra/tracing/Tracing.java
index eb5bad9..c692436 100644
--- a/src/java/org/apache/cassandra/tracing/Tracing.java
+++ b/src/java/org/apache/cassandra/tracing/Tracing.java
@@ -144,17 +144,16 @@ public class Tracing
{
assert state.get() == null;
-TraceState ts = new TraceState(localAddress, sessionId, true);
+TraceState ts = new TraceState(localAddress, sessionId);
state.set(ts);
sessions.put(sessionId, ts);
return sessionId;
}
-public void stopIfNonLocal(TraceState state)
+public void
[jira] [Commented] (CASSANDRA-5681) Refactor IESCS in Snitches
[ https://issues.apache.org/jira/browse/CASSANDRA-5681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690398#comment-13690398 ] Jonathan Ellis commented on CASSANDRA-5681: --- committed to 1.2.6 so we can roll that release; I'll leave merging to trunk to you :) Refactor IESCS in Snitches -- Key: CASSANDRA-5681 URL: https://issues.apache.org/jira/browse/CASSANDRA-5681 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.2.5 Reporter: Jason Brown Assignee: Jason Brown Priority: Minor Labels: snitch Fix For: 1.2.6, 2.0 beta 1 Attachments: 5681-v1.diff Reduce/refactor duplicated IESCS implementations in Ec2MRS, GPFS, and YPNTS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690401#comment-13690401
]
Jonathan Ellis commented on CASSANDRA-5668:
---
committed.
NPE in net.OutputTcpConnection when tracing is enabled
--
Key: CASSANDRA-5668
URL: https://issues.apache.org/jira/browse/CASSANDRA-5668
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Ryan McGuire
Assignee: Jonathan Ellis
Fix For: 1.2.6
Attachments: 5668-assert-2.txt, 5668-assert.txt, 5668-followup.txt,
5668-logs.tar.gz, 5668_npe_ddl.cql, 5668_npe_insert.cql, 5668.txt, system.log
I get multiple NullPointerException when trying to trace INSERT statements.
To reproduce:
{code}
$ ccm create -v git:trunk
$ ccm populate -n 3
$ ccm start
$ ccm node1 cqlsh 5668_npe_ddl.cql
$ ccm node1 cqlsh 5668_npe_insert.cql
{code}
And see many exceptions like this in the logs of node1:
{code}
ERROR [WRITE-/127.0.0.3] 2013-06-19 14:54:35,885 OutboundTcpConnection.java
(line 197) error writing to /127.0.0.3
java.lang.NullPointerException
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:182)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:144)
{code}
This is similar to CASSANDRA-5658 and is the reason that npe_ddl and
npe_insert are separate files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690403#comment-13690403
]
Ryan McGuire commented on CASSANDRA-5668:
-
The NullPointerException is gone, however, I see an unexpected change in the
trace after this commit.
Prior to the patch, I do this:
{code}
$ ccm create 5668
Current cluster is now: 5668
$ ccm populate -n 2:2
$ ccm start
$ ccm node1 cqlsh
Connected to 5668 at 127.0.0.1:9160.
[cqlsh 4.0.0 | Cassandra 2.0-SNAPSHOT | CQL spec 3.1.0 | Thrift protocol
19.37.0]
Use HELP for help.
cqlsh CREATE KEYSPACE test WITH replication = {'class':
'NetworkTopologyStrategy', 'dc1':2, 'dc2':2};
cqlsh CREATE TABLE test.test (id int PRIMARY KEY, value text);
cqlsh tracing on;
Now tracing requests.
cqlsh INSERT INTO test.test (id, value) VALUES ( 5, 'asdf');
Tracing session: 5d3bdaa0-da87-11e2-9eaa-35db2404c433
activity | timestamp
| source| source_elapsed
+--+---+
execute_cql3_query | 11:29:29,423
| 127.0.0.1 | 0
Parsing INSERT INTO test.test (id, value) VALUES ( 5, 'asdf'); | 11:29:29,424
| 127.0.0.1 | 1082
Peparing statement | 11:29:29,425
| 127.0.0.1 | 2241
Determining replicas for mutation | 11:29:29,425
| 127.0.0.1 | 2617
Sending message to /127.0.0.2 | 11:29:29,432
| 127.0.0.1 | 9321
Acquiring switchLock read lock | 11:29:29,433
| 127.0.0.1 | 10376
Appending to commitlog | 11:29:29,433
| 127.0.0.1 | 10424
Adding to test memtable | 11:29:29,433
| 127.0.0.1 | 10544
Sending message to /127.0.0.3 | 11:29:29,435
| 127.0.0.1 | 12658
Message received from /127.0.0.1 | 11:29:29,436
| 127.0.0.2 |102
Message received from /127.0.0.1 | 11:29:29,438
| 127.0.0.3 |103
Enqueuing forwarded write to /127.0.0.4 | 11:29:29,441
| 127.0.0.3 | 3596
Acquiring switchLock read lock | 11:29:29,441
| 127.0.0.3 | 3686
Appending to commitlog | 11:29:29,441
| 127.0.0.3 | 3704
Acquiring switchLock read lock | 11:29:29,442
| 127.0.0.2 | 5768
Adding to test memtable | 11:29:29,442
| 127.0.0.3 | 3759
Appending to commitlog | 11:29:29,442
| 127.0.0.2 | 5821
Sending message to /127.0.0.4 | 11:29:29,442
| 127.0.0.3 | 3834
Adding to test memtable | 11:29:29,442
| 127.0.0.2 | 5896
Enqueuing response to /127.0.0.1 | 11:29:29,442
| 127.0.0.3 | 4307
Enqueuing response to /127.0.0.1 | 11:29:29,443
| 127.0.0.2 | 6602
Message received from /127.0.0.3 | 11:29:29,444
| 127.0.0.4 | 92
Sending message to /127.0.0.1 | 11:29:29,444
| 127.0.0.2 | 8037
Message received from /127.0.0.2 | 11:29:29,445
| 127.0.0.1 | 36
Processing response from /127.0.0.2 | 11:29:29,445
| 127.0.0.1 |147
Acquiring switchLock read lock | 11:29:29,452
| 127.0.0.4 | 8791
Appending to commitlog | 11:29:29,452
| 127.0.0.4 | 8835
Adding to test memtable | 11:29:29,452
| 127.0.0.4 | 8897
Enqueuing response to /127.0.0.1 | 11:29:29,453
| 127.0.0.4 | 9542
Sending message to /127.0.0.1 | 11:29:29,454
| 127.0.0.4 | 10021
Message received from /127.0.0.4 | 11:29:29,454
| 127.0.0.1 | 9660
Processing response from /127.0.0.4 | 11:29:29,455
| 127.0.0.1 | 9804
Request complete | 11:29:29,435
| 127.0.0.1 | 12668
{code}
That's a 2x2 multi-dc cluster with a RF of 2 in each dc. I write a single
INSERT and I see four nodes append to their commit log. All is well.
After the patch, I see this trace:
{code}
Tracing session:
[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690406#comment-13690406
]
Jonathan Ellis commented on CASSANDRA-5668:
---
Did you get the followup patch in that? Should have addressed it.
NPE in net.OutputTcpConnection when tracing is enabled
--
Key: CASSANDRA-5668
URL: https://issues.apache.org/jira/browse/CASSANDRA-5668
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Ryan McGuire
Assignee: Jonathan Ellis
Fix For: 1.2.6
Attachments: 5668-assert-2.txt, 5668-assert.txt, 5668-followup.txt,
5668-logs.tar.gz, 5668_npe_ddl.cql, 5668_npe_insert.cql, 5668.txt, system.log
I get multiple NullPointerException when trying to trace INSERT statements.
To reproduce:
{code}
$ ccm create -v git:trunk
$ ccm populate -n 3
$ ccm start
$ ccm node1 cqlsh 5668_npe_ddl.cql
$ ccm node1 cqlsh 5668_npe_insert.cql
{code}
And see many exceptions like this in the logs of node1:
{code}
ERROR [WRITE-/127.0.0.3] 2013-06-19 14:54:35,885 OutboundTcpConnection.java
(line 197) error writing to /127.0.0.3
java.lang.NullPointerException
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:182)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:144)
{code}
This is similar to CASSANDRA-5658 and is the reason that npe_ddl and
npe_insert are separate files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690411#comment-13690411
]
Ryan McGuire commented on CASSANDRA-5668:
-
I've retested with the second patch, it's the same.
NPE in net.OutputTcpConnection when tracing is enabled
--
Key: CASSANDRA-5668
URL: https://issues.apache.org/jira/browse/CASSANDRA-5668
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Ryan McGuire
Assignee: Jonathan Ellis
Fix For: 1.2.6
Attachments: 5668-assert-2.txt, 5668-assert.txt, 5668-followup.txt,
5668-logs.tar.gz, 5668_npe_ddl.cql, 5668_npe_insert.cql, 5668.txt, system.log
I get multiple NullPointerException when trying to trace INSERT statements.
To reproduce:
{code}
$ ccm create -v git:trunk
$ ccm populate -n 3
$ ccm start
$ ccm node1 cqlsh 5668_npe_ddl.cql
$ ccm node1 cqlsh 5668_npe_insert.cql
{code}
And see many exceptions like this in the logs of node1:
{code}
ERROR [WRITE-/127.0.0.3] 2013-06-19 14:54:35,885 OutboundTcpConnection.java
(line 197) error writing to /127.0.0.3
java.lang.NullPointerException
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:182)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:144)
{code}
This is similar to CASSANDRA-5658 and is the reason that npe_ddl and
npe_insert are separate files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5561) Compaction strategy that minimizes re-compaction of old/frozen data
[ https://issues.apache.org/jira/browse/CASSANDRA-5561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690420#comment-13690420 ] Edward Capriolo commented on CASSANDRA-5561: One simple way to do this would be to specific a range in seconds like a week. A more complex but possibly better way would be having max number of acceptable sstables say 100. and we could calculate the distribution of range of time in an sstable and size. All the old weeks will likely have little or no data, they might have some bloom filter false positives so they might persist for longer then the target ttl age but they will get lumped together in the 'old stuff' sstable group. Normally the fixed sstable count and the distribution rules should make one table a week. Compaction strategy that minimizes re-compaction of old/frozen data --- Key: CASSANDRA-5561 URL: https://issues.apache.org/jira/browse/CASSANDRA-5561 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.2.3 Reporter: Tupshin Harper Fix For: 2.0.1 Neither LCS nor STCS are good for data that becomes immutable over time. The most obvious case is for time-series data where the application can guarantee that out-of-order delivery (to Cassandra) of events can't take place more than N minutes/seconds/hours/days have elapsed after the real (wall time). There are various approaches that could involve paying attention to the row keys (if they include a time component) and/or the column name (if they are TimeUUID or Integer based and are inherently time-ordered), but it might be sufficient to just look at the timestamp of the columns themselves. A possible approach: 1) Define an optional max-out-of-order window on a per-CF basis. 2) Use normal (LCS or STCS) compaction strategy for any SSTables that include any columns younger than max-out-of-order-delivery). 3) Use alternate compaction strategy (will call it TWCS time window compaction strategy for now) for any SSTables that only contain columns older than max-out-of-order-delivery. 4) TWCS will only compact sstables containing data older than max-out-of-order-delivery. 5) TWCS will only perform compaction to reduce row fragmentation (if there is any by the time it gets to TWCS or to reduce the number of small sstables. 6) To minimize re-compaction in TWCS, it should aggresively try to compact as many small sstables as possible into one large sstable that would never have to get recompacted. In the case of large datasets (e.g. 5TB per node) with LCS, there would be on the order of seven levels, and hence seven separate writes of the same data over time. With this approach, it should be possible to get about 3 compactions per column (2 in original compaction and one more once hitting TWCS) in most cases, cutting the write workload by a factor of two or more for high volume time-series applications. Note that the only workaround I can currently suggest to minimize compaction for these workloads is to programatically shard your data across time-window ranges (e.g. new CF per week), but that pushes unnecessary writing and querying logic out to the user and is not as convenient nor flexible. Also note that I am not convinced that the approach I've suggested above is the best/most general way to solve the problem, but it does appear to be a relatively easy one to implement. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
git commit: restore fetching global trace state in default .execute method
Updated Branches:
refs/heads/cassandra-1.2 e75e33fa6 - 110d283af
restore fetching global trace state in default .execute method
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/110d283a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/110d283a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/110d283a
Branch: refs/heads/cassandra-1.2
Commit: 110d283afd780774a44368b17177b5e8e781e37f
Parents: e75e33f
Author: Jonathan Ellis jbel...@apache.org
Authored: Fri Jun 21 11:13:21 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Fri Jun 21 11:13:21 2013 -0500
--
.../concurrent/DebuggableThreadPoolExecutor.java | 11 ---
1 file changed, 4 insertions(+), 7 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/110d283a/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
--
diff --git
a/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
b/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
index 26441ec..46a3216 100644
--- a/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
+++ b/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
@@ -133,7 +133,9 @@ public class DebuggableThreadPoolExecutor extends
ThreadPoolExecutor implements
public void execute(Runnable command, TraceState state)
{
-super.execute(state == null ? command : new
TraceSessionWrapperObject(command, state));
+super.execute(state == null || command instanceof TraceSessionWrapper
+ ? command
+ : new TraceSessionWrapperObject(command, state));
}
// execute does not call newTaskFor
@@ -141,7 +143,7 @@ public class DebuggableThreadPoolExecutor extends
ThreadPoolExecutor implements
public void execute(Runnable command)
{
super.execute(isTracing() !(command instanceof TraceSessionWrapper)
- ? new TraceSessionWrapperObject(command)
+ ? new
TraceSessionWrapperObject(Executors.callable(command, null))
: command);
}
@@ -261,11 +263,6 @@ public class DebuggableThreadPoolExecutor extends
ThreadPoolExecutor implements
{
private final TraceState state;
-public TraceSessionWrapper(Runnable command)
-{
-this(command, null);
-}
-
public TraceSessionWrapper(CallableT callable)
{
super(callable);
[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690434#comment-13690434
]
Jonathan Ellis commented on CASSANDRA-5668:
---
fix pushed in 110d283afd780774a44368b17177b5e8e781e37f
NPE in net.OutputTcpConnection when tracing is enabled
--
Key: CASSANDRA-5668
URL: https://issues.apache.org/jira/browse/CASSANDRA-5668
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Ryan McGuire
Assignee: Jonathan Ellis
Fix For: 1.2.6
Attachments: 5668-assert-2.txt, 5668-assert.txt, 5668-followup.txt,
5668-logs.tar.gz, 5668_npe_ddl.cql, 5668_npe_insert.cql, 5668.txt, system.log
I get multiple NullPointerException when trying to trace INSERT statements.
To reproduce:
{code}
$ ccm create -v git:trunk
$ ccm populate -n 3
$ ccm start
$ ccm node1 cqlsh 5668_npe_ddl.cql
$ ccm node1 cqlsh 5668_npe_insert.cql
{code}
And see many exceptions like this in the logs of node1:
{code}
ERROR [WRITE-/127.0.0.3] 2013-06-19 14:54:35,885 OutboundTcpConnection.java
(line 197) error writing to /127.0.0.3
java.lang.NullPointerException
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:182)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:144)
{code}
This is similar to CASSANDRA-5658 and is the reason that npe_ddl and
npe_insert are separate files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5561) Compaction strategy that minimizes re-compaction of old/frozen data
[
https://issues.apache.org/jira/browse/CASSANDRA-5561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690438#comment-13690438
]
Edward Capriolo commented on CASSANDRA-5561:
{quote}
How is this supposed to give better results than simply not compacting sstables
that we don't need to read anymore, as suggested in CASSANDRA-5515?
{quote}
That approach could work, however I think just having a simple (and dumb) fixed
time bucketing approach would solve the problem. The way it currently works, is
for sized tiered minCompactionThreshold or maxCompactionThreshold kicks in and
it mashes together old and new sstables then the entire situation becomes
unmanageable. Eventually everything gets compacted together is some
unmanageable way and then once you hit 80% disk usage you fall over a cliff and
there is no way out.
Compaction strategy that minimizes re-compaction of old/frozen data
---
Key: CASSANDRA-5561
URL: https://issues.apache.org/jira/browse/CASSANDRA-5561
Project: Cassandra
Issue Type: Improvement
Components: Core
Affects Versions: 1.2.3
Reporter: Tupshin Harper
Fix For: 2.0.1
Neither LCS nor STCS are good for data that becomes immutable over time. The
most obvious case is for time-series data where the application can guarantee
that out-of-order delivery (to Cassandra) of events can't take place more
than N minutes/seconds/hours/days have elapsed after the real (wall time).
There are various approaches that could involve paying attention to the row
keys (if they include a time component) and/or the column name (if they are
TimeUUID or Integer based and are inherently time-ordered), but it might be
sufficient to just look at the timestamp of the columns themselves.
A possible approach:
1) Define an optional max-out-of-order window on a per-CF basis.
2) Use normal (LCS or STCS) compaction strategy for any SSTables that include
any columns younger than max-out-of-order-delivery).
3) Use alternate compaction strategy (will call it TWCS time window
compaction strategy for now) for any SSTables that only contain columns older
than max-out-of-order-delivery.
4) TWCS will only compact sstables containing data older than
max-out-of-order-delivery.
5) TWCS will only perform compaction to reduce row fragmentation (if there is
any by the time it gets to TWCS or to reduce the number of small sstables.
6) To minimize re-compaction in TWCS, it should aggresively try to compact as
many small sstables as possible into one large sstable that would never have
to get recompacted.
In the case of large datasets (e.g. 5TB per node) with LCS, there would be on
the order of seven levels, and hence seven separate writes of the same data
over time. With this approach, it should be possible to get about 3
compactions per column (2 in original compaction and one more once hitting
TWCS) in most cases, cutting the write workload by a factor of two or more
for high volume time-series applications.
Note that the only workaround I can currently suggest to minimize compaction
for these workloads is to programatically shard your data across time-window
ranges (e.g. new CF per week), but that pushes unnecessary writing and
querying logic out to the user and is not as convenient nor flexible.
Also note that I am not convinced that the approach I've suggested above is
the best/most general way to solve the problem, but it does appear to be a
relatively easy one to implement.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690441#comment-13690441
]
Ryan McGuire commented on CASSANDRA-5668:
-
That works in cassandra-1.2 - if you commit to trunk, I'll test there too.
NPE in net.OutputTcpConnection when tracing is enabled
--
Key: CASSANDRA-5668
URL: https://issues.apache.org/jira/browse/CASSANDRA-5668
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Ryan McGuire
Assignee: Jonathan Ellis
Fix For: 1.2.6
Attachments: 5668-assert-2.txt, 5668-assert.txt, 5668-followup.txt,
5668-logs.tar.gz, 5668_npe_ddl.cql, 5668_npe_insert.cql, 5668.txt, system.log
I get multiple NullPointerException when trying to trace INSERT statements.
To reproduce:
{code}
$ ccm create -v git:trunk
$ ccm populate -n 3
$ ccm start
$ ccm node1 cqlsh 5668_npe_ddl.cql
$ ccm node1 cqlsh 5668_npe_insert.cql
{code}
And see many exceptions like this in the logs of node1:
{code}
ERROR [WRITE-/127.0.0.3] 2013-06-19 14:54:35,885 OutboundTcpConnection.java
(line 197) error writing to /127.0.0.3
java.lang.NullPointerException
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:182)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:144)
{code}
This is similar to CASSANDRA-5658 and is the reason that npe_ddl and
npe_insert are separate files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-5665) Gossiper.handleMajorStateChange can lose existing node ApplicationState
[ https://issues.apache.org/jira/browse/CASSANDRA-5665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690444#comment-13690444 ] Jason Brown edited comment on CASSANDRA-5665 at 6/21/13 4:25 PM: - v2 follows advice from ~jbellis about combining the proposed handleMajorStateChange and existing applyNewStates. was (Author: jasobrown): v2 follows advice from ~jbellis about combining the handleMajorStateChange and applyNewStates. Gossiper.handleMajorStateChange can lose existing node ApplicationState --- Key: CASSANDRA-5665 URL: https://issues.apache.org/jira/browse/CASSANDRA-5665 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.2.5 Reporter: Jason Brown Assignee: Jason Brown Priority: Minor Labels: gossip, upgrade Fix For: 1.2.6, 2.0 beta 1 Attachments: 5665-v1.diff, 5665-v2.diff Dovetailing on CASSANDRA-5660, I discovered that further along during an upgrade, when more nodes are on the new major version, a node the previous version can get passed some incomplete Gossip info about another, already upgraded node, and the older node drops AppStat info about that node. I think what happens is that a 1.1 node (older rev) gets gossip info from a 1.2 node (A), which includes incomplete (lacking some AppState data) gossip info about another 1.2 node (B). The 1.1 node, which has marked incorrectly kicked node B out of gossip due to the bug described in #5660, then takes that incomplete node B info and wholesale replaces any previous known state about node B in Gossiper.handleMajorStateChanged. Thus, if we previously had DC/RACK info, it'll get dropped as part of the endpointStateMap.put(endpointstate). When the data being pased is incomplete, 1.1 will start referencing node B and gets into the NPE situation in #5498. Anecdotally, this bad state is short-lived, less than a few minutes, even as short as ten seconds, until gossip catches up and properly propagates the AppState data. Furthermore, when upgrading a two datacenter, 48 node cluster, it only occurred on two nodes for less than a minute each. Thus, the scope seems limited but can occur. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5665) Gossiper.handleMajorStateChange can lose existing node ApplicationState
[ https://issues.apache.org/jira/browse/CASSANDRA-5665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Brown updated CASSANDRA-5665: --- Attachment: 5665-v2.diff v2 follows advice from ~jbellis about combining the handleMajorStateChange and applyNewStates. Gossiper.handleMajorStateChange can lose existing node ApplicationState --- Key: CASSANDRA-5665 URL: https://issues.apache.org/jira/browse/CASSANDRA-5665 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.2.5 Reporter: Jason Brown Assignee: Jason Brown Priority: Minor Labels: gossip, upgrade Fix For: 1.2.6, 2.0 beta 1 Attachments: 5665-v1.diff, 5665-v2.diff Dovetailing on CASSANDRA-5660, I discovered that further along during an upgrade, when more nodes are on the new major version, a node the previous version can get passed some incomplete Gossip info about another, already upgraded node, and the older node drops AppStat info about that node. I think what happens is that a 1.1 node (older rev) gets gossip info from a 1.2 node (A), which includes incomplete (lacking some AppState data) gossip info about another 1.2 node (B). The 1.1 node, which has marked incorrectly kicked node B out of gossip due to the bug described in #5660, then takes that incomplete node B info and wholesale replaces any previous known state about node B in Gossiper.handleMajorStateChanged. Thus, if we previously had DC/RACK info, it'll get dropped as part of the endpointStateMap.put(endpointstate). When the data being pased is incomplete, 1.1 will start referencing node B and gets into the NPE situation in #5498. Anecdotally, this bad state is short-lived, less than a few minutes, even as short as ten seconds, until gossip catches up and properly propagates the AppState data. Furthermore, when upgrading a two datacenter, 48 node cluster, it only occurred on two nodes for less than a minute each. Thus, the scope seems limited but can occur. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5665) Gossiper.handleMajorStateChange can lose existing node ApplicationState
[ https://issues.apache.org/jira/browse/CASSANDRA-5665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690451#comment-13690451 ] Jonathan Ellis commented on CASSANDRA-5665: --- LGTM. Note: is it a [pre-existing] bug that we don't call doNotifications on major state change? Gossiper.handleMajorStateChange can lose existing node ApplicationState --- Key: CASSANDRA-5665 URL: https://issues.apache.org/jira/browse/CASSANDRA-5665 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.2.5 Reporter: Jason Brown Assignee: Jason Brown Priority: Minor Labels: gossip, upgrade Fix For: 1.2.6, 2.0 beta 1 Attachments: 5665-v1.diff, 5665-v2.diff Dovetailing on CASSANDRA-5660, I discovered that further along during an upgrade, when more nodes are on the new major version, a node the previous version can get passed some incomplete Gossip info about another, already upgraded node, and the older node drops AppStat info about that node. I think what happens is that a 1.1 node (older rev) gets gossip info from a 1.2 node (A), which includes incomplete (lacking some AppState data) gossip info about another 1.2 node (B). The 1.1 node, which has marked incorrectly kicked node B out of gossip due to the bug described in #5660, then takes that incomplete node B info and wholesale replaces any previous known state about node B in Gossiper.handleMajorStateChanged. Thus, if we previously had DC/RACK info, it'll get dropped as part of the endpointStateMap.put(endpointstate). When the data being pased is incomplete, 1.1 will start referencing node B and gets into the NPE situation in #5498. Anecdotally, this bad state is short-lived, less than a few minutes, even as short as ten seconds, until gossip catches up and properly propagates the AppState data. Furthermore, when upgrading a two datacenter, 48 node cluster, it only occurred on two nodes for less than a minute each. Thus, the scope seems limited but can occur. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
git commit: Gossiper.handleMajorStateChange can lose existing node ApplicationState patch by jasobrown; reviewe4d by jbellis for CASSANDRA-5665
Updated Branches:
refs/heads/cassandra-1.2 110d283af - b7e13b89c
Gossiper.handleMajorStateChange can lose existing node ApplicationState
patch by jasobrown; reviewe4d by jbellis for CASSANDRA-5665
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b7e13b89
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b7e13b89
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b7e13b89
Branch: refs/heads/cassandra-1.2
Commit: b7e13b89c265c28acfb624a984b97a06a837c3ea
Parents: 110d283
Author: Jason Brown jasedbr...@gmail.com
Authored: Fri Jun 21 09:23:17 2013 -0700
Committer: Jason Brown jasedbr...@gmail.com
Committed: Fri Jun 21 09:23:52 2013 -0700
--
src/java/org/apache/cassandra/gms/Gossiper.java | 47 +++-
1 file changed, 26 insertions(+), 21 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/b7e13b89/src/java/org/apache/cassandra/gms/Gossiper.java
--
diff --git a/src/java/org/apache/cassandra/gms/Gossiper.java
b/src/java/org/apache/cassandra/gms/Gossiper.java
index efa9865..6b0bbe9 100644
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@ -871,6 +871,7 @@ public class Gossiper implements
IFailureDetectionEventListener, GossiperMBean
if (logger.isTraceEnabled())
logger.trace(Updating heartbeat state generation to
+ remoteGeneration + from + localGeneration + for + ep);
// major state change will handle the update by inserting
the remote state directly
+copyNewerApplicationStates(remoteState, localEpStatePtr);
handleMajorStateChange(ep, remoteState);
}
else if ( remoteGeneration == localGeneration ) // generation
has not changed, apply new states
@@ -880,11 +881,18 @@ public class Gossiper implements
IFailureDetectionEventListener, GossiperMBean
int remoteMaxVersion =
getMaxEndpointStateVersion(remoteState);
if ( remoteMaxVersion localMaxVersion )
{
-// apply states, but do not notify since there is no
major change
-applyNewStates(ep, localEpStatePtr, remoteState);
+if (logger.isTraceEnabled())
+{
+logger.trace(Updating heartbeat state version to
+ remoteState.getHeartBeatState().getHeartBeatVersion() +
+ from +
localEpStatePtr.getHeartBeatState().getHeartBeatVersion() + for + ep);
+}
+
localEpStatePtr.setHeartBeatState(remoteState.getHeartBeatState());
+MapApplicationState, VersionedValue merged =
copyNewerApplicationStates(localEpStatePtr, remoteState);
+for (EntryApplicationState, VersionedValue appState
: merged.entrySet())
+doNotifications(ep, appState.getKey(),
appState.getValue());
}
else if (logger.isTraceEnabled())
-logger.trace(Ignoring remote version +
remoteMaxVersion + = + localMaxVersion + for + ep);
+logger.trace(Ignoring remote version +
remoteMaxVersion + = + localMaxVersion + for + ep);
if (!localEpStatePtr.isAlive()
!isDeadState(localEpStatePtr)) // unless of course, it was dead
markAlive(ep, localEpStatePtr);
}
@@ -903,28 +911,25 @@ public class Gossiper implements
IFailureDetectionEventListener, GossiperMBean
}
}
-private void applyNewStates(InetAddress addr, EndpointState localState,
EndpointState remoteState)
+private MapApplicationState, VersionedValue
copyNewerApplicationStates(EndpointState toState, EndpointState fromState)
{
-// don't assert here, since if the node restarts the version will go
back to zero
-int oldVersion = localState.getHeartBeatState().getHeartBeatVersion();
-
-localState.setHeartBeatState(remoteState.getHeartBeatState());
-if (logger.isTraceEnabled())
-logger.trace(Updating heartbeat state version to +
localState.getHeartBeatState().getHeartBeatVersion() + from + oldVersion +
for + addr + ...);
-
-// we need to make two loops here, one to apply, then another to
notify, this way all states in an update are present and current when the
notifications are received
-for (EntryApplicationState, VersionedValue remoteEntry :
remoteState.getApplicationStateMap().entrySet())
+MapApplicationState,
[jira] [Commented] (CASSANDRA-5665) Gossiper.handleMajorStateChange can lose existing node ApplicationState
[ https://issues.apache.org/jira/browse/CASSANDRA-5665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690462#comment-13690462 ] Jason Brown commented on CASSANDRA-5665: [~jbellis] Yeah, i saw that we don't call doNotifications for major state change. However, we do call (in order): onRestart, onAlive, and onJoin. I suspect that probably will cover most of the same funcationality that onChange does. I can dig a little deeper, and if I suspect there might be something there, I'll create a followup ticket to fix. Immediately, though, I know we'll end up thrashing more connections in the 'reconnectable snitches' as we'll call OTCP.reset more than once. However, that might already be happening with the current scheme (at least, up until a few days ago, when we started working on all the gossiper/connection/upgrade tickets like CASSANDRA-5660). Gossiper.handleMajorStateChange can lose existing node ApplicationState --- Key: CASSANDRA-5665 URL: https://issues.apache.org/jira/browse/CASSANDRA-5665 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.2.5 Reporter: Jason Brown Assignee: Jason Brown Priority: Minor Labels: gossip, upgrade Fix For: 1.2.6, 2.0 beta 1 Attachments: 5665-v1.diff, 5665-v2.diff Dovetailing on CASSANDRA-5660, I discovered that further along during an upgrade, when more nodes are on the new major version, a node the previous version can get passed some incomplete Gossip info about another, already upgraded node, and the older node drops AppStat info about that node. I think what happens is that a 1.1 node (older rev) gets gossip info from a 1.2 node (A), which includes incomplete (lacking some AppState data) gossip info about another 1.2 node (B). The 1.1 node, which has marked incorrectly kicked node B out of gossip due to the bug described in #5660, then takes that incomplete node B info and wholesale replaces any previous known state about node B in Gossiper.handleMajorStateChanged. Thus, if we previously had DC/RACK info, it'll get dropped as part of the endpointStateMap.put(endpointstate). When the data being pased is incomplete, 1.1 will start referencing node B and gets into the NPE situation in #5498. Anecdotally, this bad state is short-lived, less than a few minutes, even as short as ten seconds, until gossip catches up and properly propagates the AppState data. Furthermore, when upgrading a two datacenter, 48 node cluster, it only occurred on two nodes for less than a minute each. Thus, the scope seems limited but can occur. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5665) Gossiper.handleMajorStateChange can lose existing node ApplicationState
[ https://issues.apache.org/jira/browse/CASSANDRA-5665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690465#comment-13690465 ] Jason Brown commented on CASSANDRA-5665: committed to 1.2 - trunk coming in a few minutes Gossiper.handleMajorStateChange can lose existing node ApplicationState --- Key: CASSANDRA-5665 URL: https://issues.apache.org/jira/browse/CASSANDRA-5665 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.2.5 Reporter: Jason Brown Assignee: Jason Brown Priority: Minor Labels: gossip, upgrade Fix For: 1.2.6, 2.0 beta 1 Attachments: 5665-v1.diff, 5665-v2.diff Dovetailing on CASSANDRA-5660, I discovered that further along during an upgrade, when more nodes are on the new major version, a node the previous version can get passed some incomplete Gossip info about another, already upgraded node, and the older node drops AppStat info about that node. I think what happens is that a 1.1 node (older rev) gets gossip info from a 1.2 node (A), which includes incomplete (lacking some AppState data) gossip info about another 1.2 node (B). The 1.1 node, which has marked incorrectly kicked node B out of gossip due to the bug described in #5660, then takes that incomplete node B info and wholesale replaces any previous known state about node B in Gossiper.handleMajorStateChanged. Thus, if we previously had DC/RACK info, it'll get dropped as part of the endpointStateMap.put(endpointstate). When the data being pased is incomplete, 1.1 will start referencing node B and gets into the NPE situation in #5498. Anecdotally, this bad state is short-lived, less than a few minutes, even as short as ten seconds, until gossip catches up and properly propagates the AppState data. Furthermore, when upgrading a two datacenter, 48 node cluster, it only occurred on two nodes for less than a minute each. Thus, the scope seems limited but can occur. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[2/3] git commit: restore fetching global trace state in default .execute method
restore fetching global trace state in default .execute method
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b8b96bdb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b8b96bdb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b8b96bdb
Branch: refs/heads/trunk
Commit: b8b96bdb733aed87197df3f01834fe47cc756ae8
Parents: ded233e
Author: Jonathan Ellis jbel...@apache.org
Authored: Fri Jun 21 11:13:21 2013 -0500
Committer: Jason Brown jasedbr...@gmail.com
Committed: Fri Jun 21 09:55:34 2013 -0700
--
.../concurrent/DebuggableThreadPoolExecutor.java | 11 ---
1 file changed, 4 insertions(+), 7 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/b8b96bdb/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
--
diff --git
a/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
b/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
index 26441ec..46a3216 100644
--- a/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
+++ b/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
@@ -133,7 +133,9 @@ public class DebuggableThreadPoolExecutor extends
ThreadPoolExecutor implements
public void execute(Runnable command, TraceState state)
{
-super.execute(state == null ? command : new
TraceSessionWrapperObject(command, state));
+super.execute(state == null || command instanceof TraceSessionWrapper
+ ? command
+ : new TraceSessionWrapperObject(command, state));
}
// execute does not call newTaskFor
@@ -141,7 +143,7 @@ public class DebuggableThreadPoolExecutor extends
ThreadPoolExecutor implements
public void execute(Runnable command)
{
super.execute(isTracing() !(command instanceof TraceSessionWrapper)
- ? new TraceSessionWrapperObject(command)
+ ? new
TraceSessionWrapperObject(Executors.callable(command, null))
: command);
}
@@ -261,11 +263,6 @@ public class DebuggableThreadPoolExecutor extends
ThreadPoolExecutor implements
{
private final TraceState state;
-public TraceSessionWrapper(Runnable command)
-{
-this(command, null);
-}
-
public TraceSessionWrapper(CallableT callable)
{
super(callable);
[3/3] git commit: refactor reconnecting snitches patch by jasobrown; reviewed by jbellis for CASSANDRA-5681
refactor reconnecting snitches
patch by jasobrown; reviewed by jbellis for CASSANDRA-5681
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ded233ef
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ded233ef
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ded233ef
Branch: refs/heads/trunk
Commit: ded233ef0c93ecfe0c1bf582cfa004e5ca574934
Parents: 36aae61
Author: Jonathan Ellis jbel...@apache.org
Authored: Fri Jun 21 10:40:31 2013 -0500
Committer: Jason Brown jasedbr...@gmail.com
Committed: Fri Jun 21 09:55:34 2013 -0700
--
.../cassandra/locator/Ec2MultiRegionSnitch.java | 71 +---
.../locator/GossipingPropertyFileSnitch.java| 63 +-
.../locator/ReconnectableSnitchHelper.java | 88
3 files changed, 94 insertions(+), 128 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ded233ef/src/java/org/apache/cassandra/locator/Ec2MultiRegionSnitch.java
--
diff --git a/src/java/org/apache/cassandra/locator/Ec2MultiRegionSnitch.java
b/src/java/org/apache/cassandra/locator/Ec2MultiRegionSnitch.java
index 9317941..bd5e091 100644
--- a/src/java/org/apache/cassandra/locator/Ec2MultiRegionSnitch.java
+++ b/src/java/org/apache/cassandra/locator/Ec2MultiRegionSnitch.java
@@ -19,16 +19,11 @@ package org.apache.cassandra.locator;
import java.io.IOException;
import java.net.InetAddress;
-import java.net.UnknownHostException;
import org.apache.cassandra.exceptions.ConfigurationException;
import org.apache.cassandra.config.DatabaseDescriptor;
import org.apache.cassandra.gms.ApplicationState;
-import org.apache.cassandra.gms.EndpointState;
import org.apache.cassandra.gms.Gossiper;
-import org.apache.cassandra.gms.IEndpointStateChangeSubscriber;
-import org.apache.cassandra.gms.VersionedValue;
-import org.apache.cassandra.net.MessagingService;
import org.apache.cassandra.service.StorageService;
/**
@@ -36,16 +31,13 @@ import org.apache.cassandra.service.StorageService;
*
* 2) Snitch will set the private IP as a Gossip application state.
*
- * 3) Snitch implements IESCS and will reset the connection if it is within the
+ * 3) Uses a helper class that implements IESCS and will reset the public IP
connection if it is within the
* same region to communicate via private IP.
*
- * Implements Ec2Snitch to inherit its functionality and extend it for
- * Multi-Region.
- *
* Operational: All the nodes in this cluster needs to be able to (modify the
* Security group settings in AWS) communicate via Public IP's.
*/
-public class Ec2MultiRegionSnitch extends Ec2Snitch implements
IEndpointStateChangeSubscriber
+public class Ec2MultiRegionSnitch extends Ec2Snitch
{
private static final String PUBLIC_IP_QUERY_URL =
http://169.254.169.254/latest/meta-data/public-ipv4;;
private static final String PRIVATE_IP_QUERY_URL =
http://169.254.169.254/latest/meta-data/local-ipv4;;
@@ -62,67 +54,10 @@ public class Ec2MultiRegionSnitch extends Ec2Snitch
implements IEndpointStateCha
DatabaseDescriptor.setBroadcastAddress(localPublicAddress);
}
-public void onJoin(InetAddress endpoint, EndpointState epState)
-{
-if (epState.getApplicationState(ApplicationState.INTERNAL_IP) != null)
-reconnect(endpoint,
epState.getApplicationState(ApplicationState.INTERNAL_IP));
-}
-
-public void onChange(InetAddress endpoint, ApplicationState state,
VersionedValue value)
-{
-if (state == ApplicationState.INTERNAL_IP)
-reconnect(endpoint, value);
-}
-
-public void onAlive(InetAddress endpoint, EndpointState state)
-{
-if (state.getApplicationState(ApplicationState.INTERNAL_IP) != null)
-reconnect(endpoint,
state.getApplicationState(ApplicationState.INTERNAL_IP));
-}
-
-public void onDead(InetAddress endpoint, EndpointState state)
-{
-// do nothing
-}
-
-public void onRestart(InetAddress endpoint, EndpointState state)
-{
-// do nothing
-}
-
-public void onRemove(InetAddress endpoint)
-{
-// do nothing.
-}
-
-private void reconnect(InetAddress publicAddress, VersionedValue
localAddressValue)
-{
-try
-{
-reconnect(publicAddress,
InetAddress.getByName(localAddressValue.value));
-}
-catch (UnknownHostException e)
-{
-logger.error(Error in getting the IP address resolved: , e);
-}
-}
-
-private void reconnect(InetAddress publicAddress, InetAddress localAddress)
-{
-if
(getDatacenter(publicAddress).equals(getDatacenter(localPublicAddress))
-
[1/3] git commit: Gossiper.handleMajorStateChange can lose existing node ApplicationState patch by jasobrown; reviewe4d by jbellis for CASSANDRA-5665
Updated Branches:
refs/heads/trunk 36aae612a - d99a6f2a2
Gossiper.handleMajorStateChange can lose existing node ApplicationState
patch by jasobrown; reviewe4d by jbellis for CASSANDRA-5665
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d99a6f2a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d99a6f2a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d99a6f2a
Branch: refs/heads/trunk
Commit: d99a6f2a2bda290404c6706e7b1db87c59dd04cb
Parents: b8b96bd
Author: Jason Brown jasedbr...@gmail.com
Authored: Fri Jun 21 09:23:17 2013 -0700
Committer: Jason Brown jasedbr...@gmail.com
Committed: Fri Jun 21 09:55:34 2013 -0700
--
src/java/org/apache/cassandra/gms/Gossiper.java | 45 +++-
1 file changed, 25 insertions(+), 20 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d99a6f2a/src/java/org/apache/cassandra/gms/Gossiper.java
--
diff --git a/src/java/org/apache/cassandra/gms/Gossiper.java
b/src/java/org/apache/cassandra/gms/Gossiper.java
index d8918f3..d1a2ab5 100644
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@ -874,6 +874,7 @@ public class Gossiper implements
IFailureDetectionEventListener, GossiperMBean
if (logger.isTraceEnabled())
logger.trace(Updating heartbeat state generation to
+ remoteGeneration + from + localGeneration + for + ep);
// major state change will handle the update by inserting
the remote state directly
+copyNewerApplicationStates(remoteState, localEpStatePtr);
handleMajorStateChange(ep, remoteState);
}
else if (remoteGeneration == localGeneration) // generation
has not changed, apply new states
@@ -883,8 +884,15 @@ public class Gossiper implements
IFailureDetectionEventListener, GossiperMBean
int remoteMaxVersion =
getMaxEndpointStateVersion(remoteState);
if (remoteMaxVersion localMaxVersion)
{
-// apply states, but do not notify since there is no
major change
-applyNewStates(ep, localEpStatePtr, remoteState);
+if (logger.isTraceEnabled())
+{
+logger.trace(Updating heartbeat state version to
+ remoteState.getHeartBeatState().getHeartBeatVersion() +
+ from +
localEpStatePtr.getHeartBeatState().getHeartBeatVersion() + for + ep);
+}
+
localEpStatePtr.setHeartBeatState(remoteState.getHeartBeatState());
+MapApplicationState, VersionedValue merged =
copyNewerApplicationStates(localEpStatePtr, remoteState);
+for (EntryApplicationState, VersionedValue appState
: merged.entrySet())
+doNotifications(ep, appState.getKey(),
appState.getValue());
}
else if (logger.isTraceEnabled())
logger.trace(Ignoring remote version +
remoteMaxVersion + = + localMaxVersion + for + ep);
@@ -906,28 +914,25 @@ public class Gossiper implements
IFailureDetectionEventListener, GossiperMBean
}
}
-private void applyNewStates(InetAddress addr, EndpointState localState,
EndpointState remoteState)
+private MapApplicationState, VersionedValue
copyNewerApplicationStates(EndpointState toState, EndpointState fromState)
{
-// don't assert here, since if the node restarts the version will go
back to zero
-int oldVersion = localState.getHeartBeatState().getHeartBeatVersion();
-
-localState.setHeartBeatState(remoteState.getHeartBeatState());
-if (logger.isTraceEnabled())
-logger.trace(Updating heartbeat state version to +
localState.getHeartBeatState().getHeartBeatVersion() + from + oldVersion +
for + addr + ...);
-
-// we need to make two loops here, one to apply, then another to
notify, this way all states in an update are present and current when the
notifications are received
-for (EntryApplicationState, VersionedValue remoteEntry :
remoteState.getApplicationStateMap().entrySet())
+MapApplicationState, VersionedValue merged = new
HashMapApplicationState, VersionedValue();
+for (EntryApplicationState, VersionedValue fromEntry :
fromState.getApplicationStateMap().entrySet())
{
-ApplicationState remoteKey = remoteEntry.getKey();
-VersionedValue remoteValue = remoteEntry.getValue();
+
Git Push Summary
Updated Tags: refs/tags/1.2.6-tentative [created] 79410e4a8
git commit: Versions and license for 1.2.6 release
Updated Branches: refs/heads/cassandra-1.2 b7e13b89c - 79410e4a8 Versions and license for 1.2.6 release Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/79410e4a Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/79410e4a Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/79410e4a Branch: refs/heads/cassandra-1.2 Commit: 79410e4a887e7adac796339c1a565436deaf254a Parents: b7e13b8 Author: Sylvain Lebresne sylv...@datastax.com Authored: Fri Jun 21 19:12:31 2013 +0200 Committer: Sylvain Lebresne sylv...@datastax.com Committed: Fri Jun 21 19:12:31 2013 +0200 -- NEWS.txt | 1 - build.xml | 2 +- debian/changelog | 6 ++ .../locator/ReconnectableSnitchHelper.java | 17 + 4 files changed, 24 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/79410e4a/NEWS.txt -- diff --git a/NEWS.txt b/NEWS.txt index dbc9aab..cb40981 100644 --- a/NEWS.txt +++ b/NEWS.txt @@ -21,7 +21,6 @@ Upgrading - hinted_handoff_throttle_in_kb is now reduced by a factor proportional to the number of nodes in the cluster (see https://issues.apache.org/jira/browse/CASSANDRA-5272). - - CQL3 syntax for CREATE CUSTOM INDEX has been updated. See CQL3 documentation for details. http://git-wip-us.apache.org/repos/asf/cassandra/blob/79410e4a/build.xml -- diff --git a/build.xml b/build.xml index 46ffbb1..4e32351 100644 --- a/build.xml +++ b/build.xml @@ -25,7 +25,7 @@ property name=debuglevel value=source,lines,vars/ !-- default version and SCM information -- -property name=base.version value=1.2.5/ +property name=base.version value=1.2.6/ property name=scm.connection value=scm:git://git.apache.org/cassandra.git/ property name=scm.developerConnection value=scm:git://git.apache.org/cassandra.git/ property name=scm.url value=http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree/ http://git-wip-us.apache.org/repos/asf/cassandra/blob/79410e4a/debian/changelog -- diff --git a/debian/changelog b/debian/changelog index 626374f..3be9ba0 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,9 @@ +cassandra (1.2.6) unstable; urgency=low + + * New release + + -- Sylvain Lebresne slebre...@apache.org Fri, 21 Jun 2013 18:58:24 +0200 + cassandra (1.2.5) unstable; urgency=low * New release http://git-wip-us.apache.org/repos/asf/cassandra/blob/79410e4a/src/java/org/apache/cassandra/locator/ReconnectableSnitchHelper.java -- diff --git a/src/java/org/apache/cassandra/locator/ReconnectableSnitchHelper.java b/src/java/org/apache/cassandra/locator/ReconnectableSnitchHelper.java index adec953..bef245e 100644 --- a/src/java/org/apache/cassandra/locator/ReconnectableSnitchHelper.java +++ b/src/java/org/apache/cassandra/locator/ReconnectableSnitchHelper.java @@ -1,3 +1,20 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * License); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an AS IS BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.apache.cassandra.locator; import java.net.InetAddress;
[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690537#comment-13690537
]
Jonathan Ellis commented on CASSANDRA-5668:
---
Merged to trunk.
NPE in net.OutputTcpConnection when tracing is enabled
--
Key: CASSANDRA-5668
URL: https://issues.apache.org/jira/browse/CASSANDRA-5668
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Ryan McGuire
Assignee: Jonathan Ellis
Fix For: 1.2.6
Attachments: 5668-assert-2.txt, 5668-assert.txt, 5668-followup.txt,
5668-logs.tar.gz, 5668_npe_ddl.cql, 5668_npe_insert.cql, 5668.txt, system.log
I get multiple NullPointerException when trying to trace INSERT statements.
To reproduce:
{code}
$ ccm create -v git:trunk
$ ccm populate -n 3
$ ccm start
$ ccm node1 cqlsh 5668_npe_ddl.cql
$ ccm node1 cqlsh 5668_npe_insert.cql
{code}
And see many exceptions like this in the logs of node1:
{code}
ERROR [WRITE-/127.0.0.3] 2013-06-19 14:54:35,885 OutboundTcpConnection.java
(line 197) error writing to /127.0.0.3
java.lang.NullPointerException
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:182)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:144)
{code}
This is similar to CASSANDRA-5658 and is the reason that npe_ddl and
npe_insert are separate files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[5/7] git commit: match dependency to shipped version
match dependency to shipped version Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/362473fb Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/362473fb Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/362473fb Branch: refs/heads/trunk Commit: 362473fb3772eae38d1241aae45dd0a33804fd5c Parents: 79410e4 Author: Jonathan Ellis jbel...@apache.org Authored: Fri Jun 21 13:32:25 2013 -0500 Committer: Jonathan Ellis jbel...@apache.org Committed: Fri Jun 21 13:32:25 2013 -0500 -- build.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/362473fb/build.xml -- diff --git a/build.xml b/build.xml index 4e32351..8d54554 100644 --- a/build.xml +++ b/build.xml @@ -340,7 +340,7 @@ dependency groupId=com.google.guava artifactId=guava version=13.0.1/ dependency groupId=commons-cli artifactId=commons-cli version=1.1/ dependency groupId=commons-codec artifactId=commons-codec version=1.2/ - dependency groupId=commons-lang artifactId=commons-lang version=2.4/ + dependency groupId=commons-lang artifactId=commons-lang version=2.6/ dependency groupId=com.googlecode.concurrentlinkedhashmap artifactId=concurrentlinkedhashmap-lru version=1.3/ dependency groupId=org.antlr artifactId=antlr version=3.2/ dependency groupId=org.slf4j artifactId=slf4j-api version=1.7.2/
[3/7] git commit: Gossiper.handleMajorStateChange can lose existing node ApplicationState patch by jasobrown; reviewe4d by jbellis for CASSANDRA-5665
Gossiper.handleMajorStateChange can lose existing node ApplicationState
patch by jasobrown; reviewe4d by jbellis for CASSANDRA-5665
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b7e13b89
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b7e13b89
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b7e13b89
Branch: refs/heads/trunk
Commit: b7e13b89c265c28acfb624a984b97a06a837c3ea
Parents: 110d283
Author: Jason Brown jasedbr...@gmail.com
Authored: Fri Jun 21 09:23:17 2013 -0700
Committer: Jason Brown jasedbr...@gmail.com
Committed: Fri Jun 21 09:23:52 2013 -0700
--
src/java/org/apache/cassandra/gms/Gossiper.java | 47 +++-
1 file changed, 26 insertions(+), 21 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/b7e13b89/src/java/org/apache/cassandra/gms/Gossiper.java
--
diff --git a/src/java/org/apache/cassandra/gms/Gossiper.java
b/src/java/org/apache/cassandra/gms/Gossiper.java
index efa9865..6b0bbe9 100644
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@ -871,6 +871,7 @@ public class Gossiper implements
IFailureDetectionEventListener, GossiperMBean
if (logger.isTraceEnabled())
logger.trace(Updating heartbeat state generation to
+ remoteGeneration + from + localGeneration + for + ep);
// major state change will handle the update by inserting
the remote state directly
+copyNewerApplicationStates(remoteState, localEpStatePtr);
handleMajorStateChange(ep, remoteState);
}
else if ( remoteGeneration == localGeneration ) // generation
has not changed, apply new states
@@ -880,11 +881,18 @@ public class Gossiper implements
IFailureDetectionEventListener, GossiperMBean
int remoteMaxVersion =
getMaxEndpointStateVersion(remoteState);
if ( remoteMaxVersion localMaxVersion )
{
-// apply states, but do not notify since there is no
major change
-applyNewStates(ep, localEpStatePtr, remoteState);
+if (logger.isTraceEnabled())
+{
+logger.trace(Updating heartbeat state version to
+ remoteState.getHeartBeatState().getHeartBeatVersion() +
+ from +
localEpStatePtr.getHeartBeatState().getHeartBeatVersion() + for + ep);
+}
+
localEpStatePtr.setHeartBeatState(remoteState.getHeartBeatState());
+MapApplicationState, VersionedValue merged =
copyNewerApplicationStates(localEpStatePtr, remoteState);
+for (EntryApplicationState, VersionedValue appState
: merged.entrySet())
+doNotifications(ep, appState.getKey(),
appState.getValue());
}
else if (logger.isTraceEnabled())
-logger.trace(Ignoring remote version +
remoteMaxVersion + = + localMaxVersion + for + ep);
+logger.trace(Ignoring remote version +
remoteMaxVersion + = + localMaxVersion + for + ep);
if (!localEpStatePtr.isAlive()
!isDeadState(localEpStatePtr)) // unless of course, it was dead
markAlive(ep, localEpStatePtr);
}
@@ -903,28 +911,25 @@ public class Gossiper implements
IFailureDetectionEventListener, GossiperMBean
}
}
-private void applyNewStates(InetAddress addr, EndpointState localState,
EndpointState remoteState)
+private MapApplicationState, VersionedValue
copyNewerApplicationStates(EndpointState toState, EndpointState fromState)
{
-// don't assert here, since if the node restarts the version will go
back to zero
-int oldVersion = localState.getHeartBeatState().getHeartBeatVersion();
-
-localState.setHeartBeatState(remoteState.getHeartBeatState());
-if (logger.isTraceEnabled())
-logger.trace(Updating heartbeat state version to +
localState.getHeartBeatState().getHeartBeatVersion() + from + oldVersion +
for + addr + ...);
-
-// we need to make two loops here, one to apply, then another to
notify, this way all states in an update are present and current when the
notifications are received
-for (EntryApplicationState, VersionedValue remoteEntry :
remoteState.getApplicationStateMap().entrySet())
+MapApplicationState, VersionedValue merged = new
HashMapApplicationState, VersionedValue();
+for
[4/7] git commit: Versions and license for 1.2.6 release
Versions and license for 1.2.6 release Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/79410e4a Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/79410e4a Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/79410e4a Branch: refs/heads/trunk Commit: 79410e4a887e7adac796339c1a565436deaf254a Parents: b7e13b8 Author: Sylvain Lebresne sylv...@datastax.com Authored: Fri Jun 21 19:12:31 2013 +0200 Committer: Sylvain Lebresne sylv...@datastax.com Committed: Fri Jun 21 19:12:31 2013 +0200 -- NEWS.txt | 1 - build.xml | 2 +- debian/changelog | 6 ++ .../locator/ReconnectableSnitchHelper.java | 17 + 4 files changed, 24 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/79410e4a/NEWS.txt -- diff --git a/NEWS.txt b/NEWS.txt index dbc9aab..cb40981 100644 --- a/NEWS.txt +++ b/NEWS.txt @@ -21,7 +21,6 @@ Upgrading - hinted_handoff_throttle_in_kb is now reduced by a factor proportional to the number of nodes in the cluster (see https://issues.apache.org/jira/browse/CASSANDRA-5272). - - CQL3 syntax for CREATE CUSTOM INDEX has been updated. See CQL3 documentation for details. http://git-wip-us.apache.org/repos/asf/cassandra/blob/79410e4a/build.xml -- diff --git a/build.xml b/build.xml index 46ffbb1..4e32351 100644 --- a/build.xml +++ b/build.xml @@ -25,7 +25,7 @@ property name=debuglevel value=source,lines,vars/ !-- default version and SCM information -- -property name=base.version value=1.2.5/ +property name=base.version value=1.2.6/ property name=scm.connection value=scm:git://git.apache.org/cassandra.git/ property name=scm.developerConnection value=scm:git://git.apache.org/cassandra.git/ property name=scm.url value=http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree/ http://git-wip-us.apache.org/repos/asf/cassandra/blob/79410e4a/debian/changelog -- diff --git a/debian/changelog b/debian/changelog index 626374f..3be9ba0 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,9 @@ +cassandra (1.2.6) unstable; urgency=low + + * New release + + -- Sylvain Lebresne slebre...@apache.org Fri, 21 Jun 2013 18:58:24 +0200 + cassandra (1.2.5) unstable; urgency=low * New release http://git-wip-us.apache.org/repos/asf/cassandra/blob/79410e4a/src/java/org/apache/cassandra/locator/ReconnectableSnitchHelper.java -- diff --git a/src/java/org/apache/cassandra/locator/ReconnectableSnitchHelper.java b/src/java/org/apache/cassandra/locator/ReconnectableSnitchHelper.java index adec953..bef245e 100644 --- a/src/java/org/apache/cassandra/locator/ReconnectableSnitchHelper.java +++ b/src/java/org/apache/cassandra/locator/ReconnectableSnitchHelper.java @@ -1,3 +1,20 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * License); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an AS IS BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.apache.cassandra.locator; import java.net.InetAddress;
[7/7] git commit: merge from 1.2
merge from 1.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7fcdd808 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7fcdd808 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7fcdd808 Branch: refs/heads/trunk Commit: 7fcdd80807103503af019785b3cd19223dd27a8e Parents: d99a6f2 362473f Author: Jonathan Ellis jbel...@apache.org Authored: Fri Jun 21 13:33:34 2013 -0500 Committer: Jonathan Ellis jbel...@apache.org Committed: Fri Jun 21 13:33:34 2013 -0500 -- NEWS.txt | 1 - debian/changelog | 6 ++ .../locator/ReconnectableSnitchHelper.java| 18 ++ 3 files changed, 24 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/7fcdd808/NEWS.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/7fcdd808/src/java/org/apache/cassandra/locator/ReconnectableSnitchHelper.java -- diff --cc src/java/org/apache/cassandra/locator/ReconnectableSnitchHelper.java index adec953,bef245e..e1353f4 --- a/src/java/org/apache/cassandra/locator/ReconnectableSnitchHelper.java +++ b/src/java/org/apache/cassandra/locator/ReconnectableSnitchHelper.java @@@ -1,3 -1,20 +1,21 @@@ + /* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * License); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an AS IS BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ ++ package org.apache.cassandra.locator; import java.net.InetAddress;
[2/7] git commit: restore fetching global trace state in default .execute method
restore fetching global trace state in default .execute method
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/110d283a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/110d283a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/110d283a
Branch: refs/heads/trunk
Commit: 110d283afd780774a44368b17177b5e8e781e37f
Parents: e75e33f
Author: Jonathan Ellis jbel...@apache.org
Authored: Fri Jun 21 11:13:21 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Fri Jun 21 11:13:21 2013 -0500
--
.../concurrent/DebuggableThreadPoolExecutor.java | 11 ---
1 file changed, 4 insertions(+), 7 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/110d283a/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
--
diff --git
a/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
b/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
index 26441ec..46a3216 100644
--- a/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
+++ b/src/java/org/apache/cassandra/concurrent/DebuggableThreadPoolExecutor.java
@@ -133,7 +133,9 @@ public class DebuggableThreadPoolExecutor extends
ThreadPoolExecutor implements
public void execute(Runnable command, TraceState state)
{
-super.execute(state == null ? command : new
TraceSessionWrapperObject(command, state));
+super.execute(state == null || command instanceof TraceSessionWrapper
+ ? command
+ : new TraceSessionWrapperObject(command, state));
}
// execute does not call newTaskFor
@@ -141,7 +143,7 @@ public class DebuggableThreadPoolExecutor extends
ThreadPoolExecutor implements
public void execute(Runnable command)
{
super.execute(isTracing() !(command instanceof TraceSessionWrapper)
- ? new TraceSessionWrapperObject(command)
+ ? new
TraceSessionWrapperObject(Executors.callable(command, null))
: command);
}
@@ -261,11 +263,6 @@ public class DebuggableThreadPoolExecutor extends
ThreadPoolExecutor implements
{
private final TraceState state;
-public TraceSessionWrapper(Runnable command)
-{
-this(command, null);
-}
-
public TraceSessionWrapper(CallableT callable)
{
super(callable);
[6/7] git commit: match dependency to shipped version
match dependency to shipped version Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/362473fb Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/362473fb Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/362473fb Branch: refs/heads/cassandra-1.2 Commit: 362473fb3772eae38d1241aae45dd0a33804fd5c Parents: 79410e4 Author: Jonathan Ellis jbel...@apache.org Authored: Fri Jun 21 13:32:25 2013 -0500 Committer: Jonathan Ellis jbel...@apache.org Committed: Fri Jun 21 13:32:25 2013 -0500 -- build.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/362473fb/build.xml -- diff --git a/build.xml b/build.xml index 4e32351..8d54554 100644 --- a/build.xml +++ b/build.xml @@ -340,7 +340,7 @@ dependency groupId=com.google.guava artifactId=guava version=13.0.1/ dependency groupId=commons-cli artifactId=commons-cli version=1.1/ dependency groupId=commons-codec artifactId=commons-codec version=1.2/ - dependency groupId=commons-lang artifactId=commons-lang version=2.4/ + dependency groupId=commons-lang artifactId=commons-lang version=2.6/ dependency groupId=com.googlecode.concurrentlinkedhashmap artifactId=concurrentlinkedhashmap-lru version=1.3/ dependency groupId=org.antlr artifactId=antlr version=3.2/ dependency groupId=org.slf4j artifactId=slf4j-api version=1.7.2/
[1/7] git commit: refactor reconnecting snitches patch by jasobrown; reviewed by jbellis for CASSANDRA-5681
Updated Branches:
refs/heads/cassandra-1.2 79410e4a8 - 362473fb3
refs/heads/trunk d99a6f2a2 - 7fcdd8080
refactor reconnecting snitches
patch by jasobrown; reviewed by jbellis for CASSANDRA-5681
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e75e33fa
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e75e33fa
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e75e33fa
Branch: refs/heads/trunk
Commit: e75e33fa6dc5e2a3fe061d747cc98679a65ef960
Parents: 18f3a79
Author: Jonathan Ellis jbel...@apache.org
Authored: Fri Jun 21 10:40:31 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Fri Jun 21 10:40:31 2013 -0500
--
.../cassandra/locator/Ec2MultiRegionSnitch.java | 71 +---
.../locator/GossipingPropertyFileSnitch.java| 63 +-
.../locator/ReconnectableSnitchHelper.java | 88
3 files changed, 94 insertions(+), 128 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e75e33fa/src/java/org/apache/cassandra/locator/Ec2MultiRegionSnitch.java
--
diff --git a/src/java/org/apache/cassandra/locator/Ec2MultiRegionSnitch.java
b/src/java/org/apache/cassandra/locator/Ec2MultiRegionSnitch.java
index 9317941..bd5e091 100644
--- a/src/java/org/apache/cassandra/locator/Ec2MultiRegionSnitch.java
+++ b/src/java/org/apache/cassandra/locator/Ec2MultiRegionSnitch.java
@@ -19,16 +19,11 @@ package org.apache.cassandra.locator;
import java.io.IOException;
import java.net.InetAddress;
-import java.net.UnknownHostException;
import org.apache.cassandra.exceptions.ConfigurationException;
import org.apache.cassandra.config.DatabaseDescriptor;
import org.apache.cassandra.gms.ApplicationState;
-import org.apache.cassandra.gms.EndpointState;
import org.apache.cassandra.gms.Gossiper;
-import org.apache.cassandra.gms.IEndpointStateChangeSubscriber;
-import org.apache.cassandra.gms.VersionedValue;
-import org.apache.cassandra.net.MessagingService;
import org.apache.cassandra.service.StorageService;
/**
@@ -36,16 +31,13 @@ import org.apache.cassandra.service.StorageService;
*
* 2) Snitch will set the private IP as a Gossip application state.
*
- * 3) Snitch implements IESCS and will reset the connection if it is within the
+ * 3) Uses a helper class that implements IESCS and will reset the public IP
connection if it is within the
* same region to communicate via private IP.
*
- * Implements Ec2Snitch to inherit its functionality and extend it for
- * Multi-Region.
- *
* Operational: All the nodes in this cluster needs to be able to (modify the
* Security group settings in AWS) communicate via Public IP's.
*/
-public class Ec2MultiRegionSnitch extends Ec2Snitch implements
IEndpointStateChangeSubscriber
+public class Ec2MultiRegionSnitch extends Ec2Snitch
{
private static final String PUBLIC_IP_QUERY_URL =
http://169.254.169.254/latest/meta-data/public-ipv4;;
private static final String PRIVATE_IP_QUERY_URL =
http://169.254.169.254/latest/meta-data/local-ipv4;;
@@ -62,67 +54,10 @@ public class Ec2MultiRegionSnitch extends Ec2Snitch
implements IEndpointStateCha
DatabaseDescriptor.setBroadcastAddress(localPublicAddress);
}
-public void onJoin(InetAddress endpoint, EndpointState epState)
-{
-if (epState.getApplicationState(ApplicationState.INTERNAL_IP) != null)
-reconnect(endpoint,
epState.getApplicationState(ApplicationState.INTERNAL_IP));
-}
-
-public void onChange(InetAddress endpoint, ApplicationState state,
VersionedValue value)
-{
-if (state == ApplicationState.INTERNAL_IP)
-reconnect(endpoint, value);
-}
-
-public void onAlive(InetAddress endpoint, EndpointState state)
-{
-if (state.getApplicationState(ApplicationState.INTERNAL_IP) != null)
-reconnect(endpoint,
state.getApplicationState(ApplicationState.INTERNAL_IP));
-}
-
-public void onDead(InetAddress endpoint, EndpointState state)
-{
-// do nothing
-}
-
-public void onRestart(InetAddress endpoint, EndpointState state)
-{
-// do nothing
-}
-
-public void onRemove(InetAddress endpoint)
-{
-// do nothing.
-}
-
-private void reconnect(InetAddress publicAddress, VersionedValue
localAddressValue)
-{
-try
-{
-reconnect(publicAddress,
InetAddress.getByName(localAddressValue.value));
-}
-catch (UnknownHostException e)
-{
-logger.error(Error in getting the IP address resolved: , e);
-}
-}
-
-private void reconnect(InetAddress publicAddress, InetAddress localAddress)
-{
-if
git commit: ninja-remove an assert that is not really relevent (and now causing errors due to refactor in CASSANDRA-5665)
Updated Branches:
refs/heads/cassandra-1.2 362473fb3 - 043a63c63
ninja-remove an assert that is not really relevent (and now causing errors
due to refactor in CASSANDRA-5665)
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/043a63c6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/043a63c6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/043a63c6
Branch: refs/heads/cassandra-1.2
Commit: 043a63c63b92b0bf68047da6b5e379da35f3c688
Parents: 362473f
Author: Jason Brown jasedbr...@gmail.com
Authored: Fri Jun 21 11:37:22 2013 -0700
Committer: Jason Brown jasedbr...@gmail.com
Committed: Fri Jun 21 11:39:02 2013 -0700
--
src/java/org/apache/cassandra/gms/Gossiper.java | 1 -
1 file changed, 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/043a63c6/src/java/org/apache/cassandra/gms/Gossiper.java
--
diff --git a/src/java/org/apache/cassandra/gms/Gossiper.java
b/src/java/org/apache/cassandra/gms/Gossiper.java
index 6b0bbe9..b629824 100644
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@ -918,7 +918,6 @@ public class Gossiper implements
IFailureDetectionEventListener, GossiperMBean
{
ApplicationState key = fromEntry.getKey();
VersionedValue value = fromEntry.getValue();
-assert fromState.getHeartBeatState().getGeneration() ==
toState.getHeartBeatState().getGeneration();
if ( (toState.applicationState.containsKey(key)
toState.applicationState.get(key).compareTo(value) 0)
|| !toState.applicationState.containsKey(key) )
[1/2] git commit: ninja-remove an assert that is not really relevent (and now causing errors due to refactor in CASSANDRA-5665)
Updated Branches:
refs/heads/trunk 7fcdd8080 - 2feb81ead
ninja-remove an assert that is not really relevent (and now causing errors
due to refactor in CASSANDRA-5665)
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/043a63c6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/043a63c6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/043a63c6
Branch: refs/heads/trunk
Commit: 043a63c63b92b0bf68047da6b5e379da35f3c688
Parents: 362473f
Author: Jason Brown jasedbr...@gmail.com
Authored: Fri Jun 21 11:37:22 2013 -0700
Committer: Jason Brown jasedbr...@gmail.com
Committed: Fri Jun 21 11:39:02 2013 -0700
--
src/java/org/apache/cassandra/gms/Gossiper.java | 1 -
1 file changed, 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/043a63c6/src/java/org/apache/cassandra/gms/Gossiper.java
--
diff --git a/src/java/org/apache/cassandra/gms/Gossiper.java
b/src/java/org/apache/cassandra/gms/Gossiper.java
index 6b0bbe9..b629824 100644
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@ -918,7 +918,6 @@ public class Gossiper implements
IFailureDetectionEventListener, GossiperMBean
{
ApplicationState key = fromEntry.getKey();
VersionedValue value = fromEntry.getValue();
-assert fromState.getHeartBeatState().getGeneration() ==
toState.getHeartBeatState().getGeneration();
if ( (toState.applicationState.containsKey(key)
toState.applicationState.get(key).compareTo(value) 0)
|| !toState.applicationState.containsKey(key) )
[2/2] git commit: Merge branch 'cassandra-1.2' into trunk
Merge branch 'cassandra-1.2' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2feb81ea Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2feb81ea Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2feb81ea Branch: refs/heads/trunk Commit: 2feb81ead63207c966281bb5e42c875a0302aec4 Parents: 7fcdd80 043a63c Author: Jason Brown jasedbr...@gmail.com Authored: Fri Jun 21 11:40:50 2013 -0700 Committer: Jason Brown jasedbr...@gmail.com Committed: Fri Jun 21 11:40:50 2013 -0700 -- src/java/org/apache/cassandra/gms/Gossiper.java | 1 - 1 file changed, 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/2feb81ea/src/java/org/apache/cassandra/gms/Gossiper.java --
[jira] [Commented] (CASSANDRA-4775) Counters 2.0
[ https://issues.apache.org/jira/browse/CASSANDRA-4775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690590#comment-13690590 ] Theodore Hong commented on CASSANDRA-4775: -- Regarding retries, it seems like CASSANDRA-3199 would have at least partially addressed the retry problem - why was that closed? Counters 2.0 Key: CASSANDRA-4775 URL: https://issues.apache.org/jira/browse/CASSANDRA-4775 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Arya Goudarzi Assignee: Aleksey Yeschenko Labels: counters Fix For: 2.1 The existing partitioned counters remain a source of frustration for most users almost two years after being introduced. The remaining problems are inherent in the design, not something that can be fixed given enough time/eyeballs. Ideally a solution would give us - similar performance - less special cases in the code - potential for a retry mechanism -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5665) Gossiper.handleMajorStateChange can lose existing node ApplicationState
[ https://issues.apache.org/jira/browse/CASSANDRA-5665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690607#comment-13690607 ] Jason Brown commented on CASSANDRA-5665: committed to trunk. Gossiper.handleMajorStateChange can lose existing node ApplicationState --- Key: CASSANDRA-5665 URL: https://issues.apache.org/jira/browse/CASSANDRA-5665 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.2.5 Reporter: Jason Brown Assignee: Jason Brown Priority: Minor Labels: gossip, upgrade Fix For: 1.2.6, 2.0 beta 1 Attachments: 5665-v1.diff, 5665-v2.diff Dovetailing on CASSANDRA-5660, I discovered that further along during an upgrade, when more nodes are on the new major version, a node the previous version can get passed some incomplete Gossip info about another, already upgraded node, and the older node drops AppStat info about that node. I think what happens is that a 1.1 node (older rev) gets gossip info from a 1.2 node (A), which includes incomplete (lacking some AppState data) gossip info about another 1.2 node (B). The 1.1 node, which has marked incorrectly kicked node B out of gossip due to the bug described in #5660, then takes that incomplete node B info and wholesale replaces any previous known state about node B in Gossiper.handleMajorStateChanged. Thus, if we previously had DC/RACK info, it'll get dropped as part of the endpointStateMap.put(endpointstate). When the data being pased is incomplete, 1.1 will start referencing node B and gets into the NPE situation in #5498. Anecdotally, this bad state is short-lived, less than a few minutes, even as short as ten seconds, until gossip catches up and properly propagates the AppState data. Furthermore, when upgrading a two datacenter, 48 node cluster, it only occurred on two nodes for less than a minute each. Thus, the scope seems limited but can occur. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5272) Hinted Handoff Throttle based on cluster size
[ https://issues.apache.org/jira/browse/CASSANDRA-5272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan McGuire updated CASSANDRA-5272: Tester: dmeyer Hinted Handoff Throttle based on cluster size - Key: CASSANDRA-5272 URL: https://issues.apache.org/jira/browse/CASSANDRA-5272 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.2.0 Reporter: Rick Branson Assignee: Jonathan Ellis Priority: Minor Labels: lhf Fix For: 1.2.6 Attachments: 5272.txt For a 12-node EC2 m1.xlarge cluster, restarting a node causes it to get completely overloaded with the default 2-thread, 1024KB setting in 1.2.x. This seemed to be a smaller problem when it was 6-nodes, but still required us to abort handoffs. The old defaults in 1.1.x were WAY more conservative. I've dropped this way down to 128KB on our production cluster which is really conservative, but appears to have solved it. The default seems way too high on any cluster that is non-trivial in size. After putting some thought to this, it seems that this should really be based on cluster size, making the throttle a target for how much write load a single node can swallow. As the cluster grows, the amount of hints that can be delivered by each other node in the cluster goes down, so the throttle should self-adjust to take that into account. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
git commit: remove unused dependencies (aws-s3, opt-simple, whirr-cli, whirr-core)
Updated Branches:
refs/heads/trunk 2feb81ead - c66ee7884
remove unused dependencies (aws-s3, opt-simple, whirr-cli, whirr-core)
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c66ee788
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c66ee788
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c66ee788
Branch: refs/heads/trunk
Commit: c66ee7884f1f07c22453365265b206b59582d842
Parents: 2feb81e
Author: Dave Brosius dbros...@apache.org
Authored: Fri Jun 21 18:00:26 2013 -0400
Committer: Dave Brosius dbros...@apache.org
Committed: Fri Jun 21 18:00:26 2013 -0400
--
build.xml | 10 --
1 file changed, 10 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c66ee788/build.xml
--
diff --git a/build.xml b/build.xml
index f5a21be..05ed9e7 100644
--- a/build.xml
+++ b/build.xml
@@ -364,15 +364,10 @@
/dependency
dependency groupId=org.apache.hadoop artifactId=hadoop-core
version=1.0.3/
dependency groupId=org.apache.pig artifactId=pig
version=0.10.0/
- dependency groupId=net.sf.jopt-simple artifactId=jopt-simple
version=3.2/
dependency groupId=net.java.dev.jna artifactId=jna
version=3.2.7/
dependency groupId=net.sourceforge.cobertura
artifactId=cobertura version=${cobertura.version}/
- dependency groupId=org.apache.whirr artifactId=whirr-core
version=0.4.0-incubating/
- dependency groupId=org.apache.whirr artifactId=whirr-cli
version=0.4.0-incubating/
- dependency groupId=org.jclouds.provider artifactId=aws-s3
version=1.0-beta-9b /
-
dependency groupId=log4j artifactId=log4j version=1.2.16 /
dependency groupId=org.apache.cassandra
artifactId=cassandra-all version=${version} /
dependency groupId=org.apache.cassandra
artifactId=cassandra-thrift version=${version} /
@@ -408,7 +403,6 @@
dependency groupId=org.apache.rat artifactId=apache-rat/
dependency groupId=org.apache.hadoop artifactId=hadoop-core/
dependency groupId=org.apache.pig artifactId=pig/
-dependency groupId=net.sf.jopt-simple artifactId=jopt-simple/
dependency groupId=net.java.dev.jna artifactId=jna/
/artifact:pom
@@ -426,9 +420,6 @@
parent groupId=org.apache.cassandra
artifactId=cassandra-parent
version=${version}/
-dependency groupId=org.apache.whirr artifactId=whirr-core/
-dependency groupId=org.apache.whirr artifactId=whirr-cli/
-dependency groupId=org.jclouds.provider artifactId=aws-s3/
/artifact:pom
!-- now the pom's for artifacts being deployed to Maven Central --
@@ -548,7 +539,6 @@
dependencyRefsBuildFile=${build.dir}/test-dependencies.xml
remoteRepository refid=apache/
remoteRepository refid=central/
-remoteRepository refid=jclouds/
remoteRepository refid=oauth/
/artifact:dependencies
copy todir=${test.lib}/jars
[jira] [Created] (CASSANDRA-5687) Move C* Python Thrift tests into dtests
Ryan McGuire created CASSANDRA-5687: --- Summary: Move C* Python Thrift tests into dtests Key: CASSANDRA-5687 URL: https://issues.apache.org/jira/browse/CASSANDRA-5687 Project: Cassandra Issue Type: Test Reporter: Ryan McGuire Assignee: Daniel Meyer Priority: Minor There's several good tests currently sitting in the C* source tree under test/system/test_thrift_server.py - these tests are not run via 'ant test' and no buildbot is regularly running them. Let's move them into cassandra-dtest so that we're not wasting valid tests. It appears they will need some refactoring to automatically start clusters via ccm, like the rest of dtests do. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4415) Add cursor API/auto paging to the native CQL protocol
[
https://issues.apache.org/jira/browse/CASSANDRA-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690854#comment-13690854
]
Aleksey Yeschenko commented on CASSANDRA-4415:
--
Mostly LGTM. So far a couple issues and some nits:
- native proto spec: section 7 mentions BATCH supporting result_page_size
- CassandraServer.get_count() calculates pageSize, but never actually passes it
to QueryPagers.pageQuery() - always passes COUNT_PAGE_SIZE instead
- QueryPagers.pageQuery() in computeNext() doesn't handle
pager.fetchPage(pageSize) returning an empty collection and throws NPE in the
next line
I think some other iterator code has similar issues, and count is slightly
broken (I had AQP.firstName() throw NPE, too)
These are the two tests that are broken (test/system/test_thrift_server.py):
1. TestMutations.test_count
2. TestMutations.test_count_around_page_size
And some nits:
- native proto spec: 4.1.4 QUERY 'The body message must be:' summary-line does
not include result_page_size
- AbstractQueryPager.discardLast() creates an iterator that's never used
(IteratorColumn iter = cf.iterator();)
- there are redundantly specialized Collections.InferrableemptyList() calls
(in AbstractQueryPager.fetchPage(), others)
Add cursor API/auto paging to the native CQL protocol
-
Key: CASSANDRA-4415
URL: https://issues.apache.org/jira/browse/CASSANDRA-4415
Project: Cassandra
Issue Type: New Feature
Affects Versions: 1.2.0 beta 1
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Labels: cql, protocol
Fix For: 2.0 beta 1
The goal here would be to use a query paging mechanism to the CQL native
protocol. Typically the client/server with that would look something like
this:
{noformat}
C sends query to S.
S sends N first rows matching the query + flag saying the response is not
complete
C requests the next N rows
S sends N next rows + flag saying whether there is more
C requests the next N rows
...
S sends last rows + flag saying there is no more result
{noformat}
The clear goal is for user to not have to worry about limiting queries and
doing manual paging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
git commit: fetch the batch mutation just once
Updated Branches:
refs/heads/trunk c66ee7884 - c06e0b577
fetch the batch mutation just once
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c06e0b57
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c06e0b57
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c06e0b57
Branch: refs/heads/trunk
Commit: c06e0b57784690af9a30e639b09d7169d5fa1588
Parents: c66ee78
Author: Dave Brosius dbros...@apache.org
Authored: Fri Jun 21 19:36:06 2013 -0400
Committer: Dave Brosius dbros...@apache.org
Committed: Fri Jun 21 19:36:06 2013 -0400
--
src/java/org/apache/cassandra/db/BatchlogManager.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c06e0b57/src/java/org/apache/cassandra/db/BatchlogManager.java
--
diff --git a/src/java/org/apache/cassandra/db/BatchlogManager.java
b/src/java/org/apache/cassandra/db/BatchlogManager.java
index 9d46739..2e1b1a7 100644
--- a/src/java/org/apache/cassandra/db/BatchlogManager.java
+++ b/src/java/org/apache/cassandra/db/BatchlogManager.java
@@ -179,7 +179,8 @@ public class BatchlogManager implements BatchlogManagerMBean
try
{
-replaySerializedMutations(result.one().getBytes(data),
result.one().getLong(written_at));
+UntypedResultSet.Row batch = result.one();
+replaySerializedMutations(batch.getBytes(data),
batch.getLong(written_at));
}
catch (IOException e)
{
git commit: passing an empty string to StringBuilder ctor is silly
Updated Branches:
refs/heads/trunk c06e0b577 - 5c5fdd40e
passing an empty string to StringBuilder ctor is silly
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5c5fdd40
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5c5fdd40
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5c5fdd40
Branch: refs/heads/trunk
Commit: 5c5fdd40ebac875ee0b4f9de1a7d1eea3b3788b9
Parents: c06e0b5
Author: Dave Brosius dbros...@apache.org
Authored: Fri Jun 21 19:42:49 2013 -0400
Committer: Dave Brosius dbros...@apache.org
Committed: Fri Jun 21 19:42:49 2013 -0400
--
src/java/org/apache/cassandra/net/MessageIn.java | 2 +-
src/java/org/apache/cassandra/net/MessageOut.java | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5c5fdd40/src/java/org/apache/cassandra/net/MessageIn.java
--
diff --git a/src/java/org/apache/cassandra/net/MessageIn.java
b/src/java/org/apache/cassandra/net/MessageIn.java
index b9d94b5..e0efefe 100644
--- a/src/java/org/apache/cassandra/net/MessageIn.java
+++ b/src/java/org/apache/cassandra/net/MessageIn.java
@@ -112,7 +112,7 @@ public class MessageInT
public String toString()
{
-StringBuilder sbuf = new StringBuilder();
+StringBuilder sbuf = new StringBuilder();
sbuf.append(FROM:).append(from).append(
TYPE:).append(getMessageType()).append( VERB:).append(verb);
return sbuf.toString();
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5c5fdd40/src/java/org/apache/cassandra/net/MessageOut.java
--
diff --git a/src/java/org/apache/cassandra/net/MessageOut.java
b/src/java/org/apache/cassandra/net/MessageOut.java
index da9eda4..e10fcee 100644
--- a/src/java/org/apache/cassandra/net/MessageOut.java
+++ b/src/java/org/apache/cassandra/net/MessageOut.java
@@ -95,7 +95,7 @@ public class MessageOutT
public String toString()
{
-StringBuilder sbuf = new StringBuilder();
+StringBuilder sbuf = new StringBuilder();
sbuf.append(TYPE:).append(getStage()).append( VERB:).append(verb);
return sbuf.toString();
}
git commit: remove unused field
Updated Branches: refs/heads/trunk 5c5fdd40e - e8970976c remove unused field Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e8970976 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e8970976 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e8970976 Branch: refs/heads/trunk Commit: e8970976ce91f29faa7b7e24e1ae11b2d15fa475 Parents: 5c5fdd4 Author: Dave Brosius dbros...@apache.org Authored: Fri Jun 21 19:50:46 2013 -0400 Committer: Dave Brosius dbros...@apache.org Committed: Fri Jun 21 19:50:46 2013 -0400 -- src/java/org/apache/cassandra/db/compaction/Upgrader.java | 2 -- 1 file changed, 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/e8970976/src/java/org/apache/cassandra/db/compaction/Upgrader.java -- diff --git a/src/java/org/apache/cassandra/db/compaction/Upgrader.java b/src/java/org/apache/cassandra/db/compaction/Upgrader.java index 17b68ec..56de6e3 100644 --- a/src/java/org/apache/cassandra/db/compaction/Upgrader.java +++ b/src/java/org/apache/cassandra/db/compaction/Upgrader.java @@ -53,8 +53,6 @@ public class Upgrader private final AbstractCompactionStrategy strategy; private final long estimatedRows; -private final int gcBefore = CompactionManager.NO_GC; - private final OutputHandler outputHandler; public Upgrader(ColumnFamilyStore cfs, SSTableReader sstable, OutputHandler outputHandler)
[jira] [Updated] (CASSANDRA-5524) Allow upgradesstables to be run against a specified directory
[ https://issues.apache.org/jira/browse/CASSANDRA-5524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cathy Daw updated CASSANDRA-5524: - Tester: alexzar (was: enigmacurry) Allow upgradesstables to be run against a specified directory - Key: CASSANDRA-5524 URL: https://issues.apache.org/jira/browse/CASSANDRA-5524 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Tyler Hobbs Assignee: Nick Bailey Priority: Minor Fix For: 1.2.6 Attachments: 0001-Add-a-snapshot-upgrade-tool.patch, 0002-Rename-snapshotupgrade-to-sstableupgrade.patch, 0003-Update-NEWS.txt-and-debian-scripts.patch Currently, upgradesstables only modifies live SSTables. Because sstableloader cannot stream old SSTable formats, this makes it difficult to restore data from a snapshot taken in a previous major version of Cassandra. Allowing the user to specify a directory for upgradesstables would resolve this, but it may also be nice to upgrade SSTables in snapshot directories automatically or with a separate flag. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5684) Multi-DC not working between 1.1.10 and 1.2.4 version
[ https://issues.apache.org/jira/browse/CASSANDRA-5684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690913#comment-13690913 ] Ananth Gundabattula commented on CASSANDRA-5684: Here is our situation which prompted us to take that path: (The path of setting up a new cluster and migrate from an old cluster rather than performing a rolling upgrade) We have two small clusters one running 1.1.4 and one running 1.1.10. We are having loads of issues with the 1.1.10 cluster (primarily could be because of the load patterns it experiences and our experiments have shown that the 1.1.10 also degraded in performance over time possibly because of hardware as well). This has led us to the decision to move to new hardware. Our new hardware being LCS capable ( we were running with hard disks on the older clusters ) and at the same time we cant afford a gradual rip off Size tiered compaction to LCS based SSTables because it will impact our latencies to a very great extent. Unfortunately we have no backup clusters right now. Our other tests also made us think about 1.2.4 because of the heap data structures that 1.2.4 provides us with. GCs have really became a headache for us and our application requires DB calls to be returned below 15ms time range (as shown in opscenter). In our tests we saw about 125 % gain in performance as compared to the hard disk based 1.1.x versions with the same number of nodes. In this context, we want to merge both the old clusters into a single new hardware based one and running 1.2.4 with the least amount of downtimes and application layer impacts. Multi-DC not working between 1.1.10 and 1.2.4 version - Key: CASSANDRA-5684 URL: https://issues.apache.org/jira/browse/CASSANDRA-5684 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.2.4 Environment: Linux Reporter: Ananth Gundabattula We wanted to upgrade to a higher version of cassandra ( a custom build of 1.2.4 ) from current cassandra version 1.1.10. The way we wanted to upgrade is to build a new cluster and enable multi-dc replication; rebuild and switch the application layer to the new cluster. This enabled us to have the least downtime. However, I was not able to get a multi-dc going between these two versions. I followed pretty much the instructions here ( for the 1.2.4 version ) http://www.datastax.com/documentation/cassandra/1.2/index.html#cassandra/operations/ops_add_dc_to_cluster_t.html. I tried enabling multi-dc using these instructions for the 1.1.10 cluster http://www.datastax.com/docs/1.0/initialize/cluster_init_multi_dc Mixing both the approaches did not work either. Specifying a 1.2.4 node as one of the 1.1.10 cluster seed nodes caused errors in the startup of 1.2.4 node. The error was not an exception but a message looking like this in the logs : ClusterName mismatch from /10.xxx.xx.xx DC1!=DC2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-5684) Multi-DC not working between 1.1.10 and 1.2.4 version
[ https://issues.apache.org/jira/browse/CASSANDRA-5684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ananth Gundabattula resolved CASSANDRA-5684. Resolution: Invalid Multi-DC not working between 1.1.10 and 1.2.4 version - Key: CASSANDRA-5684 URL: https://issues.apache.org/jira/browse/CASSANDRA-5684 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.2.4 Environment: Linux Reporter: Ananth Gundabattula We wanted to upgrade to a higher version of cassandra ( a custom build of 1.2.4 ) from current cassandra version 1.1.10. The way we wanted to upgrade is to build a new cluster and enable multi-dc replication; rebuild and switch the application layer to the new cluster. This enabled us to have the least downtime. However, I was not able to get a multi-dc going between these two versions. I followed pretty much the instructions here ( for the 1.2.4 version ) http://www.datastax.com/documentation/cassandra/1.2/index.html#cassandra/operations/ops_add_dc_to_cluster_t.html. I tried enabling multi-dc using these instructions for the 1.1.10 cluster http://www.datastax.com/docs/1.0/initialize/cluster_init_multi_dc Mixing both the approaches did not work either. Specifying a 1.2.4 node as one of the 1.1.10 cluster seed nodes caused errors in the startup of 1.2.4 node. The error was not an exception but a message looking like this in the logs : ClusterName mismatch from /10.xxx.xx.xx DC1!=DC2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5684) Multi-DC not working between 1.1.10 and 1.2.4 version
[ https://issues.apache.org/jira/browse/CASSANDRA-5684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13691025#comment-13691025 ] Ananth Gundabattula commented on CASSANDRA-5684: Looks like I got my definition of cluster and datacenter mixed up. All the while I was thinking that a datacenter can have multiple clusters hosted while it looks like the opposite is the actual case and hence the confusion. Closing this ticket Multi-DC not working between 1.1.10 and 1.2.4 version - Key: CASSANDRA-5684 URL: https://issues.apache.org/jira/browse/CASSANDRA-5684 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.2.4 Environment: Linux Reporter: Ananth Gundabattula We wanted to upgrade to a higher version of cassandra ( a custom build of 1.2.4 ) from current cassandra version 1.1.10. The way we wanted to upgrade is to build a new cluster and enable multi-dc replication; rebuild and switch the application layer to the new cluster. This enabled us to have the least downtime. However, I was not able to get a multi-dc going between these two versions. I followed pretty much the instructions here ( for the 1.2.4 version ) http://www.datastax.com/documentation/cassandra/1.2/index.html#cassandra/operations/ops_add_dc_to_cluster_t.html. I tried enabling multi-dc using these instructions for the 1.1.10 cluster http://www.datastax.com/docs/1.0/initialize/cluster_init_multi_dc Mixing both the approaches did not work either. Specifying a 1.2.4 node as one of the 1.1.10 cluster seed nodes caused errors in the startup of 1.2.4 node. The error was not an exception but a message looking like this in the logs : ClusterName mismatch from /10.xxx.xx.xx DC1!=DC2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5619) CAS UPDATE for a lost race: save round trip by returning column values
[ https://issues.apache.org/jira/browse/CASSANDRA-5619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13691038#comment-13691038 ] Blair Zajac commented on CASSANDRA-5619: Sweet, thanks guys! CAS UPDATE for a lost race: save round trip by returning column values -- Key: CASSANDRA-5619 URL: https://issues.apache.org/jira/browse/CASSANDRA-5619 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 2.0 beta 1 Reporter: Blair Zajac Assignee: Sylvain Lebresne Fix For: 2.0 beta 1 Attachments: 5619.txt Looking at the new CAS CQL3 support examples [1], if one lost a race for an UPDATE, to save a round trip to get the current values to decide if you need to perform your work, could the columns that were used in the IF clause also be returned to the caller? Maybe the columns values as part of the SET part could also be returned. I don't know if this is generally useful though. In the case of creating a new user account with a given username which is the partition key, if one lost the race to another person creating an account with the same username, it doesn't matter to the loser what the column values are, just that they lost. I'm new to Cassandra, so maybe there's other use cases, such as doing incremental amount of work on a row. In pure Java projects I've done while loops around AtomicReference.html#compareAndSet() until the work was done on the referenced object to handle multiple threads each making forward progress in updating the references object. [1] https://github.com/riptano/cassandra-dtest/blob/master/cql_tests.py#L3044 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-5688) Update debian packaging for 2.0
Blair Zajac created CASSANDRA-5688:
--
Summary: Update debian packaging for 2.0
Key: CASSANDRA-5688
URL: https://issues.apache.org/jira/browse/CASSANDRA-5688
Project: Cassandra
Issue Type: Bug
Components: Packaging
Affects Versions: 2.0 beta 1
Environment: Ubuntu precise
Reporter: Blair Zajac
Building trunk on an Ubuntu Precise VM fails with the following output:
{code}
$ git describe
cassandra-1.2.5-983-g96a1bb0
$ dpkg-buildpackage
...
...
gen-cql3-grammar:
[echo] Building Grammar
/home/blair/Code/Cassandra/cassandra-0.2.0.0.1.2.5.982/src/java/org/apache/cassandra/cql3/Cql.g
...
build-project:
[echo] apache-cassandra:
/home/blair/Code/Cassandra/cassandra-0.2.0.0.1.2.5.982/build.xml
[javac] Compiling 41 source files to
/home/blair/Code/Cassandra/cassandra-0.2.0.0.1.2.5.982/build/classes/thrift
[javac] javac: invalid target release: 1.7
[javac] Usage: javac options source files
[javac] use -help for a list of possible options
BUILD FAILED
{code}
I'm working on changes to the files in debian/ to support this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
