[jira] [Commented] (CASSANDRA-6238) LOCAL_ONE doesn't work for SimpleStrategy
[ https://issues.apache.org/jira/browse/CASSANDRA-6238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815693#comment-13815693 ] Jason Brown commented on CASSANDRA-6238: committed to 1.2, 2.0, and trunk. > LOCAL_ONE doesn't work for SimpleStrategy > - > > Key: CASSANDRA-6238 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6238 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Alex Liu >Assignee: Alex Liu > Fix For: 1.2.12, 2.0.3 > > Attachments: 6238-v2.txt, 6238.txt > > > LOCAL_ONE only works for NetworkTopologyStrategy which has DC specification. > Any other strategy fails. > If there is no DC specified in the strategy, we should treat LOCAL_ONE as ONE -- This message was sent by Atlassian JIRA (v6.1#6144)
[3/3] git commit: Merge branch 'cassandra-2.0' into trunk
Merge branch 'cassandra-2.0' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c2bd0d8d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c2bd0d8d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c2bd0d8d Branch: refs/heads/trunk Commit: c2bd0d8d171d52dbc8f5520e9c5a2e6a24fbd1de Parents: c65c15c 0e9906e Author: Jason Brown Authored: Wed Nov 6 22:09:24 2013 -0800 Committer: Jason Brown Committed: Wed Nov 6 22:09:24 2013 -0800 -- CHANGES.txt| 1 + src/java/org/apache/cassandra/db/ConsistencyLevel.java | 6 -- 2 files changed, 1 insertion(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/c2bd0d8d/CHANGES.txt --
[2/3] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0
Merge branch 'cassandra-1.2' into cassandra-2.0
Conflicts:
src/java/org/apache/cassandra/db/ConsistencyLevel.java
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0e9906e6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0e9906e6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0e9906e6
Branch: refs/heads/trunk
Commit: 0e9906e6d068f6236996093220563dc8cee54ed6
Parents: b402097 e5715f4
Author: Jason Brown
Authored: Wed Nov 6 22:08:12 2013 -0800
Committer: Jason Brown
Committed: Wed Nov 6 22:08:12 2013 -0800
--
CHANGES.txt| 1 +
src/java/org/apache/cassandra/db/ConsistencyLevel.java | 6 --
2 files changed, 1 insertion(+), 6 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0e9906e6/CHANGES.txt
--
diff --cc CHANGES.txt
index 4a1fd05,d27c495..77749d6
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -30,42 -12,10 +30,43 @@@ Merged from 1.2
* make calculatePendingRanges asynchronous (CASSANDRA-6244)
* Remove blocking flushes in gossip thread (CASSANDRA-6297)
* Fix potential socket leak in connectionpool creation (CASSANDRA-6308)
+ * Allow LOCAL_ONE/LOCAL_QUORUM to work with SimpleSrrategy (CASSANDRA-6238)
-1.2.11
+2.0.2
+ * Update FailureDetector to use nanontime (CASSANDRA-4925)
+ * Fix FileCacheService regressions (CASSANDRA-6149)
+ * Never return WriteTimeout for CL.ANY (CASSANDRA-6032)
+ * Fix race conditions in bulk loader (CASSANDRA-6129)
+ * Add configurable metrics reporting (CASSANDRA-4430)
+ * drop queries exceeding a configurable number of tombstones (CASSANDRA-6117)
+ * Track and persist sstable read activity (CASSANDRA-5515)
+ * Fixes for speculative retry (CASSANDRA-5932, CASSANDRA-6194)
+ * Improve memory usage of metadata min/max column names (CASSANDRA-6077)
+ * Fix thrift validation refusing row markers on CQL3 tables (CASSANDRA-6081)
+ * Fix insertion of collections with CAS (CASSANDRA-6069)
+ * Correctly send metadata on SELECT COUNT (CASSANDRA-6080)
+ * Track clients' remote addresses in ClientState (CASSANDRA-6070)
+ * Create snapshot dir if it does not exist when migrating
+ leveled manifest (CASSANDRA-6093)
+ * make sequential nodetool repair the default (CASSANDRA-5950)
+ * Add more hooks for compaction strategy implementations (CASSANDRA-6111)
+ * Fix potential NPE on composite 2ndary indexes (CASSANDRA-6098)
+ * Delete can potentially be skipped in batch (CASSANDRA-6115)
+ * Allow alter keyspace on system_traces (CASSANDRA-6016)
+ * Disallow empty column names in cql (CASSANDRA-6136)
+ * Use Java7 file-handling APIs and fix file moving on Windows
(CASSANDRA-5383)
+ * Save compaction history to system keyspace (CASSANDRA-5078)
+ * Fix NPE if StorageService.getOperationMode() is executed before full
startup (CASSANDRA-6166)
+ * CQL3: support pre-epoch longs for TimestampType (CASSANDRA-6212)
+ * Add reloadtriggers command to nodetool (CASSANDRA-4949)
+ * cqlsh: ignore empty 'value alias' in DESCRIBE (CASSANDRA-6139)
+ * Fix sstable loader (CASSANDRA-6205)
+ * Reject bootstrapping if the node already exists in gossip (CASSANDRA-5571)
+ * Fix NPE while loading paxos state (CASSANDRA-6211)
+ * cqlsh: add SHOW SESSION command (CASSANDRA-6228)
+Merged from 1.2:
+ * (Hadoop) Require CFRR batchSize to be at least 2 (CASSANDRA-6114)
* Add a warning for small LCS sstable size (CASSANDRA-6191)
* Add ability to list specific KS/CF combinations in nodetool cfstats
(CASSANDRA-4191)
* Mark CF clean if a mutation raced the drop and got it marked dirty
(CASSANDRA-5946)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0e9906e6/src/java/org/apache/cassandra/db/ConsistencyLevel.java
--
diff --cc src/java/org/apache/cassandra/db/ConsistencyLevel.java
index 5f17907,25fb25b..4fffc8a
--- a/src/java/org/apache/cassandra/db/ConsistencyLevel.java
+++ b/src/java/org/apache/cassandra/db/ConsistencyLevel.java
@@@ -286,46 -280,10 +282,44 @@@ public enum ConsistencyLeve
}
}
-public void validateForWrite(String table) throws InvalidRequestException
+public void validateForWrite(String keyspaceName) throws
InvalidRequestException
{
-if(this == EACH_QUORUM)
-requireNetworkTopologyStrategy(table);
+switch (this)
+{
+case LOCAL_QUORUM:
+case EACH_QUORUM:
+case LOCAL_ONE:
+requireNetworkTopologyStrategy(keyspaceName);
+break;
+case SERIAL:
+case LOCAL_SERIAL:
+throw new InvalidRequestException("You must use conditional
upd
[1/3] git commit: Allow LOCAL_ONE/LOCAL_QUORUM to work with SimpleStrategy patch by Alex Liu; reviewed by jasobrown for CASSANDRA-6238
Updated Branches:
refs/heads/trunk c65c15cc0 -> c2bd0d8d1
Allow LOCAL_ONE/LOCAL_QUORUM to work with SimpleStrategy
patch by Alex Liu; reviewed by jasobrown for CASSANDRA-6238
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e5715f4e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e5715f4e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e5715f4e
Branch: refs/heads/trunk
Commit: e5715f4e9618e933fca4d4537ff84fd4ee24a641
Parents: 8c24044
Author: Jason Brown
Authored: Wed Nov 6 22:02:49 2013 -0800
Committer: Jason Brown
Committed: Wed Nov 6 22:02:49 2013 -0800
--
CHANGES.txt | 1 +
.../org/apache/cassandra/db/ConsistencyLevel.java | 14 ++
2 files changed, 3 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e5715f4e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index eab185a..d27c495 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -12,6 +12,7 @@
* make calculatePendingRanges asynchronous (CASSANDRA-6244)
* Remove blocking flushes in gossip thread (CASSANDRA-6297)
* Fix potential socket leak in connectionpool creation (CASSANDRA-6308)
+ * Allow LOCAL_ONE/LOCAL_QUORUM to work with SimpleSrrategy (CASSANDRA-6238)
1.2.11
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e5715f4e/src/java/org/apache/cassandra/db/ConsistencyLevel.java
--
diff --git a/src/java/org/apache/cassandra/db/ConsistencyLevel.java
b/src/java/org/apache/cassandra/db/ConsistencyLevel.java
index 1ce019b..25fb25b 100644
--- a/src/java/org/apache/cassandra/db/ConsistencyLevel.java
+++ b/src/java/org/apache/cassandra/db/ConsistencyLevel.java
@@ -275,10 +275,6 @@ public enum ConsistencyLevel
{
case ANY:
throw new InvalidRequestException("ANY ConsistencyLevel is
only supported for writes");
-case LOCAL_QUORUM:
-case LOCAL_ONE:
-requireNetworkTopologyStrategy(table);
-break;
case EACH_QUORUM:
throw new InvalidRequestException("EACH_QUORUM
ConsistencyLevel is only supported for writes");
}
@@ -286,14 +282,8 @@ public enum ConsistencyLevel
public void validateForWrite(String table) throws InvalidRequestException
{
-switch (this)
-{
-case LOCAL_QUORUM:
-case EACH_QUORUM:
-case LOCAL_ONE:
-requireNetworkTopologyStrategy(table);
-break;
-}
+if(this == EACH_QUORUM)
+requireNetworkTopologyStrategy(table);
}
public void validateCounterForWrite(CFMetaData metadata) throws
InvalidRequestException
[1/2] git commit: Allow LOCAL_ONE/LOCAL_QUORUM to work with SimpleStrategy patch by Alex Liu; reviewed by jasobrown for CASSANDRA-6238
Updated Branches:
refs/heads/cassandra-2.0 b4020979c -> 0e9906e6d
Allow LOCAL_ONE/LOCAL_QUORUM to work with SimpleStrategy
patch by Alex Liu; reviewed by jasobrown for CASSANDRA-6238
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e5715f4e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e5715f4e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e5715f4e
Branch: refs/heads/cassandra-2.0
Commit: e5715f4e9618e933fca4d4537ff84fd4ee24a641
Parents: 8c24044
Author: Jason Brown
Authored: Wed Nov 6 22:02:49 2013 -0800
Committer: Jason Brown
Committed: Wed Nov 6 22:02:49 2013 -0800
--
CHANGES.txt | 1 +
.../org/apache/cassandra/db/ConsistencyLevel.java | 14 ++
2 files changed, 3 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e5715f4e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index eab185a..d27c495 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -12,6 +12,7 @@
* make calculatePendingRanges asynchronous (CASSANDRA-6244)
* Remove blocking flushes in gossip thread (CASSANDRA-6297)
* Fix potential socket leak in connectionpool creation (CASSANDRA-6308)
+ * Allow LOCAL_ONE/LOCAL_QUORUM to work with SimpleSrrategy (CASSANDRA-6238)
1.2.11
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e5715f4e/src/java/org/apache/cassandra/db/ConsistencyLevel.java
--
diff --git a/src/java/org/apache/cassandra/db/ConsistencyLevel.java
b/src/java/org/apache/cassandra/db/ConsistencyLevel.java
index 1ce019b..25fb25b 100644
--- a/src/java/org/apache/cassandra/db/ConsistencyLevel.java
+++ b/src/java/org/apache/cassandra/db/ConsistencyLevel.java
@@ -275,10 +275,6 @@ public enum ConsistencyLevel
{
case ANY:
throw new InvalidRequestException("ANY ConsistencyLevel is
only supported for writes");
-case LOCAL_QUORUM:
-case LOCAL_ONE:
-requireNetworkTopologyStrategy(table);
-break;
case EACH_QUORUM:
throw new InvalidRequestException("EACH_QUORUM
ConsistencyLevel is only supported for writes");
}
@@ -286,14 +282,8 @@ public enum ConsistencyLevel
public void validateForWrite(String table) throws InvalidRequestException
{
-switch (this)
-{
-case LOCAL_QUORUM:
-case EACH_QUORUM:
-case LOCAL_ONE:
-requireNetworkTopologyStrategy(table);
-break;
-}
+if(this == EACH_QUORUM)
+requireNetworkTopologyStrategy(table);
}
public void validateCounterForWrite(CFMetaData metadata) throws
InvalidRequestException
[2/2] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0
Merge branch 'cassandra-1.2' into cassandra-2.0
Conflicts:
src/java/org/apache/cassandra/db/ConsistencyLevel.java
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0e9906e6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0e9906e6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0e9906e6
Branch: refs/heads/cassandra-2.0
Commit: 0e9906e6d068f6236996093220563dc8cee54ed6
Parents: b402097 e5715f4
Author: Jason Brown
Authored: Wed Nov 6 22:08:12 2013 -0800
Committer: Jason Brown
Committed: Wed Nov 6 22:08:12 2013 -0800
--
CHANGES.txt| 1 +
src/java/org/apache/cassandra/db/ConsistencyLevel.java | 6 --
2 files changed, 1 insertion(+), 6 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0e9906e6/CHANGES.txt
--
diff --cc CHANGES.txt
index 4a1fd05,d27c495..77749d6
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -30,42 -12,10 +30,43 @@@ Merged from 1.2
* make calculatePendingRanges asynchronous (CASSANDRA-6244)
* Remove blocking flushes in gossip thread (CASSANDRA-6297)
* Fix potential socket leak in connectionpool creation (CASSANDRA-6308)
+ * Allow LOCAL_ONE/LOCAL_QUORUM to work with SimpleSrrategy (CASSANDRA-6238)
-1.2.11
+2.0.2
+ * Update FailureDetector to use nanontime (CASSANDRA-4925)
+ * Fix FileCacheService regressions (CASSANDRA-6149)
+ * Never return WriteTimeout for CL.ANY (CASSANDRA-6032)
+ * Fix race conditions in bulk loader (CASSANDRA-6129)
+ * Add configurable metrics reporting (CASSANDRA-4430)
+ * drop queries exceeding a configurable number of tombstones (CASSANDRA-6117)
+ * Track and persist sstable read activity (CASSANDRA-5515)
+ * Fixes for speculative retry (CASSANDRA-5932, CASSANDRA-6194)
+ * Improve memory usage of metadata min/max column names (CASSANDRA-6077)
+ * Fix thrift validation refusing row markers on CQL3 tables (CASSANDRA-6081)
+ * Fix insertion of collections with CAS (CASSANDRA-6069)
+ * Correctly send metadata on SELECT COUNT (CASSANDRA-6080)
+ * Track clients' remote addresses in ClientState (CASSANDRA-6070)
+ * Create snapshot dir if it does not exist when migrating
+ leveled manifest (CASSANDRA-6093)
+ * make sequential nodetool repair the default (CASSANDRA-5950)
+ * Add more hooks for compaction strategy implementations (CASSANDRA-6111)
+ * Fix potential NPE on composite 2ndary indexes (CASSANDRA-6098)
+ * Delete can potentially be skipped in batch (CASSANDRA-6115)
+ * Allow alter keyspace on system_traces (CASSANDRA-6016)
+ * Disallow empty column names in cql (CASSANDRA-6136)
+ * Use Java7 file-handling APIs and fix file moving on Windows
(CASSANDRA-5383)
+ * Save compaction history to system keyspace (CASSANDRA-5078)
+ * Fix NPE if StorageService.getOperationMode() is executed before full
startup (CASSANDRA-6166)
+ * CQL3: support pre-epoch longs for TimestampType (CASSANDRA-6212)
+ * Add reloadtriggers command to nodetool (CASSANDRA-4949)
+ * cqlsh: ignore empty 'value alias' in DESCRIBE (CASSANDRA-6139)
+ * Fix sstable loader (CASSANDRA-6205)
+ * Reject bootstrapping if the node already exists in gossip (CASSANDRA-5571)
+ * Fix NPE while loading paxos state (CASSANDRA-6211)
+ * cqlsh: add SHOW SESSION command (CASSANDRA-6228)
+Merged from 1.2:
+ * (Hadoop) Require CFRR batchSize to be at least 2 (CASSANDRA-6114)
* Add a warning for small LCS sstable size (CASSANDRA-6191)
* Add ability to list specific KS/CF combinations in nodetool cfstats
(CASSANDRA-4191)
* Mark CF clean if a mutation raced the drop and got it marked dirty
(CASSANDRA-5946)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0e9906e6/src/java/org/apache/cassandra/db/ConsistencyLevel.java
--
diff --cc src/java/org/apache/cassandra/db/ConsistencyLevel.java
index 5f17907,25fb25b..4fffc8a
--- a/src/java/org/apache/cassandra/db/ConsistencyLevel.java
+++ b/src/java/org/apache/cassandra/db/ConsistencyLevel.java
@@@ -286,46 -280,10 +282,44 @@@ public enum ConsistencyLeve
}
}
-public void validateForWrite(String table) throws InvalidRequestException
+public void validateForWrite(String keyspaceName) throws
InvalidRequestException
{
-if(this == EACH_QUORUM)
-requireNetworkTopologyStrategy(table);
+switch (this)
+{
+case LOCAL_QUORUM:
+case EACH_QUORUM:
+case LOCAL_ONE:
+requireNetworkTopologyStrategy(keyspaceName);
+break;
+case SERIAL:
+case LOCAL_SERIAL:
+throw new InvalidRequestException("You must use conditio
git commit: Allow LOCAL_ONE/LOCAL_QUORUM to work with SimpleStrategy patch by Alex Liu; reviewed by jasobrown for CASSANDRA-6238
Updated Branches:
refs/heads/cassandra-1.2 8c2404493 -> e5715f4e9
Allow LOCAL_ONE/LOCAL_QUORUM to work with SimpleStrategy
patch by Alex Liu; reviewed by jasobrown for CASSANDRA-6238
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e5715f4e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e5715f4e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e5715f4e
Branch: refs/heads/cassandra-1.2
Commit: e5715f4e9618e933fca4d4537ff84fd4ee24a641
Parents: 8c24044
Author: Jason Brown
Authored: Wed Nov 6 22:02:49 2013 -0800
Committer: Jason Brown
Committed: Wed Nov 6 22:02:49 2013 -0800
--
CHANGES.txt | 1 +
.../org/apache/cassandra/db/ConsistencyLevel.java | 14 ++
2 files changed, 3 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e5715f4e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index eab185a..d27c495 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -12,6 +12,7 @@
* make calculatePendingRanges asynchronous (CASSANDRA-6244)
* Remove blocking flushes in gossip thread (CASSANDRA-6297)
* Fix potential socket leak in connectionpool creation (CASSANDRA-6308)
+ * Allow LOCAL_ONE/LOCAL_QUORUM to work with SimpleSrrategy (CASSANDRA-6238)
1.2.11
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e5715f4e/src/java/org/apache/cassandra/db/ConsistencyLevel.java
--
diff --git a/src/java/org/apache/cassandra/db/ConsistencyLevel.java
b/src/java/org/apache/cassandra/db/ConsistencyLevel.java
index 1ce019b..25fb25b 100644
--- a/src/java/org/apache/cassandra/db/ConsistencyLevel.java
+++ b/src/java/org/apache/cassandra/db/ConsistencyLevel.java
@@ -275,10 +275,6 @@ public enum ConsistencyLevel
{
case ANY:
throw new InvalidRequestException("ANY ConsistencyLevel is
only supported for writes");
-case LOCAL_QUORUM:
-case LOCAL_ONE:
-requireNetworkTopologyStrategy(table);
-break;
case EACH_QUORUM:
throw new InvalidRequestException("EACH_QUORUM
ConsistencyLevel is only supported for writes");
}
@@ -286,14 +282,8 @@ public enum ConsistencyLevel
public void validateForWrite(String table) throws InvalidRequestException
{
-switch (this)
-{
-case LOCAL_QUORUM:
-case EACH_QUORUM:
-case LOCAL_ONE:
-requireNetworkTopologyStrategy(table);
-break;
-}
+if(this == EACH_QUORUM)
+requireNetworkTopologyStrategy(table);
}
public void validateCounterForWrite(CFMetaData metadata) throws
InvalidRequestException
[jira] [Commented] (CASSANDRA-6238) LOCAL_ONE doesn't work for SimpleStrategy
[ https://issues.apache.org/jira/browse/CASSANDRA-6238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815681#comment-13815681 ] Jason Brown commented on CASSANDRA-6238: I'm fine with making LOCAL_ONE work with SS, but we should also make LOCAL_QUORUM do so, as well. If there's no objections to making the LOCAL_*'s consistent, let's move forward. I'll review the patch by the morning. > LOCAL_ONE doesn't work for SimpleStrategy > - > > Key: CASSANDRA-6238 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6238 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Alex Liu >Assignee: Alex Liu > Fix For: 1.2.12, 2.0.3 > > Attachments: 6238-v2.txt, 6238.txt > > > LOCAL_ONE only works for NetworkTopologyStrategy which has DC specification. > Any other strategy fails. > If there is no DC specified in the strategy, we should treat LOCAL_ONE as ONE -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6305) cqlsh support for User Types
[ https://issues.apache.org/jira/browse/CASSANDRA-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815680#comment-13815680 ] Mikhail Stepura commented on CASSANDRA-6305: [~iamaleksey] I can try, but I have a pretty limited bandwidth these days. what is ETA for 2.1? > cqlsh support for User Types > > > Key: CASSANDRA-6305 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6305 > Project: Cassandra > Issue Type: New Feature >Reporter: Aleksey Yeschenko > Labels: cqlsh > Fix For: 2.1 > > > We need cqlsh support for several things: > 1. Autocomplete for UPDATE/INSERT/SELECT > 2. Autocomplete for ALTER TYPE/CREATE TYPE/DROP TYPE > 3. Proper decoding of UserType values (currently showing the encoded blob) > 4. Support UserTypes in DESCRIBE > 5. Separate DESCRIBE TYPES|TYPE cmds -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-3578) Multithreaded commitlog
[
https://issues.apache.org/jira/browse/CASSANDRA-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815618#comment-13815618
]
Vijay commented on CASSANDRA-3578:
--
Hi Benedict, archiver.maybeArchive(segment.getPath(), segment.getName()) is a
blocking call and will need to be a separate thread it might involve user
defined archival.
{quote}
sync() would mark things as flushed to disk that weren't, which would result in
log messages never being persisted
{quote}
My understand is that Calling force will sync the dirty pages and if we do
a concurrent writes to the same page they will be marked as dirty and will be
synched in the next call, how will we loose the log messages?
I still like the original approach :) of creating files (it may be just me)
because of simplicity and we can be aggressive in allocator threads similar to
your patch (to create empty files and deleting them).
> Multithreaded commitlog
> ---
>
> Key: CASSANDRA-3578
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3578
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Jonathan Ellis
>Assignee: Vijay
>Priority: Minor
> Labels: performance
> Attachments: 0001-CASSANDRA-3578.patch, ComitlogStress.java,
> Current-CL.png, Multi-Threded-CL.png, parallel_commit_log_2.patch
>
>
> Brian Aker pointed out a while ago that allowing multiple threads to modify
> the commitlog simultaneously (reserving space for each with a CAS first, the
> way we do in the SlabAllocator.Region.allocate) can improve performance,
> since you're not bottlenecking on a single thread to do all the copying and
> CRC computation.
> Now that we use mmap'd CommitLog segments (CASSANDRA-3411) this becomes
> doable.
> (moved from CASSANDRA-622, which was getting a bit muddled.)
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (CASSANDRA-3578) Multithreaded commitlog
[ https://issues.apache.org/jira/browse/CASSANDRA-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815553#comment-13815553 ] Benedict commented on CASSANDRA-3578: - I think there are two further improvements that need to be made, excluding some tidying up, before we should think about integration: 1) Obviously address optimising the CL format; and 2) Consider scrapping cfLastWrite beyond a set of names present in the log file I haven't worried about (1) up until now as I'm testing on SSDs so the seeking shouldn't cause a major performance penalty, and I've mostly been interested in finding out how we can improve performance, but if we were to scrap (2) we could make (1) resilient to reading past not-yet-synced records, and we could stop worrying about linearising the write positions, eliminating a huge amount of the threading overhead. We would just increment a counter indicating we've dirtied the segment. The work we currently do would still be necessary for batch executors. The only negatives here are that we might unnecessarily flush a Cf that has already fully synced in flushOldestKeyspaces (when trying to reclaim logs) and that we will be slightly slower to reclaim log files. On the whole I think it is a cost well worth paying for reducing the CL write path. > Multithreaded commitlog > --- > > Key: CASSANDRA-3578 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3578 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Vijay >Priority: Minor > Labels: performance > Attachments: 0001-CASSANDRA-3578.patch, ComitlogStress.java, > Current-CL.png, Multi-Threded-CL.png, parallel_commit_log_2.patch > > > Brian Aker pointed out a while ago that allowing multiple threads to modify > the commitlog simultaneously (reserving space for each with a CAS first, the > way we do in the SlabAllocator.Region.allocate) can improve performance, > since you're not bottlenecking on a single thread to do all the copying and > CRC computation. > Now that we use mmap'd CommitLog segments (CASSANDRA-3411) this becomes > doable. > (moved from CASSANDRA-622, which was getting a bit muddled.) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6309) Pig CqlStorage generates ERROR 1108: Duplicate schema alias
[
https://issues.apache.org/jira/browse/CASSANDRA-6309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815473#comment-13815473
]
Alex Liu commented on CASSANDRA-6309:
-
It works in 1.2 branch. It looks like some changes to table metadata
implementation causes the issue.
> Pig CqlStorage generates ERROR 1108: Duplicate schema alias
>
>
> Key: CASSANDRA-6309
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6309
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
>Reporter: Thunder Stumpges
>
> In Pig after loading a simple CQL3 table from Cassandra 2.0.1, and dumping
> contents, I receive:
> Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 1108:
> Duplicate schema alias: author in "cm"
> cm = load 'cql://thunder_test/cassandra_messages' USING CqlStorage;
> dump cm
> ERROR org.apache.pig.tools.grunt.Grunt -
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
> open iterator for alias cm
> ...
> Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 1108:
> Duplicate schema alias: author in "cm"
> at
> org.apache.pig.newplan.logical.visitor.SchemaAliasVisitor.validate(SchemaAliasVisitor.java:75)
> running 'describe cm' gives:
> cm: {message_id: chararray,author: chararray,author: chararray,body:
> chararray,message_id: chararray}
> The original table schema in Cassandra is:
> CREATE TABLE cassandra_messages (
> message_id text,
> author text,
> body text,
> PRIMARY KEY (message_id, author)
> ) WITH
> bloom_filter_fp_chance=0.01 AND
> caching='KEYS_ONLY' AND
> comment='null' AND
> dclocal_read_repair_chance=0.00 AND
> gc_grace_seconds=864000 AND
> index_interval=128 AND
> read_repair_chance=0.10 AND
> replicate_on_write='true' AND
> populate_io_cache_on_flush='false' AND
> default_time_to_live=0 AND
> speculative_retry='NONE' AND
> memtable_flush_period_in_ms=0 AND
> compaction={'class': 'SizeTieredCompactionStrategy'} AND
> compression={'sstable_compression': 'LZ4Compressor'};
> it appears that the code in CqlStorage.getColumnMetadata at ~line 478 takes
> the "keys" columns (in my case, message_id and author) and appends the
> columns from getColumnMeta (which has all three columns). Thus the keys
> columns are duplicated.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (CASSANDRA-6309) Pig CqlStorage generates ERROR 1108: Duplicate schema alias
[
https://issues.apache.org/jira/browse/CASSANDRA-6309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815460#comment-13815460
]
Thunder Stumpges commented on CASSANDRA-6309:
-
just as a note, I'm using code from the tip of the cassandra-2.0 branch. I'm a
little confused because the code which combines the list from "getKeysMeta"
with that of "getColumnMeta" has been around a while. There was some
refactoring of getKeysMeta recently, but not sure how that could have changed
things.
I have a naive patch which simply loops the columns in getColumnMeta and only
adds them to "keys" collection if they don't already exist. Can add that as a
patch if y'all think that's an appropriate way to solve this thing.
> Pig CqlStorage generates ERROR 1108: Duplicate schema alias
>
>
> Key: CASSANDRA-6309
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6309
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
>Reporter: Thunder Stumpges
>
> In Pig after loading a simple CQL3 table from Cassandra 2.0.1, and dumping
> contents, I receive:
> Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 1108:
> Duplicate schema alias: author in "cm"
> cm = load 'cql://thunder_test/cassandra_messages' USING CqlStorage;
> dump cm
> ERROR org.apache.pig.tools.grunt.Grunt -
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
> open iterator for alias cm
> ...
> Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 1108:
> Duplicate schema alias: author in "cm"
> at
> org.apache.pig.newplan.logical.visitor.SchemaAliasVisitor.validate(SchemaAliasVisitor.java:75)
> running 'describe cm' gives:
> cm: {message_id: chararray,author: chararray,author: chararray,body:
> chararray,message_id: chararray}
> The original table schema in Cassandra is:
> CREATE TABLE cassandra_messages (
> message_id text,
> author text,
> body text,
> PRIMARY KEY (message_id, author)
> ) WITH
> bloom_filter_fp_chance=0.01 AND
> caching='KEYS_ONLY' AND
> comment='null' AND
> dclocal_read_repair_chance=0.00 AND
> gc_grace_seconds=864000 AND
> index_interval=128 AND
> read_repair_chance=0.10 AND
> replicate_on_write='true' AND
> populate_io_cache_on_flush='false' AND
> default_time_to_live=0 AND
> speculative_retry='NONE' AND
> memtable_flush_period_in_ms=0 AND
> compaction={'class': 'SizeTieredCompactionStrategy'} AND
> compression={'sstable_compression': 'LZ4Compressor'};
> it appears that the code in CqlStorage.getColumnMetadata at ~line 478 takes
> the "keys" columns (in my case, message_id and author) and appends the
> columns from getColumnMeta (which has all three columns). Thus the keys
> columns are duplicated.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (CASSANDRA-6309) Pig CqlStorage generates ERROR 1108: Duplicate schema alias
[
https://issues.apache.org/jira/browse/CASSANDRA-6309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thunder Stumpges updated CASSANDRA-6309:
Description:
In Pig after loading a simple CQL3 table from Cassandra 2.0.1, and dumping
contents, I receive:
Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 1108:
Duplicate schema alias: author in "cm"
cm = load 'cql://thunder_test/cassandra_messages' USING CqlStorage;
dump cm
ERROR org.apache.pig.tools.grunt.Grunt -
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open
iterator for alias cm
...
Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 1108:
Duplicate schema alias: author in "cm"
at
org.apache.pig.newplan.logical.visitor.SchemaAliasVisitor.validate(SchemaAliasVisitor.java:75)
running 'describe cm' gives:
cm: {message_id: chararray,author: chararray,author: chararray,body:
chararray,message_id: chararray}
The original table schema in Cassandra is:
CREATE TABLE cassandra_messages (
message_id text,
author text,
body text,
PRIMARY KEY (message_id, author)
) WITH
bloom_filter_fp_chance=0.01 AND
caching='KEYS_ONLY' AND
comment='null' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=864000 AND
index_interval=128 AND
read_repair_chance=0.10 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
default_time_to_live=0 AND
speculative_retry='NONE' AND
memtable_flush_period_in_ms=0 AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={'sstable_compression': 'LZ4Compressor'};
it appears that the code in CqlStorage.getColumnMetadata at ~line 478 takes the
"keys" columns (in my case, message_id and author) and appends the columns from
getColumnMeta (which has all three columns). Thus the keys columns are
duplicated.
was:
In Pig after loading a simple CQL3 table from Cassandra 2.0.1, and dumping
contents, I receive:
Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 1108:
Duplicate schema alias: author in "cm"
The original table schema in Cassandra is:
> Pig CqlStorage generates ERROR 1108: Duplicate schema alias
>
>
> Key: CASSANDRA-6309
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6309
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
>Reporter: Thunder Stumpges
>
> In Pig after loading a simple CQL3 table from Cassandra 2.0.1, and dumping
> contents, I receive:
> Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 1108:
> Duplicate schema alias: author in "cm"
> cm = load 'cql://thunder_test/cassandra_messages' USING CqlStorage;
> dump cm
> ERROR org.apache.pig.tools.grunt.Grunt -
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
> open iterator for alias cm
> ...
> Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 1108:
> Duplicate schema alias: author in "cm"
> at
> org.apache.pig.newplan.logical.visitor.SchemaAliasVisitor.validate(SchemaAliasVisitor.java:75)
> running 'describe cm' gives:
> cm: {message_id: chararray,author: chararray,author: chararray,body:
> chararray,message_id: chararray}
> The original table schema in Cassandra is:
> CREATE TABLE cassandra_messages (
> message_id text,
> author text,
> body text,
> PRIMARY KEY (message_id, author)
> ) WITH
> bloom_filter_fp_chance=0.01 AND
> caching='KEYS_ONLY' AND
> comment='null' AND
> dclocal_read_repair_chance=0.00 AND
> gc_grace_seconds=864000 AND
> index_interval=128 AND
> read_repair_chance=0.10 AND
> replicate_on_write='true' AND
> populate_io_cache_on_flush='false' AND
> default_time_to_live=0 AND
> speculative_retry='NONE' AND
> memtable_flush_period_in_ms=0 AND
> compaction={'class': 'SizeTieredCompactionStrategy'} AND
> compression={'sstable_compression': 'LZ4Compressor'};
> it appears that the code in CqlStorage.getColumnMetadata at ~line 478 takes
> the "keys" columns (in my case, message_id and author) and appends the
> columns from getColumnMeta (which has all three columns). Thus the keys
> columns are duplicated.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Created] (CASSANDRA-6309) Pig CqlStorage generates ERROR 1108: Duplicate schema alias
Thunder Stumpges created CASSANDRA-6309: --- Summary: Pig CqlStorage generates ERROR 1108: Duplicate schema alias Key: CASSANDRA-6309 URL: https://issues.apache.org/jira/browse/CASSANDRA-6309 Project: Cassandra Issue Type: Bug Components: Hadoop Reporter: Thunder Stumpges In Pig after loading a simple CQL3 table from Cassandra 2.0.1, and dumping contents, I receive: Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 1108: Duplicate schema alias: author in "cm" The original table schema in Cassandra is: -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6127) vnodes don't scale to hundreds of nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-6127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815379#comment-13815379 ] Brandon Williams commented on CASSANDRA-6127: - At this point, I think we should: * see if the flapping happens with vnodes (maybe Quentin already knows from his last test) * see if the flapping happens without vnodes but the same number of nodes Because if sum() in ArrivalWindow is burning the most CPU in the Gossiper task (note: not bottlenecking, each call was at most ~3ms, there were just lots of them) then that means that the problem is no longer tied to vnodes (if it ever was, since sum is per-node, not per-token) and we should probably open a new ticket (can't start a cluster of size >=X all at once, or similar) and discuss there. We know that clusters much larger than any discussed on this ticket exist, but I don't think any of them have all rebooted at once. > vnodes don't scale to hundreds of nodes > --- > > Key: CASSANDRA-6127 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6127 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Any cluster that has vnodes and consists of hundreds of > physical nodes. >Reporter: Tupshin Harper >Assignee: Jonathan Ellis > Attachments: 6000vnodes.patch, AdjustableGossipPeriod.patch, > delayEstimatorUntilStatisticallyValid.patch > > > There are a lot of gossip-related issues related to very wide clusters that > also have vnodes enabled. Let's use this ticket as a master in case there are > sub-tickets. > The most obvious symptom I've seen is with 1000 nodes in EC2 with m1.xlarge > instances. Each node configured with 32 vnodes. > Without vnodes, cluster spins up fine and is ready to handle requests within > 30 minutes or less. > With vnodes, nodes are reporting constant up/down flapping messages with no > external load on the cluster. After a couple of hours, they were still > flapping, had very high cpu load, and the cluster never looked like it was > going to stabilize or be useful for traffic. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Comment Edited] (CASSANDRA-6127) vnodes don't scale to hundreds of nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-6127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815379#comment-13815379 ] Brandon Williams edited comment on CASSANDRA-6127 at 11/6/13 10:25 PM: --- At this point, I think we should: * see if the flapping happens with vnodes in 1.2 head (maybe Quentin already knows from his last test) * see if the flapping happens without vnodes in 1.2 head but the same number of nodes Because if sum() in ArrivalWindow is burning the most CPU in the Gossiper task (note: not bottlenecking, each call was at most ~3ms, there were just lots of them) then that means that the problem is no longer tied to vnodes (if it ever was, since sum is per-node, not per-token) and we should probably open a new ticket (can't start a cluster of size >=X all at once, or similar) and discuss there. We know that clusters much larger than any discussed on this ticket exist, but I don't think any of them have all rebooted at once. was (Author: brandon.williams): At this point, I think we should: * see if the flapping happens with vnodes (maybe Quentin already knows from his last test) * see if the flapping happens without vnodes but the same number of nodes Because if sum() in ArrivalWindow is burning the most CPU in the Gossiper task (note: not bottlenecking, each call was at most ~3ms, there were just lots of them) then that means that the problem is no longer tied to vnodes (if it ever was, since sum is per-node, not per-token) and we should probably open a new ticket (can't start a cluster of size >=X all at once, or similar) and discuss there. We know that clusters much larger than any discussed on this ticket exist, but I don't think any of them have all rebooted at once. > vnodes don't scale to hundreds of nodes > --- > > Key: CASSANDRA-6127 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6127 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Any cluster that has vnodes and consists of hundreds of > physical nodes. >Reporter: Tupshin Harper >Assignee: Jonathan Ellis > Attachments: 6000vnodes.patch, AdjustableGossipPeriod.patch, > delayEstimatorUntilStatisticallyValid.patch > > > There are a lot of gossip-related issues related to very wide clusters that > also have vnodes enabled. Let's use this ticket as a master in case there are > sub-tickets. > The most obvious symptom I've seen is with 1000 nodes in EC2 with m1.xlarge > instances. Each node configured with 32 vnodes. > Without vnodes, cluster spins up fine and is ready to handle requests within > 30 minutes or less. > With vnodes, nodes are reporting constant up/down flapping messages with no > external load on the cluster. After a couple of hours, they were still > flapping, had very high cpu load, and the cluster never looked like it was > going to stabilize or be useful for traffic. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (CASSANDRA-6187) Cassandra-2.0 node seems to be pausing periodically under normal operations
[
https://issues.apache.org/jira/browse/CASSANDRA-6187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Zou updated CASSANDRA-6187:
--
Description:
Under normal operation condition without any interruption, the traffic
generator will periodically see a couple of seconds of zero (or very little)
transactions, i.e. the throughput can be quite high for a while, then comes
down to zero (or very little) transactions for 10 ~ 20 seconds.
The Cassandra system log occasionally also logs message drops. But these
message drop events do not correlate in time to the observed transaction drop
events.
Example of message dropping log:
{noformat}
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,216 MessagingService.java (line
812) 1191 MUTATION messages dropped in last 5000ms
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,217 MessagingService.java (line
812) 502 READ_REPAIR messages dropped in last 5000ms
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,217 StatusLogger.java (line 55)
Pool NameActive Pending Completed Blocked All
Time Blocked
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,246 StatusLogger.java (line 70)
ReadStage 0 0 845326 0
0
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,246 StatusLogger.java (line 70)
RequestResponseStage 0 01643358 0
0
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,246 StatusLogger.java (line 70)
ReadRepairStage 0 0 61247 0
0
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,247 StatusLogger.java (line 70)
MutationStage 0 01155502 0
0
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,247 StatusLogger.java (line 70)
ReplicateOnWriteStage 0 0 0 0
0
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,247 StatusLogger.java (line 70)
GossipStage 0 0 5391 0
0
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,248 StatusLogger.java (line 70)
AntiEntropyStage 0 0 0 0
0
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,248 StatusLogger.java (line 70)
MigrationStage0 0 14 0
0
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,248 StatusLogger.java (line 70)
MemtablePostFlusher 0 0 99 0
0
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,248 StatusLogger.java (line 70)
MemoryMeter 0 0 58 0
0
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,249 StatusLogger.java (line 70)
FlushWriter 0 0 45 0
0
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,249 StatusLogger.java (line 70)
MiscStage 0 0 0 0
0
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,249 StatusLogger.java (line 70)
commitlog_archiver0 0 0 0
0
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,250 StatusLogger.java (line 70)
InternalResponseStage 0 0 3 0
0
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,250 StatusLogger.java (line 70)
HintedHandoff 0 0 7 0
0
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,250 StatusLogger.java (line 79)
CompactionManager 0 0
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,250 StatusLogger.java (line 81)
Commitlog n/a 0
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,251 StatusLogger.java (line 93)
MessagingServicen/a 0,0
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,251 StatusLogger.java (line 103)
Cache Type Size Capacity
KeysToSave
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,251 StatusLogger.java (line 105)
KeyCache 74140104857600
all
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,251 StatusLogger.java (line 111)
RowCache 00
all
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,251 StatusLogger.java (line 118)
ColumnFamilyMemtable ops,data
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,251 StatusLogger.java (line 121)
system.schema_triggers0,0
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,252 StatusLogger.java (line 121)
system.local 18,520
INFO [ScheduledTasks:1] 2013-10-11 16:36:12,252 StatusLogger.java (line 121)
system.peers
[jira] [Commented] (CASSANDRA-6268) Poor performance of Hadoop if any DC is using VNodes
[ https://issues.apache.org/jira/browse/CASSANDRA-6268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815352#comment-13815352 ] Jeremiah Jordan commented on CASSANDRA-6268: Double check you are using the right version of thrift for each one. A bunch of the stuff in the original patch look like formatting changes. > Poor performance of Hadoop if any DC is using VNodes > > > Key: CASSANDRA-6268 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6268 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Piotr Kołaczkowski >Assignee: Piotr Kołaczkowski > Attachments: 6268-2.txt, 6268-cassandra-2.0.txt > > > Some customers are complaining about huge number of splits in Hadoop caused > by VNodes. Disabling vnodes only in Hadoop DC does not fix it. Splits are > generated from the results of describe_ring, which returns a huge number of > ranges anyways, and doesn't take into account that there will be huge number > of consecutive ranges residing on the nodes we'd like the M/R job to be run. > The proposed fix: > 1. allows for specifying the DC(s) the Hadoop job should be run in (in DSE - > defaults to all Hadoop DCs) > 2. merges consecutive ranges before generating Hadoop splits, so we don't > have artificial range splitting caused by vnodes in the other DCs > For non-DSE users this feature is turned off by default and doesn't change > the old behaviour. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6268) Poor performance of Hadoop if any DC is using VNodes
[ https://issues.apache.org/jira/browse/CASSANDRA-6268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815340#comment-13815340 ] Piotr Kołaczkowski commented on CASSANDRA-6268: --- Hmm, most of it are changes generated by thrift. But let me double check that. Maybe I messed up something. > Poor performance of Hadoop if any DC is using VNodes > > > Key: CASSANDRA-6268 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6268 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Piotr Kołaczkowski >Assignee: Piotr Kołaczkowski > Attachments: 6268-2.txt, 6268-cassandra-2.0.txt > > > Some customers are complaining about huge number of splits in Hadoop caused > by VNodes. Disabling vnodes only in Hadoop DC does not fix it. Splits are > generated from the results of describe_ring, which returns a huge number of > ranges anyways, and doesn't take into account that there will be huge number > of consecutive ranges residing on the nodes we'd like the M/R job to be run. > The proposed fix: > 1. allows for specifying the DC(s) the Hadoop job should be run in (in DSE - > defaults to all Hadoop DCs) > 2. merges consecutive ranges before generating Hadoop splits, so we don't > have artificial range splitting caused by vnodes in the other DCs > For non-DSE users this feature is turned off by default and doesn't change > the old behaviour. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6127) vnodes don't scale to hundreds of nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-6127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815297#comment-13815297 ] Tupshin Harper commented on CASSANDRA-6127: --- +1. Strongly agree with Jonathan's analysis and proposal. > vnodes don't scale to hundreds of nodes > --- > > Key: CASSANDRA-6127 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6127 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Any cluster that has vnodes and consists of hundreds of > physical nodes. >Reporter: Tupshin Harper >Assignee: Jonathan Ellis > Attachments: 6000vnodes.patch, AdjustableGossipPeriod.patch, > delayEstimatorUntilStatisticallyValid.patch > > > There are a lot of gossip-related issues related to very wide clusters that > also have vnodes enabled. Let's use this ticket as a master in case there are > sub-tickets. > The most obvious symptom I've seen is with 1000 nodes in EC2 with m1.xlarge > instances. Each node configured with 32 vnodes. > Without vnodes, cluster spins up fine and is ready to handle requests within > 30 minutes or less. > With vnodes, nodes are reporting constant up/down flapping messages with no > external load on the cluster. After a couple of hours, they were still > flapping, had very high cpu load, and the cluster never looked like it was > going to stabilize or be useful for traffic. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6127) vnodes don't scale to hundreds of nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-6127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815286#comment-13815286 ] Jonathan Ellis commented on CASSANDRA-6127: --- ISTM that FD processing Gossip updates synchronously is a fundamental problem. Any hiccup in processing will cause FD false positives. (And even if our own code is perfect, GC pauses can still do this to us.) Wouldn't it be better if we: - time heartbeats based on when they arrive instead of when Gossip processes them - teach FD to recognize that its information is only good up to the most recently processed message -- the absence of messages after that doesn't mean everyone is down unless the Gossip stage is empty > vnodes don't scale to hundreds of nodes > --- > > Key: CASSANDRA-6127 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6127 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Any cluster that has vnodes and consists of hundreds of > physical nodes. >Reporter: Tupshin Harper >Assignee: Jonathan Ellis > Attachments: 6000vnodes.patch, AdjustableGossipPeriod.patch, > delayEstimatorUntilStatisticallyValid.patch > > > There are a lot of gossip-related issues related to very wide clusters that > also have vnodes enabled. Let's use this ticket as a master in case there are > sub-tickets. > The most obvious symptom I've seen is with 1000 nodes in EC2 with m1.xlarge > instances. Each node configured with 32 vnodes. > Without vnodes, cluster spins up fine and is ready to handle requests within > 30 minutes or less. > With vnodes, nodes are reporting constant up/down flapping messages with no > external load on the cluster. After a couple of hours, they were still > flapping, had very high cpu load, and the cluster never looked like it was > going to stabilize or be useful for traffic. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-3578) Multithreaded commitlog
[ https://issues.apache.org/jira/browse/CASSANDRA-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815272#comment-13815272 ] Benedict commented on CASSANDRA-3578: - Thanks. Repos updated. > Multithreaded commitlog > --- > > Key: CASSANDRA-3578 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3578 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Vijay >Priority: Minor > Labels: performance > Attachments: 0001-CASSANDRA-3578.patch, ComitlogStress.java, > Current-CL.png, Multi-Threded-CL.png, parallel_commit_log_2.patch > > > Brian Aker pointed out a while ago that allowing multiple threads to modify > the commitlog simultaneously (reserving space for each with a CAS first, the > way we do in the SlabAllocator.Region.allocate) can improve performance, > since you're not bottlenecking on a single thread to do all the copying and > CRC computation. > Now that we use mmap'd CommitLog segments (CASSANDRA-3411) this becomes > doable. > (moved from CASSANDRA-622, which was getting a bit muddled.) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6268) Poor performance of Hadoop if any DC is using VNodes
[ https://issues.apache.org/jira/browse/CASSANDRA-6268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815270#comment-13815270 ] Jonathan Ellis commented on CASSANDRA-6268: --- Should I be worried that the 2.0 patch is half the size of the other? > Poor performance of Hadoop if any DC is using VNodes > > > Key: CASSANDRA-6268 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6268 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Piotr Kołaczkowski >Assignee: Piotr Kołaczkowski > Attachments: 6268-2.txt, 6268-cassandra-2.0.txt > > > Some customers are complaining about huge number of splits in Hadoop caused > by VNodes. Disabling vnodes only in Hadoop DC does not fix it. Splits are > generated from the results of describe_ring, which returns a huge number of > ranges anyways, and doesn't take into account that there will be huge number > of consecutive ranges residing on the nodes we'd like the M/R job to be run. > The proposed fix: > 1. allows for specifying the DC(s) the Hadoop job should be run in (in DSE - > defaults to all Hadoop DCs) > 2. merges consecutive ranges before generating Hadoop splits, so we don't > have artificial range splitting caused by vnodes in the other DCs > For non-DSE users this feature is turned off by default and doesn't change > the old behaviour. -- This message was sent by Atlassian JIRA (v6.1#6144)
[2/6] git commit: Fix potential socket leak in connectionpool creation patch by Minh Do; reviewed by jbellis for CASSANDRA-6308
Fix potential socket leak in connectionpool creation
patch by Minh Do; reviewed by jbellis for CASSANDRA-6308
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8c240449
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8c240449
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8c240449
Branch: refs/heads/cassandra-2.0
Commit: 8c2404493a6efc109cd621c02a4de6ef1f1f46aa
Parents: 8e7d728
Author: Jonathan Ellis
Authored: Wed Nov 6 12:33:45 2013 -0800
Committer: Jonathan Ellis
Committed: Wed Nov 6 12:33:45 2013 -0800
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/net/MessagingService.java | 12 +---
2 files changed, 10 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/8c240449/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index fd3af68..eab185a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -11,6 +11,7 @@
* Nodetool gets default JMX port from cassandra-env.sh (CASSANDRA-6273)
* make calculatePendingRanges asynchronous (CASSANDRA-6244)
* Remove blocking flushes in gossip thread (CASSANDRA-6297)
+ * Fix potential socket leak in connectionpool creation (CASSANDRA-6308)
1.2.11
http://git-wip-us.apache.org/repos/asf/cassandra/blob/8c240449/src/java/org/apache/cassandra/net/MessagingService.java
--
diff --git a/src/java/org/apache/cassandra/net/MessagingService.java
b/src/java/org/apache/cassandra/net/MessagingService.java
index a199e83..7e420cf 100644
--- a/src/java/org/apache/cassandra/net/MessagingService.java
+++ b/src/java/org/apache/cassandra/net/MessagingService.java
@@ -259,7 +259,7 @@ public final class MessagingService implements
MessagingServiceMBean
*/
private final ConcurrentMap
streamExecutors = new NonBlockingHashMap();
-private final NonBlockingHashMap
connectionManagers = new NonBlockingHashMap();
+private final ConcurrentMap
connectionManagers = new NonBlockingHashMap();
private static final Logger logger =
LoggerFactory.getLogger(MessagingService.class);
private static final int LOG_DROPPED_INTERVAL_IN_MS = 5000;
@@ -484,11 +484,17 @@ public final class MessagingService implements
MessagingServiceMBean
OutboundTcpConnectionPool cp = connectionManagers.get(to);
if (cp == null)
{
-connectionManagers.putIfAbsent(to, new
OutboundTcpConnectionPool(to));
-cp = connectionManagers.get(to);
+cp = new OutboundTcpConnectionPool(to);
+OutboundTcpConnectionPool existingPool =
connectionManagers.putIfAbsent(to, cp);
+if (existingPool != null)
+{
+cp.close();
+cp = existingPool;
+}
}
return cp;
}
+
public OutboundTcpConnection getConnection(InetAddress to, MessageOut msg)
{
[1/6] git commit: Fix potential socket leak in connectionpool creation patch by Minh Do; reviewed by jbellis for CASSANDRA-6308
Updated Branches:
refs/heads/cassandra-1.2 8e7d7285c -> 8c2404493
refs/heads/cassandra-2.0 dc71d2200 -> b4020979c
refs/heads/trunk a4fc13c05 -> c65c15cc0
Fix potential socket leak in connectionpool creation
patch by Minh Do; reviewed by jbellis for CASSANDRA-6308
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8c240449
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8c240449
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8c240449
Branch: refs/heads/cassandra-1.2
Commit: 8c2404493a6efc109cd621c02a4de6ef1f1f46aa
Parents: 8e7d728
Author: Jonathan Ellis
Authored: Wed Nov 6 12:33:45 2013 -0800
Committer: Jonathan Ellis
Committed: Wed Nov 6 12:33:45 2013 -0800
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/net/MessagingService.java | 12 +---
2 files changed, 10 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/8c240449/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index fd3af68..eab185a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -11,6 +11,7 @@
* Nodetool gets default JMX port from cassandra-env.sh (CASSANDRA-6273)
* make calculatePendingRanges asynchronous (CASSANDRA-6244)
* Remove blocking flushes in gossip thread (CASSANDRA-6297)
+ * Fix potential socket leak in connectionpool creation (CASSANDRA-6308)
1.2.11
http://git-wip-us.apache.org/repos/asf/cassandra/blob/8c240449/src/java/org/apache/cassandra/net/MessagingService.java
--
diff --git a/src/java/org/apache/cassandra/net/MessagingService.java
b/src/java/org/apache/cassandra/net/MessagingService.java
index a199e83..7e420cf 100644
--- a/src/java/org/apache/cassandra/net/MessagingService.java
+++ b/src/java/org/apache/cassandra/net/MessagingService.java
@@ -259,7 +259,7 @@ public final class MessagingService implements
MessagingServiceMBean
*/
private final ConcurrentMap
streamExecutors = new NonBlockingHashMap();
-private final NonBlockingHashMap
connectionManagers = new NonBlockingHashMap();
+private final ConcurrentMap
connectionManagers = new NonBlockingHashMap();
private static final Logger logger =
LoggerFactory.getLogger(MessagingService.class);
private static final int LOG_DROPPED_INTERVAL_IN_MS = 5000;
@@ -484,11 +484,17 @@ public final class MessagingService implements
MessagingServiceMBean
OutboundTcpConnectionPool cp = connectionManagers.get(to);
if (cp == null)
{
-connectionManagers.putIfAbsent(to, new
OutboundTcpConnectionPool(to));
-cp = connectionManagers.get(to);
+cp = new OutboundTcpConnectionPool(to);
+OutboundTcpConnectionPool existingPool =
connectionManagers.putIfAbsent(to, cp);
+if (existingPool != null)
+{
+cp.close();
+cp = existingPool;
+}
}
return cp;
}
+
public OutboundTcpConnection getConnection(InetAddress to, MessageOut msg)
{
[6/6] git commit: Merge branch 'cassandra-2.0' into trunk
Merge branch 'cassandra-2.0' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c65c15cc Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c65c15cc Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c65c15cc Branch: refs/heads/trunk Commit: c65c15cc0583164ffd724d1008397f9380fd7a89 Parents: a4fc13c b402097 Author: Jonathan Ellis Authored: Wed Nov 6 12:33:59 2013 -0800 Committer: Jonathan Ellis Committed: Wed Nov 6 12:33:59 2013 -0800 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/net/MessagingService.java | 12 +--- 2 files changed, 10 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/c65c15cc/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/c65c15cc/src/java/org/apache/cassandra/net/MessagingService.java --
[4/6] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0
Merge branch 'cassandra-1.2' into cassandra-2.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b4020979 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b4020979 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b4020979 Branch: refs/heads/trunk Commit: b4020979ccba9514b7d70fb47d76b5e2edaeffdc Parents: dc71d22 8c24044 Author: Jonathan Ellis Authored: Wed Nov 6 12:33:51 2013 -0800 Committer: Jonathan Ellis Committed: Wed Nov 6 12:33:51 2013 -0800 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/net/MessagingService.java | 12 +--- 2 files changed, 10 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b4020979/CHANGES.txt -- diff --cc CHANGES.txt index 7e9201a,eab185a..4a1fd05 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -29,42 -11,10 +29,43 @@@ Merged from 1.2 * Nodetool gets default JMX port from cassandra-env.sh (CASSANDRA-6273) * make calculatePendingRanges asynchronous (CASSANDRA-6244) * Remove blocking flushes in gossip thread (CASSANDRA-6297) + * Fix potential socket leak in connectionpool creation (CASSANDRA-6308) -1.2.11 +2.0.2 + * Update FailureDetector to use nanontime (CASSANDRA-4925) + * Fix FileCacheService regressions (CASSANDRA-6149) + * Never return WriteTimeout for CL.ANY (CASSANDRA-6032) + * Fix race conditions in bulk loader (CASSANDRA-6129) + * Add configurable metrics reporting (CASSANDRA-4430) + * drop queries exceeding a configurable number of tombstones (CASSANDRA-6117) + * Track and persist sstable read activity (CASSANDRA-5515) + * Fixes for speculative retry (CASSANDRA-5932, CASSANDRA-6194) + * Improve memory usage of metadata min/max column names (CASSANDRA-6077) + * Fix thrift validation refusing row markers on CQL3 tables (CASSANDRA-6081) + * Fix insertion of collections with CAS (CASSANDRA-6069) + * Correctly send metadata on SELECT COUNT (CASSANDRA-6080) + * Track clients' remote addresses in ClientState (CASSANDRA-6070) + * Create snapshot dir if it does not exist when migrating + leveled manifest (CASSANDRA-6093) + * make sequential nodetool repair the default (CASSANDRA-5950) + * Add more hooks for compaction strategy implementations (CASSANDRA-6111) + * Fix potential NPE on composite 2ndary indexes (CASSANDRA-6098) + * Delete can potentially be skipped in batch (CASSANDRA-6115) + * Allow alter keyspace on system_traces (CASSANDRA-6016) + * Disallow empty column names in cql (CASSANDRA-6136) + * Use Java7 file-handling APIs and fix file moving on Windows (CASSANDRA-5383) + * Save compaction history to system keyspace (CASSANDRA-5078) + * Fix NPE if StorageService.getOperationMode() is executed before full startup (CASSANDRA-6166) + * CQL3: support pre-epoch longs for TimestampType (CASSANDRA-6212) + * Add reloadtriggers command to nodetool (CASSANDRA-4949) + * cqlsh: ignore empty 'value alias' in DESCRIBE (CASSANDRA-6139) + * Fix sstable loader (CASSANDRA-6205) + * Reject bootstrapping if the node already exists in gossip (CASSANDRA-5571) + * Fix NPE while loading paxos state (CASSANDRA-6211) + * cqlsh: add SHOW SESSION command (CASSANDRA-6228) +Merged from 1.2: + * (Hadoop) Require CFRR batchSize to be at least 2 (CASSANDRA-6114) * Add a warning for small LCS sstable size (CASSANDRA-6191) * Add ability to list specific KS/CF combinations in nodetool cfstats (CASSANDRA-4191) * Mark CF clean if a mutation raced the drop and got it marked dirty (CASSANDRA-5946) http://git-wip-us.apache.org/repos/asf/cassandra/blob/b4020979/src/java/org/apache/cassandra/net/MessagingService.java --
[5/6] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0
Merge branch 'cassandra-1.2' into cassandra-2.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b4020979 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b4020979 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b4020979 Branch: refs/heads/cassandra-2.0 Commit: b4020979ccba9514b7d70fb47d76b5e2edaeffdc Parents: dc71d22 8c24044 Author: Jonathan Ellis Authored: Wed Nov 6 12:33:51 2013 -0800 Committer: Jonathan Ellis Committed: Wed Nov 6 12:33:51 2013 -0800 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/net/MessagingService.java | 12 +--- 2 files changed, 10 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b4020979/CHANGES.txt -- diff --cc CHANGES.txt index 7e9201a,eab185a..4a1fd05 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -29,42 -11,10 +29,43 @@@ Merged from 1.2 * Nodetool gets default JMX port from cassandra-env.sh (CASSANDRA-6273) * make calculatePendingRanges asynchronous (CASSANDRA-6244) * Remove blocking flushes in gossip thread (CASSANDRA-6297) + * Fix potential socket leak in connectionpool creation (CASSANDRA-6308) -1.2.11 +2.0.2 + * Update FailureDetector to use nanontime (CASSANDRA-4925) + * Fix FileCacheService regressions (CASSANDRA-6149) + * Never return WriteTimeout for CL.ANY (CASSANDRA-6032) + * Fix race conditions in bulk loader (CASSANDRA-6129) + * Add configurable metrics reporting (CASSANDRA-4430) + * drop queries exceeding a configurable number of tombstones (CASSANDRA-6117) + * Track and persist sstable read activity (CASSANDRA-5515) + * Fixes for speculative retry (CASSANDRA-5932, CASSANDRA-6194) + * Improve memory usage of metadata min/max column names (CASSANDRA-6077) + * Fix thrift validation refusing row markers on CQL3 tables (CASSANDRA-6081) + * Fix insertion of collections with CAS (CASSANDRA-6069) + * Correctly send metadata on SELECT COUNT (CASSANDRA-6080) + * Track clients' remote addresses in ClientState (CASSANDRA-6070) + * Create snapshot dir if it does not exist when migrating + leveled manifest (CASSANDRA-6093) + * make sequential nodetool repair the default (CASSANDRA-5950) + * Add more hooks for compaction strategy implementations (CASSANDRA-6111) + * Fix potential NPE on composite 2ndary indexes (CASSANDRA-6098) + * Delete can potentially be skipped in batch (CASSANDRA-6115) + * Allow alter keyspace on system_traces (CASSANDRA-6016) + * Disallow empty column names in cql (CASSANDRA-6136) + * Use Java7 file-handling APIs and fix file moving on Windows (CASSANDRA-5383) + * Save compaction history to system keyspace (CASSANDRA-5078) + * Fix NPE if StorageService.getOperationMode() is executed before full startup (CASSANDRA-6166) + * CQL3: support pre-epoch longs for TimestampType (CASSANDRA-6212) + * Add reloadtriggers command to nodetool (CASSANDRA-4949) + * cqlsh: ignore empty 'value alias' in DESCRIBE (CASSANDRA-6139) + * Fix sstable loader (CASSANDRA-6205) + * Reject bootstrapping if the node already exists in gossip (CASSANDRA-5571) + * Fix NPE while loading paxos state (CASSANDRA-6211) + * cqlsh: add SHOW SESSION command (CASSANDRA-6228) +Merged from 1.2: + * (Hadoop) Require CFRR batchSize to be at least 2 (CASSANDRA-6114) * Add a warning for small LCS sstable size (CASSANDRA-6191) * Add ability to list specific KS/CF combinations in nodetool cfstats (CASSANDRA-4191) * Mark CF clean if a mutation raced the drop and got it marked dirty (CASSANDRA-5946) http://git-wip-us.apache.org/repos/asf/cassandra/blob/b4020979/src/java/org/apache/cassandra/net/MessagingService.java --
[3/6] git commit: Fix potential socket leak in connectionpool creation patch by Minh Do; reviewed by jbellis for CASSANDRA-6308
Fix potential socket leak in connectionpool creation
patch by Minh Do; reviewed by jbellis for CASSANDRA-6308
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8c240449
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8c240449
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8c240449
Branch: refs/heads/trunk
Commit: 8c2404493a6efc109cd621c02a4de6ef1f1f46aa
Parents: 8e7d728
Author: Jonathan Ellis
Authored: Wed Nov 6 12:33:45 2013 -0800
Committer: Jonathan Ellis
Committed: Wed Nov 6 12:33:45 2013 -0800
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/net/MessagingService.java | 12 +---
2 files changed, 10 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/8c240449/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index fd3af68..eab185a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -11,6 +11,7 @@
* Nodetool gets default JMX port from cassandra-env.sh (CASSANDRA-6273)
* make calculatePendingRanges asynchronous (CASSANDRA-6244)
* Remove blocking flushes in gossip thread (CASSANDRA-6297)
+ * Fix potential socket leak in connectionpool creation (CASSANDRA-6308)
1.2.11
http://git-wip-us.apache.org/repos/asf/cassandra/blob/8c240449/src/java/org/apache/cassandra/net/MessagingService.java
--
diff --git a/src/java/org/apache/cassandra/net/MessagingService.java
b/src/java/org/apache/cassandra/net/MessagingService.java
index a199e83..7e420cf 100644
--- a/src/java/org/apache/cassandra/net/MessagingService.java
+++ b/src/java/org/apache/cassandra/net/MessagingService.java
@@ -259,7 +259,7 @@ public final class MessagingService implements
MessagingServiceMBean
*/
private final ConcurrentMap
streamExecutors = new NonBlockingHashMap();
-private final NonBlockingHashMap
connectionManagers = new NonBlockingHashMap();
+private final ConcurrentMap
connectionManagers = new NonBlockingHashMap();
private static final Logger logger =
LoggerFactory.getLogger(MessagingService.class);
private static final int LOG_DROPPED_INTERVAL_IN_MS = 5000;
@@ -484,11 +484,17 @@ public final class MessagingService implements
MessagingServiceMBean
OutboundTcpConnectionPool cp = connectionManagers.get(to);
if (cp == null)
{
-connectionManagers.putIfAbsent(to, new
OutboundTcpConnectionPool(to));
-cp = connectionManagers.get(to);
+cp = new OutboundTcpConnectionPool(to);
+OutboundTcpConnectionPool existingPool =
connectionManagers.putIfAbsent(to, cp);
+if (existingPool != null)
+{
+cp.close();
+cp = existingPool;
+}
}
return cp;
}
+
public OutboundTcpConnection getConnection(InetAddress to, MessageOut msg)
{
[jira] [Updated] (CASSANDRA-6308) Thread leak caused in creating OutboundTcpConnectionPool
[ https://issues.apache.org/jira/browse/CASSANDRA-6308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Minh Do updated CASSANDRA-6308: --- Attachment: patch.txt > Thread leak caused in creating OutboundTcpConnectionPool > > > Key: CASSANDRA-6308 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6308 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Minh Do >Priority: Minor > Labels: leak, thread > Fix For: 1.2.12 > > Attachments: patch.txt > > > We have seen in one of our large clusters that there are many > OutboundTcpConnection threads having the same names. From a thread dump, > OutboundTcpConnection threads have accounted for the largest shares of the > total threads (65%+) and kept growing. > Here is a portion of a grep output for threads in which names start with > "WRITE-": > "WRITE-/10.28.131.195" daemon prio=10 tid=0x2aaac4022000 nid=0x2cb5 > waiting on condition [0x2acfbacda000] > "WRITE-/10.28.131.195" daemon prio=10 tid=0x2aaac42fe000 nid=0x2cb4 > waiting on condition [0x2acfbacad000] > "WRITE-/10.30.142.49" daemon prio=10 tid=0x4084 nid=0x2cb1 > waiting on condition [0x2acfbac8] > "WRITE-/10.6.222.233" daemon prio=10 tid=0x4083e000 nid=0x2cb0 > waiting on condition [0x2acfbac53000] > "WRITE-/10.6.222.233" daemon prio=10 tid=0x4083b800 nid=0x2caf > waiting on condition [0x2acfbac26000] > "WRITE-/10.6.222.233" daemon prio=10 tid=0x40839800 nid=0x2cae > waiting on condition [0x2acfbabf9000] > "WRITE-/10.6.222.233" daemon prio=10 tid=0x40837800 nid=0x2cad > waiting on condition [0x2acfbabcc000] > "WRITE-/10.6.222.233" daemon prio=10 tid=0x404a3800 nid=0x2cac > waiting on condition [0x2acfbab9f000] > "WRITE-/10.30.142.49" daemon prio=10 tid=0x404a1800 nid=0x2cab > waiting on condition [0x2acfbab72000] > "WRITE-/10.6.222.233" daemon prio=10 tid=0x4049f800 nid=0x2caa > waiting on condition [0x2acfbab45000] > "WRITE-/10.6.222.233" daemon prio=10 tid=0x4049e000 nid=0x2ca9 > waiting on condition [0x2acfbab18000] > "WRITE-/10.6.222.233" daemon prio=10 tid=0x4049c800 nid=0x2ca8 > waiting on condition [0x2acfbaaeb000] > "WRITE-/10.157.10.134" daemon prio=10 tid=0x4049a800 nid=0x2ca7 > waiting on condition [0x2acfbaabe000] > "WRITE-/10.6.222.233" daemon prio=10 tid=0x40498800 nid=0x2ca6 > waiting on condition [0x2acfbaa91000] > "WRITE-/10.6.222.233" daemon prio=10 tid=0x40496800 nid=0x2ca5 > waiting on condition [0x2acfbaa64000] > "WRITE-/10.6.222.233" daemon prio=10 tid=0x40717800 nid=0x2ca4 > waiting on condition [0x2acfbaa37000] > "WRITE-/10.6.222.233" daemon prio=10 tid=0x40716000 nid=0x2ca3 > waiting on condition [0x2acfbaa0a000] > "WRITE-/10.30.146.195" daemon prio=10 tid=0x40714800 nid=0x2ca2 > waiting on condition [0x2acfba9dd000] > "WRITE-/10.6.222.233" daemon prio=10 tid=0x40712800 nid=0x2ca1 > waiting on condition [0x2acfba9b] > "WRITE-/10.6.222.233" daemon prio=10 tid=0x40710800 nid=0x2ca0 > waiting on condition [0x2acfba983000] > "WRITE-/10.6.222.233" daemon prio=10 tid=0x4070e800 nid=0x2c9f > waiting on condition [0x2acfba956000] > "WRITE-/10.6.222.233" daemon prio=10 tid=0x4070d000 nid=0x2c9e > waiting on condition [0x2acfba929000] > "WRITE-/10.6.222.233" daemon prio=10 tid=0x4070b800 nid=0x2c9d > waiting on condition [0x2acfba8fc000] > "WRITE-/10.6.222.233" daemon prio=10 tid=0x4070a000 nid=0x2c9c > waiting on condition [0x2acfba8cf000] > "WRITE-/10.6.222.233" daemon prio=10 tid=0x40827000 nid=0x2c9b > waiting on condition [0x2acfba8a2000] > "WRITE-/10.6.222.233" daemon prio=10 tid=0x40825000 nid=0x2c9a > waiting on condition [0x2acfba875000] > "WRITE-/10.6.222.233" daemon prio=10 tid=0x2aaac488e000 nid=0x2c99 > waiting on condition [0x2acfba848000] > "WRITE-/10.6.222.233" daemon prio=10 tid=0x40823000 nid=0x2c98 > waiting on condition [0x2acfba81b000] > "WRITE-/10.6.222.233" daemon prio=10 tid=0x40821800 nid=0x2c97 > waiting on condition [0x2acfba7ee000] > "WRITE-/10.30.146.195" daemon prio=10 tid=0x4081f000 nid=0x2c96 > waiting on condition [0x2acfba7c1000] > "WRITE-/10.6.222.233" daemon prio=10 tid=0x4081d000 nid=0x2c95 > waiting on condition [0x2acfba794000] > "WRITE-/10.6.222.233" daemon prio=10 tid=0x4081b000 nid=0x2c94 > waiting on condition [0x2acfba767000] > "WRITE-/10.6.222.233" daemon prio=10 tid=0x2aaac488b000 nid=0x2c93 > waiting on condition [0x2acfba73a000] > "WRITE-/10.6.222.233" daemon prio=10 t
[jira] [Commented] (CASSANDRA-3578) Multithreaded commitlog
[ https://issues.apache.org/jira/browse/CASSANDRA-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815253#comment-13815253 ] Jonathan Ellis commented on CASSANDRA-3578: --- Yes, the versions correspond w/ releases. > Multithreaded commitlog > --- > > Key: CASSANDRA-3578 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3578 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Vijay >Priority: Minor > Labels: performance > Attachments: 0001-CASSANDRA-3578.patch, ComitlogStress.java, > Current-CL.png, Multi-Threded-CL.png, parallel_commit_log_2.patch > > > Brian Aker pointed out a while ago that allowing multiple threads to modify > the commitlog simultaneously (reserving space for each with a CAS first, the > way we do in the SlabAllocator.Region.allocate) can improve performance, > since you're not bottlenecking on a single thread to do all the copying and > CRC computation. > Now that we use mmap'd CommitLog segments (CASSANDRA-3411) this becomes > doable. > (moved from CASSANDRA-622, which was getting a bit muddled.) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-3578) Multithreaded commitlog
[ https://issues.apache.org/jira/browse/CASSANDRA-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815249#comment-13815249 ] Benedict commented on CASSANDRA-3578: - Ah, my mistake, it looks like this is the version that was being used but it was being compared with MessagingService.version. I would guess this is because of the slightly confusingly named CommitLogDescriptor.getMessagingVersion(). Should have looked at this part of the patch more closely. Are the versions meant to correspond to release versions? i.e. VERSION_21 -> C* 2.1 and above (so we're safe using the current version)? or should I bump to VERSION_22? > Multithreaded commitlog > --- > > Key: CASSANDRA-3578 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3578 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Vijay >Priority: Minor > Labels: performance > Attachments: 0001-CASSANDRA-3578.patch, ComitlogStress.java, > Current-CL.png, Multi-Threded-CL.png, parallel_commit_log_2.patch > > > Brian Aker pointed out a while ago that allowing multiple threads to modify > the commitlog simultaneously (reserving space for each with a CAS first, the > way we do in the SlabAllocator.Region.allocate) can improve performance, > since you're not bottlenecking on a single thread to do all the copying and > CRC computation. > Now that we use mmap'd CommitLog segments (CASSANDRA-3411) this becomes > doable. > (moved from CASSANDRA-622, which was getting a bit muddled.) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-3578) Multithreaded commitlog
[ https://issues.apache.org/jira/browse/CASSANDRA-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815240#comment-13815240 ] Jonathan Ellis commented on CASSANDRA-3578: --- We do have CommitLogDescriptor.version. > Multithreaded commitlog > --- > > Key: CASSANDRA-3578 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3578 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Vijay >Priority: Minor > Labels: performance > Attachments: 0001-CASSANDRA-3578.patch, ComitlogStress.java, > Current-CL.png, Multi-Threded-CL.png, parallel_commit_log_2.patch > > > Brian Aker pointed out a while ago that allowing multiple threads to modify > the commitlog simultaneously (reserving space for each with a CAS first, the > way we do in the SlabAllocator.Region.allocate) can improve performance, > since you're not bottlenecking on a single thread to do all the copying and > CRC computation. > Now that we use mmap'd CommitLog segments (CASSANDRA-3411) this becomes > doable. > (moved from CASSANDRA-622, which was getting a bit muddled.) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-3578) Multithreaded commitlog
[ https://issues.apache.org/jira/browse/CASSANDRA-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815229#comment-13815229 ] Benedict commented on CASSANDRA-3578: - I left Vijay's solution mostly unchanged here (bar a minor fix) - sync() writes the end position to the start of the commit log before each buffer.force(). The END_OF_SEGMENT_MARKER is still read for backwards compatibility; MessagingService.VERSION is used to determine if we should read the start marker, although I'm not familiar enough with this property to know if this is suitable for deciding this. We might want to consider versioning the commit logs themselves, although we could do that easily with the name pattern. > Multithreaded commitlog > --- > > Key: CASSANDRA-3578 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3578 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Vijay >Priority: Minor > Labels: performance > Attachments: 0001-CASSANDRA-3578.patch, ComitlogStress.java, > Current-CL.png, Multi-Threded-CL.png, parallel_commit_log_2.patch > > > Brian Aker pointed out a while ago that allowing multiple threads to modify > the commitlog simultaneously (reserving space for each with a CAS first, the > way we do in the SlabAllocator.Region.allocate) can improve performance, > since you're not bottlenecking on a single thread to do all the copying and > CRC computation. > Now that we use mmap'd CommitLog segments (CASSANDRA-3411) this becomes > doable. > (moved from CASSANDRA-622, which was getting a bit muddled.) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6306) "No indexed columns" error when using equals on compound primary key
[
https://issues.apache.org/jira/browse/CASSANDRA-6306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815180#comment-13815180
]
Aaron Westendorf commented on CASSANDRA-6306:
-
That's annoying, I swear I tried that and had a similar error, but I can't
repeat that now. Apologies for the noob question.
> "No indexed columns" error when using equals on compound primary key
>
>
> Key: CASSANDRA-6306
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6306
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 2.0.2
>Reporter: Aaron Westendorf
>
> Given the following table and secondary index:
> {noformat}
> TABLE series (
> name text,
> interval text,
> i_time bigint,
> insert_time float,
> r_time bigint,
> value float,
> PRIMARY KEY (name, interval)
> );
> CREATE INDEX series_i_time_idx ON series (i_time);
> {noformat}
> I get the following error
> {noformat}
> cqlsh:kairos> select i_time, r_time, value from series where name='test' and
> interval='hour' and i_time>=50;
> Bad Request: No indexed columns present in by-columns clause with Equal
> operator
> {noformat}
> I'm new to Cassandra, but of what I've read, this query should work.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (CASSANDRA-3578) Multithreaded commitlog
[
https://issues.apache.org/jira/browse/CASSANDRA-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815159#comment-13815159
]
Jonathan Ellis commented on CASSANDRA-3578:
---
How does your branch deal with replaying?
I note that replay still checks for {{END_OF_SEGMENT_MARKER}} but nothing
actually writes that value anymore.
> Multithreaded commitlog
> ---
>
> Key: CASSANDRA-3578
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3578
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Jonathan Ellis
>Assignee: Vijay
>Priority: Minor
> Labels: performance
> Attachments: 0001-CASSANDRA-3578.patch, ComitlogStress.java,
> Current-CL.png, Multi-Threded-CL.png, parallel_commit_log_2.patch
>
>
> Brian Aker pointed out a while ago that allowing multiple threads to modify
> the commitlog simultaneously (reserving space for each with a CAS first, the
> way we do in the SlabAllocator.Region.allocate) can improve performance,
> since you're not bottlenecking on a single thread to do all the copying and
> CRC computation.
> Now that we use mmap'd CommitLog segments (CASSANDRA-3411) this becomes
> doable.
> (moved from CASSANDRA-622, which was getting a bit muddled.)
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (CASSANDRA-6177) remove all sleeps in the dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-6177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cathy Daw updated CASSANDRA-6177: - Assignee: (was: Daniel Meyer) > remove all sleeps in the dtests > --- > > Key: CASSANDRA-6177 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6177 > Project: Cassandra > Issue Type: Test >Reporter: Brandon Williams > > The dtests use a ton of sleep calls for various things, most of which is > guessing if Cassandra has finished doing something or not. Guessing is > problematic and shouldn't be necessary -- a prime example of this is creating > a ks or cf. When done over cql, we sleep and hope it's done propagating, but > when done over thrift we actually check for schema agreement. We should be > able to eliminate the sleeps and reliably detect when it's time for the next > step programmatically. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (CASSANDRA-6308) Thread leak caused in creating OutboundTcpConnectionPool
Minh Do created CASSANDRA-6308: -- Summary: Thread leak caused in creating OutboundTcpConnectionPool Key: CASSANDRA-6308 URL: https://issues.apache.org/jira/browse/CASSANDRA-6308 Project: Cassandra Issue Type: Bug Components: Core Reporter: Minh Do Priority: Minor Fix For: 1.2.12, 2.0.3 We have seen in one of our large clusters that there are many OutboundTcpConnection threads having the same names. From a thread dump, OutboundTcpConnection threads have accounted for the largest shares of the total threads (65%+) and kept growing. Here is a portion of a grep output for threads in which names start with "WRITE-": "WRITE-/10.28.131.195" daemon prio=10 tid=0x2aaac4022000 nid=0x2cb5 waiting on condition [0x2acfbacda000] "WRITE-/10.28.131.195" daemon prio=10 tid=0x2aaac42fe000 nid=0x2cb4 waiting on condition [0x2acfbacad000] "WRITE-/10.30.142.49" daemon prio=10 tid=0x4084 nid=0x2cb1 waiting on condition [0x2acfbac8] "WRITE-/10.6.222.233" daemon prio=10 tid=0x4083e000 nid=0x2cb0 waiting on condition [0x2acfbac53000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x4083b800 nid=0x2caf waiting on condition [0x2acfbac26000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x40839800 nid=0x2cae waiting on condition [0x2acfbabf9000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x40837800 nid=0x2cad waiting on condition [0x2acfbabcc000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x404a3800 nid=0x2cac waiting on condition [0x2acfbab9f000] "WRITE-/10.30.142.49" daemon prio=10 tid=0x404a1800 nid=0x2cab waiting on condition [0x2acfbab72000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x4049f800 nid=0x2caa waiting on condition [0x2acfbab45000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x4049e000 nid=0x2ca9 waiting on condition [0x2acfbab18000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x4049c800 nid=0x2ca8 waiting on condition [0x2acfbaaeb000] "WRITE-/10.157.10.134" daemon prio=10 tid=0x4049a800 nid=0x2ca7 waiting on condition [0x2acfbaabe000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x40498800 nid=0x2ca6 waiting on condition [0x2acfbaa91000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x40496800 nid=0x2ca5 waiting on condition [0x2acfbaa64000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x40717800 nid=0x2ca4 waiting on condition [0x2acfbaa37000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x40716000 nid=0x2ca3 waiting on condition [0x2acfbaa0a000] "WRITE-/10.30.146.195" daemon prio=10 tid=0x40714800 nid=0x2ca2 waiting on condition [0x2acfba9dd000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x40712800 nid=0x2ca1 waiting on condition [0x2acfba9b] "WRITE-/10.6.222.233" daemon prio=10 tid=0x40710800 nid=0x2ca0 waiting on condition [0x2acfba983000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x4070e800 nid=0x2c9f waiting on condition [0x2acfba956000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x4070d000 nid=0x2c9e waiting on condition [0x2acfba929000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x4070b800 nid=0x2c9d waiting on condition [0x2acfba8fc000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x4070a000 nid=0x2c9c waiting on condition [0x2acfba8cf000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x40827000 nid=0x2c9b waiting on condition [0x2acfba8a2000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x40825000 nid=0x2c9a waiting on condition [0x2acfba875000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x2aaac488e000 nid=0x2c99 waiting on condition [0x2acfba848000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x40823000 nid=0x2c98 waiting on condition [0x2acfba81b000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x40821800 nid=0x2c97 waiting on condition [0x2acfba7ee000] "WRITE-/10.30.146.195" daemon prio=10 tid=0x4081f000 nid=0x2c96 waiting on condition [0x2acfba7c1000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x4081d000 nid=0x2c95 waiting on condition [0x2acfba794000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x4081b000 nid=0x2c94 waiting on condition [0x2acfba767000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x2aaac488b000 nid=0x2c93 waiting on condition [0x2acfba73a000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x40819000 nid=0x2c92 waiting on condition [0x2acfba70d000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x407f9000 nid=0x2c91 waiting on condition [0x2acfba6e] "WRITE-/10.6.222.233" daemon prio=10 tid=0x407f7000 nid=0x2c90 waiting on condition [0x2acfba6b3000] "WRITE-/10.6.222.233" daemon prio=10 tid=0x407f5000 nid=0x2c8f waiting on condition [0x2acfba68600
[jira] [Created] (CASSANDRA-6307) Switch cqlsh from cassandra-dbapi2 to python-driver
Aleksey Yeschenko created CASSANDRA-6307: Summary: Switch cqlsh from cassandra-dbapi2 to python-driver Key: CASSANDRA-6307 URL: https://issues.apache.org/jira/browse/CASSANDRA-6307 Project: Cassandra Issue Type: Improvement Reporter: Aleksey Yeschenko Priority: Minor Fix For: 2.1 python-driver is hitting 1.0 soon. cassandra-dbapi2 development has stalled. It's time to switch cqlsh to native protocol and cassandra-dbapi2, especially now that 1. Some CQL3 things are not supported by Thrift transport 2. cqlsh no longer has to support CQL2 (dropped in 2.0) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-3578) Multithreaded commitlog
[ https://issues.apache.org/jira/browse/CASSANDRA-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815069#comment-13815069 ] Benedict commented on CASSANDRA-3578: - I liked the basic approach of your patch, Vijay. There were a number of problems unaddressed: - sync() would mark things as flushed to disk that weren't, which would result in log messages never being persisted - BatchCommitLogExecutor would ack the first message before syncing it - markClean would mark current segment entirely clean, as opposed to up to latest ReplayPosition; guessing this was for testing, but has performance implications. Fixed. - log replay was broken, minor fix - checksum calculation was broken, minor fix - sync() could be called simultaneously at shutdown, and wasn't threadsafe - could have persisted garbled end pointer at start of log I've made the following changes: - data is marked as written in CLS after serializing to the buffer, so sync() only advances the replay position to the most recent contiguously written point in the buffer. Initially I used a skip list for this, but this seemed to be a bit of a bottleneck. Now I buffer write positions in the order they arrive, and merge them periodically (generally asynchronously, though blocking if we run out of room to buffer). Possibly needs a slight tweak to absolutely guarantee it can never run out of buffer space, but as it stands it's pretty much impossible due to compaction of any adjacent records. - Periodic and Batch CLEs are now the same class, with the only difference being batch commit waits for a signal from the CommitLogSegment that it has been syncd to disk - PeriodicCLE is now fixed rate rather than fixed delay, and blocks any writes if the previous sync hasn't completed after one commit log poll interval. This is to maintain similar guarantees as before, although potentially we should commit every 1/2 configured period, as we can lose up to 2 periods of data if the previous sync is failing badly. - Moved flushOldestKeyspaces into the CLE thread, and it now flushes keyspaces from multiple old log files, not just the oldest, up to the number needed to bring us below our limit. Since we can rapidly add many log files now, this seemed necessary I've also made some changes to encapsulate concurrency better, to minimise risk of bugs. Allocation is all done inside CLA, as opposed to split between CLA and CL. CLA also now allocates new segments as soon as the last reserve segment is used, as opposed to every 100ms, in case we ever have a situation where we exhaust segments in < 100ms. After making all of these changes, I actually found very little improvement in performance. It turns out this was because the point of contention is moving from the CL to the switchLock, and I still see huge spikes when waiting for a flush here. I've tested these changes with my in progress patch for [6271|https://issues.apache.org/jira/browse/CASSANDRA-6271] and found up to 2x performance boost for high thread counts. It's quite likely a patch for [5549|https://issues.apache.org/jira/browse/CASSANDRA-5549] will help further. [https://github.com/belliottsmith/cassandra/tree/iss-3578] [https://github.com/belliottsmith/cassandra/tree/iss-3578+6271] - note this is far from prod ready, just useful for performance testing. might only work with stress. > Multithreaded commitlog > --- > > Key: CASSANDRA-3578 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3578 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Vijay >Priority: Minor > Labels: performance > Attachments: 0001-CASSANDRA-3578.patch, ComitlogStress.java, > Current-CL.png, Multi-Threded-CL.png, parallel_commit_log_2.patch > > > Brian Aker pointed out a while ago that allowing multiple threads to modify > the commitlog simultaneously (reserving space for each with a CAS first, the > way we do in the SlabAllocator.Region.allocate) can improve performance, > since you're not bottlenecking on a single thread to do all the copying and > CRC computation. > Now that we use mmap'd CommitLog segments (CASSANDRA-3411) this becomes > doable. > (moved from CASSANDRA-622, which was getting a bit muddled.) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (CASSANDRA-6305) cqlsh support for User Types
[ https://issues.apache.org/jira/browse/CASSANDRA-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-6305: - Labels: cqlsh (was: ) > cqlsh support for User Types > > > Key: CASSANDRA-6305 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6305 > Project: Cassandra > Issue Type: New Feature >Reporter: Aleksey Yeschenko > Labels: cqlsh > Fix For: 2.1 > > > We need cqlsh support for several things: > 1. Autocomplete for UPDATE/INSERT/SELECT > 2. Autocomplete for ALTER TYPE/CREATE TYPE/DROP TYPE > 3. Proper decoding of UserType values (currently showing the encoded blob) > 4. Support UserTypes in DESCRIBE > 5. Separate DESCRIBE TYPES|TYPE cmds -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Resolved] (CASSANDRA-6306) "No indexed columns" error when using equals on compound primary key
[
https://issues.apache.org/jira/browse/CASSANDRA-6306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko resolved CASSANDRA-6306.
--
Resolution: Invalid
Can only use equality op (=) on 2i columns. This query would work if instead of
creating an index you make i_time part of the primary key, though.
> "No indexed columns" error when using equals on compound primary key
>
>
> Key: CASSANDRA-6306
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6306
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 2.0.2
>Reporter: Aaron Westendorf
>
> Given the following table and secondary index:
> {noformat}
> TABLE series (
> name text,
> interval text,
> i_time bigint,
> insert_time float,
> r_time bigint,
> value float,
> PRIMARY KEY (name, interval)
> );
> CREATE INDEX series_i_time_idx ON series (i_time);
> {noformat}
> I get the following error
> {noformat}
> cqlsh:kairos> select i_time, r_time, value from series where name='test' and
> interval='hour' and i_time>=50;
> Bad Request: No indexed columns present in by-columns clause with Equal
> operator
> {noformat}
> I'm new to Cassandra, but of what I've read, this query should work.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (CASSANDRA-6305) cqlsh support for User Types
[ https://issues.apache.org/jira/browse/CASSANDRA-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815038#comment-13815038 ] Aleksey Yeschenko commented on CASSANDRA-6305: -- [~mishail] Mikhail, would you be interested in handling this? > cqlsh support for User Types > > > Key: CASSANDRA-6305 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6305 > Project: Cassandra > Issue Type: New Feature >Reporter: Aleksey Yeschenko > Fix For: 2.1 > > > We need cqlsh support for several things: > 1. Autocomplete for UPDATE/INSERT/SELECT > 2. Autocomplete for ALTER TYPE/CREATE TYPE/DROP TYPE > 3. Proper decoding of UserType values (currently showing the encoded blob) > 4. Support UserTypes in DESCRIBE > 5. Separate DESCRIBE TYPES|TYPE cmds -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (CASSANDRA-6306) "No indexed columns" error when using equals on compound primary key
Aaron Westendorf created CASSANDRA-6306:
---
Summary: "No indexed columns" error when using equals on compound
primary key
Key: CASSANDRA-6306
URL: https://issues.apache.org/jira/browse/CASSANDRA-6306
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Cassandra 2.0.2
Reporter: Aaron Westendorf
Given the following table and secondary index:
{noformat}
TABLE series (
name text,
interval text,
i_time bigint,
insert_time float,
r_time bigint,
value float,
PRIMARY KEY (name, interval)
);
CREATE INDEX series_i_time_idx ON series (i_time);
{noformat}
I get the following error
{noformat}
cqlsh:kairos> select i_time, r_time, value from series where name='test' and
interval='hour' and i_time>=50;
Bad Request: No indexed columns present in by-columns clause with Equal operator
{noformat}
I'm new to Cassandra, but of what I've read, this query should work.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (CASSANDRA-5590) User defined types for CQL3
[
https://issues.apache.org/jira/browse/CASSANDRA-5590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13814979#comment-13814979
]
Aleksey Yeschenko commented on CASSANDRA-5590:
--
Created CASSANDRA-6305 to handle cqlsh and CASSANDRA-6304 to deal with
authorization.
> User defined types for CQL3
> ---
>
> Key: CASSANDRA-5590
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5590
> Project: Cassandra
> Issue Type: New Feature
> Components: API, Core
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 2.1
>
> Attachments: ocd-and-corrections-patch.txt
>
>
> A typical use case for a collection could be to store a bunch of addresses in
> a user profile. An address could typically be composed of a few properties:
> say a street, a city, a postal code and maybe a few phone numbers associated
> to it.
> To model that currently with collections, you might use a {{map blob>}}, where the map key could be a string identifying the address, and the
> value would be all the infos of an address serialized manually (you can use
> {{text}} instead of {{blob}} and shove everything in a string if you prefer
> but the principle is the same).
> This ticket suggests to make this more user friendly by allowing:
> {noformat}
> CREATE TYPE address (
> street text,
> city text,
> zip_code int,
> phones set
> )
> CREATE TABLE users (
> id uuid PRIMARY KEY,
> name text,
> addresses map
> )
> {noformat}
> Under the hood, that type declaration would just be metadata on top of
> CompositeType (which does mean a limitation would be that we wouldn't allow
> re-ordering or removal of fields in a custom TYPE). Namely, the {{address}}
> type would be in practice a {{CompositeType(UTF8Type, UTF8Type, Int32Type,
> SetType(UTF8Type))}} + some metadata that records the name of each component.
> In other words, this would mostly be user-friendly syntactic sugar to create
> composite blobs.
> I'll note that this would also be useful outside collections, as it might
> sometimes be more efficient/useful to have such simple composite blob. For
> instance, you could imagine to have a:
> {noformat}
> CREATE TYPE fullname (
> firstname text,
> lastname text
> )
> {noformat}
> and to rewrite the {{users}} table above as
> {noformat}
> CREATE TABLE users (
> id uuid PRIMARY KEY,
> name fullname,
> addresses map
> )
> {noformat}
> In terms of inserts we'd need a syntax for those new "struct". Could be:
> {noformat}
> INSERT INTO users (id, name)
>VALUES (2ad..., { firstname: 'Paul', lastname: 'smith'});
> UPDATE users
>SET addresses = address + { 'home': { street: '...', city: 'SF', zip_code:
> 94102, phones: {} } }
>WHERE id=2ad...;
> {noformat}
> where the difference with a map is that the "key" would be a column name (in
> the CQL3 sense), not a value/literal. Though we might find that a bit
> confusing and find some other syntax.
> On the query side, we could optionally allow things like:
> {noformat}
> SELECT name.firstname, name.lastname FROM users WHERE id=2ad...;
> {noformat}
> One open question however is what type do we send back in the result set
> for a query like:
> {noformat}
> SELECT name FROM users WHERE id=2ad...;
> {noformat}
> We could:
> # return just that it's the user defined type named {{address}}, but that
> imply the client has to query the cluster metadata to find out the definition
> of the type.
> # return the full definition of the type every time.
> I also note that client side, it might be a tad harder to support such types
> cleanly in statically type languages than in dynamically typed ones, but
> that's not the end of the world either.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Resolved] (CASSANDRA-4043) RecentBloomFilterFalseRatio and RecentBloomFilterFalsePositives reset each other
[
https://issues.apache.org/jira/browse/CASSANDRA-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tyler Hobbs resolved CASSANDRA-4043.
Resolution: Won't Fix
bq. IMO wontfixing this is reasonable since the Recent* metrics are deprecated.
Agreed, resolving as Won't Fix.
> RecentBloomFilterFalseRatio and RecentBloomFilterFalsePositives reset each
> other
>
>
> Key: CASSANDRA-4043
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4043
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Affects Versions: 1.0.8
>Reporter: Tyler Hobbs
>Assignee: Tyler Hobbs
>Priority: Trivial
> Labels: jmx
>
> If either of the ColumnFamily JMX attributes {{RecentBloomFilterFalseRatio}}
> or {{RecentBloomFilterFalsePositives}} are read, both are reset. This means
> if you try to read both attributes at the same time (like jconsole does, for
> example), one of them is guaranteed to be 0.
> The solution might be that we store a separate false positives counter for
> the ratio and the normal count and reset them separately. Some refactoring
> should be done at the same time so that the BloomFilterTracker calculates the
> false positive ratio itself instead of having DataTracker fetch both counters
> and calculate the ratio.
> On a related note, why does nodetool not use the Recent versions of the bloom
> filter metrics?
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Created] (CASSANDRA-6305) cqlsh support for User Types
Aleksey Yeschenko created CASSANDRA-6305: Summary: cqlsh support for User Types Key: CASSANDRA-6305 URL: https://issues.apache.org/jira/browse/CASSANDRA-6305 Project: Cassandra Issue Type: New Feature Reporter: Aleksey Yeschenko Fix For: 2.1 We need cqlsh support for several things: 1. Autocomplete for UPDATE/INSERT/SELECT 2. Autocomplete for ALTER TYPE/CREATE TYPE/DROP TYPE 3. Proper decoding of UserType values (currently showing the encoded blob) 4. Support UserTypes in DESCRIBE 5. Separate DESCRIBE TYPES|TYPE cmds -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (CASSANDRA-6304) Better handling of authorization for User Types
Aleksey Yeschenko created CASSANDRA-6304: Summary: Better handling of authorization for User Types Key: CASSANDRA-6304 URL: https://issues.apache.org/jira/browse/CASSANDRA-6304 Project: Cassandra Issue Type: New Feature Reporter: Aleksey Yeschenko Assignee: Aleksey Yeschenko Fix For: 2.1 Currently, we require CREATE/ALTER/DROP on ALL KEYSPACES, which is a bit excessive, and not entirely correct (but is the best we can do atm). We should: 1. create a new IResource implementation for user types (TypeResource) 2. extend CQL3 GRANT/REVOKE to allow CREATE/ALTER/DROP on (ALL TYPES|TYPE ) 3. require CREATE/ALTER/DROP permissions instead of requiring all keyspace access We could (should?) optionally require ALTER permission on the columnfamilies affected by ALTER TYPE. Not sure about this? We also don't currently allow dropping a type that's in use by a CF. So someone might start using a type in the cf, and the 'owner' of the type would not be able to drop it. So we should either add some kind of USE permission for types, or make it possible to drop a type that's currently in use. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (CASSANDRA-6268) Poor performance of Hadoop if any DC is using VNodes
[ https://issues.apache.org/jira/browse/CASSANDRA-6268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Piotr Kołaczkowski updated CASSANDRA-6268: -- Attachment: 6268-cassandra-2.0.txt Attached a patch for C* 2.0. > Poor performance of Hadoop if any DC is using VNodes > > > Key: CASSANDRA-6268 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6268 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Piotr Kołaczkowski >Assignee: Piotr Kołaczkowski > Attachments: 6268-2.txt, 6268-cassandra-2.0.txt > > > Some customers are complaining about huge number of splits in Hadoop caused > by VNodes. Disabling vnodes only in Hadoop DC does not fix it. Splits are > generated from the results of describe_ring, which returns a huge number of > ranges anyways, and doesn't take into account that there will be huge number > of consecutive ranges residing on the nodes we'd like the M/R job to be run. > The proposed fix: > 1. allows for specifying the DC(s) the Hadoop job should be run in (in DSE - > defaults to all Hadoop DCs) > 2. merges consecutive ranges before generating Hadoop splits, so we don't > have artificial range splitting caused by vnodes in the other DCs > For non-DSE users this feature is turned off by default and doesn't change > the old behaviour. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6268) Poor performance of Hadoop if any DC is using VNodes
[ https://issues.apache.org/jira/browse/CASSANDRA-6268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13814832#comment-13814832 ] Piotr Kołaczkowski commented on CASSANDRA-6268: --- There was a bug in the patch that caused describe_local_ring to be swapped with describe_ring. Attached fixed patch. > Poor performance of Hadoop if any DC is using VNodes > > > Key: CASSANDRA-6268 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6268 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Piotr Kołaczkowski >Assignee: Piotr Kołaczkowski > Attachments: 6268-2.txt > > > Some customers are complaining about huge number of splits in Hadoop caused > by VNodes. Disabling vnodes only in Hadoop DC does not fix it. Splits are > generated from the results of describe_ring, which returns a huge number of > ranges anyways, and doesn't take into account that there will be huge number > of consecutive ranges residing on the nodes we'd like the M/R job to be run. > The proposed fix: > 1. allows for specifying the DC(s) the Hadoop job should be run in (in DSE - > defaults to all Hadoop DCs) > 2. merges consecutive ranges before generating Hadoop splits, so we don't > have artificial range splitting caused by vnodes in the other DCs > For non-DSE users this feature is turned off by default and doesn't change > the old behaviour. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (CASSANDRA-6268) Poor performance of Hadoop if any DC is using VNodes
[ https://issues.apache.org/jira/browse/CASSANDRA-6268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Piotr Kołaczkowski updated CASSANDRA-6268: -- Attachment: 6268-2.txt > Poor performance of Hadoop if any DC is using VNodes > > > Key: CASSANDRA-6268 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6268 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Piotr Kołaczkowski >Assignee: Piotr Kołaczkowski > Attachments: 6268-2.txt > > > Some customers are complaining about huge number of splits in Hadoop caused > by VNodes. Disabling vnodes only in Hadoop DC does not fix it. Splits are > generated from the results of describe_ring, which returns a huge number of > ranges anyways, and doesn't take into account that there will be huge number > of consecutive ranges residing on the nodes we'd like the M/R job to be run. > The proposed fix: > 1. allows for specifying the DC(s) the Hadoop job should be run in (in DSE - > defaults to all Hadoop DCs) > 2. merges consecutive ranges before generating Hadoop splits, so we don't > have artificial range splitting caused by vnodes in the other DCs > For non-DSE users this feature is turned off by default and doesn't change > the old behaviour. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (CASSANDRA-6268) Poor performance of Hadoop if any DC is using VNodes
[ https://issues.apache.org/jira/browse/CASSANDRA-6268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Piotr Kołaczkowski updated CASSANDRA-6268: -- Attachment: (was: 6268.txt) > Poor performance of Hadoop if any DC is using VNodes > > > Key: CASSANDRA-6268 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6268 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Piotr Kołaczkowski >Assignee: Piotr Kołaczkowski > Attachments: 6268-2.txt > > > Some customers are complaining about huge number of splits in Hadoop caused > by VNodes. Disabling vnodes only in Hadoop DC does not fix it. Splits are > generated from the results of describe_ring, which returns a huge number of > ranges anyways, and doesn't take into account that there will be huge number > of consecutive ranges residing on the nodes we'd like the M/R job to be run. > The proposed fix: > 1. allows for specifying the DC(s) the Hadoop job should be run in (in DSE - > defaults to all Hadoop DCs) > 2. merges consecutive ranges before generating Hadoop splits, so we don't > have artificial range splitting caused by vnodes in the other DCs > For non-DSE users this feature is turned off by default and doesn't change > the old behaviour. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (CASSANDRA-6303) cassandra-cli should exit with error-exit status when it can't connect to cluster
Rushan Shaymardanov created CASSANDRA-6303: -- Summary: cassandra-cli should exit with error-exit status when it can't connect to cluster Key: CASSANDRA-6303 URL: https://issues.apache.org/jira/browse/CASSANDRA-6303 Project: Cassandra Issue Type: Bug Components: Tools Reporter: Rushan Shaymardanov Priority: Minor When casasndra-cli can't connect to cassandra, it returns success: # echo "use my_keyspace;" | cassandra-cli -h myhost -f /dev/stdin org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused at org.apache.thrift.transport.TSocket.open(TSocket.java:183) at org.apache.thrift.transport.TFramedTransport.open(TFramedTransport.java:81) at org.apache.cassandra.cli.CliMain.connect(CliMain.java:70) at org.apache.cassandra.cli.CliMain.main(CliMain.java:246) Caused by: java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.thrift.transport.TSocket.open(TSocket.java:178) ... 3 more Exception connecting to boney/9160. Reason: Connection refused. Not connected to a cassandra instance. [root@boney ~]# echo $? 0 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (CASSANDRA-6303) cassandra-cli should exit with error-exit status when it can't connect to cluster
[ https://issues.apache.org/jira/browse/CASSANDRA-6303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rushan Shaymardanov updated CASSANDRA-6303: --- Description: When casasndra-cli can't connect to cassandra, it returns success: [root@myhost ~] # echo "use my_keyspace;" | cassandra-cli -h myhost -f /dev/stdin org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused at org.apache.thrift.transport.TSocket.open(TSocket.java:183) at org.apache.thrift.transport.TFramedTransport.open(TFramedTransport.java:81) at org.apache.cassandra.cli.CliMain.connect(CliMain.java:70) at org.apache.cassandra.cli.CliMain.main(CliMain.java:246) Caused by: java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.thrift.transport.TSocket.open(TSocket.java:178) ... 3 more Exception connecting to boney/9160. Reason: Connection refused. Not connected to a cassandra instance. [root@myhost ~]# echo $? 0 was: When casasndra-cli can't connect to cassandra, it returns success: # echo "use my_keyspace;" | cassandra-cli -h myhost -f /dev/stdin org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused at org.apache.thrift.transport.TSocket.open(TSocket.java:183) at org.apache.thrift.transport.TFramedTransport.open(TFramedTransport.java:81) at org.apache.cassandra.cli.CliMain.connect(CliMain.java:70) at org.apache.cassandra.cli.CliMain.main(CliMain.java:246) Caused by: java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.thrift.transport.TSocket.open(TSocket.java:178) ... 3 more Exception connecting to boney/9160. Reason: Connection refused. Not connected to a cassandra instance. [root@boney ~]# echo $? 0 > cassandra-cli should exit with error-exit status when it can't connect to > cluster > - > > Key: CASSANDRA-6303 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6303 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Rushan Shaymardanov >Priority: Minor > > When casasndra-cli can't connect to cassandra, it returns success: > [root@myhost ~] # echo "use my_keyspace;" | cassandra-cli -h myhost -f > /dev/stdin > org.apache.thrift.transport.TTransportException: java.net.ConnectException: > Connection refused > at org.apache.thrift.transport.TSocket.open(TSocket.java:183) > at > org.apache.thrift.transport.TFramedTransport.open(TFramedTransport.java:81) > at org.apache.cassandra.cli.CliMain.connect(CliMain.java:70) > at org.apache.cassandra.cli.CliMain.main(CliMain.java:246) > Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) > at > java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) > at > java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) > at > java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > at java.net.Socket.connect(Socket.java:579) > at org.apache.thrift.transport.TSocket.open(TSocket.java:178) > ... 3 more > Exception connecting to boney/9160. Reason: Connection refused. > Not connected to a cassandra instance. > [root@myhost ~]# echo $? > 0 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Assigned] (CASSANDRA-5428) CQL3 don't validate that collections haven't more than 64K elements
[ https://issues.apache.org/jira/browse/CASSANDRA-5428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne reassigned CASSANDRA-5428: --- Assignee: Sylvain Lebresne > CQL3 don't validate that collections haven't more than 64K elements > --- > > Key: CASSANDRA-5428 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5428 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Sylvain Lebresne >Assignee: Sylvain Lebresne >Priority: Minor > > This is somewhat similar to CASSANDRA-5355 but with a twist. When we > serialize collections, not only does the size of the elements is limited to > 64K, but the number of elements is too because it is also an unsigned short. > Now the same argument than in CASSANDRA-5355 that collections are "places to > denormalize small amounts of data" is true here too. So the fact that > collections are limited to 64K elements is something I could live with. > However, we don't validate that no more than 64K elements are inserted. And > in fact, we can't validate it if the elements are added one by one. > So in practice, you can insert more than 64K elements, but if you try to read > it, you will only get back some subset of the collection. And the number of > elements returned will correspond to the 2 last bytes of the real size (so a > collection of 65536 elements will be returned as 1 element). Imo, that's more > problematic. > So since unfortunately we can't validate this at insertion, I suggest that as > a first step we: > # document that limitation (in http://cassandra.apache.org/doc/cql3/CQL.html > typically) > # when we read a collection that has > 64K elements, we detect it and when > serializing that for the client, we: > ** return as much as we can, i.e. the 64K first ones > ** log a warning that something is wrong > On the longer term, for 2.0, maybe we should just change the serialization > format and use an int for the collection size, using an unsigned short was > probably misguided. Of course that changes said serialization format so we > have to bump the native protocol version for that (and thus can't do that in > 1.2). -- This message was sent by Atlassian JIRA (v6.1#6144)
