date:20140905


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122597#comment-14122597
 ] 

Sylvain Lebresne commented on CASSANDRA-7879:
-

+1

 Internal c* datatypes exposed via jmx and method signatures changed in 2.1
 --

 Key: CASSANDRA-7879
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7879
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Philip S Doctor
Assignee: Marcus Eriksson
Priority: Trivial
 Fix For: 2.1.0

 Attachments: 0001-dont-expose-internal-structures-over-jmx.patch


 In c* 2.0 the StorageService jmx has this signature:
 {noformat}
 public void forceKeyspaceCleanup 
 {noformat}
 but in 2.1 RC6 it is this
 {noformat} 
 public CompactionManager.AllSSTableOpStatus forceKeyspaceCleanup
 {noformat}
 This makes any consumer have a problem with the unmarshalling and should be a 
 native java type.
 There may be further instances, the jmx api should probably be audited for 
 similar instances.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-7882) Allow disabling slab allocation for off-heap memtables

Jay Patel created CASSANDRA-7882:


 Summary: Allow disabling slab allocation for off-heap memtables
 Key: CASSANDRA-7882
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7882
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jay Patel
 Fix For: 2.1.1


CASSANDRA-5935 allows option to disable region-based allocation for on-heap 
memtables but there is no option to disable it for off-heap memtables 
(memtable_allocation_type: offheap_objects). 

Disabling region-based allocation will allow us to pack more tables in the 
schema since minimum of 1MB region won't be allocated per table. Downside can 
be more fragmentation which should be controllable by using better allocator 
like JEMalloc.

How about below option in yaml?:
memtable_allocation_type: unslabbed_offheap_objects

Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7882) Allow disabling slab allocation for off-heap memtables


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122602#comment-14122602
 ] 

Jay Patel commented on CASSANDRA-7882:
--

I've added code for this in my loca env, and done some testing. Reduction in 
allocated memory is significant with higher number of tables (1000s to 10s of 
thousands) in a cluster. By default region-based allocation should be always-on 
but providing this option will allow flexibility for packing more tables.

I'm fine if we should keep it undocumented like CASSANDRA-5935.

 Allow disabling slab allocation for off-heap memtables
 --

 Key: CASSANDRA-7882
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7882
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jay Patel
 Fix For: 2.1.1


 CASSANDRA-5935 allows option to disable region-based allocation for on-heap 
 memtables but there is no option to disable it for off-heap memtables 
 (memtable_allocation_type: offheap_objects). 
 Disabling region-based allocation will allow us to pack more tables in the 
 schema since minimum of 1MB region won't be allocated per table. Downside can 
 be more fragmentation which should be controllable by using better allocator 
 like JEMalloc.
 How about below option in yaml?:
 memtable_allocation_type: unslabbed_offheap_objects
 Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-7882) Allow disabling slab allocation for off-heap memtables

[
https://issues.apache.org/jira/browse/CASSANDRA-7882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122602#comment-14122602
]

Jay Patel edited comment on CASSANDRA-7882 at 9/5/14 8:16 AM:
--

I've added code for this in my local env, and done some testing. Reduction in
allocated memory is significant with higher number of tables (1000s to 10s of
thousands) in a cluster. By default region-based allocation should be always-on
but providing this option will allow flexibility for packing more tables.

I'm fine if we should keep it undocumented like CASSANDRA-5935.

was (Author: pateljay3001):
I've added code for this in my loca env, and done some testing. Reduction in
allocated memory is significant with higher number of tables (1000s to 10s of
thousands) in a cluster. By default region-based allocation should be always-on
but providing this option will allow flexibility for packing more tables.

I'm fine if we should keep it undocumented like CASSANDRA-5935.

Allow disabling slab allocation for off-heap memtables
--

Key: CASSANDRA-7882
URL: https://issues.apache.org/jira/browse/CASSANDRA-7882
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Jay Patel
Fix For: 2.1.1

CASSANDRA-5935 allows option to disable region-based allocation for on-heap
memtables but there is no option to disable it for off-heap memtables
(memtable_allocation_type: offheap_objects).
Disabling region-based allocation will allow us to pack more tables in the
schema since minimum of 1MB region won't be allocated per table. Downside can
be more fragmentation which should be controllable by using better allocator
like JEMalloc.
How about below option in yaml?:
memtable_allocation_type: unslabbed_offheap_objects
Thanks.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7882) Allow disabling slab allocation for off-heap memtables

[
https://issues.apache.org/jira/browse/CASSANDRA-7882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122609#comment-14122609
]

Benedict commented on CASSANDRA-7882:
-

There's a slight issue with this approach, which is the one of the main reasons
I aborted including this in 2.1, namely that there is no easy way to establish
_how much memory_ is allocated by these allocators. i.e., the overheads are
opaque, and likely significant. 16-byte+ overheads, with padding to nearest
8/16-bytes is quite likely, but we have absolutely no way of knowing, and it
will vary. These overheads are likely to be a significant portion of the total
amount allocated, and so we are unlikely to obey the configured memory
constraints, which could result in an unstable server.

If we want to solve the problem of many tables, there are other potential
approaches, such as logarithmically scaling the size of the slab we allocate
from, say, 1K for the first allocation against any memtable, up to the current
1Mb. This should prevent the overheads from being painful.

Allow disabling slab allocation for off-heap memtables
--

Key: CASSANDRA-7882
URL: https://issues.apache.org/jira/browse/CASSANDRA-7882
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Jay Patel
Fix For: 2.1.1

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-7884) Allow setting region size for region-based memtable allocators

Jay Patel created CASSANDRA-7884:


 Summary: Allow setting region size for region-based memtable 
allocators
 Key: CASSANDRA-7884
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7884
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jay Patel


SlabAllocator and  NativeAllocators currently have 1 MB region size hard coded 
in the code. Can we make it readable from cassandra.yaml ? This will allow user 
to optimize region size based on the number of tables in the cluster and/or 
payload size.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-7883) Allow plugging JEMalloc for off-heap memtables

Jay Patel created CASSANDRA-7883:


 Summary: Allow plugging JEMalloc for off-heap memtables
 Key: CASSANDRA-7883
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7883
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jay Patel
 Fix For: 2.1.1


Off-heap memtables (memtable_allocation_type:offheap_objects) introduced by 
CASSANDRA-6694 uses native GCC allocator. Provide an option to use JEMalloc 
allocator (http://www.canonware.com/jemalloc/) which is good to reduce 
fragmentation. 

CASSANDRA-3997 adds below option for off-heap caches and metadata. But it's not 
in effect for off-heap memtables. Should be use the same option or add another?

memory_allocator: JEMallocAllocator



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/2] git commit: Attempt to fix intermittent failure of DeleteTest

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 bd396ec8a - 350a94c3e


Attempt to fix intermittent failure of DeleteTest


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c81dc4bb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c81dc4bb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c81dc4bb

Branch: refs/heads/cassandra-2.1
Commit: c81dc4bb23da7f4e6c2c7738758db9d483d69292
Parents: dd4fbbc
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Fri Sep 5 10:50:13 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Fri Sep 5 10:50:24 2014 +0200

--
 .../unit/org/apache/cassandra/cql3/DeleteTest.java | 17 ++---
 1 file changed, 14 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c81dc4bb/test/unit/org/apache/cassandra/cql3/DeleteTest.java
--
diff --git a/test/unit/org/apache/cassandra/cql3/DeleteTest.java 
b/test/unit/org/apache/cassandra/cql3/DeleteTest.java
index 7782f12..44bef05 100644
--- a/test/unit/org/apache/cassandra/cql3/DeleteTest.java
+++ b/test/unit/org/apache/cassandra/cql3/DeleteTest.java
@@ -28,6 +28,7 @@ import org.apache.cassandra.config.Schema;
 import org.apache.cassandra.exceptions.ConfigurationException;
 import org.apache.cassandra.service.EmbeddedCassandraService;
 import org.junit.Assert;
+import org.junit.AfterClass;
 import org.junit.BeforeClass;
 import org.junit.Test;
 
@@ -48,14 +49,19 @@ public class DeleteTest extends SchemaLoader
 private static PreparedStatement pstmt4;
 private static PreparedStatement pstmt5;
 
-@BeforeClass()
-public static void setup() throws ConfigurationException, IOException
+@BeforeClass
+public static void setup() throws Exception
 {
 Schema.instance.clear();
 
 cassandra = new EmbeddedCassandraService();
 cassandra.start();
 
+// Currently the native server start method return before the server 
is fully binded to the socket,
+// so we need to wait slightly before trying to connect to it. We 
should fix this but in the meantime
+// using a sleep.
+Thread.sleep(500);
+
 cluster = 
Cluster.builder().addContactPoint(127.0.0.1).withPort(DatabaseDescriptor.getNativeTransportPort()).build();
 session = cluster.connect();
 
@@ -107,7 +113,12 @@ public class DeleteTest extends SchemaLoader
 pstmt5 = session.prepare(select id, cid, inh_c, val from 
junit.tpc_inherit_c where id=? and cid=?);
 }
 
-
+@AfterClass
+public static void tearDown() throws Exception
+{
+cluster.close();
+cassandra.stop();
+}
 
 @Test
 public void lostDeletesTest()

[jira] [Updated] (CASSANDRA-7554) Make CommitLogSegment sync/close asynchronous wrt each other


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-7554:

Fix Version/s: (was: 2.1.1)
   3.0

 Make CommitLogSegment sync/close asynchronous wrt each other
 

 Key: CASSANDRA-7554
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7554
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
 Fix For: 3.0


 There are a few minor issues with CLS I wanted to tidy up after working on 
 nearby code a bit recently, namely:
 1) We use synchronized() for sync() and for various minor accessors, meaning 
 either can block on the other, which is bad since sync() is lengthy
 2) Currently close() (and hence recycle()) must wait for a sync() to 
 complete, which means even if we have room available in segments waiting to 
 be recycled an ongoing sync might prevent us from reclaiming the space, 
 prematurely bottlenecking on the disk here
 3) recycle() currently depends on close(), which depends on sync(); if we've 
 decided to recycle/close a file before it is synced, this means we do not 
 care about the contents so can actually _avoid_ syncing to disk (which is 
 great in cases where the flush writers get ahead of the CL sync)
 To solve these problems I've introduced a new fairly simple concurrency 
 primitive called AsyncLock, which only supports tryLock(), or 
 tryLock(Runnable) - with the latter executing the provided runnable on the 
 thread _currently owning the lock_ after it relinquishes it. I've used this 
 to make close() take a Runnable to be executed _when the segment is actually 
 ready to be disposed of_ - which is either immediately, or once any in 
 progress sync has completed. This means the manager thread never blocks on a 
 sync.
 There is a knock on effect here, which is that we are even less inclined to 
 obey the CL limit (which has always been a soft limit), so I will file a 
 separate minor ticket to introduce a hard limit for CL size in case users 
 want to control this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

git commit: Attempt to fix intermittent failure of DeleteTest

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1.0 dd4fbbcd8 - c81dc4bb2


Attempt to fix intermittent failure of DeleteTest


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c81dc4bb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c81dc4bb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c81dc4bb

Branch: refs/heads/cassandra-2.1.0
Commit: c81dc4bb23da7f4e6c2c7738758db9d483d69292
Parents: dd4fbbc
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Fri Sep 5 10:50:13 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Fri Sep 5 10:50:24 2014 +0200

--
 .../unit/org/apache/cassandra/cql3/DeleteTest.java | 17 ++---
 1 file changed, 14 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c81dc4bb/test/unit/org/apache/cassandra/cql3/DeleteTest.java
--
diff --git a/test/unit/org/apache/cassandra/cql3/DeleteTest.java 
b/test/unit/org/apache/cassandra/cql3/DeleteTest.java
index 7782f12..44bef05 100644
--- a/test/unit/org/apache/cassandra/cql3/DeleteTest.java
+++ b/test/unit/org/apache/cassandra/cql3/DeleteTest.java
@@ -28,6 +28,7 @@ import org.apache.cassandra.config.Schema;
 import org.apache.cassandra.exceptions.ConfigurationException;
 import org.apache.cassandra.service.EmbeddedCassandraService;
 import org.junit.Assert;
+import org.junit.AfterClass;
 import org.junit.BeforeClass;
 import org.junit.Test;
 
@@ -48,14 +49,19 @@ public class DeleteTest extends SchemaLoader
 private static PreparedStatement pstmt4;
 private static PreparedStatement pstmt5;
 
-@BeforeClass()
-public static void setup() throws ConfigurationException, IOException
+@BeforeClass
+public static void setup() throws Exception
 {
 Schema.instance.clear();
 
 cassandra = new EmbeddedCassandraService();
 cassandra.start();
 
+// Currently the native server start method return before the server 
is fully binded to the socket,
+// so we need to wait slightly before trying to connect to it. We 
should fix this but in the meantime
+// using a sleep.
+Thread.sleep(500);
+
 cluster = 
Cluster.builder().addContactPoint(127.0.0.1).withPort(DatabaseDescriptor.getNativeTransportPort()).build();
 session = cluster.connect();
 
@@ -107,7 +113,12 @@ public class DeleteTest extends SchemaLoader
 pstmt5 = session.prepare(select id, cid, inh_c, val from 
junit.tpc_inherit_c where id=? and cid=?);
 }
 
-
+@AfterClass
+public static void tearDown() throws Exception
+{
+cluster.close();
+cassandra.stop();
+}
 
 @Test
 public void lostDeletesTest()

[jira] [Updated] (CASSANDRA-7738) Permit CL overuse to be explicitly bounded


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-7738:

Fix Version/s: (was: 2.1.1)
   3.0

 Permit CL overuse to be explicitly bounded
 --

 Key: CASSANDRA-7738
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7738
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
Priority: Minor
 Fix For: 3.0


 As mentioned in CASSANDRA-7554, we do not currently offer any way to 
 explicitly bound CL growth, which can be problematic in some scenarios (e.g. 
 EC2 where the system drive is quite small). We should offer a configurable 
 amount of headroom, beyond which we stop accepting writes until the backlog 
 clears.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7759) DROP TABLE makes C* unreachable


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122693#comment-14122693
 ] 

Benedict commented on CASSANDRA-7759:
-

Could you post logs from the rc6 run?

 DROP TABLE makes C* unreachable
 ---

 Key: CASSANDRA-7759
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7759
 Project: Cassandra
  Issue Type: Bug
 Environment: CQLSH of C* 2.10.-rc5 with one or two node(s).
Reporter: Fabrice Larcher

 After a {{DROP KEYSPACE}} or {{DROP TABLE}} command from _CQLSH_ I often get 
 the output :
 {code}
 errors={}, last_host=127.0.0.1
 {code}
 Then, the application can not access the node anymore (it uses the Java 
 Driver 2.1.0-rc1). And I get the following stacktraces from the 
 {{system.log}} file ;
 {code}
 ERROR [MemtableFlushWriter:5] 2014-08-13 11:26:07,577 
 CassandraDaemon.java:166 - Exception in thread 
 Thread[MemtableFlushWriter:5,5,main]
 java.lang.RuntimeException: Last written key 
 DecoratedKey(28149890686391545-8456361251720325, 
 30313034353666392d666464362d343539362d393537372d653539336430333138396437) = 
 current key DecoratedKey(108b5f3f-fc06-4a0d-99f1-3c6484b32e04, 
 31303862356633662d666330362d346130642d393966312d336336343834623332653034) 
 writing into 
 ./../data/data/metrics/run-a3c1fe80216911e49fd1ab9c3338a2ff/metrics-run.run_probe-tmp-ka-2-Data.db
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:172)
  ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:215) 
 ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:351)
  ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:314) 
 ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
  ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
 ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
  ~[guava-16.0.jar:na]
   at 
 org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1054)
  ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  ~[na:1.7.0_45]
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  ~[na:1.7.0_45]
   at java.lang.Thread.run(Thread.java:744) ~[na:1.7.0_45]
 {code}
 I also get the following (perhaps not related) 
 {code}
 ERROR [MemtableFlushWriter:6] 2014-08-13 11:26:08,827 
 CassandraDaemon.java:166 - Exception in thread 
 Thread[MemtableFlushWriter:6,5,main]
 org.apache.cassandra.serializers.MarshalException: Invalid byte for ascii: -50
   at 
 org.apache.cassandra.serializers.AsciiSerializer.validate(AsciiSerializer.java:39)
  ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 org.apache.cassandra.db.marshal.AbstractType.getString(AbstractType.java:78) 
 ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at org.apache.cassandra.dht.LocalToken.toString(LocalToken.java:39) 
 ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at java.lang.String.valueOf(String.java:2854) ~[na:1.7.0_45]
   at java.lang.StringBuilder.append(StringBuilder.java:128) ~[na:1.7.0_45]
   at org.apache.cassandra.db.DecoratedKey.toString(DecoratedKey.java:118) 
 ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at java.lang.String.valueOf(String.java:2854) ~[na:1.7.0_45]
   at java.lang.StringBuilder.append(StringBuilder.java:128) ~[na:1.7.0_45]
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:172)
  ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:215) 
 ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:351)
  ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:314) 
 ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
  ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
 ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
  ~[guava-16.0.jar:na]
   at

[2/2] git commit: Merge branch 'cassandra-2.1.0' into cassandra-2.1

Merge branch 'cassandra-2.1.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/350a94c3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/350a94c3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/350a94c3

Branch: refs/heads/cassandra-2.1
Commit: 350a94c3e3a4fe0597043d91ac8c1bae9f780731
Parents: bd396ec c81dc4b
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Fri Sep 5 10:50:46 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Fri Sep 5 10:50:46 2014 +0200

--
 .../unit/org/apache/cassandra/cql3/DeleteTest.java | 17 ++---
 1 file changed, 14 insertions(+), 3 deletions(-)
--

[2/3] git commit: Merge branch 'cassandra-2.1.0' into cassandra-2.1

Merge branch 'cassandra-2.1.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/350a94c3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/350a94c3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/350a94c3

Branch: refs/heads/trunk
Commit: 350a94c3e3a4fe0597043d91ac8c1bae9f780731
Parents: bd396ec c81dc4b
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Fri Sep 5 10:50:46 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Fri Sep 5 10:50:46 2014 +0200

--
 .../unit/org/apache/cassandra/cql3/DeleteTest.java | 17 ++---
 1 file changed, 14 insertions(+), 3 deletions(-)
--

[3/3] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d5cfe2b4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d5cfe2b4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d5cfe2b4

Branch: refs/heads/trunk
Commit: d5cfe2b4ab4139955a92667d0fecac880cbdfa0b
Parents: b4839a3 350a94c
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Fri Sep 5 10:54:40 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Fri Sep 5 10:54:40 2014 +0200

--
 .../unit/org/apache/cassandra/cql3/DeleteTest.java | 17 ++---
 1 file changed, 14 insertions(+), 3 deletions(-)
--

[1/3] git commit: Attempt to fix intermittent failure of DeleteTest

Repository: cassandra
Updated Branches:
  refs/heads/trunk b4839a388 - d5cfe2b4a


Attempt to fix intermittent failure of DeleteTest


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c81dc4bb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c81dc4bb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c81dc4bb

Branch: refs/heads/trunk
Commit: c81dc4bb23da7f4e6c2c7738758db9d483d69292
Parents: dd4fbbc
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Fri Sep 5 10:50:13 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Fri Sep 5 10:50:24 2014 +0200

--
 .../unit/org/apache/cassandra/cql3/DeleteTest.java | 17 ++---
 1 file changed, 14 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c81dc4bb/test/unit/org/apache/cassandra/cql3/DeleteTest.java
--
diff --git a/test/unit/org/apache/cassandra/cql3/DeleteTest.java 
b/test/unit/org/apache/cassandra/cql3/DeleteTest.java
index 7782f12..44bef05 100644
--- a/test/unit/org/apache/cassandra/cql3/DeleteTest.java
+++ b/test/unit/org/apache/cassandra/cql3/DeleteTest.java
@@ -28,6 +28,7 @@ import org.apache.cassandra.config.Schema;
 import org.apache.cassandra.exceptions.ConfigurationException;
 import org.apache.cassandra.service.EmbeddedCassandraService;
 import org.junit.Assert;
+import org.junit.AfterClass;
 import org.junit.BeforeClass;
 import org.junit.Test;
 
@@ -48,14 +49,19 @@ public class DeleteTest extends SchemaLoader
 private static PreparedStatement pstmt4;
 private static PreparedStatement pstmt5;
 
-@BeforeClass()
-public static void setup() throws ConfigurationException, IOException
+@BeforeClass
+public static void setup() throws Exception
 {
 Schema.instance.clear();
 
 cassandra = new EmbeddedCassandraService();
 cassandra.start();
 
+// Currently the native server start method return before the server 
is fully binded to the socket,
+// so we need to wait slightly before trying to connect to it. We 
should fix this but in the meantime
+// using a sleep.
+Thread.sleep(500);
+
 cluster = 
Cluster.builder().addContactPoint(127.0.0.1).withPort(DatabaseDescriptor.getNativeTransportPort()).build();
 session = cluster.connect();
 
@@ -107,7 +113,12 @@ public class DeleteTest extends SchemaLoader
 pstmt5 = session.prepare(select id, cid, inh_c, val from 
junit.tpc_inherit_c where id=? and cid=?);
 }
 
-
+@AfterClass
+public static void tearDown() throws Exception
+{
+cluster.close();
+cassandra.stop();
+}
 
 @Test
 public void lostDeletesTest()

git commit: Fix test compilation

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 350a94c3e - d304b7cab


Fix test compilation


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d304b7ca
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d304b7ca
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d304b7ca

Branch: refs/heads/cassandra-2.1
Commit: d304b7cabd6ab69b737c978c4436b14f262f4b62
Parents: 350a94c
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Fri Sep 5 10:59:05 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Fri Sep 5 10:59:05 2014 +0200

--
 test/unit/org/apache/cassandra/cql3/DeleteTest.java | 1 -
 1 file changed, 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d304b7ca/test/unit/org/apache/cassandra/cql3/DeleteTest.java
--
diff --git a/test/unit/org/apache/cassandra/cql3/DeleteTest.java 
b/test/unit/org/apache/cassandra/cql3/DeleteTest.java
index 44bef05..6ca6b41 100644
--- a/test/unit/org/apache/cassandra/cql3/DeleteTest.java
+++ b/test/unit/org/apache/cassandra/cql3/DeleteTest.java
@@ -117,7 +117,6 @@ public class DeleteTest extends SchemaLoader
 public static void tearDown() throws Exception
 {
 cluster.close();
-cassandra.stop();
 }
 
 @Test

[1/2] git commit: Fix test compilation

Repository: cassandra
Updated Branches:
  refs/heads/trunk d5cfe2b4a - 719bb3928


Fix test compilation


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d304b7ca
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d304b7ca
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d304b7ca

Branch: refs/heads/trunk
Commit: d304b7cabd6ab69b737c978c4436b14f262f4b62
Parents: 350a94c
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Fri Sep 5 10:59:05 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Fri Sep 5 10:59:05 2014 +0200

--
 test/unit/org/apache/cassandra/cql3/DeleteTest.java | 1 -
 1 file changed, 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d304b7ca/test/unit/org/apache/cassandra/cql3/DeleteTest.java
--
diff --git a/test/unit/org/apache/cassandra/cql3/DeleteTest.java 
b/test/unit/org/apache/cassandra/cql3/DeleteTest.java
index 44bef05..6ca6b41 100644
--- a/test/unit/org/apache/cassandra/cql3/DeleteTest.java
+++ b/test/unit/org/apache/cassandra/cql3/DeleteTest.java
@@ -117,7 +117,6 @@ public class DeleteTest extends SchemaLoader
 public static void tearDown() throws Exception
 {
 cluster.close();
-cassandra.stop();
 }
 
 @Test

[2/2] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/719bb392
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/719bb392
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/719bb392

Branch: refs/heads/trunk
Commit: 719bb392802ba94999e2ddc0c7c9d44d3843a8c8
Parents: d5cfe2b d304b7c
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Fri Sep 5 10:59:22 2014 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Fri Sep 5 10:59:22 2014 +0200

--
 test/unit/org/apache/cassandra/cql3/DeleteTest.java | 1 -
 1 file changed, 1 deletion(-)
--

[jira] [Created] (CASSANDRA-7885) Make native Server start() method block until the server is properly bound

Sylvain Lebresne created CASSANDRA-7885:
---

 Summary: Make native Server start() method block until the server 
is properly bound
 Key: CASSANDRA-7885
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7885
 Project: Cassandra
  Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Priority: Minor
 Fix For: 2.1.1


The native Server {{start()}} method returns before making sure the bind 
operation has succeeded, so the server might not be fully ready when it 
returns. It's not a big deal but it's slightly unexpected and I think that it's 
the cause of the [intermittent failures of DeleteTest on 
jenkins|http://cassci.datastax.com/job/cassandra-2.1.0_utest/lastBuild/testReport/junit.framework/TestSuite/org_apache_cassandra_cql3_DeleteTest/].



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7882) Allow disabling slab allocation for off-heap memtables

[
https://issues.apache.org/jira/browse/CASSANDRA-7882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122706#comment-14122706
]

Jay Patel commented on CASSANDRA-7882:
--

Is this overhead from using jemalloc allocator, or this happens with native gcc
as well? BTW, I've opened a related ticket CASSANDRA-7883 to allow plugging
jemalloc. And, CASSANDRA-7884 to make the region size configurable.

I think logarithmically scaling from 1K to 1 MB option can also work. But,
should we keep the start end region-size configurable? Usually, this will be
advanced option to tune but very helpful. By default, we can keep 1 MB region
size as most use cases will not need to pack many tables.

Let me know the best way to handle this, and also you can assign it to me to
have it in 2.1.1. We're actually quite dependent on this as currently 1 MB is
putting a upper limit on number of tables in the cluster having multiple
under-utilized clusters to support high # of tables is not cost-effective.

Allow disabling slab allocation for off-heap memtables
--

Key: CASSANDRA-7882
URL: https://issues.apache.org/jira/browse/CASSANDRA-7882
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Jay Patel
Fix For: 2.1.1

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

git commit: Don't expose internal classes over JMX

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1.0 c81dc4bb2 - 45014886e


Don't expose internal classes over JMX

Patch by marcuse; reviewed by slebresne for CASSANDRA-7879


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/45014886
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/45014886
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/45014886

Branch: refs/heads/cassandra-2.1.0
Commit: 45014886e89e5bd47bf2f4c9871391d77702ad82
Parents: c81dc4b
Author: Marcus Eriksson marc...@apache.org
Authored: Fri Sep 5 11:02:29 2014 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Fri Sep 5 11:02:29 2014 +0200

--
 CHANGES.txt |  1 +
 .../db/compaction/CompactionManager.java|  9 +-
 .../cassandra/service/StorageService.java   | 12 
 .../cassandra/service/StorageServiceMBean.java  |  6 ++--
 .../org/apache/cassandra/tools/NodeProbe.java   | 30 
 5 files changed, 30 insertions(+), 28 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/45014886/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 93d77c1..befdcd7 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.0
+ * Avoid exposing internal classes over JMX (CASSANDRA-7879)
  * Add null check for keys when freezing collection (CASSANDRA-7869)
 
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/45014886/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 65c4882..09c068f 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -289,7 +289,14 @@ public class CompactionManager implements 
CompactionManagerMBean
 void execute(SSTableReader input) throws IOException;
 }
 
-public enum AllSSTableOpStatus { ABORTED, SUCCESSFUL }
+public enum AllSSTableOpStatus { ABORTED(1), SUCCESSFUL(0);
+public final int statusCode;
+
+AllSSTableOpStatus(int statusCode)
+{
+this.statusCode = statusCode;
+}
+}
 
 public AllSSTableOpStatus performScrub(final ColumnFamilyStore cfs, final 
boolean skipCorrupted) throws InterruptedException, ExecutionException
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/45014886/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 735bf7b..5c6dbd6 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -2156,7 +2156,7 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 return 
Gossiper.instance.getCurrentGenerationNumber(FBUtilities.getBroadcastAddress());
 }
 
-public CompactionManager.AllSSTableOpStatus forceKeyspaceCleanup(String 
keyspaceName, String... columnFamilies) throws IOException, ExecutionException, 
InterruptedException
+public int forceKeyspaceCleanup(String keyspaceName, String... 
columnFamilies) throws IOException, ExecutionException, InterruptedException
 {
 if (keyspaceName.equals(Keyspace.SYSTEM_KS))
 throw new RuntimeException(Cleanup of the system keyspace is 
neither necessary nor wise);
@@ -2168,10 +2168,10 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 if (oneStatus != CompactionManager.AllSSTableOpStatus.SUCCESSFUL)
 status = oneStatus;
 }
-return status;
+return status.statusCode;
 }
 
-public CompactionManager.AllSSTableOpStatus scrub(boolean disableSnapshot, 
boolean skipCorrupted, String keyspaceName, String... columnFamilies) throws 
IOException, ExecutionException, InterruptedException
+public int scrub(boolean disableSnapshot, boolean skipCorrupted, String 
keyspaceName, String... columnFamilies) throws IOException, ExecutionException, 
InterruptedException
 {
 CompactionManager.AllSSTableOpStatus status = 
CompactionManager.AllSSTableOpStatus.SUCCESSFUL;
 for (ColumnFamilyStore cfStore : getValidColumnFamilies(false, false, 
keyspaceName, columnFamilies))
@@ -2180,10 +2180,10 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE

[3/3] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk

Conflicts:
src/java/org/apache/cassandra/tools/NodeProbe.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/37a9d405
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/37a9d405
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/37a9d405

Branch: refs/heads/trunk
Commit: 37a9d405c7fe5de84450e20960d94a134d3e85e9
Parents: 719bb39 dd972e6
Author: Marcus Eriksson marc...@apache.org
Authored: Fri Sep 5 11:10:40 2014 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Fri Sep 5 11:10:40 2014 +0200

--
 CHANGES.txt |  2 ++
 .../db/compaction/CompactionManager.java|  9 +-
 .../cassandra/service/StorageService.java   | 12 
 .../cassandra/service/StorageServiceMBean.java  |  6 ++--
 .../org/apache/cassandra/tools/NodeProbe.java   | 30 
 5 files changed, 31 insertions(+), 28 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/37a9d405/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/37a9d405/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/37a9d405/src/java/org/apache/cassandra/service/StorageService.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/37a9d405/src/java/org/apache/cassandra/tools/NodeProbe.java
--
diff --cc src/java/org/apache/cassandra/tools/NodeProbe.java
index a0d4db7,3f3073d..55f462e
--- a/src/java/org/apache/cassandra/tools/NodeProbe.java
+++ b/src/java/org/apache/cassandra/tools/NodeProbe.java
@@@ -203,34 -203,28 +203,28 @@@ public class NodeProbe implements AutoC
  
  public void forceKeyspaceCleanup(PrintStream out, String keyspaceName, 
String... columnFamilies) throws IOException, ExecutionException, 
InterruptedException
  {
- switch (forceKeyspaceCleanup(keyspaceName, columnFamilies))
+ if (forceKeyspaceCleanup(keyspaceName, columnFamilies) != 0)
  {
- case ABORTED:
- failed = true;
- out.println(Aborted cleaning up atleast one table in 
keyspace +keyspaceName+, check server logs for more information.);
- break;
+ failed = true;
 -out.println(Aborted cleaning up atleast one column family in 
keyspace +keyspaceName+, check server logs for more information.);
++out.println(Aborted cleaning up atleast one table in keyspace 
+keyspaceName+, check server logs for more information.);
  }
  }
  
  public void scrub(PrintStream out, boolean disableSnapshot, boolean 
skipCorrupted, String keyspaceName, String... columnFamilies) throws 
IOException, ExecutionException, InterruptedException
  {
- switch (scrub(disableSnapshot, skipCorrupted, keyspaceName, 
columnFamilies))
+ if (scrub(disableSnapshot, skipCorrupted, keyspaceName, 
columnFamilies) != 0)
  {
- case ABORTED:
- failed = true;
- out.println(Aborted scrubbing atleast one table in keyspace 
+keyspaceName+, check server logs for more information.);
- break;
+ failed = true;
 -out.println(Aborted scrubbing atleast one column family in 
keyspace +keyspaceName+, check server logs for more information.);
++out.println(Aborted scrubbing atleast one table in keyspace 
+keyspaceName+, check server logs for more information.);
  }
  }
  
  public void upgradeSSTables(PrintStream out, String keyspaceName, boolean 
excludeCurrentVersion, String... columnFamilies) throws IOException, 
ExecutionException, InterruptedException
  {
- switch (upgradeSSTables(keyspaceName, excludeCurrentVersion, 
columnFamilies))
+ if (upgradeSSTables(keyspaceName, excludeCurrentVersion, 
columnFamilies) != 0)
  {
- case ABORTED:
- failed = true;
- out.println(Aborted upgrading sstables for atleast one table 
in keyspace +keyspaceName+, check server logs for more information.);
- break;
+ failed = true;
 -out.println(Aborted upgrading sstables for atleast one column 
family in keyspace +keyspaceName+, check server logs for more information.);
++out.println(Aborted upgrading sstables for atleast one table in 
keyspace +keyspaceName+, check server logs for more information.);
  }
  }

[1/3] git commit: Don't expose internal classes over JMX

Repository: cassandra
Updated Branches:
  refs/heads/trunk 719bb3928 - 37a9d405c


Don't expose internal classes over JMX

Patch by marcuse; reviewed by slebresne for CASSANDRA-7879


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/45014886
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/45014886
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/45014886

Branch: refs/heads/trunk
Commit: 45014886e89e5bd47bf2f4c9871391d77702ad82
Parents: c81dc4b
Author: Marcus Eriksson marc...@apache.org
Authored: Fri Sep 5 11:02:29 2014 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Fri Sep 5 11:02:29 2014 +0200

--
 CHANGES.txt |  1 +
 .../db/compaction/CompactionManager.java|  9 +-
 .../cassandra/service/StorageService.java   | 12 
 .../cassandra/service/StorageServiceMBean.java  |  6 ++--
 .../org/apache/cassandra/tools/NodeProbe.java   | 30 
 5 files changed, 30 insertions(+), 28 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/45014886/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 93d77c1..befdcd7 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.0
+ * Avoid exposing internal classes over JMX (CASSANDRA-7879)
  * Add null check for keys when freezing collection (CASSANDRA-7869)
 
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/45014886/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 65c4882..09c068f 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -289,7 +289,14 @@ public class CompactionManager implements 
CompactionManagerMBean
 void execute(SSTableReader input) throws IOException;
 }
 
-public enum AllSSTableOpStatus { ABORTED, SUCCESSFUL }
+public enum AllSSTableOpStatus { ABORTED(1), SUCCESSFUL(0);
+public final int statusCode;
+
+AllSSTableOpStatus(int statusCode)
+{
+this.statusCode = statusCode;
+}
+}
 
 public AllSSTableOpStatus performScrub(final ColumnFamilyStore cfs, final 
boolean skipCorrupted) throws InterruptedException, ExecutionException
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/45014886/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 735bf7b..5c6dbd6 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -2156,7 +2156,7 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 return 
Gossiper.instance.getCurrentGenerationNumber(FBUtilities.getBroadcastAddress());
 }
 
-public CompactionManager.AllSSTableOpStatus forceKeyspaceCleanup(String 
keyspaceName, String... columnFamilies) throws IOException, ExecutionException, 
InterruptedException
+public int forceKeyspaceCleanup(String keyspaceName, String... 
columnFamilies) throws IOException, ExecutionException, InterruptedException
 {
 if (keyspaceName.equals(Keyspace.SYSTEM_KS))
 throw new RuntimeException(Cleanup of the system keyspace is 
neither necessary nor wise);
@@ -2168,10 +2168,10 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 if (oneStatus != CompactionManager.AllSSTableOpStatus.SUCCESSFUL)
 status = oneStatus;
 }
-return status;
+return status.statusCode;
 }
 
-public CompactionManager.AllSSTableOpStatus scrub(boolean disableSnapshot, 
boolean skipCorrupted, String keyspaceName, String... columnFamilies) throws 
IOException, ExecutionException, InterruptedException
+public int scrub(boolean disableSnapshot, boolean skipCorrupted, String 
keyspaceName, String... columnFamilies) throws IOException, ExecutionException, 
InterruptedException
 {
 CompactionManager.AllSSTableOpStatus status = 
CompactionManager.AllSSTableOpStatus.SUCCESSFUL;
 for (ColumnFamilyStore cfStore : getValidColumnFamilies(false, false, 
keyspaceName, columnFamilies))
@@ -2180,10 +2180,10 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 if

[2/3] git commit: Merge branch 'cassandra-2.1.0' into cassandra-2.1

Merge branch 'cassandra-2.1.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dd972e6c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dd972e6c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dd972e6c

Branch: refs/heads/trunk
Commit: dd972e6c3f48619d30e4167dde92cd333d58ffad
Parents: d304b7c 4501488
Author: Marcus Eriksson marc...@apache.org
Authored: Fri Sep 5 11:06:57 2014 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Fri Sep 5 11:06:57 2014 +0200

--
 CHANGES.txt |  2 ++
 .../db/compaction/CompactionManager.java|  9 +-
 .../cassandra/service/StorageService.java   | 12 
 .../cassandra/service/StorageServiceMBean.java  |  6 ++--
 .../org/apache/cassandra/tools/NodeProbe.java   | 30 
 5 files changed, 31 insertions(+), 28 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/dd972e6c/CHANGES.txt
--
diff --cc CHANGES.txt
index 5404f83,befdcd7..a0ba08b
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,56 -1,9 +1,58 @@@
 +2.1.1
 + * (cqlsh): Show progress of COPY operations (CASSANDRA-7789)
 + * Add syntax to remove multiple elements from a map (CASSANDRA-6599)
 + * Support non-equals conditions in lightweight transactions (CASSANDRA-6839)
 + * Add IF [NOT] EXISTS to create/drop triggers (CASSANDRA-7606)
 + * (cqlsh) Display the current logged-in user (CASSANDRA-7785)
 + * (cqlsh) Don't ignore CTRL-C during COPY FROM execution (CASSANDRA-7815)
 + * (cqlsh) Order UDTs according to cross-type dependencies in DESCRIBE
 +   output (CASSANDRA-7659)
 + * (cqlsh) Fix handling of CAS statement results (CASSANDRA-7671)
 + * (cqlsh) COPY TO/FROM improvements (CASSANDRA-7405)
 + * Support list index operations with conditions (CASSANDRA-7499)
 + * Add max live/tombstoned cells to nodetool cfstats output (CASSANDRA-7731)
 + * Validate IPv6 wildcard addresses properly (CASSANDRA-7680)
 + * (cqlsh) Error when tracing query (CASSANDRA-7613)
 + * Avoid IOOBE when building SyntaxError message snippet (CASSANDRA-7569)
 + * SSTableExport uses correct validator to create string representation of 
partition
 +   keys (CASSANDRA-7498)
 + * Avoid NPEs when receiving type changes for an unknown keyspace 
(CASSANDRA-7689)
 + * Add support for custom 2i validation (CASSANDRA-7575)
 + * Pig support for hadoop CqlInputFormat (CASSANDRA-6454)
 + * Add listen_interface and rpc_interface options (CASSANDRA-7417)
 + * Improve schema merge performance (CASSANDRA-7444)
 + * Adjust MT depth based on # of partition validating (CASSANDRA-5263)
 + * Optimise NativeCell comparisons (CASSANDRA-6755)
 + * Configurable client timeout for cqlsh (CASSANDRA-7516)
 + * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111)
 +Merged from 2.0:
 + * Explicitly disallow mixing multi-column and single-column
 +   relations on clustering columns (CASSANDRA-7711)
 + * Better error message when condition is set on PK column (CASSANDRA-7804)
 + * Don't send schema change responses and events for no-op DDL
 +   statements (CASSANDRA-7600)
 + * (Hadoop) fix cluster initialisation for a split fetching (CASSANDRA-7774)
 + * Throw InvalidRequestException when queries contain relations on entire
 +   collection columns (CASSANDRA-7506)
 + * (cqlsh) enable CTRL-R history search with libedit (CASSANDRA-7577)
 + * (Hadoop) allow ACFRW to limit nodes to local DC (CASSANDRA-7252)
 + * (cqlsh) cqlsh should automatically disable tracing when selecting
 +   from system_traces (CASSANDRA-7641)
 + * (Hadoop) Add CqlOutputFormat (CASSANDRA-6927)
 + * Don't depend on cassandra config for nodetool ring (CASSANDRA-7508)
 + * (cqlsh) Fix failing cqlsh formatting tests (CASSANDRA-7703)
 + * Fix IncompatibleClassChangeError from hadoop2 (CASSANDRA-7229)
 + * Add 'nodetool sethintedhandoffthrottlekb' (CASSANDRA-7635)
 + * (cqlsh) Add tab-completion for CREATE/DROP USER IF [NOT] EXISTS 
(CASSANDRA-7611)
 + * Catch errors when the JVM pulls the rug out from GCInspector 
(CASSANDRA-5345)
 + * cqlsh fails when version number parts are not int (CASSANDRA-7524)
 +Merged from 1.2:
 + * Improve PasswordAuthenticator default super user setup (CASSANDRA-7788)
 +
 +
  2.1.0
+  * Avoid exposing internal classes over JMX (CASSANDRA-7879)
+  * Add null check for keys when freezing collection (CASSANDRA-7869)
 -
 -
 -2.1.0-rc7
   * Add frozen keyword and require UDT to be frozen (CASSANDRA-7857)
   * Track added sstable size correctly (CASSANDRA-7239)
   * (cqlsh) Fix case insensitivity (CASSANDRA-7834)

[1/2] git commit: Don't expose internal classes over JMX

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 d304b7cab - dd972e6c3


Don't expose internal classes over JMX

Patch by marcuse; reviewed by slebresne for CASSANDRA-7879


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/45014886
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/45014886
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/45014886

Branch: refs/heads/cassandra-2.1
Commit: 45014886e89e5bd47bf2f4c9871391d77702ad82
Parents: c81dc4b
Author: Marcus Eriksson marc...@apache.org
Authored: Fri Sep 5 11:02:29 2014 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Fri Sep 5 11:02:29 2014 +0200

--
 CHANGES.txt |  1 +
 .../db/compaction/CompactionManager.java|  9 +-
 .../cassandra/service/StorageService.java   | 12 
 .../cassandra/service/StorageServiceMBean.java  |  6 ++--
 .../org/apache/cassandra/tools/NodeProbe.java   | 30 
 5 files changed, 30 insertions(+), 28 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/45014886/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 93d77c1..befdcd7 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.0
+ * Avoid exposing internal classes over JMX (CASSANDRA-7879)
  * Add null check for keys when freezing collection (CASSANDRA-7869)
 
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/45014886/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 65c4882..09c068f 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -289,7 +289,14 @@ public class CompactionManager implements 
CompactionManagerMBean
 void execute(SSTableReader input) throws IOException;
 }
 
-public enum AllSSTableOpStatus { ABORTED, SUCCESSFUL }
+public enum AllSSTableOpStatus { ABORTED(1), SUCCESSFUL(0);
+public final int statusCode;
+
+AllSSTableOpStatus(int statusCode)
+{
+this.statusCode = statusCode;
+}
+}
 
 public AllSSTableOpStatus performScrub(final ColumnFamilyStore cfs, final 
boolean skipCorrupted) throws InterruptedException, ExecutionException
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/45014886/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 735bf7b..5c6dbd6 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -2156,7 +2156,7 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 return 
Gossiper.instance.getCurrentGenerationNumber(FBUtilities.getBroadcastAddress());
 }
 
-public CompactionManager.AllSSTableOpStatus forceKeyspaceCleanup(String 
keyspaceName, String... columnFamilies) throws IOException, ExecutionException, 
InterruptedException
+public int forceKeyspaceCleanup(String keyspaceName, String... 
columnFamilies) throws IOException, ExecutionException, InterruptedException
 {
 if (keyspaceName.equals(Keyspace.SYSTEM_KS))
 throw new RuntimeException(Cleanup of the system keyspace is 
neither necessary nor wise);
@@ -2168,10 +2168,10 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 if (oneStatus != CompactionManager.AllSSTableOpStatus.SUCCESSFUL)
 status = oneStatus;
 }
-return status;
+return status.statusCode;
 }
 
-public CompactionManager.AllSSTableOpStatus scrub(boolean disableSnapshot, 
boolean skipCorrupted, String keyspaceName, String... columnFamilies) throws 
IOException, ExecutionException, InterruptedException
+public int scrub(boolean disableSnapshot, boolean skipCorrupted, String 
keyspaceName, String... columnFamilies) throws IOException, ExecutionException, 
InterruptedException
 {
 CompactionManager.AllSSTableOpStatus status = 
CompactionManager.AllSSTableOpStatus.SUCCESSFUL;
 for (ColumnFamilyStore cfStore : getValidColumnFamilies(false, false, 
keyspaceName, columnFamilies))
@@ -2180,10 +2180,10 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE

[2/2] git commit: Merge branch 'cassandra-2.1.0' into cassandra-2.1

Merge branch 'cassandra-2.1.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dd972e6c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dd972e6c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dd972e6c

Branch: refs/heads/cassandra-2.1
Commit: dd972e6c3f48619d30e4167dde92cd333d58ffad
Parents: d304b7c 4501488
Author: Marcus Eriksson marc...@apache.org
Authored: Fri Sep 5 11:06:57 2014 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Fri Sep 5 11:06:57 2014 +0200

--
 CHANGES.txt |  2 ++
 .../db/compaction/CompactionManager.java|  9 +-
 .../cassandra/service/StorageService.java   | 12 
 .../cassandra/service/StorageServiceMBean.java  |  6 ++--
 .../org/apache/cassandra/tools/NodeProbe.java   | 30 
 5 files changed, 31 insertions(+), 28 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/dd972e6c/CHANGES.txt
--
diff --cc CHANGES.txt
index 5404f83,befdcd7..a0ba08b
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,56 -1,9 +1,58 @@@
 +2.1.1
 + * (cqlsh): Show progress of COPY operations (CASSANDRA-7789)
 + * Add syntax to remove multiple elements from a map (CASSANDRA-6599)
 + * Support non-equals conditions in lightweight transactions (CASSANDRA-6839)
 + * Add IF [NOT] EXISTS to create/drop triggers (CASSANDRA-7606)
 + * (cqlsh) Display the current logged-in user (CASSANDRA-7785)
 + * (cqlsh) Don't ignore CTRL-C during COPY FROM execution (CASSANDRA-7815)
 + * (cqlsh) Order UDTs according to cross-type dependencies in DESCRIBE
 +   output (CASSANDRA-7659)
 + * (cqlsh) Fix handling of CAS statement results (CASSANDRA-7671)
 + * (cqlsh) COPY TO/FROM improvements (CASSANDRA-7405)
 + * Support list index operations with conditions (CASSANDRA-7499)
 + * Add max live/tombstoned cells to nodetool cfstats output (CASSANDRA-7731)
 + * Validate IPv6 wildcard addresses properly (CASSANDRA-7680)
 + * (cqlsh) Error when tracing query (CASSANDRA-7613)
 + * Avoid IOOBE when building SyntaxError message snippet (CASSANDRA-7569)
 + * SSTableExport uses correct validator to create string representation of 
partition
 +   keys (CASSANDRA-7498)
 + * Avoid NPEs when receiving type changes for an unknown keyspace 
(CASSANDRA-7689)
 + * Add support for custom 2i validation (CASSANDRA-7575)
 + * Pig support for hadoop CqlInputFormat (CASSANDRA-6454)
 + * Add listen_interface and rpc_interface options (CASSANDRA-7417)
 + * Improve schema merge performance (CASSANDRA-7444)
 + * Adjust MT depth based on # of partition validating (CASSANDRA-5263)
 + * Optimise NativeCell comparisons (CASSANDRA-6755)
 + * Configurable client timeout for cqlsh (CASSANDRA-7516)
 + * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111)
 +Merged from 2.0:
 + * Explicitly disallow mixing multi-column and single-column
 +   relations on clustering columns (CASSANDRA-7711)
 + * Better error message when condition is set on PK column (CASSANDRA-7804)
 + * Don't send schema change responses and events for no-op DDL
 +   statements (CASSANDRA-7600)
 + * (Hadoop) fix cluster initialisation for a split fetching (CASSANDRA-7774)
 + * Throw InvalidRequestException when queries contain relations on entire
 +   collection columns (CASSANDRA-7506)
 + * (cqlsh) enable CTRL-R history search with libedit (CASSANDRA-7577)
 + * (Hadoop) allow ACFRW to limit nodes to local DC (CASSANDRA-7252)
 + * (cqlsh) cqlsh should automatically disable tracing when selecting
 +   from system_traces (CASSANDRA-7641)
 + * (Hadoop) Add CqlOutputFormat (CASSANDRA-6927)
 + * Don't depend on cassandra config for nodetool ring (CASSANDRA-7508)
 + * (cqlsh) Fix failing cqlsh formatting tests (CASSANDRA-7703)
 + * Fix IncompatibleClassChangeError from hadoop2 (CASSANDRA-7229)
 + * Add 'nodetool sethintedhandoffthrottlekb' (CASSANDRA-7635)
 + * (cqlsh) Add tab-completion for CREATE/DROP USER IF [NOT] EXISTS 
(CASSANDRA-7611)
 + * Catch errors when the JVM pulls the rug out from GCInspector 
(CASSANDRA-5345)
 + * cqlsh fails when version number parts are not int (CASSANDRA-7524)
 +Merged from 1.2:
 + * Improve PasswordAuthenticator default super user setup (CASSANDRA-7788)
 +
 +
  2.1.0
+  * Avoid exposing internal classes over JMX (CASSANDRA-7879)
+  * Add null check for keys when freezing collection (CASSANDRA-7869)
 -
 -
 -2.1.0-rc7
   * Add frozen keyword and require UDT to be frozen (CASSANDRA-7857)
   * Track added sstable size correctly (CASSANDRA-7239)
   * (cqlsh) Fix case insensitivity (CASSANDRA-7834)

[jira] [Comment Edited] (CASSANDRA-7882) Allow disabling slab allocation for off-heap memtables

[
https://issues.apache.org/jira/browse/CASSANDRA-7882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122706#comment-14122706
]

Jay Patel edited comment on CASSANDRA-7882 at 9/5/14 9:12 AM:
--

was (Author: pateljay3001):
Is this overhead from using jemalloc allocator, or this happens with native gcc
as well? BTW, I've opened a related ticket CASSANDRA-7883 to allow plugging
jemalloc. And, CASSANDRA-7884 to make the region size configurable.

Allow disabling slab allocation for off-heap memtables
--

Key: CASSANDRA-7882
URL: https://issues.apache.org/jira/browse/CASSANDRA-7882
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Jay Patel
Fix For: 2.1.1

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-7882) Allow disabling slab allocation for off-heap memtables


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict reassigned CASSANDRA-7882:
---

Assignee: Benedict

 Allow disabling slab allocation for off-heap memtables
 --

 Key: CASSANDRA-7882
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7882
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jay Patel
Assignee: Benedict
 Fix For: 2.1.1


 CASSANDRA-5935 allows option to disable region-based allocation for on-heap 
 memtables but there is no option to disable it for off-heap memtables 
 (memtable_allocation_type: offheap_objects). 
 Disabling region-based allocation will allow us to pack more tables in the 
 schema since minimum of 1MB region won't be allocated per table. Downside can 
 be more fragmentation which should be controllable by using better allocator 
 like JEMalloc.
 How about below option in yaml?:
 memtable_allocation_type: unslabbed_offheap_objects
 Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7885) Make native Server start() method block until the server is properly bound


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-7885:

Attachment: 7885.txt

Attaching simple patch to do this. I'll note that I haven't really be able to 
reproduce the jenkins failure on my box reliably, but it sounds very likely 
that it's the reason. I'll note that for 2.1.0 I've just added a sleep in 
{{DeleteTest}} because I don't want to touch anything we don't absolutately 
have to at this point, so this patch remove that sleep for 2.1.1.

 Make native Server start() method block until the server is properly bound
 --

 Key: CASSANDRA-7885
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7885
 Project: Cassandra
  Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Priority: Minor
 Fix For: 2.1.1

 Attachments: 7885.txt


 The native Server {{start()}} method returns before making sure the bind 
 operation has succeeded, so the server might not be fully ready when it 
 returns. It's not a big deal but it's slightly unexpected and I think that 
 it's the cause of the [intermittent failures of DeleteTest on 
 jenkins|http://cassci.datastax.com/job/cassandra-2.1.0_utest/lastBuild/testReport/junit.framework/TestSuite/org_apache_cassandra_cql3_DeleteTest/].



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7884) Allow setting region size for region-based memtable allocators


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jay Patel updated CASSANDRA-7884:
-
Fix Version/s: 2.1.1

 Allow setting region size for region-based memtable allocators
 --

 Key: CASSANDRA-7884
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7884
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jay Patel
 Fix For: 2.1.1


 SlabAllocator and  NativeAllocators currently have 1 MB region size hard 
 coded in the code. Can we make it readable from cassandra.yaml ? This will 
 allow user to optimize region size based on the number of tables in the 
 cluster and/or payload size.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7882) Allow disabling slab allocation for off-heap memtables


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-7882:

Assignee: (was: Benedict)

 Allow disabling slab allocation for off-heap memtables
 --

 Key: CASSANDRA-7882
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7882
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jay Patel
 Fix For: 2.1.1


 CASSANDRA-5935 allows option to disable region-based allocation for on-heap 
 memtables but there is no option to disable it for off-heap memtables 
 (memtable_allocation_type: offheap_objects). 
 Disabling region-based allocation will allow us to pack more tables in the 
 schema since minimum of 1MB region won't be allocated per table. Downside can 
 be more fragmentation which should be controllable by using better allocator 
 like JEMalloc.
 How about below option in yaml?:
 memtable_allocation_type: unslabbed_offheap_objects
 Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7882) Allow disabling slab allocation for off-heap memtables

[
https://issues.apache.org/jira/browse/CASSANDRA-7882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122713#comment-14122713
]

Benedict commented on CASSANDRA-7882:
-

bq. or this happens with native gcc as well?

This will happen with all allocators (they need to manage the memory somehow),
but the profile will vary with each allocator and platform

bq. should we keep the start end region-size configurable?

I think 1Kb - 1Mb is more than enough to solve the problem. 1Kb is
considerably less than the general overhead for managing a table (might even be
sensible to start around 8Kb), so not a meaningful impact, and with logarithmic
scaling we're guaranteed good occupancy, so we're only going to have memory
pressure if there's genuine memory pressure.

I'm on the fence about if the logarithmic scaling should be configurable or
just always on. But it's likely sufficient to simply have an on/off config
parameter, if any.

If you want to have a crack at patching this, please do, and I'll review. We're
always keen to expand the contributor base and it isn't a high priority for the
core team right now.

Allow disabling slab allocation for off-heap memtables
--

Key: CASSANDRA-7882
URL: https://issues.apache.org/jira/browse/CASSANDRA-7882
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Jay Patel
Fix For: 2.1.1

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7883) Allow plugging JEMalloc for off-heap memtables


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122714#comment-14122714
 ] 

Benedict commented on CASSANDRA-7883:
-

I'm not convinced by the necessity of this, if only because we allocate in 
large-ish quantities so there's not likely to be any measurable impact. That 
said, it's also trivial and no downside, so happy to include if you post a 
patch.

 Allow plugging JEMalloc for off-heap memtables
 --

 Key: CASSANDRA-7883
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7883
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jay Patel
 Fix For: 2.1.1


 Off-heap memtables (memtable_allocation_type:offheap_objects) introduced by 
 CASSANDRA-6694 uses native GCC allocator. Provide an option to use JEMalloc 
 allocator (http://www.canonware.com/jemalloc/) which is good to reduce 
 fragmentation. 
 CASSANDRA-3997 adds below option for off-heap caches and metadata. But it's 
 not in effect for off-heap memtables. Should be use the same option or add 
 another?
 memory_allocator: JEMallocAllocator



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7882) Memtable slab allocation should scale logarithmically to improve occupancy rate


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-7882:

Summary: Memtable slab allocation should scale logarithmically to improve 
occupancy rate  (was: Allow disabling slab allocation for off-heap memtables)

 Memtable slab allocation should scale logarithmically to improve occupancy 
 rate
 ---

 Key: CASSANDRA-7882
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7882
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jay Patel
 Fix For: 2.1.1


 CASSANDRA-5935 allows option to disable region-based allocation for on-heap 
 memtables but there is no option to disable it for off-heap memtables 
 (memtable_allocation_type: offheap_objects). 
 Disabling region-based allocation will allow us to pack more tables in the 
 schema since minimum of 1MB region won't be allocated per table. Downside can 
 be more fragmentation which should be controllable by using better allocator 
 like JEMalloc.
 How about below option in yaml?:
 memtable_allocation_type: unslabbed_offheap_objects
 Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-7884) Allow setting region size for region-based memtable allocators


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict resolved CASSANDRA-7884.
-
Resolution: Won't Fix

Closing in favour of CASSANDRA-7282.

 Allow setting region size for region-based memtable allocators
 --

 Key: CASSANDRA-7884
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7884
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jay Patel
 Fix For: 2.1.1


 SlabAllocator and  NativeAllocators currently have 1 MB region size hard 
 coded in the code. Can we make it readable from cassandra.yaml ? This will 
 allow user to optimize region size based on the number of tables in the 
 cluster and/or payload size.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-7884) Allow setting region size for region-based memtable allocators


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122717#comment-14122717
 ] 

Benedict edited comment on CASSANDRA-7884 at 9/5/14 9:30 AM:
-

Closing in favour of CASSANDRA-7882.


was (Author: benedict):
Closing in favour of CASSANDRA-7282.

 Allow setting region size for region-based memtable allocators
 --

 Key: CASSANDRA-7884
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7884
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jay Patel
 Fix For: 2.1.1


 SlabAllocator and  NativeAllocators currently have 1 MB region size hard 
 coded in the code. Can we make it readable from cassandra.yaml ? This will 
 allow user to optimize region size based on the number of tables in the 
 cluster and/or payload size.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7882) Memtable slab allocation should scale logarithmically to improve occupancy rate


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-7882:

Labels: performance  (was: )

 Memtable slab allocation should scale logarithmically to improve occupancy 
 rate
 ---

 Key: CASSANDRA-7882
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7882
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jay Patel
  Labels: performance
 Fix For: 2.1.1


 CASSANDRA-5935 allows option to disable region-based allocation for on-heap 
 memtables but there is no option to disable it for off-heap memtables 
 (memtable_allocation_type: offheap_objects). 
 Disabling region-based allocation will allow us to pack more tables in the 
 schema since minimum of 1MB region won't be allocated per table. Downside can 
 be more fragmentation which should be controllable by using better allocator 
 like JEMalloc.
 How about below option in yaml?:
 memtable_allocation_type: unslabbed_offheap_objects
 Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7883) Allow plugging JEMalloc for off-heap memtables


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122728#comment-14122728
 ] 

Jay Patel commented on CASSANDRA-7883:
--

Hey Benedict, 
I already have code for this. BTW, I don't see option to assign tix to myself. 
Can you assign it to me?

 Allow plugging JEMalloc for off-heap memtables
 --

 Key: CASSANDRA-7883
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7883
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jay Patel
 Fix For: 2.1.1


 Off-heap memtables (memtable_allocation_type:offheap_objects) introduced by 
 CASSANDRA-6694 uses native GCC allocator. Provide an option to use JEMalloc 
 allocator (http://www.canonware.com/jemalloc/) which is good to reduce 
 fragmentation. 
 CASSANDRA-3997 adds below option for off-heap caches and metadata. But it's 
 not in effect for off-heap memtables. Should be use the same option or add 
 another?
 memory_allocator: JEMallocAllocator



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7882) Memtable slab allocation should scale logarithmically to improve occupancy rate


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-7882:

Assignee: Sylvain Lebresne

 Memtable slab allocation should scale logarithmically to improve occupancy 
 rate
 ---

 Key: CASSANDRA-7882
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7882
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jay Patel
Assignee: Sylvain Lebresne
  Labels: performance
 Fix For: 2.1.1


 CASSANDRA-5935 allows option to disable region-based allocation for on-heap 
 memtables but there is no option to disable it for off-heap memtables 
 (memtable_allocation_type: offheap_objects). 
 Disabling region-based allocation will allow us to pack more tables in the 
 schema since minimum of 1MB region won't be allocated per table. Downside can 
 be more fragmentation which should be controllable by using better allocator 
 like JEMalloc.
 How about below option in yaml?:
 memtable_allocation_type: unslabbed_offheap_objects
 Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7882) Memtable slab allocation should scale logarithmically to improve occupancy rate


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-7882:

Assignee: (was: Sylvain Lebresne)

 Memtable slab allocation should scale logarithmically to improve occupancy 
 rate
 ---

 Key: CASSANDRA-7882
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7882
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jay Patel
  Labels: performance
 Fix For: 2.1.1


 CASSANDRA-5935 allows option to disable region-based allocation for on-heap 
 memtables but there is no option to disable it for off-heap memtables 
 (memtable_allocation_type: offheap_objects). 
 Disabling region-based allocation will allow us to pack more tables in the 
 schema since minimum of 1MB region won't be allocated per table. Downside can 
 be more fragmentation which should be controllable by using better allocator 
 like JEMalloc.
 How about below option in yaml?:
 memtable_allocation_type: unslabbed_offheap_objects
 Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7883) Allow plugging JEMalloc for off-heap memtables


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-7883:

Assignee: Jay Patel

 Allow plugging JEMalloc for off-heap memtables
 --

 Key: CASSANDRA-7883
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7883
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jay Patel
Assignee: Jay Patel
 Fix For: 2.1.1


 Off-heap memtables (memtable_allocation_type:offheap_objects) introduced by 
 CASSANDRA-6694 uses native GCC allocator. Provide an option to use JEMalloc 
 allocator (http://www.canonware.com/jemalloc/) which is good to reduce 
 fragmentation. 
 CASSANDRA-3997 adds below option for off-heap caches and metadata. But it's 
 not in effect for off-heap memtables. Should be use the same option or add 
 another?
 memory_allocator: JEMallocAllocator



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7882) Memtable slab allocation should scale logarithmically to improve occupancy rate


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-7882:

Assignee: Jay Patel

 Memtable slab allocation should scale logarithmically to improve occupancy 
 rate
 ---

 Key: CASSANDRA-7882
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7882
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jay Patel
Assignee: Jay Patel
  Labels: performance
 Fix For: 2.1.1


 CASSANDRA-5935 allows option to disable region-based allocation for on-heap 
 memtables but there is no option to disable it for off-heap memtables 
 (memtable_allocation_type: offheap_objects). 
 Disabling region-based allocation will allow us to pack more tables in the 
 schema since minimum of 1MB region won't be allocated per table. Downside can 
 be more fragmentation which should be controllable by using better allocator 
 like JEMalloc.
 How about below option in yaml?:
 memtable_allocation_type: unslabbed_offheap_objects
 Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-7886) TombstoneOverwhelmingException should not wait for timeout

Christian Spriegel created CASSANDRA-7886:
-

 Summary: TombstoneOverwhelmingException should not wait for timeout
 Key: CASSANDRA-7886
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7886
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Christian Spriegel
Priority: Minor


*Issue*
When you have TombstoneOverwhelmingExceptions occuring in queries, this will 
cause the query to be simply dropped on every data-node, but no response is 
sent back to the coordinator. Instead the coordinator waits for the specified 
read_request_timeout_in_ms.

On the application side this can cause memory issues, since the application is 
waiting for the timeout interval for every request.Therefore, if our 
application runs into TombstoneOverwhelmingExceptions, then our entire 
application cluster goes down :-(

*Proposed solution*
I think the data nodes should send a error message to the coordinator when they 
run into a TombstoneOverwhelmingException. Then the coordinator does not have 
to wait for the timeout-interval.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7882) Memtable slab allocation should scale logarithmically to improve occupancy rate


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122734#comment-14122734
 ] 

Jay Patel commented on CASSANDRA-7882:
--

Sure. Pls. assign it to me. I think better to have a on/off option initially, 
and then we can make it on by default if it is promising for all kind of use 
cases, not just high # of tables.

 Memtable slab allocation should scale logarithmically to improve occupancy 
 rate
 ---

 Key: CASSANDRA-7882
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7882
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jay Patel
Assignee: Jay Patel
  Labels: performance
 Fix For: 2.1.1


 CASSANDRA-5935 allows option to disable region-based allocation for on-heap 
 memtables but there is no option to disable it for off-heap memtables 
 (memtable_allocation_type: offheap_objects). 
 Disabling region-based allocation will allow us to pack more tables in the 
 schema since minimum of 1MB region won't be allocated per table. Downside can 
 be more fragmentation which should be controllable by using better allocator 
 like JEMalloc.
 How about below option in yaml?:
 memtable_allocation_type: unslabbed_offheap_objects
 Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7886) TombstoneOverwhelmingException should not wait for timeout


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christian Spriegel updated CASSANDRA-7886:
--
Environment: Tested with Cassandra 2.0.8

 TombstoneOverwhelmingException should not wait for timeout
 --

 Key: CASSANDRA-7886
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7886
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: Tested with Cassandra 2.0.8
Reporter: Christian Spriegel
Priority: Minor

 *Issue*
 When you have TombstoneOverwhelmingExceptions occuring in queries, this will 
 cause the query to be simply dropped on every data-node, but no response is 
 sent back to the coordinator. Instead the coordinator waits for the specified 
 read_request_timeout_in_ms.
 On the application side this can cause memory issues, since the application 
 is waiting for the timeout interval for every request.Therefore, if our 
 application runs into TombstoneOverwhelmingExceptions, then our entire 
 application cluster goes down :-(
 *Proposed solution*
 I think the data nodes should send a error message to the coordinator when 
 they run into a TombstoneOverwhelmingException. Then the coordinator does not 
 have to wait for the timeout-interval.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-6809) Compressed Commit Log

2014-09-05 Thread Branimir Lambov (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122738#comment-14122738
]

Branimir Lambov commented on CASSANDRA-6809:

I'm doing some testing before deciding what exactly to implement, and I'm
seeing that the current implementation of memory-mapped IO for commit logs does
not help slow HDDs, but has a very noticeable effect (10-20%) on SSDs. It also
appears SSD writing is CPU-bound and not helped by compression.

Based on this and the discussion above, my plan for implementing compression is
the following:
* the current CommitLogSegment will become an abstract base class with two
subclasses that define how the buffer accepting the mutation is constructed,
and how data is written from it.
* one subclass will be the current implementation, with a memory-mapped buffer
to achieve the least CPU overhead and generally provide the fastest path for
SSDs when encryption is not required.
* the other will construct in-memory buffers of the size of the CLS, and
compress the sync sections before writing to a FileChannel (in the sync
thread). It should be trivial to add encryption to this.
* compression should improve performance on any medium if done before
encryption, so I do not think it makes any sense to add support for encryption
to the memory-mapped option at all.

The ComitLogStress test I am using cannot measure the effect of recycling
commit log files, and it seems that any change in that is orthogonal to adding
support for compression. I will leave reevaluating the need for recycling for a
separate ticket.

Compressed Commit Log
-

Key: CASSANDRA-6809
URL: https://issues.apache.org/jira/browse/CASSANDRA-6809
Project: Cassandra
Issue Type: Improvement
Reporter: Benedict
Assignee: Branimir Lambov
Priority: Minor
Labels: performance
Fix For: 3.0

It seems an unnecessary oversight that we don't compress the commit log.
Doing so should improve throughput, but some care will need to be taken to
ensure we use as much of a segment as possible. I propose decoupling the
writing of the records from the segments. Basically write into a (queue of)
DirectByteBuffer, and have the sync thread compress, say, ~64K chunks every X
MB written to the CL (where X is ordinarily CLS size), and then pack as many
of the compressed chunks into a CLS as possible.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7886) TombstoneOverwhelmingException should not wait for timeout


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christian Spriegel updated CASSANDRA-7886:
--
Description: 
*Issue*
When you have TombstoneOverwhelmingExceptions occuring in queries, this will 
cause the query to be simply dropped on every data-node, but no response is 
sent back to the coordinator. Instead the coordinator waits for the specified 
read_request_timeout_in_ms.

On the application side this can cause memory issues, since the application is 
waiting for the timeout interval for every request.Therefore, if our 
application runs into TombstoneOverwhelmingExceptions, then (sooner or later) 
our entire application cluster goes down :-(

*Proposed solution*
I think the data nodes should send a error message to the coordinator when they 
run into a TombstoneOverwhelmingException. Then the coordinator does not have 
to wait for the timeout-interval.



  was:
*Issue*
When you have TombstoneOverwhelmingExceptions occuring in queries, this will 
cause the query to be simply dropped on every data-node, but no response is 
sent back to the coordinator. Instead the coordinator waits for the specified 
read_request_timeout_in_ms.

On the application side this can cause memory issues, since the application is 
waiting for the timeout interval for every request.Therefore, if our 
application runs into TombstoneOverwhelmingExceptions, then our entire 
application cluster goes down :-(

*Proposed solution*
I think the data nodes should send a error message to the coordinator when they 
run into a TombstoneOverwhelmingException. Then the coordinator does not have 
to wait for the timeout-interval.




 TombstoneOverwhelmingException should not wait for timeout
 --

 Key: CASSANDRA-7886
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7886
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: Tested with Cassandra 2.0.8
Reporter: Christian Spriegel
Priority: Minor

 *Issue*
 When you have TombstoneOverwhelmingExceptions occuring in queries, this will 
 cause the query to be simply dropped on every data-node, but no response is 
 sent back to the coordinator. Instead the coordinator waits for the specified 
 read_request_timeout_in_ms.
 On the application side this can cause memory issues, since the application 
 is waiting for the timeout interval for every request.Therefore, if our 
 application runs into TombstoneOverwhelmingExceptions, then (sooner or later) 
 our entire application cluster goes down :-(
 *Proposed solution*
 I think the data nodes should send a error message to the coordinator when 
 they run into a TombstoneOverwhelmingException. Then the coordinator does not 
 have to wait for the timeout-interval.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7886) TombstoneOverwhelmingException should not wait for timeout


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122745#comment-14122745
 ] 

Sylvain Lebresne commented on CASSANDRA-7886:
-

If having some request timeout makes your entire application cluster to go 
down, then to be honest I think you should fix your application.

Not to say that we shouldn't do this, but this will require an intra-node 
protocol change so we're talking C* 3.0 at best.

 TombstoneOverwhelmingException should not wait for timeout
 --

 Key: CASSANDRA-7886
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7886
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: Tested with Cassandra 2.0.8
Reporter: Christian Spriegel
Priority: Minor

 *Issue*
 When you have TombstoneOverwhelmingExceptions occuring in queries, this will 
 cause the query to be simply dropped on every data-node, but no response is 
 sent back to the coordinator. Instead the coordinator waits for the specified 
 read_request_timeout_in_ms.
 On the application side this can cause memory issues, since the application 
 is waiting for the timeout interval for every request.Therefore, if our 
 application runs into TombstoneOverwhelmingExceptions, then (sooner or later) 
 our entire application cluster goes down :-(
 *Proposed solution*
 I think the data nodes should send a error message to the coordinator when 
 they run into a TombstoneOverwhelmingException. Then the coordinator does not 
 have to wait for the timeout-interval.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-7887) Skip completing a streamed file if it is identical to one of local files

2014-09-05 Thread Takenori Sato (JIRA)

Takenori Sato created CASSANDRA-7887:


 Summary: Skip completing a streamed file if it is identical to one 
of local files
 Key: CASSANDRA-7887
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7887
 Project: Cassandra
  Issue Type: Improvement
Reporter: Takenori Sato


A repair could transfer files that have been done before. Under such a 
circumstance, those files would get piled up, and a user could experience 
critical problems such as:

1. overwhelming compaction load
2. disk full

This leads to insufficient repairs, and data inconsistencies.

What about skipping such a file before committing its write(calling 
SSTableWrite.closeAndOpenReader)? For example, by comparing MD5 hash of a 
streamed file with those of local files.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-6809) Compressed Commit Log


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122754#comment-14122754
 ] 

Benedict commented on CASSANDRA-6809:
-

I'm +1 on your over all intended approach, but I'd appreciate some elaboration 
on what steps you've taken for testing:

* What platforms?
* What CPU config?
* What measurement of CPU time allocation? (10-20% bump may be reasonable if 
its kernel time, but I'm a little surprised it has such an impact, as we'd be 
talking about bulk writing large quantities of linear data, which should have 
reasonably low CPU impact)

If you could post your in progress branches and tests to github so I can follow 
what you're doing, that would be great

 Compressed Commit Log
 -

 Key: CASSANDRA-6809
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6809
 Project: Cassandra
  Issue Type: Improvement
Reporter: Benedict
Assignee: Branimir Lambov
Priority: Minor
  Labels: performance
 Fix For: 3.0


 It seems an unnecessary oversight that we don't compress the commit log. 
 Doing so should improve throughput, but some care will need to be taken to 
 ensure we use as much of a segment as possible. I propose decoupling the 
 writing of the records from the segments. Basically write into a (queue of) 
 DirectByteBuffer, and have the sync thread compress, say, ~64K chunks every X 
 MB written to the CL (where X is ordinarily CLS size), and then pack as many 
 of the compressed chunks into a CLS as possible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7886) TombstoneOverwhelmingException should not wait for timeout


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122756#comment-14122756
 ] 

Christian Spriegel commented on CASSANDRA-7886:
---

[~slebresne]: Customer keep sending in requests. So if cassandra suddenly 
decides to make every request wait for 15 sec. (config increased) then we run 
out of heap, because requests pile up :-(

As a workaround we can probably decrease the timeout setting, but the behaviour 
should be changed imho.

Can we set fixversion to 3.0 already so that this ticket wont be forgotten?

 TombstoneOverwhelmingException should not wait for timeout
 --

 Key: CASSANDRA-7886
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7886
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: Tested with Cassandra 2.0.8
Reporter: Christian Spriegel
Priority: Minor

 *Issue*
 When you have TombstoneOverwhelmingExceptions occuring in queries, this will 
 cause the query to be simply dropped on every data-node, but no response is 
 sent back to the coordinator. Instead the coordinator waits for the specified 
 read_request_timeout_in_ms.
 On the application side this can cause memory issues, since the application 
 is waiting for the timeout interval for every request.Therefore, if our 
 application runs into TombstoneOverwhelmingExceptions, then (sooner or later) 
 our entire application cluster goes down :-(
 *Proposed solution*
 I think the data nodes should send a error message to the coordinator when 
 they run into a TombstoneOverwhelmingException. Then the coordinator does not 
 have to wait for the timeout-interval.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-7886) TombstoneOverwhelmingException should not wait for timeout


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122756#comment-14122756
 ] 

Christian Spriegel edited comment on CASSANDRA-7886 at 9/5/14 10:09 AM:


[~slebresne]: Customer keep sending in requests. So if cassandra suddenly 
decides to make every request wait for 15 sec. (config increased) then we run 
out of heap, because requests pile up :-(

As a workaround we can probably decrease the timeout setting, but the behaviour 
should be changed imho.

Can we set fixversion to 3.0 already so that this ticket wont be forgotten?

edit: Thanks for the fast response :-)


was (Author: christianmovi):
[~slebresne]: Customer keep sending in requests. So if cassandra suddenly 
decides to make every request wait for 15 sec. (config increased) then we run 
out of heap, because requests pile up :-(

As a workaround we can probably decrease the timeout setting, but the behaviour 
should be changed imho.

Can we set fixversion to 3.0 already so that this ticket wont be forgotten?

 TombstoneOverwhelmingException should not wait for timeout
 --

 Key: CASSANDRA-7886
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7886
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: Tested with Cassandra 2.0.8
Reporter: Christian Spriegel
Priority: Minor

 *Issue*
 When you have TombstoneOverwhelmingExceptions occuring in queries, this will 
 cause the query to be simply dropped on every data-node, but no response is 
 sent back to the coordinator. Instead the coordinator waits for the specified 
 read_request_timeout_in_ms.
 On the application side this can cause memory issues, since the application 
 is waiting for the timeout interval for every request.Therefore, if our 
 application runs into TombstoneOverwhelmingExceptions, then (sooner or later) 
 our entire application cluster goes down :-(
 *Proposed solution*
 I think the data nodes should send a error message to the coordinator when 
 they run into a TombstoneOverwhelmingException. Then the coordinator does not 
 have to wait for the timeout-interval.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7870) Cannot execute logged batch when only the coordinator node is alive

2014-09-05 Thread Sergio Bossa (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122779#comment-14122779
 ] 

Sergio Bossa commented on CASSANDRA-7870:
-

Ok, so feel free to wontfix this.

 Cannot execute logged batch when only the coordinator node is alive
 ---

 Key: CASSANDRA-7870
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7870
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Sergio Bossa
Priority: Critical

 As per issue summary.
 This is probably a bug, rather than a consequence of needing to replicate the 
 batchlog, as if only the coordinator is alive the batch cannot be partially 
 executed on other nodes (as there are no other nodes).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7254) NPE on startup if another Cassandra instance is already running

2014-09-05 Thread Katsutoshi Nagaoka (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122780#comment-14122780
 ] 

Katsutoshi Nagaoka commented on CASSANDRA-7254:
---

Thank you for this workaround. 

 NPE on startup if another Cassandra instance is already running
 ---

 Key: CASSANDRA-7254
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7254
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Tyler Hobbs
Assignee: Brandon Williams
Priority: Minor
 Fix For: 2.0.10, 2.1 rc3

 Attachments: 7254.txt


 After CASSANDRA-7087, if you try to start cassandra while another instance is 
 already running, you'll see something like this:
 {noformat}
 $ bin/cassandra -f
 Error: Exception thrown by the agent : java.lang.NullPointerException
 {noformat}
 This is probably a JVM bug, but we should confirm that, open a JVM ticket, 
 and see if we can give a more useful error message on the C* side.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7886) TombstoneOverwhelmingException should not wait for timeout


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-7886:

Fix Version/s: 3.0

 TombstoneOverwhelmingException should not wait for timeout
 --

 Key: CASSANDRA-7886
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7886
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: Tested with Cassandra 2.0.8
Reporter: Christian Spriegel
Priority: Minor
 Fix For: 3.0


 *Issue*
 When you have TombstoneOverwhelmingExceptions occuring in queries, this will 
 cause the query to be simply dropped on every data-node, but no response is 
 sent back to the coordinator. Instead the coordinator waits for the specified 
 read_request_timeout_in_ms.
 On the application side this can cause memory issues, since the application 
 is waiting for the timeout interval for every request.Therefore, if our 
 application runs into TombstoneOverwhelmingExceptions, then (sooner or later) 
 our entire application cluster goes down :-(
 *Proposed solution*
 I think the data nodes should send a error message to the coordinator when 
 they run into a TombstoneOverwhelmingException. Then the coordinator does not 
 have to wait for the timeout-interval.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-6809) Compressed Commit Log

2014-09-05 Thread Branimir Lambov (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Branimir Lambov updated CASSANDRA-6809:
---
Attachment: logtest.txt

Raw test data.

 Compressed Commit Log
 -

 Key: CASSANDRA-6809
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6809
 Project: Cassandra
  Issue Type: Improvement
Reporter: Benedict
Assignee: Branimir Lambov
Priority: Minor
  Labels: performance
 Fix For: 3.0

 Attachments: logtest.txt


 It seems an unnecessary oversight that we don't compress the commit log. 
 Doing so should improve throughput, but some care will need to be taken to 
 ensure we use as much of a segment as possible. I propose decoupling the 
 writing of the records from the segments. Basically write into a (queue of) 
 DirectByteBuffer, and have the sync thread compress, say, ~64K chunks every X 
 MB written to the CL (where X is ordinarily CLS size), and then pack as many 
 of the compressed chunks into a CLS as possible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-6809) Compressed Commit Log

2014-09-05 Thread Branimir Lambov (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122807#comment-14122807
 ] 

Branimir Lambov edited comment on CASSANDRA-6809 at 9/5/14 11:25 AM:
-

Raw test data attached in logtest.txt.


was (Author: blambov):
Raw test data.

 Compressed Commit Log
 -

 Key: CASSANDRA-6809
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6809
 Project: Cassandra
  Issue Type: Improvement
Reporter: Benedict
Assignee: Branimir Lambov
Priority: Minor
  Labels: performance
 Fix For: 3.0

 Attachments: logtest.txt


 It seems an unnecessary oversight that we don't compress the commit log. 
 Doing so should improve throughput, but some care will need to be taken to 
 ensure we use as much of a segment as possible. I propose decoupling the 
 writing of the records from the segments. Basically write into a (queue of) 
 DirectByteBuffer, and have the sync thread compress, say, ~64K chunks every X 
 MB written to the CL (where X is ordinarily CLS size), and then pack as many 
 of the compressed chunks into a CLS as possible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7886) TombstoneOverwhelmingException should not wait for timeout


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122809#comment-14122809
 ] 

Sylvain Lebresne commented on CASSANDRA-7886:
-

bq. So if cassandra suddenly decides to make every request wait for 15 sec

I meant that if *every* requests hits TombstoneOverwhelmingException, you 
either have set the threshold way too low, or you have a problem with you data 
model which generates too much tombstones. And if every requests hits 
TombstoneOverwhelmingException, even if C* was to return more quickly from it, 
you'll still won't make much progress. And if you just have a couple of 
specific requets that might end up hitting the tombstone threshold occasionally 
and that still break your application cluster, that also sound like something 
you should mitigate client side (after all, getting timeout can happen, 
TombstoneOverwhelmingException or not).

bq. Can we set fixversion to 3.0 already so that this ticket wont be forgotten?

Sure, but setting a fixversion is never a promise. 

 TombstoneOverwhelmingException should not wait for timeout
 --

 Key: CASSANDRA-7886
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7886
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: Tested with Cassandra 2.0.8
Reporter: Christian Spriegel
Priority: Minor
 Fix For: 3.0


 *Issue*
 When you have TombstoneOverwhelmingExceptions occuring in queries, this will 
 cause the query to be simply dropped on every data-node, but no response is 
 sent back to the coordinator. Instead the coordinator waits for the specified 
 read_request_timeout_in_ms.
 On the application side this can cause memory issues, since the application 
 is waiting for the timeout interval for every request.Therefore, if our 
 application runs into TombstoneOverwhelmingExceptions, then (sooner or later) 
 our entire application cluster goes down :-(
 *Proposed solution*
 I think the data nodes should send a error message to the coordinator when 
 they run into a TombstoneOverwhelmingException. Then the coordinator does not 
 have to wait for the timeout-interval.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7886) TombstoneOverwhelmingException should not wait for timeout

[
https://issues.apache.org/jira/browse/CASSANDRA-7886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122830#comment-14122830
]

Christian Spriegel commented on CASSANDRA-7886:
---

[~slebresne]:
{quote}I meant that if every requests hits TombstoneOverwhelmingException{quote}
My story is a bit longer, in normal operation this does not happen, not even
close.

But sometimes our customers mess up in their backend systems: Sometimes their
backend will send the same request in an endless loop, where they delete+create
a column in a row. This causes many tombstones to be created very quickly.

Currently this single customer brings down our entire landscape, due to his
requests piling up in our tomcat. Which also affects other customers.

If Cassandra were to fail instantly, then his requests would run into an error
(which they should, because he is using it wrong) and therefore would not pile
up.

{quote}Sure, but setting a fixversion is never a promise. {quote}
Thanks!
I know. But at least somebody will have it on his radar. (I hope) :-)

TombstoneOverwhelmingException should not wait for timeout
--

Key: CASSANDRA-7886
URL: https://issues.apache.org/jira/browse/CASSANDRA-7886
Project: Cassandra
Issue Type: Improvement
Components: Core
Environment: Tested with Cassandra 2.0.8
Reporter: Christian Spriegel
Priority: Minor
Fix For: 3.0

*Issue*
When you have TombstoneOverwhelmingExceptions occuring in queries, this will
cause the query to be simply dropped on every data-node, but no response is
sent back to the coordinator. Instead the coordinator waits for the specified
read_request_timeout_in_ms.
On the application side this can cause memory issues, since the application
is waiting for the timeout interval for every request.Therefore, if our
application runs into TombstoneOverwhelmingExceptions, then (sooner or later)
our entire application cluster goes down :-(
*Proposed solution*
I think the data nodes should send a error message to the coordinator when
they run into a TombstoneOverwhelmingException. Then the coordinator does not
have to wait for the timeout-interval.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-7870) Cannot execute logged batch when only the coordinator node is alive

2014-09-05 Thread Aleksey Yeschenko (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-7870.
--
Resolution: Won't Fix

 Cannot execute logged batch when only the coordinator node is alive
 ---

 Key: CASSANDRA-7870
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7870
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Sergio Bossa
Priority: Critical

 As per issue summary.
 This is probably a bug, rather than a consequence of needing to replicate the 
 batchlog, as if only the coordinator is alive the batch cannot be partially 
 executed on other nodes (as there are no other nodes).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7874) Validate functionality of different JSR-223 providers in UDFs


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp updated CASSANDRA-7874:

Attachment: 7874.txt

Successfully validated these JSR-223 implementations for UDFs:
* Groovy v2.3.6
* JRuby 1.7.15
* Jython 2.5.3
* Scala 2.11.2

These JSR-223 implementations do not work:
* Clojure 1.5.1/1.6 : not working compilable script support in JSR-223 
implementation
* Jaskell 1.0 : seems to be unmaintained since 7 or 8 yrs

I've introduced some directories below {{lib/}} to keep scripting engine jars 
separate from C* jars. Directory structure is like this:
{noformat}
lib/jsr223/clojure
lib/jsr223/groovy
lib/jsr223/jaskell
lib/jsr223/jruby
lib/jsr223/jython
lib/jsr223/scala
{noformat}
Each of these directories contains a {{README.txt}} with instructions how to 
install those scripting engines or why they do not work.

{{cassandra.in.sh}} has been modified to scan all directories below 
{{lib/jsr223}} for jar files to add to C* daemon classpath.
Additionally JRuby, Jython and Scala require some system properties, which are 
added, if these scripting languages are detected.


 Validate functionality of different JSR-223 providers in UDFs
 -

 Key: CASSANDRA-7874
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7874
 Project: Cassandra
  Issue Type: Task
  Components: Core
Reporter: Robert Stupp
Assignee: Robert Stupp
  Labels: udf
 Attachments: 7874.txt


 CASSANDRA-7526 introduces ability to support optional JSR-223 providers like 
 Clojure, Jython, Groovy or JRuby.
 This ticket is about to test functionality with these providers but not to 
 include them in C* distribution.
 Expected result is a how to document, wiki page or similar.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7769) Implement pg-style dollar syntax for string constants


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122867#comment-14122867
 ] 

Robert Stupp commented on CASSANDRA-7769:
-

/cc [~iamaleksey] or [~thobbs] Do you have some idea how to add proper pg-style 
string support to cqlsh without refactoring the whole SafeScanner stuff?

 Implement pg-style dollar syntax for string constants
 -

 Key: CASSANDRA-7769
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7769
 Project: Cassandra
  Issue Type: Improvement
Reporter: Robert Stupp
Assignee: Robert Stupp
 Fix For: 3.0

 Attachments: 7769.txt, 7769v2.txt


 Follow-up of CASSANDRA-7740:
 {{$function$...$function$}} in addition to string style variant.
 See also 
 http://www.postgresql.org/docs/9.1/static/sql-syntax-lexical.html#SQL-SYNTAX-DOLLAR-QUOTING



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7519) Further stress improvements to generate more realistic workloads

2014-09-05 Thread T Jake Luciani (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122877#comment-14122877
 ] 

T Jake Luciani commented on CASSANDRA-7519:
---

+1

 Further stress improvements to generate more realistic workloads
 

 Key: CASSANDRA-7519
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7519
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Benedict
Assignee: Benedict
Priority: Minor
  Labels: tools
 Fix For: 2.1.1


 We generally believe that the most common workload is for reads to 
 exponentially prefer most recently written data. However as stress currently 
 behaves we have two id generation modes: sequential and random (although 
 random can be distributed). I propose introducing a new mode which is 
 somewhat like sequential, except we essentially 'look back' from the current 
 id by some amount defined by a distribution. I may possibly make the position 
 only increment as it's first written to also, so that this mode can be run 
 from a clean slate with a mixed workload. This should allow is to generate 
 workloads that are more representative.
 At the same time, I will introduce a timestamp value generator for primary 
 key columns that is strictly ascending, i.e. has some random component but is 
 based off of the actual system time (or some shared monotonically increasing 
 state) so that we can again generate a more realistic workload. This may be 
 challenging to tie in with the new procedurally generated partitions, but I'm 
 sure it can be done without too much difficulty.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

git commit: Fix assertion encountered when using heap_buffers

2014-09-05 Thread jake

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 dd972e6c3 - 3a410175f


Fix assertion encountered when using heap_buffers

Patch by tjake; reviewed be belliottsmitch for CASSANDRA-7862


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3a410175
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3a410175
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3a410175

Branch: refs/heads/cassandra-2.1
Commit: 3a410175ff8f5e37c263f18315feb60fc217c3dc
Parents: dd972e6
Author: Jake Luciani j...@apache.org
Authored: Fri Sep 5 08:47:01 2014 -0400
Committer: Jake Luciani j...@apache.org
Committed: Fri Sep 5 08:47:01 2014 -0400

--
 .../apache/cassandra/db/composites/AbstractSimpleCellNameType.java  | 1 -
 1 file changed, 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3a410175/src/java/org/apache/cassandra/db/composites/AbstractSimpleCellNameType.java
--
diff --git 
a/src/java/org/apache/cassandra/db/composites/AbstractSimpleCellNameType.java 
b/src/java/org/apache/cassandra/db/composites/AbstractSimpleCellNameType.java
index f9c9641..b3f4778 100644
--- 
a/src/java/org/apache/cassandra/db/composites/AbstractSimpleCellNameType.java
+++ 
b/src/java/org/apache/cassandra/db/composites/AbstractSimpleCellNameType.java
@@ -86,7 +86,6 @@ public abstract class AbstractSimpleCellNameType extends 
AbstractCellNameType
 {
 ByteBuffer b1 = c1.toByteBuffer();
 ByteBuffer b2 = c2.toByteBuffer();
-assert b1.hasRemaining()  b2.hasRemaining();
 return ByteBufferUtil.compareUnsigned(b1, b2);
 }

[jira] [Comment Edited] (CASSANDRA-7874) Validate functionality of different JSR-223 providers in UDFs


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122864#comment-14122864
 ] 

Robert Stupp edited comment on CASSANDRA-7874 at 9/5/14 12:52 PM:
--

Successfully validated these JSR-223 implementations for UDFs:
* Groovy v2.3.6
* JRuby 1.7.15
* Jython 2.5.3
* Scala 2.11.2

These JSR-223 implementations do not work:
* Clojure 1.5.1/1.6 : not working compilable script support in JSR-223 
implementation
* Jaskell 1.0 : seems to be unmaintained since 7 or 8 yrs (accoring to 
https://java.net/projects/scripting/sources/svn/show/trunk/engines/jaskell?rev=236)

I've introduced some directories below {{lib/}} to keep scripting engine jars 
separate from C* jars. Directory structure is like this:
{noformat}
lib/jsr223/clojure
lib/jsr223/groovy
lib/jsr223/jaskell
lib/jsr223/jruby
lib/jsr223/jython
lib/jsr223/scala
{noformat}
Each of these directories contains a {{README.txt}} with instructions how to 
install those scripting engines or why they do not work.

{{cassandra.in.sh}} has been modified to scan all directories below 
{{lib/jsr223}} for jar files to add to C* daemon classpath.
Additionally JRuby, Jython and Scala require some system properties, which are 
added, if these scripting languages are detected.



was (Author: snazy):
Successfully validated these JSR-223 implementations for UDFs:
* Groovy v2.3.6
* JRuby 1.7.15
* Jython 2.5.3
* Scala 2.11.2

These JSR-223 implementations do not work:
* Clojure 1.5.1/1.6 : not working compilable script support in JSR-223 
implementation
* Jaskell 1.0 : seems to be unmaintained since 7 or 8 yrs

I've introduced some directories below {{lib/}} to keep scripting engine jars 
separate from C* jars. Directory structure is like this:
{noformat}
lib/jsr223/clojure
lib/jsr223/groovy
lib/jsr223/jaskell
lib/jsr223/jruby
lib/jsr223/jython
lib/jsr223/scala
{noformat}
Each of these directories contains a {{README.txt}} with instructions how to 
install those scripting engines or why they do not work.

{{cassandra.in.sh}} has been modified to scan all directories below 
{{lib/jsr223}} for jar files to add to C* daemon classpath.
Additionally JRuby, Jython and Scala require some system properties, which are 
added, if these scripting languages are detected.


 Validate functionality of different JSR-223 providers in UDFs
 -

 Key: CASSANDRA-7874
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7874
 Project: Cassandra
  Issue Type: Task
  Components: Core
Reporter: Robert Stupp
Assignee: Robert Stupp
  Labels: udf
 Attachments: 7874.txt


 CASSANDRA-7526 introduces ability to support optional JSR-223 providers like 
 Clojure, Jython, Groovy or JRuby.
 This ticket is about to test functionality with these providers but not to 
 include them in C* distribution.
 Expected result is a how to document, wiki page or similar.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/2] git commit: Merge branch 'cassandra-2.1' into trunk

2014-09-05 Thread jake

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3f1820f1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3f1820f1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3f1820f1

Branch: refs/heads/trunk
Commit: 3f1820f1d785d262c4bd7ddc9e487307bceb5e86
Parents: 37a9d40 3a41017
Author: Jake Luciani j...@apache.org
Authored: Fri Sep 5 08:57:34 2014 -0400
Committer: Jake Luciani j...@apache.org
Committed: Fri Sep 5 08:57:34 2014 -0400

--
 .../apache/cassandra/db/composites/AbstractSimpleCellNameType.java  | 1 -
 1 file changed, 1 deletion(-)
--

[1/2] git commit: Fix assertion encountered when using heap_buffers

2014-09-05 Thread jake

Repository: cassandra
Updated Branches:
  refs/heads/trunk 37a9d405c - 3f1820f1d


Fix assertion encountered when using heap_buffers

Patch by tjake; reviewed be belliottsmitch for CASSANDRA-7862


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3a410175
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3a410175
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3a410175

Branch: refs/heads/trunk
Commit: 3a410175ff8f5e37c263f18315feb60fc217c3dc
Parents: dd972e6
Author: Jake Luciani j...@apache.org
Authored: Fri Sep 5 08:47:01 2014 -0400
Committer: Jake Luciani j...@apache.org
Committed: Fri Sep 5 08:47:01 2014 -0400

--
 .../apache/cassandra/db/composites/AbstractSimpleCellNameType.java  | 1 -
 1 file changed, 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3a410175/src/java/org/apache/cassandra/db/composites/AbstractSimpleCellNameType.java
--
diff --git 
a/src/java/org/apache/cassandra/db/composites/AbstractSimpleCellNameType.java 
b/src/java/org/apache/cassandra/db/composites/AbstractSimpleCellNameType.java
index f9c9641..b3f4778 100644
--- 
a/src/java/org/apache/cassandra/db/composites/AbstractSimpleCellNameType.java
+++ 
b/src/java/org/apache/cassandra/db/composites/AbstractSimpleCellNameType.java
@@ -86,7 +86,6 @@ public abstract class AbstractSimpleCellNameType extends 
AbstractCellNameType
 {
 ByteBuffer b1 = c1.toByteBuffer();
 ByteBuffer b2 = c2.toByteBuffer();
-assert b1.hasRemaining()  b2.hasRemaining();
 return ByteBufferUtil.compareUnsigned(b1, b2);
 }

[jira] [Updated] (CASSANDRA-7858) Fix UDF dtest


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp updated CASSANDRA-7858:

Attachment: 7858.txt

attached trivial patch

 Fix UDF dtest
 -

 Key: CASSANDRA-7858
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7858
 Project: Cassandra
  Issue Type: Bug
Reporter: Robert Stupp
Assignee: Robert Stupp
 Attachments: 7858.txt


 dtest for UDFs does no longer work - ticket's about to fix the dtest



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7858) Fix UDF dtest


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp updated CASSANDRA-7858:

Attachment: 7858.txt

 Fix UDF dtest
 -

 Key: CASSANDRA-7858
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7858
 Project: Cassandra
  Issue Type: Bug
Reporter: Robert Stupp
Assignee: Robert Stupp
 Attachments: 7858.txt


 dtest for UDFs does no longer work - ticket's about to fix the dtest



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7858) Fix UDF dtest


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp updated CASSANDRA-7858:

Attachment: (was: 7858.txt)

 Fix UDF dtest
 -

 Key: CASSANDRA-7858
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7858
 Project: Cassandra
  Issue Type: Bug
Reporter: Robert Stupp
Assignee: Robert Stupp

 dtest for UDFs does no longer work - ticket's about to fix the dtest



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7858) Fix UDF dtest


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-7858:
---
Reviewer: Philip Thompson

 Fix UDF dtest
 -

 Key: CASSANDRA-7858
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7858
 Project: Cassandra
  Issue Type: Bug
Reporter: Robert Stupp
Assignee: Robert Stupp

 dtest for UDFs does no longer work - ticket's about to fix the dtest



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7858) Fix UDF dtest


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122910#comment-14122910
 ] 

Philip Thompson commented on CASSANDRA-7858:


[~snazy] Can you go ahead and also submit this as a PR at 
https://github.com/riptano/cassandra-dtest/pulls ?

 Fix UDF dtest
 -

 Key: CASSANDRA-7858
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7858
 Project: Cassandra
  Issue Type: Bug
Reporter: Robert Stupp
Assignee: Robert Stupp
 Attachments: 7858.txt


 dtest for UDFs does no longer work - ticket's about to fix the dtest



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[5/6] git commit: Merge branch 'cassandra-2.1.0' into cassandra-2.1

Merge branch 'cassandra-2.1.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c6b509d7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c6b509d7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c6b509d7

Branch: refs/heads/cassandra-2.1
Commit: c6b509d751aeb8aa511bfaecb94af0a080acd426
Parents: 3a41017 cf76ba3
Author: Brandon Williams brandonwilli...@apache.org
Authored: Fri Sep 5 08:14:01 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Fri Sep 5 08:14:17 2014 -0500

--
 test/conf/cassandra.yaml   | 2 --
 test/pig/org/apache/cassandra/pig/PigTestBase.java | 2 +-
 2 files changed, 1 insertion(+), 3 deletions(-)
--

[3/6] git commit: use correct native port in pig-test

use correct native port in pig-test


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cf76ba3d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cf76ba3d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cf76ba3d

Branch: refs/heads/trunk
Commit: cf76ba3d7e7723888a3d00393569f62312ab2703
Parents: 4501488
Author: Brandon Williams brandonwilli...@apache.org
Authored: Fri Sep 5 08:13:49 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Fri Sep 5 08:13:49 2014 -0500

--
 test/pig/org/apache/cassandra/pig/PigTestBase.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf76ba3d/test/pig/org/apache/cassandra/pig/PigTestBase.java
--
diff --git a/test/pig/org/apache/cassandra/pig/PigTestBase.java 
b/test/pig/org/apache/cassandra/pig/PigTestBase.java
index 002fbba..4b3e422 100644
--- a/test/pig/org/apache/cassandra/pig/PigTestBase.java
+++ b/test/pig/org/apache/cassandra/pig/PigTestBase.java
@@ -68,7 +68,7 @@ public class PigTestBase extends SchemaLoader
 protected static String defaultParameters= 
init_address=localhostrpc_port=9170partitioner=org.apache.cassandra.dht.ByteOrderedPartitioner;
 protected static String nativeParameters = 
core_conns=2max_conns=10min_simult_reqs=3max_simult_reqs=10native_timeout=1000
  +

native_read_timeout=1000send_buff_size=4096receive_buff_size=4096solinger=3
 +
-   
tcp_nodelay=truereuse_address=truekeep_alive=truenative_port=9052;
+   
tcp_nodelay=truereuse_address=truekeep_alive=truenative_port=9042;
 
 static
 {

[4/6] git commit: Merge branch 'cassandra-2.1.0' into cassandra-2.1

Merge branch 'cassandra-2.1.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c6b509d7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c6b509d7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c6b509d7

Branch: refs/heads/trunk
Commit: c6b509d751aeb8aa511bfaecb94af0a080acd426
Parents: 3a41017 cf76ba3
Author: Brandon Williams brandonwilli...@apache.org
Authored: Fri Sep 5 08:14:01 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Fri Sep 5 08:14:17 2014 -0500

--
 test/conf/cassandra.yaml   | 2 --
 test/pig/org/apache/cassandra/pig/PigTestBase.java | 2 +-
 2 files changed, 1 insertion(+), 3 deletions(-)
--

[1/6] git commit: use correct native port in pig-test

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 3a410175f - c6b509d75
  refs/heads/cassandra-2.1.0 45014886e - cf76ba3d7
  refs/heads/trunk 3f1820f1d - 33ceddafa


use correct native port in pig-test


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cf76ba3d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cf76ba3d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cf76ba3d

Branch: refs/heads/cassandra-2.1
Commit: cf76ba3d7e7723888a3d00393569f62312ab2703
Parents: 4501488
Author: Brandon Williams brandonwilli...@apache.org
Authored: Fri Sep 5 08:13:49 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Fri Sep 5 08:13:49 2014 -0500

--
 test/pig/org/apache/cassandra/pig/PigTestBase.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf76ba3d/test/pig/org/apache/cassandra/pig/PigTestBase.java
--
diff --git a/test/pig/org/apache/cassandra/pig/PigTestBase.java 
b/test/pig/org/apache/cassandra/pig/PigTestBase.java
index 002fbba..4b3e422 100644
--- a/test/pig/org/apache/cassandra/pig/PigTestBase.java
+++ b/test/pig/org/apache/cassandra/pig/PigTestBase.java
@@ -68,7 +68,7 @@ public class PigTestBase extends SchemaLoader
 protected static String defaultParameters= 
init_address=localhostrpc_port=9170partitioner=org.apache.cassandra.dht.ByteOrderedPartitioner;
 protected static String nativeParameters = 
core_conns=2max_conns=10min_simult_reqs=3max_simult_reqs=10native_timeout=1000
  +

native_read_timeout=1000send_buff_size=4096receive_buff_size=4096solinger=3
 +
-   
tcp_nodelay=truereuse_address=truekeep_alive=truenative_port=9052;
+   
tcp_nodelay=truereuse_address=truekeep_alive=truenative_port=9042;
 
 static
 {

[2/6] git commit: use correct native port in pig-test

use correct native port in pig-test


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cf76ba3d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cf76ba3d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cf76ba3d

Branch: refs/heads/cassandra-2.1.0
Commit: cf76ba3d7e7723888a3d00393569f62312ab2703
Parents: 4501488
Author: Brandon Williams brandonwilli...@apache.org
Authored: Fri Sep 5 08:13:49 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Fri Sep 5 08:13:49 2014 -0500

--
 test/pig/org/apache/cassandra/pig/PigTestBase.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf76ba3d/test/pig/org/apache/cassandra/pig/PigTestBase.java
--
diff --git a/test/pig/org/apache/cassandra/pig/PigTestBase.java 
b/test/pig/org/apache/cassandra/pig/PigTestBase.java
index 002fbba..4b3e422 100644
--- a/test/pig/org/apache/cassandra/pig/PigTestBase.java
+++ b/test/pig/org/apache/cassandra/pig/PigTestBase.java
@@ -68,7 +68,7 @@ public class PigTestBase extends SchemaLoader
 protected static String defaultParameters= 
init_address=localhostrpc_port=9170partitioner=org.apache.cassandra.dht.ByteOrderedPartitioner;
 protected static String nativeParameters = 
core_conns=2max_conns=10min_simult_reqs=3max_simult_reqs=10native_timeout=1000
  +

native_read_timeout=1000send_buff_size=4096receive_buff_size=4096solinger=3
 +
-   
tcp_nodelay=truereuse_address=truekeep_alive=truenative_port=9052;
+   
tcp_nodelay=truereuse_address=truekeep_alive=truenative_port=9042;
 
 static
 {

[6/6] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/33ceddaf
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/33ceddaf
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/33ceddaf

Branch: refs/heads/trunk
Commit: 33ceddafab48e7d3c216dc224bcd63d4b469f911
Parents: 3f1820f c6b509d
Author: Brandon Williams brandonwilli...@apache.org
Authored: Fri Sep 5 08:14:29 2014 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Fri Sep 5 08:14:29 2014 -0500

--
 test/conf/cassandra.yaml   | 2 --
 test/pig/org/apache/cassandra/pig/PigTestBase.java | 2 +-
 2 files changed, 1 insertion(+), 3 deletions(-)
--

[jira] [Commented] (CASSANDRA-7858) Fix UDF dtest


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122913#comment-14122913
 ] 

Robert Stupp commented on CASSANDRA-7858:
-

yep - give me a few minutes

 Fix UDF dtest
 -

 Key: CASSANDRA-7858
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7858
 Project: Cassandra
  Issue Type: Bug
Reporter: Robert Stupp
Assignee: Robert Stupp
 Attachments: 7858.txt


 dtest for UDFs does no longer work - ticket's about to fix the dtest



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7858) Fix UDF dtest


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122915#comment-14122915
 ] 

Philip Thompson commented on CASSANDRA-7858:


But +1 to the patch itself.

 Fix UDF dtest
 -

 Key: CASSANDRA-7858
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7858
 Project: Cassandra
  Issue Type: Bug
Reporter: Robert Stupp
Assignee: Robert Stupp
 Attachments: 7858.txt


 dtest for UDFs does no longer work - ticket's about to fix the dtest



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7864) Repair should do less work when RF=1

2014-09-05 Thread Marcus Eriksson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122968#comment-14122968
 ] 

Marcus Eriksson commented on CASSANDRA-7864:


+1

 Repair should do less work when RF=1
 

 Key: CASSANDRA-7864
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7864
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Tyler Hobbs
Assignee: Yuki Morishita
Priority: Minor
  Labels: repair
 Fix For: 2.0.11, 2.1.1

 Attachments: 7864-2.0.txt


 When the total RF for a keyspace is = 1, repair still calculates neighbors 
 for each range and does some unneccessary work.  We could short-circuit this 
 earlier.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-7888) Decide the best way to define user-define functions

2014-09-05 Thread Benjamin Lerer (JIRA)

Benjamin Lerer created CASSANDRA-7888:
-

 Summary: Decide the best way to define user-define functions
 Key: CASSANDRA-7888
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7888
 Project: Cassandra
  Issue Type: Improvement
Reporter: Benjamin Lerer
 Fix For: 3.0






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7888) Decide the best way to define user-define functions

2014-09-05 Thread Benjamin Lerer (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-7888:
--
Description: 
The goal of this ticket is to define what would be the best way from the ease 
of use and performance point of view for defining User Defined Scalar Function 
and User Defined Aggregate Function.
I would like to clarify this point before we add support for User Defined 
Aggregate Function as part of # 
The current version of UDF created within # is supporting only the addition of 
Scalar Function and does so by allowing a User to provide some classes 
containing static methods that can then be loaded as functions within Cassandra.
The problem with the static method approach is that it force us internally to 
perform a method call via reflection for each call of the function. So if the 
request load 10 000 rows the static method will be called 10 000 times via 
reflection.
As the Method object is cached the HotSpot compiler will optimize the method 
call after a certain amount of iterations. Nevertheless, from a performance 
point of view it is definetly not a optimal situation.

Ideally a proper solution from the performance point of view will limit the 
impact to the function loading time (when the function is first added or at 
startup time) but not at query time.

The first solution to solve that problem would be to force the designer of a 
new function to implements a specific interface like:
{code}
public interface UserDefinedScalarFunction
{
Object execute(Object... args);
}
{code}
or for aggregate function
{code}
public interface UserDefinedAggregateFunction
{
UserDefinedAggregation newAggregate();

public interface UserDefinedAggregate 
{
void add(Object... args);

Object getResult();

void reset();
}
} 
{code} 

This will allow use to create one object instance via reflection and then reuse 
that object everytime the function is called.

The problems with that approach is that we loose the type safety of the 
arguments and of the return type and by consequence we will be able to detect a 
problem only at running time.

The second solution would be to force the designer of a new function to create 
a new class in which it marks the method to execute with an annotation.

{code}
public class AbsFunction
{
@Execute
public double abs(double d)
{
return Maths.abs(d);
}
}
{code}

The same approach for aggregate functions will give:
{code}
public class AvgFunction
{
private double sum;
private int count

@Add
public void addValue(double d)
{
sum += d;
count++;
}

@Get
public double getAvg()
{
if (count == 0)
return 0;
return sum / count
}
 
@Reset
public void clear()
{
sum = 0;
count = 0;
}
}
{code}

For this approach to work we need to use, at loading time, code generation for 
extending the provided class with the method needed to adapt the class to our 
framework.
The disavantage of it is that we will need to add a new library like javaassist 
to the libraries used by C*.
Its advantage is that it will allow us to detect type mismatch at creation time.

 Decide the best way to define user-define functions
 ---

 Key: CASSANDRA-7888
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7888
 Project: Cassandra
  Issue Type: Improvement
Reporter: Benjamin Lerer
  Labels: cql
 Fix For: 3.0


 The goal of this ticket is to define what would be the best way from the ease 
 of use and performance point of view for defining User Defined Scalar 
 Function and User Defined Aggregate Function.
 I would like to clarify this point before we add support for User Defined 
 Aggregate Function as part of # 
 The current version of UDF created within # is supporting only the addition 
 of Scalar Function and does so by allowing a User to provide some classes 
 containing static methods that can then be loaded as functions within 
 Cassandra.
 The problem with the static method approach is that it force us internally to 
 perform a method call via reflection for each call of the function. So if the 
 request load 10 000 rows the static method will be called 10 000 times via 
 reflection.
 As the Method object is cached the HotSpot compiler will optimize the method 
 call after a certain amount of iterations. Nevertheless, from a performance 
 point of view it is definetly not a optimal situation.
 Ideally a proper solution from the performance point of view will limit the 
 impact to the function loading time (when the function is first added or at 
 startup time) but not at query time.
 The first solution to solve that problem would be to force the designer of a 
 new function to

[jira] [Updated] (CASSANDRA-7888) Decide the best way to define user-define functions

2014-09-05 Thread Benjamin Lerer (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-7888:
--
Description: 
The goal of this ticket is to define what would be the best way from the ease 
of use and performance point of view for defining User Defined Scalar Function 
and User Defined Aggregate Function.
I would like to clarify this point before we add support for User Defined 
Aggregate Function as part of #4914 
The current version of UDF is supporting only the addition of Scalar Function 
and does so by allowing a User to provide some classes containing static 
methods that can then be loaded as functions within Cassandra.
The problem with the static method approach is that it force us internally to 
perform a method call via reflection for each call of the function. So if the 
request load 10 000 rows the static method will be called 10 000 times via 
reflection.
As the Method object is cached the HotSpot compiler will optimize the method 
call after a certain amount of iterations. Nevertheless, from a performance 
point of view it is definetly not a optimal situation.

Ideally a proper solution from the performance point of view will limit the 
impact to the function loading time (when the function is first added or at 
startup time) but not at query time.

The first solution to solve that problem would be to force the designer of a 
new function to implements a specific interface like:
{code}
public interface UserDefinedScalarFunction
{
Object execute(Object... args);
}
{code}
or for aggregate function
{code}
public interface UserDefinedAggregateFunction
{
UserDefinedAggregation newAggregate();

public interface UserDefinedAggregate 
{
void add(Object... args);

Object getResult();

void reset();
}
} 
{code} 

This will allow use to create one object instance via reflection and then reuse 
that object everytime the function is called.

The problems with that approach is that we loose the type safety of the 
arguments and of the return type and by consequence we will be able to detect a 
problem only at running time.

The second solution would be to force the designer of a new function to create 
a new class in which it marks the method to execute with an annotation.

{code}
public class AbsFunction
{
@Execute
public double abs(double d)
{
return Maths.abs(d);
}
}
{code}

The same approach for aggregate functions will give:
{code}
public class AvgFunction
{
private double sum;
private int count

@Add
public void addValue(double d)
{
sum += d;
count++;
}

@Get
public double getAvg()
{
if (count == 0)
return 0;
return sum / count
}
 
@Reset
public void clear()
{
sum = 0;
count = 0;
}
}
{code}

For this approach to work we need to use, at loading time, code generation for 
extending the provided class with the method needed to adapt the class to our 
framework.
The disavantage of it is that we will need to add a new library like javaassist 
to the libraries used by C*.
Its advantage is that it will allow us to detect type mismatch at creation time.

  was:
The goal of this ticket is to define what would be the best way from the ease 
of use and performance point of view for defining User Defined Scalar Function 
and User Defined Aggregate Function.
I would like to clarify this point before we add support for User Defined 
Aggregate Function as part of # 
The current version of UDF created within # is supporting only the addition of 
Scalar Function and does so by allowing a User to provide some classes 
containing static methods that can then be loaded as functions within Cassandra.
The problem with the static method approach is that it force us internally to 
perform a method call via reflection for each call of the function. So if the 
request load 10 000 rows the static method will be called 10 000 times via 
reflection.
As the Method object is cached the HotSpot compiler will optimize the method 
call after a certain amount of iterations. Nevertheless, from a performance 
point of view it is definetly not a optimal situation.

Ideally a proper solution from the performance point of view will limit the 
impact to the function loading time (when the function is first added or at 
startup time) but not at query time.

The first solution to solve that problem would be to force the designer of a 
new function to implements a specific interface like:
{code}
public interface UserDefinedScalarFunction
{
Object execute(Object... args);
}
{code}
or for aggregate function
{code}
public interface UserDefinedAggregateFunction
{
UserDefinedAggregation newAggregate();

public interface UserDefinedAggregate 
{
void add(Object... args);

Object getResult();

void reset();
}
}

[jira] [Commented] (CASSANDRA-7888) Decide the best way to define user-define functions


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123003#comment-14123003
 ] 

Robert Stupp commented on CASSANDRA-7888:
-

A Java interface would work fine with 'class' UDFs (CASSANDRA-7395).
'java' (CASSANDRA-7562) UDFs and JSR223 (script) UDFs from CASSANDRA-7526 might 
get complicated.
(Although 'java' code generation could be changed.)

I thought of an alternative approach to pass some result set context object 
into UDFs as the first parameter for aggregate functions.
Means: each SELECT execution generates one result set context object for each 
used aggregate function.
The drawback of course is that such a generic result set context could not 
use primitive types ({{int}}, {{long}}, {{double}}, etc) but only the wrapped 
types ({{Integer}}, {{Long}}, {{Double}}, etc).

Or we could let the UDF implementation return some aggregate interface 
implementation which gets called for each row and for the final result. For 
example for 'class' UDFs:
{noformat}
class MyAggregateFunctionContext implements AggregateFunctionResultSetDouble {
void forEachRow(SomeRowOrCell data) {
... per row magic code
}
Double getResult() {
return resultValue;
}
}
{noformat}

for 'java' UDFs:
{noformat}
CREATE FUNCTION aggregateMagic ( input double ) RETURNS double LANGUAGE java AS 
'
return new AggregateFunctionResultSetDouble {
void forEachRow(SomeRowOrCell data) {
... per row magic code
}
Double getResult() {
return resultValue;
}
}
';
{noformat}

Maybe it's necessary to add some {{CREATE AGGREGATE FUNCTION ...}} syntax to 
distinguish between scalar and aggregation functions.

BTW: javassist has been added as part of CASSANDRA-7562.

 Decide the best way to define user-define functions
 ---

 Key: CASSANDRA-7888
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7888
 Project: Cassandra
  Issue Type: Improvement
Reporter: Benjamin Lerer
  Labels: cql
 Fix For: 3.0


 The goal of this ticket is to define what would be the best way from the ease 
 of use and performance point of view for defining User Defined Scalar 
 Function and User Defined Aggregate Function.
 I would like to clarify this point before we add support for User Defined 
 Aggregate Function as part of #4914 
 The current version of UDF is supporting only the addition of Scalar Function 
 and does so by allowing a User to provide some classes containing static 
 methods that can then be loaded as functions within Cassandra.
 The problem with the static method approach is that it force us internally to 
 perform a method call via reflection for each call of the function. So if the 
 request load 10 000 rows the static method will be called 10 000 times via 
 reflection.
 As the Method object is cached the HotSpot compiler will optimize the method 
 call after a certain amount of iterations. Nevertheless, from a performance 
 point of view it is definetly not a optimal situation.
 Ideally a proper solution from the performance point of view will limit the 
 impact to the function loading time (when the function is first added or at 
 startup time) but not at query time.
 The first solution to solve that problem would be to force the designer of a 
 new function to implements a specific interface like:
 {code}
 public interface UserDefinedScalarFunction
 {
 Object execute(Object... args);
 }
 {code}
 or for aggregate function
 {code}
 public interface UserDefinedAggregateFunction
 {
 UserDefinedAggregation newAggregate();
 public interface UserDefinedAggregate 
 {
 void add(Object... args);
 Object getResult();
 void reset();
 }
 } 
 {code} 
 This will allow use to create one object instance via reflection and then 
 reuse that object everytime the function is called.
 The problems with that approach is that we loose the type safety of the 
 arguments and of the return type and by consequence we will be able to detect 
 a problem only at running time.
 The second solution would be to force the designer of a new function to 
 create a new class in which it marks the method to execute with an annotation.
 {code}
 public class AbsFunction
 {
 @Execute
 public double abs(double d)
 {
 return Maths.abs(d);
 }
 }
 {code}
 The same approach for aggregate functions will give:
 {code}
 public class AvgFunction
 {
 private double sum;
 private int count
 @Add
 public void addValue(double d)
 {
 sum += d;
   count++;
 }
 @Get
 public double getAvg()
 {
 if (count == 0)
   return 0;
 return sum / count
 }
  
 @Reset
 public void clear()
 {
   sum = 0;

[jira] [Updated] (CASSANDRA-7858) Fix UDF dtest


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-7858:
---
Labels: qa-resolved  (was: )

 Fix UDF dtest
 -

 Key: CASSANDRA-7858
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7858
 Project: Cassandra
  Issue Type: Bug
Reporter: Robert Stupp
Assignee: Robert Stupp
  Labels: qa-resolved
 Attachments: 7858.txt


 dtest for UDFs does no longer work - ticket's about to fix the dtest



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-7888) Decide the best way to define user-define functions


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123003#comment-14123003
 ] 

Robert Stupp edited comment on CASSANDRA-7888 at 9/5/14 2:36 PM:
-

A Java interface would work fine with 'class' UDFs (CASSANDRA-7395).
'java' (CASSANDRA-7562) UDFs and JSR223 (script) UDFs from CASSANDRA-7526 might 
get complicated.
(Although 'java' code generation could be changed.)

I thought of an alternative approach to pass some result set context object 
into UDFs as the first parameter for aggregate functions.
Means: each SELECT execution generates one result set context object for each 
used aggregate function.
The drawback of course is that such a generic result set context could not 
use primitive types ({{int}}, {{long}}, {{double}}, etc) but only the wrapped 
types ({{Integer}}, {{Long}}, {{Double}}, etc).

Or we could let the UDF implementation return some aggregate interface 
implementation which gets called for each row and for the final result. For 
example for 'class' UDFs:
{noformat}
class MyAggregateFunctionContext implements AggregateFunctionResultSetDouble {
void forEachRow(SomeRowOrCell data) {
... per row magic code
}
Double getResult() {
return resultValue;
}
}
{noformat}

for 'java' UDFs:
{noformat}
CREATE FUNCTION aggregateMagic ( input double ) RETURNS double LANGUAGE java AS 
'
return new AggregateFunctionResultSetDouble {
void forEachRow(SomeRowOrCell data) {
... per row magic code
}
Double getResult() {
return resultValue;
}
}
';
{noformat}

Maybe it's necessary to add some {{CREATE AGGREGATE FUNCTION ...}} syntax to 
distinguish between scalar and aggregation functions.

BTW: javassist has been added as part of CASSANDRA-7562.

EDIT: strike that {{SomeRowOrCell}} - should read {{Double}}


was (Author: snazy):
A Java interface would work fine with 'class' UDFs (CASSANDRA-7395).
'java' (CASSANDRA-7562) UDFs and JSR223 (script) UDFs from CASSANDRA-7526 might 
get complicated.
(Although 'java' code generation could be changed.)

I thought of an alternative approach to pass some result set context object 
into UDFs as the first parameter for aggregate functions.
Means: each SELECT execution generates one result set context object for each 
used aggregate function.
The drawback of course is that such a generic result set context could not 
use primitive types ({{int}}, {{long}}, {{double}}, etc) but only the wrapped 
types ({{Integer}}, {{Long}}, {{Double}}, etc).

Or we could let the UDF implementation return some aggregate interface 
implementation which gets called for each row and for the final result. For 
example for 'class' UDFs:
{noformat}
class MyAggregateFunctionContext implements AggregateFunctionResultSetDouble {
void forEachRow(SomeRowOrCell data) {
... per row magic code
}
Double getResult() {
return resultValue;
}
}
{noformat}

for 'java' UDFs:
{noformat}
CREATE FUNCTION aggregateMagic ( input double ) RETURNS double LANGUAGE java AS 
'
return new AggregateFunctionResultSetDouble {
void forEachRow(SomeRowOrCell data) {
... per row magic code
}
Double getResult() {
return resultValue;
}
}
';
{noformat}

Maybe it's necessary to add some {{CREATE AGGREGATE FUNCTION ...}} syntax to 
distinguish between scalar and aggregation functions.

BTW: javassist has been added as part of CASSANDRA-7562.

 Decide the best way to define user-define functions
 ---

 Key: CASSANDRA-7888
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7888
 Project: Cassandra
  Issue Type: Improvement
Reporter: Benjamin Lerer
  Labels: cql
 Fix For: 3.0


 The goal of this ticket is to define what would be the best way from the ease 
 of use and performance point of view for defining User Defined Scalar 
 Function and User Defined Aggregate Function.
 I would like to clarify this point before we add support for User Defined 
 Aggregate Function as part of #4914 
 The current version of UDF is supporting only the addition of Scalar Function 
 and does so by allowing a User to provide some classes containing static 
 methods that can then be loaded as functions within Cassandra.
 The problem with the static method approach is that it force us internally to 
 perform a method call via reflection for each call of the function. So if the 
 request load 10 000 rows the static method will be called 10 000 times via 
 reflection.
 As the Method object is cached the HotSpot compiler will optimize the method 
 call after a certain amount of iterations. Nevertheless, from a performance 
 point of view it is definetly not a optimal situation.

[jira] [Commented] (CASSANDRA-7594) Disruptor Thrift server worker thread pool not adjustable


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123011#comment-14123011
 ] 

Philip Thompson commented on CASSANDRA-7594:


[~xedin] With the new jar, the test is now passing.

 Disruptor Thrift server worker thread pool not adjustable
 -

 Key: CASSANDRA-7594
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7594
 Project: Cassandra
  Issue Type: Bug
Reporter: Rick Branson
Assignee: Pavel Yaskevich
 Fix For: 2.0.11

 Attachments: CASSANDRA-7594.patch, 
 disruptor-thrift-server-0.3.6-SNAPSHOT.jar, jstack.txt, 
 thrift-server-0.3.7-SNAPSHOT.jar


 For the THsHaDisruptorServer, there may not be enough threads to run blocking 
 StorageProxy methods. The current number of worker threads is hardcoded at 2 
 per selector, so 2 * numAvailableProcessors(), or 64 threads on a 16-core 
 hyperthreaded machine. StorageProxy methods block these threads, so this puts 
 an upper bound on the throughput if hsha is enabled. If operations take 10ms 
 on average, the node can only handle a maximum of 6,400 operations per 
 second. This is a regression from hsha on 1.2.x, where the thread pool was 
 tunable using rpc_min_threads and rpc_max_threads.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7849) Server logged error messages (in binary protocol) for unexpected exceptions could be more helpful

2014-09-05 Thread graham sanderson (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

graham sanderson updated CASSANDRA-7849:

Fix Version/s: 2.1.0
   2.0.11
   1.2.19

 Server logged error messages (in binary protocol) for unexpected exceptions 
 could be more helpful
 -

 Key: CASSANDRA-7849
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7849
 Project: Cassandra
  Issue Type: Improvement
Reporter: graham sanderson
 Fix For: 1.2.19, 2.0.11, 2.1.0


 From time to time (actually quite frequently) we get error messages in the 
 server logs like this
 {code}
 ERROR [Native-Transport-Requests:288] 2014-08-29 04:48:07,118 
 ErrorMessage.java (line 222) Unexpected exception during request
 java.io.IOException: Connection reset by peer
 at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
 at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
 at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
 at sun.nio.ch.IOUtil.read(IOUtil.java:192)
 at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
 at 
 org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:64)
 at 
 org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
 at 
 org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
 at 
 org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
 at 
 org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:745)
 {code}
 These particular cases are almost certainly problems with the client driver, 
 client machine, client process, however after the fact this particular 
 exception is practically impossible to debug because there is no indication 
 in the underlying JVM/netty exception of who the peer was. I should note we 
 have lots of different types of applications running against the cluster so 
 it is very hard to correlate these to anything



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[09/12] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1.0

Merge branch 'cassandra-2.0' into cassandra-2.1.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/655d517b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/655d517b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/655d517b

Branch: refs/heads/trunk
Commit: 655d517b2532fb62bca647c047f44d65bab1eb86
Parents: cf76ba3 46ef962
Author: Yuki Morishita yu...@apache.org
Authored: Fri Sep 5 10:28:33 2014 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Fri Sep 5 10:28:33 2014 -0500

--

--

[12/12] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fbc0f447
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fbc0f447
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fbc0f447

Branch: refs/heads/trunk
Commit: fbc0f447d17ba9237ed2f28101b8b0ae1b21b80d
Parents: 33cedda 7d84043
Author: Yuki Morishita yu...@apache.org
Authored: Fri Sep 5 10:29:56 2014 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Fri Sep 5 10:29:56 2014 -0500

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/service/StorageService.java | 4 ++--
 2 files changed, 3 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/fbc0f447/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/fbc0f447/src/java/org/apache/cassandra/service/StorageService.java
--
diff --cc src/java/org/apache/cassandra/service/StorageService.java
index 7575936,fb4e6e6..6714be6
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@@ -2496,9 -2496,9 +2496,9 @@@ public class StorageService extends Not
  return forceRepairAsync(keyspace, isSequential, isLocal, ranges, 
fullRepair, columnFamilies);
  }
  
 -public int forceRepairAsync(String keyspace, boolean isSequential, 
boolean isLocal, CollectionRangeToken ranges, boolean fullRepair, String... 
columnFamilies)
 +public int forceRepairAsync(final String keyspace, final boolean 
isSequential, final boolean isLocal, final CollectionRangeToken ranges, 
final boolean fullRepair, final String... columnFamilies)
  {
- if (Keyspace.SYSTEM_KS.equals(keyspace) || ranges.isEmpty())
+ if (ranges.isEmpty() || 
Keyspace.open(keyspace).getReplicationStrategy().getReplicationFactor()  2)
  return 0;
  
  int cmd = nextRepairCommand.incrementAndGet();

[04/12] git commit: Make repair no-op when RF=1

Make repair no-op when RF=1

patch by yukim; reviewed by krummas for CASSANDRA-7864


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/46ef9628
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/46ef9628
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/46ef9628

Branch: refs/heads/cassandra-2.1
Commit: 46ef9628fa4035c26d81c4a070ebd270e59e5520
Parents: 77b036a
Author: Yuki Morishita yu...@apache.org
Authored: Fri Sep 5 10:22:33 2014 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Fri Sep 5 10:22:33 2014 -0500

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/service/StorageService.java | 6 +++---
 2 files changed, 4 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/46ef9628/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 7be8979..54fcc4f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -8,6 +8,7 @@
(CASSANDRA-7722)
  * Always send Paxos commit to all replicas (CASSANDRA-7479)
  * Make disruptor_thrift_server invocation pool configurable (CASSANDRA-7594)
+ * Make repair no-op when RF=1 (CASSANDRA-7864)
 
 
 2.0.10

http://git-wip-us.apache.org/repos/asf/cassandra/blob/46ef9628/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index f16e187..12d6420 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -2404,7 +2404,7 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 
 public int forceRepairAsync(final String keyspace, final boolean 
isSequential, final CollectionString dataCenters, final CollectionString 
hosts,  final CollectionRangeToken ranges, final String... columnFamilies)
 {
-if (Keyspace.SYSTEM_KS.equals(keyspace) || ranges.isEmpty())
+if (ranges.isEmpty() || 
Keyspace.open(keyspace).getReplicationStrategy().getReplicationFactor()  2)
 return 0;
 
 final int cmd = nextRepairCommand.incrementAndGet();
@@ -2428,7 +2428,7 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 
 public int forceRepairAsync(String keyspace, boolean isSequential, boolean 
isLocal, CollectionRangeToken ranges, String... columnFamilies)
 {
-if (Keyspace.SYSTEM_KS.equals(keyspace) || ranges.isEmpty())
+if (ranges.isEmpty() || 
Keyspace.open(keyspace).getReplicationStrategy().getReplicationFactor()  2)
 return 0;
 
 final int cmd = nextRepairCommand.incrementAndGet();
@@ -2497,7 +2497,7 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 
 public void forceKeyspaceRepairRange(final String keyspaceName, final 
CollectionRangeToken ranges, boolean isSequential, boolean isLocal, final 
String... columnFamilies) throws IOException
 {
-if (Keyspace.SYSTEM_KS.equalsIgnoreCase(keyspaceName))
+if (ranges.isEmpty() || 
Keyspace.open(keyspaceName).getReplicationStrategy().getReplicationFactor()  2)
 return;
 createRepairTask(nextRepairCommand.incrementAndGet(), keyspaceName, 
ranges, isSequential, isLocal, columnFamilies).run();
 }

[07/12] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1.0

Merge branch 'cassandra-2.0' into cassandra-2.1.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/655d517b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/655d517b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/655d517b

Branch: refs/heads/cassandra-2.1
Commit: 655d517b2532fb62bca647c047f44d65bab1eb86
Parents: cf76ba3 46ef962
Author: Yuki Morishita yu...@apache.org
Authored: Fri Sep 5 10:28:33 2014 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Fri Sep 5 10:28:33 2014 -0500

--

--

[05/12] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
src/java/org/apache/cassandra/service/StorageService.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dea5fa78
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dea5fa78
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dea5fa78

Branch: refs/heads/trunk
Commit: dea5fa78a0ca94626656bbfbf101e4b152e8d6ed
Parents: c6b509d 46ef962
Author: Yuki Morishita yu...@apache.org
Authored: Fri Sep 5 10:28:07 2014 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Fri Sep 5 10:28:07 2014 -0500

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/service/StorageService.java | 4 ++--
 2 files changed, 3 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/dea5fa78/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/dea5fa78/src/java/org/apache/cassandra/service/StorageService.java
--
diff --cc src/java/org/apache/cassandra/service/StorageService.java
index 5c6dbd6,12d6420..fb4e6e6
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@@ -2462,25 -2398,19 +2462,25 @@@ public class StorageService extends Not
  {
  throw new IllegalArgumentException(You need to run primary range 
repair on all nodes in the cluster.);
  }
 -final CollectionRangeToken ranges = primaryRange ? 
getLocalPrimaryRanges(keyspace) : getLocalRanges(keyspace);
 -return forceRepairAsync(keyspace, isSequential, dataCenters, hosts, 
ranges, columnFamilies);
 +CollectionRangeToken ranges = primaryRange ? 
getLocalPrimaryRanges(keyspace) : getLocalRanges(keyspace);
 +
 +return forceRepairAsync(keyspace, isSequential, dataCenters, hosts, 
ranges, fullRepair, columnFamilies);
  }
  
 -public int forceRepairAsync(final String keyspace, final boolean 
isSequential, final CollectionString dataCenters, final CollectionString 
hosts,  final CollectionRangeToken ranges, final String... columnFamilies)
 +public int forceRepairAsync(String keyspace, boolean isSequential, 
CollectionString dataCenters, CollectionString hosts, 
CollectionRangeToken ranges, boolean fullRepair, String... columnFamilies)
  {
- if (Keyspace.SYSTEM_KS.equals(keyspace) || ranges.isEmpty())
+ if (ranges.isEmpty() || 
Keyspace.open(keyspace).getReplicationStrategy().getReplicationFactor()  2)
  return 0;
  
 -final int cmd = nextRepairCommand.incrementAndGet();
 +int cmd = nextRepairCommand.incrementAndGet();
  if (ranges.size()  0)
  {
 -new Thread(createRepairTask(cmd, keyspace, ranges, isSequential, 
dataCenters, hosts, columnFamilies)).start();
 +if (!FBUtilities.isUnix()  isSequential)
 +{
 +logger.warn(Snapshot-based repair is not yet supported on 
Windows.  Reverting to parallel repair.);
 +isSequential = false;
 +}
 +new Thread(createRepairTask(cmd, keyspace, ranges, isSequential, 
dataCenters, hosts, fullRepair, columnFamilies)).start();
  }
  return cmd;
  }
@@@ -2492,16 -2422,16 +2492,16 @@@
  {
  throw new IllegalArgumentException(You need to run primary range 
repair on all nodes in the cluster.);
  }
 -final CollectionRangeToken ranges = primaryRange ? 
getLocalPrimaryRanges(keyspace) : getLocalRanges(keyspace);
 -return forceRepairAsync(keyspace, isSequential, isLocal, ranges, 
columnFamilies);
 +CollectionRangeToken ranges = primaryRange ? 
getLocalPrimaryRanges(keyspace) : getLocalRanges(keyspace);
 +return forceRepairAsync(keyspace, isSequential, isLocal, ranges, 
fullRepair, columnFamilies);
  }
  
 -public int forceRepairAsync(String keyspace, boolean isSequential, 
boolean isLocal, CollectionRangeToken ranges, String... columnFamilies)
 +public int forceRepairAsync(String keyspace, boolean isSequential, 
boolean isLocal, CollectionRangeToken ranges, boolean fullRepair, String... 
columnFamilies)
  {
- if (Keyspace.SYSTEM_KS.equals(keyspace) || ranges.isEmpty())
+ if (ranges.isEmpty() || 
Keyspace.open(keyspace).getReplicationStrategy().getReplicationFactor()  2)
  return 0;
  
 -final int cmd = nextRepairCommand.incrementAndGet();
 +int cmd = nextRepairCommand.incrementAndGet();
  if (!FBUtilities.isUnix()  isSequential)
  {
  logger.warn(Snapshot-based repair is not

[02/12] git commit: Make repair no-op when RF=1

Make repair no-op when RF=1

patch by yukim; reviewed by krummas for CASSANDRA-7864


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/46ef9628
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/46ef9628
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/46ef9628

Branch: refs/heads/cassandra-2.1.0
Commit: 46ef9628fa4035c26d81c4a070ebd270e59e5520
Parents: 77b036a
Author: Yuki Morishita yu...@apache.org
Authored: Fri Sep 5 10:22:33 2014 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Fri Sep 5 10:22:33 2014 -0500

--
 CHANGES.txt   | 1 +
 src/java/org/apache/cassandra/service/StorageService.java | 6 +++---
 2 files changed, 4 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/46ef9628/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 7be8979..54fcc4f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -8,6 +8,7 @@
(CASSANDRA-7722)
  * Always send Paxos commit to all replicas (CASSANDRA-7479)
  * Make disruptor_thrift_server invocation pool configurable (CASSANDRA-7594)
+ * Make repair no-op when RF=1 (CASSANDRA-7864)
 
 
 2.0.10

http://git-wip-us.apache.org/repos/asf/cassandra/blob/46ef9628/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index f16e187..12d6420 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -2404,7 +2404,7 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 
 public int forceRepairAsync(final String keyspace, final boolean 
isSequential, final CollectionString dataCenters, final CollectionString 
hosts,  final CollectionRangeToken ranges, final String... columnFamilies)
 {
-if (Keyspace.SYSTEM_KS.equals(keyspace) || ranges.isEmpty())
+if (ranges.isEmpty() || 
Keyspace.open(keyspace).getReplicationStrategy().getReplicationFactor()  2)
 return 0;
 
 final int cmd = nextRepairCommand.incrementAndGet();
@@ -2428,7 +2428,7 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 
 public int forceRepairAsync(String keyspace, boolean isSequential, boolean 
isLocal, CollectionRangeToken ranges, String... columnFamilies)
 {
-if (Keyspace.SYSTEM_KS.equals(keyspace) || ranges.isEmpty())
+if (ranges.isEmpty() || 
Keyspace.open(keyspace).getReplicationStrategy().getReplicationFactor()  2)
 return 0;
 
 final int cmd = nextRepairCommand.incrementAndGet();
@@ -2497,7 +2497,7 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 
 public void forceKeyspaceRepairRange(final String keyspaceName, final 
CollectionRangeToken ranges, boolean isSequential, boolean isLocal, final 
String... columnFamilies) throws IOException
 {
-if (Keyspace.SYSTEM_KS.equalsIgnoreCase(keyspaceName))
+if (ranges.isEmpty() || 
Keyspace.open(keyspaceName).getReplicationStrategy().getReplicationFactor()  2)
 return;
 createRepairTask(nextRepairCommand.incrementAndGet(), keyspaceName, 
ranges, isSequential, isLocal, columnFamilies).run();
 }

[01/12] git commit: Make repair no-op when RF=1