date:20141024


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14182504#comment-14182504
 ] 

Sylvain Lebresne commented on CASSANDRA-8180:
-

I'm also really curious as to how you expect this optimization to work, because 
I don't see how this could work.

 Optimize disk seek using min/max colunm name meta data when the LIMIT clause 
 is used
 

 Key: CASSANDRA-8180
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8180
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: Cassandra 2.0.10
Reporter: DOAN DuyHai
Assignee: Tyler Hobbs
Priority: Minor

 I was working on an example of sensor data table (timeseries) and face a use 
 case where C* does not optimize read on disk.
 {code}
 cqlsh:test CREATE TABLE test(id int, col int, val text, PRIMARY KEY(id,col)) 
 WITH CLUSTERING ORDER BY (col DESC);
 cqlsh:test INSERT INTO test(id, col , val ) VALUES ( 1, 10, '10');
 ...
 nodetool flush test test
 ...
 cqlsh:test INSERT INTO test(id, col , val ) VALUES ( 1, 20, '20');
 ...
 nodetool flush test test
 ...
 cqlsh:test INSERT INTO test(id, col , val ) VALUES ( 1, 30, '30');
 ...
 nodetool flush test test
 {code}
 After that, I activate request tracing:
 {code}
 cqlsh:test SELECT * FROM test WHERE id=1 LIMIT 1;
  activity  | 
 timestamp| source| source_elapsed
 ---+--+---+
 execute_cql3_query | 
 23:48:46,498 | 127.0.0.1 |  0
 Parsing SELECT * FROM test WHERE id=1 LIMIT 1; | 
 23:48:46,498 | 127.0.0.1 | 74
Preparing statement | 
 23:48:46,499 | 127.0.0.1 |253
   Executing single-partition query on test | 
 23:48:46,499 | 127.0.0.1 |930
   Acquiring sstable references | 
 23:48:46,499 | 127.0.0.1 |943
Merging memtable tombstones | 
 23:48:46,499 | 127.0.0.1 |   1032
Key cache hit for sstable 3 | 
 23:48:46,500 | 127.0.0.1 |   1160
Seeking to partition beginning in data file | 
 23:48:46,500 | 127.0.0.1 |   1173
Key cache hit for sstable 2 | 
 23:48:46,500 | 127.0.0.1 |   1889
Seeking to partition beginning in data file | 
 23:48:46,500 | 127.0.0.1 |   1901
Key cache hit for sstable 1 | 
 23:48:46,501 | 127.0.0.1 |   2373
Seeking to partition beginning in data file | 
 23:48:46,501 | 127.0.0.1 |   2384
  Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 
 23:48:46,501 | 127.0.0.1 |   2768
 Merging data from memtables and 3 sstables | 
 23:48:46,501 | 127.0.0.1 |   2784
 Read 2 live and 0 tombstoned cells | 
 23:48:46,501 | 127.0.0.1 |   2976
   Request complete | 
 23:48:46,501 | 127.0.0.1 |   3551
 {code}
 We can clearly see that C* hits 3 SSTables on disk instead of just one, 
 although it has the min/max column meta data to decide which SSTable contains 
 the most recent data.
 Funny enough, if we add a clause on the clustering column to the select, this 
 time C* optimizes the read path:
 {code}
 cqlsh:test SELECT * FROM test WHERE id=1 AND col  25 LIMIT 1;
  activity  | 
 timestamp| source| source_elapsed
 ---+--+---+
 execute_cql3_query | 
 23:52:31,888 | 127.0.0.1 |  0
Parsing SELECT * FROM test WHERE id=1 AND col  25 LIMIT 1; | 
 23:52:31,888 | 127.0.0.1 | 60
Preparing statement | 
 23:52:31,888 | 127.0.0.1 |277
   Executing single-partition query on test | 
 23:52:31,889 | 127.0.0.1 |961
   Acquiring sstable references | 
 23:52:31,889 | 127.0.0.1 |971

[jira] [Commented] (CASSANDRA-8180) Optimize disk seek using min/max colunm name meta data when the LIMIT clause is used


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14182513#comment-14182513
 ] 

Sylvain Lebresne commented on CASSANDRA-8180:
-

Oh, unless you mean to order the sstables by mean min column names and to query 
them one at a time (like we do for names filter) in the specific case where 
their column names interval don't intersect. In which case, yes, that could be 
indeed useful with the compaction strategy from CASSANDRA-6602.

 Optimize disk seek using min/max colunm name meta data when the LIMIT clause 
 is used
 

 Key: CASSANDRA-8180
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8180
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: Cassandra 2.0.10
Reporter: DOAN DuyHai
Assignee: Tyler Hobbs
Priority: Minor

 I was working on an example of sensor data table (timeseries) and face a use 
 case where C* does not optimize read on disk.
 {code}
 cqlsh:test CREATE TABLE test(id int, col int, val text, PRIMARY KEY(id,col)) 
 WITH CLUSTERING ORDER BY (col DESC);
 cqlsh:test INSERT INTO test(id, col , val ) VALUES ( 1, 10, '10');
 ...
 nodetool flush test test
 ...
 cqlsh:test INSERT INTO test(id, col , val ) VALUES ( 1, 20, '20');
 ...
 nodetool flush test test
 ...
 cqlsh:test INSERT INTO test(id, col , val ) VALUES ( 1, 30, '30');
 ...
 nodetool flush test test
 {code}
 After that, I activate request tracing:
 {code}
 cqlsh:test SELECT * FROM test WHERE id=1 LIMIT 1;
  activity  | 
 timestamp| source| source_elapsed
 ---+--+---+
 execute_cql3_query | 
 23:48:46,498 | 127.0.0.1 |  0
 Parsing SELECT * FROM test WHERE id=1 LIMIT 1; | 
 23:48:46,498 | 127.0.0.1 | 74
Preparing statement | 
 23:48:46,499 | 127.0.0.1 |253
   Executing single-partition query on test | 
 23:48:46,499 | 127.0.0.1 |930
   Acquiring sstable references | 
 23:48:46,499 | 127.0.0.1 |943
Merging memtable tombstones | 
 23:48:46,499 | 127.0.0.1 |   1032
Key cache hit for sstable 3 | 
 23:48:46,500 | 127.0.0.1 |   1160
Seeking to partition beginning in data file | 
 23:48:46,500 | 127.0.0.1 |   1173
Key cache hit for sstable 2 | 
 23:48:46,500 | 127.0.0.1 |   1889
Seeking to partition beginning in data file | 
 23:48:46,500 | 127.0.0.1 |   1901
Key cache hit for sstable 1 | 
 23:48:46,501 | 127.0.0.1 |   2373
Seeking to partition beginning in data file | 
 23:48:46,501 | 127.0.0.1 |   2384
  Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 
 23:48:46,501 | 127.0.0.1 |   2768
 Merging data from memtables and 3 sstables | 
 23:48:46,501 | 127.0.0.1 |   2784
 Read 2 live and 0 tombstoned cells | 
 23:48:46,501 | 127.0.0.1 |   2976
   Request complete | 
 23:48:46,501 | 127.0.0.1 |   3551
 {code}
 We can clearly see that C* hits 3 SSTables on disk instead of just one, 
 although it has the min/max column meta data to decide which SSTable contains 
 the most recent data.
 Funny enough, if we add a clause on the clustering column to the select, this 
 time C* optimizes the read path:
 {code}
 cqlsh:test SELECT * FROM test WHERE id=1 AND col  25 LIMIT 1;
  activity  | 
 timestamp| source| source_elapsed
 ---+--+---+
 execute_cql3_query | 
 23:52:31,888 | 127.0.0.1 |  0
Parsing SELECT * FROM test WHERE id=1 AND col  25 LIMIT 1; | 
 23:52:31,888 | 127.0.0.1 | 60
Preparing statement | 
 23:52:31,888 | 127.0.0.1 |277
   Executing single-partition query

[jira] [Updated] (CASSANDRA-8182) Comparison method violates its general contract in IndexSummaryManager.redistributeSummaries


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-8182:

Assignee: Tyler Hobbs

 Comparison method violates its general contract in 
 IndexSummaryManager.redistributeSummaries
 

 Key: CASSANDRA-8182
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8182
 Project: Cassandra
  Issue Type: Bug
 Environment: 2.1.0
Reporter: Juho Mäkinen
Assignee: Tyler Hobbs

 I found this exception while going over the logs of my new 2.1.0 production 
 candidate cluster:
 ERROR [IndexSummaryManager:1] 2014-10-23 14:37:56,727 
 CassandraDaemon.java:166 - Exception in thread 
 Thread[IndexSummaryManager:1,1,main]
 java.lang.IllegalArgumentException: Comparison method violates its general 
 contract!
 at java.util.TimSort.mergeLo(TimSort.java:747) ~[na:1.7.0_67]
 at java.util.TimSort.mergeAt(TimSort.java:483) ~[na:1.7.0_67]
 at java.util.TimSort.mergeCollapse(TimSort.java:408) ~[na:1.7.0_67]
 at java.util.TimSort.sort(TimSort.java:214) ~[na:1.7.0_67]
 at java.util.TimSort.sort(TimSort.java:173) ~[na:1.7.0_67]
 at java.util.Arrays.sort(Arrays.java:659) ~[na:1.7.0_67]
 at java.util.Collections.sort(Collections.java:217) ~[na:1.7.0_67]
 at 
 org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:277)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:238)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow(IndexSummaryManager.java:139)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:75)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
 [na:1.7.0_67]
 at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) 
 [na:1.7.0_67]
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
  [na:1.7.0_67]
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
  [na:1.7.0_67]
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  [na:1.7.0_67]
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  [na:1.7.0_67]
 at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
 There's some other cases of invalid comparison methods in the past, but not 
 from this case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8182) Comparison method violates its general contract in IndexSummaryManager.redistributeSummaries


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14182523#comment-14182523
 ] 

Sylvain Lebresne commented on CASSANDRA-8182:
-

Looks like this come from CASSANDRA-5519. I suspect the problem is that the 
comparison function uses the SSTableReader {{readMeter.fifteenMinuteRate()}}, 
but if you're unlucky this value can change in the middle of the sorting (thus 
changing the comparison value for 2 sstables in the middle of the sorting). We 
probably should save those rate in a map or something before the sort and have 
the sorting refer to those saved and stable values.

 Comparison method violates its general contract in 
 IndexSummaryManager.redistributeSummaries
 

 Key: CASSANDRA-8182
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8182
 Project: Cassandra
  Issue Type: Bug
 Environment: 2.1.0
Reporter: Juho Mäkinen
Assignee: Tyler Hobbs

 I found this exception while going over the logs of my new 2.1.0 production 
 candidate cluster:
 ERROR [IndexSummaryManager:1] 2014-10-23 14:37:56,727 
 CassandraDaemon.java:166 - Exception in thread 
 Thread[IndexSummaryManager:1,1,main]
 java.lang.IllegalArgumentException: Comparison method violates its general 
 contract!
 at java.util.TimSort.mergeLo(TimSort.java:747) ~[na:1.7.0_67]
 at java.util.TimSort.mergeAt(TimSort.java:483) ~[na:1.7.0_67]
 at java.util.TimSort.mergeCollapse(TimSort.java:408) ~[na:1.7.0_67]
 at java.util.TimSort.sort(TimSort.java:214) ~[na:1.7.0_67]
 at java.util.TimSort.sort(TimSort.java:173) ~[na:1.7.0_67]
 at java.util.Arrays.sort(Arrays.java:659) ~[na:1.7.0_67]
 at java.util.Collections.sort(Collections.java:217) ~[na:1.7.0_67]
 at 
 org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:277)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:238)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow(IndexSummaryManager.java:139)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:75)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
 [na:1.7.0_67]
 at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) 
 [na:1.7.0_67]
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
  [na:1.7.0_67]
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
  [na:1.7.0_67]
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  [na:1.7.0_67]
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  [na:1.7.0_67]
 at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
 There's some other cases of invalid comparison methods in the past, but not 
 from this case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8182) Comparison method violates its general contract in IndexSummaryManager.redistributeSummaries


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-8182:

Fix Version/s: 2.1.2

 Comparison method violates its general contract in 
 IndexSummaryManager.redistributeSummaries
 

 Key: CASSANDRA-8182
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8182
 Project: Cassandra
  Issue Type: Bug
 Environment: 2.1.0
Reporter: Juho Mäkinen
Assignee: Tyler Hobbs
 Fix For: 2.1.2


 I found this exception while going over the logs of my new 2.1.0 production 
 candidate cluster:
 ERROR [IndexSummaryManager:1] 2014-10-23 14:37:56,727 
 CassandraDaemon.java:166 - Exception in thread 
 Thread[IndexSummaryManager:1,1,main]
 java.lang.IllegalArgumentException: Comparison method violates its general 
 contract!
 at java.util.TimSort.mergeLo(TimSort.java:747) ~[na:1.7.0_67]
 at java.util.TimSort.mergeAt(TimSort.java:483) ~[na:1.7.0_67]
 at java.util.TimSort.mergeCollapse(TimSort.java:408) ~[na:1.7.0_67]
 at java.util.TimSort.sort(TimSort.java:214) ~[na:1.7.0_67]
 at java.util.TimSort.sort(TimSort.java:173) ~[na:1.7.0_67]
 at java.util.Arrays.sort(Arrays.java:659) ~[na:1.7.0_67]
 at java.util.Collections.sort(Collections.java:217) ~[na:1.7.0_67]
 at 
 org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:277)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:238)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow(IndexSummaryManager.java:139)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:75)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
 [na:1.7.0_67]
 at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) 
 [na:1.7.0_67]
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
  [na:1.7.0_67]
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
  [na:1.7.0_67]
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  [na:1.7.0_67]
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  [na:1.7.0_67]
 at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
 There's some other cases of invalid comparison methods in the past, but not 
 from this case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8181) Intermittent failure of SSTableImportTest unit test


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14182534#comment-14182534
 ] 

Benjamin Lerer commented on CASSANDRA-8181:
---

[~mshuler] do you know if it always fail on the same machine? 

 Intermittent failure of SSTableImportTest unit test
 ---

 Key: CASSANDRA-8181
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8181
 Project: Cassandra
  Issue Type: Bug
  Components: Tests
Reporter: Michael Shuler
Assignee: Benjamin Lerer
 Fix For: 2.1.2

 Attachments: SSTableImportTest_failure_system.log.gz


 {noformat}
 [junit] Testsuite: org.apache.cassandra.tools.SSTableImportTest
 [junit] Tests run: 8, Failures: 1, Errors: 0, Time elapsed: 6.712 sec
 [junit] 
 [junit] - Standard Output ---
 [junit] Counting keys to import, please wait... (NOTE: to skip this use 
 -n num_keys)
 [junit] Importing 2 keys...
 [junit] 2 keys imported successfully.
 [junit] Counting keys to import, please wait... (NOTE: to skip this use 
 -n num_keys)
 [junit] Importing 2 keys...
 [junit] 2 keys imported successfully.
 [junit] Counting keys to import, please wait... (NOTE: to skip this use 
 -n num_keys)
 [junit] Importing 2 keys...
 [junit] 2 keys imported successfully.
 [junit] Counting keys to import, please wait... (NOTE: to skip this use 
 -n num_keys)
 [junit] Importing 2 keys...
 [junit] Importing 2 keys...
 [junit] 2 keys imported successfully.
 [junit] Counting keys to import, please wait... (NOTE: to skip this use 
 -n num_keys)
 [junit] Importing 2 keys...
 [junit] 2 keys imported successfully.
 [junit] Counting keys to import, please wait... (NOTE: to skip this use 
 -n num_keys)
 [junit] Importing 1 keys...
 [junit] 1 keys imported successfully.
 [junit] Counting keys to import, please wait... (NOTE: to skip this use 
 -n num_keys)
 [junit] Importing 2 keys...
 [junit] 2 keys imported successfully.
 [junit] -  ---
 [junit] Testcase: 
 testImportWithAsciiKeyValidator(org.apache.cassandra.tools.SSTableImportTest):
 FAILED
 [junit] null
 [junit] junit.framework.AssertionFailedError
 [junit] at 
 org.apache.cassandra.tools.SSTableImportTest.testImportWithAsciiKeyValidator(SSTableImportTest.java:166)
 [junit] 
 [junit] 
 [junit] Test org.apache.cassandra.tools.SSTableImportTest FAILED
 {noformat}
 testImportWithAsciiKeyValidator was added in CASSANDRA-7498 and fails as 
 above occasionally (~10-15% of runs) in CI. Attached is the system.log from 
 the failed test on 2.1 HEAD (8e5fdc2).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8182) Comparison method violates its general contract in IndexSummaryManager.redistributeSummaries

2014-10-24 Thread JIRA


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14182533#comment-14182533
 ] 

Juho Mäkinen commented on CASSANDRA-8182:
-

I'm not familiar of the inner workings of this issue, does this affect the 
stability of the cluster?

 Comparison method violates its general contract in 
 IndexSummaryManager.redistributeSummaries
 

 Key: CASSANDRA-8182
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8182
 Project: Cassandra
  Issue Type: Bug
 Environment: 2.1.0
Reporter: Juho Mäkinen
Assignee: Tyler Hobbs
 Fix For: 2.1.2


 I found this exception while going over the logs of my new 2.1.0 production 
 candidate cluster:
 ERROR [IndexSummaryManager:1] 2014-10-23 14:37:56,727 
 CassandraDaemon.java:166 - Exception in thread 
 Thread[IndexSummaryManager:1,1,main]
 java.lang.IllegalArgumentException: Comparison method violates its general 
 contract!
 at java.util.TimSort.mergeLo(TimSort.java:747) ~[na:1.7.0_67]
 at java.util.TimSort.mergeAt(TimSort.java:483) ~[na:1.7.0_67]
 at java.util.TimSort.mergeCollapse(TimSort.java:408) ~[na:1.7.0_67]
 at java.util.TimSort.sort(TimSort.java:214) ~[na:1.7.0_67]
 at java.util.TimSort.sort(TimSort.java:173) ~[na:1.7.0_67]
 at java.util.Arrays.sort(Arrays.java:659) ~[na:1.7.0_67]
 at java.util.Collections.sort(Collections.java:217) ~[na:1.7.0_67]
 at 
 org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:277)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:238)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow(IndexSummaryManager.java:139)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:75)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
 [na:1.7.0_67]
 at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) 
 [na:1.7.0_67]
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
  [na:1.7.0_67]
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
  [na:1.7.0_67]
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  [na:1.7.0_67]
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  [na:1.7.0_67]
 at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
 There's some other cases of invalid comparison methods in the past, but not 
 from this case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8182) Comparison method violates its general contract in IndexSummaryManager.redistributeSummaries


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-8182:

Priority: Minor  (was: Major)

 Comparison method violates its general contract in 
 IndexSummaryManager.redistributeSummaries
 

 Key: CASSANDRA-8182
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8182
 Project: Cassandra
  Issue Type: Bug
 Environment: 2.1.0
Reporter: Juho Mäkinen
Assignee: Tyler Hobbs
Priority: Minor
 Fix For: 2.1.2


 I found this exception while going over the logs of my new 2.1.0 production 
 candidate cluster:
 ERROR [IndexSummaryManager:1] 2014-10-23 14:37:56,727 
 CassandraDaemon.java:166 - Exception in thread 
 Thread[IndexSummaryManager:1,1,main]
 java.lang.IllegalArgumentException: Comparison method violates its general 
 contract!
 at java.util.TimSort.mergeLo(TimSort.java:747) ~[na:1.7.0_67]
 at java.util.TimSort.mergeAt(TimSort.java:483) ~[na:1.7.0_67]
 at java.util.TimSort.mergeCollapse(TimSort.java:408) ~[na:1.7.0_67]
 at java.util.TimSort.sort(TimSort.java:214) ~[na:1.7.0_67]
 at java.util.TimSort.sort(TimSort.java:173) ~[na:1.7.0_67]
 at java.util.Arrays.sort(Arrays.java:659) ~[na:1.7.0_67]
 at java.util.Collections.sort(Collections.java:217) ~[na:1.7.0_67]
 at 
 org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:277)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:238)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow(IndexSummaryManager.java:139)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:75)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
 [na:1.7.0_67]
 at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) 
 [na:1.7.0_67]
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
  [na:1.7.0_67]
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
  [na:1.7.0_67]
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  [na:1.7.0_67]
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  [na:1.7.0_67]
 at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
 There's some other cases of invalid comparison methods in the past, but not 
 from this case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8182) Comparison method violates its general contract in IndexSummaryManager.redistributeSummaries


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14182560#comment-14182560
 ] 

Sylvain Lebresne commented on CASSANDRA-8182:
-

bq.  does this affect the stability of the cluster?

No. When this happen, the task that check if it can reduce the memory of index 
summary (as per CASSANDRA-5519) will fail, but the whole thing is not critical 
in the first place (it's more a nice to have really) and it's retried every 
hour anyway (and it almost certainly won't fail consistently). Fixing this is 
more about removing a scary error from the log, it's largly unconsequential in 
practice.

 Comparison method violates its general contract in 
 IndexSummaryManager.redistributeSummaries
 

 Key: CASSANDRA-8182
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8182
 Project: Cassandra
  Issue Type: Bug
 Environment: 2.1.0
Reporter: Juho Mäkinen
Assignee: Tyler Hobbs
 Fix For: 2.1.2


 I found this exception while going over the logs of my new 2.1.0 production 
 candidate cluster:
 ERROR [IndexSummaryManager:1] 2014-10-23 14:37:56,727 
 CassandraDaemon.java:166 - Exception in thread 
 Thread[IndexSummaryManager:1,1,main]
 java.lang.IllegalArgumentException: Comparison method violates its general 
 contract!
 at java.util.TimSort.mergeLo(TimSort.java:747) ~[na:1.7.0_67]
 at java.util.TimSort.mergeAt(TimSort.java:483) ~[na:1.7.0_67]
 at java.util.TimSort.mergeCollapse(TimSort.java:408) ~[na:1.7.0_67]
 at java.util.TimSort.sort(TimSort.java:214) ~[na:1.7.0_67]
 at java.util.TimSort.sort(TimSort.java:173) ~[na:1.7.0_67]
 at java.util.Arrays.sort(Arrays.java:659) ~[na:1.7.0_67]
 at java.util.Collections.sort(Collections.java:217) ~[na:1.7.0_67]
 at 
 org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:277)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:238)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow(IndexSummaryManager.java:139)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:75)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
 [na:1.7.0_67]
 at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) 
 [na:1.7.0_67]
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
  [na:1.7.0_67]
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
  [na:1.7.0_67]
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  [na:1.7.0_67]
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  [na:1.7.0_67]
 at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
 There's some other cases of invalid comparison methods in the past, but not 
 from this case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-8183) improve PropertyFileSnitch output

Liang Xie created CASSANDRA-8183:


 Summary: improve PropertyFileSnitch output
 Key: CASSANDRA-8183
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8183
 Project: Cassandra
  Issue Type: Improvement
  Components: Config
Reporter: Liang Xie
 Fix For: 3.1


Saw log like this: 16:37:11.825 [main] DEBUG o.a.c.locator.PropertyFileSnitch 
- loaded network topology /10.2.201.66:[Ljava.lang.String;@34e276d9, 
/10.2.201.65:[Ljava.lang.String;@7fe8bbae, /10.2.201. 
68:[Ljava.lang.String;@879de1d
Obviously, we should handle the original String Array output behavior better.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8183) improve PropertyFileSnitch output


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liang Xie updated CASSANDRA-8183:
-
Attachment: CASSANDRA-8183.txt

attached a minor patch.

 improve PropertyFileSnitch output
 -

 Key: CASSANDRA-8183
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8183
 Project: Cassandra
  Issue Type: Improvement
  Components: Config
Reporter: Liang Xie
 Fix For: 3.1

 Attachments: CASSANDRA-8183.txt


 Saw log like this: 16:37:11.825 [main] DEBUG 
 o.a.c.locator.PropertyFileSnitch - loaded network topology 
 /10.2.201.66:[Ljava.lang.String;@34e276d9, 
 /10.2.201.65:[Ljava.lang.String;@7fe8bbae, /10.2.201. 
 68:[Ljava.lang.String;@879de1d
 Obviously, we should handle the original String Array output behavior better.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8183) improve PropertyFileSnitch output


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14182574#comment-14182574
 ] 

Liang Xie commented on CASSANDRA-8183:
--

Seems i could not assign it to myself right now...

 improve PropertyFileSnitch output
 -

 Key: CASSANDRA-8183
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8183
 Project: Cassandra
  Issue Type: Improvement
  Components: Config
Reporter: Liang Xie
 Fix For: 3.1

 Attachments: CASSANDRA-8183.txt


 Saw log like this: 16:37:11.825 [main] DEBUG 
 o.a.c.locator.PropertyFileSnitch - loaded network topology 
 /10.2.201.66:[Ljava.lang.String;@34e276d9, 
 /10.2.201.65:[Ljava.lang.String;@7fe8bbae, /10.2.201. 
 68:[Ljava.lang.String;@879de1d
 Obviously, we should handle the original String Array output behavior better.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-6285) 2.0 HSHA server introduces corrupt data

2014-10-24 Thread Alexander Sterligov (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14182626#comment-14182626
 ] 

Alexander Sterligov commented on CASSANDRA-6285:


[~xedin] That NPE happend once and unfortunatelly I have not saved it. If I'll 
get it once more I'll save this sstable.
I totally removed OpsCenter keyspace (with sstables) and recreated them. I 
don't get Last written key DecoratedKey any more. By the way, this error 
definetely causees streams to hang on 100%.

I have several strange things happening now:
  - I've noticed that it takes about 30 minutes between nodetool repair and 
first pending AntiEntropySession. Is that ok?
  - Repair is already running for 24 hours (~13GB per node, 17 nodes). What's 
the number of AntiEntropySessions to finish single repair? Number of key ranges?
{quote}
Pool NameActive   Pending  Completed   Blocked  All 
time blocked
CounterMutationStage  0 0  0 0  
   0
ReadStage 0 0 392196 0  
   0
RequestResponseStage  0 05271906 0  
   0
MutationStage 0 0   19832506 0  
   0
ReadRepairStage   0 0   2280 0  
   0
GossipStage   0 0 453830 0  
   0
CacheCleanupExecutor  0 0  0 0  
   0
MigrationStage0 0  0 0  
   0
ValidationExecutor0 0  39446 0  
   0
MemtableReclaimMemory 0 0  29927 0  
   0
InternalResponseStage 0 0 588279 0  
   0
AntiEntropyStage  0 05325285 0  
   0
MiscStage 0 0  0 0  
   0
CommitLogArchiver 0 0  0 0  
   0
MemtableFlushWriter   0 0  29927 0  
   0
PendingRangeCalculator0 0 30 0  
   0
MemtablePostFlush 0 0 135734 0  
   0
CompactionExecutor   3131 502175 0  
   0
AntiEntropySessions   3 3   3446 0  
   0
HintedHandoff 0 0 44 0  
   0

Message type   Dropped
RANGE_SLICE  0
READ_REPAIR  0
PAGED_RANGE  0
BINARY   0
READ 0
MUTATION 2
_TRACE   0
REQUEST_RESPONSE 0
COUNTER_MUTATION 0
{quote}
  - Some validation compactions run for more than 100% (1923%). I thinks that 
it's CASSANDRA-7239, right?

 2.0 HSHA server introduces corrupt data
 ---

 Key: CASSANDRA-6285
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6285
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 4 nodes, shortly updated from 1.2.11 to 2.0.2
Reporter: David Sauer
Assignee: Pavel Yaskevich
Priority: Critical
 Fix For: 2.0.8

 Attachments: 6285_testnotes1.txt, 
 CASSANDRA-6285-disruptor-heap.patch, cassandra-attack-src.zip, 
 compaction_test.py, disruptor-high-cpu.patch, 
 disruptor-memory-corruption.patch, enable_reallocate_buffers.txt


 After altering everything to LCS the table OpsCenter.rollups60 amd one other 
 none OpsCenter-Table got stuck with everything hanging around in L0.
 The compaction started and ran until the logs showed this:
 ERROR [CompactionExecutor:111] 2013-11-01 19:14:53,865 CassandraDaemon.java 
 (line 187) Exception in thread Thread[CompactionExecutor:111,1,RMI Runtime]
 java.lang.RuntimeException: Last written key 
 DecoratedKey(1326283851463420237, 
 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574426c6f6f6d46696c746572537061636555736564)
  = current key DecoratedKey(954210699457429663, 
 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574546f74616c4469736b5370616365557365640b0f)
  writing into 
 /var/lib/cassandra/data/OpsCenter/rollups60/OpsCenter-rollups60-tmp-jb-58656-Data.db
   at

[jira] [Comment Edited] (CASSANDRA-6285) 2.0 HSHA server introduces corrupt data

2014-10-24 Thread Alexander Sterligov (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14182626#comment-14182626
 ] 

Alexander Sterligov edited comment on CASSANDRA-6285 at 10/24/14 9:27 AM:
--

[~xedin] That NPE happend once and unfortunatelly I have not saved it. If I'll 
get it once more I'll save this sstable.
I totally removed OpsCenter keyspace (with sstables) and recreated them. I 
don't get Last written key DecoratedKey any more. By the way, this error 
definetely causees streams to hang on 100%.

I have several strange things happening now:
  - I've noticed that it takes about 30 minutes between nodetool repair and 
first pending AntiEntropySession. Is that ok?
  - Repair is already running for 24 hours (~13GB per node, 17 nodes). What's 
the number of AntiEntropySessions to finish single repair? Number of key ranges?
{quote}
Pool NameActive   Pending  Completed   Blocked  All 
time blocked
CounterMutationStage  0 0  0 0  
   0
ReadStage 0 0 392196 0  
   0
RequestResponseStage  0 05271906 0  
   0
MutationStage 0 0   19832506 0  
   0
ReadRepairStage   0 0   2280 0  
   0
GossipStage   0 0 453830 0  
   0
CacheCleanupExecutor  0 0  0 0  
   0
MigrationStage0 0  0 0  
   0
ValidationExecutor0 0  39446 0  
   0
MemtableReclaimMemory 0 0  29927 0  
   0
InternalResponseStage 0 0 588279 0  
   0
AntiEntropyStage  0 05325285 0  
   0
MiscStage 0 0  0 0  
   0
CommitLogArchiver 0 0  0 0  
   0
MemtableFlushWriter   0 0  29927 0  
   0
PendingRangeCalculator0 0 30 0  
   0
MemtablePostFlush 0 0 135734 0  
   0
CompactionExecutor   3131 502175 0  
   0
AntiEntropySessions   3 3   3446 0  
   0
HintedHandoff 0 0 44 0  
   0

Message type   Dropped
RANGE_SLICE  0
READ_REPAIR  0
PAGED_RANGE  0
BINARY   0
READ 0
MUTATION 2
_TRACE   0
REQUEST_RESPONSE 0
COUNTER_MUTATION 0
{quote}
  - Some validation compactions run for more than 100% (1923%). I thinks that 
it's CASSANDRA-7239, right?
  - the amount of sstables for some CFs is about 15 000 and continues to grow 
during repair.


was (Author: sterligovak):
[~xedin] That NPE happend once and unfortunatelly I have not saved it. If I'll 
get it once more I'll save this sstable.
I totally removed OpsCenter keyspace (with sstables) and recreated them. I 
don't get Last written key DecoratedKey any more. By the way, this error 
definetely causees streams to hang on 100%.

I have several strange things happening now:
  - I've noticed that it takes about 30 minutes between nodetool repair and 
first pending AntiEntropySession. Is that ok?
  - Repair is already running for 24 hours (~13GB per node, 17 nodes). What's 
the number of AntiEntropySessions to finish single repair? Number of key ranges?
{quote}
Pool NameActive   Pending  Completed   Blocked  All 
time blocked
CounterMutationStage  0 0  0 0  
   0
ReadStage 0 0 392196 0  
   0
RequestResponseStage  0 05271906 0  
   0
MutationStage 0 0   19832506 0  
   0
ReadRepairStage   0 0   2280 0  
   0
GossipStage   0 0 453830 0  
   0
CacheCleanupExecutor  0 0  0 0  
   0
MigrationStage0 0  0 0  
   0
ValidationExecutor0 0  39446 0  
   0
MemtableReclaimMemory

[jira] [Comment Edited] (CASSANDRA-6285) 2.0 HSHA server introduces corrupt data

2014-10-24 Thread Alexander Sterligov (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14182626#comment-14182626
 ] 

Alexander Sterligov edited comment on CASSANDRA-6285 at 10/24/14 9:29 AM:
--

[~xedin] That NPE happend once and unfortunatelly I have not saved it. If I'll 
get it once more I'll save this sstable.
I totally removed OpsCenter keyspace (with sstables) and recreated them. I 
don't get Last written key DecoratedKey any more. By the way, this error 
definetely causees streams to hang on 100%.

I have several strange things happening now:
  - I've noticed that it takes about 30 minutes between nodetool repair and 
first pending AntiEntropySession. Is that ok?
  - Repair is already running for 24 hours (~13GB per node, 17 nodes). What's 
the number of AntiEntropySessions to finish single repair? Number of key ranges?
{quote}
Pool NameActive   Pending  Completed   Blocked  All 
time blocked
CounterMutationStage  0 0  0 0  
   0
ReadStage 0 0 392196 0  
   0
RequestResponseStage  0 05271906 0  
   0
MutationStage 0 0   19832506 0  
   0
ReadRepairStage   0 0   2280 0  
   0
GossipStage   0 0 453830 0  
   0
CacheCleanupExecutor  0 0  0 0  
   0
MigrationStage0 0  0 0  
   0
ValidationExecutor0 0  39446 0  
   0
MemtableReclaimMemory 0 0  29927 0  
   0
InternalResponseStage 0 0 588279 0  
   0
AntiEntropyStage  0 05325285 0  
   0
MiscStage 0 0  0 0  
   0
CommitLogArchiver 0 0  0 0  
   0
MemtableFlushWriter   0 0  29927 0  
   0
PendingRangeCalculator0 0 30 0  
   0
MemtablePostFlush 0 0 135734 0  
   0
CompactionExecutor   3131 502175 0  
   0
AntiEntropySessions   3 3   3446 0  
   0
HintedHandoff 0 0 44 0  
   0

Message type   Dropped
RANGE_SLICE  0
READ_REPAIR  0
PAGED_RANGE  0
BINARY   0
READ 0
MUTATION 2
_TRACE   0
REQUEST_RESPONSE 0
COUNTER_MUTATION 0
{quote}
  - Some validation compactions run for more than 100% (1923%). I thinks that 
it's CASSANDRA-7239, right?
  - the amount of sstables for some CFs is about 15 000 and continues to grow 
during repair.
  - There are several following exceptions during repair
{quote}
ERROR [RepairJobTask:80] 2014-10-24 13:27:31,717 RepairJob.java:127 - Error 
occurred during snapshot phase
java.lang.RuntimeException: Could not create snapshot at /37.140.189.163
at 
org.apache.cassandra.repair.SnapshotTask$SnapshotCallback.onFailure(SnapshotTask.java:77)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.net.MessagingService$5$1.run(MessagingService.java:347) 
~[apache-cassandra-2.1.0.jar:2.1.0]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
~[na:1.7.0_51]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) 
~[na:1.7.0_51]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
[na:1.7.0_51]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
[na:1.7.0_51]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
ERROR [AntiEntropySessions:141] 2014-10-24 13:27:31,724 RepairSession.java:303 
- [repair #da2cb020-5b5f-11e4-a45e-d9cec1206f33] session completed with the 
following error
java.io.IOException: Failed during snapshot creation.
at 
org.apache.cassandra.repair.RepairSession.failedSnapshot(RepairSession.java:344)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.repair.RepairJob$2.onFailure(RepairJob.java:128) 
~[apache-cassandra-2.1.0.jar:2.1.0]
at com.google.common.util.concurrent.Futures$4.run(Futures.java:1172) 
~[guava-16.0.jar:na]
at

[jira] [Created] (CASSANDRA-8184) Fix the typo inside TriggerExecutor

Liang Xie created CASSANDRA-8184:


 Summary: Fix the typo inside TriggerExecutor
 Key: CASSANDRA-8184
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8184
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Liang Xie
Priority: Minor


found this while reading code:  tiggerDirectory, should be triggerDirectory :0



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8184) Fix the typo inside TriggerExecutor


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liang Xie updated CASSANDRA-8184:
-
Attachment: CASSANDRA-8184.txt

 Fix the typo inside TriggerExecutor
 ---

 Key: CASSANDRA-8184
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8184
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Liang Xie
Priority: Minor
 Attachments: CASSANDRA-8184.txt


 found this while reading code:  tiggerDirectory, should be triggerDirectory :0



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8149) bump metrics-reporter-config dependency to 2.3.1

2014-10-24 Thread Omid Aladini (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Omid Aladini updated CASSANDRA-8149:

Fix Version/s: 2.0.12

 bump metrics-reporter-config dependency to 2.3.1
 

 Key: CASSANDRA-8149
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8149
 Project: Cassandra
  Issue Type: Improvement
Reporter: Pierre-Yves Ritschard
 Fix For: 2.0.12, 3.0


 It would be nice to be able to take advantage of the new reporters available 
 in
 metrics-reporter-config 2.3.1 which is now available on maven central.
 If my understanding is correct, this only entails bumping the dependency in 
 build.xml.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8149) bump metrics-reporter-config dependency to 2.3.1

2014-10-24 Thread Omid Aladini (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14182634#comment-14182634
 ] 

Omid Aladini commented on CASSANDRA-8149:
-

I just added 2.0.12 to fix version, not sure if it's ok to do so after it's 
already been assigned to. But as there is another reason (making Graphite 
reporter actually useful) besides new reporters mentioned in the issue 
description, thought it may deserve to be patched to the same branch as it was 
first introduced.

 bump metrics-reporter-config dependency to 2.3.1
 

 Key: CASSANDRA-8149
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8149
 Project: Cassandra
  Issue Type: Improvement
Reporter: Pierre-Yves Ritschard
 Fix For: 2.0.12, 3.0


 It would be nice to be able to take advantage of the new reporters available 
 in
 metrics-reporter-config 2.3.1 which is now available on maven central.
 If my understanding is correct, this only entails bumping the dependency in 
 build.xml.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8180) Optimize disk seek using min/max colunm name meta data when the LIMIT clause is used

2014-10-24 Thread DOAN DuyHai (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14182733#comment-14182733
 ] 

DOAN DuyHai commented on CASSANDRA-8180:


The idea is:

If there is no restriction on clustering columns but only on LIMIT:

1) Order SSTables by min/max column depending on the first clustering order
2) Hit the first SSTable and start sequential read until reaching LIMIT
3) If LIMIT is large enough, switch to another SSTable and so on



 Optimize disk seek using min/max colunm name meta data when the LIMIT clause 
 is used
 

 Key: CASSANDRA-8180
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8180
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: Cassandra 2.0.10
Reporter: DOAN DuyHai
Assignee: Tyler Hobbs
Priority: Minor

 I was working on an example of sensor data table (timeseries) and face a use 
 case where C* does not optimize read on disk.
 {code}
 cqlsh:test CREATE TABLE test(id int, col int, val text, PRIMARY KEY(id,col)) 
 WITH CLUSTERING ORDER BY (col DESC);
 cqlsh:test INSERT INTO test(id, col , val ) VALUES ( 1, 10, '10');
 ...
 nodetool flush test test
 ...
 cqlsh:test INSERT INTO test(id, col , val ) VALUES ( 1, 20, '20');
 ...
 nodetool flush test test
 ...
 cqlsh:test INSERT INTO test(id, col , val ) VALUES ( 1, 30, '30');
 ...
 nodetool flush test test
 {code}
 After that, I activate request tracing:
 {code}
 cqlsh:test SELECT * FROM test WHERE id=1 LIMIT 1;
  activity  | 
 timestamp| source| source_elapsed
 ---+--+---+
 execute_cql3_query | 
 23:48:46,498 | 127.0.0.1 |  0
 Parsing SELECT * FROM test WHERE id=1 LIMIT 1; | 
 23:48:46,498 | 127.0.0.1 | 74
Preparing statement | 
 23:48:46,499 | 127.0.0.1 |253
   Executing single-partition query on test | 
 23:48:46,499 | 127.0.0.1 |930
   Acquiring sstable references | 
 23:48:46,499 | 127.0.0.1 |943
Merging memtable tombstones | 
 23:48:46,499 | 127.0.0.1 |   1032
Key cache hit for sstable 3 | 
 23:48:46,500 | 127.0.0.1 |   1160
Seeking to partition beginning in data file | 
 23:48:46,500 | 127.0.0.1 |   1173
Key cache hit for sstable 2 | 
 23:48:46,500 | 127.0.0.1 |   1889
Seeking to partition beginning in data file | 
 23:48:46,500 | 127.0.0.1 |   1901
Key cache hit for sstable 1 | 
 23:48:46,501 | 127.0.0.1 |   2373
Seeking to partition beginning in data file | 
 23:48:46,501 | 127.0.0.1 |   2384
  Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 
 23:48:46,501 | 127.0.0.1 |   2768
 Merging data from memtables and 3 sstables | 
 23:48:46,501 | 127.0.0.1 |   2784
 Read 2 live and 0 tombstoned cells | 
 23:48:46,501 | 127.0.0.1 |   2976
   Request complete | 
 23:48:46,501 | 127.0.0.1 |   3551
 {code}
 We can clearly see that C* hits 3 SSTables on disk instead of just one, 
 although it has the min/max column meta data to decide which SSTable contains 
 the most recent data.
 Funny enough, if we add a clause on the clustering column to the select, this 
 time C* optimizes the read path:
 {code}
 cqlsh:test SELECT * FROM test WHERE id=1 AND col  25 LIMIT 1;
  activity  | 
 timestamp| source| source_elapsed
 ---+--+---+
 execute_cql3_query | 
 23:52:31,888 | 127.0.0.1 |  0
Parsing SELECT * FROM test WHERE id=1 AND col  25 LIMIT 1; | 
 23:52:31,888 | 127.0.0.1 | 60
Preparing statement | 
 23:52:31,888 | 127.0.0.1 |277
   Executing single-partition query on test |

[jira] [Commented] (CASSANDRA-8149) bump metrics-reporter-config dependency to 2.3.1


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14182754#comment-14182754
 ] 

Brandon Williams commented on CASSANDRA-8149:
-

We don't change dependencies in minor versions.

 bump metrics-reporter-config dependency to 2.3.1
 

 Key: CASSANDRA-8149
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8149
 Project: Cassandra
  Issue Type: Improvement
Reporter: Pierre-Yves Ritschard
 Fix For: 3.0


 It would be nice to be able to take advantage of the new reporters available 
 in
 metrics-reporter-config 2.3.1 which is now available on maven central.
 If my understanding is correct, this only entails bumping the dependency in 
 build.xml.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8149) bump metrics-reporter-config dependency to 2.3.1


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-8149:

Fix Version/s: (was: 2.0.12)

 bump metrics-reporter-config dependency to 2.3.1
 

 Key: CASSANDRA-8149
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8149
 Project: Cassandra
  Issue Type: Improvement
Reporter: Pierre-Yves Ritschard
 Fix For: 3.0


 It would be nice to be able to take advantage of the new reporters available 
 in
 metrics-reporter-config 2.3.1 which is now available on maven central.
 If my understanding is correct, this only entails bumping the dependency in 
 build.xml.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8184) Fix the typo inside TriggerExecutor

2014-10-24 Thread Rajanarayanan Thottuvaikkatumana (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-8184:

Reviewer: Vijay

 Fix the typo inside TriggerExecutor
 ---

 Key: CASSANDRA-8184
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8184
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Liang Xie
Priority: Minor
 Attachments: CASSANDRA-8184.txt


 found this while reading code:  tiggerDirectory, should be triggerDirectory :0



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8180) Optimize disk seek using min/max colunm name meta data when the LIMIT clause is used

2014-10-24 Thread DOAN DuyHai (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14182800#comment-14182800
 ] 

DOAN DuyHai commented on CASSANDRA-8180:


Another test showing that it could be optimized further:

{code}
cqlsh:test SELECT * FROM test WHERE id=1 AND col40 LIMIT 1;

 id | col | val
+-+-
  1 |  30 |  30

(1 rows)


Tracing session: 2725c710-5b86-11e4-aeed-814585a29e7b

 activity  | 
timestamp| source| source_elapsed
---+--+---+
execute_cql3_query | 
16:00:46,850 | 127.0.0.1 |  0
 Parsing SELECT * FROM test WHERE id=1 AND col40 LIMIT 1; | 
16:00:46,850 | 127.0.0.1 | 77
   Preparing statement | 
16:00:46,850 | 127.0.0.1 |244
  Executing single-partition query on test | 
16:00:46,851 | 127.0.0.1 |   1485
  Acquiring sstable references | 
16:00:46,851 | 127.0.0.1 |   1500
   Merging memtable tombstones | 
16:00:46,851 | 127.0.0.1 |   1547
   Key cache hit for sstable 3 | 
16:00:46,852 | 127.0.0.1 |   1641
 Seeking to partition indexed section in data file | 
16:00:46,852 | 127.0.0.1 |   1651
   Key cache hit for sstable 2 | 
16:00:46,854 | 127.0.0.1 |   4054
 Seeking to partition indexed section in data file | 
16:00:46,854 | 127.0.0.1 |   4068
   Key cache hit for sstable 1 | 
16:00:46,855 | 127.0.0.1 |   5232
 Seeking to partition indexed section in data file | 
16:00:46,855 | 127.0.0.1 |   5249
 Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 
16:00:46,855 | 127.0.0.1 |   5499
Merging data from memtables and 3 sstables | 
16:00:46,855 | 127.0.0.1 |   5515
Read 2 live and 0 tombstoned cells | 
16:00:46,855 | 127.0.0.1 |   5598
  Request complete | 
16:00:46,855 | 127.0.0.1 |   5997
{code}

 Now reversing the inequality on the clustering column, C* does scan 3 SSTables 
instead of just one (since LIMIT = 1)

 Optimize disk seek using min/max colunm name meta data when the LIMIT clause 
 is used
 

 Key: CASSANDRA-8180
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8180
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: Cassandra 2.0.10
Reporter: DOAN DuyHai
Assignee: Tyler Hobbs
Priority: Minor

 I was working on an example of sensor data table (timeseries) and face a use 
 case where C* does not optimize read on disk.
 {code}
 cqlsh:test CREATE TABLE test(id int, col int, val text, PRIMARY KEY(id,col)) 
 WITH CLUSTERING ORDER BY (col DESC);
 cqlsh:test INSERT INTO test(id, col , val ) VALUES ( 1, 10, '10');
 ...
 nodetool flush test test
 ...
 cqlsh:test INSERT INTO test(id, col , val ) VALUES ( 1, 20, '20');
 ...
 nodetool flush test test
 ...
 cqlsh:test INSERT INTO test(id, col , val ) VALUES ( 1, 30, '30');
 ...
 nodetool flush test test
 {code}
 After that, I activate request tracing:
 {code}
 cqlsh:test SELECT * FROM test WHERE id=1 LIMIT 1;
  activity  | 
 timestamp| source| source_elapsed
 ---+--+---+
 execute_cql3_query | 
 23:48:46,498 | 127.0.0.1 |  0
 Parsing SELECT * FROM test WHERE id=1 LIMIT 1; | 
 23:48:46,498 | 127.0.0.1 | 74
Preparing statement | 
 23:48:46,499 | 127.0.0.1 |253
   Executing single-partition query on test | 
 23:48:46,499 | 127.0.0.1 |930
   Acquiring sstable references | 
 23:48:46,499 | 127.0.0.1 |943
Merging memtable tombstones | 
 23:48:46,499 | 127.0.0.1 |   1032
Key

[jira] [Commented] (CASSANDRA-8116) HSHA fails with default rpc_max_threads setting

2014-10-24 Thread Mike Adamson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14182811#comment-14182811
 ] 

Mike Adamson commented on CASSANDRA-8116:
-

+1 on the latest patch because that will be far clearer to the user than the 
OOM they get at the moment. 

 HSHA fails with default rpc_max_threads setting
 ---

 Key: CASSANDRA-8116
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8116
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Mike Adamson
Assignee: Mike Adamson
Priority: Minor
 Fix For: 2.0.11, 2.1.1

 Attachments: 8116-throw-exc-2.0.txt, 8116.txt


 The HSHA server fails with 'Out of heap space' error if the rpc_max_threads 
 is left at its default setting (unlimited) in cassandra.yaml.
 I'm not proposing any code change for this but have submitted a patch for a 
 comment change in cassandra.yaml to indicate that rpc_max_threads needs to be 
 changed if you use HSHA.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7607) Test coverage for authorization in DDL DML statements


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14182853#comment-14182853
 ] 

Rajanarayanan Thottuvaikkatumana commented on CASSANDRA-7607:
-

I have spent some time on making the TRIGGER work on a setup based on the 
latest trunk code but no luck. I can see two conflicting setup instructions in 
the CQL documentation 
(http://www.datastax.com/documentation/cql/3.1/cql/cql_reference/trigger_r.html)
 where it says the java code has to be placed under the lib/triggers directory. 
I have followed the instructions in the examples/triggers/README.txt to do the 
build of the jar file and stuff like that. But in that it is saying you to keep 
the jar files under conf/triggers/. I have done both the versions separately 
and none of them did work. Which one is correct? Then I unpacked the jar files 
and kept it as the class file preserving the directory structure needed as per 
the package structure. Still no luck. In all the cases it says 
{code}
ErrorMessage code=2300 [Query invalid because of configuration issue] 
message=Trigger class 'org.apache.cassandra.triggers.InvertedIndex' doesn't 
exist
{code}

Any thoughts? 

Do you think making a patch for the items completed and uploading make sense 
while we are working on this? Thanks


 Test coverage for authorization in DDL  DML statements
 ---

 Key: CASSANDRA-7607
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7607
 Project: Cassandra
  Issue Type: Test
  Components: Tests
Reporter: Robert Stupp
Assignee: Rajanarayanan Thottuvaikkatumana
  Labels: lhf, unit-test
 Fix For: 2.0.12


 Similar to CASSANDRA-7604
 Check that the statements perform proper authorization (allow / reject):
 * {{CREATE KEYSPACE}}
 * {{ALTER KEYSPACE}}
 * {{DROP KEYSPACE}}
 * {{CREATE TABLE}}
 * {{ALTER TABLE}}
 * {{DROP TABLE}}
 * {{CREATE TYPE}}
 * {{ALTER TYPE}}
 * {{DROP TYPE}}
 * {{CREATE INDEX}}
 * {{DROP INDEX}}
 * {{CREATE TRIGGER}}
 * {{DROP TRIGGER}}
 * {{CREATE USER}}
 * {{ALTER USER}}
 * {{DROP USER}}
 * {{TRUNCATE}}
 * {{GRANT}}
 * {{REVOKE}}
 * {{SELECT}}
 * {{UPDATE}}
 * {{DELETE}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-8185) Removing the last item from a collection removes the entire row

2014-10-24 Thread Stuart Gunter (JIRA)

Stuart Gunter created CASSANDRA-8185:


 Summary: Removing the last item from a collection removes the 
entire row
 Key: CASSANDRA-8185
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8185
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Mac OS X Yosemite
JDK 8u25
Reporter: Stuart Gunter
Priority: Minor


When removing the last item from a list or set (and I assume the same applies 
to maps), the entire row is deleted rather than just that last item.

I've only tested this in cases where the collection column is the only non-PK 
column, but it is definitely a change between v2.0.5 and v2.0.6. I've looked 
through the 2.0.6 release notes and issues and can't find any item that might 
describe this, so I assume it's not an intended change. If such an issue 
exists, I've probably just missed it, so apologies in advance.

I've created a very simple project to reproduce the issue. If you clone the 
repo and run it in two different modes (the README explains how to do this with 
preconfigured Maven profiles), you will see the tests that pass on v2.0.5 and 
fail on v2.0.6.

Repo: https://github.com/stuartgunter/cassandra-2.0.6-bug



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-8067) NullPointerException in KeyCacheSerializer


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita resolved CASSANDRA-8067.
---
   Resolution: Duplicate
Fix Version/s: 2.1.1

This should be fixed in 2.1.1. (CASSANDRA-7784)

 NullPointerException in KeyCacheSerializer
 --

 Key: CASSANDRA-8067
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8067
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Eric Leleu
 Fix For: 2.1.1


 Hi,
 I have this stack trace in the logs of Cassandra server (v2.1)
 {code}
 ERROR [CompactionExecutor:14] 2014-10-06 23:32:02,098 
 CassandraDaemon.java:166 - Exception in thread 
 Thread[CompactionExecutor:14,1,main]
 java.lang.NullPointerException: null
 at 
 org.apache.cassandra.service.CacheService$KeyCacheSerializer.serialize(CacheService.java:475)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.service.CacheService$KeyCacheSerializer.serialize(CacheService.java:463)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.cache.AutoSavingCache$Writer.saveCache(AutoSavingCache.java:225)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.db.compaction.CompactionManager$11.run(CompactionManager.java:1061)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at java.util.concurrent.Executors$RunnableAdapter.call(Unknown 
 Source) ~[na:1.7.0]
 at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) 
 ~[na:1.7.0]
 at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0]
 at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) 
 [na:1.7.0]
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) 
 [na:1.7.0]
 at java.lang.Thread.run(Unknown Source) [na:1.7.0]
 {code}
 It may not be critical because this error occured in the AutoSavingCache. 
 However the line 475 is about the CFMetaData so it may hide bigger issue...
 {code}
  474 CFMetaData cfm = 
 Schema.instance.getCFMetaData(key.desc.ksname, key.desc.cfname);
  475 cfm.comparator.rowIndexEntrySerializer().serialize(entry, 
 out);
 {code}
 Regards,
 Eric



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8185) Removing the last item from a collection removes the entire row

2014-10-24 Thread Aleksey Yeschenko (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-8185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14182968#comment-14182968
]

Aleksey Yeschenko commented on CASSANDRA-8185:
--

Yes. The change is CASSANDRA-6782, and it was intentional (read its description
and comments for the reasoning behind the change).

Removing the last item from a collection removes the entire row
---

Key: CASSANDRA-8185
URL: https://issues.apache.org/jira/browse/CASSANDRA-8185
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Mac OS X Yosemite
JDK 8u25
Reporter: Stuart Gunter
Priority: Minor

When removing the last item from a list or set (and I assume the same applies
to maps), the entire row is deleted rather than just that last item.
I've only tested this in cases where the collection column is the only non-PK
column, but it is definitely a change between v2.0.5 and v2.0.6. I've looked
through the 2.0.6 release notes and issues and can't find any item that might
describe this, so I assume it's not an intended change. If such an issue
exists, I've probably just missed it, so apologies in advance.
I've created a very simple project to reproduce the issue. If you clone the
repo and run it in two different modes (the README explains how to do this
with preconfigured Maven profiles), you will see the tests that pass on
v2.0.5 and fail on v2.0.6.
Repo: https://github.com/stuartgunter/cassandra-2.0.6-bug

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

git commit: Fix regression introduced by CASSANDRA-8115

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 abbf634dd - 7cf3f19be


Fix regression introduced by CASSANDRA-8115

Patch by Josh McKenzie, reviewed by Philip Thompson for CASSANDRA-8115


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7cf3f19b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7cf3f19b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7cf3f19b

Branch: refs/heads/cassandra-2.1
Commit: 7cf3f19be06e9414aec36757590f9286836159d9
Parents: abbf634
Author: Joshua McKenzie jmcken...@apache.org
Authored: Fri Oct 24 11:03:22 2014 -0500
Committer: Joshua McKenzie jmcken...@apache.org
Committed: Fri Oct 24 11:03:22 2014 -0500

--
 bin/cassandra.ps1 | 12 
 1 file changed, 8 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7cf3f19b/bin/cassandra.ps1
--
diff --git a/bin/cassandra.ps1 b/bin/cassandra.ps1
index a8c8677..53286ab 100644
--- a/bin/cassandra.ps1
+++ b/bin/cassandra.ps1
@@ -80,13 +80,12 @@ Function Main
 $env:CASSANDRA_PARAMS = $env:CASSANDRA_PARAMS + ' 
-Dcassandra-pidfile=' + $pidfile + ''
 }
 
-# Parse -D JVM_OPTS
+# Parse -D and -X JVM_OPTS
 for ($i = 0; $i -lt $script:args.Length; ++$i)
 {
-if ($script:args[$i].Substring(0,2) -eq -D)
+if ($script:args[$i].StartsWith(-D) -Or 
$script:args[$i].StartsWith(-X))
 {
-$param = $script:args[$i].Substring(2)
-$env:JVM_OPTS = $env:JVM_OPTS -D$param
+$env:JVM_OPTS = $env:JVM_OPTS  + $script:args[$i]
 }
 }
 
@@ -304,6 +303,11 @@ Function CheckEmptyParam($param)
 
 for ($i = 0; $i -lt $args.count; $i++)
 {
+# Skip JVM args
+if ($args[$i].StartsWith(-D) -Or $args[$i].StartsWith(-X))
+{
+continue;
+}
 Switch($args[$i])
 {
 -install  { $install = $True }

[1/2] git commit: Fix regression introduced by CASSANDRA-8115

Repository: cassandra
Updated Branches:
  refs/heads/trunk 452dd1080 - 908900800


Fix regression introduced by CASSANDRA-8115

Patch by Josh McKenzie, reviewed by Philip Thompson for CASSANDRA-8115


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7cf3f19b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7cf3f19b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7cf3f19b

Branch: refs/heads/trunk
Commit: 7cf3f19be06e9414aec36757590f9286836159d9
Parents: abbf634
Author: Joshua McKenzie jmcken...@apache.org
Authored: Fri Oct 24 11:03:22 2014 -0500
Committer: Joshua McKenzie jmcken...@apache.org
Committed: Fri Oct 24 11:03:22 2014 -0500

--
 bin/cassandra.ps1 | 12 
 1 file changed, 8 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7cf3f19b/bin/cassandra.ps1
--
diff --git a/bin/cassandra.ps1 b/bin/cassandra.ps1
index a8c8677..53286ab 100644
--- a/bin/cassandra.ps1
+++ b/bin/cassandra.ps1
@@ -80,13 +80,12 @@ Function Main
 $env:CASSANDRA_PARAMS = $env:CASSANDRA_PARAMS + ' 
-Dcassandra-pidfile=' + $pidfile + ''
 }
 
-# Parse -D JVM_OPTS
+# Parse -D and -X JVM_OPTS
 for ($i = 0; $i -lt $script:args.Length; ++$i)
 {
-if ($script:args[$i].Substring(0,2) -eq -D)
+if ($script:args[$i].StartsWith(-D) -Or 
$script:args[$i].StartsWith(-X))
 {
-$param = $script:args[$i].Substring(2)
-$env:JVM_OPTS = $env:JVM_OPTS -D$param
+$env:JVM_OPTS = $env:JVM_OPTS  + $script:args[$i]
 }
 }
 
@@ -304,6 +303,11 @@ Function CheckEmptyParam($param)
 
 for ($i = 0; $i -lt $args.count; $i++)
 {
+# Skip JVM args
+if ($args[$i].StartsWith(-D) -Or $args[$i].StartsWith(-X))
+{
+continue;
+}
 Switch($args[$i])
 {
 -install  { $install = $True }

[2/2] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/90890080
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/90890080
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/90890080

Branch: refs/heads/trunk
Commit: 9089008007b72ccd25ada2e33c253646b3920ea8
Parents: 452dd10 7cf3f19
Author: Joshua McKenzie jmcken...@apache.org
Authored: Fri Oct 24 11:05:13 2014 -0500
Committer: Joshua McKenzie jmcken...@apache.org
Committed: Fri Oct 24 11:05:13 2014 -0500

--
 bin/cassandra.ps1 | 12 
 1 file changed, 8 insertions(+), 4 deletions(-)
--

[jira] [Resolved] (CASSANDRA-8115) Windows install scripts fail to set logdir and datadir

2014-10-24 Thread Joshua McKenzie (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie resolved CASSANDRA-8115.

Resolution: Fixed

eh, if you have to apologize for missing it I have to apologize for introducing 
it in the first place.  It happens.  ;)  Fix committed.

 Windows install scripts fail to set logdir and datadir
 --

 Key: CASSANDRA-8115
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8115
 Project: Cassandra
  Issue Type: Bug
Reporter: Joshua McKenzie
Assignee: Joshua McKenzie
  Labels: Windows
 Fix For: 2.1.2

 Attachments: 8115_fix.txt, 8115_v1.txt, 8115_v2.txt


 After CASSANDRA-7136, the install scripts to run Cassandra as a service fail 
 on both the legacy and the powershell paths.  Looks like they need to have
 {code}
 ++JvmOptions=-Dcassandra.logdir=%CASSANDRA_HOME%\logs ^
 ++JvmOptions=-Dcassandra.storagedir=%CASSANDRA_HOME%\data
 {code}
 added to function correctly.
 We should take this opportunity to make sure the source of the java options 
 is uniform for both running and installation to prevent mismatches like this 
 in the future.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7813) Decide how to deal with conflict between native and user-defined functions


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14182993#comment-14182993
 ] 

Benjamin Lerer commented on CASSANDRA-7813:
---

Here are my feedbacks:
* In {{FunctionName.isReservedKeyspace()}} you should use 
{{Keyspace.SYSTEM_KS}} instead of system and do the comparison using 
{{equalsIgnoreCase}} instead of {{equals}}. You should also moves the function 
into {{Keyspace}} as it belongs more there. It will be nice also if once you 
done that you could use the function in 
{{ThriftValidation.validateKeyspaceNotSystem}} 
* As we use FunctionName as a map key we should keep it immutable and create a 
new instance if we want to change the keyspace, so the class should be final 
and the fields also.
* In {{CreateFunctionStatement.prepareKeyspace}} you make sure that the 
keyspace is set but then on {{checkAccess}} and {{validate}} you still assume 
that it can be unset. So you can remove that code.
* In {{CreateFunctionStatement.announceMigration}} we now know the keyspace so 
we should be able to use it instead of passing null to the functions.
* The check if the keyspace exist should be done in my opinion in 
{{CreateFunctionStatement.prepareKeyspace}}  and not in the {{MigrationManager}}
* The 3 previous feedbacks also apply to {{DropFunctionStatement}}
* The {{DropFunctionStatement.prepareKeyspace}} is copy past of the one of 
{{CreateFunctionStatement}} you should change the error messages
* When the function is a native function it should be associated to the 
{{system}} keyspace so it {{FunctionName.keyspace}} should be {{system}} not an 
empty string.
* {code}
candidates = declared.get(name);
ListFunction candidatesCurrentKs = declared.get(new FunctionName(keyspace, 
name.getName()));
if (candidates.isEmpty())
candidates = candidatesCurrentKs;
else if (!candidatesCurrentKs.isEmpty())
{
ListFunction tmp = new ArrayList(candidates.size() + 
candidatesCurrentKs.size());
tmp.addAll(candidates);
tmp.addAll(candidatesCurrentKs);
candidates = tmp;
 }{code}  can be replaced by {code}candidates = new ArrayList();
candidates.addAll(declared.get(name));
candidates.addAll(declared.get(new FunctionName(keyspace, 
name.getName(;{code}
* The changes are breaking AggregationTest

 Decide how to deal with conflict between native and user-defined functions
 --

 Key: CASSANDRA-7813
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7813
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Robert Stupp
  Labels: cql
 Fix For: 3.0

 Attachments: 7813.txt, 7813v2.txt, 7813v3.txt


 We have a bunch of native/hardcoded functions (now(), dateOf(), ...) and in 
 3.0, user will be able to define new functions. Now, there is a very high 
 change that we will provide more native functions over-time (to be clear, I'm 
 not particularly for adding native functions for allthethings just because we 
 can, but it's clear that we should ultimately provide more than what we 
 have). Which begs the question: how do we want to deal with the problem of 
 adding a native function potentially breaking a previously defined 
 user-defined function?
 A priori I see the following options (maybe there is more?):
 # don't do anything specific, hoping that it won't happen often and consider 
 it a user problem if it does.
 # reserve a big number of names that we're hoping will cover all future need.
 # make native function and user-defined function syntactically distinct so it 
 cannot happen.
 I'm not a huge fan of solution 1). Solution 2) is actually what we did for 
 UDT but I think it's somewhat less practical here: there is so much types 
 that it makes sense to provide natively and so it wasn't too hard to come up 
 with a reasonably small list of types name to reserve just in case. This 
 feels a lot harder for functions to me.
 Which leaves solution 3). Since we already have the concept of namespaces for 
 functions, a simple idea would be to force user function to have namespace. 
 We could even allow that namespace to be empty as long as we force the 
 namespace separator (so we'd allow {{bar::foo}} and {{::foo}} for user 
 functions, but *not* {{foo}} which would be reserved for native function).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Git Push Summary

Repository: cassandra
Updated Tags:  refs/tags/2.1.1-tentative [deleted] 3261d5e66

Git Push Summary

Repository: cassandra
Updated Tags:  refs/tags/cassandra-2.1.1 [created] e5ca62171

git commit: fix typo in TriggerExecutor patch by Liang Xie; reviewed by Vijay for CASSANDRA-8184

2014-10-24 Thread vijay

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 7cf3f19be - 658a65b28


fix typo in TriggerExecutor
patch by Liang Xie; reviewed by Vijay for CASSANDRA-8184


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/658a65b2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/658a65b2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/658a65b2

Branch: refs/heads/cassandra-2.1
Commit: 658a65b281a2ce11b94eb7a60f54de02a1e85342
Parents: 7cf3f19
Author: Vijay vijay2...@gmail.com
Authored: Fri Oct 24 09:19:40 2014 -0700
Committer: Vijay vijay2...@gmail.com
Committed: Fri Oct 24 09:19:40 2014 -0700

--
 src/java/org/apache/cassandra/triggers/TriggerExecutor.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/658a65b2/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
--
diff --git a/src/java/org/apache/cassandra/triggers/TriggerExecutor.java 
b/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
index 001529d..677daad 100644
--- a/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
+++ b/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
@@ -53,10 +53,10 @@ public class TriggerExecutor
  */
 public void reloadClasses()
 {
-File tiggerDirectory = FBUtilities.cassandraTriggerDir();
-if (tiggerDirectory == null)
+File triggerDirectory = FBUtilities.cassandraTriggerDir();
+if (triggerDirectory == null)
 return;
-customClassLoader = new CustomClassLoader(parent, tiggerDirectory);
+customClassLoader = new CustomClassLoader(parent, triggerDirectory);
 cachedTriggers.clear();
 }

[2/2] git commit: Merge branch 'cassandra-2.1' into trunk

2014-10-24 Thread vijay

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2976e693
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2976e693
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2976e693

Branch: refs/heads/trunk
Commit: 2976e69372d261db8275e3cdba28669fab7f3c89
Parents: 9089008 658a65b
Author: Vijay vijay2...@gmail.com
Authored: Fri Oct 24 09:24:05 2014 -0700
Committer: Vijay vijay2...@gmail.com
Committed: Fri Oct 24 09:24:05 2014 -0700

--
 src/java/org/apache/cassandra/triggers/TriggerExecutor.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2976e693/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
--

[1/2] git commit: fix typo in TriggerExecutor patch by Liang Xie; reviewed by Vijay for CASSANDRA-8184

2014-10-24 Thread vijay

Repository: cassandra
Updated Branches:
  refs/heads/trunk 908900800 - 2976e6937


fix typo in TriggerExecutor
patch by Liang Xie; reviewed by Vijay for CASSANDRA-8184


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/658a65b2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/658a65b2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/658a65b2

Branch: refs/heads/trunk
Commit: 658a65b281a2ce11b94eb7a60f54de02a1e85342
Parents: 7cf3f19
Author: Vijay vijay2...@gmail.com
Authored: Fri Oct 24 09:19:40 2014 -0700
Committer: Vijay vijay2...@gmail.com
Committed: Fri Oct 24 09:19:40 2014 -0700

--
 src/java/org/apache/cassandra/triggers/TriggerExecutor.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/658a65b2/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
--
diff --git a/src/java/org/apache/cassandra/triggers/TriggerExecutor.java 
b/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
index 001529d..677daad 100644
--- a/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
+++ b/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
@@ -53,10 +53,10 @@ public class TriggerExecutor
  */
 public void reloadClasses()
 {
-File tiggerDirectory = FBUtilities.cassandraTriggerDir();
-if (tiggerDirectory == null)
+File triggerDirectory = FBUtilities.cassandraTriggerDir();
+if (triggerDirectory == null)
 return;
-customClassLoader = new CustomClassLoader(parent, tiggerDirectory);
+customClassLoader = new CustomClassLoader(parent, triggerDirectory);
 cachedTriggers.clear();
 }

[jira] [Updated] (CASSANDRA-8183) improve PropertyFileSnitch output


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-8183:
---
Reviewer: Tyler Hobbs

 improve PropertyFileSnitch output
 -

 Key: CASSANDRA-8183
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8183
 Project: Cassandra
  Issue Type: Improvement
  Components: Config
Reporter: Liang Xie
Assignee: Liang Xie
 Fix For: 3.1

 Attachments: CASSANDRA-8183.txt


 Saw log like this: 16:37:11.825 [main] DEBUG 
 o.a.c.locator.PropertyFileSnitch - loaded network topology 
 /10.2.201.66:[Ljava.lang.String;@34e276d9, 
 /10.2.201.65:[Ljava.lang.String;@7fe8bbae, /10.2.201. 
 68:[Ljava.lang.String;@879de1d
 Obviously, we should handle the original String Array output behavior better.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8183) improve PropertyFileSnitch output


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-8183:
---
Assignee: Liang Xie

 improve PropertyFileSnitch output
 -

 Key: CASSANDRA-8183
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8183
 Project: Cassandra
  Issue Type: Improvement
  Components: Config
Reporter: Liang Xie
Assignee: Liang Xie
 Fix For: 3.1

 Attachments: CASSANDRA-8183.txt


 Saw log like this: 16:37:11.825 [main] DEBUG 
 o.a.c.locator.PropertyFileSnitch - loaded network topology 
 /10.2.201.66:[Ljava.lang.String;@34e276d9, 
 /10.2.201.65:[Ljava.lang.String;@7fe8bbae, /10.2.201. 
 68:[Ljava.lang.String;@879de1d
 Obviously, we should handle the original String Array output behavior better.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8183) improve PropertyFileSnitch output


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-8183:
---
Priority: Trivial  (was: Major)

 improve PropertyFileSnitch output
 -

 Key: CASSANDRA-8183
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8183
 Project: Cassandra
  Issue Type: Improvement
  Components: Config
Reporter: Liang Xie
Assignee: Liang Xie
Priority: Trivial
 Fix For: 2.1.2

 Attachments: CASSANDRA-8183.txt


 Saw log like this: 16:37:11.825 [main] DEBUG 
 o.a.c.locator.PropertyFileSnitch - loaded network topology 
 /10.2.201.66:[Ljava.lang.String;@34e276d9, 
 /10.2.201.65:[Ljava.lang.String;@7fe8bbae, /10.2.201. 
 68:[Ljava.lang.String;@879de1d
 Obviously, we should handle the original String Array output behavior better.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8183) improve PropertyFileSnitch output


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-8183:
---
Fix Version/s: (was: 3.1)
   2.1.2

 improve PropertyFileSnitch output
 -

 Key: CASSANDRA-8183
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8183
 Project: Cassandra
  Issue Type: Improvement
  Components: Config
Reporter: Liang Xie
Assignee: Liang Xie
 Fix For: 2.1.2

 Attachments: CASSANDRA-8183.txt


 Saw log like this: 16:37:11.825 [main] DEBUG 
 o.a.c.locator.PropertyFileSnitch - loaded network topology 
 /10.2.201.66:[Ljava.lang.String;@34e276d9, 
 /10.2.201.65:[Ljava.lang.String;@7fe8bbae, /10.2.201. 
 68:[Ljava.lang.String;@879de1d
 Obviously, we should handle the original String Array output behavior better.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8168) Require Java 8


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183032#comment-14183032
 ] 

Michael Shuler commented on CASSANDRA-8168:
---

Java SE 7 EOLs April 2015, as well:  
http://www.oracle.com/technetwork/java/eol-135779.html

 Require Java 8
 --

 Key: CASSANDRA-8168
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8168
 Project: Cassandra
  Issue Type: Task
Reporter: T Jake Luciani
 Fix For: 3.0


 This is to discuss requiring Java 8 for version = 3.0  
 There are a couple big reasons for this.
 * Better support for complex async work  e.g (CASSANDRA-5239)
 http://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html
 * Use Nashorn for Javascript UDFs CASSANDRA-7395



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8183) improve PropertyFileSnitch output


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183038#comment-14183038
 ] 

Tyler Hobbs commented on CASSANDRA-8183:


Thanks for the patch!

Here's my feedback:
* I appreciate the attempt to apply a map operation, but it's pretty unwieldy 
in Java, and a simple for-loop would be clearer and shorter here.
* We don't typically use @Nullable (or @NotNull) annotations
* This can be applied to 2.0, which has the same behavior.

 improve PropertyFileSnitch output
 -

 Key: CASSANDRA-8183
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8183
 Project: Cassandra
  Issue Type: Improvement
  Components: Config
Reporter: Liang Xie
Assignee: Liang Xie
Priority: Trivial
 Fix For: 2.0.10, 2.1.2

 Attachments: CASSANDRA-8183.txt


 Saw log like this: 16:37:11.825 [main] DEBUG 
 o.a.c.locator.PropertyFileSnitch - loaded network topology 
 /10.2.201.66:[Ljava.lang.String;@34e276d9, 
 /10.2.201.65:[Ljava.lang.String;@7fe8bbae, /10.2.201. 
 68:[Ljava.lang.String;@879de1d
 Obviously, we should handle the original String Array output behavior better.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8183) improve PropertyFileSnitch output


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-8183:
---
Fix Version/s: 2.0.10

 improve PropertyFileSnitch output
 -

 Key: CASSANDRA-8183
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8183
 Project: Cassandra
  Issue Type: Improvement
  Components: Config
Reporter: Liang Xie
Assignee: Liang Xie
Priority: Trivial
 Fix For: 2.0.10, 2.1.2

 Attachments: CASSANDRA-8183.txt


 Saw log like this: 16:37:11.825 [main] DEBUG 
 o.a.c.locator.PropertyFileSnitch - loaded network topology 
 /10.2.201.66:[Ljava.lang.String;@34e276d9, 
 /10.2.201.65:[Ljava.lang.String;@7fe8bbae, /10.2.201. 
 68:[Ljava.lang.String;@879de1d
 Obviously, we should handle the original String Array output behavior better.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-8116) HSHA fails with default rpc_max_threads setting


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs reassigned CASSANDRA-8116:
--

Assignee: Tyler Hobbs  (was: Mike Adamson)

 HSHA fails with default rpc_max_threads setting
 ---

 Key: CASSANDRA-8116
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8116
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Mike Adamson
Assignee: Tyler Hobbs
Priority: Minor
 Fix For: 2.0.11, 2.1.1

 Attachments: 8116-throw-exc-2.0.txt, 8116.txt


 The HSHA server fails with 'Out of heap space' error if the rpc_max_threads 
 is left at its default setting (unlimited) in cassandra.yaml.
 I'm not proposing any code change for this but have submitted a patch for a 
 comment change in cassandra.yaml to indicate that rpc_max_threads needs to be 
 changed if you use HSHA.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8116) HSHA fails with default rpc_max_threads setting


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-8116:
---
Reviewer: Mike Adamson  (was: Brandon Williams)

 HSHA fails with default rpc_max_threads setting
 ---

 Key: CASSANDRA-8116
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8116
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Mike Adamson
Assignee: Tyler Hobbs
Priority: Minor
 Fix For: 2.0.12, 2.1.2

 Attachments: 8116-throw-exc-2.0.txt, 8116.txt


 The HSHA server fails with 'Out of heap space' error if the rpc_max_threads 
 is left at its default setting (unlimited) in cassandra.yaml.
 I'm not proposing any code change for this but have submitted a patch for a 
 comment change in cassandra.yaml to indicate that rpc_max_threads needs to be 
 changed if you use HSHA.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8116) HSHA fails with default rpc_max_threads setting


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-8116:
---
Fix Version/s: (was: 2.0.11)
   (was: 2.1.1)
   2.1.2
   2.0.12

 HSHA fails with default rpc_max_threads setting
 ---

 Key: CASSANDRA-8116
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8116
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Mike Adamson
Assignee: Tyler Hobbs
Priority: Minor
 Fix For: 2.0.12, 2.1.2

 Attachments: 8116-throw-exc-2.0.txt, 8116.txt


 The HSHA server fails with 'Out of heap space' error if the rpc_max_threads 
 is left at its default setting (unlimited) in cassandra.yaml.
 I'm not proposing any code change for this but have submitted a patch for a 
 comment change in cassandra.yaml to indicate that rpc_max_threads needs to be 
 changed if you use HSHA.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Git Push Summary

Repository: cassandra
Updated Tags:  refs/tags/2.0.11-tentative [deleted] 02b83d9a8

Git Push Summary

Repository: cassandra
Updated Tags:  refs/tags/cassandra-2.0.11 [created] e05e21060

[jira] [Commented] (CASSANDRA-8181) Intermittent failure of SSTableImportTest unit test


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183049#comment-14183049
 ] 

Michael Shuler commented on CASSANDRA-8181:
---

It has repro'ed for me on three different machines - gotta loop over the test, 
but usually fails before it reaches the 10th run.

 Intermittent failure of SSTableImportTest unit test
 ---

 Key: CASSANDRA-8181
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8181
 Project: Cassandra
  Issue Type: Bug
  Components: Tests
Reporter: Michael Shuler
Assignee: Benjamin Lerer
 Fix For: 2.1.2

 Attachments: SSTableImportTest_failure_system.log.gz


 {noformat}
 [junit] Testsuite: org.apache.cassandra.tools.SSTableImportTest
 [junit] Tests run: 8, Failures: 1, Errors: 0, Time elapsed: 6.712 sec
 [junit] 
 [junit] - Standard Output ---
 [junit] Counting keys to import, please wait... (NOTE: to skip this use 
 -n num_keys)
 [junit] Importing 2 keys...
 [junit] 2 keys imported successfully.
 [junit] Counting keys to import, please wait... (NOTE: to skip this use 
 -n num_keys)
 [junit] Importing 2 keys...
 [junit] 2 keys imported successfully.
 [junit] Counting keys to import, please wait... (NOTE: to skip this use 
 -n num_keys)
 [junit] Importing 2 keys...
 [junit] 2 keys imported successfully.
 [junit] Counting keys to import, please wait... (NOTE: to skip this use 
 -n num_keys)
 [junit] Importing 2 keys...
 [junit] Importing 2 keys...
 [junit] 2 keys imported successfully.
 [junit] Counting keys to import, please wait... (NOTE: to skip this use 
 -n num_keys)
 [junit] Importing 2 keys...
 [junit] 2 keys imported successfully.
 [junit] Counting keys to import, please wait... (NOTE: to skip this use 
 -n num_keys)
 [junit] Importing 1 keys...
 [junit] 1 keys imported successfully.
 [junit] Counting keys to import, please wait... (NOTE: to skip this use 
 -n num_keys)
 [junit] Importing 2 keys...
 [junit] 2 keys imported successfully.
 [junit] -  ---
 [junit] Testcase: 
 testImportWithAsciiKeyValidator(org.apache.cassandra.tools.SSTableImportTest):
 FAILED
 [junit] null
 [junit] junit.framework.AssertionFailedError
 [junit] at 
 org.apache.cassandra.tools.SSTableImportTest.testImportWithAsciiKeyValidator(SSTableImportTest.java:166)
 [junit] 
 [junit] 
 [junit] Test org.apache.cassandra.tools.SSTableImportTest FAILED
 {noformat}
 testImportWithAsciiKeyValidator was added in CASSANDRA-7498 and fails as 
 above occasionally (~10-15% of runs) in CI. Attached is the system.log from 
 the failed test on 2.1 HEAD (8e5fdc2).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

svn commit: r1634096 - in /cassandra/site: publish/download/index.html publish/index.html src/settings.py

Author: slebresne
Date: Fri Oct 24 17:00:54 2014
New Revision: 1634096

URL: http://svn.apache.org/r1634096
Log:
Update website for 2.1.1

Modified:
cassandra/site/publish/download/index.html
cassandra/site/publish/index.html
cassandra/site/src/settings.py

Modified: cassandra/site/publish/download/index.html
URL: 
http://svn.apache.org/viewvc/cassandra/site/publish/download/index.html?rev=1634096r1=1634095r2=1634096view=diff
==
--- cassandra/site/publish/download/index.html (original)
+++ cassandra/site/publish/download/index.html Fri Oct 24 17:00:54 2014
@@ -49,8 +49,8 @@
   Cassandra releases include the core server, the a 
href=http://wiki.apache.org/cassandra/NodeTool;nodetool/a administration 
command-line interface, and a development shell (a 
href=http://cassandra.apache.org/doc/cql/CQL.html;ttcqlsh/tt/a and the 
old ttcassandra-cli/tt).
 
   p
-  The latest stable release of Apache Cassandra is 2.1.0
-  (released on 2014-09-11).  iIf you're just
+  The latest stable release of Apache Cassandra is 2.1.1
+  (released on 2014-10-24).  iIf you're just
   starting out, download this one./i
   /p
 
@@ -59,13 +59,13 @@
   ul
 li
 a class=filename 
-   
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/2.1.0/apache-cassandra-2.1.0-bin.tar.gz;
+   
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/2.1.1/apache-cassandra-2.1.1-bin.tar.gz;
onclick=javascript: 
pageTracker._trackPageview('/clicks/binary_download');
-  apache-cassandra-2.1.0-bin.tar.gz
+  apache-cassandra-2.1.1-bin.tar.gz
 /a
-[a 
href=http://www.apache.org/dist/cassandra/2.1.0/apache-cassandra-2.1.0-bin.tar.gz.asc;PGP/a]
-[a 
href=http://www.apache.org/dist/cassandra/2.1.0/apache-cassandra-2.1.0-bin.tar.gz.md5;MD5/a]
-[a 
href=http://www.apache.org/dist/cassandra/2.1.0/apache-cassandra-2.1.0-bin.tar.gz.sha1;SHA1/a]
+[a 
href=http://www.apache.org/dist/cassandra/2.1.1/apache-cassandra-2.1.1-bin.tar.gz.asc;PGP/a]
+[a 
href=http://www.apache.org/dist/cassandra/2.1.1/apache-cassandra-2.1.1-bin.tar.gz.md5;MD5/a]
+[a 
href=http://www.apache.org/dist/cassandra/2.1.1/apache-cassandra-2.1.1-bin.tar.gz.sha1;SHA1/a]
 /li
 li
 a href=http://wiki.apache.org/cassandra/DebianPackaging;Debian 
installation instructions/a
@@ -144,13 +144,13 @@
   ul
 li
 a class=filename 
-   
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/2.1.0/apache-cassandra-2.1.0-src.tar.gz;
+   
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/2.1.1/apache-cassandra-2.1.1-src.tar.gz;
onclick=javascript: 
pageTracker._trackPageview('/clicks/source_download');
-  apache-cassandra-2.1.0-src.tar.gz
+  apache-cassandra-2.1.1-src.tar.gz
 /a
-[a 
href=http://www.apache.org/dist/cassandra/2.1.0/apache-cassandra-2.1.0-src.tar.gz.asc;PGP/a]
-[a 
href=http://www.apache.org/dist/cassandra/2.1.0/apache-cassandra-2.1.0-src.tar.gz.md5;MD5/a]
-[a 
href=http://www.apache.org/dist/cassandra/2.1.0/apache-cassandra-2.1.0-src.tar.gz.sha1;SHA1/a]
+[a 
href=http://www.apache.org/dist/cassandra/2.1.1/apache-cassandra-2.1.1-src.tar.gz.asc;PGP/a]
+[a 
href=http://www.apache.org/dist/cassandra/2.1.1/apache-cassandra-2.1.1-src.tar.gz.md5;MD5/a]
+[a 
href=http://www.apache.org/dist/cassandra/2.1.1/apache-cassandra-2.1.1-src.tar.gz.sha1;SHA1/a]
 /li
   
 li

Modified: cassandra/site/publish/index.html
URL: 
http://svn.apache.org/viewvc/cassandra/site/publish/index.html?rev=1634096r1=1634095r2=1634096view=diff
==
--- cassandra/site/publish/index.html (original)
+++ cassandra/site/publish/index.html Fri Oct 24 17:00:54 2014
@@ -76,8 +76,8 @@
   h2Download/h2
   div class=inner rc
 p
-The latest release is b2.1.0/b
-span class=relnotes(a 
href=http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/cassandra-2.1.0;Changes/a)/span
+The latest release is b2.1.1/b
+span class=relnotes(a 
href=http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/cassandra-2.1.1;Changes/a)/span
 /p
 
 pa class=filename href=/download/Download options/a/p

Modified: cassandra/site/src/settings.py
URL: 
http://svn.apache.org/viewvc/cassandra/site/src/settings.py?rev=1634096r1=1634095r2=1634096view=diff
==
--- cassandra/site/src/settings.py (original)
+++ cassandra/site/src/settings.py Fri Oct 24 17:00:54 2014
@@ -98,8 +98,8 @@ class CassandraDef(object):
 veryoldstable_version = '1.2.19'
 veryoldstable_release_date = '2014-09-18'
 veryoldstable_exists = True
-stable_version = '2.1.0'
-stable_release_date = '2014-09-11'
+stable_version = '2.1.1'
+stable_release_date = '2014-10-24'
 devel_version

[jira] [Comment Edited] (CASSANDRA-8181) Intermittent failure of SSTableImportTest unit test


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183049#comment-14183049
 ] 

Michael Shuler edited comment on CASSANDRA-8181 at 10/24/14 5:03 PM:
-

It has repro'ed for me on three different machines - gotta loop over the test, 
but usually fails before it reaches the 10th run. A 25x=good bisection on two 
different machines each landed me on the initial commit of this test, so it has 
been intermittent all along  :)


was (Author: mshuler):
It has repro'ed for me on three different machines - gotta loop over the test, 
but usually fails before it reaches the 10th run.

 Intermittent failure of SSTableImportTest unit test
 ---

 Key: CASSANDRA-8181
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8181
 Project: Cassandra
  Issue Type: Bug
  Components: Tests
Reporter: Michael Shuler
Assignee: Benjamin Lerer
 Fix For: 2.1.2

 Attachments: SSTableImportTest_failure_system.log.gz


 {noformat}
 [junit] Testsuite: org.apache.cassandra.tools.SSTableImportTest
 [junit] Tests run: 8, Failures: 1, Errors: 0, Time elapsed: 6.712 sec
 [junit] 
 [junit] - Standard Output ---
 [junit] Counting keys to import, please wait... (NOTE: to skip this use 
 -n num_keys)
 [junit] Importing 2 keys...
 [junit] 2 keys imported successfully.
 [junit] Counting keys to import, please wait... (NOTE: to skip this use 
 -n num_keys)
 [junit] Importing 2 keys...
 [junit] 2 keys imported successfully.
 [junit] Counting keys to import, please wait... (NOTE: to skip this use 
 -n num_keys)
 [junit] Importing 2 keys...
 [junit] 2 keys imported successfully.
 [junit] Counting keys to import, please wait... (NOTE: to skip this use 
 -n num_keys)
 [junit] Importing 2 keys...
 [junit] Importing 2 keys...
 [junit] 2 keys imported successfully.
 [junit] Counting keys to import, please wait... (NOTE: to skip this use 
 -n num_keys)
 [junit] Importing 2 keys...
 [junit] 2 keys imported successfully.
 [junit] Counting keys to import, please wait... (NOTE: to skip this use 
 -n num_keys)
 [junit] Importing 1 keys...
 [junit] 1 keys imported successfully.
 [junit] Counting keys to import, please wait... (NOTE: to skip this use 
 -n num_keys)
 [junit] Importing 2 keys...
 [junit] 2 keys imported successfully.
 [junit] -  ---
 [junit] Testcase: 
 testImportWithAsciiKeyValidator(org.apache.cassandra.tools.SSTableImportTest):
 FAILED
 [junit] null
 [junit] junit.framework.AssertionFailedError
 [junit] at 
 org.apache.cassandra.tools.SSTableImportTest.testImportWithAsciiKeyValidator(SSTableImportTest.java:166)
 [junit] 
 [junit] 
 [junit] Test org.apache.cassandra.tools.SSTableImportTest FAILED
 {noformat}
 testImportWithAsciiKeyValidator was added in CASSANDRA-7498 and fails as 
 above occasionally (~10-15% of runs) in CI. Attached is the system.log from 
 the failed test on 2.1 HEAD (8e5fdc2).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

git commit: Error when using hsha w/ unlimited rpc_max_threads

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 46f43cee7 - 1b332bc1c


Error when using hsha w/ unlimited rpc_max_threads

Patch by Tyler Hobbs; reviewed by Mike Adamson for CASSANDRA-8116


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1b332bc1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1b332bc1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1b332bc1

Branch: refs/heads/cassandra-2.0
Commit: 1b332bc1c02786623e2baf773e9f46af9c04f21f
Parents: 46f43ce
Author: Tyler Hobbs ty...@datastax.com
Authored: Fri Oct 24 12:02:16 2014 -0500
Committer: Tyler Hobbs ty...@datastax.com
Committed: Fri Oct 24 12:02:16 2014 -0500

--
 CHANGES.txt  | 2 ++
 src/java/org/apache/cassandra/config/DatabaseDescriptor.java | 7 +++
 src/java/org/apache/cassandra/thrift/ThriftServer.java   | 6 +++---
 3 files changed, 12 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1b332bc1/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 6c2be8c..11f6517 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,7 @@
 2.0.12:
  * Pig: Remove errant LIMIT clause in CqlNativeStorage (CASSANDRA-8166)
+ * Throw ConfigurationException when hsha is used with the default
+   rpc_max_threads setting of 'unlimited' (CASSANDRA-8116)
 
 
 2.0.11:

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1b332bc1/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index 209d6c9..92ef79a 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -27,6 +27,7 @@ import java.util.*;
 import com.google.common.annotations.VisibleForTesting;
 import com.google.common.collect.ImmutableSet;
 import com.google.common.primitives.Longs;
+import org.apache.cassandra.thrift.ThriftServer;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -333,6 +334,12 @@ public class DatabaseDescriptor
 if (conf.native_transport_max_frame_size_in_mb = 0)
 throw new 
ConfigurationException(native_transport_max_frame_size_in_mb must be 
positive);
 
+// fail early instead of OOMing (see CASSANDRA-8116)
+if (ThriftServer.HSHA.equals(conf.rpc_server_type)  
conf.rpc_max_threads == Integer.MAX_VALUE)
+throw new ConfigurationException(The hsha rpc_server_type is not 
compatible with an rpc_max_threads  +
+ setting of 'unlimited'.  Please 
see the comments in cassandra.yaml  +
+ for rpc_server_type and 
rpc_max_threads.);
+
 /* end point snitch */
 if (conf.endpoint_snitch == null)
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1b332bc1/src/java/org/apache/cassandra/thrift/ThriftServer.java
--
diff --git a/src/java/org/apache/cassandra/thrift/ThriftServer.java 
b/src/java/org/apache/cassandra/thrift/ThriftServer.java
index dbd3824..7ca7261 100644
--- a/src/java/org/apache/cassandra/thrift/ThriftServer.java
+++ b/src/java/org/apache/cassandra/thrift/ThriftServer.java
@@ -34,9 +34,9 @@ import org.apache.thrift.transport.TTransportFactory;
 public class ThriftServer implements CassandraDaemon.Server
 {
 private static Logger logger = LoggerFactory.getLogger(ThriftServer.class);
-protected final static String SYNC = sync;
-protected final static String ASYNC = async;
-protected final static String HSHA = hsha;
+public final static String SYNC = sync;
+public final static String ASYNC = async;
+public final static String HSHA = hsha;
 
 protected final InetAddress address;
 protected final int port;

[1/2] git commit: Error when using hsha w/ unlimited rpc_max_threads

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 658a65b28 - 860cde74a


Error when using hsha w/ unlimited rpc_max_threads

Patch by Tyler Hobbs; reviewed by Mike Adamson for CASSANDRA-8116


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1b332bc1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1b332bc1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1b332bc1

Branch: refs/heads/cassandra-2.1
Commit: 1b332bc1c02786623e2baf773e9f46af9c04f21f
Parents: 46f43ce
Author: Tyler Hobbs ty...@datastax.com
Authored: Fri Oct 24 12:02:16 2014 -0500
Committer: Tyler Hobbs ty...@datastax.com
Committed: Fri Oct 24 12:02:16 2014 -0500

--
 CHANGES.txt  | 2 ++
 src/java/org/apache/cassandra/config/DatabaseDescriptor.java | 7 +++
 src/java/org/apache/cassandra/thrift/ThriftServer.java   | 6 +++---
 3 files changed, 12 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1b332bc1/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 6c2be8c..11f6517 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,7 @@
 2.0.12:
  * Pig: Remove errant LIMIT clause in CqlNativeStorage (CASSANDRA-8166)
+ * Throw ConfigurationException when hsha is used with the default
+   rpc_max_threads setting of 'unlimited' (CASSANDRA-8116)
 
 
 2.0.11:

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1b332bc1/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index 209d6c9..92ef79a 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -27,6 +27,7 @@ import java.util.*;
 import com.google.common.annotations.VisibleForTesting;
 import com.google.common.collect.ImmutableSet;
 import com.google.common.primitives.Longs;
+import org.apache.cassandra.thrift.ThriftServer;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -333,6 +334,12 @@ public class DatabaseDescriptor
 if (conf.native_transport_max_frame_size_in_mb = 0)
 throw new 
ConfigurationException(native_transport_max_frame_size_in_mb must be 
positive);
 
+// fail early instead of OOMing (see CASSANDRA-8116)
+if (ThriftServer.HSHA.equals(conf.rpc_server_type)  
conf.rpc_max_threads == Integer.MAX_VALUE)
+throw new ConfigurationException(The hsha rpc_server_type is not 
compatible with an rpc_max_threads  +
+ setting of 'unlimited'.  Please 
see the comments in cassandra.yaml  +
+ for rpc_server_type and 
rpc_max_threads.);
+
 /* end point snitch */
 if (conf.endpoint_snitch == null)
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1b332bc1/src/java/org/apache/cassandra/thrift/ThriftServer.java
--
diff --git a/src/java/org/apache/cassandra/thrift/ThriftServer.java 
b/src/java/org/apache/cassandra/thrift/ThriftServer.java
index dbd3824..7ca7261 100644
--- a/src/java/org/apache/cassandra/thrift/ThriftServer.java
+++ b/src/java/org/apache/cassandra/thrift/ThriftServer.java
@@ -34,9 +34,9 @@ import org.apache.thrift.transport.TTransportFactory;
 public class ThriftServer implements CassandraDaemon.Server
 {
 private static Logger logger = LoggerFactory.getLogger(ThriftServer.class);
-protected final static String SYNC = sync;
-protected final static String ASYNC = async;
-protected final static String HSHA = hsha;
+public final static String SYNC = sync;
+public final static String ASYNC = async;
+public final static String HSHA = hsha;
 
 protected final InetAddress address;
 protected final int port;

[jira] [Commented] (CASSANDRA-7463) Update CQLSSTableWriter to allow parallel writing of SSTables on the same table within the same JVM

2014-10-24 Thread Jeremy Hanna (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183051#comment-14183051
 ] 

Jeremy Hanna commented on CASSANDRA-7463:
-

Any update on this?  Seems like it would make the new bulk loader more 
efficient but also allow for things like bulk Spark persistence to Cassandra, 
is that what you were saying [~tupshin], which would be very nice.

 Update CQLSSTableWriter to allow parallel writing of SSTables on the same 
 table within the same JVM
 ---

 Key: CASSANDRA-7463
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7463
 Project: Cassandra
  Issue Type: Improvement
Reporter: Johnny Miller
Assignee: Carl Yeksigian
 Fix For: 2.0.12

 Attachments: 7463-v2.txt, 7463.patch


 Currently it is not possible to programatically write multiple SSTables for 
 the same table in parallel using the CQLSSTableWriter. This is quite a 
 limitation and the workaround of attempting to do this in a separate JVM is 
 not a great solution.
 See: 
 http://stackoverflow.com/questions/24396902/using-cqlsstablewriter-concurrently



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/2] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/860cde74
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/860cde74
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/860cde74

Branch: refs/heads/cassandra-2.1
Commit: 860cde74a65b781cdd7e501f87ac171b20f95d20
Parents: 658a65b 1b332bc
Author: Tyler Hobbs ty...@datastax.com
Authored: Fri Oct 24 12:04:40 2014 -0500
Committer: Tyler Hobbs ty...@datastax.com
Committed: Fri Oct 24 12:04:40 2014 -0500

--
 CHANGES.txt  | 2 ++
 src/java/org/apache/cassandra/config/DatabaseDescriptor.java | 7 +++
 src/java/org/apache/cassandra/thrift/ThriftServer.java   | 6 +++---
 3 files changed, 12 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/860cde74/CHANGES.txt
--
diff --cc CHANGES.txt
index d6827db,11f6517..c136c5e
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,101 -1,10 +1,103 @@@
 -2.0.12:
 +2.1.2
 + * Fix installing as service on Windows (CASSANDRA-8115)
 + * Fix CREATE TABLE for CQL2 (CASSANDRA-8144)
 + * Avoid boxing in ColumnStats min/max trackers (CASSANDRA-8109)
 +Merged from 2.0:
   * Pig: Remove errant LIMIT clause in CqlNativeStorage (CASSANDRA-8166)
+  * Throw ConfigurationException when hsha is used with the default
+rpc_max_threads setting of 'unlimited' (CASSANDRA-8116)
  
  
 -2.0.11:
 +2.1.1
 + * Dont notify when replacing tmplink files (CASSANDRA-8157)
 + * Fix validation with multiple CONTAINS clause (CASSANDRA-8131)
 + * Fix validation of collections in TriggerExecutor (CASSANDRA-8146)
 + * Fix IllegalArgumentException when a list of IN values containing tuples
 +   is passed as a single arg to a prepared statement with the v1 or v2
 +   protocol (CASSANDRA-8062)
 + * Fix ClassCastException in DISTINCT query on static columns with
 +   query paging (CASSANDRA-8108)
 + * Fix NPE on null nested UDT inside a set (CASSANDRA-8105)
 + * Fix exception when querying secondary index on set items or map keys
 +   when some clustering columns are specified (CASSANDRA-8073)
 + * Send proper error response when there is an error during native
 +   protocol message decode (CASSANDRA-8118)
 + * Gossip should ignore generation numbers too far in the future 
(CASSANDRA-8113)
 + * Fix NPE when creating a table with frozen sets, lists (CASSANDRA-8104)
 + * Fix high memory use due to tracking reads on incrementally opened sstable
 +   readers (CASSANDRA-8066)
 + * Fix EXECUTE request with skipMetadata=false returning no metadata
 +   (CASSANDRA-8054)
 + * Allow concurrent use of CQLBulkOutputFormat (CASSANDRA-7776)
 + * Shutdown JVM on OOM (CASSANDRA-7507)
 + * Upgrade netty version and enable epoll event loop (CASSANDRA-7761)
 + * Don't duplicate sstables smaller than split size when using
 +   the sstablesplitter tool (CASSANDRA-7616)
 + * Avoid re-parsing already prepared statements (CASSANDRA-7923)
 + * Fix some Thrift slice deletions and updates of COMPACT STORAGE
 +   tables with some clustering columns omitted (CASSANDRA-7990)
 + * Fix filtering for CONTAINS on sets (CASSANDRA-8033)
 + * Properly track added size (CASSANDRA-7239)
 + * Allow compilation in java 8 (CASSANDRA-7208)
 + * Fix Assertion error on RangeTombstoneList diff (CASSANDRA-8013)
 + * Release references to overlapping sstables during compaction 
(CASSANDRA-7819)
 + * Send notification when opening compaction results early (CASSANDRA-8034)
 + * Make native server start block until properly bound (CASSANDRA-7885)
 + * (cqlsh) Fix IPv6 support (CASSANDRA-7988)
 + * Ignore fat clients when checking for endpoint collision (CASSANDRA-7939)
 + * Make sstablerepairedset take a list of files (CASSANDRA-7995)
 + * (cqlsh) Tab completeion for indexes on map keys (CASSANDRA-7972)
 + * (cqlsh) Fix UDT field selection in select clause (CASSANDRA-7891)
 + * Fix resource leak in event of corrupt sstable
 + * (cqlsh) Add command line option for cqlshrc file path (CASSANDRA-7131)
 + * Provide visibility into prepared statements churn (CASSANDRA-7921, 
CASSANDRA-7930)
 + * Invalidate prepared statements when their keyspace or table is
 +   dropped (CASSANDRA-7566)
 + * cassandra-stress: fix support for NetworkTopologyStrategy (CASSANDRA-7945)
 + * Fix saving caches when a table is dropped (CASSANDRA-7784)
 + * Add better error checking of new stress profile (CASSANDRA-7716)
 + * Use ThreadLocalRandom and remove FBUtilities.threadLocalRandom 
(CASSANDRA-7934)
 + * Prevent operator mistakes due to simultaneous bootstrap (CASSANDRA-7069)
 + * cassandra-stress supports whitelist mode for node config (CASSANDRA-7658)
 + * GCInspector more closely tracks GC; cassandra-stress and

[jira] [Updated] (CASSANDRA-8137) Prepared statement size overflow error


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-8137:

Fix Version/s: (was: 2.1.1)
   2.1.2

 Prepared statement size overflow error
 --

 Key: CASSANDRA-8137
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8137
 Project: Cassandra
  Issue Type: Bug
 Environment: Linux Mint 64 | C* 2.1.0 | Ruby-driver master
Reporter: Kishan Karunaratne
 Fix For: 2.1.2


 When using C* 2.1.0 and Ruby-driver master, I get the following error when 
 running the Ruby duration test (which prepares a lot of statements, in many 
 threads):
 {noformat}
 Prepared statement of size 4451848 bytes is larger than allowed maximum of 
 2027520 bytes.
 Prepared statement of size 4434568 bytes is larger than allowed maximum of 
 2027520 bytes.
 {noformat}
 They usually occur in batches of 1, but sometimes in multiples as seen above. 
  It happens occasionally, around 20% of the time when running the code.  
 Unfortunately I don't have a stacktrace as the error isn't recorded in the 
 system log. 
 This is my schema, and the offending prepare statement:
 {noformat}
 @session.execute(CREATE TABLE duration_test.ints (
 key INT,
 copy INT,
 value INT,
 PRIMARY KEY (key, copy))
 )
 {noformat}
 {noformat}
 select = @session.prepare(SELECT * FROM ints WHERE key=?)
 {noformat}
 Now, I notice that if I explicitly specify the keyspace in the prepare, I 
 don't get the error.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/860cde74
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/860cde74
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/860cde74

Branch: refs/heads/trunk
Commit: 860cde74a65b781cdd7e501f87ac171b20f95d20
Parents: 658a65b 1b332bc
Author: Tyler Hobbs ty...@datastax.com
Authored: Fri Oct 24 12:04:40 2014 -0500
Committer: Tyler Hobbs ty...@datastax.com
Committed: Fri Oct 24 12:04:40 2014 -0500

--
 CHANGES.txt  | 2 ++
 src/java/org/apache/cassandra/config/DatabaseDescriptor.java | 7 +++
 src/java/org/apache/cassandra/thrift/ThriftServer.java   | 6 +++---
 3 files changed, 12 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/860cde74/CHANGES.txt
--
diff --cc CHANGES.txt
index d6827db,11f6517..c136c5e
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,101 -1,10 +1,103 @@@
 -2.0.12:
 +2.1.2
 + * Fix installing as service on Windows (CASSANDRA-8115)
 + * Fix CREATE TABLE for CQL2 (CASSANDRA-8144)
 + * Avoid boxing in ColumnStats min/max trackers (CASSANDRA-8109)
 +Merged from 2.0:
   * Pig: Remove errant LIMIT clause in CqlNativeStorage (CASSANDRA-8166)
+  * Throw ConfigurationException when hsha is used with the default
+rpc_max_threads setting of 'unlimited' (CASSANDRA-8116)
  
  
 -2.0.11:
 +2.1.1
 + * Dont notify when replacing tmplink files (CASSANDRA-8157)
 + * Fix validation with multiple CONTAINS clause (CASSANDRA-8131)
 + * Fix validation of collections in TriggerExecutor (CASSANDRA-8146)
 + * Fix IllegalArgumentException when a list of IN values containing tuples
 +   is passed as a single arg to a prepared statement with the v1 or v2
 +   protocol (CASSANDRA-8062)
 + * Fix ClassCastException in DISTINCT query on static columns with
 +   query paging (CASSANDRA-8108)
 + * Fix NPE on null nested UDT inside a set (CASSANDRA-8105)
 + * Fix exception when querying secondary index on set items or map keys
 +   when some clustering columns are specified (CASSANDRA-8073)
 + * Send proper error response when there is an error during native
 +   protocol message decode (CASSANDRA-8118)
 + * Gossip should ignore generation numbers too far in the future 
(CASSANDRA-8113)
 + * Fix NPE when creating a table with frozen sets, lists (CASSANDRA-8104)
 + * Fix high memory use due to tracking reads on incrementally opened sstable
 +   readers (CASSANDRA-8066)
 + * Fix EXECUTE request with skipMetadata=false returning no metadata
 +   (CASSANDRA-8054)
 + * Allow concurrent use of CQLBulkOutputFormat (CASSANDRA-7776)
 + * Shutdown JVM on OOM (CASSANDRA-7507)
 + * Upgrade netty version and enable epoll event loop (CASSANDRA-7761)
 + * Don't duplicate sstables smaller than split size when using
 +   the sstablesplitter tool (CASSANDRA-7616)
 + * Avoid re-parsing already prepared statements (CASSANDRA-7923)
 + * Fix some Thrift slice deletions and updates of COMPACT STORAGE
 +   tables with some clustering columns omitted (CASSANDRA-7990)
 + * Fix filtering for CONTAINS on sets (CASSANDRA-8033)
 + * Properly track added size (CASSANDRA-7239)
 + * Allow compilation in java 8 (CASSANDRA-7208)
 + * Fix Assertion error on RangeTombstoneList diff (CASSANDRA-8013)
 + * Release references to overlapping sstables during compaction 
(CASSANDRA-7819)
 + * Send notification when opening compaction results early (CASSANDRA-8034)
 + * Make native server start block until properly bound (CASSANDRA-7885)
 + * (cqlsh) Fix IPv6 support (CASSANDRA-7988)
 + * Ignore fat clients when checking for endpoint collision (CASSANDRA-7939)
 + * Make sstablerepairedset take a list of files (CASSANDRA-7995)
 + * (cqlsh) Tab completeion for indexes on map keys (CASSANDRA-7972)
 + * (cqlsh) Fix UDT field selection in select clause (CASSANDRA-7891)
 + * Fix resource leak in event of corrupt sstable
 + * (cqlsh) Add command line option for cqlshrc file path (CASSANDRA-7131)
 + * Provide visibility into prepared statements churn (CASSANDRA-7921, 
CASSANDRA-7930)
 + * Invalidate prepared statements when their keyspace or table is
 +   dropped (CASSANDRA-7566)
 + * cassandra-stress: fix support for NetworkTopologyStrategy (CASSANDRA-7945)
 + * Fix saving caches when a table is dropped (CASSANDRA-7784)
 + * Add better error checking of new stress profile (CASSANDRA-7716)
 + * Use ThreadLocalRandom and remove FBUtilities.threadLocalRandom 
(CASSANDRA-7934)
 + * Prevent operator mistakes due to simultaneous bootstrap (CASSANDRA-7069)
 + * cassandra-stress supports whitelist mode for node config (CASSANDRA-7658)
 + * GCInspector more closely tracks GC; cassandra-stress and nodetool

[1/3] git commit: Error when using hsha w/ unlimited rpc_max_threads

Repository: cassandra
Updated Branches:
  refs/heads/trunk 2976e6937 - 209ea6e9b


Error when using hsha w/ unlimited rpc_max_threads

Patch by Tyler Hobbs; reviewed by Mike Adamson for CASSANDRA-8116


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1b332bc1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1b332bc1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1b332bc1

Branch: refs/heads/trunk
Commit: 1b332bc1c02786623e2baf773e9f46af9c04f21f
Parents: 46f43ce
Author: Tyler Hobbs ty...@datastax.com
Authored: Fri Oct 24 12:02:16 2014 -0500
Committer: Tyler Hobbs ty...@datastax.com
Committed: Fri Oct 24 12:02:16 2014 -0500

--
 CHANGES.txt  | 2 ++
 src/java/org/apache/cassandra/config/DatabaseDescriptor.java | 7 +++
 src/java/org/apache/cassandra/thrift/ThriftServer.java   | 6 +++---
 3 files changed, 12 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1b332bc1/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 6c2be8c..11f6517 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,7 @@
 2.0.12:
  * Pig: Remove errant LIMIT clause in CqlNativeStorage (CASSANDRA-8166)
+ * Throw ConfigurationException when hsha is used with the default
+   rpc_max_threads setting of 'unlimited' (CASSANDRA-8116)
 
 
 2.0.11:

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1b332bc1/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index 209d6c9..92ef79a 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -27,6 +27,7 @@ import java.util.*;
 import com.google.common.annotations.VisibleForTesting;
 import com.google.common.collect.ImmutableSet;
 import com.google.common.primitives.Longs;
+import org.apache.cassandra.thrift.ThriftServer;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -333,6 +334,12 @@ public class DatabaseDescriptor
 if (conf.native_transport_max_frame_size_in_mb = 0)
 throw new 
ConfigurationException(native_transport_max_frame_size_in_mb must be 
positive);
 
+// fail early instead of OOMing (see CASSANDRA-8116)
+if (ThriftServer.HSHA.equals(conf.rpc_server_type)  
conf.rpc_max_threads == Integer.MAX_VALUE)
+throw new ConfigurationException(The hsha rpc_server_type is not 
compatible with an rpc_max_threads  +
+ setting of 'unlimited'.  Please 
see the comments in cassandra.yaml  +
+ for rpc_server_type and 
rpc_max_threads.);
+
 /* end point snitch */
 if (conf.endpoint_snitch == null)
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1b332bc1/src/java/org/apache/cassandra/thrift/ThriftServer.java
--
diff --git a/src/java/org/apache/cassandra/thrift/ThriftServer.java 
b/src/java/org/apache/cassandra/thrift/ThriftServer.java
index dbd3824..7ca7261 100644
--- a/src/java/org/apache/cassandra/thrift/ThriftServer.java
+++ b/src/java/org/apache/cassandra/thrift/ThriftServer.java
@@ -34,9 +34,9 @@ import org.apache.thrift.transport.TTransportFactory;
 public class ThriftServer implements CassandraDaemon.Server
 {
 private static Logger logger = LoggerFactory.getLogger(ThriftServer.class);
-protected final static String SYNC = sync;
-protected final static String ASYNC = async;
-protected final static String HSHA = hsha;
+public final static String SYNC = sync;
+public final static String ASYNC = async;
+public final static String HSHA = hsha;
 
 protected final InetAddress address;
 protected final int port;

[3/3] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk

Conflicts:
src/java/org/apache/cassandra/config/DatabaseDescriptor.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/209ea6e9
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/209ea6e9
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/209ea6e9

Branch: refs/heads/trunk
Commit: 209ea6e9b1c02f37576ae195bf59803d31b30cbb
Parents: 2976e69 860cde7
Author: Tyler Hobbs ty...@datastax.com
Authored: Fri Oct 24 12:07:55 2014 -0500
Committer: Tyler Hobbs ty...@datastax.com
Committed: Fri Oct 24 12:07:55 2014 -0500

--
 CHANGES.txt  | 2 ++
 src/java/org/apache/cassandra/config/DatabaseDescriptor.java | 7 +++
 src/java/org/apache/cassandra/thrift/ThriftServer.java   | 6 +++---
 3 files changed, 12 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/209ea6e9/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/209ea6e9/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --cc src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index 00e875b,34bd39a..a448463
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@@ -66,6 -66,6 +66,7 @@@ import org.apache.cassandra.net.Messagi
  import org.apache.cassandra.scheduler.IRequestScheduler;
  import org.apache.cassandra.scheduler.NoScheduler;
  import org.apache.cassandra.service.CacheService;
++import org.apache.cassandra.thrift.ThriftServer;
  import org.apache.cassandra.utils.ByteBufferUtil;
  import org.apache.cassandra.utils.FBUtilities;
  import org.apache.cassandra.utils.memory.HeapPool;

[jira] [Commented] (CASSANDRA-7607) Test coverage for authorization in DDL DML statements


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183056#comment-14183056
 ] 

Michael Shuler commented on CASSANDRA-7607:
---

[~enigmacurry], [~rhatch], [~philipthompson] - any thoughts on how to include 
triggers in dtest?

 Test coverage for authorization in DDL  DML statements
 ---

 Key: CASSANDRA-7607
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7607
 Project: Cassandra
  Issue Type: Test
  Components: Tests
Reporter: Robert Stupp
Assignee: Rajanarayanan Thottuvaikkatumana
  Labels: lhf, unit-test
 Fix For: 2.0.12


 Similar to CASSANDRA-7604
 Check that the statements perform proper authorization (allow / reject):
 * {{CREATE KEYSPACE}}
 * {{ALTER KEYSPACE}}
 * {{DROP KEYSPACE}}
 * {{CREATE TABLE}}
 * {{ALTER TABLE}}
 * {{DROP TABLE}}
 * {{CREATE TYPE}}
 * {{ALTER TYPE}}
 * {{DROP TYPE}}
 * {{CREATE INDEX}}
 * {{DROP INDEX}}
 * {{CREATE TRIGGER}}
 * {{DROP TRIGGER}}
 * {{CREATE USER}}
 * {{ALTER USER}}
 * {{DROP USER}}
 * {{TRUNCATE}}
 * {{GRANT}}
 * {{REVOKE}}
 * {{SELECT}}
 * {{UPDATE}}
 * {{DELETE}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8177) sequential repair is much more expensive than parallel repair

[
https://issues.apache.org/jira/browse/CASSANDRA-8177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183058#comment-14183058
]

Yuki Morishita commented on CASSANDRA-8177:
---

Sequential repair first asks every replica to take snapshot, and then ask each
replica to calculate Merkle Tree on snapshot one by one. Parallel repair
There is no difference in I/O operation.

Calculating Merkle Tree ('validation') and streaming SSTables are the two most
time consuming part of repair.
Sequential repair does validation one node at a time, so it takes time compared
to parallel.

Which is 'write' I/O line in the graph? Is it possible that the node is
streaming SSTables each other for sequential repair, and there are much less
out of sync SSTables in parallel repair?

sequential repair is much more expensive than parallel repair
-

Key: CASSANDRA-8177
URL: https://issues.apache.org/jira/browse/CASSANDRA-8177
Project: Cassandra
Issue Type: Bug
Reporter: Sean Bridges
Assignee: Yuki Morishita
Attachments: iostats.png

This is with 2.0.10
The attached graph shows io read/write throughput (as measured with iostat)
when doing repairs.
The large hump on the left is a sequential repair of one node. The two much
smaller peaks on the right are parallel repairs.
This is a 3 node cluster using vnodes (I know vnodes on small clusters isn't
recommended). Cassandra reports load of 40 gigs.
We noticed a similar problem with a larger cluster.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7607) Test coverage for authorization in DDL DML statements

2014-10-24 Thread Philip Thompson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183062#comment-14183062
 ] 

Philip Thompson commented on CASSANDRA-7607:


The docs are wrong, the trigger code has to go in conf/triggers. Are you 
putting the triggers in the conf/triggers of your cassandra install or of the 
ccm nodes? The latter should work and I can confirm it works in at least some 
cases. There is no easy ccm command to do so though.

 Test coverage for authorization in DDL  DML statements
 ---

 Key: CASSANDRA-7607
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7607
 Project: Cassandra
  Issue Type: Test
  Components: Tests
Reporter: Robert Stupp
Assignee: Rajanarayanan Thottuvaikkatumana
  Labels: lhf, unit-test
 Fix For: 2.0.12


 Similar to CASSANDRA-7604
 Check that the statements perform proper authorization (allow / reject):
 * {{CREATE KEYSPACE}}
 * {{ALTER KEYSPACE}}
 * {{DROP KEYSPACE}}
 * {{CREATE TABLE}}
 * {{ALTER TABLE}}
 * {{DROP TABLE}}
 * {{CREATE TYPE}}
 * {{ALTER TYPE}}
 * {{DROP TYPE}}
 * {{CREATE INDEX}}
 * {{DROP INDEX}}
 * {{CREATE TRIGGER}}
 * {{DROP TRIGGER}}
 * {{CREATE USER}}
 * {{ALTER USER}}
 * {{DROP USER}}
 * {{TRUNCATE}}
 * {{GRANT}}
 * {{REVOKE}}
 * {{SELECT}}
 * {{UPDATE}}
 * {{DELETE}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-8185) Removing the last item from a collection removes the entire row


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler resolved CASSANDRA-8185.
---
Resolution: Not a Problem

 Removing the last item from a collection removes the entire row
 ---

 Key: CASSANDRA-8185
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8185
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Mac OS X Yosemite
 JDK 8u25
Reporter: Stuart Gunter
Priority: Minor

 When removing the last item from a list or set (and I assume the same applies 
 to maps), the entire row is deleted rather than just that last item.
 I've only tested this in cases where the collection column is the only non-PK 
 column, but it is definitely a change between v2.0.5 and v2.0.6. I've looked 
 through the 2.0.6 release notes and issues and can't find any item that might 
 describe this, so I assume it's not an intended change. If such an issue 
 exists, I've probably just missed it, so apologies in advance.
 I've created a very simple project to reproduce the issue. If you clone the 
 repo and run it in two different modes (the README explains how to do this 
 with preconfigured Maven profiles), you will see the tests that pass on 
 v2.0.5 and fail on v2.0.6.
 Repo: https://github.com/stuartgunter/cassandra-2.0.6-bug



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8182) Comparison method violates its general contract in IndexSummaryManager.redistributeSummaries


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-8182:
---
Description: 
I found this exception while going over the logs of my new 2.1.0 production 
candidate cluster:

{noformat}
ERROR [IndexSummaryManager:1] 2014-10-23 14:37:56,727 CassandraDaemon.java:166 
- Exception in thread Thread[IndexSummaryManager:1,1,main]
java.lang.IllegalArgumentException: Comparison method violates its general 
contract!
at java.util.TimSort.mergeLo(TimSort.java:747) ~[na:1.7.0_67]
at java.util.TimSort.mergeAt(TimSort.java:483) ~[na:1.7.0_67]
at java.util.TimSort.mergeCollapse(TimSort.java:408) ~[na:1.7.0_67]
at java.util.TimSort.sort(TimSort.java:214) ~[na:1.7.0_67]
at java.util.TimSort.sort(TimSort.java:173) ~[na:1.7.0_67]
at java.util.Arrays.sort(Arrays.java:659) ~[na:1.7.0_67]
at java.util.Collections.sort(Collections.java:217) ~[na:1.7.0_67]
at 
org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:277)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:238)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow(IndexSummaryManager.java:139)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:75)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
[na:1.7.0_67]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) 
[na:1.7.0_67]
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
 [na:1.7.0_67]
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
 [na:1.7.0_67]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
[na:1.7.0_67]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
[na:1.7.0_67]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
{noformat}

There's some other cases of invalid comparison methods in the past, but not 
from this case.

  was:
I found this exception while going over the logs of my new 2.1.0 production 
candidate cluster:

ERROR [IndexSummaryManager:1] 2014-10-23 14:37:56,727 CassandraDaemon.java:166 
- Exception in thread Thread[IndexSummaryManager:1,1,main]
java.lang.IllegalArgumentException: Comparison method violates its general 
contract!
at java.util.TimSort.mergeLo(TimSort.java:747) ~[na:1.7.0_67]
at java.util.TimSort.mergeAt(TimSort.java:483) ~[na:1.7.0_67]
at java.util.TimSort.mergeCollapse(TimSort.java:408) ~[na:1.7.0_67]
at java.util.TimSort.sort(TimSort.java:214) ~[na:1.7.0_67]
at java.util.TimSort.sort(TimSort.java:173) ~[na:1.7.0_67]
at java.util.Arrays.sort(Arrays.java:659) ~[na:1.7.0_67]
at java.util.Collections.sort(Collections.java:217) ~[na:1.7.0_67]
at 
org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:277)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:238)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow(IndexSummaryManager.java:139)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
~[apache-cassandra-2.1.0.jar:2.1.0]
at 
org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:75)
 ~[apache-cassandra-2.1.0.jar:2.1.0]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
[na:1.7.0_67]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) 
[na:1.7.0_67]
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
 [na:1.7.0_67]
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
 [na:1.7.0_67]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
[na:1.7.0_67]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
[na:1.7.0_67]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]

[jira] [Commented] (CASSANDRA-7607) Test coverage for authorization in DDL DML statements

2014-10-24 Thread Ryan McGuire (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183077#comment-14183077
 ] 

Ryan McGuire commented on CASSANDRA-7607:
-

I tried it by specifying -Dcassandra.triggers_dir=/path/to/my/triggers/dir 

In the cassandra log I get:

{code}
INFO  [SharedPool-Worker-1] 2014-10-24 13:25:59,608 CustomClassLoader.java:87 - 
Loading new jar 
/home/ryan/git/datastax/cassandra/lib/triggers/trigger-example.jar
{code}

But I still get the same error about it not existing when I try and create it :(

 Test coverage for authorization in DDL  DML statements
 ---

 Key: CASSANDRA-7607
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7607
 Project: Cassandra
  Issue Type: Test
  Components: Tests
Reporter: Robert Stupp
Assignee: Rajanarayanan Thottuvaikkatumana
  Labels: lhf, unit-test
 Fix For: 2.0.12


 Similar to CASSANDRA-7604
 Check that the statements perform proper authorization (allow / reject):
 * {{CREATE KEYSPACE}}
 * {{ALTER KEYSPACE}}
 * {{DROP KEYSPACE}}
 * {{CREATE TABLE}}
 * {{ALTER TABLE}}
 * {{DROP TABLE}}
 * {{CREATE TYPE}}
 * {{ALTER TYPE}}
 * {{DROP TYPE}}
 * {{CREATE INDEX}}
 * {{DROP INDEX}}
 * {{CREATE TRIGGER}}
 * {{DROP TRIGGER}}
 * {{CREATE USER}}
 * {{ALTER USER}}
 * {{DROP USER}}
 * {{TRUNCATE}}
 * {{GRANT}}
 * {{REVOKE}}
 * {{SELECT}}
 * {{UPDATE}}
 * {{DELETE}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8116) HSHA fails with default rpc_max_threads setting


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183080#comment-14183080
 ] 

Tyler Hobbs commented on CASSANDRA-8116:


bq. is there any reason to have a default of unlimited? it just seems like it 
would be a simple change to have a reasonable default.

I'm not sure about the rationale for that default.  Would you mind opening a 
new ticket to discuss a better default?  I'm not sure if it's something we 
would want to change in 2.0 or 2.1.

 HSHA fails with default rpc_max_threads setting
 ---

 Key: CASSANDRA-8116
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8116
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Mike Adamson
Assignee: Tyler Hobbs
Priority: Minor
 Fix For: 2.0.12, 2.1.2

 Attachments: 8116-throw-exc-2.0.txt, 8116.txt


 The HSHA server fails with 'Out of heap space' error if the rpc_max_threads 
 is left at its default setting (unlimited) in cassandra.yaml.
 I'm not proposing any code change for this but have submitted a patch for a 
 comment change in cassandra.yaml to indicate that rpc_max_threads needs to be 
 changed if you use HSHA.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

git commit: Add 'die' policy for commit log and disk failure

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 860cde74a - d4df271f7


Add 'die' policy for commit log and disk failure

patch by John Sumsion and Josh McKenzie, reviewed by blerer for CASSANDRA-7927


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d4df271f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d4df271f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d4df271f

Branch: refs/heads/cassandra-2.1
Commit: d4df271f754ac4d5afd785d4043ccaaeb907a6ad
Parents: 860cde7
Author: Joshua McKenzie jmcken...@apache.org
Authored: Fri Oct 24 12:27:20 2014 -0500
Committer: Joshua McKenzie jmcken...@apache.org
Committed: Fri Oct 24 12:27:20 2014 -0500

--
 CHANGES.txt |  1 +
 conf/cassandra.yaml |  5 +-
 .../org/apache/cassandra/config/Config.java |  2 +
 .../cassandra/db/commitlog/CommitLog.java   | 12 -
 .../org/apache/cassandra/io/util/FileUtils.java | 10 +++-
 .../cassandra/utils/JVMStabilityInspector.java  | 50 ++-
 .../org/apache/cassandra/db/CommitLogTest.java  | 45 +++--
 .../utils/JVMStabilityInspectorTest.java| 51 
 .../apache/cassandra/utils/KillerForTests.java  | 43 +
 9 files changed, 197 insertions(+), 22 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d4df271f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index c136c5e..4ed07a9 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.2
+ * Add 'die' policy for commit log and disk failure (CASSANDRA-7927)
  * Fix installing as service on Windows (CASSANDRA-8115)
  * Fix CREATE TABLE for CQL2 (CASSANDRA-8144)
  * Avoid boxing in ColumnStats min/max trackers (CASSANDRA-8109)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d4df271f/conf/cassandra.yaml
--
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index c95c68c..74a8cfb 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -104,6 +104,8 @@ partitioner: org.apache.cassandra.dht.Murmur3Partitioner
 # commitlog_directory: /var/lib/cassandra/commitlog
 
 # policy for data disk failures:
+# die: shut down gossip and Thrift and kill the JVM for any fs errors or
+#  single-sstable errors, so the node can be replaced.
 # stop_paranoid: shut down gossip and Thrift even for single-sstable errors.
 # stop: shut down gossip and Thrift, leaving the node effectively dead, but
 #   can still be inspected via JMX.
@@ -114,9 +116,10 @@ partitioner: org.apache.cassandra.dht.Murmur3Partitioner
 disk_failure_policy: stop
 
 # policy for commit disk failures:
+# die: shut down gossip and Thrift and kill the JVM, so the node can be 
replaced.
 # stop: shut down gossip and Thrift, leaving the node effectively dead, but
 #   can still be inspected via JMX.
-# stop_commit: shutdown the commit log, letting writes collect but 
+# stop_commit: shutdown the commit log, letting writes collect but
 #  continuing to service reads, as in pre-2.0.5 Cassandra
 # ignore: ignore fatal errors and let the batches fail
 commit_failure_policy: stop

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d4df271f/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index e2df89f..5f16239 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -299,6 +299,7 @@ public class Config
 stop,
 ignore,
 stop_paranoid,
+die
 }
 
 public static enum CommitFailurePolicy
@@ -306,6 +307,7 @@ public class Config
 stop,
 stop_commit,
 ignore,
+die,
 }
 
 public static enum RequestSchedulerId

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d4df271f/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
--
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLog.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
index d38c4ed..ee9ca14 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
@@ -25,9 +25,12 @@ import java.util.*;
 import javax.management.MBeanServer;
 import javax.management.ObjectName;
 
-import org.apache.commons.lang3.StringUtils;
+import com.google.common.annotations.VisibleForTesting;
 import org.slf4j.Logger;
 import

[1/2] git commit: Add 'die' policy for commit log and disk failure

Repository: cassandra
Updated Branches:
  refs/heads/trunk 209ea6e9b - e39dc56ed


Add 'die' policy for commit log and disk failure

patch by John Sumsion and Josh McKenzie, reviewed by blerer for CASSANDRA-7927


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d4df271f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d4df271f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d4df271f

Branch: refs/heads/trunk
Commit: d4df271f754ac4d5afd785d4043ccaaeb907a6ad
Parents: 860cde7
Author: Joshua McKenzie jmcken...@apache.org
Authored: Fri Oct 24 12:27:20 2014 -0500
Committer: Joshua McKenzie jmcken...@apache.org
Committed: Fri Oct 24 12:27:20 2014 -0500

--
 CHANGES.txt |  1 +
 conf/cassandra.yaml |  5 +-
 .../org/apache/cassandra/config/Config.java |  2 +
 .../cassandra/db/commitlog/CommitLog.java   | 12 -
 .../org/apache/cassandra/io/util/FileUtils.java | 10 +++-
 .../cassandra/utils/JVMStabilityInspector.java  | 50 ++-
 .../org/apache/cassandra/db/CommitLogTest.java  | 45 +++--
 .../utils/JVMStabilityInspectorTest.java| 51 
 .../apache/cassandra/utils/KillerForTests.java  | 43 +
 9 files changed, 197 insertions(+), 22 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d4df271f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index c136c5e..4ed07a9 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.2
+ * Add 'die' policy for commit log and disk failure (CASSANDRA-7927)
  * Fix installing as service on Windows (CASSANDRA-8115)
  * Fix CREATE TABLE for CQL2 (CASSANDRA-8144)
  * Avoid boxing in ColumnStats min/max trackers (CASSANDRA-8109)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d4df271f/conf/cassandra.yaml
--
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index c95c68c..74a8cfb 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -104,6 +104,8 @@ partitioner: org.apache.cassandra.dht.Murmur3Partitioner
 # commitlog_directory: /var/lib/cassandra/commitlog
 
 # policy for data disk failures:
+# die: shut down gossip and Thrift and kill the JVM for any fs errors or
+#  single-sstable errors, so the node can be replaced.
 # stop_paranoid: shut down gossip and Thrift even for single-sstable errors.
 # stop: shut down gossip and Thrift, leaving the node effectively dead, but
 #   can still be inspected via JMX.
@@ -114,9 +116,10 @@ partitioner: org.apache.cassandra.dht.Murmur3Partitioner
 disk_failure_policy: stop
 
 # policy for commit disk failures:
+# die: shut down gossip and Thrift and kill the JVM, so the node can be 
replaced.
 # stop: shut down gossip and Thrift, leaving the node effectively dead, but
 #   can still be inspected via JMX.
-# stop_commit: shutdown the commit log, letting writes collect but 
+# stop_commit: shutdown the commit log, letting writes collect but
 #  continuing to service reads, as in pre-2.0.5 Cassandra
 # ignore: ignore fatal errors and let the batches fail
 commit_failure_policy: stop

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d4df271f/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index e2df89f..5f16239 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -299,6 +299,7 @@ public class Config
 stop,
 ignore,
 stop_paranoid,
+die
 }
 
 public static enum CommitFailurePolicy
@@ -306,6 +307,7 @@ public class Config
 stop,
 stop_commit,
 ignore,
+die,
 }
 
 public static enum RequestSchedulerId

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d4df271f/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
--
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLog.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
index d38c4ed..ee9ca14 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
@@ -25,9 +25,12 @@ import java.util.*;
 import javax.management.MBeanServer;
 import javax.management.ObjectName;
 
-import org.apache.commons.lang3.StringUtils;
+import com.google.common.annotations.VisibleForTesting;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
+

[2/2] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk

Conflicts:
test/unit/org/apache/cassandra/db/CommitLogTest.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e39dc56e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e39dc56e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e39dc56e

Branch: refs/heads/trunk
Commit: e39dc56eda3e325d537da44def54d85749def9d2
Parents: 209ea6e d4df271
Author: Joshua McKenzie jmcken...@apache.org
Authored: Fri Oct 24 12:31:51 2014 -0500
Committer: Joshua McKenzie jmcken...@apache.org
Committed: Fri Oct 24 12:31:51 2014 -0500

--
 CHANGES.txt |  1 +
 conf/cassandra.yaml |  5 +-
 .../org/apache/cassandra/config/Config.java |  2 +
 .../cassandra/db/commitlog/CommitLog.java   | 12 -
 .../org/apache/cassandra/io/util/FileUtils.java | 10 +++-
 .../cassandra/utils/JVMStabilityInspector.java  | 50 ++-
 .../org/apache/cassandra/db/CommitLogTest.java  | 45 +++--
 .../utils/JVMStabilityInspectorTest.java| 51 
 .../apache/cassandra/utils/KillerForTests.java  | 43 +
 9 files changed, 197 insertions(+), 22 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e39dc56e/CHANGES.txt
--
diff --cc CHANGES.txt
index 3f16e2e,4ed07a9..8821110
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,36 -1,5 +1,37 @@@
 +3.0
 + * Extend Descriptor to include a format value and refactor reader/writer 
apis (CASSANDRA-7443)
 + * Integrate JMH for microbenchmarks (CASSANDRA-8151)
 + * Keep sstable levels when bootstrapping (CASSANDRA-7460)
 + * Add Sigar library and perform basic OS settings check on startup 
(CASSANDRA-7838)
 + * Support for aggregation functions (CASSANDRA-4914)
 + * Remove cassandra-cli (CASSANDRA-7920)
 + * Accept dollar quoted strings in CQL (CASSANDRA-7769)
 + * Make assassinate a first class command (CASSANDRA-7935)
 + * Support IN clause on any clustering column (CASSANDRA-4762)
 + * Improve compaction logging (CASSANDRA-7818)
 + * Remove YamlFileNetworkTopologySnitch (CASSANDRA-7917)
 + * Do anticompaction in groups (CASSANDRA-6851)
 + * Support pure user-defined functions (CASSANDRA-7395, 7526, 7562, 7740, 
7781, 7929,
 +   7924, 7812, 8063)
 + * Permit configurable timestamps with cassandra-stress (CASSANDRA-7416)
 + * Move sstable RandomAccessReader to nio2, which allows using the
 +   FILE_SHARE_DELETE flag on Windows (CASSANDRA-4050)
 + * Remove CQL2 (CASSANDRA-5918)
 + * Add Thrift get_multi_slice call (CASSANDRA-6757)
 + * Optimize fetching multiple cells by name (CASSANDRA-6933)
 + * Allow compilation in java 8 (CASSANDRA-7028)
 + * Make incremental repair default (CASSANDRA-7250)
 + * Enable code coverage thru JaCoCo (CASSANDRA-7226)
 + * Switch external naming of 'column families' to 'tables' (CASSANDRA-4369) 
 + * Shorten SSTable path (CASSANDRA-6962)
 + * Use unsafe mutations for most unit tests (CASSANDRA-6969)
 + * Fix race condition during calculation of pending ranges (CASSANDRA-7390)
 + * Fail on very large batch sizes (CASSANDRA-8011)
 + * improve concurrency of repair (CASSANDRA-6455)
 +
 +
  2.1.2
+  * Add 'die' policy for commit log and disk failure (CASSANDRA-7927)
   * Fix installing as service on Windows (CASSANDRA-8115)
   * Fix CREATE TABLE for CQL2 (CASSANDRA-8144)
   * Avoid boxing in ColumnStats min/max trackers (CASSANDRA-8109)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e39dc56e/conf/cassandra.yaml
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e39dc56e/src/java/org/apache/cassandra/config/Config.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e39dc56e/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e39dc56e/src/java/org/apache/cassandra/io/util/FileUtils.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e39dc56e/test/unit/org/apache/cassandra/db/CommitLogTest.java
--
diff --cc test/unit/org/apache/cassandra/db/CommitLogTest.java
index 1383d78,8a1bb0c..987d744
--- a/test/unit/org/apache/cassandra/db/CommitLogTest.java
+++ b/test/unit/org/apache/cassandra/db/CommitLogTest.java
@@@ -26,10 -26,7 +26,8 @@@ import java.util.concurrent.ExecutionEx
  import java.util.zip.CRC32;
  import java.util.zip.Checksum;
  
- import

[jira] [Commented] (CASSANDRA-7713) CommitLogTest failure causes cascading unit test failures

2014-10-24 Thread Joshua McKenzie (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183104#comment-14183104
 ] 

Joshua McKenzie commented on CASSANDRA-7713:


CASSANDRA-7927 is 2.1.2+ so we might want to backport the unit-test revision 
logic into 2.0.12 (make handleCommitError public / VisibleForTesting, pass 
exception to it rather than doing read-only directory approach).

 CommitLogTest failure causes cascading unit test failures
 -

 Key: CASSANDRA-7713
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7713
 Project: Cassandra
  Issue Type: Test
Reporter: Michael Shuler
Assignee: Bogdan Kanivets
 Fix For: 2.0.12

 Attachments: CommitLogTest.system.log.txt


 When CommitLogTest.testCommitFailurePolicy_stop fails or times out, 
 {{commitDir.setWritable(true)}} is never reached, so the 
 build/test/cassandra/commitlog directory is left without write permissions, 
 causing cascading failure of all subsequent tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

svn commit: r1634109 - in /cassandra/site: publish/download/index.html src/settings.py

Author: slebresne
Date: Fri Oct 24 18:03:52 2014
New Revision: 1634109

URL: http://svn.apache.org/r1634109
Log:
Update website for 2.0.11

Modified:
cassandra/site/publish/download/index.html
cassandra/site/src/settings.py

Modified: cassandra/site/publish/download/index.html
URL: 
http://svn.apache.org/viewvc/cassandra/site/publish/download/index.html?rev=1634109r1=1634108r2=1634109view=diff
==
--- cassandra/site/publish/download/index.html (original)
+++ cassandra/site/publish/download/index.html Fri Oct 24 18:03:52 2014
@@ -102,16 +102,16 @@
   p
   Previous stable branches of Cassandra continue to see periodic maintenance
   for some time after a new major release is made. The lastest release on the
-  2.0 branch is 2.0.10 (released on
-  2014-08-25).
+  2.0 branch is 2.0.11 (released on
+  2014-09-24).
   /p
 
   ul
 li
-a class=filename 
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/2.0.10/apache-cassandra-2.0.10-bin.tar.gz;apache-cassandra-2.0.10-bin.tar.gz/a
-[a 
href=http://www.apache.org/dist/cassandra/2.0.10/apache-cassandra-2.0.10-bin.tar.gz.asc;PGP/a]
-[a 
href=http://www.apache.org/dist/cassandra/2.0.10/apache-cassandra-2.0.10-bin.tar.gz.md5;MD5/a]
-[a 
href=http://www.apache.org/dist/cassandra/2.0.10/apache-cassandra-2.0.10-bin.tar.gz.sha1;SHA1/a]
+a class=filename 
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/2.0.11/apache-cassandra-2.0.11-bin.tar.gz;apache-cassandra-2.0.11-bin.tar.gz/a
+[a 
href=http://www.apache.org/dist/cassandra/2.0.11/apache-cassandra-2.0.11-bin.tar.gz.asc;PGP/a]
+[a 
href=http://www.apache.org/dist/cassandra/2.0.11/apache-cassandra-2.0.11-bin.tar.gz.md5;MD5/a]
+[a 
href=http://www.apache.org/dist/cassandra/2.0.11/apache-cassandra-2.0.11-bin.tar.gz.sha1;SHA1/a]
 /li
   /ul
   
@@ -154,10 +154,10 @@
 /li
   
 li
-a class=filename 
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/2.0.10/apache-cassandra-2.0.10-src.tar.gz;apache-cassandra-2.0.10-src.tar.gz/a
-[a 
href=http://www.apache.org/dist/cassandra/2.0.10/apache-cassandra-2.0.10-src.tar.gz.asc;PGP/a]
-[a 
href=http://www.apache.org/dist/cassandra/2.0.10/apache-cassandra-2.0.10-src.tar.gz.md5;MD5/a]
-[a 
href=http://www.apache.org/dist/cassandra/2.0.10/apache-cassandra-2.0.10-src.tar.gz.sha1;SHA1/a]
+a class=filename 
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/2.0.11/apache-cassandra-2.0.11-src.tar.gz;apache-cassandra-2.0.11-src.tar.gz/a
+[a 
href=http://www.apache.org/dist/cassandra/2.0.11/apache-cassandra-2.0.11-src.tar.gz.asc;PGP/a]
+[a 
href=http://www.apache.org/dist/cassandra/2.0.11/apache-cassandra-2.0.11-src.tar.gz.md5;MD5/a]
+[a 
href=http://www.apache.org/dist/cassandra/2.0.11/apache-cassandra-2.0.11-src.tar.gz.sha1;SHA1/a]
 /li
   
   

Modified: cassandra/site/src/settings.py
URL: 
http://svn.apache.org/viewvc/cassandra/site/src/settings.py?rev=1634109r1=1634108r2=1634109view=diff
==
--- cassandra/site/src/settings.py (original)
+++ cassandra/site/src/settings.py Fri Oct 24 18:03:52 2014
@@ -92,8 +92,8 @@ SITE_POST_PROCESSORS = {
 }
 
 class CassandraDef(object):
-oldstable_version = '2.0.10'
-oldstable_release_date = '2014-08-25'
+oldstable_version = '2.0.11'
+oldstable_release_date = '2014-09-24'
 oldstable_exists = True
 veryoldstable_version = '1.2.19'
 veryoldstable_release_date = '2014-09-18'

[jira] [Created] (CASSANDRA-8186) Revisit defaults for max rpc threads

2014-10-24 Thread Jeremy Hanna (JIRA)

Jeremy Hanna created CASSANDRA-8186:
---

 Summary: Revisit defaults for max rpc threads
 Key: CASSANDRA-8186
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8186
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jeremy Hanna


Currently the rpc max threads defaults to unlimited.  The dangers of doing that 
in certain circumstances were discussed on CASSANDRA-8116 and a warning was 
added to the yaml.

I've worked with two clusters in the last couple of weeks that has had the heap 
full of rpc threads for this reason.  I'm wondering if we should set a more 
reasonable default so that people don't hurt themselves unnecessarily.

Is there a reason to not default this to perhaps 2048, what's commented out in 
the yaml?  We could have a yaml comment about it as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8182) Comparison method violates its general contract in IndexSummaryManager.redistributeSummaries


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-8182:
---
Attachment: 8182.txt

8182.txt uses a map to avoid changing values during the sort.

 Comparison method violates its general contract in 
 IndexSummaryManager.redistributeSummaries
 

 Key: CASSANDRA-8182
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8182
 Project: Cassandra
  Issue Type: Bug
 Environment: 2.1.0
Reporter: Juho Mäkinen
Assignee: Tyler Hobbs
Priority: Minor
 Fix For: 2.1.2

 Attachments: 8182.txt


 I found this exception while going over the logs of my new 2.1.0 production 
 candidate cluster:
 {noformat}
 ERROR [IndexSummaryManager:1] 2014-10-23 14:37:56,727 
 CassandraDaemon.java:166 - Exception in thread 
 Thread[IndexSummaryManager:1,1,main]
 java.lang.IllegalArgumentException: Comparison method violates its general 
 contract!
 at java.util.TimSort.mergeLo(TimSort.java:747) ~[na:1.7.0_67]
 at java.util.TimSort.mergeAt(TimSort.java:483) ~[na:1.7.0_67]
 at java.util.TimSort.mergeCollapse(TimSort.java:408) ~[na:1.7.0_67]
 at java.util.TimSort.sort(TimSort.java:214) ~[na:1.7.0_67]
 at java.util.TimSort.sort(TimSort.java:173) ~[na:1.7.0_67]
 at java.util.Arrays.sort(Arrays.java:659) ~[na:1.7.0_67]
 at java.util.Collections.sort(Collections.java:217) ~[na:1.7.0_67]
 at 
 org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:277)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:238)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow(IndexSummaryManager.java:139)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:75)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
 [na:1.7.0_67]
 at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) 
 [na:1.7.0_67]
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
  [na:1.7.0_67]
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
  [na:1.7.0_67]
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  [na:1.7.0_67]
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  [na:1.7.0_67]
 at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
 {noformat}
 There's some other cases of invalid comparison methods in the past, but not 
 from this case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8182) Comparison method violates its general contract in IndexSummaryManager.redistributeSummaries

2014-10-24 Thread Rajanarayanan Thottuvaikkatumana (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-8182:
---
  Component/s: Core
Since Version: 2.1 beta1

 Comparison method violates its general contract in 
 IndexSummaryManager.redistributeSummaries
 

 Key: CASSANDRA-8182
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8182
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 2.1.0
Reporter: Juho Mäkinen
Assignee: Tyler Hobbs
Priority: Minor
 Fix For: 2.1.2

 Attachments: 8182.txt


 I found this exception while going over the logs of my new 2.1.0 production 
 candidate cluster:
 {noformat}
 ERROR [IndexSummaryManager:1] 2014-10-23 14:37:56,727 
 CassandraDaemon.java:166 - Exception in thread 
 Thread[IndexSummaryManager:1,1,main]
 java.lang.IllegalArgumentException: Comparison method violates its general 
 contract!
 at java.util.TimSort.mergeLo(TimSort.java:747) ~[na:1.7.0_67]
 at java.util.TimSort.mergeAt(TimSort.java:483) ~[na:1.7.0_67]
 at java.util.TimSort.mergeCollapse(TimSort.java:408) ~[na:1.7.0_67]
 at java.util.TimSort.sort(TimSort.java:214) ~[na:1.7.0_67]
 at java.util.TimSort.sort(TimSort.java:173) ~[na:1.7.0_67]
 at java.util.Arrays.sort(Arrays.java:659) ~[na:1.7.0_67]
 at java.util.Collections.sort(Collections.java:217) ~[na:1.7.0_67]
 at 
 org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:277)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:238)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow(IndexSummaryManager.java:139)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
 ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:75)
  ~[apache-cassandra-2.1.0.jar:2.1.0]
 at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
 [na:1.7.0_67]
 at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) 
 [na:1.7.0_67]
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
  [na:1.7.0_67]
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
  [na:1.7.0_67]
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  [na:1.7.0_67]
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  [na:1.7.0_67]
 at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
 {noformat}
 There's some other cases of invalid comparison methods in the past, but not 
 from this case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7607) Test coverage for authorization in DDL DML statements


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183279#comment-14183279
 ] 

Rajanarayanan Thottuvaikkatumana commented on CASSANDRA-7607:
-

Philip, I tried to make the trigger work in the non-ccm setup. Actually these 
cases that I am trying to automate are supposed to work in a single node setup. 
That is why I started trying that way. 

 Test coverage for authorization in DDL  DML statements
 ---

 Key: CASSANDRA-7607
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7607
 Project: Cassandra
  Issue Type: Test
  Components: Tests
Reporter: Robert Stupp
Assignee: Rajanarayanan Thottuvaikkatumana
  Labels: lhf, unit-test
 Fix For: 2.0.12


 Similar to CASSANDRA-7604
 Check that the statements perform proper authorization (allow / reject):
 * {{CREATE KEYSPACE}}
 * {{ALTER KEYSPACE}}
 * {{DROP KEYSPACE}}
 * {{CREATE TABLE}}
 * {{ALTER TABLE}}
 * {{DROP TABLE}}
 * {{CREATE TYPE}}
 * {{ALTER TYPE}}
 * {{DROP TYPE}}
 * {{CREATE INDEX}}
 * {{DROP INDEX}}
 * {{CREATE TRIGGER}}
 * {{DROP TRIGGER}}
 * {{CREATE USER}}
 * {{ALTER USER}}
 * {{DROP USER}}
 * {{TRUNCATE}}
 * {{GRANT}}
 * {{REVOKE}}
 * {{SELECT}}
 * {{UPDATE}}
 * {{DELETE}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-8187) Create long-running Test Suite

2014-10-24 Thread Philip Thompson (JIRA)

Philip Thompson created CASSANDRA-8187:
--

 Summary: Create long-running Test Suite
 Key: CASSANDRA-8187
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8187
 Project: Cassandra
  Issue Type: Test
  Components: Tests
Reporter: Philip Thompson


We need to start running tests that run for at least several hours. Our current 
dtest suite is inadequate at catching data loss bugs and compaction problems.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7463) Update CQLSSTableWriter to allow parallel writing of SSTables on the same table within the same JVM

2014-10-24 Thread Rajanarayanan Thottuvaikkatumana (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183308#comment-14183308
 ] 

Benjamin Lerer commented on CASSANDRA-7463:
---

Sorry guys, I did not realize that Carl had uploaded a patch and was waiting 
for a review. I got confused with the comments.
I will review the patch as soon as possible.

 Update CQLSSTableWriter to allow parallel writing of SSTables on the same 
 table within the same JVM
 ---

 Key: CASSANDRA-7463
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7463
 Project: Cassandra
  Issue Type: Improvement
Reporter: Johnny Miller
Assignee: Carl Yeksigian
 Fix For: 2.0.12

 Attachments: 7463-v2.txt, 7463.patch


 Currently it is not possible to programatically write multiple SSTables for 
 the same table in parallel using the CQLSSTableWriter. This is quite a 
 limitation and the workaround of attempting to do this in a separate JVM is 
 not a great solution.
 See: 
 http://stackoverflow.com/questions/24396902/using-cqlsstablewriter-concurrently



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8177) sequential repair is much more expensive than parallel repair

2014-10-24 Thread Janne Jalkanen (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Janne Jalkanen updated CASSANDRA-8177:
--
Attachment: cassc-week.png

 sequential repair is much more expensive than parallel repair
 -

 Key: CASSANDRA-8177
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8177
 Project: Cassandra
  Issue Type: Bug
Reporter: Sean Bridges
Assignee: Yuki Morishita
 Attachments: cassc-week.png, iostats.png


 This is with 2.0.10
 The attached graph shows io read/write throughput (as measured with iostat) 
 when doing repairs.
 The large hump on the left is a sequential repair of one node.  The two much 
 smaller peaks on the right are parallel repairs.
 This is a 3 node cluster using vnodes (I know vnodes on small clusters isn't 
 recommended).  Cassandra reports load of 40 gigs.
 We noticed a similar problem with a larger cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7607) Test coverage for authorization in DDL DML statements


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajanarayanan Thottuvaikkatumana updated CASSANDRA-7607:

Attachment: cassandra-dtest-7607.txt

This is the patch containing the tests for CREATE/ALTER/DROP TYPE and 
CREATE/DROP FUNCTION. The branch of the code where the changes are done is 
https://github.com/riptano/cassandra-dtest . 

 Test coverage for authorization in DDL  DML statements
 ---

 Key: CASSANDRA-7607
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7607
 Project: Cassandra
  Issue Type: Test
  Components: Tests
Reporter: Robert Stupp
Assignee: Rajanarayanan Thottuvaikkatumana
  Labels: lhf, unit-test
 Fix For: 2.0.12

 Attachments: cassandra-dtest-7607.txt


 Similar to CASSANDRA-7604
 Check that the statements perform proper authorization (allow / reject):
 * {{CREATE KEYSPACE}}
 * {{ALTER KEYSPACE}}
 * {{DROP KEYSPACE}}
 * {{CREATE TABLE}}
 * {{ALTER TABLE}}
 * {{DROP TABLE}}
 * {{CREATE TYPE}}
 * {{ALTER TYPE}}
 * {{DROP TYPE}}
 * {{CREATE INDEX}}
 * {{DROP INDEX}}
 * {{CREATE TRIGGER}}
 * {{DROP TRIGGER}}
 * {{CREATE USER}}
 * {{ALTER USER}}
 * {{DROP USER}}
 * {{TRUNCATE}}
 * {{GRANT}}
 * {{REVOKE}}
 * {{SELECT}}
 * {{UPDATE}}
 * {{DELETE}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8177) sequential repair is much more expensive than parallel repair

2014-10-24 Thread Janne Jalkanen (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183362#comment-14183362
 ] 

Janne Jalkanen commented on CASSANDRA-8177:
---

Kinda having the same problem - attaching the compactions graph from Munin.  
Started a serial compaction on 21st midnight, took something like 45 hrs.

On 23rd and 24th we ran a repair with -par.  It's a lot shorter and causes a 
lot less compaction traffic.

This is with 2.0.10 from a production cluster with 9:1 read/write ratio, 
recently upgraded from 1.2.18.


 sequential repair is much more expensive than parallel repair
 -

 Key: CASSANDRA-8177
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8177
 Project: Cassandra
  Issue Type: Bug
Reporter: Sean Bridges
Assignee: Yuki Morishita
 Attachments: cassc-week.png, iostats.png


 This is with 2.0.10
 The attached graph shows io read/write throughput (as measured with iostat) 
 when doing repairs.
 The large hump on the left is a sequential repair of one node.  The two much 
 smaller peaks on the right are parallel repairs.
 This is a 3 node cluster using vnodes (I know vnodes on small clusters isn't 
 recommended).  Cassandra reports load of 40 gigs.
 We noticed a similar problem with a larger cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8147) Secondary indexing of map keys does not work properly when mixing contains and contains_key


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-8147:
--
Attachment: CASSANDRA-8147-V2.txt

Patch resulting from the merge of the V1 patch and of the one of CASSANDRA-8155

 Secondary indexing of map keys does not work properly when mixing contains 
 and contains_key
 ---

 Key: CASSANDRA-8147
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8147
 Project: Cassandra
  Issue Type: Bug
Reporter: Benjamin Lerer
Assignee: Benjamin Lerer
Priority: Minor
 Attachments: CASSANDRA-8147-V2.txt, CASSANDRA-8147.txt


 If you have a table with a map column and an index on the map key selecting 
 data using a contains key and a contains will not return the expected data.
 The problem can be reproduced using the following unit test:
 {code}
 @Test
 public void testMapKeyContainsAndValueContains() throws Throwable
 {
 createTable(CREATE TABLE %s (account text, id int, categories 
 maptext,text, PRIMARY KEY (account, id)));
 createIndex(CREATE INDEX ON %s(keys(categories)));
 execute(INSERT INTO %s (account, id , categories) VALUES (?, ?, ?), 
 test, 5, map(lmn, foo));
 assertRows(execute(SELECT * FROM %s WHERE account = ? AND id = ? AND 
 categories CONTAINS KEY ? AND categories CONTAINS ? ALLOW FILTERING, test, 
 5, lmn, foo), row(test, 5, map(lmn, foo)));  
 }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

git commit: Fix test timeout in CQLSSTableWriterTest

2014-10-24 Thread yukim

Repository: cassandra
Updated Branches:
  refs/heads/trunk e39dc56ed - 82e26b217


Fix test timeout in CQLSSTableWriterTest

Time out is caused by NPE when creating SSTable writer.
3.0 does not have ks and cf names in SSTable file name,
when writing SSTable, it has to be under ks/cf directory.


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/82e26b21
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/82e26b21
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/82e26b21

Branch: refs/heads/trunk
Commit: 82e26b217dab3c327bab882be8cc6b8c0c257139
Parents: e39dc56
Author: Yuki Morishita yu...@apache.org
Authored: Fri Oct 24 13:47:40 2014 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Fri Oct 24 15:11:46 2014 -0500

--
 .../cassandra/io/sstable/CQLSSTableWriterTest.java | 13 +
 1 file changed, 9 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/82e26b21/test/unit/org/apache/cassandra/io/sstable/CQLSSTableWriterTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/io/sstable/CQLSSTableWriterTest.java 
b/test/unit/org/apache/cassandra/io/sstable/CQLSSTableWriterTest.java
index 5a09586..250545a 100644
--- a/test/unit/org/apache/cassandra/io/sstable/CQLSSTableWriterTest.java
+++ b/test/unit/org/apache/cassandra/io/sstable/CQLSSTableWriterTest.java
@@ -70,7 +70,7 @@ public class CQLSSTableWriterTest
 CQLSSTableWriter writer = CQLSSTableWriter.builder()
   .inDirectory(dataDir)
   .forTable(schema)
-  
.withPartitioner(StorageService.instance.getPartitioner())
+  
.withPartitioner(StorageService.getPartitioner())
   .using(insert).build();
 
 writer.addRow(0, test1, 24);
@@ -129,16 +129,21 @@ public class CQLSSTableWriterTest
 // Check that the write respect the buffer size even if we only insert 
rows withing the same partition (#7360)
 // To do that simply, we use a writer with a buffer of 1MB, and write 
2 rows in the same partition with a value
 //  1MB and validate that this created more than 1 sstable.
+String KS = ks;
+String TABLE = test;
+
 File tempdir = Files.createTempDir();
+File dataDir = new File(tempdir.getAbsolutePath() + File.separator + 
KS + File.separator + TABLE);
+assert dataDir.mkdirs();
 String schema = CREATE TABLE ks.test (
   +   k int PRIMARY KEY,
   +   v blob
   + );
 String insert = INSERT INTO ks.test (k, v) VALUES (?, ?);
 CQLSSTableWriter writer = CQLSSTableWriter.builder()
-  .inDirectory(tempdir)
+  .inDirectory(dataDir)
   .forTable(schema)
-  
.withPartitioner(StorageService.instance.getPartitioner())
+  
.withPartitioner(StorageService.getPartitioner())
   .using(insert)
   .withBufferSizeInMB(1)
   .build();
@@ -156,6 +161,6 @@ public class CQLSSTableWriterTest
 return name.endsWith(-Data.db);
 }
 };
-assert tempdir.list(filterDataFiles).length  1 : 
Arrays.toString(tempdir.list(filterDataFiles));
+assert dataDir.list(filterDataFiles).length  1 : 
Arrays.toString(dataDir.list(filterDataFiles));
 }
 }

[jira] [Commented] (CASSANDRA-8177) sequential repair is much more expensive than parallel repair

2014-10-24 Thread sankalp kohli (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183413#comment-14183413
 ] 

sankalp kohli commented on CASSANDRA-8177:
--

One of the things could be somehow snapshots are being deleted which leads to 
different Merkle tree. We should log when we generate Merkle tree from 
snapshot. 
Also we should fail the repair if not all replicas are using the same snapshot. 
[~yukim] Should I create a JIRA to also confirm that all Merkle tree is coming 
from same snapshot created for same session id? 

 sequential repair is much more expensive than parallel repair
 -

 Key: CASSANDRA-8177
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8177
 Project: Cassandra
  Issue Type: Bug
Reporter: Sean Bridges
Assignee: Yuki Morishita
 Attachments: cassc-week.png, iostats.png


 This is with 2.0.10
 The attached graph shows io read/write throughput (as measured with iostat) 
 when doing repairs.
 The large hump on the left is a sequential repair of one node.  The two much 
 smaller peaks on the right are parallel repairs.
 This is a 3 node cluster using vnodes (I know vnodes on small clusters isn't 
 recommended).  Cassandra reports load of 40 gigs.
 We noticed a similar problem with a larger cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7510) Notify clients that bootstrap is finished over binary protocol

[
https://issues.apache.org/jira/browse/CASSANDRA-7510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183495#comment-14183495
]

Brandon Williams commented on CASSANDRA-7510:
-

Argh, gossip settling.

bq. I'm not sure that we can do anything about this without adding something
new to gossip, though.

I'm not sure we can either, and it'd be strange if we did, since we'd
essentially have two events that mean 'ready' unless we did something like
suppress the rpc_address state until it was bound, but that probably breaks
clients in some other way at this point.

Notify clients that bootstrap is finished over binary protocol
--

Key: CASSANDRA-7510
URL: https://issues.apache.org/jira/browse/CASSANDRA-7510
Project: Cassandra
Issue Type: Bug
Reporter: Joost Reuzel
Assignee: Brandon Williams
Priority: Minor
Fix For: 2.0.12

Attachments: 7510.txt

Currently, Cassandra will notify clients when a new node is added to a
cluster. However, that node is typically not usable yet. It first needs to
gossip its key range and finish loading all its assigned data before it
allows clients to connect. Depending on the amount of data this may take
quite a while. The clients in the mean time have no clue about the bootstrap
status of that node. The only thing they can do is periodically check if it
will accept a connection.
My proposal would be to send an additional UP event when the bootstrap is
done, this allows clients to mark the node initially as down/unavailable and
simply wait for the UP event to arrive.
Kind regards,
Joost

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7607) Test coverage for authorization in DDL DML statements

2014-10-24 Thread Ryan McGuire (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183546#comment-14183546
 ] 

Ryan McGuire commented on CASSANDRA-7607:
-

Thanks [~rnamboodiri], I've committed your patch to cassandra-dtest. In the 
future, you can push your commits to github and issue a pull request.

 Test coverage for authorization in DDL  DML statements
 ---

 Key: CASSANDRA-7607
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7607
 Project: Cassandra
  Issue Type: Test
  Components: Tests
Reporter: Robert Stupp
Assignee: Rajanarayanan Thottuvaikkatumana
  Labels: lhf, unit-test
 Fix For: 2.0.12

 Attachments: cassandra-dtest-7607.txt


 Similar to CASSANDRA-7604
 Check that the statements perform proper authorization (allow / reject):
 * {{CREATE KEYSPACE}}
 * {{ALTER KEYSPACE}}
 * {{DROP KEYSPACE}}
 * {{CREATE TABLE}}
 * {{ALTER TABLE}}
 * {{DROP TABLE}}
 * {{CREATE TYPE}}
 * {{ALTER TYPE}}
 * {{DROP TYPE}}
 * {{CREATE INDEX}}
 * {{DROP INDEX}}
 * {{CREATE TRIGGER}}
 * {{DROP TRIGGER}}
 * {{CREATE USER}}
 * {{ALTER USER}}
 * {{DROP USER}}
 * {{TRUNCATE}}
 * {{GRANT}}
 * {{REVOKE}}
 * {{SELECT}}
 * {{UPDATE}}
 * {{DELETE}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[Cassandra Wiki] Update of ContributorsGroup by BrandonWilliams

2014-10-24 Thread Apache Wiki

Dear Wiki user,

You have subscribed to a wiki page or wiki category on Cassandra Wiki for 
change notification.

The ContributorsGroup page has been changed by BrandonWilliams:
https://wiki.apache.org/cassandra/ContributorsGroup?action=diffrev1=39rev2=40

   * OtisGospodnetic
   * CarlYeksigian
   * JohnSumsion
+  * mriou

[Cassandra Wiki] Update of FAQ by mriou

2014-10-24 Thread Apache Wiki

Dear Wiki user,

You have subscribed to a wiki page or wiki category on Cassandra Wiki for 
change notification.

The FAQ page has been changed by mriou:
https://wiki.apache.org/cassandra/FAQ?action=diffrev1=166rev2=167

Comment:
Added recommended steps when running into bootstrapping issues during streaming.

   * [[#memlock|Cassandra dies with java.lang.OutOfMemoryError: Map failed]]
   * [[#opp|Why should I avoid order-preserving partitioners?]]
   * [[#clocktie|What happens if two updates are made with the same timestamp?]]
+  * [[#bootstreamfail|Why bootstrapping a new node fails with a Stream 
failed error?]]
  
  Anchor(cant_listen_on_ip_any)
  
@@ -450, +451 @@

  
  {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}}
  
+ Anchor(bootstreamfail)
+ == Why bootstrapping a new node fails with a Stream failed error? ==
+ 
+ Two main possibilities:
+   1. the GC may be creating long pauses disrupting the streaming process
+   2. compactions happening in the background hold streaming long enough that 
the TCP connection fails
+ 
+ In the first case, regular GC tuning advices apply. In the second case, you 
need to set TCP keepalive to a lower value (default is very high on Linux). Try 
to just run the following:
+ 
+ $ sudo /sbin/sysctl -w net.ipv4.tcp_keepalive_time=60 
net.ipv4.tcp_keepalive_intvl=60 net.ipv4.tcp_keepalive_probes=5
+ 
+ To make those settings permanent, add them to your /etc/sysctl.conf file.
+ 
+ Note: GCE's firewall will always interrupt TCP connections that are inactive 
for more than 10 min. Running the above command is highly recommended in that 
environment.
+ 
+ 
+ See [Partitioners].
+

[jira] [Commented] (CASSANDRA-8177) sequential repair is much more expensive than parallel repair

2014-10-24 Thread Karl Mueller (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183674#comment-14183674
 ] 

Karl Mueller commented on CASSANDRA-8177:
-

serial repairs are also terrible for us in 2.0.10

parallel is better

 sequential repair is much more expensive than parallel repair
 -

 Key: CASSANDRA-8177
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8177
 Project: Cassandra
  Issue Type: Bug
Reporter: Sean Bridges
Assignee: Yuki Morishita
 Attachments: cassc-week.png, iostats.png


 This is with 2.0.10
 The attached graph shows io read/write throughput (as measured with iostat) 
 when doing repairs.
 The large hump on the left is a sequential repair of one node.  The two much 
 smaller peaks on the right are parallel repairs.
 This is a 3 node cluster using vnodes (I know vnodes on small clusters isn't 
 recommended).  Cassandra reports load of 40 gigs.
 We noticed a similar problem with a larger cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-6602) Compaction improvements to optimize time series data


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-6602:

Fix Version/s: 2.1.1

 Compaction improvements to optimize time series data
 

 Key: CASSANDRA-6602
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6602
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Tupshin Harper
Assignee: Björn Hegerfors
  Labels: compaction, performance
 Fix For: 2.0.11, 2.1.1

 Attachments: 1 week.txt, 8 weeks.txt, STCS 16 hours.txt, 
 TimestampViewer.java, 
 cassandra-2.0-CASSANDRA-6602-DateTieredCompactionStrategy.txt, 
 cassandra-2.0-CASSANDRA-6602-DateTieredCompactionStrategy_v2.txt, 
 cassandra-2.0-CASSANDRA-6602-DateTieredCompactionStrategy_v3.txt


 There are some unique characteristics of many/most time series use cases that 
 both provide challenges, as well as provide unique opportunities for 
 optimizations.
 One of the major challenges is in compaction. The existing compaction 
 strategies will tend to re-compact data on disk at least a few times over the 
 lifespan of each data point, greatly increasing the cpu and IO costs of that 
 write.
 Compaction exists to
 1) ensure that there aren't too many files on disk
 2) ensure that data that should be contiguous (part of the same partition) is 
 laid out contiguously
 3) deleting data due to ttls or tombstones
 The special characteristics of time series data allow us to optimize away all 
 three.
 Time series data
 1) tends to be delivered in time order, with relatively constrained exceptions
 2) often has a pre-determined and fixed expiration date
 3) Never gets deleted prior to TTL
 4) Has relatively predictable ingestion rates
 Note that I filed CASSANDRA-5561 and this ticket potentially replaces or 
 lowers the need for it. In that ticket, jbellis reasonably asks, how that 
 compaction strategy is better than disabling compaction.
 Taking that to heart, here is a compaction-strategy-less approach that could 
 be extremely efficient for time-series use cases that follow the above 
 pattern.
 (For context, I'm thinking of an example use case involving lots of streams 
 of time-series data with a 5GB per day ingestion rate, and a 1000 day 
 retention with TTL, resulting in an eventual steady state of 5TB per node)
 1) You have an extremely large memtable (preferably off heap, if/when doable) 
 for the table, and that memtable is sized to be able to hold a lengthy window 
 of time. A typical period might be one day. At the end of that period, you 
 flush the contents of the memtable to an sstable and move to the next one. 
 This is basically identical to current behaviour, but with thresholds 
 adjusted so that you can ensure flushing at predictable intervals. (Open 
 question is whether predictable intervals is actually necessary, or whether 
 just waiting until the huge memtable is nearly full is sufficient)
 2) Combine the behaviour with CASSANDRA-5228 so that sstables will be 
 efficiently dropped once all of the columns have. (Another side note, it 
 might be valuable to have a modified version of CASSANDRA-3974 that doesn't 
 bother storing per-column TTL since it is required that all columns have the 
 same TTL)
 3) Be able to mark column families as read/write only (no explicit deletes), 
 so no tombstones.
 4) Optionally add back an additional type of delete that would delete all 
 data earlier than a particular timestamp, resulting in immediate dropping of 
 obsoleted sstables.
 The result is that for in-order delivered data, Every cell will be laid out 
 optimally on disk on the first pass, and over the course of 1000 days and 5TB 
 of data, there will only be 1000 5GB sstables, so the number of filehandles 
 will be reasonable.
 For exceptions (out-of-order delivery), most cases will be caught by the 
 extended (24 hour+) memtable flush times and merged correctly automatically. 
 For those that were slightly askew at flush time, or were delivered so far 
 out of order that they go in the wrong sstable, there is relatively low 
 overhead to reading from two sstables for a time slice, instead of one, and 
 that overhead would be incurred relatively rarely unless out-of-order 
 delivery was the common case, in which case, this strategy should not be used.
 Another possible optimization to address out-of-order would be to maintain 
 more than one time-centric memtables in memory at a time (e.g. two 12 hour 
 ones), and then you always insert into whichever one of the two owns the 
 appropriate range of time. By delaying flushing the ahead one until we are 
 ready to roll writes over to a third one, we are able to avoid any 
 fragmentation as long as all deliveries come in no more than 12 hours late 
 (in

[jira] [Commented] (CASSANDRA-8177) sequential repair is much more expensive than parallel repair

[
https://issues.apache.org/jira/browse/CASSANDRA-8177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183736#comment-14183736
]

Yuki Morishita commented on CASSANDRA-8177:
---

My guess for sequential repair generating lots of IO is that, when reading from
snapshot, it is hitting disk for each snapshot SSTable to read its bloom
filters, index files etc. On the other hand, parallel repair does not create
snapshot and reads already opened SSTables, some of them are from page cache.

Sequential repair is meant to be used where validation compaction on all
replica will impact on overall cluster performance. If parallel repair does the
job, then stick with it is fine.

[~kohlisankalp] You should create new JIRA, as I need more details.

sequential repair is much more expensive than parallel repair
-

Key: CASSANDRA-8177
URL: https://issues.apache.org/jira/browse/CASSANDRA-8177
Project: Cassandra
Issue Type: Bug
Reporter: Sean Bridges
Assignee: Yuki Morishita
Attachments: cassc-week.png, iostats.png

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8177) sequential repair is much more expensive than parallel repair

2014-10-24 Thread Karl Mueller (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183757#comment-14183757
 ] 

Karl Mueller commented on CASSANDRA-8177:
-

Sequential repair is meant to be used where validation compaction on all 
replica will impact on overall cluster performance. If parallel repair does the 
job, then stick with it is fine.

Why on earth is serial repair the default then??  parallel is a better default!

 sequential repair is much more expensive than parallel repair
 -

 Key: CASSANDRA-8177
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8177
 Project: Cassandra
  Issue Type: Bug
Reporter: Sean Bridges
Assignee: Yuki Morishita
 Attachments: cassc-week.png, iostats.png


 This is with 2.0.10
 The attached graph shows io read/write throughput (as measured with iostat) 
 when doing repairs.
 The large hump on the left is a sequential repair of one node.  The two much 
 smaller peaks on the right are parallel repairs.
 This is a 3 node cluster using vnodes (I know vnodes on small clusters isn't 
 recommended).  Cassandra reports load of 40 gigs.
 We noticed a similar problem with a larger cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-8177) sequential repair is much more expensive than parallel repair

2014-10-24 Thread Sean Bridges (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183783#comment-14183783
 ] 

Sean Bridges edited comment on CASSANDRA-8177 at 10/25/14 12:18 AM:


{quote}
My guess for sequential repair generating lots of IO is that, when reading from 
snapshot, it is hitting disk for each snapshot SSTable to read its bloom 
filters, index files etc
{quote}

When you snapshot you are hardlinking the old and original sstables, they are 
the same files, so the os cache shouldn't be the difference


was (Author: sgbridges):
{quote}
My guess for sequential repair generating lots of IO is that, when reading from 
snapshot, it is hitting disk for each snapshot SSTable to read its bloom 
filters, index files etc
{quote}

When you snapshot you are hardlinking the old and original sstables, they are 
the same file, so the os cache shouldn't be the difference

 sequential repair is much more expensive than parallel repair
 -

 Key: CASSANDRA-8177
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8177
 Project: Cassandra
  Issue Type: Bug
Reporter: Sean Bridges
Assignee: Yuki Morishita
 Attachments: cassc-week.png, iostats.png


 This is with 2.0.10
 The attached graph shows io read/write throughput (as measured with iostat) 
 when doing repairs.
 The large hump on the left is a sequential repair of one node.  The two much 
 smaller peaks on the right are parallel repairs.
 This is a 3 node cluster using vnodes (I know vnodes on small clusters isn't 
 recommended).  Cassandra reports load of 40 gigs.
 We noticed a similar problem with a larger cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8177) sequential repair is much more expensive than parallel repair

2014-10-24 Thread Sean Bridges (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183783#comment-14183783
 ] 

Sean Bridges commented on CASSANDRA-8177:
-

{quote}
My guess for sequential repair generating lots of IO is that, when reading from 
snapshot, it is hitting disk for each snapshot SSTable to read its bloom 
filters, index files etc
{quote}

When you snapshot you are hardlinking the old and original sstables, they are 
the same file, so the os cache shouldn't be the difference

 sequential repair is much more expensive than parallel repair
 -

 Key: CASSANDRA-8177
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8177
 Project: Cassandra
  Issue Type: Bug
Reporter: Sean Bridges
Assignee: Yuki Morishita
 Attachments: cassc-week.png, iostats.png


 This is with 2.0.10
 The attached graph shows io read/write throughput (as measured with iostat) 
 when doing repairs.
 The large hump on the left is a sequential repair of one node.  The two much 
 smaller peaks on the right are parallel repairs.
 This is a 3 node cluster using vnodes (I know vnodes on small clusters isn't 
 recommended).  Cassandra reports load of 40 gigs.
 We noticed a similar problem with a larger cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8177) sequential repair is much more expensive than parallel repair


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14183824#comment-14183824
 ] 

Yuki Morishita commented on CASSANDRA-8177:
---

One more thing that is likely related is when snapshotting, *all* SSTables are 
snapshot and opened even if the part of them are validated.
(Fixed in CASSANDRA-7024)

 sequential repair is much more expensive than parallel repair
 -

 Key: CASSANDRA-8177
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8177
 Project: Cassandra
  Issue Type: Bug
Reporter: Sean Bridges
Assignee: Yuki Morishita
 Attachments: cassc-week.png, iostats.png


 This is with 2.0.10
 The attached graph shows io read/write throughput (as measured with iostat) 
 when doing repairs.
 The large hump on the left is a sequential repair of one node.  The two much 
 smaller peaks on the right are parallel repairs.
 This is a 3 node cluster using vnodes (I know vnodes on small clusters isn't 
 recommended).  Cassandra reports load of 40 gigs.
 We noticed a similar problem with a larger cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-8177) sequential repair is much more expensive than parallel repair