[jira] [Updated] (CASSANDRA-11070) Dispatcher.Flusher's control has duplicated/conflict control

[fujian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

fujian updated CASSANDRA-11070:
---
Attachment: 0001-fix-CASSANDRA-11070.patch

> Dispatcher.Flusher's control has duplicated/conflict control
> 
>
> Key: CASSANDRA-11070
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11070
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: fujian
>  Labels: performance
> Attachments: 0001-fix-CASSANDRA-11070.patch
>
>
> org.apache.cassandra.transport.Message.Dispatcher.Flusher
> remove duplicated control for flush message control …
> Motivation:
> the !doneWork's control is duplicated and confused with runsSinceFlush > 2
> if on the first run:the queue size is 20
> donework will be set to true and not do flush due to the size<50 and 
> runsSinceFlush<2.
> if on the second run. the queue size is 0,
> donework will be reset to false and not set to true due to no new items in 
> queue, but the flush will be triggered due to:
>   if (!doneWork || runsSinceFlush > 2 || flushed.size() > 50)
> now the runsSinceFlush is 2. so in actual, its function is similar with 
> runsSinceFlush>1.
> so it is no need to keep it so that the code is confused and duplicated.
>   
> Modifications:
> remove it
> Result:
> after remove it, it will more clear and no confused.
> check the link: 
> https://github.com/jiafu1115/cassandra/commit/5279884e6a392d36b4adc5e29f9ca5d0666cb275



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11050) SSTables not loaded after dropping column

[amorton (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116610#comment-15116610
 ] 

amorton commented on CASSANDRA-11050:
-

bq. As a side note, we should really write a dtest for this issue before 
committing (amorton would you be up for that?).

[~slebresne] sure thing, is there some docs or example I should check out ? 

bq. So, your option 2 is safe, IMO (special case 
SchemaKeyspace::calculateSchemaDigest to ignore dropped_columns).

Thanks [~iamaleksey] will create a new patch with a list for schema hash and a 
list for everything else.

[~slebresne] what should I do to make the full change in 4.0, is the trunk at 
4.0 ? (sorry am out of the loop a bit :) )

> SSTables not loaded after dropping column
> -
>
> Key: CASSANDRA-11050
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11050
> Project: Cassandra
>  Issue Type: Bug
>  Components: Distributed Metadata
>Reporter: amorton
>Assignee: amorton
> Attachments: 11050-3.0.patch
>
>
> The {{system_schema.dropped_tables}} table is not flushed when schema is 
> updated, this can result in SSTables not being loaded at startup and failure 
> to start if the commit log contains mutations with the column. 
> Reproduce on cassandra-3.0 branch by starting a node and running following in 
> cqlsh:
> {code}
> create keyspace dev WITH replication = {'class':'SimpleStrategy', 
> 'replication_factor':1};
> use dev;
> create table foo (
>  foo text primary key,
>  bar text,
>  baz text
> );
> insert into foo (foo, bar, baz) values ('foo','this is bar', 'this is baz');
> alter table foo 
> drop baz;
> {code}
> Stop the node and restart, the following errors are raised and the node does 
> not start:
> {code}
> ERROR 16:38:19 Exception in thread Thread[SSTableBatchOpen:1,5,main]
> java.lang.RuntimeException: Unknown column baz during deserialization
>   at 
> org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:485)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:374)
>  ~[main/:na]
>   at 
> org.apache.cassandra.io.sstable.format.SSTableReader$4.run(SSTableReader.java:533)
>  ~[main/:na]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_60]
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_60]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_60]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_60]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
> ...
> ERROR 16:38:19 Exiting due to error while processing commit log during 
> initialization.
> org.apache.cassandra.db.commitlog.CommitLogReplayer$CommitLogReplayException: 
> Unexpected error deserializing mutation; saved to 
> /var/folders/r2/rkv1jz3j0j74r9s1zm5xx9wcgn/T/mutation5408885979635225676dat.
>   This may be caused by replaying a mutation against a table with the same 
> name but incompatible schema.  Exception follows: java.lang.RuntimeException: 
> Unknown column baz during deserialization
>   at 
> org.apache.cassandra.db.commitlog.CommitLogReplayer.handleReplayError(CommitLogReplayer.java:633)
>  [main/:na]
>   at 
> org.apache.cassandra.db.commitlog.CommitLogReplayer.replayMutation(CommitLogReplayer.java:556)
>  [main/:na]
>   at 
> org.apache.cassandra.db.commitlog.CommitLogReplayer.replaySyncSection(CommitLogReplayer.java:509)
>  [main/:na]
>   at 
> org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:404)
>  [main/:na]
>   at 
> org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:151)
>  [main/:na]
>   at 
> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:189) 
> [main/:na]
>   at 
> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:169) 
> [main/:na]
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:283) 
> [main/:na]
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:551)
>  [main/:na]
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:679) 
> [main/:na]
> {code}
> {{dropped_columns}} is not in the list of tables to flush, 
> {{SchemaKeyspace.ALL}}. 
> It's a simple patch to add it, attached. The fix will need to go to 3.0, 3.1 
> and trunk AFAIK
> however this will change the way the schema hash is calculated in 
> {{SchemaKeyspace.calculateSchemaDigest()}} It looks like this would cause 

[jira] [Updated] (CASSANDRA-11070) Dispatcher.Flusher's control has duplicated/conflict control

[fujian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

fujian updated CASSANDRA-11070:
---
Description: 
org.apache.cassandra.transport.Message.Dispatcher.Flusher

remove duplicated control for flush message control …

Motivation:

the !doneWork's control is duplicated and confused with runsSinceFlush > 2

if on the first run:the queue size is 20
donework will be set to true and not do flush due to the size<50 and 
runsSinceFlush<2.

if on the second run. the queue size is 0,
donework will be reset to false and not set to true due to no new items in 
queue, but the flush will be triggered due to:
  if (!doneWork || runsSinceFlush > 2 || flushed.size() > 50)
now the runsSinceFlush is 2. so in actual, its function is similar with 
runsSinceFlush>1.
so it is no need to keep it so that the code is confused and duplicated.
  

Modifications:

remove it

Result:

after remove it, it will more clear and no confused.

check the link: 
https://github.com/jiafu1115/cassandra/commit/5279884e6a392d36b4adc5e29f9ca5d0666cb275

  was:

org.apache.cassandra.transport.Message.Dispatcher.Flusher

remove duplicated control for flush message control …

Motivation:

the !doneWork's control is duplicated and confused with runsSinceFlush > 2

if on the first run:the queue size is 20
donework will be set to true and not do flush due to the size<50 and 
runsSinceFlush<2.

if on the second run. the queue size is 0,
donework will be reset to false and not set to true due to no new items in 
queue, but the flush will be triggered due to:
  if (!doneWork || runsSinceFlush > 2 || flushed.size() > 50)
now the runsSinceFlush is 2. so in actual, its function is similar with 
runsSinceFlush>1.
so it is no need to keep it so that the code is confused and duplicated.
  

Modifications:

remove it

Result:

after remove it, it will more clear and no confused.


> Dispatcher.Flusher's control has duplicated/conflict control
> 
>
> Key: CASSANDRA-11070
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11070
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: fujian
>  Labels: performance
>
> org.apache.cassandra.transport.Message.Dispatcher.Flusher
> remove duplicated control for flush message control …
> Motivation:
> the !doneWork's control is duplicated and confused with runsSinceFlush > 2
> if on the first run:the queue size is 20
> donework will be set to true and not do flush due to the size<50 and 
> runsSinceFlush<2.
> if on the second run. the queue size is 0,
> donework will be reset to false and not set to true due to no new items in 
> queue, but the flush will be triggered due to:
>   if (!doneWork || runsSinceFlush > 2 || flushed.size() > 50)
> now the runsSinceFlush is 2. so in actual, its function is similar with 
> runsSinceFlush>1.
> so it is no need to keep it so that the code is confused and duplicated.
>   
> Modifications:
> remove it
> Result:
> after remove it, it will more clear and no confused.
> check the link: 
> https://github.com/jiafu1115/cassandra/commit/5279884e6a392d36b4adc5e29f9ca5d0666cb275



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7464) Replace sstable2json and json2sstable

[Chris Lohfink (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Lohfink updated CASSANDRA-7464:
-
Attachment: sstabledump.patch

> Replace sstable2json and json2sstable
> -
>
> Key: CASSANDRA-7464
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7464
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sylvain Lebresne
>Assignee: Chris Lohfink
>Priority: Minor
> Fix For: 3.x
>
> Attachments: sstable-only.patch, sstabledump.patch
>
>
> Both tools are pretty awful. They are primarily meant for debugging (there is 
> much more efficient and convenient ways to do import/export data), but their 
> output manage to be hard to handle both for humans and for tools (especially 
> as soon as you have modern stuff like composites).
> There is value to having tools to export sstable contents into a format that 
> is easy to manipulate by human and tools for debugging, small hacks and 
> general tinkering, but sstable2json and json2sstable are not that.  
> So I propose that we deprecate those tools and consider writing better 
> replacements. It shouldn't be too hard to come up with an output format that 
> is more aware of modern concepts like composites, UDTs, 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7464) Replace sstable2json and json2sstable

[Chris Lohfink (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116784#comment-15116784
 ] 

Chris Lohfink commented on CASSANDRA-7464:
--

The debugging format that may be nice for both the "one per line" and the we 
can do that pretty easily using the UnfilteredRow.toString so instead of 
{code}
[
  {
"partition" : {
  "key" : [ "127.0.0.1-getWriteLatencyHisto" ],
  "position" : 19385620
},
"rows" : [
  {
"type" : "row",
"position" : 19385664,
"clustering" : [ "694621867" ],
"cells" : [
  { "name" : "value", "value" : "00", "tstamp" : 1452861829846001, 
"ttl" : 604800, "expires_at" : 1453466629, "expired" : true }
]
  },
  {
"type" : "row",
"position" : 19385686,
"clustering" : [ "694621927" ],
"cells" : [
  { "name" : "value", "value" : "00", "tstamp" : 1452861769124000, 
"ttl" : 604800, "expires_at" : 1453466569, "expired" : true }
]
  },
  {
"type" : "row",
"position" : 19385708,
"clustering" : [ "694621987" ],
"cells" : [
  { "name" : "value", "value" : "00", "tstamp" : 1452861709303002, 
"ttl" : 604800, "expires_at" : 1453466509, "expired" : true }
]
  },
  {
"type" : "row",
"position" : 19385730,
"clustering" : [ "694622047" ],
"cells" : [
  { "name" : "value", "value" : "00", "tstamp" : 1452861649548002, 
"ttl" : 604800, "expires_at" : 1453466449, "expired" : true }
]
  },
...
{code}
it can be
{code}
[127.0.0.1-getWriteLatencyHisto]@19385620 Row[info=[ts=-9223372036854775808] ]: 
694621867 | [value=00 ts=1452861829846001 ttl=604800 ldt=1453466629]
[127.0.0.1-getWriteLatencyHisto]@19385686 Row[info=[ts=-9223372036854775808] ]: 
694621927 | [value=00 ts=1452861769124000 ttl=604800 ldt=1453466569]
[127.0.0.1-getWriteLatencyHisto]@19385708 Row[info=[ts=-9223372036854775808] ]: 
694621987 | [value=00 ts=1452861709303002 ttl=604800 ldt=1453466509]
[127.0.0.1-getWriteLatencyHisto]@19385730 Row[info=[ts=-9223372036854775808] ]: 
694622047 | [value=00 ts=1452861649548002 ttl=604800 ldt=1453466449]
...
{code}
This would also have benefit for easily splitting files for hadoop jobs etc 
since it would have a cql row per line (easing wide partition issues with the 
compact output mentioned above). It would also tie the rendering to something 
already maintained for debug logging etc so little additional work for 
refactoring/storage changes. I am kinda a fan of both. So I implemented a 
{{-d}} (could use better name) option for the 1 row per line "debuggy" compact 
option (worth noting this is very hard to read if theres a lot of cells).

Also added the current position from the scanner in the results (see above 
examples).

Until CASSANDRA-9587 I had to add an alternative not to print out clustering 
key names in the toString since its not available anywhere which is a little 
hacky but can be removed once we have the names.

> Replace sstable2json and json2sstable
> -
>
> Key: CASSANDRA-7464
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7464
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sylvain Lebresne
>Assignee: Chris Lohfink
>Priority: Minor
> Fix For: 3.x
>
> Attachments: sstable-only.patch, sstabledump.patch
>
>
> Both tools are pretty awful. They are primarily meant for debugging (there is 
> much more efficient and convenient ways to do import/export data), but their 
> output manage to be hard to handle both for humans and for tools (especially 
> as soon as you have modern stuff like composites).
> There is value to having tools to export sstable contents into a format that 
> is easy to manipulate by human and tools for debugging, small hacks and 
> general tinkering, but sstable2json and json2sstable are not that.  
> So I propose that we deprecate those tools and consider writing better 
> replacements. It shouldn't be too hard to come up with an output format that 
> is more aware of modern concepts like composites, UDTs, 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8180) Optimize disk seek using min/max column name meta data when the LIMIT clause is used

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116655#comment-15116655
 ] 

Stefania commented on CASSANDRA-8180:
-

bq. That is, your code is currently completely disabled by the {{if (... 
iterator != null) return null; check in lowerBound()?}} Is there a reason to 
want the code committed then?

It is enabled when there are no tombsones since the iterator is null. However 
we can do better, when there are tombstones we can still use the row index 
bound if there is one.


bq. On the other hand, an assertion in {{UnfilteredIteratorWithLowerBound}} 
that saves the returned {{lowerBound()}} and checks the first {{next()}} to be 
greater-than or equal may be something we want to have anyway.

I've added this. Whilst the test you proposed is fine, existing tests (e.g. 
{{DeleteTest.testDeleteWithRangeAndOneClusteringColumn()}}) started to fail but 
_only when compaction is enabled_. It seems we can have sstables with no 
tombstones, valid clustering values (e.g. 0) and yet the first item is a 
{{Bound.BOTTOM}}. 

Further, it seems that static columns could also have problems with clustering 
values so it's not just tombstones. See the comment in 
{{SinglePartitionReadCommand.shouldInclude()}}. 

Moving back to in progress, might end up modifying sstable metadata format 
after all.

> Optimize disk seek using min/max column name meta data when the LIMIT clause 
> is used
> 
>
> Key: CASSANDRA-8180
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8180
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
> Environment: Cassandra 2.0.10
>Reporter: DOAN DuyHai
>Assignee: Stefania
>Priority: Minor
> Fix For: 3.x
>
> Attachments: 8180_001.yaml, 8180_002.yaml
>
>
> I was working on an example of sensor data table (timeseries) and face a use 
> case where C* does not optimize read on disk.
> {code}
> cqlsh:test> CREATE TABLE test(id int, col int, val text, PRIMARY KEY(id,col)) 
> WITH CLUSTERING ORDER BY (col DESC);
> cqlsh:test> INSERT INTO test(id, col , val ) VALUES ( 1, 10, '10');
> ...
> >nodetool flush test test
> ...
> cqlsh:test> INSERT INTO test(id, col , val ) VALUES ( 1, 20, '20');
> ...
> >nodetool flush test test
> ...
> cqlsh:test> INSERT INTO test(id, col , val ) VALUES ( 1, 30, '30');
> ...
> >nodetool flush test test
> {code}
> After that, I activate request tracing:
> {code}
> cqlsh:test> SELECT * FROM test WHERE id=1 LIMIT 1;
>  activity  | 
> timestamp| source| source_elapsed
> ---+--+---+
> execute_cql3_query | 
> 23:48:46,498 | 127.0.0.1 |  0
> Parsing SELECT * FROM test WHERE id=1 LIMIT 1; | 
> 23:48:46,498 | 127.0.0.1 | 74
>Preparing statement | 
> 23:48:46,499 | 127.0.0.1 |253
>   Executing single-partition query on test | 
> 23:48:46,499 | 127.0.0.1 |930
>   Acquiring sstable references | 
> 23:48:46,499 | 127.0.0.1 |943
>Merging memtable tombstones | 
> 23:48:46,499 | 127.0.0.1 |   1032
>Key cache hit for sstable 3 | 
> 23:48:46,500 | 127.0.0.1 |   1160
>Seeking to partition beginning in data file | 
> 23:48:46,500 | 127.0.0.1 |   1173
>Key cache hit for sstable 2 | 
> 23:48:46,500 | 127.0.0.1 |   1889
>Seeking to partition beginning in data file | 
> 23:48:46,500 | 127.0.0.1 |   1901
>Key cache hit for sstable 1 | 
> 23:48:46,501 | 127.0.0.1 |   2373
>Seeking to partition beginning in data file | 
> 23:48:46,501 | 127.0.0.1 |   2384
>  Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 
> 23:48:46,501 | 127.0.0.1 |   2768
> Merging data from memtables and 3 sstables | 
> 23:48:46,501 | 127.0.0.1 |   2784
> Read 2 live and 0 tombstoned cells | 
> 23:48:46,501 | 127.0.0.1 |   2976
>   Request complete | 
> 23:48:46,501 | 127.0.0.1 |   3551
> {code}
> We can 

[jira] [Commented] (CASSANDRA-8180) Optimize disk seek using min/max column name meta data when the LIMIT clause is used

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116897#comment-15116897
 ] 

Stefania commented on CASSANDRA-8180:
-

I think the issue is in {{MetadataCollector}}:

{code}
public void update(Cell cell)
{
updateTimestamp(cell.timestamp());
updateTTL(cell.ttl());
updateLocalDeletionTime(cell.localDeletionTime());
}
{code}

Shouldn't we check that the cell is a tombstone before updating the local 
deletion time?

> Optimize disk seek using min/max column name meta data when the LIMIT clause 
> is used
> 
>
> Key: CASSANDRA-8180
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8180
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
> Environment: Cassandra 2.0.10
>Reporter: DOAN DuyHai
>Assignee: Stefania
>Priority: Minor
> Fix For: 3.x
>
> Attachments: 8180_001.yaml, 8180_002.yaml
>
>
> I was working on an example of sensor data table (timeseries) and face a use 
> case where C* does not optimize read on disk.
> {code}
> cqlsh:test> CREATE TABLE test(id int, col int, val text, PRIMARY KEY(id,col)) 
> WITH CLUSTERING ORDER BY (col DESC);
> cqlsh:test> INSERT INTO test(id, col , val ) VALUES ( 1, 10, '10');
> ...
> >nodetool flush test test
> ...
> cqlsh:test> INSERT INTO test(id, col , val ) VALUES ( 1, 20, '20');
> ...
> >nodetool flush test test
> ...
> cqlsh:test> INSERT INTO test(id, col , val ) VALUES ( 1, 30, '30');
> ...
> >nodetool flush test test
> {code}
> After that, I activate request tracing:
> {code}
> cqlsh:test> SELECT * FROM test WHERE id=1 LIMIT 1;
>  activity  | 
> timestamp| source| source_elapsed
> ---+--+---+
> execute_cql3_query | 
> 23:48:46,498 | 127.0.0.1 |  0
> Parsing SELECT * FROM test WHERE id=1 LIMIT 1; | 
> 23:48:46,498 | 127.0.0.1 | 74
>Preparing statement | 
> 23:48:46,499 | 127.0.0.1 |253
>   Executing single-partition query on test | 
> 23:48:46,499 | 127.0.0.1 |930
>   Acquiring sstable references | 
> 23:48:46,499 | 127.0.0.1 |943
>Merging memtable tombstones | 
> 23:48:46,499 | 127.0.0.1 |   1032
>Key cache hit for sstable 3 | 
> 23:48:46,500 | 127.0.0.1 |   1160
>Seeking to partition beginning in data file | 
> 23:48:46,500 | 127.0.0.1 |   1173
>Key cache hit for sstable 2 | 
> 23:48:46,500 | 127.0.0.1 |   1889
>Seeking to partition beginning in data file | 
> 23:48:46,500 | 127.0.0.1 |   1901
>Key cache hit for sstable 1 | 
> 23:48:46,501 | 127.0.0.1 |   2373
>Seeking to partition beginning in data file | 
> 23:48:46,501 | 127.0.0.1 |   2384
>  Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 
> 23:48:46,501 | 127.0.0.1 |   2768
> Merging data from memtables and 3 sstables | 
> 23:48:46,501 | 127.0.0.1 |   2784
> Read 2 live and 0 tombstoned cells | 
> 23:48:46,501 | 127.0.0.1 |   2976
>   Request complete | 
> 23:48:46,501 | 127.0.0.1 |   3551
> {code}
> We can clearly see that C* hits 3 SSTables on disk instead of just one, 
> although it has the min/max column meta data to decide which SSTable contains 
> the most recent data.
> Funny enough, if we add a clause on the clustering column to the select, this 
> time C* optimizes the read path:
> {code}
> cqlsh:test> SELECT * FROM test WHERE id=1 AND col > 25 LIMIT 1;
>  activity  | 
> timestamp| source| source_elapsed
> ---+--+---+
> execute_cql3_query | 
> 23:52:31,888 | 127.0.0.1 |  0
>Parsing SELECT * FROM test WHERE id=1 AND col > 25 LIMIT 1; | 
> 23:52:31,888 | 127.0.0.1 | 60

[jira] [Comment Edited] (CASSANDRA-7464) Replace sstable2json and json2sstable

[Chris Lohfink (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116784#comment-15116784
 ] 

Chris Lohfink edited comment on CASSANDRA-7464 at 1/26/16 6:44 AM:
---

The debugging format that may be nice for both the "one per line" and the we 
can do that pretty easily using the UnfilteredRow.toString so instead of 
{code}
[
  {
"partition" : {
  "key" : [ "127.0.0.1-getWriteLatencyHisto" ],
  "position" : 19385620
},
"rows" : [
  {
"type" : "row",
"position" : 19385664,
"clustering" : [ "694621867" ],
"cells" : [
  { "name" : "value", "value" : "00", "tstamp" : 1452861829846001, 
"ttl" : 604800, "expires_at" : 1453466629, "expired" : true }
]
  },
  {
"type" : "row",
"position" : 19385686,
"clustering" : [ "694621927" ],
"cells" : [
  { "name" : "value", "value" : "00", "tstamp" : 1452861769124000, 
"ttl" : 604800, "expires_at" : 1453466569, "expired" : true }
]
  },
  {
"type" : "row",
"position" : 19385708,
"clustering" : [ "694621987" ],
"cells" : [
  { "name" : "value", "value" : "00", "tstamp" : 1452861709303002, 
"ttl" : 604800, "expires_at" : 1453466509, "expired" : true }
]
  },
  {
"type" : "row",
"position" : 19385730,
"clustering" : [ "694622047" ],
"cells" : [
  { "name" : "value", "value" : "00", "tstamp" : 1452861649548002, 
"ttl" : 604800, "expires_at" : 1453466449, "expired" : true }
]
  },
...
{code}
it can be
{code}
[127.0.0.1-getWriteLatencyHisto]@19385620 Row[info=[ts=-9223372036854775808] ]: 
694621867 | [value=00 ts=1452861829846001 ttl=604800 ldt=1453466629]
[127.0.0.1-getWriteLatencyHisto]@19385686 Row[info=[ts=-9223372036854775808] ]: 
694621927 | [value=00 ts=1452861769124000 ttl=604800 ldt=1453466569]
[127.0.0.1-getWriteLatencyHisto]@19385708 Row[info=[ts=-9223372036854775808] ]: 
694621987 | [value=00 ts=1452861709303002 ttl=604800 ldt=1453466509]
[127.0.0.1-getWriteLatencyHisto]@19385730 Row[info=[ts=-9223372036854775808] ]: 
694622047 | [value=00 ts=1452861649548002 ttl=604800 ldt=1453466449]
...
{code}
This would also have benefit for easily splitting files for hadoop jobs etc 
since it would have a cql row per line (easing wide partition issues with the 
compact output from Russell discussion in other ticket). It would also tie the 
rendering to something already maintained for debug logging etc so little 
additional work for refactoring/storage changes. I am kinda a fan of both. So I 
implemented a {{-d}} (could use better name) option for the 1 row per line 
"debuggy" compact option (worth noting this is very hard to read if theres a 
lot of cells).

Also added the current position from the scanner in the results (see above 
examples).

Until CASSANDRA-9587 I had to add an alternative not to print out clustering 
key names in the toString since its not available anywhere which is a little 
hacky but can be removed once we have the names.


was (Author: cnlwsu):
The debugging format that may be nice for both the "one per line" and the we 
can do that pretty easily using the UnfilteredRow.toString so instead of 
{code}
[
  {
"partition" : {
  "key" : [ "127.0.0.1-getWriteLatencyHisto" ],
  "position" : 19385620
},
"rows" : [
  {
"type" : "row",
"position" : 19385664,
"clustering" : [ "694621867" ],
"cells" : [
  { "name" : "value", "value" : "00", "tstamp" : 1452861829846001, 
"ttl" : 604800, "expires_at" : 1453466629, "expired" : true }
]
  },
  {
"type" : "row",
"position" : 19385686,
"clustering" : [ "694621927" ],
"cells" : [
  { "name" : "value", "value" : "00", "tstamp" : 1452861769124000, 
"ttl" : 604800, "expires_at" : 1453466569, "expired" : true }
]
  },
  {
"type" : "row",
"position" : 19385708,
"clustering" : [ "694621987" ],
"cells" : [
  { "name" : "value", "value" : "00", "tstamp" : 1452861709303002, 
"ttl" : 604800, "expires_at" : 1453466509, "expired" : true }
]
  },
  {
"type" : "row",
"position" : 19385730,
"clustering" : [ "694622047" ],
"cells" : [
  { "name" : "value", "value" : "00", "tstamp" : 1452861649548002, 
"ttl" : 604800, "expires_at" : 1453466449, "expired" : true }
]
  },
...
{code}
it can be
{code}
[127.0.0.1-getWriteLatencyHisto]@19385620 Row[info=[ts=-9223372036854775808] ]: 
694621867 | [value=00 ts=1452861829846001 ttl=604800 ldt=1453466629]
[127.0.0.1-getWriteLatencyHisto]@19385686 Row[info=[ts=-9223372036854775808] ]: 
694621927 | [value=00 ts=1452861769124000 ttl=604800 ldt=1453466569]

[jira] [Created] (CASSANDRA-11070) Dispatcher.Flusher's control has duplicated/conflict control

[fujian (JIRA)]

fujian created CASSANDRA-11070:
--

 Summary: Dispatcher.Flusher's control has duplicated/conflict 
control
 Key: CASSANDRA-11070
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11070
 Project: Cassandra
  Issue Type: Bug
  Components: Streaming and Messaging
Reporter: fujian



org.apache.cassandra.transport.Message.Dispatcher.Flusher

remove duplicated control for flush message control …

Motivation:

the !doneWork's control is duplicated and confused with runsSinceFlush > 2

if on the first run:the queue size is 20
donework will be set to true and not do flush due to the size<50 and 
runsSinceFlush<2.

if on the second run. the queue size is 0,
donework will be reset to false and not set to true due to no new items in 
queue, but the flush will be triggered due to:
  if (!doneWork || runsSinceFlush > 2 || flushed.size() > 50)
now the runsSinceFlush is 2. so in actual, its function is similar with 
runsSinceFlush>1.
so it is no need to keep it so that the code is confused and duplicated.
  

Modifications:

remove it

Result:

after remove it, it will more clear and no confused.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9472) Reintroduce off heap memtables

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116758#comment-15116758
 ] 

Stefania commented on CASSANDRA-9472:
-

I've pushed one more commit 
[here|https://github.com/stef1927/cassandra/commit/f035a5a0e69c745d2257186258a1ffa113427120]
 where I fixed a compilation problem (a leftover of my changes yesterday) and 
removed the {{peer == 0}} guards. 

bq. I assumed that (as it used to be) the peer could be set to zero on free, 
and that this was how you were encountering the problem. I see now it is only 
the fake NativeCell that causes problems; I do wonder if we shouldn't simply 
allocate it for real, to avoid the check for every single call to a cell. It's 
also highly unlikely to ever appear in anyone's debugger if it's only that 
specific cell, so we could simply ignore the problem entirely.

Yes it was only the fake native cell and clustering causing problems but I was 
unable to reproduce these segmentation faults today so either some code 
underneath has changed or the root cause must have been something else.

I've launched the tests on my branch, if they are clear I think we are good to 
commit:

|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-9472-updated-testall/]|
|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-9472-updated-dtest/]|



> Reintroduce off heap memtables
> --
>
> Key: CASSANDRA-9472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9472
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Benedict
>Assignee: Benedict
> Fix For: 3.x
>
>
> CASSANDRA-8099 removes off heap memtables. We should reintroduce them ASAP.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8180) Optimize disk seek using min/max column name meta data when the LIMIT clause is used

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116781#comment-15116781
 ] 

Stefania commented on CASSANDRA-8180:
-

bq. It seems we can have sstables with no tombstones, valid clustering values 
(e.g. 0) and yet the first item is a Bound.BOTTOM. 

It's once again the metadata that is wrong: {{maxLocalDeletionTime}} is 
{{Integer.MAX_VALUE}} even though there are tombstones in the sstable, as a 
consequence {{sstable.hasTombstones()}} returns an incorrect value.

> Optimize disk seek using min/max column name meta data when the LIMIT clause 
> is used
> 
>
> Key: CASSANDRA-8180
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8180
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
> Environment: Cassandra 2.0.10
>Reporter: DOAN DuyHai
>Assignee: Stefania
>Priority: Minor
> Fix For: 3.x
>
> Attachments: 8180_001.yaml, 8180_002.yaml
>
>
> I was working on an example of sensor data table (timeseries) and face a use 
> case where C* does not optimize read on disk.
> {code}
> cqlsh:test> CREATE TABLE test(id int, col int, val text, PRIMARY KEY(id,col)) 
> WITH CLUSTERING ORDER BY (col DESC);
> cqlsh:test> INSERT INTO test(id, col , val ) VALUES ( 1, 10, '10');
> ...
> >nodetool flush test test
> ...
> cqlsh:test> INSERT INTO test(id, col , val ) VALUES ( 1, 20, '20');
> ...
> >nodetool flush test test
> ...
> cqlsh:test> INSERT INTO test(id, col , val ) VALUES ( 1, 30, '30');
> ...
> >nodetool flush test test
> {code}
> After that, I activate request tracing:
> {code}
> cqlsh:test> SELECT * FROM test WHERE id=1 LIMIT 1;
>  activity  | 
> timestamp| source| source_elapsed
> ---+--+---+
> execute_cql3_query | 
> 23:48:46,498 | 127.0.0.1 |  0
> Parsing SELECT * FROM test WHERE id=1 LIMIT 1; | 
> 23:48:46,498 | 127.0.0.1 | 74
>Preparing statement | 
> 23:48:46,499 | 127.0.0.1 |253
>   Executing single-partition query on test | 
> 23:48:46,499 | 127.0.0.1 |930
>   Acquiring sstable references | 
> 23:48:46,499 | 127.0.0.1 |943
>Merging memtable tombstones | 
> 23:48:46,499 | 127.0.0.1 |   1032
>Key cache hit for sstable 3 | 
> 23:48:46,500 | 127.0.0.1 |   1160
>Seeking to partition beginning in data file | 
> 23:48:46,500 | 127.0.0.1 |   1173
>Key cache hit for sstable 2 | 
> 23:48:46,500 | 127.0.0.1 |   1889
>Seeking to partition beginning in data file | 
> 23:48:46,500 | 127.0.0.1 |   1901
>Key cache hit for sstable 1 | 
> 23:48:46,501 | 127.0.0.1 |   2373
>Seeking to partition beginning in data file | 
> 23:48:46,501 | 127.0.0.1 |   2384
>  Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 
> 23:48:46,501 | 127.0.0.1 |   2768
> Merging data from memtables and 3 sstables | 
> 23:48:46,501 | 127.0.0.1 |   2784
> Read 2 live and 0 tombstoned cells | 
> 23:48:46,501 | 127.0.0.1 |   2976
>   Request complete | 
> 23:48:46,501 | 127.0.0.1 |   3551
> {code}
> We can clearly see that C* hits 3 SSTables on disk instead of just one, 
> although it has the min/max column meta data to decide which SSTable contains 
> the most recent data.
> Funny enough, if we add a clause on the clustering column to the select, this 
> time C* optimizes the read path:
> {code}
> cqlsh:test> SELECT * FROM test WHERE id=1 AND col > 25 LIMIT 1;
>  activity  | 
> timestamp| source| source_elapsed
> ---+--+---+
> execute_cql3_query | 
> 23:52:31,888 | 127.0.0.1 |  0
>Parsing SELECT * FROM test WHERE id=1 AND col > 25 LIMIT 1; | 
> 23:52:31,888 | 127.0.0.1 

[jira] [Updated] (CASSANDRA-10836) Data integrity flappers in upgrade_tests dtests

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-10836:
-
Priority: Major  (was: Critical)

> Data integrity flappers in upgrade_tests dtests
> ---
>
> Key: CASSANDRA-10836
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10836
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Sylvain Lebresne
> Fix For: 3.0.x, 3.x
>
>
> A number of the {{upgrade_tests}} in dtest are flapping with data integrity 
> bugs. For instance:
> http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/422/testReport/upgrade_tests.cql_tests/TestCQLNodes3RF3/conditional_delete_test/history/
> http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/421/testReport/upgrade_tests.cql_tests/TestCQLNodes3RF3/cas_and_list_index_test/history/
> This needs further digging to get good debug information; it could well be, 
> e.g., a race condition in the test. I have not reproduced locally.
> /cc [~philipthompson]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11045) Legacy SSTables for KEYS indexes can't be read in 3.0+

[Sam Tunnicliffe (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115366#comment-15115366
 ] 

Sam Tunnicliffe commented on CASSANDRA-11045:
-

||branch||testall||dtest||
|[11045-3.0|https://github.com/beobal/cassandra/tree/11046-5.0]|[testall|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-11045-3.0-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-11045-3.0-dtest]|

(dtests currently running)

> Legacy SSTables for KEYS indexes can't be read in 3.0+
> --
>
> Key: CASSANDRA-11045
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11045
> Project: Cassandra
>  Issue Type: Bug
>  Components: Distributed Metadata, Local Write-Read Paths
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Critical
> Fix For: 3.0.x, 3.x
>
>
> Trying to read an SSTable for a KEYS index created in 2.1 throws a 
> deserialization error when being read following an upgrade to 3.0. This 
> occurs whether the read comes via the query path or by running {{nodetool 
> upgradesstables}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11065) null pointer exception in CassandraDaemon.java:195

[Vassil Lunchev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vassil Lunchev updated CASSANDRA-11065:
---
Description: 
I had a keyspace called 'tests'. I dropped it. Then I checked some nodes and 
one of them still had that keyspace 'tests'. On a node that still has the 
dropped keyspace I ran:
nodetools repair tests;

In the system logs of another node that did not have keyspace 'tests' I am 
seeing a null pointer exception:

{code:java}
ERROR [AntiEntropyStage:2] 2016-01-25 15:02:46,323 
RepairMessageVerbHandler.java:161 - Got error, removing parent repair session
ERROR [AntiEntropyStage:2] 2016-01-25 15:02:46,324 CassandraDaemon.java:195 - 
Exception in thread Thread[AntiEntropyStage:2,5,main]
java.lang.RuntimeException: java.lang.NullPointerException
at 
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:164)
 ~[apache-cassandra-3.0.1.jar:3.0.1]
at 
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) 
~[apache-cassandra-3.0.1.jar:3.0.1]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
~[na:1.8.0_66-internal]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
~[na:1.8.0_66-internal]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_66-internal]
Caused by: java.lang.NullPointerException: null
at 
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:69)
 ~[apache-cassandra-3.0.1.jar:3.0.1]
... 4 common frames omitted
{code}


  was:
I had a keyspace called 'tests'. I dropped it. Then I checked some nodes and 
one of them still had that keyspace 'tests'. On a node that still has the 
dropped keyspace I ran:
nodetools repair tests;

In the system logs of another node that did not have keyspace 'tests' I am 
seeing a null pointer exception:

ERROR [AntiEntropyStage:2] 2016-01-25 15:02:46,323 
RepairMessageVerbHandler.java:161 - Got error, removing parent repair session
ERROR [AntiEntropyStage:2] 2016-01-25 15:02:46,324 CassandraDaemon.java:195 - 
Exception in thread Thread[AntiEntropyStage:2,5,main]
java.lang.RuntimeException: java.lang.NullPointerException
at 
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:164)
 ~[apache-cassandra-3.0.1.jar:3.0.1]
at 
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) 
~[apache-cassandra-3.0.1.jar:3.0.1]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
~[na:1.8.0_66-internal]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
~[na:1.8.0_66-internal]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_66-internal]
Caused by: java.lang.NullPointerException: null
at 
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:69)
 ~[apache-cassandra-3.0.1.jar:3.0.1]
... 4 common frames omitted


> null pointer exception in CassandraDaemon.java:195
> --
>
> Key: CASSANDRA-11065
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11065
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Vassil Lunchev
>Priority: Minor
> Fix For: 3.0.1
>
>
> I had a keyspace called 'tests'. I dropped it. Then I checked some nodes and 
> one of them still had that keyspace 'tests'. On a node that still has the 
> dropped keyspace I ran:
> nodetools repair tests;
> In the system logs of another node that did not have keyspace 'tests' I am 
> seeing a null pointer exception:
> {code:java}
> ERROR [AntiEntropyStage:2] 2016-01-25 15:02:46,323 
> RepairMessageVerbHandler.java:161 - Got error, removing parent repair session
> ERROR [AntiEntropyStage:2] 2016-01-25 15:02:46,324 CassandraDaemon.java:195 - 
> Exception in thread Thread[AntiEntropyStage:2,5,main]
> java.lang.RuntimeException: java.lang.NullPointerException
>   at 
> org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:164)
>  ~[apache-cassandra-3.0.1.jar:3.0.1]
>   at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) 
> ~[apache-cassandra-3.0.1.jar:3.0.1]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_66-internal]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  ~[na:1.8.0_66-internal]
>   at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_66-internal]
> Caused by: java.lang.NullPointerException: null
>   at 
> org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:69)
>  ~[apache-cassandra-3.0.1.jar:3.0.1]
>   ... 4 

[jira] [Created] (CASSANDRA-11066) Static column with secondary index on clustering column causes ArrayIndexOutOfBoundsException

[Artem Soloviov (JIRA)]

Artem Soloviov created CASSANDRA-11066:
--

 Summary: Static column with secondary index on clustering column 
causes ArrayIndexOutOfBoundsException
 Key: CASSANDRA-11066
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11066
 Project: Cassandra
  Issue Type: Bug
  Components: CQL, Local Write-Read Paths
 Environment: [cqlsh 5.0.1 | Cassandra 3.2.1 | CQL spec 3.4.0 | Native 
protocol v4]
Reporter: Artem Soloviov


Enabling secondary index on clustering column breaks static column support:
{code}
Connected to Exodentity Cassandra Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 3.2.1 | CQL spec 3.4.0 | Native protocol v4]
Use HELP for help.
cqlsh> CREATE KEYSPACE test WITH replication = {'class': 'SimpleStrategy', 
'replication_factor': 1};
cqlsh> USE test;
cqlsh:test> CREATE  TABLE t (k text , s text  static, i int, PRIMARY KEY (k, 
i));
cqlsh:test> CREATE INDEX t_idx ON t(i);
cqlsh:test> INSERT INTO t (k,s,i) VALUES ('k','static value', 0);
{code}
causes 
{noformat}
java.lang.ArrayIndexOutOfBoundsException: 0
at 
org.apache.cassandra.db.AbstractClusteringPrefix.get(AbstractClusteringPrefix.java:59)
 ~[apache-cassandra-3.2.1.jar:3.2.1]
at 
org.apache.cassandra.index.internal.composites.ClusteringColumnIndex.getIndexedValue(ClusteringColumnIndex.java:60)
 ~[apache-cassandra-3.2.1.jar:3.2.1]
at 
org.apache.cassandra.index.internal.CassandraIndex.getIndexedValue(CassandraIndex.java:598)
 ~[apache-cassandra-3.2.1.jar:3.2.1]
at 
org.apache.cassandra.index.internal.CassandraIndex.insert(CassandraIndex.java:490)
 ~[apache-cassandra-3.2.1.jar:3.2.1]
at 
org.apache.cassandra.index.internal.CassandraIndex.access$100(CassandraIndex.java:53)
 ~[apache-cassandra-3.2.1.jar:3.2.1]
at 
org.apache.cassandra.index.internal.CassandraIndex$1.indexPrimaryKey(CassandraIndex.java:437)
 ~[apache-cassandra-3.2.1.jar:3.2.1]
at 
org.apache.cassandra.index.internal.CassandraIndex$1.insertRow(CassandraIndex.java:347)
 ~[apache-cassandra-3.2.1.jar:3.2.1]
at 
org.apache.cassandra.index.SecondaryIndexManager$WriteTimeTransaction.onInserted(SecondaryIndexManager.java:803)
 ~[apache-cassandra-3.2.1.jar:3.2.1]
at 
org.apache.cassandra.db.partitions.AtomicBTreePartition$RowUpdater.apply(AtomicBTreePartition.java:275)
 ~[apache-cassandra-3.2.1.jar:3.2.1]
at 
org.apache.cassandra.db.partitions.AtomicBTreePartition.addAllWithSizeDelta(AtomicBTreePartition.java:154)
 ~[apache-cassandra-3.2.1.jar:3.2.1]
at org.apache.cassandra.db.Memtable.put(Memtable.java:239) 
~[apache-cassandra-3.2.1.jar:3.2.1]
at 
org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:1225) 
~[apache-cassandra-3.2.1.jar:3.2.1]
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:494) 
~[apache-cassandra-3.2.1.jar:3.2.1]
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:384) 
~[apache-cassandra-3.2.1.jar:3.2.1]
at org.apache.cassandra.db.Mutation.apply(Mutation.java:202) 
~[apache-cassandra-3.2.1.jar:3.2.1]
at 
org.apache.cassandra.service.StorageProxy$$Lambda$223/885149282.run(Unknown 
Source) ~[na:na]
at 
org.apache.cassandra.service.StorageProxy$7.runMayThrow(StorageProxy.java:1276) 
~[apache-cassandra-3.2.1.jar:3.2.1]
at 
org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2433)
 [apache-cassandra-3.2.1.jar:3.2.1]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
[na:1.8.0_51]
at 
org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
 [apache-cassandra-3.2.1.jar:3.2.1]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
[apache-cassandra-3.2.1.jar:3.2.1]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11064) Failed aggregate creation breaks server permanently

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115337#comment-15115337
 ] 

Robert Stupp commented on CASSANDRA-11064:
--

Got the bug. In CreateAggregateStatement, the initcond is validated by parsing 
the "raw" CQL value (in this case {{'s@foo:i@32'}}). But during creation of the 
schema-migration-mutation (which should succeed, as it has passed the 
validation) we re-construct the CQL term from the binary representation, which 
simply calls {{AbstractType.getString}} for all custom types. Will check & fix 
that.

> Failed aggregate creation breaks server permanently
> ---
>
> Key: CASSANDRA-11064
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11064
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Olivier Michallat
>Assignee: Robert Stupp
>
> While testing edge cases around aggregates, I tried the following to see if 
> custom types were supported:
> {code}
> ccm create v321 -v3.2.1 -n3
> ccm updateconf enable_user_defined_functions:true
> ccm start
> ccm node1 cqlsh
> CREATE FUNCTION id(i 'DynamicCompositeType(s => UTF8Type, i => Int32Type)')
> RETURNS NULL ON NULL INPUT
> RETURNS 'DynamicCompositeType(s => UTF8Type, i => Int32Type)'
> LANGUAGE java
> AS 'return i;';
> // function created successfully
> CREATE AGGREGATE ag()
> SFUNC id
> STYPE 'DynamicCompositeType(s => UTF8Type, i => Int32Type)'
> INITCOND 's@foo:i@32';
> ServerError:  message="java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.exceptions.SyntaxException: Failed parsing CQL term: 
> [s@foo:i@32] reason: SyntaxException line 1:1 no viable alternative at 
> character '@'">{code}
> Despite the error, the aggregate appears in system tables:
> {code}
> select * from system_schema.aggregates;
>  keyspace_name | aggregate_name | ...
> ---++ ...
>   test | ag | ...
> {code}
> But you can't drop it, and trying to drop its function produces the server 
> error again:
> {code}
> DROP AGGREGATE ag;
> InvalidRequest: code=2200 [Invalid query] message="Cannot drop non existing 
> aggregate 'test.ag'"
> DROP FUNCTION id;
> ServerError:  message="java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.exceptions.SyntaxException: Failed parsing CQL term: 
> [s@foo:i@32] reason: SyntaxException line 1:1 no viable alternative at 
> character '@'">
> {code}
> What's worse, it's now impossible to restart the server:
> {code}
> ccm stop; ccm start
> org.apache.cassandra.exceptions.SyntaxException: Failed parsing CQL term: 
> [s@foo:i@32] reason: SyntaxException line 1:1 no viable alternative at 
> character '@'
>   at 
> org.apache.cassandra.cql3.CQLFragmentParser.parseAny(CQLFragmentParser.java:48)
>   at org.apache.cassandra.cql3.Terms.asBytes(Terms.java:51)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.createUDAFromRow(SchemaKeyspace.java:1225)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchUDAs(SchemaKeyspace.java:1204)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchFunctions(SchemaKeyspace.java:1129)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:897)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:872)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:860)
>   at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:125)
>   at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:115)
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:229)
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:551)
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:680)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11046) Existing indexes are always rebuilt on upgrade to 3.0

[Sam Tunnicliffe (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115363#comment-15115363
 ] 

Sam Tunnicliffe commented on CASSANDRA-11046:
-

||branch||testall||dtest||
|[11046-3.0|https://github.com/beobal/cassandra/tree/11046-3.0]|[testall|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-11046-3.0-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-11046-3.0-dtest]|

Seems like the dtest run I kicked off when I posted the patch didn't run for 
whatever reason, so I've started another

> Existing indexes are always rebuilt on upgrade to 3.0
> -
>
> Key: CASSANDRA-11046
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11046
> Project: Cassandra
>  Issue Type: Bug
>  Components: Distributed Metadata
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Critical
> Fix For: 3.0.x, 3.x
>
>
> CASSANDRA-10127 made the naming of secondary indexes consistent internally. 
> Previously some places used just the index name, whilst others required the 
> name in the format {{table.index}}. 
> One place including the additional table name was the {{system.IndexInfo}} 
> table, which records whether or not the index has been built. On upgrade to 
> 3.0, as the node restarts and initialises and index, it checks this table to 
> determine whether an initial build task is necessary. Since 10127, this check 
> expects the row to include just the index name, but the actual row will still 
> be in the old {{table.index}} format, causing the index manager to assume the 
> index is not built and submit a build task. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-11065) null pointer exception in CassandraDaemon.java:195

[Vassil Lunchev (JIRA)]

Vassil Lunchev created CASSANDRA-11065:
--

 Summary: null pointer exception in CassandraDaemon.java:195
 Key: CASSANDRA-11065
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11065
 Project: Cassandra
  Issue Type: Bug
Reporter: Vassil Lunchev
Priority: Minor
 Fix For: 3.0.1


I had a keyspace called 'tests'. I dropped it. Then I checked some nodes and 
one of them still had that keyspace 'tests'. On a node that still has the 
dropped keyspace I ran:
nodetools repair tests;

In the system logs of another node that did not have keyspace 'tests' I am 
seeing a null pointer exception:

ERROR [AntiEntropyStage:2] 2016-01-25 15:02:46,323 
RepairMessageVerbHandler.java:161 - Got error, removing parent repair session
ERROR [AntiEntropyStage:2] 2016-01-25 15:02:46,324 CassandraDaemon.java:195 - 
Exception in thread Thread[AntiEntropyStage:2,5,main]
java.lang.RuntimeException: java.lang.NullPointerException
at 
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:164)
 ~[apache-cassandra-3.0.1.jar:3.0.1]
at 
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) 
~[apache-cassandra-3.0.1.jar:3.0.1]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
~[na:1.8.0_66-internal]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
~[na:1.8.0_66-internal]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_66-internal]
Caused by: java.lang.NullPointerException: null
at 
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:69)
 ~[apache-cassandra-3.0.1.jar:3.0.1]
... 4 common frames omitted



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8093) Add Metrics for OS and hardware usage with Sigar

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115180#comment-15115180
 ] 

Joshua McKenzie commented on CASSANDRA-8093:


[~ifesdjeen]: Assigned to you. Thanks!

> Add Metrics for OS and hardware usage with Sigar
> 
>
> Key: CASSANDRA-8093
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8093
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: T Jake Luciani
>Assignee: Alex Petrov
> Fix For: 3.x
>
>
> Now that CASSANDRA-7838 is in we can expose many OS and hardware specific 
> metrics which will be useful for operators who want a holistic view of their 
> nodes from Cassandra.
> - Disk
> - CPU
> - Memory
> - Network



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11065) null pointer exception in CassandraDaemon.java:195

[Vassil Lunchev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vassil Lunchev updated CASSANDRA-11065:
---
Description: 
I had a keyspace called 'tests'. I dropped it. Then I checked some nodes and 
one of them still had that keyspace 'tests'. On a node that still has the 
dropped keyspace I ran:
nodetools repair tests;

In the system logs of another node that did not have keyspace 'tests' I am 
seeing a null pointer exception:

{code:java}
ERROR [AntiEntropyStage:2] 2016-01-25 15:02:46,323 
RepairMessageVerbHandler.java:161 - Got error, removing parent repair session
ERROR [AntiEntropyStage:2] 2016-01-25 15:02:46,324 CassandraDaemon.java:195 - 
Exception in thread Thread[AntiEntropyStage:2,5,main]
java.lang.RuntimeException: java.lang.NullPointerException
at 
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:164)
 ~[apache-cassandra-3.0.1.jar:3.0.1]
at 
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) 
~[apache-cassandra-3.0.1.jar:3.0.1]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
~[na:1.8.0_66-internal]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
~[na:1.8.0_66-internal]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_66-internal]
Caused by: java.lang.NullPointerException: null
at 
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:69)
 ~[apache-cassandra-3.0.1.jar:3.0.1]
... 4 common frames omitted
{code}

The error appears every time I run:
nodetools repair tests;

  was:
I had a keyspace called 'tests'. I dropped it. Then I checked some nodes and 
one of them still had that keyspace 'tests'. On a node that still has the 
dropped keyspace I ran:
nodetools repair tests;

In the system logs of another node that did not have keyspace 'tests' I am 
seeing a null pointer exception:

{code:java}
ERROR [AntiEntropyStage:2] 2016-01-25 15:02:46,323 
RepairMessageVerbHandler.java:161 - Got error, removing parent repair session
ERROR [AntiEntropyStage:2] 2016-01-25 15:02:46,324 CassandraDaemon.java:195 - 
Exception in thread Thread[AntiEntropyStage:2,5,main]
java.lang.RuntimeException: java.lang.NullPointerException
at 
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:164)
 ~[apache-cassandra-3.0.1.jar:3.0.1]
at 
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) 
~[apache-cassandra-3.0.1.jar:3.0.1]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
~[na:1.8.0_66-internal]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
~[na:1.8.0_66-internal]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_66-internal]
Caused by: java.lang.NullPointerException: null
at 
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:69)
 ~[apache-cassandra-3.0.1.jar:3.0.1]
... 4 common frames omitted
{code}



> null pointer exception in CassandraDaemon.java:195
> --
>
> Key: CASSANDRA-11065
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11065
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Vassil Lunchev
>Priority: Minor
> Fix For: 3.0.1
>
>
> I had a keyspace called 'tests'. I dropped it. Then I checked some nodes and 
> one of them still had that keyspace 'tests'. On a node that still has the 
> dropped keyspace I ran:
> nodetools repair tests;
> In the system logs of another node that did not have keyspace 'tests' I am 
> seeing a null pointer exception:
> {code:java}
> ERROR [AntiEntropyStage:2] 2016-01-25 15:02:46,323 
> RepairMessageVerbHandler.java:161 - Got error, removing parent repair session
> ERROR [AntiEntropyStage:2] 2016-01-25 15:02:46,324 CassandraDaemon.java:195 - 
> Exception in thread Thread[AntiEntropyStage:2,5,main]
> java.lang.RuntimeException: java.lang.NullPointerException
>   at 
> org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:164)
>  ~[apache-cassandra-3.0.1.jar:3.0.1]
>   at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) 
> ~[apache-cassandra-3.0.1.jar:3.0.1]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_66-internal]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  ~[na:1.8.0_66-internal]
>   at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_66-internal]
> Caused by: java.lang.NullPointerException: null
>   at 
> 

[jira] [Commented] (CASSANDRA-8180) Optimize disk seek using min/max column name meta data when the LIMIT clause is used

[Branimir Lambov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115132#comment-15115132
 ] 

Branimir Lambov commented on CASSANDRA-8180:


bq. iter.partitionLevelDeletion() is currently called for all sstables by 
queryMemtableAndDiskInternal()

That is, your code is currently completely disabled by the [{{if (... iterator 
!= null) return 
null;}}|https://github.com/stef1927/cassandra/commit/d5cfc6fd56d50eda5d9c510591bae1d66e17ec59#diff-78de604e500e1cc63c6d53a2ac6d6d65R52]
 check in {{lowerBound()}}? Is there a reason to want the code committed then?


bq. We can replace BTreeRow.emptyRow(ret) with new 
RangeTombstoneBoundMarker(RangeTombstone.Bound.inclusiveOpen(filter.isReversed(),
 ret.getRawValues()), DeletionTime.LIVE) if there is still a valid reason and 
ideally a failing test would be useful.

It is not as easy to fully break it as I was expecting, but in the presence of 
tombstones you can still break a basic feature of the iterators -- the 
inequality of the returned elements. A test that does
{code}
createTable("CREATE TABLE %s (a int, b int, c text, primary key (a, 
b))");
execute("INSERT INTO %s (a, b, c) VALUES(1, 1, '1')");
execute("INSERT INTO %s (a, b, c) VALUES(1, 3, '3')");
execute("DELETE FROM %s where a=1 and b >= 2 and b <= 3");
execute("INSERT INTO %s (a, b, c) VALUES(1, 2, '2')");
flush();

execute("DELETE FROM %s where a=1 and b >= 2 and b <= 3");
flush();

execute("SELECT * FROM %s WHERE a = ?", 1);
{code}
will end up with an iterator that lists two tombstone markers with equal 
clustering. Unfortunately that's filtered out before being returned so it's not 
trivial to write a test that checks this. On the other hand, an assertion in 
{{UnfilteredIteratorWithLowerBound}} that saves the returned {{lowerBound()}} 
and checks the first {{next()}} to be greater-than or equal may be something we 
want to have anyway.


> Optimize disk seek using min/max column name meta data when the LIMIT clause 
> is used
> 
>
> Key: CASSANDRA-8180
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8180
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
> Environment: Cassandra 2.0.10
>Reporter: DOAN DuyHai
>Assignee: Stefania
>Priority: Minor
> Fix For: 3.x
>
> Attachments: 8180_001.yaml, 8180_002.yaml
>
>
> I was working on an example of sensor data table (timeseries) and face a use 
> case where C* does not optimize read on disk.
> {code}
> cqlsh:test> CREATE TABLE test(id int, col int, val text, PRIMARY KEY(id,col)) 
> WITH CLUSTERING ORDER BY (col DESC);
> cqlsh:test> INSERT INTO test(id, col , val ) VALUES ( 1, 10, '10');
> ...
> >nodetool flush test test
> ...
> cqlsh:test> INSERT INTO test(id, col , val ) VALUES ( 1, 20, '20');
> ...
> >nodetool flush test test
> ...
> cqlsh:test> INSERT INTO test(id, col , val ) VALUES ( 1, 30, '30');
> ...
> >nodetool flush test test
> {code}
> After that, I activate request tracing:
> {code}
> cqlsh:test> SELECT * FROM test WHERE id=1 LIMIT 1;
>  activity  | 
> timestamp| source| source_elapsed
> ---+--+---+
> execute_cql3_query | 
> 23:48:46,498 | 127.0.0.1 |  0
> Parsing SELECT * FROM test WHERE id=1 LIMIT 1; | 
> 23:48:46,498 | 127.0.0.1 | 74
>Preparing statement | 
> 23:48:46,499 | 127.0.0.1 |253
>   Executing single-partition query on test | 
> 23:48:46,499 | 127.0.0.1 |930
>   Acquiring sstable references | 
> 23:48:46,499 | 127.0.0.1 |943
>Merging memtable tombstones | 
> 23:48:46,499 | 127.0.0.1 |   1032
>Key cache hit for sstable 3 | 
> 23:48:46,500 | 127.0.0.1 |   1160
>Seeking to partition beginning in data file | 
> 23:48:46,500 | 127.0.0.1 |   1173
>Key cache hit for sstable 2 | 
> 23:48:46,500 | 127.0.0.1 |   1889
>Seeking to partition beginning in data file | 
> 23:48:46,500 | 127.0.0.1 |   1901
>Key cache hit for sstable 1 | 
> 23:48:46,501 | 127.0.0.1 |   2373
>   

[jira] [Commented] (CASSANDRA-11064) Failed aggregate creation breaks server permanently

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115243#comment-15115243
 ] 

Robert Stupp commented on CASSANDRA-11064:
--

Hm - usage of composite types is definitely untested. IMO we should prevent the 
use of custom-types in UDAs. WDYT?
Despite that, it's a bug that invalid {{initcond}} values make it into the 
schema.

> Failed aggregate creation breaks server permanently
> ---
>
> Key: CASSANDRA-11064
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11064
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Olivier Michallat
>Assignee: Robert Stupp
>
> While testing edge cases around aggregates, I tried the following to see if 
> custom types were supported:
> {code}
> ccm create v321 -v3.2.1 -n3
> ccm updateconf enable_user_defined_functions:true
> ccm start
> ccm node1 cqlsh
> CREATE FUNCTION id(i 'DynamicCompositeType(s => UTF8Type, i => Int32Type)')
> RETURNS NULL ON NULL INPUT
> RETURNS 'DynamicCompositeType(s => UTF8Type, i => Int32Type)'
> LANGUAGE java
> AS 'return i;';
> // function created successfully
> CREATE AGGREGATE ag()
> SFUNC id
> STYPE 'DynamicCompositeType(s => UTF8Type, i => Int32Type)'
> INITCOND 's@foo:i@32';
> ServerError:  message="java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.exceptions.SyntaxException: Failed parsing CQL term: 
> [s@foo:i@32] reason: SyntaxException line 1:1 no viable alternative at 
> character '@'">{code}
> Despite the error, the aggregate appears in system tables:
> {code}
> select * from system_schema.aggregates;
>  keyspace_name | aggregate_name | ...
> ---++ ...
>   test | ag | ...
> {code}
> But you can't drop it, and trying to drop its function produces the server 
> error again:
> {code}
> DROP AGGREGATE ag;
> InvalidRequest: code=2200 [Invalid query] message="Cannot drop non existing 
> aggregate 'test.ag'"
> DROP FUNCTION id;
> ServerError:  message="java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.exceptions.SyntaxException: Failed parsing CQL term: 
> [s@foo:i@32] reason: SyntaxException line 1:1 no viable alternative at 
> character '@'">
> {code}
> What's worse, it's now impossible to restart the server:
> {code}
> ccm stop; ccm start
> org.apache.cassandra.exceptions.SyntaxException: Failed parsing CQL term: 
> [s@foo:i@32] reason: SyntaxException line 1:1 no viable alternative at 
> character '@'
>   at 
> org.apache.cassandra.cql3.CQLFragmentParser.parseAny(CQLFragmentParser.java:48)
>   at org.apache.cassandra.cql3.Terms.asBytes(Terms.java:51)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.createUDAFromRow(SchemaKeyspace.java:1225)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchUDAs(SchemaKeyspace.java:1204)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchFunctions(SchemaKeyspace.java:1129)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:897)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:872)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:860)
>   at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:125)
>   at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:115)
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:229)
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:551)
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:680)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11064) Failed aggregate creation breaks server permanently

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115283#comment-15115283
 ] 

Sylvain Lebresne commented on CASSANDRA-11064:
--

bq. IMO we should prevent the use of custom-types in UDAs.

Why? What's the reasoning? I don't see a good reason why UDA would have to 
assume anything about types and hence see no reason to prevent their use. In 
fact, I'd say that having them not working might hint at some reasonable 
assumption made by the implementation and we should fix the root cause. In this 
example in particular, I'm not sure why the code is trying to parse within what 
is clearly a CQL string (note that I'm not saying the example of the 
description _should_ work, I believe it shouldn't because strings are not valid 
literals for a custom type (you'd have to pass a blob), only that the error 
itself seems suspicious).

bq. it's a bug that invalid initcond values make it into the schema

Right, that's _the_ bug.

> Failed aggregate creation breaks server permanently
> ---
>
> Key: CASSANDRA-11064
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11064
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Olivier Michallat
>Assignee: Robert Stupp
>
> While testing edge cases around aggregates, I tried the following to see if 
> custom types were supported:
> {code}
> ccm create v321 -v3.2.1 -n3
> ccm updateconf enable_user_defined_functions:true
> ccm start
> ccm node1 cqlsh
> CREATE FUNCTION id(i 'DynamicCompositeType(s => UTF8Type, i => Int32Type)')
> RETURNS NULL ON NULL INPUT
> RETURNS 'DynamicCompositeType(s => UTF8Type, i => Int32Type)'
> LANGUAGE java
> AS 'return i;';
> // function created successfully
> CREATE AGGREGATE ag()
> SFUNC id
> STYPE 'DynamicCompositeType(s => UTF8Type, i => Int32Type)'
> INITCOND 's@foo:i@32';
> ServerError:  message="java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.exceptions.SyntaxException: Failed parsing CQL term: 
> [s@foo:i@32] reason: SyntaxException line 1:1 no viable alternative at 
> character '@'">{code}
> Despite the error, the aggregate appears in system tables:
> {code}
> select * from system_schema.aggregates;
>  keyspace_name | aggregate_name | ...
> ---++ ...
>   test | ag | ...
> {code}
> But you can't drop it, and trying to drop its function produces the server 
> error again:
> {code}
> DROP AGGREGATE ag;
> InvalidRequest: code=2200 [Invalid query] message="Cannot drop non existing 
> aggregate 'test.ag'"
> DROP FUNCTION id;
> ServerError:  message="java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.exceptions.SyntaxException: Failed parsing CQL term: 
> [s@foo:i@32] reason: SyntaxException line 1:1 no viable alternative at 
> character '@'">
> {code}
> What's worse, it's now impossible to restart the server:
> {code}
> ccm stop; ccm start
> org.apache.cassandra.exceptions.SyntaxException: Failed parsing CQL term: 
> [s@foo:i@32] reason: SyntaxException line 1:1 no viable alternative at 
> character '@'
>   at 
> org.apache.cassandra.cql3.CQLFragmentParser.parseAny(CQLFragmentParser.java:48)
>   at org.apache.cassandra.cql3.Terms.asBytes(Terms.java:51)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.createUDAFromRow(SchemaKeyspace.java:1225)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchUDAs(SchemaKeyspace.java:1204)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchFunctions(SchemaKeyspace.java:1129)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:897)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:872)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:860)
>   at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:125)
>   at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:115)
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:229)
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:551)
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:680)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11064) Failed aggregate creation breaks server permanently

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115291#comment-15115291
 ] 

Robert Stupp commented on CASSANDRA-11064:
--

bq. What's the reasoning?

I think it's difficult to handle (Custom)CompositeTypes via the Java driver as 
you have to deal with byte buffers/arrays. I was also thinking of the idea to 
get rid of AbstractType at all - still supporting "officially" might prevent 
that (maybe I got that wrong).


> Failed aggregate creation breaks server permanently
> ---
>
> Key: CASSANDRA-11064
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11064
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Olivier Michallat
>Assignee: Robert Stupp
>
> While testing edge cases around aggregates, I tried the following to see if 
> custom types were supported:
> {code}
> ccm create v321 -v3.2.1 -n3
> ccm updateconf enable_user_defined_functions:true
> ccm start
> ccm node1 cqlsh
> CREATE FUNCTION id(i 'DynamicCompositeType(s => UTF8Type, i => Int32Type)')
> RETURNS NULL ON NULL INPUT
> RETURNS 'DynamicCompositeType(s => UTF8Type, i => Int32Type)'
> LANGUAGE java
> AS 'return i;';
> // function created successfully
> CREATE AGGREGATE ag()
> SFUNC id
> STYPE 'DynamicCompositeType(s => UTF8Type, i => Int32Type)'
> INITCOND 's@foo:i@32';
> ServerError:  message="java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.exceptions.SyntaxException: Failed parsing CQL term: 
> [s@foo:i@32] reason: SyntaxException line 1:1 no viable alternative at 
> character '@'">{code}
> Despite the error, the aggregate appears in system tables:
> {code}
> select * from system_schema.aggregates;
>  keyspace_name | aggregate_name | ...
> ---++ ...
>   test | ag | ...
> {code}
> But you can't drop it, and trying to drop its function produces the server 
> error again:
> {code}
> DROP AGGREGATE ag;
> InvalidRequest: code=2200 [Invalid query] message="Cannot drop non existing 
> aggregate 'test.ag'"
> DROP FUNCTION id;
> ServerError:  message="java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.exceptions.SyntaxException: Failed parsing CQL term: 
> [s@foo:i@32] reason: SyntaxException line 1:1 no viable alternative at 
> character '@'">
> {code}
> What's worse, it's now impossible to restart the server:
> {code}
> ccm stop; ccm start
> org.apache.cassandra.exceptions.SyntaxException: Failed parsing CQL term: 
> [s@foo:i@32] reason: SyntaxException line 1:1 no viable alternative at 
> character '@'
>   at 
> org.apache.cassandra.cql3.CQLFragmentParser.parseAny(CQLFragmentParser.java:48)
>   at org.apache.cassandra.cql3.Terms.asBytes(Terms.java:51)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.createUDAFromRow(SchemaKeyspace.java:1225)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchUDAs(SchemaKeyspace.java:1204)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchFunctions(SchemaKeyspace.java:1129)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:897)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:872)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:860)
>   at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:125)
>   at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:115)
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:229)
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:551)
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:680)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10050) Secondary Index Performance Dependent on TokenRange Searched in Analytics

[Jonathan Ellis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115370#comment-15115370
 ] 

Jonathan Ellis commented on CASSANDRA-10050:


Is there a compelling reason to use key order, or is just one of those things 
that seemed like a good idea at the time?

Predicate pushdown for scans is basically the only reason to use 2i, so we 
should probably fix this.

> Secondary Index Performance Dependent on TokenRange Searched in Analytics
> -
>
> Key: CASSANDRA-10050
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10050
> Project: Cassandra
>  Issue Type: Improvement
> Environment: Single node, macbook, 2.1.8
>Reporter: Russell Alexander Spitzer
> Fix For: 3.x
>
>
> In doing some test work on the Spark Cassandra Connector I saw some odd 
> performance when pushing down range queries with Secondary Index filters. 
> When running the queries we see huge amount of time when the C* server is not 
> doing any work and the query seem to be hanging. This investigation led to 
> the work in this document
> https://docs.google.com/spreadsheets/d/1aJg3KX7nPnY77RJ9ZT-IfaYADgJh0A--nAxItvC6hb4/edit#gid=0
> The Spark Cassandra Connector builds up token range specific queries and 
> allows the user to pushdown relevant fields to C*. Here we have two indexed 
> fields (size) and (color) being pushed down to C*. 
> {code}
> SELECT count(*) FROM ks.tab WHERE token("store") > $min AND token("store") <= 
> $max AND color = 'red' AND size = 'P' ALLOW FILTERING;{code}
> These queries will have different token ranges inserted and executed as 
> separate spark tasks. Spark tasks with token ranges near the Min(token) end 
> up executing much faster than those near Max(token) which also happen to 
> through errors.
> {code}
> Coordinator node timed out waiting for replica nodes' responses] 
> message="Operation timed out - received only 0 responses." 
> info={'received_responses': 0, 'required_responses': 1, 'consistency': 'ONE'}
> {code}
> I took the queries and ran them through CQLSH to see the difference in time. 
> A linear relationship is seen based on where the tokenRange being queried is 
> starting with only 2 second for queries near the beginning of the full token 
> spectrum and over 12 seconds at the end of the spectrum. 
> The question is, can this behavior be improved? or should we not recommend 
> using secondary indexes with Analytics workloads?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11065) null pointer exception in CassandraDaemon.java:195

[Vassil Lunchev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vassil Lunchev updated CASSANDRA-11065:
---
Description: 
I had a keyspace called 'tests'. I dropped it. Then I checked some nodes and 
one of them still had that keyspace 'tests'. On a node that still has the 
dropped keyspace I ran:
nodetools repair tests;

In the system logs of another node that did not have keyspace 'tests' I am 
seeing a null pointer exception:

{code:java}
ERROR [AntiEntropyStage:2] 2016-01-25 15:02:46,323 
RepairMessageVerbHandler.java:161 - Got error, removing parent repair session
ERROR [AntiEntropyStage:2] 2016-01-25 15:02:46,324 CassandraDaemon.java:195 - 
Exception in thread Thread[AntiEntropyStage:2,5,main]
java.lang.RuntimeException: java.lang.NullPointerException
at 
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:164)
 ~[apache-cassandra-3.0.1.jar:3.0.1]
at 
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) 
~[apache-cassandra-3.0.1.jar:3.0.1]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
~[na:1.8.0_66-internal]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
~[na:1.8.0_66-internal]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_66-internal]
Caused by: java.lang.NullPointerException: null
at 
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:69)
 ~[apache-cassandra-3.0.1.jar:3.0.1]
... 4 common frames omitted
{code}

The error appears every time I run:
nodetools repair tests;

I can see it in the logs of all nodes, including the node on which I run the 
repair.

  was:
I had a keyspace called 'tests'. I dropped it. Then I checked some nodes and 
one of them still had that keyspace 'tests'. On a node that still has the 
dropped keyspace I ran:
nodetools repair tests;

In the system logs of another node that did not have keyspace 'tests' I am 
seeing a null pointer exception:

{code:java}
ERROR [AntiEntropyStage:2] 2016-01-25 15:02:46,323 
RepairMessageVerbHandler.java:161 - Got error, removing parent repair session
ERROR [AntiEntropyStage:2] 2016-01-25 15:02:46,324 CassandraDaemon.java:195 - 
Exception in thread Thread[AntiEntropyStage:2,5,main]
java.lang.RuntimeException: java.lang.NullPointerException
at 
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:164)
 ~[apache-cassandra-3.0.1.jar:3.0.1]
at 
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) 
~[apache-cassandra-3.0.1.jar:3.0.1]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
~[na:1.8.0_66-internal]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
~[na:1.8.0_66-internal]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_66-internal]
Caused by: java.lang.NullPointerException: null
at 
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:69)
 ~[apache-cassandra-3.0.1.jar:3.0.1]
... 4 common frames omitted
{code}

The error appears every time I run:
nodetools repair tests;


> null pointer exception in CassandraDaemon.java:195
> --
>
> Key: CASSANDRA-11065
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11065
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Vassil Lunchev
>Priority: Minor
> Fix For: 3.0.1
>
>
> I had a keyspace called 'tests'. I dropped it. Then I checked some nodes and 
> one of them still had that keyspace 'tests'. On a node that still has the 
> dropped keyspace I ran:
> nodetools repair tests;
> In the system logs of another node that did not have keyspace 'tests' I am 
> seeing a null pointer exception:
> {code:java}
> ERROR [AntiEntropyStage:2] 2016-01-25 15:02:46,323 
> RepairMessageVerbHandler.java:161 - Got error, removing parent repair session
> ERROR [AntiEntropyStage:2] 2016-01-25 15:02:46,324 CassandraDaemon.java:195 - 
> Exception in thread Thread[AntiEntropyStage:2,5,main]
> java.lang.RuntimeException: java.lang.NullPointerException
>   at 
> org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:164)
>  ~[apache-cassandra-3.0.1.jar:3.0.1]
>   at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) 
> ~[apache-cassandra-3.0.1.jar:3.0.1]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_66-internal]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  ~[na:1.8.0_66-internal]
>   at java.lang.Thread.run(Thread.java:745) 

[jira] [Updated] (CASSANDRA-11065) null pointer exception in CassandraDaemon.java:195

[Vassil Lunchev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vassil Lunchev updated CASSANDRA-11065:
---
Fix Version/s: (was: 3.0.1)
  Description: 
Running Cassandra 3.0.1 installed from apt-get on debian.

I had a keyspace called 'tests'. I dropped it. Then I checked some nodes and 
one of them still had that keyspace 'tests'. On a node that still has the 
dropped keyspace I ran:
nodetools repair tests;

In the system logs of another node that did not have keyspace 'tests' I am 
seeing a null pointer exception:

{code:java}
ERROR [AntiEntropyStage:2] 2016-01-25 15:02:46,323 
RepairMessageVerbHandler.java:161 - Got error, removing parent repair session
ERROR [AntiEntropyStage:2] 2016-01-25 15:02:46,324 CassandraDaemon.java:195 - 
Exception in thread Thread[AntiEntropyStage:2,5,main]
java.lang.RuntimeException: java.lang.NullPointerException
at 
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:164)
 ~[apache-cassandra-3.0.1.jar:3.0.1]
at 
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) 
~[apache-cassandra-3.0.1.jar:3.0.1]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
~[na:1.8.0_66-internal]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
~[na:1.8.0_66-internal]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_66-internal]
Caused by: java.lang.NullPointerException: null
at 
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:69)
 ~[apache-cassandra-3.0.1.jar:3.0.1]
... 4 common frames omitted
{code}

The error appears every time I run:
nodetools repair tests;

I can see it in the logs of all nodes, including the node on which I run the 
repair.

  was:
I had a keyspace called 'tests'. I dropped it. Then I checked some nodes and 
one of them still had that keyspace 'tests'. On a node that still has the 
dropped keyspace I ran:
nodetools repair tests;

In the system logs of another node that did not have keyspace 'tests' I am 
seeing a null pointer exception:

{code:java}
ERROR [AntiEntropyStage:2] 2016-01-25 15:02:46,323 
RepairMessageVerbHandler.java:161 - Got error, removing parent repair session
ERROR [AntiEntropyStage:2] 2016-01-25 15:02:46,324 CassandraDaemon.java:195 - 
Exception in thread Thread[AntiEntropyStage:2,5,main]
java.lang.RuntimeException: java.lang.NullPointerException
at 
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:164)
 ~[apache-cassandra-3.0.1.jar:3.0.1]
at 
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) 
~[apache-cassandra-3.0.1.jar:3.0.1]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
~[na:1.8.0_66-internal]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
~[na:1.8.0_66-internal]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_66-internal]
Caused by: java.lang.NullPointerException: null
at 
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:69)
 ~[apache-cassandra-3.0.1.jar:3.0.1]
... 4 common frames omitted
{code}

The error appears every time I run:
nodetools repair tests;

I can see it in the logs of all nodes, including the node on which I run the 
repair.


> null pointer exception in CassandraDaemon.java:195
> --
>
> Key: CASSANDRA-11065
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11065
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Vassil Lunchev
>Priority: Minor
>
> Running Cassandra 3.0.1 installed from apt-get on debian.
> I had a keyspace called 'tests'. I dropped it. Then I checked some nodes and 
> one of them still had that keyspace 'tests'. On a node that still has the 
> dropped keyspace I ran:
> nodetools repair tests;
> In the system logs of another node that did not have keyspace 'tests' I am 
> seeing a null pointer exception:
> {code:java}
> ERROR [AntiEntropyStage:2] 2016-01-25 15:02:46,323 
> RepairMessageVerbHandler.java:161 - Got error, removing parent repair session
> ERROR [AntiEntropyStage:2] 2016-01-25 15:02:46,324 CassandraDaemon.java:195 - 
> Exception in thread Thread[AntiEntropyStage:2,5,main]
> java.lang.RuntimeException: java.lang.NullPointerException
>   at 
> org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:164)
>  ~[apache-cassandra-3.0.1.jar:3.0.1]
>   at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) 
> ~[apache-cassandra-3.0.1.jar:3.0.1]
>   at 
> 

[jira] [Assigned] (CASSANDRA-11064) Failed aggregate creation breaks server permanently

[Robert Stupp (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp reassigned CASSANDRA-11064:


Assignee: Robert Stupp

> Failed aggregate creation breaks server permanently
> ---
>
> Key: CASSANDRA-11064
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11064
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Olivier Michallat
>Assignee: Robert Stupp
>
> While testing edge cases around aggregates, I tried the following to see if 
> custom types were supported:
> {code}
> ccm create v321 -v3.2.1 -n3
> ccm updateconf enable_user_defined_functions:true
> ccm start
> ccm node1 cqlsh
> CREATE FUNCTION id(i 'DynamicCompositeType(s => UTF8Type, i => Int32Type)')
> RETURNS NULL ON NULL INPUT
> RETURNS 'DynamicCompositeType(s => UTF8Type, i => Int32Type)'
> LANGUAGE java
> AS 'return i;';
> // function created successfully
> CREATE AGGREGATE ag()
> SFUNC id
> STYPE 'DynamicCompositeType(s => UTF8Type, i => Int32Type)'
> INITCOND 's@foo:i@32';
> ServerError:  message="java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.exceptions.SyntaxException: Failed parsing CQL term: 
> [s@foo:i@32] reason: SyntaxException line 1:1 no viable alternative at 
> character '@'">{code}
> Despite the error, the aggregate appears in system tables:
> {code}
> select * from system_schema.aggregates;
>  keyspace_name | aggregate_name | ...
> ---++ ...
>   test | ag | ...
> {code}
> But you can't drop it, and trying to drop its function produces the server 
> error again:
> {code}
> DROP AGGREGATE ag;
> InvalidRequest: code=2200 [Invalid query] message="Cannot drop non existing 
> aggregate 'test.ag'"
> DROP FUNCTION id;
> ServerError:  message="java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.exceptions.SyntaxException: Failed parsing CQL term: 
> [s@foo:i@32] reason: SyntaxException line 1:1 no viable alternative at 
> character '@'">
> {code}
> What's worse, it's now impossible to restart the server:
> {code}
> ccm stop; ccm start
> org.apache.cassandra.exceptions.SyntaxException: Failed parsing CQL term: 
> [s@foo:i@32] reason: SyntaxException line 1:1 no viable alternative at 
> character '@'
>   at 
> org.apache.cassandra.cql3.CQLFragmentParser.parseAny(CQLFragmentParser.java:48)
>   at org.apache.cassandra.cql3.Terms.asBytes(Terms.java:51)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.createUDAFromRow(SchemaKeyspace.java:1225)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchUDAs(SchemaKeyspace.java:1204)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchFunctions(SchemaKeyspace.java:1129)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:897)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:872)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:860)
>   at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:125)
>   at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:115)
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:229)
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:551)
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:680)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7281) SELECT on tuple relations are broken for mixed ASC/DESC clustering order

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115316#comment-15115316
 ] 

Benjamin Lerer commented on CASSANDRA-7281:
---

Thanks for the work.

The code freeze for 3.3 will be next monday. By consequence, end of week will 
probably be too late for the review process.
I will take it over. 


> SELECT on tuple relations are broken for mixed ASC/DESC clustering order
> 
>
> Key: CASSANDRA-7281
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7281
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Marcin Szymaniuk
> Fix For: 3.3, 2.2.x, 3.0.x
>
> Attachments: 
> 0001-CASSANDRA-7281-SELECT-on-tuple-relations-are-broken-.patch, 
> 0001-CASSANDRA-7281-SELECT-on-tuple-relations-are-broken-v2.patch, 
> 0001-CASSANDRA-7281-SELECT-on-tuple-relations-are-broken-v3.patch, 
> 0001-CASSANDRA-7281-SELECT-on-tuple-relations-are-broken-v4.patch, 
> 0001-CASSANDRA-7281-SELECT-on-tuple-relations-are-broken-v5.patch, 
> 0001-Some-tests-for-CASSANDRA-7281.patch, 7281_unit_tests.txt
>
>
> As noted on 
> [CASSANDRA-6875|https://issues.apache.org/jira/browse/CASSANDRA-6875?focusedCommentId=13992153=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13992153],
>  the tuple notation is broken when the clustering order mixes ASC and DESC 
> directives because the range of data they describe don't correspond to a 
> single continuous slice internally. To copy the example from CASSANDRA-6875:
> {noformat}
> cqlsh:ks> create table foo (a int, b int, c int, PRIMARY KEY (a, b, c)) WITH 
> CLUSTERING ORDER BY (b DESC, c ASC);
> cqlsh:ks> INSERT INTO foo (a, b, c) VALUES (0, 2, 0);
> cqlsh:ks> INSERT INTO foo (a, b, c) VALUES (0, 1, 0);
> cqlsh:ks> INSERT INTO foo (a, b, c) VALUES (0, 1, 1);
> cqlsh:ks> INSERT INTO foo (a, b, c) VALUES (0, 0, 0);
> cqlsh:ks> SELECT * FROM foo WHERE a=0;
>  a | b | c
> ---+---+---
>  0 | 2 | 0
>  0 | 1 | 0
>  0 | 1 | 1
>  0 | 0 | 0
> (4 rows)
> cqlsh:ks> SELECT * FROM foo WHERE a=0 AND (b, c) > (1, 0);
>  a | b | c
> ---+---+---
>  0 | 2 | 0
> (1 rows)
> {noformat}
> The last query should really return {{(0, 2, 0)}} and {{(0, 1, 1)}}.
> For that specific example we should generate 2 internal slices, but I believe 
> that with more clustering columns we may have more slices.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8093) Add Metrics for OS and hardware usage with Sigar

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-8093:
---
Assignee: Alex Petrov

> Add Metrics for OS and hardware usage with Sigar
> 
>
> Key: CASSANDRA-8093
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8093
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: T Jake Luciani
>Assignee: Alex Petrov
> Fix For: 3.x
>
>
> Now that CASSANDRA-7838 is in we can expose many OS and hardware specific 
> metrics which will be useful for operators who want a holistic view of their 
> nodes from Cassandra.
> - Disk
> - CPU
> - Memory
> - Network



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9633) Add ability to encrypt sstables

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115270#comment-15115270
 ] 

Jason Brown commented on CASSANDRA-9633:


Pushed a first-pass implementation 
[here|https://github.com/apache/cassandra/compare/trunk...jasobrown:9633]

This submission does piggy back off the {{ICompressor}} interface, and plugs in 
reasonably well. Due to the stateful nature of {{EncryptingCompressor}} (see 
[the 
note|https://github.com/jasobrown/cassandra/blob/9633/src/java/org/apache/cassandra/io/compress/EncryptingCompressor.java#L47]
 at the top of the class file), it needs a unqiue copy of the compressor when 
creating new SSTables; hence the reason for 
{{CompressionParams.copyWithFreshCompressor()}}. I'm open to sugesstions on 
improving this.

Through testing this, I uncovered some inefficiencies with {{CipherFactory}} 
(added with CASSANDRA-9945), so I refactored it a bit to cache instances of 
{{KeyProvider}}. Encrypted commit logs (CASSANDRA-6018) and hints 
(CASSANDRA-11040) are not really affected by this refactor.

One oddity, though, will be for users when they do want to encrypt a table; 
they will have to set the compression options on the table - which doesn't map, 
in an obvious manner, to encryption. I'm wide open to suggestions on how to 
improve the "UI" for this; however, I'd be a bit reticent to create a parallel 
set of of classes similar to what we have for compression, as I suspect 
encryption won't be as widely employed as compression (and I'm not sure the 
extra infrastructure is worth it). 

> Add ability to encrypt sstables
> ---
>
> Key: CASSANDRA-9633
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9633
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Jason Brown
>Assignee: Jason Brown
>  Labels: encryption, security, sstable
> Fix For: 3.x
>
>
> Add option to allow encrypting of sstables.
> I have a version of this functionality built on cassandra 2.0 that 
> piggy-backs on the existing sstable compression functionality and ICompressor 
> interface (similar in nature to what DataStax Enterprise does). However, if 
> we're adding the feature to the main OSS product, I'm not sure if we want to 
> use the pluggable compression framework or if it's worth investigating a 
> different path. I think there's a lot of upside in reusing the sstable 
> compression scheme, but perhaps add a new component in cqlsh for table 
> encryption and a corresponding field in CFMD.
> Encryption configuration in the yaml can use the same mechanism as 
> CASSANDRA-6018 (which is currently pending internal review).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11064) Failed aggregate creation breaks server permanently

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115306#comment-15115306
 ] 

Sylvain Lebresne commented on CASSANDRA-11064:
--

But we're talking of UDA here right? Custom types are not gonna anywhere 
anytime soon and there is no plan that I'm aware of removing them (which 
doesn't mean we can change how they are defined, so we do have some flexibility 
regarding AbstractType). This goes for their usability: having to deal with 
blobs is not more difficult for UDA than for any other place in CQL.
 
Don't get me wrong, I'm not saying their support is the most important thing 
ever. But they are part of CQL and are supported anywhere else so excluding 
them from UDA now would be inconsistent. More importantly, it's not the problem 
here so let's focus on the true issue: that validation of {{INITCONT}} is 
broken.

> Failed aggregate creation breaks server permanently
> ---
>
> Key: CASSANDRA-11064
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11064
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Olivier Michallat
>Assignee: Robert Stupp
>
> While testing edge cases around aggregates, I tried the following to see if 
> custom types were supported:
> {code}
> ccm create v321 -v3.2.1 -n3
> ccm updateconf enable_user_defined_functions:true
> ccm start
> ccm node1 cqlsh
> CREATE FUNCTION id(i 'DynamicCompositeType(s => UTF8Type, i => Int32Type)')
> RETURNS NULL ON NULL INPUT
> RETURNS 'DynamicCompositeType(s => UTF8Type, i => Int32Type)'
> LANGUAGE java
> AS 'return i;';
> // function created successfully
> CREATE AGGREGATE ag()
> SFUNC id
> STYPE 'DynamicCompositeType(s => UTF8Type, i => Int32Type)'
> INITCOND 's@foo:i@32';
> ServerError:  message="java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.exceptions.SyntaxException: Failed parsing CQL term: 
> [s@foo:i@32] reason: SyntaxException line 1:1 no viable alternative at 
> character '@'">{code}
> Despite the error, the aggregate appears in system tables:
> {code}
> select * from system_schema.aggregates;
>  keyspace_name | aggregate_name | ...
> ---++ ...
>   test | ag | ...
> {code}
> But you can't drop it, and trying to drop its function produces the server 
> error again:
> {code}
> DROP AGGREGATE ag;
> InvalidRequest: code=2200 [Invalid query] message="Cannot drop non existing 
> aggregate 'test.ag'"
> DROP FUNCTION id;
> ServerError:  message="java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.exceptions.SyntaxException: Failed parsing CQL term: 
> [s@foo:i@32] reason: SyntaxException line 1:1 no viable alternative at 
> character '@'">
> {code}
> What's worse, it's now impossible to restart the server:
> {code}
> ccm stop; ccm start
> org.apache.cassandra.exceptions.SyntaxException: Failed parsing CQL term: 
> [s@foo:i@32] reason: SyntaxException line 1:1 no viable alternative at 
> character '@'
>   at 
> org.apache.cassandra.cql3.CQLFragmentParser.parseAny(CQLFragmentParser.java:48)
>   at org.apache.cassandra.cql3.Terms.asBytes(Terms.java:51)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.createUDAFromRow(SchemaKeyspace.java:1225)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchUDAs(SchemaKeyspace.java:1204)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchFunctions(SchemaKeyspace.java:1129)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:897)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:872)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:860)
>   at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:125)
>   at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:115)
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:229)
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:551)
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:680)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11045) Legacy SSTables for KEYS indexes can't be read in 3.0+

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-11045:
-
Reviewer: Sylvain Lebresne

> Legacy SSTables for KEYS indexes can't be read in 3.0+
> --
>
> Key: CASSANDRA-11045
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11045
> Project: Cassandra
>  Issue Type: Bug
>  Components: Distributed Metadata, Local Write-Read Paths
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Critical
> Fix For: 3.0.x, 3.x
>
>
> Trying to read an SSTable for a KEYS index created in 2.1 throws a 
> deserialization error when being read following an upgrade to 3.0. This 
> occurs whether the read comes via the query path or by running {{nodetool 
> upgradesstables}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11046) Existing indexes are always rebuilt on upgrade to 3.0

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-11046:
-
Reviewer: T Jake Luciani

> Existing indexes are always rebuilt on upgrade to 3.0
> -
>
> Key: CASSANDRA-11046
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11046
> Project: Cassandra
>  Issue Type: Bug
>  Components: Distributed Metadata
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Critical
> Fix For: 3.0.x, 3.x
>
>
> CASSANDRA-10127 made the naming of secondary indexes consistent internally. 
> Previously some places used just the index name, whilst others required the 
> name in the format {{table.index}}. 
> One place including the additional table name was the {{system.IndexInfo}} 
> table, which records whether or not the index has been built. On upgrade to 
> 3.0, as the node restarts and initialises and index, it checks this table to 
> determine whether an initial build task is necessary. Since 10127, this check 
> expects the row to include just the index name, but the actual row will still 
> be in the old {{table.index}} format, causing the index manager to assume the 
> index is not built and submit a build task. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-11064) Failed aggregate creation breaks server permanently

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115337#comment-15115337
 ] 

Robert Stupp edited comment on CASSANDRA-11064 at 1/25/16 3:09 PM:
---

Got the bug. In CreateAggregateStatement, the initcond is validated by parsing 
the "raw" CQL value (in this case {{'s@foo:i@32'}}). But during creation of the 
schema-migration-mutation (which should succeed, as it has passed the 
validation) we re-construct the CQL term from the binary representation, which 
simply calls {{AbstractType.getString}} for all custom types - and this returns 
{{s@foo:i@32}} (without the single quotes). Will check & fix that.


was (Author: snazy):
Got the bug. In CreateAggregateStatement, the initcond is validated by parsing 
the "raw" CQL value (in this case {{'s@foo:i@32'}}). But during creation of the 
schema-migration-mutation (which should succeed, as it has passed the 
validation) we re-construct the CQL term from the binary representation, which 
simply calls {{AbstractType.getString}} for all custom types. Will check & fix 
that.

> Failed aggregate creation breaks server permanently
> ---
>
> Key: CASSANDRA-11064
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11064
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Olivier Michallat
>Assignee: Robert Stupp
>
> While testing edge cases around aggregates, I tried the following to see if 
> custom types were supported:
> {code}
> ccm create v321 -v3.2.1 -n3
> ccm updateconf enable_user_defined_functions:true
> ccm start
> ccm node1 cqlsh
> CREATE FUNCTION id(i 'DynamicCompositeType(s => UTF8Type, i => Int32Type)')
> RETURNS NULL ON NULL INPUT
> RETURNS 'DynamicCompositeType(s => UTF8Type, i => Int32Type)'
> LANGUAGE java
> AS 'return i;';
> // function created successfully
> CREATE AGGREGATE ag()
> SFUNC id
> STYPE 'DynamicCompositeType(s => UTF8Type, i => Int32Type)'
> INITCOND 's@foo:i@32';
> ServerError:  message="java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.exceptions.SyntaxException: Failed parsing CQL term: 
> [s@foo:i@32] reason: SyntaxException line 1:1 no viable alternative at 
> character '@'">{code}
> Despite the error, the aggregate appears in system tables:
> {code}
> select * from system_schema.aggregates;
>  keyspace_name | aggregate_name | ...
> ---++ ...
>   test | ag | ...
> {code}
> But you can't drop it, and trying to drop its function produces the server 
> error again:
> {code}
> DROP AGGREGATE ag;
> InvalidRequest: code=2200 [Invalid query] message="Cannot drop non existing 
> aggregate 'test.ag'"
> DROP FUNCTION id;
> ServerError:  message="java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.exceptions.SyntaxException: Failed parsing CQL term: 
> [s@foo:i@32] reason: SyntaxException line 1:1 no viable alternative at 
> character '@'">
> {code}
> What's worse, it's now impossible to restart the server:
> {code}
> ccm stop; ccm start
> org.apache.cassandra.exceptions.SyntaxException: Failed parsing CQL term: 
> [s@foo:i@32] reason: SyntaxException line 1:1 no viable alternative at 
> character '@'
>   at 
> org.apache.cassandra.cql3.CQLFragmentParser.parseAny(CQLFragmentParser.java:48)
>   at org.apache.cassandra.cql3.Terms.asBytes(Terms.java:51)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.createUDAFromRow(SchemaKeyspace.java:1225)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchUDAs(SchemaKeyspace.java:1204)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchFunctions(SchemaKeyspace.java:1129)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:897)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:872)
>   at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:860)
>   at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:125)
>   at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:115)
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:229)
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:551)
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:680)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-11064) Failed aggregate creation breaks server permanently

[Olivier Michallat (JIRA)]

Olivier Michallat created CASSANDRA-11064:
-

 Summary: Failed aggregate creation breaks server permanently
 Key: CASSANDRA-11064
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11064
 Project: Cassandra
  Issue Type: Bug
Reporter: Olivier Michallat


While testing edge cases around aggregates, I tried the following to see if 
custom types were supported:

{code}
ccm create v321 -v3.2.1 -n3
ccm updateconf enable_user_defined_functions:true
ccm start
ccm node1 cqlsh

CREATE FUNCTION id(i 'DynamicCompositeType(s => UTF8Type, i => Int32Type)')
RETURNS NULL ON NULL INPUT
RETURNS 'DynamicCompositeType(s => UTF8Type, i => Int32Type)'
LANGUAGE java
AS 'return i;';
// function created successfully

CREATE AGGREGATE ag()
SFUNC id
STYPE 'DynamicCompositeType(s => UTF8Type, i => Int32Type)'
INITCOND 's@foo:i@32';

ServerError: {code}

Despite the error, the aggregate appears in system tables:
{code}
select * from system_schema.aggregates;
 keyspace_name | aggregate_name | ...
---++ ...
  test | ag | ...
{code}

But you can't drop it, and trying to drop its function produces the server 
error again:
{code}
DROP AGGREGATE ag;
InvalidRequest: code=2200 [Invalid query] message="Cannot drop non existing 
aggregate 'test.ag'"

DROP FUNCTION id;
ServerError: 
{code}

What's worse, it's now impossible to restart the server:
{code}
ccm stop; ccm start

org.apache.cassandra.exceptions.SyntaxException: Failed parsing CQL term: 
[s@foo:i@32] reason: SyntaxException line 1:1 no viable alternative at 
character '@'
at 
org.apache.cassandra.cql3.CQLFragmentParser.parseAny(CQLFragmentParser.java:48)
at org.apache.cassandra.cql3.Terms.asBytes(Terms.java:51)
at 
org.apache.cassandra.schema.SchemaKeyspace.createUDAFromRow(SchemaKeyspace.java:1225)
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchUDAs(SchemaKeyspace.java:1204)
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchFunctions(SchemaKeyspace.java:1129)
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:897)
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:872)
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:860)
at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:125)
at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:115)
at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:229)
at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:551)
at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:680)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11046) Existing indexes are always rebuilt on upgrade to 3.0

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115353#comment-15115353
 ] 

T Jake Luciani commented on CASSANDRA-11046:


[~beobal] makes sense to me, can you post test run links?

> Existing indexes are always rebuilt on upgrade to 3.0
> -
>
> Key: CASSANDRA-11046
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11046
> Project: Cassandra
>  Issue Type: Bug
>  Components: Distributed Metadata
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Critical
> Fix For: 3.0.x, 3.x
>
>
> CASSANDRA-10127 made the naming of secondary indexes consistent internally. 
> Previously some places used just the index name, whilst others required the 
> name in the format {{table.index}}. 
> One place including the additional table name was the {{system.IndexInfo}} 
> table, which records whether or not the index has been built. On upgrade to 
> 3.0, as the node restarts and initialises and index, it checks this table to 
> determine whether an initial build task is necessary. Since 10127, this check 
> expects the row to include just the index name, but the actual row will still 
> be in the old {{table.index}} format, causing the index manager to assume the 
> index is not built and submit a build task. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8093) Add Metrics for OS and hardware usage with Sigar

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115440#comment-15115440
 ] 

Aleksey Yeschenko commented on CASSANDRA-8093:
--

FWIW last time this discussion came up there was opposition - including from me 
- on the grounds that exposing this sort of metrics is not a responsibility of 
Cassandra.

> Add Metrics for OS and hardware usage with Sigar
> 
>
> Key: CASSANDRA-8093
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8093
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: T Jake Luciani
>Assignee: Alex Petrov
> Fix For: 3.x
>
>
> Now that CASSANDRA-7838 is in we can expose many OS and hardware specific 
> metrics which will be useful for operators who want a holistic view of their 
> nodes from Cassandra.
> - Disk
> - CPU
> - Memory
> - Network



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-11067) Improve SASI syntax

[Jonathan Ellis (JIRA)]

Jonathan Ellis created CASSANDRA-11067:
--

 Summary: Improve SASI syntax
 Key: CASSANDRA-11067
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11067
 Project: Cassandra
  Issue Type: Task
  Components: CQL
Reporter: Jonathan Ellis
Assignee: Sam Tunnicliffe


I think everyone agrees that a LIKE operator would be ideal, but that's 
probably not in scope for an initial 3.4 release.

Still, I'm uncomfortable with the initial approach of overloading = to mean 
"satisfies index expression."  The problem is that it will be very difficult to 
back out of this behavior once people are using it.

I propose adding a new operator in the interim instead.  Call it SLIKE, maybe.  
With the exact same behavior that SASI currently exposes, just with a separate 
operator rather than being rolled into =.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11067) Improve SASI syntax

[Jonathan Ellis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115458#comment-15115458
 ] 

Jonathan Ellis commented on CASSANDRA-11067:


/cc [~xedin] [~jrwest]

> Improve SASI syntax
> ---
>
> Key: CASSANDRA-11067
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11067
> Project: Cassandra
>  Issue Type: Task
>  Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Sam Tunnicliffe
>
> I think everyone agrees that a LIKE operator would be ideal, but that's 
> probably not in scope for an initial 3.4 release.
> Still, I'm uncomfortable with the initial approach of overloading = to mean 
> "satisfies index expression."  The problem is that it will be very difficult 
> to back out of this behavior once people are using it.
> I propose adding a new operator in the interim instead.  Call it SLIKE, 
> maybe.  With the exact same behavior that SASI currently exposes, just with a 
> separate operator rather than being rolled into =.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11067) Improve SASI syntax

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-11067:
---
Description: 
I think everyone agrees that a LIKE operator would be ideal, but that's 
probably not in scope for an initial 3.4 release.

Still, I'm uncomfortable with the initial approach of overloading = to mean 
"satisfies index expression."  The problem is that it will be very difficult to 
back out of this behavior once people are using it.

I propose adding a new operator in the interim instead.  Call it MATCHES, 
maybe.  With the exact same behavior that SASI currently exposes, just with a 
separate operator rather than being rolled into =.

  was:
I think everyone agrees that a LIKE operator would be ideal, but that's 
probably not in scope for an initial 3.4 release.

Still, I'm uncomfortable with the initial approach of overloading = to mean 
"satisfies index expression."  The problem is that it will be very difficult to 
back out of this behavior once people are using it.

I propose adding a new operator in the interim instead.  Call it SLIKE, maybe.  
With the exact same behavior that SASI currently exposes, just with a separate 
operator rather than being rolled into =.


> Improve SASI syntax
> ---
>
> Key: CASSANDRA-11067
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11067
> Project: Cassandra
>  Issue Type: Task
>  Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Sam Tunnicliffe
>
> I think everyone agrees that a LIKE operator would be ideal, but that's 
> probably not in scope for an initial 3.4 release.
> Still, I'm uncomfortable with the initial approach of overloading = to mean 
> "satisfies index expression."  The problem is that it will be very difficult 
> to back out of this behavior once people are using it.
> I propose adding a new operator in the interim instead.  Call it MATCHES, 
> maybe.  With the exact same behavior that SASI currently exposes, just with a 
> separate operator rather than being rolled into =.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11068) Entire row is compacted away if remaining cells are tombstones expiring after gc_grace_seconds

[Andy Tolbert (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andy Tolbert updated CASSANDRA-11068:
-
Description: 
Assuming the following schema:

{code}
CREATE TABLE simple.data (
k text PRIMARY KEY,
v int
) WITH gc_grace_seconds = 300;
{code}

And the following queries:

{code}
insert into simple.data (k, v) values ('blah', 1);
delete v from simple.data where k='blah';
{code}

Performing a {{select *}} from this table will return 1 row with a null value:

{code}
cqlsh> select * from simple.data;

 k | v
---+-
  blah |null
{code}

Prior the 3.0, if I were to do a flush, the sstable representation of this 
table would include an empty cell and a tombstone:

{code}
[
{"key": "blah",
 "cells": [["","",1453747038457027],
   ["v",1453747112,1453747112383096,"d"]]}
]
{code}

As my gc_grace_seconds value is 300, if I wait 5 minutes and perform a 
compaction, the new sstable would omit the tombstone, but the empty cell would 
still be present:

{code}
[
{"key": "blah",
 "cells": [["","",1453747038457027]]}
]
{code}

Performing the {{select *}} query would still yield the same result because of 
this.

However, in 3.2.1 this does not seem to be the behavior, after deleting the 'v' 
cell, performing a flush and then waiting 5 minutes and doing a compact, what 
ends up happening is that the sstable completely disappears (presumably because 
there is no remaining data) and the select query emits 0 rows:

{code}
cqlsh> select * from simple.data;

 k | v
---+-

(0 rows)
{code}

I'm unsure if this is by design or a bug, but it does represent a change 
between C* versions.

I have not tried this for a table with clustering columns yet, but I assume 
that the behavior will be the same.

  was:
Assuming the following schema:

{code}
CREATE TABLE simple.data (
k text PRIMARY KEY,
v int
) WITH gc_grace_seconds = 300;
{code}

And the following queries:

{code}
insert into simple.data (k, v) values ('blah', 1);
delete reading from simple.data where k='blah';
{code}

Performing a {{select *}} from this table will return 1 row with a null value:

{code}
cqlsh> select * from simple.data;

 k | v
---+-
  blah |null
{code}

Prior the 3.0, if I were to do a flush, the sstable representation of this 
table would include an empty cell and a tombstone:

{code}
[
{"key": "blah",
 "cells": [["","",1453747038457027],
   ["v",1453747112,1453747112383096,"d"]]}
]
{code}

As my gc_grace_seconds value is 300, if I wait 5 minutes and perform a 
compaction, the new sstable would omit the tombstone, but the empty cell would 
still be present:

{code}
[
{"key": "blah",
 "cells": [["","",1453747038457027]]}
]
{code}

Performing the {{select *}} query would still yield the same result because of 
this.

However, in 3.2.1 this does not seem to be the behavior, after deleting the 'v' 
cell, performing a flush and then waiting 5 minutes and doing a compact, what 
ends up happening is that the sstable completely disappears (presumably because 
there is no remaining data) and the select query emits 0 rows:

{code}
cqlsh> select * from simple.data;

 k | v
---+-

(0 rows)
{code}

I'm unsure if this is by design or a bug, but it does represent a change 
between C* versions.

I have not tried this for a table with clustering columns yet, but I assume 
that the behavior will be the same.


> Entire row is compacted away if remaining cells are tombstones expiring after 
> gc_grace_seconds
> --
>
> Key: CASSANDRA-11068
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11068
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Andy Tolbert
>
> Assuming the following schema:
> {code}
> CREATE TABLE simple.data (
> k text PRIMARY KEY,
> v int
> ) WITH gc_grace_seconds = 300;
> {code}
> And the following queries:
> {code}
> insert into simple.data (k, v) values ('blah', 1);
> delete v from simple.data where k='blah';
> {code}
> Performing a {{select *}} from this table will return 1 row with a null value:
> {code}
> cqlsh> select * from simple.data;
>  k | v
> ---+-
>   blah |null
> {code}
> Prior the 3.0, if I were to do a flush, the sstable representation of this 
> table would include an empty cell and a tombstone:
> {code}
> [
> {"key": "blah",
>  "cells": [["","",1453747038457027],
>["v",1453747112,1453747112383096,"d"]]}
> ]
> {code}
> As my gc_grace_seconds value is 300, if I wait 5 minutes and perform a 
> compaction, the new sstable would omit the tombstone, but the empty cell 
> would still be present:
> {code}
> [
> {"key": "blah",
>  "cells": [["","",1453747038457027]]}
> ]
> {code}
> 

[jira] [Updated] (CASSANDRA-11068) Entire row is compacted away if remaining cells are tombstones expiring after gc_grace_seconds

[Andy Tolbert (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andy Tolbert updated CASSANDRA-11068:
-
Description: 
Assuming the following schema:

{code}
CREATE TABLE simple.data (
k text PRIMARY KEY,
v int
) WITH gc_grace_seconds = 300;
{code}

And the following queries:

{code}
insert into simple.data (k, v) values ('blah', 1);
delete v from simple.data where k='blah';
{code}

Performing a {{select *}} from this table will return 1 row with a null value:

{code}
cqlsh> select * from simple.data;

 k | v
---+-
  blah |null
{code}

Prior the 3.0, if I were to do a flush, the sstable representation of this 
table would include an empty cell and a tombstone:

{code}
[
{"key": "blah",
 "cells": [["","",1453747038457027],
   ["v",1453747112,1453747112383096,"d"]]}
]
{code}

As my gc_grace_seconds value is 300, if I wait 5 minutes and perform a 
compaction, the new sstable would omit the tombstone, but the empty cell would 
still be present:

{code}
[
{"key": "blah",
 "cells": [["","",1453747038457027]]}
]
{code}

Performing the {{select *}} query would still yield the same result because of 
this.

However, in 3.2.1 this does not seem to be the behavior, after deleting the 'v' 
cell, performing a flush and then waiting 5 minutes and doing a compact, what 
ends up happening is that the sstable completely disappears (presumably because 
there is no remaining data) and the select query emits 0 rows:

{code}
cqlsh> select * from simple.data;

 k | v
---+-

(0 rows)
{code}

I'm unsure if this is by design or a bug, but it does represent a change 
between C* versions.

-I have not tried this for a table with clustering columns yet, but I assume 
that the behavior will be the same.- (The problem only manifests for tables 
with no clustering columns).

  was:
Assuming the following schema:

{code}
CREATE TABLE simple.data (
k text PRIMARY KEY,
v int
) WITH gc_grace_seconds = 300;
{code}

And the following queries:

{code}
insert into simple.data (k, v) values ('blah', 1);
delete v from simple.data where k='blah';
{code}

Performing a {{select *}} from this table will return 1 row with a null value:

{code}
cqlsh> select * from simple.data;

 k | v
---+-
  blah |null
{code}

Prior the 3.0, if I were to do a flush, the sstable representation of this 
table would include an empty cell and a tombstone:

{code}
[
{"key": "blah",
 "cells": [["","",1453747038457027],
   ["v",1453747112,1453747112383096,"d"]]}
]
{code}

As my gc_grace_seconds value is 300, if I wait 5 minutes and perform a 
compaction, the new sstable would omit the tombstone, but the empty cell would 
still be present:

{code}
[
{"key": "blah",
 "cells": [["","",1453747038457027]]}
]
{code}

Performing the {{select *}} query would still yield the same result because of 
this.

However, in 3.2.1 this does not seem to be the behavior, after deleting the 'v' 
cell, performing a flush and then waiting 5 minutes and doing a compact, what 
ends up happening is that the sstable completely disappears (presumably because 
there is no remaining data) and the select query emits 0 rows:

{code}
cqlsh> select * from simple.data;

 k | v
---+-

(0 rows)
{code}

I'm unsure if this is by design or a bug, but it does represent a change 
between C* versions.

I have not tried this for a table with clustering columns yet, but I assume 
that the behavior will be the same.


> Entire row is compacted away if remaining cells are tombstones expiring after 
> gc_grace_seconds
> --
>
> Key: CASSANDRA-11068
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11068
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Andy Tolbert
>
> Assuming the following schema:
> {code}
> CREATE TABLE simple.data (
> k text PRIMARY KEY,
> v int
> ) WITH gc_grace_seconds = 300;
> {code}
> And the following queries:
> {code}
> insert into simple.data (k, v) values ('blah', 1);
> delete v from simple.data where k='blah';
> {code}
> Performing a {{select *}} from this table will return 1 row with a null value:
> {code}
> cqlsh> select * from simple.data;
>  k | v
> ---+-
>   blah |null
> {code}
> Prior the 3.0, if I were to do a flush, the sstable representation of this 
> table would include an empty cell and a tombstone:
> {code}
> [
> {"key": "blah",
>  "cells": [["","",1453747038457027],
>["v",1453747112,1453747112383096,"d"]]}
> ]
> {code}
> As my gc_grace_seconds value is 300, if I wait 5 minutes and perform a 
> compaction, the new sstable would omit the tombstone, but the empty cell 
> would still be present:
> {code}
> [
> {"key": 

[jira] [Updated] (CASSANDRA-10134) Always require replace_address to replace existing address

[Joel Knighton (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Knighton updated CASSANDRA-10134:
--
Reviewer: Joel Knighton

> Always require replace_address to replace existing address
> --
>
> Key: CASSANDRA-10134
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10134
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Distributed Metadata
>Reporter: Tyler Hobbs
>Assignee: Stefania
> Fix For: 3.x
>
>
> Normally, when a node is started from a clean state with the same address as 
> an existing down node, it will fail to start with an error like this:
> {noformat}
> ERROR [main] 2015-08-19 15:07:51,577 CassandraDaemon.java:554 - Exception 
> encountered during startup
> java.lang.RuntimeException: A node with address /127.0.0.3 already exists, 
> cancelling join. Use cassandra.replace_address if you want to replace this 
> node.
>   at 
> org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:543)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:783)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:720)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:611)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378) 
> [main/:na]
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:537)
>  [main/:na]
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:626) 
> [main/:na]
> {noformat}
> However, if {{auto_bootstrap}} is set to false or the node is in its own seed 
> list, it will not throw this error and will start normally.  The new node 
> then takes over the host ID of the old node (even if the tokens are 
> different), and the only message you will see is a warning in the other 
> nodes' logs:
> {noformat}
> logger.warn("Changing {}'s host ID from {} to {}", endpoint, storedId, 
> hostId);
> {noformat}
> This could cause an operator to accidentally wipe out the token information 
> for a down node without replacing it.  To fix this, we should check for an 
> endpoint collision even if {{auto_bootstrap}} is false or the node is a seed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8093) Add Metrics for OS and hardware usage with Sigar

[Alex Petrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115453#comment-15115453
 ] 

Alex Petrov commented on CASSANDRA-8093:


I kind of had the same feeling although didn't trust myself.

Might be better if something like DS agent had that functionality.
On Mon, 25 Jan 2016 at 16:57, Aleksey Yeschenko (JIRA) 

-- 
Alex


> Add Metrics for OS and hardware usage with Sigar
> 
>
> Key: CASSANDRA-8093
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8093
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: T Jake Luciani
>Assignee: Alex Petrov
> Fix For: 3.x
>
>
> Now that CASSANDRA-7838 is in we can expose many OS and hardware specific 
> metrics which will be useful for operators who want a holistic view of their 
> nodes from Cassandra.
> - Disk
> - CPU
> - Memory
> - Network



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8093) Add Metrics for OS and hardware usage with Sigar

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115451#comment-15115451
 ] 

T Jake Luciani commented on CASSANDRA-8093:
---

Yeah, over a year later I'm not sure this makes sense vs getting these from 
collectd.

If there are specific metrics that we can add which help C* operators and 
require OS metrics as part of the calculation then we should consider those.
One Idea might be to track other processes which may be running on the box 
causing load.  Also tracking the process size according to the OS which 
includes all the off heap stuff.

> Add Metrics for OS and hardware usage with Sigar
> 
>
> Key: CASSANDRA-8093
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8093
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: T Jake Luciani
>Assignee: Alex Petrov
> Fix For: 3.x
>
>
> Now that CASSANDRA-7838 is in we can expose many OS and hardware specific 
> metrics which will be useful for operators who want a holistic view of their 
> nodes from Cassandra.
> - Disk
> - CPU
> - Memory
> - Network



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11045) Legacy SSTables for KEYS indexes can't be read in 3.0+

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115403#comment-15115403
 ] 

Sylvain Lebresne commented on CASSANDRA-11045:
--

+1 (pending test results of course)

> Legacy SSTables for KEYS indexes can't be read in 3.0+
> --
>
> Key: CASSANDRA-11045
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11045
> Project: Cassandra
>  Issue Type: Bug
>  Components: Distributed Metadata, Local Write-Read Paths
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Critical
> Fix For: 3.0.x, 3.x
>
>
> Trying to read an SSTable for a KEYS index created in 2.1 throws a 
> deserialization error when being read following an upgrade to 3.0. This 
> occurs whether the read comes via the query path or by running {{nodetool 
> upgradesstables}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11067) Improve SASI syntax

[Jonathan Ellis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115546#comment-15115546
 ] 

Jonathan Ellis commented on CASSANDRA-11067:


If we can get a LIKE approach in for 3.4, so much the better.  I'm saying that 
I don't think overloading = is acceptable to ship, and proposing the simplest 
possible alternative.

> Improve SASI syntax
> ---
>
> Key: CASSANDRA-11067
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11067
> Project: Cassandra
>  Issue Type: Task
>  Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Sam Tunnicliffe
>
> I think everyone agrees that a LIKE operator would be ideal, but that's 
> probably not in scope for an initial 3.4 release.
> Still, I'm uncomfortable with the initial approach of overloading = to mean 
> "satisfies index expression."  The problem is that it will be very difficult 
> to back out of this behavior once people are using it.
> I propose adding a new operator in the interim instead.  Call it MATCHES, 
> maybe.  With the exact same behavior that SASI currently exposes, just with a 
> separate operator rather than being rolled into =.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11046) Existing indexes are always rebuilt on upgrade to 3.0

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115545#comment-15115545
 ] 

T Jake Luciani commented on CASSANDRA-11046:


+1

> Existing indexes are always rebuilt on upgrade to 3.0
> -
>
> Key: CASSANDRA-11046
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11046
> Project: Cassandra
>  Issue Type: Bug
>  Components: Distributed Metadata
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Critical
> Fix For: 3.0.x, 3.x
>
>
> CASSANDRA-10127 made the naming of secondary indexes consistent internally. 
> Previously some places used just the index name, whilst others required the 
> name in the format {{table.index}}. 
> One place including the additional table name was the {{system.IndexInfo}} 
> table, which records whether or not the index has been built. On upgrade to 
> 3.0, as the node restarts and initialises and index, it checks this table to 
> determine whether an initial build task is necessary. Since 10127, this check 
> expects the row to include just the index name, but the actual row will still 
> be in the old {{table.index}} format, causing the index manager to assume the 
> index is not built and submit a build task. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11067) Improve SASI syntax

[Jack Krupansky (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115501#comment-15115501
 ] 

Jack Krupansky commented on CASSANDRA-11067:


How about a severely restricted LIKE that only permits patterns ending with a 
"%" for prefix query (LIKE 'J%') or with a "%" at either end for contains (LIKE 
'%abc%') or ending with a % for suffix query (LIKE '%smith')? Then it would be 
fully compatible with SQL.

In any case, "=" would then attempt an exact match using the SASI index? That 
would allow both exact and inexact matching for each column using a single 
index.

If we can't have this restricted LIKE, descriptive keyword operators like 
SUFFIX and PREFIX would seem desirable. Could the existing CONTAINS operator 
also be used? They would also handle the case where the prefix/suffix/contains 
string is a parameter - otherwise the user has to do a messy concat.

> Improve SASI syntax
> ---
>
> Key: CASSANDRA-11067
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11067
> Project: Cassandra
>  Issue Type: Task
>  Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Sam Tunnicliffe
>
> I think everyone agrees that a LIKE operator would be ideal, but that's 
> probably not in scope for an initial 3.4 release.
> Still, I'm uncomfortable with the initial approach of overloading = to mean 
> "satisfies index expression."  The problem is that it will be very difficult 
> to back out of this behavior once people are using it.
> I propose adding a new operator in the interim instead.  Call it MATCHES, 
> maybe.  With the exact same behavior that SASI currently exposes, just with a 
> separate operator rather than being rolled into =.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8093) Add Metrics for OS and hardware usage with Sigar

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115565#comment-15115565
 ] 

Joshua McKenzie commented on CASSANDRA-8093:


bq. FWIW last time this discussion came up there was opposition ... not a 
responsibility of Cassandra.
Now that you mention it, I do recall that. After thinking about this for a bit 
(rather than just assigning and running...): my .02 is that we should avoid the 
bottomless rabbit-hole of trying to either give people environmental 
information they can get through other (more mature) vectors or trying to 
prevent them from shooting themselves in the foot (re: other procs on box).

There are plenty of robust tools to handle the above; I'd rather users go to 
specialized tools rather than settling for a cobbled together version from us.

> Add Metrics for OS and hardware usage with Sigar
> 
>
> Key: CASSANDRA-8093
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8093
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: T Jake Luciani
>Assignee: Alex Petrov
> Fix For: 3.x
>
>
> Now that CASSANDRA-7838 is in we can expose many OS and hardware specific 
> metrics which will be useful for operators who want a holistic view of their 
> nodes from Cassandra.
> - Disk
> - CPU
> - Memory
> - Network



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-8093) Add Metrics for OS and hardware usage with Sigar

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne resolved CASSANDRA-8093.
-
Resolution: Won't Fix

Seems we all agree on what to do with this.

> Add Metrics for OS and hardware usage with Sigar
> 
>
> Key: CASSANDRA-8093
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8093
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: T Jake Luciani
>Assignee: Alex Petrov
> Fix For: 3.x
>
>
> Now that CASSANDRA-7838 is in we can expose many OS and hardware specific 
> metrics which will be useful for operators who want a holistic view of their 
> nodes from Cassandra.
> - Disk
> - CPU
> - Memory
> - Network



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11063) Unable to compute ceiling for max when histogram overflowed

[Navjyot Nishant (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115648#comment-15115648
 ] 

Navjyot Nishant commented on CASSANDRA-11063:
-

Thanks Marcus.. Yes we did identified this data modelling issue and wonking to 
fix this, but as you also mentioned this shouldn't fail like this anyways, do 
we have a workaround to fix this? Our system.log is full of this error.. Errors 
are unstoppable even when no compaction is running. At this point we just want 
to stop this spam.

> Unable to compute ceiling for max when histogram overflowed
> ---
>
> Key: CASSANDRA-11063
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11063
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: Cassandra 2.1.9 on RHEL
>Reporter: Navjyot Nishant
>  Labels: Compaction, thread
>
> Issue https://issues.apache.org/jira/browse/CASSANDRA-8028 seems related with 
> error we are getting. But we are getting this with Cassandra 2.1.9 when 
> autocompaction is running it keeps throwing following errors, we are unsure 
> if its a bug or can be resolved, please suggest.
> {code}
> WARN  [CompactionExecutor:3] 2016-01-23 13:30:40,907 SSTableWriter.java:240 - 
> Compacting large partition gccatlgsvcks/category_name_dedup:66611300 
> (138152195 bytes)
> ERROR [CompactionExecutor:1] 2016-01-23 13:30:50,267 CassandraDaemon.java:223 
> - Exception in thread Thread[CompactionExecutor:1,1,main]
> java.lang.IllegalStateException: Unable to compute ceiling for max when 
> histogram overflowed
> at 
> org.apache.cassandra.utils.EstimatedHistogram.mean(EstimatedHistogram.java:203)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.sstable.metadata.StatsMetadata.getEstimatedDroppableTombstoneRatio(StatsMetadata.java:98)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.sstable.SSTableReader.getEstimatedDroppableTombstoneRatio(SSTableReader.java:1987)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.db.compaction.AbstractCompactionStrategy.worthDroppingTombstones(AbstractCompactionStrategy.java:370)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy.getNextBackgroundSSTables(SizeTieredCompactionStrategy.java:96)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy.getNextBackgroundTask(SizeTieredCompactionStrategy.java:179)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy.getNextBackgroundTask(WrappingCompactionStrategy.java:84)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:230)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> ~[na:1.7.0_51]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262) 
> ~[na:1.7.0_51]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  ~[na:1.7.0_51]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  [na:1.7.0_51]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
> {code}
> h3. Additional info:
> *cfstats is running fine for that table...*
> {code}
> ~ $ nodetool cfstats gccatlgsvcks.category_name_dedup
> Keyspace: gccatlgsvcks
> Read Count: 0
> Read Latency: NaN ms.
> Write Count: 0
> Write Latency: NaN ms.
> Pending Flushes: 0
> Table: category_name_dedup
> SSTable count: 6
> Space used (live): 836314727
> Space used (total): 836314727
> Space used by snapshots (total): 3621519
> Off heap memory used (total): 6930368
> SSTable Compression Ratio: 0.03725358753117693
> Number of keys (estimate): 3004
> Memtable cell count: 0
> Memtable data size: 0
> Memtable off heap memory used: 0
> Memtable switch count: 0
> Local read count: 0
> Local read latency: NaN ms
> Local write count: 0
> Local write latency: NaN ms
> Pending flushes: 0
> Bloom filter false positives: 0
> Bloom filter false ratio: 0.0
> Bloom filter space used: 5240
> Bloom filter off heap memory used: 5192
> Index summary off heap memory used: 1200
> 

[jira] [Updated] (CASSANDRA-11067) Improve SASI syntax

[Pavel Yaskevich (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Yaskevich updated CASSANDRA-11067:

Fix Version/s: 3.4

> Improve SASI syntax
> ---
>
> Key: CASSANDRA-11067
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11067
> Project: Cassandra
>  Issue Type: Task
>  Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Pavel Yaskevich
> Fix For: 3.4
>
>
> I think everyone agrees that a LIKE operator would be ideal, but that's 
> probably not in scope for an initial 3.4 release.
> Still, I'm uncomfortable with the initial approach of overloading = to mean 
> "satisfies index expression."  The problem is that it will be very difficult 
> to back out of this behavior once people are using it.
> I propose adding a new operator in the interim instead.  Call it MATCHES, 
> maybe.  With the exact same behavior that SASI currently exposes, just with a 
> separate operator rather than being rolled into =.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-11067) Improve SASI syntax

[Pavel Yaskevich (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Yaskevich reassigned CASSANDRA-11067:
---

Assignee: Pavel Yaskevich  (was: Sam Tunnicliffe)

> Improve SASI syntax
> ---
>
> Key: CASSANDRA-11067
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11067
> Project: Cassandra
>  Issue Type: Task
>  Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Pavel Yaskevich
> Fix For: 3.4
>
>
> I think everyone agrees that a LIKE operator would be ideal, but that's 
> probably not in scope for an initial 3.4 release.
> Still, I'm uncomfortable with the initial approach of overloading = to mean 
> "satisfies index expression."  The problem is that it will be very difficult 
> to back out of this behavior once people are using it.
> I propose adding a new operator in the interim instead.  Call it MATCHES, 
> maybe.  With the exact same behavior that SASI currently exposes, just with a 
> separate operator rather than being rolled into =.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8093) Add Metrics for OS and hardware usage with Sigar

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-8093:
-
Fix Version/s: (was: 3.x)

> Add Metrics for OS and hardware usage with Sigar
> 
>
> Key: CASSANDRA-8093
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8093
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: T Jake Luciani
>Assignee: Alex Petrov
>
> Now that CASSANDRA-7838 is in we can expose many OS and hardware specific 
> metrics which will be useful for operators who want a holistic view of their 
> nodes from Cassandra.
> - Disk
> - CPU
> - Memory
> - Network



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11067) Improve SASI syntax

[Pavel Yaskevich (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Yaskevich updated CASSANDRA-11067:

Reviewer: Sam Tunnicliffe

> Improve SASI syntax
> ---
>
> Key: CASSANDRA-11067
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11067
> Project: Cassandra
>  Issue Type: Task
>  Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Pavel Yaskevich
> Fix For: 3.4
>
>
> I think everyone agrees that a LIKE operator would be ideal, but that's 
> probably not in scope for an initial 3.4 release.
> Still, I'm uncomfortable with the initial approach of overloading = to mean 
> "satisfies index expression."  The problem is that it will be very difficult 
> to back out of this behavior once people are using it.
> I propose adding a new operator in the interim instead.  Call it MATCHES, 
> maybe.  With the exact same behavior that SASI currently exposes, just with a 
> separate operator rather than being rolled into =.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11067) Improve SASI syntax

[Pavel Yaskevich (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115734#comment-15115734
 ] 

Pavel Yaskevich commented on CASSANDRA-11067:
-

[~beobal] [~jbellis] I'm going to take on this since I was about it do it 
anyway, LIKE itself is not a big change for SASI so I would rather go with that 
than MATCHES, should be able to make it work pretty fast. 

> Improve SASI syntax
> ---
>
> Key: CASSANDRA-11067
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11067
> Project: Cassandra
>  Issue Type: Task
>  Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Sam Tunnicliffe
> Fix For: 3.4
>
>
> I think everyone agrees that a LIKE operator would be ideal, but that's 
> probably not in scope for an initial 3.4 release.
> Still, I'm uncomfortable with the initial approach of overloading = to mean 
> "satisfies index expression."  The problem is that it will be very difficult 
> to back out of this behavior once people are using it.
> I propose adding a new operator in the interim instead.  Call it MATCHES, 
> maybe.  With the exact same behavior that SASI currently exposes, just with a 
> separate operator rather than being rolled into =.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10855) Use Caffeine (W-TinyLFU) for on-heap caches

[Ben Manes (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115737#comment-15115737
 ] 

Ben Manes commented on CASSANDRA-10855:
---

[2.1.0|https://github.com/ben-manes/caffeine/releases/tag/v2.1.0] was released 
which includes the above mentioned optimizations. So the cache should seem 
artificially better for an artificial workload =)

I haven't had a chance to try to add more workload profiles to the Cassandra 
tool. It would be nice if we knew what real-world distributions were like, as 
Zipf-like is what researchers published. From the traces I've experimented 
with, I am fairly confident in a net positive result.

P.S. An 
[article|http://highscalability.com/blog/2016/1/25/design-of-a-modern-cache.html]
 on HighScalability describes the overall design.

> Use Caffeine (W-TinyLFU) for on-heap caches
> ---
>
> Key: CASSANDRA-10855
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10855
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Ben Manes
>  Labels: performance
>
> Cassandra currently uses 
> [ConcurrentLinkedHashMap|https://code.google.com/p/concurrentlinkedhashmap] 
> for performance critical caches (key, counter) and Guava's cache for 
> non-critical (auth, metrics, security). All of these usages have been 
> replaced by [Caffeine|https://github.com/ben-manes/caffeine], written by the 
> author of the previously mentioned libraries.
> The primary incentive is to switch from LRU policy to W-TinyLFU, which 
> provides [near optimal|https://github.com/ben-manes/caffeine/wiki/Efficiency] 
> hit rates. It performs particularly well in database and search traces, is 
> scan resistant, and as adds a very small time/space overhead to LRU.
> Secondarily, Guava's caches never obtained similar 
> [performance|https://github.com/ben-manes/caffeine/wiki/Benchmarks] to CLHM 
> due to some optimizations not being ported over. This change results in 
> faster reads and not creating garbage as a side-effect.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11067) Improve SASI syntax

[Jonathan Ellis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115742#comment-15115742
 ] 

Jonathan Ellis commented on CASSANDRA-11067:


Sounds good.  Thanks!

> Improve SASI syntax
> ---
>
> Key: CASSANDRA-11067
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11067
> Project: Cassandra
>  Issue Type: Task
>  Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Pavel Yaskevich
> Fix For: 3.4
>
>
> I think everyone agrees that a LIKE operator would be ideal, but that's 
> probably not in scope for an initial 3.4 release.
> Still, I'm uncomfortable with the initial approach of overloading = to mean 
> "satisfies index expression."  The problem is that it will be very difficult 
> to back out of this behavior once people are using it.
> I propose adding a new operator in the interim instead.  Call it MATCHES, 
> maybe.  With the exact same behavior that SASI currently exposes, just with a 
> separate operator rather than being rolled into =.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10779) Mutations do not block for completion under view lock contention

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115679#comment-15115679
 ] 

T Jake Luciani commented on CASSANDRA-10779:


I'm trying to explain the reason for non-blocking the Mutation stage.  For 
example this patch would block all non-mv mutations (including system 
keyspaces) if we encounter (write stage pool size MV writes on the same lock).

If we decide to go with the non completable future approach then I would 
suggest we reintroduce the MV Write stage. So we don't block non MV mutations.

> Mutations do not block for completion under view lock contention
> 
>
> Key: CASSANDRA-10779
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10779
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
> Environment: Windows 7 64-bit, Cassandra v3.0.0, Java 1.8u60
>Reporter: Will Zhang
>Assignee: Tyler Hobbs
> Fix For: 3.0.x, 3.x
>
>
> Hi guys,
> I encountered the following warning message when I was testing to upgrade 
> from v2.2.2 to v3.0.0. 
> It looks like a write time-out but in an uncaught exception. Could this be an 
> easy fix?
> Log file section below. Thank you!
> {code}
>   WARN  [SharedPool-Worker-64] 2015-11-26 14:04:24,678 
> AbstractTracingAwareExecutorService.java:169 - Uncaught exception on thread 
> Thread[SharedPool-Worker-64,10,main]: {}
> org.apache.cassandra.exceptions.WriteTimeoutException: Operation timed out - 
> received only 0 responses.
>   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:427) 
> ~[apache-cassandra-3.0.0.jar:3.0.0]
>   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:386) 
> ~[apache-cassandra-3.0.0.jar:3.0.0]
>   at org.apache.cassandra.db.Mutation.apply(Mutation.java:205) 
> ~[apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.db.Keyspace.lambda$apply$59(Keyspace.java:435) 
> ~[apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_60]
>   at 
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
>  ~[apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [apache-cassandra-3.0.0.jar:3.0.0]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
>   INFO  [IndexSummaryManager:1] 2015-11-26 14:41:10,527 
> IndexSummaryManager.java:257 - Redistributing index summaries
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10688) Stack overflow from SSTableReader$InstanceTidier.runOnClose in Leak Detector

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116092#comment-15116092
 ] 

Ariel Weisberg commented on CASSANDRA-10688:


I looked at doing this. I think it's more than just a simple stack because I am 
trying to bound the memory usage and not just pool. It actually still fails at 
that because {{path}} still allocates a large array, but at least it is 
proportionally much less than the memory allocated to {{InProgressVisit}}s. 
It's a bit finicky to try and clear the stack of things we don't want pooled at 
some point while also avoiding walking the entire thing to clear it frequently.

I would like to either remove pooling or leave it as is.

> Stack overflow from SSTableReader$InstanceTidier.runOnClose in Leak Detector
> 
>
> Key: CASSANDRA-10688
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10688
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths, Testing
>Reporter: Jeremiah Jordan
>Assignee: Ariel Weisberg
> Fix For: 3.0.x
>
>
> Running some tests against cassandra-3.0 
> 9fc957cf3097e54ccd72e51b2d0650dc3e83eae0
> The tests are just running cassandra-stress write and read while adding and 
> removing nodes from the cluster.  After the test runs when I go back through 
> logs I find the following Stackoverflow fairly often:
> ERROR [Strong-Reference-Leak-Detector:1] 2015-11-11 00:04:10,638  
> Ref.java:413 - Stackoverflow [private java.lang.Runnable 
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier.runOnClose,
>  final java.lang.Runnable 
> org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache.andThen, 
> final org.apache.cassandra.cache.InstrumentingCache 
> org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys.cache, private 
> final org.apache.cassandra.cache.ICache 
> org.apache.cassandra.cache.InstrumentingCache.map, private final 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap 
> org.apache.cassandra.cache.ConcurrentLinkedHashCache.map, final 
> com.googlecode.concurrentlinkedhashmap.LinkedDeque 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap.evictionDeque, 
> com.googlecode.concurrentlinkedhashmap.Linked 
> com.googlecode.concurrentlinkedhashmap.LinkedDeque.first, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> ... (repeated a whole bunch more)  
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> final java.lang.Object 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.key, 
> public final byte[] org.apache.cassandra.cache.KeyCacheKey.key



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10699) Make schema alterations strongly consistent

[Jack Krupansky (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116229#comment-15116229
 ] 

Jack Krupansky commented on CASSANDRA-10699:


Will resolution of this ticket enable concurrent clients to successfully 
perform CREATE TABLE IF NOT EXISTS? Or will that still be problematic? I just 
want to know if this is the ticket to point people to for concurrent CREATE 
TABLE IF NOT EXISTS issues.

In the mean time, should we update the doc to effectively say that concurrent 
CREATE TABLE IF NOT EXISTS is not supported and that it is the responsibility 
of the user to absolutely refrain from attempting any potentially concurrent 
attempts to CREATE TABLE IF NOT EXISTS for a given table?

A related doc issue is how the user can tell that the CREATE TABLE has 
successfully completed around the ring. IOW, if cqlsh returns success, is the 
table really created on all nodes? Is a nodetool tablestats a reliable check - 
if all nodes are listed then the CREATE TABLE has succeeded/completed on all 
nodes?


> Make schema alterations strongly consistent
> ---
>
> Key: CASSANDRA-10699
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10699
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Aleksey Yeschenko
> Fix For: 3.x
>
>
> Schema changes do not necessarily commute. This has been the case before 
> CASSANDRA-5202, but now is particularly problematic.
> We should employ a strongly consistent protocol instead of relying on 
> marshalling {{Mutation}} objects with schema changes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10928) SSTableExportTest.testExportColumnsWithMetadata randomly fails

[Brandon Williams (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-10928:
-
Reviewer: Jason Brown  (was: Joshua McKenzie)

> SSTableExportTest.testExportColumnsWithMetadata randomly fails
> --
>
> Key: CASSANDRA-10928
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10928
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Michael Kjellman
>Assignee: sankalp kohli
> Fix For: 2.1.12
>
> Attachments: CASSANDRA_10928_2.1.diff
>
>
> The SSTableExportTest.testExportColumnsWithMetadata test will randomly fail 
> (bogusly). Currently, the string check used won’t work if the JSON generated 
> happened to order the elements in the array differently.
> {code}
> assertEquals(
> "unexpected serialization format for topLevelDeletion",
> "{\"markedForDeleteAt\":0,\"localDeletionTime\":0}",
> serializedDeletionInfo.toJSONString());
> {code}
> {noformat}
> [junit] Testcase: 
> testExportColumnsWithMetadata(org.apache.cassandra.tools.SSTableExportTest):  
>   FAILED
> [junit] unexpected serialization format for topLevelDeletion 
> expected:<{"[markedForDeleteAt":0,"localDeletionTime]":0}> but 
> was:<{"[localDeletionTime":0,"markedForDeleteAt]":0}>
> [junit] junit.framework.AssertionFailedError: unexpected serialization 
> format for topLevelDeletion 
> expected:<{"[markedForDeleteAt":0,"localDeletionTime]":0}> but 
> was:<{"[localDeletionTime":0,"markedForDeleteAt]":0}>
> [junit]   at 
> org.apache.cassandra.tools.SSTableExportTest.testExportColumnsWithMetadata(SSTableExportTest.java:299)
> [junit]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11068) Entire row is compacted away if remaining cells are tombstones expiring after gc_grace_seconds

[Andy Tolbert (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andy Tolbert updated CASSANDRA-11068:
-
Reproduced In: 3.2.1

> Entire row is compacted away if remaining cells are tombstones expiring after 
> gc_grace_seconds
> --
>
> Key: CASSANDRA-11068
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11068
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Andy Tolbert
>
> Assuming the following schema:
> {code}
> CREATE TABLE simple.data (
> k text PRIMARY KEY,
> v int
> ) WITH gc_grace_seconds = 300;
> {code}
> And the following queries:
> {code}
> insert into simple.data (k, v) values ('blah', 1);
> delete reading from simple.data where k='blah';
> {code}
> Performing a {{select *}} from this table will return 1 row with a null value:
> {code}
> cqlsh> select * from simple.data;
>  k | v
> ---+-
>   blah |null
> {code}
> Prior the 3.0, if I were to do a flush, the sstable representation of this 
> table would include an empty cell and a tombstone:
> {code}
> [
> {"key": "blah",
>  "cells": [["","",1453747038457027],
>["v",1453747112,1453747112383096,"d"]]}
> ]
> {code}
> As my gc_grace_seconds value is 300, if I wait 5 minutes and perform a 
> compaction, the new sstable would omit the tombstone, but the empty cell 
> would still be present:
> {code}
> [
> {"key": "blah",
>  "cells": [["","",1453747038457027]]}
> ]
> {code}
> Performing the {{select *}} query would still yield the same result because 
> of this.
> However, in 3.2.1 this does not seem to be the behavior, after deleting the 
> 'v' cell, performing a flush and then waiting 5 minutes and doing a compact, 
> what ends up happening is that the sstable completely disappears (presumably 
> because there is no remaining data) and the select query emits 0 rows:
> {code}
> cqlsh> select * from simple.data;
>  k | v
> ---+-
> (0 rows)
> {code}
> I'm unsure if this is by design or a bug, but it does represent a change 
> between C* versions.
> I have not tried this for a table with clustering columns yet, but I assume 
> that the behavior will be the same.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-11068) Entire row is compacted away if remaining cells are tombstones expiring after gc_grace_seconds

[Andy Tolbert (JIRA)]

Andy Tolbert created CASSANDRA-11068:


 Summary: Entire row is compacted away if remaining cells are 
tombstones expiring after gc_grace_seconds
 Key: CASSANDRA-11068
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11068
 Project: Cassandra
  Issue Type: Bug
Reporter: Andy Tolbert


Assuming the following schema:

{code}
CREATE TABLE simple.data (
k text PRIMARY KEY,
v int
) WITH gc_grace_seconds = 300;
{code}

And the following queries:

{code}
insert into simple.data (k, v) values ('blah', 1);
delete reading from simple.data where k='blah';
{code}

Performing a {{select *}} from this table will return 1 row with a null value:

{code}
cqlsh> select * from simple.data;

 k | v
---+-
  blah |null
{code}

Prior the 3.0, if I were to do a flush, the sstable representation of this 
table would include an empty cell and a tombstone:

{code}
[
{"key": "blah",
 "cells": [["","",1453747038457027],
   ["v",1453747112,1453747112383096,"d"]]}
]
{code}

As my gc_grace_seconds value is 300, if I wait 5 minutes and perform a 
compaction, the new sstable would omit the tombstone, but the empty cell would 
still be present:

{code}
[
{"key": "blah",
 "cells": [["","",1453747038457027]]}
]
{code}

Performing the {{select *}} query would still yield the same result because of 
this.

However, in 3.2.1 this does not seem to be the behavior, after deleting the 'v' 
cell, performing a flush and then waiting 5 minutes and doing a compact, what 
ends up happening is that the sstable completely disappears (presumably because 
there is no remaining data) and the select query emits 0 rows:

{code}
cqlsh> select * from simple.data;

 k | v
---+-

(0 rows)
{code}

I'm unsure if this is by design or a bug, but it does represent a change 
between C* versions.

I have not tried this for a table with clustering columns yet, but I assume 
that the behavior will be the same.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11068) Entire row is compacted away if remaining cells are tombstones expiring after gc_grace_seconds

[Andy Tolbert (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115804#comment-15115804
 ] 

Andy Tolbert commented on CASSANDRA-11068:
--

It looks like this behavior only exists when there are no clustering key 
columns.  The behavior seems to be consistent with Cassandra 2.X when 
clustering keys exist.

> Entire row is compacted away if remaining cells are tombstones expiring after 
> gc_grace_seconds
> --
>
> Key: CASSANDRA-11068
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11068
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Andy Tolbert
>
> Assuming the following schema:
> {code}
> CREATE TABLE simple.data (
> k text PRIMARY KEY,
> v int
> ) WITH gc_grace_seconds = 300;
> {code}
> And the following queries:
> {code}
> insert into simple.data (k, v) values ('blah', 1);
> delete v from simple.data where k='blah';
> {code}
> Performing a {{select *}} from this table will return 1 row with a null value:
> {code}
> cqlsh> select * from simple.data;
>  k | v
> ---+-
>   blah |null
> {code}
> Prior the 3.0, if I were to do a flush, the sstable representation of this 
> table would include an empty cell and a tombstone:
> {code}
> [
> {"key": "blah",
>  "cells": [["","",1453747038457027],
>["v",1453747112,1453747112383096,"d"]]}
> ]
> {code}
> As my gc_grace_seconds value is 300, if I wait 5 minutes and perform a 
> compaction, the new sstable would omit the tombstone, but the empty cell 
> would still be present:
> {code}
> [
> {"key": "blah",
>  "cells": [["","",1453747038457027]]}
> ]
> {code}
> Performing the {{select *}} query would still yield the same result because 
> of this.
> However, in 3.2.1 this does not seem to be the behavior, after deleting the 
> 'v' cell, performing a flush and then waiting 5 minutes and doing a compact, 
> what ends up happening is that the sstable completely disappears (presumably 
> because there is no remaining data) and the select query emits 0 rows:
> {code}
> cqlsh> select * from simple.data;
>  k | v
> ---+-
> (0 rows)
> {code}
> I'm unsure if this is by design or a bug, but it does represent a change 
> between C* versions.
> I have not tried this for a table with clustering columns yet, but I assume 
> that the behavior will be the same.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10855) Use Caffeine (W-TinyLFU) for on-heap caches

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115839#comment-15115839
 ] 

Ariel Weisberg commented on CASSANDRA-10855:


stress supports [Weibull 
distribution|https://en.wikipedia.org/wiki/Weibull_distribution] so you 
shouldn't need to do a code change to find an alternative to uniform. Not sure 
if Weibull can fill the same niche as Zipf.

> Use Caffeine (W-TinyLFU) for on-heap caches
> ---
>
> Key: CASSANDRA-10855
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10855
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Ben Manes
>  Labels: performance
>
> Cassandra currently uses 
> [ConcurrentLinkedHashMap|https://code.google.com/p/concurrentlinkedhashmap] 
> for performance critical caches (key, counter) and Guava's cache for 
> non-critical (auth, metrics, security). All of these usages have been 
> replaced by [Caffeine|https://github.com/ben-manes/caffeine], written by the 
> author of the previously mentioned libraries.
> The primary incentive is to switch from LRU policy to W-TinyLFU, which 
> provides [near optimal|https://github.com/ben-manes/caffeine/wiki/Efficiency] 
> hit rates. It performs particularly well in database and search traces, is 
> scan resistant, and as adds a very small time/space overhead to LRU.
> Secondarily, Guava's caches never obtained similar 
> [performance|https://github.com/ben-manes/caffeine/wiki/Benchmarks] to CLHM 
> due to some optimizations not being ported over. This change results in 
> faster reads and not creating garbage as a side-effect.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11069) Materialised views require all collections to be selected.

[Vassil Lunchev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vassil Lunchev updated CASSANDRA-11069:
---
Description: 
Running Cassandra 3.0.2

Using the official example from: 
http://www.datastax.com/dev/blog/new-in-cassandra-3-0-materialized-views
The only difference is that I have added a map column to the base table.

{code:cql}
CREATE TABLE scores
(
  user TEXT,
  game TEXT,
  year INT,
  month INT,
  day INT,
  score INT,
  a_map map,
  PRIMARY KEY (user, game, year, month, day)
);

CREATE MATERIALIZED VIEW alltimehigh AS
   SELECT user FROM scores
   WHERE game IS NOT NULL AND score IS NOT NULL AND user IS NOT NULL AND 
year IS NOT NULL AND month IS NOT NULL AND day IS NOT NULL
   PRIMARY KEY (game, score, user, year, month, day)
   WITH CLUSTERING ORDER BY (score desc);

INSERT INTO scores (user, game, year, month, day, score) VALUES ('pcmanus', 
'Coup', 2015, 06, 02, 2000);
SELECT * FROM scores;
SELECT * FROM alltimehigh;
{code}

All of the above works perfectly fine. Until you insert a row where the 'a_map' 
column is not null.

{code:cql}
INSERT INTO scores (user, game, year, month, day, score, a_map) VALUES 
('pcmanus_2', 'Coup', 2015, 06, 02, 2000, {1: 'text'});
{code}

This results in:
{code}
Traceback (most recent call last):
  File "/Users/vassil/apache-cassandra-3.0.2/bin/cqlsh.py", line 1258, in 
perform_simple_statement
result = future.result()
  File 
"/Users/vassil/apache-cassandra-3.0.2/bin/../lib/cassandra-driver-internal-only-3.0.0-6af642d.zip/cassandra-driver-3.0.0-6af642d/cassandra/cluster.py",
 line 3122, in result
raise self._final_exception
WriteFailure: code=1500 [Replica(s) failed to execute write] message="Operation 
failed - received 0 responses and 1 failures" info={'failures': 1, 
'received_responses': 0, 'required_responses': 1, 'consistency': 'ONE'}
{code}

Selecting the base table and the materialised view is also interesting:
{code}
SELECT * FROM scores;
SELECT * FROM alltimehigh;
{code}

The result is:
{code}
cqlsh:tests> SELECT * FROM scores;

 user| game | year | month | day | a_map | score
-+--+--+---+-+---+---
 pcmanus | Coup | 2015 | 6 |   2 |  null |  2000

(1 rows)
cqlsh:tests> SELECT * FROM alltimehigh;

 game | score | user  | year | month | day
--+---+---+--+---+-
 Coup |  2000 |   pcmanus | 2015 | 6 |   2
 Coup |  2000 | pcmanus_2 | 2015 | 6 |   2

(2 rows)
{code}

In the logs you can see:
{code:java}
ERROR [SharedPool-Worker-2] 2016-01-26 03:25:27,456 Keyspace.java:484 - Unknown 
exception caught while attempting to update MaterializedView! tests.scores
java.lang.IllegalStateException: [ColumnDefinition{name=a_map, 
type=org.apache.cassandra.db.marshal.MapType(org.apache.cassandra.db.marshal.Int32Type,org.apache.cassandra.db.marshal.UTF8Type),
 kind=REGULAR, position=-1}] is not a subset of []
at 
org.apache.cassandra.db.Columns$Serializer.encodeBitmap(Columns.java:531) 
~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.Columns$Serializer.serializedSubsetSize(Columns.java:483)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.rows.UnfilteredSerializer.serializedRowBodySize(UnfilteredSerializer.java:275)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.rows.UnfilteredSerializer.serializedSize(UnfilteredSerializer.java:247)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.rows.UnfilteredSerializer.serializedSize(UnfilteredSerializer.java:234)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.rows.UnfilteredSerializer.serializedSize(UnfilteredSerializer.java:227)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serializedSize(UnfilteredRowIteratorSerializer.java:169)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.serializedSize(PartitionUpdate.java:683)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.Mutation$MutationSerializer.serializedSize(Mutation.java:354)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:259) 
~[apache-cassandra-3.0.2.jar:3.0.2]
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:461) 
[apache-cassandra-3.0.2.jar:3.0.2]
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:384) 
[apache-cassandra-3.0.2.jar:3.0.2]
at org.apache.cassandra.db.Mutation.apply(Mutation.java:210) 
[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:703) 
~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:149)
 

[jira] [Created] (CASSANDRA-11069) Materialised views require all collections to be selected.

[Vassil Lunchev (JIRA)]

Vassil Lunchev created CASSANDRA-11069:
--

 Summary: Materialised views require all collections to be selected.
 Key: CASSANDRA-11069
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11069
 Project: Cassandra
  Issue Type: Bug
Reporter: Vassil Lunchev


Running Cassandra 3.0.2

Using the official example from: 
http://www.datastax.com/dev/blog/new-in-cassandra-3-0-materialized-views
The only difference is that the I have added a map column to the base table.

{code:cql}
CREATE TABLE scores
(
  user TEXT,
  game TEXT,
  year INT,
  month INT,
  day INT,
  score INT,
  a_map map,
  PRIMARY KEY (user, game, year, month, day)
);

CREATE MATERIALIZED VIEW alltimehigh AS
   SELECT user FROM scores
   WHERE game IS NOT NULL AND score IS NOT NULL AND user IS NOT NULL AND 
year IS NOT NULL AND month IS NOT NULL AND day IS NOT NULL
   PRIMARY KEY (game, score, user, year, month, day)
   WITH CLUSTERING ORDER BY (score desc);

INSERT INTO scores (user, game, year, month, day, score) VALUES ('pcmanus', 
'Coup', 2015, 06, 02, 2000);
SELECT * FROM scores;
SELECT * FROM alltimehigh;
{code}

All of the above works perfectly fine. Until you insert a row that where the 
'a_map' column is not null.

{code:cql}
INSERT INTO scores (user, game, year, month, day, score, a_map) VALUES 
('pcmanus_2', 'Coup', 2015, 06, 02, 2000, {1: 'text'});
{code}

This results in:
{code}
Traceback (most recent call last):
  File "/Users/vassil/apache-cassandra-3.0.2/bin/cqlsh.py", line 1258, in 
perform_simple_statement
result = future.result()
  File 
"/Users/vassil/apache-cassandra-3.0.2/bin/../lib/cassandra-driver-internal-only-3.0.0-6af642d.zip/cassandra-driver-3.0.0-6af642d/cassandra/cluster.py",
 line 3122, in result
raise self._final_exception
WriteFailure: code=1500 [Replica(s) failed to execute write] message="Operation 
failed - received 0 responses and 1 failures" info={'failures': 1, 
'received_responses': 0, 'required_responses': 1, 'consistency': 'ONE'}
{code}

Selecting the base table and the materialised view is also interesting:
{code}
SELECT * FROM scores;
SELECT * FROM alltimehigh;
{code}

The result is:
{code}
cqlsh:tests> SELECT * FROM scores;

 user| game | year | month | day | a_map | score
-+--+--+---+-+---+---
 pcmanus | Coup | 2015 | 6 |   2 |  null |  2000

(1 rows)
cqlsh:tests> SELECT * FROM alltimehigh;

 game | score | user  | year | month | day
--+---+---+--+---+-
 Coup |  2000 |   pcmanus | 2015 | 6 |   2
 Coup |  2000 | pcmanus_2 | 2015 | 6 |   2

(2 rows)
{code}

In the logs you can see:
{code:java}
ERROR [SharedPool-Worker-2] 2016-01-26 03:25:27,456 Keyspace.java:484 - Unknown 
exception caught while attempting to update MaterializedView! tests.scores
java.lang.IllegalStateException: [ColumnDefinition{name=a_map, 
type=org.apache.cassandra.db.marshal.MapType(org.apache.cassandra.db.marshal.Int32Type,org.apache.cassandra.db.marshal.UTF8Type),
 kind=REGULAR, position=-1}] is not a subset of []
at 
org.apache.cassandra.db.Columns$Serializer.encodeBitmap(Columns.java:531) 
~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.Columns$Serializer.serializedSubsetSize(Columns.java:483)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.rows.UnfilteredSerializer.serializedRowBodySize(UnfilteredSerializer.java:275)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.rows.UnfilteredSerializer.serializedSize(UnfilteredSerializer.java:247)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.rows.UnfilteredSerializer.serializedSize(UnfilteredSerializer.java:234)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.rows.UnfilteredSerializer.serializedSize(UnfilteredSerializer.java:227)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serializedSize(UnfilteredRowIteratorSerializer.java:169)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.serializedSize(PartitionUpdate.java:683)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.Mutation$MutationSerializer.serializedSize(Mutation.java:354)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:259) 
~[apache-cassandra-3.0.2.jar:3.0.2]
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:461) 
[apache-cassandra-3.0.2.jar:3.0.2]
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:384) 
[apache-cassandra-3.0.2.jar:3.0.2]
at org.apache.cassandra.db.Mutation.apply(Mutation.java:210) 
[apache-cassandra-3.0.2.jar:3.0.2]
at 

[jira] [Updated] (CASSANDRA-11069) Materialised views require all collections to be selected.

[Vassil Lunchev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vassil Lunchev updated CASSANDRA-11069:
---
Description: 
Running Cassandra 3.0.2

Using the official example from: 
http://www.datastax.com/dev/blog/new-in-cassandra-3-0-materialized-views
The only difference is that I have added a map column to the base table.

{code:cql}
CREATE TABLE scores
(
  user TEXT,
  game TEXT,
  year INT,
  month INT,
  day INT,
  score INT,
  a_map map,
  PRIMARY KEY (user, game, year, month, day)
);

CREATE MATERIALIZED VIEW alltimehigh AS
   SELECT user FROM scores
   WHERE game IS NOT NULL AND score IS NOT NULL AND user IS NOT NULL AND 
year IS NOT NULL AND month IS NOT NULL AND day IS NOT NULL
   PRIMARY KEY (game, score, user, year, month, day)
   WITH CLUSTERING ORDER BY (score desc);

INSERT INTO scores (user, game, year, month, day, score) VALUES ('pcmanus', 
'Coup', 2015, 06, 02, 2000);
SELECT * FROM scores;
SELECT * FROM alltimehigh;
{code}

All of the above works perfectly fine. Until you insert a row that where the 
'a_map' column is not null.

{code:cql}
INSERT INTO scores (user, game, year, month, day, score, a_map) VALUES 
('pcmanus_2', 'Coup', 2015, 06, 02, 2000, {1: 'text'});
{code}

This results in:
{code}
Traceback (most recent call last):
  File "/Users/vassil/apache-cassandra-3.0.2/bin/cqlsh.py", line 1258, in 
perform_simple_statement
result = future.result()
  File 
"/Users/vassil/apache-cassandra-3.0.2/bin/../lib/cassandra-driver-internal-only-3.0.0-6af642d.zip/cassandra-driver-3.0.0-6af642d/cassandra/cluster.py",
 line 3122, in result
raise self._final_exception
WriteFailure: code=1500 [Replica(s) failed to execute write] message="Operation 
failed - received 0 responses and 1 failures" info={'failures': 1, 
'received_responses': 0, 'required_responses': 1, 'consistency': 'ONE'}
{code}

Selecting the base table and the materialised view is also interesting:
{code}
SELECT * FROM scores;
SELECT * FROM alltimehigh;
{code}

The result is:
{code}
cqlsh:tests> SELECT * FROM scores;

 user| game | year | month | day | a_map | score
-+--+--+---+-+---+---
 pcmanus | Coup | 2015 | 6 |   2 |  null |  2000

(1 rows)
cqlsh:tests> SELECT * FROM alltimehigh;

 game | score | user  | year | month | day
--+---+---+--+---+-
 Coup |  2000 |   pcmanus | 2015 | 6 |   2
 Coup |  2000 | pcmanus_2 | 2015 | 6 |   2

(2 rows)
{code}

In the logs you can see:
{code:java}
ERROR [SharedPool-Worker-2] 2016-01-26 03:25:27,456 Keyspace.java:484 - Unknown 
exception caught while attempting to update MaterializedView! tests.scores
java.lang.IllegalStateException: [ColumnDefinition{name=a_map, 
type=org.apache.cassandra.db.marshal.MapType(org.apache.cassandra.db.marshal.Int32Type,org.apache.cassandra.db.marshal.UTF8Type),
 kind=REGULAR, position=-1}] is not a subset of []
at 
org.apache.cassandra.db.Columns$Serializer.encodeBitmap(Columns.java:531) 
~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.Columns$Serializer.serializedSubsetSize(Columns.java:483)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.rows.UnfilteredSerializer.serializedRowBodySize(UnfilteredSerializer.java:275)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.rows.UnfilteredSerializer.serializedSize(UnfilteredSerializer.java:247)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.rows.UnfilteredSerializer.serializedSize(UnfilteredSerializer.java:234)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.rows.UnfilteredSerializer.serializedSize(UnfilteredSerializer.java:227)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serializedSize(UnfilteredRowIteratorSerializer.java:169)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.serializedSize(PartitionUpdate.java:683)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.Mutation$MutationSerializer.serializedSize(Mutation.java:354)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:259) 
~[apache-cassandra-3.0.2.jar:3.0.2]
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:461) 
[apache-cassandra-3.0.2.jar:3.0.2]
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:384) 
[apache-cassandra-3.0.2.jar:3.0.2]
at org.apache.cassandra.db.Mutation.apply(Mutation.java:210) 
[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:703) 
~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:149)
 

[jira] [Updated] (CASSANDRA-11069) Materialised views require all collections to be selected.

[Vassil Lunchev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vassil Lunchev updated CASSANDRA-11069:
---
Description: 
Running Cassandra 3.0.2

Using the official example from: 
http://www.datastax.com/dev/blog/new-in-cassandra-3-0-materialized-views
The only difference is that I have added a map column to the base table.

{code:cql}
CREATE TABLE scores
(
  user TEXT,
  game TEXT,
  year INT,
  month INT,
  day INT,
  score INT,
  a_map map,
  PRIMARY KEY (user, game, year, month, day)
);

CREATE MATERIALIZED VIEW alltimehigh AS
   SELECT user FROM scores
   WHERE game IS NOT NULL AND score IS NOT NULL AND user IS NOT NULL AND 
year IS NOT NULL AND month IS NOT NULL AND day IS NOT NULL
   PRIMARY KEY (game, score, user, year, month, day)
   WITH CLUSTERING ORDER BY (score desc);

INSERT INTO scores (user, game, year, month, day, score) VALUES ('pcmanus', 
'Coup', 2015, 06, 02, 2000);
SELECT * FROM scores;
SELECT * FROM alltimehigh;
{code}

All of the above works perfectly fine. Until you insert a row where the 'a_map' 
column is not null.

{code:cql}
INSERT INTO scores (user, game, year, month, day, score, a_map) VALUES 
('pcmanus_2', 'Coup', 2015, 06, 02, 2000, {1: 'text'});
{code}

This results in:
{code}
Traceback (most recent call last):
  File "/Users/vassil/apache-cassandra-3.0.2/bin/cqlsh.py", line 1258, in 
perform_simple_statement
result = future.result()
  File 
"/Users/vassil/apache-cassandra-3.0.2/bin/../lib/cassandra-driver-internal-only-3.0.0-6af642d.zip/cassandra-driver-3.0.0-6af642d/cassandra/cluster.py",
 line 3122, in result
raise self._final_exception
WriteFailure: code=1500 [Replica(s) failed to execute write] message="Operation 
failed - received 0 responses and 1 failures" info={'failures': 1, 
'received_responses': 0, 'required_responses': 1, 'consistency': 'ONE'}
{code}

Selecting the base table and the materialised view is also interesting:
{code}
SELECT * FROM scores;
SELECT * FROM alltimehigh;
{code}

The result is:
{code}
cqlsh:tests> SELECT * FROM scores;

 user| game | year | month | day | a_map | score
-+--+--+---+-+---+---
 pcmanus | Coup | 2015 | 6 |   2 |  null |  2000

(1 rows)
cqlsh:tests> SELECT * FROM alltimehigh;

 game | score | user  | year | month | day
--+---+---+--+---+-
 Coup |  2000 |   pcmanus | 2015 | 6 |   2
 Coup |  2000 | pcmanus_2 | 2015 | 6 |   2

(2 rows)
{code}

In the logs you can see:
{code:java}
ERROR [SharedPool-Worker-2] 2016-01-26 03:25:27,456 Keyspace.java:484 - Unknown 
exception caught while attempting to update MaterializedView! tests.scores
java.lang.IllegalStateException: [ColumnDefinition{name=a_map, 
type=org.apache.cassandra.db.marshal.MapType(org.apache.cassandra.db.marshal.Int32Type,org.apache.cassandra.db.marshal.UTF8Type),
 kind=REGULAR, position=-1}] is not a subset of []
at 
org.apache.cassandra.db.Columns$Serializer.encodeBitmap(Columns.java:531) 
~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.Columns$Serializer.serializedSubsetSize(Columns.java:483)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.rows.UnfilteredSerializer.serializedRowBodySize(UnfilteredSerializer.java:275)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.rows.UnfilteredSerializer.serializedSize(UnfilteredSerializer.java:247)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.rows.UnfilteredSerializer.serializedSize(UnfilteredSerializer.java:234)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.rows.UnfilteredSerializer.serializedSize(UnfilteredSerializer.java:227)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serializedSize(UnfilteredRowIteratorSerializer.java:169)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.serializedSize(PartitionUpdate.java:683)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.Mutation$MutationSerializer.serializedSize(Mutation.java:354)
 ~[apache-cassandra-3.0.2.jar:3.0.2]
at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:259) 
~[apache-cassandra-3.0.2.jar:3.0.2]
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:461) 
[apache-cassandra-3.0.2.jar:3.0.2]
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:384) 
[apache-cassandra-3.0.2.jar:3.0.2]
at org.apache.cassandra.db.Mutation.apply(Mutation.java:210) 
[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:703) 
~[apache-cassandra-3.0.2.jar:3.0.2]
at 
org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:149)
 

[jira] [Updated] (CASSANDRA-10767) Checking version of Cassandra command creates `cassandra.logdir_IS_UNDEFINED/`

[Yuki Morishita (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-10767:
---
Priority: Minor  (was: Major)

> Checking version of Cassandra command creates `cassandra.logdir_IS_UNDEFINED/`
> --
>
> Key: CASSANDRA-10767
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10767
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: $ cassandra -v   
>  
> 2.1.2
> MacOSX 10.9.5
> $ brew info cassandra 
>  [14:15:41]
> cassandra: stable 2.2.3 (bottled)
> Eventually consistent, distributed key-value store
> https://cassandra.apache.org
> /usr/local/Cellar/cassandra/2.1.2 (3975 files, 92M) *
>   Built from source
> From: 
> https://github.com/Homebrew/homebrew/blob/master/Library/Formula/cassandra.rb
> ==> Caveats
> To have launchd start cassandra at login:
>   ln -sfv /usr/local/opt/cassandra/*.plist ~/Library/LaunchAgents
> Then to load cassandra now:
>   launchctl load ~/Library/LaunchAgents/homebrew.mxcl.cassandra.plist
>Reporter: Jens Rantil
>Priority: Minor
>
> When I execute `cassandra -v` on the terminal the directory 
> `cassandra.logdir_IS_UNDEFINED` is created in my CWD:
> {noformat}
> $ tree cassandra.logdir_IS_UNDEFINED
> cassandra.logdir_IS_UNDEFINED
> └── system.log
> 0 directories, 1 file
> {noformat}
> Expected: That no log file nor directory is created when I'm simply checking 
> the version of Cassandra. Feels a bit ridiculous.
> Additionals: Just double checking, is this a bundling issue that should be 
> reported to Homebrew? Probably not, right?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-6036) When SStableloader exits, allow for a non-zero exit code if an exception is encounterd

[Yuki Morishita (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita resolved CASSANDRA-6036.
---
Resolution: Not A Problem

As Brandon mentioned, this should be taken care of in recent versions.

> When SStableloader exits, allow for a non-zero exit code if an exception is 
> encounterd
> --
>
> Key: CASSANDRA-6036
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6036
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Laura Adney
>
> Request from customer to have sstableloader exit with non-zero exit code when 
> an exception is encountered.
> Customer reports:1379107056.40 ERROR [Thread-108] 2013-09-13 17:17:36,344 
> CassandraDaemon.java (line 192) Exception in thread Thread[Thread-108,5,main]
> 1379107056.40 java.lang.AssertionError
> 1379107056.40 at 
> org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:168)
> 1379107056.40 at 
> org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:138)
> 1379107056.40 at 
> org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:238)
> 1379107056.40 at 
> org.apache.cassandra.net.IncomingTcpConnection.handleStream(IncomingTcpConnection.java:178)
> 1379107056.40 at 
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:78)
> 1379107056.40 ERROR [Thread-108] 17:17:36,344 Exception in thread 
> Thread[Thread-108,5,main]
> 1379107056.40 java.lang.AssertionError
> 1379107056.40 at 
> org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:168)
> 1379107056.40 at 
> org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:138)
> 1379107056.40 at 
> org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:238)
> 1379107056.40 at 
> org.apache.cassandra.net.IncomingTcpConnection.handleStream(IncomingTcpConnection.java:178)
> 1379107056.40 at 
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:78)
> And finally: sstableloader exits 0 in this condition, as opposed to non-zero, 
> so our tools blindly pass over this and don't realize that the import failed. 
> That's pretty bad, too.
> zd6699



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[10/10] cassandra git commit: Merge branch 'cassandra-3.3' into trunk

[marcuse]

Merge branch 'cassandra-3.3' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4abd9936
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4abd9936
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4abd9936

Branch: refs/heads/trunk
Commit: 4abd9936342f5faf3e23c3d395e24278a3b3bfbd
Parents: 72790dc f217428
Author: Marcus Eriksson 
Authored: Mon Jan 25 10:09:31 2016 +0100
Committer: Marcus Eriksson 
Committed: Mon Jan 25 10:09:31 2016 +0100

--
 CHANGES.txt |   1 +
 src/java/org/apache/cassandra/db/Memtable.java  |   9 +
 .../db/compaction/CompactionController.java |  19 +-
 .../db/compaction/CompactionControllerTest.java | 195 +++
 4 files changed, 221 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4abd9936/CHANGES.txt
--

[07/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[marcuse]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/442f4737
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/442f4737
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/442f4737

Branch: refs/heads/trunk
Commit: 442f4737ceddd34f14210da49cee4d48b468f01e
Parents: 2f8e5f3 b5d6d4f
Author: Marcus Eriksson 
Authored: Mon Jan 25 10:05:26 2016 +0100
Committer: Marcus Eriksson 
Committed: Mon Jan 25 10:05:59 2016 +0100

--
 CHANGES.txt |   1 +
 src/java/org/apache/cassandra/db/Memtable.java  |   9 +
 .../db/compaction/CompactionController.java |  19 +-
 .../db/compaction/CompactionControllerTest.java | 195 +++
 4 files changed, 221 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/442f4737/CHANGES.txt
--
diff --cc CHANGES.txt
index bb26b20,cdc3b34..70abffe
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,30 -1,8 +1,31 @@@
 +3.0.3
 + * Update CQL documentation (CASSANDRA-10899)
 + * Check the column name, not cell name, for dropped columns when reading
 +   legacy sstables (CASSANDRA-11018)
 + * Don't attempt to index clustering values of static rows (CASSANDRA-11021)
 + * Remove checksum files after replaying hints (CASSANDRA-10947)
 + * Support passing base table metadata to custom 2i validation 
(CASSANDRA-10924)
 + * Ensure stale index entries are purged during reads (CASSANDRA-11013)
 + * Fix AssertionError when removing from list using UPDATE (CASSANDRA-10954)
 + * Fix UnsupportedOperationException when reading old sstable with range
 +   tombstone (CASSANDRA-10743)
 + * MV should use the maximum timestamp of the primary key (CASSANDRA-10910)
 + * Fix potential assertion error during compaction (CASSANDRA-10944)
 + * Fix counting of received sstables in streaming (CASSANDRA-10949)
 + * Implement hints compression (CASSANDRA-9428)
 + * Fix potential assertion error when reading static columns (CASSANDRA-10903)
 + * Avoid NoSuchElementException when executing empty batch (CASSANDRA-10711)
 + * Avoid building PartitionUpdate in toString (CASSANDRA-10897)
 + * Reduce heap spent when receiving many SSTables (CASSANDRA-10797)
 + * Add back support for 3rd party auth providers to bulk loader 
(CASSANDRA-10873)
 + * Eliminate the dependency on jgrapht for UDT resolution (CASSANDRA-10653)
 + * (Hadoop) Close Clusters and Sessions in Hadoop Input/Output classes 
(CASSANDRA-10837)
 + * Fix sstableloader not working with upper case keyspace name 
(CASSANDRA-10806)
 +Merged from 2.2:
  2.2.5
+  * maxPurgeableTimestamp needs to check memtables too (CASSANDRA-9949)
   * Apply change to compaction throughput in real time (CASSANDRA-10025)
   * Fix potential NPE on ORDER BY queries with IN (CASSANDRA-10955)
 - * Avoid over-fetching during the page of range queries (CASSANDRA-8521)
   * Start L0 STCS-compactions even if there is a L0 -> L1 compaction
 going (CASSANDRA-10979)
   * Make UUID LSB unique per process (CASSANDRA-7925)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/442f4737/src/java/org/apache/cassandra/db/Memtable.java
--
diff --cc src/java/org/apache/cassandra/db/Memtable.java
index 96b1775,fb4da72..5d5f7bf
--- a/src/java/org/apache/cassandra/db/Memtable.java
+++ b/src/java/org/apache/cassandra/db/Memtable.java
@@@ -237,12 -227,24 +240,13 @@@ public class Memtable implements Compar
  }
  }
  
 -final AtomicBTreeColumns.ColumnUpdater updater = 
previous.addAllWithSizeDelta(cf, allocator, opGroup, indexer);
 -minTimestamp = Math.min(minTimestamp, updater.minTimestamp);
 -liveDataSize.addAndGet(initialSize + updater.dataSize);
 -currentOperations.addAndGet(cf.getColumnCount() + 
(cf.isMarkedForDelete() ? 1 : 0) + cf.deletionInfo().rangeCount());
 -return updater.colUpdateTimeDelta;
 -}
 -
 -// for debugging
 -public String contents()
 -{
 -StringBuilder builder = new StringBuilder();
 -builder.append("{");
 -for (Map.Entry entry : 
rows.entrySet())
 -{
 -builder.append(entry.getKey()).append(": 
").append(entry.getValue()).append(", ");
 -}
 -builder.append("}");
 -return builder.toString();
 +long[] pair = previous.addAllWithSizeDelta(update, opGroup, indexer);
++minTimestamp = Math.min(minTimestamp, previous.stats().minTimestamp);
 +liveDataSize.addAndGet(initialSize + pair[0]);
 +columnsCollector.update(update.columns());
 +

[02/10] cassandra git commit: maxPurgeableTimestamp needs to check memtables too

[marcuse]

maxPurgeableTimestamp needs to check memtables too

Patch by Stefania Alborghetti; reviewed by marcuse for CASSANDRA-9949


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b5d6d4f7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b5d6d4f7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b5d6d4f7

Branch: refs/heads/cassandra-3.0
Commit: b5d6d4f72299a0b08ce3279aade507e2a999acc6
Parents: b4d67c9
Author: Stefania Alborghetti 
Authored: Mon Jan 4 16:24:51 2016 +0100
Committer: Stefania Alborghetti 
Committed: Mon Jan 25 13:30:05 2016 +0800

--
 CHANGES.txt |   1 +
 .../apache/cassandra/db/AtomicBTreeColumns.java |  14 +-
 src/java/org/apache/cassandra/db/Memtable.java  |  15 +-
 .../db/compaction/CompactionController.java |  21 +-
 .../db/compaction/LazilyCompactedRow.java   |   2 +-
 .../db/compaction/CompactionControllerTest.java | 191 +++
 6 files changed, 232 insertions(+), 12 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b5d6d4f7/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 59997ff..cdc3b34 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.5
+ * maxPurgeableTimestamp needs to check memtables too (CASSANDRA-9949)
  * Apply change to compaction throughput in real time (CASSANDRA-10025)
  * Fix potential NPE on ORDER BY queries with IN (CASSANDRA-10955)
  * Avoid over-fetching during the page of range queries (CASSANDRA-8521)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b5d6d4f7/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java 
b/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
index 710b289..f5b7712 100644
--- a/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
+++ b/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
@@ -198,7 +198,7 @@ public class AtomicBTreeColumns extends ColumnFamily
  *
  * @return the difference in size seen after merging the given columns
  */
-public Pair addAllWithSizeDelta(final ColumnFamily cm, 
MemtableAllocator allocator, OpOrder.Group writeOp, Updater indexer)
+public ColumnUpdater addAllWithSizeDelta(final ColumnFamily cm, 
MemtableAllocator allocator, OpOrder.Group writeOp, Updater indexer)
 {
 ColumnUpdater updater = new ColumnUpdater(this, cm.metadata, 
allocator, writeOp, indexer);
 DeletionInfo inputDeletionInfoCopy = null;
@@ -237,7 +237,7 @@ public class AtomicBTreeColumns extends ColumnFamily
 {
 indexer.updateRowLevelIndexes();
 updater.finish();
-return Pair.create(updater.dataSize, 
updater.colUpdateTimeDelta);
+return updater;
 }
 else if (!monitorOwned)
 {
@@ -429,7 +429,7 @@ public class AtomicBTreeColumns extends ColumnFamily
 }
 
 // the function we provide to the btree utilities to perform any column 
replacements
-private static final class ColumnUpdater implements UpdateFunction
+static final class ColumnUpdater implements UpdateFunction
 {
 final AtomicBTreeColumns updating;
 final CFMetaData metadata;
@@ -442,6 +442,7 @@ public class AtomicBTreeColumns extends ColumnFamily
 long colUpdateTimeDelta = Long.MAX_VALUE;
 final MemtableAllocator.DataReclaimer reclaimer;
 List inserted; // TODO: replace with walk of aborted BTree
+long minTimestamp = Long.MAX_VALUE;
 
 private ColumnUpdater(AtomicBTreeColumns updating, CFMetaData 
metadata, MemtableAllocator allocator, OpOrder.Group writeOp, Updater indexer)
 {
@@ -462,6 +463,7 @@ public class AtomicBTreeColumns extends ColumnFamily
 if (inserted == null)
 inserted = new ArrayList<>();
 inserted.add(insert);
+minTimestamp = Math.min(minTimestamp, insert.timestamp());
 return insert;
 }
 
@@ -469,6 +471,11 @@ public class AtomicBTreeColumns extends ColumnFamily
 {
 Cell reconciled = existing.reconcile(update);
 indexer.update(existing, reconciled);
+// pick the smallest timestamp because we want to be consistent 
with the logic applied when inserting
+// a cell in apply(Cell insert) above. For example given 3 
timestamps where T3 < T2 < T1 then we want
+// [apply(T1) -> apply(T2) -> apply(T3)] and [apply(T3) -> 
apply(T2) -> apply(T1)] to 

[08/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.3

[marcuse]

Merge branch 'cassandra-3.0' into cassandra-3.3


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f2174280
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f2174280
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f2174280

Branch: refs/heads/cassandra-3.3
Commit: f2174280d6dcbb951b539dc558d1e1ae81f43b5c
Parents: acb7fed 442f473
Author: Marcus Eriksson 
Authored: Mon Jan 25 10:09:15 2016 +0100
Committer: Marcus Eriksson 
Committed: Mon Jan 25 10:09:15 2016 +0100

--
 CHANGES.txt |   1 +
 src/java/org/apache/cassandra/db/Memtable.java  |   9 +
 .../db/compaction/CompactionController.java |  19 +-
 .../db/compaction/CompactionControllerTest.java | 195 +++
 4 files changed, 221 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f2174280/CHANGES.txt
--
diff --cc CHANGES.txt
index ed56a78,70abffe..81657f9
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -15,9 -11,20 +15,10 @@@ Merged from 3.0
 tombstone (CASSANDRA-10743)
   * MV should use the maximum timestamp of the primary key (CASSANDRA-10910)
   * Fix potential assertion error during compaction (CASSANDRA-10944)
 - * Fix counting of received sstables in streaming (CASSANDRA-10949)
 - * Implement hints compression (CASSANDRA-9428)
 - * Fix potential assertion error when reading static columns (CASSANDRA-10903)
 - * Avoid NoSuchElementException when executing empty batch (CASSANDRA-10711)
 - * Avoid building PartitionUpdate in toString (CASSANDRA-10897)
 - * Reduce heap spent when receiving many SSTables (CASSANDRA-10797)
 - * Add back support for 3rd party auth providers to bulk loader 
(CASSANDRA-10873)
 - * Eliminate the dependency on jgrapht for UDT resolution (CASSANDRA-10653)
 - * (Hadoop) Close Clusters and Sessions in Hadoop Input/Output classes 
(CASSANDRA-10837)
 - * Fix sstableloader not working with upper case keyspace name 
(CASSANDRA-10806)
  Merged from 2.2:
 -2.2.5
+  * maxPurgeableTimestamp needs to check memtables too (CASSANDRA-9949)
   * Apply change to compaction throughput in real time (CASSANDRA-10025)
 + * (cqlsh) encode input correctly when saving history
   * Fix potential NPE on ORDER BY queries with IN (CASSANDRA-10955)
   * Start L0 STCS-compactions even if there is a L0 -> L1 compaction
 going (CASSANDRA-10979)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f2174280/src/java/org/apache/cassandra/db/Memtable.java
--
diff --cc src/java/org/apache/cassandra/db/Memtable.java
index 8e7a43c,5d5f7bf..952c045
--- a/src/java/org/apache/cassandra/db/Memtable.java
+++ b/src/java/org/apache/cassandra/db/Memtable.java
@@@ -334,37 -316,24 +338,42 @@@ public class Memtable implements Compar
  return creationTime;
  }
  
+ public long getMinTimestamp()
+ {
+ return minTimestamp;
+ }
+ 
 -class FlushRunnable extends DiskAwareRunnable
 +class FlushRunnable implements Callable
  {
 -private final ReplayPosition context;
 +public final ReplayPosition context;
  private final long estimatedSize;
 +private final ConcurrentNavigableMap toFlush;
  
  private final boolean isBatchLogTable;
 +private final SSTableMultiWriter writer;
 +
 +// keeping these to be able to log what we are actually flushing
 +private final PartitionPosition from;
 +private final PartitionPosition to;
  
 -FlushRunnable(ReplayPosition context)
 +FlushRunnable(ReplayPosition context, PartitionPosition from, 
PartitionPosition to, Directories.DataDirectory flushLocation, 
LifecycleTransaction txn)
  {
 -this.context = context;
 +this(context, partitions.subMap(from, to), flushLocation, from, 
to, txn);
 +}
 +
 +FlushRunnable(ReplayPosition context, LifecycleTransaction txn)
 +{
 +this(context, partitions, null, null, null, txn);
 +}
  
 +FlushRunnable(ReplayPosition context, 
ConcurrentNavigableMap toFlush, 
Directories.DataDirectory flushLocation, PartitionPosition from, 
PartitionPosition to, LifecycleTransaction txn)
 +{
 +this.context = context;
 +this.toFlush = toFlush;
 +this.from = from;
 +this.to = to;
  long keySize = 0;
 -for (PartitionPosition key : partitions.keySet())
 +for (PartitionPosition key : toFlush.keySet())
  {
  //  make sure we don't write 

[03/10] cassandra git commit: maxPurgeableTimestamp needs to check memtables too

[marcuse]

maxPurgeableTimestamp needs to check memtables too

Patch by Stefania Alborghetti; reviewed by marcuse for CASSANDRA-9949


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b5d6d4f7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b5d6d4f7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b5d6d4f7

Branch: refs/heads/cassandra-3.3
Commit: b5d6d4f72299a0b08ce3279aade507e2a999acc6
Parents: b4d67c9
Author: Stefania Alborghetti 
Authored: Mon Jan 4 16:24:51 2016 +0100
Committer: Stefania Alborghetti 
Committed: Mon Jan 25 13:30:05 2016 +0800

--
 CHANGES.txt |   1 +
 .../apache/cassandra/db/AtomicBTreeColumns.java |  14 +-
 src/java/org/apache/cassandra/db/Memtable.java  |  15 +-
 .../db/compaction/CompactionController.java |  21 +-
 .../db/compaction/LazilyCompactedRow.java   |   2 +-
 .../db/compaction/CompactionControllerTest.java | 191 +++
 6 files changed, 232 insertions(+), 12 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b5d6d4f7/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 59997ff..cdc3b34 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.5
+ * maxPurgeableTimestamp needs to check memtables too (CASSANDRA-9949)
  * Apply change to compaction throughput in real time (CASSANDRA-10025)
  * Fix potential NPE on ORDER BY queries with IN (CASSANDRA-10955)
  * Avoid over-fetching during the page of range queries (CASSANDRA-8521)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b5d6d4f7/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java 
b/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
index 710b289..f5b7712 100644
--- a/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
+++ b/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
@@ -198,7 +198,7 @@ public class AtomicBTreeColumns extends ColumnFamily
  *
  * @return the difference in size seen after merging the given columns
  */
-public Pair addAllWithSizeDelta(final ColumnFamily cm, 
MemtableAllocator allocator, OpOrder.Group writeOp, Updater indexer)
+public ColumnUpdater addAllWithSizeDelta(final ColumnFamily cm, 
MemtableAllocator allocator, OpOrder.Group writeOp, Updater indexer)
 {
 ColumnUpdater updater = new ColumnUpdater(this, cm.metadata, 
allocator, writeOp, indexer);
 DeletionInfo inputDeletionInfoCopy = null;
@@ -237,7 +237,7 @@ public class AtomicBTreeColumns extends ColumnFamily
 {
 indexer.updateRowLevelIndexes();
 updater.finish();
-return Pair.create(updater.dataSize, 
updater.colUpdateTimeDelta);
+return updater;
 }
 else if (!monitorOwned)
 {
@@ -429,7 +429,7 @@ public class AtomicBTreeColumns extends ColumnFamily
 }
 
 // the function we provide to the btree utilities to perform any column 
replacements
-private static final class ColumnUpdater implements UpdateFunction
+static final class ColumnUpdater implements UpdateFunction
 {
 final AtomicBTreeColumns updating;
 final CFMetaData metadata;
@@ -442,6 +442,7 @@ public class AtomicBTreeColumns extends ColumnFamily
 long colUpdateTimeDelta = Long.MAX_VALUE;
 final MemtableAllocator.DataReclaimer reclaimer;
 List inserted; // TODO: replace with walk of aborted BTree
+long minTimestamp = Long.MAX_VALUE;
 
 private ColumnUpdater(AtomicBTreeColumns updating, CFMetaData 
metadata, MemtableAllocator allocator, OpOrder.Group writeOp, Updater indexer)
 {
@@ -462,6 +463,7 @@ public class AtomicBTreeColumns extends ColumnFamily
 if (inserted == null)
 inserted = new ArrayList<>();
 inserted.add(insert);
+minTimestamp = Math.min(minTimestamp, insert.timestamp());
 return insert;
 }
 
@@ -469,6 +471,11 @@ public class AtomicBTreeColumns extends ColumnFamily
 {
 Cell reconciled = existing.reconcile(update);
 indexer.update(existing, reconciled);
+// pick the smallest timestamp because we want to be consistent 
with the logic applied when inserting
+// a cell in apply(Cell insert) above. For example given 3 
timestamps where T3 < T2 < T1 then we want
+// [apply(T1) -> apply(T2) -> apply(T3)] and [apply(T3) -> 
apply(T2) -> apply(T1)] to 

[jira] [Commented] (CASSANDRA-11063) Unable to compute ceiling for max when histogram overflowed

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15114984#comment-15114984
 ] 

Marcus Eriksson commented on CASSANDRA-11063:
-

we should note that you seem to have a 30GB partition: {{Compacted partition 
maximum bytes: 30753941057}} - we should ofc not fail like this anyway, but you 
might want to fix your data model

> Unable to compute ceiling for max when histogram overflowed
> ---
>
> Key: CASSANDRA-11063
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11063
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: Cassandra 2.1.9 on RHEL
>Reporter: Navjyot Nishant
>  Labels: Compaction, thread
>
> Issue https://issues.apache.org/jira/browse/CASSANDRA-8028 seems related with 
> error we are getting. But we are getting this with Cassandra 2.1.9 when 
> autocompaction is running it keeps throwing following errors, we are unsure 
> if its a bug or can be resolved, please suggest.
> {code}
> WARN  [CompactionExecutor:3] 2016-01-23 13:30:40,907 SSTableWriter.java:240 - 
> Compacting large partition gccatlgsvcks/category_name_dedup:66611300 
> (138152195 bytes)
> ERROR [CompactionExecutor:1] 2016-01-23 13:30:50,267 CassandraDaemon.java:223 
> - Exception in thread Thread[CompactionExecutor:1,1,main]
> java.lang.IllegalStateException: Unable to compute ceiling for max when 
> histogram overflowed
> at 
> org.apache.cassandra.utils.EstimatedHistogram.mean(EstimatedHistogram.java:203)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.sstable.metadata.StatsMetadata.getEstimatedDroppableTombstoneRatio(StatsMetadata.java:98)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.io.sstable.SSTableReader.getEstimatedDroppableTombstoneRatio(SSTableReader.java:1987)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.db.compaction.AbstractCompactionStrategy.worthDroppingTombstones(AbstractCompactionStrategy.java:370)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy.getNextBackgroundSSTables(SizeTieredCompactionStrategy.java:96)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy.getNextBackgroundTask(SizeTieredCompactionStrategy.java:179)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy.getNextBackgroundTask(WrappingCompactionStrategy.java:84)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:230)
>  ~[apache-cassandra-2.1.9.jar:2.1.9]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> ~[na:1.7.0_51]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262) 
> ~[na:1.7.0_51]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  ~[na:1.7.0_51]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  [na:1.7.0_51]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
> {code}
> h3. Additional info:
> *cfstats is running fine for that table...*
> {code}
> ~ $ nodetool cfstats gccatlgsvcks.category_name_dedup
> Keyspace: gccatlgsvcks
> Read Count: 0
> Read Latency: NaN ms.
> Write Count: 0
> Write Latency: NaN ms.
> Pending Flushes: 0
> Table: category_name_dedup
> SSTable count: 6
> Space used (live): 836314727
> Space used (total): 836314727
> Space used by snapshots (total): 3621519
> Off heap memory used (total): 6930368
> SSTable Compression Ratio: 0.03725358753117693
> Number of keys (estimate): 3004
> Memtable cell count: 0
> Memtable data size: 0
> Memtable off heap memory used: 0
> Memtable switch count: 0
> Local read count: 0
> Local read latency: NaN ms
> Local write count: 0
> Local write latency: NaN ms
> Pending flushes: 0
> Bloom filter false positives: 0
> Bloom filter false ratio: 0.0
> Bloom filter space used: 5240
> Bloom filter off heap memory used: 5192
> Index summary off heap memory used: 1200
> Compression metadata off heap memory used: 6923976
> Compacted partition minimum bytes: 125
> Compacted partition maximum 

[09/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.3

[marcuse]

Merge branch 'cassandra-3.0' into cassandra-3.3


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f2174280
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f2174280
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f2174280

Branch: refs/heads/trunk
Commit: f2174280d6dcbb951b539dc558d1e1ae81f43b5c
Parents: acb7fed 442f473
Author: Marcus Eriksson 
Authored: Mon Jan 25 10:09:15 2016 +0100
Committer: Marcus Eriksson 
Committed: Mon Jan 25 10:09:15 2016 +0100

--
 CHANGES.txt |   1 +
 src/java/org/apache/cassandra/db/Memtable.java  |   9 +
 .../db/compaction/CompactionController.java |  19 +-
 .../db/compaction/CompactionControllerTest.java | 195 +++
 4 files changed, 221 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f2174280/CHANGES.txt
--
diff --cc CHANGES.txt
index ed56a78,70abffe..81657f9
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -15,9 -11,20 +15,10 @@@ Merged from 3.0
 tombstone (CASSANDRA-10743)
   * MV should use the maximum timestamp of the primary key (CASSANDRA-10910)
   * Fix potential assertion error during compaction (CASSANDRA-10944)
 - * Fix counting of received sstables in streaming (CASSANDRA-10949)
 - * Implement hints compression (CASSANDRA-9428)
 - * Fix potential assertion error when reading static columns (CASSANDRA-10903)
 - * Avoid NoSuchElementException when executing empty batch (CASSANDRA-10711)
 - * Avoid building PartitionUpdate in toString (CASSANDRA-10897)
 - * Reduce heap spent when receiving many SSTables (CASSANDRA-10797)
 - * Add back support for 3rd party auth providers to bulk loader 
(CASSANDRA-10873)
 - * Eliminate the dependency on jgrapht for UDT resolution (CASSANDRA-10653)
 - * (Hadoop) Close Clusters and Sessions in Hadoop Input/Output classes 
(CASSANDRA-10837)
 - * Fix sstableloader not working with upper case keyspace name 
(CASSANDRA-10806)
  Merged from 2.2:
 -2.2.5
+  * maxPurgeableTimestamp needs to check memtables too (CASSANDRA-9949)
   * Apply change to compaction throughput in real time (CASSANDRA-10025)
 + * (cqlsh) encode input correctly when saving history
   * Fix potential NPE on ORDER BY queries with IN (CASSANDRA-10955)
   * Start L0 STCS-compactions even if there is a L0 -> L1 compaction
 going (CASSANDRA-10979)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f2174280/src/java/org/apache/cassandra/db/Memtable.java
--
diff --cc src/java/org/apache/cassandra/db/Memtable.java
index 8e7a43c,5d5f7bf..952c045
--- a/src/java/org/apache/cassandra/db/Memtable.java
+++ b/src/java/org/apache/cassandra/db/Memtable.java
@@@ -334,37 -316,24 +338,42 @@@ public class Memtable implements Compar
  return creationTime;
  }
  
+ public long getMinTimestamp()
+ {
+ return minTimestamp;
+ }
+ 
 -class FlushRunnable extends DiskAwareRunnable
 +class FlushRunnable implements Callable
  {
 -private final ReplayPosition context;
 +public final ReplayPosition context;
  private final long estimatedSize;
 +private final ConcurrentNavigableMap toFlush;
  
  private final boolean isBatchLogTable;
 +private final SSTableMultiWriter writer;
 +
 +// keeping these to be able to log what we are actually flushing
 +private final PartitionPosition from;
 +private final PartitionPosition to;
  
 -FlushRunnable(ReplayPosition context)
 +FlushRunnable(ReplayPosition context, PartitionPosition from, 
PartitionPosition to, Directories.DataDirectory flushLocation, 
LifecycleTransaction txn)
  {
 -this.context = context;
 +this(context, partitions.subMap(from, to), flushLocation, from, 
to, txn);
 +}
 +
 +FlushRunnable(ReplayPosition context, LifecycleTransaction txn)
 +{
 +this(context, partitions, null, null, null, txn);
 +}
  
 +FlushRunnable(ReplayPosition context, 
ConcurrentNavigableMap toFlush, 
Directories.DataDirectory flushLocation, PartitionPosition from, 
PartitionPosition to, LifecycleTransaction txn)
 +{
 +this.context = context;
 +this.toFlush = toFlush;
 +this.from = from;
 +this.to = to;
  long keySize = 0;
 -for (PartitionPosition key : partitions.keySet())
 +for (PartitionPosition key : toFlush.keySet())
  {
  //  make sure we don't write 

[06/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[marcuse]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/442f4737
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/442f4737
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/442f4737

Branch: refs/heads/cassandra-3.0
Commit: 442f4737ceddd34f14210da49cee4d48b468f01e
Parents: 2f8e5f3 b5d6d4f
Author: Marcus Eriksson 
Authored: Mon Jan 25 10:05:26 2016 +0100
Committer: Marcus Eriksson 
Committed: Mon Jan 25 10:05:59 2016 +0100

--
 CHANGES.txt |   1 +
 src/java/org/apache/cassandra/db/Memtable.java  |   9 +
 .../db/compaction/CompactionController.java |  19 +-
 .../db/compaction/CompactionControllerTest.java | 195 +++
 4 files changed, 221 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/442f4737/CHANGES.txt
--
diff --cc CHANGES.txt
index bb26b20,cdc3b34..70abffe
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,30 -1,8 +1,31 @@@
 +3.0.3
 + * Update CQL documentation (CASSANDRA-10899)
 + * Check the column name, not cell name, for dropped columns when reading
 +   legacy sstables (CASSANDRA-11018)
 + * Don't attempt to index clustering values of static rows (CASSANDRA-11021)
 + * Remove checksum files after replaying hints (CASSANDRA-10947)
 + * Support passing base table metadata to custom 2i validation 
(CASSANDRA-10924)
 + * Ensure stale index entries are purged during reads (CASSANDRA-11013)
 + * Fix AssertionError when removing from list using UPDATE (CASSANDRA-10954)
 + * Fix UnsupportedOperationException when reading old sstable with range
 +   tombstone (CASSANDRA-10743)
 + * MV should use the maximum timestamp of the primary key (CASSANDRA-10910)
 + * Fix potential assertion error during compaction (CASSANDRA-10944)
 + * Fix counting of received sstables in streaming (CASSANDRA-10949)
 + * Implement hints compression (CASSANDRA-9428)
 + * Fix potential assertion error when reading static columns (CASSANDRA-10903)
 + * Avoid NoSuchElementException when executing empty batch (CASSANDRA-10711)
 + * Avoid building PartitionUpdate in toString (CASSANDRA-10897)
 + * Reduce heap spent when receiving many SSTables (CASSANDRA-10797)
 + * Add back support for 3rd party auth providers to bulk loader 
(CASSANDRA-10873)
 + * Eliminate the dependency on jgrapht for UDT resolution (CASSANDRA-10653)
 + * (Hadoop) Close Clusters and Sessions in Hadoop Input/Output classes 
(CASSANDRA-10837)
 + * Fix sstableloader not working with upper case keyspace name 
(CASSANDRA-10806)
 +Merged from 2.2:
  2.2.5
+  * maxPurgeableTimestamp needs to check memtables too (CASSANDRA-9949)
   * Apply change to compaction throughput in real time (CASSANDRA-10025)
   * Fix potential NPE on ORDER BY queries with IN (CASSANDRA-10955)
 - * Avoid over-fetching during the page of range queries (CASSANDRA-8521)
   * Start L0 STCS-compactions even if there is a L0 -> L1 compaction
 going (CASSANDRA-10979)
   * Make UUID LSB unique per process (CASSANDRA-7925)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/442f4737/src/java/org/apache/cassandra/db/Memtable.java
--
diff --cc src/java/org/apache/cassandra/db/Memtable.java
index 96b1775,fb4da72..5d5f7bf
--- a/src/java/org/apache/cassandra/db/Memtable.java
+++ b/src/java/org/apache/cassandra/db/Memtable.java
@@@ -237,12 -227,24 +240,13 @@@ public class Memtable implements Compar
  }
  }
  
 -final AtomicBTreeColumns.ColumnUpdater updater = 
previous.addAllWithSizeDelta(cf, allocator, opGroup, indexer);
 -minTimestamp = Math.min(minTimestamp, updater.minTimestamp);
 -liveDataSize.addAndGet(initialSize + updater.dataSize);
 -currentOperations.addAndGet(cf.getColumnCount() + 
(cf.isMarkedForDelete() ? 1 : 0) + cf.deletionInfo().rangeCount());
 -return updater.colUpdateTimeDelta;
 -}
 -
 -// for debugging
 -public String contents()
 -{
 -StringBuilder builder = new StringBuilder();
 -builder.append("{");
 -for (Map.Entry entry : 
rows.entrySet())
 -{
 -builder.append(entry.getKey()).append(": 
").append(entry.getValue()).append(", ");
 -}
 -builder.append("}");
 -return builder.toString();
 +long[] pair = previous.addAllWithSizeDelta(update, opGroup, indexer);
++minTimestamp = Math.min(minTimestamp, previous.stats().minTimestamp);
 +liveDataSize.addAndGet(initialSize + pair[0]);
 +columnsCollector.update(update.columns());
 +

[04/10] cassandra git commit: maxPurgeableTimestamp needs to check memtables too

[marcuse]

maxPurgeableTimestamp needs to check memtables too

Patch by Stefania Alborghetti; reviewed by marcuse for CASSANDRA-9949


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b5d6d4f7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b5d6d4f7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b5d6d4f7

Branch: refs/heads/trunk
Commit: b5d6d4f72299a0b08ce3279aade507e2a999acc6
Parents: b4d67c9
Author: Stefania Alborghetti 
Authored: Mon Jan 4 16:24:51 2016 +0100
Committer: Stefania Alborghetti 
Committed: Mon Jan 25 13:30:05 2016 +0800

--
 CHANGES.txt |   1 +
 .../apache/cassandra/db/AtomicBTreeColumns.java |  14 +-
 src/java/org/apache/cassandra/db/Memtable.java  |  15 +-
 .../db/compaction/CompactionController.java |  21 +-
 .../db/compaction/LazilyCompactedRow.java   |   2 +-
 .../db/compaction/CompactionControllerTest.java | 191 +++
 6 files changed, 232 insertions(+), 12 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b5d6d4f7/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 59997ff..cdc3b34 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.5
+ * maxPurgeableTimestamp needs to check memtables too (CASSANDRA-9949)
  * Apply change to compaction throughput in real time (CASSANDRA-10025)
  * Fix potential NPE on ORDER BY queries with IN (CASSANDRA-10955)
  * Avoid over-fetching during the page of range queries (CASSANDRA-8521)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b5d6d4f7/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java 
b/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
index 710b289..f5b7712 100644
--- a/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
+++ b/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
@@ -198,7 +198,7 @@ public class AtomicBTreeColumns extends ColumnFamily
  *
  * @return the difference in size seen after merging the given columns
  */
-public Pair addAllWithSizeDelta(final ColumnFamily cm, 
MemtableAllocator allocator, OpOrder.Group writeOp, Updater indexer)
+public ColumnUpdater addAllWithSizeDelta(final ColumnFamily cm, 
MemtableAllocator allocator, OpOrder.Group writeOp, Updater indexer)
 {
 ColumnUpdater updater = new ColumnUpdater(this, cm.metadata, 
allocator, writeOp, indexer);
 DeletionInfo inputDeletionInfoCopy = null;
@@ -237,7 +237,7 @@ public class AtomicBTreeColumns extends ColumnFamily
 {
 indexer.updateRowLevelIndexes();
 updater.finish();
-return Pair.create(updater.dataSize, 
updater.colUpdateTimeDelta);
+return updater;
 }
 else if (!monitorOwned)
 {
@@ -429,7 +429,7 @@ public class AtomicBTreeColumns extends ColumnFamily
 }
 
 // the function we provide to the btree utilities to perform any column 
replacements
-private static final class ColumnUpdater implements UpdateFunction
+static final class ColumnUpdater implements UpdateFunction
 {
 final AtomicBTreeColumns updating;
 final CFMetaData metadata;
@@ -442,6 +442,7 @@ public class AtomicBTreeColumns extends ColumnFamily
 long colUpdateTimeDelta = Long.MAX_VALUE;
 final MemtableAllocator.DataReclaimer reclaimer;
 List inserted; // TODO: replace with walk of aborted BTree
+long minTimestamp = Long.MAX_VALUE;
 
 private ColumnUpdater(AtomicBTreeColumns updating, CFMetaData 
metadata, MemtableAllocator allocator, OpOrder.Group writeOp, Updater indexer)
 {
@@ -462,6 +463,7 @@ public class AtomicBTreeColumns extends ColumnFamily
 if (inserted == null)
 inserted = new ArrayList<>();
 inserted.add(insert);
+minTimestamp = Math.min(minTimestamp, insert.timestamp());
 return insert;
 }
 
@@ -469,6 +471,11 @@ public class AtomicBTreeColumns extends ColumnFamily
 {
 Cell reconciled = existing.reconcile(update);
 indexer.update(existing, reconciled);
+// pick the smallest timestamp because we want to be consistent 
with the logic applied when inserting
+// a cell in apply(Cell insert) above. For example given 3 
timestamps where T3 < T2 < T1 then we want
+// [apply(T1) -> apply(T2) -> apply(T3)] and [apply(T3) -> 
apply(T2) -> apply(T1)] to both return 

[01/10] cassandra git commit: maxPurgeableTimestamp needs to check memtables too

[marcuse]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.2 b4d67c9e5 -> b5d6d4f72
  refs/heads/cassandra-3.0 2f8e5f346 -> 442f4737c
  refs/heads/cassandra-3.3 acb7fed1d -> f2174280d
  refs/heads/trunk 72790dc8e -> 4abd99363


maxPurgeableTimestamp needs to check memtables too

Patch by Stefania Alborghetti; reviewed by marcuse for CASSANDRA-9949


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b5d6d4f7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b5d6d4f7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b5d6d4f7

Branch: refs/heads/cassandra-2.2
Commit: b5d6d4f72299a0b08ce3279aade507e2a999acc6
Parents: b4d67c9
Author: Stefania Alborghetti 
Authored: Mon Jan 4 16:24:51 2016 +0100
Committer: Stefania Alborghetti 
Committed: Mon Jan 25 13:30:05 2016 +0800

--
 CHANGES.txt |   1 +
 .../apache/cassandra/db/AtomicBTreeColumns.java |  14 +-
 src/java/org/apache/cassandra/db/Memtable.java  |  15 +-
 .../db/compaction/CompactionController.java |  21 +-
 .../db/compaction/LazilyCompactedRow.java   |   2 +-
 .../db/compaction/CompactionControllerTest.java | 191 +++
 6 files changed, 232 insertions(+), 12 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b5d6d4f7/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 59997ff..cdc3b34 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.5
+ * maxPurgeableTimestamp needs to check memtables too (CASSANDRA-9949)
  * Apply change to compaction throughput in real time (CASSANDRA-10025)
  * Fix potential NPE on ORDER BY queries with IN (CASSANDRA-10955)
  * Avoid over-fetching during the page of range queries (CASSANDRA-8521)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b5d6d4f7/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java 
b/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
index 710b289..f5b7712 100644
--- a/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
+++ b/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
@@ -198,7 +198,7 @@ public class AtomicBTreeColumns extends ColumnFamily
  *
  * @return the difference in size seen after merging the given columns
  */
-public Pair addAllWithSizeDelta(final ColumnFamily cm, 
MemtableAllocator allocator, OpOrder.Group writeOp, Updater indexer)
+public ColumnUpdater addAllWithSizeDelta(final ColumnFamily cm, 
MemtableAllocator allocator, OpOrder.Group writeOp, Updater indexer)
 {
 ColumnUpdater updater = new ColumnUpdater(this, cm.metadata, 
allocator, writeOp, indexer);
 DeletionInfo inputDeletionInfoCopy = null;
@@ -237,7 +237,7 @@ public class AtomicBTreeColumns extends ColumnFamily
 {
 indexer.updateRowLevelIndexes();
 updater.finish();
-return Pair.create(updater.dataSize, 
updater.colUpdateTimeDelta);
+return updater;
 }
 else if (!monitorOwned)
 {
@@ -429,7 +429,7 @@ public class AtomicBTreeColumns extends ColumnFamily
 }
 
 // the function we provide to the btree utilities to perform any column 
replacements
-private static final class ColumnUpdater implements UpdateFunction
+static final class ColumnUpdater implements UpdateFunction
 {
 final AtomicBTreeColumns updating;
 final CFMetaData metadata;
@@ -442,6 +442,7 @@ public class AtomicBTreeColumns extends ColumnFamily
 long colUpdateTimeDelta = Long.MAX_VALUE;
 final MemtableAllocator.DataReclaimer reclaimer;
 List inserted; // TODO: replace with walk of aborted BTree
+long minTimestamp = Long.MAX_VALUE;
 
 private ColumnUpdater(AtomicBTreeColumns updating, CFMetaData 
metadata, MemtableAllocator allocator, OpOrder.Group writeOp, Updater indexer)
 {
@@ -462,6 +463,7 @@ public class AtomicBTreeColumns extends ColumnFamily
 if (inserted == null)
 inserted = new ArrayList<>();
 inserted.add(insert);
+minTimestamp = Math.min(minTimestamp, insert.timestamp());
 return insert;
 }
 
@@ -469,6 +471,11 @@ public class AtomicBTreeColumns extends ColumnFamily
 {
 Cell reconciled = existing.reconcile(update);
 indexer.update(existing, reconciled);
+// pick the smallest timestamp because we want to be consistent 
with the logic applied 

[05/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[marcuse]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/442f4737
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/442f4737
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/442f4737

Branch: refs/heads/cassandra-3.3
Commit: 442f4737ceddd34f14210da49cee4d48b468f01e
Parents: 2f8e5f3 b5d6d4f
Author: Marcus Eriksson 
Authored: Mon Jan 25 10:05:26 2016 +0100
Committer: Marcus Eriksson 
Committed: Mon Jan 25 10:05:59 2016 +0100

--
 CHANGES.txt |   1 +
 src/java/org/apache/cassandra/db/Memtable.java  |   9 +
 .../db/compaction/CompactionController.java |  19 +-
 .../db/compaction/CompactionControllerTest.java | 195 +++
 4 files changed, 221 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/442f4737/CHANGES.txt
--
diff --cc CHANGES.txt
index bb26b20,cdc3b34..70abffe
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,30 -1,8 +1,31 @@@
 +3.0.3
 + * Update CQL documentation (CASSANDRA-10899)
 + * Check the column name, not cell name, for dropped columns when reading
 +   legacy sstables (CASSANDRA-11018)
 + * Don't attempt to index clustering values of static rows (CASSANDRA-11021)
 + * Remove checksum files after replaying hints (CASSANDRA-10947)
 + * Support passing base table metadata to custom 2i validation 
(CASSANDRA-10924)
 + * Ensure stale index entries are purged during reads (CASSANDRA-11013)
 + * Fix AssertionError when removing from list using UPDATE (CASSANDRA-10954)
 + * Fix UnsupportedOperationException when reading old sstable with range
 +   tombstone (CASSANDRA-10743)
 + * MV should use the maximum timestamp of the primary key (CASSANDRA-10910)
 + * Fix potential assertion error during compaction (CASSANDRA-10944)
 + * Fix counting of received sstables in streaming (CASSANDRA-10949)
 + * Implement hints compression (CASSANDRA-9428)
 + * Fix potential assertion error when reading static columns (CASSANDRA-10903)
 + * Avoid NoSuchElementException when executing empty batch (CASSANDRA-10711)
 + * Avoid building PartitionUpdate in toString (CASSANDRA-10897)
 + * Reduce heap spent when receiving many SSTables (CASSANDRA-10797)
 + * Add back support for 3rd party auth providers to bulk loader 
(CASSANDRA-10873)
 + * Eliminate the dependency on jgrapht for UDT resolution (CASSANDRA-10653)
 + * (Hadoop) Close Clusters and Sessions in Hadoop Input/Output classes 
(CASSANDRA-10837)
 + * Fix sstableloader not working with upper case keyspace name 
(CASSANDRA-10806)
 +Merged from 2.2:
  2.2.5
+  * maxPurgeableTimestamp needs to check memtables too (CASSANDRA-9949)
   * Apply change to compaction throughput in real time (CASSANDRA-10025)
   * Fix potential NPE on ORDER BY queries with IN (CASSANDRA-10955)
 - * Avoid over-fetching during the page of range queries (CASSANDRA-8521)
   * Start L0 STCS-compactions even if there is a L0 -> L1 compaction
 going (CASSANDRA-10979)
   * Make UUID LSB unique per process (CASSANDRA-7925)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/442f4737/src/java/org/apache/cassandra/db/Memtable.java
--
diff --cc src/java/org/apache/cassandra/db/Memtable.java
index 96b1775,fb4da72..5d5f7bf
--- a/src/java/org/apache/cassandra/db/Memtable.java
+++ b/src/java/org/apache/cassandra/db/Memtable.java
@@@ -237,12 -227,24 +240,13 @@@ public class Memtable implements Compar
  }
  }
  
 -final AtomicBTreeColumns.ColumnUpdater updater = 
previous.addAllWithSizeDelta(cf, allocator, opGroup, indexer);
 -minTimestamp = Math.min(minTimestamp, updater.minTimestamp);
 -liveDataSize.addAndGet(initialSize + updater.dataSize);
 -currentOperations.addAndGet(cf.getColumnCount() + 
(cf.isMarkedForDelete() ? 1 : 0) + cf.deletionInfo().rangeCount());
 -return updater.colUpdateTimeDelta;
 -}
 -
 -// for debugging
 -public String contents()
 -{
 -StringBuilder builder = new StringBuilder();
 -builder.append("{");
 -for (Map.Entry entry : 
rows.entrySet())
 -{
 -builder.append(entry.getKey()).append(": 
").append(entry.getValue()).append(", ");
 -}
 -builder.append("}");
 -return builder.toString();
 +long[] pair = previous.addAllWithSizeDelta(update, opGroup, indexer);
++minTimestamp = Math.min(minTimestamp, previous.stats().minTimestamp);
 +liveDataSize.addAndGet(initialSize + pair[0]);
 +columnsCollector.update(update.columns());
 +

[jira] [Commented] (CASSANDRA-8180) Optimize disk seek using min/max column name meta data when the LIMIT clause is used

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15114909#comment-15114909
 ] 

Stefania commented on CASSANDRA-8180:
-

Pushed a 
[commit|https://github.com/stef1927/cassandra/commit/d5cfc6fd56d50eda5d9c510591bae1d66e17ec59]
 where we don't use the lower bound in the presence of tombstones and 
{{DeleteTest}} is now passing. CI is still pending however. I would like to 
point out that {{iter.partitionLevelDeletion()}} is currently called for all 
sstables by {{queryMemtableAndDiskInternal()}} and therefore, in the presence 
of tombstones, we have to access the sstable anyway.

bq. Tombstones. A DELETE WHERE pk = ? AND ck1 = ? in a table with key (pk, ck1, 
ck2) will generate one.

This case existed already in 
{{DeleteTest.testDeleteWithRangeAndTwoClusteringColumns()}}. It does not fail 
though, because the clustering comparator compares prefix values from first to 
last and so it works fine with incomplete prefixes.

bq. An empty row will not work correctly as a lower bound. It does not sort as 
needed with respect to tombstone bounds, which should also be included in the 
test (more specifically, one that adds a row, flushes, deletes same row, 
flushes again, then checks if it resurfaces-- I believe this would break with 
the current code). Use a RangeTombstoneBound with DeletionTime.LIVE as the 
deletion time and a bound obtained by RangeTombstone.Bound.inclusiveOpen, which 
should do the right thing in both directions.

I don't think this matters any longer if we never replace a tombstone with a 
lower bound but, I could not reproduce any failures even when using a lower 
bound for tombstones (other than for one sided range tombstones). Here is what 
I tried:

- add a row, flush, delete same row, flush again => FINE (we don't even use 
merge iterator)
- add two rows in same partition, flush, delete one and flush, delete other one 
and flush, FINE again
- add three rows in same partition, flush, delete one and flush, delete other 
one and flush, FINE again

We can replace {{BTreeRow.emptyRow(ret)}} with {{new 
RangeTombstoneBoundMarker(RangeTombstone.Bound.inclusiveOpen(filter.isReversed(),
 ret.getRawValues()), DeletionTime.LIVE)}} if there is still a valid reason and 
ideally a failing test would be useful. I will clean up {{lowerBound()}} and 
add some comments to it in the next round once we have reached a decision.

> Optimize disk seek using min/max column name meta data when the LIMIT clause 
> is used
> 
>
> Key: CASSANDRA-8180
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8180
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
> Environment: Cassandra 2.0.10
>Reporter: DOAN DuyHai
>Assignee: Stefania
>Priority: Minor
> Fix For: 3.x
>
> Attachments: 8180_001.yaml, 8180_002.yaml
>
>
> I was working on an example of sensor data table (timeseries) and face a use 
> case where C* does not optimize read on disk.
> {code}
> cqlsh:test> CREATE TABLE test(id int, col int, val text, PRIMARY KEY(id,col)) 
> WITH CLUSTERING ORDER BY (col DESC);
> cqlsh:test> INSERT INTO test(id, col , val ) VALUES ( 1, 10, '10');
> ...
> >nodetool flush test test
> ...
> cqlsh:test> INSERT INTO test(id, col , val ) VALUES ( 1, 20, '20');
> ...
> >nodetool flush test test
> ...
> cqlsh:test> INSERT INTO test(id, col , val ) VALUES ( 1, 30, '30');
> ...
> >nodetool flush test test
> {code}
> After that, I activate request tracing:
> {code}
> cqlsh:test> SELECT * FROM test WHERE id=1 LIMIT 1;
>  activity  | 
> timestamp| source| source_elapsed
> ---+--+---+
> execute_cql3_query | 
> 23:48:46,498 | 127.0.0.1 |  0
> Parsing SELECT * FROM test WHERE id=1 LIMIT 1; | 
> 23:48:46,498 | 127.0.0.1 | 74
>Preparing statement | 
> 23:48:46,499 | 127.0.0.1 |253
>   Executing single-partition query on test | 
> 23:48:46,499 | 127.0.0.1 |930
>   Acquiring sstable references | 
> 23:48:46,499 | 127.0.0.1 |943
>Merging memtable tombstones | 
> 23:48:46,499 | 127.0.0.1 |   1032
>Key cache hit for sstable 3 | 
> 23:48:46,500 | 127.0.0.1 |   1160
>Seeking to partition beginning in data file 

[jira] [Commented] (CASSANDRA-10657) Re-enable/improve value skipping

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15114953#comment-15114953
 ] 

Sylvain Lebresne commented on CASSANDRA-10657:
--

bq. As this is a follow-up to 10655, is the new behavior expected?

Sorry for missing that but yes, it is expected. I've created a [pull 
request|https://github.com/riptano/cassandra-dtest/pull/769] to modify that 
test so it doesn't make such strong (and now invalid) assumption while 
hopefully still testing something useful.

> Re-enable/improve value skipping
> 
>
> Key: CASSANDRA-10657
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10657
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.4
>
>
> This is a followup to CASSANDRA-10655, to re-enable the optimization of 
> skipping values for the columns that are not requested by users in a CQL 
> query. See CASSANDRA-10655 for why it was disabled, the goal here is to 
> re-enable it minus the bugs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-11060) Allow DTCS old SSTable filtering to use min timestamp instead of max

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson resolved CASSANDRA-11060.
-
Resolution: Won't Fix

Using max_sstable_age_days has been deprecated in favor of limiting the size of 
the time windows, background here: CASSANDRA-10280

> Allow DTCS old SSTable filtering to use min timestamp instead of max
> 
>
> Key: CASSANDRA-11060
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11060
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sam Bisbee
>  Labels: dtcs
>
> We have observed a DTCS behavior when using TTLs where SSTables are never or 
> very rarely fully expired due to compaction, allowing expired data to be 
> "stuck" in large partially expired SSTables.
> This is because compaction filtering is performed on the max timestamp, which 
> continues to grow as SSTables are compacted together. This means they will 
> never move past max_sstable_age_days. With a sufficiently large TTL, like 30 
> days, this allows old but not expired SSTables to continue combining and 
> never become fully expired, even with a max_sstable_age_days of 1.
> As a result we have seen expired data hang around in large SSTables for over 
> six months longer than it should have. This is obviously wasteful and a disk 
> capacity issue.
> As a result we have been running an extended version of DTCS called MTCS in 
> some deployments. The only change is that it uses min timestamp instead of 
> max for compaction filtering (filterOldSSTables()). This allows SSTables to 
> move beyond max_sstable_age_days and stop compacting, which means the entire 
> SSTable can become fully expired and be dropped off disk as intended.
> You can see and test MTCS here: https://github.com/threatstack/mtcs
> I am not advocating that MTCS be its own stand alone compaction strategy. 
> However, I would like to see a configuration option for DTCS that allows you 
> to specify whether old SSTables should be filtered on min or max timestamp.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11060) Allow DTCS old SSTable filtering to use min timestamp instead of max

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15114977#comment-15114977
 ] 

Marcus Eriksson commented on CASSANDRA-11060:
-

also see CASSANDRA-11056

> Allow DTCS old SSTable filtering to use min timestamp instead of max
> 
>
> Key: CASSANDRA-11060
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11060
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sam Bisbee
>  Labels: dtcs
>
> We have observed a DTCS behavior when using TTLs where SSTables are never or 
> very rarely fully expired due to compaction, allowing expired data to be 
> "stuck" in large partially expired SSTables.
> This is because compaction filtering is performed on the max timestamp, which 
> continues to grow as SSTables are compacted together. This means they will 
> never move past max_sstable_age_days. With a sufficiently large TTL, like 30 
> days, this allows old but not expired SSTables to continue combining and 
> never become fully expired, even with a max_sstable_age_days of 1.
> As a result we have seen expired data hang around in large SSTables for over 
> six months longer than it should have. This is obviously wasteful and a disk 
> capacity issue.
> As a result we have been running an extended version of DTCS called MTCS in 
> some deployments. The only change is that it uses min timestamp instead of 
> max for compaction filtering (filterOldSSTables()). This allows SSTables to 
> move beyond max_sstable_age_days and stop compacting, which means the entire 
> SSTable can become fully expired and be dropped off disk as intended.
> You can see and test MTCS here: https://github.com/threatstack/mtcs
> I am not advocating that MTCS be its own stand alone compaction strategy. 
> However, I would like to see a configuration option for DTCS that allows you 
> to specify whether old SSTables should be filtered on min or max timestamp.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9472) Reintroduce off heap memtables

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115010#comment-15115010
 ] 

Benedict commented on CASSANDRA-9472:
-

Thanks.

bq. Did you mean to override deletionInfo() in AtomicBTreePartition? I've 
removed it in the nits below.

I did, I just forgot to put corresponding functionality into {{EnsureOnHeap}}, 
I've reintroduced it to my copy of the branch, with it added back

bq. Looking at the last commit, is it necessary to make a local copy of 
this.peer? It's a final field initialized in the constructor so why the copy?

I assumed that (as it used to be) the peer could be set to zero on free, and 
that this was how you were encountering the problem.  I see now it is only the 
fake NativeCell that causes problems; I do wonder if we shouldn't simply 
allocate it for real, to avoid the check for every single call to a cell.  It's 
also highly unlikely to ever appear in anyone's debugger if it's only that 
specific cell, so we could simply ignore the problem entirely.

bq. How do you run the tests given that you've changed cassandra.yaml to use 
heap_buffers again

You're right.  I ran the first batch with it offheap, and then changed it back 
to restore it as I don't like having that modified, but it was premature. I've 
restored offheap_objects to the base cassandra.yaml

> Reintroduce off heap memtables
> --
>
> Key: CASSANDRA-9472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9472
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Benedict
>Assignee: Benedict
> Fix For: 3.x
>
>
> CASSANDRA-8099 removes off heap memtables. We should reintroduce them ASAP.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11034) consistent_reads_after_move_test is failing on trunk

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-11034:
-
Assignee: Marcus Eriksson

> consistent_reads_after_move_test is failing on trunk
> 
>
> Key: CASSANDRA-11034
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11034
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Philip Thompson
>Assignee: Marcus Eriksson
>  Labels: dtest
> Fix For: 3.x
>
> Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log, 
> node3.log, node3_debug.log
>
>
> The novnode dtest 
> {{consistent_bootstrap_test.TestBootstrapConsistency.consistent_reads_after_move_test}}
>  is failing on trunk. See an example failure 
> [here|http://cassci.datastax.com/job/trunk_novnode_dtest/274/testReport/consistent_bootstrap_test/TestBootstrapConsistency/consistent_reads_after_move_test/].
> On trunk I am getting an OOM of one of my C* nodes [node3], which is what 
> causes the nodetool move to fail. Logs are attached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11034) consistent_reads_after_move_test is failing on trunk

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115020#comment-15115020
 ] 

Sylvain Lebresne commented on CASSANDRA-11034:
--

Assigning to [~krummas] because from the CI history, this look like it is 
likely a regression from CASSANDRA-6696. That is, it starts failing 
[there|http://cassci.datastax.com/job/trunk_novnode_dtest/259/changes], which 
has a few other minor changes, but the server side OOM happens in {{Splitter}} 
which was, I believe, introduced by this patch (see trace 
[here|http://cassci.datastax.com/job/trunk_novnode_dtest/259/testReport/consistent_bootstrap_test/TestBootstrapConsistency/consistent_reads_after_move_test_2/]).

> consistent_reads_after_move_test is failing on trunk
> 
>
> Key: CASSANDRA-11034
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11034
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Philip Thompson
>Assignee: Marcus Eriksson
>  Labels: dtest
> Fix For: 3.x
>
> Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log, 
> node3.log, node3_debug.log
>
>
> The novnode dtest 
> {{consistent_bootstrap_test.TestBootstrapConsistency.consistent_reads_after_move_test}}
>  is failing on trunk. See an example failure 
> [here|http://cassci.datastax.com/job/trunk_novnode_dtest/274/testReport/consistent_bootstrap_test/TestBootstrapConsistency/consistent_reads_after_move_test/].
> On trunk I am getting an OOM of one of my C* nodes [node3], which is what 
> causes the nodetool move to fail. Logs are attached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: 9986: Remove SliceableUnfilteredRowIterator

[benedict]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 4abd99363 -> 4fb559b58


9986: Remove SliceableUnfilteredRowIterator

Unifies more read path access paths onto Slices, by constructing a Slices object
for all ClusteringIndexFilter that would have previously sliced the iterator 
themselves.
All iterators now accept a Slices object, and iterate over its contents as 
necessary.


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4fb559b5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4fb559b5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4fb559b5

Branch: refs/heads/trunk
Commit: 4fb559b58a8a8419e91a786e44f25f6506f4d037
Parents: 4abd993
Author: Benedict Elliott Smith 
Authored: Sun Sep 13 01:35:07 2015 +0100
Committer: Benedict Elliott Smith 
Committed: Mon Jan 25 10:52:04 2016 +

--
 .../org/apache/cassandra/db/ReadResponse.java   |  3 +-
 .../db/SinglePartitionReadCommand.java  |  8 +-
 src/java/org/apache/cassandra/db/Slices.java| 80 
 .../columniterator/AbstractSSTableIterator.java | 33 ++--
 .../db/columniterator/SSTableIterator.java  |  8 +-
 .../columniterator/SSTableReversedIterator.java |  8 +-
 .../db/filter/ClusteringIndexFilter.java|  9 +--
 .../db/filter/ClusteringIndexNamesFilter.java   | 50 ++--
 .../db/filter/ClusteringIndexSliceFilter.java   |  6 +-
 .../db/partitions/AbstractBTreePartition.java   | 34 -
 .../db/partitions/PartitionUpdate.java  | 11 +--
 .../db/rows/SliceableUnfilteredRowIterator.java | 39 --
 .../io/sstable/format/SSTableReader.java|  6 +-
 .../io/sstable/format/big/BigTableReader.java   | 19 +++--
 .../io/sstable/format/big/BigTableScanner.java  |  2 +-
 .../partition/PartitionImplementationTest.java  | 33 
 .../cassandra/io/sstable/LegacySSTableTest.java |  7 +-
 .../sstable/format/ClientModeSSTableTest.java   |  8 +-
 18 files changed, 86 insertions(+), 278 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4fb559b5/src/java/org/apache/cassandra/db/ReadResponse.java
--
diff --git a/src/java/org/apache/cassandra/db/ReadResponse.java 
b/src/java/org/apache/cassandra/db/ReadResponse.java
index 41f0d5d..c114315 100644
--- a/src/java/org/apache/cassandra/db/ReadResponse.java
+++ b/src/java/org/apache/cassandra/db/ReadResponse.java
@@ -283,13 +283,12 @@ public abstract class ReadResponse
 {
 ImmutableBTreePartition partition = toReturn.get(idx++);
 
-
 ClusteringIndexFilter filter = 
command.clusteringIndexFilter(partition.partitionKey());
 
 // Pre-3.0, we didn't have a way to express exclusivity 
for non-composite comparators, so all slices were
 // inclusive on both ends. If we have exclusive slice 
ends, we need to filter the results here.
 if (!command.metadata().isCompound())
-return 
filter.filter(partition.sliceableUnfilteredIterator(command.columnFilter(), 
filter.isReversed()));
+return 
partition.unfilteredIterator(command.columnFilter(), 
filter.getSlices(command.metadata()), filter.isReversed());
 
 return 
partition.unfilteredIterator(command.columnFilter(), Slices.ALL, 
filter.isReversed());
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4fb559b5/src/java/org/apache/cassandra/db/SinglePartitionReadCommand.java
--
diff --git a/src/java/org/apache/cassandra/db/SinglePartitionReadCommand.java 
b/src/java/org/apache/cassandra/db/SinglePartitionReadCommand.java
index a1de3d6..9ad9ba3 100644
--- a/src/java/org/apache/cassandra/db/SinglePartitionReadCommand.java
+++ b/src/java/org/apache/cassandra/db/SinglePartitionReadCommand.java
@@ -549,7 +549,7 @@ public class SinglePartitionReadCommand extends ReadCommand
 
 sstable.incrementReadCount();
 @SuppressWarnings("resource") // 'iter' is added to iterators 
which is closed on exception, or through the closing of the final merged 
iterator
-UnfilteredRowIterator iter = 
filter.filter(sstable.iterator(partitionKey(), columnFilter(), 
filter.isReversed(), isForThrift()));
+UnfilteredRowIterator iter = sstable.iterator(partitionKey(), 
filter.getSlices(metadata()), columnFilter(), filter.isReversed(), 
isForThrift());
 if (!sstable.isRepaired())
 oldestUnrepairedTombstone = 
Math.min(oldestUnrepairedTombstone, sstable.getMinLocalDeletionTime());
 
@@ -569,7 +569,7 @@ public class 

[jira] [Updated] (CASSANDRA-10912) resumable_bootstrap_test dtest flaps

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-10912:
-
Assignee: Yuki Morishita

> resumable_bootstrap_test dtest flaps
> 
>
> Key: CASSANDRA-10912
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10912
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Yuki Morishita
> Fix For: 3.0.x
>
>
> {{bootstrap_test.py:TestBootstrap.resumable_bootstrap_test}} test flaps when 
> a node fails to start listening for connections via CQL:
> {code}
> 21 Dec 2015 10:07:48 [node3] Missing: ['Starting listening for CQL clients']:
> {code}
> I've seen it on 2.2 HEAD:
> http://cassci.datastax.com/job/cassandra-2.2_dtest/449/testReport/bootstrap_test/TestBootstrap/resumable_bootstrap_test/history/
> and 3.0 HEAD:
> http://cassci.datastax.com/job/cassandra-3.0_dtest/444/testReport/junit/bootstrap_test/TestBootstrap/resumable_bootstrap_test/
> and trunk:
> http://cassci.datastax.com/job/trunk_dtest/838/testReport/junit/bootstrap_test/TestBootstrap/resumable_bootstrap_test/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10836) Data integrity flappers in upgrade_tests dtests

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115044#comment-15115044
 ] 

Sylvain Lebresne commented on CASSANDRA-10836:
--

I strongly suspect this is an issue with the tests. All those tests are doing 
conditional updates but use non SERIAL reads, and since the consistency for 
both commit and read defaults to ONE, it is expected that there is no specific 
consistency guarantees and the flapping is not surprising. I've created a [pull 
request|https://github.com/riptano/cassandra-dtest/pull/770] to fix that and 
use SERIAL reads. There was also a mix of conditional and non-conditional 
updates in those tests, which is equally dodgy, so the pull request changes 
those so we can hopefully get more reliable testing.

I've checked that all the modified test pass with the change, but given the 
tests were flapping somewhat irregularly, I think we'll have to commit and wait 
to see if this does fix the issue on the long run (but I'm confident we _at 
least_ need the changes of the pull request if we want reliable test).

> Data integrity flappers in upgrade_tests dtests
> ---
>
> Key: CASSANDRA-10836
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10836
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Jim Witschey
>Priority: Critical
> Fix For: 3.0.x, 3.x
>
>
> A number of the {{upgrade_tests}} in dtest are flapping with data integrity 
> bugs. For instance:
> http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/422/testReport/upgrade_tests.cql_tests/TestCQLNodes3RF3/conditional_delete_test/history/
> http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/421/testReport/upgrade_tests.cql_tests/TestCQLNodes3RF3/cas_and_list_index_test/history/
> This needs further digging to get good debug information; it could well be, 
> e.g., a race condition in the test. I have not reproduced locally.
> /cc [~philipthompson]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10688) Stack overflow from SSTableReader$InstanceTidier.runOnClose in Leak Detector

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115053#comment-15115053
 ] 

Benedict commented on CASSANDRA-10688:
--

OK, LGTM.

Personally, I would have wrapped the Stack, so that fetch/return of 
{{InProgressVisit}} were guaranteed to be paired, and introduced no extra 
boilerplate in the main loop (so that the stack pop simply zeroed the top 
{{InProgressVisit}} instead of removing it and placing it on another queue).  
It's up to you if you want to make that change or not.


> Stack overflow from SSTableReader$InstanceTidier.runOnClose in Leak Detector
> 
>
> Key: CASSANDRA-10688
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10688
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths, Testing
>Reporter: Jeremiah Jordan
>Assignee: Ariel Weisberg
> Fix For: 3.0.x
>
>
> Running some tests against cassandra-3.0 
> 9fc957cf3097e54ccd72e51b2d0650dc3e83eae0
> The tests are just running cassandra-stress write and read while adding and 
> removing nodes from the cluster.  After the test runs when I go back through 
> logs I find the following Stackoverflow fairly often:
> ERROR [Strong-Reference-Leak-Detector:1] 2015-11-11 00:04:10,638  
> Ref.java:413 - Stackoverflow [private java.lang.Runnable 
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier.runOnClose,
>  final java.lang.Runnable 
> org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache.andThen, 
> final org.apache.cassandra.cache.InstrumentingCache 
> org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys.cache, private 
> final org.apache.cassandra.cache.ICache 
> org.apache.cassandra.cache.InstrumentingCache.map, private final 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap 
> org.apache.cassandra.cache.ConcurrentLinkedHashCache.map, final 
> com.googlecode.concurrentlinkedhashmap.LinkedDeque 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap.evictionDeque, 
> com.googlecode.concurrentlinkedhashmap.Linked 
> com.googlecode.concurrentlinkedhashmap.LinkedDeque.first, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> ... (repeated a whole bunch more)  
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.next, 
> final java.lang.Object 
> com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node.key, 
> public final byte[] org.apache.cassandra.cache.KeyCacheKey.key



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10697) Leak detected while running offline scrub

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115066#comment-15115066
 ] 

Benedict commented on CASSANDRA-10697:
--

I've pushed a fix for this 
[here|https://github.com/belliottsmith/cassandra/tree/10697-2.1]

It would be helpful if test engineering could have a look at the dtest that is 
meant to check this functionality, to try and get it to exercise this problem. 
I'm afraid my python is very rusty and I'm only here for a few more days.

In the process of merging this patch upwards, I found that it looks very much 
like scrub is not working at all in 2.2 upwards, as the transaction we are 
using doesn't get committed.  So the newly scrubbed sstables should be being 
deleted, and the old ones restored.  Could test engineering also check why the 
dtest isn't catching this problem?

> Leak detected while running offline scrub
> -
>
> Key: CASSANDRA-10697
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10697
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: C* 2.1.9 on Debian Wheezy
>Reporter: mlowicki
>Assignee: Benedict
>Priority: Critical
>
> I got couple of those:
> {code}
> ERROR 05:09:15 LEAK DETECTED: a reference 
> (org.apache.cassandra.utils.concurrent.Ref$State@3b60e162) to class 
> org.apache.cassandra.io.sstable.SSTableReader$InstanceTidier@1433208674:/var/lib/cassandra/data/sync/entity2-e24b5040199b11e5a30f75bb514ae072/sync-entity2-ka-405434
>  was not released before the reference was garbage collected
> {code}
> and then:
> {code}
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
> at 
> org.apache.cassandra.io.compress.CompressedRandomAccessReader.decompressChunk(CompressedRandomAccessReader.java:99)
> at 
> org.apache.cassandra.io.compress.CompressedRandomAccessReader.reBuffer(CompressedRandomAccessReader.java:81)
> at 
> org.apache.cassandra.io.util.RandomAccessReader.read(RandomAccessReader.java:353)
> at java.io.RandomAccessFile.readFully(RandomAccessFile.java:444)
> at java.io.RandomAccessFile.readFully(RandomAccessFile.java:424)
> at 
> org.apache.cassandra.io.util.RandomAccessReader.readBytes(RandomAccessReader.java:378)
> at 
> org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:348)
> at 
> org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:327)
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:397)
> at 
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:381)
> at 
> org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:75)
> at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:52)
> at 
> org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:46)
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> at 
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.hasNext(SSTableIdentityIterator.java:120)
> at 
> org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:202)
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> at 
> com.google.common.collect.Iterators$7.computeNext(Iterators.java:645)
> at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> at 
> org.apache.cassandra.db.ColumnIndex$Builder.buildForCompaction(ColumnIndex.java:165)
> at 
> org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:121)
> at 
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:192)
> at 
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:127)
> at 
> org.apache.cassandra.io.sstable.SSTableRewriter.tryAppend(SSTableRewriter.java:158)
> at 
> org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:220)
> at 
> org.apache.cassandra.tools.StandaloneScrubber.main(StandaloneScrubber.java:116)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10174) x

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-10174:
-
Summary: x  (was: Range tombstones through thrift don't handle static 
columns probably)

> x
> -
>
> Key: CASSANDRA-10174
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10174
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.0.0 rc2
>
>
> When inserting range tombstones through thrift in a "static" compact table, 
> the current code doesn't properly delete the static components of the table 
> (even when it should).
> To the best of my knowledge, we don't have a test for this (though in all 
> fairness, range deletions through thrift is a pretty late addition and has 
> little coverage) so the first step will probably be to reproduce.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10174) Range tombstones through thrift don't handle static columns probably

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-10174:
-
Summary: Range tombstones through thrift don't handle static columns 
probably  (was: x)

> Range tombstones through thrift don't handle static columns probably
> 
>
> Key: CASSANDRA-10174
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10174
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.0.0 rc2
>
>
> When inserting range tombstones through thrift in a "static" compact table, 
> the current code doesn't properly delete the static components of the table 
> (even when it should).
> To the best of my knowledge, we don't have a test for this (though in all 
> fairness, range deletions through thrift is a pretty late addition and has 
> little coverage) so the first step will probably be to reproduce.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10767) Checking version of Cassandra command creates `cassandra.logdir_IS_UNDEFINED/`

[Yuki Morishita (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-10767:
---
Priority: Trivial  (was: Minor)

> Checking version of Cassandra command creates `cassandra.logdir_IS_UNDEFINED/`
> --
>
> Key: CASSANDRA-10767
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10767
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: $ cassandra -v   
>  
> 2.1.2
> MacOSX 10.9.5
> $ brew info cassandra 
>  [14:15:41]
> cassandra: stable 2.2.3 (bottled)
> Eventually consistent, distributed key-value store
> https://cassandra.apache.org
> /usr/local/Cellar/cassandra/2.1.2 (3975 files, 92M) *
>   Built from source
> From: 
> https://github.com/Homebrew/homebrew/blob/master/Library/Formula/cassandra.rb
> ==> Caveats
> To have launchd start cassandra at login:
>   ln -sfv /usr/local/opt/cassandra/*.plist ~/Library/LaunchAgents
> Then to load cassandra now:
>   launchctl load ~/Library/LaunchAgents/homebrew.mxcl.cassandra.plist
>Reporter: Jens Rantil
>Assignee: Yuki Morishita
>Priority: Trivial
> Fix For: 2.1.x, 2.2.x, 3.0.x
>
>
> When I execute `cassandra -v` on the terminal the directory 
> `cassandra.logdir_IS_UNDEFINED` is created in my CWD:
> {noformat}
> $ tree cassandra.logdir_IS_UNDEFINED
> cassandra.logdir_IS_UNDEFINED
> └── system.log
> 0 directories, 1 file
> {noformat}
> Expected: That no log file nor directory is created when I'm simply checking 
> the version of Cassandra. Feels a bit ridiculous.
> Additionals: Just double checking, is this a bundling issue that should be 
> reported to Homebrew? Probably not, right?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)