[jira] [Created] (CASSANDRA-12030) Range tombstones that are masked by row tombstones should not be written out

[Nachiket Patil (JIRA)]

Nachiket Patil created CASSANDRA-12030:
--

 Summary: Range tombstones that are masked by row tombstones should 
not be written out
 Key: CASSANDRA-12030
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12030
 Project: Cassandra
  Issue Type: Improvement
Reporter: Nachiket Patil
Assignee: Nachiket Patil
Priority: Minor


During compaction, if a table has range tombstone and a row delete with higher 
timestamp than range tombstone, both are written out to disk. Some problems 
seen because of this behavior:
1. The Range tombstone is expensive to maintain.
2. Range queries see timeouts when there are too many range tombstones present 
which may be masked by row tombstones.
This can be avoided with simple optimization to not write out range tombstone 
if it is masked by row delete.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8523) Writes should be sent to a replacement node while it is streaming in data

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-8523:
---
Status: Patch Available  (was: Awaiting Feedback)

> Writes should be sent to a replacement node while it is streaming in data
> -
>
> Key: CASSANDRA-8523
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8523
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Richard Wagner
>Assignee: Paulo Motta
> Fix For: 2.1.x
>
>
> In our operations, we make heavy use of replace_address (or 
> replace_address_first_boot) in order to replace broken nodes. We now realize 
> that writes are not sent to the replacement nodes while they are in hibernate 
> state and streaming in data. This runs counter to what our expectations were, 
> especially since we know that writes ARE sent to nodes when they are 
> bootstrapped into the ring.
> It seems like cassandra should arrange to send writes to a node that is in 
> the process of replacing another node, just like it does for a nodes that are 
> bootstraping. I hesitate to phrase this as "we should send writes to a node 
> in hibernate" because the concept of hibernate may be useful in other 
> contexts, as per CASSANDRA-8336. Maybe a new state is needed here?
> Among other things, the fact that we don't get writes during this period 
> makes subsequent repairs more expensive, proportional to the number of writes 
> that we miss (and depending on the amount of data that needs to be streamed 
> during replacement and the time it may take to rebuild secondary indexes, we 
> could miss many many hours worth of writes). It also leaves us more exposed 
> to consistency violations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8523) Writes should be sent to a replacement node while it is streaming in data

[Richard Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337328#comment-15337328
 ] 

Richard Low commented on CASSANDRA-8523:


It's not those writes that matter - the replacement will get those writes from 
the other nodes during streaming. The hints that you might care about are 
writes dropped during the replacement on the replacing node. But those should 
be extremely rare, and a price well worth paying for getting the vast majority 
of the writes vs none today.

> Writes should be sent to a replacement node while it is streaming in data
> -
>
> Key: CASSANDRA-8523
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8523
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Richard Wagner
>Assignee: Paulo Motta
> Fix For: 2.1.x
>
>
> In our operations, we make heavy use of replace_address (or 
> replace_address_first_boot) in order to replace broken nodes. We now realize 
> that writes are not sent to the replacement nodes while they are in hibernate 
> state and streaming in data. This runs counter to what our expectations were, 
> especially since we know that writes ARE sent to nodes when they are 
> bootstrapped into the ring.
> It seems like cassandra should arrange to send writes to a node that is in 
> the process of replacing another node, just like it does for a nodes that are 
> bootstraping. I hesitate to phrase this as "we should send writes to a node 
> in hibernate" because the concept of hibernate may be useful in other 
> contexts, as per CASSANDRA-8336. Maybe a new state is needed here?
> Among other things, the fact that we don't get writes during this period 
> makes subsequent repairs more expensive, proportional to the number of writes 
> that we miss (and depending on the amount of data that needs to be streamed 
> during replacement and the time it may take to rebuild secondary indexes, we 
> could miss many many hours worth of writes). It also leaves us more exposed 
> to consistency violations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8523) Writes should be sent to a replacement node while it is streaming in data

[Brandon Williams (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337301#comment-15337301
 ] 

Brandon Williams commented on CASSANDRA-8523:
-

.bq Since it's no longer necessary to forward hints to the replacement node 
when replace_address != broadcast_address, the replacement node does not need 
to inherit the same ID of the original node.

I don't think that's necessarily true, if a node dies at point A, and then the 
replacement is issued at point N, the hints between points B and M are still 
missed if the host id is not the same.

> Writes should be sent to a replacement node while it is streaming in data
> -
>
> Key: CASSANDRA-8523
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8523
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Richard Wagner
>Assignee: Paulo Motta
> Fix For: 2.1.x
>
>
> In our operations, we make heavy use of replace_address (or 
> replace_address_first_boot) in order to replace broken nodes. We now realize 
> that writes are not sent to the replacement nodes while they are in hibernate 
> state and streaming in data. This runs counter to what our expectations were, 
> especially since we know that writes ARE sent to nodes when they are 
> bootstrapped into the ring.
> It seems like cassandra should arrange to send writes to a node that is in 
> the process of replacing another node, just like it does for a nodes that are 
> bootstraping. I hesitate to phrase this as "we should send writes to a node 
> in hibernate" because the concept of hibernate may be useful in other 
> contexts, as per CASSANDRA-8336. Maybe a new state is needed here?
> Among other things, the fact that we don't get writes during this period 
> makes subsequent repairs more expensive, proportional to the number of writes 
> that we miss (and depending on the amount of data that needs to be streamed 
> during replacement and the time it may take to rebuild secondary indexes, we 
> could miss many many hours worth of writes). It also leaves us more exposed 
> to consistency violations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-8523) Writes should be sent to a replacement node while it is streaming in data

[Brandon Williams (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337301#comment-15337301
 ] 

Brandon Williams edited comment on CASSANDRA-8523 at 6/18/16 12:26 AM:
---

bq. Since it's no longer necessary to forward hints to the replacement node 
when replace_address != broadcast_address, the replacement node does not need 
to inherit the same ID of the original node.

I don't think that's necessarily true, if a node dies at point A, and then the 
replacement is issued at point N, the hints between points B and M are still 
missed if the host id is not the same.


was (Author: brandon.williams):
.bq Since it's no longer necessary to forward hints to the replacement node 
when replace_address != broadcast_address, the replacement node does not need 
to inherit the same ID of the original node.

I don't think that's necessarily true, if a node dies at point A, and then the 
replacement is issued at point N, the hints between points B and M are still 
missed if the host id is not the same.

> Writes should be sent to a replacement node while it is streaming in data
> -
>
> Key: CASSANDRA-8523
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8523
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Richard Wagner
>Assignee: Paulo Motta
> Fix For: 2.1.x
>
>
> In our operations, we make heavy use of replace_address (or 
> replace_address_first_boot) in order to replace broken nodes. We now realize 
> that writes are not sent to the replacement nodes while they are in hibernate 
> state and streaming in data. This runs counter to what our expectations were, 
> especially since we know that writes ARE sent to nodes when they are 
> bootstrapped into the ring.
> It seems like cassandra should arrange to send writes to a node that is in 
> the process of replacing another node, just like it does for a nodes that are 
> bootstraping. I hesitate to phrase this as "we should send writes to a node 
> in hibernate" because the concept of hibernate may be useful in other 
> contexts, as per CASSANDRA-8336. Maybe a new state is needed here?
> Among other things, the fact that we don't get writes during this period 
> makes subsequent repairs more expensive, proportional to the number of writes 
> that we miss (and depending on the amount of data that needs to be streamed 
> during replacement and the time it may take to rebuild secondary indexes, we 
> could miss many many hours worth of writes). It also leaves us more exposed 
> to consistency violations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8523) Writes should be sent to a replacement node while it is streaming in data

[Richard Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337290#comment-15337290
 ] 

Richard Low commented on CASSANDRA-8523:


Thanks! I'm happy to review.

> Writes should be sent to a replacement node while it is streaming in data
> -
>
> Key: CASSANDRA-8523
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8523
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Richard Wagner
>Assignee: Paulo Motta
> Fix For: 2.1.x
>
>
> In our operations, we make heavy use of replace_address (or 
> replace_address_first_boot) in order to replace broken nodes. We now realize 
> that writes are not sent to the replacement nodes while they are in hibernate 
> state and streaming in data. This runs counter to what our expectations were, 
> especially since we know that writes ARE sent to nodes when they are 
> bootstrapped into the ring.
> It seems like cassandra should arrange to send writes to a node that is in 
> the process of replacing another node, just like it does for a nodes that are 
> bootstraping. I hesitate to phrase this as "we should send writes to a node 
> in hibernate" because the concept of hibernate may be useful in other 
> contexts, as per CASSANDRA-8336. Maybe a new state is needed here?
> Among other things, the fact that we don't get writes during this period 
> makes subsequent repairs more expensive, proportional to the number of writes 
> that we miss (and depending on the amount of data that needs to be streamed 
> during replacement and the time it may take to rebuild secondary indexes, we 
> could miss many many hours worth of writes). It also leaves us more exposed 
> to consistency violations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8523) Writes should be sent to a replacement node while it is streaming in data

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337288#comment-15337288
 ] 

Paulo Motta commented on CASSANDRA-8523:


Due to the limitations of forwarding writes to replacement nodes with the same 
IP, I propose initially adding this support only to replacement nodes with a 
different IP, since it's much simpler and we can do it in a backward-compatible 
way so it can probably go on 2.2+.

After CASSANDRA-11559, we can extend this support to nodes with the same IP 
quite easily by setting an inactive flag on nodes being replaced and ignore 
these nodes on read.

The central idea is:
{quote}
* Add a new non-dead gossip state for replace BOOT_REPLACE
* When receiving BOOT_REPLACE, other node adds the replacing node as 
bootstrapping endpoint
* Pending ranges are calculated, and writes are sent to the replacing node 
during replace
* When replacing node changes state to NORMAL, the old node is removed and the 
new node becomes a natural endpoint on TokenMetadata
* The final step is to change the original node state to REMOVED_TOKEN so other 
nodes evict the original node from gossip
{quote}

Since it's no longer necessary to forward hints to the replacement node when 
{{replace_address != broadcast_address}}, the replacement node does not need to 
inherit the same ID of the original node.

The replacing process remains unchanged when the replacement node has the same 
IP as the original node. If that's the case, I added a warn message so users 
know they need to run repair if the node is down for longer than 
{{max_hint_window_in_ms}}:
{noformat}
Writes will not be redirected to this node while it is performing replace 
because it has the same address as the node to be replaced ({}). 
If that node has been down for longer than max_hint_window_in_ms, repair must 
be run after the replacement process in order to make this node consistent.
{noformat}

I adapted current dtests to test replace_address for both the old and the new 
path, and when {{replace_address != broadcast_address}} make sure writes are 
being redirected to the replacement node.

Initial patch and tests below (will provide 2.2+ patches after initial review):
||2.2||dtest||
|[branch|https://github.com/apache/cassandra/compare/cassandra-2.2...pauloricardomg:2.2-8523]|[branch|https://github.com/riptano/cassandra-dtest/compare/master...pauloricardomg:8523]|
|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-2.2-8523-testall/lastCompletedBuild/testReport/]|
|[dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-2.2-8523-dtest/lastCompletedBuild/testReport/]|

> Writes should be sent to a replacement node while it is streaming in data
> -
>
> Key: CASSANDRA-8523
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8523
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Richard Wagner
>Assignee: Paulo Motta
> Fix For: 2.1.x
>
>
> In our operations, we make heavy use of replace_address (or 
> replace_address_first_boot) in order to replace broken nodes. We now realize 
> that writes are not sent to the replacement nodes while they are in hibernate 
> state and streaming in data. This runs counter to what our expectations were, 
> especially since we know that writes ARE sent to nodes when they are 
> bootstrapped into the ring.
> It seems like cassandra should arrange to send writes to a node that is in 
> the process of replacing another node, just like it does for a nodes that are 
> bootstraping. I hesitate to phrase this as "we should send writes to a node 
> in hibernate" because the concept of hibernate may be useful in other 
> contexts, as per CASSANDRA-8336. Maybe a new state is needed here?
> Among other things, the fact that we don't get writes during this period 
> makes subsequent repairs more expensive, proportional to the number of writes 
> that we miss (and depending on the amount of data that needs to be streamed 
> during replacement and the time it may take to rebuild secondary indexes, we 
> could miss many many hours worth of writes). It also leaves us more exposed 
> to consistency violations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10786) Include hash of result set metadata in prepared statement id

[Andy Tolbert (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337240#comment-15337240
 ] 

Andy Tolbert edited comment on CASSANDRA-10786 at 6/17/16 11:54 PM:


{quote}
What's our normal process for replacing the bundled Java driver when there's a 
protocol version change? Andy Tolbert any preferences on how to handle the 
review of the driver changes?
{quote}

Good question, I recall with protocol v4 and with C* 3.0 that there was a 
period of time where a pre-released driver jar was put in lib which I think is 
reasonable, but maybe there is a better way to do this, will discuss with the 
team on Monday.  There was [a 
PR|https://github.com/datastax/java-driver/pull/675] for reviewing this, but we 
will need to update it with the commit from [~ifesdjeen]'s 
[branch|https://github.com/ifesdjeen/java-driver/tree/10786-v5].


was (Author: andrew.tolbert):
{quote}
What's our normal process for replacing the bundled Java driver when there's a 
protocol version change? Andy Tolbert any preferences on how to handle the 
review of the driver changes?
{quote}

Good question, I recall with protocol v4 and with C* 3.0 that there was a 
period of time where a pre-released driver jar was put in lib which I think is 
reasonable, but maybe there is a better way to do this, will discuss with the 
team on Monday.  There was [a 
PR|https://github.com/datastax/java-driver/pull/675] for reviewing this, but we 
will need to update it with the commit from 
[~ifesdjeen's|https://github.com/ifesdjeen/java-driver/tree/10786-v5] branch.

> Include hash of result set metadata in prepared statement id
> 
>
> Key: CASSANDRA-10786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10786
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Olivier Michallat
>Assignee: Alex Petrov
>Priority: Minor
>  Labels: client-impacting, doc-impacting, protocolv5
> Fix For: 3.x
>
>
> This is a follow-up to CASSANDRA-7910, which was about invalidating a 
> prepared statement when the table is altered, to force clients to update 
> their local copy of the metadata.
> There's still an issue if multiple clients are connected to the same host. 
> The first client to execute the query after the cache was invalidated will 
> receive an UNPREPARED response, re-prepare, and update its local metadata. 
> But other clients might miss it entirely (the MD5 hasn't changed), and they 
> will keep using their old metadata. For example:
> # {{SELECT * ...}} statement is prepared in Cassandra with md5 abc123, 
> clientA and clientB both have a cache of the metadata (columns b and c) 
> locally
> # column a gets added to the table, C* invalidates its cache entry
> # clientA sends an EXECUTE request for md5 abc123, gets UNPREPARED response, 
> re-prepares on the fly and updates its local metadata to (a, b, c)
> # prepared statement is now in C*’s cache again, with the same md5 abc123
> # clientB sends an EXECUTE request for id abc123. Because the cache has been 
> populated again, the query succeeds. But clientB still has not updated its 
> metadata, it’s still (b,c)
> One solution that was suggested is to include a hash of the result set 
> metadata in the md5. This way the md5 would change at step 3, and any client 
> using the old md5 would get an UNPREPARED, regardless of whether another 
> client already reprepared.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10786) Include hash of result set metadata in prepared statement id

[Andy Tolbert (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337240#comment-15337240
 ] 

Andy Tolbert commented on CASSANDRA-10786:
--

{quote}
What's our normal process for replacing the bundled Java driver when there's a 
protocol version change? Andy Tolbert any preferences on how to handle the 
review of the driver changes?
{quote}

Good question, I recall with protocol v4 and with C* 3.0 that there was a 
period of time where a pre-released driver jar was put in lib which I think is 
reasonable, but maybe there is a better way to do this, will discuss with the 
team on Monday.  There was [a 
PR|https://github.com/datastax/java-driver/pull/675] for reviewing this, but we 
will need to update it with the commit from 
[~ifesdjeen's|https://github.com/ifesdjeen/java-driver/tree/10786-v5] branch.

> Include hash of result set metadata in prepared statement id
> 
>
> Key: CASSANDRA-10786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10786
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Olivier Michallat
>Assignee: Alex Petrov
>Priority: Minor
>  Labels: client-impacting, doc-impacting, protocolv5
> Fix For: 3.x
>
>
> This is a follow-up to CASSANDRA-7910, which was about invalidating a 
> prepared statement when the table is altered, to force clients to update 
> their local copy of the metadata.
> There's still an issue if multiple clients are connected to the same host. 
> The first client to execute the query after the cache was invalidated will 
> receive an UNPREPARED response, re-prepare, and update its local metadata. 
> But other clients might miss it entirely (the MD5 hasn't changed), and they 
> will keep using their old metadata. For example:
> # {{SELECT * ...}} statement is prepared in Cassandra with md5 abc123, 
> clientA and clientB both have a cache of the metadata (columns b and c) 
> locally
> # column a gets added to the table, C* invalidates its cache entry
> # clientA sends an EXECUTE request for md5 abc123, gets UNPREPARED response, 
> re-prepares on the fly and updates its local metadata to (a, b, c)
> # prepared statement is now in C*’s cache again, with the same md5 abc123
> # clientB sends an EXECUTE request for id abc123. Because the cache has been 
> populated again, the query succeeds. But clientB still has not updated its 
> metadata, it’s still (b,c)
> One solution that was suggested is to include a hash of the result set 
> metadata in the md5. This way the md5 would change at step 3, and any client 
> using the old md5 would get an UNPREPARED, regardless of whether another 
> client already reprepared.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11913) BufferUnderFlowException in CompressorTest

[Rei Odaira (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337225#comment-15337225
 ] 

Rei Odaira commented on CASSANDRA-11913:


I have updated the patch.

> BufferUnderFlowException in CompressorTest
> --
>
> Key: CASSANDRA-11913
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11913
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
> Environment: Non-x86 environments
>Reporter: Rei Odaira
>Assignee: Rei Odaira
>Priority: Minor
> Fix For: 2.2.x, 3.0.x, 3.x
>
> Attachments: 11913-2.2.txt
>
>
> org.apache.cassandra.io.compress.CompressorTest causes 
> java.nio.BufferUnderflowException on environments where FastByteOperations 
> uses PureJavaOperations. The root cause is that 
> CompressorTest.testArrayUncompress() copies data from a ByteBuffer to a byte 
> array beyond the limit of the ByteBuffer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11913) BufferUnderFlowException in CompressorTest

[Rei Odaira (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rei Odaira updated CASSANDRA-11913:
---
Attachment: 11913-2.2.txt

> BufferUnderFlowException in CompressorTest
> --
>
> Key: CASSANDRA-11913
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11913
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
> Environment: Non-x86 environments
>Reporter: Rei Odaira
>Assignee: Rei Odaira
>Priority: Minor
> Fix For: 2.2.x, 3.0.x, 3.x
>
> Attachments: 11913-2.2.txt
>
>
> org.apache.cassandra.io.compress.CompressorTest causes 
> java.nio.BufferUnderflowException on environments where FastByteOperations 
> uses PureJavaOperations. The root cause is that 
> CompressorTest.testArrayUncompress() copies data from a ByteBuffer to a byte 
> array beyond the limit of the ByteBuffer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11913) BufferUnderFlowException in CompressorTest

[Rei Odaira (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rei Odaira updated CASSANDRA-11913:
---
Attachment: (was: 11913-2.2.txt)

> BufferUnderFlowException in CompressorTest
> --
>
> Key: CASSANDRA-11913
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11913
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
> Environment: Non-x86 environments
>Reporter: Rei Odaira
>Assignee: Rei Odaira
>Priority: Minor
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> org.apache.cassandra.io.compress.CompressorTest causes 
> java.nio.BufferUnderflowException on environments where FastByteOperations 
> uses PureJavaOperations. The root cause is that 
> CompressorTest.testArrayUncompress() copies data from a ByteBuffer to a byte 
> array beyond the limit of the ByteBuffer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11038) Is node being restarted treated as node joining?

[Joel Knighton (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337209#comment-15337209
 ] 

Joel Knighton commented on CASSANDRA-11038:
---

LGTM.

Patches look good, testall runs look good. Looks like your dtests got branched 
at a bad time when there was a brief problem on the dtest branch - I rebased 
your dtest branch on master and pushed at 
[jkni/cassandra-dtest/11038|https://github.com/jkni/cassandra-dtest/tree/11038].
 Because ccm has changes for latest trunk after the CDC merge, I've rebased 
your trunk branch and pushed it at 
[jkni/cassandra/11038-trunk-rebase|https://github.com/jkni/cassandra/tree/11038-trunk-rebase].
 The rebase was clean so you can just rebase your trunk branch on commit. Dtest 
runs after this update are clean for 
[2.2|http://cassci.datastax.com/view/Dev/view/jkni/job/jkni-11038-2.2-dtest/], 
[3.0|http://cassci.datastax.com/view/Dev/view/jkni/job/jkni-11038-3.0-dtest/], 
and 
[trunk|http://cassci.datastax.com/view/Dev/view/jkni/job/jkni-11038-trunk-dtest/].

Stefania has a PR open for her [CASSANDRA-11731] fixes 
[here|https://github.com/riptano/cassandra-dtest/pull/983]. I don't see a PR 
for your added test. Do you want to OK the 11731 PR and PR your added test on 
commit [~beobal], or do you want me to?

> Is node being restarted treated as node joining?
> 
>
> Key: CASSANDRA-11038
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11038
> Project: Cassandra
>  Issue Type: Bug
>  Components: Distributed Metadata
>Reporter: cheng ren
>Assignee: Sam Tunnicliffe
>Priority: Minor
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> Hi, 
> What we found recently is that every time we restart a node, all other nodes 
> in the cluster treat the restarted node as a new node joining and issue node 
> joining notification to clients. We have traced the code path being hit when 
> a peer node detected a restarted node:
> src/java/org/apache/cassandra/gms/Gossiper.java
> {code}
> private void handleMajorStateChange(InetAddress ep, EndpointState epState)
> {
> if (!isDeadState(epState))
> {
> if (endpointStateMap.get(ep) != null)
> logger.info("Node {} has restarted, now UP", ep);
> else
> logger.info("Node {} is now part of the cluster", ep);
> }
> if (logger.isTraceEnabled())
> logger.trace("Adding endpoint state for " + ep);
> endpointStateMap.put(ep, epState);
> // the node restarted: it is up to the subscriber to take whatever 
> action is necessary
> for (IEndpointStateChangeSubscriber subscriber : subscribers)
> subscriber.onRestart(ep, epState);
> if (!isDeadState(epState))
> markAlive(ep, epState);
> else
> {
> logger.debug("Not marking " + ep + " alive due to dead state");
> markDead(ep, epState);
> }
> for (IEndpointStateChangeSubscriber subscriber : subscribers)
> subscriber.onJoin(ep, epState);
> }
> {code}
> subscriber.onJoin(ep, epState) ends up with calling onJoinCluster in 
> Server.java
> {code}
> src/java/org/apache/cassandra/transport/Server.java
> public void onJoinCluster(InetAddress endpoint)
> {
> server.connectionTracker.send(Event.TopologyChange.newNode(getRpcAddress(endpoint),
>  server.socket.getPort()));
> }
> {code}
> We have a full trace of code path and skip some intermedia function calls 
> here for being brief. 
> Upon receiving the node joining notification, clients would go and scan 
> system peer table to fetch the latest topology information. Since we have 
> tens of thousands of client connections, scans from all of them put an 
> enormous load to our cluster. 
> Although in the newer version of driver, client skips fetching peer table if 
> the new node has already existed in local metadata, we are still curious why 
> node being restarted is handled as node joining on server side? Did we hit a 
> bug or this is the way supposed to be? Our old java driver version is 1.0.4 
> and cassandra version is 2.0.12.
> Thanks!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10786) Include hash of result set metadata in prepared statement id

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337161#comment-15337161
 ] 

Tyler Hobbs commented on CASSANDRA-10786:
-

The Cassandra patch looks good to me, nicely done.  We'll need to start the v5 
protocol spec document (and add a "changes" section), but I can do that on 
commit.

What's our normal process for replacing the bundled Java driver when there's a 
protocol version change?  [~andrew.tolbert] any preferences on how to handle 
the review of the driver changes?  It looks like there are a couple of minor 
things in that patch that may need adjusting (like the min cassandra version 
for the v5 protocol, for example).

> Include hash of result set metadata in prepared statement id
> 
>
> Key: CASSANDRA-10786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10786
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Olivier Michallat
>Assignee: Alex Petrov
>Priority: Minor
>  Labels: client-impacting, doc-impacting, protocolv5
> Fix For: 3.x
>
>
> This is a follow-up to CASSANDRA-7910, which was about invalidating a 
> prepared statement when the table is altered, to force clients to update 
> their local copy of the metadata.
> There's still an issue if multiple clients are connected to the same host. 
> The first client to execute the query after the cache was invalidated will 
> receive an UNPREPARED response, re-prepare, and update its local metadata. 
> But other clients might miss it entirely (the MD5 hasn't changed), and they 
> will keep using their old metadata. For example:
> # {{SELECT * ...}} statement is prepared in Cassandra with md5 abc123, 
> clientA and clientB both have a cache of the metadata (columns b and c) 
> locally
> # column a gets added to the table, C* invalidates its cache entry
> # clientA sends an EXECUTE request for md5 abc123, gets UNPREPARED response, 
> re-prepares on the fly and updates its local metadata to (a, b, c)
> # prepared statement is now in C*’s cache again, with the same md5 abc123
> # clientB sends an EXECUTE request for id abc123. Because the cache has been 
> populated again, the query succeeds. But clientB still has not updated its 
> metadata, it’s still (b,c)
> One solution that was suggested is to include a hash of the result set 
> metadata in the md5. This way the md5 would change at step 3, and any client 
> using the old md5 would get an UNPREPARED, regardless of whether another 
> client already reprepared.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11920) bloom_filter_fp_chance needs to be validated up front

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336970#comment-15336970
 ] 

Tyler Hobbs commented on CASSANDRA-11920:
-

Thanks for the patch!

I've created a new unit test to exercise this.  I also backported the patch to 
2.2, since that's affected as well.

Here are the patches and pending CI test runs:
||branch||testall||dtest||
|[CASSANDRA-11920-2.2|https://github.com/thobbs/cassandra/tree/CASSANDRA-11920-2.2]|[testall|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-11920-2.2-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-11920-2.2-dtest]|
|[CASSANDRA-11920-3.0|https://github.com/thobbs/cassandra/tree/CASSANDRA-11920-3.0]|[testall|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-11920-3.0-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-11920-3.0-dtest]|
|[CASSANDRA-11920-trunk|https://github.com/thobbs/cassandra/tree/CASSANDRA-11920-trunk]|[testall|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-11920-trunk-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-11920-trunk-dtest]|

If the tests look good, I'll commit this.

> bloom_filter_fp_chance needs to be validated up front
> -
>
> Key: CASSANDRA-11920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11920
> Project: Cassandra
>  Issue Type: Bug
>  Components: Lifecycle, Local Write-Read Paths
>Reporter: ADARSH KUMAR
>Assignee: Arindam Gupta
>Priority: Minor
>  Labels: lhf
> Attachments: 11920-3.0.txt
>
>
> Hi,
> I was doing some bench-marking on bloom_filter_fp_chance values. Everything 
> worked fine for values .01(default for STCS), .001, .0001. But when I set 
> bloom_filter_fp_chance = .1 i observed following behaviour:
> 1). Reads and writes looked normal from cqlsh.
> 2). SSttables are never created.
> 3). It just creates two files (*-Data.db and *-index.db) of size 0kb.
> 4). nodetool flush does not work and produce following exception:
> java.lang.UnsupportedOperationException: Unable to satisfy 1.0E-5 with 20 
> buckets per element
> at 
> org.apache.cassandra.utils.BloomCalculations.computeBloomSpec(BloomCalculations.java:150)
>  .
> I checked BloomCalculations class and following lines are responsible for 
> this exception:
> if (maxFalsePosProb < probs[maxBucketsPerElement][maxK]) {
>   throw new UnsupportedOperationException(String.format("Unable to 
> satisfy %s with %s buckets per element",
>  maxFalsePosProb, 
> maxBucketsPerElement));
>   }
> From  the code it looks like a hard coaded validation (unless we can change 
> the nuber of buckets).
> So, if this validation is hard coaded then why it is even allowed to set such 
> value of bloom_fileter_fp_chance, that can prevent ssTable generation?
> Please correct this issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8844) Change Data Capture (CDC)

[Carl Yeksigian (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336883#comment-15336883
 ] 

Carl Yeksigian commented on CASSANDRA-8844:
---

CDC is part of 3.8, which will be released during July.

The feature is already committed to trunk, so you could [pull Cassandra and 
build it|http://wiki.apache.org/cassandra/HowToBuild] yourself if you wanted to 
test it before we have a release including this.

> Change Data Capture (CDC)
> -
>
> Key: CASSANDRA-8844
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8844
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Coordination, Local Write-Read Paths
>Reporter: Tupshin Harper
>Assignee: Joshua McKenzie
>Priority: Critical
> Fix For: 3.8
>
>
> "In databases, change data capture (CDC) is a set of software design patterns 
> used to determine (and track) the data that has changed so that action can be 
> taken using the changed data. Also, Change data capture (CDC) is an approach 
> to data integration that is based on the identification, capture and delivery 
> of the changes made to enterprise data sources."
> -Wikipedia
> As Cassandra is increasingly being used as the Source of Record (SoR) for 
> mission critical data in large enterprises, it is increasingly being called 
> upon to act as the central hub of traffic and data flow to other systems. In 
> order to try to address the general need, we (cc [~brianmhess]), propose 
> implementing a simple data logging mechanism to enable per-table CDC patterns.
> h2. The goals:
> # Use CQL as the primary ingestion mechanism, in order to leverage its 
> Consistency Level semantics, and in order to treat it as the single 
> reliable/durable SoR for the data.
> # To provide a mechanism for implementing good and reliable 
> (deliver-at-least-once with possible mechanisms for deliver-exactly-once ) 
> continuous semi-realtime feeds of mutations going into a Cassandra cluster.
> # To eliminate the developmental and operational burden of users so that they 
> don't have to do dual writes to other systems.
> # For users that are currently doing batch export from a Cassandra system, 
> give them the opportunity to make that realtime with a minimum of coding.
> h2. The mechanism:
> We propose a durable logging mechanism that functions similar to a commitlog, 
> with the following nuances:
> - Takes place on every node, not just the coordinator, so RF number of copies 
> are logged.
> - Separate log per table.
> - Per-table configuration. Only tables that are specified as CDC_LOG would do 
> any logging.
> - Per DC. We are trying to keep the complexity to a minimum to make this an 
> easy enhancement, but most likely use cases would prefer to only implement 
> CDC logging in one (or a subset) of the DCs that are being replicated to
> - In the critical path of ConsistencyLevel acknowledgment. Just as with the 
> commitlog, failure to write to the CDC log should fail that node's write. If 
> that means the requested consistency level was not met, then clients *should* 
> experience UnavailableExceptions.
> - Be written in a Row-centric manner such that it is easy for consumers to 
> reconstitute rows atomically.
> - Written in a simple format designed to be consumed *directly* by daemons 
> written in non JVM languages
> h2. Nice-to-haves
> I strongly suspect that the following features will be asked for, but I also 
> believe that they can be deferred for a subsequent release, and to guage 
> actual interest.
> - Multiple logs per table. This would make it easy to have multiple 
> "subscribers" to a single table's changes. A workaround would be to create a 
> forking daemon listener, but that's not a great answer.
> - Log filtering. Being able to apply filters, including UDF-based filters 
> would make Casandra a much more versatile feeder into other systems, and 
> again, reduce complexity that would otherwise need to be built into the 
> daemons.
> h2. Format and Consumption
> - Cassandra would only write to the CDC log, and never delete from it. 
> - Cleaning up consumed logfiles would be the client daemon's responibility
> - Logfile size should probably be configurable.
> - Logfiles should be named with a predictable naming schema, making it 
> triivial to process them in order.
> - Daemons should be able to checkpoint their work, and resume from where they 
> left off. This means they would have to leave some file artifact in the CDC 
> log's directory.
> - A sophisticated daemon should be able to be written that could 
> -- Catch up, in written-order, even when it is multiple logfiles behind in 
> processing
> -- Be able to continuously "tail" the most recent logfile and get 
> low-latency(ms?) access to the data as it is written.
> h2. Alternate approach
> In order 

[jira] [Commented] (CASSANDRA-11988) NullPointerExpception when reading/compacting table

[Carl Yeksigian (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336877#comment-15336877
 ] 

Carl Yeksigian commented on CASSANDRA-11988:


I had a [fix|https://github.com/carlyeks/cassandra/tree/ticket/11988/3.0], but 
there seems to be a dependency between the change to return null for empty 
static rows and early reopen, so there are failing unit tests.

I'll work on a different fix on Monday which will just target the possibly bad 
usages.

> NullPointerExpception when reading/compacting table
> ---
>
> Key: CASSANDRA-11988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11988
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Nimi Wariboko Jr.
>Assignee: Carl Yeksigian
> Fix For: 3.6
>
>
> I have a table that suddenly refuses to be read or compacted. Issuing a read 
> on the table causes a NPE.
> On compaction, it returns the error
> {code}
> ERROR [CompactionExecutor:6] 2016-06-09 17:10:15,724 CassandraDaemon.java:213 
> - Exception in thread Thread[CompactionExecutor:6,1,main]
> java.lang.NullPointerException: null
>   at 
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:38)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:64)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:24)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:76)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:226)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:182)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:82)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:264)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_45]
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_45]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_45]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_45]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> {code}
> Schema:
> {code}
> CREATE TABLE cmpayments.report_payments (
> reportid timeuuid,
> userid timeuuid,
> adjustedearnings decimal,
> deleted set static,
> earnings map,
> gross map,
> organizationid text,
> payall timestamp static,
> status text,
> PRIMARY KEY (reportid, userid)
> ) WITH CLUSTERING ORDER BY (userid ASC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND comment = ''
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class': 
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8700) replace the wiki with docs in the git repo

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336860#comment-15336860
 ] 

Sylvain Lebresne commented on CASSANDRA-8700:
-

bq. Let me know what you think.

I'll look at it next week cause it's getting late, but I have no doubt it beats 
the hell out of doing this manually. That's great.

Anyway, outside of that pull request (and the next one on sstable and 
memtables) the branch includes everything I got so far, including the section 
on compaction by Marcus and Jeff that Marcus sent me by email. I'm myself still 
working on the CQL doc.

Overall, I think it's safe to say that we'll have something decent to publish 
by the end of next week (we might still have a handful of missing sections by 
then, but they should be few enough that we can put it online).

> replace the wiki with docs in the git repo
> --
>
> Key: CASSANDRA-8700
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8700
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation and Website
>Reporter: Jon Haddad
>Assignee: Sylvain Lebresne
>Priority: Blocker
> Fix For: 3.8
>
> Attachments: TombstonesAndGcGrace.md, bloom_filters.md, 
> compression.md, contributing.zip, getting_started.zip, hardware.md
>
>
> The wiki as it stands is pretty terrible.  It takes several minutes to apply 
> a single update, and as a result, it's almost never updated.  The information 
> there has very little context as to what version it applies to.  Most people 
> I've talked to that try to use the information they find there find it is 
> more confusing than helpful.
> I'd like to propose that instead of using the wiki, the doc directory in the 
> cassandra repo be used for docs (already used for CQL3 spec) in a format that 
> can be built to a variety of output formats like HTML / epub / etc.  I won't 
> start the bikeshedding on which markup format is preferable - but there are 
> several options that can work perfectly fine.  I've personally use sphinx w/ 
> restructured text, and markdown.  Both can build easily and as an added bonus 
> be pushed to readthedocs (or something similar) automatically.  For an 
> example, see cqlengine's documentation, which I think is already 
> significantly better than the wiki: 
> http://cqlengine.readthedocs.org/en/latest/
> In addition to being overall easier to maintain, putting the documentation in 
> the git repo adds context, since it evolves with the versions of Cassandra.
> If the wiki were kept even remotely up to date, I wouldn't bother with this, 
> but not having at least some basic documentation in the repo, or anywhere 
> associated with the project, is frustrating.
> For reference, the last 3 updates were:
> 1/15/15 - updating committers list
> 1/08/15 - updating contributers and how to contribute
> 12/16/14 - added a link to CQL docs from wiki frontpage (by me)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11920) bloom_filter_fp_chance needs to be validated up front

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-11920:

Priority: Minor  (was: Major)

> bloom_filter_fp_chance needs to be validated up front
> -
>
> Key: CASSANDRA-11920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11920
> Project: Cassandra
>  Issue Type: Bug
>  Components: Lifecycle, Local Write-Read Paths
>Reporter: ADARSH KUMAR
>Assignee: Arindam Gupta
>Priority: Minor
>  Labels: lhf
> Attachments: 11920-3.0.txt
>
>
> Hi,
> I was doing some bench-marking on bloom_filter_fp_chance values. Everything 
> worked fine for values .01(default for STCS), .001, .0001. But when I set 
> bloom_filter_fp_chance = .1 i observed following behaviour:
> 1). Reads and writes looked normal from cqlsh.
> 2). SSttables are never created.
> 3). It just creates two files (*-Data.db and *-index.db) of size 0kb.
> 4). nodetool flush does not work and produce following exception:
> java.lang.UnsupportedOperationException: Unable to satisfy 1.0E-5 with 20 
> buckets per element
> at 
> org.apache.cassandra.utils.BloomCalculations.computeBloomSpec(BloomCalculations.java:150)
>  .
> I checked BloomCalculations class and following lines are responsible for 
> this exception:
> if (maxFalsePosProb < probs[maxBucketsPerElement][maxK]) {
>   throw new UnsupportedOperationException(String.format("Unable to 
> satisfy %s with %s buckets per element",
>  maxFalsePosProb, 
> maxBucketsPerElement));
>   }
> From  the code it looks like a hard coaded validation (unless we can change 
> the nuber of buckets).
> So, if this validation is hard coaded then why it is even allowed to set such 
> value of bloom_fileter_fp_chance, that can prevent ssTable generation?
> Please correct this issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11920) bloom_filter_fp_chance needs to be validated up front

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-11920:

Assignee: Arindam Gupta

> bloom_filter_fp_chance needs to be validated up front
> -
>
> Key: CASSANDRA-11920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11920
> Project: Cassandra
>  Issue Type: Bug
>  Components: Lifecycle, Local Write-Read Paths
>Reporter: ADARSH KUMAR
>Assignee: Arindam Gupta
>  Labels: lhf
> Attachments: 11920-3.0.txt
>
>
> Hi,
> I was doing some bench-marking on bloom_filter_fp_chance values. Everything 
> worked fine for values .01(default for STCS), .001, .0001. But when I set 
> bloom_filter_fp_chance = .1 i observed following behaviour:
> 1). Reads and writes looked normal from cqlsh.
> 2). SSttables are never created.
> 3). It just creates two files (*-Data.db and *-index.db) of size 0kb.
> 4). nodetool flush does not work and produce following exception:
> java.lang.UnsupportedOperationException: Unable to satisfy 1.0E-5 with 20 
> buckets per element
> at 
> org.apache.cassandra.utils.BloomCalculations.computeBloomSpec(BloomCalculations.java:150)
>  .
> I checked BloomCalculations class and following lines are responsible for 
> this exception:
> if (maxFalsePosProb < probs[maxBucketsPerElement][maxK]) {
>   throw new UnsupportedOperationException(String.format("Unable to 
> satisfy %s with %s buckets per element",
>  maxFalsePosProb, 
> maxBucketsPerElement));
>   }
> From  the code it looks like a hard coaded validation (unless we can change 
> the nuber of buckets).
> So, if this validation is hard coaded then why it is even allowed to set such 
> value of bloom_fileter_fp_chance, that can prevent ssTable generation?
> Please correct this issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8844) Change Data Capture (CDC)

[Vedant (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336839#comment-15336839
 ] 

Vedant commented on CASSANDRA-8844:
---

When is GA release planned for this feature? How can I evaluate it in the mean 
time for my use case ?

> Change Data Capture (CDC)
> -
>
> Key: CASSANDRA-8844
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8844
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Coordination, Local Write-Read Paths
>Reporter: Tupshin Harper
>Assignee: Joshua McKenzie
>Priority: Critical
> Fix For: 3.8
>
>
> "In databases, change data capture (CDC) is a set of software design patterns 
> used to determine (and track) the data that has changed so that action can be 
> taken using the changed data. Also, Change data capture (CDC) is an approach 
> to data integration that is based on the identification, capture and delivery 
> of the changes made to enterprise data sources."
> -Wikipedia
> As Cassandra is increasingly being used as the Source of Record (SoR) for 
> mission critical data in large enterprises, it is increasingly being called 
> upon to act as the central hub of traffic and data flow to other systems. In 
> order to try to address the general need, we (cc [~brianmhess]), propose 
> implementing a simple data logging mechanism to enable per-table CDC patterns.
> h2. The goals:
> # Use CQL as the primary ingestion mechanism, in order to leverage its 
> Consistency Level semantics, and in order to treat it as the single 
> reliable/durable SoR for the data.
> # To provide a mechanism for implementing good and reliable 
> (deliver-at-least-once with possible mechanisms for deliver-exactly-once ) 
> continuous semi-realtime feeds of mutations going into a Cassandra cluster.
> # To eliminate the developmental and operational burden of users so that they 
> don't have to do dual writes to other systems.
> # For users that are currently doing batch export from a Cassandra system, 
> give them the opportunity to make that realtime with a minimum of coding.
> h2. The mechanism:
> We propose a durable logging mechanism that functions similar to a commitlog, 
> with the following nuances:
> - Takes place on every node, not just the coordinator, so RF number of copies 
> are logged.
> - Separate log per table.
> - Per-table configuration. Only tables that are specified as CDC_LOG would do 
> any logging.
> - Per DC. We are trying to keep the complexity to a minimum to make this an 
> easy enhancement, but most likely use cases would prefer to only implement 
> CDC logging in one (or a subset) of the DCs that are being replicated to
> - In the critical path of ConsistencyLevel acknowledgment. Just as with the 
> commitlog, failure to write to the CDC log should fail that node's write. If 
> that means the requested consistency level was not met, then clients *should* 
> experience UnavailableExceptions.
> - Be written in a Row-centric manner such that it is easy for consumers to 
> reconstitute rows atomically.
> - Written in a simple format designed to be consumed *directly* by daemons 
> written in non JVM languages
> h2. Nice-to-haves
> I strongly suspect that the following features will be asked for, but I also 
> believe that they can be deferred for a subsequent release, and to guage 
> actual interest.
> - Multiple logs per table. This would make it easy to have multiple 
> "subscribers" to a single table's changes. A workaround would be to create a 
> forking daemon listener, but that's not a great answer.
> - Log filtering. Being able to apply filters, including UDF-based filters 
> would make Casandra a much more versatile feeder into other systems, and 
> again, reduce complexity that would otherwise need to be built into the 
> daemons.
> h2. Format and Consumption
> - Cassandra would only write to the CDC log, and never delete from it. 
> - Cleaning up consumed logfiles would be the client daemon's responibility
> - Logfile size should probably be configurable.
> - Logfiles should be named with a predictable naming schema, making it 
> triivial to process them in order.
> - Daemons should be able to checkpoint their work, and resume from where they 
> left off. This means they would have to leave some file artifact in the CDC 
> log's directory.
> - A sophisticated daemon should be able to be written that could 
> -- Catch up, in written-order, even when it is multiple logfiles behind in 
> processing
> -- Be able to continuously "tail" the most recent logfile and get 
> low-latency(ms?) access to the data as it is written.
> h2. Alternate approach
> In order to make consuming a change log easy and efficient to do with low 
> latency, the following could supplement the approach outlined above
> - Instead of writing to a logfile, by 

[jira] [Commented] (CASSANDRA-8700) replace the wiki with docs in the git repo

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336835#comment-15336835
 ] 

Tyler Hobbs commented on CASSANDRA-8700:


Memtable and SSTable architecture: https://github.com/pcmanus/cassandra/pull/58

> replace the wiki with docs in the git repo
> --
>
> Key: CASSANDRA-8700
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8700
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation and Website
>Reporter: Jon Haddad
>Assignee: Sylvain Lebresne
>Priority: Blocker
> Fix For: 3.8
>
> Attachments: TombstonesAndGcGrace.md, bloom_filters.md, 
> compression.md, contributing.zip, getting_started.zip, hardware.md
>
>
> The wiki as it stands is pretty terrible.  It takes several minutes to apply 
> a single update, and as a result, it's almost never updated.  The information 
> there has very little context as to what version it applies to.  Most people 
> I've talked to that try to use the information they find there find it is 
> more confusing than helpful.
> I'd like to propose that instead of using the wiki, the doc directory in the 
> cassandra repo be used for docs (already used for CQL3 spec) in a format that 
> can be built to a variety of output formats like HTML / epub / etc.  I won't 
> start the bikeshedding on which markup format is preferable - but there are 
> several options that can work perfectly fine.  I've personally use sphinx w/ 
> restructured text, and markdown.  Both can build easily and as an added bonus 
> be pushed to readthedocs (or something similar) automatically.  For an 
> example, see cqlengine's documentation, which I think is already 
> significantly better than the wiki: 
> http://cqlengine.readthedocs.org/en/latest/
> In addition to being overall easier to maintain, putting the documentation in 
> the git repo adds context, since it evolves with the versions of Cassandra.
> If the wiki were kept even remotely up to date, I wouldn't bother with this, 
> but not having at least some basic documentation in the repo, or anywhere 
> associated with the project, is frustrating.
> For reference, the last 3 updates were:
> 1/15/15 - updating committers list
> 1/08/15 - updating contributers and how to contribute
> 12/16/14 - added a link to CQL docs from wiki frontpage (by me)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-4663) Streaming sends one file at a time serially.

[Anubhav Kale (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-4663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336787#comment-15336787
 ] 

Anubhav Kale commented on CASSANDRA-4663:
-

I made a change to RangeStreamer to created multiple StreamSessions per host 
(Split token ranges into chunks equal to the number of sockets). I saw a 
performance improvement (time-wise) of ~33%. 

Since the same code is used for bootstrap and nodetool rebuild, it will help in 
both cases. The one side-effect that operators need to be aware of is the 
number of SS Tables created on destination (since they will blow up 
corresponding to number of splits).

I suggest we could add a -par option for nodetool rebuild command and let 
operators provide number of connections. For bootstrap, we can provide yaml 
setting and default to 1. (If we do decide to add yaml setting, do I need to 
worry about any version breaking stuff?)

If that makes sense, I will create a patch for trunk. 

> Streaming sends one file at a time serially. 
> -
>
> Key: CASSANDRA-4663
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4663
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Priority: Minor
>
> This is not fast enough when someone is using SSD and may be 10G link. We 
> should try to create multiple connections and send multiple files in 
> parallel. 
> Current approach under utilize the link(even 1G).
> This change will improve the bootstrapping time of a node. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11327) Maintain a histogram of times when writes are blocked due to no available memory

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11327:

Status: Ready to Commit  (was: Patch Available)

> Maintain a histogram of times when writes are blocked due to no available 
> memory
> 
>
> Key: CASSANDRA-11327
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11327
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Core
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>
> I have a theory that part of the reason C* is so sensitive to timeouts during 
> saturating write load is that throughput is basically a sawtooth with valleys 
> at zero. This is something I have observed and it gets worse as you add 2i to 
> a table or do anything that decreases the throughput of flushing.
> I think the fix for this is to incrementally release memory pinned by 
> memtables and 2i during flushing instead of releasing it all at once. I know 
> that's not really possible, but we can fake it with memory accounting that 
> tracks how close to completion flushing is and releases permits for 
> additional memory. This will lead to a bit of a sawtooth in real memory 
> usage, but we can account for that so the peak footprint is the same.
> I think the end result of this change will be a sawtooth, but the valley of 
> the sawtooth will not be zero it will be the rate at which flushing 
> progresses. Optimizing the rate at which flushing progresses and it's 
> fairness with other work can then be tackled separately.
> Before we do this I think we should demonstrate that pinned memory due to 
> flushing is actually the issue by getting better visibility into the 
> distribution of instances of not having any memory by maintaining a histogram 
> of spans of time where no memory is available and a thread is blocked.
> [MemtableAllocatr$SubPool.allocate(long)|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/utils/memory/MemtableAllocator.java#L186]
>  should be a relatively straightforward entry point for this. The first 
> thread to block can mark the start of memory starvation and the last thread 
> out can mark the end. Have a periodic task that tracks the amount of time 
> spent blocked per interval of time and if it is greater than some threshold 
> log with more details, possibly at debug.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11327) Maintain a histogram of times when writes are blocked due to no available memory

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11327:

Reviewer: Chris Lohfink

> Maintain a histogram of times when writes are blocked due to no available 
> memory
> 
>
> Key: CASSANDRA-11327
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11327
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Core
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>
> I have a theory that part of the reason C* is so sensitive to timeouts during 
> saturating write load is that throughput is basically a sawtooth with valleys 
> at zero. This is something I have observed and it gets worse as you add 2i to 
> a table or do anything that decreases the throughput of flushing.
> I think the fix for this is to incrementally release memory pinned by 
> memtables and 2i during flushing instead of releasing it all at once. I know 
> that's not really possible, but we can fake it with memory accounting that 
> tracks how close to completion flushing is and releases permits for 
> additional memory. This will lead to a bit of a sawtooth in real memory 
> usage, but we can account for that so the peak footprint is the same.
> I think the end result of this change will be a sawtooth, but the valley of 
> the sawtooth will not be zero it will be the rate at which flushing 
> progresses. Optimizing the rate at which flushing progresses and it's 
> fairness with other work can then be tackled separately.
> Before we do this I think we should demonstrate that pinned memory due to 
> flushing is actually the issue by getting better visibility into the 
> distribution of instances of not having any memory by maintaining a histogram 
> of spans of time where no memory is available and a thread is blocked.
> [MemtableAllocatr$SubPool.allocate(long)|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/utils/memory/MemtableAllocator.java#L186]
>  should be a relatively straightforward entry point for this. The first 
> thread to block can mark the start of memory starvation and the last thread 
> out can mark the end. Have a periodic task that tracks the amount of time 
> spent blocked per interval of time and if it is greater than some threshold 
> log with more details, possibly at debug.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11327) Maintain a histogram of times when writes are blocked due to no available memory

[Chris Lohfink (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336749#comment-15336749
 ] 

Chris Lohfink commented on CASSANDRA-11327:
---

I looked at it a few days ago, seeing metric name change now I'm +1

> Maintain a histogram of times when writes are blocked due to no available 
> memory
> 
>
> Key: CASSANDRA-11327
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11327
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Core
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>
> I have a theory that part of the reason C* is so sensitive to timeouts during 
> saturating write load is that throughput is basically a sawtooth with valleys 
> at zero. This is something I have observed and it gets worse as you add 2i to 
> a table or do anything that decreases the throughput of flushing.
> I think the fix for this is to incrementally release memory pinned by 
> memtables and 2i during flushing instead of releasing it all at once. I know 
> that's not really possible, but we can fake it with memory accounting that 
> tracks how close to completion flushing is and releases permits for 
> additional memory. This will lead to a bit of a sawtooth in real memory 
> usage, but we can account for that so the peak footprint is the same.
> I think the end result of this change will be a sawtooth, but the valley of 
> the sawtooth will not be zero it will be the rate at which flushing 
> progresses. Optimizing the rate at which flushing progresses and it's 
> fairness with other work can then be tackled separately.
> Before we do this I think we should demonstrate that pinned memory due to 
> flushing is actually the issue by getting better visibility into the 
> distribution of instances of not having any memory by maintaining a histogram 
> of spans of time where no memory is available and a thread is blocked.
> [MemtableAllocatr$SubPool.allocate(long)|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/utils/memory/MemtableAllocator.java#L186]
>  should be a relatively straightforward entry point for this. The first 
> thread to block can mark the start of memory starvation and the last thread 
> out can mark the end. Have a periodic task that tracks the amount of time 
> spent blocked per interval of time and if it is greater than some threshold 
> log with more details, possibly at debug.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8700) replace the wiki with docs in the git repo

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336742#comment-15336742
 ] 

Tyler Hobbs commented on CASSANDRA-8700:


Pull request to automatically convert cassandra.yaml into rst: 
https://github.com/pcmanus/cassandra/pull/57

This works pretty well overall.  There are a few minor downsides:
* I couldn't figure out a good way to handle complex options (lists and dicts) 
without hard coding the option names in the script.  I considered using a yaml 
parser to detect these, but that would add a dependency and wouldn't work for 
commented-out complex options.  Fortunately, it's a pretty small list, and 
should rarely change.  If it does change without us updating the script, it 
won't break anything, but the option body will end up in the next section.
* Some of the existing comments needed to be formatted slightly differently to 
look good in rst.
* A couple of comments and default values don't make as much sense in the rst 
version.  For example, {{allocate_tokens_for_keyspace}} has a default value of 
{{KEYSPACE}}, which is really more of a placeholder.  In the rst version, this 
is shown as the default value, so it's a little weird.  The number of quirks 
like this is fairly small, though, and they're not terrible.

Let me know what you think.

> replace the wiki with docs in the git repo
> --
>
> Key: CASSANDRA-8700
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8700
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation and Website
>Reporter: Jon Haddad
>Assignee: Sylvain Lebresne
>Priority: Blocker
> Fix For: 3.8
>
> Attachments: TombstonesAndGcGrace.md, bloom_filters.md, 
> compression.md, contributing.zip, getting_started.zip, hardware.md
>
>
> The wiki as it stands is pretty terrible.  It takes several minutes to apply 
> a single update, and as a result, it's almost never updated.  The information 
> there has very little context as to what version it applies to.  Most people 
> I've talked to that try to use the information they find there find it is 
> more confusing than helpful.
> I'd like to propose that instead of using the wiki, the doc directory in the 
> cassandra repo be used for docs (already used for CQL3 spec) in a format that 
> can be built to a variety of output formats like HTML / epub / etc.  I won't 
> start the bikeshedding on which markup format is preferable - but there are 
> several options that can work perfectly fine.  I've personally use sphinx w/ 
> restructured text, and markdown.  Both can build easily and as an added bonus 
> be pushed to readthedocs (or something similar) automatically.  For an 
> example, see cqlengine's documentation, which I think is already 
> significantly better than the wiki: 
> http://cqlengine.readthedocs.org/en/latest/
> In addition to being overall easier to maintain, putting the documentation in 
> the git repo adds context, since it evolves with the versions of Cassandra.
> If the wiki were kept even remotely up to date, I wouldn't bother with this, 
> but not having at least some basic documentation in the repo, or anywhere 
> associated with the project, is frustrating.
> For reference, the last 3 updates were:
> 1/15/15 - updating committers list
> 1/08/15 - updating contributers and how to contribute
> 12/16/14 - added a link to CQL docs from wiki frontpage (by me)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8700) replace the wiki with docs in the git repo

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336734#comment-15336734
 ] 

T Jake Luciani commented on CASSANDRA-8700:
---

bq. cherry-picked because I like my branches like my complexities: linear

Oh good. I was going to ask that you keep the commits so attribution/blame 
works.

> replace the wiki with docs in the git repo
> --
>
> Key: CASSANDRA-8700
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8700
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation and Website
>Reporter: Jon Haddad
>Assignee: Sylvain Lebresne
>Priority: Blocker
> Fix For: 3.8
>
> Attachments: TombstonesAndGcGrace.md, bloom_filters.md, 
> compression.md, contributing.zip, getting_started.zip, hardware.md
>
>
> The wiki as it stands is pretty terrible.  It takes several minutes to apply 
> a single update, and as a result, it's almost never updated.  The information 
> there has very little context as to what version it applies to.  Most people 
> I've talked to that try to use the information they find there find it is 
> more confusing than helpful.
> I'd like to propose that instead of using the wiki, the doc directory in the 
> cassandra repo be used for docs (already used for CQL3 spec) in a format that 
> can be built to a variety of output formats like HTML / epub / etc.  I won't 
> start the bikeshedding on which markup format is preferable - but there are 
> several options that can work perfectly fine.  I've personally use sphinx w/ 
> restructured text, and markdown.  Both can build easily and as an added bonus 
> be pushed to readthedocs (or something similar) automatically.  For an 
> example, see cqlengine's documentation, which I think is already 
> significantly better than the wiki: 
> http://cqlengine.readthedocs.org/en/latest/
> In addition to being overall easier to maintain, putting the documentation in 
> the git repo adds context, since it evolves with the versions of Cassandra.
> If the wiki were kept even remotely up to date, I wouldn't bother with this, 
> but not having at least some basic documentation in the repo, or anywhere 
> associated with the project, is frustrating.
> For reference, the last 3 updates were:
> 1/15/15 - updating committers list
> 1/08/15 - updating contributers and how to contribute
> 12/16/14 - added a link to CQL docs from wiki frontpage (by me)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10862) LCS repair: compact tables before making available in L0

[Chen Shen (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336712#comment-15336712
 ] 

Chen Shen commented on CASSANDRA-10862:
---

Thank you [~pauloricardomg]!, I will address your comments with a new patch 
soon.

> LCS repair: compact tables before making available in L0
> 
>
> Key: CASSANDRA-10862
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10862
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction, Streaming and Messaging
>Reporter: Jeff Ferland
>Assignee: Chen Shen
>
> When doing repair on a system with lots of mismatched ranges, the number of 
> tables in L0 goes up dramatically, as correspondingly goes the number of 
> tables referenced for a query. Latency increases dramatically in tandem.
> Eventually all the copied tables are compacted down in L0, then copied into 
> L1 (which may be a very large copy), finally reducing the number of SSTables 
> per query into the manageable range.
> It seems to me that the cleanest answer is to compact after streaming, then 
> mark tables available rather than marking available when the file itself is 
> complete.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12029) Use off-heap memtables by default

[Robert Stupp (JIRA)]

Robert Stupp created CASSANDRA-12029:


 Summary: Use off-heap memtables by default
 Key: CASSANDRA-12029
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12029
 Project: Cassandra
  Issue Type: Improvement
Reporter: Robert Stupp
Priority: Minor


A memtable configuration for 0mb on-heap and >0 mb off-heap is currently 
rejected.

I'm not fully sure whether this is intentional, but making the memtable 
completely off-heap could reduce heap usage and GC pressure using a 
configuration like this:
{code}
memtable_heap_space_in_mb: 0
memtable_offheap_space_in_mb: 1024
memtable_allocation_type: offheap_objects
{code}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12028) Proposal to change concurrent_* and native_transport_max_threads defaults

[Robert Stupp (JIRA)]

Robert Stupp created CASSANDRA-12028:


 Summary: Proposal to change concurrent_* and 
native_transport_max_threads defaults
 Key: CASSANDRA-12028
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12028
 Project: Cassandra
  Issue Type: Improvement
Reporter: Robert Stupp
Priority: Minor


The current recommendations in c.yaml regarding 
{{concurrent_reads/writes/counter_writes/materialized_view_writes}} and 
{{native_transport_max_threads}} feel oversized.

While working on some load/stress tests, reducing these to the number of CPUs 
and also lowering {{native_transport_max_threads}} to 2xCPUs had no influence 
to the actual performance but reduced the risk of allocating too many threads 
when a node ran into a longer GC and requests against that node piled up.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12027) Provide metrics for thread pool heap and CPU usage

[Robert Stupp (JIRA)]

Robert Stupp created CASSANDRA-12027:


 Summary: Provide metrics for thread pool heap and CPU usage
 Key: CASSANDRA-12027
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12027
 Project: Cassandra
  Issue Type: Improvement
Reporter: Robert Stupp
Assignee: Robert Stupp
Priority: Minor
 Fix For: 3.x


While working on some load tests and digging through the causes of certain 
things, it was very useful to know which thread pools actually consume how much 
heap and CPU.

I already have some general implementation in place, but this assumes that the 
thread names stay constant. CASSANDRA-11966 is about to change this, so the 
general approach no longer works. Instead, measuring CPU and heap usage would 
need to be accumulated in SEPWorker for the actual (logical) thread pool.

Purpose of this ticket is to provide the following metrics:
* CPU and heap usage per statically named C* thread
* CPU and heap usage per non-shared C* thread pool
* CPU and heap usage per shared C* thread pool
* CPU and heap usage for Netty thread pools

This information would be exposed as "ordinary" metrics.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12026) update NEWS.txt to explain system schema exceptions during partial cluster upgrade

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12026:

Attachment: 12026_v1.txt

Patch attached.

> update NEWS.txt to explain system schema exceptions during partial cluster 
> upgrade
> --
>
> Key: CASSANDRA-12026
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12026
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Russ Hatch
>Assignee: Joshua McKenzie
> Attachments: 12026_v1.txt
>
>
> Upgrade tests found this exception occuring during upgrades:
> {noformat}
> node2: ERROR [MessagingService-Incoming-/127.0.0.1] 2016-06-16 20:14:59,268 
> CassandraDaemon.java:217 - Exception in thread 
> Thread[MessagingService-Incoming-/127.0.0.1,5,main]
> java.lang.RuntimeException: Unknown column cdc during deserialization
>   at 
> org.apache.cassandra.db.Columns$Serializer.deserialize(Columns.java:433) 
> ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.SerializationHeader$Serializer.deserializeForMessaging(SerializationHeader.java:407)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.deserializeHeader(UnfilteredRowIteratorSerializer.java:192)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize30(PartitionUpdate.java:668)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize(PartitionUpdate.java:656)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:341)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:350)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.service.MigrationManager$MigrationsSerializer.deserialize(MigrationManager.java:610)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.service.MigrationManager$MigrationsSerializer.deserialize(MigrationManager.java:593)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at org.apache.cassandra.net.MessageIn.read(MessageIn.java:114) 
> ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:190)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:178)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:92)
>  ~[apache-cassandra-3.7.jar:3.7]
> {noformat}
> Which is apparently normal and should subside after full cluster upgrade to 
> post-3.0 versions.
> NEWS.txt needs an update to let users know this is not a problem during their 
> upgrade.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12026) update NEWS.txt to explain system schema exceptions during partial cluster upgrade

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12026:

Reviewer: Carl Yeksigian
  Status: Patch Available  (was: Open)

> update NEWS.txt to explain system schema exceptions during partial cluster 
> upgrade
> --
>
> Key: CASSANDRA-12026
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12026
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Russ Hatch
>Assignee: Joshua McKenzie
> Attachments: 12026_v1.txt
>
>
> Upgrade tests found this exception occuring during upgrades:
> {noformat}
> node2: ERROR [MessagingService-Incoming-/127.0.0.1] 2016-06-16 20:14:59,268 
> CassandraDaemon.java:217 - Exception in thread 
> Thread[MessagingService-Incoming-/127.0.0.1,5,main]
> java.lang.RuntimeException: Unknown column cdc during deserialization
>   at 
> org.apache.cassandra.db.Columns$Serializer.deserialize(Columns.java:433) 
> ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.SerializationHeader$Serializer.deserializeForMessaging(SerializationHeader.java:407)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.deserializeHeader(UnfilteredRowIteratorSerializer.java:192)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize30(PartitionUpdate.java:668)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize(PartitionUpdate.java:656)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:341)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:350)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.service.MigrationManager$MigrationsSerializer.deserialize(MigrationManager.java:610)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.service.MigrationManager$MigrationsSerializer.deserialize(MigrationManager.java:593)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at org.apache.cassandra.net.MessageIn.read(MessageIn.java:114) 
> ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:190)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:178)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:92)
>  ~[apache-cassandra-3.7.jar:3.7]
> {noformat}
> Which is apparently normal and should subside after full cluster upgrade to 
> post-3.0 versions.
> NEWS.txt needs an update to let users know this is not a problem during their 
> upgrade.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12026) update NEWS.txt to explain system schema exceptions during partial cluster upgrade

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12026:

Assignee: Joshua McKenzie

> update NEWS.txt to explain system schema exceptions during partial cluster 
> upgrade
> --
>
> Key: CASSANDRA-12026
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12026
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Russ Hatch
>Assignee: Joshua McKenzie
>
> Upgrade tests found this exception occuring during upgrades:
> {noformat}
> node2: ERROR [MessagingService-Incoming-/127.0.0.1] 2016-06-16 20:14:59,268 
> CassandraDaemon.java:217 - Exception in thread 
> Thread[MessagingService-Incoming-/127.0.0.1,5,main]
> java.lang.RuntimeException: Unknown column cdc during deserialization
>   at 
> org.apache.cassandra.db.Columns$Serializer.deserialize(Columns.java:433) 
> ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.SerializationHeader$Serializer.deserializeForMessaging(SerializationHeader.java:407)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.deserializeHeader(UnfilteredRowIteratorSerializer.java:192)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize30(PartitionUpdate.java:668)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize(PartitionUpdate.java:656)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:341)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:350)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.service.MigrationManager$MigrationsSerializer.deserialize(MigrationManager.java:610)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.service.MigrationManager$MigrationsSerializer.deserialize(MigrationManager.java:593)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at org.apache.cassandra.net.MessageIn.read(MessageIn.java:114) 
> ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:190)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:178)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:92)
>  ~[apache-cassandra-3.7.jar:3.7]
> {noformat}
> Which is apparently normal and should subside after full cluster upgrade to 
> post-3.0 versions.
> NEWS.txt needs an update to let users know this is not a problem during their 
> upgrade.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12026) update NEWS.txt to explain system schema exceptions during partial cluster upgrade

[Russ Hatch (JIRA)]

Russ Hatch created CASSANDRA-12026:
--

 Summary: update NEWS.txt to explain system schema exceptions 
during partial cluster upgrade
 Key: CASSANDRA-12026
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12026
 Project: Cassandra
  Issue Type: Bug
Reporter: Russ Hatch


Upgrade tests found this exception occuring during upgrades:
{noformat}
node2: ERROR [MessagingService-Incoming-/127.0.0.1] 2016-06-16 20:14:59,268 
CassandraDaemon.java:217 - Exception in thread 
Thread[MessagingService-Incoming-/127.0.0.1,5,main]
java.lang.RuntimeException: Unknown column cdc during deserialization
at 
org.apache.cassandra.db.Columns$Serializer.deserialize(Columns.java:433) 
~[apache-cassandra-3.7.jar:3.7]
at 
org.apache.cassandra.db.SerializationHeader$Serializer.deserializeForMessaging(SerializationHeader.java:407)
 ~[apache-cassandra-3.7.jar:3.7]
at 
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.deserializeHeader(UnfilteredRowIteratorSerializer.java:192)
 ~[apache-cassandra-3.7.jar:3.7]
at 
org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize30(PartitionUpdate.java:668)
 ~[apache-cassandra-3.7.jar:3.7]
at 
org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize(PartitionUpdate.java:656)
 ~[apache-cassandra-3.7.jar:3.7]
at 
org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:341)
 ~[apache-cassandra-3.7.jar:3.7]
at 
org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:350)
 ~[apache-cassandra-3.7.jar:3.7]
at 
org.apache.cassandra.service.MigrationManager$MigrationsSerializer.deserialize(MigrationManager.java:610)
 ~[apache-cassandra-3.7.jar:3.7]
at 
org.apache.cassandra.service.MigrationManager$MigrationsSerializer.deserialize(MigrationManager.java:593)
 ~[apache-cassandra-3.7.jar:3.7]
at org.apache.cassandra.net.MessageIn.read(MessageIn.java:114) 
~[apache-cassandra-3.7.jar:3.7]
at 
org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:190)
 ~[apache-cassandra-3.7.jar:3.7]
at 
org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:178)
 ~[apache-cassandra-3.7.jar:3.7]
at 
org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:92)
 ~[apache-cassandra-3.7.jar:3.7]
{noformat}

Which is apparently normal and should subside after full cluster upgrade to 
post-3.0 versions.

NEWS.txt needs an update to let users know this is not a problem during their 
upgrade.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11961) Nonfatal NPE in CompactionMetrics

[Robert Stupp (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp updated CASSANDRA-11961:
-
   Resolution: Fixed
Fix Version/s: 3.8
   Status: Resolved  (was: Patch Available)

+1

Thanks!
Committed as 4754cddfb980226326994b60383ef83baed06714 to trunk

Test results:
||trunk|[branch|https://github.com/apache/cassandra/compare/trunk...snazy:11961-trunk]|[testall|http://cassci.datastax.com/view/Dev/view/snazy/job/snazy-11961-trunk-testall/lastSuccessfulBuild/]|[dtest|http://cassci.datastax.com/view/Dev/view/snazy/job/snazy-11961-trunk-dtest/lastSuccessfulBuild/]



> Nonfatal NPE in CompactionMetrics
> -
>
> Key: CASSANDRA-11961
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11961
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Robert Stupp
>Assignee: Achal Shah
>Priority: Minor
>  Labels: lhf
> Fix For: 3.8
>
>
> Just saw the following NPE on trunk. Means, that {{metaData}} from 
> {{CFMetaData metaData = compaction.getCompactionInfo().getCFMetaData();}} is 
> {{null}}. A simple {{if (metaData == null) continue;}} should fix this.
> {code}
> Caused by: java.lang.NullPointerException: null
>   at 
> org.apache.cassandra.metrics.CompactionMetrics$2.getValue(CompactionMetrics.java:103)
>  ~[main/:na]
>   at 
> org.apache.cassandra.metrics.CompactionMetrics$2.getValue(CompactionMetrics.java:78)
>  ~[main/:na]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Fix non-fatal NPE in CompactionMetrics

[snazy]

Repository: cassandra
Updated Branches:
  refs/heads/trunk c35e55de4 -> 4754cddfb


Fix non-fatal NPE in CompactionMetrics

patch by Achal Shah; reviewed by Robert Stupp for CASSANDRA-11961


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4754cddf
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4754cddf
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4754cddf

Branch: refs/heads/trunk
Commit: 4754cddfb980226326994b60383ef83baed06714
Parents: c35e55d
Author: Achal Shah 
Authored: Fri Jun 17 20:17:08 2016 +0200
Committer: Robert Stupp 
Committed: Fri Jun 17 20:17:08 2016 +0200

--
 src/java/org/apache/cassandra/metrics/CompactionMetrics.java | 4 
 1 file changed, 4 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4754cddf/src/java/org/apache/cassandra/metrics/CompactionMetrics.java
--
diff --git a/src/java/org/apache/cassandra/metrics/CompactionMetrics.java 
b/src/java/org/apache/cassandra/metrics/CompactionMetrics.java
index f3a9d00..9d2863f 100644
--- a/src/java/org/apache/cassandra/metrics/CompactionMetrics.java
+++ b/src/java/org/apache/cassandra/metrics/CompactionMetrics.java
@@ -100,6 +100,10 @@ public class CompactionMetrics implements 
CompactionManager.CompactionExecutorSt
 for (CompactionInfo.Holder compaction : compactions)
 {
 CFMetaData metaData = 
compaction.getCompactionInfo().getCFMetaData();
+if (metaData == null)
+{
+continue;
+}
 if (!resultMap.containsKey(metaData.ksName))
 {
 resultMap.put(metaData.ksName, new HashMap<>());

[jira] [Updated] (CASSANDRA-11798) Allow specification of 'time' column value as number in CQL query.

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11798:

   Resolution: Fixed
Fix Version/s: 3.0.8
   3.8
   Status: Resolved  (was: Ready to Commit)

Looks like a simple oversight when I originally implemented the type.

[Committed|https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=commit;h=863dbc78777101eab08f23877a4d9ffc92ffb6b4]
 to 3.0 and trunk since 3.7 is already cut.

Thanks for adding the test in there as well!

> Allow specification of 'time' column value as number in CQL query.
> --
>
> Key: CASSANDRA-11798
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11798
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Andy Tolbert
>Assignee: Alex Petrov
>Priority: Minor
> Fix For: 3.8, 3.0.8
>
>
> The 'time' cql type is internally stored and sent over the protocol as an 
> 8-byte long value representing nanoseconds since midnight.   
> When specifying a time column value as a number in a simple statement,, C* 
> currently responds with:
> {noformat}
> InvalidRequest: code=2200 [Invalid query] message="Invalid INTEGER constant 
> (420) for "time" of type time"
> {noformat}
> Instead one must provide this value as a string (i.e. {{'420'}}) or 
> use an {{HH.MM.SS.s}} format (i.e. {{'00:00:42.0'}}).  It 
> would be nice if it supported unquoted numbers as well.
> Example:
> {code}
> cqlsh:simple> CREATE TABLE timeentity (id varchar PRIMARY KEY, time time);
> # Doesn't work
> cqlsh:simple> INSERT into timeentity (id, time) values ('1', 420);
> InvalidRequest: code=2200 [Invalid query] message="Invalid INTEGER constant 
> (420) for "time" of type time"
> # Works
> cqlsh:simple> INSERT into timeentity (id, time) values ('1', '420');
> {code}
> When using prepared statements or simple statements with parameters, one 
> could provide a long value, depending on the driver implementation.  I.E. the 
> java driver has {{setTime(int index, long v)}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11798) Allow specification of 'time' column value as number in CQL query.

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11798:

Status: Ready to Commit  (was: Patch Available)

> Allow specification of 'time' column value as number in CQL query.
> --
>
> Key: CASSANDRA-11798
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11798
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Andy Tolbert
>Assignee: Alex Petrov
>Priority: Minor
>
> The 'time' cql type is internally stored and sent over the protocol as an 
> 8-byte long value representing nanoseconds since midnight.   
> When specifying a time column value as a number in a simple statement,, C* 
> currently responds with:
> {noformat}
> InvalidRequest: code=2200 [Invalid query] message="Invalid INTEGER constant 
> (420) for "time" of type time"
> {noformat}
> Instead one must provide this value as a string (i.e. {{'420'}}) or 
> use an {{HH.MM.SS.s}} format (i.e. {{'00:00:42.0'}}).  It 
> would be nice if it supported unquoted numbers as well.
> Example:
> {code}
> cqlsh:simple> CREATE TABLE timeentity (id varchar PRIMARY KEY, time time);
> # Doesn't work
> cqlsh:simple> INSERT into timeentity (id, time) values ('1', 420);
> InvalidRequest: code=2200 [Invalid query] message="Invalid INTEGER constant 
> (420) for "time" of type time"
> # Works
> cqlsh:simple> INSERT into timeentity (id, time) values ('1', '420');
> {code}
> When using prepared statements or simple statements with parameters, one 
> could provide a long value, depending on the driver implementation.  I.E. the 
> java driver has {{setTime(int index, long v)}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/3] cassandra git commit: Allow specification of 'time' column value as number in CQL query

[jmckenzie]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 dbc40b1ed -> 863dbc787
  refs/heads/trunk 6c9c09118 -> c35e55de4


Allow specification of 'time' column value as number in CQL query

Patch by Alex Petrov; reviewed by jmckenzie for CASSANDRA-11798


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/863dbc78
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/863dbc78
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/863dbc78

Branch: refs/heads/cassandra-3.0
Commit: 863dbc78777101eab08f23877a4d9ffc92ffb6b4
Parents: dbc40b1
Author: Alex Petrov 
Authored: Wed May 25 16:19:47 2016 +0200
Committer: Josh McKenzie 
Committed: Fri Jun 17 13:42:39 2016 -0400

--
 .../org/apache/cassandra/cql3/Constants.java|  1 +
 .../cassandra/db/marshal/TimeTypeTest.java  | 27 ++--
 2 files changed, 26 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/863dbc78/src/java/org/apache/cassandra/cql3/Constants.java
--
diff --git a/src/java/org/apache/cassandra/cql3/Constants.java 
b/src/java/org/apache/cassandra/cql3/Constants.java
index 4982c49..a2bacdf 100644
--- a/src/java/org/apache/cassandra/cql3/Constants.java
+++ b/src/java/org/apache/cassandra/cql3/Constants.java
@@ -196,6 +196,7 @@ public abstract class Constants
 case FLOAT:
 case INT:
 case SMALLINT:
+case TIME:
 case TIMESTAMP:
 case TINYINT:
 case VARINT:

http://git-wip-us.apache.org/repos/asf/cassandra/blob/863dbc78/test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java 
b/test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java
index 699c805..d61d2c6 100644
--- a/test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java
+++ b/test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java
@@ -23,11 +23,14 @@ import java.nio.ByteBuffer;
 import java.util.concurrent.TimeUnit;
 
 import org.junit.Test;
+
+import org.apache.cassandra.cql3.*;
 import org.apache.cassandra.serializers.TimeSerializer;
 
-public class TimeTypeTest
+public class TimeTypeTest extends CQLTester
 {
-@Test public void TestComparison()
+@Test
+public void testComparison()
 {
 Long t1 = TimeSerializer.timeStringToLong("01:00:00.123456789");
 Long t2 = new Long((1L * 60L * 60L * 1000L * 1000L * 1000L) + 
123456789);
@@ -58,4 +61,24 @@ public class TimeTypeTest
 b2 = TimeSerializer.instance.serialize(0L);
 assert TimeType.instance.compare(b1, b2) > 0 : "Failed > comparison 
against max range.";
 }
+
+/**
+ * Check that time is correctly recognized and validated
+ */
+@Test
+public void testTime() throws Throwable
+{
+createTable("CREATE TABLE %s (id int, cl int, ts time, PRIMARY KEY(id, 
cl));");
+
+execute("INSERT into %s (id, cl, ts) values (1, 1, 420);");
+execute("INSERT into %s (id, cl, ts) values (1, 2, '420');");
+execute("INSERT into %s (id, cl, ts) values (1, 3, 
'00:00:42.0');");
+execute("INSERT into %s (id, cl, ts) values (1, 4, ?);", 420L);
+
+assertRows(execute("SELECT * FROM %s"),
+   row(1, 1, 420L),
+   row(1, 2, 420L),
+   row(1, 3, 420L),
+   row(1, 4, 420L));
+}
 }

[3/3] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[jmckenzie]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c35e55de
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c35e55de
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c35e55de

Branch: refs/heads/trunk
Commit: c35e55de455dc790264f0a9e72be19967cbadc8a
Parents: 6c9c091 863dbc7
Author: Josh McKenzie 
Authored: Fri Jun 17 13:43:56 2016 -0400
Committer: Josh McKenzie 
Committed: Fri Jun 17 13:43:56 2016 -0400

--
 .../org/apache/cassandra/cql3/Constants.java|  1 +
 .../cassandra/db/marshal/TimeTypeTest.java  | 27 ++--
 2 files changed, 26 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c35e55de/src/java/org/apache/cassandra/cql3/Constants.java
--

[2/3] cassandra git commit: Allow specification of 'time' column value as number in CQL query

[jmckenzie]

Allow specification of 'time' column value as number in CQL query

Patch by Alex Petrov; reviewed by jmckenzie for CASSANDRA-11798


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/863dbc78
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/863dbc78
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/863dbc78

Branch: refs/heads/trunk
Commit: 863dbc78777101eab08f23877a4d9ffc92ffb6b4
Parents: dbc40b1
Author: Alex Petrov 
Authored: Wed May 25 16:19:47 2016 +0200
Committer: Josh McKenzie 
Committed: Fri Jun 17 13:42:39 2016 -0400

--
 .../org/apache/cassandra/cql3/Constants.java|  1 +
 .../cassandra/db/marshal/TimeTypeTest.java  | 27 ++--
 2 files changed, 26 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/863dbc78/src/java/org/apache/cassandra/cql3/Constants.java
--
diff --git a/src/java/org/apache/cassandra/cql3/Constants.java 
b/src/java/org/apache/cassandra/cql3/Constants.java
index 4982c49..a2bacdf 100644
--- a/src/java/org/apache/cassandra/cql3/Constants.java
+++ b/src/java/org/apache/cassandra/cql3/Constants.java
@@ -196,6 +196,7 @@ public abstract class Constants
 case FLOAT:
 case INT:
 case SMALLINT:
+case TIME:
 case TIMESTAMP:
 case TINYINT:
 case VARINT:

http://git-wip-us.apache.org/repos/asf/cassandra/blob/863dbc78/test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java 
b/test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java
index 699c805..d61d2c6 100644
--- a/test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java
+++ b/test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java
@@ -23,11 +23,14 @@ import java.nio.ByteBuffer;
 import java.util.concurrent.TimeUnit;
 
 import org.junit.Test;
+
+import org.apache.cassandra.cql3.*;
 import org.apache.cassandra.serializers.TimeSerializer;
 
-public class TimeTypeTest
+public class TimeTypeTest extends CQLTester
 {
-@Test public void TestComparison()
+@Test
+public void testComparison()
 {
 Long t1 = TimeSerializer.timeStringToLong("01:00:00.123456789");
 Long t2 = new Long((1L * 60L * 60L * 1000L * 1000L * 1000L) + 
123456789);
@@ -58,4 +61,24 @@ public class TimeTypeTest
 b2 = TimeSerializer.instance.serialize(0L);
 assert TimeType.instance.compare(b1, b2) > 0 : "Failed > comparison 
against max range.";
 }
+
+/**
+ * Check that time is correctly recognized and validated
+ */
+@Test
+public void testTime() throws Throwable
+{
+createTable("CREATE TABLE %s (id int, cl int, ts time, PRIMARY KEY(id, 
cl));");
+
+execute("INSERT into %s (id, cl, ts) values (1, 1, 420);");
+execute("INSERT into %s (id, cl, ts) values (1, 2, '420');");
+execute("INSERT into %s (id, cl, ts) values (1, 3, 
'00:00:42.0');");
+execute("INSERT into %s (id, cl, ts) values (1, 4, ?);", 420L);
+
+assertRows(execute("SELECT * FROM %s"),
+   row(1, 1, 420L),
+   row(1, 2, 420L),
+   row(1, 3, 420L),
+   row(1, 4, 420L));
+}
 }

[jira] [Updated] (CASSANDRA-11948) Wrong ByteBuffer comparisons in TimeTypeTest

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11948:

   Resolution: Fixed
Fix Version/s: (was: 3.0.x)
   (was: 2.2.x)
   (was: 3.x)
   3.0.8
   3.8
   2.2.7
   Status: Resolved  (was: Ready to Commit)

[Committed|https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=commit;h=fd91e1593941d39c536d28542af539e5009be0d4]
 and merged up.

Having written those original tests, the break from the "-1, 0, 1" idiom on 
comparison is quite the surprise to me. Good catch!

> Wrong ByteBuffer comparisons in TimeTypeTest
> 
>
> Key: CASSANDRA-11948
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11948
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Rei Odaira
>Assignee: Rei Odaira
>Priority: Minor
> Fix For: 2.2.7, 3.8, 3.0.8
>
> Attachments: 11948-2.2.txt
>
>
> The last three tests in org.apache.cassandra.db.marshal.TimeTypeTest compare 
> the returned values of AbstractType.compare() with -1 or 1. However,  
> AbstractType.compare() is supposed to return a negative integer, zero, or a 
> positive integer, and it does not necessarily return -1 or 1.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11948) Wrong ByteBuffer comparisons in TimeTypeTest

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11948:

Status: Ready to Commit  (was: Patch Available)

> Wrong ByteBuffer comparisons in TimeTypeTest
> 
>
> Key: CASSANDRA-11948
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11948
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Rei Odaira
>Assignee: Rei Odaira
>Priority: Minor
> Fix For: 2.2.x, 3.0.x, 3.x
>
> Attachments: 11948-2.2.txt
>
>
> The last three tests in org.apache.cassandra.db.marshal.TimeTypeTest compare 
> the returned values of AbstractType.compare() with -1 or 1. However,  
> AbstractType.compare() is supposed to return a negative integer, zero, or a 
> positive integer, and it does not necessarily return -1 or 1.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/6] cassandra git commit: Fix TimeTypeTest - comparing AbstractType.compare() with -1 or 1.

[jmckenzie]

Fix TimeTypeTest - comparing AbstractType.compare() with -1 or 1.

Patch by rodaira; reviewed by jmckenzie for CASSANDRA-11948


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fd91e159
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fd91e159
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fd91e159

Branch: refs/heads/cassandra-3.0
Commit: fd91e1593941d39c536d28542af539e5009be0d4
Parents: 0d8e7b1
Author: Rei Odaira 
Authored: Thu Jun 2 16:01:17 2016 -0500
Committer: Josh McKenzie 
Committed: Fri Jun 17 12:49:20 2016 -0400

--
 test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/fd91e159/test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java 
b/test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java
index 3057b9c..699c805 100644
--- a/test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java
+++ b/test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java
@@ -48,14 +48,14 @@ public class TimeTypeTest
 
 b1 = TimeSerializer.instance.serialize(0L);
 b2 = TimeSerializer.instance.serialize(1000L);
-assert TimeType.instance.compare(b1, b2) == -1 : "Failed < comparison 
on 0";
+assert TimeType.instance.compare(b1, b2) < 0 : "Failed < comparison on 
0";
 
 b1 = TimeSerializer.instance.serialize(0L);
 b2 = TimeSerializer.instance.serialize(TimeUnit.DAYS.toNanos(1));
-assert TimeType.instance.compare(b1, b2) == -1 : "Failed < comparison 
against max range.";
+assert TimeType.instance.compare(b1, b2) < 0 : "Failed < comparison 
against max range.";
 
 b1 = TimeSerializer.instance.serialize(TimeUnit.DAYS.toNanos(1));
 b2 = TimeSerializer.instance.serialize(0L);
-assert TimeType.instance.compare(b1, b2) == 1 : "Failed > comparison 
against max range.";
+assert TimeType.instance.compare(b1, b2) > 0 : "Failed > comparison 
against max range.";
 }
 }

[3/6] cassandra git commit: Fix TimeTypeTest - comparing AbstractType.compare() with -1 or 1.

[jmckenzie]

Fix TimeTypeTest - comparing AbstractType.compare() with -1 or 1.

Patch by rodaira; reviewed by jmckenzie for CASSANDRA-11948


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fd91e159
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fd91e159
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fd91e159

Branch: refs/heads/trunk
Commit: fd91e1593941d39c536d28542af539e5009be0d4
Parents: 0d8e7b1
Author: Rei Odaira 
Authored: Thu Jun 2 16:01:17 2016 -0500
Committer: Josh McKenzie 
Committed: Fri Jun 17 12:49:20 2016 -0400

--
 test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/fd91e159/test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java 
b/test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java
index 3057b9c..699c805 100644
--- a/test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java
+++ b/test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java
@@ -48,14 +48,14 @@ public class TimeTypeTest
 
 b1 = TimeSerializer.instance.serialize(0L);
 b2 = TimeSerializer.instance.serialize(1000L);
-assert TimeType.instance.compare(b1, b2) == -1 : "Failed < comparison 
on 0";
+assert TimeType.instance.compare(b1, b2) < 0 : "Failed < comparison on 
0";
 
 b1 = TimeSerializer.instance.serialize(0L);
 b2 = TimeSerializer.instance.serialize(TimeUnit.DAYS.toNanos(1));
-assert TimeType.instance.compare(b1, b2) == -1 : "Failed < comparison 
against max range.";
+assert TimeType.instance.compare(b1, b2) < 0 : "Failed < comparison 
against max range.";
 
 b1 = TimeSerializer.instance.serialize(TimeUnit.DAYS.toNanos(1));
 b2 = TimeSerializer.instance.serialize(0L);
-assert TimeType.instance.compare(b1, b2) == 1 : "Failed > comparison 
against max range.";
+assert TimeType.instance.compare(b1, b2) > 0 : "Failed > comparison 
against max range.";
 }
 }

[6/6] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[jmckenzie]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6c9c0911
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6c9c0911
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6c9c0911

Branch: refs/heads/trunk
Commit: 6c9c0911852c6d198c52d89d1fdb7da8036ef44d
Parents: 0ff8ab0 dbc40b1
Author: Josh McKenzie 
Authored: Fri Jun 17 12:50:24 2016 -0400
Committer: Josh McKenzie 
Committed: Fri Jun 17 12:50:24 2016 -0400

--
 test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--

[4/6] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[jmckenzie]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dbc40b1e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dbc40b1e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dbc40b1e

Branch: refs/heads/trunk
Commit: dbc40b1ed8dc0bd018b7e31c24363bb9e3e0e0c9
Parents: 70115ff fd91e15
Author: Josh McKenzie 
Authored: Fri Jun 17 12:50:02 2016 -0400
Committer: Josh McKenzie 
Committed: Fri Jun 17 12:50:02 2016 -0400

--
 test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--

[1/6] cassandra git commit: Fix TimeTypeTest - comparing AbstractType.compare() with -1 or 1.

[jmckenzie]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.2 0d8e7b18e -> fd91e1593
  refs/heads/cassandra-3.0 70115ff19 -> dbc40b1ed
  refs/heads/trunk 0ff8ab0c2 -> 6c9c09118


Fix TimeTypeTest - comparing AbstractType.compare() with -1 or 1.

Patch by rodaira; reviewed by jmckenzie for CASSANDRA-11948


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fd91e159
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fd91e159
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fd91e159

Branch: refs/heads/cassandra-2.2
Commit: fd91e1593941d39c536d28542af539e5009be0d4
Parents: 0d8e7b1
Author: Rei Odaira 
Authored: Thu Jun 2 16:01:17 2016 -0500
Committer: Josh McKenzie 
Committed: Fri Jun 17 12:49:20 2016 -0400

--
 test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/fd91e159/test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java 
b/test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java
index 3057b9c..699c805 100644
--- a/test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java
+++ b/test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java
@@ -48,14 +48,14 @@ public class TimeTypeTest
 
 b1 = TimeSerializer.instance.serialize(0L);
 b2 = TimeSerializer.instance.serialize(1000L);
-assert TimeType.instance.compare(b1, b2) == -1 : "Failed < comparison 
on 0";
+assert TimeType.instance.compare(b1, b2) < 0 : "Failed < comparison on 
0";
 
 b1 = TimeSerializer.instance.serialize(0L);
 b2 = TimeSerializer.instance.serialize(TimeUnit.DAYS.toNanos(1));
-assert TimeType.instance.compare(b1, b2) == -1 : "Failed < comparison 
against max range.";
+assert TimeType.instance.compare(b1, b2) < 0 : "Failed < comparison 
against max range.";
 
 b1 = TimeSerializer.instance.serialize(TimeUnit.DAYS.toNanos(1));
 b2 = TimeSerializer.instance.serialize(0L);
-assert TimeType.instance.compare(b1, b2) == 1 : "Failed > comparison 
against max range.";
+assert TimeType.instance.compare(b1, b2) > 0 : "Failed > comparison 
against max range.";
 }
 }

[5/6] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[jmckenzie]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dbc40b1e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dbc40b1e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dbc40b1e

Branch: refs/heads/cassandra-3.0
Commit: dbc40b1ed8dc0bd018b7e31c24363bb9e3e0e0c9
Parents: 70115ff fd91e15
Author: Josh McKenzie 
Authored: Fri Jun 17 12:50:02 2016 -0400
Committer: Josh McKenzie 
Committed: Fri Jun 17 12:50:02 2016 -0400

--
 test/unit/org/apache/cassandra/db/marshal/TimeTypeTest.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--

[jira] [Updated] (CASSANDRA-12023) Schema upgrade bug with super columns

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12023:

Assignee: Tyler Hobbs

> Schema upgrade bug with super columns
> -
>
> Key: CASSANDRA-12023
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12023
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Jeremiah Jordan
>Assignee: Tyler Hobbs
>Priority: Critical
> Fix For: 3.0.x, 3.x
>
>
> Doing some upgrade tests starting on 2.0 to 2.1 to 3.0 we hit the follow bug 
> that prevents 3.0 nodes from starting.  Running the test a few times with 
> different waits and flushing sometimes or not I have seen the following 
> errors:
> {code}
> ERROR [main] 2016-06-17 10:42:40,112 CassandraDaemon.java:698 - Exception 
> encountered during startup
> org.apache.cassandra.serializers.MarshalException: cannot parse 'value' as 
> hex bytes
>   at 
> org.apache.cassandra.db.marshal.BytesType.fromString(BytesType.java:45) 
> ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.createColumnFromColumnRow(LegacySchemaMigrator.java:682)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.createColumnsFromColumnRows(LegacySchemaMigrator.java:641)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.decodeTableMetadata(LegacySchemaMigrator.java:316)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.readTableMetadata(LegacySchemaMigrator.java:273)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.readTable(LegacySchemaMigrator.java:244)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.lambda$readTables$7(LegacySchemaMigrator.java:237)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at java.util.ArrayList.forEach(ArrayList.java:1249) ~[na:1.8.0_66]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.readTables(LegacySchemaMigrator.java:237)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.readKeyspace(LegacySchemaMigrator.java:186)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.lambda$readSchema$4(LegacySchemaMigrator.java:177)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at java.util.ArrayList.forEach(ArrayList.java:1249) ~[na:1.8.0_66]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.readSchema(LegacySchemaMigrator.java:177)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.migrate(LegacySchemaMigrator.java:77)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:229) 
> [apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:557)
>  [apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:685) 
> [apache-cassandra-3.0.7.jar:3.0.7]
> Caused by: java.lang.NumberFormatException: An hex string representing bytes 
> must have an even length
>   at org.apache.cassandra.utils.Hex.hexToBytes(Hex.java:57) 
> ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.db.marshal.BytesType.fromString(BytesType.java:41) 
> ~[apache-cassandra-3.0.7.jar:3.0.7]
>   ... 16 common frames omitted
> {code}
> {code}
> ERROR [main] 2016-06-17 10:49:21,326 CassandraDaemon.java:698 - Exception 
> encountered during startup
> java.lang.RuntimeException: org.codehaus.jackson.JsonParseException: 
> Unexpected character ('K' (code 75)): expected a valid value (number, String, 
> array, object, 'true', 'false' or 'null')
>  at [Source: java.io.StringReader@60d4475f; line: 1, column: 2]
>   at 
> org.apache.cassandra.utils.FBUtilities.fromJsonMap(FBUtilities.java:561) 
> ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.decodeTableParams(LegacySchemaMigrator.java:442)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.decodeTableMetadata(LegacySchemaMigrator.java:365)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.readTableMetadata(LegacySchemaMigrator.java:273)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.readTable(LegacySchemaMigrator.java:244)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> 

[jira] [Updated] (CASSANDRA-11576) Add support for JNA mlockall(2) on POWER

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11576:

Status: Ready to Commit  (was: Patch Available)

> Add support for JNA mlockall(2) on POWER
> 
>
> Key: CASSANDRA-11576
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11576
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
> Environment: POWER architecture
>Reporter: Rei Odaira
>Assignee: Rei Odaira
>Priority: Minor
> Fix For: 2.1.15, 2.2.7, 3.8, 3.0.8
>
> Attachments: 11576-2.1.txt
>
>
> org.apache.cassandra.utils.CLibrary contains hard-coded C-macro values to be 
> passed to system calls through JNA. These values are system-dependent, and as 
> far as I investigated, Linux and AIX on the IBM POWER architecture define 
> {{MCL_CURRENT}} and {{MCL_FUTURE}} (for mlockall(2)) as different values than 
> the current hard-coded values.  As a result, mlockall(2) fails on these 
> platforms.
> {code}
> WARN  18:51:51 Unknown mlockall error 22
> {code}
> I am going to provide a patch to support JNA mlockall(2) on POWER.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11576) Add support for JNA mlockall(2) on POWER

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11576:

   Resolution: Fixed
Fix Version/s: (was: 3.0.x)
   (was: 2.2.x)
   (was: 2.1.x)
   (was: 3.x)
   3.0.8
   3.8
   2.2.7
   2.1.15
   Status: Resolved  (was: Ready to Commit)

Went ahead and 
[commit|https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=commit;h=a56bc1600bc8b98b9949751873e911ad5b237fbe]
 to 2.1 and merged up, as this is a very non-invasive change.

Thanks for the patch!

> Add support for JNA mlockall(2) on POWER
> 
>
> Key: CASSANDRA-11576
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11576
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
> Environment: POWER architecture
>Reporter: Rei Odaira
>Assignee: Rei Odaira
>Priority: Minor
> Fix For: 2.1.15, 2.2.7, 3.8, 3.0.8
>
> Attachments: 11576-2.1.txt
>
>
> org.apache.cassandra.utils.CLibrary contains hard-coded C-macro values to be 
> passed to system calls through JNA. These values are system-dependent, and as 
> far as I investigated, Linux and AIX on the IBM POWER architecture define 
> {{MCL_CURRENT}} and {{MCL_FUTURE}} (for mlockall(2)) as different values than 
> the current hard-coded values.  As a result, mlockall(2) fails on these 
> platforms.
> {code}
> WARN  18:51:51 Unknown mlockall error 22
> {code}
> I am going to provide a patch to support JNA mlockall(2) on POWER.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[01/10] cassandra git commit: Set correct MCL_CURRENT and MCL_FUTURE values on IBM POWER to support mlockall(2) through JNA

[jmckenzie]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 615bf3721 -> a56bc1600
  refs/heads/cassandra-2.2 dbefa854b -> 0d8e7b18e
  refs/heads/cassandra-3.0 05bacc756 -> 70115ff19
  refs/heads/trunk 057c32997 -> 0ff8ab0c2


Set correct MCL_CURRENT and MCL_FUTURE values on IBM POWER to support 
mlockall(2) through JNA

Patch by rodaira; reviewed by jmckenzie for CASSANDRA-11576


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a56bc160
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a56bc160
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a56bc160

Branch: refs/heads/cassandra-2.1
Commit: a56bc1600bc8b98b9949751873e911ad5b237fbe
Parents: 615bf37
Author: Rei Odaira 
Authored: Fri Apr 15 10:50:45 2016 -0500
Committer: Josh McKenzie 
Committed: Fri Jun 17 12:38:25 2016 -0400

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/utils/CLibrary.java| 28 ++--
 2 files changed, 27 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a56bc160/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index ec2b48e..64cfdaa 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.15
+ * Support mlockall on IBM POWER arch (CASSANDRA-11576)
  * Cache local ranges when calculating repair neighbors (CASSANDRA-11933)
  * Allow LWT operation on static column with only partition keys 
(CASSANDRA-10532)
  * Create interval tree over canonical sstables to avoid missing sstables 
during streaming (CASSANDRA-11886)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a56bc160/src/java/org/apache/cassandra/utils/CLibrary.java
--
diff --git a/src/java/org/apache/cassandra/utils/CLibrary.java 
b/src/java/org/apache/cassandra/utils/CLibrary.java
index 1c771af..48cb67c 100644
--- a/src/java/org/apache/cassandra/utils/CLibrary.java
+++ b/src/java/org/apache/cassandra/utils/CLibrary.java
@@ -33,8 +33,8 @@ public final class CLibrary
 {
 private static final Logger logger = 
LoggerFactory.getLogger(CLibrary.class);
 
-private static final int MCL_CURRENT = 1;
-private static final int MCL_FUTURE = 2;
+private static final int MCL_CURRENT;
+private static final int MCL_FUTURE;
 
 private static final int ENOMEM = 12;
 
@@ -75,6 +75,30 @@ public final class CLibrary
 logger.warn("Obsolete version of JNA present; unable to register C 
library. Upgrade to JNA 3.2.7 or later");
 jnaAvailable = false;
 }
+
+if (System.getProperty("os.arch").toLowerCase().contains("ppc"))
+{
+if (System.getProperty("os.name").toLowerCase().contains("linux"))
+{
+   MCL_CURRENT = 0x2000;
+   MCL_FUTURE = 0x4000;
+}
+else if 
(System.getProperty("os.name").toLowerCase().contains("aix"))
+{
+MCL_CURRENT = 0x100;
+MCL_FUTURE = 0x200;
+}
+else
+{
+MCL_CURRENT = 1;
+MCL_FUTURE = 2;
+}
+}
+else
+{
+MCL_CURRENT = 1;
+MCL_FUTURE = 2;
+}
 }
 
 private static native int mlockall(int flags) throws LastErrorException;

[06/10] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[jmckenzie]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0d8e7b18
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0d8e7b18
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0d8e7b18

Branch: refs/heads/cassandra-2.2
Commit: 0d8e7b18e54a378deae93f361ecd8ed593e0729b
Parents: dbefa85 a56bc16
Author: Josh McKenzie 
Authored: Fri Jun 17 12:39:37 2016 -0400
Committer: Josh McKenzie 
Committed: Fri Jun 17 12:39:37 2016 -0400

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/utils/CLibrary.java| 28 ++--
 2 files changed, 27 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0d8e7b18/CHANGES.txt
--
diff --cc CHANGES.txt
index d9afaa3,64cfdaa..2c66ef9
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,30 -1,5 +1,31 @@@
 -2.1.15
 +2.2.7
 + * StorageService shutdown hook should use a volatile variable 
(CASSANDRA-11984)
 + * Persist local metadata earlier in startup sequence (CASSANDRA-11742)
 + * Run CommitLog tests with different compression settings (CASSANDRA-9039)
 + * cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664)
 + * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587)
 + * Fix possible race condition in CommitLog.recover (CASSANDRA-11743)
 + * Enable client encryption in sstableloader with cli options 
(CASSANDRA-11708)
 + * Possible memory leak in NIODataInputStream (CASSANDRA-11867)
 + * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669)
 + * Add seconds to cqlsh tracing session duration (CASSANDRA-11753)
 + * Prohibit Reverse Counter type as part of the PK (CASSANDRA-9395)
 + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626)
 + * Exit JVM if JMX server fails to startup (CASSANDRA-11540)
 + * Produce a heap dump when exiting on OOM (CASSANDRA-9861)
 + * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427)
 + * Restore ability to filter on clustering columns when using a 2i 
(CASSANDRA-11510)
 + * JSON datetime formatting needs timezone (CASSANDRA-11137)
 + * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502)
 + * Remove unnescessary file existence check during anticompaction 
(CASSANDRA-11660)
 + * Add missing files to debian packages (CASSANDRA-11642)
 + * Avoid calling Iterables::concat in loops during 
ModificationStatement::getFunctions (CASSANDRA-11621)
 + * cqlsh: COPY FROM should use regular inserts for single statement batches 
and
 +   report errors correctly if workers processes crash on initialization 
(CASSANDRA-11474)
 + * Always close cluster with connection in CqlRecordWriter (CASSANDRA-11553)
 + * Fix slice queries on ordered COMPACT tables (CASSANDRA-10988)
 +Merged from 2.1:
+  * Support mlockall on IBM POWER arch (CASSANDRA-11576)
   * Cache local ranges when calculating repair neighbors (CASSANDRA-11933)
   * Allow LWT operation on static column with only partition keys 
(CASSANDRA-10532)
   * Create interval tree over canonical sstables to avoid missing sstables 
during streaming (CASSANDRA-11886)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0d8e7b18/src/java/org/apache/cassandra/utils/CLibrary.java
--

[04/10] cassandra git commit: Set correct MCL_CURRENT and MCL_FUTURE values on IBM POWER to support mlockall(2) through JNA

[jmckenzie]

Set correct MCL_CURRENT and MCL_FUTURE values on IBM POWER to support 
mlockall(2) through JNA

Patch by rodaira; reviewed by jmckenzie for CASSANDRA-11576


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a56bc160
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a56bc160
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a56bc160

Branch: refs/heads/trunk
Commit: a56bc1600bc8b98b9949751873e911ad5b237fbe
Parents: 615bf37
Author: Rei Odaira 
Authored: Fri Apr 15 10:50:45 2016 -0500
Committer: Josh McKenzie 
Committed: Fri Jun 17 12:38:25 2016 -0400

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/utils/CLibrary.java| 28 ++--
 2 files changed, 27 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a56bc160/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index ec2b48e..64cfdaa 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.15
+ * Support mlockall on IBM POWER arch (CASSANDRA-11576)
  * Cache local ranges when calculating repair neighbors (CASSANDRA-11933)
  * Allow LWT operation on static column with only partition keys 
(CASSANDRA-10532)
  * Create interval tree over canonical sstables to avoid missing sstables 
during streaming (CASSANDRA-11886)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a56bc160/src/java/org/apache/cassandra/utils/CLibrary.java
--
diff --git a/src/java/org/apache/cassandra/utils/CLibrary.java 
b/src/java/org/apache/cassandra/utils/CLibrary.java
index 1c771af..48cb67c 100644
--- a/src/java/org/apache/cassandra/utils/CLibrary.java
+++ b/src/java/org/apache/cassandra/utils/CLibrary.java
@@ -33,8 +33,8 @@ public final class CLibrary
 {
 private static final Logger logger = 
LoggerFactory.getLogger(CLibrary.class);
 
-private static final int MCL_CURRENT = 1;
-private static final int MCL_FUTURE = 2;
+private static final int MCL_CURRENT;
+private static final int MCL_FUTURE;
 
 private static final int ENOMEM = 12;
 
@@ -75,6 +75,30 @@ public final class CLibrary
 logger.warn("Obsolete version of JNA present; unable to register C 
library. Upgrade to JNA 3.2.7 or later");
 jnaAvailable = false;
 }
+
+if (System.getProperty("os.arch").toLowerCase().contains("ppc"))
+{
+if (System.getProperty("os.name").toLowerCase().contains("linux"))
+{
+   MCL_CURRENT = 0x2000;
+   MCL_FUTURE = 0x4000;
+}
+else if 
(System.getProperty("os.name").toLowerCase().contains("aix"))
+{
+MCL_CURRENT = 0x100;
+MCL_FUTURE = 0x200;
+}
+else
+{
+MCL_CURRENT = 1;
+MCL_FUTURE = 2;
+}
+}
+else
+{
+MCL_CURRENT = 1;
+MCL_FUTURE = 2;
+}
 }
 
 private static native int mlockall(int flags) throws LastErrorException;

[08/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[jmckenzie]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/70115ff1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/70115ff1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/70115ff1

Branch: refs/heads/cassandra-3.0
Commit: 70115ff190e9d2d1187b328a8e24fc217ce9305a
Parents: 05bacc7 0d8e7b1
Author: Josh McKenzie 
Authored: Fri Jun 17 12:40:46 2016 -0400
Committer: Josh McKenzie 
Committed: Fri Jun 17 12:40:46 2016 -0400

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/utils/CLibrary.java| 28 ++--
 2 files changed, 27 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/70115ff1/CHANGES.txt
--
diff --cc CHANGES.txt
index 3eb420d,2c66ef9..4c0d9a0
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -79,11 -23,22 +79,12 @@@ Merged from 2.2
   * cqlsh: COPY FROM should use regular inserts for single statement batches 
and
 report errors correctly if workers processes crash on initialization 
(CASSANDRA-11474)
   * Always close cluster with connection in CqlRecordWriter (CASSANDRA-11553)
 - * Fix slice queries on ordered COMPACT tables (CASSANDRA-10988)
 + * Allow only DISTINCT queries with partition keys restrictions 
(CASSANDRA-11339)
 + * CqlConfigHelper no longer requires both a keystore and truststore to work 
(CASSANDRA-11532)
 + * Make deprecated repair methods backward-compatible with previous 
notification service (CASSANDRA-11430)
 + * IncomingStreamingConnection version check message wrong (CASSANDRA-11462)
  Merged from 2.1:
+  * Support mlockall on IBM POWER arch (CASSANDRA-11576)
 - * Cache local ranges when calculating repair neighbors (CASSANDRA-11933)
 - * Allow LWT operation on static column with only partition keys 
(CASSANDRA-10532)
 - * Create interval tree over canonical sstables to avoid missing sstables 
during streaming (CASSANDRA-11886)
 - * cqlsh COPY FROM: shutdown parent cluster after forking, to avoid 
corrupting SSL connections (CASSANDRA-11749)
 - * cqlsh: apply current keyspace to source command (CASSANDRA-11152)
 - * Backport CASSANDRA-11578 (CASSANDRA-11750)
 - * Clear out parent repair session if repair coordinator dies 
(CASSANDRA-11824)
 - * Set default streaming_socket_timeout_in_ms to 24 hours (CASSANDRA-11840)
 - * Do not consider local node a valid source during replace (CASSANDRA-11848)
 - * Avoid holding SSTableReaders for duration of incremental repair 
(CASSANDRA-11739)
 - * Add message dropped tasks to nodetool netstats (CASSANDRA-11855)
 - * Don't compute expensive MaxPurgeableTimestamp until we've verified there's 
an 
 -   expired tombstone (CASSANDRA-11834)
   * Add option to disable use of severity in DynamicEndpointSnitch 
(CASSANDRA-11737)
   * cqlsh COPY FROM fails for null values with non-prepared statements 
(CASSANDRA-11631)
   * Make cython optional in pylib/setup.py (CASSANDRA-11630)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/70115ff1/src/java/org/apache/cassandra/utils/CLibrary.java
--

[03/10] cassandra git commit: Set correct MCL_CURRENT and MCL_FUTURE values on IBM POWER to support mlockall(2) through JNA

[jmckenzie]

Set correct MCL_CURRENT and MCL_FUTURE values on IBM POWER to support 
mlockall(2) through JNA

Patch by rodaira; reviewed by jmckenzie for CASSANDRA-11576


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a56bc160
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a56bc160
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a56bc160

Branch: refs/heads/cassandra-3.0
Commit: a56bc1600bc8b98b9949751873e911ad5b237fbe
Parents: 615bf37
Author: Rei Odaira 
Authored: Fri Apr 15 10:50:45 2016 -0500
Committer: Josh McKenzie 
Committed: Fri Jun 17 12:38:25 2016 -0400

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/utils/CLibrary.java| 28 ++--
 2 files changed, 27 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a56bc160/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index ec2b48e..64cfdaa 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.15
+ * Support mlockall on IBM POWER arch (CASSANDRA-11576)
  * Cache local ranges when calculating repair neighbors (CASSANDRA-11933)
  * Allow LWT operation on static column with only partition keys 
(CASSANDRA-10532)
  * Create interval tree over canonical sstables to avoid missing sstables 
during streaming (CASSANDRA-11886)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a56bc160/src/java/org/apache/cassandra/utils/CLibrary.java
--
diff --git a/src/java/org/apache/cassandra/utils/CLibrary.java 
b/src/java/org/apache/cassandra/utils/CLibrary.java
index 1c771af..48cb67c 100644
--- a/src/java/org/apache/cassandra/utils/CLibrary.java
+++ b/src/java/org/apache/cassandra/utils/CLibrary.java
@@ -33,8 +33,8 @@ public final class CLibrary
 {
 private static final Logger logger = 
LoggerFactory.getLogger(CLibrary.class);
 
-private static final int MCL_CURRENT = 1;
-private static final int MCL_FUTURE = 2;
+private static final int MCL_CURRENT;
+private static final int MCL_FUTURE;
 
 private static final int ENOMEM = 12;
 
@@ -75,6 +75,30 @@ public final class CLibrary
 logger.warn("Obsolete version of JNA present; unable to register C 
library. Upgrade to JNA 3.2.7 or later");
 jnaAvailable = false;
 }
+
+if (System.getProperty("os.arch").toLowerCase().contains("ppc"))
+{
+if (System.getProperty("os.name").toLowerCase().contains("linux"))
+{
+   MCL_CURRENT = 0x2000;
+   MCL_FUTURE = 0x4000;
+}
+else if 
(System.getProperty("os.name").toLowerCase().contains("aix"))
+{
+MCL_CURRENT = 0x100;
+MCL_FUTURE = 0x200;
+}
+else
+{
+MCL_CURRENT = 1;
+MCL_FUTURE = 2;
+}
+}
+else
+{
+MCL_CURRENT = 1;
+MCL_FUTURE = 2;
+}
 }
 
 private static native int mlockall(int flags) throws LastErrorException;

[10/10] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[jmckenzie]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0ff8ab0c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0ff8ab0c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0ff8ab0c

Branch: refs/heads/trunk
Commit: 0ff8ab0c2bf3983eec63e07daf96d32decf916cd
Parents: 057c329 70115ff
Author: Josh McKenzie 
Authored: Fri Jun 17 12:41:03 2016 -0400
Committer: Josh McKenzie 
Committed: Fri Jun 17 12:41:03 2016 -0400

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/utils/CLibrary.java| 28 ++--
 2 files changed, 27 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0ff8ab0c/CHANGES.txt
--

[07/10] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[jmckenzie]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0d8e7b18
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0d8e7b18
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0d8e7b18

Branch: refs/heads/trunk
Commit: 0d8e7b18e54a378deae93f361ecd8ed593e0729b
Parents: dbefa85 a56bc16
Author: Josh McKenzie 
Authored: Fri Jun 17 12:39:37 2016 -0400
Committer: Josh McKenzie 
Committed: Fri Jun 17 12:39:37 2016 -0400

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/utils/CLibrary.java| 28 ++--
 2 files changed, 27 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0d8e7b18/CHANGES.txt
--
diff --cc CHANGES.txt
index d9afaa3,64cfdaa..2c66ef9
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,30 -1,5 +1,31 @@@
 -2.1.15
 +2.2.7
 + * StorageService shutdown hook should use a volatile variable 
(CASSANDRA-11984)
 + * Persist local metadata earlier in startup sequence (CASSANDRA-11742)
 + * Run CommitLog tests with different compression settings (CASSANDRA-9039)
 + * cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664)
 + * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587)
 + * Fix possible race condition in CommitLog.recover (CASSANDRA-11743)
 + * Enable client encryption in sstableloader with cli options 
(CASSANDRA-11708)
 + * Possible memory leak in NIODataInputStream (CASSANDRA-11867)
 + * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669)
 + * Add seconds to cqlsh tracing session duration (CASSANDRA-11753)
 + * Prohibit Reverse Counter type as part of the PK (CASSANDRA-9395)
 + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626)
 + * Exit JVM if JMX server fails to startup (CASSANDRA-11540)
 + * Produce a heap dump when exiting on OOM (CASSANDRA-9861)
 + * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427)
 + * Restore ability to filter on clustering columns when using a 2i 
(CASSANDRA-11510)
 + * JSON datetime formatting needs timezone (CASSANDRA-11137)
 + * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502)
 + * Remove unnescessary file existence check during anticompaction 
(CASSANDRA-11660)
 + * Add missing files to debian packages (CASSANDRA-11642)
 + * Avoid calling Iterables::concat in loops during 
ModificationStatement::getFunctions (CASSANDRA-11621)
 + * cqlsh: COPY FROM should use regular inserts for single statement batches 
and
 +   report errors correctly if workers processes crash on initialization 
(CASSANDRA-11474)
 + * Always close cluster with connection in CqlRecordWriter (CASSANDRA-11553)
 + * Fix slice queries on ordered COMPACT tables (CASSANDRA-10988)
 +Merged from 2.1:
+  * Support mlockall on IBM POWER arch (CASSANDRA-11576)
   * Cache local ranges when calculating repair neighbors (CASSANDRA-11933)
   * Allow LWT operation on static column with only partition keys 
(CASSANDRA-10532)
   * Create interval tree over canonical sstables to avoid missing sstables 
during streaming (CASSANDRA-11886)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0d8e7b18/src/java/org/apache/cassandra/utils/CLibrary.java
--

[09/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[jmckenzie]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/70115ff1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/70115ff1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/70115ff1

Branch: refs/heads/trunk
Commit: 70115ff190e9d2d1187b328a8e24fc217ce9305a
Parents: 05bacc7 0d8e7b1
Author: Josh McKenzie 
Authored: Fri Jun 17 12:40:46 2016 -0400
Committer: Josh McKenzie 
Committed: Fri Jun 17 12:40:46 2016 -0400

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/utils/CLibrary.java| 28 ++--
 2 files changed, 27 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/70115ff1/CHANGES.txt
--
diff --cc CHANGES.txt
index 3eb420d,2c66ef9..4c0d9a0
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -79,11 -23,22 +79,12 @@@ Merged from 2.2
   * cqlsh: COPY FROM should use regular inserts for single statement batches 
and
 report errors correctly if workers processes crash on initialization 
(CASSANDRA-11474)
   * Always close cluster with connection in CqlRecordWriter (CASSANDRA-11553)
 - * Fix slice queries on ordered COMPACT tables (CASSANDRA-10988)
 + * Allow only DISTINCT queries with partition keys restrictions 
(CASSANDRA-11339)
 + * CqlConfigHelper no longer requires both a keystore and truststore to work 
(CASSANDRA-11532)
 + * Make deprecated repair methods backward-compatible with previous 
notification service (CASSANDRA-11430)
 + * IncomingStreamingConnection version check message wrong (CASSANDRA-11462)
  Merged from 2.1:
+  * Support mlockall on IBM POWER arch (CASSANDRA-11576)
 - * Cache local ranges when calculating repair neighbors (CASSANDRA-11933)
 - * Allow LWT operation on static column with only partition keys 
(CASSANDRA-10532)
 - * Create interval tree over canonical sstables to avoid missing sstables 
during streaming (CASSANDRA-11886)
 - * cqlsh COPY FROM: shutdown parent cluster after forking, to avoid 
corrupting SSL connections (CASSANDRA-11749)
 - * cqlsh: apply current keyspace to source command (CASSANDRA-11152)
 - * Backport CASSANDRA-11578 (CASSANDRA-11750)
 - * Clear out parent repair session if repair coordinator dies 
(CASSANDRA-11824)
 - * Set default streaming_socket_timeout_in_ms to 24 hours (CASSANDRA-11840)
 - * Do not consider local node a valid source during replace (CASSANDRA-11848)
 - * Avoid holding SSTableReaders for duration of incremental repair 
(CASSANDRA-11739)
 - * Add message dropped tasks to nodetool netstats (CASSANDRA-11855)
 - * Don't compute expensive MaxPurgeableTimestamp until we've verified there's 
an 
 -   expired tombstone (CASSANDRA-11834)
   * Add option to disable use of severity in DynamicEndpointSnitch 
(CASSANDRA-11737)
   * cqlsh COPY FROM fails for null values with non-prepared statements 
(CASSANDRA-11631)
   * Make cython optional in pylib/setup.py (CASSANDRA-11630)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/70115ff1/src/java/org/apache/cassandra/utils/CLibrary.java
--

[05/10] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[jmckenzie]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0d8e7b18
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0d8e7b18
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0d8e7b18

Branch: refs/heads/cassandra-3.0
Commit: 0d8e7b18e54a378deae93f361ecd8ed593e0729b
Parents: dbefa85 a56bc16
Author: Josh McKenzie 
Authored: Fri Jun 17 12:39:37 2016 -0400
Committer: Josh McKenzie 
Committed: Fri Jun 17 12:39:37 2016 -0400

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/utils/CLibrary.java| 28 ++--
 2 files changed, 27 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0d8e7b18/CHANGES.txt
--
diff --cc CHANGES.txt
index d9afaa3,64cfdaa..2c66ef9
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,30 -1,5 +1,31 @@@
 -2.1.15
 +2.2.7
 + * StorageService shutdown hook should use a volatile variable 
(CASSANDRA-11984)
 + * Persist local metadata earlier in startup sequence (CASSANDRA-11742)
 + * Run CommitLog tests with different compression settings (CASSANDRA-9039)
 + * cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664)
 + * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587)
 + * Fix possible race condition in CommitLog.recover (CASSANDRA-11743)
 + * Enable client encryption in sstableloader with cli options 
(CASSANDRA-11708)
 + * Possible memory leak in NIODataInputStream (CASSANDRA-11867)
 + * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669)
 + * Add seconds to cqlsh tracing session duration (CASSANDRA-11753)
 + * Prohibit Reverse Counter type as part of the PK (CASSANDRA-9395)
 + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626)
 + * Exit JVM if JMX server fails to startup (CASSANDRA-11540)
 + * Produce a heap dump when exiting on OOM (CASSANDRA-9861)
 + * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427)
 + * Restore ability to filter on clustering columns when using a 2i 
(CASSANDRA-11510)
 + * JSON datetime formatting needs timezone (CASSANDRA-11137)
 + * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502)
 + * Remove unnescessary file existence check during anticompaction 
(CASSANDRA-11660)
 + * Add missing files to debian packages (CASSANDRA-11642)
 + * Avoid calling Iterables::concat in loops during 
ModificationStatement::getFunctions (CASSANDRA-11621)
 + * cqlsh: COPY FROM should use regular inserts for single statement batches 
and
 +   report errors correctly if workers processes crash on initialization 
(CASSANDRA-11474)
 + * Always close cluster with connection in CqlRecordWriter (CASSANDRA-11553)
 + * Fix slice queries on ordered COMPACT tables (CASSANDRA-10988)
 +Merged from 2.1:
+  * Support mlockall on IBM POWER arch (CASSANDRA-11576)
   * Cache local ranges when calculating repair neighbors (CASSANDRA-11933)
   * Allow LWT operation on static column with only partition keys 
(CASSANDRA-10532)
   * Create interval tree over canonical sstables to avoid missing sstables 
during streaming (CASSANDRA-11886)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0d8e7b18/src/java/org/apache/cassandra/utils/CLibrary.java
--

[02/10] cassandra git commit: Set correct MCL_CURRENT and MCL_FUTURE values on IBM POWER to support mlockall(2) through JNA

[jmckenzie]

Set correct MCL_CURRENT and MCL_FUTURE values on IBM POWER to support 
mlockall(2) through JNA

Patch by rodaira; reviewed by jmckenzie for CASSANDRA-11576


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a56bc160
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a56bc160
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a56bc160

Branch: refs/heads/cassandra-2.2
Commit: a56bc1600bc8b98b9949751873e911ad5b237fbe
Parents: 615bf37
Author: Rei Odaira 
Authored: Fri Apr 15 10:50:45 2016 -0500
Committer: Josh McKenzie 
Committed: Fri Jun 17 12:38:25 2016 -0400

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/utils/CLibrary.java| 28 ++--
 2 files changed, 27 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a56bc160/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index ec2b48e..64cfdaa 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.15
+ * Support mlockall on IBM POWER arch (CASSANDRA-11576)
  * Cache local ranges when calculating repair neighbors (CASSANDRA-11933)
  * Allow LWT operation on static column with only partition keys 
(CASSANDRA-10532)
  * Create interval tree over canonical sstables to avoid missing sstables 
during streaming (CASSANDRA-11886)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/a56bc160/src/java/org/apache/cassandra/utils/CLibrary.java
--
diff --git a/src/java/org/apache/cassandra/utils/CLibrary.java 
b/src/java/org/apache/cassandra/utils/CLibrary.java
index 1c771af..48cb67c 100644
--- a/src/java/org/apache/cassandra/utils/CLibrary.java
+++ b/src/java/org/apache/cassandra/utils/CLibrary.java
@@ -33,8 +33,8 @@ public final class CLibrary
 {
 private static final Logger logger = 
LoggerFactory.getLogger(CLibrary.class);
 
-private static final int MCL_CURRENT = 1;
-private static final int MCL_FUTURE = 2;
+private static final int MCL_CURRENT;
+private static final int MCL_FUTURE;
 
 private static final int ENOMEM = 12;
 
@@ -75,6 +75,30 @@ public final class CLibrary
 logger.warn("Obsolete version of JNA present; unable to register C 
library. Upgrade to JNA 3.2.7 or later");
 jnaAvailable = false;
 }
+
+if (System.getProperty("os.arch").toLowerCase().contains("ppc"))
+{
+if (System.getProperty("os.name").toLowerCase().contains("linux"))
+{
+   MCL_CURRENT = 0x2000;
+   MCL_FUTURE = 0x4000;
+}
+else if 
(System.getProperty("os.name").toLowerCase().contains("aix"))
+{
+MCL_CURRENT = 0x100;
+MCL_FUTURE = 0x200;
+}
+else
+{
+MCL_CURRENT = 1;
+MCL_FUTURE = 2;
+}
+}
+else
+{
+MCL_CURRENT = 1;
+MCL_FUTURE = 2;
+}
 }
 
 private static native int mlockall(int flags) throws LastErrorException;

[jira] [Commented] (CASSANDRA-8700) replace the wiki with docs in the git repo

[Sam Tunnicliffe (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336400#comment-15336400
 ] 

Sam Tunnicliffe commented on CASSANDRA-8700:


Pull request for security: https://github.com/pcmanus/cassandra/pull/56

> replace the wiki with docs in the git repo
> --
>
> Key: CASSANDRA-8700
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8700
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation and Website
>Reporter: Jon Haddad
>Assignee: Sylvain Lebresne
>Priority: Blocker
> Fix For: 3.8
>
> Attachments: TombstonesAndGcGrace.md, bloom_filters.md, 
> compression.md, contributing.zip, getting_started.zip, hardware.md
>
>
> The wiki as it stands is pretty terrible.  It takes several minutes to apply 
> a single update, and as a result, it's almost never updated.  The information 
> there has very little context as to what version it applies to.  Most people 
> I've talked to that try to use the information they find there find it is 
> more confusing than helpful.
> I'd like to propose that instead of using the wiki, the doc directory in the 
> cassandra repo be used for docs (already used for CQL3 spec) in a format that 
> can be built to a variety of output formats like HTML / epub / etc.  I won't 
> start the bikeshedding on which markup format is preferable - but there are 
> several options that can work perfectly fine.  I've personally use sphinx w/ 
> restructured text, and markdown.  Both can build easily and as an added bonus 
> be pushed to readthedocs (or something similar) automatically.  For an 
> example, see cqlengine's documentation, which I think is already 
> significantly better than the wiki: 
> http://cqlengine.readthedocs.org/en/latest/
> In addition to being overall easier to maintain, putting the documentation in 
> the git repo adds context, since it evolves with the versions of Cassandra.
> If the wiki were kept even remotely up to date, I wouldn't bother with this, 
> but not having at least some basic documentation in the repo, or anywhere 
> associated with the project, is frustrating.
> For reference, the last 3 updates were:
> 1/15/15 - updating committers list
> 1/08/15 - updating contributers and how to contribute
> 12/16/14 - added a link to CQL docs from wiki frontpage (by me)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12023) Schema upgrade bug with super columns

[Jeremiah Jordan (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336401#comment-15336401
 ] 

Jeremiah Jordan commented on CASSANDRA-12023:
-

You don't actually need the 2.0 to 2.1 upgrade in here.  Making the super 
column tables in 2.1 has the same problem when going to 3.0.

{code}
ccm create -n 1 -v 2.1.14 upgrade
ccm start
sleep 30
# run 2.0 stress that makes super columns
~/.ccm/repository/2.0.17/tools/bin/cassandra-stress
ccm stop
ccm node1 setdir --version 3.0.7
ccm start
sleep 30
tail -f -n 100 ~/.ccm/upgrade/node1/logs/system.log
{code}

> Schema upgrade bug with super columns
> -
>
> Key: CASSANDRA-12023
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12023
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Jeremiah Jordan
>Priority: Critical
> Fix For: 3.0.x, 3.x
>
>
> Doing some upgrade tests starting on 2.0 to 2.1 to 3.0 we hit the follow bug 
> that prevents 3.0 nodes from starting.  Running the test a few times with 
> different waits and flushing sometimes or not I have seen the following 
> errors:
> {code}
> ERROR [main] 2016-06-17 10:42:40,112 CassandraDaemon.java:698 - Exception 
> encountered during startup
> org.apache.cassandra.serializers.MarshalException: cannot parse 'value' as 
> hex bytes
>   at 
> org.apache.cassandra.db.marshal.BytesType.fromString(BytesType.java:45) 
> ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.createColumnFromColumnRow(LegacySchemaMigrator.java:682)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.createColumnsFromColumnRows(LegacySchemaMigrator.java:641)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.decodeTableMetadata(LegacySchemaMigrator.java:316)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.readTableMetadata(LegacySchemaMigrator.java:273)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.readTable(LegacySchemaMigrator.java:244)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.lambda$readTables$7(LegacySchemaMigrator.java:237)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at java.util.ArrayList.forEach(ArrayList.java:1249) ~[na:1.8.0_66]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.readTables(LegacySchemaMigrator.java:237)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.readKeyspace(LegacySchemaMigrator.java:186)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.lambda$readSchema$4(LegacySchemaMigrator.java:177)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at java.util.ArrayList.forEach(ArrayList.java:1249) ~[na:1.8.0_66]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.readSchema(LegacySchemaMigrator.java:177)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.migrate(LegacySchemaMigrator.java:77)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:229) 
> [apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:557)
>  [apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:685) 
> [apache-cassandra-3.0.7.jar:3.0.7]
> Caused by: java.lang.NumberFormatException: An hex string representing bytes 
> must have an even length
>   at org.apache.cassandra.utils.Hex.hexToBytes(Hex.java:57) 
> ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.db.marshal.BytesType.fromString(BytesType.java:41) 
> ~[apache-cassandra-3.0.7.jar:3.0.7]
>   ... 16 common frames omitted
> {code}
> {code}
> ERROR [main] 2016-06-17 10:49:21,326 CassandraDaemon.java:698 - Exception 
> encountered during startup
> java.lang.RuntimeException: org.codehaus.jackson.JsonParseException: 
> Unexpected character ('K' (code 75)): expected a valid value (number, String, 
> array, object, 'true', 'false' or 'null')
>  at [Source: java.io.StringReader@60d4475f; line: 1, column: 2]
>   at 
> org.apache.cassandra.utils.FBUtilities.fromJsonMap(FBUtilities.java:561) 
> ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.decodeTableParams(LegacySchemaMigrator.java:442)
>  ~[apache-cassandra-3.0.7.jar:3.0.7]
>   at 
> org.apache.cassandra.schema.LegacySchemaMigrator.decodeTableMetadata(LegacySchemaMigrator.java:365)
>  

[jira] [Created] (CASSANDRA-12025) dtest failure in paging_test.TestPagingData.test_paging_with_filtering_on_counter_columns

[Sean McCarthy (JIRA)]

Sean McCarthy created CASSANDRA-12025:
-

 Summary: dtest failure in 
paging_test.TestPagingData.test_paging_with_filtering_on_counter_columns
 Key: CASSANDRA-12025
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12025
 Project: Cassandra
  Issue Type: Test
Reporter: Sean McCarthy
Assignee: DS Test Eng
 Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log

example failure:

http://cassci.datastax.com/job/trunk_dtest/1276/testReport/paging_test/TestPagingData/test_paging_with_filtering_on_counter_columns

Failed on CassCI build trunk_dtest #1276

{code}
Error Message

Lists differ: [[4, 7, 8, 9], [4, 9, 10, 11]] != [[4, 7, 8, 9], [4, 8, 9, 10], 
...

First differing element 1:
[4, 9, 10, 11]
[4, 8, 9, 10]

Second list contains 1 additional elements.
First extra element 2:
[4, 9, 10, 11]

- [[4, 7, 8, 9], [4, 9, 10, 11]]
+ [[4, 7, 8, 9], [4, 8, 9, 10], [4, 9, 10, 11]]
?+++  
{code}
{code}
Stacktrace

  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
  File "/home/automaton/cassandra-dtest/tools.py", line 288, in wrapped
f(obj)
  File "/home/automaton/cassandra-dtest/paging_test.py", line 1148, in 
test_paging_with_filtering_on_counter_columns
self._test_paging_with_filtering_on_counter_columns(session, True)
  File "/home/automaton/cassandra-dtest/paging_test.py", line 1107, in 
_test_paging_with_filtering_on_counter_columns
[4, 9, 10, 11]])
  File "/usr/lib/python2.7/unittest/case.py", line 513, in assertEqual
assertion_func(first, second, msg=msg)
  File "/usr/lib/python2.7/unittest/case.py", line 742, in assertListEqual
self.assertSequenceEqual(list1, list2, msg, seq_type=list)
  File "/usr/lib/python2.7/unittest/case.py", line 724, in assertSequenceEqual
self.fail(msg)
  File "/usr/lib/python2.7/unittest/case.py", line 410, in fail
raise self.failureException(msg)
{code}
Logs are attached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12023) Schema upgrade bug with super columns

[Jeremiah Jordan (JIRA)]

Jeremiah Jordan created CASSANDRA-12023:
---

 Summary: Schema upgrade bug with super columns
 Key: CASSANDRA-12023
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12023
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jeremiah Jordan
Priority: Critical
 Fix For: 3.0.x, 3.x


Doing some upgrade tests starting on 2.0 to 2.1 to 3.0 we hit the follow bug 
that prevents 3.0 nodes from starting.  Running the test a few times with 
different waits and flushing sometimes or not I have seen the following errors:

{code}
ERROR [main] 2016-06-17 10:42:40,112 CassandraDaemon.java:698 - Exception 
encountered during startup
org.apache.cassandra.serializers.MarshalException: cannot parse 'value' as hex 
bytes
at 
org.apache.cassandra.db.marshal.BytesType.fromString(BytesType.java:45) 
~[apache-cassandra-3.0.7.jar:3.0.7]
at 
org.apache.cassandra.schema.LegacySchemaMigrator.createColumnFromColumnRow(LegacySchemaMigrator.java:682)
 ~[apache-cassandra-3.0.7.jar:3.0.7]
at 
org.apache.cassandra.schema.LegacySchemaMigrator.createColumnsFromColumnRows(LegacySchemaMigrator.java:641)
 ~[apache-cassandra-3.0.7.jar:3.0.7]
at 
org.apache.cassandra.schema.LegacySchemaMigrator.decodeTableMetadata(LegacySchemaMigrator.java:316)
 ~[apache-cassandra-3.0.7.jar:3.0.7]
at 
org.apache.cassandra.schema.LegacySchemaMigrator.readTableMetadata(LegacySchemaMigrator.java:273)
 ~[apache-cassandra-3.0.7.jar:3.0.7]
at 
org.apache.cassandra.schema.LegacySchemaMigrator.readTable(LegacySchemaMigrator.java:244)
 ~[apache-cassandra-3.0.7.jar:3.0.7]
at 
org.apache.cassandra.schema.LegacySchemaMigrator.lambda$readTables$7(LegacySchemaMigrator.java:237)
 ~[apache-cassandra-3.0.7.jar:3.0.7]
at java.util.ArrayList.forEach(ArrayList.java:1249) ~[na:1.8.0_66]
at 
org.apache.cassandra.schema.LegacySchemaMigrator.readTables(LegacySchemaMigrator.java:237)
 ~[apache-cassandra-3.0.7.jar:3.0.7]
at 
org.apache.cassandra.schema.LegacySchemaMigrator.readKeyspace(LegacySchemaMigrator.java:186)
 ~[apache-cassandra-3.0.7.jar:3.0.7]
at 
org.apache.cassandra.schema.LegacySchemaMigrator.lambda$readSchema$4(LegacySchemaMigrator.java:177)
 ~[apache-cassandra-3.0.7.jar:3.0.7]
at java.util.ArrayList.forEach(ArrayList.java:1249) ~[na:1.8.0_66]
at 
org.apache.cassandra.schema.LegacySchemaMigrator.readSchema(LegacySchemaMigrator.java:177)
 ~[apache-cassandra-3.0.7.jar:3.0.7]
at 
org.apache.cassandra.schema.LegacySchemaMigrator.migrate(LegacySchemaMigrator.java:77)
 ~[apache-cassandra-3.0.7.jar:3.0.7]
at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:229) 
[apache-cassandra-3.0.7.jar:3.0.7]
at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:557) 
[apache-cassandra-3.0.7.jar:3.0.7]
at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:685) 
[apache-cassandra-3.0.7.jar:3.0.7]
Caused by: java.lang.NumberFormatException: An hex string representing bytes 
must have an even length
at org.apache.cassandra.utils.Hex.hexToBytes(Hex.java:57) 
~[apache-cassandra-3.0.7.jar:3.0.7]
at 
org.apache.cassandra.db.marshal.BytesType.fromString(BytesType.java:41) 
~[apache-cassandra-3.0.7.jar:3.0.7]
... 16 common frames omitted
{code}

{code}
ERROR [main] 2016-06-17 10:49:21,326 CassandraDaemon.java:698 - Exception 
encountered during startup
java.lang.RuntimeException: org.codehaus.jackson.JsonParseException: Unexpected 
character ('K' (code 75)): expected a valid value (number, String, array, 
object, 'true', 'false' or 'null')
 at [Source: java.io.StringReader@60d4475f; line: 1, column: 2]
at 
org.apache.cassandra.utils.FBUtilities.fromJsonMap(FBUtilities.java:561) 
~[apache-cassandra-3.0.7.jar:3.0.7]
at 
org.apache.cassandra.schema.LegacySchemaMigrator.decodeTableParams(LegacySchemaMigrator.java:442)
 ~[apache-cassandra-3.0.7.jar:3.0.7]
at 
org.apache.cassandra.schema.LegacySchemaMigrator.decodeTableMetadata(LegacySchemaMigrator.java:365)
 ~[apache-cassandra-3.0.7.jar:3.0.7]
at 
org.apache.cassandra.schema.LegacySchemaMigrator.readTableMetadata(LegacySchemaMigrator.java:273)
 ~[apache-cassandra-3.0.7.jar:3.0.7]
at 
org.apache.cassandra.schema.LegacySchemaMigrator.readTable(LegacySchemaMigrator.java:244)
 ~[apache-cassandra-3.0.7.jar:3.0.7]
at 
org.apache.cassandra.schema.LegacySchemaMigrator.lambda$readTables$7(LegacySchemaMigrator.java:237)
 ~[apache-cassandra-3.0.7.jar:3.0.7]
at java.util.ArrayList.forEach(ArrayList.java:1249) ~[na:1.8.0_66]
at 
org.apache.cassandra.schema.LegacySchemaMigrator.readTables(LegacySchemaMigrator.java:237)
 ~[apache-cassandra-3.0.7.jar:3.0.7]
at 

[jira] [Commented] (CASSANDRA-11198) Materialized view inconsistency

[justin wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336354#comment-15336354
 ] 

justin wu commented on CASSANDRA-11198:
---

I also encountered this issue.  Any update plan?

> Materialized view inconsistency
> ---
>
> Key: CASSANDRA-11198
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11198
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Gábor Auth
>Assignee: Carl Yeksigian
> Attachments: CASSANDRA-11198.trace
>
>
> Here is a materialized view:
> {code}
> > DESCRIBE MATERIALIZED VIEW unit_by_transport ;
> CREATE MATERIALIZED VIEW unit_by_transport AS
> SELECT *
> FROM unit
> WHERE transportid IS NOT NULL AND type IS NOT NULL
> PRIMARY KEY (transportid, id)
> WITH CLUSTERING ORDER BY (id ASC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND comment = ''
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class': 
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> {code}
> I cannot reproduce this but sometimes and somehow happened the same issue 
> (https://issues.apache.org/jira/browse/CASSANDRA-10910):
> {code}
> > SELECT transportid, id, type FROM unit_by_transport WHERE 
> > transportid=24f90d20-d61f-11e5-9d3c-8fc3ad6906e2 and 
> > id=99c05a70-d686-11e5-a169-97287061d5d1;
>  transportid  | id   
> | type
> --+--+--
>  24f90d20-d61f-11e5-9d3c-8fc3ad6906e2 | 99c05a70-d686-11e5-a169-97287061d5d1 
> | null
> (1 rows)
> > SELECT transportid, id, type FROM unit WHERE 
> > id=99c05a70-d686-11e5-a169-97287061d5d1;
>  transportid | id | type
> -++--
> (0 rows)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12024) dtest failure in cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_copy_to_with_child_process_crashing

[Sean McCarthy (JIRA)]

Sean McCarthy created CASSANDRA-12024:
-

 Summary: dtest failure in 
cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_copy_to_with_child_process_crashing
 Key: CASSANDRA-12024
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12024
 Project: Cassandra
  Issue Type: Test
Reporter: Sean McCarthy
Assignee: DS Test Eng
 Attachments: node1.log

example failure:

http://cassci.datastax.com/job/cassandra-2.1_offheap_dtest/360/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_copy_to_with_child_process_crashing

Failed on CassCI build cassandra-2.1_offheap_dtest #360

{code}
Stacktrace

  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
  File "/home/automaton/cassandra-dtest/dtest.py", line 889, in wrapped
f(obj)
  File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", line 
2701, in test_copy_to_with_child_process_crashing
self.assertIn('some records might be missing', err)
  File "/usr/lib/python2.7/unittest/case.py", line 803, in assertIn
self.fail(self._formatMessage(msg, standardMsg))
  File "/usr/lib/python2.7/unittest/case.py", line 410, in fail
raise self.failureException(msg)

Error Message

'some records might be missing' not found in ''
{code}

Logs are attached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12022) dtest failure in repair_tests.repair_test.TestRepair.simple_repair_order_preserving_test

[Sean McCarthy (JIRA)]

Sean McCarthy created CASSANDRA-12022:
-

 Summary: dtest failure in 
repair_tests.repair_test.TestRepair.simple_repair_order_preserving_test
 Key: CASSANDRA-12022
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12022
 Project: Cassandra
  Issue Type: Test
Reporter: Sean McCarthy
Assignee: Sean McCarthy
Priority: Trivial


example failure:

http://cassci.datastax.com/job/cassandra-2.1_novnode_dtest/254/testReport/repair_tests.repair_test/TestRepair/simple_repair_order_preserving_test

Failed on CassCI build cassandra-2.1_novnode_dtest #254

Cluster doesn't exist when called.

{code}
File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
  File "/home/automaton/cassandra-dtest/repair_tests/repair_test.py", line 225, 
in simple_repair_order_preserving_test
self._simple_repair(order_preserving_partitioner=True)
  File "/home/automaton/cassandra-dtest/repair_tests/repair_test.py", line 273, 
in _simple_repair
cluster.set_partitioner('org.apache.cassandra.dht.ByteOrderedPartitioner')
"global name 'cluster' is not defined
{code}

The call to cluster is wrapped in an if clause, but isn't defined before that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Issue Comment Deleted] (CASSANDRA-11940) Look into better default file_cache_size for 2.2

[T Jake Luciani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-11940:
---
Comment: was deleted

(was: I think this is actually the same issue as CASSANDRA-8729

Recycling the Mapped byte buffer is causing read before write.)

> Look into better default file_cache_size for 2.2
> 
>
> Key: CASSANDRA-11940
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11940
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: T Jake Luciani
>Assignee: T Jake Luciani
> Fix For: 2.2.x
>
> Attachments: abnormal disk read throughput.png
>
>
> CASSANDRA-8464 added support for mmapped decompression where in version <= 
> 2.1 the reads were all decompressed in standard heap buffers.
> Since the usage of the file_cache is based solely on the buffer capacity we 
> should/can make this much larger in 2.2 when the disk access mode is mmap.  
> The downside of this cache being too small is made worse by 8464 since the 
> buffers are mmapped/unmapped causing explicit page faults.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12017) Allow configuration of inter DC compression

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-12017:

Fix Version/s: 3.x

> Allow configuration of inter DC compression 
> 
>
> Key: CASSANDRA-12017
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12017
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Thom Valley
> Fix For: 3.x
>
>
> With larger and more extensively geographically distributed clusters, users 
> are beginning to need the ability to reduce bandwidth consumption as much as 
> possible.
> With larger workloads, the limits of even large intercontinental data links 
> (55MBps is pretty typical) are beginning to be stretched.
> InterDC SSL is currently hard coded to use the fastest (not highest) 
> compression settings.  LZ4 is a great option, but being able to raise the 
> compression at the cost of some additional CPU may save as much as 10% 
> (perhaps slightly more depending on the data).  10% of a 55MBps link, if 
> running at or near capacity is substantial.
> This also has a large impact on the overhead and rate possible for 
> instantiating new DCs as well as rebuilding a DC after a failure.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9842) Inconsistent behavior for '= null' conditions on static columns

[Alex Petrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336284#comment-15336284
 ] 

Alex Petrov commented on CASSANDRA-9842:


As [CASSANDRA-10532] is merged now, I've updated the tests with conditional 
deletion (although only on {{NULL}} static columns), both with and without 
batch.

All branches are rebased:

|| ||2.1||2.2||3.0||trunk|
||code|[2.1|https://github.com/ifesdjeen/cassandra/tree/9842-2.1]|[2.2|https://github.com/ifesdjeen/cassandra/tree/9842-2.2]|[3.0|https://github.com/ifesdjeen/cassandra/tree/9842-3.0]|[trunk|https://github.com/ifesdjeen/cassandra/tree/9842-trunk]|
||utest|[2.1|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-9842-2.1-testall/]|[2.2|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-9842-2.2-testall/]|[3.0|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-9842-3.0-testall/]|[trunk|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-9842-trunk-testall/]|
||dtest|[2.1|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-9842-2.1-dtest/]|[2.2|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-9842-2.2-dtest/]|[3.0|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-9842-3.0-dtest/]|[trunk|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-9842-trunk-dtest/]|

The test failures seem unrelated..
Note for committer: tests commits for {{2.1}} and {{2.2}} are different from 
ones to {{3.0}} and {{trunk}} (there's a subtle difference because of value 
being returned in {{2.x}} together with {{applied}} column) , and change commit 
itself is same for {{2.1}} and {{2.2}}, not required for {{3.0}} and {{trunk}}. 

> Inconsistent behavior for '= null' conditions on static columns
> ---
>
> Key: CASSANDRA-9842
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9842
> Project: Cassandra
>  Issue Type: Bug
> Environment: cassandra-2.1.8 on Ubuntu 15.04
>Reporter: Chandra Sekar
>Assignee: Alex Petrov
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
>
> Both inserting a row (in a non-existent partition) and updating a static 
> column in the same LWT fails. Creating the partition before performing the 
> LWT works.
> h3. Table Definition
> {code}
> create table txtable(pcol bigint, ccol bigint, scol bigint static, ncol text, 
> primary key((pcol), ccol));
> {code}
> h3. Inserting row in non-existent partition and updating static column in one 
> LWT
> {code}
> begin batch
> insert into txtable (pcol, ccol, ncol) values (1, 1, 'A');
> update txtable set scol = 1 where pcol = 1 if scol = null;
> apply batch;
> [applied]
> ---
>  False
> {code}
> h3. Creating partition before LWT
> {code}
> insert into txtable (pcol, scol) values (1, null) if not exists;
> begin batch
> insert into txtable (pcol, ccol, ncol) values (1, 1, 'A');
> update txtable set scol = 1 where pcol = 1 if scol = null;
> apply batch;
> [applied]
> ---
>  True
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11960) Hints are not seekable

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11960:

Reviewer: Branimir Lambov

> Hints are not seekable
> --
>
> Key: CASSANDRA-11960
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11960
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Robert Stupp
>Assignee: Stefan Podkowinski
>
> Got the following error message on trunk. No idea how to reproduce. But the 
> only thing the (not overridden) seek method does is throwing this exception.
> {code}
> ERROR [HintsDispatcher:2] 2016-06-05 18:51:09,397 CassandraDaemon.java:222 - 
> Exception in thread Thread[HintsDispatcher:2,1,main]
> java.lang.UnsupportedOperationException: Hints are not seekable.
>   at org.apache.cassandra.hints.HintsReader.seek(HintsReader.java:114) 
> ~[main/:na]
>   at 
> org.apache.cassandra.hints.HintsDispatcher.seek(HintsDispatcher.java:79) 
> ~[main/:na]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.deliver(HintsDispatchExecutor.java:257)
>  ~[main/:na]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:242)
>  ~[main/:na]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:220)
>  ~[main/:na]
>   at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:199)
>  ~[main/:na]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_91]
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_91]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_91]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_91]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11327) Maintain a histogram of times when writes are blocked due to no available memory

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336276#comment-15336276
 ] 

Joshua McKenzie commented on CASSANDRA-11327:
-

[~cnlwsu]: you good taking review on this?

> Maintain a histogram of times when writes are blocked due to no available 
> memory
> 
>
> Key: CASSANDRA-11327
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11327
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Core
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>
> I have a theory that part of the reason C* is so sensitive to timeouts during 
> saturating write load is that throughput is basically a sawtooth with valleys 
> at zero. This is something I have observed and it gets worse as you add 2i to 
> a table or do anything that decreases the throughput of flushing.
> I think the fix for this is to incrementally release memory pinned by 
> memtables and 2i during flushing instead of releasing it all at once. I know 
> that's not really possible, but we can fake it with memory accounting that 
> tracks how close to completion flushing is and releases permits for 
> additional memory. This will lead to a bit of a sawtooth in real memory 
> usage, but we can account for that so the peak footprint is the same.
> I think the end result of this change will be a sawtooth, but the valley of 
> the sawtooth will not be zero it will be the rate at which flushing 
> progresses. Optimizing the rate at which flushing progresses and it's 
> fairness with other work can then be tackled separately.
> Before we do this I think we should demonstrate that pinned memory due to 
> flushing is actually the issue by getting better visibility into the 
> distribution of instances of not having any memory by maintaining a histogram 
> of spans of time where no memory is available and a thread is blocked.
> [MemtableAllocatr$SubPool.allocate(long)|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/utils/memory/MemtableAllocator.java#L186]
>  should be a relatively straightforward entry point for this. The first 
> thread to block can mark the start of memory starvation and the last thread 
> out can mark the end. Have a periodic task that tracks the amount of time 
> spent blocked per interval of time and if it is greater than some threshold 
> log with more details, possibly at debug.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11755) nodetool info should run with "readonly" jmx access

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-11755:

Reproduced In: 3.5, 2.1.10  (was: 2.1.10, 3.5)
 Reviewer: Paulo Motta

> nodetool info should run with "readonly" jmx access
> ---
>
> Key: CASSANDRA-11755
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11755
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Jérôme Mainaud
>Priority: Minor
>  Labels: security
> Fix For: 2.1.14
>
> Attachments: 11755-2.1.patch, 
> nodetool-info-exception-when-readonly.txt
>
>
> nodetool info crash when granted with readonly jmx access
> In the example given in attachment, the jmxremote.access file gives readonly 
> access to the cassandra jmx role.
> When the role is granted to readwrite access, everything works.
> The main reason is that node datacenter and rack info are fetched by an 
> operation invocation instead of by an attribute read. The former one is not 
> allowed to the role with readonly access.
> This is a security concern because nodetool info could be called by a 
> monitoring agent (Nagios for instance) and enterprise policy often don't 
> allow these agents to connect to JMX with higher privileges than "readonly".



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12013) Optimize the isEOF() checking in RAR

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12013:

Reviewer: Stefania

> Optimize the isEOF() checking in RAR
> 
>
> Key: CASSANDRA-12013
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12013
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Dikang Gu
>Assignee: Dikang Gu
>Priority: Minor
>  Labels: compaction
> Fix For: 2.2.x
>
> Attachments: 0001-Optimize-the-isEOF-checking-in-RAR.patch, 
> Screenshot 2016-06-15 11.20.51.png
>
>
> When I profile compaction in 2.1, I find the isEOF() is an expensive 
> operation, and we do the check for each read in RAR, which I think is 
> unnecessary.
> I attach a patch to only check isEOF() when we need to rebuffer, and then I 
> do not see it from hotspot.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11988) NullPointerExpception when reading/compacting table

[Carl Yeksigian (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336253#comment-15336253
 ] 

Carl Yeksigian commented on CASSANDRA-11988:


[~bkowalczyyk]: It is actually the deletion of the static columns that is 
causing this issue, but I don't think there is presently any data corruption - 
we need a fix to be able to compact and read this data, but it shouldn't be 
changed on disk at this point (as we can't compact it away).

I'm working today on a fix.

> NullPointerExpception when reading/compacting table
> ---
>
> Key: CASSANDRA-11988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11988
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Nimi Wariboko Jr.
>Assignee: Carl Yeksigian
> Fix For: 3.6
>
>
> I have a table that suddenly refuses to be read or compacted. Issuing a read 
> on the table causes a NPE.
> On compaction, it returns the error
> {code}
> ERROR [CompactionExecutor:6] 2016-06-09 17:10:15,724 CassandraDaemon.java:213 
> - Exception in thread Thread[CompactionExecutor:6,1,main]
> java.lang.NullPointerException: null
>   at 
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:38)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:64)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:24)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:76)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:226)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:182)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:82)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:264)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_45]
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_45]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_45]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_45]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> {code}
> Schema:
> {code}
> CREATE TABLE cmpayments.report_payments (
> reportid timeuuid,
> userid timeuuid,
> adjustedearnings decimal,
> deleted set static,
> earnings map,
> gross map,
> organizationid text,
> payall timestamp static,
> status text,
> PRIMARY KEY (reportid, userid)
> ) WITH CLUSTERING ORDER BY (userid ASC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND comment = ''
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class': 
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-12021) Consider removing Util.toString

[Michael Mior (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Mior resolved CASSANDRA-12021.
--
Resolution: Invalid

Very sorry. My default project appears to have changed. This issue is not for 
Cassandra.

> Consider removing Util.toString
> ---
>
> Key: CASSANDRA-12021
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12021
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Michael Mior
>Priority: Minor
>
> {{Util.toString}} seems unnecessary in light of 
> {{com.google.common.base.Joiner}}. There are ~16 total usages of this across 
> the entire code base. One inside SubstitutionVisitor and the rest scattered 
> around the Mongo, Cassandra, and Elasticsearch adapters. If there are no 
> objections, I'll remove this and replace with {{Joiner}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12021) Consider removing Util.toString

[Michael Mior (JIRA)]

Michael Mior created CASSANDRA-12021:


 Summary: Consider removing Util.toString
 Key: CASSANDRA-12021
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12021
 Project: Cassandra
  Issue Type: Improvement
Reporter: Michael Mior
Priority: Minor


{{Util.toString}} seems unnecessary in light of 
{{com.google.common.base.Joiner}}. There are ~16 total usages of this across 
the entire code base. One inside SubstitutionVisitor and the rest scattered 
around the Mongo, Cassandra, and Elasticsearch adapters. If there are no 
objections, I'll remove this and replace with {{Joiner}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10857) Allow dropping COMPACT STORAGE flag from tables in 3.X

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336235#comment-15336235
 ] 

Sylvain Lebresne commented on CASSANDRA-10857:
--

bq. The grammar is updated to support empty strings for identifiers, too.

Note that if we do that (and I certainly wouldn't mind a better solution cause 
that's still ugly at the end of the day), I suggest still rejecting it when 
declaring columns/field (for table, MVs, UDT, function parameters, ...). We 
only want to solve our problem of selecting/inserting into that server-side 
defined column, but I doubt allowing user to create column with empty names is 
a good idea.

> Allow dropping COMPACT STORAGE flag from tables in 3.X
> --
>
> Key: CASSANDRA-10857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10857
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL, Distributed Metadata
>Reporter: Aleksey Yeschenko
>Assignee: Alex Petrov
> Fix For: 3.x
>
>
> Thrift allows users to define flexible mixed column families - where certain 
> columns would have explicitly pre-defined names, potentially non-default 
> validation types, and be indexed.
> Example:
> {code}
> create column family foo
> and default_validation_class = UTF8Type
> and column_metadata = [
> {column_name: bar, validation_class: Int32Type, index_type: KEYS},
> {column_name: baz, validation_class: UUIDType, index_type: KEYS}
> ];
> {code}
> Columns named {{bar}} and {{baz}} will be validated as {{Int32Type}} and 
> {{UUIDType}}, respectively, and be indexed. Columns with any other name will 
> be validated by {{UTF8Type}} and will not be indexed.
> With CASSANDRA-8099, {{bar}} and {{baz}} would be mapped to static columns 
> internally. However, being {{WITH COMPACT STORAGE}}, the table will only 
> expose {{bar}} and {{baz}} columns. Accessing any dynamic columns (any column 
> not named {{bar}} and {{baz}}) right now requires going through Thrift.
> This is blocking Thrift -> CQL migration for users who have mixed 
> dynamic/static column families. That said, it *shouldn't* be hard to allow 
> users to drop the {{compact}} flag to expose the table as it is internally 
> now, and be able to access all columns.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11873) Add duration type

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336215#comment-15336215
 ] 

Sylvain Lebresne commented on CASSANDRA-11873:
--

bq. I understand the rational of the suggestions but I would be in favor of 
keeping what I proposed.

I concur (and will note that we're really just following the [internal standard 
for those unit|https://en.wikipedia.org/wiki/Second#SI_multiples], which 
weigths more in my book than any particular database choices).

That said, I do agree that making the unit case-insensitive would be nice 
(because most stuffs in CQL as case-insensitive, and because having the 
difference been only base on the case is error prone). Which is why I would 
personally suggest using {{mo}} for months (I guess you could alternatively 
keep {{m}} for months and force {{min}} for {{minutes}}, but I think it's way 
more common to abbreviate minutes than month (in life in general) so I think 
using something slightly irregular for month is less surprising).

> Add duration type
> -
>
> Key: CASSANDRA-11873
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11873
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>  Labels: client-impacting, doc-impacting
> Fix For: 3.x
>
>
> For CASSANDRA-11871 or to allow queries with {{WHERE}} clause like:
> {{... WHERE reading_time < now() - 2h}}, we need to support some duration 
> type.
> In my opinion, it should be represented internally as a number of 
> microseconds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11940) Look into better default file_cache_size for 2.2

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336209#comment-15336209
 ] 

T Jake Luciani commented on CASSANDRA-11940:


I think this is actually the same issue as CASSANDRA-8729

Recycling the Mapped byte buffer is causing read before write.

> Look into better default file_cache_size for 2.2
> 
>
> Key: CASSANDRA-11940
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11940
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: T Jake Luciani
>Assignee: T Jake Luciani
> Fix For: 2.2.x
>
> Attachments: abnormal disk read throughput.png
>
>
> CASSANDRA-8464 added support for mmapped decompression where in version <= 
> 2.1 the reads were all decompressed in standard heap buffers.
> Since the usage of the file_cache is based solely on the buffer capacity we 
> should/can make this much larger in 2.2 when the disk access mode is mmap.  
> The downside of this cache being too small is made worse by 8464 since the 
> buffers are mmapped/unmapped causing explicit page faults.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10857) Allow dropping COMPACT STORAGE flag from tables in 3.X

[Alex Petrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15335953#comment-15335953
 ] 

Alex Petrov edited comment on CASSANDRA-10857 at 6/17/16 2:14 PM:
--

bq. {{ALTER TABLE foo DROP COMPACT STORAGE}} or something along that line, 
rather than "random" table property. It's of course mainly a minor syntactic 
detail.

I'll add the {{DROP COMPACT STORAGE}} instead of the flag (it was the quickest 
way plus the initial patch as actually worked through 

And will add the expanded comments to the {{NEWS.txt}} section. We can also add 
a special document as SASI does, where we explain thrift updates.

As regards (2), (3), (4) - I mostly wanted to pinpoint it. We may briefly 
mention it in case someone wonders what happens in this case when migrating 
from thrift or even just dropping {{COMPACT STORAGE}}. 
I agree that the default {{BytesType}} is not  problem. Might be worth 
mentioning it in docs and maybe add some notes about types.

bq. "" (though it's worth checking that such name is actually allowed in 
queries).

-Unfortunately, we are not allowing selecting such name at the moment. When try 
try type cast, the empty string literal will appear in result instead of the 
actual column. We can add '' as a selector.- 
The grammar is updated to support empty strings for identifiers, too.


was (Author: ifesdjeen):
bq. {{ALTER TABLE foo DROP COMPACT STORAGE}} or something along that line, 
rather than "random" table property. It's of course mainly a minor syntactic 
detail.

I'll add the {{DROP COMPACT STORAGE}} instead of the flag (it was the quickest 
way plus the initial patch as actually worked through 

And will add the expanded comments to the {{NEWS.txt}} section. We can also add 
a special document as SASI does, where we explain thrift updates.

As regards (2), (3), (4) - I mostly wanted to pinpoint it. We may briefly 
mention it in case someone wonders what happens in this case when migrating 
from thrift or even just dropping {{COMPACT STORAGE}}. 
I agree that the default {{BytesType}} is not  problem. Might be worth 
mentioning it in docs and maybe add some notes about types.

bq. "" (though it's worth checking that such name is actually allowed in 
queries).

Unfortunately, we are not allowing selecting such name at the moment. When try 
try type cast, the empty string literal will appear in result instead of the 
actual column. We can add '' as a selector.

> Allow dropping COMPACT STORAGE flag from tables in 3.X
> --
>
> Key: CASSANDRA-10857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10857
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL, Distributed Metadata
>Reporter: Aleksey Yeschenko
>Assignee: Alex Petrov
> Fix For: 3.x
>
>
> Thrift allows users to define flexible mixed column families - where certain 
> columns would have explicitly pre-defined names, potentially non-default 
> validation types, and be indexed.
> Example:
> {code}
> create column family foo
> and default_validation_class = UTF8Type
> and column_metadata = [
> {column_name: bar, validation_class: Int32Type, index_type: KEYS},
> {column_name: baz, validation_class: UUIDType, index_type: KEYS}
> ];
> {code}
> Columns named {{bar}} and {{baz}} will be validated as {{Int32Type}} and 
> {{UUIDType}}, respectively, and be indexed. Columns with any other name will 
> be validated by {{UTF8Type}} and will not be indexed.
> With CASSANDRA-8099, {{bar}} and {{baz}} would be mapped to static columns 
> internally. However, being {{WITH COMPACT STORAGE}}, the table will only 
> expose {{bar}} and {{baz}} columns. Accessing any dynamic columns (any column 
> not named {{bar}} and {{baz}}) right now requires going through Thrift.
> This is blocking Thrift -> CQL migration for users who have mixed 
> dynamic/static column families. That said, it *shouldn't* be hard to allow 
> users to drop the {{compact}} flag to expose the table as it is internally 
> now, and be able to access all columns.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11873) Add duration type

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336117#comment-15336117
 ] 

Benjamin Lerer commented on CASSANDRA-11873:


bq. What CQL type has nanosecond resolution?

{{Time}} has nanosecond resolution. By consequence I strongly believe that we 
need microseconds and nanoseconds support.

{quote}
1. Change "us" to "u" to be consistent with InfluxDB, and to keep the 
abbreviations consistently to 1 letter.
2. Change the "ns" to "n" to keep all the abbreviations consistently to one 
letter. 
{quote}

I think that looking at what other are doing is a good thing but I do not think 
that we should bother following exactly what they are doing if they are the 
only one doing it.
In my opinion, we should keep {{ms}} for milliseconds as we will have to find 
something less natural otherwise. In this case it seems to me that we can also 
keep {{us}} and {{ns}}.

I understand the rational of the suggestions but I would be in favor of keeping 
what I proposed.


> Add duration type
> -
>
> Key: CASSANDRA-11873
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11873
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>  Labels: client-impacting, doc-impacting
> Fix For: 3.x
>
>
> For CASSANDRA-11871 or to allow queries with {{WHERE}} clause like:
> {{... WHERE reading_time < now() - 2h}}, we need to support some duration 
> type.
> In my opinion, it should be represented internally as a number of 
> microseconds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-12012) CQLSSTableWriter and composite clustering keys trigger NPE

[Alex Petrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336113#comment-15336113
 ] 

Alex Petrov edited comment on CASSANDRA-12012 at 6/17/16 1:46 PM:
--

[~pierz] I was working on some related issue (unset values), thought there 
might be some intersection.
It seems that in provided code keyspace is not initialised and the 
{{DatabaseDescriptor#getFlushWriters}} throws NPE. Maybe 
[this|https://github.com/apache/cassandra/blob/cassandra-3.0/test/unit/org/apache/cassandra/io/sstable/CQLSSTableWriterTest.java#L54-L56]
 will help you (in test, the server is initialised prior to creating the 
writer).


was (Author: ifesdjeen):
[~pierz] I was working on some related issue (unset values), thought there 
might be some intersection.
It seems that in provided code keyspace is not initialised and the 
{{DatabaseDescriptor#getFlushWriters}} throws NPE. Maybe 
[this|https://github.com/apache/cassandra/blob/cassandra-3.0/test/unit/org/apache/cassandra/io/sstable/CQLSSTableWriterTest.java#L54-L56]
 will help you.

> CQLSSTableWriter and composite clustering keys trigger NPE
> --
>
> Key: CASSANDRA-12012
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12012
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Pierre N.
>Assignee: Mahdi Mohammadi
>
> It triggers when using multiple clustering keys in the primary keys
> {code}
> package tests;
> import java.io.File;
> import org.apache.cassandra.io.sstable.CQLSSTableWriter;
> import org.apache.cassandra.config.Config;
> public class DefaultWriter {
> 
> public static void main(String[] args) throws Exception {
> Config.setClientMode(true);
> 
> String createTableQuery = "CREATE TABLE ks_test.table_test ("
> + "pk1 int,"
> + "ck1 int,"
> + "ck2 int,"
> + "PRIMARY KEY ((pk1), ck1, ck2)"
> + ");";
> String insertQuery = "INSERT INTO ks_test.table_test(pk1, ck1, ck2) 
> VALUES(?,?,?)";
> 
> CQLSSTableWriter writer = CQLSSTableWriter.builder()
> .inDirectory(Files.createTempDirectory("sst").toFile())
> .forTable(createTableQuery)
> .using(insertQuery)
> .build();
> writer.close();
> }
> }
> {code}
> Exception : 
> {code}
> Exception in thread "main" java.lang.ExceptionInInitializerError
>   at org.apache.cassandra.db.Keyspace.initCf(Keyspace.java:368)
>   at org.apache.cassandra.db.Keyspace.(Keyspace.java:305)
>   at org.apache.cassandra.db.Keyspace.open(Keyspace.java:129)
>   at org.apache.cassandra.db.Keyspace.open(Keyspace.java:106)
>   at org.apache.cassandra.db.Keyspace.openAndGetStore(Keyspace.java:159)
>   at 
> org.apache.cassandra.cql3.restrictions.PrimaryKeyRestrictionSet.hasSupportingIndex(PrimaryKeyRestrictionSet.java:156)
>   at 
> org.apache.cassandra.cql3.restrictions.PrimaryKeyRestrictionSet.(PrimaryKeyRestrictionSet.java:118)
>   at 
> org.apache.cassandra.cql3.restrictions.PrimaryKeyRestrictionSet.mergeWith(PrimaryKeyRestrictionSet.java:213)
>   at 
> org.apache.cassandra.cql3.restrictions.StatementRestrictions.addSingleColumnRestriction(StatementRestrictions.java:266)
>   at 
> org.apache.cassandra.cql3.restrictions.StatementRestrictions.addRestriction(StatementRestrictions.java:250)
>   at 
> org.apache.cassandra.cql3.restrictions.StatementRestrictions.(StatementRestrictions.java:159)
>   at 
> org.apache.cassandra.cql3.statements.UpdateStatement$ParsedInsert.prepareInternal(UpdateStatement.java:183)
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement$Parsed.prepare(ModificationStatement.java:782)
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement$Parsed.prepare(ModificationStatement.java:768)
>   at 
> org.apache.cassandra.cql3.QueryProcessor.getStatement(QueryProcessor.java:505)
>   at 
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.getStatement(CQLSSTableWriter.java:508)
>   at 
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.using(CQLSSTableWriter.java:439)
>   at tests.DefaultWriter.main(DefaultWriter.java:29)
> Caused by: java.lang.NullPointerException
>   at 
> org.apache.cassandra.config.DatabaseDescriptor.getFlushWriters(DatabaseDescriptor.java:1188)
>   at 
> org.apache.cassandra.db.ColumnFamilyStore.(ColumnFamilyStore.java:127)
>   ... 18 more
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12012) CQLSSTableWriter and composite clustering keys trigger NPE

[Alex Petrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336113#comment-15336113
 ] 

Alex Petrov commented on CASSANDRA-12012:
-

[~pierz] I was working on some related issue (unset values), thought there 
might be some intersection.
It seems that in provided code keyspace is not initialised and the 
{{DatabaseDescriptor#getFlushWriters}} throws NPE. Maybe 
[this|https://github.com/apache/cassandra/blob/cassandra-3.0/test/unit/org/apache/cassandra/io/sstable/CQLSSTableWriterTest.java#L54-L56]
 will help you.

> CQLSSTableWriter and composite clustering keys trigger NPE
> --
>
> Key: CASSANDRA-12012
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12012
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Pierre N.
>Assignee: Mahdi Mohammadi
>
> It triggers when using multiple clustering keys in the primary keys
> {code}
> package tests;
> import java.io.File;
> import org.apache.cassandra.io.sstable.CQLSSTableWriter;
> import org.apache.cassandra.config.Config;
> public class DefaultWriter {
> 
> public static void main(String[] args) throws Exception {
> Config.setClientMode(true);
> 
> String createTableQuery = "CREATE TABLE ks_test.table_test ("
> + "pk1 int,"
> + "ck1 int,"
> + "ck2 int,"
> + "PRIMARY KEY ((pk1), ck1, ck2)"
> + ");";
> String insertQuery = "INSERT INTO ks_test.table_test(pk1, ck1, ck2) 
> VALUES(?,?,?)";
> 
> CQLSSTableWriter writer = CQLSSTableWriter.builder()
> .inDirectory(Files.createTempDirectory("sst").toFile())
> .forTable(createTableQuery)
> .using(insertQuery)
> .build();
> writer.close();
> }
> }
> {code}
> Exception : 
> {code}
> Exception in thread "main" java.lang.ExceptionInInitializerError
>   at org.apache.cassandra.db.Keyspace.initCf(Keyspace.java:368)
>   at org.apache.cassandra.db.Keyspace.(Keyspace.java:305)
>   at org.apache.cassandra.db.Keyspace.open(Keyspace.java:129)
>   at org.apache.cassandra.db.Keyspace.open(Keyspace.java:106)
>   at org.apache.cassandra.db.Keyspace.openAndGetStore(Keyspace.java:159)
>   at 
> org.apache.cassandra.cql3.restrictions.PrimaryKeyRestrictionSet.hasSupportingIndex(PrimaryKeyRestrictionSet.java:156)
>   at 
> org.apache.cassandra.cql3.restrictions.PrimaryKeyRestrictionSet.(PrimaryKeyRestrictionSet.java:118)
>   at 
> org.apache.cassandra.cql3.restrictions.PrimaryKeyRestrictionSet.mergeWith(PrimaryKeyRestrictionSet.java:213)
>   at 
> org.apache.cassandra.cql3.restrictions.StatementRestrictions.addSingleColumnRestriction(StatementRestrictions.java:266)
>   at 
> org.apache.cassandra.cql3.restrictions.StatementRestrictions.addRestriction(StatementRestrictions.java:250)
>   at 
> org.apache.cassandra.cql3.restrictions.StatementRestrictions.(StatementRestrictions.java:159)
>   at 
> org.apache.cassandra.cql3.statements.UpdateStatement$ParsedInsert.prepareInternal(UpdateStatement.java:183)
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement$Parsed.prepare(ModificationStatement.java:782)
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement$Parsed.prepare(ModificationStatement.java:768)
>   at 
> org.apache.cassandra.cql3.QueryProcessor.getStatement(QueryProcessor.java:505)
>   at 
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.getStatement(CQLSSTableWriter.java:508)
>   at 
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.using(CQLSSTableWriter.java:439)
>   at tests.DefaultWriter.main(DefaultWriter.java:29)
> Caused by: java.lang.NullPointerException
>   at 
> org.apache.cassandra.config.DatabaseDescriptor.getFlushWriters(DatabaseDescriptor.java:1188)
>   at 
> org.apache.cassandra.db.ColumnFamilyStore.(ColumnFamilyStore.java:127)
>   ... 18 more
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-12020) sstablemetadata and sstabledump need better testing

[Chris Lohfink (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Lohfink reassigned CASSANDRA-12020:
-

Assignee: Chris Lohfink  (was: DS Test Eng)

> sstablemetadata and sstabledump need better testing
> ---
>
> Key: CASSANDRA-12020
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12020
> Project: Cassandra
>  Issue Type: Test
>  Components: Tools
>Reporter: Stefania
>Assignee: Chris Lohfink
>
> There is one dtest for sstabledump but it doesn't cover sstables with a local 
> partitioner, which is why a user reported CASSANDRA-12002 on the mailing 
> list; sstablemetadata has no tests at all, it is only used to check the 
> repair status by repair tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-12015) Rebuilding from another DC should use different sources

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336013#comment-15336013
 ] 

Paulo Motta edited comment on CASSANDRA-12015 at 6/17/16 1:11 PM:
--

well, in this case I think the easiest way is to add a new 
{{IEndpointSnitch.sortByRackDc}} method that has a default implementation that 
doesn't do any sorting, and is only overridden by 
{{AbstractNetworkTopologySnitch}}.


was (Author: pauloricardomg):
well, in this case I think the easiest way is to add a new 
{{IEndpointSnitch.sortByRackDc}} method that has a default implementation that 
doesn't do any sorting, and is only overridden by {{NetworkTopologyStrategy}}.

> Rebuilding from another DC should use different sources
> ---
>
> Key: CASSANDRA-12015
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12015
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Fabien Rousseau
>
> Currently, when adding a new DC (ex: DC2) and rebuilding it from an existing 
> DC (ex: DC1), only the closest replica is used as a "source of data".
> It works but is not optimal, because in case of an RF=3 and 3 nodes cluster, 
> only one node in DC1 is streaming the data to DC2. 
> To build the new DC in a reasonable time, it would be better, in that case, 
> to stream from multiple sources, thus distributing more evenly the load.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12015) Rebuilding from another DC should use different sources

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336013#comment-15336013
 ] 

Paulo Motta commented on CASSANDRA-12015:
-

well, in this case I think the easiest way is to add a new 
{{IEndpointSnitch.sortByRackDc}} method that has a default implementation that 
doesn't do any sorting, and is only overridden by {{NetworkTopologyStrategy}}.

> Rebuilding from another DC should use different sources
> ---
>
> Key: CASSANDRA-12015
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12015
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Fabien Rousseau
>
> Currently, when adding a new DC (ex: DC2) and rebuilding it from an existing 
> DC (ex: DC1), only the closest replica is used as a "source of data".
> It works but is not optimal, because in case of an RF=3 and 3 nodes cluster, 
> only one node in DC1 is streaming the data to DC2. 
> To build the new DC in a reasonable time, it would be better, in that case, 
> to stream from multiple sources, thus distributing more evenly the load.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12015) Rebuilding from another DC should use different sources

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336005#comment-15336005
 ] 

Paulo Motta commented on CASSANDRA-12015:
-

bq. How about just copying the sortByProximity impl from AbstractEndpointSnitch 
to RangeStreamer ?

you'd probably need to also copy the {{compareEndpoints}} from 
{{AbstractNetworkTopologySnitch}}, maybe abstract to a common static method so 
it can be reuse... and we'd also need to special case the {{SimpleSnitch}} case 
I guess. yes, definitely not a one-liner but should be feasible..

> Rebuilding from another DC should use different sources
> ---
>
> Key: CASSANDRA-12015
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12015
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Fabien Rousseau
>
> Currently, when adding a new DC (ex: DC2) and rebuilding it from an existing 
> DC (ex: DC1), only the closest replica is used as a "source of data".
> It works but is not optimal, because in case of an RF=3 and 3 nodes cluster, 
> only one node in DC1 is streaming the data to DC2. 
> To build the new DC in a reasonable time, it would be better, in that case, 
> to stream from multiple sources, thus distributing more evenly the load.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12015) Rebuilding from another DC should use different sources

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15335990#comment-15335990
 ] 

Paulo Motta commented on CASSANDRA-12015:
-

How about just copying the {{sortByProximity}} impl from 
{{AbstractEndpointSnitch}} to {{RangeStreamer}} ?

> Rebuilding from another DC should use different sources
> ---
>
> Key: CASSANDRA-12015
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12015
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Fabien Rousseau
>
> Currently, when adding a new DC (ex: DC2) and rebuilding it from an existing 
> DC (ex: DC1), only the closest replica is used as a "source of data".
> It works but is not optimal, because in case of an RF=3 and 3 nodes cluster, 
> only one node in DC1 is streaming the data to DC2. 
> To build the new DC in a reasonable time, it would be better, in that case, 
> to stream from multiple sources, thus distributing more evenly the load.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11873) Add duration type

[Brian Hess (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15335986#comment-15335986
 ] 

 Brian Hess commented on CASSANDRA-11873:
-

That largely covers my thoughts, with a couple things:
1. Change "us" to "u" to be consistent with InfluxDB, and to keep the 
abbreviations consistently to 1 letter.
2. Change the "ns" to "n" to keep all the abbreviations consistently to one 
letter. 
3. Change "y" to "Y" so year and month are both capital. I'm not sure exactly, 
but I would be inclined to changing "d" to "D" and "w" to "W" so units smaller 
than a day are lower-case and the others are upper-case, for user-friendliness. 

Two other suggestions/questions:
a) Is it possible to change Month or Minute to be something other than M/m so 
that we could make these units case insensitive? I don't have a great 
suggestion there, but I have no good suggestion there (maybe M for month and N 
for minute? Like I said, not great).
b) What CQL type has nanosecond resolution? Is there any way someone can do any 
arithmetic on a CQL type where it would do anything with "4ns"? now() is a 
timestamp which has millisecond resolution. So, I actually don't see the data 
type that has anything fine grained enough here. If that's the case we can get 
rid of "ns". I'm not sure what had microsecond resolution either now that I 
look at our docs, so perhaps the same for "u". 

> Add duration type
> -
>
> Key: CASSANDRA-11873
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11873
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>  Labels: client-impacting, doc-impacting
> Fix For: 3.x
>
>
> For CASSANDRA-11871 or to allow queries with {{WHERE}} clause like:
> {{... WHERE reading_time < now() - 2h}}, we need to support some duration 
> type.
> In my opinion, it should be represented internally as a number of 
> microseconds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-11920) bloom_filter_fp_chance needs to be validated up front

[Arindam Gupta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15335833#comment-15335833
 ] 

Arindam Gupta edited comment on CASSANDRA-11920 at 6/17/16 12:57 PM:
-

Attached a patch with the fix. I have not added any unit test as could not find 
corresponding unit test files for these classes, however tested it with cqlsh. 
Please review.


was (Author: arindamg):
attached the patch with the fix. I have not added any unit test as could not 
find corresponding unit test files for these classes, however tested it with 
cqlsh.

> bloom_filter_fp_chance needs to be validated up front
> -
>
> Key: CASSANDRA-11920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11920
> Project: Cassandra
>  Issue Type: Bug
>  Components: Lifecycle, Local Write-Read Paths
>Reporter: ADARSH KUMAR
>  Labels: lhf
> Attachments: 11920-3.0.txt
>
>
> Hi,
> I was doing some bench-marking on bloom_filter_fp_chance values. Everything 
> worked fine for values .01(default for STCS), .001, .0001. But when I set 
> bloom_filter_fp_chance = .1 i observed following behaviour:
> 1). Reads and writes looked normal from cqlsh.
> 2). SSttables are never created.
> 3). It just creates two files (*-Data.db and *-index.db) of size 0kb.
> 4). nodetool flush does not work and produce following exception:
> java.lang.UnsupportedOperationException: Unable to satisfy 1.0E-5 with 20 
> buckets per element
> at 
> org.apache.cassandra.utils.BloomCalculations.computeBloomSpec(BloomCalculations.java:150)
>  .
> I checked BloomCalculations class and following lines are responsible for 
> this exception:
> if (maxFalsePosProb < probs[maxBucketsPerElement][maxK]) {
>   throw new UnsupportedOperationException(String.format("Unable to 
> satisfy %s with %s buckets per element",
>  maxFalsePosProb, 
> maxBucketsPerElement));
>   }
> From  the code it looks like a hard coaded validation (unless we can change 
> the nuber of buckets).
> So, if this validation is hard coaded then why it is even allowed to set such 
> value of bloom_fileter_fp_chance, that can prevent ssTable generation?
> Please correct this issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Issue Comment Deleted] (CASSANDRA-11920) bloom_filter_fp_chance needs to be validated up front

[Arindam Gupta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arindam Gupta updated CASSANDRA-11920:
--
Comment: was deleted

(was: patch fix for 11920)

> bloom_filter_fp_chance needs to be validated up front
> -
>
> Key: CASSANDRA-11920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11920
> Project: Cassandra
>  Issue Type: Bug
>  Components: Lifecycle, Local Write-Read Paths
>Reporter: ADARSH KUMAR
>  Labels: lhf
> Attachments: 11920-3.0.txt
>
>
> Hi,
> I was doing some bench-marking on bloom_filter_fp_chance values. Everything 
> worked fine for values .01(default for STCS), .001, .0001. But when I set 
> bloom_filter_fp_chance = .1 i observed following behaviour:
> 1). Reads and writes looked normal from cqlsh.
> 2). SSttables are never created.
> 3). It just creates two files (*-Data.db and *-index.db) of size 0kb.
> 4). nodetool flush does not work and produce following exception:
> java.lang.UnsupportedOperationException: Unable to satisfy 1.0E-5 with 20 
> buckets per element
> at 
> org.apache.cassandra.utils.BloomCalculations.computeBloomSpec(BloomCalculations.java:150)
>  .
> I checked BloomCalculations class and following lines are responsible for 
> this exception:
> if (maxFalsePosProb < probs[maxBucketsPerElement][maxK]) {
>   throw new UnsupportedOperationException(String.format("Unable to 
> satisfy %s with %s buckets per element",
>  maxFalsePosProb, 
> maxBucketsPerElement));
>   }
> From  the code it looks like a hard coaded validation (unless we can change 
> the nuber of buckets).
> So, if this validation is hard coaded then why it is even allowed to set such 
> value of bloom_fileter_fp_chance, that can prevent ssTable generation?
> Please correct this issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Issue Comment Deleted] (CASSANDRA-11920) bloom_filter_fp_chance needs to be validated up front

[Arindam Gupta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arindam Gupta updated CASSANDRA-11920:
--
Comment: was deleted

(was: please review the patch.)

> bloom_filter_fp_chance needs to be validated up front
> -
>
> Key: CASSANDRA-11920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11920
> Project: Cassandra
>  Issue Type: Bug
>  Components: Lifecycle, Local Write-Read Paths
>Reporter: ADARSH KUMAR
>  Labels: lhf
> Attachments: 11920-3.0.txt
>
>
> Hi,
> I was doing some bench-marking on bloom_filter_fp_chance values. Everything 
> worked fine for values .01(default for STCS), .001, .0001. But when I set 
> bloom_filter_fp_chance = .1 i observed following behaviour:
> 1). Reads and writes looked normal from cqlsh.
> 2). SSttables are never created.
> 3). It just creates two files (*-Data.db and *-index.db) of size 0kb.
> 4). nodetool flush does not work and produce following exception:
> java.lang.UnsupportedOperationException: Unable to satisfy 1.0E-5 with 20 
> buckets per element
> at 
> org.apache.cassandra.utils.BloomCalculations.computeBloomSpec(BloomCalculations.java:150)
>  .
> I checked BloomCalculations class and following lines are responsible for 
> this exception:
> if (maxFalsePosProb < probs[maxBucketsPerElement][maxK]) {
>   throw new UnsupportedOperationException(String.format("Unable to 
> satisfy %s with %s buckets per element",
>  maxFalsePosProb, 
> maxBucketsPerElement));
>   }
> From  the code it looks like a hard coaded validation (unless we can change 
> the nuber of buckets).
> So, if this validation is hard coaded then why it is even allowed to set such 
> value of bloom_fileter_fp_chance, that can prevent ssTable generation?
> Please correct this issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12020) sstablemetadata and sstabledump need better testing

[Jim Witschey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Witschey updated CASSANDRA-12020:
-
Assignee: DS Test Eng

> sstablemetadata and sstabledump need better testing
> ---
>
> Key: CASSANDRA-12020
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12020
> Project: Cassandra
>  Issue Type: Test
>  Components: Tools
>Reporter: Stefania
>Assignee: DS Test Eng
>
> There is one dtest for sstabledump but it doesn't cover sstables with a local 
> partitioner, which is why a user reported CASSANDRA-12002 on the mailing 
> list; sstablemetadata has no tests at all, it is only used to check the 
> repair status by repair tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10857) Allow dropping COMPACT STORAGE flag from tables in 3.X

[Alex Petrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15335953#comment-15335953
 ] 

Alex Petrov commented on CASSANDRA-10857:
-

bq. {{ALTER TABLE foo DROP COMPACT STORAGE}} or something along that line, 
rather than "random" table property. It's of course mainly a minor syntactic 
detail.

I'll add the {{DROP COMPACT STORAGE}} instead of the flag (it was the quickest 
way plus the initial patch as actually worked through 

And will add the expanded comments to the {{NEWS.txt}} section. We can also add 
a special document as SASI does, where we explain thrift updates.

As regards (2), (3), (4) - I mostly wanted to pinpoint it. We may briefly 
mention it in case someone wonders what happens in this case when migrating 
from thrift or even just dropping {{COMPACT STORAGE}}. 
I agree that the default {{BytesType}} is not  problem. Might be worth 
mentioning it in docs and maybe add some notes about types.

bq. "" (though it's worth checking that such name is actually allowed in 
queries).

Unfortunately, we are not allowing selecting such name at the moment. When try 
try type cast, the empty string literal will appear in result instead of the 
actual column. We can add '' as a selector.

> Allow dropping COMPACT STORAGE flag from tables in 3.X
> --
>
> Key: CASSANDRA-10857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10857
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL, Distributed Metadata
>Reporter: Aleksey Yeschenko
>Assignee: Alex Petrov
> Fix For: 3.x
>
>
> Thrift allows users to define flexible mixed column families - where certain 
> columns would have explicitly pre-defined names, potentially non-default 
> validation types, and be indexed.
> Example:
> {code}
> create column family foo
> and default_validation_class = UTF8Type
> and column_metadata = [
> {column_name: bar, validation_class: Int32Type, index_type: KEYS},
> {column_name: baz, validation_class: UUIDType, index_type: KEYS}
> ];
> {code}
> Columns named {{bar}} and {{baz}} will be validated as {{Int32Type}} and 
> {{UUIDType}}, respectively, and be indexed. Columns with any other name will 
> be validated by {{UTF8Type}} and will not be indexed.
> With CASSANDRA-8099, {{bar}} and {{baz}} would be mapped to static columns 
> internally. However, being {{WITH COMPACT STORAGE}}, the table will only 
> expose {{bar}} and {{baz}} columns. Accessing any dynamic columns (any column 
> not named {{bar}} and {{baz}}) right now requires going through Thrift.
> This is blocking Thrift -> CQL migration for users who have mixed 
> dynamic/static column families. That said, it *shouldn't* be hard to allow 
> users to drop the {{compact}} flag to expose the table as it is internally 
> now, and be able to access all columns.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12015) Rebuilding from another DC should use different sources

[Fabien Rousseau (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15335942#comment-15335942
 ] 

Fabien Rousseau commented on CASSANDRA-12015:
-

Hum after looking at the code of 2.1 branch, I don't see how to bypass the 
dynamic snitch...

It is NOT possible to obtain the snitch used by the dynamic snitch (no accessor 
method), and looking at the implementation of 
DynamicSnitch.getSortedListByProximity() it just calls 
DynamicSnitch.sortByProximity(), so calling the latter method won't have any 
effect...

It seems to me that this won't be done by a simple "one-liner"

> Rebuilding from another DC should use different sources
> ---
>
> Key: CASSANDRA-12015
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12015
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Fabien Rousseau
>
> Currently, when adding a new DC (ex: DC2) and rebuilding it from an existing 
> DC (ex: DC1), only the closest replica is used as a "source of data".
> It works but is not optimal, because in case of an RF=3 and 3 nodes cluster, 
> only one node in DC1 is streaming the data to DC2. 
> To build the new DC in a reasonable time, it would be better, in that case, 
> to stream from multiple sources, thus distributing more evenly the load.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)