[jira] [Updated] (CASSANDRA-12490) Add sequence distribution type to cassandra stress

[Ben Slater (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ben Slater updated CASSANDRA-12490:
---
Status: Patch Available  (was: Reopened)

Patch attached.

> Add sequence distribution type to cassandra stress
> --
>
> Key: CASSANDRA-12490
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12490
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Ben Slater
>Assignee: Ben Slater
>Priority: Minor
> Fix For: 3.10
>
> Attachments: 12490-trunk.patch, 12490.yaml, 12490update-trunk.patch, 
> cqlstress-seq-example.yaml
>
>
> When using the write command, cassandra stress sequentially generates seeds. 
> This ensures generated values don't overlap (unless the sequence wraps) 
> providing more predictable number of inserted records (and generating a base 
> set of data without wasted writes).
> When using a yaml stress spec there is no sequenced distribution available. 
> It think it would be useful to have this for doing initial load of data for 
> testing 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12490) Add sequence distribution type to cassandra stress

[Ben Slater (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ben Slater updated CASSANDRA-12490:
---
Attachment: 12490update-trunk.patch

I've attached the patch to fix the implementation of setSeed() (update to 
current trunk rather than complete new patch). [~aboudreault] the same change 
also fixes the issue you mentioned where things did work properly with multiple 
threads.

> Add sequence distribution type to cassandra stress
> --
>
> Key: CASSANDRA-12490
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12490
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Ben Slater
>Assignee: Ben Slater
>Priority: Minor
> Fix For: 3.10
>
> Attachments: 12490-trunk.patch, 12490.yaml, 12490update-trunk.patch, 
> cqlstress-seq-example.yaml
>
>
> When using the write command, cassandra stress sequentially generates seeds. 
> This ensures generated values don't overlap (unless the sequence wraps) 
> providing more predictable number of inserted records (and generating a base 
> set of data without wasted writes).
> When using a yaml stress spec there is no sequenced distribution available. 
> It think it would be useful to have this for doing initial load of data for 
> testing 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-12777) Optimize the vnode allocation for single replica per DC

[Dikang Gu (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572934#comment-15572934
 ] 

Dikang Gu edited comment on CASSANDRA-12777 at 10/15/16 4:21 AM:
-

It's not perfect yet, but I'd like to send it out to let people take a look 
first: 

Currently it supports Murmur3Partitioner and RandomPartitioner. [~blambov] do 
you mind to take a look?

I will continue to clean it up.
Thanks!


was (Author: dikanggu):
It's not perfect yet, but I'd like to send it out to let people take a look 
first: 
https://github.com/DikangGu/cassandra/commit/3711f9c00f47aba768ceba70fd9bd54f1d64

Currently it supports Murmur3Partitioner and RandomPartitioner. [~blambov] do 
you mind to take a look?

I will continue to clean it up.
Thanks!

> Optimize the vnode allocation for single replica per DC
> ---
>
> Key: CASSANDRA-12777
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12777
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Dikang Gu
>Assignee: Dikang Gu
> Fix For: 3.x
>
>
> The new vnode allocation algorithm introduced in CASSANDRA-7032 is optimized 
> for the situation that there are multiple replicas per DC.
> In our production environment, most cluster only has one replica, in this 
> case, the algorithm does not work perfectly. It always tries to split token 
> ranges by half, so that the ownership of "min" node could go as low as ~60% 
> compared to avg.
> So for single replica case, I'm working on a new algorithm, which is based on 
> Branimir's previous commit, to split token ranges by "some" percentage, 
> instead of always by half. In this way, we can get a very small variation of 
> the ownership among different nodes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12777) Optimize the vnode allocation for single replica per DC

[Dikang Gu (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15577330#comment-15577330
 ] 

Dikang Gu commented on CASSANDRA-12777:
---

[~blambov] thanks a lot for the review! I addressed your comments and here is a 
new commit: 
https://github.com/DikangGu/cassandra/commit/402050e32732e67055935689951a56f92b9be281

1. wraparound calculation, for BigIntegerToken, if it's above maximum, I will 
`mod` the max token. I also add test and validation for it.
2. I put the createTokenInfo in constructor because I need to populate the unit 
info according to the tokens.
3. agree, removed the createTokenInfo
4. I tried different fractions, from 0.50 - 0.99, 0.50 will fail the assertion, 
otherwise, they do not make much difference. 
https://gist.github.com/DikangGu/acd8f568f67b11082443419a8d503b01, I put 0.75 
in this commit.
5. Add the `TokenAllocatorBase`, I keep the factory class because I think it's 
cleaner to keep the factory method there.

Thanks!

> Optimize the vnode allocation for single replica per DC
> ---
>
> Key: CASSANDRA-12777
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12777
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Dikang Gu
>Assignee: Dikang Gu
> Fix For: 3.x
>
>
> The new vnode allocation algorithm introduced in CASSANDRA-7032 is optimized 
> for the situation that there are multiple replicas per DC.
> In our production environment, most cluster only has one replica, in this 
> case, the algorithm does not work perfectly. It always tries to split token 
> ranges by half, so that the ownership of "min" node could go as low as ~60% 
> compared to avg.
> So for single replica case, I'm working on a new algorithm, which is based on 
> Branimir's previous commit, to split token ranges by "some" percentage, 
> instead of always by half. In this way, we can get a very small variation of 
> the ownership among different nodes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11873) Add duration type

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-11873:

Status: Ready to Commit  (was: Patch Available)

> Add duration type
> -
>
> Key: CASSANDRA-11873
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11873
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>  Labels: client-impacting, doc-impacting
> Fix For: 3.x
>
>
> For CASSANDRA-11871 or to allow queries with {{WHERE}} clause like:
> {{... WHERE reading_time < now() - 2h}}, we need to support some duration 
> type.
> In my opinion, it should be represented internally as a number of 
> microseconds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11873) Add duration type

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576856#comment-15576856
 ] 

Tyler Hobbs commented on CASSANDRA-11873:
-

+1 after the new changes, they look good.

bq. In my opinion having some checks will be probably less confusing for the 
user. With the current binary representation used for duration, the order will 
be completely random.

A completely random order is better than an almost-correct ordering, I think.  
In any case, it would be easy to go from the current restrictions to allowing 
durations to be used anywhere (but with random ordering) if we need to, so I 
think starting with this is fine.

bq. I did not tackle that part but I thought that there will be more work to do 
on the python driver side.

Ah, yes, you're correct.  We just need to make sure that driver work includes 
updating the reserved keywords, since that's easy to miss. (cc [~aholmber])

> Add duration type
> -
>
> Key: CASSANDRA-11873
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11873
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>  Labels: client-impacting, doc-impacting
> Fix For: 3.x
>
>
> For CASSANDRA-11871 or to allow queries with {{WHERE}} clause like:
> {{... WHERE reading_time < now() - 2h}}, we need to support some duration 
> type.
> In my opinion, it should be represented internally as a number of 
> microseconds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12728) Handling partially written hint files

[Garvit Juniwal (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576548#comment-15576548
 ] 

Garvit Juniwal commented on CASSANDRA-12728:


[~iamaleksey] Can you please take a look at the patch? Is that the correct fix? 
Here is my commit summary:

If a hints file gets truncated due to a crash, the last hint might have
been written incompletely. Presence of a corrupted hints file can put
cassandra in a crash loop trying o read a corrupted hint over and over.
Ignore and discard the last incomplete hint instead of throwing a
FSReadError exception.

> Handling partially written hint files
> -
>
> Key: CASSANDRA-12728
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12728
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sharvanath Pathak
>Assignee: Aleksey Yeschenko
> Attachments: CASSANDRA-12728.patch
>
>
> {noformat}
> ERROR [HintsDispatcher:1] 2016-09-28 17:44:43,397 
> HintsDispatchExecutor.java:225 - Failed to dispatch hints file 
> d5d7257c-9f81-49b2-8633-6f9bda6e3dea-1474892654160-1.hints: file is corrupted 
> ({})
> org.apache.cassandra.io.FSReadError: java.io.EOFException
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:282)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:252)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.sendHints(HintsDispatcher.java:156)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.sendHintsAndAwait(HintsDispatcher.java:137)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:119) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:91) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.deliver(HintsDispatchExecutor.java:259)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:242)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:220)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:199)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_77]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> [na:1.8.0_77]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_77]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_77]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
> Caused by: java.io.EOFException: null
> at 
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:68)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:60)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.ChecksummedDataInput.readFully(ChecksummedDataInput.java:126)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:402) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.readBuffer(HintsReader.java:310)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNextInternal(HintsReader.java:301)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:278)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> ... 15 common frames omitted
> {noformat}
> We've found out that the hint file was truncated because there was a hard 
> reboot around the time of last write to the file. I think we basically need 
> to handle partially written hint files. Also, the CRC file does not exist in 
> this case (probably because it crashed while writing the hints file). May be 
> ignoring and cleaning up such partially written hint files can be a way to 
> fix this?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-12728) Handling partially written hint files

[Garvit Juniwal (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576548#comment-15576548
 ] 

Garvit Juniwal edited comment on CASSANDRA-12728 at 10/14/16 9:30 PM:
--

[~iamaleksey] Can you please take a look at the patch? Is that the correct fix? 
Here is my commit summary:

If a hints file gets truncated due to a crash, the last hint might have
been written incompletely. Presence of a corrupted hints file can put
cassandra in a crash loop trying to read a corrupted hint over and over.
Ignore and discard the last incomplete hint instead of throwing a
FSReadError exception.


was (Author: garvitjuniwal):
[~iamaleksey] Can you please take a look at the patch? Is that the correct fix? 
Here is my commit summary:

If a hints file gets truncated due to a crash, the last hint might have
been written incompletely. Presence of a corrupted hints file can put
cassandra in a crash loop trying o read a corrupted hint over and over.
Ignore and discard the last incomplete hint instead of throwing a
FSReadError exception.

> Handling partially written hint files
> -
>
> Key: CASSANDRA-12728
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12728
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sharvanath Pathak
>Assignee: Aleksey Yeschenko
> Attachments: CASSANDRA-12728.patch
>
>
> {noformat}
> ERROR [HintsDispatcher:1] 2016-09-28 17:44:43,397 
> HintsDispatchExecutor.java:225 - Failed to dispatch hints file 
> d5d7257c-9f81-49b2-8633-6f9bda6e3dea-1474892654160-1.hints: file is corrupted 
> ({})
> org.apache.cassandra.io.FSReadError: java.io.EOFException
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:282)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:252)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.sendHints(HintsDispatcher.java:156)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.sendHintsAndAwait(HintsDispatcher.java:137)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:119) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:91) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.deliver(HintsDispatchExecutor.java:259)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:242)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:220)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:199)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_77]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> [na:1.8.0_77]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_77]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_77]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
> Caused by: java.io.EOFException: null
> at 
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:68)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:60)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.ChecksummedDataInput.readFully(ChecksummedDataInput.java:126)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:402) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.readBuffer(HintsReader.java:310)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNextInternal(HintsReader.java:301)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:278)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> ... 15 common frames 

[jira] [Updated] (CASSANDRA-12728) Handling partially written hint files

[Garvit Juniwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Garvit Juniwal updated CASSANDRA-12728:
---
 Reviewer: Aleksey Yeschenko
Reproduced In: 3.0.9
   Status: Patch Available  (was: Open)

> Handling partially written hint files
> -
>
> Key: CASSANDRA-12728
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12728
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sharvanath Pathak
>Assignee: Aleksey Yeschenko
> Attachments: CASSANDRA-12728.patch
>
>
> {noformat}
> ERROR [HintsDispatcher:1] 2016-09-28 17:44:43,397 
> HintsDispatchExecutor.java:225 - Failed to dispatch hints file 
> d5d7257c-9f81-49b2-8633-6f9bda6e3dea-1474892654160-1.hints: file is corrupted 
> ({})
> org.apache.cassandra.io.FSReadError: java.io.EOFException
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:282)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:252)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.sendHints(HintsDispatcher.java:156)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.sendHintsAndAwait(HintsDispatcher.java:137)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:119) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:91) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.deliver(HintsDispatchExecutor.java:259)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:242)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:220)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:199)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_77]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> [na:1.8.0_77]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_77]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_77]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
> Caused by: java.io.EOFException: null
> at 
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:68)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:60)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.ChecksummedDataInput.readFully(ChecksummedDataInput.java:126)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:402) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.readBuffer(HintsReader.java:310)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNextInternal(HintsReader.java:301)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:278)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> ... 15 common frames omitted
> {noformat}
> We've found out that the hint file was truncated because there was a hard 
> reboot around the time of last write to the file. I think we basically need 
> to handle partially written hint files. Also, the CRC file does not exist in 
> this case (probably because it crashed while writing the hints file). May be 
> ignoring and cleaning up such partially written hint files can be a way to 
> fix this?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12728) Handling partially written hint files

[Garvit Juniwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Garvit Juniwal updated CASSANDRA-12728:
---
Attachment: CASSANDRA-12728.patch

If a hints file gets truncated due to a crash, the last hint might have
been written incompletely. Presence of a corrupted hints file can put
Cassandra in a crash loop trying o read a corrupted hint over and over.
Ignore and discard the last incomplete hint instead of throwing a
FSReadError exception.

> Handling partially written hint files
> -
>
> Key: CASSANDRA-12728
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12728
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sharvanath Pathak
>Assignee: Aleksey Yeschenko
> Attachments: CASSANDRA-12728.patch
>
>
> {noformat}
> ERROR [HintsDispatcher:1] 2016-09-28 17:44:43,397 
> HintsDispatchExecutor.java:225 - Failed to dispatch hints file 
> d5d7257c-9f81-49b2-8633-6f9bda6e3dea-1474892654160-1.hints: file is corrupted 
> ({})
> org.apache.cassandra.io.FSReadError: java.io.EOFException
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:282)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:252)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.sendHints(HintsDispatcher.java:156)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.sendHintsAndAwait(HintsDispatcher.java:137)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:119) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:91) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.deliver(HintsDispatchExecutor.java:259)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:242)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:220)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:199)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_77]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> [na:1.8.0_77]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_77]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_77]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
> Caused by: java.io.EOFException: null
> at 
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:68)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:60)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.ChecksummedDataInput.readFully(ChecksummedDataInput.java:126)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:402) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.readBuffer(HintsReader.java:310)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNextInternal(HintsReader.java:301)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:278)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> ... 15 common frames omitted
> {noformat}
> We've found out that the hint file was truncated because there was a hard 
> reboot around the time of last write to the file. I think we basically need 
> to handle partially written hint files. Also, the CRC file does not exist in 
> this case (probably because it crashed while writing the hints file). May be 
> ignoring and cleaning up such partially written hint files can be a way to 
> fix this?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Issue Comment Deleted] (CASSANDRA-12728) Handling partially written hint files

[Garvit Juniwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Garvit Juniwal updated CASSANDRA-12728:
---
Comment: was deleted

(was: If a hints file gets truncated due to a crash, the last hint might have
been written incompletely. Presence of a corrupted hints file can put
Cassandra in a crash loop trying o read a corrupted hint over and over.
Ignore and discard the last incomplete hint instead of throwing a
FSReadError exception.)

> Handling partially written hint files
> -
>
> Key: CASSANDRA-12728
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12728
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sharvanath Pathak
>Assignee: Aleksey Yeschenko
> Attachments: CASSANDRA-12728.patch
>
>
> {noformat}
> ERROR [HintsDispatcher:1] 2016-09-28 17:44:43,397 
> HintsDispatchExecutor.java:225 - Failed to dispatch hints file 
> d5d7257c-9f81-49b2-8633-6f9bda6e3dea-1474892654160-1.hints: file is corrupted 
> ({})
> org.apache.cassandra.io.FSReadError: java.io.EOFException
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:282)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:252)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.sendHints(HintsDispatcher.java:156)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.sendHintsAndAwait(HintsDispatcher.java:137)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:119) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:91) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.deliver(HintsDispatchExecutor.java:259)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:242)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:220)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:199)
>  [apache-cassandra-3.0.6.jar:3.0.6]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_77]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> [na:1.8.0_77]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_77]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_77]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
> Caused by: java.io.EOFException: null
> at 
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:68)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:60)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.ChecksummedDataInput.readFully(ChecksummedDataInput.java:126)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:402) 
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.readBuffer(HintsReader.java:310)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNextInternal(HintsReader.java:301)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> at 
> org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:278)
>  ~[apache-cassandra-3.0.6.jar:3.0.6]
> ... 15 common frames omitted
> {noformat}
> We've found out that the hint file was truncated because there was a hard 
> reboot around the time of last write to the file. I think we basically need 
> to handle partially written hint files. Also, the CRC file does not exist in 
> this case (probably because it crashed while writing the hints file). May be 
> ignoring and cleaning up such partially written hint files can be a way to 
> fix this?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9754) Make index info heap friendly for large CQL partitions

[Michael Kjellman (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576497#comment-15576497
 ] 

Michael Kjellman commented on CASSANDRA-9754:
-

All of the threads that were responsible for generating load in the control 
cluster for the two large partition read and write workloads had died because 
the cluster became so unstable. As soon as I just restarted the stress load 60% 
of the instances in the cluster OOMed within 2 minutes of restarting the load. 

At this point I don't think I can drive any more data into the partitions with 
the old code and I'm going to declare defeat and say that 17GB as the absolute 
max partition size possible with the old/previous/current index implementation 
(given the JVM parameters as I detailed in the test description above).

I'm going to leave the load at the current read and write rates in the two 
Birch clusters until things explode to see the theoretical max partition size 
possible with the Birch implementation today. After that I'll wipe the clusters 
and restart the same load at 2x the read and write rates to see how things go 
with that configuration.

> Make index info heap friendly for large CQL partitions
> --
>
> Key: CASSANDRA-9754
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9754
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Assignee: Michael Kjellman
>Priority: Minor
> Fix For: 4.x
>
> Attachments: gc_collection_times_with_birch.png, 
> gc_collection_times_without_birch.png, gc_counts_with_birch.png, 
> gc_counts_without_birch.png, 
> perf_cluster_1_with_birch_read_latency_and_counts.png, 
> perf_cluster_1_with_birch_write_latency_and_counts.png, 
> perf_cluster_2_with_birch_read_latency_and_counts.png, 
> perf_cluster_2_with_birch_write_latency_and_counts.png, 
> perf_cluster_3_without_birch_read_latency_and_counts.png, 
> perf_cluster_3_without_birch_write_latency_and_counts.png
>
>
>  Looking at a heap dump of 2.0 cluster, I found that majority of the objects 
> are IndexInfo and its ByteBuffers. This is specially bad in endpoints with 
> large CQL partitions. If a CQL partition is say 6,4GB, it will have 100K 
> IndexInfo objects and 200K ByteBuffers. This will create a lot of churn for 
> GC. Can this be improved by not creating so many objects?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9754) Make index info heap friendly for large CQL partitions

[Michael Kjellman (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576390#comment-15576390
 ] 

Michael Kjellman commented on CASSANDRA-9754:
-

I have even more great news! My two test clusters just crossed 53GB for the 
90th percentile for max row size. The 50th percentile for mean row size is 
~8.5GB. Read and write latencies are still the same as the numbers I posted 
above from 3 days ago. So you could have an entire cluster of 10GB rows and 
still be stable :) 

> Make index info heap friendly for large CQL partitions
> --
>
> Key: CASSANDRA-9754
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9754
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Assignee: Michael Kjellman
>Priority: Minor
> Fix For: 4.x
>
> Attachments: gc_collection_times_with_birch.png, 
> gc_collection_times_without_birch.png, gc_counts_with_birch.png, 
> gc_counts_without_birch.png, 
> perf_cluster_1_with_birch_read_latency_and_counts.png, 
> perf_cluster_1_with_birch_write_latency_and_counts.png, 
> perf_cluster_2_with_birch_read_latency_and_counts.png, 
> perf_cluster_2_with_birch_write_latency_and_counts.png, 
> perf_cluster_3_without_birch_read_latency_and_counts.png, 
> perf_cluster_3_without_birch_write_latency_and_counts.png
>
>
>  Looking at a heap dump of 2.0 cluster, I found that majority of the objects 
> are IndexInfo and its ByteBuffers. This is specially bad in endpoints with 
> large CQL partitions. If a CQL partition is say 6,4GB, it will have 100K 
> IndexInfo objects and 200K ByteBuffers. This will create a lot of churn for 
> GC. Can this be improved by not creating so many objects?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10857) Allow dropping COMPACT STORAGE flag from tables in 3.X

[Alex Petrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576188#comment-15576188
 ] 

Alex Petrov commented on CASSANDRA-10857:
-

The empty column selection from this issue will be irrelevant after 
[CASSANDRA-12373].

> Allow dropping COMPACT STORAGE flag from tables in 3.X
> --
>
> Key: CASSANDRA-10857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10857
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL, Distributed Metadata
>Reporter: Aleksey Yeschenko
>Assignee: Alex Petrov
> Fix For: 3.x
>
>
> Thrift allows users to define flexible mixed column families - where certain 
> columns would have explicitly pre-defined names, potentially non-default 
> validation types, and be indexed.
> Example:
> {code}
> create column family foo
> and default_validation_class = UTF8Type
> and column_metadata = [
> {column_name: bar, validation_class: Int32Type, index_type: KEYS},
> {column_name: baz, validation_class: UUIDType, index_type: KEYS}
> ];
> {code}
> Columns named {{bar}} and {{baz}} will be validated as {{Int32Type}} and 
> {{UUIDType}}, respectively, and be indexed. Columns with any other name will 
> be validated by {{UTF8Type}} and will not be indexed.
> With CASSANDRA-8099, {{bar}} and {{baz}} would be mapped to static columns 
> internally. However, being {{WITH COMPACT STORAGE}}, the table will only 
> expose {{bar}} and {{baz}} columns. Accessing any dynamic columns (any column 
> not named {{bar}} and {{baz}}) right now requires going through Thrift.
> This is blocking Thrift -> CQL migration for users who have mixed 
> dynamic/static column families. That said, it *shouldn't* be hard to allow 
> users to drop the {{compact}} flag to expose the table as it is internally 
> now, and be able to access all columns.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-12384) Include info about sstable on "Compacting large row” message

[sankalp kohli (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576131#comment-15576131
 ] 

sankalp kohli edited comment on CASSANDRA-12384 at 10/14/16 6:37 PM:
-

Attached v2 and we dont need Dtest changes


was (Author: kohlisankalp):
If you ignore the Dtest attached...It will work. 

> Include info about sstable on "Compacting large row” message
> 
>
> Key: CASSANDRA-12384
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12384
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Assignee: sankalp kohli
>Priority: Trivial
> Attachments: CASSANDRA-12384_3.0.txt, CASSANDRA-12384_trunk.txt, 
> CASSANDRA_12384_v2.diff
>
>
> On a message like this one, it would be helpful to understand which sstable 
> this large row is going in
> Compacting large row abc/xyz:38956kjhawf (xyz bytes) incrementally



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12384) Include info about sstable on "Compacting large row” message

[sankalp kohli (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

sankalp kohli updated CASSANDRA-12384:
--
Attachment: CASSANDRA_12384_v2.diff

> Include info about sstable on "Compacting large row” message
> 
>
> Key: CASSANDRA-12384
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12384
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Assignee: sankalp kohli
>Priority: Trivial
> Attachments: CASSANDRA-12384_3.0.txt, CASSANDRA-12384_trunk.txt, 
> CASSANDRA_12384_v2.diff
>
>
> On a message like this one, it would be helpful to understand which sstable 
> this large row is going in
> Compacting large row abc/xyz:38956kjhawf (xyz bytes) incrementally



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12384) Include info about sstable on "Compacting large row” message

[sankalp kohli (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

sankalp kohli updated CASSANDRA-12384:
--
Attachment: (was: Dtest.txt)

> Include info about sstable on "Compacting large row” message
> 
>
> Key: CASSANDRA-12384
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12384
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Assignee: sankalp kohli
>Priority: Trivial
> Attachments: CASSANDRA-12384_3.0.txt, CASSANDRA-12384_trunk.txt, 
> CASSANDRA_12384_v2.diff
>
>
> On a message like this one, it would be helpful to understand which sstable 
> this large row is going in
> Compacting large row abc/xyz:38956kjhawf (xyz bytes) incrementally



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12384) Include info about sstable on "Compacting large row” message

[sankalp kohli (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576131#comment-15576131
 ] 

sankalp kohli commented on CASSANDRA-12384:
---

If you ignore the Dtest attached...It will work. 

> Include info about sstable on "Compacting large row” message
> 
>
> Key: CASSANDRA-12384
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12384
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Assignee: sankalp kohli
>Priority: Trivial
> Attachments: CASSANDRA-12384_3.0.txt, CASSANDRA-12384_trunk.txt, 
> Dtest.txt
>
>
> On a message like this one, it would be helpful to understand which sstable 
> this large row is going in
> Compacting large row abc/xyz:38956kjhawf (xyz bytes) incrementally



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12535) Occasionally seeing AccessControlException, CodecNotFoundException when executing a User Defined Aggregate

[Carl Yeksigian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Yeksigian updated CASSANDRA-12535:
---
Status: Open  (was: Patch Available)

> Occasionally seeing AccessControlException, CodecNotFoundException when 
> executing a User Defined Aggregate
> --
>
> Key: CASSANDRA-12535
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12535
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 3.7 (via brew install), Mac OS X 10.11.6
>Reporter: Pat Patterson
>Assignee: Robert Stupp
>Priority: Minor
> Fix For: 3.0.x, 3.x
>
>
> I have defined a UDA to implement standard deviation:
> {noformat}
> cqlsh:mykeyspace> CREATE OR REPLACE FUNCTION sdState ( state 
> tuple, val double ) CALLED ON NULL INPUT RETURNS 
> tuple LANGUAGE java AS 
>  ... 'int n = state.getInt(0); double mean = state.getDouble(1); double m2 = 
> state.getDouble(2); n++; double delta = val - mean; mean += delta / n; m2 += 
> delta * (val - mean); state.setInt(0, n); state.setDouble(1, mean); 
> state.setDouble(2, m2); return state;'; 
> cqlsh:mykeyspace> CREATE OR REPLACE FUNCTION sdFinal ( state 
> tuple ) CALLED ON NULL INPUT RETURNS double LANGUAGE java 
> AS 
>  ... 'int n = state.getInt(0); double m2 = state.getDouble(2); if (n < 1) { 
> return null; } return Math.sqrt(m2 / (n - 1));';
> cqlsh:mykeyspace> CREATE AGGREGATE IF NOT EXISTS stdev ( double ) 
>  ... SFUNC sdState STYPE tuple FINALFUNC sdFinal INITCOND 
> (0,0,0);
> {noformat}
> My table:
> {noformat}
> CREATE TABLE readings (
> sensor_id int,
> time timestamp,
> temperature double,
> status text,
> PRIMARY KEY (sensor_id, time)
> ) WITH CLUSTERING ORDER BY (time ASC);
> {noformat}
> I'm inserting a row every 0.1 seconds. The data looks like this:
> {noformat}
> cqlsh:mykeyspace> select * from readings limit 10;
>  sensor_id | time| status | temperature
> ---+-++-
>  5 | 2016-08-24 19:11:34.896000+ | OK |9.97
>  5 | 2016-08-24 19:11:43.933000+ | OK |   10.28
>  5 | 2016-08-24 19:11:49.958000+ | OK |7.65
>  5 | 2016-08-24 19:11:51.968000+ | OK |   10.11
>  5 | 2016-08-24 19:12:58.512000+ |  Fault |   10.41
>  5 | 2016-08-24 19:13:04.542000+ | OK |9.66
>  5 | 2016-08-24 19:13:16.593000+ | OK |10.9
>  5 | 2016-08-24 19:13:37.692000+ | OK |11.2
>  5 | 2016-08-24 19:13:46.738000+ | OK |   10.34
>  5 | 2016-08-24 19:13:49.757000+ | OK |10.6
> {noformat}
> I'm running a query every few seconds with my UDA - like this (timestamps are 
> different each time):
> {noformat}
> select avg(temperature), stdev(temperature) from readings where sensor_id = 1 
> and time > 1472066523193;
> {noformat}
> Most of the time, this works just fine:
> {noformat}
> cqlsh:mykeyspace> select avg(temperature), stdev(temperature) from readings 
> where sensor_id = 1 and time > 1472066523193;
>  system.avg(temperature) | mykeyspace.stdev(temperature)
> -+---
>   9.9291 |   0.94179
> (1 rows)
> {noformat}
> But, occasionally, it fails with one of two exceptions:
> {noformat}
> cqlsh:mykeyspace> select avg(temperature), stdev(temperature) from readings 
> where sensor_id = 1 and time > 1472066523193;
> Traceback (most recent call last):
>   File "/usr/local/Cellar/cassandra/3.7/libexec/bin/cqlsh.py", line 1277, in 
> perform_simple_statement
> result = future.result()
>   File "cassandra/cluster.py", line 3629, in 
> cassandra.cluster.ResponseFuture.result (cassandra/cluster.c:69369)
> raise self._final_exception
> FunctionFailure: Error from server: code=1400 [User Defined Function failure] 
> message="execution of 'mykeyspace.sdstate[frozen>, 
> double]' failed: java.security.AccessControlException: access denied 
> ("java.io.FilePermission" "/usr/local/etc/cassandra/logback.xml" "read")"
> {noformat}
> or
> {noformat}
> cqlsh:mykeyspace> select count(*), avg(temperature), stdev(temperature) from 
> readings where sensor_id = 1 and time > '2016-08-24 15:00:00.000+';
> Traceback (most recent call last):
>   File "/usr/local/Cellar/cassandra/3.7/libexec/bin/cqlsh.py", line 1277, in 
> perform_simple_statement
> result = future.result()
>   File "cassandra/cluster.py", line 3629, in 
> cassandra.cluster.ResponseFuture.result (cassandra/cluster.c:69369)
> 

[jira] [Commented] (CASSANDRA-12535) Occasionally seeing AccessControlException, CodecNotFoundException when executing a User Defined Aggregate

[Carl Yeksigian (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15575963#comment-15575963
 ] 

Carl Yeksigian commented on CASSANDRA-12535:


Looks like the logback reload tests failed in CI.

Is there a way for us to suppress the logging attempts made from a secured 
thread? While this seems like it will fix the issue at hand, just worried that 
future updates to logback or the logger that we use will cause other subtle 
issues with our sandboxing.

> Occasionally seeing AccessControlException, CodecNotFoundException when 
> executing a User Defined Aggregate
> --
>
> Key: CASSANDRA-12535
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12535
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 3.7 (via brew install), Mac OS X 10.11.6
>Reporter: Pat Patterson
>Assignee: Robert Stupp
>Priority: Minor
> Fix For: 3.0.x, 3.x
>
>
> I have defined a UDA to implement standard deviation:
> {noformat}
> cqlsh:mykeyspace> CREATE OR REPLACE FUNCTION sdState ( state 
> tuple, val double ) CALLED ON NULL INPUT RETURNS 
> tuple LANGUAGE java AS 
>  ... 'int n = state.getInt(0); double mean = state.getDouble(1); double m2 = 
> state.getDouble(2); n++; double delta = val - mean; mean += delta / n; m2 += 
> delta * (val - mean); state.setInt(0, n); state.setDouble(1, mean); 
> state.setDouble(2, m2); return state;'; 
> cqlsh:mykeyspace> CREATE OR REPLACE FUNCTION sdFinal ( state 
> tuple ) CALLED ON NULL INPUT RETURNS double LANGUAGE java 
> AS 
>  ... 'int n = state.getInt(0); double m2 = state.getDouble(2); if (n < 1) { 
> return null; } return Math.sqrt(m2 / (n - 1));';
> cqlsh:mykeyspace> CREATE AGGREGATE IF NOT EXISTS stdev ( double ) 
>  ... SFUNC sdState STYPE tuple FINALFUNC sdFinal INITCOND 
> (0,0,0);
> {noformat}
> My table:
> {noformat}
> CREATE TABLE readings (
> sensor_id int,
> time timestamp,
> temperature double,
> status text,
> PRIMARY KEY (sensor_id, time)
> ) WITH CLUSTERING ORDER BY (time ASC);
> {noformat}
> I'm inserting a row every 0.1 seconds. The data looks like this:
> {noformat}
> cqlsh:mykeyspace> select * from readings limit 10;
>  sensor_id | time| status | temperature
> ---+-++-
>  5 | 2016-08-24 19:11:34.896000+ | OK |9.97
>  5 | 2016-08-24 19:11:43.933000+ | OK |   10.28
>  5 | 2016-08-24 19:11:49.958000+ | OK |7.65
>  5 | 2016-08-24 19:11:51.968000+ | OK |   10.11
>  5 | 2016-08-24 19:12:58.512000+ |  Fault |   10.41
>  5 | 2016-08-24 19:13:04.542000+ | OK |9.66
>  5 | 2016-08-24 19:13:16.593000+ | OK |10.9
>  5 | 2016-08-24 19:13:37.692000+ | OK |11.2
>  5 | 2016-08-24 19:13:46.738000+ | OK |   10.34
>  5 | 2016-08-24 19:13:49.757000+ | OK |10.6
> {noformat}
> I'm running a query every few seconds with my UDA - like this (timestamps are 
> different each time):
> {noformat}
> select avg(temperature), stdev(temperature) from readings where sensor_id = 1 
> and time > 1472066523193;
> {noformat}
> Most of the time, this works just fine:
> {noformat}
> cqlsh:mykeyspace> select avg(temperature), stdev(temperature) from readings 
> where sensor_id = 1 and time > 1472066523193;
>  system.avg(temperature) | mykeyspace.stdev(temperature)
> -+---
>   9.9291 |   0.94179
> (1 rows)
> {noformat}
> But, occasionally, it fails with one of two exceptions:
> {noformat}
> cqlsh:mykeyspace> select avg(temperature), stdev(temperature) from readings 
> where sensor_id = 1 and time > 1472066523193;
> Traceback (most recent call last):
>   File "/usr/local/Cellar/cassandra/3.7/libexec/bin/cqlsh.py", line 1277, in 
> perform_simple_statement
> result = future.result()
>   File "cassandra/cluster.py", line 3629, in 
> cassandra.cluster.ResponseFuture.result (cassandra/cluster.c:69369)
> raise self._final_exception
> FunctionFailure: Error from server: code=1400 [User Defined Function failure] 
> message="execution of 'mykeyspace.sdstate[frozen>, 
> double]' failed: java.security.AccessControlException: access denied 
> ("java.io.FilePermission" "/usr/local/etc/cassandra/logback.xml" "read")"
> {noformat}
> or
> {noformat}
> cqlsh:mykeyspace> select count(*), avg(temperature), stdev(temperature) from 
> readings where sensor_id = 1 and time > '2016-08-24 

[jira] [Comment Edited] (CASSANDRA-12787) Reduce instanceOf() type checking to improve performance

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15575845#comment-15575845
 ] 

T Jake Luciani edited comment on CASSANDRA-12787 at 10/14/16 4:53 PM:
--

I ran a test on a 24 core machine with 64G ram and wasn't able to see a 
difference with the following 
[branch|https://github.com/tjake/cassandra/tree/instanceof-fix].

I was using a pure write workload.

!instaceof.png!


was (Author: tjake):
I ran a test on a 24 core machine with 64G ram and wasn't able to see a 
difference with the following 
[branch|https://github.com/tjake/cassandra/tree/instanceof-fix].

I was using a pure write workload

> Reduce instanceOf() type checking to improve performance
> 
>
> Key: CASSANDRA-12787
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12787
> Project: Cassandra
>  Issue Type: Improvement
> Environment: The tests and examples stated were run on:
> Intel (R) Xeon (R) CPU E5-2699 v3 @ 2.30GHz, HT Enabled
> Oracle JDK 1.8
> Cassandra 3.10-SNAPSHOT
> Linux kernel 4.7
>Reporter: shylaja kokoori
> Fix For: 3.x
>
> Attachments: instanceof.png, reduce_instanceof_typechecking.pdf
>
>
> While performance profiling Cassandra with cassandra-stress test on a  pure 
> write workload, we noticed that one of the hot methods for Cassandra server 
> is the compareTo( PartitionPosition pos) function in 
> org.apache.cassandra.db.DecoratedKey. The actual root cause of the problem is 
> a slowdown in Java's instanceof operator.
> Our initial performance testing using a hacked JVM shows about 61% increase 
> in throughput and about 15% reduction in 99 percentile latency. Data shows 
> that improvements are from the removal of thread contention caused by 
> Instanceof operator.
> Currently, we are working on a JDK fix to solve this issue. In the meantime, 
> we think it will be beneficial to address this issue at Java application 
> level as well. We are in the process of creating a patch to Cassandra 
> replacing instanceof check with virtual method calls. The patch will be 
> available in this thread for review soon. Please let us know your feedback 
> and comments.
> For more details please refer to the attached pdf.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12787) Reduce instanceOf() type checking to improve performance

[T Jake Luciani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani updated CASSANDRA-12787:
---
Attachment: instanceof.png

I ran a test on a 24 core machine with 64G ram and wasn't able to see a 
difference with the following 
[branch|https://github.com/tjake/cassandra/tree/instanceof-fix].

I was using a pure write workload

> Reduce instanceOf() type checking to improve performance
> 
>
> Key: CASSANDRA-12787
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12787
> Project: Cassandra
>  Issue Type: Improvement
> Environment: The tests and examples stated were run on:
> Intel (R) Xeon (R) CPU E5-2699 v3 @ 2.30GHz, HT Enabled
> Oracle JDK 1.8
> Cassandra 3.10-SNAPSHOT
> Linux kernel 4.7
>Reporter: shylaja kokoori
> Fix For: 3.x
>
> Attachments: instanceof.png, reduce_instanceof_typechecking.pdf
>
>
> While performance profiling Cassandra with cassandra-stress test on a  pure 
> write workload, we noticed that one of the hot methods for Cassandra server 
> is the compareTo( PartitionPosition pos) function in 
> org.apache.cassandra.db.DecoratedKey. The actual root cause of the problem is 
> a slowdown in Java's instanceof operator.
> Our initial performance testing using a hacked JVM shows about 61% increase 
> in throughput and about 15% reduction in 99 percentile latency. Data shows 
> that improvements are from the removal of thread contention caused by 
> Instanceof operator.
> Currently, we are working on a JDK fix to solve this issue. In the meantime, 
> we think it will be beneficial to address this issue at Java application 
> level as well. We are in the process of creating a patch to Cassandra 
> replacing instanceof check with virtual method calls. The patch will be 
> available in this thread for review soon. Please let us know your feedback 
> and comments.
> For more details please refer to the attached pdf.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12761) Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes in CASSANDRA-10876

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15575758#comment-15575758
 ] 

Aleksey Yeschenko commented on CASSANDRA-12761:
---

Committed as 
[e722046406bb2c037a96ea9605e0e4878e49e596|https://github.com/apache/cassandra/commit/e722046406bb2c037a96ea9605e0e4878e49e596]
 to 3.X and merged into trunk, thanks.

> Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes in 
> CASSANDRA-10876  
> -
>
> Key: CASSANDRA-12761
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12761
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration
>Reporter: Guy Bolton King
>Assignee: Guy Bolton King
>Priority: Trivial
> Fix For: 3.x, 4.x
>
> Attachments: 
> 0001-Update-cassandra.yaml-documentation-for-batch_size-t.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12761) Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes in CASSANDRA-10876

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-12761:
--
   Resolution: Fixed
Fix Version/s: (was: 4.x)
   (was: 3.x)
   3.10
   Status: Resolved  (was: Ready to Commit)

> Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes in 
> CASSANDRA-10876  
> -
>
> Key: CASSANDRA-12761
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12761
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration
>Reporter: Guy Bolton King
>Assignee: Guy Bolton King
>Priority: Trivial
> Fix For: 3.10
>
> Attachments: 
> 0001-Update-cassandra.yaml-documentation-for-batch_size-t.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] cassandra git commit: Update cassandra.yaml documentation for batch_size thresholds

[aleksey]

Update cassandra.yaml documentation for batch_size thresholds

Take account of changes in CASSANDRA-10876

Patch by Guy Bolton King; reviewed by Joel Knighton for CASSANDRA-12761


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e7220464
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e7220464
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e7220464

Branch: refs/heads/trunk
Commit: e722046406bb2c037a96ea9605e0e4878e49e596
Parents: 206a7bb
Author: Guy Bolton King 
Authored: Tue Sep 20 14:55:24 2016 +0100
Committer: Aleksey Yeschenko 
Committed: Fri Oct 14 17:14:57 2016 +0100

--
 CHANGES.txt | 1 +
 conf/cassandra.yaml | 4 ++--
 2 files changed, 3 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e7220464/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0045ba5..5023590 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.10
+ * Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes 
in CASSANDRA-10876 (CASSANDRA-12761)
  * cqlsh fails to format collections when using aliases (CASSANDRA-11534)
  * Check for hash conflicts in prepared statements (CASSANDRA-12733)
  * Exit query parsing upon first error (CASSANDRA-12598)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e7220464/conf/cassandra.yaml
--
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index 8a0b3ee..986913b 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -1150,11 +1150,11 @@ transparent_data_encryption_options:
 tombstone_warn_threshold: 1000
 tombstone_failure_threshold: 10
 
-# Log WARN on any batch size exceeding this value. 5kb per batch by default.
+# Log WARN on any multiple-partition batch size exceeding this value. 5kb per 
batch by default.
 # Caution should be taken on increasing the size of this threshold as it can 
lead to node instability.
 batch_size_warn_threshold_in_kb: 5
 
-# Fail any batch exceeding this value. 50kb (10x warn threshold) by default.
+# Fail any multiple-partition batch exceeding this value. 50kb (10x warn 
threshold) by default.
 batch_size_fail_threshold_in_kb: 50
 
 # Log WARN on any batches not of type LOGGED than span across more partitions 
than this limit

[1/3] cassandra git commit: Update cassandra.yaml documentation for batch_size thresholds

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.X 206a7bbac -> e72204640
  refs/heads/trunk 17bd6adc8 -> b3a0130d1


Update cassandra.yaml documentation for batch_size thresholds

Take account of changes in CASSANDRA-10876

Patch by Guy Bolton King; reviewed by Joel Knighton for CASSANDRA-12761


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e7220464
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e7220464
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e7220464

Branch: refs/heads/cassandra-3.X
Commit: e722046406bb2c037a96ea9605e0e4878e49e596
Parents: 206a7bb
Author: Guy Bolton King 
Authored: Tue Sep 20 14:55:24 2016 +0100
Committer: Aleksey Yeschenko 
Committed: Fri Oct 14 17:14:57 2016 +0100

--
 CHANGES.txt | 1 +
 conf/cassandra.yaml | 4 ++--
 2 files changed, 3 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e7220464/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0045ba5..5023590 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.10
+ * Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes 
in CASSANDRA-10876 (CASSANDRA-12761)
  * cqlsh fails to format collections when using aliases (CASSANDRA-11534)
  * Check for hash conflicts in prepared statements (CASSANDRA-12733)
  * Exit query parsing upon first error (CASSANDRA-12598)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e7220464/conf/cassandra.yaml
--
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index 8a0b3ee..986913b 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -1150,11 +1150,11 @@ transparent_data_encryption_options:
 tombstone_warn_threshold: 1000
 tombstone_failure_threshold: 10
 
-# Log WARN on any batch size exceeding this value. 5kb per batch by default.
+# Log WARN on any multiple-partition batch size exceeding this value. 5kb per 
batch by default.
 # Caution should be taken on increasing the size of this threshold as it can 
lead to node instability.
 batch_size_warn_threshold_in_kb: 5
 
-# Fail any batch exceeding this value. 50kb (10x warn threshold) by default.
+# Fail any multiple-partition batch exceeding this value. 50kb (10x warn 
threshold) by default.
 batch_size_fail_threshold_in_kb: 50
 
 # Log WARN on any batches not of type LOGGED than span across more partitions 
than this limit

[3/3] cassandra git commit: Merge branch 'cassandra-3.X' into trunk

[aleksey]

Merge branch 'cassandra-3.X' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b3a0130d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b3a0130d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b3a0130d

Branch: refs/heads/trunk
Commit: b3a0130d1d76af903c1e5bcc2b65a2ef120e3626
Parents: 17bd6ad e722046
Author: Aleksey Yeschenko 
Authored: Fri Oct 14 17:15:22 2016 +0100
Committer: Aleksey Yeschenko 
Committed: Fri Oct 14 17:15:22 2016 +0100

--
 CHANGES.txt | 1 +
 conf/cassandra.yaml | 4 ++--
 2 files changed, 3 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b3a0130d/CHANGES.txt
--
diff --cc CHANGES.txt
index 0aef01c,5023590..63ed4a6
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,11 -1,5 +1,12 @@@
 +4.0
 + * Add column definition kind to dropped columns in schema (CASSANDRA-12705)
 + * Add (automate) Nodetool Documentation (CASSANDRA-12672)
 + * Update bundled cqlsh python driver to 3.7.0 (CASSANDRA-12736)
 + * Reject invalid replication settings when creating or altering a keyspace 
(CASSANDRA-12681)
 +
 +
  3.10
+  * Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes 
in CASSANDRA-10876 (CASSANDRA-12761)
   * cqlsh fails to format collections when using aliases (CASSANDRA-11534)
   * Check for hash conflicts in prepared statements (CASSANDRA-12733)
   * Exit query parsing upon first error (CASSANDRA-12598)

[jira] [Updated] (CASSANDRA-12791) MessageIn logic to determine if the message is cross-node is wrong

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-12791:
-
Reviewer: Stefania
  Status: Patch Available  (was: Open)

I'm attaching a patch with the changes described above:
| [12791-trunk|https://github.com/pcmanus/cassandra/commits/12791-trunk] | 
[utests|http://cassci.datastax.com/job/pcmanus-12791-trunk-testall] | 
[dtests|http://cassci.datastax.com/job/pcmanus-12791-trunk-dtest] |

The patch actually does 2 (less important) things on top of the points above:
# it uses {{ApproximateTime.currentTimeMillis()}} for anything related to the 
message "construction time" timestamp. This was currently a bit inconsistent in 
that the timestamp was set using {{System.currentTimeMillis()}} but later 
compared to {{ApproximateTime.currentTimeMillis()}}. This is not really a 
problem in practice but I'm a stickler for consistency so that was bugging me.
# the patch gets rid of the {{ConstructionTime}} class. This is arguably a bit 
of a personal opinion here since the behavior of that class is still there, 
it's just somewhat "inlined" in {{MessageIn}}/{{Monitorable}}. The reason is 
that after this patch, grouping the 2 infos of that class (timestamp and 
isCrossNode boolean) together doesn't make sense to me: the {{isCrossNode}} 
property is one of message/operation, not of the timestamp.

Also, the patch is currently against trunk. It's probably worth at least 
putting it 3.X though, so unless object to that I'll rebase to that branch once 
I have test results for trunk. I'm not sure it's worth bothering with 3.0 
though: as said in the description, I consider it kind of a bug in that not 
having "crossNode" metrics be incremented by default will almost surely 
surprise users, but it's also fairly minor and could be called a "weird 
behavior".


> MessageIn logic to determine if the message is cross-node is wrong
> --
>
> Key: CASSANDRA-12791
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12791
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
>Priority: Minor
>
> {{MessageIn}} has the following code to read the 'creation time' of the 
> message on the receiving side:
> {noformat}
> public static ConstructionTime readTimestamp(InetAddress from, DataInputPlus 
> input, long timestamp) throws IOException
> {
> // make sure to readInt, even if cross_node_to is not enabled
> int partial = input.readInt();
> long crossNodeTimestamp = (timestamp & 0xL) | (((partial 
> & 0xL) << 2) >> 2);
> if (timestamp > crossNodeTimestamp)
> {
> MessagingService.instance().metrics.addTimeTaken(from, timestamp - 
> crossNodeTimestamp);
> }
> if(DatabaseDescriptor.hasCrossNodeTimeout())
> {
> return new ConstructionTime(crossNodeTimestamp, timestamp != 
> crossNodeTimestamp);
> }
> else
> {
> return new ConstructionTime();
> }
> }
> {noformat}
> where {{timestamp}} is really the local time on the receiving node when 
> calling that method.
> The incorrect part, I believe, is the {{timestamp != crossNodeTimestamp}} 
> used to set the {{isCrossNode}} field of {{ConstructionTime}}. A first 
> problem is that this will basically always be {{true}}: for it to be 
> {{false}}, we'd need the low-bytes of the timestamp taken on the sending node 
> to coincide exactly with the ones taken on the receiving side, which is 
> _very_ unlikely. It is also a relatively meaningless test: having that test 
> be {{false}} basically means the lack of clock sync between the 2 nodes is 
> exactly the time the 2 calls to {{System.currentTimeMillis()}} (on sender and 
> receiver), which is definitively not what we care about.
> What the result of this test is used for is to determine if the message was 
> crossNode or local. It's used to increment different metrics (we separate 
> metric local versus crossNode dropped messages) in {{MessagingService}} for 
> instance. And that's where this is kind of a bug: not only the {{timestamp != 
> crossNodeTimestamp}}, but if {{DatabaseDescriptor.hasCrossNodeTimeout()}}, we 
> *always* have this {{isCrossNode}} false, which means we'll never increment 
> the "cross-node dropped messages" metric, which is imo unexpected.
> That is, it is true that if {{DatabaseDescriptor.hasCrossNodeTimeout() == 
> false}}, then we end using the receiver side timestamp to timeout messages, 
> and so you end up only dropping messages that timeout locally. And _in that 
> sense_, always incrementing the "locally" dropped messages metric is not 
> completely illogical. But I doubt most users are aware of those pretty 
> specific nuance when looking at the related metrics, and 

[jira] [Created] (CASSANDRA-12791) MessageIn logic to determine if the message is cross-node is wrong

[Sylvain Lebresne (JIRA)]

Sylvain Lebresne created CASSANDRA-12791:


 Summary: MessageIn logic to determine if the message is cross-node 
is wrong
 Key: CASSANDRA-12791
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12791
 Project: Cassandra
  Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Priority: Minor


{{MessageIn}} has the following code to read the 'creation time' of the message 
on the receiving side:
{noformat}
public static ConstructionTime readTimestamp(InetAddress from, DataInputPlus 
input, long timestamp) throws IOException
{
// make sure to readInt, even if cross_node_to is not enabled
int partial = input.readInt();
long crossNodeTimestamp = (timestamp & 0xL) | (((partial & 
0xL) << 2) >> 2);
if (timestamp > crossNodeTimestamp)
{
MessagingService.instance().metrics.addTimeTaken(from, timestamp - 
crossNodeTimestamp);
}
if(DatabaseDescriptor.hasCrossNodeTimeout())
{
return new ConstructionTime(crossNodeTimestamp, timestamp != 
crossNodeTimestamp);
}
else
{
return new ConstructionTime();
}
}
{noformat}
where {{timestamp}} is really the local time on the receiving node when calling 
that method.

The incorrect part, I believe, is the {{timestamp != crossNodeTimestamp}} used 
to set the {{isCrossNode}} field of {{ConstructionTime}}. A first problem is 
that this will basically always be {{true}}: for it to be {{false}}, we'd need 
the low-bytes of the timestamp taken on the sending node to coincide exactly 
with the ones taken on the receiving side, which is _very_ unlikely. It is also 
a relatively meaningless test: having that test be {{false}} basically means 
the lack of clock sync between the 2 nodes is exactly the time the 2 calls to 
{{System.currentTimeMillis()}} (on sender and receiver), which is definitively 
not what we care about.

What the result of this test is used for is to determine if the message was 
crossNode or local. It's used to increment different metrics (we separate 
metric local versus crossNode dropped messages) in {{MessagingService}} for 
instance. And that's where this is kind of a bug: not only the {{timestamp != 
crossNodeTimestamp}}, but if {{DatabaseDescriptor.hasCrossNodeTimeout()}}, we 
*always* have this {{isCrossNode}} false, which means we'll never increment the 
"cross-node dropped messages" metric, which is imo unexpected.

That is, it is true that if {{DatabaseDescriptor.hasCrossNodeTimeout() == 
false}}, then we end using the receiver side timestamp to timeout messages, and 
so you end up only dropping messages that timeout locally. And _in that sense_, 
always incrementing the "locally" dropped messages metric is not completely 
illogical. But I doubt most users are aware of those pretty specific nuance 
when looking at the related metrics, and I'm relatively sure users expect a 
metrics named {{droppedCrossNodeTimeout}} to actually count cross-node messages 
by default (keep in mind that {{DatabaseDescriptor.hasCrossNodeTimeout()}} is 
actually false by default).

Anyway, to sum it up I suggest that the following change should be done:
# the {{timestamp != crossNodeTimestamp}} test is definitively not what we 
want. We should at a minimum just replace it to {{true}} as that's basically 
what it ends up being except for very rare and arguably random cases.
# given how the {{ConstructionTime.isCrossNode}} is used, I suggest that we 
really want it to mean if the message has shipped cross-node, not just be a 
synonymous of {{DatabaseDescriptor.hasCrossNodeTimeout()}}. It should be 
whether the message shipped cross-node, i.e. whether {{from == 
BroadcastAdress()}} or not.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-12252) Support RR and Speculative Retry for EACH_QUORUM reads

[Carl Yeksigian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Yeksigian reassigned CASSANDRA-12252:
--

Assignee: Carl Yeksigian

> Support RR and Speculative Retry for EACH_QUORUM reads
> --
>
> Key: CASSANDRA-12252
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12252
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Carl Yeksigian
>Assignee: Carl Yeksigian
>Priority: Minor
>
> We should properly support read repair and speculative retry for EACH_QUORUM 
> reads.
> To support these, we need to make sure that we count responses for each DC 
> separately, so that we can properly monitor the number of responses from 
> each. For speculative retry, we should try to replace each host that we are 
> retrying with one in the same DC; that will make sure that we actually have 
> the correct responses at the end if the speculative retries are successful.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (CASSANDRA-12790) Support InfluxDb metrics reporter config

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer reopened CASSANDRA-12790:

  Assignee: Benjamin Lerer

> Support InfluxDb metrics reporter config
> 
>
> Key: CASSANDRA-12790
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12790
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Configuration, Packaging
> Environment: Ubuntu 14.04
> Oracle Java 1.8.0_65
> Cassandra 3.0.9
>Reporter: Achmad Nasirudin Sandi
>Assignee: Benjamin Lerer
>Priority: Minor
>  Labels: easyfix, easytest, newbie
> Attachments: metrics-reporter-config-influx.yaml
>
>
> InfluxDb is a great tool for storing metrics. The main advantage over 
> graphite protocol is its tag mechanism that allows host info to be excluded 
> from metric name.
> I managed to send metrics into InfluxDb by putting 
> {{metrics-influxdb-1.1.7.jar}} into {{lib}} folder and replacing these 
> dependencies jar:
>  
> {code}
> commons-codec-1.2.jar -> commons-codec-1.9.jar
> reporter-config3-3.0.0.jar -> reporter-config3-3.0.3.jar
> reporter-config-base-3.0.0.jar -> reporter-config-base-3.0.3.jar
> {code}
> Although it seems that dropwizard team is working on official [package for 
> InfluxDb|https://github.com/dropwizard/metrics/tree/4.0-development], it 
> would be great it those libs are available in default distribution by 
> upgrading {{commons-codec}} and {{metrics}} dependencies. I was wondering 
> what effect are there to Cassandra functionality. Where can I help?
> Attached file is my configuration example.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12787) Reduce instanceOf() type checking to improve performance

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15575566#comment-15575566
 ] 

T Jake Luciani commented on CASSANDRA-12787:


We already have a .kind() call.  I'm going to verify the issue on a high core 
system.  [~shylaja.koko...@intel.com] thanks so much for discovering this.  May 
I ask what tool you used to see this? I am able to see DK.compare was a hotspot 
but how do you see the jvm cpu cycles?  

> Reduce instanceOf() type checking to improve performance
> 
>
> Key: CASSANDRA-12787
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12787
> Project: Cassandra
>  Issue Type: Improvement
> Environment: The tests and examples stated were run on:
> Intel (R) Xeon (R) CPU E5-2699 v3 @ 2.30GHz, HT Enabled
> Oracle JDK 1.8
> Cassandra 3.10-SNAPSHOT
> Linux kernel 4.7
>Reporter: shylaja kokoori
> Fix For: 3.x
>
> Attachments: reduce_instanceof_typechecking.pdf
>
>
> While performance profiling Cassandra with cassandra-stress test on a  pure 
> write workload, we noticed that one of the hot methods for Cassandra server 
> is the compareTo( PartitionPosition pos) function in 
> org.apache.cassandra.db.DecoratedKey. The actual root cause of the problem is 
> a slowdown in Java's instanceof operator.
> Our initial performance testing using a hacked JVM shows about 61% increase 
> in throughput and about 15% reduction in 99 percentile latency. Data shows 
> that improvements are from the removal of thread contention caused by 
> Instanceof operator.
> Currently, we are working on a JDK fix to solve this issue. In the meantime, 
> we think it will be beneficial to address this issue at Java application 
> level as well. We are in the process of creating a patch to Cassandra 
> replacing instanceof check with virtual method calls. The patch will be 
> available in this thread for review soon. Please let us know your feedback 
> and comments.
> For more details please refer to the attached pdf.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-12790) Support InfluxDb metrics reporter config

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15575636#comment-15575636
 ] 

Benjamin Lerer edited comment on CASSANDRA-12790 at 10/14/16 3:20 PM:
--

[~carlyeks] it seems that {{commons-codec}} also need to be upgraded.


was (Author: blerer):
[~carlyeks] I agree but it seems that {{commons-codec}} also need to be 
upgraded.

> Support InfluxDb metrics reporter config
> 
>
> Key: CASSANDRA-12790
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12790
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Configuration, Packaging
> Environment: Ubuntu 14.04
> Oracle Java 1.8.0_65
> Cassandra 3.0.9
>Reporter: Achmad Nasirudin Sandi
>Priority: Minor
>  Labels: easyfix, easytest, newbie
> Attachments: metrics-reporter-config-influx.yaml
>
>
> InfluxDb is a great tool for storing metrics. The main advantage over 
> graphite protocol is its tag mechanism that allows host info to be excluded 
> from metric name.
> I managed to send metrics into InfluxDb by putting 
> {{metrics-influxdb-1.1.7.jar}} into {{lib}} folder and replacing these 
> dependencies jar:
>  
> {code}
> commons-codec-1.2.jar -> commons-codec-1.9.jar
> reporter-config3-3.0.0.jar -> reporter-config3-3.0.3.jar
> reporter-config-base-3.0.0.jar -> reporter-config-base-3.0.3.jar
> {code}
> Although it seems that dropwizard team is working on official [package for 
> InfluxDb|https://github.com/dropwizard/metrics/tree/4.0-development], it 
> would be great it those libs are available in default distribution by 
> upgrading {{commons-codec}} and {{metrics}} dependencies. I was wondering 
> what effect are there to Cassandra functionality. Where can I help?
> Attached file is my configuration example.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12790) Support InfluxDb metrics reporter config

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15575636#comment-15575636
 ] 

Benjamin Lerer commented on CASSANDRA-12790:


[~carlyeks] I agree but it seems that {{commons-codec}} also need to be 
upgraded.

> Support InfluxDb metrics reporter config
> 
>
> Key: CASSANDRA-12790
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12790
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Configuration, Packaging
> Environment: Ubuntu 14.04
> Oracle Java 1.8.0_65
> Cassandra 3.0.9
>Reporter: Achmad Nasirudin Sandi
>Priority: Minor
>  Labels: easyfix, easytest, newbie
> Attachments: metrics-reporter-config-influx.yaml
>
>
> InfluxDb is a great tool for storing metrics. The main advantage over 
> graphite protocol is its tag mechanism that allows host info to be excluded 
> from metric name.
> I managed to send metrics into InfluxDb by putting 
> {{metrics-influxdb-1.1.7.jar}} into {{lib}} folder and replacing these 
> dependencies jar:
>  
> {code}
> commons-codec-1.2.jar -> commons-codec-1.9.jar
> reporter-config3-3.0.0.jar -> reporter-config3-3.0.3.jar
> reporter-config-base-3.0.0.jar -> reporter-config-base-3.0.3.jar
> {code}
> Although it seems that dropwizard team is working on official [package for 
> InfluxDb|https://github.com/dropwizard/metrics/tree/4.0-development], it 
> would be great it those libs are available in default distribution by 
> upgrading {{commons-codec}} and {{metrics}} dependencies. I was wondering 
> what effect are there to Cassandra functionality. Where can I help?
> Attached file is my configuration example.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-12790) Support InfluxDb metrics reporter config

[Carl Yeksigian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Yeksigian resolved CASSANDRA-12790.

   Resolution: Duplicate
 Assignee: (was: Benjamin Lerer)
Fix Version/s: 3.10
Reproduced In:   (was: 3.0.9)

The update to metrics-reporter 3.0.3 has already been committed to 3.10.

> Support InfluxDb metrics reporter config
> 
>
> Key: CASSANDRA-12790
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12790
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Configuration, Packaging
> Environment: Ubuntu 14.04
> Oracle Java 1.8.0_65
> Cassandra 3.0.9
>Reporter: Achmad Nasirudin Sandi
>Priority: Minor
>  Labels: easyfix, easytest, newbie
> Fix For: 3.10
>
> Attachments: metrics-reporter-config-influx.yaml
>
>
> InfluxDb is a great tool for storing metrics. The main advantage over 
> graphite protocol is its tag mechanism that allows host info to be excluded 
> from metric name.
> I managed to send metrics into InfluxDb by putting 
> {{metrics-influxdb-1.1.7.jar}} into {{lib}} folder and replacing these 
> dependencies jar:
>  
> {code}
> commons-codec-1.2.jar -> commons-codec-1.9.jar
> reporter-config3-3.0.0.jar -> reporter-config3-3.0.3.jar
> reporter-config-base-3.0.0.jar -> reporter-config-base-3.0.3.jar
> {code}
> Although it seems that dropwizard team is working on official [package for 
> InfluxDb|https://github.com/dropwizard/metrics/tree/4.0-development], it 
> would be great it those libs are available in default distribution by 
> upgrading {{commons-codec}} and {{metrics}} dependencies. I was wondering 
> what effect are there to Cassandra functionality. Where can I help?
> Attached file is my configuration example.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12790) Support InfluxDb metrics reporter config

[Carl Yeksigian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Yeksigian updated CASSANDRA-12790:
---
Fix Version/s: (was: 3.10)

> Support InfluxDb metrics reporter config
> 
>
> Key: CASSANDRA-12790
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12790
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Configuration, Packaging
> Environment: Ubuntu 14.04
> Oracle Java 1.8.0_65
> Cassandra 3.0.9
>Reporter: Achmad Nasirudin Sandi
>Priority: Minor
>  Labels: easyfix, easytest, newbie
> Attachments: metrics-reporter-config-influx.yaml
>
>
> InfluxDb is a great tool for storing metrics. The main advantage over 
> graphite protocol is its tag mechanism that allows host info to be excluded 
> from metric name.
> I managed to send metrics into InfluxDb by putting 
> {{metrics-influxdb-1.1.7.jar}} into {{lib}} folder and replacing these 
> dependencies jar:
>  
> {code}
> commons-codec-1.2.jar -> commons-codec-1.9.jar
> reporter-config3-3.0.0.jar -> reporter-config3-3.0.3.jar
> reporter-config-base-3.0.0.jar -> reporter-config-base-3.0.3.jar
> {code}
> Although it seems that dropwizard team is working on official [package for 
> InfluxDb|https://github.com/dropwizard/metrics/tree/4.0-development], it 
> would be great it those libs are available in default distribution by 
> upgrading {{commons-codec}} and {{metrics}} dependencies. I was wondering 
> what effect are there to Cassandra functionality. Where can I help?
> Attached file is my configuration example.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12788) AbstractReplicationStrategy is not extendable from outside the package

[Jeremiah Jordan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeremiah Jordan updated CASSANDRA-12788:

Issue Type: New Feature  (was: Bug)

> AbstractReplicationStrategy is not extendable from outside the package
> --
>
> Key: CASSANDRA-12788
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12788
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Kurt Greaves
> Attachments: 12788.patch
>
>
> The constructor for ARS is package-private (no modifier), which means that 
> custom RS implementations cannot call it. That is, calling super(...) fails 
> unless you create your replication strategy inside 
> org.apache.cassandra.locator. This obviously isn't ideal.
> I propose adding protected to the constructor in ARS. The attached patch was 
> against trunk however I believe it should work for 2.1, 2.2, 3.0, and 3.x.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-12144) Undeletable / duplicate rows after upgrading from 2.2.4 to 3.0.7

[Jakub Lida (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15575533#comment-15575533
 ] 

Jakub Lida edited comment on CASSANDRA-12144 at 10/14/16 2:59 PM:
--

Do you know if this bug also affects upgrading from 2.1.9 to 3.0.8?


was (Author: jlida):
Do you know if this bug also affects upgrading from 2.0.9 to 3.0.8?

> Undeletable / duplicate rows after upgrading from 2.2.4 to 3.0.7
> 
>
> Key: CASSANDRA-12144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12144
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stanislav Vishnevskiy
>Assignee: Alex Petrov
> Fix For: 3.0.9, 3.8
>
>
> We upgraded our cluster today and now have a some rows that refuse to delete.
> Here are some example traces.
> https://gist.github.com/vishnevskiy/36aa18c468344ea22d14f9fb9b99171d
> Even weirder.
> Updating the row and querying it back results in 2 rows even though the id is 
> the clustering key.
> {noformat}
> user_id| id | since| type
> ---++--+--
> 116138050710536192 | 153047019424972800 | null |0
> 116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> And then deleting it again only removes the new one.
> {noformat}
> cqlsh:discord_relationships> DELETE FROM relationships WHERE user_id = 
> 116138050710536192 AND id = 153047019424972800;
> cqlsh:discord_relationships> SELECT * FROM relationships WHERE user_id = 
> 116138050710536192 AND id = 153047019424972800;
>  user_id| id | since| type
> ++--+--
>  116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> We tried repairing, compacting, scrubbing. No Luck.
> Not sure what to do. Is anyone aware of this?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12144) Undeletable / duplicate rows after upgrading from 2.2.4 to 3.0.7

[Alex Petrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15575565#comment-15575565
 ] 

Alex Petrov commented on CASSANDRA-12144:
-

Yes, {{3.0.8}} is affected. You should upgrade to {{3.0.9}} for correct 
behaviour. 

> Undeletable / duplicate rows after upgrading from 2.2.4 to 3.0.7
> 
>
> Key: CASSANDRA-12144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12144
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stanislav Vishnevskiy
>Assignee: Alex Petrov
> Fix For: 3.0.9, 3.8
>
>
> We upgraded our cluster today and now have a some rows that refuse to delete.
> Here are some example traces.
> https://gist.github.com/vishnevskiy/36aa18c468344ea22d14f9fb9b99171d
> Even weirder.
> Updating the row and querying it back results in 2 rows even though the id is 
> the clustering key.
> {noformat}
> user_id| id | since| type
> ---++--+--
> 116138050710536192 | 153047019424972800 | null |0
> 116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> And then deleting it again only removes the new one.
> {noformat}
> cqlsh:discord_relationships> DELETE FROM relationships WHERE user_id = 
> 116138050710536192 AND id = 153047019424972800;
> cqlsh:discord_relationships> SELECT * FROM relationships WHERE user_id = 
> 116138050710536192 AND id = 153047019424972800;
>  user_id| id | since| type
> ++--+--
>  116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> We tried repairing, compacting, scrubbing. No Luck.
> Not sure what to do. Is anyone aware of this?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12144) Undeletable / duplicate rows after upgrading from 2.2.4 to 3.0.7

[Jakub Lida (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15575533#comment-15575533
 ] 

Jakub Lida commented on CASSANDRA-12144:


Do you know if this bug also affects upgrading from 2.0.9 to 3.0.8?

> Undeletable / duplicate rows after upgrading from 2.2.4 to 3.0.7
> 
>
> Key: CASSANDRA-12144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12144
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stanislav Vishnevskiy
>Assignee: Alex Petrov
> Fix For: 3.0.9, 3.8
>
>
> We upgraded our cluster today and now have a some rows that refuse to delete.
> Here are some example traces.
> https://gist.github.com/vishnevskiy/36aa18c468344ea22d14f9fb9b99171d
> Even weirder.
> Updating the row and querying it back results in 2 rows even though the id is 
> the clustering key.
> {noformat}
> user_id| id | since| type
> ---++--+--
> 116138050710536192 | 153047019424972800 | null |0
> 116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> And then deleting it again only removes the new one.
> {noformat}
> cqlsh:discord_relationships> DELETE FROM relationships WHERE user_id = 
> 116138050710536192 AND id = 153047019424972800;
> cqlsh:discord_relationships> SELECT * FROM relationships WHERE user_id = 
> 116138050710536192 AND id = 153047019424972800;
>  user_id| id | since| type
> ++--+--
>  116138050710536192 | 153047019424972800 | 2016-05-30 14:53:08+ |2
> {noformat}
> We tried repairing, compacting, scrubbing. No Luck.
> Not sure what to do. Is anyone aware of this?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12268) Make MV Index creation robust for wide referent rows

[Carl Yeksigian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Yeksigian updated CASSANDRA-12268:
---
Fix Version/s: (was: 3.0.x)
   (was: 3.x)
   4.0
   3.10
   3.0.10

> Make MV Index creation robust for wide referent rows
> 
>
> Key: CASSANDRA-12268
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12268
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Jonathan Shook
>Assignee: Carl Yeksigian
> Fix For: 3.0.10, 3.10, 4.0
>
> Attachments: 12268.py
>
>
> When creating an index for a materialized view for extant data, heap pressure 
> is very dependent on the cardinality of of rows associated with each index 
> value. With the way that per-index value rows are created within the index, 
> this can cause unbounded heap pressure, which can cause OOM. This appears to 
> be a side-effect of how each index row is applied atomically as with batches.
> The commit logs can accumulate enough during the process to prevent the node 
> from being restarted. Given that this occurs during global index creation, 
> this can happen on multiple nodes, making stable recovery of a node set 
> difficult, as co-replicas become unavailable to assist in back-filling data 
> from commitlogs.
> While it is understandable that you want to avoid having relatively wide rows 
>  even in materialized views, this represents a particularly difficult 
> scenario for triage.
> The basic recommendation for improving this is to sub-group the index 
> creation into smaller chunks internally, providing a maximal bound against 
> the heap pressure when it is needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11803) Creating a materialized view on a table with "token" column breaks the cluster

[Carl Yeksigian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Yeksigian updated CASSANDRA-11803:
---
Fix Version/s: (was: 3.0.x)
   4.0
   3.10
   3.0.10

> Creating a materialized view on a table with "token" column breaks the cluster
> --
>
> Key: CASSANDRA-11803
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11803
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Kernel:
> Linux 4.4.8-20.46.amzn1.x86_64
> Java:
> Java OpenJDK Runtime Environment (build 1.8.0_91-b14)
> OpenJDK 64-Bit Server VM (build 25.91-b14, mixed mode)
> Cassandra: 
> datastax-ddc-3.3.0-1.noarch
> datastax-ddc-tools-3.3.0-1.noarch
>Reporter: Victor Trac
>Assignee: Carl Yeksigian
> Fix For: 3.0.10, 3.10, 4.0
>
>
> On a new Cassandra cluster, if we create a table with a field called "token" 
> (with quotes) and then create a materialized view that uses "token", the 
> cluster breaks. A ServerError is returned, and no further nodetool operations 
> on the sstables work. Restarting the Cassandra server will also fail. It 
> seems like the entire cluster is hosed.
> We tried this on Cassandra 3.3 and 3.5. 
> Here's how to produce (on an new, empty cassandra 3.5 docker container):
> {code}
> [cqlsh 5.0.1 | Cassandra 3.5 | CQL spec 3.4.0 | Native protocol v4]
> Use HELP for help.
> cqlsh> CREATE KEYSPACE account WITH REPLICATION = { 'class' : 
> 'SimpleStrategy', 'replication_factor' : 1 };
> cqlsh> CREATE TABLE account.session  (
>...   "token" blob,
>...   account_id uuid,
>...   PRIMARY KEY("token")
>... )WITH compaction={'class': 'LeveledCompactionStrategy'} AND
>...   compression={'sstable_compression': 'LZ4Compressor'};
> cqlsh> CREATE MATERIALIZED VIEW account.account_session AS
>...SELECT account_id,"token" FROM account.session
>...WHERE "token" IS NOT NULL and account_id IS NOT NULL
>...PRIMARY KEY (account_id, "token");
> ServerError:  message="java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.exceptions.SyntaxException: line 1:25 no viable 
> alternative at input 'FROM' (SELECT account_id, token [FROM]...)">
> cqlsh> drop table account.session;
> ServerError:  message="java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.exceptions.SyntaxException: line 1:25 no viable 
> alternative at input 'FROM' (SELECT account_id, token [FROM]...)">
> {code}
> When any sstable*, nodetool, or when the Cassandra process is restarted, this 
> is emitted on startup and Cassandra exits (copied from a server w/ data):
> {code}
> INFO  [main] 2016-05-12 23:25:30,074 ColumnFamilyStore.java:395 - 
> Initializing system_schema.indexes
> DEBUG [SSTableBatchOpen:1] 2016-05-12 23:25:30,075 SSTableReader.java:480 - 
> Opening 
> /mnt/cassandra/data/system_schema/indexes-0feb57ac311f382fba6d9024d305702f/ma-4-big
>  (91 bytes)
> ERROR [main] 2016-05-12 23:25:30,143 CassandraDaemon.java:697 - Exception 
> encountered during startup
> org.apache.cassandra.exceptions.SyntaxException: line 1:59 no viable 
> alternative at input 'FROM' (..., expire_at, last_used, token [FROM]...)
> at 
> org.apache.cassandra.cql3.ErrorCollector.throwFirstSyntaxError(ErrorCollector.java:101)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.cql3.CQLFragmentParser.parseAnyUnhandled(CQLFragmentParser.java:80)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.cql3.QueryProcessor.parseStatement(QueryProcessor.java:512)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchView(SchemaKeyspace.java:1128)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchViews(SchemaKeyspace.java:1092)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:903)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:879)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:867)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:134) 
> ~[apache-cassandra-3.5.0.jar:3.5.0]
> at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:124) 
> ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:229) 
> [apache-cassandra-3.5.0.jar:3.5.0]
> at 
> 

[jira] [Updated] (CASSANDRA-11803) Creating a materialized view on a table with "token" column breaks the cluster

[Carl Yeksigian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Yeksigian updated CASSANDRA-11803:
---
Resolution: Fixed
Status: Resolved  (was: Ready to Commit)

Thanks for the review [~ifesdjeen].

Committed as 
[153583b|https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=commit;h=153583be55e2a0bba74102bf1d5fc7a79d314b1f].

> Creating a materialized view on a table with "token" column breaks the cluster
> --
>
> Key: CASSANDRA-11803
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11803
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Kernel:
> Linux 4.4.8-20.46.amzn1.x86_64
> Java:
> Java OpenJDK Runtime Environment (build 1.8.0_91-b14)
> OpenJDK 64-Bit Server VM (build 25.91-b14, mixed mode)
> Cassandra: 
> datastax-ddc-3.3.0-1.noarch
> datastax-ddc-tools-3.3.0-1.noarch
>Reporter: Victor Trac
>Assignee: Carl Yeksigian
> Fix For: 3.0.x
>
>
> On a new Cassandra cluster, if we create a table with a field called "token" 
> (with quotes) and then create a materialized view that uses "token", the 
> cluster breaks. A ServerError is returned, and no further nodetool operations 
> on the sstables work. Restarting the Cassandra server will also fail. It 
> seems like the entire cluster is hosed.
> We tried this on Cassandra 3.3 and 3.5. 
> Here's how to produce (on an new, empty cassandra 3.5 docker container):
> {code}
> [cqlsh 5.0.1 | Cassandra 3.5 | CQL spec 3.4.0 | Native protocol v4]
> Use HELP for help.
> cqlsh> CREATE KEYSPACE account WITH REPLICATION = { 'class' : 
> 'SimpleStrategy', 'replication_factor' : 1 };
> cqlsh> CREATE TABLE account.session  (
>...   "token" blob,
>...   account_id uuid,
>...   PRIMARY KEY("token")
>... )WITH compaction={'class': 'LeveledCompactionStrategy'} AND
>...   compression={'sstable_compression': 'LZ4Compressor'};
> cqlsh> CREATE MATERIALIZED VIEW account.account_session AS
>...SELECT account_id,"token" FROM account.session
>...WHERE "token" IS NOT NULL and account_id IS NOT NULL
>...PRIMARY KEY (account_id, "token");
> ServerError:  message="java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.exceptions.SyntaxException: line 1:25 no viable 
> alternative at input 'FROM' (SELECT account_id, token [FROM]...)">
> cqlsh> drop table account.session;
> ServerError:  message="java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.exceptions.SyntaxException: line 1:25 no viable 
> alternative at input 'FROM' (SELECT account_id, token [FROM]...)">
> {code}
> When any sstable*, nodetool, or when the Cassandra process is restarted, this 
> is emitted on startup and Cassandra exits (copied from a server w/ data):
> {code}
> INFO  [main] 2016-05-12 23:25:30,074 ColumnFamilyStore.java:395 - 
> Initializing system_schema.indexes
> DEBUG [SSTableBatchOpen:1] 2016-05-12 23:25:30,075 SSTableReader.java:480 - 
> Opening 
> /mnt/cassandra/data/system_schema/indexes-0feb57ac311f382fba6d9024d305702f/ma-4-big
>  (91 bytes)
> ERROR [main] 2016-05-12 23:25:30,143 CassandraDaemon.java:697 - Exception 
> encountered during startup
> org.apache.cassandra.exceptions.SyntaxException: line 1:59 no viable 
> alternative at input 'FROM' (..., expire_at, last_used, token [FROM]...)
> at 
> org.apache.cassandra.cql3.ErrorCollector.throwFirstSyntaxError(ErrorCollector.java:101)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.cql3.CQLFragmentParser.parseAnyUnhandled(CQLFragmentParser.java:80)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.cql3.QueryProcessor.parseStatement(QueryProcessor.java:512)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchView(SchemaKeyspace.java:1128)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchViews(SchemaKeyspace.java:1092)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:903)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:879)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:867)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:134) 
> ~[apache-cassandra-3.5.0.jar:3.5.0]
> at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:124) 
> ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> 

[3/8] cassandra git commit: Preserve quoted reserved keyword column names in MV creation

[carl]

Preserve quoted reserved keyword column names in MV creation

Patch by Carl Yeksigian; reviewed by Alex Petrov for CASSANDRA-11803


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/153583be
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/153583be
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/153583be

Branch: refs/heads/trunk
Commit: 153583be55e2a0bba74102bf1d5fc7a79d314b1f
Parents: a889852
Author: Carl Yeksigian 
Authored: Fri Oct 14 10:12:14 2016 -0400
Committer: Carl Yeksigian 
Committed: Fri Oct 14 10:28:00 2016 -0400

--
 CHANGES.txt |   1 +
 .../apache/cassandra/cql3/ColumnIdentifier.java |   3 +-
 src/java/org/apache/cassandra/cql3/Cql.g|   2 +
 .../apache/cassandra/cql3/ReservedKeywords.java | 118 +++
 .../org/apache/cassandra/cql3/ViewTest.java |  20 
 .../db/ColumnFamilyStoreCQLHelperTest.java  |   4 +-
 6 files changed, 145 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/153583be/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f291674..e82eedd 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.10
+ * Preserve quoted reserved keyword column names in MV creation 
(CASSANDRA-11803)
  * nodetool stopdaemon errors out (CASSANDRA-12646)
  * Split materialized view mutations on build to prevent OOM (CASSANDRA-12268)
  * mx4j does not work in 3.0.8 (CASSANDRA-12274)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/153583be/src/java/org/apache/cassandra/cql3/ColumnIdentifier.java
--
diff --git a/src/java/org/apache/cassandra/cql3/ColumnIdentifier.java 
b/src/java/org/apache/cassandra/cql3/ColumnIdentifier.java
index 1e25b23..ceb81be 100644
--- a/src/java/org/apache/cassandra/cql3/ColumnIdentifier.java
+++ b/src/java/org/apache/cassandra/cql3/ColumnIdentifier.java
@@ -362,8 +362,9 @@ public class ColumnIdentifier extends Selectable implements 
IMeasurableMemory, C
 
 public static String maybeQuote(String text)
 {
-if (UNQUOTED_IDENTIFIER.matcher(text).matches())
+if (UNQUOTED_IDENTIFIER.matcher(text).matches() && 
!ReservedKeywords.isReserved(text))
 return text;
+
 return '"' + 
PATTERN_DOUBLE_QUOTE.matcher(text).replaceAll(Matcher.quoteReplacement("\"\"")) 
+ '"';
 }
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/153583be/src/java/org/apache/cassandra/cql3/Cql.g
--
diff --git a/src/java/org/apache/cassandra/cql3/Cql.g 
b/src/java/org/apache/cassandra/cql3/Cql.g
index d50b979..3123877 100644
--- a/src/java/org/apache/cassandra/cql3/Cql.g
+++ b/src/java/org/apache/cassandra/cql3/Cql.g
@@ -1634,6 +1634,8 @@ basic_unreserved_keyword returns [String str]
 ;
 
 // Case-insensitive keywords
+// When adding a new reserved keyword, add entry to 
o.a.c.cql3.ReservedKeywords as well
+// When adding a new unreserved keyword, add entry to list above
 K_SELECT:  S E L E C T;
 K_FROM:F R O M;
 K_AS:  A S;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/153583be/src/java/org/apache/cassandra/cql3/ReservedKeywords.java
--
diff --git a/src/java/org/apache/cassandra/cql3/ReservedKeywords.java 
b/src/java/org/apache/cassandra/cql3/ReservedKeywords.java
new file mode 100644
index 000..91b7e61
--- /dev/null
+++ b/src/java/org/apache/cassandra/cql3/ReservedKeywords.java
@@ -0,0 +1,118 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.cql3;
+
+import java.util.Set;
+
+import com.google.common.collect.ImmutableSet;
+
+class ReservedKeywords
+{
+private static final String[] reservedKeywords = new String[]
+ {

[5/8] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.X

[carl]

Merge branch 'cassandra-3.0' into cassandra-3.X


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/206a7bba
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/206a7bba
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/206a7bba

Branch: refs/heads/trunk
Commit: 206a7bbac70d8ad5f27eff7b22ccfc8ab703a31b
Parents: 771a1a2 153583b
Author: Carl Yeksigian 
Authored: Fri Oct 14 10:28:58 2016 -0400
Committer: Carl Yeksigian 
Committed: Fri Oct 14 10:28:58 2016 -0400

--
 CHANGES.txt |   1 +
 src/antlr/Lexer.g   |   2 +
 src/antlr/Parser.g  |   2 +-
 .../apache/cassandra/cql3/ColumnIdentifier.java |   2 +-
 .../apache/cassandra/cql3/ReservedKeywords.java | 118 +++
 .../org/apache/cassandra/cql3/ViewTest.java |  20 
 .../db/ColumnFamilyStoreCQLHelperTest.java  |   4 +-
 7 files changed, 145 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/206a7bba/CHANGES.txt
--
diff --cc CHANGES.txt
index bc5dcd1,e82eedd..0045ba5
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,85 -1,5 +1,86 @@@
 -3.0.10
 +3.10
 + * cqlsh fails to format collections when using aliases (CASSANDRA-11534)
 + * Check for hash conflicts in prepared statements (CASSANDRA-12733)
 + * Exit query parsing upon first error (CASSANDRA-12598)
 + * Fix cassandra-stress to use single seed in UUID generation 
(CASSANDRA-12729)
 + * CQLSSTableWriter does not allow Update statement (CASSANDRA-12450)
 + * Config class uses boxed types but DD exposes primitive types 
(CASSANDRA-12199)
 + * Add pre- and post-shutdown hooks to Storage Service (CASSANDRA-12461)
 + * Add hint delivery metrics (CASSANDRA-12693)
 + * Remove IndexInfo cache from FileIndexInfoRetriever (CASSANDRA-12731)
 + * ColumnIndex does not reuse buffer (CASSANDRA-12502)
 + * cdc column addition still breaks schema migration tasks (CASSANDRA-12697)
 + * Upgrade metrics-reporter dependencies (CASSANDRA-12089)
 + * Tune compaction thread count via nodetool (CASSANDRA-12248)
 + * Add +=/-= shortcut syntax for update queries (CASSANDRA-12232)
 + * Include repair session IDs in repair start message (CASSANDRA-12532)
 + * Add a blocking task to Index, run before joining the ring (CASSANDRA-12039)
 + * Fix NPE when using CQLSSTableWriter (CASSANDRA-12667)
 + * Support optional backpressure strategies at the coordinator 
(CASSANDRA-9318)
 + * Make randompartitioner work with new vnode allocation (CASSANDRA-12647)
 + * Fix cassandra-stress graphing (CASSANDRA-12237)
 + * Allow filtering on partition key columns for queries without secondary 
indexes (CASSANDRA-11031)
 + * Fix Cassandra Stress reporting thread model and precision (CASSANDRA-12585)
 + * Add JMH benchmarks.jar (CASSANDRA-12586)
 + * Add row offset support to SASI (CASSANDRA-11990)
 + * Cleanup uses of AlterTableStatementColumn (CASSANDRA-12567)
 + * Add keep-alive to streaming (CASSANDRA-11841)
 + * Tracing payload is passed through newSession(..) (CASSANDRA-11706)
 + * avoid deleting non existing sstable files and improve related log messages 
(CASSANDRA-12261)
 + * json/yaml output format for nodetool compactionhistory (CASSANDRA-12486)
 + * Retry all internode messages once after a connection is
 +   closed and reopened (CASSANDRA-12192)
 + * Add support to rebuild from targeted replica (CASSANDRA-9875)
 + * Add sequence distribution type to cassandra stress (CASSANDRA-12490)
 + * "SELECT * FROM foo LIMIT ;" does not error out (CASSANDRA-12154)
 + * Define executeLocally() at the ReadQuery Level (CASSANDRA-12474)
 + * Extend read/write failure messages with a map of replica addresses
 +   to error codes in the v5 native protocol (CASSANDRA-12311)
 + * Fix rebuild of SASI indexes with existing index files (CASSANDRA-12374)
 + * Let DatabaseDescriptor not implicitly startup services (CASSANDRA-9054, 
12550)
 + * Fix clustering indexes in presence of static columns in SASI 
(CASSANDRA-12378)
 + * Fix queries on columns with reversed type on SASI indexes (CASSANDRA-12223)
 + * Added slow query log (CASSANDRA-12403)
 + * Count full coordinated request against timeout (CASSANDRA-12256)
 + * Allow TTL with null value on insert and update (CASSANDRA-12216)
 + * Make decommission operation resumable (CASSANDRA-12008)
 + * Add support to one-way targeted repair (CASSANDRA-9876)
 + * Remove clientutil jar (CASSANDRA-11635)
 + * Fix compaction throughput throttle (CASSANDRA-12366, CASSANDRA-12717)
 + * Delay releasing Memtable memory on flush until PostFlush has finished 
running (CASSANDRA-12358)
 + * Cassandra stress should dump all setting on startup (CASSANDRA-11914)
 + * 

[6/8] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.X

[carl]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/206a7bba/src/java/org/apache/cassandra/cql3/ColumnIdentifier.java
--
diff --cc src/java/org/apache/cassandra/cql3/ColumnIdentifier.java
index 7a23231,ceb81be..70f7590
--- a/src/java/org/apache/cassandra/cql3/ColumnIdentifier.java
+++ b/src/java/org/apache/cassandra/cql3/ColumnIdentifier.java
@@@ -218,11 -232,139 +218,11 @@@ public class ColumnIdentifier implement
  return ByteBufferUtil.compareUnsigned(this.bytes, that.bytes);
  }
  
 -/**
 - * Because Thrift-created tables may have a non-text comparator, we 
cannot determine the proper 'key' until
 - * we know the comparator. ColumnIdentifier.Raw is a placeholder that can 
be converted to a real ColumnIdentifier
 - * once the comparator is known with prepare(). This should only be used 
with identifiers that are actual
 - * column names. See CASSANDRA-8178 for more background.
 - */
 -public static interface Raw extends Selectable.Raw
 -{
 -
 -public ColumnIdentifier prepare(CFMetaData cfm);
 -
 -/**
 - * Returns a string representation of the identifier that is safe to 
use directly in CQL queries.
 - * In necessary, the string will be double-quoted, and any quotes 
inside the string will be escaped.
 - */
 -public String toCQLString();
 -}
 -
 -public static class Literal implements Raw
 -{
 -private final String rawText;
 -private final String text;
 -
 -public Literal(String rawText, boolean keepCase)
 -{
 -this.rawText = rawText;
 -this.text =  keepCase ? rawText : rawText.toLowerCase(Locale.US);
 -}
 -
 -public ColumnIdentifier prepare(CFMetaData cfm)
 -{
 -if (!cfm.isStaticCompactTable())
 -return getInterned(text, true);
 -
 -AbstractType thriftColumnNameType = cfm.thriftColumnNameType();
 -if (thriftColumnNameType instanceof UTF8Type)
 -return getInterned(text, true);
 -
 -// We have a Thrift-created table with a non-text comparator. 
Check if we have a match column, otherwise assume we should use
 -// thriftColumnNameType
 -ByteBuffer bufferName = ByteBufferUtil.bytes(text);
 -for (ColumnDefinition def : cfm.allColumns())
 -{
 -if (def.name.bytes.equals(bufferName))
 -return def.name;
 -}
 -return getInterned(thriftColumnNameType, 
thriftColumnNameType.fromString(rawText), text);
 -}
 -
 -public boolean processesSelection()
 -{
 -return false;
 -}
 -
 -@Override
 -public final int hashCode()
 -{
 -return text.hashCode();
 -}
 -
 -@Override
 -public final boolean equals(Object o)
 -{
 -if(!(o instanceof Literal))
 -return false;
 -
 -Literal that = (Literal) o;
 -return text.equals(that.text);
 -}
 -
 -@Override
 -public String toString()
 -{
 -return text;
 -}
 -
 -public String toCQLString()
 -{
 -return maybeQuote(text);
 -}
 -}
 -
 -public static class ColumnIdentifierValue implements Raw
 -{
 -private final ColumnIdentifier identifier;
 -
 -public ColumnIdentifierValue(ColumnIdentifier identifier)
 -{
 -this.identifier = identifier;
 -}
 -
 -public ColumnIdentifier prepare(CFMetaData cfm)
 -{
 -return identifier;
 -}
 -
 -public boolean processesSelection()
 -{
 -return false;
 -}
 -
 -@Override
 -public final int hashCode()
 -{
 -return identifier.hashCode();
 -}
 -
 -@Override
 -public final boolean equals(Object o)
 -{
 -if(!(o instanceof ColumnIdentifierValue))
 -return false;
 -ColumnIdentifierValue that = (ColumnIdentifierValue) o;
 -return identifier.equals(that.identifier);
 -}
 -
 -@Override
 -public String toString()
 -{
 -return identifier.toString();
 -}
 -
 -public String toCQLString()
 -{
 -return maybeQuote(identifier.text);
 -}
 -}
 -
 +@VisibleForTesting
  public static String maybeQuote(String text)
  {
- if (UNQUOTED_IDENTIFIER.matcher(text).matches())
+ if (UNQUOTED_IDENTIFIER.matcher(text).matches() && 
!ReservedKeywords.isReserved(text))
  return text;
 -
 -return '"' + 
PATTERN_DOUBLE_QUOTE.matcher(text).replaceAll(Matcher.quoteReplacement("\"\"")) 
+ '"';
 +return '"' + 

[7/8] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.X

[carl]

Merge branch 'cassandra-3.0' into cassandra-3.X


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/206a7bba
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/206a7bba
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/206a7bba

Branch: refs/heads/cassandra-3.X
Commit: 206a7bbac70d8ad5f27eff7b22ccfc8ab703a31b
Parents: 771a1a2 153583b
Author: Carl Yeksigian 
Authored: Fri Oct 14 10:28:58 2016 -0400
Committer: Carl Yeksigian 
Committed: Fri Oct 14 10:28:58 2016 -0400

--
 CHANGES.txt |   1 +
 src/antlr/Lexer.g   |   2 +
 src/antlr/Parser.g  |   2 +-
 .../apache/cassandra/cql3/ColumnIdentifier.java |   2 +-
 .../apache/cassandra/cql3/ReservedKeywords.java | 118 +++
 .../org/apache/cassandra/cql3/ViewTest.java |  20 
 .../db/ColumnFamilyStoreCQLHelperTest.java  |   4 +-
 7 files changed, 145 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/206a7bba/CHANGES.txt
--
diff --cc CHANGES.txt
index bc5dcd1,e82eedd..0045ba5
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,85 -1,5 +1,86 @@@
 -3.0.10
 +3.10
 + * cqlsh fails to format collections when using aliases (CASSANDRA-11534)
 + * Check for hash conflicts in prepared statements (CASSANDRA-12733)
 + * Exit query parsing upon first error (CASSANDRA-12598)
 + * Fix cassandra-stress to use single seed in UUID generation 
(CASSANDRA-12729)
 + * CQLSSTableWriter does not allow Update statement (CASSANDRA-12450)
 + * Config class uses boxed types but DD exposes primitive types 
(CASSANDRA-12199)
 + * Add pre- and post-shutdown hooks to Storage Service (CASSANDRA-12461)
 + * Add hint delivery metrics (CASSANDRA-12693)
 + * Remove IndexInfo cache from FileIndexInfoRetriever (CASSANDRA-12731)
 + * ColumnIndex does not reuse buffer (CASSANDRA-12502)
 + * cdc column addition still breaks schema migration tasks (CASSANDRA-12697)
 + * Upgrade metrics-reporter dependencies (CASSANDRA-12089)
 + * Tune compaction thread count via nodetool (CASSANDRA-12248)
 + * Add +=/-= shortcut syntax for update queries (CASSANDRA-12232)
 + * Include repair session IDs in repair start message (CASSANDRA-12532)
 + * Add a blocking task to Index, run before joining the ring (CASSANDRA-12039)
 + * Fix NPE when using CQLSSTableWriter (CASSANDRA-12667)
 + * Support optional backpressure strategies at the coordinator 
(CASSANDRA-9318)
 + * Make randompartitioner work with new vnode allocation (CASSANDRA-12647)
 + * Fix cassandra-stress graphing (CASSANDRA-12237)
 + * Allow filtering on partition key columns for queries without secondary 
indexes (CASSANDRA-11031)
 + * Fix Cassandra Stress reporting thread model and precision (CASSANDRA-12585)
 + * Add JMH benchmarks.jar (CASSANDRA-12586)
 + * Add row offset support to SASI (CASSANDRA-11990)
 + * Cleanup uses of AlterTableStatementColumn (CASSANDRA-12567)
 + * Add keep-alive to streaming (CASSANDRA-11841)
 + * Tracing payload is passed through newSession(..) (CASSANDRA-11706)
 + * avoid deleting non existing sstable files and improve related log messages 
(CASSANDRA-12261)
 + * json/yaml output format for nodetool compactionhistory (CASSANDRA-12486)
 + * Retry all internode messages once after a connection is
 +   closed and reopened (CASSANDRA-12192)
 + * Add support to rebuild from targeted replica (CASSANDRA-9875)
 + * Add sequence distribution type to cassandra stress (CASSANDRA-12490)
 + * "SELECT * FROM foo LIMIT ;" does not error out (CASSANDRA-12154)
 + * Define executeLocally() at the ReadQuery Level (CASSANDRA-12474)
 + * Extend read/write failure messages with a map of replica addresses
 +   to error codes in the v5 native protocol (CASSANDRA-12311)
 + * Fix rebuild of SASI indexes with existing index files (CASSANDRA-12374)
 + * Let DatabaseDescriptor not implicitly startup services (CASSANDRA-9054, 
12550)
 + * Fix clustering indexes in presence of static columns in SASI 
(CASSANDRA-12378)
 + * Fix queries on columns with reversed type on SASI indexes (CASSANDRA-12223)
 + * Added slow query log (CASSANDRA-12403)
 + * Count full coordinated request against timeout (CASSANDRA-12256)
 + * Allow TTL with null value on insert and update (CASSANDRA-12216)
 + * Make decommission operation resumable (CASSANDRA-12008)
 + * Add support to one-way targeted repair (CASSANDRA-9876)
 + * Remove clientutil jar (CASSANDRA-11635)
 + * Fix compaction throughput throttle (CASSANDRA-12366, CASSANDRA-12717)
 + * Delay releasing Memtable memory on flush until PostFlush has finished 
running (CASSANDRA-12358)
 + * Cassandra stress should dump all setting on startup (CASSANDRA-11914)
 

[8/8] cassandra git commit: Merge branch 'cassandra-3.X' into trunk

[carl]

Merge branch 'cassandra-3.X' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/17bd6adc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/17bd6adc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/17bd6adc

Branch: refs/heads/trunk
Commit: 17bd6adc8f35dcb44e3e4848e267c256c0c59182
Parents: 56bc77c 206a7bb
Author: Carl Yeksigian 
Authored: Fri Oct 14 10:29:48 2016 -0400
Committer: Carl Yeksigian 
Committed: Fri Oct 14 10:29:48 2016 -0400

--
 CHANGES.txt |   1 +
 src/antlr/Lexer.g   |   2 +
 src/antlr/Parser.g  |   2 +-
 .../apache/cassandra/cql3/ColumnIdentifier.java |   2 +-
 .../apache/cassandra/cql3/ReservedKeywords.java | 118 +++
 .../org/apache/cassandra/cql3/ViewTest.java |  20 
 .../db/ColumnFamilyStoreCQLHelperTest.java  |   4 +-
 7 files changed, 145 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/17bd6adc/CHANGES.txt
--

[2/8] cassandra git commit: Preserve quoted reserved keyword column names in MV creation

[carl]

Preserve quoted reserved keyword column names in MV creation

Patch by Carl Yeksigian; reviewed by Alex Petrov for CASSANDRA-11803


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/153583be
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/153583be
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/153583be

Branch: refs/heads/cassandra-3.X
Commit: 153583be55e2a0bba74102bf1d5fc7a79d314b1f
Parents: a889852
Author: Carl Yeksigian 
Authored: Fri Oct 14 10:12:14 2016 -0400
Committer: Carl Yeksigian 
Committed: Fri Oct 14 10:28:00 2016 -0400

--
 CHANGES.txt |   1 +
 .../apache/cassandra/cql3/ColumnIdentifier.java |   3 +-
 src/java/org/apache/cassandra/cql3/Cql.g|   2 +
 .../apache/cassandra/cql3/ReservedKeywords.java | 118 +++
 .../org/apache/cassandra/cql3/ViewTest.java |  20 
 .../db/ColumnFamilyStoreCQLHelperTest.java  |   4 +-
 6 files changed, 145 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/153583be/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f291674..e82eedd 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.10
+ * Preserve quoted reserved keyword column names in MV creation 
(CASSANDRA-11803)
  * nodetool stopdaemon errors out (CASSANDRA-12646)
  * Split materialized view mutations on build to prevent OOM (CASSANDRA-12268)
  * mx4j does not work in 3.0.8 (CASSANDRA-12274)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/153583be/src/java/org/apache/cassandra/cql3/ColumnIdentifier.java
--
diff --git a/src/java/org/apache/cassandra/cql3/ColumnIdentifier.java 
b/src/java/org/apache/cassandra/cql3/ColumnIdentifier.java
index 1e25b23..ceb81be 100644
--- a/src/java/org/apache/cassandra/cql3/ColumnIdentifier.java
+++ b/src/java/org/apache/cassandra/cql3/ColumnIdentifier.java
@@ -362,8 +362,9 @@ public class ColumnIdentifier extends Selectable implements 
IMeasurableMemory, C
 
 public static String maybeQuote(String text)
 {
-if (UNQUOTED_IDENTIFIER.matcher(text).matches())
+if (UNQUOTED_IDENTIFIER.matcher(text).matches() && 
!ReservedKeywords.isReserved(text))
 return text;
+
 return '"' + 
PATTERN_DOUBLE_QUOTE.matcher(text).replaceAll(Matcher.quoteReplacement("\"\"")) 
+ '"';
 }
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/153583be/src/java/org/apache/cassandra/cql3/Cql.g
--
diff --git a/src/java/org/apache/cassandra/cql3/Cql.g 
b/src/java/org/apache/cassandra/cql3/Cql.g
index d50b979..3123877 100644
--- a/src/java/org/apache/cassandra/cql3/Cql.g
+++ b/src/java/org/apache/cassandra/cql3/Cql.g
@@ -1634,6 +1634,8 @@ basic_unreserved_keyword returns [String str]
 ;
 
 // Case-insensitive keywords
+// When adding a new reserved keyword, add entry to 
o.a.c.cql3.ReservedKeywords as well
+// When adding a new unreserved keyword, add entry to list above
 K_SELECT:  S E L E C T;
 K_FROM:F R O M;
 K_AS:  A S;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/153583be/src/java/org/apache/cassandra/cql3/ReservedKeywords.java
--
diff --git a/src/java/org/apache/cassandra/cql3/ReservedKeywords.java 
b/src/java/org/apache/cassandra/cql3/ReservedKeywords.java
new file mode 100644
index 000..91b7e61
--- /dev/null
+++ b/src/java/org/apache/cassandra/cql3/ReservedKeywords.java
@@ -0,0 +1,118 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.cql3;
+
+import java.util.Set;
+
+import com.google.common.collect.ImmutableSet;
+
+class ReservedKeywords
+{
+private static final String[] reservedKeywords = new String[]
+   

[4/8] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.X

[carl]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/206a7bba/src/java/org/apache/cassandra/cql3/ColumnIdentifier.java
--
diff --cc src/java/org/apache/cassandra/cql3/ColumnIdentifier.java
index 7a23231,ceb81be..70f7590
--- a/src/java/org/apache/cassandra/cql3/ColumnIdentifier.java
+++ b/src/java/org/apache/cassandra/cql3/ColumnIdentifier.java
@@@ -218,11 -232,139 +218,11 @@@ public class ColumnIdentifier implement
  return ByteBufferUtil.compareUnsigned(this.bytes, that.bytes);
  }
  
 -/**
 - * Because Thrift-created tables may have a non-text comparator, we 
cannot determine the proper 'key' until
 - * we know the comparator. ColumnIdentifier.Raw is a placeholder that can 
be converted to a real ColumnIdentifier
 - * once the comparator is known with prepare(). This should only be used 
with identifiers that are actual
 - * column names. See CASSANDRA-8178 for more background.
 - */
 -public static interface Raw extends Selectable.Raw
 -{
 -
 -public ColumnIdentifier prepare(CFMetaData cfm);
 -
 -/**
 - * Returns a string representation of the identifier that is safe to 
use directly in CQL queries.
 - * In necessary, the string will be double-quoted, and any quotes 
inside the string will be escaped.
 - */
 -public String toCQLString();
 -}
 -
 -public static class Literal implements Raw
 -{
 -private final String rawText;
 -private final String text;
 -
 -public Literal(String rawText, boolean keepCase)
 -{
 -this.rawText = rawText;
 -this.text =  keepCase ? rawText : rawText.toLowerCase(Locale.US);
 -}
 -
 -public ColumnIdentifier prepare(CFMetaData cfm)
 -{
 -if (!cfm.isStaticCompactTable())
 -return getInterned(text, true);
 -
 -AbstractType thriftColumnNameType = cfm.thriftColumnNameType();
 -if (thriftColumnNameType instanceof UTF8Type)
 -return getInterned(text, true);
 -
 -// We have a Thrift-created table with a non-text comparator. 
Check if we have a match column, otherwise assume we should use
 -// thriftColumnNameType
 -ByteBuffer bufferName = ByteBufferUtil.bytes(text);
 -for (ColumnDefinition def : cfm.allColumns())
 -{
 -if (def.name.bytes.equals(bufferName))
 -return def.name;
 -}
 -return getInterned(thriftColumnNameType, 
thriftColumnNameType.fromString(rawText), text);
 -}
 -
 -public boolean processesSelection()
 -{
 -return false;
 -}
 -
 -@Override
 -public final int hashCode()
 -{
 -return text.hashCode();
 -}
 -
 -@Override
 -public final boolean equals(Object o)
 -{
 -if(!(o instanceof Literal))
 -return false;
 -
 -Literal that = (Literal) o;
 -return text.equals(that.text);
 -}
 -
 -@Override
 -public String toString()
 -{
 -return text;
 -}
 -
 -public String toCQLString()
 -{
 -return maybeQuote(text);
 -}
 -}
 -
 -public static class ColumnIdentifierValue implements Raw
 -{
 -private final ColumnIdentifier identifier;
 -
 -public ColumnIdentifierValue(ColumnIdentifier identifier)
 -{
 -this.identifier = identifier;
 -}
 -
 -public ColumnIdentifier prepare(CFMetaData cfm)
 -{
 -return identifier;
 -}
 -
 -public boolean processesSelection()
 -{
 -return false;
 -}
 -
 -@Override
 -public final int hashCode()
 -{
 -return identifier.hashCode();
 -}
 -
 -@Override
 -public final boolean equals(Object o)
 -{
 -if(!(o instanceof ColumnIdentifierValue))
 -return false;
 -ColumnIdentifierValue that = (ColumnIdentifierValue) o;
 -return identifier.equals(that.identifier);
 -}
 -
 -@Override
 -public String toString()
 -{
 -return identifier.toString();
 -}
 -
 -public String toCQLString()
 -{
 -return maybeQuote(identifier.text);
 -}
 -}
 -
 +@VisibleForTesting
  public static String maybeQuote(String text)
  {
- if (UNQUOTED_IDENTIFIER.matcher(text).matches())
+ if (UNQUOTED_IDENTIFIER.matcher(text).matches() && 
!ReservedKeywords.isReserved(text))
  return text;
 -
 -return '"' + 
PATTERN_DOUBLE_QUOTE.matcher(text).replaceAll(Matcher.quoteReplacement("\"\"")) 
+ '"';
 +return '"' + 

[1/8] cassandra git commit: Preserve quoted reserved keyword column names in MV creation

[carl]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 a88985270 -> 153583be5
  refs/heads/cassandra-3.X 771a1a2fa -> 206a7bbac
  refs/heads/trunk 56bc77c4e -> 17bd6adc8


Preserve quoted reserved keyword column names in MV creation

Patch by Carl Yeksigian; reviewed by Alex Petrov for CASSANDRA-11803


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/153583be
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/153583be
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/153583be

Branch: refs/heads/cassandra-3.0
Commit: 153583be55e2a0bba74102bf1d5fc7a79d314b1f
Parents: a889852
Author: Carl Yeksigian 
Authored: Fri Oct 14 10:12:14 2016 -0400
Committer: Carl Yeksigian 
Committed: Fri Oct 14 10:28:00 2016 -0400

--
 CHANGES.txt |   1 +
 .../apache/cassandra/cql3/ColumnIdentifier.java |   3 +-
 src/java/org/apache/cassandra/cql3/Cql.g|   2 +
 .../apache/cassandra/cql3/ReservedKeywords.java | 118 +++
 .../org/apache/cassandra/cql3/ViewTest.java |  20 
 .../db/ColumnFamilyStoreCQLHelperTest.java  |   4 +-
 6 files changed, 145 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/153583be/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f291674..e82eedd 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.10
+ * Preserve quoted reserved keyword column names in MV creation 
(CASSANDRA-11803)
  * nodetool stopdaemon errors out (CASSANDRA-12646)
  * Split materialized view mutations on build to prevent OOM (CASSANDRA-12268)
  * mx4j does not work in 3.0.8 (CASSANDRA-12274)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/153583be/src/java/org/apache/cassandra/cql3/ColumnIdentifier.java
--
diff --git a/src/java/org/apache/cassandra/cql3/ColumnIdentifier.java 
b/src/java/org/apache/cassandra/cql3/ColumnIdentifier.java
index 1e25b23..ceb81be 100644
--- a/src/java/org/apache/cassandra/cql3/ColumnIdentifier.java
+++ b/src/java/org/apache/cassandra/cql3/ColumnIdentifier.java
@@ -362,8 +362,9 @@ public class ColumnIdentifier extends Selectable implements 
IMeasurableMemory, C
 
 public static String maybeQuote(String text)
 {
-if (UNQUOTED_IDENTIFIER.matcher(text).matches())
+if (UNQUOTED_IDENTIFIER.matcher(text).matches() && 
!ReservedKeywords.isReserved(text))
 return text;
+
 return '"' + 
PATTERN_DOUBLE_QUOTE.matcher(text).replaceAll(Matcher.quoteReplacement("\"\"")) 
+ '"';
 }
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/153583be/src/java/org/apache/cassandra/cql3/Cql.g
--
diff --git a/src/java/org/apache/cassandra/cql3/Cql.g 
b/src/java/org/apache/cassandra/cql3/Cql.g
index d50b979..3123877 100644
--- a/src/java/org/apache/cassandra/cql3/Cql.g
+++ b/src/java/org/apache/cassandra/cql3/Cql.g
@@ -1634,6 +1634,8 @@ basic_unreserved_keyword returns [String str]
 ;
 
 // Case-insensitive keywords
+// When adding a new reserved keyword, add entry to 
o.a.c.cql3.ReservedKeywords as well
+// When adding a new unreserved keyword, add entry to list above
 K_SELECT:  S E L E C T;
 K_FROM:F R O M;
 K_AS:  A S;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/153583be/src/java/org/apache/cassandra/cql3/ReservedKeywords.java
--
diff --git a/src/java/org/apache/cassandra/cql3/ReservedKeywords.java 
b/src/java/org/apache/cassandra/cql3/ReservedKeywords.java
new file mode 100644
index 000..91b7e61
--- /dev/null
+++ b/src/java/org/apache/cassandra/cql3/ReservedKeywords.java
@@ -0,0 +1,118 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.cql3;
+
+import java.util.Set;
+
+import 

[jira] [Commented] (CASSANDRA-12661) Make gc_log and gc_warn settable at runtime

[Edward Capriolo (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15575464#comment-15575464
 ] 

Edward Capriolo commented on CASSANDRA-12661:
-

Ping: Still applies cleanly. 

> Make gc_log and gc_warn settable at runtime
> ---
>
> Key: CASSANDRA-12661
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12661
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>Priority: Minor
>
> Changes:
> * Move gc_log_threshold_in_ms and gc_warn_threshold_in_ms close together in 
> the config
> * rename variables to match properties
> * add unit tests to ensure hybration
> * add unit tests to ensure variables are set propertly
> * minor perf (do not consturct string from buffer f not logging)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12627) Provide new seed providers

[Edward Capriolo (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15575487#comment-15575487
 ] 

Edward Capriolo commented on CASSANDRA-12627:
-

These are massively helpful in cloud environments as the user can provision 
without much hacking of the yaml file.

> Provide new seed providers
> --
>
> Key: CASSANDRA-12627
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12627
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>
> SeedProvider is plugable, however only one implementation exists.
> Changes:
> * Create a SeedProvider that reads properties from System properties or env
> * Provide a SeedProvider that scans ranges of IP addresses to find peers.
> * Refactor interface to abstract class because all seed providers must 
> provide a constructor that accepts Map 
> * correct error messages
> * Do not catch Exception use MultiCatch and catch typed exceptions



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12643) Estimated histograms tend to overflow

[Edward Capriolo (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15575478#comment-15575478
 ] 

Edward Capriolo commented on CASSANDRA-12643:
-

This patch would help administrators understand what overflows and why. It can 
be used to refine the algorithm to fail less. Comments appreciated.

> Estimated histograms tend to overflow
> -
>
> Key: CASSANDRA-12643
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12643
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10825) OverloadedException is untested

[Edward Capriolo (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15575459#comment-15575459
 ] 

Edward Capriolo commented on CASSANDRA-10825:
-

[~iamaleksey] Looping around on all my patch_available patches and being a 
pest. Do you need anything else from me?

> OverloadedException is untested
> ---
>
> Key: CASSANDRA-10825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10825
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Ariel Weisberg
>Assignee: Edward Capriolo
> Attachments: jmx-hint.png
>
>
> If you grep test/src and cassandra-dtest you will find that the string 
> OverloadedException doesn't appear anywhere.
> In CASSANDRA-10477 it was found that there were cases where Paxos should 
> back-pressure and throw OverloadedException but didn't.
> If OverloadedException is used for functional purposes then we should test 
> that it is thrown under expected conditions. If there are behaviors driven by 
> catching or tracking OverloadedException we should test those as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12252) Support RR and Speculative Retry for EACH_QUORUM reads

[Carl Yeksigian (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15575432#comment-15575432
 ] 

Carl Yeksigian commented on CASSANDRA-12252:


Since EACH_QUORUM is not a typical consistency level for users, I think it 
makes sense to try to continue to use the simple ReadCallback that we currently 
have which does not need to track each DC independently, only the number of 
responses, for all other consistency levels, and only add the complexity of a 
DC aware counter if we are using a DC-aware consistency level (possibly adding 
this support for LOCAL_* consistency levels as well).

For speculative retry, we can't select a single node to retry before we send 
the first set of requests - we need to know which DCs have failed to responded 
before we can properly choose the next node to try. In this case, we will need 
the information from the read callback to determine which DC hasn't responded 
yet, and then can choose the next replica to which to send an additional 
request. This can be done with a new SpeculativeRetry type since this will not 
be the usual case.

> Support RR and Speculative Retry for EACH_QUORUM reads
> --
>
> Key: CASSANDRA-12252
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12252
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Carl Yeksigian
>Priority: Minor
>
> We should properly support read repair and speculative retry for EACH_QUORUM 
> reads.
> To support these, we need to make sure that we count responses for each DC 
> separately, so that we can properly monitor the number of responses from 
> each. For speculative retry, we should try to replace each host that we are 
> retrying with one in the same DC; that will make sure that we actually have 
> the correct responses at the end if the speculative retries are successful.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12490) Add sequence distribution type to cassandra stress

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15575388#comment-15575388
 ] 

T Jake Luciani commented on CASSANDRA-12490:


Regarding the distribution issues I also noticed this and opened 
CASSANDRA-12744.   

> Add sequence distribution type to cassandra stress
> --
>
> Key: CASSANDRA-12490
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12490
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Ben Slater
>Assignee: Ben Slater
>Priority: Minor
> Fix For: 3.10
>
> Attachments: 12490-trunk.patch, 12490.yaml, cqlstress-seq-example.yaml
>
>
> When using the write command, cassandra stress sequentially generates seeds. 
> This ensures generated values don't overlap (unless the sequence wraps) 
> providing more predictable number of inserted records (and generating a base 
> set of data without wasted writes).
> When using a yaml stress spec there is no sequenced distribution available. 
> It think it would be useful to have this for doing initial load of data for 
> testing 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12490) Add sequence distribution type to cassandra stress

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15575377#comment-15575377
 ] 

T Jake Luciani commented on CASSANDRA-12490:


Ah you are right!  I'm not sure why but for runs with an older version of 
stress I was getting read errors and assumed it was validation but perhaps it's 
a different bug.

> Add sequence distribution type to cassandra stress
> --
>
> Key: CASSANDRA-12490
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12490
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Ben Slater
>Assignee: Ben Slater
>Priority: Minor
> Fix For: 3.10
>
> Attachments: 12490-trunk.patch, 12490.yaml, cqlstress-seq-example.yaml
>
>
> When using the write command, cassandra stress sequentially generates seeds. 
> This ensures generated values don't overlap (unless the sequence wraps) 
> providing more predictable number of inserted records (and generating a base 
> set of data without wasted writes).
> When using a yaml stress spec there is no sequenced distribution available. 
> It think it would be useful to have this for doing initial load of data for 
> testing 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12790) Support InfluxDb metrics reporter config

[Achmad Nasirudin Sandi (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15575051#comment-15575051
 ] 

Achmad Nasirudin Sandi commented on CASSANDRA-12790:


Thank you.

> Support InfluxDb metrics reporter config
> 
>
> Key: CASSANDRA-12790
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12790
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Configuration, Packaging
> Environment: Ubuntu 14.04
> Oracle Java 1.8.0_65
> Cassandra 3.0.9
>Reporter: Achmad Nasirudin Sandi
>Assignee: Benjamin Lerer
>Priority: Minor
>  Labels: easyfix, easytest, newbie
> Attachments: metrics-reporter-config-influx.yaml
>
>
> InfluxDb is a great tool for storing metrics. The main advantage over 
> graphite protocol is its tag mechanism that allows host info to be excluded 
> from metric name.
> I managed to send metrics into InfluxDb by putting 
> {{metrics-influxdb-1.1.7.jar}} into {{lib}} folder and replacing these 
> dependencies jar:
>  
> {code}
> commons-codec-1.2.jar -> commons-codec-1.9.jar
> reporter-config3-3.0.0.jar -> reporter-config3-3.0.3.jar
> reporter-config-base-3.0.0.jar -> reporter-config-base-3.0.3.jar
> {code}
> Although it seems that dropwizard team is working on official [package for 
> InfluxDb|https://github.com/dropwizard/metrics/tree/4.0-development], it 
> would be great it those libs are available in default distribution by 
> upgrading {{commons-codec}} and {{metrics}} dependencies. I was wondering 
> what effect are there to Cassandra functionality. Where can I help?
> Attached file is my configuration example.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12790) Support InfluxDb metrics reporter config

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574859#comment-15574859
 ] 

Benjamin Lerer commented on CASSANDRA-12790:


What I can do for you, next week, is to run the tests on the {{3.0.X}} version 
with the upgraded jars. At least you will know if there are some obvious 
problems. Nevertheless, it does not means that you will be fully safe. We try 
to cover as much things as we can on our tests but they are not 100% bullet 
proof.

> Support InfluxDb metrics reporter config
> 
>
> Key: CASSANDRA-12790
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12790
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Configuration, Packaging
> Environment: Ubuntu 14.04
> Oracle Java 1.8.0_65
> Cassandra 3.0.9
>Reporter: Achmad Nasirudin Sandi
>Assignee: Benjamin Lerer
>Priority: Minor
>  Labels: easyfix, easytest, newbie
> Attachments: metrics-reporter-config-influx.yaml
>
>
> InfluxDb is a great tool for storing metrics. The main advantage over 
> graphite protocol is its tag mechanism that allows host info to be excluded 
> from metric name.
> I managed to send metrics into InfluxDb by putting 
> {{metrics-influxdb-1.1.7.jar}} into {{lib}} folder and replacing these 
> dependencies jar:
>  
> {code}
> commons-codec-1.2.jar -> commons-codec-1.9.jar
> reporter-config3-3.0.0.jar -> reporter-config3-3.0.3.jar
> reporter-config-base-3.0.0.jar -> reporter-config-base-3.0.3.jar
> {code}
> Although it seems that dropwizard team is working on official [package for 
> InfluxDb|https://github.com/dropwizard/metrics/tree/4.0-development], it 
> would be great it those libs are available in default distribution by 
> upgrading {{commons-codec}} and {{metrics}} dependencies. I was wondering 
> what effect are there to Cassandra functionality. Where can I help?
> Attached file is my configuration example.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12651) Failure in SecondaryIndexTest.testAllowFilteringOnPartitionKeyWithSecondaryIndex

[Alex Petrov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-12651:

Description: 
This has failed with/without compression.

Stacktrace:
{code}
junit.framework.AssertionFailedError: Got less rows than expected. Expected 2 
but got 0
at org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:909)
at 
org.apache.cassandra.cql3.validation.entities.SecondaryIndexTest.lambda$testAllowFilteringOnPartitionKeyWithSecondaryIndex$78(SecondaryIndexTest.java:1228)
at 
org.apache.cassandra.cql3.validation.entities.SecondaryIndexTest$$Lambda$293/218688965.apply(Unknown
 Source)
at 
org.apache.cassandra.cql3.CQLTester.beforeAndAfterFlush(CQLTester.java:1215)
at 
org.apache.cassandra.cql3.validation.entities.SecondaryIndexTest.testAllowFilteringOnPartitionKeyWithSecondaryIndex(SecondaryIndexTest.java:1218)
{code}

Examples:
http://cassci.datastax.com/job/trunk_testall/1176/testReport/org.apache.cassandra.cql3.validation.entities/SecondaryIndexTest/testAllowFilteringOnPartitionKeyWithSecondaryIndex/
http://cassci.datastax.com/job/trunk_testall/1176/testReport/org.apache.cassandra.cql3.validation.entities/SecondaryIndexTest/testAllowFilteringOnPartitionKeyWithSecondaryIndex_compression/

http://cassci.datastax.com/job/trunk_testall/1219/testReport/org.apache.cassandra.cql3.validation.entities/SecondaryIndexTest/testAllowFilteringOnPartitionKeyWithSecondaryIndex/
http://cassci.datastax.com/job/trunk_testall/1216/testReport/org.apache.cassandra.cql3.validation.entities/SecondaryIndexTest/testAllowFilteringOnPartitionKeyWithSecondaryIndex/
http://cassci.datastax.com/job/trunk_testall/1208/testReport/org.apache.cassandra.cql3.validation.entities/SecondaryIndexTest/testAllowFilteringOnPartitionKeyWithSecondaryIndex/
http://cassci.datastax.com/job/trunk_testall/1176/testReport/org.apache.cassandra.cql3.validation.entities/SecondaryIndexTest/testAllowFilteringOnPartitionKeyWithSecondaryIndex/
http://cassci.datastax.com/job/trunk_testall/1175/testReport/org.apache.cassandra.cql3.validation.entities/SecondaryIndexTest/testAllowFilteringOnPartitionKeyWithSecondaryIndex/

May or may not be related, but there's a test failure (index duplicate):
http://cassci.datastax.com/view/Dev/view/carlyeks/job/carlyeks-ticket-11803-3.X-testall/lastCompletedBuild/testReport/org.apache.cassandra.index.internal/CassandraIndexTest/indexOnFirstClusteringColumn_compression/


  was:
This has failed with/without compression.

Stacktrace:
{code}
junit.framework.AssertionFailedError: Got less rows than expected. Expected 2 
but got 0
at org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:909)
at 
org.apache.cassandra.cql3.validation.entities.SecondaryIndexTest.lambda$testAllowFilteringOnPartitionKeyWithSecondaryIndex$78(SecondaryIndexTest.java:1228)
at 
org.apache.cassandra.cql3.validation.entities.SecondaryIndexTest$$Lambda$293/218688965.apply(Unknown
 Source)
at 
org.apache.cassandra.cql3.CQLTester.beforeAndAfterFlush(CQLTester.java:1215)
at 
org.apache.cassandra.cql3.validation.entities.SecondaryIndexTest.testAllowFilteringOnPartitionKeyWithSecondaryIndex(SecondaryIndexTest.java:1218)
{code}

Examples:
http://cassci.datastax.com/job/trunk_testall/1176/testReport/org.apache.cassandra.cql3.validation.entities/SecondaryIndexTest/testAllowFilteringOnPartitionKeyWithSecondaryIndex/
http://cassci.datastax.com/job/trunk_testall/1176/testReport/org.apache.cassandra.cql3.validation.entities/SecondaryIndexTest/testAllowFilteringOnPartitionKeyWithSecondaryIndex_compression/


> Failure in 
> SecondaryIndexTest.testAllowFilteringOnPartitionKeyWithSecondaryIndex
> 
>
> Key: CASSANDRA-12651
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12651
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Joel Knighton
>Assignee: Alex Petrov
>
> This has failed with/without compression.
> Stacktrace:
> {code}
> junit.framework.AssertionFailedError: Got less rows than expected. Expected 2 
> but got 0
>   at org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:909)
>   at 
> org.apache.cassandra.cql3.validation.entities.SecondaryIndexTest.lambda$testAllowFilteringOnPartitionKeyWithSecondaryIndex$78(SecondaryIndexTest.java:1228)
>   at 
> org.apache.cassandra.cql3.validation.entities.SecondaryIndexTest$$Lambda$293/218688965.apply(Unknown
>  Source)
>   at 
> org.apache.cassandra.cql3.CQLTester.beforeAndAfterFlush(CQLTester.java:1215)
>   at 
> org.apache.cassandra.cql3.validation.entities.SecondaryIndexTest.testAllowFilteringOnPartitionKeyWithSecondaryIndex(SecondaryIndexTest.java:1218)
> {code}
> Examples:
> 

[jira] [Commented] (CASSANDRA-12790) Support InfluxDb metrics reporter config

[Achmad Nasirudin Sandi (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574796#comment-15574796
 ] 

Achmad Nasirudin Sandi commented on CASSANDRA-12790:


I understand the versioning policy.

But I expect nothing breaks in the 
[tests|http://cassandra.apache.org/doc/latest/development/testing.html#dtests] 
when I did change the dependencies as I did. Is my assumption correct? Thank 
you.

> Support InfluxDb metrics reporter config
> 
>
> Key: CASSANDRA-12790
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12790
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Configuration, Packaging
> Environment: Ubuntu 14.04
> Oracle Java 1.8.0_65
> Cassandra 3.0.9
>Reporter: Achmad Nasirudin Sandi
>Assignee: Benjamin Lerer
>Priority: Minor
>  Labels: easyfix, easytest, newbie
> Attachments: metrics-reporter-config-influx.yaml
>
>
> InfluxDb is a great tool for storing metrics. The main advantage over 
> graphite protocol is its tag mechanism that allows host info to be excluded 
> from metric name.
> I managed to send metrics into InfluxDb by putting 
> {{metrics-influxdb-1.1.7.jar}} into {{lib}} folder and replacing these 
> dependencies jar:
>  
> {code}
> commons-codec-1.2.jar -> commons-codec-1.9.jar
> reporter-config3-3.0.0.jar -> reporter-config3-3.0.3.jar
> reporter-config-base-3.0.0.jar -> reporter-config-base-3.0.3.jar
> {code}
> Although it seems that dropwizard team is working on official [package for 
> InfluxDb|https://github.com/dropwizard/metrics/tree/4.0-development], it 
> would be great it those libs are available in default distribution by 
> upgrading {{commons-codec}} and {{metrics}} dependencies. I was wondering 
> what effect are there to Cassandra functionality. Where can I help?
> Attached file is my configuration example.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12784) ReplicationAwareTokenAllocatorTest times out almost every time for 3.X and trunk

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574732#comment-15574732
 ] 

Stefania commented on CASSANDRA-12784:
--

bq. testExistingCluster is supposed to be deterministic as it uses a seeded 
random and should either always fail or always succeed. That random must be 
somehow changing in the multiplexed run.

Thank you, I'll have a look on Monday at why the random generator is changing.

> ReplicationAwareTokenAllocatorTest times out almost every time for 3.X and 
> trunk
> 
>
> Key: CASSANDRA-12784
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12784
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 3.x
>
> Attachments: ReplicationAwareTokenAllocatorTest.jfr.gz
>
>
> Example failure: 
> http://cassci.datastax.com/view/cassandra-3.X/job/cassandra-3.X_testall/lastCompletedBuild/testReport/org.apache.cassandra.dht.tokenallocator/ReplicationAwareTokenAllocatorTest/testNewClusterWithMurmur3Partitioner/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-12784) ReplicationAwareTokenAllocatorTest times out almost every time for 3.X and trunk

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574654#comment-15574654
 ] 

Stefania edited comment on CASSANDRA-12784 at 10/14/16 9:02 AM:


Thanks for the review.

The multiplexed run returned a number of 
[failures|https://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-testall-multiplex/26/]
 in {{RandomReplicationAwareTokenAllocatorTest.testExistingCluster}}, is this 
test also expected to be flaky or is this a problem?

Splitting the test caused a failure in testall, I've fixed it and relaunched.

bq. The stack printout for single flakes could be useful to track the history 
of a failure; I would prefer not to lose it, but I wouldn't stop the commit if 
that is something you think is worth sacrificing.

The stack printout is still there, the logger is smart enough to work it out, 
I've tested it locally, see sample output below.

bq. I would rename flakyTestNewCluster in the base class to just testNewCluster 
since the individual runner is the one that declares it flaky and handles that.

Done.

bq. Is there a reason to post the the commits view instead of the the branch 
one? There's no direct way to get from the commits view to compare which is the 
one most useful for reviews.

Not really:

||3.X||trunk||
|[patch|https://github.com/stef1927/cassandra/tree/12784-3.X]|[patch|https://github.com/stef1927/cassandra/tree/12784]|
|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-12784-3.X-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-12784-testall/]|

--

Sample stack printout obtained by inserting a fake {{Assert.fail("Test 
message")}}:

{code}
  [junit] INFO  [main] 2016-10-14 16:10:44,810 ?:? - Test failed. It tends to 
fail sometimes due to the random selection of the tokens in the first few nodes.
[junit] java.lang.AssertionError: Test message
[junit] at org.junit.Assert.fail(Assert.java:88) ~[junit-4.12.jar:4.12]
[junit] at 
org.apache.cassandra.dht.tokenallocator.AbstractReplicationAwareTokenAllocatorTest.testNewCluster(AbstractReplicationAwareTokenAllocatorTest.java:592)
 ~[classes/:na]
[junit] at 
org.apache.cassandra.dht.tokenallocator.AbstractReplicationAwareTokenAllocatorTest.testNewCluster(AbstractReplicationAwareTokenAllocatorTest.java:563)
 ~[classes/:na]
[junit] at 
org.apache.cassandra.dht.tokenallocator.RandomReplicationAwareTokenAllocatorTest.flakyTestNewCluster(RandomReplicationAwareTokenAllocatorTest.java:51)
 [classes/:na]
[junit] at 
org.apache.cassandra.Util.runCatchingAssertionError(Util.java:581) 
~[classes/:na]
[junit] at org.apache.cassandra.Util.flakyTest(Util.java:611) 
~[classes/:na]
[junit] at 
org.apache.cassandra.dht.tokenallocator.RandomReplicationAwareTokenAllocatorTest.testNewClusterr(RandomReplicationAwareTokenAllocatorTest.java:44)
 [classes/:na]
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
~[na:1.8.0_101]
[junit] at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
~[na:1.8.0_101]
[junit] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 ~[na:1.8.0_101]
[junit] at java.lang.reflect.Method.invoke(Method.java:498) 
~[na:1.8.0_101]
[junit] at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
 [junit-4.12.jar:4.12]
[junit] at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
 [junit-4.12.jar:4.12]
[junit] at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
 [junit-4.12.jar:4.12]
[junit] at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
 [junit-4.12.jar:4.12]
[junit] at 
org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) 
[junit-4.12.jar:4.12]
[junit] at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
 [junit-4.12.jar:4.12]
[junit] at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
 [junit-4.12.jar:4.12]
[junit] at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) 
[junit-4.12.jar:4.12]
[junit] at 
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) 
[junit-4.12.jar:4.12]
[junit] at 
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) 
[junit-4.12.jar:4.12]
[junit] at 
org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) 
[junit-4.12.jar:4.12]
[junit] at 
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) 
[junit-4.12.jar:4.12]
[junit] at org.junit.runners.ParentRunner.run(ParentRunner.java:363) 
[junit-4.12.jar:4.12]
[junit] at 

[jira] [Commented] (CASSANDRA-12777) Optimize the vnode allocation for single replica per DC

[Branimir Lambov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574714#comment-15574714
 ] 

Branimir Lambov commented on CASSANDRA-12777:
-

"Percent" implies 0-100 which isn't what you are using (rightly so). "Ratio" is 
a better term for 0-1 multiplier.

How do you handle result above maximum in [the wraparound 
calculation|https://github.com/DikangGu/cassandra/commit/3711f9c00f47aba768ceba70fd9bd54f1d64#diff-fbd2bf80d0bb22ddfaac9973e58558f4R100]?
 For {{LongToken}} that's not a problem as the token gets wrapped on conversion 
to long, but I don't think that happens for {{BigIntegerToken}}. This needs a 
test as well (including a random one using {{a.size(split(a, b, x))}} within a 
couple of ulps from {{x * a.size(b)}}, also including a validity check for the 
returned token).

[{{createTokenInfo}} in 
constructor|https://github.com/DikangGu/cassandra/commit/3711f9c00f47aba768ceba70fd9bd54f1d64#diff-3ee50470ce492c51246b32b35fae5cfcR51]
 appears superfluous.

Most of the comments in the original implementation add important information 
and should be preserved (e.g. why [the + 
2|https://github.com/DikangGu/cassandra/commit/3711f9c00f47aba768ceba70fd9bd54f1d64#diff-3ee50470ce492c51246b32b35fae5cfcR155]).

Did you try lower fractions than 0.99 for takeovers? I would go lower, perhaps 
0.9 or even 0.75 (try the simulation out).

Nit: From a design perspective I believe it would be cleaner to leave the 
{{TokenAllocator}} interface as interface, put the factory method there, and 
move the abstract base class to a {{TokenAllocatorBase}}.

> Optimize the vnode allocation for single replica per DC
> ---
>
> Key: CASSANDRA-12777
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12777
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Dikang Gu
>Assignee: Dikang Gu
> Fix For: 3.x
>
>
> The new vnode allocation algorithm introduced in CASSANDRA-7032 is optimized 
> for the situation that there are multiple replicas per DC.
> In our production environment, most cluster only has one replica, in this 
> case, the algorithm does not work perfectly. It always tries to split token 
> ranges by half, so that the ownership of "min" node could go as low as ~60% 
> compared to avg.
> So for single replica case, I'm working on a new algorithm, which is based on 
> Branimir's previous commit, to split token ranges by "some" percentage, 
> instead of always by half. In this way, we can get a very small variation of 
> the ownership among different nodes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12789) ViewTest.testCompaction releases reference twice

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574715#comment-15574715
 ] 

Stefania commented on CASSANDRA-12789:
--

Multiplexing this [patch|https://github.com/stef1927/cassandra/tree/12789-3.0] 
100 times 
[here|https://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-testall-multiplex/27/].

> ViewTest.testCompaction releases reference twice
> 
>
> Key: CASSANDRA-12789
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12789
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 3.0.x, 3.x
>
>
> Sample failures:
> http://cassci.datastax.com/view/trunk/job/trunk_testall/1233/testReport/junit/org.apache.cassandra.db.lifecycle/ViewTest/testCompaction/
> http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_testall/684/testReport/junit/org.apache.cassandra.db.lifecycle/ViewTest/testCompaction/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-12790) Support InfluxDb metrics reporter config

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer reassigned CASSANDRA-12790:
--

Assignee: Benjamin Lerer

> Support InfluxDb metrics reporter config
> 
>
> Key: CASSANDRA-12790
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12790
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Configuration, Packaging
> Environment: Ubuntu 14.04
> Oracle Java 1.8.0_65
> Cassandra 3.0.9
>Reporter: Achmad Nasirudin Sandi
>Assignee: Benjamin Lerer
>Priority: Minor
>  Labels: easyfix, easytest, newbie
> Attachments: metrics-reporter-config-influx.yaml
>
>
> InfluxDb is a great tool for storing metrics. The main advantage over 
> graphite protocol is its tag mechanism that allows host info to be excluded 
> from metric name.
> I managed to send metrics into InfluxDb by putting 
> {{metrics-influxdb-1.1.7.jar}} into {{lib}} folder and replacing these 
> dependencies jar:
>  
> {code}
> commons-codec-1.2.jar -> commons-codec-1.9.jar
> reporter-config3-3.0.0.jar -> reporter-config3-3.0.3.jar
> reporter-config-base-3.0.0.jar -> reporter-config-base-3.0.3.jar
> {code}
> Although it seems that dropwizard team is working on official [package for 
> InfluxDb|https://github.com/dropwizard/metrics/tree/4.0-development], it 
> would be great it those libs are available in default distribution by 
> upgrading {{commons-codec}} and {{metrics}} dependencies. I was wondering 
> what effect are there to Cassandra functionality. Where can I help?
> Attached file is my configuration example.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11803) Creating a materialized view on a table with "token" column breaks the cluster

[Alex Petrov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-11803:

Status: Ready to Commit  (was: Patch Available)

> Creating a materialized view on a table with "token" column breaks the cluster
> --
>
> Key: CASSANDRA-11803
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11803
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Kernel:
> Linux 4.4.8-20.46.amzn1.x86_64
> Java:
> Java OpenJDK Runtime Environment (build 1.8.0_91-b14)
> OpenJDK 64-Bit Server VM (build 25.91-b14, mixed mode)
> Cassandra: 
> datastax-ddc-3.3.0-1.noarch
> datastax-ddc-tools-3.3.0-1.noarch
>Reporter: Victor Trac
>Assignee: Carl Yeksigian
> Fix For: 3.0.x
>
>
> On a new Cassandra cluster, if we create a table with a field called "token" 
> (with quotes) and then create a materialized view that uses "token", the 
> cluster breaks. A ServerError is returned, and no further nodetool operations 
> on the sstables work. Restarting the Cassandra server will also fail. It 
> seems like the entire cluster is hosed.
> We tried this on Cassandra 3.3 and 3.5. 
> Here's how to produce (on an new, empty cassandra 3.5 docker container):
> {code}
> [cqlsh 5.0.1 | Cassandra 3.5 | CQL spec 3.4.0 | Native protocol v4]
> Use HELP for help.
> cqlsh> CREATE KEYSPACE account WITH REPLICATION = { 'class' : 
> 'SimpleStrategy', 'replication_factor' : 1 };
> cqlsh> CREATE TABLE account.session  (
>...   "token" blob,
>...   account_id uuid,
>...   PRIMARY KEY("token")
>... )WITH compaction={'class': 'LeveledCompactionStrategy'} AND
>...   compression={'sstable_compression': 'LZ4Compressor'};
> cqlsh> CREATE MATERIALIZED VIEW account.account_session AS
>...SELECT account_id,"token" FROM account.session
>...WHERE "token" IS NOT NULL and account_id IS NOT NULL
>...PRIMARY KEY (account_id, "token");
> ServerError:  message="java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.exceptions.SyntaxException: line 1:25 no viable 
> alternative at input 'FROM' (SELECT account_id, token [FROM]...)">
> cqlsh> drop table account.session;
> ServerError:  message="java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.exceptions.SyntaxException: line 1:25 no viable 
> alternative at input 'FROM' (SELECT account_id, token [FROM]...)">
> {code}
> When any sstable*, nodetool, or when the Cassandra process is restarted, this 
> is emitted on startup and Cassandra exits (copied from a server w/ data):
> {code}
> INFO  [main] 2016-05-12 23:25:30,074 ColumnFamilyStore.java:395 - 
> Initializing system_schema.indexes
> DEBUG [SSTableBatchOpen:1] 2016-05-12 23:25:30,075 SSTableReader.java:480 - 
> Opening 
> /mnt/cassandra/data/system_schema/indexes-0feb57ac311f382fba6d9024d305702f/ma-4-big
>  (91 bytes)
> ERROR [main] 2016-05-12 23:25:30,143 CassandraDaemon.java:697 - Exception 
> encountered during startup
> org.apache.cassandra.exceptions.SyntaxException: line 1:59 no viable 
> alternative at input 'FROM' (..., expire_at, last_used, token [FROM]...)
> at 
> org.apache.cassandra.cql3.ErrorCollector.throwFirstSyntaxError(ErrorCollector.java:101)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.cql3.CQLFragmentParser.parseAnyUnhandled(CQLFragmentParser.java:80)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.cql3.QueryProcessor.parseStatement(QueryProcessor.java:512)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchView(SchemaKeyspace.java:1128)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchViews(SchemaKeyspace.java:1092)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:903)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:879)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:867)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:134) 
> ~[apache-cassandra-3.5.0.jar:3.5.0]
> at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:124) 
> ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:229) 
> [apache-cassandra-3.5.0.jar:3.5.0]
> at 
> 

[jira] [Commented] (CASSANDRA-11803) Creating a materialized view on a table with "token" column breaks the cluster

[Alex Petrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574693#comment-15574693
 ] 

Alex Petrov commented on CASSANDRA-11803:
-

Great, thank you! 

+1

> Creating a materialized view on a table with "token" column breaks the cluster
> --
>
> Key: CASSANDRA-11803
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11803
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Kernel:
> Linux 4.4.8-20.46.amzn1.x86_64
> Java:
> Java OpenJDK Runtime Environment (build 1.8.0_91-b14)
> OpenJDK 64-Bit Server VM (build 25.91-b14, mixed mode)
> Cassandra: 
> datastax-ddc-3.3.0-1.noarch
> datastax-ddc-tools-3.3.0-1.noarch
>Reporter: Victor Trac
>Assignee: Carl Yeksigian
> Fix For: 3.0.x
>
>
> On a new Cassandra cluster, if we create a table with a field called "token" 
> (with quotes) and then create a materialized view that uses "token", the 
> cluster breaks. A ServerError is returned, and no further nodetool operations 
> on the sstables work. Restarting the Cassandra server will also fail. It 
> seems like the entire cluster is hosed.
> We tried this on Cassandra 3.3 and 3.5. 
> Here's how to produce (on an new, empty cassandra 3.5 docker container):
> {code}
> [cqlsh 5.0.1 | Cassandra 3.5 | CQL spec 3.4.0 | Native protocol v4]
> Use HELP for help.
> cqlsh> CREATE KEYSPACE account WITH REPLICATION = { 'class' : 
> 'SimpleStrategy', 'replication_factor' : 1 };
> cqlsh> CREATE TABLE account.session  (
>...   "token" blob,
>...   account_id uuid,
>...   PRIMARY KEY("token")
>... )WITH compaction={'class': 'LeveledCompactionStrategy'} AND
>...   compression={'sstable_compression': 'LZ4Compressor'};
> cqlsh> CREATE MATERIALIZED VIEW account.account_session AS
>...SELECT account_id,"token" FROM account.session
>...WHERE "token" IS NOT NULL and account_id IS NOT NULL
>...PRIMARY KEY (account_id, "token");
> ServerError:  message="java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.exceptions.SyntaxException: line 1:25 no viable 
> alternative at input 'FROM' (SELECT account_id, token [FROM]...)">
> cqlsh> drop table account.session;
> ServerError:  message="java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
> org.apache.cassandra.exceptions.SyntaxException: line 1:25 no viable 
> alternative at input 'FROM' (SELECT account_id, token [FROM]...)">
> {code}
> When any sstable*, nodetool, or when the Cassandra process is restarted, this 
> is emitted on startup and Cassandra exits (copied from a server w/ data):
> {code}
> INFO  [main] 2016-05-12 23:25:30,074 ColumnFamilyStore.java:395 - 
> Initializing system_schema.indexes
> DEBUG [SSTableBatchOpen:1] 2016-05-12 23:25:30,075 SSTableReader.java:480 - 
> Opening 
> /mnt/cassandra/data/system_schema/indexes-0feb57ac311f382fba6d9024d305702f/ma-4-big
>  (91 bytes)
> ERROR [main] 2016-05-12 23:25:30,143 CassandraDaemon.java:697 - Exception 
> encountered during startup
> org.apache.cassandra.exceptions.SyntaxException: line 1:59 no viable 
> alternative at input 'FROM' (..., expire_at, last_used, token [FROM]...)
> at 
> org.apache.cassandra.cql3.ErrorCollector.throwFirstSyntaxError(ErrorCollector.java:101)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.cql3.CQLFragmentParser.parseAnyUnhandled(CQLFragmentParser.java:80)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.cql3.QueryProcessor.parseStatement(QueryProcessor.java:512)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchView(SchemaKeyspace.java:1128)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchViews(SchemaKeyspace.java:1092)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:903)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:879)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:867)
>  ~[apache-cassandra-3.5.0.jar:3.5.0]
> at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:134) 
> ~[apache-cassandra-3.5.0.jar:3.5.0]
> at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:124) 
> ~[apache-cassandra-3.5.0.jar:3.5.0]
> at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:229) 
> [apache-cassandra-3.5.0.jar:3.5.0]
> at 
> 

[jira] [Commented] (CASSANDRA-12787) Reduce instanceOf() type checking to improve performance

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574680#comment-15574680
 ] 

Benjamin Lerer commented on CASSANDRA-12787:


Based on how the scala bug was fixed, it seems that something like 
[this|https://github.com/apache/cassandra/compare/trunk...blerer:12787-3.X] 
should fix it. Am I wrong?


> Reduce instanceOf() type checking to improve performance
> 
>
> Key: CASSANDRA-12787
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12787
> Project: Cassandra
>  Issue Type: Improvement
> Environment: The tests and examples stated were run on:
> Intel (R) Xeon (R) CPU E5-2699 v3 @ 2.30GHz, HT Enabled
> Oracle JDK 1.8
> Cassandra 3.10-SNAPSHOT
> Linux kernel 4.7
>Reporter: shylaja kokoori
> Fix For: 3.x
>
> Attachments: reduce_instanceof_typechecking.pdf
>
>
> While performance profiling Cassandra with cassandra-stress test on a  pure 
> write workload, we noticed that one of the hot methods for Cassandra server 
> is the compareTo( PartitionPosition pos) function in 
> org.apache.cassandra.db.DecoratedKey. The actual root cause of the problem is 
> a slowdown in Java's instanceof operator.
> Our initial performance testing using a hacked JVM shows about 61% increase 
> in throughput and about 15% reduction in 99 percentile latency. Data shows 
> that improvements are from the removal of thread contention caused by 
> Instanceof operator.
> Currently, we are working on a JDK fix to solve this issue. In the meantime, 
> we think it will be beneficial to address this issue at Java application 
> level as well. We are in the process of creating a patch to Cassandra 
> replacing instanceof check with virtual method calls. The patch will be 
> available in this thread for review soon. Please let us know your feedback 
> and comments.
> For more details please refer to the attached pdf.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12784) ReplicationAwareTokenAllocatorTest times out almost every time for 3.X and trunk

[Branimir Lambov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574676#comment-15574676
 ] 

Branimir Lambov commented on CASSANDRA-12784:
-

{{testExistingCluster}} is supposed to be deterministic as it uses a seeded 
random and should either always fail or always succeed. That random must be 
somehow changing in the multiplexed run.

bq. The stack printout still there

That's great, I did not know this trick.

> ReplicationAwareTokenAllocatorTest times out almost every time for 3.X and 
> trunk
> 
>
> Key: CASSANDRA-12784
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12784
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 3.x
>
> Attachments: ReplicationAwareTokenAllocatorTest.jfr.gz
>
>
> Example failure: 
> http://cassci.datastax.com/view/cassandra-3.X/job/cassandra-3.X_testall/lastCompletedBuild/testReport/org.apache.cassandra.dht.tokenallocator/ReplicationAwareTokenAllocatorTest/testNewClusterWithMurmur3Partitioner/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12790) Support InfluxDb metrics reporter config

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574658#comment-15574658
 ] 

Benjamin Lerer commented on CASSANDRA-12790:


bq. Currently we need ver 3.0.X to maintain compatibility against 
Spark-Cassandra Connector

I understand your problem but then you have to do it on your side. {{3.0.X}} is 
a bug fix branch.
I will look into upgrading {{common-codec}} into {{3.X}}.

> Support InfluxDb metrics reporter config
> 
>
> Key: CASSANDRA-12790
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12790
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Configuration, Packaging
> Environment: Ubuntu 14.04
> Oracle Java 1.8.0_65
> Cassandra 3.0.9
>Reporter: Achmad Nasirudin Sandi
>Priority: Minor
>  Labels: easyfix, easytest, newbie
> Attachments: metrics-reporter-config-influx.yaml
>
>
> InfluxDb is a great tool for storing metrics. The main advantage over 
> graphite protocol is its tag mechanism that allows host info to be excluded 
> from metric name.
> I managed to send metrics into InfluxDb by putting 
> {{metrics-influxdb-1.1.7.jar}} into {{lib}} folder and replacing these 
> dependencies jar:
>  
> {code}
> commons-codec-1.2.jar -> commons-codec-1.9.jar
> reporter-config3-3.0.0.jar -> reporter-config3-3.0.3.jar
> reporter-config-base-3.0.0.jar -> reporter-config-base-3.0.3.jar
> {code}
> Although it seems that dropwizard team is working on official [package for 
> InfluxDb|https://github.com/dropwizard/metrics/tree/4.0-development], it 
> would be great it those libs are available in default distribution by 
> upgrading {{commons-codec}} and {{metrics}} dependencies. I was wondering 
> what effect are there to Cassandra functionality. Where can I help?
> Attached file is my configuration example.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12784) ReplicationAwareTokenAllocatorTest times out almost every time for 3.X and trunk

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574654#comment-15574654
 ] 

Stefania commented on CASSANDRA-12784:
--

Thanks for the review.

The multiplexed run returned a number of 
[failures|https://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-testall-multiplex/26/]
 in {{RandomReplicationAwareTokenAllocatorTest.testExistingCluster}}, is this 
test also expected to be flaky or is this a problem?

Splitting the test caused a failure in testall, I've fixed it and relaunched.

bq. The stack printout for single flakes could be useful to track the history 
of a failure; I would prefer not to lose it, but I wouldn't stop the commit if 
that is something you think is worth sacrificing.

The stack printout still there, the logger is smart enough to work it out, I've 
tested it locally, see sample output below.

bq. I would rename flakyTestNewCluster in the base class to just testNewCluster 
since the individual runner is the one that declares it flaky and handles that.

Done.

bq. Is there a reason to post the the commits view instead of the the branch 
one? There's no direct way to get from the commits view to compare which is the 
one most useful for reviews.

Not really:

||3.X||trunk||
|[patch|https://github.com/stef1927/cassandra/tree/12784-3.X]|[patch|https://github.com/stef1927/cassandra/tree/12784]|
|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-12784-3.X-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-12784-testall/]|

--

Sample stack printout obtained by inserting a fake {{Assert.fail("Test 
message")}}:

{code}
  [junit] INFO  [main] 2016-10-14 16:10:44,810 ?:? - Test failed. It tends to 
fail sometimes due to the random selection of the tokens in the first few nodes.
[junit] java.lang.AssertionError: Test message
[junit] at org.junit.Assert.fail(Assert.java:88) ~[junit-4.12.jar:4.12]
[junit] at 
org.apache.cassandra.dht.tokenallocator.AbstractReplicationAwareTokenAllocatorTest.testNewCluster(AbstractReplicationAwareTokenAllocatorTest.java:592)
 ~[classes/:na]
[junit] at 
org.apache.cassandra.dht.tokenallocator.AbstractReplicationAwareTokenAllocatorTest.testNewCluster(AbstractReplicationAwareTokenAllocatorTest.java:563)
 ~[classes/:na]
[junit] at 
org.apache.cassandra.dht.tokenallocator.RandomReplicationAwareTokenAllocatorTest.flakyTestNewCluster(RandomReplicationAwareTokenAllocatorTest.java:51)
 [classes/:na]
[junit] at 
org.apache.cassandra.Util.runCatchingAssertionError(Util.java:581) 
~[classes/:na]
[junit] at org.apache.cassandra.Util.flakyTest(Util.java:611) 
~[classes/:na]
[junit] at 
org.apache.cassandra.dht.tokenallocator.RandomReplicationAwareTokenAllocatorTest.testNewClusterr(RandomReplicationAwareTokenAllocatorTest.java:44)
 [classes/:na]
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
~[na:1.8.0_101]
[junit] at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
~[na:1.8.0_101]
[junit] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 ~[na:1.8.0_101]
[junit] at java.lang.reflect.Method.invoke(Method.java:498) 
~[na:1.8.0_101]
[junit] at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
 [junit-4.12.jar:4.12]
[junit] at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
 [junit-4.12.jar:4.12]
[junit] at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
 [junit-4.12.jar:4.12]
[junit] at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
 [junit-4.12.jar:4.12]
[junit] at 
org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) 
[junit-4.12.jar:4.12]
[junit] at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
 [junit-4.12.jar:4.12]
[junit] at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
 [junit-4.12.jar:4.12]
[junit] at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) 
[junit-4.12.jar:4.12]
[junit] at 
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) 
[junit-4.12.jar:4.12]
[junit] at 
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) 
[junit-4.12.jar:4.12]
[junit] at 
org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) 
[junit-4.12.jar:4.12]
[junit] at 
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) 
[junit-4.12.jar:4.12]
[junit] at org.junit.runners.ParentRunner.run(ParentRunner.java:363) 
[junit-4.12.jar:4.12]
[junit] at 
junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:38) 
[junit-4.12.jar:na]
[junit] at 

[jira] [Updated] (CASSANDRA-12784) ReplicationAwareTokenAllocatorTest times out almost every time for 3.X and trunk

[Stefania (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefania updated CASSANDRA-12784:
-
Reviewer: Branimir Lambov

> ReplicationAwareTokenAllocatorTest times out almost every time for 3.X and 
> trunk
> 
>
> Key: CASSANDRA-12784
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12784
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 3.x
>
> Attachments: ReplicationAwareTokenAllocatorTest.jfr.gz
>
>
> Example failure: 
> http://cassci.datastax.com/view/cassandra-3.X/job/cassandra-3.X_testall/lastCompletedBuild/testReport/org.apache.cassandra.dht.tokenallocator/ReplicationAwareTokenAllocatorTest/testNewClusterWithMurmur3Partitioner/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-12790) Support InfluxDb metrics reporter config

[Achmad Nasirudin Sandi (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574622#comment-15574622
 ] 

Achmad Nasirudin Sandi edited comment on CASSANDRA-12790 at 10/14/16 8:15 AM:
--

It is required by {{metrics-influxdb-1.1.7.jar}}. I tested by inserting some 
record and did SELECT query. It is fine so far. Currently we need ver 3.0.X to 
maintain compatibility against [Spark-Cassandra 
Connector|https://github.com/datastax/spark-cassandra-connector/]


was (Author: achmadns):
It is required by {{metrics-influxdb-1.1.7.jar}}. I tested by inserting some 
record and did SELECT query. It is fine so far.

> Support InfluxDb metrics reporter config
> 
>
> Key: CASSANDRA-12790
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12790
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Configuration, Packaging
> Environment: Ubuntu 14.04
> Oracle Java 1.8.0_65
> Cassandra 3.0.9
>Reporter: Achmad Nasirudin Sandi
>Priority: Minor
>  Labels: easyfix, easytest, newbie
> Attachments: metrics-reporter-config-influx.yaml
>
>
> InfluxDb is a great tool for storing metrics. The main advantage over 
> graphite protocol is its tag mechanism that allows host info to be excluded 
> from metric name.
> I managed to send metrics into InfluxDb by putting 
> {{metrics-influxdb-1.1.7.jar}} into {{lib}} folder and replacing these 
> dependencies jar:
>  
> {code}
> commons-codec-1.2.jar -> commons-codec-1.9.jar
> reporter-config3-3.0.0.jar -> reporter-config3-3.0.3.jar
> reporter-config-base-3.0.0.jar -> reporter-config-base-3.0.3.jar
> {code}
> Although it seems that dropwizard team is working on official [package for 
> InfluxDb|https://github.com/dropwizard/metrics/tree/4.0-development], it 
> would be great it those libs are available in default distribution by 
> upgrading {{commons-codec}} and {{metrics}} dependencies. I was wondering 
> what effect are there to Cassandra functionality. Where can I help?
> Attached file is my configuration example.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12787) Reduce instanceOf() type checking to improve performance

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-12787:
---
Assignee: (was: Benjamin Lerer)

> Reduce instanceOf() type checking to improve performance
> 
>
> Key: CASSANDRA-12787
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12787
> Project: Cassandra
>  Issue Type: Improvement
> Environment: The tests and examples stated were run on:
> Intel (R) Xeon (R) CPU E5-2699 v3 @ 2.30GHz, HT Enabled
> Oracle JDK 1.8
> Cassandra 3.10-SNAPSHOT
> Linux kernel 4.7
>Reporter: shylaja kokoori
> Fix For: 3.x
>
> Attachments: reduce_instanceof_typechecking.pdf
>
>
> While performance profiling Cassandra with cassandra-stress test on a  pure 
> write workload, we noticed that one of the hot methods for Cassandra server 
> is the compareTo( PartitionPosition pos) function in 
> org.apache.cassandra.db.DecoratedKey. The actual root cause of the problem is 
> a slowdown in Java's instanceof operator.
> Our initial performance testing using a hacked JVM shows about 61% increase 
> in throughput and about 15% reduction in 99 percentile latency. Data shows 
> that improvements are from the removal of thread contention caused by 
> Instanceof operator.
> Currently, we are working on a JDK fix to solve this issue. In the meantime, 
> we think it will be beneficial to address this issue at Java application 
> level as well. We are in the process of creating a patch to Cassandra 
> replacing instanceof check with virtual method calls. The patch will be 
> available in this thread for review soon. Please let us know your feedback 
> and comments.
> For more details please refer to the attached pdf.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12790) Support InfluxDb metrics reporter config

[Achmad Nasirudin Sandi (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574622#comment-15574622
 ] 

Achmad Nasirudin Sandi commented on CASSANDRA-12790:


It is required by {{metrics-influxdb-1.1.7.jar}}. I tested by inserting some 
record and did SELECT query. It is fine so far.

> Support InfluxDb metrics reporter config
> 
>
> Key: CASSANDRA-12790
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12790
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Configuration, Packaging
> Environment: Ubuntu 14.04
> Oracle Java 1.8.0_65
> Cassandra 3.0.9
>Reporter: Achmad Nasirudin Sandi
>Priority: Minor
>  Labels: easyfix, easytest, newbie
> Attachments: metrics-reporter-config-influx.yaml
>
>
> InfluxDb is a great tool for storing metrics. The main advantage over 
> graphite protocol is its tag mechanism that allows host info to be excluded 
> from metric name.
> I managed to send metrics into InfluxDb by putting 
> {{metrics-influxdb-1.1.7.jar}} into {{lib}} folder and replacing these 
> dependencies jar:
>  
> {code}
> commons-codec-1.2.jar -> commons-codec-1.9.jar
> reporter-config3-3.0.0.jar -> reporter-config3-3.0.3.jar
> reporter-config-base-3.0.0.jar -> reporter-config-base-3.0.3.jar
> {code}
> Although it seems that dropwizard team is working on official [package for 
> InfluxDb|https://github.com/dropwizard/metrics/tree/4.0-development], it 
> would be great it those libs are available in default distribution by 
> upgrading {{commons-codec}} and {{metrics}} dependencies. I was wondering 
> what effect are there to Cassandra functionality. Where can I help?
> Attached file is my configuration example.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-12787) Reduce instanceOf() type checking to improve performance

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer reassigned CASSANDRA-12787:
--

Assignee: Benjamin Lerer

> Reduce instanceOf() type checking to improve performance
> 
>
> Key: CASSANDRA-12787
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12787
> Project: Cassandra
>  Issue Type: Improvement
> Environment: The tests and examples stated were run on:
> Intel (R) Xeon (R) CPU E5-2699 v3 @ 2.30GHz, HT Enabled
> Oracle JDK 1.8
> Cassandra 3.10-SNAPSHOT
> Linux kernel 4.7
>Reporter: shylaja kokoori
>Assignee: Benjamin Lerer
> Fix For: 3.x
>
> Attachments: reduce_instanceof_typechecking.pdf
>
>
> While performance profiling Cassandra with cassandra-stress test on a  pure 
> write workload, we noticed that one of the hot methods for Cassandra server 
> is the compareTo( PartitionPosition pos) function in 
> org.apache.cassandra.db.DecoratedKey. The actual root cause of the problem is 
> a slowdown in Java's instanceof operator.
> Our initial performance testing using a hacked JVM shows about 61% increase 
> in throughput and about 15% reduction in 99 percentile latency. Data shows 
> that improvements are from the removal of thread contention caused by 
> Instanceof operator.
> Currently, we are working on a JDK fix to solve this issue. In the meantime, 
> we think it will be beneficial to address this issue at Java application 
> level as well. We are in the process of creating a patch to Cassandra 
> replacing instanceof check with virtual method calls. The patch will be 
> available in this thread for review soon. Please let us know your feedback 
> and comments.
> For more details please refer to the attached pdf.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12790) Support InfluxDb metrics reporter config

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574617#comment-15574617
 ] 

Benjamin Lerer commented on CASSANDRA-12790:


As it is an improvement, it can only be done in 3.X (next release being 3.10). 
I checked that branch and the reporter config dependencies have already been 
upgraded to the version you need.
Do you really need commons-codec to be upgraded to 1.9?

> Support InfluxDb metrics reporter config
> 
>
> Key: CASSANDRA-12790
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12790
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Configuration, Packaging
> Environment: Ubuntu 14.04
> Oracle Java 1.8.0_65
> Cassandra 3.0.9
>Reporter: Achmad Nasirudin Sandi
>Priority: Minor
>  Labels: easyfix, easytest, newbie
> Attachments: metrics-reporter-config-influx.yaml
>
>
> InfluxDb is a great tool for storing metrics. The main advantage over 
> graphite protocol is its tag mechanism that allows host info to be excluded 
> from metric name.
> I managed to send metrics into InfluxDb by putting 
> {{metrics-influxdb-1.1.7.jar}} into {{lib}} folder and replacing these 
> dependencies jar:
>  
> {code}
> commons-codec-1.2.jar -> commons-codec-1.9.jar
> reporter-config3-3.0.0.jar -> reporter-config3-3.0.3.jar
> reporter-config-base-3.0.0.jar -> reporter-config-base-3.0.3.jar
> {code}
> Although it seems that dropwizard team is working on official [package for 
> InfluxDb|https://github.com/dropwizard/metrics/tree/4.0-development], it 
> would be great it those libs are available in default distribution by 
> upgrading {{commons-codec}} and {{metrics}} dependencies. I was wondering 
> what effect are there to Cassandra functionality. Where can I help?
> Attached file is my configuration example.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12790) Support InfluxDb metrics reporter config

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-12790:
---
Priority: Minor  (was: Blocker)

> Support InfluxDb metrics reporter config
> 
>
> Key: CASSANDRA-12790
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12790
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Configuration, Packaging
> Environment: Ubuntu 14.04
> Oracle Java 1.8.0_65
> Cassandra 3.0.9
>Reporter: Achmad Nasirudin Sandi
>Priority: Minor
>  Labels: easyfix, easytest, newbie
> Attachments: metrics-reporter-config-influx.yaml
>
>
> InfluxDb is a great tool for storing metrics. The main advantage over 
> graphite protocol is its tag mechanism that allows host info to be excluded 
> from metric name.
> I managed to send metrics into InfluxDb by putting 
> {{metrics-influxdb-1.1.7.jar}} into {{lib}} folder and replacing these 
> dependencies jar:
>  
> {code}
> commons-codec-1.2.jar -> commons-codec-1.9.jar
> reporter-config3-3.0.0.jar -> reporter-config3-3.0.3.jar
> reporter-config-base-3.0.0.jar -> reporter-config-base-3.0.3.jar
> {code}
> Although it seems that dropwizard team is working on official [package for 
> InfluxDb|https://github.com/dropwizard/metrics/tree/4.0-development], it 
> would be great it those libs are available in default distribution by 
> upgrading {{commons-codec}} and {{metrics}} dependencies. I was wondering 
> what effect are there to Cassandra functionality. Where can I help?
> Attached file is my configuration example.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12784) ReplicationAwareTokenAllocatorTest times out almost every time for 3.X and trunk

[Branimir Lambov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Branimir Lambov updated CASSANDRA-12784:

Status: Ready to Commit  (was: Patch Available)

> ReplicationAwareTokenAllocatorTest times out almost every time for 3.X and 
> trunk
> 
>
> Key: CASSANDRA-12784
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12784
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 3.x
>
> Attachments: ReplicationAwareTokenAllocatorTest.jfr.gz
>
>
> Example failure: 
> http://cassci.datastax.com/view/cassandra-3.X/job/cassandra-3.X_testall/lastCompletedBuild/testReport/org.apache.cassandra.dht.tokenallocator/ReplicationAwareTokenAllocatorTest/testNewClusterWithMurmur3Partitioner/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12784) ReplicationAwareTokenAllocatorTest times out almost every time for 3.X and trunk

[Branimir Lambov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574514#comment-15574514
 ] 

Branimir Lambov commented on CASSANDRA-12784:
-

+1. Nits:

- The stack printout for single flakes could be useful to track the history of 
a failure; I would prefer not to lose it, but I wouldn't stop the commit if 
that is something you think is worth sacrificing.

- I would rename [{{flakyTestNewCluster}} in the base 
class|https://github.com/apache/cassandra/compare/trunk...stef1927:12784-3.X#diff-f32c9e3d5921a2a50fe56db4612f14b4R548]
 to just {{testNewCluster}} since the individual runner is the one that 
declares it flaky and handles that.

Is there a reason to post the the commits view instead of the [the branch 
one|https://github.com/stef1927/cassandra/tree/12784-3.X]? There's no direct 
way to get from the commits view to 
[compare|https://github.com/apache/cassandra/compare/trunk...stef1927:12784-3.X]
 which is the one most useful for reviews.



> ReplicationAwareTokenAllocatorTest times out almost every time for 3.X and 
> trunk
> 
>
> Key: CASSANDRA-12784
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12784
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 3.x
>
> Attachments: ReplicationAwareTokenAllocatorTest.jfr.gz
>
>
> Example failure: 
> http://cassci.datastax.com/view/cassandra-3.X/job/cassandra-3.X_testall/lastCompletedBuild/testReport/org.apache.cassandra.dht.tokenallocator/ReplicationAwareTokenAllocatorTest/testNewClusterWithMurmur3Partitioner/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12790) Support InfluxDb metrics reporter config

[Achmad Nasirudin Sandi (JIRA)]

Achmad Nasirudin Sandi created CASSANDRA-12790:
--

 Summary: Support InfluxDb metrics reporter config
 Key: CASSANDRA-12790
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12790
 Project: Cassandra
  Issue Type: Improvement
  Components: Configuration, Packaging
 Environment: Ubuntu 14.04
Oracle Java 1.8.0_65
Cassandra 3.0.9
Reporter: Achmad Nasirudin Sandi
Priority: Blocker
 Attachments: metrics-reporter-config-influx.yaml

InfluxDb is a great tool for storing metrics. The main advantage over graphite 
protocol is its tag mechanism that allows host info to be excluded from metric 
name.

I managed to send metrics into InfluxDb by putting 
{{metrics-influxdb-1.1.7.jar}} into {{lib}} folder and replacing these 
dependencies jar:
 
{code}
commons-codec-1.2.jar -> commons-codec-1.9.jar
reporter-config3-3.0.0.jar -> reporter-config3-3.0.3.jar
reporter-config-base-3.0.0.jar -> reporter-config-base-3.0.3.jar
{code}

Although it seems that dropwizard team is working on official [package for 
InfluxDb|https://github.com/dropwizard/metrics/tree/4.0-development], it would 
be great it those libs are available in default distribution by upgrading 
{{commons-codec}} and {{metrics}} dependencies. I was wondering what effect are 
there to Cassandra functionality. Where can I help?

Attached file is my configuration example.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12789) ViewTest.testCompaction releases reference twice

[Stefania (JIRA)]

Stefania created CASSANDRA-12789:


 Summary: ViewTest.testCompaction releases reference twice
 Key: CASSANDRA-12789
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12789
 Project: Cassandra
  Issue Type: Bug
Reporter: Stefania
Assignee: Stefania
 Fix For: 3.0.x, 3.x


Sample failures:

http://cassci.datastax.com/view/trunk/job/trunk_testall/1233/testReport/junit/org.apache.cassandra.db.lifecycle/ViewTest/testCompaction/

http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_testall/684/testReport/junit/org.apache.cassandra.db.lifecycle/ViewTest/testCompaction/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12788) AbstractReplicationStrategy is not extendable from outside the package

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-12788:
---
Reproduced In: 3.0.9, 2.2.8, 2.1.16, 2.0.17, 3.x  (was: 2.0.17, 2.1.16, 
2.2.8, 3.0.9, 3.x)
   Status: Patch Available  (was: Awaiting Feedback)

> AbstractReplicationStrategy is not extendable from outside the package
> --
>
> Key: CASSANDRA-12788
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12788
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Kurt Greaves
> Attachments: 12788.patch
>
>
> The constructor for ARS is package-private (no modifier), which means that 
> custom RS implementations cannot call it. That is, calling super(...) fails 
> unless you create your replication strategy inside 
> org.apache.cassandra.locator. This obviously isn't ideal.
> I propose adding protected to the constructor in ARS. The attached patch was 
> against trunk however I believe it should work for 2.1, 2.2, 3.0, and 3.x.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)