date:20170726


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102750#comment-16102750
 ] 

Amitkumar Ghatwal edited comment on CASSANDRA-13615 at 7/27/17 5:54 AM:


thanks and  acknowledged - [~jjirsa] .how do we push this  
"libsigar-ppc64le-linux.so" to cassandra's mainline "/lib" folder 

"sigar-1.6.4.jar"  in "/lib" does not include a ppc64le library, so had to 
install libsigar-ppc64le-linux.so manually. Since the sigar community has been 
inactive for long, probably we could directly propose to include the ppc64le 
library attached to this jira which is specific to power and wont hamper other 
platforms.

Also dropping this ppc64le object file into a x86-64 directory does not cause 
problems, it's  properly ignored for x86_64.




was (Author: amitkumar_ghatwal):
thanks and  acknowledged - [~jjirsa] .how do we push this  
"libsigar-ppc64le-linux.so" to cassandra's mainline "/lib" folder 

"sigar-1.6.4.jar"  in "/lib" does not include a ppc64le library, so had to 
install libsigar-ppc64le-linux.so manually. Since the sigar community has been 
inactive for long, probably we could directly propose to include the ppc64le 
library attached to this jira which is specific to power and wont hamper other 
platforms.



> Include 'ppc64le' library for sigar-1.6.4.jar
> -
>
> Key: CASSANDRA-13615
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13615
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Libraries
> Environment: # arch
> ppc64le
>Reporter: Amitkumar Ghatwal
>  Labels: easyfix
> Fix For: 3.0.x, 3.11.x, 4.x
>
> Attachments: libsigar-ppc64le-linux.so
>
>
> Hi All,
> sigar-1.6.4.jar does not include a ppc64le library, so we had to install 
> libsigar-ppc64le-linux.so.As the community has been inactive for long 
> (https://github.com/hyperic/sigar), requesting the community to include the 
> ppc64le library directly here.
> Attaching the ppc64le library ( *.so) file to be included under 
> "/lib/sigar-bin". let me know of issues/dependency if any.
> FYI - [~ReiOdaira],[~jjirsa], [~mshuler]
> Regards,
> Amit



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13615) Include 'ppc64le' library for sigar-1.6.4.jar


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102750#comment-16102750
 ] 

Amitkumar Ghatwal commented on CASSANDRA-13615:
---

thanks and  acknowledged - [~jjirsa] .how do we push this  
"libsigar-ppc64le-linux.so" to cassandra's mainline "/lib" folder 

"sigar-1.6.4.jar"  in "/lib" does not include a ppc64le library, so had to 
install libsigar-ppc64le-linux.so manually. Since the sigar community has been 
inactive for long, probably we could directly propose to include the ppc64le 
library attached to this jira which is specific to power and wont hamper other 
platforms.



> Include 'ppc64le' library for sigar-1.6.4.jar
> -
>
> Key: CASSANDRA-13615
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13615
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Libraries
> Environment: # arch
> ppc64le
>Reporter: Amitkumar Ghatwal
>  Labels: easyfix
> Fix For: 3.0.x, 3.11.x, 4.x
>
> Attachments: libsigar-ppc64le-linux.so
>
>
> Hi All,
> sigar-1.6.4.jar does not include a ppc64le library, so we had to install 
> libsigar-ppc64le-linux.so.As the community has been inactive for long 
> (https://github.com/hyperic/sigar), requesting the community to include the 
> ppc64le library directly here.
> Attaching the ppc64le library ( *.so) file to be included under 
> "/lib/sigar-bin". let me know of issues/dependency if any.
> FYI - [~ReiOdaira],[~jjirsa], [~mshuler]
> Regards,
> Amit



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13641) Properly evict pstmts from prepared statements cache


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-13641:
---
Since Version: 3.10

> Properly evict pstmts from prepared statements cache
> 
>
> Key: CASSANDRA-13641
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13641
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
> Fix For: 3.11.1
>
>
> Prepared statements that are evicted from the prepared statements cache are 
> not removed from the underlying table {{system.prepared_statements}}. This 
> can lead to issues during startup.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13615) Include 'ppc64le' library for sigar-1.6.4.jar


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102722#comment-16102722
 ] 

Jeff Jirsa commented on CASSANDRA-13615:


I don't control the hadoop slaves, [~amitkumar_ghatwal] - they're provided by 
the hadoop project.


> Include 'ppc64le' library for sigar-1.6.4.jar
> -
>
> Key: CASSANDRA-13615
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13615
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Libraries
> Environment: # arch
> ppc64le
>Reporter: Amitkumar Ghatwal
>  Labels: easyfix
> Fix For: 3.0.x, 3.11.x, 4.x
>
> Attachments: libsigar-ppc64le-linux.so
>
>
> Hi All,
> sigar-1.6.4.jar does not include a ppc64le library, so we had to install 
> libsigar-ppc64le-linux.so.As the community has been inactive for long 
> (https://github.com/hyperic/sigar), requesting the community to include the 
> ppc64le library directly here.
> Attaching the ppc64le library ( *.so) file to be included under 
> "/lib/sigar-bin". let me know of issues/dependency if any.
> FYI - [~ReiOdaira],[~jjirsa], [~mshuler]
> Regards,
> Amit



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13615) Include 'ppc64le' library for sigar-1.6.4.jar


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102711#comment-16102711
 ] 

Amitkumar Ghatwal commented on CASSANDRA-13615:
---

[~jjirsa] -  can u please check and enable those `ppc64le` slaves -  
hadoop-ppc64le-1/ubuntu-ppc64le on jenkins . 

> Include 'ppc64le' library for sigar-1.6.4.jar
> -
>
> Key: CASSANDRA-13615
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13615
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Libraries
> Environment: # arch
> ppc64le
>Reporter: Amitkumar Ghatwal
>  Labels: easyfix
> Fix For: 3.0.x, 3.11.x, 4.x
>
> Attachments: libsigar-ppc64le-linux.so
>
>
> Hi All,
> sigar-1.6.4.jar does not include a ppc64le library, so we had to install 
> libsigar-ppc64le-linux.so.As the community has been inactive for long 
> (https://github.com/hyperic/sigar), requesting the community to include the 
> ppc64le library directly here.
> Attaching the ppc64le library ( *.so) file to be included under 
> "/lib/sigar-bin". let me know of issues/dependency if any.
> FYI - [~ReiOdaira],[~jjirsa], [~mshuler]
> Regards,
> Amit



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-9988) Introduce leaf-only iterator


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102704#comment-16102704
 ] 

Jay Zhuang edited comment on CASSANDRA-9988 at 7/27/17 4:57 AM:


Hi [~benedict], re-read [your comments 1.5 years' 
ago|https://issues.apache.org/jira/browse/CASSANDRA-9988?focusedCommentId=15117219=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15117219],
 now I see your point:
{quote}
2. We could probably get uniformly better behaviour by implementing exponential 
search for the leaf iterator; this would bring our total number of comparisons 
down from n lg n to n when searching *an approximately equal set of items*, 
without harming our best case complexity.
{quote}
In that case, {{i}} would be very small ({{i}} is where the target is), so if i 
= 1, expSearch ({{log( i ) + log( i )}}) just needs to do 2 comparisons. vs. 
binarySearch 5 comparisons: log(32) = 5 in worst case. Now the question is how 
we should define the test: How many re-seeks should we do? How we choose next 
target? If we randomly select the next target, it's unfair to expSearch, if we 
always select next value (basically use search to iterator the tree) is unfair 
to binSearch. It really depends on the user scenario. Do you have any 
suggestion?


was (Author: jay.zhuang):
Hi [~benedict], re-read [your comments 1.5 years' 
ago|https://issues.apache.org/jira/browse/CASSANDRA-9988?focusedCommentId=15117219=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15117219],
 now I see your point:
{quote}
2. We could probably get uniformly better behaviour by implementing exponential 
search for the leaf iterator; this would bring our total number of comparisons 
down from n lg n to n when searching *an approximately equal set of items*, 
without harming our best case complexity.
{quote}
In that case, {{i}} would be very small ({{i}} is where the target is), so 
expSearch ({{log( i ) + log( i )}}) just needs to do 2 comparisons. vs. 
binarySearch 5 comparisons: log(32) = 5 in worst case. Now the question is how 
we should define the test: How many re-seeks should we do? How we choose next 
target? If we randomly select the next target, it's unfair to expSearch, if we 
always select next value (basically use search to iterator the tree) is unfair 
to binSearch. It really depends on the user scenario. Do you have any 
suggestion?

> Introduce leaf-only iterator
> 
>
> Key: CASSANDRA-9988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9988
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Benedict
>Assignee: Jay Zhuang
>Priority: Minor
>  Labels: patch
> Fix For: 4.0
>
> Attachments: 9988-data.png, 9988-result2.png, 9988-result3.png, 
> 9988-result.png, 9988-test-result3.png, 9988-test-result-expsearch.xlsx, 
> 9988-test-result-raw.png, 9988-test-result.xlsx, 9988-trunk-new.txt, 
> 9988-trunk-new-update.txt, trunk-9988.txt
>
>
> In many cases we have small btrees, small enough to fit in a single leaf 
> page. In this case it _may_ be more efficient to specialise our iterator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-9988) Introduce leaf-only iterator


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102704#comment-16102704
 ] 

Jay Zhuang commented on CASSANDRA-9988:
---

Hi [~benedict], re-read [your comments 1.5 years' 
ago|https://issues.apache.org/jira/browse/CASSANDRA-9988?focusedCommentId=15117219=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15117219],
 now I see your point:
{quote}
2. We could probably get uniformly better behaviour by implementing exponential 
search for the leaf iterator; this would bring our total number of comparisons 
down from n lg n to n when searching *an approximately equal set of items*, 
without harming our best case complexity.
{quote}
In that case, {{i}} would be very small ({{i}} is where the target is), so 
expSearch ({{log( i ) + log( i )}}) just needs to do 2 comparisons. vs. 
binarySearch 5 comparisons: log(32) = 5 in worst case. Now the question is how 
we should define the test: How many re-seeks should we do? How we choose next 
target? If we randomly select the next target, it's unfair to expSearch, if we 
always select next value (basically use search to iterator the tree) is unfair 
to binSearch. It really depends on the user scenario. Do you have any 
suggestion?

> Introduce leaf-only iterator
> 
>
> Key: CASSANDRA-9988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9988
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Benedict
>Assignee: Jay Zhuang
>Priority: Minor
>  Labels: patch
> Fix For: 4.0
>
> Attachments: 9988-data.png, 9988-result2.png, 9988-result3.png, 
> 9988-result.png, 9988-test-result3.png, 9988-test-result-expsearch.xlsx, 
> 9988-test-result-raw.png, 9988-test-result.xlsx, 9988-trunk-new.txt, 
> 9988-trunk-new-update.txt, trunk-9988.txt
>
>
> In many cases we have small btrees, small enough to fit in a single leaf 
> page. In this case it _may_ be more efficient to specialise our iterator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-13711) Invalid writetime for null columns in cqlsh


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa reassigned CASSANDRA-13711:
--

Assignee: Jeff Jirsa

> Invalid writetime for null columns in cqlsh
> ---
>
> Key: CASSANDRA-13711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13711
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> From the user list:
> https://lists.apache.org/thread.html/448731c029eee72e499fc6acd44d257d1671193f850a68521c2c6681@%3Cuser.cassandra.apache.org%3E
> {code}
> (oss-ccm) MacBook-Pro:~ jjirsa$ ccm create test -n 1 -s -v 3.0.10
> Current cluster is now: test
> (oss-ccm) MacBook-Pro:~ jjirsa$ ccm node1 cqlsh
> Connected to test at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.0.10 | CQL spec 3.4.0 | Native protocol v4]
> Use HELP for help.
> cqlsh> CREATE KEYSPACE test WITH replication = {'class':'SimpleStrategy', 
> 'replication_factor': 1};
> cqlsh> CREATE TABLE test.t ( a text primary key, b text );
> cqlsh> insert into test.t(a) values('z');
> cqlsh> insert into test.t(a) values('w');
> cqlsh> insert into test.t(a) values('e');
> cqlsh> insert into test.t(a) values('r');
> cqlsh> insert into test.t(a) values('t');
> cqlsh> select a,b, writetime (b) from test.t;
> a | b | writetime(b)
> ---+--+--
> z | null | null
> e | null | null
> r | null | null
> w | null | null
> t | null | null
> (5 rows)
> cqlsh>
> cqlsh> insert into test.t(a,b) values('t','x');
> cqlsh> insert into test.t(a) values('b');
> cqlsh> select a,b, writetime (b) from test.t;
>  a | b| writetime(b)
> ---+--+--
>  z | null | null
>  e | null | null
>  r | null | null
>  w | null | null
>  t |x | 1500565131354883
>  b | null | 1500565131354883
> (6 rows)
> {code}
> Data on disk:
> {code}
> MacBook-Pro:~ jjirsa$ ~/.ccm/repository/3.0.14/tools/bin/sstabledump 
> /Users/jjirsa/.ccm/test/node1/data0/test/t-bed196006d0511e7904be9daad294861/mc-1-big-Data.db
> [
>   {
> "partition" : {
>   "key" : [ "z" ],
>   "position" : 0
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 20,
> "liveness_info" : { "tstamp" : "2017-07-20T04:41:54.818118Z" },
> "cells" : [ ]
>   }
> ]
>   },
>   {
> "partition" : {
>   "key" : [ "e" ],
>   "position" : 21
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 44,
> "liveness_info" : { "tstamp" : "2017-07-20T04:42:04.288547Z" },
> "cells" : [ ]
>   }
> ]
>   },
>   {
> "partition" : {
>   "key" : [ "r" ],
>   "position" : 45
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 68,
> "liveness_info" : { "tstamp" : "2017-07-20T04:42:08.991417Z" },
> "cells" : [ ]
>   }
> ]
>   },
>   {
> "partition" : {
>   "key" : [ "w" ],
>   "position" : 69
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 92,
> "liveness_info" : { "tstamp" : "2017-07-20T04:41:59.005382Z" },
> "cells" : [ ]
>   }
> ]
>   },
>   {
> "partition" : {
>   "key" : [ "t" ],
>   "position" : 93
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 120,
> "liveness_info" : { "tstamp" : "2017-07-20T15:38:51.354883Z" },
> "cells" : [
>   { "name" : "b", "value" : "x" }
> ]
>   }
> ]
>   },
>   {
> "partition" : {
>   "key" : [ "b" ],
>   "position" : 121
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 146,
> "liveness_info" : { "tstamp" : "2017-07-20T15:39:03.631297Z" },
> "cells" : [ ]
>   }
> ]
>   }
> ]MacBook-Pro:~ jjirsa$
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-9988) Introduce leaf-only iterator


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102593#comment-16102593
 ] 

Jay Zhuang commented on CASSANDRA-9988:
---

{quote}
However, these are iterators, and the expectation is that there will be 
multiple seeks during the iterator's lifetime. I had assumed that your 
searchFound and searchNotFound enumerated (searched for) a sequence of found / 
not found keys. There should probably be such variants.
{quote}
Okay, I can add more tests for that. But I don't think expSearch would make 
multiple-seeks within a leaf node better, as multiple seeks would make the 
{{n}} even smaller. And even in general, here is an article comparing 
binarySearch vs. exponentialSearch (galloping Search): [Beating the binary 
search algorithm – interpolation search, galloping 
search|http://blog.teamleadnet.com/2014/06/beating-binary-search-algorithm.html]
 Gallop(exponentialSearch) is actually not as good as JAVA default API 
{{Arrays.binarySearch}}:!http://3.bp.blogspot.com/-mPLIu6llPBY/U5JoP-6xivI/BMU/D9N5mhLNbOk/s1600/SearchTime.png!

For consistency concern, binarySearch() is wildly used in BTree code:
[BTree.java|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/utils/btree/BTree.java],
 
[NodeCursor.java|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/utils/btree/NodeCursor.java],
 
[TreeCursor.java|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/utils/btree/TreeCursor.java],
 
[NodeBuilder.java|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/utils/btree/NodeBuilder.java],
 
[BTreeRemoval.java|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/utils/btree/BTreeRemoval.java]

> Introduce leaf-only iterator
> 
>
> Key: CASSANDRA-9988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9988
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Benedict
>Assignee: Jay Zhuang
>Priority: Minor
>  Labels: patch
> Fix For: 4.0
>
> Attachments: 9988-data.png, 9988-result2.png, 9988-result3.png, 
> 9988-result.png, 9988-test-result3.png, 9988-test-result-expsearch.xlsx, 
> 9988-test-result-raw.png, 9988-test-result.xlsx, 9988-trunk-new.txt, 
> 9988-trunk-new-update.txt, trunk-9988.txt
>
>
> In many cases we have small btrees, small enough to fit in a single leaf 
> page. In this case it _may_ be more efficient to specialise our iterator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13711) Invalid writetime for null columns in cqlsh


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102556#comment-16102556
 ] 

Jeff Jirsa commented on CASSANDRA-13711:


FWIW, this is not a cqlsh bug, repro's on python drivers, and does not repro on 
2.1



> Invalid writetime for null columns in cqlsh
> ---
>
> Key: CASSANDRA-13711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13711
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jeff Jirsa
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> From the user list:
> https://lists.apache.org/thread.html/448731c029eee72e499fc6acd44d257d1671193f850a68521c2c6681@%3Cuser.cassandra.apache.org%3E
> {code}
> (oss-ccm) MacBook-Pro:~ jjirsa$ ccm create test -n 1 -s -v 3.0.10
> Current cluster is now: test
> (oss-ccm) MacBook-Pro:~ jjirsa$ ccm node1 cqlsh
> Connected to test at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.0.10 | CQL spec 3.4.0 | Native protocol v4]
> Use HELP for help.
> cqlsh> CREATE KEYSPACE test WITH replication = {'class':'SimpleStrategy', 
> 'replication_factor': 1};
> cqlsh> CREATE TABLE test.t ( a text primary key, b text );
> cqlsh> insert into test.t(a) values('z');
> cqlsh> insert into test.t(a) values('w');
> cqlsh> insert into test.t(a) values('e');
> cqlsh> insert into test.t(a) values('r');
> cqlsh> insert into test.t(a) values('t');
> cqlsh> select a,b, writetime (b) from test.t;
> a | b | writetime(b)
> ---+--+--
> z | null | null
> e | null | null
> r | null | null
> w | null | null
> t | null | null
> (5 rows)
> cqlsh>
> cqlsh> insert into test.t(a,b) values('t','x');
> cqlsh> insert into test.t(a) values('b');
> cqlsh> select a,b, writetime (b) from test.t;
>  a | b| writetime(b)
> ---+--+--
>  z | null | null
>  e | null | null
>  r | null | null
>  w | null | null
>  t |x | 1500565131354883
>  b | null | 1500565131354883
> (6 rows)
> {code}
> Data on disk:
> {code}
> MacBook-Pro:~ jjirsa$ ~/.ccm/repository/3.0.14/tools/bin/sstabledump 
> /Users/jjirsa/.ccm/test/node1/data0/test/t-bed196006d0511e7904be9daad294861/mc-1-big-Data.db
> [
>   {
> "partition" : {
>   "key" : [ "z" ],
>   "position" : 0
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 20,
> "liveness_info" : { "tstamp" : "2017-07-20T04:41:54.818118Z" },
> "cells" : [ ]
>   }
> ]
>   },
>   {
> "partition" : {
>   "key" : [ "e" ],
>   "position" : 21
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 44,
> "liveness_info" : { "tstamp" : "2017-07-20T04:42:04.288547Z" },
> "cells" : [ ]
>   }
> ]
>   },
>   {
> "partition" : {
>   "key" : [ "r" ],
>   "position" : 45
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 68,
> "liveness_info" : { "tstamp" : "2017-07-20T04:42:08.991417Z" },
> "cells" : [ ]
>   }
> ]
>   },
>   {
> "partition" : {
>   "key" : [ "w" ],
>   "position" : 69
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 92,
> "liveness_info" : { "tstamp" : "2017-07-20T04:41:59.005382Z" },
> "cells" : [ ]
>   }
> ]
>   },
>   {
> "partition" : {
>   "key" : [ "t" ],
>   "position" : 93
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 120,
> "liveness_info" : { "tstamp" : "2017-07-20T15:38:51.354883Z" },
> "cells" : [
>   { "name" : "b", "value" : "x" }
> ]
>   }
> ]
>   },
>   {
> "partition" : {
>   "key" : [ "b" ],
>   "position" : 121
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 146,
> "liveness_info" : { "tstamp" : "2017-07-20T15:39:03.631297Z" },
> "cells" : [ ]
>   }
> ]
>   }
> ]MacBook-Pro:~ jjirsa$
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13733) nodetool toptables


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-13733:
---
Component/s: Observability
 Metrics

> nodetool toptables
> --
>
> Key: CASSANDRA-13733
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13733
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Metrics, Observability
>Reporter: Jon Haddad
>  Labels: lhf, low-hanging-fruit
>
> In the same way toppartitions samples the activity on a table, toptables 
> should be able to report which tables are used the most.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13733) nodetool toptables


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-13733:
---
Labels: lhf low-hanging-fruit  (was: )

> nodetool toptables
> --
>
> Key: CASSANDRA-13733
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13733
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Metrics, Observability
>Reporter: Jon Haddad
>  Labels: lhf, low-hanging-fruit
>
> In the same way toppartitions samples the activity on a table, toptables 
> should be able to report which tables are used the most.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13733) nodetool toptables

2017-07-26 Thread Chris Lohfink (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102545#comment-16102545
 ] 

Chris Lohfink commented on CASSANDRA-13733:
---

This could be as simple as printing top X tables by looking at their Latency 
metrics (rate in particular), that would require no changes just a view of the 
data. a new jmx operation could make it a lot faster so it doesnt require N jmx 
calls (n=number of tables).

> nodetool toptables
> --
>
> Key: CASSANDRA-13733
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13733
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jon Haddad
>
> In the same way toppartitions samples the activity on a table, toptables 
> should be able to report which tables are used the most.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13732) GH PR #131 - Refactoring to primitive functional interfaces in AuthCache.java


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102505#comment-16102505
 ] 

Jeff Jirsa commented on CASSANDRA-13732:


Assigning to myself, actual author is {{Ameya Ketkar}}

Pushed to CI here: 
Unit tests: https://circleci.com/gh/jeffjirsa/cassandra/tree/cassandra-13732
Dtests : 
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/149/
 (I think, but ASF Jenkins is unhappy right now)


> GH PR #131 - Refactoring to primitive functional interfaces in AuthCache.java
> -
>
> Key: CASSANDRA-13732
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13732
> Project: Cassandra
>  Issue Type: Bug
>  Components: Auth
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
> Fix For: 4.x
>
>
> https://github.com/apache/cassandra/pull/131
> Refactor to avoid unnecessary boxing/unboxing in auth cache.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-9988) Introduce leaf-only iterator


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102503#comment-16102503
 ] 

Jay Zhuang commented on CASSANDRA-9988:
---

{quote}
Also, what exactly are these graphs showing us?
{quote}
It's the throughput improvements before and after the change ({{($before - 
$after)/$before}}), for example for {{SearchFound}} test, when the treeSize is 
1, it has negative performance impact about {{-18%}}, when the treeSize is 16 
or 32, it is about -65%. I attached the excel files with raw data to the 
ticket.[^9988-test-result.xlsx][^9988-test-result-expsearch.xlsx]



> Introduce leaf-only iterator
> 
>
> Key: CASSANDRA-9988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9988
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Benedict
>Assignee: Jay Zhuang
>Priority: Minor
>  Labels: patch
> Fix For: 4.0
>
> Attachments: 9988-data.png, 9988-result2.png, 9988-result3.png, 
> 9988-result.png, 9988-test-result3.png, 9988-test-result-expsearch.xlsx, 
> 9988-test-result-raw.png, 9988-test-result.xlsx, 9988-trunk-new.txt, 
> 9988-trunk-new-update.txt, trunk-9988.txt
>
>
> In many cases we have small btrees, small enough to fit in a single leaf 
> page. In this case it _may_ be more efficient to specialise our iterator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-13732) GH PR #131 - Refactoring to primitive functional interfaces in AuthCache.java


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa reassigned CASSANDRA-13732:
--

Assignee: Jeff Jirsa

> GH PR #131 - Refactoring to primitive functional interfaces in AuthCache.java
> -
>
> Key: CASSANDRA-13732
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13732
> Project: Cassandra
>  Issue Type: Bug
>  Components: Auth
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
> Fix For: 4.x
>
>
> https://github.com/apache/cassandra/pull/131
> Refactor to avoid unnecessary boxing/unboxing in auth cache.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13696) Digest mismatch Exception if hints file has UnknownColumnFamily


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-13696:
---
   Resolution: Fixed
Fix Version/s: (was: 3.11.x)
   (was: 4.x)
   (was: 3.0.x)
   4.0
   3.11.1
   3.0.15
   Status: Resolved  (was: Ready to Commit)

Committed to 3.0 as {{f919cf4a478cdbcb7864e8b47814a40bfcb343a7}} and merged to 
3.11 and trunk. Thanks [~jay.zhuang], good find!


> Digest mismatch Exception if hints file has UnknownColumnFamily
> ---
>
> Key: CASSANDRA-13696
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13696
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Blocker
> Fix For: 3.0.15, 3.11.1, 4.0
>
>
> {noformat}
> WARN  [HintsDispatcher:2] 2017-07-16 22:00:32,579 HintsReader.java:235 - 
> Failed to read a hint for /127.0.0.2: a2b7daf1-a6a4-4dfc-89de-32d12d2d48b0 - 
> table with id 3882bbb0-6a71-11e7-9bca-2759083e3964 is unknown in file 
> a2b7daf1-a6a4-4dfc-89de-32d12d2d48b0-1500242103097-1.hints
> ERROR [HintsDispatcher:2] 2017-07-16 22:00:32,580 
> HintsDispatchExecutor.java:234 - Failed to dispatch hints file 
> a2b7daf1-a6a4-4dfc-89de-32d12d2d48b0-1500242103097-1.hints: file is corrupted 
> ({})
> org.apache.cassandra.io.FSReadError: java.io.IOException: Digest mismatch 
> exception
> at 
> org.apache.cassandra.hints.HintsReader$HintsIterator.computeNext(HintsReader.java:199)
>  ~[main/:na]
> at 
> org.apache.cassandra.hints.HintsReader$HintsIterator.computeNext(HintsReader.java:164)
>  ~[main/:na]
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[main/:na]
> at 
> org.apache.cassandra.hints.HintsDispatcher.sendHints(HintsDispatcher.java:157)
>  ~[main/:na]
> at 
> org.apache.cassandra.hints.HintsDispatcher.sendHintsAndAwait(HintsDispatcher.java:139)
>  ~[main/:na]
> at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:123) 
> ~[main/:na]
> at 
> org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:95) 
> ~[main/:na]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.deliver(HintsDispatchExecutor.java:268)
>  [main/:na]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:251)
>  [main/:na]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:229)
>  [main/:na]
> at 
> org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:208)
>  [main/:na]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_111]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_111]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_111]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_111]
> at 
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
>  [main/:na]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_111]
> Caused by: java.io.IOException: Digest mismatch exception
> at 
> org.apache.cassandra.hints.HintsReader$HintsIterator.computeNextInternal(HintsReader.java:216)
>  ~[main/:na]
> at 
> org.apache.cassandra.hints.HintsReader$HintsIterator.computeNext(HintsReader.java:190)
>  ~[main/:na]
> ... 16 common frames omitted
> {noformat}
> It causes multiple cassandra nodes stop [by 
> default|https://github.com/apache/cassandra/blob/cassandra-3.0/conf/cassandra.yaml#L188].
> Here is the reproduce steps on a 3 nodes cluster, RF=3:
> 1. stop node1
> 2. send some data with quorum (or one), it will generate hints file on 
> node2/node3
> 3. drop the table
> 4. start node1
> node2/node3 will report "corrupted hints file" and stop. The impact is very 
> bad for a large cluster, when it happens, almost all the nodes are down at 
> the same time and we have to remove all the hints files (which contain the 
> dropped table) to bring the node back.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-9988) Introduce leaf-only iterator


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jay Zhuang updated CASSANDRA-9988:
--
Attachment: 9988-test-result-expsearch.xlsx

> Introduce leaf-only iterator
> 
>
> Key: CASSANDRA-9988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9988
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Benedict
>Assignee: Jay Zhuang
>Priority: Minor
>  Labels: patch
> Fix For: 4.0
>
> Attachments: 9988-data.png, 9988-result2.png, 9988-result3.png, 
> 9988-result.png, 9988-test-result3.png, 9988-test-result-expsearch.xlsx, 
> 9988-test-result-raw.png, 9988-test-result.xlsx, 9988-trunk-new.txt, 
> 9988-trunk-new-update.txt, trunk-9988.txt
>
>
> In many cases we have small btrees, small enough to fit in a single leaf 
> page. In this case it _may_ be more efficient to specialise our iterator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-9988) Introduce leaf-only iterator


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jay Zhuang updated CASSANDRA-9988:
--
Attachment: (was: 9988-test-result2.xlsx)

> Introduce leaf-only iterator
> 
>
> Key: CASSANDRA-9988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9988
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Benedict
>Assignee: Jay Zhuang
>Priority: Minor
>  Labels: patch
> Fix For: 4.0
>
> Attachments: 9988-data.png, 9988-result2.png, 9988-result3.png, 
> 9988-result.png, 9988-test-result3.png, 9988-test-result-expsearch.xlsx, 
> 9988-test-result-raw.png, 9988-test-result.xlsx, 9988-trunk-new.txt, 
> 9988-trunk-new-update.txt, trunk-9988.txt
>
>
> In many cases we have small btrees, small enough to fit in a single leaf 
> page. In this case it _may_ be more efficient to specialise our iterator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[1/6] cassandra git commit: Fix Digest mismatch Exception if hints file has UnknownColumnFamily

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 19adec1f8 -> f919cf4a4
  refs/heads/cassandra-3.11 1aa14bc46 -> 7ff4a653f
  refs/heads/trunk cf4a0576a -> 33f94dec0


Fix Digest mismatch Exception if hints file has UnknownColumnFamily

Patch by Jay Zhuang; Reviewed by Jeff Jirsa for CASSANDRA-13696


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f919cf4a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f919cf4a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f919cf4a

Branch: refs/heads/cassandra-3.0
Commit: f919cf4a478cdbcb7864e8b47814a40bfcb343a7
Parents: 19adec1
Author: Jeff Jirsa 
Authored: Wed Jul 26 16:56:11 2017 -0700
Committer: Jeff Jirsa 
Committed: Wed Jul 26 16:56:11 2017 -0700

--
 CHANGES.txt |   1 +
 .../cassandra/hints/ChecksummedDataInput.java   |   2 +-
 .../apache/cassandra/hints/HintsDescriptor.java |   2 +-
 .../org/apache/cassandra/hints/HintsReader.java |   2 +-
 .../apache/cassandra/hints/HintsReaderTest.java | 172 +++
 5 files changed, 176 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f919cf4a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3247277..5e6d189 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,6 +1,7 @@
 3.0.15
  * Fixed ambiguous output of nodetool tablestats command (CASSANDRA-13722)
  * JMXEnabledThreadPoolExecutor with corePoolSize equal to maxPoolSize 
(Backport CASSANDRA-13329)
+ * Fix Digest mismatch Exception if hints file has UnknownColumnFamily 
(CASSANDRA-13696)
  * Purge tombstones created by expired cells (CASSANDRA-13643)
  * Make concat work with iterators that have different subsets of columns 
(CASSANDRA-13482)
  * Set test.runners based on cores and memory size (CASSANDRA-13078)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f919cf4a/src/java/org/apache/cassandra/hints/ChecksummedDataInput.java
--
diff --git a/src/java/org/apache/cassandra/hints/ChecksummedDataInput.java 
b/src/java/org/apache/cassandra/hints/ChecksummedDataInput.java
index 095d7f4..39f46a4 100644
--- a/src/java/org/apache/cassandra/hints/ChecksummedDataInput.java
+++ b/src/java/org/apache/cassandra/hints/ChecksummedDataInput.java
@@ -107,7 +107,7 @@ public class ChecksummedDataInput extends 
RandomAccessReader.RandomAccessReaderW
 {
 updateCrc();
 
-// we must diable crc updates in case we rebuffer
+// we must disable crc updates in case we rebuffer
 // when called source.readInt()
 crcUpdateDisabled = true;
 return ((int) crc.getValue()) == readInt();

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f919cf4a/src/java/org/apache/cassandra/hints/HintsDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/hints/HintsDescriptor.java 
b/src/java/org/apache/cassandra/hints/HintsDescriptor.java
index f5296b3..916da4e 100644
--- a/src/java/org/apache/cassandra/hints/HintsDescriptor.java
+++ b/src/java/org/apache/cassandra/hints/HintsDescriptor.java
@@ -120,7 +120,7 @@ final class HintsDescriptor
 switch (hintsVersion)
 {
 case VERSION_30:
-return MessagingService.VERSION_30;
+return MessagingService.FORCE_3_0_PROTOCOL_VERSION ? 
MessagingService.VERSION_30 : MessagingService.VERSION_3014;
 default:
 throw new AssertionError();
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f919cf4a/src/java/org/apache/cassandra/hints/HintsReader.java
--
diff --git a/src/java/org/apache/cassandra/hints/HintsReader.java 
b/src/java/org/apache/cassandra/hints/HintsReader.java
index 8104051..973a15e 100644
--- a/src/java/org/apache/cassandra/hints/HintsReader.java
+++ b/src/java/org/apache/cassandra/hints/HintsReader.java
@@ -239,7 +239,7 @@ class HintsReader implements AutoCloseable, 
Iterable
 descriptor.fileName());
 input.skipBytes(Ints.checkedCast(size - 
input.bytesPastLimit()));
 
-return null;
+hint = null; // set the return value to null and let following 
code to update/check the CRC
 }
 
 if (input.checkCrc())

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f919cf4a/test/unit/org/apache/cassandra/hints/HintsReaderTest.java
--
diff --git

[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7ff4a653
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7ff4a653
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7ff4a653

Branch: refs/heads/trunk
Commit: 7ff4a653f24ed4951a56859de2e5032d802693bd
Parents: 1aa14bc f919cf4
Author: Jeff Jirsa 
Authored: Wed Jul 26 16:56:51 2017 -0700
Committer: Jeff Jirsa 
Committed: Wed Jul 26 16:57:34 2017 -0700

--
 CHANGES.txt |   1 +
 .../cassandra/hints/ChecksummedDataInput.java   |   2 +-
 .../apache/cassandra/hints/HintsDescriptor.java |   2 +-
 .../org/apache/cassandra/hints/HintsReader.java |   2 +-
 .../apache/cassandra/hints/HintsReaderTest.java | 172 +++
 5 files changed, 176 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7ff4a653/CHANGES.txt
--
diff --cc CHANGES.txt
index cfa4a24,5e6d189..dd8f88b
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,11 -1,7 +1,12 @@@
 +3.11.1
 + * "ignore" option is ignored in sstableloader (CASSANDRA-13721)
 + * Deadlock in AbstractCommitLogSegmentManager (CASSANDRA-13652)
 + * Duplicate the buffer before passing it to analyser in SASI operation 
(CASSANDRA-13512)
 + * Properly evict pstmts from prepared statements cache (CASSANDRA-13641)
 +Merged from 3.0:
  3.0.15
   * Fixed ambiguous output of nodetool tablestats command (CASSANDRA-13722)
 - * JMXEnabledThreadPoolExecutor with corePoolSize equal to maxPoolSize 
(Backport CASSANDRA-13329)
+  * Fix Digest mismatch Exception if hints file has UnknownColumnFamily 
(CASSANDRA-13696)
   * Purge tombstones created by expired cells (CASSANDRA-13643)
   * Make concat work with iterators that have different subsets of columns 
(CASSANDRA-13482)
   * Set test.runners based on cores and memory size (CASSANDRA-13078)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7ff4a653/src/java/org/apache/cassandra/hints/ChecksummedDataInput.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7ff4a653/src/java/org/apache/cassandra/hints/HintsDescriptor.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7ff4a653/src/java/org/apache/cassandra/hints/HintsReader.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/33f94dec
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/33f94dec
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/33f94dec

Branch: refs/heads/trunk
Commit: 33f94dec093d70f1276fc86a5051ee04b1fa058e
Parents: cf4a057 7ff4a65
Author: Jeff Jirsa 
Authored: Wed Jul 26 16:57:57 2017 -0700
Committer: Jeff Jirsa 
Committed: Wed Jul 26 16:59:20 2017 -0700

--
 CHANGES.txt |   1 +
 .../cassandra/hints/ChecksummedDataInput.java   |   2 +-
 .../org/apache/cassandra/hints/HintsReader.java |   2 +-
 .../apache/cassandra/hints/HintsReaderTest.java | 174 +++
 4 files changed, 177 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/33f94dec/CHANGES.txt
--
diff --cc CHANGES.txt
index 7189cb4,dd8f88b..2500043
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -107,7 -4,9 +107,8 @@@
   * Duplicate the buffer before passing it to analyser in SASI operation 
(CASSANDRA-13512)
   * Properly evict pstmts from prepared statements cache (CASSANDRA-13641)
  Merged from 3.0:
 -3.0.15
   * Fixed ambiguous output of nodetool tablestats command (CASSANDRA-13722)
+  * Fix Digest mismatch Exception if hints file has UnknownColumnFamily 
(CASSANDRA-13696)
   * Purge tombstones created by expired cells (CASSANDRA-13643)
   * Make concat work with iterators that have different subsets of columns 
(CASSANDRA-13482)
   * Set test.runners based on cores and memory size (CASSANDRA-13078)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/33f94dec/src/java/org/apache/cassandra/hints/HintsReader.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/33f94dec/test/unit/org/apache/cassandra/hints/HintsReaderTest.java
--
diff --cc test/unit/org/apache/cassandra/hints/HintsReaderTest.java
index 000,70cf6e7..d95bd56
mode 00,100644..100644
--- a/test/unit/org/apache/cassandra/hints/HintsReaderTest.java
+++ b/test/unit/org/apache/cassandra/hints/HintsReaderTest.java
@@@ -1,0 -1,172 +1,174 @@@
+ /*
+  * Licensed to the Apache Software Foundation (ASF) under one
+  * or more contributor license agreements.  See the NOTICE file
+  * distributed with this work for additional information
+  * regarding copyright ownership.  The ASF licenses this file
+  * to you under the Apache License, Version 2.0 (the
+  * "License"); you may not use this file except in compliance
+  * with the License.  You may obtain a copy of the License at
+  *
+  * http://www.apache.org/licenses/LICENSE-2.0
+  *
+  * Unless required by applicable law or agreed to in writing, software
+  * distributed under the License is distributed on an "AS IS" BASIS,
+  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  * See the License for the specific language governing permissions and
+  * limitations under the License.
+  */
+ 
+ package org.apache.cassandra.hints;
+ 
+ import java.io.File;
+ import java.io.IOException;
+ import java.nio.ByteBuffer;
+ import java.nio.file.Files;
+ import java.util.Iterator;
+ import java.util.UUID;
+ import java.util.concurrent.TimeUnit;
+ 
+ import com.google.common.collect.Iterables;
+ import org.junit.BeforeClass;
+ import org.junit.Test;
+ 
+ import org.apache.cassandra.SchemaLoader;
 -import org.apache.cassandra.config.CFMetaData;
 -import org.apache.cassandra.config.Schema;
+ import org.apache.cassandra.db.Mutation;
+ import org.apache.cassandra.db.RowUpdateBuilder;
+ import org.apache.cassandra.db.rows.Cell;
+ import org.apache.cassandra.db.rows.Row;
+ import org.apache.cassandra.io.util.FileUtils;
+ import org.apache.cassandra.schema.KeyspaceParams;
++import org.apache.cassandra.schema.MigrationManager;
++import org.apache.cassandra.schema.Schema;
++import org.apache.cassandra.schema.TableMetadata;
+ 
+ import static junit.framework.Assert.assertEquals;
+ import static junit.framework.Assert.assertNotNull;
+ import static org.apache.cassandra.Util.dk;
+ import static org.apache.cassandra.utils.ByteBufferUtil.bytes;
+ 
+ public class HintsReaderTest
+ {
+ private static final String CF_STANDARD1 = "Standard1";
+ private static final String CF_STANDARD2 = "Standard2";
+ 
+ private static HintsDescriptor descriptor;
+ 
+ private static File directory;
+ 
+ @BeforeClass
+ public static void defineSchema() throws Exception
+ {
+ SchemaLoader.prepareServer();
+ 
+ descriptor = new HintsDescriptor(UUID.randomUUID(),

[2/6] cassandra git commit: Fix Digest mismatch Exception if hints file has UnknownColumnFamily

Fix Digest mismatch Exception if hints file has UnknownColumnFamily

Patch by Jay Zhuang; Reviewed by Jeff Jirsa for CASSANDRA-13696


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f919cf4a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f919cf4a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f919cf4a

Branch: refs/heads/cassandra-3.11
Commit: f919cf4a478cdbcb7864e8b47814a40bfcb343a7
Parents: 19adec1
Author: Jeff Jirsa 
Authored: Wed Jul 26 16:56:11 2017 -0700
Committer: Jeff Jirsa 
Committed: Wed Jul 26 16:56:11 2017 -0700

--
 CHANGES.txt |   1 +
 .../cassandra/hints/ChecksummedDataInput.java   |   2 +-
 .../apache/cassandra/hints/HintsDescriptor.java |   2 +-
 .../org/apache/cassandra/hints/HintsReader.java |   2 +-
 .../apache/cassandra/hints/HintsReaderTest.java | 172 +++
 5 files changed, 176 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f919cf4a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3247277..5e6d189 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,6 +1,7 @@
 3.0.15
  * Fixed ambiguous output of nodetool tablestats command (CASSANDRA-13722)
  * JMXEnabledThreadPoolExecutor with corePoolSize equal to maxPoolSize 
(Backport CASSANDRA-13329)
+ * Fix Digest mismatch Exception if hints file has UnknownColumnFamily 
(CASSANDRA-13696)
  * Purge tombstones created by expired cells (CASSANDRA-13643)
  * Make concat work with iterators that have different subsets of columns 
(CASSANDRA-13482)
  * Set test.runners based on cores and memory size (CASSANDRA-13078)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f919cf4a/src/java/org/apache/cassandra/hints/ChecksummedDataInput.java
--
diff --git a/src/java/org/apache/cassandra/hints/ChecksummedDataInput.java 
b/src/java/org/apache/cassandra/hints/ChecksummedDataInput.java
index 095d7f4..39f46a4 100644
--- a/src/java/org/apache/cassandra/hints/ChecksummedDataInput.java
+++ b/src/java/org/apache/cassandra/hints/ChecksummedDataInput.java
@@ -107,7 +107,7 @@ public class ChecksummedDataInput extends 
RandomAccessReader.RandomAccessReaderW
 {
 updateCrc();
 
-// we must diable crc updates in case we rebuffer
+// we must disable crc updates in case we rebuffer
 // when called source.readInt()
 crcUpdateDisabled = true;
 return ((int) crc.getValue()) == readInt();

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f919cf4a/src/java/org/apache/cassandra/hints/HintsDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/hints/HintsDescriptor.java 
b/src/java/org/apache/cassandra/hints/HintsDescriptor.java
index f5296b3..916da4e 100644
--- a/src/java/org/apache/cassandra/hints/HintsDescriptor.java
+++ b/src/java/org/apache/cassandra/hints/HintsDescriptor.java
@@ -120,7 +120,7 @@ final class HintsDescriptor
 switch (hintsVersion)
 {
 case VERSION_30:
-return MessagingService.VERSION_30;
+return MessagingService.FORCE_3_0_PROTOCOL_VERSION ? 
MessagingService.VERSION_30 : MessagingService.VERSION_3014;
 default:
 throw new AssertionError();
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f919cf4a/src/java/org/apache/cassandra/hints/HintsReader.java
--
diff --git a/src/java/org/apache/cassandra/hints/HintsReader.java 
b/src/java/org/apache/cassandra/hints/HintsReader.java
index 8104051..973a15e 100644
--- a/src/java/org/apache/cassandra/hints/HintsReader.java
+++ b/src/java/org/apache/cassandra/hints/HintsReader.java
@@ -239,7 +239,7 @@ class HintsReader implements AutoCloseable, 
Iterable
 descriptor.fileName());
 input.skipBytes(Ints.checkedCast(size - 
input.bytesPastLimit()));
 
-return null;
+hint = null; // set the return value to null and let following 
code to update/check the CRC
 }
 
 if (input.checkCrc())

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f919cf4a/test/unit/org/apache/cassandra/hints/HintsReaderTest.java
--
diff --git a/test/unit/org/apache/cassandra/hints/HintsReaderTest.java 
b/test/unit/org/apache/cassandra/hints/HintsReaderTest.java
new file mode 100644
index 000..70cf6e7
--- /dev/null
+++

[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7ff4a653
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7ff4a653
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7ff4a653

Branch: refs/heads/cassandra-3.11
Commit: 7ff4a653f24ed4951a56859de2e5032d802693bd
Parents: 1aa14bc f919cf4
Author: Jeff Jirsa 
Authored: Wed Jul 26 16:56:51 2017 -0700
Committer: Jeff Jirsa 
Committed: Wed Jul 26 16:57:34 2017 -0700

--
 CHANGES.txt |   1 +
 .../cassandra/hints/ChecksummedDataInput.java   |   2 +-
 .../apache/cassandra/hints/HintsDescriptor.java |   2 +-
 .../org/apache/cassandra/hints/HintsReader.java |   2 +-
 .../apache/cassandra/hints/HintsReaderTest.java | 172 +++
 5 files changed, 176 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7ff4a653/CHANGES.txt
--
diff --cc CHANGES.txt
index cfa4a24,5e6d189..dd8f88b
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,11 -1,7 +1,12 @@@
 +3.11.1
 + * "ignore" option is ignored in sstableloader (CASSANDRA-13721)
 + * Deadlock in AbstractCommitLogSegmentManager (CASSANDRA-13652)
 + * Duplicate the buffer before passing it to analyser in SASI operation 
(CASSANDRA-13512)
 + * Properly evict pstmts from prepared statements cache (CASSANDRA-13641)
 +Merged from 3.0:
  3.0.15
   * Fixed ambiguous output of nodetool tablestats command (CASSANDRA-13722)
 - * JMXEnabledThreadPoolExecutor with corePoolSize equal to maxPoolSize 
(Backport CASSANDRA-13329)
+  * Fix Digest mismatch Exception if hints file has UnknownColumnFamily 
(CASSANDRA-13696)
   * Purge tombstones created by expired cells (CASSANDRA-13643)
   * Make concat work with iterators that have different subsets of columns 
(CASSANDRA-13482)
   * Set test.runners based on cores and memory size (CASSANDRA-13078)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7ff4a653/src/java/org/apache/cassandra/hints/ChecksummedDataInput.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7ff4a653/src/java/org/apache/cassandra/hints/HintsDescriptor.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7ff4a653/src/java/org/apache/cassandra/hints/HintsReader.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[3/6] cassandra git commit: Fix Digest mismatch Exception if hints file has UnknownColumnFamily

Fix Digest mismatch Exception if hints file has UnknownColumnFamily

Patch by Jay Zhuang; Reviewed by Jeff Jirsa for CASSANDRA-13696


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f919cf4a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f919cf4a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f919cf4a

Branch: refs/heads/trunk
Commit: f919cf4a478cdbcb7864e8b47814a40bfcb343a7
Parents: 19adec1
Author: Jeff Jirsa 
Authored: Wed Jul 26 16:56:11 2017 -0700
Committer: Jeff Jirsa 
Committed: Wed Jul 26 16:56:11 2017 -0700

--
 CHANGES.txt |   1 +
 .../cassandra/hints/ChecksummedDataInput.java   |   2 +-
 .../apache/cassandra/hints/HintsDescriptor.java |   2 +-
 .../org/apache/cassandra/hints/HintsReader.java |   2 +-
 .../apache/cassandra/hints/HintsReaderTest.java | 172 +++
 5 files changed, 176 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f919cf4a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3247277..5e6d189 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,6 +1,7 @@
 3.0.15
  * Fixed ambiguous output of nodetool tablestats command (CASSANDRA-13722)
  * JMXEnabledThreadPoolExecutor with corePoolSize equal to maxPoolSize 
(Backport CASSANDRA-13329)
+ * Fix Digest mismatch Exception if hints file has UnknownColumnFamily 
(CASSANDRA-13696)
  * Purge tombstones created by expired cells (CASSANDRA-13643)
  * Make concat work with iterators that have different subsets of columns 
(CASSANDRA-13482)
  * Set test.runners based on cores and memory size (CASSANDRA-13078)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f919cf4a/src/java/org/apache/cassandra/hints/ChecksummedDataInput.java
--
diff --git a/src/java/org/apache/cassandra/hints/ChecksummedDataInput.java 
b/src/java/org/apache/cassandra/hints/ChecksummedDataInput.java
index 095d7f4..39f46a4 100644
--- a/src/java/org/apache/cassandra/hints/ChecksummedDataInput.java
+++ b/src/java/org/apache/cassandra/hints/ChecksummedDataInput.java
@@ -107,7 +107,7 @@ public class ChecksummedDataInput extends 
RandomAccessReader.RandomAccessReaderW
 {
 updateCrc();
 
-// we must diable crc updates in case we rebuffer
+// we must disable crc updates in case we rebuffer
 // when called source.readInt()
 crcUpdateDisabled = true;
 return ((int) crc.getValue()) == readInt();

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f919cf4a/src/java/org/apache/cassandra/hints/HintsDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/hints/HintsDescriptor.java 
b/src/java/org/apache/cassandra/hints/HintsDescriptor.java
index f5296b3..916da4e 100644
--- a/src/java/org/apache/cassandra/hints/HintsDescriptor.java
+++ b/src/java/org/apache/cassandra/hints/HintsDescriptor.java
@@ -120,7 +120,7 @@ final class HintsDescriptor
 switch (hintsVersion)
 {
 case VERSION_30:
-return MessagingService.VERSION_30;
+return MessagingService.FORCE_3_0_PROTOCOL_VERSION ? 
MessagingService.VERSION_30 : MessagingService.VERSION_3014;
 default:
 throw new AssertionError();
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f919cf4a/src/java/org/apache/cassandra/hints/HintsReader.java
--
diff --git a/src/java/org/apache/cassandra/hints/HintsReader.java 
b/src/java/org/apache/cassandra/hints/HintsReader.java
index 8104051..973a15e 100644
--- a/src/java/org/apache/cassandra/hints/HintsReader.java
+++ b/src/java/org/apache/cassandra/hints/HintsReader.java
@@ -239,7 +239,7 @@ class HintsReader implements AutoCloseable, 
Iterable
 descriptor.fileName());
 input.skipBytes(Ints.checkedCast(size - 
input.bytesPastLimit()));
 
-return null;
+hint = null; // set the return value to null and let following 
code to update/check the CRC
 }
 
 if (input.checkCrc())

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f919cf4a/test/unit/org/apache/cassandra/hints/HintsReaderTest.java
--
diff --git a/test/unit/org/apache/cassandra/hints/HintsReaderTest.java 
b/test/unit/org/apache/cassandra/hints/HintsReaderTest.java
new file mode 100644
index 000..70cf6e7
--- /dev/null
+++

[jira] [Updated] (CASSANDRA-9988) Introduce leaf-only iterator


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jay Zhuang updated CASSANDRA-9988:
--
Attachment: 9988-test-result.xlsx
9988-test-result2.xlsx

> Introduce leaf-only iterator
> 
>
> Key: CASSANDRA-9988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9988
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Benedict
>Assignee: Jay Zhuang
>Priority: Minor
>  Labels: patch
> Fix For: 4.0
>
> Attachments: 9988-data.png, 9988-result2.png, 9988-result3.png, 
> 9988-result.png, 9988-test-result2.xlsx, 9988-test-result3.png, 
> 9988-test-result-raw.png, 9988-test-result.xlsx, 9988-trunk-new.txt, 
> 9988-trunk-new-update.txt, trunk-9988.txt
>
>
> In many cases we have small btrees, small enough to fit in a single leaf 
> page. In this case it _may_ be more efficient to specialise our iterator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-9988) Introduce leaf-only iterator


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jay Zhuang updated CASSANDRA-9988:
--
Attachment: 9988-test-result-raw.png

> Introduce leaf-only iterator
> 
>
> Key: CASSANDRA-9988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9988
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Benedict
>Assignee: Jay Zhuang
>Priority: Minor
>  Labels: patch
> Fix For: 4.0
>
> Attachments: 9988-data.png, 9988-result2.png, 9988-result3.png, 
> 9988-result.png, 9988-test-result3.png, 9988-test-result-raw.png, 
> 9988-trunk-new.txt, 9988-trunk-new-update.txt, trunk-9988.txt
>
>
> In many cases we have small btrees, small enough to fit in a single leaf 
> page. In this case it _may_ be more efficient to specialise our iterator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13699) Allow to set batch_size_warn_threshold_in_kb via JMX


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-13699:
---
   Resolution: Fixed
Fix Version/s: (was: 4.x)
   4.0
   Status: Resolved  (was: Ready to Commit)

Thanks! Committed as {{cf4a0576a6f2b8f2d828a8b14140f212803adb7c}}


> Allow to set batch_size_warn_threshold_in_kb via JMX
> 
>
> Key: CASSANDRA-13699
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13699
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Romain Hardouin
>Assignee: Romain Hardouin
>Priority: Minor
> Fix For: 4.0
>
> Attachments: 13699-trunk.txt
>
>
> We can set {{batch_size_fail_threshold_in_kb}} via JMX but not 
> {{batch_size_warn_threshold_in_kb}}. 
> The patch allows to set it dynamically and adds a INFO log for both 
> thresholds.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13699) Allow to set batch_size_warn_threshold_in_kb via JMX


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-13699:
---
Status: Ready to Commit  (was: Patch Available)

> Allow to set batch_size_warn_threshold_in_kb via JMX
> 
>
> Key: CASSANDRA-13699
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13699
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Romain Hardouin
>Assignee: Romain Hardouin
>Priority: Minor
> Fix For: 4.0
>
> Attachments: 13699-trunk.txt
>
>
> We can set {{batch_size_fail_threshold_in_kb}} via JMX but not 
> {{batch_size_warn_threshold_in_kb}}. 
> The patch allows to set it dynamically and adds a INFO log for both 
> thresholds.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

cassandra git commit: Allow to set batch_size_warn_threshold_in_kb via JMX

Repository: cassandra
Updated Branches:
  refs/heads/trunk 6e19e81db -> cf4a0576a


Allow to set batch_size_warn_threshold_in_kb via JMX

patch by Romain Hardouin; reviewed by Jeff Jirsa for CASSANDRA-13699


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cf4a0576
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cf4a0576
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cf4a0576

Branch: refs/heads/trunk
Commit: cf4a0576a6f2b8f2d828a8b14140f212803adb7c
Parents: 6e19e81
Author: Romain Hardouin 
Authored: Wed Jul 19 22:10:59 2017 +0200
Committer: Jeff Jirsa 
Committed: Wed Jul 26 16:29:53 2017 -0700

--
 CHANGES.txt |  5 +++--
 .../org/apache/cassandra/config/DatabaseDescriptor.java |  5 +
 .../org/apache/cassandra/service/StorageService.java| 12 
 .../apache/cassandra/service/StorageServiceMBean.java   |  5 +
 4 files changed, 25 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf4a0576/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b4880fb..7189cb4 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,7 +1,8 @@
 4.0
+ * batch_size_warn_threshold_in_kb can now be set at runtime (CASSANDRA-13699)
  * Avoid always rebuilding secondary indexes at startup (CASSANDRA-13725)
- * Upgrade JMH from 1.13 to 1.19
- * Upgrade SLF4J from 1.7.7 to 1.7.25
+ * Upgrade JMH from 1.13 to 1.19 (CASSANDRA-13727)
+ * Upgrade SLF4J from 1.7.7 to 1.7.25 (CASSANDRA-12996)
  * Default for start_native_transport now true if not set in config 
(CASSANDRA-13656)
  * Don't add localhost to the graph when calculating where to stream from 
(CASSANDRA-13583)
  * Allow skipping equality-restricted clustering columns in ORDER BY clause 
(CASSANDRA-10271)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf4a0576/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index e369982..208605d 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -1157,6 +1157,11 @@ public class DatabaseDescriptor
 return conf.batch_size_warn_threshold_in_kb * 1024;
 }
 
+public static int getBatchSizeWarnThresholdInKB()
+{
+return conf.batch_size_warn_threshold_in_kb;
+}
+
 public static long getBatchSizeFailThreshold()
 {
 return conf.batch_size_fail_threshold_in_kb * 1024L;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf4a0576/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index a3d3791..9070e89 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -4982,6 +4982,18 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 public void setBatchSizeFailureThreshold(int threshold)
 {
 DatabaseDescriptor.setBatchSizeFailThresholdInKB(threshold);
+logger.info("Updated batch_size_fail_threshold_in_kb to {}", 
threshold);
+}
+
+public int getBatchSizeWarnThreshold()
+{
+return DatabaseDescriptor.getBatchSizeWarnThresholdInKB();
+}
+
+public void setBatchSizeWarnThreshold(int threshold)
+{
+DatabaseDescriptor.setBatchSizeWarnThresholdInKB(threshold);
+logger.info("Updated batch_size_warn_threshold_in_kb to {}", 
threshold);
 }
 
 public void setHintedHandoffThrottleInKB(int throttleInKB)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf4a0576/src/java/org/apache/cassandra/service/StorageServiceMBean.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageServiceMBean.java 
b/src/java/org/apache/cassandra/service/StorageServiceMBean.java
index bd8aca6..36c43fd 100644
--- a/src/java/org/apache/cassandra/service/StorageServiceMBean.java
+++ b/src/java/org/apache/cassandra/service/StorageServiceMBean.java
@@ -620,6 +620,11 @@ public interface StorageServiceMBean extends 
NotificationEmitter
 /** Sets the threshold for rejecting queries due to a large batch size */
 public void setBatchSizeFailureThreshold(int batchSizeDebugThreshold);
 
+/** Returns the

[jira] [Updated] (CASSANDRA-13711) Invalid writetime for null columns in cqlsh


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-13711:
---
Priority: Major  (was: Minor)

> Invalid writetime for null columns in cqlsh
> ---
>
> Key: CASSANDRA-13711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13711
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jeff Jirsa
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> From the user list:
> https://lists.apache.org/thread.html/448731c029eee72e499fc6acd44d257d1671193f850a68521c2c6681@%3Cuser.cassandra.apache.org%3E
> {code}
> (oss-ccm) MacBook-Pro:~ jjirsa$ ccm create test -n 1 -s -v 3.0.10
> Current cluster is now: test
> (oss-ccm) MacBook-Pro:~ jjirsa$ ccm node1 cqlsh
> Connected to test at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.0.10 | CQL spec 3.4.0 | Native protocol v4]
> Use HELP for help.
> cqlsh> CREATE KEYSPACE test WITH replication = {'class':'SimpleStrategy', 
> 'replication_factor': 1};
> cqlsh> CREATE TABLE test.t ( a text primary key, b text );
> cqlsh> insert into test.t(a) values('z');
> cqlsh> insert into test.t(a) values('w');
> cqlsh> insert into test.t(a) values('e');
> cqlsh> insert into test.t(a) values('r');
> cqlsh> insert into test.t(a) values('t');
> cqlsh> select a,b, writetime (b) from test.t;
> a | b | writetime(b)
> ---+--+--
> z | null | null
> e | null | null
> r | null | null
> w | null | null
> t | null | null
> (5 rows)
> cqlsh>
> cqlsh> insert into test.t(a,b) values('t','x');
> cqlsh> insert into test.t(a) values('b');
> cqlsh> select a,b, writetime (b) from test.t;
>  a | b| writetime(b)
> ---+--+--
>  z | null | null
>  e | null | null
>  r | null | null
>  w | null | null
>  t |x | 1500565131354883
>  b | null | 1500565131354883
> (6 rows)
> {code}
> Data on disk:
> {code}
> MacBook-Pro:~ jjirsa$ ~/.ccm/repository/3.0.14/tools/bin/sstabledump 
> /Users/jjirsa/.ccm/test/node1/data0/test/t-bed196006d0511e7904be9daad294861/mc-1-big-Data.db
> [
>   {
> "partition" : {
>   "key" : [ "z" ],
>   "position" : 0
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 20,
> "liveness_info" : { "tstamp" : "2017-07-20T04:41:54.818118Z" },
> "cells" : [ ]
>   }
> ]
>   },
>   {
> "partition" : {
>   "key" : [ "e" ],
>   "position" : 21
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 44,
> "liveness_info" : { "tstamp" : "2017-07-20T04:42:04.288547Z" },
> "cells" : [ ]
>   }
> ]
>   },
>   {
> "partition" : {
>   "key" : [ "r" ],
>   "position" : 45
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 68,
> "liveness_info" : { "tstamp" : "2017-07-20T04:42:08.991417Z" },
> "cells" : [ ]
>   }
> ]
>   },
>   {
> "partition" : {
>   "key" : [ "w" ],
>   "position" : 69
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 92,
> "liveness_info" : { "tstamp" : "2017-07-20T04:41:59.005382Z" },
> "cells" : [ ]
>   }
> ]
>   },
>   {
> "partition" : {
>   "key" : [ "t" ],
>   "position" : 93
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 120,
> "liveness_info" : { "tstamp" : "2017-07-20T15:38:51.354883Z" },
> "cells" : [
>   { "name" : "b", "value" : "x" }
> ]
>   }
> ]
>   },
>   {
> "partition" : {
>   "key" : [ "b" ],
>   "position" : 121
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 146,
> "liveness_info" : { "tstamp" : "2017-07-20T15:39:03.631297Z" },
> "cells" : [ ]
>   }
> ]
>   }
> ]MacBook-Pro:~ jjirsa$
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13711) Invalid writetime for null columns in cqlsh


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-13711:
---
Priority: Minor  (was: Major)

> Invalid writetime for null columns in cqlsh
> ---
>
> Key: CASSANDRA-13711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13711
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jeff Jirsa
>Priority: Minor
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> From the user list:
> https://lists.apache.org/thread.html/448731c029eee72e499fc6acd44d257d1671193f850a68521c2c6681@%3Cuser.cassandra.apache.org%3E
> {code}
> (oss-ccm) MacBook-Pro:~ jjirsa$ ccm create test -n 1 -s -v 3.0.10
> Current cluster is now: test
> (oss-ccm) MacBook-Pro:~ jjirsa$ ccm node1 cqlsh
> Connected to test at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.0.10 | CQL spec 3.4.0 | Native protocol v4]
> Use HELP for help.
> cqlsh> CREATE KEYSPACE test WITH replication = {'class':'SimpleStrategy', 
> 'replication_factor': 1};
> cqlsh> CREATE TABLE test.t ( a text primary key, b text );
> cqlsh> insert into test.t(a) values('z');
> cqlsh> insert into test.t(a) values('w');
> cqlsh> insert into test.t(a) values('e');
> cqlsh> insert into test.t(a) values('r');
> cqlsh> insert into test.t(a) values('t');
> cqlsh> select a,b, writetime (b) from test.t;
> a | b | writetime(b)
> ---+--+--
> z | null | null
> e | null | null
> r | null | null
> w | null | null
> t | null | null
> (5 rows)
> cqlsh>
> cqlsh> insert into test.t(a,b) values('t','x');
> cqlsh> insert into test.t(a) values('b');
> cqlsh> select a,b, writetime (b) from test.t;
>  a | b| writetime(b)
> ---+--+--
>  z | null | null
>  e | null | null
>  r | null | null
>  w | null | null
>  t |x | 1500565131354883
>  b | null | 1500565131354883
> (6 rows)
> {code}
> Data on disk:
> {code}
> MacBook-Pro:~ jjirsa$ ~/.ccm/repository/3.0.14/tools/bin/sstabledump 
> /Users/jjirsa/.ccm/test/node1/data0/test/t-bed196006d0511e7904be9daad294861/mc-1-big-Data.db
> [
>   {
> "partition" : {
>   "key" : [ "z" ],
>   "position" : 0
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 20,
> "liveness_info" : { "tstamp" : "2017-07-20T04:41:54.818118Z" },
> "cells" : [ ]
>   }
> ]
>   },
>   {
> "partition" : {
>   "key" : [ "e" ],
>   "position" : 21
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 44,
> "liveness_info" : { "tstamp" : "2017-07-20T04:42:04.288547Z" },
> "cells" : [ ]
>   }
> ]
>   },
>   {
> "partition" : {
>   "key" : [ "r" ],
>   "position" : 45
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 68,
> "liveness_info" : { "tstamp" : "2017-07-20T04:42:08.991417Z" },
> "cells" : [ ]
>   }
> ]
>   },
>   {
> "partition" : {
>   "key" : [ "w" ],
>   "position" : 69
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 92,
> "liveness_info" : { "tstamp" : "2017-07-20T04:41:59.005382Z" },
> "cells" : [ ]
>   }
> ]
>   },
>   {
> "partition" : {
>   "key" : [ "t" ],
>   "position" : 93
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 120,
> "liveness_info" : { "tstamp" : "2017-07-20T15:38:51.354883Z" },
> "cells" : [
>   { "name" : "b", "value" : "x" }
> ]
>   }
> ]
>   },
>   {
> "partition" : {
>   "key" : [ "b" ],
>   "position" : 121
> },
> "rows" : [
>   {
> "type" : "row",
> "position" : 146,
> "liveness_info" : { "tstamp" : "2017-07-20T15:39:03.631297Z" },
> "cells" : [ ]
>   }
> ]
>   }
> ]MacBook-Pro:~ jjirsa$
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13733) nodetool toptables


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102438#comment-16102438
 ] 

Jeff Jirsa commented on CASSANDRA-13733:


I presume - based on your comparison - you want top-k sampling - would it be 
sufficient to just sample the table-level metrics ? 

> nodetool toptables
> --
>
> Key: CASSANDRA-13733
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13733
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jon Haddad
>
> In the same way toppartitions samples the activity on a table, toptables 
> should be able to report which tables are used the most.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-9988) Introduce leaf-only iterator

2017-07-26 Thread Benedict (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102361#comment-16102361
 ] 

Benedict commented on CASSANDRA-9988:
-

Also, what exactly are these graphs showing us?

> Introduce leaf-only iterator
> 
>
> Key: CASSANDRA-9988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9988
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Benedict
>Assignee: Jay Zhuang
>Priority: Minor
>  Labels: patch
> Fix For: 4.0
>
> Attachments: 9988-data.png, 9988-result2.png, 9988-result3.png, 
> 9988-result.png, 9988-test-result3.png, 9988-trunk-new.txt, 
> 9988-trunk-new-update.txt, trunk-9988.txt
>
>
> In many cases we have small btrees, small enough to fit in a single leaf 
> page. In this case it _may_ be more efficient to specialise our iterator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-9988) Introduce leaf-only iterator

2017-07-26 Thread Benedict (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102358#comment-16102358
 ] 

Benedict commented on CASSANDRA-9988:
-

So, I took a quick peek at your actual test cases, and it seems you're only 
ever seeking into an iterator the once with a uniform key across its domain, 
then discarding it.  In this scenario, yes, you would expect exponential search 
to be more costly.

However, these are iterators, and the expectation is that there will be 
multiple seeks during the iterator's lifetime.  I had assumed that your 
searchFound and searchNotFound enumerated (searched for) a sequence of found / 
not found keys.  There should probably be such variants.

All that said, I feel like this ticket has dragged on long enough and we're 
probably well past diminishing returns for the change itself.  But these 
benchmarks are likely to persist a lot better than this discussion, so it is 
probably worthwhile getting them right.  It's also the case that we already use 
exponential search for the branch iterator (though this was decided back on 2.2 
storage format, so our N was much larger), and we should probably be consistent 
and settle it once and for all.

> Introduce leaf-only iterator
> 
>
> Key: CASSANDRA-9988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9988
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Benedict
>Assignee: Jay Zhuang
>Priority: Minor
>  Labels: patch
> Fix For: 4.0
>
> Attachments: 9988-data.png, 9988-result2.png, 9988-result3.png, 
> 9988-result.png, 9988-test-result3.png, 9988-trunk-new.txt, 
> 9988-trunk-new-update.txt, trunk-9988.txt
>
>
> In many cases we have small btrees, small enough to fit in a single leaf 
> page. In this case it _may_ be more efficient to specialise our iterator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-9988) Introduce leaf-only iterator


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102332#comment-16102332
 ] 

Jay Zhuang edited comment on CASSANDRA-9988 at 7/26/17 9:46 PM:


They're good feedbacks. I did the tests for expSearch:
!9988-test-result3.png|test!
I think the exponentialSearch didn't out perform the binaraySearch is because:
# data size is too small. In this case, it only searches inside of one leaf, 
the maximum data size is 32 (default node size).
In theory, the performance of exponential search is {{[O(log( i ) + log( i 
))|https://en.wikipedia.org/wiki/Exponential_search#Performance]}} ({{i}} is 
where the target is), vs. binary search is {{O(log( n ))}}.
Because exponentialSearch has 2 stages, it could do more comparisons when the 
{{n}} is small.
# Maybe Java API {{Arrays.binarySearch()}} is already doing some optimizations.

So for this JIRA, I don't think it worth doing exponentialSearch.


was (Author: jay.zhuang):
They're good feedbacks. I did the tests for expSearch:
!9988-test-result3.png|test!
I think the exponentialSearch didn't out perform the binaraySearch is because:
# data size is too small. In this case, it only searches inside of one leaf, 
the maximum data size is 32 (default node size).
In theroy, the performance of exponential search is {{[O(log( i ) + log( i 
))|https://en.wikipedia.org/wiki/Exponential_search#Performance]}} ({{i}} is 
where the target is), vs. binary search is {{O(log( n ))}}.
Because exponentialSearch has 2 stages, it could do more compares when the 
{{n}} is small.
# Maybe Java API {{Arrays.binarySearch()}} is already doing some optimizations.

So for this JIRA, I don't think it worth doing exponentialSearch.

> Introduce leaf-only iterator
> 
>
> Key: CASSANDRA-9988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9988
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Benedict
>Assignee: Jay Zhuang
>Priority: Minor
>  Labels: patch
> Fix For: 4.0
>
> Attachments: 9988-data.png, 9988-result2.png, 9988-result3.png, 
> 9988-result.png, 9988-test-result3.png, 9988-trunk-new.txt, 
> 9988-trunk-new-update.txt, trunk-9988.txt
>
>
> In many cases we have small btrees, small enough to fit in a single leaf 
> page. In this case it _may_ be more efficient to specialise our iterator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-9988) Introduce leaf-only iterator


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102332#comment-16102332
 ] 

Jay Zhuang commented on CASSANDRA-9988:
---

They're good feedbacks. I did the tests for expSearch:
!9988-test-result3.png|test!
I think the exponentialSearch didn't out perform the binaraySearch is because:
# data size is too small. In this case, it only searches inside of one leaf, 
the maximum data size is 32 (default node size).
In theroy, the performance of exponential search is {{[O(log( i ) + log( i 
))|https://en.wikipedia.org/wiki/Exponential_search#Performance]}} ({{i}} is 
where the target is), vs. binary search is {{O(log( n ))}}.
Because exponentialSearch has 2 stages, it could do more compares when the 
{{n}} is small.
# Maybe Java API {{Arrays.binarySearch()}} is already doing some optimizations.

So for this JIRA, I don't think it worth doing exponentialSearch.

> Introduce leaf-only iterator
> 
>
> Key: CASSANDRA-9988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9988
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Benedict
>Assignee: Jay Zhuang
>Priority: Minor
>  Labels: patch
> Fix For: 4.0
>
> Attachments: 9988-data.png, 9988-result2.png, 9988-result3.png, 
> 9988-result.png, 9988-test-result3.png, 9988-trunk-new.txt, 
> 9988-trunk-new-update.txt, trunk-9988.txt
>
>
> In many cases we have small btrees, small enough to fit in a single leaf 
> page. In this case it _may_ be more efficient to specialise our iterator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-13733) nodetool toptables

2017-07-26 Thread Jon Haddad (JIRA)

Jon Haddad created CASSANDRA-13733:
--

 Summary: nodetool toptables
 Key: CASSANDRA-13733
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13733
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jon Haddad


In the same way toppartitions samples the activity on a table, toptables should 
be able to report which tables are used the most.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13713) Move processing of EchoMessage response to gossip stage

2017-07-26 Thread Jason Brown (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102323#comment-16102323
 ] 

Jason Brown commented on CASSANDRA-13713:
-

bq. do you think it's worth doing something to map the echo response directly 
to the gossip stage and gossip connection?

CASSANDRA-13714 :)

> Move processing of EchoMessage response to gossip stage
> ---
>
> Key: CASSANDRA-13713
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13713
> Project: Cassandra
>  Issue Type: Bug
>  Components: Distributed Metadata
>Reporter: Jason Brown
>Assignee: Jason Brown
>Priority: Minor
> Fix For: 4.x
>
>
> Currently, when a node receives an {{EchoMessage}}, is sends a simple ACK 
> reply back (see {{EchoVerbHandler}}). The ACK is sent on the small message 
> connection, and because it is 'generically' typed as 
> {{Verb.REQUEST_RESPONSE}}, is consumed on a {{Stage.REQUEST_RESPONSE}} 
> thread. The proper thread for this response to be consumed is 
> {{Stage.GOSSIP}}, that way we can move more of the updating of the gossip 
> state to a single, centralized thread, and less abuse of gossip's shared 
> mutable state can occur.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-13732) GH PR #131 - Refactoring to primitive functional interfaces in AuthCache.java

Jeff Jirsa created CASSANDRA-13732:
--

 Summary: GH PR #131 - Refactoring to primitive functional 
interfaces in AuthCache.java
 Key: CASSANDRA-13732
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13732
 Project: Cassandra
  Issue Type: Bug
  Components: Auth
Reporter: Jeff Jirsa
 Fix For: 4.x


https://github.com/apache/cassandra/pull/131

Refactor to avoid unnecessary boxing/unboxing in auth cache.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-9988) Introduce leaf-only iterator


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jay Zhuang updated CASSANDRA-9988:
--
Attachment: 9988-test-result3.png

> Introduce leaf-only iterator
> 
>
> Key: CASSANDRA-9988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9988
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Benedict
>Assignee: Jay Zhuang
>Priority: Minor
>  Labels: patch
> Fix For: 4.0
>
> Attachments: 9988-data.png, 9988-result2.png, 9988-result3.png, 
> 9988-result.png, 9988-test-result3.png, 9988-trunk-new.txt, 
> 9988-trunk-new-update.txt, trunk-9988.txt
>
>
> In many cases we have small btrees, small enough to fit in a single leaf 
> page. In this case it _may_ be more efficient to specialise our iterator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13713) Move processing of EchoMessage response to gossip stage

2017-07-26 Thread Joel Knighton (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102175#comment-16102175
 ] 

Joel Knighton commented on CASSANDRA-13713:
---

The simple change here seems good - do you think it's worth doing something to 
map the echo response directly to the gossip stage and gossip connection? This 
seems like a good fix for the present state, but I think perhaps a different 
verb or a way to map individual REQUEST_RESPONSE verbs to a stage/connection 
makes sense on a longer term.

> Move processing of EchoMessage response to gossip stage
> ---
>
> Key: CASSANDRA-13713
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13713
> Project: Cassandra
>  Issue Type: Bug
>  Components: Distributed Metadata
>Reporter: Jason Brown
>Assignee: Jason Brown
>Priority: Minor
> Fix For: 4.x
>
>
> Currently, when a node receives an {{EchoMessage}}, is sends a simple ACK 
> reply back (see {{EchoVerbHandler}}). The ACK is sent on the small message 
> connection, and because it is 'generically' typed as 
> {{Verb.REQUEST_RESPONSE}}, is consumed on a {{Stage.REQUEST_RESPONSE}} 
> thread. The proper thread for this response to be consumed is 
> {{Stage.GOSSIP}}, that way we can move more of the updating of the gossip 
> state to a single, centralized thread, and less abuse of gossip's shared 
> mutable state can occur.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-11825) NPE in gossip

2017-07-26 Thread Joel Knighton (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-11825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102145#comment-16102145
 ] 

Joel Knighton commented on CASSANDRA-11825:
---

Executing {{#start(int)}} and {{#stop()}} on the gossip stage seems safe to me 
at first glance. As an alternative, I think it's safe to simply pass the 
generation we use at comparison time down the chain of methods and make sure to 
use that in the endpoint state we send, since the problem is a mismatch between 
generation at comparison time and generation at message sending time. Then, 
we'll send everything on the next message when we compare and see the 
generation difference.

Either seems fine to me, but I could be missing something on either. I'd spend 
more time on a proper review.

> NPE in gossip
> -
>
> Key: CASSANDRA-11825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11825
> Project: Cassandra
>  Issue Type: Bug
>Reporter: T Jake Luciani
>Assignee: Joel Knighton
>  Labels: fallout
> Fix For: 3.0.x
>
>
> We have a test that causes an NPE in gossip code:
> It's basically calling nodetool enable/disable gossip
> From the debug log
> {quote}
> WARN  [RMI TCP Connection(17)-54.153.70.214] 2016-05-17 18:58:44,423 
> StorageService.java:395 - Starting gossip by operator request
> DEBUG [RMI TCP Connection(17)-54.153.70.214] 2016-05-17 18:58:44,424 
> StorageService.java:1996 - Node /172.31.24.76 state NORMAL, token 
> [-9223372036854775808]
> INFO  [RMI TCP Connection(17)-54.153.70.214] 2016-05-17 18:58:44,424 
> StorageService.java:1999 - Node /172.31.24.76 state jump to NORMAL
> DEBUG [RMI TCP Connection(17)-54.153.70.214] 2016-05-17 18:58:44,424 
> YamlConfigurationLoader.java:102 - Loading settings from 
> file:/mnt/ephemeral/automaton/cassandra-src/conf/cassandra.yaml
> DEBUG [PendingRangeCalculator:1] 2016-05-17 18:58:44,425 
> PendingRangeCalculatorService.java:66 - finished calculation for 5 keyspaces 
> in 0ms
> DEBUG [GossipStage:1] 2016-05-17 18:58:45,346 FailureDetector.java:456 - 
> Ignoring interval time of 75869093776 for /172.31.31.1
> DEBUG [GossipStage:1] 2016-05-17 18:58:45,347 FailureDetector.java:456 - 
> Ignoring interval time of 75869214424 for /172.31.17.32
> INFO  [GossipStage:1] 2016-05-17 18:58:45,347 Gossiper.java:1028 - Node 
> /172.31.31.1 has restarted, now UP
> DEBUG [GossipStage:1] 2016-05-17 18:58:45,347 StorageService.java:1996 - Node 
> /172.31.31.1 state NORMAL, token [-3074457345618258603]
> INFO  [GossipStage:1] 2016-05-17 18:58:45,347 StorageService.java:1999 - Node 
> /172.31.31.1 state jump to NORMAL
> INFO  [HANDSHAKE-/172.31.31.1] 2016-05-17 18:58:45,348 
> OutboundTcpConnection.java:514 - Handshaking version with /172.31.31.1
> ERROR [GossipStage:1] 2016-05-17 18:58:45,354 CassandraDaemon.java:195 - 
> Exception in thread Thread[GossipStage:1,5,main]
> java.lang.NullPointerException: null
>   at org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:846) 
> ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:2008)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:1729)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageService.onJoin(StorageService.java:2446) 
> ~[main/:na]
>   at 
> org.apache.cassandra.gms.Gossiper.handleMajorStateChange(Gossiper.java:1050) 
> ~[main/:na]
>   at 
> org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1133) 
> ~[main/:na]
>   at 
> org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:49)
>  ~[main/:na]
>   at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) 
> ~[main/:na]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_40]
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_40]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_40]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_40]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_40]
> INFO  [GossipStage:1] 2016-05-17 18:58:45,355 Gossiper.java:1028 - Node 
> /172.31.17.32 has restarted, now UP
> DEBUG [GossipStage:1] 2016-05-17 18:58:45,355 StorageService.java:1996 - Node 
> /172.31.17.32 state NORMAL, token [3074457345618258602]
> INFO  [GossipStage:1] 2016-05-17 18:58:45,356 StorageService.java:1999 - Node 
> /172.31.17.32 state jump to NORMAL
> INFO  [HANDSHAKE-/172.31.17.32] 2016-05-17 18:58:45,356 
> OutboundTcpConnection.java:514 - Handshaking version with /172.31.17.32
> DEBUG [PendingRangeCalculator:1] 2016-05-17 18:58:45,357 
>

[jira] [Commented] (CASSANDRA-8457) nio MessagingService

2017-07-26 Thread Jason Brown (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102128#comment-16102128
 ] 

Jason Brown commented on CASSANDRA-8457:


dtests running locally, including upgrade and no_vodes tests, are passing (with 
the exception of those not working on trunk, and one other that is related to 
CASSANDRA-12229). 

[~slebresne] and I discussed offline, and he agreed the 
{{OutboundConnectionParams}} taking a {{Consumer}} was 
reasonable. Hence, that was implemented. Current branch also has a minor fix 
for reconnecting when a connection is rejected due to protocol version mismatch 
(discovered via dtest upgrade tests)

> nio MessagingService
> 
>
> Key: CASSANDRA-8457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8457
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Jonathan Ellis
>Assignee: Jason Brown
>Priority: Minor
>  Labels: netty, performance
> Fix For: 4.x
>
> Attachments: 8457-load.tgz
>
>
> Thread-per-peer (actually two each incoming and outbound) is a big 
> contributor to context switching, especially for larger clusters.  Let's look 
> at switching to nio, possibly via Netty.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-9988) Introduce leaf-only iterator

2017-07-26 Thread Benedict (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101956#comment-16101956
 ] 

Benedict commented on CASSANDRA-9988:
-

Right. 

That, and my allusion to a "representative complexity" of object are meant to 
account for the fact that in C* proper we can expect comparisons to:
# miss L1/2 cache (and on first access all layers of cache; I don't know how 
many times we iterate objects, though; almost certainly more than once)
# be costly, as
## we will have many different comparators and object types to compare (so 
likely megamorphic call sites and no inlining)
## many of the object will be heavyweight
## neighbouring objects are likely to have lengthy common prefixes, so 
comparisons cannot abort early

Since exponential search's main benefit is reducing the number of comparisons, 
each of these mischaracterisations is likely to underrepresent its potential in 
the wild.

_Ideally_ we should be mitigating these confounders, else our decisions will 
potentially be inaccurate.

> Introduce leaf-only iterator
> 
>
> Key: CASSANDRA-9988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9988
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Benedict
>Assignee: Jay Zhuang
>Priority: Minor
>  Labels: patch
> Fix For: 4.0
>
> Attachments: 9988-data.png, 9988-result2.png, 9988-result3.png, 
> 9988-result.png, 9988-trunk-new.txt, 9988-trunk-new-update.txt, trunk-9988.txt
>
>
> In many cases we have small btrees, small enough to fit in a single leaf 
> page. In this case it _may_ be more efficient to specialise our iterator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-9988) Introduce leaf-only iterator

2017-07-26 Thread Ariel Weisberg (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101919#comment-16101919
 ] 

Ariel Weisberg commented on CASSANDRA-9988:
---

I think Benedict was looking for a comparison of exponential search to binary 
search with a large number of small trees. The comparison you have is with many 
trees is with and without the optimization using binary search?

Since you already did the work of implementing exponential search it seems like 
it's worth hooking it up and testing btree size from 1-32 just to see so we 
don't have to revisit it later to decide if exponential search is a good idea.

> Introduce leaf-only iterator
> 
>
> Key: CASSANDRA-9988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9988
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Benedict
>Assignee: Jay Zhuang
>Priority: Minor
>  Labels: patch
> Fix For: 4.0
>
> Attachments: 9988-data.png, 9988-result2.png, 9988-result3.png, 
> 9988-result.png, 9988-trunk-new.txt, 9988-trunk-new-update.txt, trunk-9988.txt
>
>
> In many cases we have small btrees, small enough to fit in a single leaf 
> page. In this case it _may_ be more efficient to specialise our iterator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13691) Fix incorrect [2.1 <— 3.0] serialization of counter cells with pre-2.1 local shards

2017-07-26 Thread Aleksey Yeschenko (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-13691:
--
Reviewer: Sam Tunnicliffe

> Fix incorrect [2.1 <— 3.0] serialization of counter cells with pre-2.1 local 
> shards
> ---
>
> Key: CASSANDRA-13691
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13691
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>  Labels: counters, upgrade
> Fix For: 3.0.x, 3.11.x
>
>
> We stopped generating local shards in C* 2.1, after CASSANDRA-6504 (Counters 
> 2.0). But it’s still possible to have counter cell values
> around, remaining from 2.0 times, on 2.1, 3.0, 3.11, and even trunk nodes, if 
> they’ve never been overwritten.
> In 2.1, we used two classes for two kinds of counter columns:
> {{CounterCell}} class to store counters - internally as collections of 
> {{CounterContext}} blobs, encoding collections of (host id, count, clock) 
> tuples
> {{CounterUpdateCell}} class to represent unapplied increments - essentially a 
> single long value; this class was never written to commit log, memtables, or 
> sstables, and was only used inside {{Mutation}} object graph - in memory, and 
> marshalled over network in cases when counter write coordinator and counter 
> write leader were different nodes
> 3.0 got rid of {{CounterCell}} and {{CounterUpdateCell}}, among other 
> {{Cell}} classes. In order to represent these unapplied increments - 
> equivalents of 2.1 {{CounterUpdateCell}} - in 3.0 we encode them as regular 
> counter columns, with a ‘special’ {{CounterContext}} value. I.e. a counter 
> context with a single local shard. We do that so that we can reuse local 
> shard reconcile logic (summing up) to seamlessly support counters with same 
> names collapsing to single increments in batches. See 
> {{UpdateParameters.addCounter()}} method comments 
> [here|https://github.com/apache/cassandra/blob/cassandra-3.0.14/src/java/org/apache/cassandra/cql3/UpdateParameters.java#L157-L171]
>  for details. It also assumes that nothing else can generate a counter with 
> local shards.
> It works fine in pure 3.0 clusters, and in mixed 2.1/3.0 clusters, assuming 
> that there are no counters with legacy local shards remaining from 2.0 era. 
> It breaks down badly if there are.
> {{LegacyLayout.serializeAsLegacyPartition()}} and consequently 
> {{LegacyCell.isCounterUpdate()}} - classes responsible for serializing and 
> deserialising in 2.1 format for compatibility - use the following logic to 
> tell if a cell of {{COUNTER}} kind is a regular final counter or an unapplied 
> increment:
> {code}
> private boolean isCounterUpdate()
> {
> // See UpdateParameters.addCounter() for more details on this
> return isCounter() && CounterContext.instance().isLocal(value);
> }
> {code}
> {{CounterContext.isLocal()}} method here looks at the first shard of the 
> collection of tuples and returns true if it’s a local one.
> This method would correctly identify a cell generated by 
> {{UpdateParameters.addCounter()}} as a counter update and serialize it 
> correctly as a 2.1 {{CounterUpdateCell}}. However, it would also incorrectly 
> flag any regular counter cell that just so happens to have a local shard as 
> the first tuple of the counter context as a counter update. If a 2.1 node as 
> a coordinator of a read requests fetches such a value from a 3.0 node, during 
> a rolling upgrade, instead of the expected {{CounterCell}} object it will 
> receive a {{CounterUpdateCell}}, breaking all the things. In the best case 
> scenario it will cause an assert in {{AbstractCell.reconcileCounter()}} to be 
> raised.
> To fix the problem we must find an unambiguous way, without false positives 
> or false negatives, to represent and identify unapplied counter updates on 
> 3.0 side. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12125) ERROR [MemtableFlushWriter:4] 2016-07-01 06:20:41,137 CassandraDaemon.java:185 - Exception in thread Thread[MemtableFlushWriter:4,5,main] java.lang.RuntimeExcepti

2017-07-26 Thread Lewis DiFelice (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-12125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101909#comment-16101909
 ] 

Lewis DiFelice commented on CASSANDRA-12125:


Table in question with keyspace name changed

CREATE TABLE  master.accounts (
accountid bigint PRIMARY KEY,
accountname text,
createdate timestamp,
friendaffectionexpiredate timestamp,
lastloginbonus timestamp,
lastworldid bigint,
lastworldtype int,
maxmasterydate timestamp,
modifydate timestamp,
namereset int,
towerleaderdate timestamp
) WITH bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class': 
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class': 
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';
CREATE INDEX accounts_account_name ON master.accounts (accountname);


> ERROR [MemtableFlushWriter:4] 2016-07-01 06:20:41,137 
> CassandraDaemon.java:185 - Exception in thread 
> Thread[MemtableFlushWriter:4,5,main]  java.lang.RuntimeException: Last 
> written key DecoratedKey(.XX, X) >= current key DecoratedKey
> 
>
> Key: CASSANDRA-12125
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12125
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: RHEL-6.5 64-bit Apache Cassandra 2.2.5v
>Reporter: Relish Chackochan
> Fix For: 2.2.x
>
>
> We are running on RHEL-6.5 64-bit with Apache Cassandra 2.2.5v on 4 node 
> cluster and getting the following error on multiple node while running the 
> repair job and when getting the error repair job is hang.
> Can some one help to identify the issue.
> {code}
> ERROR [MemtableFlushWriter:4] 2016-07-01 06:20:41,137 
> CassandraDaemon.java:185 - Exception in thread 
> Thread[MemtableFlushWriter:4,5,main]
> java.lang.RuntimeException: Last written key DecoratedKey(1467371986.8870, 
> 313436373337313938362e38383730) >= current key DecoratedKey(, 
> 313436373337323030312e38383730) writing into 
> /opt/cassandra/data/proddb/log_data1-0a5092a0a4fa11e5872fc1ce0a46dc27/.maxdatetimeindex_idx/tmp-la-470-big-Data.db
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13725) Secondary indexes are always rebuilt at startup


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrés de la Peña updated CASSANDRA-13725:
--
Resolution: Fixed
Status: Resolved  (was: Ready to Commit)

> Secondary indexes are always rebuilt at startup
> ---
>
> Key: CASSANDRA-13725
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13725
> Project: Cassandra
>  Issue Type: Bug
>  Components: Secondary Indexes
>Reporter: Sergio Bossa
>Assignee: Sergio Bossa
>Priority: Critical
> Fix For: 4.0
>
>
> Following CASSANDRA-10130, a bug has been introduced that causes a 2i to be 
> rebuilt at startup, even if such index is already built.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12125) ERROR [MemtableFlushWriter:4] 2016-07-01 06:20:41,137 CassandraDaemon.java:185 - Exception in thread Thread[MemtableFlushWriter:4,5,main] java.lang.RuntimeExcepti

2017-07-26 Thread Lewis DiFelice (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-12125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101904#comment-16101904
 ] 

Lewis DiFelice commented on CASSANDRA-12125:


Having same problem with 3.6.   In my case I + Suspect it is happening when the 
app renames the secondary key. 

7/26/17
{{2:44:37.694 PM
ERROR [PerDiskMemtableFlushWriter_0:50359] 2017-07-26 14:44:37,694 
CassandraDaemon.java:213 - Exception in thread 
Thread[PerDiskMemtableFlushWriter_0:50359,5,main]
java.lang.RuntimeException: Last written key DecoratedKey(GhezaliBoumedien, 
4768657a616c69426f756d656469656e) >= current key 
DecoratedKey(\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,
 4768657a616c69426f756d656469656e2c3339303138393633) writing into 
/opt/cassandra/data/kiwi_live_ps4_eu_master/accounts-a43d0260bcd011e6a220bd11b0c79695/.accounts_account_name/ma-20604-big-Data.db
at 
org.apache.cassandra.io.sstable.format.big.BigTableWriter.beforeAppend(BigTableWriter.java:114)
 ~[apache-cassandra-3.6.0.jar:3.6.0]
at 
org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:153)
 ~[apache-cassandra-3.6.0.jar:3.6.0]
at 
org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:48)
 ~[apache-cassandra-3.6.0.jar:3.6.0]
at 
org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:445)
 ~[apache-cassandra-3.6.0.jar:3.6.0]
at 
org.apache.cassandra.db.Memtable$FlushRunnable.call(Memtable.java:479) 
~[apache-cassandra-3.6.0.jar:3.6.0]
at 
org.apache.cassandra.db.Memtable$FlushRunnable.call(Memtable.java:365) 
~[apache-cassandra-3.6.0.jar:3.6.0]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
~[na:1.8.0_45]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
~[na:1.8.0_45]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]}}

> ERROR [MemtableFlushWriter:4] 2016-07-01 06:20:41,137 
> CassandraDaemon.java:185 - Exception in thread 
> Thread[MemtableFlushWriter:4,5,main]  java.lang.RuntimeException: Last 
> written key DecoratedKey(.XX, X) >= current key DecoratedKey
> 
>
> Key: CASSANDRA-12125
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12125
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: RHEL-6.5 64-bit Apache Cassandra 2.2.5v
>Reporter: Relish Chackochan
> Fix For: 2.2.x
>
>
> We are running on RHEL-6.5 64-bit with Apache Cassandra 2.2.5v on 4 node 
> cluster and getting the following error on multiple node while running the 
> repair job and when getting the error repair job is hang.
> Can some one help to identify the issue.
> {code}
> ERROR [MemtableFlushWriter:4] 2016-07-01 06:20:41,137 
> CassandraDaemon.java:185 - Exception in thread 
> Thread[MemtableFlushWriter:4,5,main]
> java.lang.RuntimeException: Last written key DecoratedKey(1467371986.8870, 
> 313436373337313938362e38383730) >= current key DecoratedKey(, 
> 313436373337323030312e38383730) writing into 
> /opt/cassandra/data/proddb/log_data1-0a5092a0a4fa11e5872fc1ce0a46dc27/.maxdatetimeindex_idx/tmp-la-470-big-Data.db
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13691) Fix incorrect [2.1 <— 3.0] serialization of counter cells with pre-2.1 local shards

2017-07-26 Thread Aleksey Yeschenko (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101898#comment-16101898
 ] 

Aleksey Yeschenko commented on CASSANDRA-13691:
---

Dtest [here|https://github.com/iamaleksey/cassandra-dtest/commits/13691] - 
requires latest ccm with [issue 463|https://github.com/pcmanus/ccm/issues/463] 
addressed.

> Fix incorrect [2.1 <— 3.0] serialization of counter cells with pre-2.1 local 
> shards
> ---
>
> Key: CASSANDRA-13691
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13691
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>  Labels: counters, upgrade
> Fix For: 3.0.x, 3.11.x
>
>
> We stopped generating local shards in C* 2.1, after CASSANDRA-6504 (Counters 
> 2.0). But it’s still possible to have counter cell values
> around, remaining from 2.0 times, on 2.1, 3.0, 3.11, and even trunk nodes, if 
> they’ve never been overwritten.
> In 2.1, we used two classes for two kinds of counter columns:
> {{CounterCell}} class to store counters - internally as collections of 
> {{CounterContext}} blobs, encoding collections of (host id, count, clock) 
> tuples
> {{CounterUpdateCell}} class to represent unapplied increments - essentially a 
> single long value; this class was never written to commit log, memtables, or 
> sstables, and was only used inside {{Mutation}} object graph - in memory, and 
> marshalled over network in cases when counter write coordinator and counter 
> write leader were different nodes
> 3.0 got rid of {{CounterCell}} and {{CounterUpdateCell}}, among other 
> {{Cell}} classes. In order to represent these unapplied increments - 
> equivalents of 2.1 {{CounterUpdateCell}} - in 3.0 we encode them as regular 
> counter columns, with a ‘special’ {{CounterContext}} value. I.e. a counter 
> context with a single local shard. We do that so that we can reuse local 
> shard reconcile logic (summing up) to seamlessly support counters with same 
> names collapsing to single increments in batches. See 
> {{UpdateParameters.addCounter()}} method comments 
> [here|https://github.com/apache/cassandra/blob/cassandra-3.0.14/src/java/org/apache/cassandra/cql3/UpdateParameters.java#L157-L171]
>  for details. It also assumes that nothing else can generate a counter with 
> local shards.
> It works fine in pure 3.0 clusters, and in mixed 2.1/3.0 clusters, assuming 
> that there are no counters with legacy local shards remaining from 2.0 era. 
> It breaks down badly if there are.
> {{LegacyLayout.serializeAsLegacyPartition()}} and consequently 
> {{LegacyCell.isCounterUpdate()}} - classes responsible for serializing and 
> deserialising in 2.1 format for compatibility - use the following logic to 
> tell if a cell of {{COUNTER}} kind is a regular final counter or an unapplied 
> increment:
> {code}
> private boolean isCounterUpdate()
> {
> // See UpdateParameters.addCounter() for more details on this
> return isCounter() && CounterContext.instance().isLocal(value);
> }
> {code}
> {{CounterContext.isLocal()}} method here looks at the first shard of the 
> collection of tuples and returns true if it’s a local one.
> This method would correctly identify a cell generated by 
> {{UpdateParameters.addCounter()}} as a counter update and serialize it 
> correctly as a 2.1 {{CounterUpdateCell}}. However, it would also incorrectly 
> flag any regular counter cell that just so happens to have a local shard as 
> the first tuple of the counter context as a counter update. If a 2.1 node as 
> a coordinator of a read requests fetches such a value from a 3.0 node, during 
> a rolling upgrade, instead of the expected {{CounterCell}} object it will 
> receive a {{CounterUpdateCell}}, breaking all the things. In the best case 
> scenario it will cause an assert in {{AbstractCell.reconcileCounter()}} to be 
> raised.
> To fix the problem we must find an unambiguous way, without false positives 
> or false negatives, to represent and identify unapplied counter updates on 
> 3.0 side. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13725) Secondary indexes are always rebuilt at startup


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101865#comment-16101865
 ] 

Andrés de la Peña commented on CASSANDRA-13725:
---

Dtests committed as 
[b724df80d3bbb55b6b41845633e3a9034116f3be|https://github.com/apache/cassandra-dtest/commit/b724df80d3bbb55b6b41845633e3a9034116f3be].

> Secondary indexes are always rebuilt at startup
> ---
>
> Key: CASSANDRA-13725
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13725
> Project: Cassandra
>  Issue Type: Bug
>  Components: Secondary Indexes
>Reporter: Sergio Bossa
>Assignee: Sergio Bossa
>Priority: Critical
> Fix For: 4.0
>
>
> Following CASSANDRA-10130, a bug has been introduced that causes a 2i to be 
> rebuilt at startup, even if such index is already built.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

cassandra-dtest git commit: Added test to verify indexes are not rebuilt at startup if not actually needed.

2017-07-26 Thread adelapena

Repository: cassandra-dtest
Updated Branches:
  refs/heads/master 894bc92c9 -> b724df80d


Added test to verify indexes are not rebuilt at startup if not actually needed.


Project: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/commit/b724df80
Tree: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/tree/b724df80
Diff: http://git-wip-us.apache.org/repos/asf/cassandra-dtest/diff/b724df80

Branch: refs/heads/master
Commit: b724df80d3bbb55b6b41845633e3a9034116f3be
Parents: 894bc92
Author: Sergio Bossa 
Authored: Mon Jul 24 14:08:32 2017 +0100
Committer: Sergio Bossa 
Committed: Tue Jul 25 18:47:50 2017 +0100

--
 secondary_indexes_test.py | 69 --
 tools/data.py | 18 ++-
 2 files changed, 37 insertions(+), 50 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra-dtest/blob/b724df80/secondary_indexes_test.py
--
diff --git a/secondary_indexes_test.py b/secondary_indexes_test.py
index 1edd30e..11cd3af 100644
--- a/secondary_indexes_test.py
+++ b/secondary_indexes_test.py
@@ -14,11 +14,10 @@ from cassandra.query import BatchStatement, SimpleStatement
 from dtest import (DISABLE_VNODES, OFFHEAP_MEMTABLES, DtestTimeoutError,
Tester, debug, CASSANDRA_VERSION_FROM_BUILD, create_ks, 
create_cf)
 from tools.assertions import assert_bootstrap_state, assert_invalid, 
assert_none, assert_one, assert_row_count
-from tools.data import index_is_built, rows_to_list
+from tools.data import block_until_index_is_built, index_is_built, rows_to_list
 from tools.decorators import since
 from tools.misc import new_node
 
-
 class TestSecondaryIndexes(Tester):
 
 @staticmethod
@@ -306,28 +305,14 @@ class TestSecondaryIndexes(Tester):
 lookup_value = session.execute('select "C0" from standard1 limit 
1')[0].C0
 session.execute('CREATE INDEX ix_c0 ON standard1("C0");')
 
-start = time.time()
-while time.time() < start + 30:
-debug("waiting for index to build")
-time.sleep(1)
-if index_is_built(node1, session, 'keyspace1', 'standard1', 
'ix_c0'):
-break
-else:
-raise DtestTimeoutError()
+block_until_index_is_built(node1, session, 'keyspace1', 'standard1', 
'ix_c0')
 
 stmt = session.prepare('select * from standard1 where "C0" = ?')
 self.assertEqual(1, len(list(session.execute(stmt, [lookup_value]
 before_files = self._index_sstables_files(node1, 'keyspace1', 
'standard1', 'ix_c0')
 
 node1.nodetool("rebuild_index keyspace1 standard1 ix_c0")
-start = time.time()
-while time.time() < start + 30:
-debug("waiting for index to rebuild")
-time.sleep(1)
-if index_is_built(node1, session, 'keyspace1', 'standard1', 
'ix_c0'):
-break
-else:
-raise DtestTimeoutError()
+block_until_index_is_built(node1, session, 'keyspace1', 'standard1', 
'ix_c0')
 
 after_files = self._index_sstables_files(node1, 'keyspace1', 
'standard1', 'ix_c0')
 self.assertNotEqual(before_files, after_files)
@@ -447,39 +432,39 @@ class TestSecondaryIndexes(Tester):
'Cannot execute this query as it might involve data 
filtering')
 
 @since('4.0')
-def test_index_is_not_always_rebuilt_at_start(self):
+def test_index_is_not_rebuilt_at_restart(self):
 """
-@jira_ticket CASSANDRA-10130
+@jira_ticket CASSANDRA-13725
 
-Tests the management of index status during manual index rebuilding 
failures.
+Tests the index is not rebuilt at restart if already built.
 """
 
 cluster = self.cluster
-cluster.populate(1, 
install_byteman=True).start(wait_for_binary_proto=True)
+cluster.populate(1).start(wait_for_binary_proto=True)
 node = cluster.nodelist()[0]
 
 session = self.patient_cql_connection(node)
 create_ks(session, 'k', 1)
 session.execute("CREATE TABLE k.t (k int PRIMARY KEY, v int)")
-session.execute("CREATE INDEX idx ON k.t(v)")
 session.execute("INSERT INTO k.t(k, v) VALUES (0, 1)")
-session.execute("INSERT INTO k.t(k, v) VALUES (2, 3)")
 
-# Verify that the index is marked as built and it can answer queries
+debug("Create the index")
+session.execute("CREATE INDEX idx ON k.t(v)")
+block_until_index_is_built(node, session, 'k', 't', 'idx')
+before_files = self._index_sstables_files(node, 'k', 't', 'idx')
+
+debug("Verify the index is marked as built and it can be queried")

[jira] [Commented] (CASSANDRA-13725) Secondary indexes are always rebuilt at startup


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101857#comment-16101857
 ] 

Andrés de la Peña commented on CASSANDRA-13725:
---

Committed as 
[6e19e81db8e4c43bf5ef33308de1ae79916bb61c|https://github.com/apache/cassandra/commit/6e19e81db8e4c43bf5ef33308de1ae79916bb61c].

> Secondary indexes are always rebuilt at startup
> ---
>
> Key: CASSANDRA-13725
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13725
> Project: Cassandra
>  Issue Type: Bug
>  Components: Secondary Indexes
>Reporter: Sergio Bossa
>Assignee: Sergio Bossa
>Priority: Critical
> Fix For: 4.0
>
>
> Following CASSANDRA-10130, a bug has been introduced that causes a 2i to be 
> rebuilt at startup, even if such index is already built.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

cassandra git commit: Avoid always rebuilding secondary indexes at startup

2017-07-26 Thread adelapena

Repository: cassandra
Updated Branches:
  refs/heads/trunk 1d032e635 -> 6e19e81db


Avoid always rebuilding secondary indexes at startup

patch by Sergio Bossa; reviewed by Andres de la PeÃ±a for CASSANDRA-13725


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6e19e81d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6e19e81d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6e19e81d

Branch: refs/heads/trunk
Commit: 6e19e81db8e4c43bf5ef33308de1ae79916bb61c
Parents: 1d032e6
Author: AndreÌs de la PenÌa 
Authored: Wed Jul 26 16:56:59 2017 +0100
Committer: AndreÌs de la PenÌa 
Committed: Wed Jul 26 16:56:59 2017 +0100

--
 CHANGES.txt |  1 +
 .../apache/cassandra/db/ColumnFamilyStore.java  |  2 +-
 .../cassandra/index/SecondaryIndexManager.java  | 36 
 .../apache/cassandra/db/RangeTombstoneTest.java |  4 +--
 .../apache/cassandra/db/SecondaryIndexTest.java |  2 +-
 5 files changed, 26 insertions(+), 19 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6e19e81d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 46bbe3c..b4880fb 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0
+ * Avoid always rebuilding secondary indexes at startup (CASSANDRA-13725)
  * Upgrade JMH from 1.13 to 1.19
  * Upgrade SLF4J from 1.7.7 to 1.7.25
  * Default for start_native_transport now true if not set in config 
(CASSANDRA-13656)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6e19e81d/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java 
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 98ee7df..73e7db6 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -456,7 +456,7 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 // create the private ColumnFamilyStores for the secondary column 
indexes
 indexManager = new SecondaryIndexManager(this);
 for (IndexMetadata info : metadata.get().indexes)
-indexManager.addIndex(info);
+indexManager.addIndex(info, true);
 
 if (registerBookeeping)
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6e19e81d/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
--
diff --git a/src/java/org/apache/cassandra/index/SecondaryIndexManager.java 
b/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
index e253f3b..bc17e19 100644
--- a/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
+++ b/src/java/org/apache/cassandra/index/SecondaryIndexManager.java
@@ -186,7 +186,7 @@ public class SecondaryIndexManager implements 
IndexRegistry, INotificationConsum
 // we call add for every index definition in the collection as
 // some may not have been created here yet, only added to schema
 for (IndexMetadata tableIndex : tableIndexes)
-addIndex(tableIndex);
+addIndex(tableIndex, false);
 }
 
 private Future reloadIndex(IndexMetadata indexDef)
@@ -199,13 +199,12 @@ public class SecondaryIndexManager implements 
IndexRegistry, INotificationConsum
 }
 
 @SuppressWarnings("unchecked")
-private synchronized Future createIndex(IndexMetadata indexDef)
+private synchronized Future createIndex(IndexMetadata indexDef, boolean 
isNewCF)
 {
 final Index index = createInstance(indexDef);
 index.register(this);
 
-// now mark as building prior to initializing
-markIndexesBuilding(ImmutableSet.of(index), true);
+markIndexesBuilding(ImmutableSet.of(index), true, isNewCF);
 
 Callable initialBuildTask = null;
 // if the index didn't register itself, we can probably assume that no 
initialization needs to happen
@@ -255,13 +254,15 @@ public class SecondaryIndexManager implements 
IndexRegistry, INotificationConsum
  * Adds and builds a index
  *
  * @param indexDef the IndexMetadata describing the index
+ * @param isNewCF true if the index is added as part of a new 
table/columnfamily (i.e. loading a CF at startup), 
+ * false for all other cases (i.e. newly added index)
  */
-public synchronized Future addIndex(IndexMetadata indexDef)
+public synchronized Future addIndex(IndexMetadata indexDef, boolean 
isNewCF)
 {
 if (indexes.containsKey(indexDef.name))
 return

[jira] [Comment Edited] (CASSANDRA-11500) Obsolete MV entry may not be properly deleted

[
https://issues.apache.org/jira/browse/CASSANDRA-11500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082241#comment-16082241
]

ZhaoYang edited comment on CASSANDRA-11500 at 7/26/17 3:46 PM:
---

h3. Relation: base -> view

First of all, I think all of us should agree on what cases view row should
exists.

IMO, there are two main cases:

1. base pk and view pk are the same (order doesn't matter) and view has no
filter conditions or only conditions on base pk.
(filter condition mean: {{c = 1}} in view's where clause. filter condition is
not a concern here, since no previous view data to be cleared.)

view row exists if any of following is true:
* a. base row pk has live livenessInfo(timestamp) and base row pk satifies
view's filter conditions if any.
* b. or one of base row columns selected in view has live timestamp (via
update) and base row pk satifies view's filter conditions if any. this is
handled by existing mechanism of liveness and tombstone since all info are
included in view row
* c. or one of base row columns not selected in view has live timestamp (via
update) and base row pk satifies view's filter conditions if any. Those
unselected columns' timestamp/ttl/cell-deletion info are not currently stored
on view row.

2. base column used in view pk or view has filter conditions on base non-key
column which can also lead to entire view row being wiped.

view row exists if any of following is true:
* a. base row pk has live livenessInfo(timestamp) && base column used in view
pk is not null but no timestamp && conditions are satisfied. ( pk having live
livenesInfo means it is not deleted by tombstone)
* b. or base row column in view pk has timestamp (via update) && conditions are
satisfied. eg. if base column used in view pk is TTLed, entire view row should
be wiped.

Next thing is to model "view's tombstone and livenessInfo" to maintain view
data based on above cases.

h3. Previous known issues:
(I might miss some issues, feel free to ping me..)

ttl
* view row is not wiped when TTLed on base column used in view pk or TTLed on
base non-key column with filter condition
* cells with same timestamp, merging ttls are not deterministic.

partial update on base columns not selected in view
* it results in no view data. because of current update semantics, no view
updates are generated
* corresponding view row' liveness is not depending on liveness of base columns

filter conditions or base column used in view pk causes
* view row is shadowed after a few modification on base column used in view pk
if the base non-key column has TS greater than base pk's ts and view key
column's ts. (as mentioned by sylvain: we need to be able to re-insert an entry
when a prior one had been deleted need to be careful to hanlde timestamp tie)

tombstone merging is not commutative
* in current code, shadowable tombstone doesn't co-exist with regular tombstone

sstabledump not supporting current shadowable tombstone

h3. Model

I can think of two ways to ship all required base column info to view:
* make base columns that are not selected in view as "virtual cell" and
store their imestamp/ttl to view without their actual values. so we can reuse
current ts/tb/ttl mechanism with additional validation logic to check if a view
row is alive.
* or storing those info on view's livenessInfo/deletion with addition merge
logic to make sure view liveness/deletion are ordered properly even in the case
of timestamp tie. It's like a replacement stragy which uses inserted view row
to replace old view row. (In regular table, reconciliation is at cell level.
need to research more about the concurrent view update cases, fow now, it looks
fine). It also implies that every modification on base, view row will get
replaced entirely..

I will go ahead with -second way since there is an existing shadowable
tombstone mechanism.- VirtualCells to avoid changing low level timestamp
comparison..

{code}
ColumnInfo: // generated from base column as it is.
0. timestamp
1. ttl
2. localDeletionTime: could be used to represent tombstone or TTLed
depends on if there is ttl

supersedes(): if timestamps are different, greater timestamp
supersedes; if timestamps are same, greater localDeletionTime supersedes.

Row: // VirtualCells(keyOrConditions and unselected) are always merged with
another row's during row merging process

// base column that are used in view pk or has filter condition on
non-pk column.
// if any column is not live, entire view row is wiped.
// if a column in base is filtered and not selected, it's stored here.
// during base modification, if a view row is removed due to
base-column-in-view-pk or filter-contiions, then no deletion is issue,
// the virtual cell tombstone is added to Row's keyOrConditions.
2.

[jira] [Updated] (CASSANDRA-12334) HP Fortify Analysis

2017-07-26 Thread Joshua McKenzie (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12334:

Component/s: Testing

> HP Fortify Analysis
> ---
>
> Key: CASSANDRA-12334
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12334
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Jonathan Ellis
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-12211) Avoid giving every command the same latency number

2017-07-26 Thread Joshua McKenzie (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12211:

Component/s: Tools
 Observability

> Avoid giving every command the same latency number
> --
>
> Key: CASSANDRA-12211
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12211
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability, Tools
>Reporter: Robert Stupp
>Priority: Minor
>
> While reviewing CASSANDRA-7384, I found a _TODO avoid giving every command 
> the same latency number.  Can fix this in CASSANDRA-5329_ 
> [here|https://github.com/apache/cassandra/blob/70059726f08a98ea21af91ce3855bf62f6f4b652/src/java/org/apache/cassandra/service/StorageProxy.java#L1631]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-11500) Obsolete MV entry may not be properly deleted


[ 
https://issues.apache.org/jira/browse/CASSANDRA-11500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101603#comment-16101603
 ] 

ZhaoYang edited comment on CASSANDRA-11500 at 7/26/17 3:34 PM:
---

WIP 
[branch|https://github.com/jasonstack/cassandra/commits/CASSANDRA-11500-cell]

Changed:
Extra "VirtualCell"(kind of special LivenessInfo for MV) in Row. It stores: 1. 
base column used in view PK and base column used in view filter conditions. if 
any of such column dead, entire view row dead, regardless LivenessInfo or 
DeletionTime status. 2. unselected base columns. if any of such column alive, 
view's pk should be alive if it's not deleted by DeletionTime or those columns 
in <1>. 

Todo: 
optimize in-memory/storage representation for "virtual-cells" to re-use 
AbstractRow/BTree
more dtest
optimize ViewUpdateGenerator process to reduce payload. eg, if view row has 
live pk or other live column, "virtualCells' unselected payload" is not 
necessary.
support collection type in unselected column (for now, it won't generate 
"virtual-cells" for non-frozen collection type)


was (Author: jasonstack):
WIP 
[branch|https://github.com/jasonstack/cassandra/commits/CASSANDRA-11500-cell]

Changed:
Extra "VirtualCell"(kind of special LivenessInfo for MV) in Row. It stores: 1. 
base column used in view PK and base column used in view filter conditions. if 
any of such column dead, entire view row dead, regardless LivenessInfo or 
DeletionTime status. 2. unselected base columns. if any of such column alive, 
view's pk should be alive if it's not deleted by DeletionTime or those columns 
in <1>. 

Todo: 
optimize in-memory/storage representation for "virtual-cells" to re-use 
AbstractRow/BTree
discard dead view row shadowed by virtualCells in compaction after gc_grace
more dtest
support collection type in unselected column (for now, it won't generate 
"virtual-cells" for non-frozen collection type)

> Obsolete MV entry may not be properly deleted
> -
>
> Key: CASSANDRA-11500
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11500
> Project: Cassandra
>  Issue Type: Bug
>  Components: Materialized Views
>Reporter: Sylvain Lebresne
>Assignee: ZhaoYang
>
> When a Materialized View uses a non-PK base table column in its PK, if an 
> update changes that column value, we add the new view entry and remove the 
> old one. When doing that removal, the current code uses the same timestamp 
> than for the liveness info of the new entry, which is the max timestamp for 
> any columns participating to the view PK. This is not correct for the 
> deletion as the old view entry could have other columns with higher timestamp 
> which won't be deleted as can easily shown by the failing of the following 
> test:
> {noformat}
> CREATE TABLE t (k int PRIMARY KEY, a int, b int);
> CREATE MATERIALIZED VIEW mv AS SELECT * FROM t WHERE k IS NOT NULL AND a IS 
> NOT NULL PRIMARY KEY (k, a);
> INSERT INTO t(k, a, b) VALUES (1, 1, 1) USING TIMESTAMP 0;
> UPDATE t USING TIMESTAMP 4 SET b = 2 WHERE k = 1;
> UPDATE t USING TIMESTAMP 2 SET a = 2 WHERE k = 1;
> SELECT * FROM mv WHERE k = 1; // This currently return 2 entries, the old 
> (invalid) and the new one
> {noformat}
> So the correct timestamp to use for the deletion is the biggest timestamp in 
> the old view entry (which we know since we read the pre-existing base row), 
> and that is what CASSANDRA-11475 does (the test above thus doesn't fail on 
> that branch).
> Unfortunately, even then we can still have problems if further updates 
> requires us to overide the old entry. Consider the following case:
> {noformat}
> CREATE TABLE t (k int PRIMARY KEY, a int, b int);
> CREATE MATERIALIZED VIEW mv AS SELECT * FROM t WHERE k IS NOT NULL AND a IS 
> NOT NULL PRIMARY KEY (k, a);
> INSERT INTO t(k, a, b) VALUES (1, 1, 1) USING TIMESTAMP 0;
> UPDATE t USING TIMESTAMP 10 SET b = 2 WHERE k = 1;
> UPDATE t USING TIMESTAMP 2 SET a = 2 WHERE k = 1; // This will delete the 
> entry for a=1 with timestamp 10
> UPDATE t USING TIMESTAMP 3 SET a = 1 WHERE k = 1; // This needs to re-insert 
> an entry for a=1 but shouldn't be deleted by the prior deletion
> UPDATE t USING TIMESTAMP 4 SET a = 2 WHERE k = 1; // ... and we can play this 
> game more than once
> UPDATE t USING TIMESTAMP 5 SET a = 1 WHERE k = 1;
> ...
> {noformat}
> In a way, this is saying that the "shadowable" deletion mechanism is not 
> general enough: we need to be able to re-insert an entry when a prior one had 
> been deleted before, but we can't rely on timestamps being strictly bigger on 
> the re-insert. In that sense, this can be though as a similar problem than 
> CASSANDRA-10965, though the solution there of a single flag is not enough 
> since we can have to replace more than once.
> I think the proper solution

[jira] [Updated] (CASSANDRA-8871) Non-null paging state returned if last page

2017-07-26 Thread Andy Tolbert (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andy Tolbert updated CASSANDRA-8871:

Description: 
When retrieving the next page from the result of a simple statement, the result 
will return a non-null paging state even if it's the last page of the query. 
This only happens if it's the last page, and the results of the last page 
exactly matches the paging size.

Schema:
{noformat}
  CREATE KEYSPACE simplex WITH replication = {'class': 'SimpleStrategy', 
'replication_factor': 3};
  USE simplex;
  CREATE TABLE test (k text, v int, PRIMARY KEY (k, v));
  INSERT INTO test (k, v) VALUES ('a', 0);
  INSERT INTO test (k, v) VALUES ('b', 1);
  INSERT INTO test (k, v) VALUES ('c', 2);
  INSERT INTO test (k, v) VALUES ('d', 3);
  INSERT INTO test (k, v) VALUES ('e', 4);
{noformat}

Query:
{noformat}
  result  = session.execute("SELECT * FROM test", page_size: 5)

  loop do
puts "last page? #{result.last_page?}"
puts "page size: #{result.size}"

result.each do |row|
  puts row
end
puts ""

break if result.last_page?
result = result.next_page
  end
{noformat}

Result:
{noformat}
  last page? false
  page size: 5
  {"k"=>"a", "v"=>0}
  {"k"=>"c", "v"=>2}
  {"k"=>"m", "v"=>12}
  {"k"=>"f", "v"=>5}
  {"k"=>"o", "v"=>14}
  
  last page? true
  page size: 0
{noformat}


  was:
tWhen retrieving the next page from the result of a simple statement, the 
result will return a non-null paging state even if it's the last page of the 
query. This only happens if it's the last page, and the results of the last 
page exactly matches the paging size.

Schema:
{noformat}
  CREATE KEYSPACE simplex WITH replication = {'class': 'SimpleStrategy', 
'replication_factor': 3};
  USE simplex;
  CREATE TABLE test (k text, v int, PRIMARY KEY (k, v));
  INSERT INTO test (k, v) VALUES ('a', 0);
  INSERT INTO test (k, v) VALUES ('b', 1);
  INSERT INTO test (k, v) VALUES ('c', 2);
  INSERT INTO test (k, v) VALUES ('d', 3);
  INSERT INTO test (k, v) VALUES ('e', 4);
{noformat}

Query:
{noformat}
  result  = session.execute("SELECT * FROM test", page_size: 5)

  loop do
puts "last page? #{result.last_page?}"
puts "page size: #{result.size}"

result.each do |row|
  puts row
end
puts ""

break if result.last_page?
result = result.next_page
  end
{noformat}

Result:
{noformat}
  last page? false
  page size: 5
  {"k"=>"a", "v"=>0}
  {"k"=>"c", "v"=>2}
  {"k"=>"m", "v"=>12}
  {"k"=>"f", "v"=>5}
  {"k"=>"o", "v"=>14}
  
  last page? true
  page size: 0
{noformat}



> Non-null paging state returned if last page
> ---
>
> Key: CASSANDRA-8871
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8871
> Project: Cassandra
>  Issue Type: Bug
> Environment: ruby-driver 2.1.0 | C* 2.0.12 | C* 2.1.3
>Reporter: Kishan Karunaratne
>Assignee: Tyler Hobbs
>
> When retrieving the next page from the result of a simple statement, the 
> result will return a non-null paging state even if it's the last page of the 
> query. This only happens if it's the last page, and the results of the last 
> page exactly matches the paging size.
> Schema:
> {noformat}
>   CREATE KEYSPACE simplex WITH replication = {'class': 'SimpleStrategy', 
> 'replication_factor': 3};
>   USE simplex;
>   CREATE TABLE test (k text, v int, PRIMARY KEY (k, v));
>   INSERT INTO test (k, v) VALUES ('a', 0);
>   INSERT INTO test (k, v) VALUES ('b', 1);
>   INSERT INTO test (k, v) VALUES ('c', 2);
>   INSERT INTO test (k, v) VALUES ('d', 3);
>   INSERT INTO test (k, v) VALUES ('e', 4);
> {noformat}
> Query:
> {noformat}
>   result  = session.execute("SELECT * FROM test", page_size: 5)
>   loop do
> puts "last page? #{result.last_page?}"
> puts "page size: #{result.size}"
> result.each do |row|
>   puts row
> end
> puts ""
> break if result.last_page?
> result = result.next_page
>   end
> {noformat}
> Result:
> {noformat}
>   last page? false
>   page size: 5
>   {"k"=>"a", "v"=>0}
>   {"k"=>"c", "v"=>2}
>   {"k"=>"m", "v"=>12}
>   {"k"=>"f", "v"=>5}
>   {"k"=>"o", "v"=>14}
>   
>   last page? true
>   page size: 0
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-8871) Non-null paging state returned if last page

2017-07-26 Thread Andy Tolbert (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andy Tolbert updated CASSANDRA-8871:

Description: 
tWhen retrieving the next page from the result of a simple statement, the 
result will return a non-null paging state even if it's the last page of the 
query. This only happens if it's the last page, and the results of the last 
page exactly matches the paging size.

Schema:
{noformat}
  CREATE KEYSPACE simplex WITH replication = {'class': 'SimpleStrategy', 
'replication_factor': 3};
  USE simplex;
  CREATE TABLE test (k text, v int, PRIMARY KEY (k, v));
  INSERT INTO test (k, v) VALUES ('a', 0);
  INSERT INTO test (k, v) VALUES ('b', 1);
  INSERT INTO test (k, v) VALUES ('c', 2);
  INSERT INTO test (k, v) VALUES ('d', 3);
  INSERT INTO test (k, v) VALUES ('e', 4);
{noformat}

Query:
{noformat}
  result  = session.execute("SELECT * FROM test", page_size: 5)

  loop do
puts "last page? #{result.last_page?}"
puts "page size: #{result.size}"

result.each do |row|
  puts row
end
puts ""

break if result.last_page?
result = result.next_page
  end
{noformat}

Result:
{noformat}
  last page? false
  page size: 5
  {"k"=>"a", "v"=>0}
  {"k"=>"c", "v"=>2}
  {"k"=>"m", "v"=>12}
  {"k"=>"f", "v"=>5}
  {"k"=>"o", "v"=>14}
  
  last page? true
  page size: 0
{noformat}


  was:
When retrieving the next page from the result of a simple statement, the result 
will return a non-null paging state even if it's the last page of the query. 
This only happens if it's the last page, and the results of the last page 
exactly matches the paging size.

Schema:
{noformat}
  CREATE KEYSPACE simplex WITH replication = {'class': 'SimpleStrategy', 
'replication_factor': 3};
  USE simplex;
  CREATE TABLE test (k text, v int, PRIMARY KEY (k, v));
  INSERT INTO test (k, v) VALUES ('a', 0);
  INSERT INTO test (k, v) VALUES ('b', 1);
  INSERT INTO test (k, v) VALUES ('c', 2);
  INSERT INTO test (k, v) VALUES ('d', 3);
  INSERT INTO test (k, v) VALUES ('e', 4);
{noformat}

Query:
{noformat}
  result  = session.execute("SELECT * FROM test", page_size: 5)

  loop do
puts "last page? #{result.last_page?}"
puts "page size: #{result.size}"

result.each do |row|
  puts row
end
puts ""

break if result.last_page?
result = result.next_page
  end
{noformat}

Result:
{noformat}
  last page? false
  page size: 5
  {"k"=>"a", "v"=>0}
  {"k"=>"c", "v"=>2}
  {"k"=>"m", "v"=>12}
  {"k"=>"f", "v"=>5}
  {"k"=>"o", "v"=>14}
  
  last page? true
  page size: 0
{noformat}



> Non-null paging state returned if last page
> ---
>
> Key: CASSANDRA-8871
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8871
> Project: Cassandra
>  Issue Type: Bug
> Environment: ruby-driver 2.1.0 | C* 2.0.12 | C* 2.1.3
>Reporter: Kishan Karunaratne
>Assignee: Tyler Hobbs
>
> tWhen retrieving the next page from the result of a simple statement, the 
> result will return a non-null paging state even if it's the last page of the 
> query. This only happens if it's the last page, and the results of the last 
> page exactly matches the paging size.
> Schema:
> {noformat}
>   CREATE KEYSPACE simplex WITH replication = {'class': 'SimpleStrategy', 
> 'replication_factor': 3};
>   USE simplex;
>   CREATE TABLE test (k text, v int, PRIMARY KEY (k, v));
>   INSERT INTO test (k, v) VALUES ('a', 0);
>   INSERT INTO test (k, v) VALUES ('b', 1);
>   INSERT INTO test (k, v) VALUES ('c', 2);
>   INSERT INTO test (k, v) VALUES ('d', 3);
>   INSERT INTO test (k, v) VALUES ('e', 4);
> {noformat}
> Query:
> {noformat}
>   result  = session.execute("SELECT * FROM test", page_size: 5)
>   loop do
> puts "last page? #{result.last_page?}"
> puts "page size: #{result.size}"
> result.each do |row|
>   puts row
> end
> puts ""
> break if result.last_page?
> result = result.next_page
>   end
> {noformat}
> Result:
> {noformat}
>   last page? false
>   page size: 5
>   {"k"=>"a", "v"=>0}
>   {"k"=>"c", "v"=>2}
>   {"k"=>"m", "v"=>12}
>   {"k"=>"f", "v"=>5}
>   {"k"=>"o", "v"=>14}
>   
>   last page? true
>   page size: 0
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13615) Include 'ppc64le' library for sigar-1.6.4.jar

2017-07-26 Thread Michael Shuler (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101705#comment-16101705
 ] 

Michael Shuler commented on CASSANDRA-13615:


Neither of the {{ppc64le}} label machines in Jenkins are online, as of this 
comment, and they have not been for quite some time.
https://builds.apache.org/label/ppc64le/

> Include 'ppc64le' library for sigar-1.6.4.jar
> -
>
> Key: CASSANDRA-13615
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13615
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Libraries
> Environment: # arch
> ppc64le
>Reporter: Amitkumar Ghatwal
>  Labels: easyfix
> Fix For: 3.0.x, 3.11.x, 4.x
>
> Attachments: libsigar-ppc64le-linux.so
>
>
> Hi All,
> sigar-1.6.4.jar does not include a ppc64le library, so we had to install 
> libsigar-ppc64le-linux.so.As the community has been inactive for long 
> (https://github.com/hyperic/sigar), requesting the community to include the 
> ppc64le library directly here.
> Attaching the ppc64le library ( *.so) file to be included under 
> "/lib/sigar-bin". let me know of issues/dependency if any.
> FYI - [~ReiOdaira],[~jjirsa], [~mshuler]
> Regards,
> Amit



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13724) Cleanup build.xml dependencies

2017-07-26 Thread Stefan Podkowinski (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101667#comment-16101667
 ] 

Stefan Podkowinski commented on CASSANDRA-13724:


The biggest benefit is that you're more open to community contributions with a 
maven based build process. Our home-grown ant solution works, but most devs 
will only ever have to build, test and manage dependencies. Standard tasks like 
this shouldn't require all the complexity that comes with build.xml.

What I'd like to try out at some point is to see if we can have both solutions 
in the future, the ant process working as is and additionally maven, which 
would be able to compile, test and package the project. Ant targets could then 
be migrated to maven over time, or just [executed by 
maven|http://maven.apache.org/plugins/maven-antrun-plugin/].

It's really not that hard to do this step by step. I managed to replace the 
maven-ant-tasks (project not maintained for many years) with the Aether 
dependency resolver that is used starting in maven 3. Doing so allows you to 
externalize pom files with your dependencies and we could continue to take this 
further by calling from ant into maven or the other way around (but probably 
not in this ticket). See 
[WIP-13724|https://github.com/spodkowinski/cassandra/tree/WIP-13724]


> Cleanup build.xml dependencies
> --
>
> Key: CASSANDRA-13724
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13724
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Build
>Reporter: Stefan Podkowinski
>Priority: Minor
> Fix For: 4.x
>
>
> As discussed in CASSANDRA-12996, the build.xml has become pretty messy and 
> could use some cleanup and additional inline comments. Maybe parts of it 
> could be even split up or externalized.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13725) Secondary indexes are always rebuilt at startup


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrés de la Peña updated CASSANDRA-13725:
--
Status: Ready to Commit  (was: Patch Available)

> Secondary indexes are always rebuilt at startup
> ---
>
> Key: CASSANDRA-13725
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13725
> Project: Cassandra
>  Issue Type: Bug
>  Components: Secondary Indexes
>Reporter: Sergio Bossa
>Assignee: Sergio Bossa
>Priority: Critical
> Fix For: 4.0
>
>
> Following CASSANDRA-10130, a bug has been introduced that causes a 2i to be 
> rebuilt at startup, even if such index is already built.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13725) Secondary indexes are always rebuilt at startup


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101609#comment-16101609
 ] 

Andrés de la Peña commented on CASSANDRA-13725:
---

Both the patch and the dtest look good to me, and the CI results seem ok, +1.

> Secondary indexes are always rebuilt at startup
> ---
>
> Key: CASSANDRA-13725
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13725
> Project: Cassandra
>  Issue Type: Bug
>  Components: Secondary Indexes
>Reporter: Sergio Bossa
>Assignee: Sergio Bossa
>Priority: Critical
> Fix For: 4.0
>
>
> Following CASSANDRA-10130, a bug has been introduced that causes a 2i to be 
> rebuilt at startup, even if such index is already built.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-11500) Obsolete MV entry may not be properly deleted


[ 
https://issues.apache.org/jira/browse/CASSANDRA-11500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101603#comment-16101603
 ] 

ZhaoYang commented on CASSANDRA-11500:
--

WIP 
[branch|https://github.com/jasonstack/cassandra/commits/CASSANDRA-11500-cell]

Changed:
Extra "VirtualCell"(kind of special LivenessInfo for MV) in Row. It stores: 1. 
base column used in view PK and base column used in view filter conditions. if 
any of such column dead, entire view row dead, regardless LivenessInfo or 
DeletionTime status. 2. unselected base columns. if any of such column alive, 
view's pk should be alive if it's not deleted by DeletionTime or those columns 
in <1>. 

Todo: 
optimize in-memory/storage representation for "virtual-cells" to re-use 
AbstractRow/BTree
discard dead view row shadowed by virtualCells in compaction after gc_grace
more dtest
support collection type in unselected column (for now, it won't generate 
"virtual-cells" for non-frozen collection type)

> Obsolete MV entry may not be properly deleted
> -
>
> Key: CASSANDRA-11500
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11500
> Project: Cassandra
>  Issue Type: Bug
>  Components: Materialized Views
>Reporter: Sylvain Lebresne
>Assignee: ZhaoYang
>
> When a Materialized View uses a non-PK base table column in its PK, if an 
> update changes that column value, we add the new view entry and remove the 
> old one. When doing that removal, the current code uses the same timestamp 
> than for the liveness info of the new entry, which is the max timestamp for 
> any columns participating to the view PK. This is not correct for the 
> deletion as the old view entry could have other columns with higher timestamp 
> which won't be deleted as can easily shown by the failing of the following 
> test:
> {noformat}
> CREATE TABLE t (k int PRIMARY KEY, a int, b int);
> CREATE MATERIALIZED VIEW mv AS SELECT * FROM t WHERE k IS NOT NULL AND a IS 
> NOT NULL PRIMARY KEY (k, a);
> INSERT INTO t(k, a, b) VALUES (1, 1, 1) USING TIMESTAMP 0;
> UPDATE t USING TIMESTAMP 4 SET b = 2 WHERE k = 1;
> UPDATE t USING TIMESTAMP 2 SET a = 2 WHERE k = 1;
> SELECT * FROM mv WHERE k = 1; // This currently return 2 entries, the old 
> (invalid) and the new one
> {noformat}
> So the correct timestamp to use for the deletion is the biggest timestamp in 
> the old view entry (which we know since we read the pre-existing base row), 
> and that is what CASSANDRA-11475 does (the test above thus doesn't fail on 
> that branch).
> Unfortunately, even then we can still have problems if further updates 
> requires us to overide the old entry. Consider the following case:
> {noformat}
> CREATE TABLE t (k int PRIMARY KEY, a int, b int);
> CREATE MATERIALIZED VIEW mv AS SELECT * FROM t WHERE k IS NOT NULL AND a IS 
> NOT NULL PRIMARY KEY (k, a);
> INSERT INTO t(k, a, b) VALUES (1, 1, 1) USING TIMESTAMP 0;
> UPDATE t USING TIMESTAMP 10 SET b = 2 WHERE k = 1;
> UPDATE t USING TIMESTAMP 2 SET a = 2 WHERE k = 1; // This will delete the 
> entry for a=1 with timestamp 10
> UPDATE t USING TIMESTAMP 3 SET a = 1 WHERE k = 1; // This needs to re-insert 
> an entry for a=1 but shouldn't be deleted by the prior deletion
> UPDATE t USING TIMESTAMP 4 SET a = 2 WHERE k = 1; // ... and we can play this 
> game more than once
> UPDATE t USING TIMESTAMP 5 SET a = 1 WHERE k = 1;
> ...
> {noformat}
> In a way, this is saying that the "shadowable" deletion mechanism is not 
> general enough: we need to be able to re-insert an entry when a prior one had 
> been deleted before, but we can't rely on timestamps being strictly bigger on 
> the re-insert. In that sense, this can be though as a similar problem than 
> CASSANDRA-10965, though the solution there of a single flag is not enough 
> since we can have to replace more than once.
> I think the proper solution would be to ship enough information to always be 
> able to decide when a view deletion is shadowed. Which means that both 
> liveness info (for updates) and shadowable deletion would need to ship the 
> timestamp of any base table column that is part the view PK (so {{a}} in the 
> example below).  It's doable (and not that hard really), but it does require 
> a change to the sstable and intra-node protocol, which makes this a bit 
> painful right now.
> But I'll also note that as CASSANDRA-1096 shows, the timestamp is not even 
> enough since on equal timestamp the value can be the deciding factor. So in 
> theory we'd have to ship the value of those columns (in the case of a 
> deletion at least since we have it in the view PK for updates). That said, on 
> that last problem, my preference would be that we start prioritizing 
> CASSANDRA-6123 seriously so we don't have to care about conflicting timestamp 
> anymore, which would make this problem go

[jira] [Updated] (CASSANDRA-13708) Warn or error when receiving stream requests or inbound streams that contain unowned token ranges

2017-07-26 Thread Sam Tunnicliffe (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-13708:

Description: 
If a node receives a StreamRequest that includes ranges outside of those owned 
by the node, a warning should be logged. 

On the receiving side, when deserializing a streamed SSTable we should also log 
a warning if we encounter keys outside of the node's owned ranges. Again, we 
could also record a metric for such events and optionally fail the streaming 
session.

  was:
f a node receives a StreamRequest that includes ranges outside of those owned 
by the node, a warning should be logged. 

On the receiving side, when deserializing a streamed SSTable we should also log 
a warning if we encounter keys outside of the node's owned ranges. Again, we 
could also record a metric for such events and optionally fail the streaming 
session.


> Warn or error when receiving stream requests or inbound streams that contain 
> unowned token ranges
> -
>
> Key: CASSANDRA-13708
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13708
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Coordination, Observability, Streaming and Messaging
>Reporter: Sam Tunnicliffe
>
> If a node receives a StreamRequest that includes ranges outside of those 
> owned by the node, a warning should be logged. 
> On the receiving side, when deserializing a streamed SSTable we should also 
> log a warning if we encounter keys outside of the node's owned ranges. Again, 
> we could also record a metric for such events and optionally fail the 
> streaming session.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13547) Filtered materialized views missing data


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhaoYang updated CASSANDRA-13547:
-
Resolution: Duplicate
Status: Resolved  (was: Patch Available)

thanks, mark it as duplicate, move to 11500 with new MV timestamp

> Filtered materialized views missing data
> 
>
> Key: CASSANDRA-13547
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13547
> Project: Cassandra
>  Issue Type: Bug
>  Components: Materialized Views
> Environment: Official Cassandra 3.10 Docker image (ID 154b919bf8ce).
>Reporter: Craig Nicholson
>Assignee: Krishna Dattu Koneru
>Priority: Blocker
>  Labels: materializedviews
> Fix For: 3.11.x
>
>
> When creating a materialized view against a base table the materialized view 
> does not always reflect the correct data.
> Using the following test schema:
> {code:title=Schema|language=sql}
> DROP KEYSPACE IF EXISTS test;
> CREATE KEYSPACE test
>   WITH REPLICATION = { 
>'class' : 'SimpleStrategy', 
>'replication_factor' : 1 
>   };
> CREATE TABLE test.table1 (
> id int,
> name text,
> enabled boolean,
> foo text,
> PRIMARY KEY (id, name));
> CREATE MATERIALIZED VIEW test.table1_mv1 AS SELECT id, name, foo
> FROM test.table1
> WHERE id IS NOT NULL 
> AND name IS NOT NULL 
> AND enabled = TRUE
> PRIMARY KEY ((name), id);
> CREATE MATERIALIZED VIEW test.table1_mv2 AS SELECT id, name, foo, enabled
> FROM test.table1
> WHERE id IS NOT NULL 
> AND name IS NOT NULL 
> AND enabled = TRUE
> PRIMARY KEY ((name), id);
> {code}
> When I insert a row into the base table the materialized views are updated 
> appropriately. (+)
> {code:title=Insert row|language=sql}
> cqlsh> INSERT INTO test.table1 (id, name, enabled, foo) VALUES (1, 'One', 
> TRUE, 'Bar');
> cqlsh> SELECT * FROM test.table1;
>  id | name | enabled | foo
> +--+-+-
>   1 |  One |True | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
>  name | id | foo
> --++-
>   One |  1 | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
>  name | id | enabled | foo
> --++-+-
>   One |  1 |True | Bar
> (1 rows)
> {code}
> Updating the record in the base table and setting enabled to FALSE will 
> filter the record from both materialized views. (+)
> {code:title=Disable the row|language=sql}
> cqlsh> UPDATE test.table1 SET enabled = FALSE WHERE id = 1 AND name = 'One';
> cqlsh> SELECT * FROM test.table1;
>  id | name | enabled | foo
> +--+-+-
>   1 |  One |   False | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
>  name | id | foo
> --++-
> (0 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
>  name | id | enabled | foo
> --++-+-
> (0 rows)
> {code}
> However a further update to the base table setting enabled to TRUE should 
> include the record in both materialzed views, however only one view 
> (table1_mv2) gets updated. (-)
> It appears that only the view (table1_mv2) that returns the filtered column 
> (enabled) is updated. (-)
> Additionally columns that are not part of the partiion or clustering key are 
> not updated. You can see that the foo column has a null value in table1_mv2. 
> (-)
> {code:title=Enable the row|language=sql}
> cqlsh> UPDATE test.table1 SET enabled = TRUE WHERE id = 1 AND name = 'One';
> cqlsh> SELECT * FROM test.table1;
>  id | name | enabled | foo
> +--+-+-
>   1 |  One |True | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
>  name | id | foo
> --++-
> (0 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
>  name | id | enabled | foo
> --++-+--
>   One |  1 |True | null
> (1 rows)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13127) Materialized Views: View row expires too soon


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhaoYang updated CASSANDRA-13127:
-
Resolution: Duplicate
Status: Resolved  (was: Patch Available)

mark this as duplicate, move to 11500 with new MV timestamp

> Materialized Views: View row expires too soon
> -
>
> Key: CASSANDRA-13127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13127
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths, Materialized Views
>Reporter: Duarte Nunes
>Assignee: ZhaoYang
>
> Consider the following commands, ran against trunk:
> {code}
> echo "DROP MATERIALIZED VIEW ks.mv; DROP TABLE ks.base;" | bin/cqlsh
> echo "CREATE TABLE ks.base (p int, c int, v int, PRIMARY KEY (p, c));" | 
> bin/cqlsh
> echo "CREATE MATERIALIZED VIEW ks.mv AS SELECT p, c FROM base WHERE p IS NOT 
> NULL AND c IS NOT NULL PRIMARY KEY (c, p);" | bin/cqlsh
> echo "INSERT INTO ks.base (p, c) VALUES (0, 0) USING TTL 10;" | bin/cqlsh
> # wait for row liveness to get closer to expiration
> sleep 6;
> echo "UPDATE ks.base USING TTL 8 SET v = 0 WHERE p = 0 and c = 0;" | bin/cqlsh
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
>  p | c | ttl(v)
> ---+---+
>  0 | 0 |  7
> (1 rows)
>  c | p
> ---+---
>  0 | 0
> (1 rows)
> # wait for row liveness to expire
> sleep 4;
> echo "SELECT p, c, ttl(v) FROM ks.base; SELECT * FROM ks.mv;" | bin/cqlsh
>  p | c | ttl(v)
> ---+---+
>  0 | 0 |  3
> (1 rows)
>  c | p
> ---+---
> (0 rows)
> {code}
> Notice how the view row is removed even though the base row is still live. I 
> would say this is because in ViewUpdateGenerator#computeLivenessInfoForEntry 
> the TTLs are compared instead of the expiration times, but I'm not sure I'm 
> getting that far ahead in the code when updating a column that's not in the 
> view.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13731) Timeout while setting keyspace


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101341#comment-16101341
 ] 

peng xiao commented on CASSANDRA-13731:
---

maybe this should not raised here? as per  
https://datastax-oss.atlassian.net/browse/JAVA-1002
we just need to upgrade the driver to 3.0.1.Could you please confirm.

> Timeout while setting keyspace
> --
>
> Key: CASSANDRA-13731
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13731
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 2.1.13
>Reporter: peng xiao
>
> Hi,
> We are troubling a strange issue.Currently we have a Cluster with Cassandra 
> 2.1.13.
> when the applications start,it will print the following warings.And it takes 
> long time for applications to start.
> Could you please advise ?
> https://datastax-oss.atlassian.net/browse/JAVA-1002
> this one says prepares a PreparedStatement caused this issue and has been 
> fixed.
> but we are still experiencing this in 2.1.13.
> 2017-07-26 15:49:20.676  WARN 11706 --- [-] [cluster1-nio-worker-2] 
> com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
> Connection[/xxx.xxx.xxx.xxx:9042-3, inFlight=5, closed=false]. This should 
> not happen but is not critical (it will be retried)
> 2017-07-26 15:49:20.677  WARN 11706 --- [-] [cluster1-nio-worker-3] 
> com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
> Connection[/xxx.16.42.xxx:9042-3, inFlight=5, closed=false]. This should not 
> happen but is not critical (it will be retried)
> 2017-07-26 15:49:20.676  WARN 11706 --- [-] [cluster1-nio-worker-0] 
> com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
> Connection[/xxx.16.42.xxx:9042-3, inFlight=5, closed=false]. This should not 
> happen but is not critical (it will be retried)
> 2017-07-26 15:49:20.676  WARN 11706 --- [-] [cluster1-nio-worker-1] 
> com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
> Connection[/xxx.16.42.xxx:9042-3, inFlight=5, closed=false]. This should not 
> happen but is not critical (it will be retried)
> 2017-07-26 15:49:32.777  WARN 11706 --- [-] [main] 
> com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
> Connection[/xxx.xx.xx.xxx:9042-3, inFlight=1, closed=false]. This should not 
> happen but is not critical (it will be retried)
> Thanks



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13731) Timeout while setting keyspace


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

peng xiao updated CASSANDRA-13731:
--
Description: 
Hi,
We are troubling a strange issue.Currently we have a Cluster with Cassandra 
2.1.13.
when the applications start,it will print the following warings.And it takes 
long time for applications to start.
Could you please advise ?
https://datastax-oss.atlassian.net/browse/JAVA-1002
this one says prepares a PreparedStatement caused this issue and has been fixed.
but we are still experiencing this in 2.1.13.

2017-07-26 15:49:20.676  WARN 11706 --- [-] [cluster1-nio-worker-2] 
com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
Connection[/xxx.xxx.xxx.xxx:9042-3, inFlight=5, closed=false]. This should not 
happen but is not critical (it will be retried)
2017-07-26 15:49:20.677  WARN 11706 --- [-] [cluster1-nio-worker-3] 
com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
Connection[/xxx.16.42.xxx:9042-3, inFlight=5, closed=false]. This should not 
happen but is not critical (it will be retried)
2017-07-26 15:49:20.676  WARN 11706 --- [-] [cluster1-nio-worker-0] 
com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
Connection[/xxx.16.42.xxx:9042-3, inFlight=5, closed=false]. This should not 
happen but is not critical (it will be retried)
2017-07-26 15:49:20.676  WARN 11706 --- [-] [cluster1-nio-worker-1] 
com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
Connection[/xxx.16.42.xxx:9042-3, inFlight=5, closed=false]. This should not 
happen but is not critical (it will be retried)
2017-07-26 15:49:32.777  WARN 11706 --- [-] [main] 
com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
Connection[/xxx.xx.xx.xxx:9042-3, inFlight=1, closed=false]. This should not 
happen but is not critical (it will be retried)

Thanks

  was:
2017-07-26 15:49:20.676  WARN 11706 --- [-] [cluster1-nio-worker-2] 
com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
Connection[/xxx.xxx.xxx.xxx:9042-3, inFlight=5, closed=false]. This should not 
happen but is not critical (it will be retried)
2017-07-26 15:49:20.677  WARN 11706 --- [-] [cluster1-nio-worker-3] 
com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
Connection[/xxx.16.42.xxx:9042-3, inFlight=5, closed=false]. This should not 
happen but is not critical (it will be retried)
2017-07-26 15:49:20.676  WARN 11706 --- [-] [cluster1-nio-worker-0] 
com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
Connection[/xxx.16.42.xxx:9042-3, inFlight=5, closed=false]. This should not 
happen but is not critical (it will be retried)
2017-07-26 15:49:20.676  WARN 11706 --- [-] [cluster1-nio-worker-1] 
com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
Connection[/xxx.16.42.xxx:9042-3, inFlight=5, closed=false]. This should not 
happen but is not critical (it will be retried)
2017-07-26 15:49:32.777  WARN 11706 --- [-] [main] 
com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
Connection[/xxx.xx.xx.xxx:9042-3, inFlight=1, closed=false]. This should not 
happen but is not critical (it will be retried)


> Timeout while setting keyspace
> --
>
> Key: CASSANDRA-13731
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13731
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 2.1.13
>Reporter: peng xiao
>
> Hi,
> We are troubling a strange issue.Currently we have a Cluster with Cassandra 
> 2.1.13.
> when the applications start,it will print the following warings.And it takes 
> long time for applications to start.
> Could you please advise ?
> https://datastax-oss.atlassian.net/browse/JAVA-1002
> this one says prepares a PreparedStatement caused this issue and has been 
> fixed.
> but we are still experiencing this in 2.1.13.
> 2017-07-26 15:49:20.676  WARN 11706 --- [-] [cluster1-nio-worker-2] 
> com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
> Connection[/xxx.xxx.xxx.xxx:9042-3, inFlight=5, closed=false]. This should 
> not happen but is not critical (it will be retried)
> 2017-07-26 15:49:20.677  WARN 11706 --- [-] [cluster1-nio-worker-3] 
> com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
> Connection[/xxx.16.42.xxx:9042-3, inFlight=5, closed=false]. This should not 
> happen but is not critical (it will be retried)
> 2017-07-26 15:49:20.676  WARN 11706 --- [-] [cluster1-nio-worker-0] 
> com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
> Connection[/xxx.16.42.xxx:9042-3, inFlight=5, closed=false]. This should not 
> happen but is not critical (it will be retried)
> 2017-07-26 15:49:20.676  WARN 11706 --- [-]

[jira] [Updated] (CASSANDRA-13731) Timeout while setting keyspace


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

peng xiao updated CASSANDRA-13731:
--
Description: 
2017-07-26 15:49:20.676  WARN 11706 --- [-] [cluster1-nio-worker-2] 
com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
Connection[/xxx.xxx.xxx.xxx:9042-3, inFlight=5, closed=false]. This should not 
happen but is not critical (it will be retried)
2017-07-26 15:49:20.677  WARN 11706 --- [-] [cluster1-nio-worker-3] 
com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
Connection[/xxx.16.42.xxx:9042-3, inFlight=5, closed=false]. This should not 
happen but is not critical (it will be retried)
2017-07-26 15:49:20.676  WARN 11706 --- [-] [cluster1-nio-worker-0] 
com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
Connection[/xxx.16.42.xxx:9042-3, inFlight=5, closed=false]. This should not 
happen but is not critical (it will be retried)
2017-07-26 15:49:20.676  WARN 11706 --- [-] [cluster1-nio-worker-1] 
com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
Connection[/xxx.16.42.xxx:9042-3, inFlight=5, closed=false]. This should not 
happen but is not critical (it will be retried)
2017-07-26 15:49:32.777  WARN 11706 --- [-] [main] 
com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
Connection[/xxx.xx.xx.xxx:9042-3, inFlight=1, closed=false]. This should not 
happen but is not critical (it will be retried)

  was:
2017-07-26 15:49:20.676  WARN 11706 --- [-] [cluster1-nio-worker-2] 
com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
Connection[/xxx.xxx.xxx.xxx:9042-3, inFlight=5, closed=false]. This should not 
happen but is not critical (it will be retried)
2017-07-26 15:49:20.677  WARN 11706 --- [-] [cluster1-nio-worker-3] 
com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
Connection[/xxx.16.42.xxx:9042-3, inFlight=5, closed=false]. This should not 
happen but is not critical (it will be retried)
2017-07-26 15:49:20.676  WARN 11706 --- [-] [cluster1-nio-worker-0] 
com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
Connection[/xxx.16.42.xxx:9042-3, inFlight=5, closed=false]. This should not 
happen but is not critical (it will be retried)
2017-07-26 15:49:20.676  WARN 11706 --- [-] [cluster1-nio-worker-1] 
com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
Connection[/xxx.16.42.xxx:9042-3, inFlight=5, closed=false]. This should not 
happen but is not critical (it will be retried)
2017-07-26 15:49:32.777  WARN 11706 --- [-] [main] 
com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
Connection[/172.16.42.113:9042-3, inFlight=1, closed=false]. This should not 
happen but is not critical (it will be retried)


> Timeout while setting keyspace
> --
>
> Key: CASSANDRA-13731
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13731
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra 2.1.13
>Reporter: peng xiao
>
> 2017-07-26 15:49:20.676  WARN 11706 --- [-] [cluster1-nio-worker-2] 
> com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
> Connection[/xxx.xxx.xxx.xxx:9042-3, inFlight=5, closed=false]. This should 
> not happen but is not critical (it will be retried)
> 2017-07-26 15:49:20.677  WARN 11706 --- [-] [cluster1-nio-worker-3] 
> com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
> Connection[/xxx.16.42.xxx:9042-3, inFlight=5, closed=false]. This should not 
> happen but is not critical (it will be retried)
> 2017-07-26 15:49:20.676  WARN 11706 --- [-] [cluster1-nio-worker-0] 
> com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
> Connection[/xxx.16.42.xxx:9042-3, inFlight=5, closed=false]. This should not 
> happen but is not critical (it will be retried)
> 2017-07-26 15:49:20.676  WARN 11706 --- [-] [cluster1-nio-worker-1] 
> com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
> Connection[/xxx.16.42.xxx:9042-3, inFlight=5, closed=false]. This should not 
> happen but is not critical (it will be retried)
> 2017-07-26 15:49:32.777  WARN 11706 --- [-] [main] 
> com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
> Connection[/xxx.xx.xx.xxx:9042-3, inFlight=1, closed=false]. This should not 
> happen but is not critical (it will be retried)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-13731) Timeout while setting keyspace

peng xiao created CASSANDRA-13731:
-

 Summary: Timeout while setting keyspace
 Key: CASSANDRA-13731
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13731
 Project: Cassandra
  Issue Type: Bug
 Environment: Cassandra 2.1.13
Reporter: peng xiao


2017-07-26 15:49:20.676  WARN 11706 --- [-] [cluster1-nio-worker-2] 
com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
Connection[/xxx.xxx.xxx.xxx:9042-3, inFlight=5, closed=false]. This should not 
happen but is not critical (it will be retried)
2017-07-26 15:49:20.677  WARN 11706 --- [-] [cluster1-nio-worker-3] 
com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
Connection[/xxx.16.42.xxx:9042-3, inFlight=5, closed=false]. This should not 
happen but is not critical (it will be retried)
2017-07-26 15:49:20.676  WARN 11706 --- [-] [cluster1-nio-worker-0] 
com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
Connection[/xxx.16.42.xxx:9042-3, inFlight=5, closed=false]. This should not 
happen but is not critical (it will be retried)
2017-07-26 15:49:20.676  WARN 11706 --- [-] [cluster1-nio-worker-1] 
com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
Connection[/xxx.16.42.xxx:9042-3, inFlight=5, closed=false]. This should not 
happen but is not critical (it will be retried)
2017-07-26 15:49:32.777  WARN 11706 --- [-] [main] 
com.datastax.driver.core.Connection  : Timeout while setting keyspace on 
Connection[/172.16.42.113:9042-3, inFlight=1, closed=false]. This should not 
happen but is not critical (it will be retried)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13601) Changes requested to the cassandra's debian + rpm installers packages


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101333#comment-16101333
 ] 

Amitkumar Ghatwal commented on CASSANDRA-13601:
---

also any comments for this ticket attached patch -   
ppc64le_unaligned_memory_access.patch ?


> Changes requested to the cassandra's debian + rpm installers packages
> -
>
> Key: CASSANDRA-13601
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13601
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Packaging
> Environment: ~$ lscpu
> Architecture:  ppc64le
> Byte Order:Little Endian
>Reporter: Amitkumar Ghatwal
>Priority: Minor
>  Labels: lhf
> Fix For: 3.0.x, 3.11.x, 4.x
>
> Attachments: ppc64le_unaligned_memory_access.patch
>
>
> Hi All,
> Thanks [~mshuler] for helping in installing cassandra using arch independent 
> installers  for debian + rpm packages from here : 
> http://cassandra.apache.org/download/
> For my architecture - " ppc64le" , the installation process from debian + rpm 
> wasn't straightforward. And needed below configuration level changes.
> For Ubuntu- Cassandra 3.10 release - below changes were needed
> 1) echo "deb [arch=amd64] http://www.apache.org/dist/cassandra/debian 310x 
> main" | sudo tee -a /etc/apt/sources.list.d/cassandra.sources.list 
> 2) sed -i -e s/Xss256/Xss512/g /etc/cassandra/jvm.options
> 3) Removing jna-4.0.0.jar and replacing it with latest jna-4.4.0.jar in 
> (/usr/share/cassandra/lib)- Downloaded from here . 
> 4) Restart cassandra service
> For RHEL - Cassandra 3.0.13 release - below changes were needed
> 1) sed -i -e s/Xss256/Xss512/g /etc/cassandra/default.conf/cassandra-env.sh
> 3) Removing jna-4.0.0.jar and replacing it with latest jna-4.4.0.jar in 
> (/usr/share/cassandra/lib)- Downloaded from here . 
> 4) Restart cassandra service
> Could you please help in introducing above changes so that cassandra can be 
> installed from the debian + rpm pcakages and will indeed become architecture 
> independent.
> Regards,
> Amit



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13601) Changes requested to the cassandra's debian + rpm installers packages


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101329#comment-16101329
 ] 

Amitkumar Ghatwal commented on CASSANDRA-13601:
---

Is it possible to increase the stack size specific to power to 512k using above 
method (cassandra-env.sh) - [~jjirsa] [~mshuler]

> Changes requested to the cassandra's debian + rpm installers packages
> -
>
> Key: CASSANDRA-13601
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13601
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Packaging
> Environment: ~$ lscpu
> Architecture:  ppc64le
> Byte Order:Little Endian
>Reporter: Amitkumar Ghatwal
>Priority: Minor
>  Labels: lhf
> Fix For: 3.0.x, 3.11.x, 4.x
>
> Attachments: ppc64le_unaligned_memory_access.patch
>
>
> Hi All,
> Thanks [~mshuler] for helping in installing cassandra using arch independent 
> installers  for debian + rpm packages from here : 
> http://cassandra.apache.org/download/
> For my architecture - " ppc64le" , the installation process from debian + rpm 
> wasn't straightforward. And needed below configuration level changes.
> For Ubuntu- Cassandra 3.10 release - below changes were needed
> 1) echo "deb [arch=amd64] http://www.apache.org/dist/cassandra/debian 310x 
> main" | sudo tee -a /etc/apt/sources.list.d/cassandra.sources.list 
> 2) sed -i -e s/Xss256/Xss512/g /etc/cassandra/jvm.options
> 3) Removing jna-4.0.0.jar and replacing it with latest jna-4.4.0.jar in 
> (/usr/share/cassandra/lib)- Downloaded from here . 
> 4) Restart cassandra service
> For RHEL - Cassandra 3.0.13 release - below changes were needed
> 1) sed -i -e s/Xss256/Xss512/g /etc/cassandra/default.conf/cassandra-env.sh
> 3) Removing jna-4.0.0.jar and replacing it with latest jna-4.4.0.jar in 
> (/usr/share/cassandra/lib)- Downloaded from here . 
> 4) Restart cassandra service
> Could you please help in introducing above changes so that cassandra can be 
> installed from the debian + rpm pcakages and will indeed become architecture 
> independent.
> Regards,
> Amit



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13615) Include 'ppc64le' library for sigar-1.6.4.jar


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101325#comment-16101325
 ] 

Amitkumar Ghatwal commented on CASSANDRA-13615:
---

[~jjirsa] [~mshuler]  - any comments/updates on the above please ?

> Include 'ppc64le' library for sigar-1.6.4.jar
> -
>
> Key: CASSANDRA-13615
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13615
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Libraries
> Environment: # arch
> ppc64le
>Reporter: Amitkumar Ghatwal
>  Labels: easyfix
> Fix For: 3.0.x, 3.11.x, 4.x
>
> Attachments: libsigar-ppc64le-linux.so
>
>
> Hi All,
> sigar-1.6.4.jar does not include a ppc64le library, so we had to install 
> libsigar-ppc64le-linux.so.As the community has been inactive for long 
> (https://github.com/hyperic/sigar), requesting the community to include the 
> ppc64le library directly here.
> Attaching the ppc64le library ( *.so) file to be included under 
> "/lib/sigar-bin". let me know of issues/dependency if any.
> FYI - [~ReiOdaira],[~jjirsa], [~mshuler]
> Regards,
> Amit



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13387) Metrics for repair

2017-07-26 Thread Stefan Podkowinski (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101279#comment-16101279
 ] 

Stefan Podkowinski commented on CASSANDRA-13387:


Nothing really object here. Good catch on the IOException. Lets kick off the 
tests:

* [testall|https://circleci.com/gh/spodkowinski/cassandra/tree/CASSANDRA-13387]
* 
[dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/148/]



> Metrics for repair
> --
>
> Key: CASSANDRA-13387
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13387
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Simon Zhou
>Assignee: Simon Zhou
>Priority: Minor
>
> We're missing metrics for repair, especially for errors. From what I observed 
> now, the exception will be caught by UncaughtExceptionHandler set in 
> CassandraDaemon and is categorized as StorageMetrics.exceptions. This is one 
> example:
> {code}
> ERROR [AntiEntropyStage:1] 2017-03-27 18:17:08,385 CassandraDaemon.java:207 - 
> Exception in thread Thread[AntiEntropyStage:1,5,main]
> java.lang.RuntimeException: Parent repair session with id = 
> 8c85d260-1319-11e7-82a2-25090a89015f has failed.
> at 
> org.apache.cassandra.service.ActiveRepairService.getParentRepairSession(ActiveRepairService.java:377)
>  ~[apache-cassandra-3.0.10.jar:3.0.10]
> at 
> org.apache.cassandra.service.ActiveRepairService.removeParentRepairSession(ActiveRepairService.java:392)
>  ~[apache-cassandra-3.0.10.jar:3.0.10]
> at 
> org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:172)
>  ~[apache-cassandra-3.0.10.jar:3.0.10]
> at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) 
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_121]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_121]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_121]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13723) Trivial format error of digest in StorageProxy