[jira] [Commented] (CASSANDRA-14525) streaming failure during bootstrap makes new node into inconsistent state

[Kurt Greaves (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523222#comment-16523222
 ] 

Kurt Greaves commented on CASSANDRA-14525:
--

A few things:
 If we fail streaming and {{isSurveyMode}} is true we still get the NPE if auth 
is enabled when trying to connect to C* on that node. Not much we can do about 
this because auth isn't initialised until we join the ring, but I'm not sure 
why we should handle this situation differently, and also it's currently kind 
of broken. At the moment if you resume bootstrap after a streaming failure 
_while in write survey mode_, you will leave write survey mode on completion of 
bootstrapping (ouch).


 I think we should handle write survey bootstrapping the same as normal 
bootstrap, where if we get an error during streaming we don't start transports. 
Then, on resume, handle survey mode so that we _don't_ join the ring on 
completion of bootstrapping, but we do still start transports.


 On top of that, seeing as we're in this code anyway, I think it would be 
reasonable if we could look at handling the auth case a bit better when write 
survey is enabled as well. Ideally, if auth is required I see no point in 
starting the transports seeing as you'll always get an NPE, so maybe we can add 
a check for that in {{CassandraDaemon#start()?}}

{{DatabaseDescriptor.getAuthenticator().requireAuthentication()}} should be 
enough here I think.

 

Some things regarding the error message:
 We've got repeated information in our error:
{code:java}
WARN  [main] 2018-06-25 09:13:24,136 StorageService.java:935 - Some data 
streaming failed. Use nodetool to check bootstrap state and resume. For more, 
see `nodetool help bootstrap`. IN_PROGRESS
ERROR [main] 2018-06-25 09:13:32,190 CassandraDaemon.java:445 - Node is not yet 
bootstraped hence not enabling native transport. Use nodetool to check 
bootstrap state and resume. For more, see `nodetool helpbootstrap`
{code}
I think our new message should either be INFO or WARN (INFO is in line with 
other messages in {{start()}}, and I think it would make more sense if the 
original message in \{{StorageService}} was ERROR. We could change the message 
in CassandraDaemon to:
{code:java}
Not starting client transports as bootstrap has not completed.{code}
or something similar, to be more in line with the other info messages.

Finally, with your patch if we resume bootstrap we don't start thrift. As per 
Vince's patch, daemon.start() is desirable here over startNativeTransport so 
that we always start thrift and CQL.

> streaming failure during bootstrap makes new node into inconsistent state
> -
>
> Key: CASSANDRA-14525
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14525
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Jaydeepkumar Chovatia
>Assignee: Jaydeepkumar Chovatia
>Priority: Major
> Fix For: 4.0, 2.2.x, 3.0.x
>
>
> If bootstrap fails for newly joining node (most common reason is due to 
> streaming failure) then Cassandra state remains in {{joining}} state which is 
> fine but Cassandra also enables Native transport which makes overall state 
> inconsistent. This further creates NullPointer exception if auth is enabled 
> on the new node, please find reproducible steps here:
> For example if bootstrap fails due to streaming errors like
> {quote}java.util.concurrent.ExecutionException: 
> org.apache.cassandra.streaming.StreamException: Stream failed
>  at 
> com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299)
>  ~[guava-18.0.jar:na]
>  at 
> com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286)
>  ~[guava-18.0.jar:na]
>  at 
> com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116) 
> ~[guava-18.0.jar:na]
>  at 
> org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:1256)
>  [apache-cassandra-3.0.16.jar:3.0.16]
>  at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:894)
>  [apache-cassandra-3.0.16.jar:3.0.16]
>  at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:660)
>  [apache-cassandra-3.0.16.jar:3.0.16]
>  at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:573)
>  [apache-cassandra-3.0.16.jar:3.0.16]
>  at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:330) 
> [apache-cassandra-3.0.16.jar:3.0.16]
>  at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:567)
>  [apache-cassandra-3.0.16.jar:3.0.16]
>  at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:695) 
> [apache-cassandra-3.0.16.jar:3.0.16]
>  Caused by: 

[jira] [Commented] (CASSANDRA-14543) Hinted handoff to replay purgeable tombstones

[Jay Zhuang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523125#comment-16523125
 ] 

Jay Zhuang commented on CASSANDRA-14543:


Thanks [~KurtG] for the link. I think [~slebresne] is talking about not doing 
RR on post-gcgs tombstone, which I agree. Repairing post-gcgs tombstone will 
cause more unnecessary RR, the tombstones will bounce around nodes and not get 
compacted away: CASSANDRA-11427.

But hints-handoff will only happen once, and we know the target node is missing 
that deletion. It may not need it if the tombstone is repaired from another 
node within GCGS, otherwise, it's the best (only) way to delete the data on 
that node.



> Hinted handoff to replay purgeable tombstones 
> --
>
> Key: CASSANDRA-14543
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14543
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jay Zhuang
>Priority: Minor
>
> Hinted-handoff currently only dispatches and applies the mutations that are 
> within GCGS: 
> [{{Hint.java:97}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/hints/Hint.java#L97].
>  Which is to make sure it won't resurrect any deleted data.
> But replaying tombstones should be safe, it could reduce the chance to have 
> [un-repairable inconsistent 
> data|https://lists.apache.org/thread.html/2d3d39d960143d4d2146ed2530821504ff855e832713dec7d0afd8ac@%3Cdev.cassandra.apache.org%3E].
> Here is the user scenario it tries to fix:
> {noformat}
> 1. Create a 3 nodes cluster
> 2. Create a table with small gc_grace_seconds (for reproducing purpose):
> CREATE KEYSPACE foo WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 3};
> CREATE TABLE foo.bar (
> id int PRIMARY KEY,
> name text
> ) WITH gc_grace_seconds=30;
> 3. Insert data with consistency all:
> INSERT INTO foo.bar (id, name) VALUES(1, 'cstar');
> 4. stop 1 node
> $ ccm node2 stop
> 5. Delete the data with consistency quorum:
> DELETE FROM foo.bar WHERE id=1;
> 6. Wait 30 seconds and then start node2:
> $ ccm node2 start
> {noformat}
> Now, node2 has the data, node1/node3 have the purgeable tombstone. It 
> triggers RR every time which sends data from node2 to node1/node3 but repairs 
> nothing.
> With purgeable tombstones hints handoff, it at least will dispatch the 
> tombstone and delete the data on node2. It won't fix the root cause but 
> reduce the chance to have this issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14543) Hinted handoff to replay purgeable tombstones

[Kurt Greaves (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523042#comment-16523042
 ] 

Kurt Greaves commented on CASSANDRA-14543:
--

I'm not sure about this because it only "works" if you assume the delete 
happened within the HH window, and then the node was down for an additional 
(*GCGS - HH*). It does reduce the number of cases where users could be bitten 
by this, but not significantly, and I suspect this will become another layer of 
confusion when someone does hit the problem. 

I think also [~slebresne]'s comment 
[here|https://issues.apache.org/jira/browse/CASSANDRA-14532?focusedCommentId=16519023=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16519023]
 does a good job of explaining why sending tombstones after GCGS is a bad idea. 

 While I don't think the constant unnecessary read repair is good, there's not 
really many good solutions here. Ideally the read repair wouldn't happen in the 
first place and we exclude post gcgs tombstones from digest calc but that's 
likely very risky. So far it seems the best of a bunch of terrible solutions 
here is to either repair within GCGS, or never delete things. Going forward 
only_purge_repaired_tombstones w/ incremental repair should fix the underlying 
design problem here.

> Hinted handoff to replay purgeable tombstones 
> --
>
> Key: CASSANDRA-14543
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14543
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jay Zhuang
>Priority: Minor
>
> Hinted-handoff currently only dispatches and applies the mutations that are 
> within GCGS: 
> [{{Hint.java:97}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/hints/Hint.java#L97].
>  Which is to make sure it won't resurrect any deleted data.
> But replaying tombstones should be safe, it could reduce the chance to have 
> [un-repairable inconsistent 
> data|https://lists.apache.org/thread.html/2d3d39d960143d4d2146ed2530821504ff855e832713dec7d0afd8ac@%3Cdev.cassandra.apache.org%3E].
> Here is the user scenario it tries to fix:
> {noformat}
> 1. Create a 3 nodes cluster
> 2. Create a table with small gc_grace_seconds (for reproducing purpose):
> CREATE KEYSPACE foo WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 3};
> CREATE TABLE foo.bar (
> id int PRIMARY KEY,
> name text
> ) WITH gc_grace_seconds=30;
> 3. Insert data with consistency all:
> INSERT INTO foo.bar (id, name) VALUES(1, 'cstar');
> 4. stop 1 node
> $ ccm node2 stop
> 5. Delete the data with consistency quorum:
> DELETE FROM foo.bar WHERE id=1;
> 6. Wait 30 seconds and then start node2:
> $ ccm node2 start
> {noformat}
> Now, node2 has the data, node1/node3 have the purgeable tombstone. It 
> triggers RR every time which sends data from node2 to node1/node3 but repairs 
> nothing.
> With purgeable tombstones hints handoff, it at least will dispatch the 
> tombstone and delete the data on node2. It won't fix the root cause but 
> reduce the chance to have this issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14536) Bump the hints message version to match the current one

[Jay Zhuang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523015#comment-16523015
 ] 

Jay Zhuang commented on CASSANDRA-14536:


I think we should add a new hint file version instead of bumping the existing 
one's messagingVersion. It should be the same as CASSANDRA-13161. I updated the 
patch and added the unittest for that. Please review.

> Bump the hints message version to match the current one
> ---
>
> Key: CASSANDRA-14536
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14536
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Hints
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Minor
>
> Currently in trunk the 
> [{{Hints.messagingVersion()}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/hints/HintsDescriptor.java#L217]
>  is {{VERSION_30}}. The current {{messagingVersion}} is {{VERSION_40}}. Which 
> causes ineffeicient hints dispatch: 
> [{{HintsDispatcher.java:128}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/hints/HintsDispatcher.java#L128]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14114) uTest failed: NettyFactoryTest.createServerChannel_UnbindableAddress()

[Jay Zhuang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522973#comment-16522973
 ] 

Jay Zhuang commented on CASSANDRA-14114:


Is there any use case that the user will use the non-local IP? The same 
question for the native port, here is a test case to set up a cluster listening 
on a non-local IP if {{ip_nonlocal_bind == 1}}: 
https://github.com/cooldoger/cassandra/commit/05eafc3e73702f22a251eb9c269b88c523245c73

Is it possible that the user uses a virtual IP on EC2, so they could re-assign 
the IP (and copy the sstables) to switch the host quickly?

> uTest failed: NettyFactoryTest.createServerChannel_UnbindableAddress()
> --
>
> Key: CASSANDRA-14114
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14114
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jay Zhuang
>Assignee: Dinesh Joshi
>Priority: Minor
>  Labels: Testing
>
> {noformat}
> [junit] Testcase: 
> createServerChannel_UnbindableAddress(org.apache.cassandra.net.async.NettyFactoryTest):
>FAILED
> [junit] Expected exception: 
> org.apache.cassandra.exceptions.ConfigurationException
> [junit] junit.framework.AssertionFailedError: Expected exception: 
> org.apache.cassandra.exceptions.ConfigurationException
> [junit]
> [junit]
> [junit] Test org.apache.cassandra.net.async.NettyFactoryTest FAILED
> {noformat}
> I'm unable to reproduce the problem on a mac or circleCI, but on some hosts 
> (Linux 4.4.38), it's able to bind IP {{1.1.1.1}}, or any other valid IP 
> (which breaks the testcase):
> {noformat}
> ...
> [junit] INFO  [main] 2017-12-13 21:20:48,470 NettyFactory.java:190 - 
> Starting Messaging Service on /1.1.1.1:9876 , encryption: disabled
> ...
> {noformat}
> Is it because a network/kernal configuration?
> +[~jasobrown]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period

[Jay Zhuang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522950#comment-16522950
 ] 

Jay Zhuang commented on CASSANDRA-6434:
---

Is there a follow-up ticket or plan to make {{only_purge_repaired_tombstones}} 
default and get ride of GCGS? As the purgeable tombstones are not repairable, 
so keeping the unrepaired tombstones (purgeable) seems not helpful.

> Repair-aware gc grace period 
> -
>
> Key: CASSANDRA-6434
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6434
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: sankalp kohli
>Assignee: Marcus Eriksson
>Priority: Major
> Fix For: 3.0 beta 1
>
>
> Since the reason for gcgs is to ensure that we don't purge tombstones until 
> every replica has been notified, it's redundant in a world where we're 
> tracking repair times per sstable (and repairing frequentily), i.e., a world 
> where we default to incremental repair a la CASSANDRA-5351.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14543) Hinted handoff to replay purgeable tombstones

[Jay Zhuang (JIRA)]

Jay Zhuang created CASSANDRA-14543:
--

 Summary: Hinted handoff to replay purgeable tombstones 
 Key: CASSANDRA-14543
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14543
 Project: Cassandra
  Issue Type: Improvement
Reporter: Jay Zhuang


Hinted-handoff currently only dispatches and applies the mutations that are 
within GCGS: 
[{{Hint.java:97}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/hints/Hint.java#L97].
 Which is to make sure it won't resurrect any deleted data.

But replaying tombstones should be safe, it could reduce the chance to have 
[un-repairable inconsistent 
data|https://lists.apache.org/thread.html/2d3d39d960143d4d2146ed2530821504ff855e832713dec7d0afd8ac@%3Cdev.cassandra.apache.org%3E].

Here is the user scenario it tries to fix:
{noformat}
1. Create a 3 nodes cluster
2. Create a table with small gc_grace_seconds (for reproducing purpose):

CREATE KEYSPACE foo WITH replication = {'class': 'SimpleStrategy',
'replication_factor': 3};
CREATE TABLE foo.bar (
id int PRIMARY KEY,
name text
) WITH gc_grace_seconds=30;

3. Insert data with consistency all:

INSERT INTO foo.bar (id, name) VALUES(1, 'cstar');

4. stop 1 node

$ ccm node2 stop

5. Delete the data with consistency quorum:

DELETE FROM foo.bar WHERE id=1;

6. Wait 30 seconds and then start node2:

$ ccm node2 start
{noformat}
Now, node2 has the data, node1/node3 have the purgeable tombstone. It triggers 
RR every time which sends data from node2 to node1/node3 but repairs nothing.
With purgeable tombstones hints handoff, it at least will dispatch the 
tombstone and delete the data on node2. It won't fix the root cause but reduce 
the chance to have this issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14543) Hinted handoff to replay purgeable tombstones

[Jay Zhuang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jay Zhuang updated CASSANDRA-14543:
---
Status: Awaiting Feedback  (was: Open)

> Hinted handoff to replay purgeable tombstones 
> --
>
> Key: CASSANDRA-14543
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14543
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jay Zhuang
>Priority: Minor
>
> Hinted-handoff currently only dispatches and applies the mutations that are 
> within GCGS: 
> [{{Hint.java:97}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/hints/Hint.java#L97].
>  Which is to make sure it won't resurrect any deleted data.
> But replaying tombstones should be safe, it could reduce the chance to have 
> [un-repairable inconsistent 
> data|https://lists.apache.org/thread.html/2d3d39d960143d4d2146ed2530821504ff855e832713dec7d0afd8ac@%3Cdev.cassandra.apache.org%3E].
> Here is the user scenario it tries to fix:
> {noformat}
> 1. Create a 3 nodes cluster
> 2. Create a table with small gc_grace_seconds (for reproducing purpose):
> CREATE KEYSPACE foo WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 3};
> CREATE TABLE foo.bar (
> id int PRIMARY KEY,
> name text
> ) WITH gc_grace_seconds=30;
> 3. Insert data with consistency all:
> INSERT INTO foo.bar (id, name) VALUES(1, 'cstar');
> 4. stop 1 node
> $ ccm node2 stop
> 5. Delete the data with consistency quorum:
> DELETE FROM foo.bar WHERE id=1;
> 6. Wait 30 seconds and then start node2:
> $ ccm node2 start
> {noformat}
> Now, node2 has the data, node1/node3 have the purgeable tombstone. It 
> triggers RR every time which sends data from node2 to node1/node3 but repairs 
> nothing.
> With purgeable tombstones hints handoff, it at least will dispatch the 
> tombstone and delete the data on node2. It won't fix the root cause but 
> reduce the chance to have this issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13669) Error when starting cassandra: Unable to make UUID from 'aa' (SASI index)

[Michael Shuler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler updated CASSANDRA-13669:
---
Fix Version/s: (was: 3.11.x)
   3.11.3

> Error when starting cassandra: Unable to make UUID from 'aa' (SASI index)
> -
>
> Key: CASSANDRA-13669
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13669
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
> Environment: Tested on:
> * macOS Sierra 10.12.5
> * Ubuntu 14.04.5 LTS
>Reporter: Lukasz Biedrycki
>Assignee: ZhaoYang
>Priority: Critical
>  Labels: sasi
> Fix For: 3.11.3
>
>
> Recently I experienced a problem that prevents me to restart cassandra.
> I narrowed it down to SASI Index when added on uuid field.
> Steps to reproduce:
> 1. start cassandra (./bin/cassandra -f)
> 2. create keyspace, table, index and add data:
> {noformat}
> CREATE KEYSPACE testkeyspace
> WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '1'} 
>AND durable_writes = true;
> use testkeyspace ;
> CREATE TABLE testtable (
>col1 uuid,
>col2 uuid,
>ts timeuuid,
>col3 uuid,
>PRIMARY KEY((col1, col2), ts) ) with clustering order by (ts desc);
> CREATE CUSTOM INDEX col3_testtable_idx ON testtable(col3)
> USING 'org.apache.cassandra.index.sasi.SASIIndex'
> WITH OPTIONS = {'analyzer_class': 
> 'org.apache.cassandra.index.sasi.analyzer.StandardAnalyzer', 'mode': 
> 'PREFIX'};
> INSERT INTO testtable(col1, col2, ts, col3)
> VALUES(898e0014-6161-11e7-b9b7-238ea83bd70b,
>898e0014-6161-11e7-b9b7-238ea83bd70b,
>now(), 898e0014-6161-11e7-b9b7-238ea83bd70b);
> {noformat}
> 3. restart cassandra
> It crashes with an error (sorry it's huge):
> {noformat}
> DEBUG 09:09:20 Writing Memtable-testtable@1005362073(0.075KiB serialized 
> bytes, 1 ops, 0%/0% of on/off-heap limit), flushed range = 
> (min(-9223372036854775808), max(9223372036854775807)]
> ERROR 09:09:20 Exception in thread 
> Thread[PerDiskMemtableFlushWriter_0:1,5,main]
> org.apache.cassandra.serializers.MarshalException: Unable to make UUID from 
> 'aa'
>   at 
> org.apache.cassandra.db.marshal.UUIDType.fromString(UUIDType.java:118) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.index.sasi.analyzer.StandardAnalyzer.hasNext(StandardAnalyzer.java:168)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter$Index.add(PerSSTableIndexWriter.java:208)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter.lambda$nextUnfilteredCluster$0(PerSSTableIndexWriter.java:132)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at java.util.Collections$SingletonSet.forEach(Collections.java:4767) 
> ~[na:1.8.0_131]
>   at 
> org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter.nextUnfilteredCluster(PerSSTableIndexWriter.java:119)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.ColumnIndex.lambda$add$1(ColumnIndex.java:233) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_131]
>   at org.apache.cassandra.db.ColumnIndex.add(ColumnIndex.java:233) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.ColumnIndex.buildRowIndex(ColumnIndex.java:107) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:169)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:48)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:458)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.call(Memtable.java:493) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.call(Memtable.java:380) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_131]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_131]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_131]
>   at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
> Exception (java.lang.RuntimeException) encountered during startup: 
> java.util.concurrent.ExecutionException: java.lang.RuntimeException: 
> java.util.concurrent.ExecutionException: 
> org.apache.cassandra.serializers.MarshalException: Unable to make UUID from 
> 'aa'
> java.lang.RuntimeException: 

[jira] [Updated] (CASSANDRA-14538) Add virtual table to view cache information

[Chris Lohfink (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Lohfink updated CASSANDRA-14538:
--
Description: 
 Add a table for displaying the general cache information (like in {{nodetool 
info}})

{code}
cqlsh> SELECT * FROM system_views.caches;

 name| entries | hit_ratio | hits | recent_hits_per_sec | 
recent_requests_per_sec | requests | size_max | size_used
-+-+---+--+-+-+--+--+---
 counter |   0 |   NaN |0 |   0 |   
0 |0 | 12582912 | 0
 key |  19 |  0.922509 |  250 |  11 |   
   13 |  271 | 25165824 |  1728
 row |   0 |   NaN |0 |   0 |   
0 |0 | 16777216 | 0

(3 rows)
{code}

  was:
 Add a table for displaying the general cache information (like in {{nodetool 
info}})

{code}
cqlsh> SELECT * FROM system_views.caches;

 name| entries | hit_ratio | hits | recent_hits_per_sec | 
recent_requests_per_sec | requests | size_max | size_used
-+-+---+--+-+-+--+--+---
 counter |   0 |   NaN |0 |   0 |   
0 |0 | 12582912 | 0
 key |  19 |  0.922509 |  250 |  11 |   
   13 |  271 | 25165824 |  1728
 row |   0 |   NaN |0 |   0 |   
0 |0 | 16777216 | 0

(3 rows)


> Add virtual table to view cache information
> ---
>
> Key: CASSANDRA-14538
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14538
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
>  Labels: virtual-tables
>
>  Add a table for displaying the general cache information (like in {{nodetool 
> info}})
> {code}
> cqlsh> SELECT * FROM system_views.caches;
>  name| entries | hit_ratio | hits | recent_hits_per_sec | 
> recent_requests_per_sec | requests | size_max | size_used
> -+-+---+--+-+-+--+--+---
>  counter |   0 |   NaN |0 |   0 | 
>   0 |0 | 12582912 | 0
>  key |  19 |  0.922509 |  250 |  11 | 
>  13 |  271 | 25165824 |  1728
>  row |   0 |   NaN |0 |   0 | 
>   0 |0 | 16777216 | 0
> (3 rows)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14538) Add virtual table to view cache information

[Chris Lohfink (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522565#comment-16522565
 ] 

Chris Lohfink commented on CASSANDRA-14538:
---

agreed, removed that table

> Add virtual table to view cache information
> ---
>
> Key: CASSANDRA-14538
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14538
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
>  Labels: virtual-tables
>
>  Add a table for displaying the general cache information (like in {{nodetool 
> info}})
> {code}
> cqlsh> SELECT * FROM system_views.caches;
>  name| entries | hit_ratio | hits | recent_hits_per_sec | 
> recent_requests_per_sec | requests | size_max | size_used
> -+-+---+--+-+-+--+--+---
>  counter |   0 |   NaN |0 |   0 | 
>   0 |0 | 12582912 | 0
>  key |  19 |  0.922509 |  250 |  11 | 
>  13 |  271 | 25165824 |  1728
>  row |   0 |   NaN |0 |   0 | 
>   0 |0 | 16777216 | 0
> (3 rows)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14538) Add virtual table to view cache information

[Chris Lohfink (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Lohfink updated CASSANDRA-14538:
--
Description: 
 Add a table for displaying the general cache information (like in {{nodetool 
info}})

{code}
cqlsh> SELECT * FROM system_views.caches;

 name| entries | hit_ratio | hits | recent_hits_per_sec | 
recent_requests_per_sec | requests | size_max | size_used
-+-+---+--+-+-+--+--+---
 counter |   0 |   NaN |0 |   0 |   
0 |0 | 12582912 | 0
 key |  19 |  0.922509 |  250 |  11 |   
   13 |  271 | 25165824 |  1728
 row |   0 |   NaN |0 |   0 |   
0 |0 | 16777216 | 0

(3 rows)

  was:
 Add a couple tables, one for displaying the general cache information (like in 
{{nodetool info}}) and another to view the contents (ish) of keycache. This 
would be pretty useful in tests to tell if things are in the cache and in 
finding wide partitions and performance issues.

Possibility to discuss: DELETE on the key cache to evict specific records, I 
wrote the code to do it but im not sure if its valuable or worth complexity. 
TRUNCATE support on virtual tables might be nice as a way to clear the keycache 
as well. These can be followup tickets though if it seems like a good idea as 
truncate and delete are currently disabled in virtual tables and that behavior 
should maybe be its own ticket/discussion.
 
{code}
cqlsh> SELECT * FROM system_views.caches;

 name| entries | hit_ratio | hits | recent_hits_per_sec | 
recent_requests_per_sec | requests | size_max | size_used
-+-+---+--+-+-+--+--+---
 counter |   0 |   NaN |0 |   0 |   
0 |0 | 12582912 | 0
 key |  19 |  0.922509 |  250 |  11 |   
   13 |  271 | 25165824 |  1728
 row |   0 |   NaN |0 |   0 |   
0 |0 | 16777216 | 0

(3 rows)
cqlsh> SELECT * FROM system_views.key_cache;

 keyspace_name | table_name   | key | sstable | size
---+--+-+-+--
 basic | wide |row1 |   5 |   64
 basic | wide |row1 |   6 |   64
system |local |   local |  15 |   24
system |local |   local |  16 |   24
system |local |   local |  17 |   24
system | sstable_activity | system_auth:roles:1 |  13 |   24
   system_auth |roles |   cassandra |   1 |   24
 system_schema |   tables |   basic |  13 |   24
 system_schema |   tables | system_auth |  13 |   24
 system_schema |   tables |  system_distributed |  13 |   24
 system_schema |   tables |   system_traces |  13 |   24
 system_schema |  columns |   basic |  13 |   24
 system_schema |  columns | system_auth |  13 |   24
 system_schema |  columns |  system_distributed |  13 |   24
 system_schema |  columns |   system_traces |  13 |   24
 system_schema |keyspaces |   basic |  13 |   24
 system_schema |keyspaces | system_auth |  13 |   24
 system_schema |keyspaces |  system_distributed |  13 |   24
 system_schema |keyspaces |   system_traces |  13 |   24

(19 rows)
{code}


> Add virtual table to view cache information
> ---
>
> Key: CASSANDRA-14538
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14538
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
>  Labels: virtual-tables
>
>  Add a table for displaying the general cache information (like in {{nodetool 
> info}})
> {code}
> cqlsh> SELECT * FROM system_views.caches;
>  name| entries | hit_ratio | hits | recent_hits_per_sec | 
> recent_requests_per_sec | requests | size_max | size_used
> -+-+---+--+-+-+--+--+---
>  counter |   0 |   NaN |0 |   0 | 
>   0 |0 | 12582912 | 0
>  key |  19 |  0.922509 |  250 |  11 | 

[jira] [Commented] (CASSANDRA-14538) Add virtual table to view cache information

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522474#comment-16522474
 ] 

Aleksey Yeschenko commented on CASSANDRA-14538:
---

I think this might be too big to materialize fully in memory, and probably 
shouldn't be using {{SimpleDataSet}}.

> Add virtual table to view cache information
> ---
>
> Key: CASSANDRA-14538
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14538
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
>  Labels: virtual-tables
>
>  Add a couple tables, one for displaying the general cache information (like 
> in {{nodetool info}}) and another to view the contents (ish) of keycache. 
> This would be pretty useful in tests to tell if things are in the cache and 
> in finding wide partitions and performance issues.
> Possibility to discuss: DELETE on the key cache to evict specific records, I 
> wrote the code to do it but im not sure if its valuable or worth complexity. 
> TRUNCATE support on virtual tables might be nice as a way to clear the 
> keycache as well. These can be followup tickets though if it seems like a 
> good idea as truncate and delete are currently disabled in virtual tables and 
> that behavior should maybe be its own ticket/discussion.
>  
> {code}
> cqlsh> SELECT * FROM system_views.caches;
>  name| entries | hit_ratio | hits | recent_hits_per_sec | 
> recent_requests_per_sec | requests | size_max | size_used
> -+-+---+--+-+-+--+--+---
>  counter |   0 |   NaN |0 |   0 | 
>   0 |0 | 12582912 | 0
>  key |  19 |  0.922509 |  250 |  11 | 
>  13 |  271 | 25165824 |  1728
>  row |   0 |   NaN |0 |   0 | 
>   0 |0 | 16777216 | 0
> (3 rows)
> cqlsh> SELECT * FROM system_views.key_cache;
>  keyspace_name | table_name   | key | sstable | size
> ---+--+-+-+--
>  basic | wide |row1 |   5 |   64
>  basic | wide |row1 |   6 |   64
> system |local |   local |  15 |   24
> system |local |   local |  16 |   24
> system |local |   local |  17 |   24
> system | sstable_activity | system_auth:roles:1 |  13 |   24
>system_auth |roles |   cassandra |   1 |   24
>  system_schema |   tables |   basic |  13 |   24
>  system_schema |   tables | system_auth |  13 |   24
>  system_schema |   tables |  system_distributed |  13 |   24
>  system_schema |   tables |   system_traces |  13 |   24
>  system_schema |  columns |   basic |  13 |   24
>  system_schema |  columns | system_auth |  13 |   24
>  system_schema |  columns |  system_distributed |  13 |   24
>  system_schema |  columns |   system_traces |  13 |   24
>  system_schema |keyspaces |   basic |  13 |   24
>  system_schema |keyspaces | system_auth |  13 |   24
>  system_schema |keyspaces |  system_distributed |  13 |   24
>  system_schema |keyspaces |   system_traces |  13 |   24
> (19 rows)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14444) Got NPE when querying Cassandra 3.11.2

[Xiaodong Xie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522393#comment-16522393
 ] 

Xiaodong Xie commented on CASSANDRA-1:
--

Thanks for the comments, [~eperott]. (y)

We've already successfully upgraded our cluster (all 6 nodes) to 3.11.2 (from 
2.2.6 as mentioned) with the patch. And we are running 3.11.2 plus the patch 
since then for about 1 month, everything was fine. 

 

> Got NPE when querying Cassandra 3.11.2
> --
>
> Key: CASSANDRA-1
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Ubuntu 14.04, JDK 1.8.0_171. 
> Cassandra 3.11.2
>Reporter: Xiaodong Xie
>Priority: Blocker
>
> We just upgraded our Cassandra cluster from 2.2.6 to 3.11.2
> After upgrading, we immediately got exceptions in Cassandra like this one: 
>  
> {code}
> ERROR [Native-Transport-Requests-1] 2018-05-11 17:10:21,994 
> QueryMessage.java:129 - Unexpected error during query
> java.lang.NullPointerException: null
> at 
> org.apache.cassandra.dht.RandomPartitioner.getToken(RandomPartitioner.java:248)
>  ~[apache-cassandra-3.11.2.jar:3.11.2]
> at 
> org.apache.cassandra.dht.RandomPartitioner.decorateKey(RandomPartitioner.java:92)
>  ~[apache-cassandra-3.11.2.jar:3.11.2]
> at org.apache.cassandra.config.CFMetaData.decorateKey(CFMetaData.java:666) 
> ~[apache-cassandra-3.11.2.jar:3.11.2]
> at 
> org.apache.cassandra.service.pager.PartitionRangeQueryPager.(PartitionRangeQueryPager.java:44)
>  ~[apache-cassandra-3.11.2.jar:3.11.2]
> at 
> org.apache.cassandra.db.PartitionRangeReadCommand.getPager(PartitionRangeReadCommand.java:268)
>  ~[apache-cassandra-3.11.2.jar:3.11.2]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.getPager(SelectStatement.java:475)
>  ~[apache-cassandra-3.11.2.jar:3.11.2]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:288)
>  ~[apache-cassandra-3.11.2.jar:3.11.2]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:118)
>  ~[apache-cassandra-3.11.2.jar:3.11.2]
> at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:224)
>  ~[apache-cassandra-3.11.2.jar:3.11.2]
> at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:255) 
> ~[apache-cassandra-3.11.2.jar:3.11.2]
> at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:240) 
> ~[apache-cassandra-3.11.2.jar:3.11.2]
> at 
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:116)
>  ~[apache-cassandra-3.11.2.jar:3.11.2]
> at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:517)
>  [apache-cassandra-3.11.2.jar:3.11.2]
> at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:410)
>  [apache-cassandra-3.11.2.jar:3.11.2]
> at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)
>  [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:35)
>  [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:348)
>  [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_171]
> at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
>  [apache-cassandra-3.11.2.jar:3.11.2]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109) 
> [apache-cassandra-3.11.2.jar:3.11.2]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_171]
> {code}
>  
> The table schema is like:
> {code}
> CREATE TABLE example.example_table (
>  id bigint,
>  hash text,
>  json text,
>  PRIMARY KEY (id, hash)
> ) WITH COMPACT STORAGE
> {code}
>  
> The query is something like:
> {code}
> "select * from example.example_table;" // (We do know this is bad practise, 
> and we are trying to fix that right now)
> {code}
> with fetch-size as 200, using DataStax Java driver. 
> This table contains about 20k rows. 
>  
> Actually, the fix is quite simple, 
>  
> {code}
> --- a/src/java/org/apache/cassandra/service/pager/PagingState.java
> +++ b/src/java/org/apache/cassandra/service/pager/PagingState.java
> @@ -46,7 +46,7 @@ public class PagingState
> public PagingState(ByteBuffer partitionKey, RowMark rowMark, int remaining, 
> int remainingInPartition)
>  {
> - this.partitionKey = partitionKey;
> + this.partitionKey = 

[jira] [Commented] (CASSANDRA-14444) Got NPE when querying Cassandra 3.11.2

[JIRA]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522314#comment-16522314
 ] 

Per Otterström commented on CASSANDRA-1:


I'm able to reproduce this but only when I run mixed versions (2.2.11 and 
3.11.2 in my case).

I believe this is a duplicate of CASSANDRA-10880. In short, even though 2.2 and 
3.11 both support protocol v4, the two Cassandra versions represent the paging 
state differently. The result is this rather ugly NPE in the server. The 
proposed solution is to force client to use protocol v3 while upgrading.

If you need the features of v4, then another solution (that we've been using 
when upgrading to 3.0) is to create your own load balancing policy and a custom 
StatementWrapper on the client side to make paging requests "sticky" in terms 
of node-versions.

More details regarding the issue 
[here|https://www.mail-archive.com/user@cassandra.apache.org/msg45381.html].

I have not reviewed or tried your patch yet. The NPE is less than elegant, so 
this could be an improvement of error handling.

Did you verify your patch in a cluster with mixed versions?

> Got NPE when querying Cassandra 3.11.2
> --
>
> Key: CASSANDRA-1
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Ubuntu 14.04, JDK 1.8.0_171. 
> Cassandra 3.11.2
>Reporter: Xiaodong Xie
>Priority: Blocker
>
> We just upgraded our Cassandra cluster from 2.2.6 to 3.11.2
> After upgrading, we immediately got exceptions in Cassandra like this one: 
>  
> {code}
> ERROR [Native-Transport-Requests-1] 2018-05-11 17:10:21,994 
> QueryMessage.java:129 - Unexpected error during query
> java.lang.NullPointerException: null
> at 
> org.apache.cassandra.dht.RandomPartitioner.getToken(RandomPartitioner.java:248)
>  ~[apache-cassandra-3.11.2.jar:3.11.2]
> at 
> org.apache.cassandra.dht.RandomPartitioner.decorateKey(RandomPartitioner.java:92)
>  ~[apache-cassandra-3.11.2.jar:3.11.2]
> at org.apache.cassandra.config.CFMetaData.decorateKey(CFMetaData.java:666) 
> ~[apache-cassandra-3.11.2.jar:3.11.2]
> at 
> org.apache.cassandra.service.pager.PartitionRangeQueryPager.(PartitionRangeQueryPager.java:44)
>  ~[apache-cassandra-3.11.2.jar:3.11.2]
> at 
> org.apache.cassandra.db.PartitionRangeReadCommand.getPager(PartitionRangeReadCommand.java:268)
>  ~[apache-cassandra-3.11.2.jar:3.11.2]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.getPager(SelectStatement.java:475)
>  ~[apache-cassandra-3.11.2.jar:3.11.2]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:288)
>  ~[apache-cassandra-3.11.2.jar:3.11.2]
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:118)
>  ~[apache-cassandra-3.11.2.jar:3.11.2]
> at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:224)
>  ~[apache-cassandra-3.11.2.jar:3.11.2]
> at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:255) 
> ~[apache-cassandra-3.11.2.jar:3.11.2]
> at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:240) 
> ~[apache-cassandra-3.11.2.jar:3.11.2]
> at 
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:116)
>  ~[apache-cassandra-3.11.2.jar:3.11.2]
> at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:517)
>  [apache-cassandra-3.11.2.jar:3.11.2]
> at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:410)
>  [apache-cassandra-3.11.2.jar:3.11.2]
> at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)
>  [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:35)
>  [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at 
> io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:348)
>  [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_171]
> at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
>  [apache-cassandra-3.11.2.jar:3.11.2]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109) 
> [apache-cassandra-3.11.2.jar:3.11.2]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_171]
> {code}
>  
> The table schema is like:
> {code}
> CREATE TABLE example.example_table (
>  id bigint,
>  hash text,
>  json text,
>  PRIMARY KEY (id, hash)
> ) 

[jira] [Updated] (CASSANDRA-14485) Optimize internode messaging protocol

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-14485:

   Resolution: Fixed
Fix Version/s: (was: 4.0.x)
   4.0
   Status: Resolved  (was: Ready to Commit)

committed as sha {{06209037ea56b5a2a49615a99f1542d6ea1b2947}}

> Optimize internode messaging protocol
> -
>
> Key: CASSANDRA-14485
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14485
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Streaming and Messaging
>Reporter: Jason Brown
>Assignee: Jason Brown
>Priority: Major
> Fix For: 4.0
>
>
> There's some dead wood and places for optimization in the internode messaging 
> protocol. Currently, we include the sender's \{{IPAddressAndPort}} in *every* 
> internode message, even though we already sent that in the handshake that 
> established the connection/session. Further, there are several places where 
> we can use vints instead of a fixed, 4-byte integer value- especially as 
> those values will almost always be less than one byte.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14540) Internode messaging handshake sends wrong messaging version number

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-14540:

   Resolution: Fixed
Fix Version/s: (was: 4.x)
   4.0
   Status: Resolved  (was: Patch Available)

committed as sha {{5db822b71ad7278ca6443455d029dd79e22388d8}}.

> Internode messaging handshake sends wrong messaging version number
> --
>
> Key: CASSANDRA-14540
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14540
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
>Reporter: Jason Brown
>Assignee: Jason Brown
>Priority: Blocker
> Fix For: 4.0
>
>
> With the refactor of internode messaging to netty in 4.0, we abstracted the 
> protocol handshakes messages into a class and handlers. There is a bug when 
> the initiator of the connection sends, in the third message of the handshake, 
> it's own default protocol version number 
> ({{MessagingService.current_version}}), rather than the negotiated version. 
> This was not causing any obvious problems when CASSANDRA-8457 was initially 
> committed, but the bug is exposed after CASSANDRA-7544. The problem is during 
> rolling upgrades of 3.0/3.X to 4.0, nodes cannot correctly connect. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

cassandra git commit: Optimize internode messaging protocol

[jasobrown]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 5db822b71 -> 06209037e


Optimize internode messaging protocol

patch by jasobrown; reviewed by Dinesh Joshi for CASSANDRA-14485


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/06209037
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/06209037
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/06209037

Branch: refs/heads/trunk
Commit: 06209037ea56b5a2a49615a99f1542d6ea1b2947
Parents: 5db822b
Author: Jason Brown 
Authored: Tue May 29 19:21:10 2018 -0700
Committer: Jason Brown 
Committed: Mon Jun 25 06:41:30 2018 -0700

--
 CHANGES.txt |   1 +
 .../org/apache/cassandra/net/MessageOut.java| 172 ++
 .../net/async/BaseMessageInHandler.java | 148 
 .../net/async/InboundHandshakeHandler.java  |   6 +-
 .../cassandra/net/async/MessageInHandler.java   | 220 +++---
 .../net/async/MessageInHandlerPre40.java| 231 +++
 .../apache/cassandra/utils/vint/VIntCoding.java |  37 +++
 .../test/microbench/MessageOutBench.java| 129 +++
 .../net/async/HandshakeHandlersTest.java|  31 ++-
 .../net/async/MessageInHandlerTest.java | 105 ++---
 .../cassandra/utils/vint/VIntCodingTest.java|  15 ++
 11 files changed, 832 insertions(+), 263 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/06209037/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index fb14e40..e99c9ea 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0
+ * Optimize internode messaging protocol (CASSANDRA-14485)
  * Internode messaging handshake sends wrong messaging version number 
(CASSANDRA-14540)
  * Add a virtual table to expose active client connections (CASSANDRA-14458)
  * Clean up and refactor client metrics (CASSANDRA-14524)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/06209037/src/java/org/apache/cassandra/net/MessageOut.java
--
diff --git a/src/java/org/apache/cassandra/net/MessageOut.java 
b/src/java/org/apache/cassandra/net/MessageOut.java
index 30968df..834435e 100644
--- a/src/java/org/apache/cassandra/net/MessageOut.java
+++ b/src/java/org/apache/cassandra/net/MessageOut.java
@@ -18,6 +18,7 @@
 
 package org.apache.cassandra.net;
 
+import java.io.IOError;
 import java.io.IOException;
 import java.util.ArrayList;
 import java.util.List;
@@ -29,47 +30,43 @@ import com.google.common.primitives.Ints;
 import org.apache.cassandra.concurrent.Stage;
 import org.apache.cassandra.db.TypeSizes;
 import org.apache.cassandra.io.IVersionedSerializer;
+import org.apache.cassandra.io.util.DataOutputBuffer;
 import org.apache.cassandra.io.util.DataOutputPlus;
 import org.apache.cassandra.locator.InetAddressAndPort;
 import 
org.apache.cassandra.net.async.OutboundConnectionIdentifier.ConnectionType;
 import org.apache.cassandra.tracing.Tracing;
 import org.apache.cassandra.utils.FBUtilities;
-import org.apache.cassandra.utils.Pair;
+import org.apache.cassandra.utils.vint.VIntCoding;
 
 import static org.apache.cassandra.tracing.Tracing.isTracing;
 
 /**
  * Each message contains a header with several fixed fields, an optional 
key-value parameters section, and then
- * the message payload itself. Note: the IP address in the header may be 
either IPv4 (4 bytes) or IPv6 (16 bytes).
- * The diagram below shows the IPv4 address for brevity.
+ * the message payload itself. Note: the legacy IP address (pre-4.0) in the 
header may be either IPv4 (4 bytes)
+ * or IPv6 (16 bytes). The diagram below shows the IPv4 address for brevity. 
In pre-4.0, the payloadSize was
+ * encoded as a 4-byte integer; in 4.0 and up it is an unsigned byte (255 
parameters should be enough for anyone).
  *
  * 
  * {@code
- *1 1 1 1 1 2 2 2 2 2 3 3 3 3 3 4 4 4 4 4 5 5 5 5 5 6 6
- *  0 2 4 6 8 0 2 4 6 8 0 2 4 6 8 0 2 4 6 8 0 2 4 6 8 0 2 4 6 8 0 2
- * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
- * |   PROTOCOL MAGIC  |
- * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
- * | Message ID|
- * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
- * | Timestamp |
- * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
- * |  Addr len |   IP Address (IPv4)   /
- * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
- * /   | Verb  /
- * 

cassandra git commit: Internode messaging handshake sends wrong messaging version number

[jasobrown]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 2bad5d5b6 -> 5db822b71


Internode messaging handshake sends wrong messaging version number

patch by jasobrown; reviewed by Dinesh Joshi for CASSANDRA-14540


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5db822b7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5db822b7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5db822b7

Branch: refs/heads/trunk
Commit: 5db822b71ad7278ca6443455d029dd79e22388d8
Parents: 2bad5d5
Author: Jason Brown 
Authored: Fri Jun 22 13:56:17 2018 -0700
Committer: Jason Brown 
Committed: Mon Jun 25 06:36:59 2018 -0700

--
 CHANGES.txt |  1 +
 .../net/async/OutboundHandshakeHandler.java |  2 +-
 .../net/async/OutboundHandshakeHandlerTest.java | 59 ++--
 3 files changed, 55 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5db822b7/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 047689e..fb14e40 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0
+ * Internode messaging handshake sends wrong messaging version number 
(CASSANDRA-14540)
  * Add a virtual table to expose active client connections (CASSANDRA-14458)
  * Clean up and refactor client metrics (CASSANDRA-14524)
  * Nodetool import row cache invalidation races with adding sstables to 
tracker (CASSANDRA-14529)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5db822b7/src/java/org/apache/cassandra/net/async/OutboundHandshakeHandler.java
--
diff --git 
a/src/java/org/apache/cassandra/net/async/OutboundHandshakeHandler.java 
b/src/java/org/apache/cassandra/net/async/OutboundHandshakeHandler.java
index c555bed..3ccbf49 100644
--- a/src/java/org/apache/cassandra/net/async/OutboundHandshakeHandler.java
+++ b/src/java/org/apache/cassandra/net/async/OutboundHandshakeHandler.java
@@ -177,7 +177,7 @@ public class OutboundHandshakeHandler extends 
ByteToMessageDecoder
 
 try
 {
-ctx.writeAndFlush(new 
ThirdHandshakeMessage(MessagingService.current_version, 
connectionId.local()).encode(ctx.alloc()));
+ctx.writeAndFlush(new ThirdHandshakeMessage(peerMessagingVersion, 
connectionId.local()).encode(ctx.alloc()));
 ChannelWriter channelWriter = setupPipeline(ctx.channel(), 
peerMessagingVersion);
 callback.accept(HandshakeResult.success(channelWriter, 
peerMessagingVersion));
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5db822b7/test/unit/org/apache/cassandra/net/async/OutboundHandshakeHandlerTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/net/async/OutboundHandshakeHandlerTest.java 
b/test/unit/org/apache/cassandra/net/async/OutboundHandshakeHandlerTest.java
index be71fd4..2d377af 100644
--- a/test/unit/org/apache/cassandra/net/async/OutboundHandshakeHandlerTest.java
+++ b/test/unit/org/apache/cassandra/net/async/OutboundHandshakeHandlerTest.java
@@ -18,7 +18,6 @@
 
 package org.apache.cassandra.net.async;
 
-import java.net.InetSocketAddress;
 import java.util.LinkedList;
 import java.util.List;
 import java.util.Optional;
@@ -42,6 +41,7 @@ import org.apache.cassandra.config.DatabaseDescriptor;
 import org.apache.cassandra.locator.InetAddressAndPort;
 import org.apache.cassandra.net.MessagingService;
 import org.apache.cassandra.net.async.HandshakeProtocol.SecondHandshakeMessage;
+import org.apache.cassandra.net.async.HandshakeProtocol.ThirdHandshakeMessage;
 import org.apache.cassandra.net.async.OutboundHandshakeHandler.HandshakeResult;
 
 import static 
org.apache.cassandra.net.async.OutboundHandshakeHandler.HandshakeResult.UNKNOWN_PROTOCOL_VERSION;
@@ -102,20 +102,31 @@ public class OutboundHandshakeHandlerTest
 }
 
 @Test
-public void decode_HappyPath() throws Exception
+public void decode_HappyPath()
 {
 buf = new 
SecondHandshakeMessage(MESSAGING_VERSION).encode(PooledByteBufAllocator.DEFAULT);
 channel.writeInbound(buf);
 Assert.assertEquals(1, channel.outboundMessages().size());
 Assert.assertTrue(channel.isOpen());
-Assert.assertTrue(channel.releaseOutbound()); // throw away any 
responses from decode()
 
 Assert.assertEquals(MESSAGING_VERSION, 
callbackHandler.result.negotiatedMessagingVersion);
 Assert.assertEquals(HandshakeResult.Outcome.SUCCESS, 
callbackHandler.result.outcome);
+Assert.assertFalse(channel.outboundMessages().isEmpty());
+
+ByteBuf thridMsgBuf = (ByteBuf) channel.outboundMessages().poll();
+try
+   

[jira] [Resolved] (CASSANDRA-14470) Repair validation failed/unable to create merkle tree

[Kurt Greaves (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kurt Greaves resolved CASSANDRA-14470.
--
Resolution: Duplicate

> Repair validation failed/unable to create merkle tree
> -
>
> Key: CASSANDRA-14470
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14470
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Harry Hough
>Priority: Major
>
> I had trouble repairing with a full repair across all nodes and keyspaces so 
> I swapped to doing table by table. This table will not repair even after 
> scrub/restart of all nodes. I am using command:
> {code:java}
> nodetool repair -full -seq keyspace table
> {code}
> {code:java}
> [2018-05-25 19:26:36,525] Repair session 0198ee50-6050-11e8-a3b7-9d0793eab507 
> for range [(165598500763544933,166800441975877433], 
> (-5455068259072262254,-5445777107512274819], 
> (-4614366950466274594,-4609359222424798148], 
> (3417371506258365094,3421921915575816226], 
> (5221788898381458942,5222846663270250559], 
> (3421921915575816226,3429175540277204991], 
> (3276484330153091115,3282213186258578546], 
> (-3306169730424140596,-3303439264231406101], 
> (5228704360821395206,5242415853745535023], 
> (5808045095951939338,5808562658315740708], 
> (-3303439264231406101,-3302592736123212969]] finished (progress: 1%)
> [2018-05-25 19:27:23,848] Repair session 0180f980-6050-11e8-a3b7-9d0793eab507 
> for range [(-8495158945319933291,-8482949618583319581], 
> (1803296697741516342,1805330812863783941], 
> (8633191319643427141,8637771071728131257], 
> (2214097236323810344,2218253238829661319], 
> (8637771071728131257,8639627594735133685], 
> (2195525904029414718,2214097236323810344], 
> (-8500127431270773970,-8495158945319933291], 
> (7151693083782264341,7152162989417914407], 
> (-8482949618583319581,-8481973749935314249]] finished (progress: 1%)
> [2018-05-25 19:30:32,590] Repair session 01ac9d62-6050-11e8-a3b7-9d0793eab507 
> for range [(7887346492105510731,7893062759268864220], 
> (-153277717939330979,-151986584968539220], 
> (-6351665356961460262,-6336288442758847669], 
> (7881942012672602731,7887346492105510731], 
> (-5884528383037906783,-5878097817437987368], 
> (6054625594262089428,6060773114960761336], 
> (-6354401100436622515,-6351665356961460262], 
> (3358411934943460772,336336663817876], 
> (6255644242745576360,6278718135193665575], 
> (-6321106762570843270,-6316788220143151823], 
> (1754319239259058661,1759314644652031521], 
> (7893062759268864220,7894890594190784729], 
> (-8012293411840276426,-8011781808288431224]] failed with error [repair 
> #01ac9d62-6050-11e8-a3b7-9d0793eab507 on keyspace/table, 
> [(7887346492105510731,7893062759268864220], 
> (-153277717939330979,-151986584968539220], 
> (-6351665356961460262,-6336288442758847669], 
> (7881942012672602731,7887346492105510731],
> (-5884528383037906783,-5878097817437987368], 
> (6054625594262089428,6060773114960761336], 
> (-6354401100436622515,-6351665356961460262], 
> (3358411934943460772,336336663817876], 
> (6255644242745576360,6278718135193665575], 
> (-6321106762570843270,-6316788220143151823], 
> (1754319239259058661,1759314644652031521], 
> (7893062759268864220,7894890594190784729], 
> (-8012293411840276426,-8011781808288431224]]] Validation failed in 
> /192.168.8.64 (progress: 1%)
> [2018-05-25 19:30:38,744] Repair session 01ab16c1-6050-11e8-a3b7-9d0793eab507 
> for range [(4474598255414218354,4477186372547790770], 
> (-8368931070988054567,-8367389908801757978], 
> (4445104759712094068,4445123832517144036], 
> (6749641233379918040,6749879473217708908], 
> (717627050679001698,729408043324000761], 
> (8984622403893999385,8990662643404904110], 
> (4457612694557846994,4474598255414218354], 
> (5589049422573545528,5593079877787783784], 
> (3609693317839644945,3613727999875360405], 
> (8499016262183246473,8504603366117127178], 
> (-5421277973540712245,-5417725796037372830], 
> (5586405751301680690,5589049422573545528], 
> (-2611069890590917549,-2603911539353128123], 
> (2424772330724108233,2427564448454334730], 
> (3172651438220766183,3175226710613527829], 
> (4445123832517144036,4457612694557846994], 
> (-6827531712183440570,-6800863837312326365], 
> (5593079877787783784,5596020904874304252], 
> (716705770783505310,717627050679001698], 
> (115377252345874298,119626359210683992], 
> (239394377432130766,240250561347730054]] failed with error [repair 
> #01ab16c1-6050-11e8-a3b7-9d0793eab507 on keyspace/table, 
> [(4474598255414218354,4477186372547790770], 
> (-8368931070988054567,-8367389908801757978], 
> (4445104759712094068,4445123832517144036], 
> (6749641233379918040,6749879473217708908], 
> (717627050679001698,729408043324000761], 
> (8984622403893999385,8990662643404904110], 
> (4457612694557846994,4474598255414218354], 
> 

[jira] [Commented] (CASSANDRA-13669) Error when starting cassandra: Unable to make UUID from 'aa' (SASI index)

[JIRA]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522155#comment-16522155
 ] 

Andrés de la Peña commented on CASSANDRA-13669:
---

Committed to 3.11 as 
[ea62d8862c311e3d9b64d622bea0a68d3825aa7d|https://github.com/apache/cassandra/commit/ea62d8862c311e3d9b64d622bea0a68d3825aa7d]
 and merged to trunk.

> Error when starting cassandra: Unable to make UUID from 'aa' (SASI index)
> -
>
> Key: CASSANDRA-13669
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13669
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
> Environment: Tested on:
> * macOS Sierra 10.12.5
> * Ubuntu 14.04.5 LTS
>Reporter: Lukasz Biedrycki
>Assignee: ZhaoYang
>Priority: Critical
>  Labels: sasi
> Fix For: 3.11.x
>
>
> Recently I experienced a problem that prevents me to restart cassandra.
> I narrowed it down to SASI Index when added on uuid field.
> Steps to reproduce:
> 1. start cassandra (./bin/cassandra -f)
> 2. create keyspace, table, index and add data:
> {noformat}
> CREATE KEYSPACE testkeyspace
> WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '1'} 
>AND durable_writes = true;
> use testkeyspace ;
> CREATE TABLE testtable (
>col1 uuid,
>col2 uuid,
>ts timeuuid,
>col3 uuid,
>PRIMARY KEY((col1, col2), ts) ) with clustering order by (ts desc);
> CREATE CUSTOM INDEX col3_testtable_idx ON testtable(col3)
> USING 'org.apache.cassandra.index.sasi.SASIIndex'
> WITH OPTIONS = {'analyzer_class': 
> 'org.apache.cassandra.index.sasi.analyzer.StandardAnalyzer', 'mode': 
> 'PREFIX'};
> INSERT INTO testtable(col1, col2, ts, col3)
> VALUES(898e0014-6161-11e7-b9b7-238ea83bd70b,
>898e0014-6161-11e7-b9b7-238ea83bd70b,
>now(), 898e0014-6161-11e7-b9b7-238ea83bd70b);
> {noformat}
> 3. restart cassandra
> It crashes with an error (sorry it's huge):
> {noformat}
> DEBUG 09:09:20 Writing Memtable-testtable@1005362073(0.075KiB serialized 
> bytes, 1 ops, 0%/0% of on/off-heap limit), flushed range = 
> (min(-9223372036854775808), max(9223372036854775807)]
> ERROR 09:09:20 Exception in thread 
> Thread[PerDiskMemtableFlushWriter_0:1,5,main]
> org.apache.cassandra.serializers.MarshalException: Unable to make UUID from 
> 'aa'
>   at 
> org.apache.cassandra.db.marshal.UUIDType.fromString(UUIDType.java:118) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.index.sasi.analyzer.StandardAnalyzer.hasNext(StandardAnalyzer.java:168)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter$Index.add(PerSSTableIndexWriter.java:208)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter.lambda$nextUnfilteredCluster$0(PerSSTableIndexWriter.java:132)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at java.util.Collections$SingletonSet.forEach(Collections.java:4767) 
> ~[na:1.8.0_131]
>   at 
> org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter.nextUnfilteredCluster(PerSSTableIndexWriter.java:119)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.ColumnIndex.lambda$add$1(ColumnIndex.java:233) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_131]
>   at org.apache.cassandra.db.ColumnIndex.add(ColumnIndex.java:233) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.ColumnIndex.buildRowIndex(ColumnIndex.java:107) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:169)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:48)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:458)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.call(Memtable.java:493) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.call(Memtable.java:380) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_131]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_131]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_131]
>   at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
> Exception (java.lang.RuntimeException) encountered during startup: 
> java.util.concurrent.ExecutionException: java.lang.RuntimeException: 
> 

[jira] [Updated] (CASSANDRA-13669) Error when starting cassandra: Unable to make UUID from 'aa' (SASI index)

[JIRA]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrés de la Peña updated CASSANDRA-13669:
--
   Resolution: Fixed
Reproduced In: 3.11.1, 3.11.0, 4.0  (was: 3.11.0, 3.11.1, 4.0)
   Status: Resolved  (was: Ready to Commit)

> Error when starting cassandra: Unable to make UUID from 'aa' (SASI index)
> -
>
> Key: CASSANDRA-13669
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13669
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
> Environment: Tested on:
> * macOS Sierra 10.12.5
> * Ubuntu 14.04.5 LTS
>Reporter: Lukasz Biedrycki
>Assignee: ZhaoYang
>Priority: Critical
>  Labels: sasi
> Fix For: 3.11.x
>
>
> Recently I experienced a problem that prevents me to restart cassandra.
> I narrowed it down to SASI Index when added on uuid field.
> Steps to reproduce:
> 1. start cassandra (./bin/cassandra -f)
> 2. create keyspace, table, index and add data:
> {noformat}
> CREATE KEYSPACE testkeyspace
> WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '1'} 
>AND durable_writes = true;
> use testkeyspace ;
> CREATE TABLE testtable (
>col1 uuid,
>col2 uuid,
>ts timeuuid,
>col3 uuid,
>PRIMARY KEY((col1, col2), ts) ) with clustering order by (ts desc);
> CREATE CUSTOM INDEX col3_testtable_idx ON testtable(col3)
> USING 'org.apache.cassandra.index.sasi.SASIIndex'
> WITH OPTIONS = {'analyzer_class': 
> 'org.apache.cassandra.index.sasi.analyzer.StandardAnalyzer', 'mode': 
> 'PREFIX'};
> INSERT INTO testtable(col1, col2, ts, col3)
> VALUES(898e0014-6161-11e7-b9b7-238ea83bd70b,
>898e0014-6161-11e7-b9b7-238ea83bd70b,
>now(), 898e0014-6161-11e7-b9b7-238ea83bd70b);
> {noformat}
> 3. restart cassandra
> It crashes with an error (sorry it's huge):
> {noformat}
> DEBUG 09:09:20 Writing Memtable-testtable@1005362073(0.075KiB serialized 
> bytes, 1 ops, 0%/0% of on/off-heap limit), flushed range = 
> (min(-9223372036854775808), max(9223372036854775807)]
> ERROR 09:09:20 Exception in thread 
> Thread[PerDiskMemtableFlushWriter_0:1,5,main]
> org.apache.cassandra.serializers.MarshalException: Unable to make UUID from 
> 'aa'
>   at 
> org.apache.cassandra.db.marshal.UUIDType.fromString(UUIDType.java:118) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.index.sasi.analyzer.StandardAnalyzer.hasNext(StandardAnalyzer.java:168)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter$Index.add(PerSSTableIndexWriter.java:208)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter.lambda$nextUnfilteredCluster$0(PerSSTableIndexWriter.java:132)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at java.util.Collections$SingletonSet.forEach(Collections.java:4767) 
> ~[na:1.8.0_131]
>   at 
> org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter.nextUnfilteredCluster(PerSSTableIndexWriter.java:119)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.ColumnIndex.lambda$add$1(ColumnIndex.java:233) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_131]
>   at org.apache.cassandra.db.ColumnIndex.add(ColumnIndex.java:233) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.ColumnIndex.buildRowIndex(ColumnIndex.java:107) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:169)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:48)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:458)
>  ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.call(Memtable.java:493) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.call(Memtable.java:380) 
> ~[apache-cassandra-3.9.jar:3.9]
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_131]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_131]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_131]
>   at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
> Exception (java.lang.RuntimeException) encountered during startup: 
> java.util.concurrent.ExecutionException: java.lang.RuntimeException: 
> java.util.concurrent.ExecutionException: 
> 

[3/3] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[adelapena]

Merge branch 'cassandra-3.11' into trunk

# Conflicts:
#   src/java/org/apache/cassandra/index/sasi/SASIIndex.java
#   test/unit/org/apache/cassandra/index/sasi/SASIIndexTest.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2bad5d5b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2bad5d5b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2bad5d5b

Branch: refs/heads/trunk
Commit: 2bad5d5b6d2134ecd3db63d02aa2274299d1d748
Parents: 0f5443d ea62d88
Author: Andrés de la Peña 
Authored: Mon Jun 25 12:11:46 2018 +0100
Committer: Andrés de la Peña 
Committed: Mon Jun 25 12:11:46 2018 +0100

--
 CHANGES.txt |  1 +
 .../apache/cassandra/index/sasi/SASIIndex.java  |  5 +-
 .../index/sasi/analyzer/AbstractAnalyzer.java   |  8 ++
 .../index/sasi/analyzer/DelimiterAnalyzer.java  | 12 ++-
 .../index/sasi/analyzer/NoOpAnalyzer.java   |  6 ++
 .../sasi/analyzer/NonTokenizingAnalyzer.java|  6 ++
 .../index/sasi/analyzer/StandardAnalyzer.java   | 18 +
 .../cassandra/index/sasi/conf/IndexMode.java| 20 -
 .../cassandra/index/sasi/SASIIndexTest.java | 77 
 9 files changed, 147 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2bad5d5b/CHANGES.txt
--
diff --cc CHANGES.txt
index ce945df,6f6e009..047689e
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,260 -1,5 +1,261 @@@
 +4.0
 + * Add a virtual table to expose active client connections (CASSANDRA-14458)
 + * Clean up and refactor client metrics (CASSANDRA-14524)
 + * Nodetool import row cache invalidation races with adding sstables to 
tracker (CASSANDRA-14529)
 + * Fix assertions in LWTs after TableMetadata was made immutable 
(CASSANDRA-14356)
 + * Abort compactions quicker (CASSANDRA-14397)
 + * Support light-weight transactions in cassandra-stress (CASSANDRA-13529)
 + * Make AsyncOneResponse use the correct timeout (CASSANDRA-14509)
 + * Add option to sanity check tombstones on reads/compactions 
(CASSANDRA-14467)
 + * Add a virtual table to expose all running sstable tasks (CASSANDRA-14457)
 + * Let nodetool import take a list of directories (CASSANDRA-14442)
 + * Avoid unneeded memory allocations / cpu for disabled log levels 
(CASSANDRA-14488)
 + * Implement virtual keyspace interface (CASSANDRA-7622)
 + * nodetool import cleanup and improvements (CASSANDRA-14417)
 + * Bump jackson version to >= 2.9.5 (CASSANDRA-14427)
 + * Allow nodetool toppartitions without specifying table (CASSANDRA-14360)
 + * Audit logging for database activity (CASSANDRA-12151)
 + * Clean up build artifacts in docs container (CASSANDRA-14432)
 + * Minor network authz improvements (Cassandra-14413)
 + * Automatic sstable upgrades (CASSANDRA-14197)
 + * Replace deprecated junit.framework.Assert usages with org.junit.Assert 
(CASSANDRA-14431)
 + * Cassandra-stress throws NPE if insert section isn't specified in user 
profile (CASSSANDRA-14426)
 + * List clients by protocol versions `nodetool clientstats --by-protocol` 
(CASSANDRA-14335)
 + * Improve LatencyMetrics performance by reducing write path processing 
(CASSANDRA-14281)
 + * Add network authz (CASSANDRA-13985)
 + * Use the correct IP/Port for Streaming when localAddress is left unbound 
(CASSANDRA-14389)
 + * nodetool listsnapshots is missing local system keyspace snapshots 
(CASSANDRA-14381)
 + * Remove StreamCoordinator.streamExecutor thread pool (CASSANDRA-14402)
 + * Rename nodetool --with-port to --print-port to disambiguate from --port 
(CASSANDRA-14392)
 + * Client TOPOLOGY_CHANGE messages have wrong port. (CASSANDRA-14398)
 + * Add ability to load new SSTables from a separate directory (CASSANDRA-6719)
 + * Eliminate background repair and probablistic read_repair_chance table 
options
 +   (CASSANDRA-13910)
 + * Bind to correct local address in 4.0 streaming (CASSANDRA-14362)
 + * Use standard Amazon naming for datacenter and rack in Ec2Snitch 
(CASSANDRA-7839)
 + * Fix junit failure for SSTableReaderTest (CASSANDRA-14387)
 + * Abstract write path for pluggable storage (CASSANDRA-14118)
 + * nodetool describecluster should be more informative (CASSANDRA-13853)
 + * Compaction performance improvements (CASSANDRA-14261) 
 + * Refactor Pair usage to avoid boxing ints/longs (CASSANDRA-14260)
 + * Add options to nodetool tablestats to sort and limit output 
(CASSANDRA-13889)
 + * Rename internals to reflect CQL vocabulary (CASSANDRA-14354)
 + * Add support for hybrid MIN(), MAX() speculative retry policies
 +   (CASSANDRA-14293, CASSANDRA-14338, CASSANDRA-14352)
 + * Fix some regressions caused by 14058 (CASSANDRA-14353)
 + * Abstract repair for pluggable storage (CASSANDRA-14116)
 + * Add meaningful 

[1/3] cassandra git commit: Validate supported column type with SASI analyzer

[adelapena]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.11 08a515dc7 -> ea62d8862
  refs/heads/trunk 0f5443d9c -> 2bad5d5b6


Validate supported column type with SASI analyzer

patch by Zhao Yang; reviewed by Andres de la Peña for CASSANDRA-13669


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ea62d886
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ea62d886
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ea62d886

Branch: refs/heads/cassandra-3.11
Commit: ea62d8862c311e3d9b64d622bea0a68d3825aa7d
Parents: 08a515d
Author: Zhao Yang 
Authored: Tue May 15 09:43:39 2018 +0800
Committer: Andrés de la Peña 
Committed: Mon Jun 25 11:51:15 2018 +0100

--
 CHANGES.txt |  1 +
 .../apache/cassandra/index/sasi/SASIIndex.java  |  5 +-
 .../index/sasi/analyzer/AbstractAnalyzer.java   |  8 ++
 .../index/sasi/analyzer/DelimiterAnalyzer.java  | 12 ++-
 .../index/sasi/analyzer/NoOpAnalyzer.java   |  6 ++
 .../sasi/analyzer/NonTokenizingAnalyzer.java|  6 ++
 .../index/sasi/analyzer/StandardAnalyzer.java   | 18 +
 .../cassandra/index/sasi/conf/IndexMode.java| 20 -
 .../cassandra/index/sasi/SASIIndexTest.java | 77 
 9 files changed, 147 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ea62d886/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d9c62ea..6f6e009 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.11.3
+ * Validate supported column type with SASI analyzer (CASSANDRA-13669)
  * Remove BTree.Builder Recycler to reduce memory usage (CASSANDRA-13929)
  * Reduce nodetool GC thread count (CASSANDRA-14475)
  * Fix New SASI view creation during Index Redistribution (CASSANDRA-14055)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ea62d886/src/java/org/apache/cassandra/index/sasi/SASIIndex.java
--
diff --git a/src/java/org/apache/cassandra/index/sasi/SASIIndex.java 
b/src/java/org/apache/cassandra/index/sasi/SASIIndex.java
index f127748..2c1d088 100644
--- a/src/java/org/apache/cassandra/index/sasi/SASIIndex.java
+++ b/src/java/org/apache/cassandra/index/sasi/SASIIndex.java
@@ -121,6 +121,9 @@ public class SASIIndex implements Index, 
INotificationConsumer
 CompactionManager.instance.submitIndexBuild(new 
SASIIndexBuilder(baseCfs, toRebuild));
 }
 
+/**
+ * Called via reflection at {@link 
IndexMetadata#validateCustomIndexOptions}
+ */
 public static Map validateOptions(Map 
options, CFMetaData cfm)
 {
 if (!(cfm.partitioner instanceof Murmur3Partitioner))
@@ -140,7 +143,7 @@ public class SASIIndex implements Index, 
INotificationConsumer
 if (target.left.isPartitionKey())
 throw new ConfigurationException("partition key columns are not 
yet supported by SASI");
 
-IndexMode.validateAnalyzer(options);
+IndexMode.validateAnalyzer(options, target.left);
 
 IndexMode mode = IndexMode.getMode(target.left, options);
 if (mode.mode == Mode.SPARSE)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ea62d886/src/java/org/apache/cassandra/index/sasi/analyzer/AbstractAnalyzer.java
--
diff --git 
a/src/java/org/apache/cassandra/index/sasi/analyzer/AbstractAnalyzer.java 
b/src/java/org/apache/cassandra/index/sasi/analyzer/AbstractAnalyzer.java
index 31c66cc..e3bb7a2 100644
--- a/src/java/org/apache/cassandra/index/sasi/analyzer/AbstractAnalyzer.java
+++ b/src/java/org/apache/cassandra/index/sasi/analyzer/AbstractAnalyzer.java
@@ -43,6 +43,14 @@ public abstract class AbstractAnalyzer implements 
Iterator
 public abstract void reset(ByteBuffer input);
 
 /**
+ * Test whether the given validator is compatible with the underlying 
analyzer.
+ *
+ * @param validator
+ * @return
+ */
+public abstract boolean isCompatibleWith(AbstractType validator);
+
+/**
  * @return true if current analyzer provides text tokenization, false 
otherwise.
  */
 public boolean isTokenizing()

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ea62d886/src/java/org/apache/cassandra/index/sasi/analyzer/DelimiterAnalyzer.java
--
diff --git 
a/src/java/org/apache/cassandra/index/sasi/analyzer/DelimiterAnalyzer.java 
b/src/java/org/apache/cassandra/index/sasi/analyzer/DelimiterAnalyzer.java
index 24acef4..fea4b4f 100644
--- a/src/java/org/apache/cassandra/index/sasi/analyzer/DelimiterAnalyzer.java
+++ 

[2/3] cassandra git commit: Validate supported column type with SASI analyzer

[adelapena]

Validate supported column type with SASI analyzer

patch by Zhao Yang; reviewed by Andres de la Peña for CASSANDRA-13669


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ea62d886
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ea62d886
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ea62d886

Branch: refs/heads/trunk
Commit: ea62d8862c311e3d9b64d622bea0a68d3825aa7d
Parents: 08a515d
Author: Zhao Yang 
Authored: Tue May 15 09:43:39 2018 +0800
Committer: Andrés de la Peña 
Committed: Mon Jun 25 11:51:15 2018 +0100

--
 CHANGES.txt |  1 +
 .../apache/cassandra/index/sasi/SASIIndex.java  |  5 +-
 .../index/sasi/analyzer/AbstractAnalyzer.java   |  8 ++
 .../index/sasi/analyzer/DelimiterAnalyzer.java  | 12 ++-
 .../index/sasi/analyzer/NoOpAnalyzer.java   |  6 ++
 .../sasi/analyzer/NonTokenizingAnalyzer.java|  6 ++
 .../index/sasi/analyzer/StandardAnalyzer.java   | 18 +
 .../cassandra/index/sasi/conf/IndexMode.java| 20 -
 .../cassandra/index/sasi/SASIIndexTest.java | 77 
 9 files changed, 147 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ea62d886/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d9c62ea..6f6e009 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.11.3
+ * Validate supported column type with SASI analyzer (CASSANDRA-13669)
  * Remove BTree.Builder Recycler to reduce memory usage (CASSANDRA-13929)
  * Reduce nodetool GC thread count (CASSANDRA-14475)
  * Fix New SASI view creation during Index Redistribution (CASSANDRA-14055)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ea62d886/src/java/org/apache/cassandra/index/sasi/SASIIndex.java
--
diff --git a/src/java/org/apache/cassandra/index/sasi/SASIIndex.java 
b/src/java/org/apache/cassandra/index/sasi/SASIIndex.java
index f127748..2c1d088 100644
--- a/src/java/org/apache/cassandra/index/sasi/SASIIndex.java
+++ b/src/java/org/apache/cassandra/index/sasi/SASIIndex.java
@@ -121,6 +121,9 @@ public class SASIIndex implements Index, 
INotificationConsumer
 CompactionManager.instance.submitIndexBuild(new 
SASIIndexBuilder(baseCfs, toRebuild));
 }
 
+/**
+ * Called via reflection at {@link 
IndexMetadata#validateCustomIndexOptions}
+ */
 public static Map validateOptions(Map 
options, CFMetaData cfm)
 {
 if (!(cfm.partitioner instanceof Murmur3Partitioner))
@@ -140,7 +143,7 @@ public class SASIIndex implements Index, 
INotificationConsumer
 if (target.left.isPartitionKey())
 throw new ConfigurationException("partition key columns are not 
yet supported by SASI");
 
-IndexMode.validateAnalyzer(options);
+IndexMode.validateAnalyzer(options, target.left);
 
 IndexMode mode = IndexMode.getMode(target.left, options);
 if (mode.mode == Mode.SPARSE)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ea62d886/src/java/org/apache/cassandra/index/sasi/analyzer/AbstractAnalyzer.java
--
diff --git 
a/src/java/org/apache/cassandra/index/sasi/analyzer/AbstractAnalyzer.java 
b/src/java/org/apache/cassandra/index/sasi/analyzer/AbstractAnalyzer.java
index 31c66cc..e3bb7a2 100644
--- a/src/java/org/apache/cassandra/index/sasi/analyzer/AbstractAnalyzer.java
+++ b/src/java/org/apache/cassandra/index/sasi/analyzer/AbstractAnalyzer.java
@@ -43,6 +43,14 @@ public abstract class AbstractAnalyzer implements 
Iterator
 public abstract void reset(ByteBuffer input);
 
 /**
+ * Test whether the given validator is compatible with the underlying 
analyzer.
+ *
+ * @param validator
+ * @return
+ */
+public abstract boolean isCompatibleWith(AbstractType validator);
+
+/**
  * @return true if current analyzer provides text tokenization, false 
otherwise.
  */
 public boolean isTokenizing()

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ea62d886/src/java/org/apache/cassandra/index/sasi/analyzer/DelimiterAnalyzer.java
--
diff --git 
a/src/java/org/apache/cassandra/index/sasi/analyzer/DelimiterAnalyzer.java 
b/src/java/org/apache/cassandra/index/sasi/analyzer/DelimiterAnalyzer.java
index 24acef4..fea4b4f 100644
--- a/src/java/org/apache/cassandra/index/sasi/analyzer/DelimiterAnalyzer.java
+++ b/src/java/org/apache/cassandra/index/sasi/analyzer/DelimiterAnalyzer.java
@@ -37,10 +37,10 @@ import org.apache.cassandra.utils.AbstractIterator;
 public class DelimiterAnalyzer extends 

[jira] [Commented] (CASSANDRA-14344) Support filtering using IN restrictions

[Venkata Harikrishna Nukala (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522138#comment-16522138
 ] 

Venkata Harikrishna Nukala commented on CASSANDRA-14344:


[~blerer] Added test cases and uploaded new patch  [^14344-trunk-2.txt] with 
small fixes. {{RowFilter.Expression}} is accepting ByteBuffer for value which 
is restricting all values to be serialized into a single buffer. And they 
should be deserialized to compare. This approach forces deserialization for 
every check.

I am also trying another approach. Created a new class 
{{RowFilter.InExpression}} for evaluating the {{IN}} expressions. The 
expression instance is common across evaluations, so it can hold deserialized 
values and reuse them. Attached patch  [^14344-trunk-inexpression-approach.txt] 
of this approach. Introducing a new expression type has side effects on 
Serialization i.e. {{RowFilter.Expression.Serializer}} class. Not sure how much 
it impacts yet. This patch is still half-baked (sorry for that). Just want to 
give some idea of the approach.

> Support filtering using IN restrictions
> ---
>
> Key: CASSANDRA-14344
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14344
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Dikang Gu
>Assignee: Venkata Harikrishna Nukala
>Priority: Major
> Attachments: 14344-trunk-2.txt, 
> 14344-trunk-inexpression-approach.txt, 14344-trunk.txt
>
>
> Support IN filter query like this:
>  
> CREATE TABLE ks1.t1 (
>     key int,
>     col1 int,
>     col2 int,
>     value int,
>     PRIMARY KEY (key, col1, col2)
> ) WITH CLUSTERING ORDER BY (col1 ASC, col2 ASC)
>  
> cqlsh:ks1> select * from t1 where key = 1 and col2 in (1) allow filtering;
>  
>  key | col1 | col2 | value
> -+--+--+---
>    1 |    1 |    1 |     1
>    1 |    2 |    1 |     3
>  
> (2 rows)
> cqlsh:ks1> select * from t1 where key = 1 and col2 in (1, 2) allow filtering;
> *{color:#ff}InvalidRequest: Error from server: code=2200 [Invalid query] 
> message="IN restrictions are not supported on indexed columns"{color}*
> cqlsh:ks1>



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14344) Support filtering using IN restrictions

[Venkata Harikrishna Nukala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Venkata Harikrishna Nukala updated CASSANDRA-14344:
---
Attachment: 14344-trunk-inexpression-approach.txt
14344-trunk-2.txt

> Support filtering using IN restrictions
> ---
>
> Key: CASSANDRA-14344
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14344
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Dikang Gu
>Assignee: Venkata Harikrishna Nukala
>Priority: Major
> Attachments: 14344-trunk-2.txt, 
> 14344-trunk-inexpression-approach.txt, 14344-trunk.txt
>
>
> Support IN filter query like this:
>  
> CREATE TABLE ks1.t1 (
>     key int,
>     col1 int,
>     col2 int,
>     value int,
>     PRIMARY KEY (key, col1, col2)
> ) WITH CLUSTERING ORDER BY (col1 ASC, col2 ASC)
>  
> cqlsh:ks1> select * from t1 where key = 1 and col2 in (1) allow filtering;
>  
>  key | col1 | col2 | value
> -+--+--+---
>    1 |    1 |    1 |     1
>    1 |    2 |    1 |     3
>  
> (2 rows)
> cqlsh:ks1> select * from t1 where key = 1 and col2 in (1, 2) allow filtering;
> *{color:#ff}InvalidRequest: Error from server: code=2200 [Invalid query] 
> message="IN restrictions are not supported on indexed columns"{color}*
> cqlsh:ks1>



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Resolved] (CASSANDRA-13023) Add droppable tombstone metrics

[Stefan Podkowinski (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Podkowinski resolved CASSANDRA-13023.

Resolution: Not A Problem

Thanks for letting me know, [~arodrime]! It's been a while since I created the 
ticket, so I'm not fully sure if I had something else in mind back then, or 
simply wasn't aware of the metric at table level, which seems to be sufficient.

> Add droppable tombstone metrics
> ---
>
> Key: CASSANDRA-13023
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13023
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability
>Reporter: Stefan Podkowinski
>Priority: Major
> Attachments: droppableTombstoneRatio.png
>
>
> Currently it's only possible to inspect the ratio of droppable tombstones on 
> sstable basis using the {{sstablemetadata}} tool. It would be very useful to 
> have this information provided as part of Cassandra metrics as well, e.g. to 
> get an idea on the effectiveness of tombstone eviction during compactions 
> over time.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14537) Expose metrics through virtual tables

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521945#comment-16521945
 ] 

Benjamin Lerer commented on CASSANDRA-14537:


[~cnlwsu] I did not notice that you already opened some tickets to expose 
metrics. My plan was to do all of them in a single ticket.
Now, it my make sense to use that ticket as an umbrella ticket. What do you 
think?  

> Expose metrics through virtual tables
> -
>
> Key: CASSANDRA-14537
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14537
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Major
>
> The goal of this ticket is to expose the metrics information through some 
> system views (virtual tables).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14538) Add virtual table to view cache information

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-14538:
---
Reviewer: Benjamin Lerer

> Add virtual table to view cache information
> ---
>
> Key: CASSANDRA-14538
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14538
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
>  Labels: virtual-tables
>
>  Add a couple tables, one for displaying the general cache information (like 
> in {{nodetool info}}) and another to view the contents (ish) of keycache. 
> This would be pretty useful in tests to tell if things are in the cache and 
> in finding wide partitions and performance issues.
> Possibility to discuss: DELETE on the key cache to evict specific records, I 
> wrote the code to do it but im not sure if its valuable or worth complexity. 
> TRUNCATE support on virtual tables might be nice as a way to clear the 
> keycache as well. These can be followup tickets though if it seems like a 
> good idea as truncate and delete are currently disabled in virtual tables and 
> that behavior should maybe be its own ticket/discussion.
>  
> {code}
> cqlsh> SELECT * FROM system_views.caches;
>  name| entries | hit_ratio | hits | recent_hits_per_sec | 
> recent_requests_per_sec | requests | size_max | size_used
> -+-+---+--+-+-+--+--+---
>  counter |   0 |   NaN |0 |   0 | 
>   0 |0 | 12582912 | 0
>  key |  19 |  0.922509 |  250 |  11 | 
>  13 |  271 | 25165824 |  1728
>  row |   0 |   NaN |0 |   0 | 
>   0 |0 | 16777216 | 0
> (3 rows)
> cqlsh> SELECT * FROM system_views.key_cache;
>  keyspace_name | table_name   | key | sstable | size
> ---+--+-+-+--
>  basic | wide |row1 |   5 |   64
>  basic | wide |row1 |   6 |   64
> system |local |   local |  15 |   24
> system |local |   local |  16 |   24
> system |local |   local |  17 |   24
> system | sstable_activity | system_auth:roles:1 |  13 |   24
>system_auth |roles |   cassandra |   1 |   24
>  system_schema |   tables |   basic |  13 |   24
>  system_schema |   tables | system_auth |  13 |   24
>  system_schema |   tables |  system_distributed |  13 |   24
>  system_schema |   tables |   system_traces |  13 |   24
>  system_schema |  columns |   basic |  13 |   24
>  system_schema |  columns | system_auth |  13 |   24
>  system_schema |  columns |  system_distributed |  13 |   24
>  system_schema |  columns |   system_traces |  13 |   24
>  system_schema |keyspaces |   basic |  13 |   24
>  system_schema |keyspaces | system_auth |  13 |   24
>  system_schema |keyspaces |  system_distributed |  13 |   24
>  system_schema |keyspaces |   system_traces |  13 |   24
> (19 rows)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14523) Thread pool stats virtual table

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-14523:
---
Reviewer: Benjamin Lerer

> Thread pool stats virtual table
> ---
>
> Key: CASSANDRA-14523
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14523
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
>  Labels: virtual-tables
>
> Expose the thread pools like in status logger/tpstats. Additionally be nice 
> to include the scheduled executor pools that are currently unmonitored.
> {code:java}
> cqlsh> select * from system_views.thread_pools;
>  thread_pool  | active | active_max | completed | pending 
> | tasks_blocked | total_blocked
> --+++---+-+---+---
>anti_entropy_stage |  0 |  1 | 0 |   0 
> | 0 | 0
>cache_cleanup_executor |  0 |  1 | 0 |   0 
> | 0 | 0
>   compaction_executor |  0 |  4 |41 |   0 
> | 0 | 0
>counter_mutation_stage |  0 | 32 | 0 |   0 
> | 0 | 0
>  gossip_stage |  0 |  1 | 0 |   0 
> | 0 | 0
>  hints_dispatcher |  0 |  2 | 0 |   0 
> | 0 | 0
>   internal_response_stage |  0 |  8 | 0 |   0 
> | 0 | 0
> memtable_flush_writer |  0 |  2 | 5 |   0 
> | 0 | 0
>   memtable_post_flush |  0 |  1 |20 |   0 
> | 0 | 0
>   memtable_reclaim_memory |  0 |  1 | 5 |   0 
> | 0 | 0
>   migration_stage |  0 |  1 | 0 |   0 
> | 0 | 0
>misc_stage |  0 |  1 | 0 |   0 
> | 0 | 0
>mutation_stage |  0 | 32 |   247 |   0 
> | 0 | 0
> native_transport_requests |  1 |128 |28 |   0 
> | 0 | 0
>  pending_range_calculator |  0 |  1 | 2 |   0 
> | 0 | 0
>  per_disk_memtable_flush_writer_0 |  0 |  2 | 5 |   0 
> | 0 | 0
> read_repair_stage |  0 |  8 | 0 |   0 
> | 0 | 0
>read_stage |  0 | 32 |13 |   0 
> | 0 | 0
>   repair_task |  0 | 2147483647 | 0 |   0 
> | 0 | 0
>request_response_stage |  0 |  8 | 0 |   0 
> | 0 | 0
>   sampler |  0 |  1 | 0 |   0 
> | 0 | 0
>  scheduled_fast_tasks |  0 | 2147483647 |  1398 |   1 
> | 0 | 0
>   scheduled_heartbeat |  0 | 2147483647 |14 |   1 
> | 0 | 0
> scheduled_hotness_tracker |  0 | 2147483647 | 0 |   1 
> | 0 | 0
>  scheduled_non_periodic_tasks |  0 | 2147483647 |10 |   0 
> | 0 | 0
>  scheduled_optional_tasks |  0 | 2147483647 | 5 |   8 
> | 0 | 0
> scheduled_summary_builder |  0 | 2147483647 | 0 |   1 
> | 0 | 0
>   scheduled_tasks |  0 | 2147483647 |   194 |  74 
> | 0 | 0
>secondary_index_management |  0 |  1 | 0 |   0 
> | 0 | 0
>   validation_executor |  0 | 2147483647 | 0 |   0 
> | 0 | 0
>   view_build_executor |  0 |  1 | 0 |   0 
> | 0 | 0
>   view_mutation_stage |  0 | 32 | 0 |   0 
> | 0 | 0
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands,