Re: Deflate compressor

[Jeff Jirsa]

On 2017-07-01 02:50 (-0700), Cogumelos Maravilha  
wrote: 
> Hi list,
> 
> Is there a way to set Deflate level of compression?
> 
> Brotli sounds good but unstable.
> 
> I just need more compression ratio.
> 
> I'm using C* 3.11.0
> 

LZ4 compression levels are configurable as of cassandra 3.6 ( 
https://issues.apache.org/jira/browse/CASSANDRA-11051 ) . 

"The attached patch adds the following optional parameters 
'lz4_compressor_type' and 'lz4_high_compressor_level' to the LZ4Compressor."

Glancing at the source: 
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/compress/LZ4Compressor.java#L41-L49

lz4_high_compressor_level defaults to 9. 
lz4_compressor_type defaults to 'fast' ('high' is also available).




-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

回复：hi, i have a question.

[牟宗传]

hi jirsa,thank you very much! i'll try it .
best regards!   

--发件人：Jeff 
Jirsa 发送时间：2017年7月6日(星期四) 13:15收件人：user 
主　题：Re: hi, i have a question.


On 2017-07-05 19:29 (-0700), "牟宗传"  wrote: 
> 
> 
> hi , i have a question .    how could i connect to cassandra-cluster( 
>"TEST CLUSTER")? i would not connect to one node IP of cluster ,because when 
>the node down, i could not do something? Â Â Â Â could you help me ?
> thanks 
> 
> 

Typically you provide a list of IPs in a config or property file (so that you 
won't need to recompile your application if those IPs change). If any one node 
is down, the other IPs should allow the client to connect and discover the rest 
of the cluster.

Depending on the level of automation at your company, you may find that this is 
easier accomplished with DNS tricks, config-management queries (chef search or 
similar), or flat text files.



-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

Re: Does select queries objects will be allocated in new_heap ??

[Jeff Jirsa]

On 2017-07-05 21:50 (-0700), Pranay akula  wrote: 
> Hai,
> 
> I have a small doubt i am not much familiar with java, so when a select or
> insert query is executed against a node does the output of that query will
> be stored in heap ??
> 

SELECT will create a number of short-lived objects on the heap, yes, but it's 
not like the memtable - there is not a read-side memtable. Most of the objects 
are transient and should be collected before being promoted to old gen. There 
are some exceptions - the key and partition caches are counterexamples.



> For insert query the output will stored in Memtables will is part on
> new_heap right ?? does select and other queries also use the new_heap or
> just writes ??
> 

Just writes.


> Is there any document available correlating queries and heap objects.
> 

Very much version dependent, there are some great videos around about the read 
path - Tyler's is probably as authoritative as you'll find 
https://www.youtube.com/watch?v=HuDJBTPdaOA



-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

Re: hi, i have a question.

[Jeff Jirsa]

On 2017-07-05 19:29 (-0700), "牟宗传"  wrote: 
> 
> 
> hi , i have a question .    how could i connect to cassandra-cluster( 
> "TEST CLUSTER")? i would not connect to one node IP of cluster ,because when 
> the node down, i could not do something?     could you help me ?
> thanks 
> 
> 

Typically you provide a list of IPs in a config or property file (so that you 
won't need to recompile your application if those IPs change). If any one node 
is down, the other IPs should allow the client to connect and discover the rest 
of the cluster.

Depending on the level of automation at your company, you may find that this is 
easier accomplished with DNS tricks, config-management queries (chef search or 
similar), or flat text files.



-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

Does select queries objects will be allocated in new_heap ??

[Pranay akula]

Hai,

I have a small doubt i am not much familiar with java, so when a select or
insert query is executed against a node does the output of that query will
be stored in heap ??

For insert query the output will stored in Memtables will is part on
new_heap right ?? does select and other queries also use the new_heap or
just writes ??

Is there any document available correlating queries and heap objects.


Thanks
Pranay.

Re: Merkle trees requests hanging

[Anuj Wadehra]

Hi Jean, 
Ensure that your firewall is not timing out idle connections. Nodes should time 
out idle connections first (using tcp keep alive settings before firewall does 
it). Please refer 
http://docs.datastax.com/en/archived/cassandra/2.0/cassandra/troubleshooting/trblshootIdleFirewall.html.
 

ThanksAnuj

Sent from Yahoo Mail on Android 
 
  On Tue, 4 Jul 2017 at 19:41, Jean Carlo wrote:   
Hello.

What if a node send a merkle tree to its replica but this one would never 
received by any network issues. The repair will be hanging eternally ? or 
Should I modify  the parameter 
# streaming_socket_timeout_in_ms: 0 to avoid this ?

Saludos

Jean Carlo
"The best way to predict the future is to invent it" Alan Kay
  

hi, i have a question.

[牟宗传]

hi , i have a question .how could i connect to cassandra-cluster( "TEST 
CLUSTER")? i would not connect to one node IP of cluster ,because when the node 
down, i could not do something? could you help me ?
thanks 

Re: commitlog_total_space_in_mb tuning

[Jay Zhuang]

Thanks Jeff for the quick response.

We're running with 3.0.13 which doesn't have commitlog_segment_recycling
option. So it should be disabled.

I think the CL flush is because commitlog full. The commitlog size is
closing to 8G:
#mbean =
org.apache.cassandra.metrics:name=TotalCommitLogSize,type=CommitLog:
Value = 8489271296;

We're running with 128G memory and 30G heap size. Maybe it's good idea
to increase the commitlog_total_space. On the other hand, even with  8G
commitlog_total_space, replaying CL after restart takes more than 5 minutes.

In our case, the actual problem is it's causing lots of read repair
timeouts as the repair mutations are dropped. Which causes Cassandra JVM
hang or sometimes crash.

/Jay

On 7/5/17 2:45 PM, Jeff Jirsa wrote:
> 
> 
> On 2017-07-05 14:32 (-0700), Jay Zhuang  wrote: 
>> Hi,
>>
>> commitlog_total_space_in_mb is increased from 1G to 8G in
>> CASSANDRA-7031. Sometimes we saw the number of dropped mutations spikes.
>> Not sure if it's a sign that we should increase the
>> commitlog_total_space_in_mb?
> 
> 8G seems pretty typical. The real litmus test is in Benedict's comment:
> 
>> We can find that we force memtable flushes as a result of log utilisation 
>> instead of memtable occupancy quite often
> 
> Are your memtable flushes because the memtable is full, or because the 
> commitlog is full?
> 
> Also, what version are you running, and do you have segment recycling 
> enabled? (hopefully not: https://issues.apache.org/jira/browse/CASSANDRA-8771 
> )
> 
> 
> 
> -
> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: user-h...@cassandra.apache.org
> 

-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

Re: Merkle trees requests hanging

[Jeff Jirsa]

On 2017-07-04 07:11 (-0700), Jean Carlo  wrote: 
> Hello.
> 
> What if a node send a merkle tree to its replica but this one would never
> received by any network issues. The repair will be hanging eternally ? or
> Should I modify  the parameter
> # streaming_socket_timeout_in_ms: 0 to avoid this ?
> 

Yes, having a non-zero streaming timeout is generally encouraged for the reason 
you describe.


-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

Re: UDF for sorting

[Jeff Jirsa]

On 2017-07-03 16:19 (-0700), Justin Cameron  wrote: 
> While you can't do this with Cassandra, you can get the functionality you
> want with the cassandra-lucene-plugin (
> https://github.com/Stratio/cassandra-lucene-index/blob/branch-3.0.10/doc/documentation.rst#searching
> ).
> 
> Keep in mind that as with any secondary index there are performance-related
> limitations:
> https://github.com/Stratio/cassandra-lucene-index/blob/branch-3.0.10/doc/documentation.rst#performance-tips


We just added a "Plugins" section to the docs ( 
https://github.com/apache/cassandra/blob/trunk/doc/source/plugins/index.rst )  
- it would be nice if someone would add the Cassandra-Lucene-Index plugin there.




-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

Re: commitlog_total_space_in_mb tuning

[Jeff Jirsa]

On 2017-07-05 14:32 (-0700), Jay Zhuang  wrote: 
> Hi,
> 
> commitlog_total_space_in_mb is increased from 1G to 8G in
> CASSANDRA-7031. Sometimes we saw the number of dropped mutations spikes.
> Not sure if it's a sign that we should increase the
> commitlog_total_space_in_mb?

8G seems pretty typical. The real litmus test is in Benedict's comment:

> We can find that we force memtable flushes as a result of log utilisation 
> instead of memtable occupancy quite often

Are your memtable flushes because the memtable is full, or because the 
commitlog is full?

Also, what version are you running, and do you have segment recycling enabled? 
(hopefully not: https://issues.apache.org/jira/browse/CASSANDRA-8771 )



-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

commitlog_total_space_in_mb tuning

[Jay Zhuang]

Hi,

commitlog_total_space_in_mb is increased from 1G to 8G in
CASSANDRA-7031. Sometimes we saw the number of dropped mutations spikes.
Not sure if it's a sign that we should increase the
commitlog_total_space_in_mb?

For bean:
org.apache.cassandra.metrics:name=WaitingOnSegmentAllocation,type=CommitLog
Mean is 48684 microseconds
Max is 1386179 microseconds

I think it's relatively high, compare to our other clusters. Does anyone
tune that parameter? Any suggestion on that?

Thanks,
Jay

-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

Re: Best practice to add(bootstrap) multiple nodes to cluster at once

[Alain RODRIGUEZ]

Sorry about the delay, but a colleague wrote this, that I believe might be
of interest:
http://thelastpickle.com/blog/2017/05/23/auto-bootstrapping-part1.html.

Specially this part:
http://thelastpickle.com/blog/2017/05/23/auto-bootstrapping-part1.html#adding-multiple-nodes
.

Still unless it is really needed or in independent data center, It is a
good idea to add nodes one by one in case of doubt on what are the impacts.

C*heers,
---
Alain Rodriguez - @arodream - al...@thelastpickle.com
France

The Last Pickle - Apache Cassandra Consulting
http://www.thelastpickle.com

2017-06-20 17:14 GMT+01:00 ZAIDI, ASAD A :

> adding multiple nodes at once tax system more and caused me issues on
> existing nodes. I prefer to add one node at a time …
>
>
>
>
>
> *From:* techpyaasa . [mailto:techpya...@gmail.com]
> *Sent:* Tuesday, June 20, 2017 9:32 AM
> *To:* user@cassandra.apache.org
> *Subject:* Best practice to add(bootstrap) multiple nodes to cluster at
> once
>
>
>
> Hi,
>
>
>
> What is the best practice to add(bootstrap) multiple nodes at once to c*
> cluster.
>
>
>
> Using c*-2.1.17 , 2 DCs , 3 groups in each DC
>
>
>
> Thanks
>
> TechPyaasa
>

Re: Seed gossip version will not connect with that version

[Nitan Kainth]

Restart could help with hung gossip but check you network as possible root 
cause for this.

Sent from my iPhone

> On Jul 5, 2017, at 7:23 AM, Jean Carlo  wrote:
> 
> Hello
> 
> I have repairs that hangs because this problem
> 
> WARN  [MessagingService-Outgoing-/10.0.0.143] 2017-07-04 10:29:50,076 
> OutboundTcpConnection.java:416 - Seed gossip version is -2147483648; will not 
> connect with that version
> INFO  [HANDSHAKE-/10.0.0.143] 2017-07-04 10:29:50,076 
> OutboundTcpConnection.java:496 - Cannot handshake version with /10.0.0.143
> INFO  [HANDSHAKE-/10.0.0.143] 2017-07-04 10:29:50,090 
> OutboundTcpConnection.java:487 - Handshaking version with /10.0.0.143
> 
> 
> Is it enough to solve by restarting cassandra both nodes ?
> 
> 
> Best regards
> 
> 
> Jean Carlo
> 
> "The best way to predict the future is to invent it" Alan Kay

Seed gossip version will not connect with that version

[Jean Carlo]

Hello

I have repairs that hangs because this problem

WARN  [MessagingService-Outgoing-/10.0.0.143] 2017-07-04 10:29:50,076
OutboundTcpConnection.java:416 - Seed gossip version is -2147483648; will
not connect with that version
INFO  [HANDSHAKE-/10.0.0.143] 2017-07-04 10:29:50,076
OutboundTcpConnection.java:496 - Cannot handshake version with /10.0.0.143
INFO  [HANDSHAKE-/10.0.0.143] 2017-07-04 10:29:50,090
OutboundTcpConnection.java:487 - Handshaking version with /10.0.0.143


Is it enough to solve by restarting cassandra both nodes ?


Best regards


Jean Carlo

"The best way to predict the future is to invent it" Alan Kay

stress tool: random seed

[Vlad]

Hi,
I'm populating database with YAML. Each time I run stress tool I get the same 
rows, i.e. the same data generated and no new rows appear.
Is there any option to generate each time new data? I would like to test 
growing database, but don't want insert each time all data.
I found a kind of workaround by changing partition key distribution boundaries, 
but is there better way?
Regards, Vlad

Re: Error when starting cassandra: Unable to make UUID from 'aa' (SASI index)

[Łukasz Biedrycki]

Yes, I tested 3.11, it exists. Thank you, I'll open up a Jira.

On Wed, Jul 5, 2017 at 1:27 PM, Jonathan Haddad  wrote:

> You should check if the same error exists in 3.11. If so, open up a Jira.
> On Wed, Jul 5, 2017 at 10:38 AM Łukasz Biedrycki <
> lukasz.biedry...@gmail.com> wrote:
>
>> Hey,
>>
>> I am using Cassandra 3.9.
>>
>> Recently I experienced a problem that prevents me to restart cassandra. I
>> narrowed it down to SASI Index.
>>
>> Steps to reproduce:
>> 1. start cassandra (./bin/cassandra -f)
>> 2. create keyspace, table, index and add data:
>>
>> CREATE KEYSPACE testkeyspace
>> WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '1'}
>>AND durable_writes = true;
>>
>> use testkeyspace ;
>>
>> CREATE TABLE testtable (
>>col1 uuid,
>>col2 uuid,
>>ts timeuuid,
>>col3 uuid,
>>PRIMARY KEY((col1, col2), ts) ) with clustering order by (ts desc);
>>
>> CREATE CUSTOM INDEX col3_testtable_idx ON testtable(col3)
>> USING 'org.apache.cassandra.index.sasi.SASIIndex'
>> WITH OPTIONS = {'analyzer_class': 'org.apache.cassandra.index.
>> sasi.analyzer.StandardAnalyzer', 'mode': 'PREFIX'};
>>
>> INSERT INTO testtable(col1, col2, ts, col3)
>> VALUES(898e0014-6161-11e7-b9b7-238ea83bd70b,
>>898e0014-6161-11e7-b9b7-238ea83bd70b,
>>now(), 898e0014-6161-11e7-b9b7-238ea83bd70b);
>>
>> 3. restart cassandra
>>
>> It crashes with an error (sorry it's huge):
>> DEBUG 09:09:20 Writing Memtable-testtable@1005362073(0.075KiB serialized
>> bytes, 1 ops, 0%/0% of on/off-heap limit), flushed range =
>> (min(-9223372036854775808), max(9223372036854775807)]
>> ERROR 09:09:20 Exception in thread Thread[PerDiskMemtableFlushWriter_0:
>> 1,5,main]
>> org.apache.cassandra.serializers.MarshalException: Unable to make UUID
>> from 'aa'
>> at org.apache.cassandra.db.marshal.UUIDType.fromString(UUIDType.java:118)
>> ~[apache-cassandra-3.9.jar:3.9]
>> at 
>> org.apache.cassandra.index.sasi.analyzer.StandardAnalyzer.hasNext(StandardAnalyzer.java:168)
>> ~[apache-cassandra-3.9.jar:3.9]
>> at org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter$Index.
>> add(PerSSTableIndexWriter.java:208) ~[apache-cassandra-3.9.jar:3.9]
>> at org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter.lambda$
>> nextUnfilteredCluster$0(PerSSTableIndexWriter.java:132)
>> ~[apache-cassandra-3.9.jar:3.9]
>> at java.util.Collections$SingletonSet.forEach(Collections.java:4767)
>> ~[na:1.8.0_131]
>> at org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter.
>> nextUnfilteredCluster(PerSSTableIndexWriter.java:119)
>> ~[apache-cassandra-3.9.jar:3.9]
>> at org.apache.cassandra.db.ColumnIndex.lambda$add$1(ColumnIndex.java:233)
>> ~[apache-cassandra-3.9.jar:3.9]
>> at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_131]
>> at org.apache.cassandra.db.ColumnIndex.add(ColumnIndex.java:233)
>> ~[apache-cassandra-3.9.jar:3.9]
>> at org.apache.cassandra.db.ColumnIndex.buildRowIndex(ColumnIndex.java:107)
>> ~[apache-cassandra-3.9.jar:3.9]
>> at 
>> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:169)
>> ~[apache-cassandra-3.9.jar:3.9]
>> at org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(
>> SimpleSSTableMultiWriter.java:48) ~[apache-cassandra-3.9.jar:3.9]
>> at org.apache.cassandra.db.Memtable$FlushRunnable.
>> writeSortedContents(Memtable.java:458) ~[apache-cassandra-3.9.jar:3.9]
>> at org.apache.cassandra.db.Memtable$FlushRunnable.call(Memtable.java:493)
>> ~[apache-cassandra-3.9.jar:3.9]
>> at org.apache.cassandra.db.Memtable$FlushRunnable.call(Memtable.java:380)
>> ~[apache-cassandra-3.9.jar:3.9]
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> ~[na:1.8.0_131]
>> at 
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> ~[na:1.8.0_131]
>> at 
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> [na:1.8.0_131]
>> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
>> Exception (java.lang.RuntimeException) encountered during startup:
>> java.util.concurrent.ExecutionException: java.lang.RuntimeException:
>> java.util.concurrent.ExecutionException: 
>> org.apache.cassandra.serializers.MarshalException:
>> Unable to make UUID from 'aa'
>> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
>> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
>> org.apache.cassandra.serializers.MarshalException: Unable to make UUID
>> from 'aa'
>> at org.apache.cassandra.utils.Throwables.maybeFail(Throwables.java:51)
>> ERROR 09:09:20 Exception encountered during startup
>> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
>> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
>> org.apache.cassandra.serializers.MarshalException: Unable to make UUID
>> from 'aa'
>> at org.apache.cassandra.utils.Throwables.maybeFail(Throwables.java:51)
>> 

Re: Error when starting cassandra: Unable to make UUID from 'aa' (SASI index)

[Jonathan Haddad]

You should check if the same error exists in 3.11. If so, open up a Jira.
On Wed, Jul 5, 2017 at 10:38 AM Łukasz Biedrycki 
wrote:

> Hey,
>
> I am using Cassandra 3.9.
>
> Recently I experienced a problem that prevents me to restart cassandra. I
> narrowed it down to SASI Index.
>
> Steps to reproduce:
> 1. start cassandra (./bin/cassandra -f)
> 2. create keyspace, table, index and add data:
>
> CREATE KEYSPACE testkeyspace
> WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '1'}
>AND durable_writes = true;
>
> use testkeyspace ;
>
> CREATE TABLE testtable (
>col1 uuid,
>col2 uuid,
>ts timeuuid,
>col3 uuid,
>PRIMARY KEY((col1, col2), ts) ) with clustering order by (ts desc);
>
> CREATE CUSTOM INDEX col3_testtable_idx ON testtable(col3)
> USING 'org.apache.cassandra.index.sasi.SASIIndex'
> WITH OPTIONS = {'analyzer_class':
> 'org.apache.cassandra.index.sasi.analyzer.StandardAnalyzer', 'mode':
> 'PREFIX'};
>
> INSERT INTO testtable(col1, col2, ts, col3)
> VALUES(898e0014-6161-11e7-b9b7-238ea83bd70b,
>898e0014-6161-11e7-b9b7-238ea83bd70b,
>now(), 898e0014-6161-11e7-b9b7-238ea83bd70b);
>
> 3. restart cassandra
>
> It crashes with an error (sorry it's huge):
> DEBUG 09:09:20 Writing Memtable-testtable@1005362073(0.075KiB serialized
> bytes, 1 ops, 0%/0% of on/off-heap limit), flushed range =
> (min(-9223372036854775808), max(9223372036854775807)]
> ERROR 09:09:20 Exception in thread
> Thread[PerDiskMemtableFlushWriter_0:1,5,main]
> org.apache.cassandra.serializers.MarshalException: Unable to make UUID
> from 'aa'
> at org.apache.cassandra.db.marshal.UUIDType.fromString(UUIDType.java:118)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.index.sasi.analyzer.StandardAnalyzer.hasNext(StandardAnalyzer.java:168)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter$Index.add(PerSSTableIndexWriter.java:208)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter.lambda$nextUnfilteredCluster$0(PerSSTableIndexWriter.java:132)
> ~[apache-cassandra-3.9.jar:3.9]
> at java.util.Collections$SingletonSet.forEach(Collections.java:4767)
> ~[na:1.8.0_131]
> at
> org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter.nextUnfilteredCluster(PerSSTableIndexWriter.java:119)
> ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.db.ColumnIndex.lambda$add$1(ColumnIndex.java:233)
> ~[apache-cassandra-3.9.jar:3.9]
> at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_131]
> at org.apache.cassandra.db.ColumnIndex.add(ColumnIndex.java:233)
> ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.db.ColumnIndex.buildRowIndex(ColumnIndex.java:107)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:169)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:48)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:458)
> ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.db.Memtable$FlushRunnable.call(Memtable.java:493)
> ~[apache-cassandra-3.9.jar:3.9]
> at org.apache.cassandra.db.Memtable$FlushRunnable.call(Memtable.java:380)
> ~[apache-cassandra-3.9.jar:3.9]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_131]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_131]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_131]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
> Exception (java.lang.RuntimeException) encountered during startup:
> java.util.concurrent.ExecutionException: java.lang.RuntimeException:
> java.util.concurrent.ExecutionException:
> org.apache.cassandra.serializers.MarshalException: Unable to make UUID from
> 'aa'
> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
> org.apache.cassandra.serializers.MarshalException: Unable to make UUID from
> 'aa'
> at org.apache.cassandra.utils.Throwables.maybeFail(Throwables.java:51)
> ERROR 09:09:20 Exception encountered during startup
> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
> org.apache.cassandra.serializers.MarshalException: Unable to make UUID from
> 'aa'
> at org.apache.cassandra.utils.Throwables.maybeFail(Throwables.java:51)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.utils.FBUtilities.waitOnFutures(FBUtilities.java:391)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.db.commitlog.CommitLogReplayer.blockForWrites(CommitLogReplayer.java:168)
> ~[apache-cassandra-3.9.jar:3.9]
> 

Re: Adding a New Node With The Same IP of an Old Node

[Shalom Sagges]

Thanks Nitan!

Eventually it was a firewall issue related to the Centos7 node.

Once fixed, the rolling restart resolved the issue completely.

Thanks again!



Shalom Sagges
DBA
T: +972-74-700-4035
 
 We Create Meaningful Connections



On Tue, Jul 4, 2017 at 7:52 PM, Nitan Kainth  wrote:

> Try rolling restart of cluster as solution for schema version mismatch.
>
>
> Sent from my iPhone
>
> On Jul 4, 2017, at 8:31 AM, Shalom Sagges  wrote:
>
> Hi Experts,
>
> My plan is to upgrade the C* nodes' OS from Centos6 to Centos7.
> Since an upgrade wasn't recommended I needed to install new machines with
> Centos7 and join them to the cluster.
> I didn't want to decommission/bootstrap dozens of nodes, so I decided to
> do the following:
>
>- Create a new machine
>- Copy all data, schema data and commit logs using rsync from the old
>node to the new one
>- Stop Cassandra on the old node, copy the delta and shut the node down
>- Change the hostname and IP address of the new node to the hostname
>and IP of the old node.
>- Start Cassandra on the new node.
>
> My thought was that the "new" node will join the cluster as the "old" node
> since it has all of its data and metadata, however, after Cassandra started
> up, it saw all its peers in DN state. The same goes for the other nodes,
> that saw the new node is DN state.
>
> I didn't find any errors or warnings in the logs, but I did see that the 
> *schema
> version* on the new node was different from the others( I'm assuming
> that's the issue here?)
>
>
> *Functioning node*
> nodetool describecluster
> Cluster Information:
> Name: MyCluster
> Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
> Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
> Schema versions:
> 503bf151-c59c-35a0-8350-ca73e82098f5: [x.x.x.32,
> x.x.x.126, x.x.x.35, x.x.x.1, x.x.x.28, x.x.x.15, x.x.x.252, x.x.x.253,
> x.x.x.31]
>
> UNREACHABLE: [x.x.x.2]
>
>
> *New node*
> nodetool describecluster
> Cluster Information:
> Name: MyCluster
> Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
> Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
> Schema versions:
> dab67163-65d4-3895-82e6-ffc07bf5c17a: [x.x.x.2]
>
> UNREACHABLE: [x.x.x.31, x.x.x.1, x.x.x.28, x.x.x.252,
> x.x.x.253, x.x.x.15, x.x.x.126, x.x.x.35, x.x.x.32]
>
>
> I'd really REALLY appreciate some guidance. Did I do something wrong? Is
> there a way to fix this?
>
>
> Thanks a lot!
>
> Shalom Sagges
> DBA
>  
>  We Create Meaningful Connections
>
>
>
> This message may contain confidential and/or privileged information.
> If you are not the addressee or authorized to receive this on behalf of
> the addressee you must not use, copy, disclose or take action based on this
> message or any information herein.
> If you have received this message in error, please advise the sender
> immediately by reply email and delete this message. Thank you.
>
>

-- 
This message may contain confidential and/or privileged information. 
If you are not the addressee or authorized to receive this on behalf of the 
addressee you must not use, copy, disclose or take action based on this 
message or any information herein. 
If you have received this message in error, please advise the sender 
immediately by reply email and delete this message. Thank you.

Error when starting cassandra: Unable to make UUID from 'aa' (SASI index)

[Łukasz Biedrycki]

Hey,

I am using Cassandra 3.9.

Recently I experienced a problem that prevents me to restart cassandra. I
narrowed it down to SASI Index.

Steps to reproduce:
1. start cassandra (./bin/cassandra -f)
2. create keyspace, table, index and add data:

CREATE KEYSPACE testkeyspace
WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '1'}
   AND durable_writes = true;

use testkeyspace ;

CREATE TABLE testtable (
   col1 uuid,
   col2 uuid,
   ts timeuuid,
   col3 uuid,
   PRIMARY KEY((col1, col2), ts) ) with clustering order by (ts desc);

CREATE CUSTOM INDEX col3_testtable_idx ON testtable(col3)
USING 'org.apache.cassandra.index.sasi.SASIIndex'
WITH OPTIONS = {'analyzer_class':
'org.apache.cassandra.index.sasi.analyzer.StandardAnalyzer', 'mode':
'PREFIX'};

INSERT INTO testtable(col1, col2, ts, col3)
VALUES(898e0014-6161-11e7-b9b7-238ea83bd70b,
   898e0014-6161-11e7-b9b7-238ea83bd70b,
   now(), 898e0014-6161-11e7-b9b7-238ea83bd70b);

3. restart cassandra

It crashes with an error (sorry it's huge):
DEBUG 09:09:20 Writing Memtable-testtable@1005362073(0.075KiB serialized
bytes, 1 ops, 0%/0% of on/off-heap limit), flushed range =
(min(-9223372036854775808), max(9223372036854775807)]
ERROR 09:09:20 Exception in thread
Thread[PerDiskMemtableFlushWriter_0:1,5,main]
org.apache.cassandra.serializers.MarshalException: Unable to make UUID from
'aa'
at org.apache.cassandra.db.marshal.UUIDType.fromString(UUIDType.java:118)
~[apache-cassandra-3.9.jar:3.9]
at
org.apache.cassandra.index.sasi.analyzer.StandardAnalyzer.hasNext(StandardAnalyzer.java:168)
~[apache-cassandra-3.9.jar:3.9]
at
org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter$Index.add(PerSSTableIndexWriter.java:208)
~[apache-cassandra-3.9.jar:3.9]
at
org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter.lambda$nextUnfilteredCluster$0(PerSSTableIndexWriter.java:132)
~[apache-cassandra-3.9.jar:3.9]
at java.util.Collections$SingletonSet.forEach(Collections.java:4767)
~[na:1.8.0_131]
at
org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter.nextUnfilteredCluster(PerSSTableIndexWriter.java:119)
~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.ColumnIndex.lambda$add$1(ColumnIndex.java:233)
~[apache-cassandra-3.9.jar:3.9]
at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_131]
at org.apache.cassandra.db.ColumnIndex.add(ColumnIndex.java:233)
~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.ColumnIndex.buildRowIndex(ColumnIndex.java:107)
~[apache-cassandra-3.9.jar:3.9]
at
org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:169)
~[apache-cassandra-3.9.jar:3.9]
at
org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:48)
~[apache-cassandra-3.9.jar:3.9]
at
org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:458)
~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.Memtable$FlushRunnable.call(Memtable.java:493)
~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.Memtable$FlushRunnable.call(Memtable.java:380)
~[apache-cassandra-3.9.jar:3.9]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_131]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_131]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_131]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
Exception (java.lang.RuntimeException) encountered during startup:
java.util.concurrent.ExecutionException: java.lang.RuntimeException:
java.util.concurrent.ExecutionException:
org.apache.cassandra.serializers.MarshalException: Unable to make UUID from
'aa'
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
org.apache.cassandra.serializers.MarshalException: Unable to make UUID from
'aa'
at org.apache.cassandra.utils.Throwables.maybeFail(Throwables.java:51)
ERROR 09:09:20 Exception encountered during startup
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
org.apache.cassandra.serializers.MarshalException: Unable to make UUID from
'aa'
at org.apache.cassandra.utils.Throwables.maybeFail(Throwables.java:51)
~[apache-cassandra-3.9.jar:3.9]
at
org.apache.cassandra.utils.FBUtilities.waitOnFutures(FBUtilities.java:391)
~[apache-cassandra-3.9.jar:3.9]
at
org.apache.cassandra.db.commitlog.CommitLogReplayer.blockForWrites(CommitLogReplayer.java:168)
~[apache-cassandra-3.9.jar:3.9]
at
org.apache.cassandra.db.commitlog.CommitLog.recoverFiles(CommitLog.java:188)
~[apache-cassandra-3.9.jar:3.9]
at
org.apache.cassandra.db.commitlog.CommitLog.recoverSegmentsOnDisk(CommitLog.java:167)
~[apache-cassandra-3.9.jar:3.9]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:323)
[apache-cassandra-3.9.jar:3.9]
at