Mix and match SASI with standard indexes not allowed?

[Voytek Jarnot]

Scenario (running 3.9, by the way):

CREATE TABLE atc_test1.idxtest (
pk text PRIMARY KEY,
col1 text,
col2 text);
CREATE CUSTOM INDEX idx2 ON atc_test1.idxtest (col2) USING
'org.apache.cassandra.index.sasi.SASIIndex';
CREATE INDEX idx1 ON atc_test1.idxtest (col1);

Queries:

Works: select * from idxtest where col1='asdf';
Works: select * from idxtest where col2='asdf';

Does not work: select * from idxtest where col1='asdf' and col2='asdf'
allow filtering;

Cassandra logs the following:
java.lang.ClassCastException:
org.apache.cassandra.index.internal.composites.RegularColumnIndex cannot be
cast to org.apache.cassandra.index.sasi.SASIIndex

I'm guessing the takeaway is that once you go SASI you're committed to only
SASI?  Unfortunately - in our case - the non-SASI index is in place because
one can't do a SASI index on a map column.

Questions:

Anyone else encounter this?  Any workarounds you're willing to share?

Seems like throwing a runtime exception is not a terrific way of handling
this... makes me wonder if there's something amiss with my particular
instance/config or if this is simply the way it is...

Re: unsubscribe

[SmartCat - Scott Hirleman]

That doesn't work. You need to send an email to
user-unsubscr...@cassandra.apache.org

On Thu, Oct 27, 2016 at 11:22 AM, Anseh Danesharasteh <
anseh.dan...@gmail.com> wrote:

> unsubscribe
>



-- 
*Scott Hirleman*
*Head of US Marketing and Sales*
www.smartcat.io
https://github.com/smartcat-labs 

Re: cqlsh fails to connect

[Michael Shuler]

It may be possible that you were using the old version of cqlsh? `which
cqlsh` on your upgraded nodes might point to the old install path, or a
copied version somewhere in your $PATH, perhaps.

Doing a fresh install and checking was a good idea, and it does show
that using the current version should be working OK.

-- 
Michael

On 10/28/2016 02:13 PM, Ioannis Zafiropoulos wrote:
> Ok, I tried with a new empty one node cluster of the same DSE version
> and cqlsh works without hiccups.
> So, the whole issue exists because I upgraded from Cassandra 2.1.11.
> 
> The procedure I followed for the upgrade was very simple:
> - nodetool drain (on all nodes)
> - shutdown all nodes
> - Uncompressed the DSE tarball os new version in a new path
> - modified new cassandra.conf
> - started all nodes
> - nodetool upgradesstables 
> 
> The cluster(the one with the problematic cqlsh)  is up and running
> without problems, I am able to connect with DBeaver and via Java.
> What could have gone wrong so that the latest python drivers (3.7.1 &
> 3.6.0) will not let my connect from python?
> 
> Thanks
> 
> On Fri, Oct 28, 2016 at 12:50 PM, Ioannis Zafiropoulos
> > wrote:
> 
> Hi Rajesh,
> 
> I just tried python 2.711 & 2.7.12 and I get the same error 'invalid
> continuation byte'.
> 
> On Fri, Oct 28, 2016 at 11:53 AM, Rajesh Radhakrishnan
>  > wrote:
> 
> 
> Hi John Z,
> 
> Did you tried running with latest Python 2.7.11 or 2.7.12?
> 
> Kind regards,
> Rajesh Radhakrishnan 
> 
> 
> 
> *From:* Ioannis Zafiropoulos [john...@gmail.com
> ]
> *Sent:* 27 October 2016 22:16
> *To:* user@cassandra.apache.org 
> *Subject:* cqlsh fails to connect
> 
> I upgraded DSE 4.8.9 to 5.0.3, that is, from Cassandra 2.1.11 to
> 3.0.9
> I used DSE 5.0.3 tarball installation. Cassandra cluster is up
> and running OK and I am able to connect through DBeaver.
> 
> Tried a lot of things and cannot connect with cqlsh:
> 
> Connection error: ('Unable to connect to any servers',
> {'x.x.x.x': UnicodeDecodeError('utf8',
> '+\x00\x00H\x08\x00\xf0$+\x00\x00\x10\x7f-\xe7S+\x00\x00`B\xb3\xe5S',
> 6, 7, 'invalid continuation byte')})
> 
> Versions
> 
> $ pip freeze | grep cas
> cassandra-driver==3.6.0
> cassandra-driver-dse==1.0.3
> 
> $ python
> Python 2.7.5 (default, Nov 20 2015, 02:00:19)
> [GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
> Type "help", "copyright", "credits" or "license" for more
> information.
> 
> $ cat /etc/redhat-release
> CentOS Linux release 7.2.1511 (Core)
> 
> $ which cqlsh (DSE's cqlsh client)
> /opt/dse/bin/cqlsh
> 
> Tried also
> $ export CQLSH_NO_BUNDLED=false
> 
> Also tried
> --
> Tried to install via pip in a fresh clean box the cqlsh client.
> I ended up with the latest cassandra-driver 3.7.1
> 
> $ pip freeze | grep cas
> cassandra-driver==3.7.1
> 
> $ pip freeze | grep cql
> cql==1.4.0
> cqlsh==5.0.3
> 
> And got this:
> Connection error: ('Unable to connect to any servers',
> {'x.x.x.x': ProtocolError("cql_version '3.3.1' is not supported
> by remote (w/ native protocol). Supported versions: [u'3.4.0']",)})
> 
> Tried to force it:
> $ cqlsh XX.XX.XX.XX --cqlversion="3.4.0"
> Got the original-first error message:
> Connection error: ('Unable to connect to any servers',
> {'x.x.x.x': UnicodeDecodeError('utf8',
> '+\x00\x00H\x08\x00\xf0$+\x00\x00\x10\x7f-\xe7S+\x00\x00`B\xb3\xe5S',
> 6, 7, 'invalid continuation byte')})
> 
> At some point I got this message too, but I don't remember what
> I did:
> Connection error: ('Unable to connect to any servers',
> {'x.x.x.x': DriverException(u'Failed decoding result column
> "table_name" of type varchar: ',)})
> 
> Thank you in advance for you help,
> John Z
> 
> 
> 
> 
> 
> 
> **
> The information contained in the EMail and any attachments is
> confidential and intended solely and for the attention and use
> of the named addressee(s). It may not be disclosed to any other
> person without the express authority of Public Health England,
> or the intended recipient, or both. If you are not the intended
> recipient, you must not disclose, copy, distribute or retain
> this 

Re: how to get the size of the particular partition key belonging to an sstable ??

[Edward Capriolo]

There are actually multiple tickets for different size functions. Examples
include computing size of collections, number of rows, and physical sizes
server side.

I also have a patch to make the warn and info settable at runtime.

https://issues.apache.org/jira/browse/CASSANDRA-12661?filter=-1

It is an ask that has come up multiple times now.

On Fri, Oct 28, 2016 at 3:40 PM, Jeff Jirsa 
wrote:

> See also  https://issues.apache.org/jira/browse/CASSANDRA-12367
>
>
>
>
>
>
>
> *From: *Justin Cameron 
> *Reply-To: *"user@cassandra.apache.org" 
> *Date: *Friday, October 28, 2016 at 12:35 PM
> *To: *"user@cassandra.apache.org" 
> *Subject: *Re: how to get the size of the particular partition key
> belonging to an sstable ??
>
>
>
>
>
> nodetool cfhistograms / nodetool tablehistograms will also output
> partition size statistics for a given table: http://docs.datastax.com/en/
> cassandra/3.0/cassandra/tools/toolsTablehisto.html
> 
>
>
>
>
>
> On Fri, 28 Oct 2016 at 12:32 Justin Cameron 
> wrote:
>
> If you're trying to determine this in order to diagnose wide row issues,
> then you can check your logs - Cassandra will log a warning for partitions
> > 100MB during compaction.
>
>
>
> See https://docs.datastax.com/en/cassandra/3.x/cassandra/
> configuration/configCassandra_yaml.html#configCassandra_
> yaml__compaction_large_partition_warning_threshold_mb
> 
>
>
>
>
>
> On Fri, 28 Oct 2016 at 01:49 Oleg Krayushkin  wrote:
>
> Hi,
>
>
>
> I guess it's about getting particular Partition Size on disk. If so, I
> would like to know this too.
>
>
>
> 2016-10-28 9:09 GMT+03:00 Vladimir Yudovin :
>
> Hi,
>
>
>
> >size of a particular partition key
>
> Can you please elucidate this? Key can be just number, or string, or
> several values.
>
>
>
> Best regards, Vladimir Yudovin,
>
>
> *Winguzone
> 
> - Hosted Cloud CassandraLaunch your cluster in minutes.*
>
>
>
>
>
>  On Thu, 27 Oct 2016 11:45:47 -0400*Pranay akula
> >* wrote 
>
>
>
> how can i get the size of a particular partition key belonging to an
> sstable ?? can we find it using index or summary or Statistics.db files ??
> does reading the hexdump of these files help ??
>
>
>
>
>
>
>
> Thanks
>
> Pranay.
>
>
>
>
>
>
>
> --
>
>
>
> Oleg Krayushkin
>
> --
>
> *Justin Cameron*
>
> Senior Software Engineer | Instaclustr
>
>
>
>
>
>
> This email has been sent on behalf of Instaclustr Pty Ltd (Australia) and
> Instaclustr Inc (USA).
>
> This email and any attachments may contain confidential and legally
> privileged information.  If you are not the intended recipient, do not copy
> or disclose its content, but please reply to this email immediately and
> highlight the error to the sender and then immediately delete the message.
>
>
>
> --
>
> *Justin Cameron*
>
> Senior Software Engineer | Instaclustr
>
>
>
>
>
>
> This email has been sent on behalf of Instaclustr Pty Ltd (Australia) and
> Instaclustr Inc (USA).
>
> This email and any attachments may contain confidential and legally
> privileged information.  If you are not the intended recipient, do not copy
> or disclose its content, but please reply to this email immediately and
> highlight the error to the sender and then immediately delete the message.
>
>
> CONFIDENTIALITY NOTE: This e-mail and any attachments are confidential and
> may be legally privileged. If you are not the intended recipient, do not
> disclose, copy, distribute, or use this email or any attachments. If you
> have received this in error please let the sender know and then delete the
> email and all attachments.
>

Re: how to get the size of the particular partition key belonging to an sstable ??

[Jeff Jirsa]

See also  https://issues.apache.org/jira/browse/CASSANDRA-12367

 

 

 

From: Justin Cameron 
Reply-To: "user@cassandra.apache.org" 
Date: Friday, October 28, 2016 at 12:35 PM
To: "user@cassandra.apache.org" 
Subject: Re: how to get the size of the particular partition key belonging to 
an sstable ??

 

 

nodetool cfhistograms / nodetool tablehistograms will also output partition 
size statistics for a given table: 
http://docs.datastax.com/en/cassandra/3.0/cassandra/tools/toolsTablehisto.html

 

 

On Fri, 28 Oct 2016 at 12:32 Justin Cameron  wrote:

If you're trying to determine this in order to diagnose wide row issues, then 
you can check your logs - Cassandra will log a warning for partitions > 100MB 
during compaction. 

 

See 
https://docs.datastax.com/en/cassandra/3.x/cassandra/configuration/configCassandra_yaml.html#configCassandra_yaml__compaction_large_partition_warning_threshold_mb

 

 

On Fri, 28 Oct 2016 at 01:49 Oleg Krayushkin  wrote:

Hi, 

 

I guess it's about getting particular Partition Size on disk. If so, I would 
like to know this too.

 

2016-10-28 9:09 GMT+03:00 Vladimir Yudovin :

Hi,

 

>size of a particular partition key

Can you please elucidate this? Key can be just number, or string, or several 
values. 

 

Best regards, Vladimir Yudovin, 

Winguzone - Hosted Cloud Cassandra
Launch your cluster in minutes.

 

 

 On Thu, 27 Oct 2016 11:45:47 -0400Pranay akula 
 wrote 

 

how can i get the size of a particular partition key belonging to an sstable ?? 
can we find it using index or summary or Statistics.db files ?? does reading 
the hexdump of these files help ??

 

 

 

Thanks

Pranay.

 



 

-- 

 

Oleg Krayushkin

-- 

Justin Cameron

Senior Software Engineer | Instaclustr




 


This email has been sent on behalf of Instaclustr Pty Ltd (Australia) and 
Instaclustr Inc (USA).

This email and any attachments may contain confidential and legally privileged 
information.  If you are not the intended recipient, do not copy or disclose 
its content, but please reply to this email immediately and highlight the error 
to the sender and then immediately delete the message.

 

-- 

Justin Cameron

Senior Software Engineer | Instaclustr




 


This email has been sent on behalf of Instaclustr Pty Ltd (Australia) and 
Instaclustr Inc (USA).

This email and any attachments may contain confidential and legally privileged 
information.  If you are not the intended recipient, do not copy or disclose 
its content, but please reply to this email immediately and highlight the error 
to the sender and then immediately delete the message.

 

CONFIDENTIALITY NOTE: This e-mail and any attachments are confidential and may 
be legally privileged. If you are not the intended recipient, do not disclose, 
copy, distribute, or use this email or any attachments. If you have received 
this in error please let the sender know and then delete the email and all 
attachments.


smime.p7s
Description: S/MIME cryptographic signature

Re: how to get the size of the particular partition key belonging to an sstable ??

[Justin Cameron]

nodetool cfhistograms / nodetool tablehistograms will also output partition
size statistics for a given table:
http://docs.datastax.com/en/cassandra/3.0/cassandra/tools/toolsTablehisto.html


On Fri, 28 Oct 2016 at 12:32 Justin Cameron  wrote:

> If you're trying to determine this in order to diagnose wide row issues,
> then you can check your logs - Cassandra will log a warning for partitions
> > 100MB during compaction.
>
> See
> https://docs.datastax.com/en/cassandra/3.x/cassandra/configuration/configCassandra_yaml.html#configCassandra_yaml__compaction_large_partition_warning_threshold_mb
>
>
> On Fri, 28 Oct 2016 at 01:49 Oleg Krayushkin  wrote:
>
> Hi,
>
> I guess it's about getting particular Partition Size on disk. If so, I
> would like to know this too.
>
> 2016-10-28 9:09 GMT+03:00 Vladimir Yudovin :
>
> Hi,
>
> >size of a particular partition key
> Can you please elucidate this? Key can be just number, or string, or
> several values.
>
> Best regards, Vladimir Yudovin,
>
> *Winguzone  - Hosted Cloud
> CassandraLaunch your cluster in minutes.*
>
>
>  On Thu, 27 Oct 2016 11:45:47 -0400*Pranay akula
> >* wrote 
>
> how can i get the size of a particular partition key belonging to an
> sstable ?? can we find it using index or summary or Statistics.db files ??
> does reading the hexdump of these files help ??
>
>
>
> Thanks
> Pranay.
>
>
>
>
>
> --
>
> Oleg Krayushkin
>
> --
>
> Justin Cameron
>
> Senior Software Engineer | Instaclustr
>
>
>
>
> This email has been sent on behalf of Instaclustr Pty Ltd (Australia) and
> Instaclustr Inc (USA).
>
> This email and any attachments may contain confidential and legally
> privileged information.  If you are not the intended recipient, do not copy
> or disclose its content, but please reply to this email immediately and
> highlight the error to the sender and then immediately delete the message.
>
> --

Justin Cameron

Senior Software Engineer | Instaclustr




This email has been sent on behalf of Instaclustr Pty Ltd (Australia) and
Instaclustr Inc (USA).

This email and any attachments may contain confidential and legally
privileged information.  If you are not the intended recipient, do not copy
or disclose its content, but please reply to this email immediately and
highlight the error to the sender and then immediately delete the message.

Re: how to get the size of the particular partition key belonging to an sstable ??

[Justin Cameron]

If you're trying to determine this in order to diagnose wide row issues,
then you can check your logs - Cassandra will log a warning for partitions
> 100MB during compaction.

See
https://docs.datastax.com/en/cassandra/3.x/cassandra/configuration/configCassandra_yaml.html#configCassandra_yaml__compaction_large_partition_warning_threshold_mb


On Fri, 28 Oct 2016 at 01:49 Oleg Krayushkin  wrote:

> Hi,
>
> I guess it's about getting particular Partition Size on disk. If so, I
> would like to know this too.
>
> 2016-10-28 9:09 GMT+03:00 Vladimir Yudovin :
>
> Hi,
>
> >size of a particular partition key
> Can you please elucidate this? Key can be just number, or string, or
> several values.
>
> Best regards, Vladimir Yudovin,
>
> *Winguzone  - Hosted Cloud
> CassandraLaunch your cluster in minutes.*
>
>
>  On Thu, 27 Oct 2016 11:45:47 -0400*Pranay akula
> >* wrote 
>
> how can i get the size of a particular partition key belonging to an
> sstable ?? can we find it using index or summary or Statistics.db files ??
> does reading the hexdump of these files help ??
>
>
>
> Thanks
> Pranay.
>
>
>
>
>
> --
>
> Oleg Krayushkin
>
-- 

Justin Cameron

Senior Software Engineer | Instaclustr




This email has been sent on behalf of Instaclustr Pty Ltd (Australia) and
Instaclustr Inc (USA).

This email and any attachments may contain confidential and legally
privileged information.  If you are not the intended recipient, do not copy
or disclose its content, but please reply to this email immediately and
highlight the error to the sender and then immediately delete the message.

Re: cqlsh fails to connect

[Ioannis Zafiropoulos]

Ok, I tried with a new empty one node cluster of the same DSE version and
cqlsh works without hiccups.
So, the whole issue exists because I upgraded from Cassandra 2.1.11.

The procedure I followed for the upgrade was very simple:
- nodetool drain (on all nodes)
- shutdown all nodes
- Uncompressed the DSE tarball os new version in a new path
- modified new cassandra.conf
- started all nodes
- nodetool upgradesstables

The cluster(the one with the problematic cqlsh)  is up and running without
problems, I am able to connect with DBeaver and via Java.
What could have gone wrong so that the latest python drivers (3.7.1 &
3.6.0) will not let my connect from python?

Thanks

On Fri, Oct 28, 2016 at 12:50 PM, Ioannis Zafiropoulos 
wrote:

> Hi Rajesh,
>
> I just tried python 2.711 & 2.7.12 and I get the same error 'invalid
> continuation byte'.
>
> On Fri, Oct 28, 2016 at 11:53 AM, Rajesh Radhakrishnan <
> rajesh.radhakrish...@phe.gov.uk> wrote:
>
>>
>> Hi John Z,
>>
>> Did you tried running with latest Python 2.7.11 or 2.7.12?
>>
>> Kind regards,
>> Rajesh Radhakrishnan
>>
>> --
>> *From:* Ioannis Zafiropoulos [john...@gmail.com]
>> *Sent:* 27 October 2016 22:16
>> *To:* user@cassandra.apache.org
>> *Subject:* cqlsh fails to connect
>>
>> I upgraded DSE 4.8.9 to 5.0.3, that is, from Cassandra 2.1.11 to 3.0.9
>> I used DSE 5.0.3 tarball installation. Cassandra cluster is up and
>> running OK and I am able to connect through DBeaver.
>>
>> Tried a lot of things and cannot connect with cqlsh:
>>
>> Connection error: ('Unable to connect to any servers', {'x.x.x.x':
>> UnicodeDecodeError('utf8', '+\x00\x00H\x08\x00\xf0$+\x00\
>> x00\x10\x7f-\xe7S+\x00\x00`B\xb3\xe5S', 6, 7, 'invalid continuation
>> byte')})
>>
>> Versions
>> 
>> $ pip freeze | grep cas
>> cassandra-driver==3.6.0
>> cassandra-driver-dse==1.0.3
>>
>> $ python
>> Python 2.7.5 (default, Nov 20 2015, 02:00:19)
>> [GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>>
>> $ cat /etc/redhat-release
>> CentOS Linux release 7.2.1511 (Core)
>>
>> $ which cqlsh (DSE's cqlsh client)
>> /opt/dse/bin/cqlsh
>>
>> Tried also
>> $ export CQLSH_NO_BUNDLED=false
>>
>> Also tried
>> --
>> Tried to install via pip in a fresh clean box the cqlsh client. I ended
>> up with the latest cassandra-driver 3.7.1
>>
>> $ pip freeze | grep cas
>> cassandra-driver==3.7.1
>>
>> $ pip freeze | grep cql
>> cql==1.4.0
>> cqlsh==5.0.3
>>
>> And got this:
>> Connection error: ('Unable to connect to any servers', {'x.x.x.x':
>> ProtocolError("cql_version '3.3.1' is not supported by remote (w/ native
>> protocol). Supported versions: [u'3.4.0']",)})
>>
>> Tried to force it:
>> $ cqlsh XX.XX.XX.XX --cqlversion="3.4.0"
>> Got the original-first error message:
>> Connection error: ('Unable to connect to any servers', {'x.x.x.x':
>> UnicodeDecodeError('utf8', '+\x00\x00H\x08\x00\xf0$+\x00\
>> x00\x10\x7f-\xe7S+\x00\x00`B\xb3\xe5S', 6, 7, 'invalid continuation
>> byte')})
>>
>> At some point I got this message too, but I don't remember what I did:
>> Connection error: ('Unable to connect to any servers', {'x.x.x.x':
>> DriverException(u'Failed decoding result column "table_name" of type
>> varchar: ',)})
>>
>> Thank you in advance for you help,
>> John Z
>>
>>
>>
>>
>>
>> 
>> **
>> The information contained in the EMail and any attachments is
>> confidential and intended solely and for the attention and use of the named
>> addressee(s). It may not be disclosed to any other person without the
>> express authority of Public Health England, or the intended recipient, or
>> both. If you are not the intended recipient, you must not disclose, copy,
>> distribute or retain this message or any part of it. This footnote also
>> confirms that this EMail has been swept for computer viruses by
>> Symantec.Cloud, but please re-sweep any attachments before opening or
>> saving. http://www.gov.uk/PHE
>> 
>> **
>>
>
>

Re: Tools to manage repairs

[Jonathan Haddad]

Vincent: currently big partitions, even if you're using paging & slicing by
clustering keys, will give you performance problems over time.  Please read
the JIRAs that Alex linked to, they provide in depth explanations as to
why, from some of the best Cassandra operators in the world :)

On Fri, Oct 28, 2016 at 9:50 AM Vincent Rischmann  wrote:

> Well I only asked that because I wanted to make sure that we're not doing
> it wrong, because that's actually how we query stuff,  we always provide a
> cluster key or a range of cluster keys.
>
> But yes, I understand that compactions may suffer and/or there may be
> hidden bottlenecks because of big partitions, so it's definitely good to
> know, and I'll definitely work on reducing partition sizes.
>
> On Fri, Oct 28, 2016, at 06:32 PM, Edward Capriolo wrote:
>
>
>
> On Fri, Oct 28, 2016 at 11:21 AM, Vincent Rischmann 
> wrote:
>
>
> Doesn't paging help with this ? Also if we select a range via the cluster
> key we're never really selecting the full partition. Or is that wrong ?
>
>
> On Fri, Oct 28, 2016, at 05:00 PM, Edward Capriolo wrote:
>
> Big partitions are an anti-pattern here is why:
>
> First Cassandra is not an analytic datastore. Sure it has some UDFs and
> aggregate UDFs, but the true purpose of the data store is to satisfy point
> reads. Operations have strict timeouts:
>
> # How long the coordinator should wait for read operations to complete
> read_request_timeout_in_ms: 5000
>
> # How long the coordinator should wait for seq or index scans to complete
> range_request_timeout_in_ms: 1
>
> This means you need to be able to satisfy the operation in 5 seconds.
> Which is not only the "think time" for 1 server, but if you are doing a
> quorum the operation has to complete and compare on 2 or more servers.
> Beyond these cutoffs are thread pools which fill up and start dropping
> requests once full.
>
> Something has to give, either functionality or physics. Particularly the
> physics of aggregating an ever-growing data set across N replicas in less
> than 5 seconds.  How many 2ms point reads will be blocked by 50 ms queries
> etc.
>
> I do not see the technical limitations of big partitions on disk is the
> only hurdle to climb here.
>
>
> On Fri, Oct 28, 2016 at 10:39 AM, Alexander Dejanovski <
> a...@thelastpickle.com> wrote:
>
> Hi Eric,
>
> that would be https://issues.apache.org/jira/browse/CASSANDRA-9754 by
> Michael Kjellman and https://issues.apache.org/jira/browse/CASSANDRA-11206 by
> Robert Stupp.
> If you haven't seen it yet, Robert's summit talk on big partitions is
> totally worth it :
> Video : https://www.youtube.com/watch?v=N3mGxgnUiRY
> Slides :
> http://www.slideshare.net/DataStax/myths-of-big-partitions-robert-stupp-datastax-cassandra-summit-2016
>
> Cheers,
>
>
> On Fri, Oct 28, 2016 at 4:09 PM Eric Evans 
> wrote:
>
> On Thu, Oct 27, 2016 at 4:13 PM, Alexander Dejanovski
>  wrote:
> > A few patches are pushing the limits of partition sizes so we may soon be
> > more comfortable with big partitions.
>
> You don't happen to have Jira links to these handy, do you?
>
>
>
> --
> Eric Evans
> john.eric.ev...@gmail.com
>
>
>
> --
> -
> Alexander Dejanovski
> France
> @alexanderdeja
>
> Consultant
> Apache Cassandra Consulting
> http://www.thelastpickle.com
>
>
>
>
> "Doesn't paging help with this ? Also if we select a range via the
> cluster key we're never really selecting the full partition. Or is that
> wrong ?"
>
> What I am suggestion is that the data store has had this practical
> limitation on size of partition since inception. As a result the common use
> case is not to use it in such a way. For example, the compaction manager
> may not be optimized for this cases, queries running across large
> partitions may cause more contention or lots of young gen garbage , queries
> running across large partitions may occupy the slots of the read stage etc.
>
>
>
> http://mail-archives.apache.org/mod_mbox/cassandra-user/201602.mbox/%3CCAJjpQyTS2eaCcRBVa=zmm-hcbx5nf4ovc1enw+sffgwvngo...@mail.gmail.com%3E
>
> I think there is possibly some more "little details" to discover. Not in a
> bad thing. I just do not think it you can hand-waive like a specific thing
> someone is working on now or paging solves it. If it was that easy it would
> be solved by now :)
>
>
>

Re: Slow performance after upgrading from 2.0.9 to 2.1.11

[Paulo Motta]

Haven't seen this before, but perhaps it's related to CASSANDRA-10433? This
is just a wild guess as it's in a related codepath, but maybe worth trying
out the patch available to see if it helps anything...

2016-10-28 15:03 GMT-02:00 Dikang Gu :

> We are seeing huge cpu regression when upgrading one of our 2.0.16 cluster
> to 2.1.14 as well. The 2.1.14 node is not able to handle the same amount of
> read traffic as the 2.0.16 node, actually, it's less than 50%.
>
> And in the perf results, the first line could go as high as 50%, as we
> turn up the read traffic, which never appeared in 2.0.16.
>
> Any thoughts?
> Thanks
>
>
> Samples: 952K of event 'cycles', Event count (approx.): 229681774560
> Overhead  Shared Object  Symbol
>6.52%  perf-196410.map[.]
> Lorg/apache/cassandra/db/marshal/IntegerType;.compare in
> Lorg/apache/cassandra/db/composites/AbstractSimpleCellNameType;.compare
>4.84%  libzip.so  [.] adler32
>2.88%  perf-196410.map[.]
> Ljava/nio/HeapByteBuffer;.get in Lorg/apache/cassandra/db/marsh
> al/IntegerType;.compare
>2.39%  perf-196410.map[.]
> Ljava/nio/Buffer;.checkIndex in Lorg/apache/cassandra/db/marsh
> al/IntegerType;.findMostSignificantByte
>2.03%  perf-196410.map[.]
> Ljava/math/BigInteger;.compareTo in Lorg/apache/cassandra/db/Decor
> atedKey;.compareTo
>1.65%  perf-196410.map[.] vtable chunks
>1.44%  perf-196410.map[.]
> Lorg/apache/cassandra/db/DecoratedKey;.compareTo in
> Ljava/util/concurrent/ConcurrentSkipListMap;.findNode
>1.02%  perf-196410.map[.]
> Lorg/apache/cassandra/db/composites/AbstractSimpleCellNameType;.compare
>1.00%  snappy-1.0.5.2-libsnappyjava.so[.] 0x3804
>0.87%  perf-196410.map[.]
> Ljava/io/DataInputStream;.readFully in Lorg/apache/cassandra/db/Abstr
> actCell$1;.computeNext
>0.82%  snappy-1.0.5.2-libsnappyjava.so[.] 0x36dc
>0.79%  [kernel]   [k]
> copy_user_generic_string
>0.73%  perf-196410.map[.] vtable chunks
>0.71%  perf-196410.map[.]
> Lorg/apache/cassandra/db/OnDiskAtom$Serializer;.deserializeFromSSTable in
> Lorg/apache/cassandra/db/AbstractCell$1;.computeNext
>0.70%  [kernel]   [k] find_busiest_group
>0.69%  perf-196410.map[.] <80>H3^?
>0.68%  perf-196410.map[.]
> Lorg/apache/cassandra/db/DecoratedKey;.compareTo
>0.65%  perf-196410.map[.]
> jbyte_disjoint_arraycopy
>0.64%  [kernel]   [k] _raw_spin_lock
>0.63%  [kernel]   [k] __schedule
>0.45%  snappy-1.0.5.2-libsnappyjava.so[.] 0x36df
>
> On Fri, Jan 29, 2016 at 2:11 PM, Corry Opdenakker 
> wrote:
>
>> @JC, Get the pid of your target java process (something like "ps -ef |
>> grep -i cassandra") .
>> Then do a kill -3  (at unix/linux)
>> Check the stdout logfile of the process.
>>  it should contain the threaddump.
>> If you found it, then great!
>> Let that kill -3 loop for about 2 or 3 minutes.
>> Herafter copy paste and load the stdout file into one if the mentioned
>> tools.
>> If you are not familiar with the java internals, then those threaddumps
>> will learn you a lot:)
>>
>>
>>
>>
>> Op vrijdag 29 januari 2016 heeft Jean Carlo 
>> het volgende geschreven:
>>
>>> I am having the same issue after upgrade cassandra 2.1.12 from 2.0.10. I
>>> am not good on jvm so I would like to know how to do what @
>>> CorryOpdenakker  propose with cassandra.
>>>
>>> :)
>>>
>>> I check concurrent_compactors
>>>
>>>
>>> Saludos
>>>
>>> Jean Carlo
>>>
>>> "The best way to predict the future is to invent it" Alan Kay
>>>
>>> On Fri, Jan 29, 2016 at 9:24 PM, Corry Opdenakker 
>>> wrote:
>>>
 Hi guys,
 Cassandra is still new for me, but I have a lot of java tuning
 experience.

 For root cause detection of performance degradations its always good to
 start with collecting a series of java thread dumps. Take at problem
 occurrence using a loopscript for example 60 thread dumps with an interval
 of 1 or 2 seconds.
 Then load those dumps into IBM thread dump analyzer or in "eclipse mat"
 or any similar tool and see which methods appear to be most active or
 blocking others.

 Its really very useful

 Same can be be done in a normal situation to compare the difference.

 That should give more insights.

 Cheers, Corry


 Op vrijdag 29 januari 2016 heeft Peddi, Praveen  het
 volgende geschreven:

> Hello,
> 

Upgrade from 3.6 to 3.9

[Gábor Auth]

Hi,

Anyone been updated Cassandra to version 3.9 on live system?

I've updated our two DC's Cassandra cluster from 3.6 to 3.9 and on the new
nodes I experienced relative huge CPU and memory footprint during the
handshake with the old nodes (sometimes run out of memory:
OutOfMemoryError: Java heap space).

Bye,
Gábor Auth

Re: Slow performance after upgrading from 2.0.9 to 2.1.11

[Dikang Gu]

We are seeing huge cpu regression when upgrading one of our 2.0.16 cluster
to 2.1.14 as well. The 2.1.14 node is not able to handle the same amount of
read traffic as the 2.0.16 node, actually, it's less than 50%.

And in the perf results, the first line could go as high as 50%, as we turn
up the read traffic, which never appeared in 2.0.16.

Any thoughts?
Thanks


Samples: 952K of event 'cycles', Event count (approx.): 229681774560
Overhead  Shared Object  Symbol
   6.52%  perf-196410.map[.]
Lorg/apache/cassandra/db/marshal/IntegerType;.compare in
Lorg/apache/cassandra/db/composites/AbstractSimpleCellNameType;.compare
   4.84%  libzip.so  [.] adler32
   2.88%  perf-196410.map[.]
Ljava/nio/HeapByteBuffer;.get in
Lorg/apache/cassandra/db/marshal/IntegerType;.compare
   2.39%  perf-196410.map[.]
Ljava/nio/Buffer;.checkIndex in
Lorg/apache/cassandra/db/marshal/IntegerType;.findMostSignificantByte
   2.03%  perf-196410.map[.]
Ljava/math/BigInteger;.compareTo in
Lorg/apache/cassandra/db/DecoratedKey;.compareTo
   1.65%  perf-196410.map[.] vtable chunks
   1.44%  perf-196410.map[.]
Lorg/apache/cassandra/db/DecoratedKey;.compareTo in
Ljava/util/concurrent/ConcurrentSkipListMap;.findNode
   1.02%  perf-196410.map[.]
Lorg/apache/cassandra/db/composites/AbstractSimpleCellNameType;.compare
   1.00%  snappy-1.0.5.2-libsnappyjava.so[.] 0x3804
   0.87%  perf-196410.map[.]
Ljava/io/DataInputStream;.readFully in
Lorg/apache/cassandra/db/AbstractCell$1;.computeNext
   0.82%  snappy-1.0.5.2-libsnappyjava.so[.] 0x36dc
   0.79%  [kernel]   [k]
copy_user_generic_string
   0.73%  perf-196410.map[.] vtable chunks
   0.71%  perf-196410.map[.]
Lorg/apache/cassandra/db/OnDiskAtom$Serializer;.deserializeFromSSTable in
Lorg/apache/cassandra/db/AbstractCell$1;.computeNext
   0.70%  [kernel]   [k] find_busiest_group
   0.69%  perf-196410.map[.] <80>H3^?
   0.68%  perf-196410.map[.]
Lorg/apache/cassandra/db/DecoratedKey;.compareTo
   0.65%  perf-196410.map[.]
jbyte_disjoint_arraycopy
   0.64%  [kernel]   [k] _raw_spin_lock
   0.63%  [kernel]   [k] __schedule
   0.45%  snappy-1.0.5.2-libsnappyjava.so[.] 0x36df

On Fri, Jan 29, 2016 at 2:11 PM, Corry Opdenakker  wrote:

> @JC, Get the pid of your target java process (something like "ps -ef |
> grep -i cassandra") .
> Then do a kill -3  (at unix/linux)
> Check the stdout logfile of the process.
>  it should contain the threaddump.
> If you found it, then great!
> Let that kill -3 loop for about 2 or 3 minutes.
> Herafter copy paste and load the stdout file into one if the mentioned
> tools.
> If you are not familiar with the java internals, then those threaddumps
> will learn you a lot:)
>
>
>
>
> Op vrijdag 29 januari 2016 heeft Jean Carlo 
> het volgende geschreven:
>
>> I am having the same issue after upgrade cassandra 2.1.12 from 2.0.10. I
>> am not good on jvm so I would like to know how to do what @
>> CorryOpdenakker  propose with cassandra.
>>
>> :)
>>
>> I check concurrent_compactors
>>
>>
>> Saludos
>>
>> Jean Carlo
>>
>> "The best way to predict the future is to invent it" Alan Kay
>>
>> On Fri, Jan 29, 2016 at 9:24 PM, Corry Opdenakker 
>> wrote:
>>
>>> Hi guys,
>>> Cassandra is still new for me, but I have a lot of java tuning
>>> experience.
>>>
>>> For root cause detection of performance degradations its always good to
>>> start with collecting a series of java thread dumps. Take at problem
>>> occurrence using a loopscript for example 60 thread dumps with an interval
>>> of 1 or 2 seconds.
>>> Then load those dumps into IBM thread dump analyzer or in "eclipse mat"
>>> or any similar tool and see which methods appear to be most active or
>>> blocking others.
>>>
>>> Its really very useful
>>>
>>> Same can be be done in a normal situation to compare the difference.
>>>
>>> That should give more insights.
>>>
>>> Cheers, Corry
>>>
>>>
>>> Op vrijdag 29 januari 2016 heeft Peddi, Praveen  het
>>> volgende geschreven:
>>>
 Hello,
 We have another update on performance on 2.1.11. compression_chunk_size
  didn’t really help much but We changed concurrent_compactors from default
 to 64 in 2.1.11 and read latencies improved significantly. However, 2.1.11
 read latencies are still 1.5 slower than 2.0.9. One thing we noticed in JMX
 metric that could affect read latencies is that 2.1.11 is running
 ReadRepairedBackground and ReadRepairedBlocking too 

Re: Tools to manage repairs

[Vincent Rischmann]

Well I only asked that because I wanted to make sure that we're not
doing it wrong, because that's actually how we query stuff,  we always
provide a cluster key or a range of cluster keys.

But yes, I understand that compactions may suffer and/or there may be
hidden bottlenecks because of big partitions, so it's definitely good to
know, and I'll definitely work on reducing partition sizes.

On Fri, Oct 28, 2016, at 06:32 PM, Edward Capriolo wrote:
>
>
> On Fri, Oct 28, 2016 at 11:21 AM, Vincent Rischmann
>  wrote:
>> __
>> Doesn't paging help with this ? Also if we select a range via the
>> cluster key we're never really selecting the full partition. Or is
>> that wrong ?
>>
>>
>> On Fri, Oct 28, 2016, at 05:00 PM, Edward Capriolo wrote:
>>> Big partitions are an anti-pattern here is why:
>>>
>>> First Cassandra is not an analytic datastore. Sure it has some UDFs
>>> and aggregate UDFs, but the true purpose of the data store is to
>>> satisfy point reads. Operations have strict timeouts:
>>>
>>> # How long the coordinator should wait for read operations to
>>> # complete
>>> read_request_timeout_in_ms: 5000
>>>
>>> # How long the coordinator should wait for seq or index scans to
>>> # complete
>>> range_request_timeout_in_ms: 1
>>>
>>> This means you need to be able to satisfy the operation in 5
>>> seconds. Which is not only the "think time" for 1 server, but if you
>>> are doing a quorum the operation has to complete and compare on 2 or
>>> more servers. Beyond these cutoffs are thread pools which fill up
>>> and start dropping requests once full.
>>>
>>> Something has to give, either functionality or physics. Particularly
>>> the physics of aggregating an ever-growing data set across N
>>> replicas in less than 5 seconds.  How many 2ms point reads will be
>>> blocked by 50 ms queries etc.
>>>
>>> I do not see the technical limitations of big partitions on disk is
>>> the only hurdle to climb here.
>>>
>>>
>>> On Fri, Oct 28, 2016 at 10:39 AM, Alexander Dejanovski
>>>  wrote:
 Hi Eric,

 that would be https://issues.apache.org/jira/browse/CASSANDRA-9754
 by Michael Kjellman and
 https://issues.apache.org/jira/browse/CASSANDRA-11206 by Robert
 Stupp.
 If you haven't seen it yet, Robert's summit talk on big partitions
 is totally worth it :
 Video : https://www.youtube.com/watch?v=N3mGxgnUiRY
 Slides :
 http://www.slideshare.net/DataStax/myths-of-big-partitions-robert-stupp-datastax-cassandra-summit-2016

 Cheers,


 On Fri, Oct 28, 2016 at 4:09 PM Eric Evans
  wrote:
> On Thu, Oct 27, 2016 at 4:13 PM, Alexander Dejanovski
>  wrote:
> > A few patches are pushing the limits of partition sizes so we
> > may soon be
> > more comfortable with big partitions.
>
> You don't happen to have Jira links to these handy, do you?
>
>
>
> --
>  Eric Evans john.eric.ev...@gmail.com
>


 --
 -
 Alexander Dejanovski
 France
 @alexanderdeja

 Consultant
 Apache Cassandra Consulting
 http://www.thelastpickle.com[1]


>>
>
> "Doesn't paging help with this ? Also if we select a range via the
> cluster key we're never really selecting the full partition. Or is
> that wrong ?"
>
> What I am suggestion is that the data store has had this practical
> limitation on size of partition since inception. As a result the
> common use case is not to use it in such a way. For example, the
> compaction manager may not be optimized for this cases, queries
> running across large partitions may cause more contention or lots of
> young gen garbage , queries running across large partitions may occupy
> the slots of the read stage etc.
>
>
> http://mail-archives.apache.org/mod_mbox/cassandra-user/201602.mbox/%3CCAJjpQyTS2eaCcRBVa=zmm-hcbx5nf4ovc1enw+sffgwvngo...@mail.gmail.com%3E
>
> I think there is possibly some more "little details" to discover. Not
> in a bad thing. I just do not think it you can hand-waive like a
> specific thing someone is working on now or paging solves it. If it
> was that easy it would be solved by now :)
>


Links:

  1. http://www.thelastpickle.com/

Re: cqlsh fails to connect

[Ioannis Zafiropoulos]

Hi Rajesh,

I just tried python 2.711 & 2.7.12 and I get the same error 'invalid
continuation byte'.

On Fri, Oct 28, 2016 at 11:53 AM, Rajesh Radhakrishnan <
rajesh.radhakrish...@phe.gov.uk> wrote:

>
> Hi John Z,
>
> Did you tried running with latest Python 2.7.11 or 2.7.12?
>
> Kind regards,
> Rajesh Radhakrishnan
>
> --
> *From:* Ioannis Zafiropoulos [john...@gmail.com]
> *Sent:* 27 October 2016 22:16
> *To:* user@cassandra.apache.org
> *Subject:* cqlsh fails to connect
>
> I upgraded DSE 4.8.9 to 5.0.3, that is, from Cassandra 2.1.11 to 3.0.9
> I used DSE 5.0.3 tarball installation. Cassandra cluster is up and running
> OK and I am able to connect through DBeaver.
>
> Tried a lot of things and cannot connect with cqlsh:
>
> Connection error: ('Unable to connect to any servers', {'x.x.x.x':
> UnicodeDecodeError('utf8', '+\x00\x00H\x08\x00\xf0$+\x00\
> x00\x10\x7f-\xe7S+\x00\x00`B\xb3\xe5S', 6, 7, 'invalid continuation
> byte')})
>
> Versions
> 
> $ pip freeze | grep cas
> cassandra-driver==3.6.0
> cassandra-driver-dse==1.0.3
>
> $ python
> Python 2.7.5 (default, Nov 20 2015, 02:00:19)
> [GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>
> $ cat /etc/redhat-release
> CentOS Linux release 7.2.1511 (Core)
>
> $ which cqlsh (DSE's cqlsh client)
> /opt/dse/bin/cqlsh
>
> Tried also
> $ export CQLSH_NO_BUNDLED=false
>
> Also tried
> --
> Tried to install via pip in a fresh clean box the cqlsh client. I ended up
> with the latest cassandra-driver 3.7.1
>
> $ pip freeze | grep cas
> cassandra-driver==3.7.1
>
> $ pip freeze | grep cql
> cql==1.4.0
> cqlsh==5.0.3
>
> And got this:
> Connection error: ('Unable to connect to any servers', {'x.x.x.x':
> ProtocolError("cql_version '3.3.1' is not supported by remote (w/ native
> protocol). Supported versions: [u'3.4.0']",)})
>
> Tried to force it:
> $ cqlsh XX.XX.XX.XX --cqlversion="3.4.0"
> Got the original-first error message:
> Connection error: ('Unable to connect to any servers', {'x.x.x.x':
> UnicodeDecodeError('utf8', '+\x00\x00H\x08\x00\xf0$+\x00\
> x00\x10\x7f-\xe7S+\x00\x00`B\xb3\xe5S', 6, 7, 'invalid continuation
> byte')})
>
> At some point I got this message too, but I don't remember what I did:
> Connection error: ('Unable to connect to any servers', {'x.x.x.x':
> DriverException(u'Failed decoding result column "table_name" of type
> varchar: ',)})
>
> Thank you in advance for you help,
> John Z
>
>
>
>
>
> **
> The information contained in the EMail and any attachments is confidential
> and intended solely and for the attention and use of the named
> addressee(s). It may not be disclosed to any other person without the
> express authority of Public Health England, or the intended recipient, or
> both. If you are not the intended recipient, you must not disclose, copy,
> distribute or retain this message or any part of it. This footnote also
> confirms that this EMail has been swept for computer viruses by
> Symantec.Cloud, but please re-sweep any attachments before opening or
> saving. http://www.gov.uk/PHE
> **
>

Re: Tools to manage repairs

[Edward Capriolo]

On Fri, Oct 28, 2016 at 11:21 AM, Vincent Rischmann 
wrote:

> Doesn't paging help with this ? Also if we select a range via the cluster
> key we're never really selecting the full partition. Or is that wrong ?
>
>
> On Fri, Oct 28, 2016, at 05:00 PM, Edward Capriolo wrote:
>
> Big partitions are an anti-pattern here is why:
>
> First Cassandra is not an analytic datastore. Sure it has some UDFs and
> aggregate UDFs, but the true purpose of the data store is to satisfy point
> reads. Operations have strict timeouts:
>
> # How long the coordinator should wait for read operations to complete
> read_request_timeout_in_ms: 5000
>
> # How long the coordinator should wait for seq or index scans to complete
> range_request_timeout_in_ms: 1
>
> This means you need to be able to satisfy the operation in 5 seconds.
> Which is not only the "think time" for 1 server, but if you are doing a
> quorum the operation has to complete and compare on 2 or more servers.
> Beyond these cutoffs are thread pools which fill up and start dropping
> requests once full.
>
> Something has to give, either functionality or physics. Particularly the
> physics of aggregating an ever-growing data set across N replicas in less
> than 5 seconds.  How many 2ms point reads will be blocked by 50 ms queries
> etc.
>
> I do not see the technical limitations of big partitions on disk is the
> only hurdle to climb here.
>
>
> On Fri, Oct 28, 2016 at 10:39 AM, Alexander Dejanovski <
> a...@thelastpickle.com> wrote:
>
> Hi Eric,
>
> that would be https://issues.apache.org/jira/browse/CASSANDRA-9754 by
> Michael Kjellman and https://issues.apache.org/jira/browse/CASSANDRA-11206 by
> Robert Stupp.
> If you haven't seen it yet, Robert's summit talk on big partitions is
> totally worth it :
> Video : https://www.youtube.com/watch?v=N3mGxgnUiRY
> Slides : http://www.slideshare.net/DataStax/myths-of-big-partitions
> -robert-stupp-datastax-cassandra-summit-2016
>
> Cheers,
>
>
> On Fri, Oct 28, 2016 at 4:09 PM Eric Evans 
> wrote:
>
> On Thu, Oct 27, 2016 at 4:13 PM, Alexander Dejanovski
>  wrote:
> > A few patches are pushing the limits of partition sizes so we may soon be
> > more comfortable with big partitions.
>
> You don't happen to have Jira links to these handy, do you?
>
>
> --
> Eric Evans
> john.eric.ev...@gmail.com
>
>
> --
> -
> Alexander Dejanovski
> France
> @alexanderdeja
>
> Consultant
> Apache Cassandra Consulting
> http://www.thelastpickle.com
>
>
>
"Doesn't paging help with this ? Also if we select a range via the cluster
key we're never really selecting the full partition. Or is that wrong ?"

What I am suggestion is that the data store has had this practical
limitation on size of partition since inception. As a result the common use
case is not to use it in such a way. For example, the compaction manager
may not be optimized for this cases, queries running across large
partitions may cause more contention or lots of young gen garbage , queries
running across large partitions may occupy the slots of the read stage etc.


http://mail-archives.apache.org/mod_mbox/cassandra-user/201602.mbox/%3CCAJjpQyTS2eaCcRBVa=zmm-hcbx5nf4ovc1enw+sffgwvngo...@mail.gmail.com%3E

I think there is possibly some more "little details" to discover. Not in a
bad thing. I just do not think it you can hand-waive like a specific thing
someone is working on now or paging solves it. If it was that easy it would
be solved by now :)

RE: cqlsh fails to connect

[Rajesh Radhakrishnan]

Hi John Z,

Did you tried running with latest Python 2.7.11 or 2.7.12?

Kind regards,
Rajesh Radhakrishnan


From: Ioannis Zafiropoulos [john...@gmail.com]
Sent: 27 October 2016 22:16
To: user@cassandra.apache.org
Subject: cqlsh fails to connect

I upgraded DSE 4.8.9 to 5.0.3, that is, from Cassandra 2.1.11 to 3.0.9
I used DSE 5.0.3 tarball installation. Cassandra cluster is up and running OK 
and I am able to connect through DBeaver.

Tried a lot of things and cannot connect with cqlsh:

Connection error: ('Unable to connect to any servers', {'x.x.x.x': 
UnicodeDecodeError('utf8', 
'+\x00\x00H\x08\x00\xf0$+\x00\x00\x10\x7f-\xe7S+\x00\x00`B\xb3\xe5S', 6, 7, 
'invalid continuation byte')})

Versions

$ pip freeze | grep cas
cassandra-driver==3.6.0
cassandra-driver-dse==1.0.3

$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

$ cat /etc/redhat-release
CentOS Linux release 7.2.1511 (Core)

$ which cqlsh (DSE's cqlsh client)
/opt/dse/bin/cqlsh

Tried also
$ export CQLSH_NO_BUNDLED=false

Also tried
--
Tried to install via pip in a fresh clean box the cqlsh client. I ended up with 
the latest cassandra-driver 3.7.1

$ pip freeze | grep cas
cassandra-driver==3.7.1

$ pip freeze | grep cql
cql==1.4.0
cqlsh==5.0.3

And got this:
Connection error: ('Unable to connect to any servers', {'x.x.x.x': 
ProtocolError("cql_version '3.3.1' is not supported by remote (w/ native 
protocol). Supported versions: [u'3.4.0']",)})

Tried to force it:
$ cqlsh XX.XX.XX.XX --cqlversion="3.4.0"
Got the original-first error message:
Connection error: ('Unable to connect to any servers', {'x.x.x.x': 
UnicodeDecodeError('utf8', 
'+\x00\x00H\x08\x00\xf0$+\x00\x00\x10\x7f-\xe7S+\x00\x00`B\xb3\xe5S', 6, 7, 
'invalid continuation byte')})

At some point I got this message too, but I don't remember what I did:
Connection error: ('Unable to connect to any servers', {'x.x.x.x': 
DriverException(u'Failed decoding result column "table_name" of type varchar: 
',)})

Thank you in advance for you help,
John Z





**
The information contained in the EMail and any attachments is confidential and 
intended solely and for the attention and use of the named addressee(s). It may 
not be disclosed to any other person without the express authority of Public 
Health England, or the intended recipient, or both. If you are not the intended 
recipient, you must not disclose, copy, distribute or retain this message or 
any part of it. This footnote also confirms that this EMail has been swept for 
computer viruses by Symantec.Cloud, but please re-sweep any attachments before 
opening or saving. http://www.gov.uk/PHE
**

Re: Tools to manage repairs

[Vincent Rischmann]

Doesn't paging help with this ? Also if we select a range via the
cluster key we're never really selecting the full partition. Or is
that wrong ?


On Fri, Oct 28, 2016, at 05:00 PM, Edward Capriolo wrote:
> Big partitions are an anti-pattern here is why:
>
> First Cassandra is not an analytic datastore. Sure it has some UDFs
> and aggregate UDFs, but the true purpose of the data store is to
> satisfy point reads. Operations have strict timeouts:
>
> # How long the coordinator should wait for read operations to complete
> read_request_timeout_in_ms: 5000
>
> # How long the coordinator should wait for seq or index scans to
> # complete
> range_request_timeout_in_ms: 1
>
> This means you need to be able to satisfy the operation in 5 seconds.
> Which is not only the "think time" for 1 server, but if you are doing
> a quorum the operation has to complete and compare on 2 or more
> servers. Beyond these cutoffs are thread pools which fill up and start
> dropping requests once full.
>
> Something has to give, either functionality or physics. Particularly
> the physics of aggregating an ever-growing data set across N replicas
> in less than 5 seconds.  How many 2ms point reads will be blocked by
> 50 ms queries etc.
>
> I do not see the technical limitations of big partitions on disk is
> the only hurdle to climb here.
>
>
> On Fri, Oct 28, 2016 at 10:39 AM, Alexander Dejanovski
>  wrote:
>> Hi Eric,
>>
>> that would be
>> https://issues.apache.org/jira/browse/CASSANDRA-9754 by
>> Michael Kjellman and
>> https://issues.apache.org/jira/browse/CASSANDRA-11206 by
>> Robert Stupp.
>> If you haven't seen it yet, Robert's summit talk on big partitions is
>> totally worth it :
>> Video : https://www.youtube.com/watch?v=N3mGxgnUiRY
>> Slides :
>> http://www.slideshare.net/DataStax/myths-of-big-partitions-robert-stupp-datastax-cassandra-summit-2016
>>
>> Cheers,
>>
>>
>> On Fri, Oct 28, 2016 at 4:09 PM Eric Evans
>>  wrote:
>>> On Thu, Oct 27, 2016 at 4:13 PM, Alexander Dejanovski
>>>   wrote:
>>>  > A few patches are pushing the limits of partition sizes so we may
>>>  > soon be
>>>  > more comfortable with big partitions.
>>>
>>>  You don't happen to have Jira links to these handy, do you?
>>>
>>>
>>>  --
>>>  Eric Evans john.eric.ev...@gmail.com
>>
>> --
>> -
>> Alexander Dejanovski
>> France
>> @alexanderdeja
>>
>> Consultant
>> Apache Cassandra Consulting
>> http://www.thelastpickle.com[1]
>>


Links:

  1. http://www.thelastpickle.com/

Re: Tools to manage repairs

[Edward Capriolo]

Big partitions are an anti-pattern here is why:

First Cassandra is not an analytic datastore. Sure it has some UDFs and
aggregate UDFs, but the true purpose of the data store is to satisfy point
reads. Operations have strict timeouts:

# How long the coordinator should wait for read operations to complete
read_request_timeout_in_ms: 5000

# How long the coordinator should wait for seq or index scans to complete
range_request_timeout_in_ms: 1

This means you need to be able to satisfy the operation in 5 seconds. Which
is not only the "think time" for 1 server, but if you are doing a quorum
the operation has to complete and compare on 2 or more servers. Beyond
these cutoffs are thread pools which fill up and start dropping requests
once full.

Something has to give, either functionality or physics. Particularly the
physics of aggregating an ever-growing data set across N replicas in less
than 5 seconds.  How many 2ms point reads will be blocked by 50 ms queries
etc.

I do not see the technical limitations of big partitions on disk is the
only hurdle to climb here.




On Fri, Oct 28, 2016 at 10:39 AM, Alexander Dejanovski <
a...@thelastpickle.com> wrote:

> Hi Eric,
>
> that would be https://issues.apache.org/jira/browse/CASSANDRA-9754 by
> Michael Kjellman and https://issues.apache.org/jira/browse/CASSANDRA-11206 by
> Robert Stupp.
> If you haven't seen it yet, Robert's summit talk on big partitions is
> totally worth it :
> Video : https://www.youtube.com/watch?v=N3mGxgnUiRY
> Slides : http://www.slideshare.net/DataStax/myths-of-big-
> partitions-robert-stupp-datastax-cassandra-summit-2016
>
> Cheers,
>
>
> On Fri, Oct 28, 2016 at 4:09 PM Eric Evans 
> wrote:
>
>> On Thu, Oct 27, 2016 at 4:13 PM, Alexander Dejanovski
>>  wrote:
>> > A few patches are pushing the limits of partition sizes so we may soon
>> be
>> > more comfortable with big partitions.
>>
>> You don't happen to have Jira links to these handy, do you?
>>
>>
>> --
>> Eric Evans
>> john.eric.ev...@gmail.com
>>
> --
> -
> Alexander Dejanovski
> France
> @alexanderdeja
>
> Consultant
> Apache Cassandra Consulting
> http://www.thelastpickle.com
>

Re: Tools to manage repairs

[Alexander Dejanovski]

Hi Eric,

that would be https://issues.apache.org/jira/browse/CASSANDRA-9754 by
Michael Kjellman and https://issues.apache.org/jira/browse/CASSANDRA-11206 by
Robert Stupp.
If you haven't seen it yet, Robert's summit talk on big partitions is
totally worth it :
Video : https://www.youtube.com/watch?v=N3mGxgnUiRY
Slides :
http://www.slideshare.net/DataStax/myths-of-big-partitions-robert-stupp-datastax-cassandra-summit-2016

Cheers,


On Fri, Oct 28, 2016 at 4:09 PM Eric Evans 
wrote:

> On Thu, Oct 27, 2016 at 4:13 PM, Alexander Dejanovski
>  wrote:
> > A few patches are pushing the limits of partition sizes so we may soon be
> > more comfortable with big partitions.
>
> You don't happen to have Jira links to these handy, do you?
>
>
> --
> Eric Evans
> john.eric.ev...@gmail.com
>
-- 
-
Alexander Dejanovski
France
@alexanderdeja

Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com

Re: Tools to manage repairs

[Eric Evans]

On Thu, Oct 27, 2016 at 4:13 PM, Alexander Dejanovski
 wrote:
> A few patches are pushing the limits of partition sizes so we may soon be
> more comfortable with big partitions.

You don't happen to have Jira links to these handy, do you?


-- 
Eric Evans
john.eric.ev...@gmail.com

Re: Cassandra failure during read query at consistency QUORUM (2 responses were required but only 0 replica responded, 2 failed)

[Edward Capriolo]

This looks like another case of an assert bubbling through try catch that
don't catch assert

On Fri, Oct 28, 2016 at 6:30 AM, Denis Mikhaylov  wrote:

> Hi!
>
> We’re running Cassandra 3.9
>
> On the application side I see failed reads with this exception
> com.datastax.driver.core.exceptions.ReadFailureException: Cassandra
> failure during read query at consistency QUORUM (2 responses were required
> but only 0 replica responded, 2 failed)
>
> On the server side we see:
>
> WARN  [SharedPool-Worker-3] 2016-10-28 13:28:22,965
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread
> Thread[SharedPool-Worker-3,5,
> main]: {}
> java.lang.AssertionError: null
> at org.apache.cassandra.db.rows.BTreeRow.getCell(BTreeRow.java:212)
> ~[apache-cassandra-3.7.jar:3.7]
> at org.apache.cassandra.db.SinglePartitionReadCommand.
> canRemoveRow(SinglePartitionReadCommand.java:899)
> ~[apache-cassandra-3.7.jar:3.7]
> at org.apache.cassandra.db.SinglePartitionReadCommand.
> reduceFilter(SinglePartitionReadCommand.java:863)
> ~[apache-cassandra-3.7.jar:3.7]
> at org.apache.cassandra.db.SinglePartitionReadCommand.
> queryMemtableAndSSTablesInTimestampOrder(SinglePartitionReadCommand.java:748)
> ~[apache-cassan
> dra-3.7.jar:3.7]
> at org.apache.cassandra.db.SinglePartitionReadCommand.
> queryMemtableAndDiskInternal(SinglePartitionReadCommand.java:519)
> ~[apache-cassandra-3.7.jar:
> 3.7]
> at org.apache.cassandra.db.SinglePartitionReadCommand.
> queryMemtableAndDisk(SinglePartitionReadCommand.java:496)
> ~[apache-cassandra-3.7.jar:3.7]
> at org.apache.cassandra.db.SinglePartitionReadCommand.
> queryStorage(SinglePartitionReadCommand.java:358)
> ~[apache-cassandra-3.7.jar:3.7]
> at 
> org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:366)
> ~[apache-cassandra-3.7.jar:3.7]
> at org.apache.cassandra.db.ReadCommandVerbHandler.doVerb(
> ReadCommandVerbHandler.java:48) ~[apache-cassandra-3.7.jar:3.7]
> at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:64)
> ~[apache-cassandra-3.7.jar:3.7]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_102]
> at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorServ
> ice$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
> ~[apache-cassandra-
> 3.7.jar:3.7]
> at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorServ
> ice$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
> [apache
> -cassandra-3.7.jar:3.7]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> [apache-cassandra-3.7.jar:3.7]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_102]
>
> It’s only affect single table. Sadly both on test (3.9) and production
> (3.7) deployments of cassandra.
>
> What could be the problem? Please help.

Cassandra failure during read query at consistency QUORUM (2 responses were required but only 0 replica responded, 2 failed)

[Denis Mikhaylov]

Hi!

We’re running Cassandra 3.9

On the application side I see failed reads with this exception 
com.datastax.driver.core.exceptions.ReadFailureException: Cassandra failure 
during read query at consistency QUORUM (2 responses were required but only 0 
replica responded, 2 failed)

On the server side we see:

WARN  [SharedPool-Worker-3] 2016-10-28 13:28:22,965 
AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread 
Thread[SharedPool-Worker-3,5,
main]: {}
java.lang.AssertionError: null
at org.apache.cassandra.db.rows.BTreeRow.getCell(BTreeRow.java:212) 
~[apache-cassandra-3.7.jar:3.7]
at 
org.apache.cassandra.db.SinglePartitionReadCommand.canRemoveRow(SinglePartitionReadCommand.java:899)
 ~[apache-cassandra-3.7.jar:3.7]
at 
org.apache.cassandra.db.SinglePartitionReadCommand.reduceFilter(SinglePartitionReadCommand.java:863)
 ~[apache-cassandra-3.7.jar:3.7]
at 
org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndSSTablesInTimestampOrder(SinglePartitionReadCommand.java:748)
 ~[apache-cassan
dra-3.7.jar:3.7]
at 
org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDiskInternal(SinglePartitionReadCommand.java:519)
 ~[apache-cassandra-3.7.jar:
3.7]
at 
org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDisk(SinglePartitionReadCommand.java:496)
 ~[apache-cassandra-3.7.jar:3.7]
at 
org.apache.cassandra.db.SinglePartitionReadCommand.queryStorage(SinglePartitionReadCommand.java:358)
 ~[apache-cassandra-3.7.jar:3.7]
at 
org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:366) 
~[apache-cassandra-3.7.jar:3.7]
at 
org.apache.cassandra.db.ReadCommandVerbHandler.doVerb(ReadCommandVerbHandler.java:48)
 ~[apache-cassandra-3.7.jar:3.7]
at 
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:64) 
~[apache-cassandra-3.7.jar:3.7]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
~[na:1.8.0_102]
at 
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
 ~[apache-cassandra-
3.7.jar:3.7]
at 
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
 [apache
-cassandra-3.7.jar:3.7]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
[apache-cassandra-3.7.jar:3.7]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_102]

It’s only affect single table. Sadly both on test (3.9) and production (3.7) 
deployments of cassandra.

What could be the problem? Please help.

Re: How does the "batch" commit log sync works

[Benedict Elliott Smith]

That is the maximum length of time that queries may be batched together
for, not the minimum. If there is a break in the flow of queries for the
commit log, it will commit those outstanding immediately.  It will anyway
commit in clusters of commit log file size (default 32Mb).

I know the documentation used to disagree with itself in a few places, and
with actual behaviour, but I thought that had been fixed.  I suggest you
file a ticket if you find a mention that does not match this description.

Really the batch period is a near useless parameter.  If it were to be
honoured as a minimum, performance would decline due to the threading model
in Cassandra (and it will be years before this and memory management
improve enough to support that behaviour).

Conversely honouring it as a maximum is only possible for very small
values, just by nature of queueing theory.

I believe I proposed removing the parameter entirely some time ago, though
it is lost in the mists of time.

Anyway, many people do indeed use this commitlog mode
successfully, although it is by far less common than periodic mode.  This
behaviour does not mean your data is in anyway unsafe.

On Friday, 28 October 2016, Edward Capriolo  wrote:

> I mentioned during my Cassandra.yaml presentation at the summit that I
> never saw anyone use these settings. Things off by default are typically
> not highly not covered well by tests. It sounds like it is not working.
> Quick suggestion: go back in time maybe to a version like 1.2.X or 0.7 and
> see if it behaves like the yaml suggests it should.
>
> On Thu, Oct 27, 2016 at 11:48 PM, Hiroyuki Yamada  > wrote:
>
>> Hello Satoshi and the community,
>>
>> I am also using commitlog_sync for durability, but I have never
>> modified commitlog_sync_batch_window_in_ms parameter yet,
>> so I wondered if it is working or not.
>>
>> As Satoshi said, I also changed commitlog_sync_batch_window_in_ms (to
>> 1) and restarted C* and
>> issued some INSERT command.
>> But, it actually returned immediately right after issuing.
>>
>> So, it seems like the parameter is not working correctly.
>> Are we missing something ?
>>
>> Thanks,
>> Hiro
>>
>> On Thu, Oct 27, 2016 at 5:58 PM, Satoshi Hikida > > wrote:
>> > Hi, all.
>> >
>> > I have a question about "batch" commit log sync behavior with C* version
>> > 2.2.8.
>> >
>> > Here's what I have done:
>> >
>> > * set commitlog_sync to the "batch" mode as follows:
>> >
>> >> commitlog_sync: batch
>> >> commitlog_sync_batch_window_in_ms: 1
>> >
>> > * ran a script which inserts the data to a table
>> > * prepared a disk dedicated to store the commit logs
>> >
>> > According to the DataStax document, I expected that fsync is done once
>> in a
>> > batch window (one fsync per 10sec in this case) and writes issued within
>> > this batch window are blocked until fsync is completed.
>> >
>> > In my experiment, however, it seems that the write requests returned
>> almost
>> > immediately (within 300~400 ms).
>> >
>> > Am I misunderstanding something? If so, can someone give me any advices
>> as
>> > to the reason why C* behaves like this?
>> >
>> >
>> > I referred to this document:
>> > https://docs.datastax.com/en/cassandra/2.2/cassandra/configu
>> ration/configCassandra_yaml.html#configCassandra_yaml__
>> PerformanceTuningProps
>> >
>> > Regards,
>> > Satoshi
>> >
>>
>
>

Re: Property file snitch and unbalanced cluster

[Carlos Alonso]

Hi Robert,

What has happened is that you have now two datacenters in your cluster. The
way they replicate information will depend on your keyspace settings.

Regarding your process I don't think it is safe to do it that way. I'd
start off by decommissioning nodes 4 and 5 so that your cluster is back to
1 datacenter with 3 nodes and then add them sequentially again making sure
the configuration in the Snitch is the proper one.

Regards.

Carlos Alonso | Software Engineer | @calonso 

On 28 October 2016 at 10:06, Robert Sicoie  wrote:

> Hi guys,
>
> I have a cluster with 5 nodes, cassandra 3.0.5, RF=2, PropertyFileSnitch.
> Three of the nodes were added initially, DC=PRD. The other two were added
> more recently.
>
> The default for unknown nodes default=DC1:r1 (which is bad in my case, as
> it would have been nice to have PRD as default)
>
> As the three initial do not have the information about the later two in
> the properties file, they see node4 and node5 as in DC1:r1.
>
> I assume that:
>
>- 1-2-3 have mode data than 4 and 5 because all writes that come to
>1-2-3 are replicated only on 1-2-3 (as there is no schema for DC1) and this
>is why 1,2,3 have more data and the cluster is unbalanced.
>- all writes that come to 4 and 5 are replicated on 1,2,3,4 and 5 as
>the property file snitch config has info about the entire cluster on 4 and
>5.
>
> Is it save if I just update the network topology file on 1, 2, 3 and add
> lines about node 4 and 5, and then roll restart 4 and 5? Will the cluster
> balance itself? Currently 1, 2, 3 have about 16GB each and 4,5 about 6 GB
> each.
>
> Thanks,
>
> Robert
>

Re: how to get the size of the particular partition key belonging to an sstable ??

[Oleg Krayushkin]

Hi,

I guess it's about getting particular Partition Size on disk. If so, I
would like to know this too.

2016-10-28 9:09 GMT+03:00 Vladimir Yudovin :

> Hi,
>
> >size of a particular partition key
> Can you please elucidate this? Key can be just number, or string, or
> several values.
>
> Best regards, Vladimir Yudovin,
>
> *Winguzone  - Hosted Cloud
> CassandraLaunch your cluster in minutes.*
>
>
>  On Thu, 27 Oct 2016 11:45:47 -0400*Pranay akula
> >* wrote 
>
> how can i get the size of a particular partition key belonging to an
> sstable ?? can we find it using index or summary or Statistics.db files ??
> does reading the hexdump of these files help ??
>
>
>
> Thanks
> Pranay.
>
>
>


-- 

Oleg Krayushkin

Property file snitch and unbalanced cluster

[Robert Sicoie]

Hi guys,

I have a cluster with 5 nodes, cassandra 3.0.5, RF=2, PropertyFileSnitch.
Three of the nodes were added initially, DC=PRD. The other two were added
more recently.

The default for unknown nodes default=DC1:r1 (which is bad in my case, as
it would have been nice to have PRD as default)

As the three initial do not have the information about the later two in the
properties file, they see node4 and node5 as in DC1:r1.

I assume that:

   - 1-2-3 have mode data than 4 and 5 because all writes that come to
   1-2-3 are replicated only on 1-2-3 (as there is no schema for DC1) and this
   is why 1,2,3 have more data and the cluster is unbalanced.
   - all writes that come to 4 and 5 are replicated on 1,2,3,4 and 5 as the
   property file snitch config has info about the entire cluster on 4 and 5.

Is it save if I just update the network topology file on 1, 2, 3 and add
lines about node 4 and 5, and then roll restart 4 and 5? Will the cluster
balance itself? Currently 1, 2, 3 have about 16GB each and 4,5 about 6 GB
each.

Thanks,

Robert

Unsubscribe

[Kunal Gaikwad]

Unsubscribe

Regards,
Kunal Gaikwad

Re: how to get the size of the particular partition key belonging to an sstable ??

[Vladimir Yudovin]

Hi,



size of a particular partition key

Can you please elucidate this? Key can be just number, or string, or several 
values. 



Best regards, Vladimir Yudovin, 

Winguzone - Hosted Cloud Cassandra
Launch your cluster in minutes.





 On Thu, 27 Oct 2016 11:45:47 -0400Pranay akula 
pranay.akula2...@gmail.com wrote 




how can i get the size of a particular partition key belonging to an sstable ?? 
can we find it using index or summary or Statistics.db files ?? does reading 
the hexdump of these files help ??







Thanks

Pranay.

Re: Re : Generic keystore when enabling SSL

[Vladimir Yudovin]

Hi Jacob,



there is no problem to use the same certificate (whether issued by some 
authority or self signed) on all nodes until it's present in truststore. CN 
doesn't matter in this case, it can be any string you want. 

Would this impact client-to-node encryption

Nu, but clients should either add nodes certificate to their truststore or 
disable validation (each Cassandra driver does this in its own way).



Best regards, Vladimir Yudovin, 

Winguzone - Hosted Cloud Cassandra
Launch your cluster in minutes.





 On Thu, 27 Oct 2016 16:45:48 -0400Jacob Shadix 
jacobsha...@gmail.com wrote 




I am interested if anyone has taken this approach to share the same keystore 
across all the nodes with the 3rd party root/intermediate CA existing only in 
the truststore. If so, please share your experience and lessons learned. Would 
this impact client-to-node encryption as the certificates used in internode 
would not have the hostnames represented in CN?



-- Jacob Shadix 








On Wed, Sep 21, 2016 at 11:40 AM, sai krishnam raju potturi 
pskraj...@gmail.com wrote:

hi Evans;

   rather than having one individual certificate for every node, we are looking 
at getting one Comodo wild-card certificate, and importing that into the 
keystore. along with the intermediate CA provided by Comodo. As far as the 
trust-store is concerned, we are looking at importing the intermediate CA 
provided along with the signed wild-card cert by Comodo.



   So in this case we'll be having just one keystore (generic), and truststore 
we'll be copying to all the nodes. We've run into issues however, and are 
trying to iron that out. Interested to know if anybody in the community has 
taken a similar approach.

 

   We are pretty much going on the lines of following post by LastPickle 
http://thelastpickle.com/blog/2015/09/30/hardening-cassandra-step-by-step-part-1-server-to-server.html.
 Instead of creating our own CA, we are relying on Comodo.



thanks

Sai





On Wed, Sep 21, 2016 at 10:30 AM, Eric Evans john.eric.ev...@gmail.com 
wrote:

On Tue, Sep 20, 2016 at 12:57 PM, sai krishnam raju potturi
 pskraj...@gmail.com wrote:
  Due to the security policies in our company, we were asked to use 3rd 
party
  signed certs. Since we'll require to manage 100's of individual certs, we
  wanted to know if there is a work around with a generic keystore and
  truststore.
 
 Can you explain what you mean by "generic keystore"?  Are you looking
 to create keystores signed by a self-signed root CA (distributed via a

 truststore)?

 
 --
 Eric Evans
 john.eric.ev...@gmail.com