Re: cqlsh problem

2016-10-13 Thread Stefania Alborghetti 
CASSANDRA-10959  made
the control connection timeout be the same as the connect timeout. This
patch is available since 2.2.5, 3.0.3, 3.3.

On Fri, Oct 14, 2016 at 11:45 AM, joseph gao  wrote:

> I've found the problem, in cqlsh file, find all Cluster 's construct
> method like '  conn = Cluster(xxx,xxx). At the end, add
> parameter control_connection_timeout=float(_SOME_MS_VALUE_). As below:
> conn = Cluster(contact_points=(self.hostname,), port=self.port,
> cql_version=self.conn.cql_version,
>protocol_version=DEFAULT_PROTOCOL_VERSION,
>auth_provider=auth_provider,
>ssl_options=self.conn.ssl_options,
>load_balancing_policy=WhiteListRoundRobinPolicy([
> self.hostname]),
>control_connection_timeout=float(36000))
> this will fix the problem, thanks all.
>
> 2016-09-20 17:23 GMT+08:00 George Sigletos :
>
>> Apologies. The entire error stack:
>>
>> ERROR [SharedPool-Worker-5] 2016-09-20 11:23:20,039 ErrorMessage.java:251
>> - Unexpected exception during request
>> com.google.common.util.concurrent.UncheckedExecutionException:
>> java.lang.RuntimeException: 
>> org.apache.cassandra.exceptions.ReadTimeoutException:
>> Operation timed out - received only 2 responses.
>> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
>> ~[guava-16.0.jar:na]
>> at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
>> ~[guava-16.0.jar:na]
>> at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
>> ~[guava-16.0.jar:na]
>> at 
>> com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
>> ~[guava-16.0.jar:na]
>> at 
>> org.apache.cassandra.auth.PermissionsCache.getPermissions(PermissionsCache.java:72)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at org.apache.cassandra.auth.Auth.getPermissions(Auth.java:75)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at 
>> org.apache.cassandra.service.ClientState.authorize(ClientState.java:353)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at 
>> org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:251)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at 
>> org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:245)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at 
>> org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:229)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at 
>> org.apache.cassandra.cql3.statements.SelectStatement.checkAccess(SelectStatement.java:195)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at 
>> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:235)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at 
>> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:260)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at 
>> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:122)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at 
>> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:439)
>> [apache-cassandra-2.1.14.jar:2.1.14]
>> at 
>> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:335)
>> [apache-cassandra-2.1.14.jar:2.1.14]
>> at io.netty.channel.SimpleChannelInboundHandler.channelRead(Sim
>> pleChannelInboundHandler.java:105) [netty-all-4.0.23.Final.jar:4.
>> 0.23.Final]
>> at io.netty.channel.AbstractChannelHandlerContext.invokeChannel
>> Read(AbstractChannelHandlerContext.java:333)
>> [netty-all-4.0.23.Final.jar:4.0.23.Final]
>> at io.netty.channel.AbstractChannelHandlerContext.access$700(Ab
>> stractChannelHandlerContext.java:32) [netty-all-4.0.23.Final.jar:4.
>> 0.23.Final]
>> at io.netty.channel.AbstractChannelHandlerContext$8.run(Abstrac
>> tChannelHandlerContext.java:324) [netty-all-4.0.23.Final.jar:4.
>> 0.23.Final]
>> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown
>> Source) [na:1.7.0_79]
>> at org.apache.cassandra.concurrent.AbstractTracingAwareExecutor
>> Service$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
>> [apache-cassandra-2.1.14.jar:2.1.14]
>> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
>> [apache-cassandra-2.1.14.jar:2.1.14]
>> at java.lang.Thread.run(Unknown Source) [na:1.7.0_79]
>> Caused by: java.lang.RuntimeException: 
>> org.apache.cassandra.exceptions.ReadTimeoutException:
>> Operation timed out - received only 2 responses.
>> at org.apache.cassandra.auth.Auth.selectUser(Auth.java:276)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at org.apache.cassandra.auth.Auth.isSuperuser(Auth.java:97)
>> ~[apache-cassandra-2.1.14.jar:2.1.14]
>> at 
>> org.apache.cassandra.auth.AuthenticatedUser.isSuper(AuthenticatedUser.java:50)
>>

Re: are there any free Cassandra -> ElasticSearch connector / plugin ?

2016-10-13 Thread Jonathan Haddad 
Yep - exactly right Kurt.  I strongly doubt there can be over 100K LOC
changed across 1K files and not touch anything else as a side effect.

git diff --stat cassandra-3.0.9..HEAD
 1093 files changed, 98490 insertions(+), 16870 deletions(-)


On Thu, Oct 13, 2016 at 9:27 PM kurt Greaves  wrote:

> New features don't necessarily restrict bugs only to those features. (only
> in our dreams). Often features may touch on parts of the code that could
> cause issues for other parts of the system.
>
> To clarify, just because you don't use new features, doesn't mean you are
> free from the risk of their bugs.
>
> On 14 October 2016 at 00:23, Jonathan Haddad  wrote:
>
> If you're not using the features why use a release that nobody else (read:
> experienced users) is?
>
> What do you need in 3.x that's not available in 3.0?
>
> On Thu, Oct 13, 2016 at 5:23 PM Eric Ho  wrote:
>
> But if I'm not doing anything fancy w/ C* (i.e. don't use new features in
> 3.{2,4,6}) then I'll be fine, right ?
>
>
> -eric ho
>
>
> On Thu, Oct 13, 2016 at 5:09 PM, Jonathan Haddad 
> wrote:
>
> I listed my reasons, please check my previous email.
>
> On Thu, Oct 13, 2016 at 4:55 PM Eric Ho  wrote:
>
> Why 3.0.x ?  Why not use 3.2.x or 3.4.x ? or 3.6.x ?
> Shouldn't 3.6.x be more stable than say 3.2.x ?
>
>
> -eric ho
>
>
> On Thu, Oct 13, 2016 at 3:48 PM, Jonathan Haddad 
> wrote:
>
> Here's your basic options:
>
> 1. Triggers (avoid like the plague)
> 2. CDC (really new, tricky to avoid RF operations as is, probably avoid)
> 3. Do it in your app
> 4. Put Kafka in front of your data, write as many consumers as you want to
> write the data in as many ways as you want
>
> Also, how long have you been using Cassandra?  Unless you're comfortable
> rolling your own builds and merging in bugfixes from upstream, I really
> suggest using a 3.0.x release instead of a 3.7.
>
> 3.7 falls under the Tick Tock release cycle, which is almost completely
> untested in production by experienced operators.  In the cases where it has
> been tested, there have been numerous bugs found which I (and I think most
> people on this list) consider to be show stoppers.  Additionally, the Tick
> Tock release cycle puts the operator in the uncomfortable position of
> having to decide between upgrading to a new version with new features
> (probably new bugs) or back porting bug fixes from future versions
> themselves.There will never be a 3.7.1 release which fixes bugs in 3.7
> without adding new features.
>
> https://github.com/apache/cassandra/blob/trunk/NEWS.txt
>
> For new projects I recommend starting with the recently released 3.0.9.
>
> Assuming the project changes it's policy on releases (all signs point to
> yes), then by the time 4.0 rolls out a lot of the features which have been
> released in the 3.x series will have matured a bit, so it's very possible
> 4.0 will stabilize faster than the usual 6 months it takes for a major
> release.
>
> All that said, there's nothing wrong with doing compatibility & smoke tests
> against the latest 3.x release as well as 3.0 and reporting bugs back to
> the Apache Cassandra JIRA, I'm sure it would be greatly appreciated.
>
> https://issues.apache.org/jira/secure/Dashboard.jspa
>
> Jon
>
>
>
> On Thu, Oct 13, 2016 at 3:15 PM Eric Ho  wrote:
>
> Some suggested Elassandra.  But that is based on Cassandra 2.2.
> I would like to use Cassandra 3.7 and up...
>
>
>
> -eric ho
>
>
> On Thu, Oct 13, 2016 at 3:04 PM, vincent gromakowski <
> vincent.gromakow...@gmail.com> wrote:
>
> Elassandra
> https://github.com/vroyer/elassandra
>
> Le 14 oct. 2016 12:02 AM, "Eric Ho"  a écrit :
>
> I don't want to change my code to write into C* and then to ES.
> So, I'm looking for some sort of a sync tool that will sync my C* table
> into ES and it should be smart enough to avoid duplicates or gaps.
> Is there such a tool / plugin ?
> I'm using stock apache Cassandra 3.7.
> I know that some premium Cassandra has ES builtin or integrated but I
> can't afford premium right now...
> Thanks.
>
> -eric ho
>
>
>
>
>
>
>
> --
> Kurt Greaves
> k...@instaclustr.com
> www.instaclustr.com
>

Re: Repair in Multi Datacenter - Should you use -dc Datacenter repair or repair with -pr

2016-10-13 Thread kurt Greaves 
Don't do pr repairs when using incremental repair, you'll just end up with
loads of anti-compactions.

On 12 October 2016 at 19:11, Harikrishnan Pillai 
wrote:

> In my experience dc local repair node by node with
> Pr and par options is best .full repair increased sstables
> A lot and take days to compact it back or another
> Easy option for repair is use a spark job ,read all data with
> Consistency all and increase read repair chance to
> 100 % or use Netflix tickler
>
> Sent from my iPhone
>
> On Oct 12, 2016, at 11:44 AM, Anuj Wadehra  wrote:
>
> Hi Leena,
>
> First thing you should be concerned about is : Why the repair -pr
> operation doesnt complete ?
> Second comes the question : Which repair option is best?
>
>
> One probable cause of stuck repairs is : if the firewall between DCs is
> closing TCP connections and Cassandra is trying to use such connections,
> repairs will hang. Please refer https://docs.datastax.com/en/
> cassandra/2.0/cassandra/troubleshooting/trblshootIdleFirewall.html . We
> faced that.
>
> Also make sure you comply with basic bandwidth requirement between DCs.
> Recommended is 1000 Mb/s (1 gigabit) or greater.
>
> Answers for specific questions:
> 1.As per my understanding, all replicas will not participate in dc local
> repairs and thus repair would be ineffective. You need to make sure that
> all replicas of a data in all dcs are in sync.
>
> 2. Every DC is not a ring. All DCs together form a token ring. So, I think
> yes you should run repair -pr on all nodes.
>
> 3. Yes. I dont have experience with incremental repairs. But you can run
> repair -pr on all nodes of all DCs.
>
> Regarding Best approach of repair, you should see some repair
> presentations of Cassandra Summit 2016. All are online now.
>
> I attended the summit and people using large clusters generally use sub
> range repairs to repair their clusters. But such large deployments are on
> older Cassandra versions and these deployments generally dont use vnodes.
> So people know easily which nodes hold which token range.
>
>
>
> Thanks
> Anuj
>
> --
> *From: *Leena Ghatpande ;
> *To: *user@cassandra.apache.org ;
> *Subject: *Repair in Multi Datacenter - Should you use -dc Datacenter
> repair or repair with -pr
> *Sent: *Wed, Oct 12, 2016 2:15:51 PM
>
> Please advice. Cannot find any clear documentation on what is the best
> strategy for repairing nodes on a regular basis with multiple datacenters
> involved.
>
>
> We are running cassandra 3.7 in multi datacenter with 4 nodes in each data
> center. We are trying to run repairs every other night to keep the nodes in
> good state.We currently run repair with -pr option , but the repair process
> gets hung and does not complete gracefully. Dont see any errors in the logs
> either.
>
>
> What is the best way to perform repairs on multiple data centers on large
> tables.
>
> 1. Can we run Datacenter repair using -dc option for each data center? Do
> we need to run repair on each node in that case or will it repair all nodes
> within the datacenter?
>
> 2. Is running repair with -pr across all nodes required , if we perform
> the step 1 every night?
>
> 3. Is cross data center repair required and if so whats the best option?
>
>
> Thanks
>
>
> Leena
>
>
>
>


-- 
Kurt Greaves
k...@instaclustr.com
www.instaclustr.com

Re: are there any free Cassandra -> ElasticSearch connector / plugin ?

2016-10-13 Thread kurt Greaves 
New features don't necessarily restrict bugs only to those features. (only
in our dreams). Often features may touch on parts of the code that could
cause issues for other parts of the system.

To clarify, just because you don't use new features, doesn't mean you are
free from the risk of their bugs.

On 14 October 2016 at 00:23, Jonathan Haddad  wrote:

> If you're not using the features why use a release that nobody else (read:
> experienced users) is?
>
> What do you need in 3.x that's not available in 3.0?
>
> On Thu, Oct 13, 2016 at 5:23 PM Eric Ho  wrote:
>
>> But if I'm not doing anything fancy w/ C* (i.e. don't use new features in
>> 3.{2,4,6}) then I'll be fine, right ?
>>
>>
>> -eric ho
>>
>>
>> On Thu, Oct 13, 2016 at 5:09 PM, Jonathan Haddad 
>> wrote:
>>
>> I listed my reasons, please check my previous email.
>>
>> On Thu, Oct 13, 2016 at 4:55 PM Eric Ho  wrote:
>>
>> Why 3.0.x ?  Why not use 3.2.x or 3.4.x ? or 3.6.x ?
>> Shouldn't 3.6.x be more stable than say 3.2.x ?
>>
>>
>> -eric ho
>>
>>
>> On Thu, Oct 13, 2016 at 3:48 PM, Jonathan Haddad 
>> wrote:
>>
>> Here's your basic options:
>>
>> 1. Triggers (avoid like the plague)
>> 2. CDC (really new, tricky to avoid RF operations as is, probably avoid)
>> 3. Do it in your app
>> 4. Put Kafka in front of your data, write as many consumers as you want
>> to write the data in as many ways as you want
>>
>> Also, how long have you been using Cassandra?  Unless you're comfortable
>> rolling your own builds and merging in bugfixes from upstream, I really
>> suggest using a 3.0.x release instead of a 3.7.
>>
>> 3.7 falls under the Tick Tock release cycle, which is almost completely
>> untested in production by experienced operators.  In the cases where it
>> has
>> been tested, there have been numerous bugs found which I (and I think most
>> people on this list) consider to be show stoppers.  Additionally, the Tick
>> Tock release cycle puts the operator in the uncomfortable position of
>> having to decide between upgrading to a new version with new features
>> (probably new bugs) or back porting bug fixes from future versions
>> themselves.There will never be a 3.7.1 release which fixes bugs in 3.7
>> without adding new features.
>>
>> https://github.com/apache/cassandra/blob/trunk/NEWS.txt
>>
>> For new projects I recommend starting with the recently released 3.0.9.
>>
>> Assuming the project changes it's policy on releases (all signs point to
>> yes), then by the time 4.0 rolls out a lot of the features which have been
>> released in the 3.x series will have matured a bit, so it's very possible
>> 4.0 will stabilize faster than the usual 6 months it takes for a major
>> release.
>>
>> All that said, there's nothing wrong with doing compatibility & smoke
>> tests
>> against the latest 3.x release as well as 3.0 and reporting bugs back to
>> the Apache Cassandra JIRA, I'm sure it would be greatly appreciated.
>>
>> https://issues.apache.org/jira/secure/Dashboard.jspa
>>
>> Jon
>>
>>
>>
>> On Thu, Oct 13, 2016 at 3:15 PM Eric Ho  wrote:
>>
>> Some suggested Elassandra.  But that is based on Cassandra 2.2.
>> I would like to use Cassandra 3.7 and up...
>>
>>
>>
>> -eric ho
>>
>>
>> On Thu, Oct 13, 2016 at 3:04 PM, vincent gromakowski <
>> vincent.gromakow...@gmail.com> wrote:
>>
>> Elassandra
>> https://github.com/vroyer/elassandra
>>
>> Le 14 oct. 2016 12:02 AM, "Eric Ho"  a écrit :
>>
>> I don't want to change my code to write into C* and then to ES.
>> So, I'm looking for some sort of a sync tool that will sync my C* table
>> into ES and it should be smart enough to avoid duplicates or gaps.
>> Is there such a tool / plugin ?
>> I'm using stock apache Cassandra 3.7.
>> I know that some premium Cassandra has ES builtin or integrated but I
>> can't afford premium right now...
>> Thanks.
>>
>> -eric ho
>>
>>
>>
>>
>>


-- 
Kurt Greaves
k...@instaclustr.com
www.instaclustr.com

Re: what does this Note mean?

2016-10-13 Thread Jeff Jirsa 
Nodetool status / nodetool ring will show an estimated ownership. They estimate 
this by looking at the replication settings of one of the non-system keyspaces 
(presumably those that contain actual data).

 

If they all have the same replication settings (say, RF=3 in each DC using 
NetworkTopologyStrategy), then the ring/status ownership estimation can be 
reasonably certain it’s correct – whatever value it will calculate is true for 
all keyspaces.

If they differ, then it can print some ownership % from whatever keyspace it 
selects, but that % will be different than if you chose a different keyspace. 
In this case, pass the keyspace as the optional last argument to specify which 
keyspace matters to you.

 

-  Jeff

 

From: Kant Kodali 
Reply-To: "user@cassandra.apache.org" 
Date: Thursday, October 13, 2016 at 7:08 PM
To: "user@cassandra.apache.org" 
Subject: what does this Note mean?

 

 

Note: Non-system keyspaces don't have the same replication settings, effective 
ownership information is meaningless


CONFIDENTIALITY NOTE: This e-mail and any attachments are confidential and may 
be legally privileged. If you are not the intended recipient, do not disclose, 
copy, distribute, or use this email or any attachments. If you have received 
this in error please let the sender know and then delete the email and all 
attachments.


smime.p7s
Description: S/MIME cryptographic signature

Re: cqlsh problem

2016-10-13 Thread joseph gao 
I've found the problem, in cqlsh file, find all Cluster 's construct method
like '  conn = Cluster(xxx,xxx). At the end, add
parameter control_connection_timeout=float(_SOME_MS_VALUE_). As below:
conn = Cluster(contact_points=(self.hostname,), port=self.port,
cql_version=self.conn.cql_version,
   protocol_version=DEFAULT_PROTOCOL_VERSION,
   auth_provider=auth_provider,
   ssl_options=self.conn.ssl_options,

 load_balancing_policy=WhiteListRoundRobinPolicy([self.hostname]),
   control_connection_timeout=float(36000))
this will fix the problem, thanks all.

2016-09-20 17:23 GMT+08:00 George Sigletos :

> Apologies. The entire error stack:
>
> ERROR [SharedPool-Worker-5] 2016-09-20 11:23:20,039 ErrorMessage.java:251
> - Unexpected exception during request
> com.google.common.util.concurrent.UncheckedExecutionException:
> java.lang.RuntimeException: 
> org.apache.cassandra.exceptions.ReadTimeoutException:
> Operation timed out - received only 2 responses.
> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
> ~[guava-16.0.jar:na]
> at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
> ~[guava-16.0.jar:na]
> at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
> ~[guava-16.0.jar:na]
> at 
> com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
> ~[guava-16.0.jar:na]
> at 
> org.apache.cassandra.auth.PermissionsCache.getPermissions(PermissionsCache.java:72)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at org.apache.cassandra.auth.Auth.getPermissions(Auth.java:75)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.service.ClientState.authorize(ClientState.java:353)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:251)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:245)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:229)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at org.apache.cassandra.cql3.statements.SelectStatement.
> checkAccess(SelectStatement.java:195) ~[apache-cassandra-2.1.14.jar:
> 2.1.14]
> at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:235)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:260)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:122)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:439)
> [apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:335)
> [apache-cassandra-2.1.14.jar:2.1.14]
> at io.netty.channel.SimpleChannelInboundHandler.channelRead(
> SimpleChannelInboundHandler.java:105) [netty-all-4.0.23.Final.jar:4.
> 0.23.Final]
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(
> AbstractChannelHandlerContext.java:333) [netty-all-4.0.23.Final.jar:4.
> 0.23.Final]
> at io.netty.channel.AbstractChannelHandlerContext.access$700(
> AbstractChannelHandlerContext.java:32) [netty-all-4.0.23.Final.jar:4.
> 0.23.Final]
> at io.netty.channel.AbstractChannelHandlerContext$8.run(
> AbstractChannelHandlerContext.java:324) [netty-all-4.0.23.Final.jar:4.
> 0.23.Final]
> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown
> Source) [na:1.7.0_79]
> at org.apache.cassandra.concurrent.AbstractTracingAwareExecutorSe
> rvice$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
> [apache-cassandra-2.1.14.jar:2.1.14]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> [apache-cassandra-2.1.14.jar:2.1.14]
> at java.lang.Thread.run(Unknown Source) [na:1.7.0_79]
> Caused by: java.lang.RuntimeException: 
> org.apache.cassandra.exceptions.ReadTimeoutException:
> Operation timed out - received only 2 responses.
> at org.apache.cassandra.auth.Auth.selectUser(Auth.java:276)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at org.apache.cassandra.auth.Auth.isSuperuser(Auth.java:97)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.auth.AuthenticatedUser.isSuper(AuthenticatedUser.java:50)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.auth.CassandraAuthorizer.authorize(CassandraAuthorizer.java:67)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.auth.PermissionsCache$1.load(PermissionsCache.java:124)
> ~[apache-cassandra-2.1.14.jar:2.1.14]
> at 
> org.apache.cassandra.auth.PermissionsCache$1.load(PermissionsCache.java:121)
>

Re: what does this Note mean?

2016-10-13 Thread Sebastian Estevez 
It means for accurate load statistics add the keyspace name after nodetool
status.

nodetool status 




All the best,




On Thu, Oct 13, 2016 at 10:08 PM, Kant Kodali  wrote:

>
> Note: Non-system keyspaces don't have the same replication settings,
> effective ownership information is meaningless
>

what does this Note mean?

2016-10-13 Thread Kant Kodali 
Note: Non-system keyspaces don't have the same replication settings,
effective ownership information is meaningless

Re: Can not delete record with counter column in a batch.

2016-10-13 Thread Michael Shuler 
On 10/13/2016 08:11 PM, Alexei Levashov wrote:
> Previously I did it in a logged batch without any issues but upgrading
> from Cassandra 1.2.x to 2.1 brings the following Exception message:
> "Error executing batch. Message: Cannot include a counter statement in a
> logged batch"
> 
> Question: 
> It looks like exception thrown from this line:
> 
> if (isLogged() && statement.isCounter()) throw new
> InvalidRequestException("Cannot include a counter statement in a logged
> batch");
> 
> Why does DELETE statement for the table containing counter field return
> isCounter() == true? 
> Is it a bug or expected behavior?

This appears to be expected behavior.

If you look at blame on the line you linked, it was commit ee401cf8
which you can follow to the commit:

https://github.com/apache/cassandra/commit/ee401cf8131a779069805cbe9ef4ab05d4a63b9a

Which notes: CASSANDRA-7351
 https://issues.apache.org/jira/browse/CASSANDRA-7351

It looks like you may use UNLOGGED batch.

-- 
Kind regards,
Michael

Can not delete record with counter column in a batch.

2016-10-13 Thread Alexei Levashov 
Configuration : Cassandra 2.1
(if it matters Datastax Java driver 3.0.0 - driver users list couldn't help)
Platform Unix.

I have two tables:

CREATE TABLE unmapped_doc_type (
application_id uuid,
doc_type_id text,
metadata text,
PRIMARY KEY (application_id, doc_type_id)
)
CREATE TABLE unmapped_doc_type_counter (
application_id uuid,
doc_type_id text,
occurrences counter,
PRIMARY KEY (application_id, doc_type_id)
)

Table unmapped_doc_type has metadata for doc_type and
unmapped_doc_type_counter one has the number of occurences this doc type
was requested.
When I delete record in unmapped_doc_type I have to delete the same (same
values of application_id, doc_type_id) in unmapped_doc_type_counter table.
I am building DELETE statements using QueryBuilder, wrapping them into
com.datastax.driver.core.querybuilder.Batch as new Batch(statements, true)
and calling session.execute(batch) with this batch.

Previously I did it in a logged batch without any issues but upgrading from
Cassandra 1.2.x to 2.1 brings the following Exception message:
"Error executing batch. Message: Cannot include a counter statement in a
logged batch"
Sample of statement that fails:
"
BEGIN BATCH
DELETE FROM unmapped_doc_type
WHERE doc_type_id='567cfa05-aa41-4d4d-b65d-42dfd198ef2b' AND
application_id=41367e0c-5d6f-4b2f-897b-e25d6a52ee0e;
DELETE FROM unmapped_doc_type_counter
WHERE doc_type_id='567cfa05-aa41-4d4d-b65d-42dfd198ef2b' AND
application_id=41367e0c-5d6f-4b2f-897b-e25d6a52ee0e;
APPLY BATCH;
".

Question:
It looks like exception thrown from this line:

if (isLogged() && statement.isCounter()) throw new
InvalidRequestException("Cannot include a counter statement in a logged
batch");

Why does DELETE statement for the table containing counter field return
isCounter() == true?
Is it a bug or expected behavior?
Thank you,
Alexei.

Re: are there any free Cassandra -> ElasticSearch connector / plugin ?

2016-10-13 Thread Jonathan Haddad 
If you're not using the features why use a release that nobody else (read:
experienced users) is?

What do you need in 3.x that's not available in 3.0?

On Thu, Oct 13, 2016 at 5:23 PM Eric Ho  wrote:

> But if I'm not doing anything fancy w/ C* (i.e. don't use new features in
> 3.{2,4,6}) then I'll be fine, right ?
>
>
> -eric ho
>
>
> On Thu, Oct 13, 2016 at 5:09 PM, Jonathan Haddad 
> wrote:
>
> I listed my reasons, please check my previous email.
>
> On Thu, Oct 13, 2016 at 4:55 PM Eric Ho  wrote:
>
> Why 3.0.x ?  Why not use 3.2.x or 3.4.x ? or 3.6.x ?
> Shouldn't 3.6.x be more stable than say 3.2.x ?
>
>
> -eric ho
>
>
> On Thu, Oct 13, 2016 at 3:48 PM, Jonathan Haddad 
> wrote:
>
> Here's your basic options:
>
> 1. Triggers (avoid like the plague)
> 2. CDC (really new, tricky to avoid RF operations as is, probably avoid)
> 3. Do it in your app
> 4. Put Kafka in front of your data, write as many consumers as you want to
> write the data in as many ways as you want
>
> Also, how long have you been using Cassandra?  Unless you're comfortable
> rolling your own builds and merging in bugfixes from upstream, I really
> suggest using a 3.0.x release instead of a 3.7.
>
> 3.7 falls under the Tick Tock release cycle, which is almost completely
> untested in production by experienced operators.  In the cases where it has
> been tested, there have been numerous bugs found which I (and I think most
> people on this list) consider to be show stoppers.  Additionally, the Tick
> Tock release cycle puts the operator in the uncomfortable position of
> having to decide between upgrading to a new version with new features
> (probably new bugs) or back porting bug fixes from future versions
> themselves.There will never be a 3.7.1 release which fixes bugs in 3.7
> without adding new features.
>
> https://github.com/apache/cassandra/blob/trunk/NEWS.txt
>
> For new projects I recommend starting with the recently released 3.0.9.
>
> Assuming the project changes it's policy on releases (all signs point to
> yes), then by the time 4.0 rolls out a lot of the features which have been
> released in the 3.x series will have matured a bit, so it's very possible
> 4.0 will stabilize faster than the usual 6 months it takes for a major
> release.
>
> All that said, there's nothing wrong with doing compatibility & smoke tests
> against the latest 3.x release as well as 3.0 and reporting bugs back to
> the Apache Cassandra JIRA, I'm sure it would be greatly appreciated.
>
> https://issues.apache.org/jira/secure/Dashboard.jspa
>
> Jon
>
>
>
> On Thu, Oct 13, 2016 at 3:15 PM Eric Ho  wrote:
>
> Some suggested Elassandra.  But that is based on Cassandra 2.2.
> I would like to use Cassandra 3.7 and up...
>
>
>
> -eric ho
>
>
> On Thu, Oct 13, 2016 at 3:04 PM, vincent gromakowski <
> vincent.gromakow...@gmail.com> wrote:
>
> Elassandra
> https://github.com/vroyer/elassandra
>
> Le 14 oct. 2016 12:02 AM, "Eric Ho"  a écrit :
>
> I don't want to change my code to write into C* and then to ES.
> So, I'm looking for some sort of a sync tool that will sync my C* table
> into ES and it should be smart enough to avoid duplicates or gaps.
> Is there such a tool / plugin ?
> I'm using stock apache Cassandra 3.7.
> I know that some premium Cassandra has ES builtin or integrated but I
> can't afford premium right now...
> Thanks.
>
> -eric ho
>
>
>
>
>

Re: are there any free Cassandra -> ElasticSearch connector / plugin ?

2016-10-13 Thread Eric Ho 
But if I'm not doing anything fancy w/ C* (i.e. don't use new features in
3.{2,4,6}) then I'll be fine, right ?


-eric ho


On Thu, Oct 13, 2016 at 5:09 PM, Jonathan Haddad  wrote:

> I listed my reasons, please check my previous email.
>
> On Thu, Oct 13, 2016 at 4:55 PM Eric Ho  wrote:
>
>> Why 3.0.x ?  Why not use 3.2.x or 3.4.x ? or 3.6.x ?
>> Shouldn't 3.6.x be more stable than say 3.2.x ?
>>
>>
>> -eric ho
>>
>>
>> On Thu, Oct 13, 2016 at 3:48 PM, Jonathan Haddad 
>> wrote:
>>
>> Here's your basic options:
>>
>> 1. Triggers (avoid like the plague)
>> 2. CDC (really new, tricky to avoid RF operations as is, probably avoid)
>> 3. Do it in your app
>> 4. Put Kafka in front of your data, write as many consumers as you want
>> to write the data in as many ways as you want
>>
>> Also, how long have you been using Cassandra?  Unless you're comfortable
>> rolling your own builds and merging in bugfixes from upstream, I really
>> suggest using a 3.0.x release instead of a 3.7.
>>
>> 3.7 falls under the Tick Tock release cycle, which is almost completely
>> untested in production by experienced operators.  In the cases where it
>> has
>> been tested, there have been numerous bugs found which I (and I think most
>> people on this list) consider to be show stoppers.  Additionally, the Tick
>> Tock release cycle puts the operator in the uncomfortable position of
>> having to decide between upgrading to a new version with new features
>> (probably new bugs) or back porting bug fixes from future versions
>> themselves.There will never be a 3.7.1 release which fixes bugs in 3.7
>> without adding new features.
>>
>> https://github.com/apache/cassandra/blob/trunk/NEWS.txt
>>
>> For new projects I recommend starting with the recently released 3.0.9.
>>
>> Assuming the project changes it's policy on releases (all signs point to
>> yes), then by the time 4.0 rolls out a lot of the features which have been
>> released in the 3.x series will have matured a bit, so it's very possible
>> 4.0 will stabilize faster than the usual 6 months it takes for a major
>> release.
>>
>> All that said, there's nothing wrong with doing compatibility & smoke
>> tests
>> against the latest 3.x release as well as 3.0 and reporting bugs back to
>> the Apache Cassandra JIRA, I'm sure it would be greatly appreciated.
>>
>> https://issues.apache.org/jira/secure/Dashboard.jspa
>>
>> Jon
>>
>>
>>
>> On Thu, Oct 13, 2016 at 3:15 PM Eric Ho  wrote:
>>
>> Some suggested Elassandra.  But that is based on Cassandra 2.2.
>> I would like to use Cassandra 3.7 and up...
>>
>>
>>
>> -eric ho
>>
>>
>> On Thu, Oct 13, 2016 at 3:04 PM, vincent gromakowski <
>> vincent.gromakow...@gmail.com> wrote:
>>
>> Elassandra
>> https://github.com/vroyer/elassandra
>>
>> Le 14 oct. 2016 12:02 AM, "Eric Ho"  a écrit :
>>
>> I don't want to change my code to write into C* and then to ES.
>> So, I'm looking for some sort of a sync tool that will sync my C* table
>> into ES and it should be smart enough to avoid duplicates or gaps.
>> Is there such a tool / plugin ?
>> I'm using stock apache Cassandra 3.7.
>> I know that some premium Cassandra has ES builtin or integrated but I
>> can't afford premium right now...
>> Thanks.
>>
>> -eric ho
>>
>>
>>
>>

Re: are there any free Cassandra -> ElasticSearch connector / plugin ?

2016-10-13 Thread Jonathan Haddad 
I listed my reasons, please check my previous email.

On Thu, Oct 13, 2016 at 4:55 PM Eric Ho  wrote:

> Why 3.0.x ?  Why not use 3.2.x or 3.4.x ? or 3.6.x ?
> Shouldn't 3.6.x be more stable than say 3.2.x ?
>
>
> -eric ho
>
>
> On Thu, Oct 13, 2016 at 3:48 PM, Jonathan Haddad 
> wrote:
>
> Here's your basic options:
>
> 1. Triggers (avoid like the plague)
> 2. CDC (really new, tricky to avoid RF operations as is, probably avoid)
> 3. Do it in your app
> 4. Put Kafka in front of your data, write as many consumers as you want to
> write the data in as many ways as you want
>
> Also, how long have you been using Cassandra?  Unless you're comfortable
> rolling your own builds and merging in bugfixes from upstream, I really
> suggest using a 3.0.x release instead of a 3.7.
>
> 3.7 falls under the Tick Tock release cycle, which is almost completely
> untested in production by experienced operators.  In the cases where it has
> been tested, there have been numerous bugs found which I (and I think most
> people on this list) consider to be show stoppers.  Additionally, the Tick
> Tock release cycle puts the operator in the uncomfortable position of
> having to decide between upgrading to a new version with new features
> (probably new bugs) or back porting bug fixes from future versions
> themselves.There will never be a 3.7.1 release which fixes bugs in 3.7
> without adding new features.
>
> https://github.com/apache/cassandra/blob/trunk/NEWS.txt
>
> For new projects I recommend starting with the recently released 3.0.9.
>
> Assuming the project changes it's policy on releases (all signs point to
> yes), then by the time 4.0 rolls out a lot of the features which have been
> released in the 3.x series will have matured a bit, so it's very possible
> 4.0 will stabilize faster than the usual 6 months it takes for a major
> release.
>
> All that said, there's nothing wrong with doing compatibility & smoke tests
> against the latest 3.x release as well as 3.0 and reporting bugs back to
> the Apache Cassandra JIRA, I'm sure it would be greatly appreciated.
>
> https://issues.apache.org/jira/secure/Dashboard.jspa
>
> Jon
>
>
>
> On Thu, Oct 13, 2016 at 3:15 PM Eric Ho  wrote:
>
> Some suggested Elassandra.  But that is based on Cassandra 2.2.
> I would like to use Cassandra 3.7 and up...
>
>
>
> -eric ho
>
>
> On Thu, Oct 13, 2016 at 3:04 PM, vincent gromakowski <
> vincent.gromakow...@gmail.com> wrote:
>
> Elassandra
> https://github.com/vroyer/elassandra
>
> Le 14 oct. 2016 12:02 AM, "Eric Ho"  a écrit :
>
> I don't want to change my code to write into C* and then to ES.
> So, I'm looking for some sort of a sync tool that will sync my C* table
> into ES and it should be smart enough to avoid duplicates or gaps.
> Is there such a tool / plugin ?
> I'm using stock apache Cassandra 3.7.
> I know that some premium Cassandra has ES builtin or integrated but I
> can't afford premium right now...
> Thanks.
>
> -eric ho
>
>
>
>

Re: are there any free Cassandra -> ElasticSearch connector / plugin ?

2016-10-13 Thread Eric Ho 
Why 3.0.x ?  Why not use 3.2.x or 3.4.x ? or 3.6.x ?
Shouldn't 3.6.x be more stable than say 3.2.x ?


-eric ho


On Thu, Oct 13, 2016 at 3:48 PM, Jonathan Haddad  wrote:

> Here's your basic options:
>
> 1. Triggers (avoid like the plague)
> 2. CDC (really new, tricky to avoid RF operations as is, probably avoid)
> 3. Do it in your app
> 4. Put Kafka in front of your data, write as many consumers as you want to
> write the data in as many ways as you want
>
> Also, how long have you been using Cassandra?  Unless you're comfortable
> rolling your own builds and merging in bugfixes from upstream, I really
> suggest using a 3.0.x release instead of a 3.7.
>
> 3.7 falls under the Tick Tock release cycle, which is almost completely
> untested in production by experienced operators.  In the cases where it has
> been tested, there have been numerous bugs found which I (and I think most
> people on this list) consider to be show stoppers.  Additionally, the Tick
> Tock release cycle puts the operator in the uncomfortable position of
> having to decide between upgrading to a new version with new features
> (probably new bugs) or back porting bug fixes from future versions
> themselves.There will never be a 3.7.1 release which fixes bugs in 3.7
> without adding new features.
>
> https://github.com/apache/cassandra/blob/trunk/NEWS.txt
>
> For new projects I recommend starting with the recently released 3.0.9.
>
> Assuming the project changes it's policy on releases (all signs point to
> yes), then by the time 4.0 rolls out a lot of the features which have been
> released in the 3.x series will have matured a bit, so it's very possible
> 4.0 will stabilize faster than the usual 6 months it takes for a major
> release.
>
> All that said, there's nothing wrong with doing compatibility & smoke tests
> against the latest 3.x release as well as 3.0 and reporting bugs back to
> the Apache Cassandra JIRA, I'm sure it would be greatly appreciated.
>
> https://issues.apache.org/jira/secure/Dashboard.jspa
>
> Jon
>
>
>
> On Thu, Oct 13, 2016 at 3:15 PM Eric Ho  wrote:
>
>> Some suggested Elassandra.  But that is based on Cassandra 2.2.
>> I would like to use Cassandra 3.7 and up...
>>
>>
>>
>> -eric ho
>>
>>
>> On Thu, Oct 13, 2016 at 3:04 PM, vincent gromakowski <
>> vincent.gromakow...@gmail.com> wrote:
>>
>> Elassandra
>> https://github.com/vroyer/elassandra
>>
>> Le 14 oct. 2016 12:02 AM, "Eric Ho"  a écrit :
>>
>> I don't want to change my code to write into C* and then to ES.
>> So, I'm looking for some sort of a sync tool that will sync my C* table
>> into ES and it should be smart enough to avoid duplicates or gaps.
>> Is there such a tool / plugin ?
>> I'm using stock apache Cassandra 3.7.
>> I know that some premium Cassandra has ES builtin or integrated but I
>> can't afford premium right now...
>> Thanks.
>>
>> -eric ho
>>
>>
>>

Re: are there any free Cassandra -> ElasticSearch connector / plugin ?

2016-10-13 Thread Jonathan Haddad 
Here's your basic options:

1. Triggers (avoid like the plague)
2. CDC (really new, tricky to avoid RF operations as is, probably avoid)
3. Do it in your app
4. Put Kafka in front of your data, write as many consumers as you want to
write the data in as many ways as you want

Also, how long have you been using Cassandra?  Unless you're comfortable
rolling your own builds and merging in bugfixes from upstream, I really
suggest using a 3.0.x release instead of a 3.7.

3.7 falls under the Tick Tock release cycle, which is almost completely
untested in production by experienced operators.  In the cases where it has
been tested, there have been numerous bugs found which I (and I think most
people on this list) consider to be show stoppers.  Additionally, the Tick
Tock release cycle puts the operator in the uncomfortable position of
having to decide between upgrading to a new version with new features
(probably new bugs) or back porting bug fixes from future versions
themselves.There will never be a 3.7.1 release which fixes bugs in 3.7
without adding new features.

https://github.com/apache/cassandra/blob/trunk/NEWS.txt

For new projects I recommend starting with the recently released 3.0.9.

Assuming the project changes it's policy on releases (all signs point to
yes), then by the time 4.0 rolls out a lot of the features which have been
released in the 3.x series will have matured a bit, so it's very possible
4.0 will stabilize faster than the usual 6 months it takes for a major
release.

All that said, there's nothing wrong with doing compatibility & smoke tests
against the latest 3.x release as well as 3.0 and reporting bugs back to
the Apache Cassandra JIRA, I'm sure it would be greatly appreciated.

https://issues.apache.org/jira/secure/Dashboard.jspa

Jon



On Thu, Oct 13, 2016 at 3:15 PM Eric Ho  wrote:

> Some suggested Elassandra.  But that is based on Cassandra 2.2.
> I would like to use Cassandra 3.7 and up...
>
>
>
> -eric ho
>
>
> On Thu, Oct 13, 2016 at 3:04 PM, vincent gromakowski <
> vincent.gromakow...@gmail.com> wrote:
>
> Elassandra
> https://github.com/vroyer/elassandra
>
> Le 14 oct. 2016 12:02 AM, "Eric Ho"  a écrit :
>
> I don't want to change my code to write into C* and then to ES.
> So, I'm looking for some sort of a sync tool that will sync my C* table
> into ES and it should be smart enough to avoid duplicates or gaps.
> Is there such a tool / plugin ?
> I'm using stock apache Cassandra 3.7.
> I know that some premium Cassandra has ES builtin or integrated but I
> can't afford premium right now...
> Thanks.
>
> -eric ho
>
>
>

Re: are there any free Cassandra -> ElasticSearch connector / plugin ?

2016-10-13 Thread Eric Ho 
Some suggested Elassandra.  But that is based on Cassandra 2.2.
I would like to use Cassandra 3.7 and up...



-eric ho


On Thu, Oct 13, 2016 at 3:04 PM, vincent gromakowski <
vincent.gromakow...@gmail.com> wrote:

> Elassandra
> https://github.com/vroyer/elassandra
>
> Le 14 oct. 2016 12:02 AM, "Eric Ho"  a écrit :
>
>> I don't want to change my code to write into C* and then to ES.
>> So, I'm looking for some sort of a sync tool that will sync my C* table
>> into ES and it should be smart enough to avoid duplicates or gaps.
>> Is there such a tool / plugin ?
>> I'm using stock apache Cassandra 3.7.
>> I know that some premium Cassandra has ES builtin or integrated but I
>> can't afford premium right now...
>> Thanks.
>>
>> -eric ho
>>
>>

Re: are there any free Cassandra -> ElasticSearch connector / plugin ?

2016-10-13 Thread Brian O'Neill 
I haven't used it yet, but
https://github.com/vroyer/elassandra 

-- 
Brian O'Neill
Principal Architect @ Monetate
m: 215.588.6024
bone...@monetate.com 


> On Oct 13, 2016, at 6:02 PM, Eric Ho  wrote:
> 
> I don't want to change my code to write into C* and then to ES.
> So, I'm looking for some sort of a sync tool that will sync my C* table into 
> ES and it should be smart enough to avoid duplicates or gaps.
> Is there such a tool / plugin ?
> I'm using stock apache Cassandra 3.7.
> I know that some premium Cassandra has ES builtin or integrated but I can't 
> afford premium right now...
> Thanks.
> 
> -eric ho
>

Re: are there any free Cassandra -> ElasticSearch connector / plugin ?

2016-10-13 Thread vincent gromakowski 
Elassandra
https://github.com/vroyer/elassandra

Le 14 oct. 2016 12:02 AM, "Eric Ho"  a écrit :

> I don't want to change my code to write into C* and then to ES.
> So, I'm looking for some sort of a sync tool that will sync my C* table
> into ES and it should be smart enough to avoid duplicates or gaps.
> Is there such a tool / plugin ?
> I'm using stock apache Cassandra 3.7.
> I know that some premium Cassandra has ES builtin or integrated but I
> can't afford premium right now...
> Thanks.
>
> -eric ho
>
>

are there any free Cassandra -> ElasticSearch connector / plugin ?

2016-10-13 Thread Eric Ho 
I don't want to change my code to write into C* and then to ES.
So, I'm looking for some sort of a sync tool that will sync my C* table
into ES and it should be smart enough to avoid duplicates or gaps.
Is there such a tool / plugin ?
I'm using stock apache Cassandra 3.7.
I know that some premium Cassandra has ES builtin or integrated but I can't
afford premium right now...
Thanks.

-eric ho

Re: Does increment/decrement by 0 generate any commits ?

2016-10-13 Thread Ben Bromhead 
According to https://issues.apache.org/jira/browse/CASSANDRA-7304 unset
values in a prepared statement for a counter does not change the value of
the counter. This applies for versions of Cassandra 2.2 and above.

I would also look to verify the claimed behaviour myself.

On Tue, 11 Oct 2016 at 09:49 Dorian Hoxha  wrote:

> I just have a bunch of counters in 1 row, and I want to selectively update
> them. And I want to keep prepared queries. But I don't want to keep 30
> prepared queries (1 for each counter column, but keep only 1). So in most
> cases, I will increment 1 column by positive integer and the others by 0.
>
> Makes sense ?
>
-- 
Ben Bromhead
CTO | Instaclustr 
+1 650 284 9692
Managed Cassandra / Spark on AWS, Azure and Softlayer

New node overstreaming data ?

2016-10-13 Thread Anubhav Kale 
Hello,

We run 2.1.13 and seeing an odd issue. A node went down, and stayed down for a 
while so it went out of gossip. When we try to bootstrap it again (as a new 
node), it overstreams from other nodes, eventually disk becomes full and 
crashes. This repeated 3 times.

Does anyone have any insights on what to try next (both in terms of root 
causing, and working around) ? To work around, we tried increasing #compactors 
and reducing stream throughput so that at least incoming #SSTables would be 
controlled.

This has happened to us few times in the past too, so I am wondering if this is 
a known problem (I couldn't find any JIRAs).

Thanks !

Re: Gossip status: hibernate

2016-10-13 Thread Joel Knighton 
In the normal case, absolutely. That should happen quickly. I think it is
likely the case that you've hit another race condition where this phantom
node is not correctly marked as dead. In this case, even if it is removed
from some nodes in the cluster, it will get re-added by the node that
doesn't have the phantom node marked down. I can't say definitively without
some more information though.

On Thu, Oct 13, 2016 at 7:11 AM, Kasper Petersen 
wrote:

> Thanks for the details.
>
> I don't know what happened on that node. It's a long time ago I think. I
> wasn't aware of it earlier.
>
> Will a node in hibernating state that failed joining and subsequently was
> discarded get removed from gossip at some point?
>
> On Wed, Oct 12, 2016 at 5:23 PM, Joel Knighton  > wrote:
>
>> 1. A hibernating node is participating in gossip but intentionally hasn't
>> yet joined the ring. The two cases where a node would set a hibernating
>> status are when the node was started with "-Dcassandra.join_ring=False" and
>> has tokens or when the node was started to replace another node (using
>> "-Dcassandra.replace_address" or "-Dcassandra.replace_address_f
>> irst_boot").
>>
>> 2. A rolling restart is probably your best bet. You may have more luck
>> with an assassinate in the case that you connect to a node that is not
>> continuously removing/adding the state. I suspect that this node will have
>> an alive status for this endpoint state. As usual, you should wield
>> assassinate with lots of caution.
>>
>> This issue sounds most similar to CASSANDRA-10371. If you provide
>> debugging information similar to that requested on the above ticket as well
>> as what operation you were performing on the node (was it a failed attempt
>> at replacing? etc) on a JIRA ticket, someone might have a chance to look
>> into this further.
>>
>> On Wed, Oct 12, 2016 at 9:48 AM, Kasper Petersen 
>> wrote:
>>
>>> Hi,
>>>
>>> I've recently upgraded our Cassandra cluster from 2.1 to 3.9. By
>>> default(?) 3.9 creates a debug.log file containing a ton of lines (a new
>>> one every second) with:
>>>
>>> DEBUG [GossipTasks:1] 2016-10-12 14:43:38,761 Gossiper.java:337 -
 Convicting /172.31.137.65 with status hibernate - alive false
>>>
>>>
>>> That node has not been around for a very long time now.
>>>
>>> It does not show up in nodetool status and nodetool gossipinfo returns
>>> the following output about that node:
>>>
>>> /172.31.137.65
   generation:1433571405
   heartbeat:232
   STATUS:3:hibernate,true
   LOAD:225:96445.0
   SCHEMA:53:e2d1a288-581c-3f35-b492-1b9d5a803631
   DC:9:us-east
   RACK:11:1b
   RELEASE_VERSION:7:2.1.5
   RPC_ADDRESS:6:172.31.137.65
   SEVERITY:231:0.2512562870979309
   NET_VERSION:4:8
   HOST_ID:5:7988d3c9-dec8-4b71-b5a9-0b962aad0680
   TOKENS:2:
>>>
>>>
>>> nodetool removenode 7988d3c9-dec8-4b71-b5a9-0b962aad0680 resulted in:
>>>
>>> error: Host ID not found.

>>>
>>> Now my questions are:
>>>
>>>1. What does it mean for a node to be "hibernating"? How does it end
>>>up in that state?
>>>2. How do I get rid of it? Its not coming back.
>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Kasper Middelboe Petersen
>>>
>>> *Lead Backend Developer*
>>>
>>> *SYBO Games ApS*
>>> Jorcks Passage 1A, 4th.
>>> 1162 Copenhagen K
>>>
>>
>>
>
>
> --
> Best regards,
> Kasper Middelboe Petersen
>
> *Lead Backend Developer*
>
> *SYBO Games ApS*
> Jorcks Passage 1A, 4th.
> 1162 Copenhagen K
>

Re: Gossip status: hibernate

2016-10-13 Thread Kasper Petersen 
Thanks for the details.

I don't know what happened on that node. It's a long time ago I think. I
wasn't aware of it earlier.

Will a node in hibernating state that failed joining and subsequently was
discarded get removed from gossip at some point?

On Wed, Oct 12, 2016 at 5:23 PM, Joel Knighton 
wrote:

> 1. A hibernating node is participating in gossip but intentionally hasn't
> yet joined the ring. The two cases where a node would set a hibernating
> status are when the node was started with "-Dcassandra.join_ring=False" and
> has tokens or when the node was started to replace another node (using
> "-Dcassandra.replace_address" or "-Dcassandra.replace_address_
> first_boot").
>
> 2. A rolling restart is probably your best bet. You may have more luck
> with an assassinate in the case that you connect to a node that is not
> continuously removing/adding the state. I suspect that this node will have
> an alive status for this endpoint state. As usual, you should wield
> assassinate with lots of caution.
>
> This issue sounds most similar to CASSANDRA-10371. If you provide
> debugging information similar to that requested on the above ticket as well
> as what operation you were performing on the node (was it a failed attempt
> at replacing? etc) on a JIRA ticket, someone might have a chance to look
> into this further.
>
> On Wed, Oct 12, 2016 at 9:48 AM, Kasper Petersen 
> wrote:
>
>> Hi,
>>
>> I've recently upgraded our Cassandra cluster from 2.1 to 3.9. By
>> default(?) 3.9 creates a debug.log file containing a ton of lines (a new
>> one every second) with:
>>
>> DEBUG [GossipTasks:1] 2016-10-12 14:43:38,761 Gossiper.java:337 -
>>> Convicting /172.31.137.65 with status hibernate - alive false
>>
>>
>> That node has not been around for a very long time now.
>>
>> It does not show up in nodetool status and nodetool gossipinfo returns
>> the following output about that node:
>>
>> /172.31.137.65
>>>   generation:1433571405
>>>   heartbeat:232
>>>   STATUS:3:hibernate,true
>>>   LOAD:225:96445.0
>>>   SCHEMA:53:e2d1a288-581c-3f35-b492-1b9d5a803631
>>>   DC:9:us-east
>>>   RACK:11:1b
>>>   RELEASE_VERSION:7:2.1.5
>>>   RPC_ADDRESS:6:172.31.137.65
>>>   SEVERITY:231:0.2512562870979309
>>>   NET_VERSION:4:8
>>>   HOST_ID:5:7988d3c9-dec8-4b71-b5a9-0b962aad0680
>>>   TOKENS:2:
>>
>>
>> nodetool removenode 7988d3c9-dec8-4b71-b5a9-0b962aad0680 resulted in:
>>
>> error: Host ID not found.
>>>
>>
>> Now my questions are:
>>
>>1. What does it mean for a node to be "hibernating"? How does it end
>>up in that state?
>>2. How do I get rid of it? Its not coming back.
>>
>>
>>
>> --
>> Best regards,
>> Kasper Middelboe Petersen
>>
>> *Lead Backend Developer*
>>
>> *SYBO Games ApS*
>> Jorcks Passage 1A, 4th.
>> 1162 Copenhagen K
>>
>
>


-- 
Best regards,
Kasper Middelboe Petersen

*Lead Backend Developer*

*SYBO Games ApS*
Jorcks Passage 1A, 4th.
1162 Copenhagen K