Re: Does "nodetool repair" need to be run on each node for a given table?

[Max Campos]

For anyone with questions about how repair works, why repair, partial range 
repair, incremental vs. non-incremental repair, best practices for repair, etc. 
 I highly recommend this talk from Alexander Dejanovski, The Last Pickle at 
Cassandra Summit 2016:

https://www.youtube.com/watch?v=1Sz_K8UID6E 


Alexander went a little too crazy on the auto-repeat animated GIFs, but the 
information is excellent — thanks Alexander if you’re out there - I really got 
a lot out of your talk.  :-)

- Max

> On Mar 14, 2017, at 12:48 pm, Thakrar, Jayesh  
> wrote:
> 
> Thank you for the links, Meg - very helpful!
>  
> From: Meg Mara >
> Date: Tuesday, March 14, 2017 at 2:08 PM
> To: "user@cassandra.apache.org " 
> >
> Subject: RE: Does "nodetool repair" need to be run on each node for a given 
> table?
>  
> Yes, whichever node initiates “nodetool repair” becomes the co-ordinator node 
> for that repair operation and only the token ranges it owns are repaired. 
> Other nodes which own data in that “repaired token range” will also be 
> involved in this repair operation, create their merkle trees and send it to 
> coordinator for comparison.
>  
> There is a lot to explain when it comes to repair and its many options (-pr, 
> -inc, -par etc). I recommend the following two links which provide the 
> answers to all your questions.
>  
> https://www.datastax.com/dev/blog/repair-in-cassandra 
> 
> https://www.pythian.com/blog/effective-anti-entropy-repair-cassandra/ 
> 
>  
> Thanks,
> -  Meg Mara
>  
> From: Thakrar, Jayesh [mailto:jthak...@conversantmedia.com 
> ] 
> Sent: Tuesday, March 14, 2017 1:26 PM
> To: daemeon reiydelle >; 
> user@cassandra.apache.org 
> Subject: Re: Does "nodetool repair" need to be run on each node for a given 
> table?
>  
> Thanks Daemon - it’s the documentation that prompted me to ask the question.
>  
> The doc points to further info on incremental, and would I be correct to 
> interpret that nodetool initiates a repair on a specific node (i.e. the node 
> specified with -h or the current node)?
>  
> http://www.datastax.com/dev/blog/more-efficient-repairs 
> 
>  
> 
>  
> From: daemeon reiydelle >
> Date: Monday, March 13, 2017 at 5:02 PM
> To: >
> Subject: Re: Does "nodetool repair" need to be run on each node for a given 
> table?
>  
> I
> ​ find it helpful to read the manual first. After review, I would be happy to 
> answer specific questions.
> 
> https://docs.datastax.com/en/cassandra/2.1/cassandra/tools/toolsRepair.html 
> ​
> 
> 
> ...
> 
> Daemeon C.M. Reiydelle
> USA (+1) 415.501.0198
> London (+44) (0) 20 8144 9872
>  
> On Mon, Mar 13, 2017 at 1:30 PM, Thakrar, Jayesh 
> > wrote:
> I understand that the nodetool command connects to a specific server and for 
> many of the commands, e.g. "info", "compactionstats", etc, the information is 
> for that specific node.
> While for some other commands like "status", the info is for the whole 
> cluster.
>  
> So is "nodetool repair" that operates at a single node level (i.e. repairs 
> the partitions contained on the target node?).
> If so, what is the recommended approach to doing repairs?
>  
> E.g. we have a large number of tables (20+), large amount of data (40+ TB) 
> and a number of nodes (40+).
> Do I need to iterate through each server AND each table?
>  
> Thanks,
> Jayesh
>  
>  
>  
>  

Re: COPY FROM performance

[Stefania Alborghetti]

Launch cqlsh withe the "--debug" option: cqlsh --debug. You should see
which Python driver it is using. My guess is that it is not using the
installed driver, which by default should be Cythonized, but it is still
using the embedded driver.

This is what is shown on my machine for the embedded driver:

Using CQL driver: 

And this is for an installed driver:

Using CQL driver: 

You should also notice that cqlsh takes a little bit longer to start when
using an external driver.

If you want to double check if the driver is Cythonized, any installed
driver should be, you can do "unzip -l" on the egg, for example:

unzip -l
/usr/local/lib/python2.7/dist-packages/cassandra_driver-3.7.1.post0-py2.7-linux-x86_64.egg

You should see files with extension ".pyc" and ".so", they indicate that
the driver was compiled with Cython.

Lastly, regarding point 3, Cython does not ship with C*. However, you don't
need it, unless you want to compile with Cython *pylib/cqlshlib/copyutil.py*
as well, but in my opinion, this is not worth it, the biggest improvement
is when the driver is compiled with Cython, at least from the tests I did.
Therefore, your steps above look correct.

It would be very odd that you see no performance gain with a Cythonized
driver, but like I said performance depends on the schema type, perhaps
your schema has complex types like collections, where the parsing is the
dominant factor - but in this case I would have expected cassandra-loader
to outperform COPY FROM. The parsing is done by *copyutil.py* by the way,
in which case you may well want to Cythonize it too (instructions in the
blog), but because the Python code in *copyutil.py* is not optimized for
Cython, don't expect huge gains.

If you want to move the burden of parsing from the client to the cluster,
you can do so with PREPAREDSTATEMENTS=False, but I only recommend this if
the cluster machines are idle.

Finally, make sure to try out some COPY FROM parameters, especially
NUMPROCESSES
and CHUNKSIZE. For the first parameter, observe the CPU and increase the
number of processes if you have idle CPU on the client, decrease it if the
CPU is blocked (dstat -lvrn 10). As for CHUNKSIZE, because it is in rows,
it may be that for your schema the ideal value is higher or smaller than
5,000, so try different values, such as 500 and 50,000, and see what this
does to performance.



On Tue, Mar 14, 2017 at 9:23 PM, Artur R  wrote:

> HI!
>
> I am trying to increase performance of COPY FROM by installing "*Cython
>  and libev
>  C extensions"* as
> described here: https://www.datastax.com/dev/blog/six-
> parameters-affecting-cqlsh-copy-from-performance.
>
> My steps are as the following:
>
>1. Install Cassandra 3.10, start it, add keyspace and table
>
>2. Install C Extensions:
>sudo apt-get install gcc python-dev
>
>3. Don't install Cython because as far as I understand it ships with
>C* 3 by default on step 1), so skip it
>
>4. Install libev:
>sudo apt-get install libev4 libev-dev
>
>5. Reinstall C* driver (because as far as I understand it shipped with
>C* on step 1):
>sudo pip install cassandra-driver
>
>6. export CQLSH_NO_BUNDLED=TRUE
>
>7. cqlsh to node and try COPY FROM
>
> And after all these steps above performance of COPY FROM is the same as
> before.
> I tested it with single node cluster and with multiple nodes cluster - it
> doesn't impact on performance.
> However, I see that COPY FROM is CPU bounded on my machines, so these
> steps should definitely increase the performance.
>
>
> The question: what I did wrong? Maybe some step is missed.
> How to check that COPY FROM really uses "*Cython
>  and libev
>  C extensions"* ?
>



-- 



STEFANIA ALBORGHETTI

Software engineer | +852 6114 9265 | stefania.alborghe...@datastax.com


[image: http://www.datastax.com/cloud-applications]

Re: Internal Security - Authentication & Authorization

[Jai Bheemsen Rao Dhanwada]

I have similar question. when we create users or roles what is the
consistency level used?

I know, If NOT EXISTS will use SERIAL consistency. what consistency will be
used if just use CREATE USER ?

On Mon, Mar 13, 2017 at 7:09 PM, Jacob Shadix  wrote:

> I'm looking for a deeper understanding of how Cassandra interacts with the
> system_auth keyspace to authenticate/authorize users.
>
> Here is what I have pieced together. Please let me know if I am on the
> right track.
>
> A user attempts to connect to Cassandra. Cassandra checks against
> system_auth for that user @ LOCAL_ONE - - If the user exists, a connection
> is established. When CQL is executed, C* again checks system_auth for that
> user @ LOCAL_ONE to determine if it has the correct privileges to perform
> the CQL. If so, it executes the CQL and the permissions are stored in a
> cache. During the cache validity timeframe, future requests for ANY user
> stored in the cache do not require a lookup against system_auth. After the
> cache validity runs out, any new requests will require a lookup against
> system_auth.
>
> -- Jacob Shadix
>

Re: changing compaction strategy

[Mike Torra]

Some more info:
- running C* 3.9
- I tried `nodetool flush` on the column family this change applies to, and
while it does seem to trigger compactions, there is still one pending that
won't seem to run
- I tried `nodetool compact` on the column family as well, with a similar
affect

Is there a way to tell when/if the local node has successfully updated the
compaction strategy? Looking at the sstable files, it seems like they are
still based on STCS but I don't know how to be sure.

Appreciate any tips or suggestions!

On Mon, Mar 13, 2017 at 5:30 PM, Mike Torra  wrote:

> I'm trying to change compaction strategy one node at a time. I'm using
> jmxterm like this:
>
> `echo 'set -b 
> org.apache.cassandra.db:type=ColumnFamilies,keyspace=my_ks,columnfamily=my_cf
> CompactionParametersJson \{"class":"TimeWindowCompactionStrategy",
> "compaction_window_unit":"HOURS","compaction_window_size":"6"\}' | java
> -jar jmxterm-1.0-alpha-4-uber.jar --url localhost:7199`
>
> and I see this in the cassandra logs:
>
> INFO  [RMI TCP Connection(37)-127.0.0.1] 2017-03-13 20:29:08,251
> CompactionStrategyManager.java:841 - Switching local compaction strategy
> from 
> CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,
> options={max_threshold=32, min_threshold=4}} to CompactionParams{class=org.
> apache.cassandra.db.compaction.TimeWindowCompactionStrategy,
> options={min_threshold=4, max_threshold=32, compaction_window_unit=HOURS,
> compaction_window_size=6}}}
>
> After doing this, `nodetool compactionstats` shows 1 pending compaction,
> but none running. Also, cqlsh describe shows the old compaction strategy
> still. Am I missing a step?
>

Re: Change the IP of a live node

[George Sigletos]

To give a complete picture, my node has actually two network interfaces:
eth0 for 192.168.xx.xx and eth1 for 10.179.xx.xx

On Tue, Mar 14, 2017 at 7:46 PM, George Sigletos 
wrote:

> Hello,
>
> I am trying to change the IP of a live node (I am not replacing a dead
> one).
>
> So I stop the service on my node (not a seed node), I change the IP from
> 192.168.xx.xx to 10.179.xx.xx, and modify "listen_address" and
> "rpc_address" in the cassandra.yaml, while I also set auto_bootstrap:
> false. Then I restart but it fails to see the rest of the cluster:
>
> Datacenter: DC1
> ===
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens  OwnsHost
> ID   Rack
> DN  192.168.xx.xx  ?  256 ?   
> 241f3002-8f89-4433-a521-4fa4b070b704
> r1
> UN  10.179.xx.xx  3.45 TB256 ?   
> 3b07df3b-683b-4e2d-b307-3c48190c8f1c
> RAC1
> DN  192.168.xx.xx  ?  256 ?   
> 19636f1e-9417-4354-8364-6617b8d3d20b
> r1
> DN  192.168.xx.xx?  256 ?   
> 9c65c71c-f5dd-4267-af9e-a20881cf3d48
> r1
> DN  192.168.xx.xx   ?  256 ?   
> ee75219f-0f2c-4be0-bd6d-038315212728
> r1
>
> Am I doing anything wrong? Thanks in advance
>
> Kind regards,
> George
>

Change the IP of a live node

[George Sigletos]

Hello,

I am trying to change the IP of a live node (I am not replacing a dead
one).

So I stop the service on my node (not a seed node), I change the IP from
192.168.xx.xx to 10.179.xx.xx, and modify "listen_address" and
"rpc_address" in the cassandra.yaml, while I also set auto_bootstrap:
false. Then I restart but it fails to see the rest of the cluster:

Datacenter: DC1
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  AddressLoad   Tokens  OwnsHost
ID   Rack
DN  192.168.xx.xx  ?  256 ?
241f3002-8f89-4433-a521-4fa4b070b704  r1
UN  10.179.xx.xx  3.45 TB256 ?
3b07df3b-683b-4e2d-b307-3c48190c8f1c  RAC1
DN  192.168.xx.xx  ?  256 ?
19636f1e-9417-4354-8364-6617b8d3d20b  r1
DN  192.168.xx.xx?  256 ?
9c65c71c-f5dd-4267-af9e-a20881cf3d48  r1
DN  192.168.xx.xx   ?  256 ?
ee75219f-0f2c-4be0-bd6d-038315212728  r1

Am I doing anything wrong? Thanks in advance

Kind regards,
George

Re: scylladb

[Dor Laor]

On Tue, Mar 14, 2017 at 7:43 AM, Eric Evans 
wrote:

> On Sun, Mar 12, 2017 at 4:01 PM, James Carman
>  wrote:
> > Does all of this Scylla talk really even belong on the Cassandra user
> > mailing list in the first place?
>
> I personally found it interesting, informative, and on-topic when it
> was about justification of the 10x performance claim, numa,
> scheduling, concurrency, etc.  At some point it got a little sales-y
> on the one side, and caremad on the other.
>

I tried to just provide accurate answers. There is a meme on twitter that
matches it:

Pi Day is just a fake holiday created by math companies to sell more math.

It's that good so I couldn't resist.


> ¯\_(ツ)_/¯
>
>
> --
> Eric Evans
> john.eric.ev...@gmail.com
>

Re: Does "nodetool repair" need to be run on each node for a given table?

[daemeon reiydelle]

Am I unreasonable in expecting a poster to have looked at the documentation
before posting? And that reposting the same query WITHOUT reading the
documents (when pointed out to them) when asked to do so is not
appropriate? Do we have a way to blackball such?


*...*



*Daemeon C.M. ReiydelleUSA (+1) 415.501.0198London (+44) (0) 20 8144 9872*

On Mon, Mar 13, 2017 at 1:30 PM, Thakrar, Jayesh <
jthak...@conversantmedia.com> wrote:

> I understand that the nodetool command connects to a specific server and
> for many of the commands, e.g. "info", "compactionstats", etc, the
> information is for that specific node.
>
> While for some other commands like "status", the info is for the whole
> cluster.
>
>
>
> So is "nodetool repair" that operates at a single node level (i.e. repairs
> the partitions contained on the target node?).
>
> If so, what is the recommended approach to doing repairs?
>
>
>
> E.g. we have a large number of tables (20+), large amount of data (40+ TB)
> and a number of nodes (40+).
>
> Do I need to iterate through each server AND each table?
>
>
>
> Thanks,
>
> Jayesh
>
>
>
>
>
>
>

results differ on two queries, based on secondary index key and partition key

[ferit baver elhuseyni]

Hi all,


We are using a C* 2.2.8 cluster in our production system, composed of 5
nodes in 1 DC with RF=3. Our clients mostly write with CL.ALL and read with
CL.ONE (both will be switched to quorum soon).

We face several problems while trying to persist classical "follow
relationship". Did anyone of you have similar problems / or have any idea
on what could be wrong?

1) First our model. It is based on two tables : follower and following,
that should be identical. First one is for queries on getting followers of
a user, latter is for getting who a user is following.

followings (uid bigint, ts timeuuid, fid bigint, PRIMARY KEY (uid, ts))
WITH CLUSTERING ORDER BY (ts DESC);

followers (uid bigint, ts timeuuid, fid bigint, PRIMARY KEY (uid, ts)) WITH
CLUSTERING ORDER BY (ts DESC);


2) Both tables have secondary indexes on fid columns.

3) Definitely, a new follow relationship should insert one row to each
table and delete should work on both too.



*Problems :*

1) We have a serious discrepancy problems between tables. With "nodetool
cfstats" followings is 18mb, follower is 19mb in total. For demonstration
purposes of this problem, I got followers of the most-followed user from
both tables.

A) select * from followers where uid=12345678
B) select * from followings where fid=12345678

using a small script on unix, i could find out this info on sets A and B:
count( A < B ) = 1247
count( B < A ) = 185
count( A ∩ B ) = 20894


2) Even more interesting than that is, if I query follower table on
secondary index, I don't get a row that I normally get with filtering just
on partition key. Let me try to visualize it :

select uid,ts,fid from followers where fid=X (cannot find uid=12345678)
 A | BBB | X
 C | DDD | X
 E | FFF | X

select uid,ts,fid from followers where uid=12345678 | grep X
 12345678 | GGG | *X*


*My thoughts :*

1) Currently, we don't use batches during inserts and deletes to both
tables. Would this help with our problems?

2) I was first suspicious of a corruption in secondary indexes. But
actually, through the use of secondary index, I get consistent results.

3) I also thought, there could be the case of zombie rows. However we
didn't have any long downtimes with our nodes. But, to our shame, we
haven't been running any scheduled repairs on the cluster.

4) Finally, do you think that there may be problem with our modelling?


Thanks in advance.

Re: scylladb

[Eric Evans]

On Sun, Mar 12, 2017 at 4:01 PM, James Carman
 wrote:
> Does all of this Scylla talk really even belong on the Cassandra user
> mailing list in the first place?

I personally found it interesting, informative, and on-topic when it
was about justification of the 10x performance claim, numa,
scheduling, concurrency, etc.  At some point it got a little sales-y
on the one side, and caremad on the other.

¯\_(ツ)_/¯


-- 
Eric Evans
john.eric.ev...@gmail.com

Does "nodetool repair" need to be run on each node for a given table?

[Thakrar, Jayesh]

I understand that the nodetool command connects to a specific server and for 
many of the commands, e.g. "info", "compactionstats", etc, the information is 
for that specific node.
While for some other commands like "status", the info is for the whole cluster.

So is "nodetool repair" that operates at a single node level (i.e. repairs the 
partitions contained on the target node?).
If so, what is the recommended approach to doing repairs?

E.g. we have a large number of tables (20+), large amount of data (40+ TB) and 
a number of nodes (40+).
Do I need to iterate through each server AND each table?

Thanks,
Jayesh

COPY FROM performance

[Artur R]

HI!

I am trying to increase performance of COPY FROM by installing "*Cython
 and libev
 C extensions"* as
described here:
https://www.datastax.com/dev/blog/six-parameters-affecting-cqlsh-copy-from-performance
.

My steps are as the following:

   1. Install Cassandra 3.10, start it, add keyspace and table

   2. Install C Extensions:
   sudo apt-get install gcc python-dev

   3. Don't install Cython because as far as I understand it ships with C*
   3 by default on step 1), so skip it

   4. Install libev:
   sudo apt-get install libev4 libev-dev

   5. Reinstall C* driver (because as far as I understand it shipped with
   C* on step 1):
   sudo pip install cassandra-driver

   6. export CQLSH_NO_BUNDLED=TRUE

   7. cqlsh to node and try COPY FROM

And after all these steps above performance of COPY FROM is the same as
before.
I tested it with single node cluster and with multiple nodes cluster - it
doesn't impact on performance.
However, I see that COPY FROM is CPU bounded on my machines, so these steps
should definitely increase the performance.


The question: what I did wrong? Maybe some step is missed.
How to check that COPY FROM really uses "*Cython
 and libev
 C extensions"* ?

Re: HELP with bulk loading

[Artur R]

Thank you all!
It turns out that the fastest ways are: https://github.com/brianmhess/
cassandra-loader and COPY FROM.

So I decided to stick with COPY FROM as it built-in and easy-to-use.

On Fri, Mar 10, 2017 at 2:22 PM, Ahmed Eljami 
wrote:

> Hi,
>
> >3. sstableloader is slow too. Assuming that I have new empty C* cluster,
> how can I improve the upload speed? Maybe disable replication or some other
> settings while streaming and then turn it back?
>
> Maybe you can accelerate you load with the option -cph (connection per
> host): https://issues.apache.org/jira/browse/CASSANDRA-3668 and -t=1000
>
> With cph=12 and t=1000,  I went from 56min (default value) to 11min for
> table of 50Gb.
>
>
>
> 2017-03-10 2:09 GMT+01:00 Stefania Alborghetti  datastax.com>:
>
>> When I tested cqlsh COPY FROM for CASSANDRA-11053
>> ,
>> I was able to import about 20 GB in under 4 minutes on a cluster with 8
>> nodes using the same benchmark created for cassandra-loader, provided the
>> driver was Cythonized, instructions in this blog post
>> .
>> The performance was similar to cassandra-loader.
>>
>> Depending on your schema, one or the other may do slightly better.
>>
>> On Fri, Mar 10, 2017 at 8:11 AM, Ryan Svihla  wrote:
>>
>>> I suggest using cassandra loader
>>>
>>> https://github.com/brianmhess/cassandra-loader
>>>
>>> On Mar 9, 2017 5:30 PM, "Artur R"  wrote:
>>>
 Hello all!

 There are ~500gb of CSV files and I am trying to find the way how to
 upload them to C* table (new empty C* cluster of 3 nodes, replication
 factor 2) within reasonable time (say, 10 hours using 3-4 instance of
 c3.8xlarge EC2 nodes).

 My first impulse was to use CQLSSTableWriter, but it is too slow is of
 single instance and I can't efficiently parallelize it (just creating Java
 threads) because after some moment it always "hangs" (looks like GC is
 overstressed) and eats all available memory.

 So the questions are:
 1. What is the best way to bulk-load huge amount of data to new C*
 cluster?

 This comment here: https://issues.apache.org/jira/browse/CASSANDRA-9323
 :

 The preferred way to bulk load is now COPY; see CASSANDRA-11053
>  and linked
> tickets


 is confusing because I read that the CQLSSTableWriter + sstableloader
 is much faster than COPY. Who is right?

 2. Is there any real examples of multi-threaded using of
 CQLSSTableWriter?
 Maybe ready to use libraries like: https://github.com/spotify/hdfs2cass
 ?

 3. sstableloader is slow too. Assuming that I have new empty C*
 cluster, how can I improve the upload speed? Maybe disable replication or
 some other settings while streaming and then turn it back?

 Thanks!
 Artur.

>>>
>>
>>
>> --
>>
>> 
>>
>> STEFANIA ALBORGHETTI
>>
>> Software engineer | +852 6114 9265 <+852%206114%209265> |
>> stefania.alborghe...@datastax.com
>>
>>
>> [image: http://www.datastax.com/cloud-applications]
>> 
>>
>>
>>
>>
>
>
> --
> Cordialement;
>
> Ahmed ELJAMI
>

Re: A Single Dropped Node Fails Entire Read Queries

[Shalom Sagges]

Thanks a lot Joel!

I'll go ahead and upgrade.

Thanks again!


Shalom Sagges
DBA
T: +972-74-700-4035
 
 We Create Meaningful Connections



On Mon, Mar 13, 2017 at 7:27 PM, Joel Knighton 
wrote:

> It's possible that you're hitting https://issues.apache.
> org/jira/browse/CASSANDRA-13009 .
>
> In (simplified) summary, the read query picks the right number of
> endpoints fairly early in its execution. Because the down node has not been
> detected as down yet, it may be one of the nodes. When this node doesn't
> answer, it is likely that speculative retry will kick in after a certain
> amount of time and query an up node. This feature is present and working in
> the earlier releases you tested. Unfortunately, percentile-based
> speculative retry wasn't working as intended in 2.2+ until fixed in
> CASSANDRA-13009, which went into 2.2.9/3.0.11+.
>
> It may be worth evaluating the latest 3.0.x release.
>
> On Mon, Mar 13, 2017 at 11:48 AM, Shalom Sagges 
> wrote:
>
>> Just some more info, I've tried the same scenario on 2.0.14 and 2.1.15
>> and didn't encounter such errors.
>> What I did find is that the timeout errors appear only until the node is
>> discovered as "DN" in nodetool status. Once the node is in DN status, the
>> errors stop and the data is retrieved.
>>
>> Could this be a bug in 3.0.9? Or some sort of misconfiguration I missed?
>>
>> Thanks!
>>
>>
>>
>> Shalom Sagges
>> DBA
>> T: +972-74-700-4035 <+972%2074-700-4035>
>>  
>>  We Create Meaningful Connections
>> 
>>
>>
>> On Sun, Mar 12, 2017 at 10:21 AM, Shalom Sagges 
>> wrote:
>>
>>> Hi Michael,
>>>
>>> If a node suddenly fails, and there are other replicas that can still
>>> satisfy the consistency level, shouldn't the request succeed regardless of
>>> the failed node?
>>>
>>> Thanks!
>>>
>>>
>>>
>>>
>>>
>>> Shalom Sagges
>>> DBA
>>> T: +972-74-700-4035 <+972%2074-700-4035>
>>>  
>>>  We Create Meaningful Connections
>>> 
>>>
>>>
>>> On Fri, Mar 10, 2017 at 6:25 PM, Michael Shuler 
>>> wrote:
>>>
 I may be mistaken on the exact configuration option for the timeout
 you're hitting, but I believe this may be the general
 `request_timeout_in_ms: 1` in conf/cassandra.yaml.

 A reasonable timeout for a "node down" discovery/processing is needed to
 prevent random flapping of nodes with a super short timeout interval.
 Applications should also retry on a host unavailable exception like
 this, because in the long run, this should be expected from time to time
 for network partitions, node failure, maintenance cycles, etc.

 --
 Kind regards,
 Michael

 On 03/10/2017 04:07 AM, Shalom Sagges wrote:
 > Hi daniel,
 >
 > I don't think that's a network issue, because ~10 seconds after the
 node
 > stopped, the queries were successful again without any timeout issues.
 >
 > Thanks!
 >
 >
 > Shalom Sagges
 > DBA
 > T: +972-74-700-4035
 > 
 >    
 >
 >   We Create Meaningful Connections
 >
 > 
 >
 >
 >
 > On Fri, Mar 10, 2017 at 12:01 PM, Daniel Hölbling-Inzko
 >  > wrote:
 >
 > Could there be network issues in connecting between the nodes? If
 > node a gets To be the query coordinator but can't reach b and c is
 > obviously down it won't get a quorum.
 >
 > Greetings
 >
 > Shalom Sagges  > schrieb am Fr. 10. März 2017 um
 10:55:
 >
 > @Ryan, my keyspace replication settings are as follows:
 > CREATE KEYSPACE mykeyspace WITH replication = {'class':
 > 'NetworkTopologyStrategy', 'DC1': '3', 'DC2: '3', 'DC3': '3'}
 >  AND durable_writes = true;
 >
 > CREATE TABLE mykeyspace.test (
 > column1 text,
 > column2 text,
 > column3 text,
 > PRIMARY KEY (column1, column2)
 >
 > The query is */select * from mykeyspace.test where
 > column1='x';/*
 >
 > @Daniel, the replication factor is 3. That's why I don't
 >