Re: Ye old singleton debate

[Ali Akhtar]

+1. Would be awesome if this could be mocked / tested.

On Thu, Mar 16, 2017 at 3:47 AM, Edward Capriolo 
wrote:

> This question came up today:
>
> OK, say you mock, how do you construct a working multi-process
> representation of how C* actually works from within a unit test without
> running the code that actually constructs the cluster?
>
> 1) Don't do that (construct a multinode cluster in a test) just mock the
> crap out of it.
>
> http://www.baeldung.com/mockito-verify
>
> 2) dtests
> Dtest don't actually do this in the classic sense. One challenge is
> code-coverage. For many projects I use cobertura. http://www.
> mojohaus.org/cobertura-maven-plugin/. Cobertura can't (as far as know)
> can not instrument N JVMs and give you coverage. Bringing up a full on
> cluster to test something is slow, compute intensive, and quite hard.
>
> 3) Fix it
> https://issues.apache.org/jira/browse/CASSANDRA-7837
> https://issues.apache.org/jira/browse/CASSANDRA-10283
>
> *Impossible you say!  No NoSQL JAVA DATABASE CAN DO THIS!!!*
>
> https://www.elastic.co/guide/en/elasticsearch/reference/
> current/integration-tests.html
>
> Wouldn't that be just the bees knees???
>
>
>
>
>
>
>

Ye old singleton debate

[Edward Capriolo]

This question came up today:

OK, say you mock, how do you construct a working multi-process
representation of how C* actually works from within a unit test without
running the code that actually constructs the cluster?

1) Don't do that (construct a multinode cluster in a test) just mock the
crap out of it.

http://www.baeldung.com/mockito-verify

2) dtests
Dtest don't actually do this in the classic sense. One challenge is
code-coverage. For many projects I use cobertura.
http://www.mojohaus.org/cobertura-maven-plugin/. Cobertura can't (as far as
know) can not instrument N JVMs and give you coverage. Bringing up a full
on cluster to test something is slow, compute intensive, and quite hard.

3) Fix it
https://issues.apache.org/jira/browse/CASSANDRA-7837
https://issues.apache.org/jira/browse/CASSANDRA-10283

*Impossible you say!  No NoSQL JAVA DATABASE CAN DO THIS!!!*

https://www.elastic.co/guide/en/elasticsearch/reference/current/integration-tests.html

Wouldn't that be just the bees knees???

Re: Does "nodetool repair" need to be run on each node for a given table?

[Thakrar, Jayesh]

Thank you Eric for helping out.
The reason I sent the question a second time is because I did not see my 
question and the first reply from the usergroup.
After I sent the question a second time, I got a personal flame from somebody 
else too and so examined my "spam" folders and that's where I found both.
All the same, I got some very valuable pointers and am pasting here as a summary

https://docs.datastax.com/en/cassandra/2.1/cassandra/tools/toolsRepair.html

https://www.datastax.com/dev/blog/repair-in-cassandra

https://www.pythian.com/blog/effective-anti-entropy-repair-cassandra/

https://www.youtube.com/watch?v=1Sz_K8UID6E




On 3/15/17, 10:22 AM, "Eric Evans"  wrote:

On Tue, Mar 14, 2017 at 12:04 PM, daemeon reiydelle  
wrote:
> Am I unreasonable in expecting a poster to have looked at the 
documentation
> before posting? And that reposting the same query WITHOUT reading the
> documents (when pointed out to them) when asked to do so is not 
appropriate?

It may be reasonable to assume that people will make a good faith
effort to read the docs, but what is unreasonable is to presume that
an RTFM will do any good whatsoever.  In my experience, there is
almost always a more productive approach.

> Do we have a way to blackball such?

No, but no one is making you respond, either.

-- 
Eric Evans
john.eric.ev...@gmail.com

Re: TransportException - Consistency LOCAL_ONE - EC2

[Ryan Svihla]

give it a try see how it behaves

On Mar 15, 2017 10:09 AM, "Frank Hughes"  wrote:

> Thanks Ryan, appreciated again. getPolicy just had this:
>
> Policy policy = new TokenAwarePolicy(DCAwareRoundRobinPolicy.
> builder().build());
>
> so i guess i need
>
> Policy policy = new 
> TokenAwarePolicy(DCAwareRoundRobinPolicy.builder().build(),
> false);
>
> Frank
>
> On 2017-03-15 13:45 (-), Ryan Svihla  wrote:
> > I don't see what getPolicy is retrieving but you want to use TokenAware
> > with the shuffle false option in the ctor, it defaults to shuffle true so
> > that load is spread when people have horribly fat partitions.
> >
> > On Wed, Mar 15, 2017 at 9:41 AM, Frank Hughes 
> > wrote:
> >
> > > Thanks for reply. Much appreciated.
> > >
> > > I should have included more detail. So I am using replication factor 2,
> > > and the code is using a token aware method of distributing the work so
> that
> > > only data that is primarily owned by the node is read on that local
> > > machine. So i guess this points to the logic im using to determine
> what is
> > > primarily owned by a node. I guess this is verging into something that
> > > should be posted to the java driver list, but i'll post here in case
> its
> > > useful or theres an obvious problem:
> > >
> > > PoolingOptions poolingOpts = new PoolingOptions();
> > > poolingOpts.setCoreConnectionsPerHost(HostDistance.REMOTE,
> this.coreConn);
> > > poolingOpts.setMaxConnectionsPerHost(HostDistance.REMOTE,
> this.maxConn);
> > > poolingOpts.setMaxRequestsPerConnection(HostDistance.LOCAL, 32768);
> > > poolingOpts.setMaxRequestsPerConnection(HostDistance.REMOTE, 2000);
> > >
> > > SocketOptions socketOptions = new SocketOptions();
> > > socketOptions.setReadTimeoutMillis(15000);
> > >
> > > Cluster.Builder builder = Cluster.builder();
> > > for(String contactPoint: contactPoints){
> > > builder.addContactPoint(contactPoint.trim());
> > > builder.withPoolingOptions(poolingOpts);
> > > builder.withSocketOptions(socketOptions);
> > > }
> > >
> > > builder.withLoadBalancingPolicy(getPolicy())
> > > .withQueryOptions(new QueryOptions()
> > > .setPrepareOnAllHosts(true)
> > > .setMetadataEnabled(true)
> > > );
> > >
> > > Cluster cluster = builder.build();
> > > Metadata metadata = cluster.getMetadata();
> > > Session session = cluster.connect(keyspaceName);
> > > Set allHosts = metadata.getAllHosts();
> > > int numberOfHost = 4;
> > >
> > > Host localHost = null;
> > > for (Host host : allHosts) {
> > > if(host.getAddress().getHostAddress().equalsIgnoreCase(local))
> > > localHost = host;
> > > }
> > >
> > > Map replicaCount = new HashMap > > List>();
> > > TokenRange[] tokenRanges = unwrapTokenRanges(metadata.
> getTokenRanges()).toArray(new
> > > TokenRange[0]);
> > >
> > > List tokenRangeList = Arrays.asList(tokenRanges);
> > > tokenRangeList.sort(new Comparator() {
> > > @Override
> > > public int compare(TokenRange o1, TokenRange o2) {
> > > return o1.getStart().compareTo(o2.getStart());
> > > }
> > > });
> > >
> > > int numberOfHost = metadata.getAllHosts().size();
> > > int rangesPerHost = tokenRanges.length / numberOfHost;
> > >
> > > for(TokenRange tokenRange : tokenRangeList){
> > >
> > > Set hosts = metadata.getReplicas(keyspaceName, tokenRange);
> > >
> > > String rangeHosts = "";
> > > Iterator iter = hosts.iterator();
> > > while(iter.hasNext()){
> > > Host host = iter.next();
> > >
> > > List tokenRangesForHost = replicaCount.get(host);
> > > if(tokenRangesForHost == null){
> > > tokenRangesForHost = new ArrayList();
> > > }
> > >
> > > if(tokenRangesForHost.size() < rangesPerHost ||
> !iter.hasNext()){
> > > tokenRangesForHost.add(tokenRange);
> > > replicaCount.put(host, tokenRangesForHost);
> > > break;
> > > }
> > >
> > > rangeHosts += host.getAddress().toString();
> > > }
> > > }
> > >
> > > for(Host replica : replicaCount.keySet()){
> > > List allocatedRanges = replicaCount.get(replica);
> > > for(TokenRange tr : replicaCount.get(replica)){
> > > System.out.println(tr.getStart() + " to " + tr.getEnd());
> > > }
> > > }
> > >
> > > //get a list of token ranges for this host
> > > List tokenRangesForHost = replicaCount.get(localHost);
> > >
> > > Again, any thoughts are much appreciated.
> > >
> > > Thanks
> > >
> > > Frank
> > >
> > >
> > > On 2017-03-15 12:38 (-), Ryan Svihla  wrote:
> > > > LOCAL_ONE just means local to the datacenter by default the
> tokenaware
> > > > policy will go to a replica that owns that data (primary or any
> replica
> > > > depends on the driver) and that may or may not be the node the driver
> > > > process is running on.
> > > >
> > > > So to put this more concretely if 

Re: Does "nodetool repair" need to be run on each node for a given table?

[Eric Evans]

On Tue, Mar 14, 2017 at 12:04 PM, daemeon reiydelle  wrote:
> Am I unreasonable in expecting a poster to have looked at the documentation
> before posting? And that reposting the same query WITHOUT reading the
> documents (when pointed out to them) when asked to do so is not appropriate?

It may be reasonable to assume that people will make a good faith
effort to read the docs, but what is unreasonable is to presume that
an RTFM will do any good whatsoever.  In my experience, there is
almost always a more productive approach.

> Do we have a way to blackball such?

No, but no one is making you respond, either.

-- 
Eric Evans
john.eric.ev...@gmail.com

Re: TransportException - Consistency LOCAL_ONE - EC2

[Frank Hughes]

Thanks Ryan, appreciated again. getPolicy just had this:

Policy policy = new TokenAwarePolicy(DCAwareRoundRobinPolicy.builder().build());

so i guess i need 

Policy policy = new TokenAwarePolicy(DCAwareRoundRobinPolicy.builder().build(), 
false);

Frank

On 2017-03-15 13:45 (-), Ryan Svihla  wrote: 
> I don't see what getPolicy is retrieving but you want to use TokenAware
> with the shuffle false option in the ctor, it defaults to shuffle true so
> that load is spread when people have horribly fat partitions.
> 
> On Wed, Mar 15, 2017 at 9:41 AM, Frank Hughes 
> wrote:
> 
> > Thanks for reply. Much appreciated.
> >
> > I should have included more detail. So I am using replication factor 2,
> > and the code is using a token aware method of distributing the work so that
> > only data that is primarily owned by the node is read on that local
> > machine. So i guess this points to the logic im using to determine what is
> > primarily owned by a node. I guess this is verging into something that
> > should be posted to the java driver list, but i'll post here in case its
> > useful or theres an obvious problem:
> >
> > PoolingOptions poolingOpts = new PoolingOptions();
> > poolingOpts.setCoreConnectionsPerHost(HostDistance.REMOTE, this.coreConn);
> > poolingOpts.setMaxConnectionsPerHost(HostDistance.REMOTE, this.maxConn);
> > poolingOpts.setMaxRequestsPerConnection(HostDistance.LOCAL, 32768);
> > poolingOpts.setMaxRequestsPerConnection(HostDistance.REMOTE, 2000);
> >
> > SocketOptions socketOptions = new SocketOptions();
> > socketOptions.setReadTimeoutMillis(15000);
> >
> > Cluster.Builder builder = Cluster.builder();
> > for(String contactPoint: contactPoints){
> > builder.addContactPoint(contactPoint.trim());
> > builder.withPoolingOptions(poolingOpts);
> > builder.withSocketOptions(socketOptions);
> > }
> >
> > builder.withLoadBalancingPolicy(getPolicy())
> > .withQueryOptions(new QueryOptions()
> > .setPrepareOnAllHosts(true)
> > .setMetadataEnabled(true)
> > );
> >
> > Cluster cluster = builder.build();
> > Metadata metadata = cluster.getMetadata();
> > Session session = cluster.connect(keyspaceName);
> > Set allHosts = metadata.getAllHosts();
> > int numberOfHost = 4;
> >
> > Host localHost = null;
> > for (Host host : allHosts) {
> > if(host.getAddress().getHostAddress().equalsIgnoreCase(local))
> > localHost = host;
> > }
> >
> > Map replicaCount = new HashMap > List>();
> > TokenRange[] tokenRanges = 
> > unwrapTokenRanges(metadata.getTokenRanges()).toArray(new
> > TokenRange[0]);
> >
> > List tokenRangeList = Arrays.asList(tokenRanges);
> > tokenRangeList.sort(new Comparator() {
> > @Override
> > public int compare(TokenRange o1, TokenRange o2) {
> > return o1.getStart().compareTo(o2.getStart());
> > }
> > });
> >
> > int numberOfHost = metadata.getAllHosts().size();
> > int rangesPerHost = tokenRanges.length / numberOfHost;
> >
> > for(TokenRange tokenRange : tokenRangeList){
> >
> > Set hosts = metadata.getReplicas(keyspaceName, tokenRange);
> >
> > String rangeHosts = "";
> > Iterator iter = hosts.iterator();
> > while(iter.hasNext()){
> > Host host = iter.next();
> >
> > List tokenRangesForHost = replicaCount.get(host);
> > if(tokenRangesForHost == null){
> > tokenRangesForHost = new ArrayList();
> > }
> >
> > if(tokenRangesForHost.size() < rangesPerHost || !iter.hasNext()){
> > tokenRangesForHost.add(tokenRange);
> > replicaCount.put(host, tokenRangesForHost);
> > break;
> > }
> >
> > rangeHosts += host.getAddress().toString();
> > }
> > }
> >
> > for(Host replica : replicaCount.keySet()){
> > List allocatedRanges = replicaCount.get(replica);
> > for(TokenRange tr : replicaCount.get(replica)){
> > System.out.println(tr.getStart() + " to " + tr.getEnd());
> > }
> > }
> >
> > //get a list of token ranges for this host
> > List tokenRangesForHost = replicaCount.get(localHost);
> >
> > Again, any thoughts are much appreciated.
> >
> > Thanks
> >
> > Frank
> >
> >
> > On 2017-03-15 12:38 (-), Ryan Svihla  wrote:
> > > LOCAL_ONE just means local to the datacenter by default the tokenaware
> > > policy will go to a replica that owns that data (primary or any replica
> > > depends on the driver) and that may or may not be the node the driver
> > > process is running on.
> > >
> > > So to put this more concretely if you have RF 2 with that 4 node cluster
> > so
> > > 2 nodes will be responsible for that data and if your local process is
> > not
> > > running on one of those 2 nodes it will definitely HAVE to go to another
> > > node.
> > >
> > > Therefore, if you wanted to pin behavior to a local replica you'd have to
> > > send your work out in a token aware fashion where said work 

Re: Change the IP of a live node

[Ryan Svihla]

I've actually changed the ip address quite a bit (gossip complains on
startup and happily picks up the new address),  I think this maybe easier
such as..can those ip addresses route to one another ?

As in can the first node with 192.168.xx.xx hit the node with 10.179.xx.xx
on that interface?

On Wed, Mar 15, 2017 at 9:37 AM, kurt greaves  wrote:

> Cassandra uses the IP address for more or less everything. It's possible
> to change it through some hackery however probably not a great idea. The
> nodes system tables will still reference the old IP which is likely your
> problem here.
>
> On 14 March 2017 at 18:58, George Sigletos  wrote:
>
>> To give a complete picture, my node has actually two network interfaces:
>> eth0 for 192.168.xx.xx and eth1 for 10.179.xx.xx
>>
>> On Tue, Mar 14, 2017 at 7:46 PM, George Sigletos 
>> wrote:
>>
>>> Hello,
>>>
>>> I am trying to change the IP of a live node (I am not replacing a dead
>>> one).
>>>
>>> So I stop the service on my node (not a seed node), I change the IP from
>>> 192.168.xx.xx to 10.179.xx.xx, and modify "listen_address" and
>>> "rpc_address" in the cassandra.yaml, while I also set auto_bootstrap:
>>> false. Then I restart but it fails to see the rest of the cluster:
>>>
>>> Datacenter: DC1
>>> ===
>>> Status=Up/Down
>>> |/ State=Normal/Leaving/Joining/Moving
>>> --  AddressLoad   Tokens  OwnsHost
>>> ID   Rack
>>> DN  192.168.xx.xx  ?  256 ?
>>> 241f3002-8f89-4433-a521-4fa4b070b704  r1
>>> UN  10.179.xx.xx  3.45 TB256 ?
>>> 3b07df3b-683b-4e2d-b307-3c48190c8f1c  RAC1
>>> DN  192.168.xx.xx  ?  256 ?
>>> 19636f1e-9417-4354-8364-6617b8d3d20b  r1
>>> DN  192.168.xx.xx?  256 ?
>>> 9c65c71c-f5dd-4267-af9e-a20881cf3d48  r1
>>> DN  192.168.xx.xx   ?  256 ?
>>> ee75219f-0f2c-4be0-bd6d-038315212728  r1
>>>
>>> Am I doing anything wrong? Thanks in advance
>>>
>>> Kind regards,
>>> George
>>>
>>
>>
>


-- 

Thanks,
Ryan Svihla

Re: TransportException - Consistency LOCAL_ONE - EC2

[Ryan Svihla]

I don't see what getPolicy is retrieving but you want to use TokenAware
with the shuffle false option in the ctor, it defaults to shuffle true so
that load is spread when people have horribly fat partitions.

On Wed, Mar 15, 2017 at 9:41 AM, Frank Hughes 
wrote:

> Thanks for reply. Much appreciated.
>
> I should have included more detail. So I am using replication factor 2,
> and the code is using a token aware method of distributing the work so that
> only data that is primarily owned by the node is read on that local
> machine. So i guess this points to the logic im using to determine what is
> primarily owned by a node. I guess this is verging into something that
> should be posted to the java driver list, but i'll post here in case its
> useful or theres an obvious problem:
>
> PoolingOptions poolingOpts = new PoolingOptions();
> poolingOpts.setCoreConnectionsPerHost(HostDistance.REMOTE, this.coreConn);
> poolingOpts.setMaxConnectionsPerHost(HostDistance.REMOTE, this.maxConn);
> poolingOpts.setMaxRequestsPerConnection(HostDistance.LOCAL, 32768);
> poolingOpts.setMaxRequestsPerConnection(HostDistance.REMOTE, 2000);
>
> SocketOptions socketOptions = new SocketOptions();
> socketOptions.setReadTimeoutMillis(15000);
>
> Cluster.Builder builder = Cluster.builder();
> for(String contactPoint: contactPoints){
> builder.addContactPoint(contactPoint.trim());
> builder.withPoolingOptions(poolingOpts);
> builder.withSocketOptions(socketOptions);
> }
>
> builder.withLoadBalancingPolicy(getPolicy())
> .withQueryOptions(new QueryOptions()
> .setPrepareOnAllHosts(true)
> .setMetadataEnabled(true)
> );
>
> Cluster cluster = builder.build();
> Metadata metadata = cluster.getMetadata();
> Session session = cluster.connect(keyspaceName);
> Set allHosts = metadata.getAllHosts();
> int numberOfHost = 4;
>
> Host localHost = null;
> for (Host host : allHosts) {
> if(host.getAddress().getHostAddress().equalsIgnoreCase(local))
> localHost = host;
> }
>
> Map replicaCount = new HashMap List>();
> TokenRange[] tokenRanges = 
> unwrapTokenRanges(metadata.getTokenRanges()).toArray(new
> TokenRange[0]);
>
> List tokenRangeList = Arrays.asList(tokenRanges);
> tokenRangeList.sort(new Comparator() {
> @Override
> public int compare(TokenRange o1, TokenRange o2) {
> return o1.getStart().compareTo(o2.getStart());
> }
> });
>
> int numberOfHost = metadata.getAllHosts().size();
> int rangesPerHost = tokenRanges.length / numberOfHost;
>
> for(TokenRange tokenRange : tokenRangeList){
>
> Set hosts = metadata.getReplicas(keyspaceName, tokenRange);
>
> String rangeHosts = "";
> Iterator iter = hosts.iterator();
> while(iter.hasNext()){
> Host host = iter.next();
>
> List tokenRangesForHost = replicaCount.get(host);
> if(tokenRangesForHost == null){
> tokenRangesForHost = new ArrayList();
> }
>
> if(tokenRangesForHost.size() < rangesPerHost || !iter.hasNext()){
> tokenRangesForHost.add(tokenRange);
> replicaCount.put(host, tokenRangesForHost);
> break;
> }
>
> rangeHosts += host.getAddress().toString();
> }
> }
>
> for(Host replica : replicaCount.keySet()){
> List allocatedRanges = replicaCount.get(replica);
> for(TokenRange tr : replicaCount.get(replica)){
> System.out.println(tr.getStart() + " to " + tr.getEnd());
> }
> }
>
> //get a list of token ranges for this host
> List tokenRangesForHost = replicaCount.get(localHost);
>
> Again, any thoughts are much appreciated.
>
> Thanks
>
> Frank
>
>
> On 2017-03-15 12:38 (-), Ryan Svihla  wrote:
> > LOCAL_ONE just means local to the datacenter by default the tokenaware
> > policy will go to a replica that owns that data (primary or any replica
> > depends on the driver) and that may or may not be the node the driver
> > process is running on.
> >
> > So to put this more concretely if you have RF 2 with that 4 node cluster
> so
> > 2 nodes will be responsible for that data and if your local process is
> not
> > running on one of those 2 nodes it will definitely HAVE to go to another
> > node.
> >
> > Therefore, if you wanted to pin behavior to a local replica you'd have to
> > send your work out in a token aware fashion where said work only goes to
> > the primary token owner of that data, and remove any shuffling of
> replicas
> > in the process (is only on by default in the java driver to my
> knowledge).
> >
> > On Wed, Mar 15, 2017 at 6:38 AM, Frank Hughes 
> > wrote:
> >
> > > Hi there,
> > >
> > > Im running a java process on a 4 node cassandra 3.9 cluster on EC2
> > > (instance type t2.2xlarge), the process running separately on each of
> the
> > > nodes (i.e. 4 running JVMs).
> > > The process is just doing reads from Cassandra and building a SOLR
> index
> > > 

Re: Change the IP of a live node

[kurt greaves]

Cassandra uses the IP address for more or less everything. It's possible to
change it through some hackery however probably not a great idea. The nodes
system tables will still reference the old IP which is likely your problem
here.

On 14 March 2017 at 18:58, George Sigletos  wrote:

> To give a complete picture, my node has actually two network interfaces:
> eth0 for 192.168.xx.xx and eth1 for 10.179.xx.xx
>
> On Tue, Mar 14, 2017 at 7:46 PM, George Sigletos 
> wrote:
>
>> Hello,
>>
>> I am trying to change the IP of a live node (I am not replacing a dead
>> one).
>>
>> So I stop the service on my node (not a seed node), I change the IP from
>> 192.168.xx.xx to 10.179.xx.xx, and modify "listen_address" and
>> "rpc_address" in the cassandra.yaml, while I also set auto_bootstrap:
>> false. Then I restart but it fails to see the rest of the cluster:
>>
>> Datacenter: DC1
>> ===
>> Status=Up/Down
>> |/ State=Normal/Leaving/Joining/Moving
>> --  AddressLoad   Tokens  OwnsHost
>> ID   Rack
>> DN  192.168.xx.xx  ?  256 ?
>> 241f3002-8f89-4433-a521-4fa4b070b704  r1
>> UN  10.179.xx.xx  3.45 TB256 ?
>> 3b07df3b-683b-4e2d-b307-3c48190c8f1c  RAC1
>> DN  192.168.xx.xx  ?  256 ?
>> 19636f1e-9417-4354-8364-6617b8d3d20b  r1
>> DN  192.168.xx.xx?  256 ?
>> 9c65c71c-f5dd-4267-af9e-a20881cf3d48  r1
>> DN  192.168.xx.xx   ?  256 ?
>> ee75219f-0f2c-4be0-bd6d-038315212728  r1
>>
>> Am I doing anything wrong? Thanks in advance
>>
>> Kind regards,
>> George
>>
>
>

Re: TransportException - Consistency LOCAL_ONE - EC2

[Frank Hughes]

Thanks for reply. Much appreciated.

I should have included more detail. So I am using replication factor 2, and the 
code is using a token aware method of distributing the work so that only data 
that is primarily owned by the node is read on that local machine. So i guess 
this points to the logic im using to determine what is primarily owned by a 
node. I guess this is verging into something that should be posted to the java 
driver list, but i'll post here in case its useful or theres an obvious problem:

PoolingOptions poolingOpts = new PoolingOptions();
poolingOpts.setCoreConnectionsPerHost(HostDistance.REMOTE, this.coreConn);
poolingOpts.setMaxConnectionsPerHost(HostDistance.REMOTE, this.maxConn);
poolingOpts.setMaxRequestsPerConnection(HostDistance.LOCAL, 32768);
poolingOpts.setMaxRequestsPerConnection(HostDistance.REMOTE, 2000);

SocketOptions socketOptions = new SocketOptions();
socketOptions.setReadTimeoutMillis(15000);

Cluster.Builder builder = Cluster.builder();
for(String contactPoint: contactPoints){
builder.addContactPoint(contactPoint.trim());
builder.withPoolingOptions(poolingOpts);
builder.withSocketOptions(socketOptions);
}

builder.withLoadBalancingPolicy(getPolicy())
.withQueryOptions(new QueryOptions()
.setPrepareOnAllHosts(true)
.setMetadataEnabled(true)
);

Cluster cluster = builder.build();   
Metadata metadata = cluster.getMetadata();
Session session = cluster.connect(keyspaceName);
Set allHosts = metadata.getAllHosts();
int numberOfHost = 4;

Host localHost = null;
for (Host host : allHosts) {
if(host.getAddress().getHostAddress().equalsIgnoreCase(local))
localHost = host;
}

Map replicaCount = new HashMap();
TokenRange[] tokenRanges = 
unwrapTokenRanges(metadata.getTokenRanges()).toArray(new TokenRange[0]);

List tokenRangeList = Arrays.asList(tokenRanges);
tokenRangeList.sort(new Comparator() {
@Override
public int compare(TokenRange o1, TokenRange o2) {
return o1.getStart().compareTo(o2.getStart());
}
});

int numberOfHost = metadata.getAllHosts().size();
int rangesPerHost = tokenRanges.length / numberOfHost;

for(TokenRange tokenRange : tokenRangeList){

Set hosts = metadata.getReplicas(keyspaceName, tokenRange);

String rangeHosts = "";
Iterator iter = hosts.iterator();
while(iter.hasNext()){
Host host = iter.next();

List tokenRangesForHost = replicaCount.get(host);
if(tokenRangesForHost == null){
tokenRangesForHost = new ArrayList();
}

if(tokenRangesForHost.size() < rangesPerHost || !iter.hasNext()){
tokenRangesForHost.add(tokenRange);
replicaCount.put(host, tokenRangesForHost);
break;
}

rangeHosts += host.getAddress().toString();
}
}

for(Host replica : replicaCount.keySet()){
List allocatedRanges = replicaCount.get(replica);
for(TokenRange tr : replicaCount.get(replica)){
System.out.println(tr.getStart() + " to " + tr.getEnd());
}
}

//get a list of token ranges for this host
List tokenRangesForHost = replicaCount.get(localHost);

Again, any thoughts are much appreciated.

Thanks

Frank


On 2017-03-15 12:38 (-), Ryan Svihla  wrote: 
> LOCAL_ONE just means local to the datacenter by default the tokenaware
> policy will go to a replica that owns that data (primary or any replica
> depends on the driver) and that may or may not be the node the driver
> process is running on.
> 
> So to put this more concretely if you have RF 2 with that 4 node cluster so
> 2 nodes will be responsible for that data and if your local process is not
> running on one of those 2 nodes it will definitely HAVE to go to another
> node.
> 
> Therefore, if you wanted to pin behavior to a local replica you'd have to
> send your work out in a token aware fashion where said work only goes to
> the primary token owner of that data, and remove any shuffling of replicas
> in the process (is only on by default in the java driver to my knowledge).
> 
> On Wed, Mar 15, 2017 at 6:38 AM, Frank Hughes 
> wrote:
> 
> > Hi there,
> >
> > Im running a java process on a 4 node cassandra 3.9 cluster on EC2
> > (instance type t2.2xlarge), the process running separately on each of the
> > nodes (i.e. 4 running JVMs).
> > The process is just doing reads from Cassandra and building a SOLR index
> > and using the java driver with consistency level LOCAL_ONE.
> > However, the following exception is through:
> >
> > com.datastax.driver.core.exceptions.TransportException: [/10.0.0.2:9042]
> > Connection has been closed
> > at com.datastax.driver.core.exceptions.TransportException.
> > copy(TransportException.java:38)
> > at com.datastax.driver.core.exceptions.TransportException.
> > copy(TransportException.java:24)
> > at com.datastax.driver.core.DriverThrowables.propagateCause(
> > 

Re: changing compaction strategy

[kurt greaves]

The rogue pending task is likely a non-issue. If your jmx command went
through without errors and you got the log message you can assume it
worked. It won't show in the schema unless you run the ALTER statement
which affects the whole cluster.

If you were switching from STCS then you wouldn't expect any recompaction,
as SSTables will just be calculated to be in their relevant window. On the
other hand switching from LCS would generate many compactions.
​
The only way to really tell is to confirm SSTables are expiring as you
would expect them to, or verify that new SSTables are not being compacted
with older ones. You might need to wait for at least one time window to
pass to check this.

Re: Internal Security - Authentication & Authorization

[Sam Tunnicliffe]

>
> Here is what I have pieced together. Please let me know if I am on the
> right track.


You're more or less right regarding the built in
authenticator/authorizer/role manager (which are usually referred to as
"internal" as they store their data in Cassandra tables). One important
thing to note is that using the default superuser credentials (i.e. logging
in as 'cassandra') will make these reads happen at QUORUM, not LOCAL_ONE,
which is one of the reasons you shouldn't use those credentials
after initial setup.

There are 2 settings which govern the lifetime of cached auth data. Once an
item has been cached, it becomes eligible for refresh after the update
interval has passed. A get from the cache will trigger this refresh, which
happens in the background and while it's running, the old (maybe stale)
entry is served from the cache. When the validity period expires for a
cache entry, it is removed from the cache and subsequent reads trigger a
blocking fetch from storage.

There is further detail in the docs here:
http://cassandra.apache.org/doc/latest/operating/security.html

 If NOT EXISTS will use SERIAL consistency


This isn't actually true. Because internal storage is just one
implementation of role/user management, it doesn't rely on LWT. Instead,
the configured role manager is consulted before executing the statement,
which is similar to how IF NOT EXISTS in schema updates work.


On Tue, Mar 14, 2017 at 11:44 PM, Jai Bheemsen Rao Dhanwada <
jaibheem...@gmail.com> wrote:

> I have similar question. when we create users or roles what is the
> consistency level used?
>
> I know, If NOT EXISTS will use SERIAL consistency. what consistency will
> be used if just use CREATE USER ?
>
> On Mon, Mar 13, 2017 at 7:09 PM, Jacob Shadix 
> wrote:
>
>> I'm looking for a deeper understanding of how Cassandra interacts with
>> the system_auth keyspace to authenticate/authorize users.
>>
>> Here is what I have pieced together. Please let me know if I am on the
>> right track.
>>
>> A user attempts to connect to Cassandra. Cassandra checks against
>> system_auth for that user @ LOCAL_ONE - - If the user exists, a connection
>> is established. When CQL is executed, C* again checks system_auth for that
>> user @ LOCAL_ONE to determine if it has the correct privileges to perform
>> the CQL. If so, it executes the CQL and the permissions are stored in a
>> cache. During the cache validity timeframe, future requests for ANY user
>> stored in the cache do not require a lookup against system_auth. After the
>> cache validity runs out, any new requests will require a lookup against
>> system_auth.
>>
>> -- Jacob Shadix
>>
>
>

Re: Internal Security - Authentication & Authorization

[kurt greaves]

Jacob, seems you are on the right track however my understanding is that
only the user that was auth'd has their permissions/roles/creds cached.

Also. Cassandra will query at QUORUM for the "cassandra" user, and at
LOCAL_ONE for *all* other users. This is the same for creating users/roles.

Re: TransportException - Consistency LOCAL_ONE - EC2

[Ryan Svihla]

LOCAL_ONE just means local to the datacenter by default the tokenaware
policy will go to a replica that owns that data (primary or any replica
depends on the driver) and that may or may not be the node the driver
process is running on.

So to put this more concretely if you have RF 2 with that 4 node cluster so
2 nodes will be responsible for that data and if your local process is not
running on one of those 2 nodes it will definitely HAVE to go to another
node.

Therefore, if you wanted to pin behavior to a local replica you'd have to
send your work out in a token aware fashion where said work only goes to
the primary token owner of that data, and remove any shuffling of replicas
in the process (is only on by default in the java driver to my knowledge).

On Wed, Mar 15, 2017 at 6:38 AM, Frank Hughes 
wrote:

> Hi there,
>
> Im running a java process on a 4 node cassandra 3.9 cluster on EC2
> (instance type t2.2xlarge), the process running separately on each of the
> nodes (i.e. 4 running JVMs).
> The process is just doing reads from Cassandra and building a SOLR index
> and using the java driver with consistency level LOCAL_ONE.
> However, the following exception is through:
>
> com.datastax.driver.core.exceptions.TransportException: [/10.0.0.2:9042]
> Connection has been closed
> at com.datastax.driver.core.exceptions.TransportException.
> copy(TransportException.java:38)
> at com.datastax.driver.core.exceptions.TransportException.
> copy(TransportException.java:24)
> at com.datastax.driver.core.DriverThrowables.propagateCause(
> DriverThrowables.java:37)
> at com.datastax.driver.core.ArrayBackedResultSet$
> MultiPage.prepareNextRow(ArrayBackedResultSet.java:313)
> at com.datastax.driver.core.ArrayBackedResultSet$
> MultiPage.isExhausted(ArrayBackedResultSet.java:269)
> at com.datastax.driver.core.ArrayBackedResultSet$1.
> hasNext(ArrayBackedResultSet.java:143)
>
> where 10.0.0.2 is not the local machine. So my questions:
>
> - Should this happen when Im using consistency level LOCAL_ONE and just
> doing reads ?
> - Does this suggest non-local reads are happening ?
>
> Many thanks for any help/ideas.
>
> Frank
>
>
>


-- 

Thanks,
Ryan Svihla

TransportException - Consistency LOCAL_ONE - EC2

[Frank Hughes]

Hi there,

Im running a java process on a 4 node cassandra 3.9 cluster on EC2 (instance 
type t2.2xlarge), the process running separately on each of the nodes (i.e. 4 
running JVMs).
The process is just doing reads from Cassandra and building a SOLR index and 
using the java driver with consistency level LOCAL_ONE.
However, the following exception is through:

com.datastax.driver.core.exceptions.TransportException: [/10.0.0.2:9042] 
Connection has been closed
at 
com.datastax.driver.core.exceptions.TransportException.copy(TransportException.java:38)
at 
com.datastax.driver.core.exceptions.TransportException.copy(TransportException.java:24)
at 
com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
at 
com.datastax.driver.core.ArrayBackedResultSet$MultiPage.prepareNextRow(ArrayBackedResultSet.java:313)
at 
com.datastax.driver.core.ArrayBackedResultSet$MultiPage.isExhausted(ArrayBackedResultSet.java:269)
at 
com.datastax.driver.core.ArrayBackedResultSet$1.hasNext(ArrayBackedResultSet.java:143)

where 10.0.0.2 is not the local machine. So my questions:

- Should this happen when Im using consistency level LOCAL_ONE and just doing 
reads ?
- Does this suggest non-local reads are happening ?

Many thanks for any help/ideas.

Frank

Re: Slow repair

[Ben Slater]

When you say you’re running repair to “rebalance” do you mean to populate
the new DC? If so, the normal/correct procedure is to use nodetool rebuild
rather than repair. See
https://docs.datastax.com/en/cassandra/2.1/cassandra/operations/ops_add_dc_to_cluster_t.html
for
the full details.

Cheers
Ben

On Wed, 15 Mar 2017 at 21:14 Gábor Auth  wrote:

> Hi,
>
> We are working with a two DCs Cassandra cluster (EU and US), so that the
> distance is over 160 ms between them. I've added a new DC to this cluster,
> modified the keyspace's replication factor and trying to rebalance it with
> repair but the repair is very slow (over 10-15 minutes per node per
> keyspace with ~40 column families). Is it normal with this network latency
> or something wrong with the cluster or the network connection? :)
>
> [2017-03-15 05:52:38,255] Starting repair command #4, repairing keyspace
> test20151222 with repair options (parallelism: parallel, primary range:
> true, incremental: false, job threads: 1, ColumnFamilies: [], dataCenters:
> [], hosts: [], # of ranges: 32)
> [2017-03-15 05:54:11,913] Repair session
> 988bd850-0943-11e7-9c1f-f5ba092c6aea for range
> [(-3328182031191101706,-3263206086630594139],
> (-449681117114180865,-426983008087217811],
> (-4940101276128910421,-4726878962587262390],
> (-4999008077542282524,-4940101276128910421]] finished (progress: 11%)
> [2017-03-15 05:55:39,721] Repair session
> 9a6fda92-0943-11e7-9c1f-f5ba092c6aea for range
> [(7538662821591320245,7564364667721298414],
> (8095771383100385537,8112071444788258953],
> (-1625703837190283897,-1600176580612824092],
> (-1075557915997532230,-1072724867906442440], (-9152
> 563942239372475,-9123254980705325471],
> (7485905313674392326,7513617239634230698]] finished (progress: 14%)
> [2017-03-15 05:57:05,718] Repair session
> 9de181b1-0943-11e7-9c1f-f5ba092c6aea for range
> [(-6471953894734787784,-6420063839816736750],
> (1372322727565611879,1480899944406172322],
> (1176263633569625668,1177285361971054591],
> (440549646067640682,491840653569315468], (-43128299
> 75221321282,-4177428401237878410]] finished (progress: 17%)
> [2017-03-15 05:58:39,997] Repair session
> a18bc500-0943-11e7-9c1f-f5ba092c6aea for range
> [(5327651902976749177,5359189884199963589],
> (-5362946313988105342,-5348008210198062914],
> (-5756557262823877856,-5652851311492822149],
> (-5400778420101537991,-5362946313988105342], (668
> 2536072120412021,6904193483670147322]] finished (progress: 20%)
> [2017-03-15 05:59:11,791] Repair session
> a44f2ac2-0943-11e7-9c1f-f5ba092c6aea for range
> [(952873612468870228,1042958763135655298],
> (558544893991295379,572114658167804730]] finished (progress: 22%)
> [2017-03-15 05:59:56,197] Repair session
> a5e13c71-0943-11e7-9c1f-f5ba092c6aea for range
> [(1914238614647876002,1961526714897144472],
> (3610056520286573718,3619622957324752442],
> (-3506227577233676363,-3504718440405535976],
> (-4120686433235827731,-4098515820338981500], (56515
> 94158011135924,5668698324546997949]] finished (progress: 25%)
> [2017-03-15 06:00:45,610] Repair session
> a897a9e1-0943-11e7-9c1f-f5ba092c6aea for range
> [(-9007733666337543056,-8979974976044921941]] finished (progress: 28%)
> [2017-03-15 06:01:58,826] Repair session
> a927b4e1-0943-11e7-9c1f-f5ba092c6aea for range
> [(3599745202434925817,3608662806723095677],
> (3390003128426746316,3391135639180043521],
> (3391135639180043521,3529019003015169892]] finished (progress: 31%)
> [2017-03-15 06:03:15,440] Repair session
> aae06160-0943-11e7-9c1f-f5ba092c6aea for range
> [(-7542303048667795773,-7300899534947316960]] finished (progress: 34%)
> [2017-03-15 06:03:17,786] Repair completed successfully
> [2017-03-15 06:03:17,787] Repair command #4 finished in 10 minutes 39
> seconds
>
> Bye,
> Gábor Auth
>
> --


*Ben Slater*

*Chief Product Officer *

   


Read our latest technical blog posts here
.

This email has been sent on behalf of Instaclustr Pty. Limited (Australia)
and Instaclustr Inc (USA).

This email and any attachments may contain confidential and legally
privileged information.  If you are not the intended recipient, do not copy
or disclose its content, but please reply to this email immediately and
highlight the error to the sender and then immediately delete the message.

Slow repair

[Gábor Auth]

Hi,

We are working with a two DCs Cassandra cluster (EU and US), so that the
distance is over 160 ms between them. I've added a new DC to this cluster,
modified the keyspace's replication factor and trying to rebalance it with
repair but the repair is very slow (over 10-15 minutes per node per
keyspace with ~40 column families). Is it normal with this network latency
or something wrong with the cluster or the network connection? :)

[2017-03-15 05:52:38,255] Starting repair command #4, repairing keyspace
test20151222 with repair options (parallelism: parallel, primary range:
true, incremental: false, job threads: 1, ColumnFamilies: [], dataCenters:
[], hosts: [], # of ranges: 32)
[2017-03-15 05:54:11,913] Repair session
988bd850-0943-11e7-9c1f-f5ba092c6aea for range
[(-3328182031191101706,-3263206086630594139],
(-449681117114180865,-426983008087217811],
(-4940101276128910421,-4726878962587262390],
(-4999008077542282524,-4940101276128910421]] finished (progress: 11%)
[2017-03-15 05:55:39,721] Repair session
9a6fda92-0943-11e7-9c1f-f5ba092c6aea for range
[(7538662821591320245,7564364667721298414],
(8095771383100385537,8112071444788258953],
(-1625703837190283897,-1600176580612824092],
(-1075557915997532230,-1072724867906442440], (-9152
563942239372475,-9123254980705325471],
(7485905313674392326,7513617239634230698]] finished (progress: 14%)
[2017-03-15 05:57:05,718] Repair session
9de181b1-0943-11e7-9c1f-f5ba092c6aea for range
[(-6471953894734787784,-6420063839816736750],
(1372322727565611879,1480899944406172322],
(1176263633569625668,1177285361971054591],
(440549646067640682,491840653569315468], (-43128299
75221321282,-4177428401237878410]] finished (progress: 17%)
[2017-03-15 05:58:39,997] Repair session
a18bc500-0943-11e7-9c1f-f5ba092c6aea for range
[(5327651902976749177,5359189884199963589],
(-5362946313988105342,-5348008210198062914],
(-5756557262823877856,-5652851311492822149],
(-5400778420101537991,-5362946313988105342], (668
2536072120412021,6904193483670147322]] finished (progress: 20%)
[2017-03-15 05:59:11,791] Repair session
a44f2ac2-0943-11e7-9c1f-f5ba092c6aea for range
[(952873612468870228,1042958763135655298],
(558544893991295379,572114658167804730]] finished (progress: 22%)
[2017-03-15 05:59:56,197] Repair session
a5e13c71-0943-11e7-9c1f-f5ba092c6aea for range
[(1914238614647876002,1961526714897144472],
(3610056520286573718,3619622957324752442],
(-3506227577233676363,-3504718440405535976],
(-4120686433235827731,-4098515820338981500], (56515
94158011135924,5668698324546997949]] finished (progress: 25%)
[2017-03-15 06:00:45,610] Repair session
a897a9e1-0943-11e7-9c1f-f5ba092c6aea for range
[(-9007733666337543056,-8979974976044921941]] finished (progress: 28%)
[2017-03-15 06:01:58,826] Repair session
a927b4e1-0943-11e7-9c1f-f5ba092c6aea for range
[(3599745202434925817,3608662806723095677],
(3390003128426746316,3391135639180043521],
(3391135639180043521,3529019003015169892]] finished (progress: 31%)
[2017-03-15 06:03:15,440] Repair session
aae06160-0943-11e7-9c1f-f5ba092c6aea for range
[(-7542303048667795773,-7300899534947316960]] finished (progress: 34%)
[2017-03-15 06:03:17,786] Repair completed successfully
[2017-03-15 06:03:17,787] Repair command #4 finished in 10 minutes 39
seconds

Bye,
Gábor Auth