[jira] [Commented] (CASSANDRA-12627) Provide new seed providers

[Edward Capriolo (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951153#comment-15951153
 ] 

Edward Capriolo commented on CASSANDRA-12627:
-

Giving up on this one. Maybe I dont understand why exists a "plugable" seed 
provider so that there can only be 1 implementation of it.

> Provide new seed providers
> --
>
> Key: CASSANDRA-12627
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12627
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Edward Capriolo
>
> SeedProvider is plugable, however only one implementation exists.
> Changes:
> * Create a SeedProvider that reads properties from System properties or env
> * Provide a SeedProvider that scans ranges of IP addresses to find peers.
> * Refactor interface to abstract class because all seed providers must 
> provide a constructor that accepts Map 
> * correct error messages
> * Do not catch Exception use MultiCatch and catch typed exceptions



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CASSANDRA-12627) Provide new seed providers

[Edward Capriolo (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15670567#comment-15670567
 ] 

Edward Capriolo commented on CASSANDRA-12627:
-

[~jasobrown] We attempted to discussion this offline but have been playing 
phone tag for ~ two weeks now.

Here is what I am going to do.

* Rebase
* Remove NeighborSeedProvider
* Rename PropertyOrEnvironmentSeedProvider to PropertyBasedSeedProvider

I think my last comment demonstrates why PropertyBasedSeedProvider it is 
useful. It allows people to integrate with a general set of things. Including 
shell scripts, http servers, and dns servers.


> Provide new seed providers
> --
>
> Key: CASSANDRA-12627
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12627
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>
> SeedProvider is plugable, however only one implementation exists.
> Changes:
> * Create a SeedProvider that reads properties from System properties or env
> * Provide a SeedProvider that scans ranges of IP addresses to find peers.
> * Refactor interface to abstract class because all seed providers must 
> provide a constructor that accepts Map 
> * correct error messages
> * Do not catch Exception use MultiCatch and catch typed exceptions



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12627) Provide new seed providers

[Edward Capriolo (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15638193#comment-15638193
 ] 

Edward Capriolo commented on CASSANDRA-12627:
-

I am a bit confused. I am looking at the code you are sending me here:
{quote}
 String priamSeeds = 
DataFetcher.fetchData("http://127.0.0.1:8080/Priam/REST/v1/cassconfig/get_seeds;);
for (String seed : priamSeeds.split(","))
seeds.add(InetAddress.getByName(seed));
{quote}

With the feature I am proposing you can replace that with:

CASSANDRA_SEED_LIST=`wget 
http://127.0.0.1:8080/Priam/REST/v1/cassconfig/get_seeds` bin/cassandra

You could also do

CASSANDRA_SEED_LIST=`dig -t txt mydomain.com` bin/cassandra

Doesn't this seem extremely useful and straight forward?

> Provide new seed providers
> --
>
> Key: CASSANDRA-12627
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12627
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>
> SeedProvider is plugable, however only one implementation exists.
> Changes:
> * Create a SeedProvider that reads properties from System properties or env
> * Provide a SeedProvider that scans ranges of IP addresses to find peers.
> * Refactor interface to abstract class because all seed providers must 
> provide a constructor that accepts Map 
> * correct error messages
> * Do not catch Exception use MultiCatch and catch typed exceptions



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12627) Provide new seed providers

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15638108#comment-15638108
 ] 

Jason Brown commented on CASSANDRA-12627:
-

bq. Operators should not actually need to muck around with the yaml file anyway

Well, you already need to edit the yaml in order to define the 
{{SeedProdvider}}, let alone setting addresses, ports, and the like. For better 
or worse, operators will need to deal with the yaml. Thus, I'm -1 on 
{[PropertyOrEnvironmentSeedProvider}}.

{{NeighborSeedProvider}} gets into equally an unexpected space, as well. By 
naively setting the {{scan.distance}} too wide, we'd include nodes in the list 
that do not exist or are not c* nodes. In {{Gossiper}}, we will try to connect 
and gossip with those nodes as seeds ({{Gossiper#maybeGossipToSeed()}}). That 
means extra unix sockets (really not a problem), but also extra threads due to 
the existing internode messaging service implementation (see 
{{OutboundTcpConnectionPool}}). Those extra threads are expensive in large 
clusters. I'm not convinced that setting explicit addresses as the seed(s) is 
more difficult or requires more config understanding than defining a 
{{scan.distance}}. Again, I'm -1 on {{NeighborSeedProvider}}

bq. why is the configuration so obtuse and plugable if only one implementation 
exists?

For the record, there are other provider implementations, some public (like 
[Priam|https://github.com/Netflix/Priam/blob/master/priam-cass-extensions/src/main/java/com/netflix/priam/cassandra/extensions/NFSeedProvider.java]),
 some private. In those cases, special functionality is needed, and the space 
to plug it in is provided. The trade off, of course, is an increased config 
complexity that affects all users, in some small way.

I agree that the yaml config is non-trivial, but even that is pluggable. If we 
want to entertain a "config-lite" for new users/operator to get something stood 
up quickly, that might be a great thing - but outside the scope of this ticket.


> Provide new seed providers
> --
>
> Key: CASSANDRA-12627
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12627
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>
> SeedProvider is plugable, however only one implementation exists.
> Changes:
> * Create a SeedProvider that reads properties from System properties or env
> * Provide a SeedProvider that scans ranges of IP addresses to find peers.
> * Refactor interface to abstract class because all seed providers must 
> provide a constructor that accepts Map 
> * correct error messages
> * Do not catch Exception use MultiCatch and catch typed exceptions



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12627) Provide new seed providers

[Edward Capriolo (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15637420#comment-15637420
 ] 

Edward Capriolo commented on CASSANDRA-12627:
-

PropertyOrEnvironmentSeedProvider 

{quote}
muck about with the yaml anyways
{quote}

Operators should not actually need to muck around with the yaml file anyway. In 
the old cassandra days we ONLY had the configuration file to find seeds. Now 
the known hosts are stored in system tables anyway. Thus the seeds defined in 
the yaml file are fairly useless after initialization just like the tokens. We 
should break free of maintaining this. 

NeighborSeedProvider you are correct to say that this implementation is not 
useful in a cloud environment with random IP allocation. However in non-cloud 
environments machines are typically given IP addresses in a subnet and they for 
large networks and administrator will pre-allocate a subnet.

For example, an administrator will say we are going to use the network 
10.1.1.0/255.255.255.0 as a class C network for Cassandra servers. Given our 
own IP address we can know where the others will be. 

In general you can argue the merits of each provider. The next logical question 
is why is the configuration so obtuse and plugable if only one implementation 
exists?

> Provide new seed providers
> --
>
> Key: CASSANDRA-12627
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12627
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>
> SeedProvider is plugable, however only one implementation exists.
> Changes:
> * Create a SeedProvider that reads properties from System properties or env
> * Provide a SeedProvider that scans ranges of IP addresses to find peers.
> * Refactor interface to abstract class because all seed providers must 
> provide a constructor that accepts Map 
> * correct error messages
> * Do not catch Exception use MultiCatch and catch typed exceptions



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12627) Provide new seed providers

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15637148#comment-15637148
 ] 

Jason Brown commented on CASSANDRA-12627:
-

Thanks for the patch , [~appodictic]. Here's my initial thoughts:

- {{SeedProvider}}, I agree that anyone wanting to implement their own provider 
would need to know (and probably trip up on) that we require a constructor that 
takes a {{Map}} as an argument. That change makes sense.

- {{PropertyOrEnvironmentSeedProvider}}, I'm trying to understand the upside 
value of this. Sure, it's hypothetically "simpler" to pass in a {{-D}} prop or 
set an env variable, but won't operators already need to muck about with the 
yaml anyways, for other values? By introducing this new seed provider, do we 
just spread out the configuration burden to more places?

- {{NeighborSeedProvider}}, can you explain what the value is of this class? It 
appears to naively add a range of {{InetAddress}} centered around the current 
node's address - and then use those as seeds. That seed list would only be 
visible on that node as we don't have a shared, distributed notion of what 
nodes are "seeds". I'm trying to imagine where {{NeighborSeedProvider}} would 
be useful, especially in a "cloud environment", where IP address assignment, at 
least from my previous EC2 experience, is largely random.

> Provide new seed providers
> --
>
> Key: CASSANDRA-12627
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12627
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>
> SeedProvider is plugable, however only one implementation exists.
> Changes:
> * Create a SeedProvider that reads properties from System properties or env
> * Provide a SeedProvider that scans ranges of IP addresses to find peers.
> * Refactor interface to abstract class because all seed providers must 
> provide a constructor that accepts Map 
> * correct error messages
> * Do not catch Exception use MultiCatch and catch typed exceptions



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12627) Provide new seed providers

[Edward Capriolo (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15575487#comment-15575487
 ] 

Edward Capriolo commented on CASSANDRA-12627:
-

These are massively helpful in cloud environments as the user can provision 
without much hacking of the yaml file.

> Provide new seed providers
> --
>
> Key: CASSANDRA-12627
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12627
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>
> SeedProvider is plugable, however only one implementation exists.
> Changes:
> * Create a SeedProvider that reads properties from System properties or env
> * Provide a SeedProvider that scans ranges of IP addresses to find peers.
> * Refactor interface to abstract class because all seed providers must 
> provide a constructor that accepts Map 
> * correct error messages
> * Do not catch Exception use MultiCatch and catch typed exceptions



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12627) Provide new seed providers

[Edward Capriolo (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15480385#comment-15480385
 ] 

Edward Capriolo commented on CASSANDRA-12627:
-

https://github.com/edwardcapriolo/cassandra/tree/CASSANDRA-12627

> Provide new seed providers
> --
>
> Key: CASSANDRA-12627
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12627
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>
> SeedProvider is plugable, however only one implementation exists.
> Changes:
> * Create a SeedProvider that reads properties from System properties or env
> * Provide a SeedProvider that scans ranges of IP addresses to find peers.
> * Refactor interface to abstract class because all seed providers must 
> provide a constructor that accepts Map 
> * correct error messages
> * Do not catch Exception use MultiCatch and catch typed exceptions



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)