* David Pilato <[email protected]> [2014-04-11 08:06:28]:
> And all your nodes have, right?
>
> network:
> publish_host: _ec2:publicDns_
Yes. I'm using Puppet so every node gets the same configuration.
> I think some nodes are only master and some others are only data nodes, right?
Yes. The main difference is the data nodes have EBS volumes attached and
mounted for storing the indices, and they're bigger instances.
> Any chance you could put in your gist all individual generated
> elasticsearch.yml file?
The only difference so far between the configurations is the data nodes
have indices.memory.index_buffer_size set and obviously the values for
node.data and node.master are flipped. All other settings are identical.
> I don't understand why one successfully uses
>
> "publish_address" :
> "inet[ec2-54-206-x-x.ap-southeast-2.compute.amazonaws.com/172.31.x.x:9300]"
>
> And the other ones:
>
> "publish_address" : "inet[/172.31.x.x:9300]"
That's what I'm confused about too.
I understand "inet[ec2-54-206-x-x.amazonaws.com/172.31.x.x:9300]" is
just a textual representation of the transport endpoint, but does that
mean that another node will use the result of the DNS lookup of the
hostname in preference to the IP address that's also reported?
> With the exact same parameters, it should not happen.
> Could you try using _ec2:publicIp_ and see how it goes?
I tried to use the public IP instead, but I couldn't get the cluster to
successfully associate, I think that's the security groups interfering
so I'll have another attempt.
However, as I understand it, using the public IP means traffic between
cluster nodes performs worse and costs more, (Amazon classes it as
intra-region traffic, as if I was transfering between availability
zones). That's when I discovered that the public DNS name internally
still resolves to the private IP address so I assumed using the public
DNS names everywhere keeps the internal traffic using the private IP
addresses and external traffic (from the tribe nodes) still also work.
I've also just tried using the cloud-aws plugin on the tribe node and
adapted the configuration like so:
tribe:
blocks:
metadata: true
sydney:
cloud:
aws:
access_key: abc123
region: ap-southeast-2
secret_key: secret
cluster:
name: logstash
discovery:
ec2:
groups: elasticsearch
host_type: public_dns
type: ec2
zen:
minimum_master_nodes: 2
ping:
multicast:
enabled: false
I can see it makes a query to find all the nodes in the cluster, with
their public DNS names and IP addresses but then when it connects to
them it starts trying to use the private IP addresses again and I'm in
the same situation as before.
Matt
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/20140411130118.GC2245%40simulant.bodgit-n-scarper.com.
For more options, visit https://groups.google.com/d/optout.
