Finally, I fixed my problem.
There was a mistake for the field "discovery.ec2.groups". Instead of a
string, I had to put an array of string.
And I also forgot to add the tag "platform:prod" to CloudFormation when
launching my stack.
Fixed!
On Friday, 7 February 2014 14:54:05 UTC+1, Thomas FATTAL wrote:
>
> Hi,
>
> I'm trying to configure two Elasticsearch nodes in AWS in the same
> autoscaling group (CloudFormation).
> I am having some problems with them discovering each other.
>
> The following shows the elasticsearch.log I have on the first machine with
> the instance-id "i-2db5db03".
> The second machine has an instance-id "i-324e6612".
>
> It seems that both nodes recognize each other, thanks to
> discovery.ec2.tag.* field I added but then there are some problems that
> make them not to join together:
>
> [2014-02-07 13:17:08,852][INFO ][node ]
> [ip-10-238-225-133.ec2.internal] version[1.0.0.Beta2], pid[15342],
> build[296cfbe/2013-12-02T15:46:27Z]
> [2014-02-07 13:17:08,853][INFO ][node ]
> [ip-10-238-225-133.ec2.internal] initializing ...
> [2014-02-07 13:17:08,917][INFO ][plugins ]
> [ip-10-238-225-133.ec2.internal] loaded [cloud-aws], sites [paramedic]
> [2014-02-07 13:17:15,452][DEBUG][discovery.zen.ping.unicast]
> [ip-10-238-225-133.ec2.internal] using initial hosts [], with
> concurrent_connects [10]
> [2014-02-07 13:17:15,455][DEBUG][discovery.ec2 ]
> [ip-10-238-225-133.ec2.internal] using ping.timeout [3s],
> master_election.filter_client [true], master_election.filter_data [false]
> [2014-02-07 13:17:15,456][DEBUG][discovery.zen.elect ]
> [ip-10-238-225-133.ec2.internal] using minimum_master_nodes [1]
> [2014-02-07 13:17:15,457][DEBUG][discovery.zen.fd ]
> [ip-10-238-225-133.ec2.internal] [master] uses ping_interval [1s],
> ping_timeout [30s], ping_retries [3]
> [2014-02-07 13:17:15,500][DEBUG][discovery.zen.fd ]
> [ip-10-238-225-133.ec2.internal] [node ] uses ping_interval [1s],
> ping_timeout [30s], ping_retries [3]
> [2014-02-07 13:17:16,769][DEBUG][discovery.ec2 ]
> [ip-10-238-225-133.ec2.internal] using host_type [PRIVATE_IP], tags
> [{platform=prod}], groups [[]] with any_group [true], availability_zones
> [[]]
> [2014-02-07 13:17:19,930][INFO ][node ]
> [ip-10-238-225-133.ec2.internal] initialized
> [2014-02-07 13:17:19,931][INFO ][node ]
> [ip-10-238-225-133.ec2.internal] starting ...
> [2014-02-07 13:17:20,455][INFO ][transport ]
> [ip-10-238-225-133.ec2.internal] bound_address {inet[/0.0.0.0:9300]},
> publish_address {inet[/10.238.225.133:9300]}
> [2014-02-07 13:17:20,527][TRACE][discovery ]
> [ip-10-238-225-133.ec2.internal] waiting for 30s for the initial state to
> be set by the discovery
> [2014-02-07 13:17:21,981][TRACE][discovery.ec2 ]
> [ip-10-238-225-133.ec2.internal] building dynamic unicast discovery nodes...
> [2014-02-07 13:17:21,982][TRACE][discovery.ec2 ]
> [ip-10-238-225-133.ec2.internal] filtering out instance i-2db5db03 based
> tags {platform=prod}, not part of [{Key: aws:cloudformation:stack-id,
> Value:
> arn:aws:cloudformation:us-east-1:876119091332:stack/ES-10/daf53050-8ff8-11e3-bdce-50e241629418,
>
> }, {Key: aws:cloudformation:stack-name, Value: ES-10, }, {Key:
> aws:cloudformation:logical-id, Value: ESASG, }, {Key:
> aws:autoscaling:groupName, Value: ES-10-ESASG-BHGX7KKQ9QPR, }]
> [2014-02-07 13:17:21,983][TRACE][discovery.ec2 ]
> [ip-10-238-225-133.ec2.internal] filtering out instance i-324e6612 based
> tags {platform=prod}, not part of [{Key: aws:cloudformation:logical-id,
> Value: ESASG, }, {Key: aws:cloudformation:stack-id, Value:
> arn:aws:cloudformation:us-east-1:876119091332:stack/ES-10/daf53050-8ff8-11e3-bdce-50e241629418,
>
> }, {Key: aws:cloudformation:stack-name, Value: ES-10, }, {Key:
> aws:autoscaling:groupName, Value: ES-10-ESASG-BHGX7KKQ9QPR, }]
> [2014-02-07 13:17:21,983][DEBUG][discovery.ec2 ]
> [ip-10-238-225-133.ec2.internal] using dynamic discovery nodes []
> [2014-02-07 13:17:23,744][TRACE][discovery.ec2 ]
> [ip-10-238-225-133.ec2.internal] building dynamic unicast discovery nodes...
> [2014-02-07 13:17:23,745][TRACE][discovery.ec2 ]
> [ip-10-238-225-133.ec2.internal] filtering out instance i-2db5db03 based
> tags {platform=prod}, not part of [{Key: aws:cloudformation:stack-id,
> Value:
> arn:aws:cloudformation:us-east-1:876119091332:stack/ES-10/daf53050-8ff8-11e3-bdce-50e241629418,
>
> }, {Key: aws:cloudformation:stack-name, Value: ES-10, }, {Key:
> aws:cloudformation:logical-id, Value: ESASG, }, {Key:
> aws:autoscaling:groupName, Value: ES-10-ESASG-BHGX7KKQ9QPR, }]
> [2014-02-07 13:17:23,745][TRACE][discovery.ec2 ]
> [ip-10-238-225-133.ec2.internal] filtering out instance i-324e6612 based
> tags {platform=prod}, not part of [{Key: aws:cloudformation:logical-id,
> Value: ESASG, }, {Key: aws:cloudformation:stack-id, Value:
> arn:aws:cloudformation:us-east-1:876119091332:stack/ES-10/daf53050-8ff8-11e3-bdce-50e241629418,
>
> }, {Key: aws:cloudformation:stack-name, Value: ES-10, }, {Key:
> aws:autoscaling:groupName, Value: ES-10-ESASG-BHGX7KKQ9QPR, }]
> [2014-02-07 13:17:25,604][DEBUG][discovery.ec2 ]
> [ip-10-238-225-133.ec2.internal] using dynamic discovery nodes []
> [2014-02-07 13:17:25,604][TRACE][discovery.ec2 ]
> [ip-10-238-225-133.ec2.internal] full ping responses: {none}
> [2014-02-07 13:17:25,605][DEBUG][discovery.ec2 ]
> [ip-10-238-225-133.ec2.internal] filtered ping responses:
> (filter_client[true], filter_data[false]) {none}
> [2014-02-07 13:17:25,618][INFO ][cluster.service ]
> [ip-10-238-225-133.ec2.internal] new_master
> [ip-10-238-225-133.ec2.internal][tUBe7JNdTquROI6UWVLNpw][inet[/10.238.225.133:9300]]{aws_availability_zone=us-east-1d,
>
> max_local_storage_nodes=1}, reason: zen-disco-join (elected_as_master)
> [2014-02-07 13:17:25,752][TRACE][discovery ]
> [ip-10-238-225-133.ec2.internal] initial state set from discovery
> [2014-02-07 13:17:25,752][INFO ][discovery ]
> [ip-10-238-225-133.ec2.internal] elasticsearch/tUBe7JNdTquROI6UWVLNpw
> [2014-02-07 13:17:25,892][INFO ][http ]
> [ip-10-238-225-133.ec2.internal] bound_address {inet[/0.0.0.0:9200]},
> publish_address {inet[/10.238.225.133:9200]}
> [2014-02-07 13:17:25,893][INFO ][node ]
> [ip-10-238-225-133.ec2.internal] started
> [2014-02-07 13:17:25,924][INFO ][gateway ]
> [ip-10-238-225-133.ec2.internal] recovered [0] indices into cluster_state
>
>
>
> I'm using Chef to configure my node with the cookbook of Karmi (
> https://github.com/elasticsearch/cookbook-elasticsearch).
> The node.json is the following:
>
> "/etc/chef/node.json": {
> "content": {
> "run_list": [
> "recipe[java]",
> "recipe[monit]",
> "recipe[elasticsearch]",
> "recipe[elasticsearch::plugins]",
> "recipe[elasticsearch::aws]",
> "recipe[elasticsearch::monit]"
> ],
> "java": {
> "install_flavor": "oracle",
> "oracle": {
> "accept_oracle_download_terms": true
> },
> "jdk_version": "7"
>
> },
> "elasticsearch": {
> "version": "1.0.0.Beta2",
> "logging": {
> "discovery": "TRACE"
> },
> "cluster": {
> "name": "elasticsearch"
> },
> "bootstrap": {
> "mlockall": false
> },
> "discovery": {
> "type": "ec2",
> "ec2": {
> "groups": {"Ref": "ESSecurityGroup"},
> "tag": {
> "platform": "prod"
> }
> },
> "zen": {
> "ping": {
> "multicast": {
> "enabled": false
> }
> }
> }
> },
> "cloud": {
> "aws": {
> "access_key": {"Ref": "ESHostKeys"},
> "secret_key": {"Fn::GetAtt": [
> "ESHostKeys", "SecretAccessKey"]},
> "region": "us-east-1"
> },
> "ec2": {
> "security_group": {"Ref":
> "ESSecurityGroup"}
> }
> },
> "monit": {
> "notify_email": "my_email_address",
> "mail_format": {
> "from": "from_email_address",
> "subject":
> "[monit]$SERVICE$EVENTon$HOST",
> "message": "$SERVICE$ACTION:
> $DESCRIPTION"
> }
> },
> "plugins": {
> "karmi/elasticsearch-paramedic" : {}
> }
> }
> },
> "mode": "000644",
> "owner": "root",
> "group": "root"
> }
> }
>
>
> I'm using Oracle 7 for ElasticSearch. The machines are Ubuntu 12.04 and
> I'm using Elasticsearch 1.0.0 RC2.
>
> It seems that the first node is seeing the second node but refuse to join
> it because the tags "aws:cloudformation:stack-id", "
> aws:cloudformation:stack-name", "aws:cloudformation:logical-id" and
> "aws:autoscaling:groupName"
> are not the same. However, I checked these tags for both instances and it's
> the same (as the two instances have been created in the same Autoscaling
> group).
>
> Any ideas?
> Thanks!
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/754216cd-c396-4396-a1e6-b1e6a9cc63c3%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
