Hi Nathan,
it seems your Elasticsearch config is still wrong. Both nodes only bind to
localhost:
ES node 1:
> [2016-08-02 09:19:16,184][INFO ][transport ] [Betty Ross Banner]
> publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300},
> {127.0.0.1:9300}
>
ES node 2:
> [2016-08-02 09:19:16,064][INFO ][transport ] [Invisible Woman]
> publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300},
> {127.0.0.1:9300}
I suggest you double check the configuration files and do the changes I
suggested in the numerous mails before.
Cheers,
Jochen
On Tuesday, 2 August 2016 18:43:16 UTC+2, Nathan Mace wrote:
>
> Please see attached files. I got the elasticsearch.log file from
> /var/log/elasticsearch on both nodes. Additionally I got graylog.log from
> the same location on both nodes. Even though node 2 doesn't have graylog
> installed it had a log file for it. Not sure why that is.
>
> Thanks!
>
> Nathan
>
> On Tuesday, August 2, 2016 at 11:10:49 AM UTC-4, Jochen Schalanda wrote:
>>
>> Hi Nathan,
>>
>> please post the *complete* log files of your Elasticsearch and Graylog
>> nodes.
>>
>> Cheers,
>> Jochen
>>
>> On Tuesday, 2 August 2016 16:56:58 UTC+2, Nathan Mace wrote:
>>>
>>> Removing the leading whitespaces didn't help.
>>>
>>> However in looking through the logs I found this in the primary node's
>>> graylog.log file:
>>>
>>> ConnectTransportException[[ansted-search-01][x.x.x.149:9300]
>>> connect_timeout[30s]]; nested: ConnectException[Connection refused:
>>> /x.x.x.149:9300];
>>> at
>>> org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:987)
>>> at
>>> org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:920)
>>> at
>>> org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:893)
>>> at
>>> org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:260)
>>> at
>>> org.elasticsearch.discovery.zen.ZenDiscovery.joinElectedMaster(ZenDiscovery.java:434)
>>> at
>>> org.elasticsearch.discovery.zen.ZenDiscovery.innerJoinCluster(ZenDiscovery.java:386)
>>> at
>>> org.elasticsearch.discovery.zen.ZenDiscovery.access$4800(ZenDiscovery.java:91)
>>> at
>>> org.elasticsearch.discovery.zen.ZenDiscovery$JoinThreadControl$1.run(ZenDiscovery.java:1237)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> at java.lang.Thread.run(Thread.java:745)
>>>
>>> It was repeated several times. That is it trying to connect to the
>>> second node on port 9300 and not being able to. I see in the documentation
>>> that 9300 is the default port and I have nothing in either of the ES YML
>>> files referencing that port number, so it seems to be all default. If I do
>>> a netstat on both hosts they are both listening on port 9200 and 9300. It
>>> would seem that it is listening, but only allowing connections to 9300 from
>>> localhost? What would I need to change to allow a connect from the other
>>> node?
>>>
>>> Nathan
>>>
>>> On Tuesday, August 2, 2016 at 10:22:44 AM UTC-4, Jochen Schalanda wrote:
>>>>
>>>> Hi Nathan,
>>>>
>>>> I'm not sure how Elasticsearch handles leading whitespace in their
>>>> configuration file. I'd recommend making sure that the configuration
>>>> settings really start at the beginning of a line.
>>>>
>>>> Additionally, please post the complete log files of your Elasticsearch
>>>> and Graylog nodes.
>>>>
>>>> Cheers,
>>>> Jochen
>>>>
>>>> On Tuesday, 2 August 2016 16:00:47 UTC+2, Nathan Mace wrote:
>>>>>
>>>>> Oh good grief! Clearly been staring at this problem to long, I
>>>>> completely missed those hash signs.
>>>>>
>>>>> OK, now ES is happily running on the proper IP addresses. I can
>>>>> access it via curl from other hosts. So that's a large improvement.
>>>>> However Graylog still only reports 1 node in the web interface. I've
>>>>> attached the current versions of the config files (vs copy/paste). Given
>>>>> my tunnel vision on the hash signs, this seems like it will be something
>>>>> obvious but I can't find it.
>>>>>
>>>>> Thank you so much for the help!
>>>>>
>>>>> Nathan
>>>>>
>>>>> On Tuesday, August 2, 2016 at 9:30:58 AM UTC-4, Jochen Schalanda wrote:
>>>>>>
>>>>>> Hi Nathan,
>>>>>>
>>>>>> leading hash signs (the # character) mean that the line is commented
>>>>>> out.
>>>>>>
>>>>>> For example the following line is completely ignored:
>>>>>>
>>>>>> # discovery.zen.ping.unicast.hosts: ["x.x.x.146", "x.x.x.149"]
>>>>>>
>>>>>>
>>>>>> While this line is "active" and will be obeyed:
>>>>>>
>>>>>> cluster.name: graylog
>>>>>>
>>>>>>
>>>>>> Maybe you've only copy & pasted your configuration files in a strange
>>>>>> way (which is why I would always recommend to send them as attachments),
>>>>>> but that's how it looks like.
>>>>>>
>>>>>> Cheers,
>>>>>> Jochen
>>>>>>
>>>>>> On Tuesday, 2 August 2016 15:23:22 UTC+2, Nathan Mace wrote:
>>>>>>>
>>>>>>> Thanks Jochen. I will make the changes. However I am very confused
>>>>>>> by your comment about the second node having the cluster.name
>>>>>>> setting unset. I'm showing that it is set to "graylog" just like the
>>>>>>> first
>>>>>>> node. I'm not sure at all what you mean.
>>>>>>>
>>>>>>> Nathan
>>>>>>>
>>>>>>> On Tuesday, August 2, 2016 at 6:38:45 AM UTC-4, Jochen Schalanda
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hi Nathan,
>>>>>>>>
>>>>>>>> check the elasticsearch_network_host setting of your Graylog
>>>>>>>> nodes. It should be set to one (and only one!) public IP address of
>>>>>>>> the
>>>>>>>> Graylog node which can be accessed by all other Elasticsearch nodes in
>>>>>>>> the
>>>>>>>> cluster. elasticsearch_discovery_zen_ping_unicast_hosts should be
>>>>>>>> a comma-separated list of host/port pairs containing the addresses of
>>>>>>>> the
>>>>>>>> Elasticsearch nodes, for example:
>>>>>>>>
>>>>>>>> elasticsearch_discovery_zen_ping_unicast_hosts = x.x.x.146:9300,
>>>>>>>> x.x.x.149
>>>>>>>>
>>>>>>>>
>>>>>>>> See
>>>>>>>> http://docs.graylog.org/en/2.0/pages/configuration/elasticsearch.html#network-setup
>>>>>>>>
>>>>>>>> for details.
>>>>>>>>
>>>>>>>> Additionally, the cluster.name of your second Elasticsearch node
>>>>>>>> is unset, which makes it default to "elasticsearch". The logs of that
>>>>>>>> Elasticsearch node should show this pretty clearly.
>>>>>>>>
>>>>>>>> Also take a look at the network.host settings of both your
>>>>>>>> Elasticsearch nodes. This setting must be customized to your network
>>>>>>>> setup,
>>>>>>>> otherwise they'll only bind to the local network interface (i. e.
>>>>>>>> 127.0.0.1 or ::1). See
>>>>>>>> https://www.elastic.co/guide/en/elasticsearch/reference/2.3/modules-network.html#common-network-settings
>>>>>>>>
>>>>>>>> for details.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Jochen
>>>>>>>>
>>>>>>>> On Monday, 1 August 2016 22:15:32 UTC+2, Nathan Mace wrote:
>>>>>>>>>
>>>>>>>>> Primary node (MonoDB, Graylog, and ES): IP Address: x.x.x.146
>>>>>>>>> Secondary Node (ES Only): IP Address: x.x.x.149
>>>>>>>>>
>>>>>>>>> Both on the same subnet. Can ping each other.
>>>>>>>>> […]
>>>>>>>>>
>>>>>>>>
--
You received this message because you are subscribed to the Google Groups
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/graylog2/388534db-6490-4e70-90a6-4e7e76ad38fc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
