Removing the leading whitespaces didn't help. However in looking through the logs I found this in the primary node's graylog.log file:
ConnectTransportException[[ansted-search-01][x.x.x.149:9300] connect_timeout[30s]]; nested: ConnectException[Connection refused: /x.x.x.149:9300]; at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:987) at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:920) at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:893) at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:260) at org.elasticsearch.discovery.zen.ZenDiscovery.joinElectedMaster(ZenDiscovery.java:434) at org.elasticsearch.discovery.zen.ZenDiscovery.innerJoinCluster(ZenDiscovery.java:386) at org.elasticsearch.discovery.zen.ZenDiscovery.access$4800(ZenDiscovery.java:91) at org.elasticsearch.discovery.zen.ZenDiscovery$JoinThreadControl$1.run(ZenDiscovery.java:1237) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) It was repeated several times. That is it trying to connect to the second node on port 9300 and not being able to. I see in the documentation that 9300 is the default port and I have nothing in either of the ES YML files referencing that port number, so it seems to be all default. If I do a netstat on both hosts they are both listening on port 9200 and 9300. It would seem that it is listening, but only allowing connections to 9300 from localhost? What would I need to change to allow a connect from the other node? Nathan On Tuesday, August 2, 2016 at 10:22:44 AM UTC-4, Jochen Schalanda wrote: > > Hi Nathan, > > I'm not sure how Elasticsearch handles leading whitespace in their > configuration file. I'd recommend making sure that the configuration > settings really start at the beginning of a line. > > Additionally, please post the complete log files of your Elasticsearch and > Graylog nodes. > > Cheers, > Jochen > > On Tuesday, 2 August 2016 16:00:47 UTC+2, Nathan Mace wrote: >> >> Oh good grief! Clearly been staring at this problem to long, I >> completely missed those hash signs. >> >> OK, now ES is happily running on the proper IP addresses. I can access >> it via curl from other hosts. So that's a large improvement. However >> Graylog still only reports 1 node in the web interface. I've attached the >> current versions of the config files (vs copy/paste). Given my tunnel >> vision on the hash signs, this seems like it will be something obvious but >> I can't find it. >> >> Thank you so much for the help! >> >> Nathan >> >> On Tuesday, August 2, 2016 at 9:30:58 AM UTC-4, Jochen Schalanda wrote: >>> >>> Hi Nathan, >>> >>> leading hash signs (the # character) mean that the line is commented >>> out. >>> >>> For example the following line is completely ignored: >>> >>> # discovery.zen.ping.unicast.hosts: ["x.x.x.146", "x.x.x.149"] >>> >>> >>> While this line is "active" and will be obeyed: >>> >>> cluster.name: graylog >>> >>> >>> Maybe you've only copy & pasted your configuration files in a strange >>> way (which is why I would always recommend to send them as attachments), >>> but that's how it looks like. >>> >>> Cheers, >>> Jochen >>> >>> On Tuesday, 2 August 2016 15:23:22 UTC+2, Nathan Mace wrote: >>>> >>>> Thanks Jochen. I will make the changes. However I am very confused by >>>> your comment about the second node having the cluster.name setting >>>> unset. I'm showing that it is set to "graylog" just like the first node. >>>> I'm not sure at all what you mean. >>>> >>>> Nathan >>>> >>>> On Tuesday, August 2, 2016 at 6:38:45 AM UTC-4, Jochen Schalanda wrote: >>>>> >>>>> Hi Nathan, >>>>> >>>>> check the elasticsearch_network_host setting of your Graylog nodes. >>>>> It should be set to one (and only one!) public IP address of the Graylog >>>>> node which can be accessed by all other Elasticsearch nodes in the >>>>> cluster. elasticsearch_discovery_zen_ping_unicast_hosts should be a >>>>> comma-separated list of host/port pairs containing the addresses of the >>>>> Elasticsearch nodes, for example: >>>>> >>>>> elasticsearch_discovery_zen_ping_unicast_hosts = x.x.x.146:9300, >>>>> x.x.x.149 >>>>> >>>>> >>>>> See >>>>> http://docs.graylog.org/en/2.0/pages/configuration/elasticsearch.html#network-setup >>>>> >>>>> for details. >>>>> >>>>> Additionally, the cluster.name of your second Elasticsearch node is >>>>> unset, which makes it default to "elasticsearch". The logs of that >>>>> Elasticsearch node should show this pretty clearly. >>>>> >>>>> Also take a look at the network.host settings of both your >>>>> Elasticsearch nodes. This setting must be customized to your network >>>>> setup, >>>>> otherwise they'll only bind to the local network interface (i. e. >>>>> 127.0.0.1 or ::1). See >>>>> https://www.elastic.co/guide/en/elasticsearch/reference/2.3/modules-network.html#common-network-settings >>>>> >>>>> for details. >>>>> >>>>> Cheers, >>>>> Jochen >>>>> >>>>> On Monday, 1 August 2016 22:15:32 UTC+2, Nathan Mace wrote: >>>>>> >>>>>> Primary node (MonoDB, Graylog, and ES): IP Address: x.x.x.146 >>>>>> Secondary Node (ES Only): IP Address: x.x.x.149 >>>>>> >>>>>> Both on the same subnet. Can ping each other. >>>>>> […] >>>>>> >>>>> -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/9995dc87-b7c3-44ac-9c4e-a10f21edc95e%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
