Hi, please make sure that the other nodes of the Elasticsearch cluster are able to communicate with the embedded ES node in Graylog. Currently, the embedded ES node in Graylog is announcing 127.0.0.1 as its address, to which the other ES nodes can't connect to.
You can change the address of the embedded ES node using the elasticsearch_network_host configuration setting in your Graylog configuration file (see https://github.com/Graylog2/graylog2-server/blob/2.0.0/misc/graylog.conf#L187-L194 ). Cheers, Jochen On Monday, 9 May 2016 14:48:16 UTC+2, Prakashkumar S wrote: > > Hi, > > I have the following setup. > > GRAY LOG SERVER(172.16.16.102) ---> ElasticSearch - node1 (172.16.16.100) > --> ElasticSearch -node2 (172.16.16.101) > > > Everything Seems to be normal if I run over normal HTTP transport. Things > break if I enable TLS on Graylog server and I am getting the error attached > in this mail. > > Debugs > > 2016-05-09T18:09:18.307+05:30 INFO [zen] > [graylog-12c27c84-87b7-4975-b43f-1adc8abc477a] failed to send join request > to master > [{elasticsearch-node1}{8hAPDQKkS3CwFCfSeAn1Pw}{172.16.16.100}{172.16.16.100:9300}], > > reason > [RemoteTransportException[[elasticsearch-node1][172.16.16.100:9300][internal:discovery/zen/join]]; > > nested: > ConnectTransportException[[graylog-12c27c84-87b7-4975-b43f-1adc8abc477a][ > 127.0.0.1:9350] connect_timeout[30s]]; nested: > NotSerializableExceptionWrapper[connect_exception: Connection refused: / > 127.0.0.1:9350]; ] > 2016-05-09T18:09:21.332+05:30 INFO [zen] > [graylog-12c27c84-87b7-4975-b43f-1adc8abc477a] failed to send join request > to master > [{elasticsearch-node1}{8hAPDQKkS3CwFCfSeAn1Pw}{172.16.16.100}{172.16.16.100:9300}], > > reason > [RemoteTransportException[[elasticsearch-node1][172.16.16.100:9300][internal:discovery/zen/join]]; > > nested: > ConnectTransportException[[graylog-12c27c84-87b7-4975-b43f-1adc8abc477a][ > 127.0.0.1:9350] connect_timeout[30s]]; nested: > NotSerializableExceptionWrapper[connect_exception: Connection refused: / > 127.0.0.1:9350]; ] > 2016-05-09T18:09:24.362+05:30 INFO [zen] > [graylog-12c27c84-87b7-4975-b43f-1adc8abc477a] failed to send join request > to master > [{elasticsearch-node1}{8hAPDQKkS3CwFCfSeAn1Pw}{172.16.16.100}{172.16.16.100:9300}], > > reason > [RemoteTransportException[[elasticsearch-node1][172.16.16.100:9300][internal:discovery/zen/join]]; > > nested: > ConnectTransportException[[graylog-12c27c84-87b7-4975-b43f-1adc8abc477a][ > 127.0.0.1:9350] connect_timeout[30s]]; nested: > NotSerializableExceptionWrapper[connect_exception: Connection refused: / > 127.0.0.1:9350]; ] > 2016-05-09T18:09:27.391+05:30 INFO [zen] > [graylog-12c27c84-87b7-4975-b43f-1adc8abc477a] failed to send join request > to master > [{elasticsearch-node1}{8hAPDQKkS3CwFCfSeAn1Pw}{172.16.16.100}{172.16.16.100:9300}], > > reason > [RemoteTransportException[[elasticsearch-node1][172.16.16.100:9300][internal:discovery/zen/join]]; > > nested: > ConnectTransportException[[graylog-12c27c84-87b7-4975-b43f-1adc8abc477a][ > 127.0.0.1:9350] connect_timeout[30s]]; nested: > NotSerializableExceptionWrapper[connect_exception: Connection refused: / > 127.0.0.1:9350]; ] > 2016-05-09T18:09:30.416+05:30 INFO [zen] > [graylog-12c27c84-87b7-4975-b43f-1adc8abc477a] failed to send join request > to master > [{elasticsearch-node1}{8hAPDQKkS3CwFCfSeAn1Pw}{172.16.16.100}{172.16.16.100:9300}], > > reason > [RemoteTransportException[[elasticsearch-node1][172.16.16.100:9300][internal:discovery/zen/join]]; > > nested: > ConnectTransportException[[graylog-12c27c84-87b7-4975-b43f-1adc8abc477a][ > 127.0.0.1:9350] connect_timeout[30s]]; nested: > NotSerializableExceptionWrapper[connect_exception: Connection refused: / > 127.0.0.1:9350]; ] > 2016-05-09T18:09:33.431+05:30 INFO [zen] > [graylog-12c27c84-87b7-4975-b43f-1adc8abc477a] failed to send join request > to master > [{elasticsearch-node1}{8hAPDQKkS3CwFCfSeAn1Pw}{172.16.16.100}{172.16.16.100:9300}], > > reason > [RemoteTransportException[[elasticsearch-node1][172.16.16.100:9300][internal:discovery/zen/join]]; > > nested: > ConnectTransportException[[graylog-12c27c84-87b7-4975-b43f-1adc8abc477a][ > 127.0.0.1:9350] connect_timeout[30s]]; nested: > NotSerializableExceptionWrapper[connect_exception: Connection refused: / > 127.0.0.1:9350]; ] > 2016-05-09T18:09:34.282+05:30 INFO [IndexerClusterCheckerThread] Indexer > not fully initialized yet. Skipping periodic cluster check. > 2016-05-09T18:09:36.443+05:30 INFO [zen] > [graylog-12c27c84-87b7-4975-b43f-1adc8abc477a] failed to send join request > to master > [{elasticsearch-node1}{8hAPDQKkS3CwFCfSeAn1Pw}{172.16.16.100}{172.16.16.100:9300}], > > reason > [RemoteTransportException[[elasticsearch-node1][172.16.16.100:9300][internal:discovery/zen/join]]; > > nested: > ConnectTransportException[[graylog-12c27c84-87b7-4975-b43f-1adc8abc477a][ > 127.0.0.1:9350] connect_timeout[30s]]; nested: > NotSerializableExceptionWrapper[connect_exception: Connection refused: / > 127.0.0.1:9350]; ] > 2016-05-09T18:09:39.466+05:30 INFO [zen] > [graylog-12c27c84-87b7-4975-b43f-1adc8abc477a] failed to send join request > to master > [{elasticsearch-node1}{8hAPDQKkS3CwFCfSeAn1Pw}{172.16.16.100}{172.16.16.100:9300}], > > reason > [RemoteTransportException[[elasticsearch-node1][172.16.16.100:9300][internal:discovery/zen/join]]; > > nested: > ConnectTransportException[[graylog-12c27c84-87b7-4975-b43f-1adc8abc477a][ > 127.0.0.1:9350] connect_timeout[30s]]; nested: > NotSerializableExceptionWrapper[connect_exception: Connection refused: / > 127.0.0.1:9350]; ] > 2016-05-09T18:09:42.502+05:30 INFO [zen] > [graylog-12c27c84-87b7-4975-b43f-1adc8abc477a] failed to send join request > to master > [{elasticsearch-node1}{8hAPDQKkS3CwFCfSeAn1Pw}{172.16.16.100}{172.16.16.100:9300}], > > reason > [RemoteTransportException[[elasticsearch-node1][172.16.16.100:9300][internal:discovery/zen/join]]; > > nested: > ConnectTransportException[[graylog-12c27c84-87b7-4975-b43f-1adc8abc477a][ > 127.0.0.1:9350] connect_timeout[30s]]; nested: > NotSerializableExceptionWrapper[connect_exception: Connection refused: / > 127.0.0.1:9350]; ] > 2016-05-09T18:09:45.536+05:30 INFO [zen] > [graylog-12c27c84-87b7-4975-b43f-1adc8abc477a] failed to send join request > to master > [{elasticsearch-node1}{8hAPDQKkS3CwFCfSeAn1Pw}{172.16.16.100}{172.16.16.100:9300}], > > reason > [RemoteTransportException[[elasticsearch-node1][172.16.16.100:9300][internal:discovery/zen/join]]; > > nested: > ConnectTransportException[[graylog-12c27c84-87b7-4975-b43f-1adc8abc477a][ > 127.0.0.1:9350] connect_timeout[30s]]; nested: > NotSerializableExceptionWrapper[connect_exception: Connection refused: / > 127.0.0.1:9350]; ] > 2016-05-09T18:09:48.561+05:30 INFO [zen] > [graylog-12c27c84-87b7-4975-b43f-1adc8abc477a] failed to send join request > to master > [{elasticsearch-node1}{8hAPDQKkS3CwFCfSeAn1Pw}{172.16.16.100}{172.16.16.100:9300}], > > reason > [RemoteTransportException[[elasticsearch-node1][172.16.16.100:9300][internal:discovery/zen/join]]; > > nested: > ConnectTransportException[[graylog-12c27c84-87b7-4975-b43f-1adc8abc477a][ > 127.0.0.1:9350] connect_timeout[30s]]; nested: > NotSerializableExceptionWrapper[connect_exception: Connection refused: / > 127.0.0.1:9350]; ] > 2016-05-09T18:09:51.592+05:30 INFO [zen] > [graylog-12c27c84-87b7-4975-b43f-1adc8abc477a] failed to send join request > to master > [{elasticsearch-node1}{8hAPDQKkS3CwFCfSeAn1Pw}{172.16.16.100}{172.16.16.100:9300}], > > reason > [RemoteTransportException[[elasticsearch-node1][172.16.16.100:9300][internal:discovery/zen/join]]; > > nested: > ConnectTransportException[[graylog-12c27c84-87b7-4975-b43f-1adc8abc477a][ > 127.0.0.1:9350] connect_timeout[30s]]; nested: > NotSerializableExceptionWrapper[connect_exception: Connection refused: / > 127.0.0.1:9350]; ] > 2016-05-09T18:09:54.624+05:30 INFO [zen] > [graylog-12c27c84-87b7-4975-b43f-1adc8abc477a] failed to send join request > to master > [{elasticsearch-node1}{8hAPDQKkS3CwFCfSeAn1Pw}{172.16.16.100}{172.16.16.100:9300}], > > reason > [RemoteTransportException[[elasticsearch-node1][172.16.16.100:9300][internal:discovery/zen/join]]; > > nested: > ConnectTransportException[[graylog-12c27c84-87b7-4975-b43f-1adc8abc477a][ > 127.0.0.1:9350] connect_timeout[30s]]; nested: > NotSerializableExceptionWrapper[connect_exception: Connection refused: / > 127.0.0.1:9350]; ] > 2016-05-09T18:09:57.640+05:30 INFO [zen] > [graylog-12c27c84-87b7-4975-b43f-1adc8abc477a] failed to send join request > to master > [{elasticsearch-node1}{8hAPDQKkS3CwFCfSeAn1Pw}{172.16.16.100}{172.16.16.100:9300}], > > reason > [RemoteTransportException[[elasticsearch-node1][172.16.16.100:9300][internal:discovery/zen/join]]; > > nested: > ConnectTransportException[[graylog-12c27c84-87b7-4975-b43f-1adc8abc477a][ > 127.0.0.1:9350] connect_timeout[30s]]; nested: > NotSerializableExceptionWrapper[connect_exception: Connection refused: / > 127.0.0.1:9350]; ] > 2016-05-09T18:10:00.667+05:30 INFO [zen] > [graylog-12c27c84-87b7-4975-b43f-1adc8abc477a] failed to send join request > to master > [{elasticsearch-node1}{8hAPDQKkS3CwFCfSeAn1Pw}{172.16.16.100}{172.16.16.100:9300}], > > reason > [RemoteTransportException[[elasticsearch-node1][172.16.16.100:9300][internal:discovery/zen/join]]; > > nested: > ConnectTransportException[[graylog-12c27c84-87b7-4975-b43f-1adc8abc477a][ > 127.0.0.1:9350] connect_timeout[30s]]; nested: > NotSerializableExceptionWrapper[connect_exception: Connection refused: / > 127.0.0.1:9350]; ] > 2016-05-09T18:10:03.694+05:30 INFO [zen] > [graylog-12c27c84-87b7-4975-b43f-1adc8abc477a] failed to send join request > to master > [{elasticsearch-node1}{8hAPDQKkS3CwFCfSeAn1Pw}{172.16.16.100}{172.16.16.100:9300}], > > reason > [RemoteTransportException[[elasticsearch-node1][172.16.16.100:9300][internal:discovery/zen/join]]; > > nested: > ConnectTransportException[[graylog-12c27c84-87b7-4975-b43f-1adc8abc477a][ > 127.0.0.1:9350] connect_timeout[30s]]; nested: > NotSerializableExceptionWrapper[connect_exception: Connection refused: / > 127.0.0.1:9350]; ] > 2016-05-09T18:10:04.287+05:30 INFO [IndexerClusterCheckerThread] Indexer > not fully initialized yet. Skipping periodic cluster check. > > not sure What I am missing here to make it work over TLS. Could you give > me some pointers ? Thanks > -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/a2b76b09-16d3-46ae-b062-b1826a768b1e%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
