Hi Tim,
are there any more messages in the logs of Graylog? The interesting part is
probably just before the first message you've posted…
Cheers,
Jochen
On Friday, 10 July 2015 10:46:39 UTC+2, tim lewis wrote:
>
> All,
>
>
> I am able to connect my server node to the database. This is successful,
> but inevitably fails after about 10 seconds. The connections get dropped
> reliably after about that time.
>
>
> Can anyone give some pointers on troubleshooting this. I have graylog
> clusters (EC2 ASG's) for each of graylog's roles. I am trying to connect
> the server to the data nodes, but ubuntu@graylog:/opt/graylog/server$
> ./bin/graylogctl status shows this:
>
>
>
> #=============================================================
>
>
> graylog-server running with PID 6818
>
> ubuntu@graylog:/opt/graylog/server$ ./bin/graylogctl status
>
> graylog-server running with PID 6818
>
> ubuntu@graylog:/opt/graylog/server$ ./bin/graylogctl status
>
> graylog-server running with PID 6818
>
> ubuntu@graylog:/opt/graylog/server$ ./bin/graylogctl status
>
> graylog-server running with PID 6818
>
> ubuntu@graylog:/opt/graylog/server$ ./bin/graylogctl status
>
> graylog-server running with PID 6818
>
> ubuntu@graylog:/opt/graylog/server$ ./bin/graylogctl status
>
> graylog-server not running
>
>
> On Mongo terminal(running on same box as server), I see this:
>
>
> #=============================================================
>
> 55520 #3 (3 connections now open)
>
> 2015-07-09T19:52:28.040+0000 [initandlisten] connection accepted from
> 127.0.0.1:55521 #4 (4 connections now open)
>
> 2015-07-09T19:52:28.040+0000 [initandlisten] connection accepted from
> 127.0.0.1:55522 #5 (5 connections now open)
>
> 2015-07-09T19:52:29.069+0000 [conn3] build index on:
> graylog2.content_packs properties: { v: 1, key: { _id: 1 }, name: "_id_",
> ns: "graylog2.content_packs" }
>
> 2015-07-09T19:52:29.069+0000 [conn3] added index to empty collection
>
> 2015-07-09T19:52:32.610+0000 [clientcursormon] mem (MB) res:43 virt:507
>
> 2015-07-09T19:52:32.610+0000 [clientcursormon] mapped (incl journal
> view):320
>
> 2015-07-09T19:52:32.610+0000 [clientcursormon] connections:5
>
> 2015-07-09T19:52:43.554+0000 [conn3] build index on:
> graylog2.system_messages properties: { v: 1, key: { _id: 1 }, name: "_id_",
> ns: "graylog2.system_messages" }
>
> 2015-07-09T19:52:43.554+0000 [conn3] added index to empty collection
>
> 2015-07-09T19:52:47.892+0000 [conn1] end connection 127.0.0.1:55518 (4
> connections now open)
>
> 2015-07-09T19:52:47.892+0000 [conn2] end connection 127.0.0.1:55519 (3
> connections now open)
>
> 2015-07-09T19:52:47.892+0000 [conn3] end connection 127.0.0.1:55520 (2
> connections now open)
>
> 2015-07-09T19:52:47.892+0000 [conn4] end connection 127.0.0.1:55521 (1
> connection now open)
>
> 2015-07-09T19:52:47.892+0000 [conn5] end connection 127.0.0.1:55522 (0
> connections now open)
>
>
>
>
> The graylog server log file
>
>
> #=============================================================
>
>
> WARN ][discovery.zen.ping.unicast] [Jamal Afari] [1] failed send ping to
> [#zen_unicast_1#][graylog][inet[/172.31.198.14:9300]]
>
> org.elasticsearch.ElasticsearchIllegalStateException: can't add nodes to a
> stopped transport
>
> at
> org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:719)
>
> at
> org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:699)
>
> at
> org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:220)
>
> at
> org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:373)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:745)
>
> [2015-07-09 17:44:17,774][INFO ][node ] [Jamal Afari]
> stopped
>
> [2015-07-09 17:44:17,774][INFO ][node ] [Jamal Afari]
> closing ...
>
> [2015-07-09 17:44:17,779][INFO ][node ] [Jamal Afari]
> closed
>
> [2015-07-09 17:44:33,959][INFO ][node ] [Lunatica]
> version[1.5.2], pid[2275], build[62ff986/2015-04-27T09:21:06Z]
>
> [2015-07-09 17:44:33,959][INFO ][node ] [Lunatica]
> initializing ...
>
> [2015-07-09 17:44:33,963][INFO ][plugins ] [Lunatica]
> loaded [], sites []
>
> [2015-07-09 17:44:36,314][INFO ][node ] [Lunatica]
> initialized
>
> [2015-07-09 17:44:36,314][INFO ][node ] [Lunatica]
> starting ...
>
> [2015-07-09 17:44:36,369][INFO ][transport ] [Lunatica]
> bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/
> 172.30.198.14:9300]}
>
> [2015-07-09 17:44:36,377][INFO ][discovery ] [Lunatica]
> graylog/vseaJqvdTimJCB1Tjz5Jkw
>
> [2015-07-09 17:45:06,377][WARN ][discovery ] [Lunatica]
> waited for 30s and no initial state was set by the discovery
>
> [2015-07-09 17:45:06,382][INFO ][http ] [Lunatica]
> bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/
> 172.30.198.14:9200]}
>
> [2015-07-09 17:45:06,383][INFO ][node ] [Lunatica]
> started
>
> [2015-07-09 17:46:06,539][INFO ][cluster.service ] [Lunatica]
> new_master
> [Lunatica][vseaJqvdTimJCB1Tjz5Jkw][graylog][inet[/172.30.198.14:9300]],
> reason: zen-disco-join (elected_as_master)
>
> [2015-07-09 17:46:07,035][INFO ][gateway ] [Lunatica]
> recovered [1] indices into cluster_state
>
> [2015-07-09 17:52:25,143][INFO ][node ] [Lunatica]
> stopping ...
>
> [2015-07-09 17:52:25,166][INFO ][node ] [Lunatica]
> stopped
>
> [2015-07-09 17:52:25,166][INFO ][node ] [Lunatica]
> closing ...
>
> [2015-07-09 17:52:25,170][INFO ][node ] [Lunatica]
> closed
>
> [2015-07-09 18:01:44,217][INFO ][node ] [Spider
> Doppelganger] version[1.5.2], pid[2407], build[62ff986/2015-04-27T09:21:06Z]
>
> [2015-07-09 18:01:44,217][INFO ][node ] [Spider
> Doppelganger] initializing ...
>
> [2015-07-09 18:01:44,221][INFO ][plugins ] [Spider
> Doppelganger] loaded [], sites []
>
> [2015-07-09 18:01:46,542][INFO ][node ] [Spider
> Doppelganger] initialized
>
> [2015-07-09 18:01:46,543][INFO ][node ] [Spider
> Doppelganger] starting ...
>
> [2015-07-09 18:01:46,595][INFO ][transport ] [Spider
> Doppelganger] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address
> {inet[/172.30.198.14:9300]}
>
> [2015-07-09 18:01:46,603][INFO ][discovery ] [Spider
> Doppelganger] graylog/Sd2WCbx-Rw-k1HxDRGRGtg
>
> [2015-07-09 18:02:16,603][WARN ][discovery ] [Spider
> Doppelganger] waited for 30s and no initial state was set by the discovery
>
> [2015-07-09 18:02:16,608][INFO ][http ] [Spider
> Doppelganger] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address
> {inet[/172.30.198.14:9200]}
>
> [2015-07-09 18:02:16,608][INFO ][node ] [Spider
> Doppelganger] started
>
> [2015-07-09 18:03:16,766][INFO ][cluster.service ] [Spider
> Doppelganger] new_master [Spider
> Doppelganger][Sd2WCbx-Rw-k1HxDRGRGtg][graylog][inet[/172.30.198.14:9300]],
> reason: zen-disco-join (elected_as_master)
>
> [2015-07-09 18:03:17,256][INFO ][gateway ] [Spider
> Doppelganger] recovered [1] indices into cluster_state
>
> [2015-07-09 19:20:16,978][INFO ][node ] [Spider
> Doppelganger] stopping ...
>
> [2015-07-09 19:20:17,013][INFO ][node ] [Spider
> Doppelganger] stopped
>
> [2015-07-09 19:20:17,013][INFO ][node ] [Spider
> Doppelganger] closing ...
>
> [2015-07-09 19:20:17,016][INFO ][node ] [Spider
> Doppelganger] closed
>
> [2015-07-09 19:23:08,161][INFO ][node ] [M]
> version[1.5.2], pid[2570], build[62ff986/2015-04-27T09:21:06Z]
>
> [2015-07-09 19:23:08,162][INFO ][node ] [M]
> initializing ...
>
> [2015-07-09 19:23:08,166][INFO ][plugins ] [M] loaded [],
> sites []
>
> [2015-07-09 19:23:10,504][INFO ][node ] [M] initialized
>
> [2015-07-09 19:23:10,505][INFO ][node ] [M] starting
> ...
>
> [2015-07-09 19:23:10,559][INFO ][transport ] [M]
> bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/
> 172.30.198.14:9300]}
>
> [2015-07-09 19:23:10,567][INFO ][discovery ] [M]
> graylog/nTUnv4H8RvKLx1A0HbBcLg
>
> [2015-07-09 19:23:40,567][WARN ][discovery ] [M] waited for
> 30s and no initial state was set by the discovery
>
> [2015-07-09 19:23:40,571][INFO ][http ] [M]
> bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/
> 172.30.198.14:9200]}
>
> [2015-07-09 19:23:40,572][INFO ][node ] [M] started
>
> [2015-07-09 19:24:40,730][INFO ][cluster.service ] [M] new_master
> [M][nTUnv4H8RvKLx1A0HbBcLg][graylog][inet[/172.30.198.14:9300]], reason:
> zen-disco-join (elected_as_master)
>
> [2015-07-09 19:24:41,223][INFO ][gateway ] [M] recovered
> [1] indices into cluster_state
>
> [2015-07-09 19:47:46,609][INFO ][node ] [M] stopping
> ...
>
> [2015-07-09 19:47:46,645][INFO ][node ] [M] stopped
>
> [2015-07-09 19:47:46,645][INFO ][node ] [M] closing ...
>
> [2015-07-09 19:47:46,651][INFO ][node ] [M] closed
>
> [2015-07-09 19:51:57,942][INFO ][node ] [Nuklo]
> version[1.5.2], pid[2760], build[62ff986/2015-04-27T09:21:06Z]
>
> [2015-07-09 19:51:57,942][INFO ][node ] [Nuklo]
> initializing ...
>
> [2015-07-09 19:51:57,946][INFO ][plugins ] [Nuklo] loaded
> [], sites []
>
> [2015-07-09 19:52:00,319][INFO ][node ] [Nuklo]
> initialized
>
> [2015-07-09 19:52:00,320][INFO ][node ] [Nuklo]
> starting ...
>
> [2015-07-09 19:52:00,372][INFO ][transport ] [Nuklo]
> bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/
> 172.30.198.14:9300]}
>
> [2015-07-09 19:52:00,380][INFO ][discovery ] [Nuklo]
> graylog/Bsr-NI-nTTCUdeFL2q05Bg
>
> [2015-07-09 19:52:30,381][WARN ][discovery ] [Nuklo] waited
> for 30s and no initial state was set by the discovery
>
> [2015-07-09 19:52:30,385][INFO ][http ] [Nuklo]
> bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/
> 172.30.198.14:9200]}
>
> [2015-07-09 19:52:30,386][INFO ][node ] [Nuklo] started
>
> [2015-07-09 19:53:30,542][INFO ][cluster.service ] [Nuklo]
> new_master
> [Nuklo][Bsr-NI-nTTCUdeFL2q05Bg][graylog][inet[/172.30.198.14:9300]],
> reason: zen-disco-join (elected_as_master)
>
> [2015-07-09 19:53:31,056][INFO ][gateway ] [Nuklo]
> recovered [1] indices into cluster_state
>
> [2015-07-09 19:58:31,867][INFO ][node ] [Nuklo]
> stopping ...
>
> [2015-07-09 19:58:31,897][INFO ][node ] [Nuklo] stopped
>
> [2015-07-09 19:58:31,897][INFO ][node ] [Nuklo]
> closing ...
>
> [2015-07-09 19:58:31,901][INFO ][node
>
>
>
>
> Thanks,
>
>
> Tim
>
--
You received this message because you are subscribed to the Google Groups
"graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
