We can of course all edit the docs, if there's something we can add to help people or make the defaults more clear we will happily do that, no matter if a support contract is involved or not. We always want to give the best support possible, regardless if someone pays us or not. If that helps revenue the better :)
It's been a very long day for me, let's catch up tomorrow to collect the results, if that's fine for you. Best, Kay On Feb 7, 2014 12:00 AM, "Scotty H" <[email protected]> wrote: > Yes, I understand that - the assumption would be to use the configuration > file instead if you needed those options, but there are problems with zen > discovery -as you saw with my issues. > Are all members of the graylog/torch team allowed to update public > documentation at will, or are there hurdles associated with that? > My line of reasoning isL the more successful installs around the globe, > the more potential for support contracts there will be. To accomplish that, > more detailed documentation associated with graylog server install and > troubleshooting would help facilitate that. > > On Thursday, February 6, 2014 4:57:41 PM UTC-5, Kay Röpke wrote: >> >> Please note that the elasticsearch_* options are limited, I.e. not all >> settings will be accepted in the graylog2 config file. >> Only those that are listed in the sample file are actually doing >> anything. We should probably be clearer about that... >> On Feb 6, 2014 4:30 PM, "Scotty H" <[email protected]> wrote: >> >>> Yes, the publish address was 165 earlier in the thread, It's operating >>> entirely on .166 now. >>> Bad: elasticsearch_discovery_zen_ping_unicast_hosts = [" >>> 172.16.1.166:9300"] >>> Good: elasticsearch_discovery_zen_ping_unicast_hosts = 172.16.1.166:9300 >>> >>> It will connect now, but I'm not comfortable with it just yet. I'll do >>> some more testing and report back if I have any more issues. >>> >>> It sounds like I can't use the custom elasticsearch-graylog >>> configuration at all. >>> elasticsearch_network.publish_host = 172.16.1.166 appears to work. >>> >>> >>> >>> On Wednesday, February 5, 2014 3:59:21 PM UTC-5, Kay Röpke wrote: >>>> >>>> The extreme log output might be because something is already writing >>>> logs to graylog2. >>>> I'm on my phone only right now and will be out of the office tomorrow. >>>> >>>> I noticed that your graylog2 server has a publish address of .165 for >>>> the elastic search client. Can .166 talk to .165? It seems to me that the >>>> client sees the es master, but that the master cannot connect back. >>>> Try setting the network.host settings explicitly to .166 everywhere. >>>> Also, your ES node does not need to be configured with unicast hosts >>>> (it is set to its own address in your config). I'm unsure if that can have >>>> an adverse effect, but something I noticed. >>>> >>>> What does the elasticsearch log say? The graylog2 output says it can >>>> connect, but I'm willing to bet that the other side can't. >>>> >>>> Best, >>>> Kay >>>> On Feb 5, 2014 8:40 PM, "Scotty H" <[email protected]> wrote: >>>> >>>>> Yes, very odd - The very strange thing to me is the configuration >>>>> related to zen discovery, iptables, etc didn't change when it stopped >>>>> working. >>>>> Yes, 172.16.1.166 and 172.16.1.165 are the same host. It has two >>>>> interfaces on it - I want to dedicate one (.165) to logstash, and >>>>> graylog/ES will be .166 >>>>> One single node, for now. This is a monolithic box: Dual 4 core >>>>> hyperthreaded Xeons @ 3.2ghz, 128Gb ECC RAM, 15TB RAID6 with a high >>>>> performance areca hardware controller. I may look into spinning up >>>>> multiple >>>>> ES/graylog nodes on the same host as demand increases - but I need to get >>>>> discovery working consistently first. >>>>> >>>>> I've tried setting it to each individual interface >>>>> in elasticsearch.yml and elasticsearch-graylog.conf, to no avail. For >>>>> posterity, I've applied that and here we are: >>>>> elasticsearch.yml <http://pastebin.com/RjT9Thdb> >>>>> elasticsearch-graylog.conf <http://pastebin.com/ga8B7NtK> >>>>> graylog-server --debug <http://pastebin.com/EpxvvRih> >>>>> No connection. >>>>> >>>>> >>>>> Taking your additional advice and removing the >>>>> elasticsearch_config_file alltogether and relying only on the >>>>> graylog2.conf >>>>> file, here's the config file: >>>>> graylog2.conf - No elasticsearch_config_file<http://pastebin.com/6fkyVbBe> >>>>> graylog2-server.jar --debug <http://pastebin.com/B1HsY91g> >>>>> >>>>> The first time I ran it, my 10,000 lines of scrollback cleared out >>>>> within 2 seconds of error messages, so something is happening. I posted >>>>> the >>>>> part 2nd to pastebin so you could see the initialization. >>>>> If I run it as a service, I get a 4.5MB log file of errors. It's >>>>> attached to this post. I cleared it out before I restarted the service. >>>>> >>>>> So, something else is happening, which may or may not be good. >>>>> >>>>> >>>>> On Wednesday, February 5, 2014 12:18:46 PM UTC-5, Kay Röpke wrote: >>>>>> >>>>>> Beyond odd. >>>>>> The only thing that strikes me as unusual is that 127.0.0.1:9300 is >>>>>> in the unicast host list. >>>>>> Usually you would omit that, and only put the members in the >>>>>> elasticsearch cluster into it. >>>>>> Do you have multiple elasticsearch nodes? Or is the one running on >>>>>> .166:9300 the only one? >>>>>> I guess 165:9300 is actually the same host right? >>>>>> >>>>>> I would change the elasticsearch-graylog.conf to say: >>>>>> >>>>>> 1. discovery.zen.ping.unicast.hosts: ["172.16.1.166:9300"] >>>>>> >>>>>> >>>>>> The same for the actual elasticsearch.yml, that should not list any >>>>>> other nodes if you don't have them, especially not the graylog2 client >>>>>> node. >>>>>> >>>>>> Then it should simply discover the cluster. >>>>>> >>>>>> In fact, I would probably leave out the elasticsearch_config_file >>>>>> option, and configure everything in the graylog2.conf. >>>>>> Here's what I would configure: >>>>>> >>>>>> elasticsearch_cluster_name = graylog2 >>>>>> elasticsearch_node_name = graylog2-server >>>>>> elasticsearch_node_master = false >>>>>> elasticsearch_node_data = false >>>>>> elasticsearch_transport_tcp_port = 9350 >>>>>> elasticsearch_http_enabled = false >>>>>> elasticsearch_discovery_zen_ping_multicast_enabled = false >>>>>> elasticsearch_discovery_zen_ping_unicast_hosts = 172.1.16.166:9300 >>>>>> >>>>>> Basically all of these except the last two lines are the default >>>>>> values for those settings. >>>>>> You should also be able to see the graylog2 server node join the >>>>>> cluster in the elasticsearch log output. >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Feb 5, 2014 at 5:54 PM, Scotty H <[email protected]> wrote: >>>>>> >>>>>>> Bigdesk screenshot. <http://i.imgur.com/WvxRiTl.png> >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "graylog2" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>>>> >>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "graylog2" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "graylog2" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/groups/opt_out. >>> >> -- > You received this message because you are subscribed to the Google Groups > "graylog2" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
