63 indices with a total of 1,278,495,089 messages under management. I'm going to upgrade to RC2 in just a few moments, retaining my existing configuration files.
On Thursday, February 6, 2014 6:21:08 PM UTC-5, Kay Röpke wrote: > > We can of course all edit the docs, if there's something we can add to > help people or make the defaults more clear we will happily do that, no > matter if a support contract is involved or not. > We always want to give the best support possible, regardless if someone > pays us or not. > If that helps revenue the better :) > > It's been a very long day for me, let's catch up tomorrow to collect the > results, if that's fine for you. > > Best, > Kay > On Feb 7, 2014 12:00 AM, "Scotty H" <[email protected] <javascript:>> > wrote: > >> Yes, I understand that - the assumption would be to use the configuration >> file instead if you needed those options, but there are problems with zen >> discovery -as you saw with my issues. >> Are all members of the graylog/torch team allowed to update public >> documentation at will, or are there hurdles associated with that? >> My line of reasoning isL the more successful installs around the globe, >> the more potential for support contracts there will be. To accomplish that, >> more detailed documentation associated with graylog server install and >> troubleshooting would help facilitate that. >> >> On Thursday, February 6, 2014 4:57:41 PM UTC-5, Kay Röpke wrote: >>> >>> Please note that the elasticsearch_* options are limited, I.e. not all >>> settings will be accepted in the graylog2 config file. >>> Only those that are listed in the sample file are actually doing >>> anything. We should probably be clearer about that... >>> On Feb 6, 2014 4:30 PM, "Scotty H" <[email protected]> wrote: >>> >>>> Yes, the publish address was 165 earlier in the thread, It's operating >>>> entirely on .166 now. >>>> Bad: elasticsearch_discovery_zen_ping_unicast_hosts = [" >>>> 172.16.1.166:9300"] >>>> Good: elasticsearch_discovery_zen_ping_unicast_hosts = >>>> 172.16.1.166:9300 >>>> >>>> It will connect now, but I'm not comfortable with it just yet. I'll do >>>> some more testing and report back if I have any more issues. >>>> >>>> It sounds like I can't use the custom elasticsearch-graylog >>>> configuration at all. >>>> elasticsearch_network.publish_host = 172.16.1.166 appears to work. >>>> >>>> >>>> >>>> On Wednesday, February 5, 2014 3:59:21 PM UTC-5, Kay Röpke wrote: >>>>> >>>>> The extreme log output might be because something is already writing >>>>> logs to graylog2. >>>>> I'm on my phone only right now and will be out of the office tomorrow. >>>>> >>>>> I noticed that your graylog2 server has a publish address of .165 for >>>>> the elastic search client. Can .166 talk to .165? It seems to me that the >>>>> client sees the es master, but that the master cannot connect back. >>>>> Try setting the network.host settings explicitly to .166 everywhere. >>>>> Also, your ES node does not need to be configured with unicast hosts >>>>> (it is set to its own address in your config). I'm unsure if that can >>>>> have >>>>> an adverse effect, but something I noticed. >>>>> >>>>> What does the elasticsearch log say? The graylog2 output says it can >>>>> connect, but I'm willing to bet that the other side can't. >>>>> >>>>> Best, >>>>> Kay >>>>> On Feb 5, 2014 8:40 PM, "Scotty H" <[email protected]> wrote: >>>>> >>>>>> Yes, very odd - The very strange thing to me is the configuration >>>>>> related to zen discovery, iptables, etc didn't change when it stopped >>>>>> working. >>>>>> Yes, 172.16.1.166 and 172.16.1.165 are the same host. It has two >>>>>> interfaces on it - I want to dedicate one (.165) to logstash, and >>>>>> graylog/ES will be .166 >>>>>> One single node, for now. This is a monolithic box: Dual 4 core >>>>>> hyperthreaded Xeons @ 3.2ghz, 128Gb ECC RAM, 15TB RAID6 with a high >>>>>> performance areca hardware controller. I may look into spinning up >>>>>> multiple >>>>>> ES/graylog nodes on the same host as demand increases - but I need to >>>>>> get >>>>>> discovery working consistently first. >>>>>> >>>>>> I've tried setting it to each individual interface >>>>>> in elasticsearch.yml and elasticsearch-graylog.conf, to no avail. For >>>>>> posterity, I've applied that and here we are: >>>>>> elasticsearch.yml <http://pastebin.com/RjT9Thdb> >>>>>> elasticsearch-graylog.conf <http://pastebin.com/ga8B7NtK> >>>>>> graylog-server --debug <http://pastebin.com/EpxvvRih> >>>>>> No connection. >>>>>> >>>>>> >>>>>> Taking your additional advice and removing the >>>>>> elasticsearch_config_file alltogether and relying only on the >>>>>> graylog2.conf >>>>>> file, here's the config file: >>>>>> graylog2.conf - No >>>>>> elasticsearch_config_file<http://pastebin.com/6fkyVbBe> >>>>>> graylog2-server.jar --debug <http://pastebin.com/B1HsY91g> >>>>>> >>>>>> The first time I ran it, my 10,000 lines of scrollback cleared out >>>>>> within 2 seconds of error messages, so something is happening. I posted >>>>>> the >>>>>> part 2nd to pastebin so you could see the initialization. >>>>>> If I run it as a service, I get a 4.5MB log file of errors. It's >>>>>> attached to this post. I cleared it out before I restarted the service. >>>>>> >>>>>> So, something else is happening, which may or may not be good. >>>>>> >>>>>> >>>>>> On Wednesday, February 5, 2014 12:18:46 PM UTC-5, Kay Röpke wrote: >>>>>>> >>>>>>> Beyond odd. >>>>>>> The only thing that strikes me as unusual is that 127.0.0.1:9300 is >>>>>>> in the unicast host list. >>>>>>> Usually you would omit that, and only put the members in the >>>>>>> elasticsearch cluster into it. >>>>>>> Do you have multiple elasticsearch nodes? Or is the one running on >>>>>>> .166:9300 the only one? >>>>>>> I guess 165:9300 is actually the same host right? >>>>>>> >>>>>>> I would change the elasticsearch-graylog.conf to say: >>>>>>> >>>>>>> 1. discovery.zen.ping.unicast.hosts: ["172.16.1.166:9300"] >>>>>>> >>>>>>> >>>>>>> The same for the actual elasticsearch.yml, that should not list any >>>>>>> other nodes if you don't have them, especially not the graylog2 client >>>>>>> node. >>>>>>> >>>>>>> Then it should simply discover the cluster. >>>>>>> >>>>>>> In fact, I would probably leave out the elasticsearch_config_file >>>>>>> option, and configure everything in the graylog2.conf. >>>>>>> Here's what I would configure: >>>>>>> >>>>>>> elasticsearch_cluster_name = graylog2 >>>>>>> elasticsearch_node_name = graylog2-server >>>>>>> elasticsearch_node_master = false >>>>>>> elasticsearch_node_data = false >>>>>>> elasticsearch_transport_tcp_port = 9350 >>>>>>> elasticsearch_http_enabled = false >>>>>>> elasticsearch_discovery_zen_ping_multicast_enabled = false >>>>>>> elasticsearch_discovery_zen_ping_unicast_hosts = 172.1.16.166:9300 >>>>>>> >>>>>>> Basically all of these except the last two lines are the default >>>>>>> values for those settings. >>>>>>> You should also be able to see the graylog2 server node join the >>>>>>> cluster in the elasticsearch log output. >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Wed, Feb 5, 2014 at 5:54 PM, Scotty H <[email protected]> wrote: >>>>>>> >>>>>>>> Bigdesk screenshot. <http://i.imgur.com/WvxRiTl.png> >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "graylog2" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>>>>> >>>>>>> >>>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "graylog2" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>>> >>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "graylog2" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> For more options, visit https://groups.google.com/groups/opt_out. >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "graylog2" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> For more options, visit https://groups.google.com/groups/opt_out. >> > -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
