The values from the external elasticsearch.yml will override the ones specified in graylog2.conf. You can omit them from the graylog2.conf for clarity, but it should not make any difference.
The ping timeout is only applicable for the pings the ES master sends when already connected, IIRC. Can you verify that you can actually reach the ES nodes on those IPs from the server where graylog2 is running on? curl -XGET http://172.16.1.166:9200/_cluster/nodes?pretty The cluster name seems to be properly configured, because that's often a problem, but looking at the log output and config files that isn't the case. -k On Wed, Feb 5, 2014 at 5:04 PM, Scotty H <[email protected]> wrote: > Thanks for the quick reply! > Yes, I forgot to set multicast.enabled back to false after my testing > yesterday - I was trying anything I could to try and get it functional > again. I made some modifications to our multicast randevous points (sup 720 > 6500-E cisco routers) in our network late yesterday to try and accommodate > elasticsearch's multicast discovery, that setting is simply residual from > that. I've set it back to false and I still get the same problem. > Here's the debug output after I set it to false: > http://pastebin.com/TGMVKzxt > Give me a few moments to try the IPV4 preference and I'll report back my > findings. I'm going to put it on both elasticsearch and graylog's java > wrapper. > > About the elasticsearch_ prefixed settings from graylog2.conf. If I > specify a config file, should I comment out all of the prefixed ones in > graylog2.conf and rely only on the elasticsearch_config_file's settings? > It's not explicitly documented, I assume the prefixed lines will override > the config file's lines. > > An interesting problem I've also seen is discovery.zen.ping.timeout: 5s seems > to not have any effect on graylog's elasticsearch client when it attempts > to connect/find a cluster. It's always 3 seconds. I wonder if my ES master > is too slow to respond? (I doubt that.) > > > On Wednesday, February 5, 2014 9:36:09 AM UTC-5, Kay Röpke wrote: > >> Hi! >> >> Thank you for the kind words :) >> >> I think your discovery issue is that your elasticsearch-graylog.conf >> contains: >> >> 1. discovery.zen.ping.multicast.enabled: true >> >> >> as well as the unicast setting. >> I believe it will still try multicast in that case, could you retry with >> setting that to false? >> Since your elasticsearch nodes are set to disable multicast discovery I >> believe that to prevent successful discovery. >> >> To have elasticsearch listen on a specific ip you can set this entry in >> elasticsearch.yml: >> >> 1. network.bind_host: 0.0.0.0 >> >> >> You are correct with the two config files, the elasticsearch_config_file >> is used for the client node inside graylog2, the content of that file and >> the elasticsearch_ prefixed settings from graylog2.conf are merged. The >> ones with the prefixes are simply provided to allow not having to have the >> second elasticsearch config file for simpler setups (because having two >> similarly named files confused many new users). >> >> IPv6 can sometimes also interfere, to force Java to use IPv4 you can add >> the system property to the java command line: >> -Djava.net.preferIPv4Stack=true (for graylog2 server) >> (refer to http://stackoverflow.com/questions/9882357/how-to-set- >> java-net-preferipv4stack-true-in-the-java-code) >> >> The output you provided does look like it's using IPv4 already thought. >> My bet is on multicast discovery screwing it up. >> >> Let us know if you need additional assistance, you can also find us on >> IRC in #graylog2 on freenode. Or here :) >> >> Best, >> Kay >> >> >> On Wed, Feb 5, 2014 at 3:26 PM, Scotty H <[email protected]> wrote: >> >>> This new version is vastly faster than the previous revisions. It's very >>> nice. Thank you all for putting in the effort. Hopefully I'll be able to >>> convince management to secure a support contract once .20 becomes more >>> release-worthy. >>> >>> Now, I'm about to pull my hair out. I had a working implementation and >>> one afternoon I altered start up configuration to allow for more >>> elasticsearch and graylog heap memory, and I simply cannot get it to >>> connect to the cluster. I've always had zen discovery problems, even years >>> ago with older versions. When I upgrade to the new RC1.1, I blew up my old >>> elasticsearch data (about 8TB worth of logdata) and wiped my mongo DB to >>> start fresh. >>> Bigdesk has my cluster status as green, all of the indices appear to be >>> accessible. >>> >>> Here's the nitty gritty of our environment: >>> >>> Server: CentOS release 6.5 (Final) >>> graylog2.conf <http://pastebin.com/Z14c6K1k> >>> elasticsearch-graylog.conf <http://pastebin.com/GMJMehAx> >>> elasticsearch.yml <http://pastebin.com/CMNyJANz> >>> ifconfig <http://pastebin.com/k7wBwEgX> >>> graylog2-server.jar --debug <http://pastebin.com/95r4ZuZn> >>> Elasticsearch's Start Log <http://pastebin.com/DqRqqCKy> >>> Non-debug graylog-server service start <http://pastebin.com/fUagNfiA> >>> >>> Here's some stuff out of BigDesk elasticsearch plugin running on port >>> 9200: >>> Elasticsearch version: 0.90.10 >>> Indices Docs count: 705,052,192 >>> HTTP address: inet[/172.16.1.166:9200] >>> Bound address: inet[/0:0:0:0:0:0:0:0:9200] >>> Publish address: inet[/172.16.1.166:9200] >>> Transport address: inet[/172.16.1.166:9300] >>> Bound address: inet[/0:0:0:0:0:0:0:0:9300] >>> Publish address: inet[/172.16.1.166:9300] >>> >>> >>> - I've followed this guide on Torch.sh: http://support. >>> torch.sh/help/kb/graylog2-server/configuring-and-tuning- >>> >>> elasticsearch-for-graylog2-v0200<http://support.torch.sh/help/kb/graylog2-server/configuring-and-tuning-elasticsearch-for-graylog2-v0200> >>> - It doesn't matter if iptables is running or not. >>> - Multicast won't work in our environment. >>> - Setting specific bind addresses for either elasticsearch or >>> graylog won't appear to work. As you can see, I have two physical >>> interfaces. I want to listen on .166 for my graylog inputs, .165 is for >>> logstash for cisco log reformatting. I honestly don't care what >>> interface >>> elasticsearch listens on as long as it's consistant. >>> - Graylog is configured to NOT be a master or a data node both in >>> graylog.conf and elasticsearch-graylog.con (the elasticsearch_config_file >>> I'm using - it is DIFFERENT than the elasticsearch config file, that's >>> correct, right?). Commenting those master/data lines out in graylog2.conf >>> doesn't seem to help the situation either. >>> >>> What is so perplexing is the setup was working just fine previously - I >>> could start/stop the services without issue. I'm running out of things to >>> try and elasticsearch articles and threads on this problem to read. >>> Am I completely missing something? Is IPv6 breaking things? >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "graylog2" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> >>> For more options, visit https://groups.google.com/groups/opt_out. >>> >> >> -- > You received this message because you are subscribed to the Google Groups > "graylog2" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
