Interesting, I had previously found that many (but not all) ES params are extensively processed by code fairly deep in the guts of ES itself (unrelated to Metron’s use of it), to allow this kind of flexibility. Thanks very much for sharing your results, Mike. Looks like we’ll continue with the release.
Laurens, I hope you can find what’s going on, on your system. I would suggest that, if it isn’t being written through to the yml file, it seems to be an Ambari or install issue rather than an ES issue per se. Thanks, --Matt From: Michael Miklavcic <[email protected]> Date: Wednesday, September 13, 2017 at 2:08 PM To: Matt Foley <[email protected]> Cc: Laurens Vets <[email protected]>, "[email protected]" <[email protected]> Subject: Re: Elasticsearch masters_also_are_datanodes doesn't work. Here's what I found. Data is flowing into ES indexes. I can change the "masters_also_are_datanodes" property and it is reflected in the node.data property in elasticsearch.yml. Here's my search for this property in the source code. It is properly being used int the ES mpack scripts afaik. Targets Occurrences of 'masters_also_are_datanodes' in Directory devprojects/metron/metron-deployment Found Occurrences (6 usages found) Unclassified occurrence (5 usages found) metron-deployment (2 usages found) metron-deployment/other-examples/manual-install (1 usage found) Manual_Install_CentOS6.md (1 usage found) 810 "masters_also_are_datanodes" to "true" metron-deployment/roles/ambari_config/vars (1 usage found) single_node_vm.yml (1 usage found) 104 masters_also_are_datanodes: "1" metron_mpack (3 usages found) common-services.ELASTICSEARCH.2.3.3.configuration (1 usage found) elastic-site.xml (1 usage found) 30 <name>masters_also_are_datanodes</name> common-services.ELASTICSEARCH.2.3.3.package.scripts (1 usage found) params.py (1 usage found) 26 masters_also_are_datanodes = config['configurations']['elastic-site']['masters_also_are_datanodes'] common-services.ELASTICSEARCH.2.3.3.package.templates (1 usage found) elasticsearch.master.yaml.j2 (1 usage found) 36 data: {{ masters_also_are_datanodes }} Usage in string constants (1 usage found) metron_mpack (1 usage found) common-services.ELASTICSEARCH.2.3.3.package.scripts (1 usage found) params.py (1 usage found) 26 masters_also_are_datanodes = config['configurations']['elastic-site']['masters_also_are_datanodes'] It should be noted that regardless of the ES documentation, this seems to work with a variety of values including "true" (with quotes) and 1 (without quotes). When I set the value to garbage through Ambari, I see this in /var/log/elasticsearch/metron.log java.lang.IllegalArgumentException: value cannot be parsed to boolean [ true/1/on/yes OR false/0/off/no ] at org.elasticsearch.common.Booleans.parseBooleanExact(Booleans.java:97) at org.elasticsearch.cluster.node.DiscoveryNode.dataNode(DiscoveryNode.java:256) at org.elasticsearch.cluster.node.DiscoveryNodes$Builder.build(DiscoveryNodes.java:659) at org.elasticsearch.cluster.ClusterState$Builder.nodes(ClusterState.java:571) at org.elasticsearch.cluster.service.InternalClusterService.doStart(InternalClusterService.java:205) at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:68) at org.elasticsearch.node.Node.start(Node.java:279) at org.elasticsearch.bootstrap.Bootstrap.start(Bootstrap.java:206) at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:272) at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35) Apparently, even yes/no will work! I tried it, and can confirm they work with quotes. But surprisingly, this does NOT work if you pass true/false or yes/no without quotes. In summary, here are the supported values: * 1 * 0 * "true" * "false" * "yes" * "no" Mike On Wed, Sep 13, 2017 at 2:17 PM, Matt Foley <[email protected]<mailto:[email protected]>> wrote: Sorry, you’ve beat me. Mike, what have you found in full-dev? Does ES work correctly there? From: Laurens Vets <[email protected]<mailto:[email protected]>> Date: Wednesday, September 13, 2017 at 1:14 PM To: Matt Foley <[email protected]<mailto:[email protected]>> Cc: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>>, Michael Miklavcic <[email protected]<mailto:[email protected]>> Subject: Re: Elasticsearch masters_also_are_datanodes doesn't work. Yes, I'm saving the changes. The orange bar pops up saying I have to restart elasticsearch, which I do. I'm logged in with the admin user, it has privileges to do everything else. I'm using a bare metal install via Ambari. Everything else seems to work through Ambari except this :) On 2017-09-13 12:04, Matt Foley wrote: That it doesn't work in Ambari, at least to the point of writing thru to the config file (whether or not it actually works with ES), is quite mystifying. Silly question, but, are you pressing the "Save" button after changing the field? Are you perhaps not logged into Ambari with an ambari user id that has privs to change this config? Did you use non-default user configuration with Ambari, and if so are you running with an admin-priv ambari account? From: Laurens Vets <[email protected]<mailto:[email protected]>> Date: Tuesday, September 12, 2017 at 8:32 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Cc: Michael Miklavcic <[email protected]<mailto:[email protected]>>, Matt Foley <[email protected]<mailto:[email protected]>> Subject: Re: Elasticsearch masters_also_are_datanodes doesn't work. See inline. Now, Laurens, two question for you: First, please confirm: Are you setting the value of masters_also_are_datanodes via the masters_also_are_datanodes field in the Ambari configuration GUI? Or directly editing files? I'm using Ambari to edit the configuration. However, I have to manually edit the file locally as the Ambari configuration doesn't work. Second, do you set it at the initial install time, or afterward? I'm pretty sure it needs to be set at initial install time, before you ever launch the Elasticsearch nodes. I'm dubious about whether an Elasticsearch master can learn to be a datanode if it first wakes up and configures itself to not be one. But I'm not an Elasticsearch expert, so if someone else knows different, please say so. In this case, I forgot to set it initially, so I tried to change it afterwards to no avail. Hope this helps more than obfuscates. --Matt From: Michael Miklavcic <[email protected]<mailto:[email protected]>> Date: Tuesday, September 12, 2017 at 4:54 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Cc: Matt Foley <[email protected]<mailto:[email protected]>> Subject: Re: Elasticsearch masters_also_are_datanodes doesn't work. At the very least, the value provided by default seems to have changed to a "1" instead of "true" without the tooltip having been updated to match. Error! Filename not specified. On Tue, Sep 12, 2017 at 4:00 PM, Michael Miklavcic <[email protected]<mailto:[email protected]>> wrote: I think this is our default setup for full dev. It's only a 1-node VM, so I'm pretty sure that it would not work otherwise. I'm spinning up full dev now and will look into it also. On Tue, Sep 12, 2017 at 3:04 PM, Laurens Vets <[email protected]<mailto:[email protected]>> wrote: https://issues.apache.org/jira/browse/METRON-1181 I'll also spin up a full-dev environment and see what happens there. On 2017-09-10 21:50, Matt Foley wrote: Laurens, please open a jira. Altho it may seem obvious, please include full repro. This may be a showstopper, as it presumably (?) prevents ES from working on a single-node deployment? On 9/10/17, 4:01 PM, "Laurens Vets" <[email protected]<mailto:[email protected]>> wrote: Another issue I noticed. Setting "masters_also_are_datanodes" in Ambari to "true" does not work. The settings in /etc/elasticsearch/elasticsearch.yml will always be false when restarting elasticsearch...
