[
https://issues.apache.org/jira/browse/METRON-609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15726355#comment-15726355
]
Matt Foley commented on METRON-609:
-----------------------------------
Hi [~justinleet], my thinking on this:
* Provide a boolean config parameter "masters_also_are_datanodes" which both
gets transcribed to master's "node.data", and could be used in calculations of
effective number of datanodes. (I can't figure out how to do use those
calculations as constraints either, but I haven't talked to the Ambari guys
yet.)
* Thus, you could either install "dedicated Masters and dedicated datanodes
(aka Slaves)", or "(non-dedicated) Master/datanodes and additional dedicated
datanodes". There's no evident reason to allow both dedicated and
non-dedicated Masters in the cluster, IMO.
* I think we have less need to enforce the recommended production config (1+
dedicated Masters and 3+ dedicated Slaves). For example, a 3-node combined
Master/datanode configuration should be reasonable for small Metron clusters,
even in production. And we want to allow 1- or 2-node clusters for testing and
lab work. We should just clearly document the recommended configs, and that ES
clusters with less than 3 datanodes are supported only for lab or testing
purposes.
Obvious improvements that are also desirable, are to use the actual Master and
Slave assignments to capture the number of nodes installed and to populate the
list of Master hosts in zen_discovery_ping_unicast_hosts. Again I'll have to
talk to the Ambari guys for this, because I can't find it in the documentation.
> Enhance Mpack to handle single-node and small-cluster installs of
> Elasticsearch
> -------------------------------------------------------------------------------
>
> Key: METRON-609
> URL: https://issues.apache.org/jira/browse/METRON-609
> Project: Metron
> Issue Type: Improvement
> Reporter: Matt Foley
> Assignee: Matt Foley
>
> The current Mpack for Ambari install of Metron requires that Elasticsearch be
> installed with 1+ dedicated Masters and 3+ dedicated Slaves. It does not
> support combined master/datanode configuration (non-"dedicated" Master),
> which is the recommended way to run a single-node configuration of
> Elasticsearch; nor does it allow less than 3 dedicated Slaves, thereby
> requiring the cluster have at least 4 nodes.
> This jira proposes to enable 1-, 2-, and 3-node clusters to have a working
> Elasticsearch install via the Mpack. It will also allow non-dedicated
> master/datanodes, which are required for single-node and useful for few-node
> clusters.
> I intend to also include the following enhancements and fixes:
> * Determine whether elastic-sysconfig:data_dir and elastic-site:path_data
> should have same default value and if so fix it. (I think they should, and
> they don't. Probably there should only be one variable instead of two.)
> * Provide Quick Links in Ambari service page for Elasticsearch to the
> self-report pages for Elasticsearch health, node list, and other "_cat"
> status. May include some "_cluster" status.
> * Improve various mouse-over Description fields in the GUI
> * Improve the order of fields in the GUI, to group at the beginning the
> fields that must be modified or are commonly modified. Possibly separate
> "basic" and "advanced" configs.
> * Provide a README.md that describes what is going on with the various
> resource files in the Mpack src
> * Enhance the wiki page for cluster installation, with regard to single-node
> install with Mpack.
> * Database-preserving upgrade from version 1.0.0.0.
> Also see
> https://github.com/apache/incubator-metron/tree/master/metron-deployment#current-limitations
> for some other known issues. And the documentation in that README.md page
> is significantly out of date wrt the Mpack.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
