How would the nodes on the other side of the network become master with the formula (total num of nodes / 2 + 1) one side will always become inaccessible?
On Mon, Jan 13, 2014 at 3:50 PM, Mark Walkom <[email protected]>wrote: > The logs shows that a node was not contactable and so was removed from the > cluster. > That could be a load issue on the node, network issues, or something else. > Check the logs on the node that disconnected and see what you find. > > If you leave the defaults, you risk the chance of a split-brain situation, > where nodes lose contact with each other and then decide to become a > master. This can cause you to end up with multiple masters in multiple > clusters, which is not good for data integrity. > > Setting the min masters means that if a node cannot contact N masters, it > will refuse connections. > > Regards, > Mark Walkom > > Infrastructure Engineer > Campaign Monitor > email: [email protected] > web: www.campaignmonitor.com > > > On 14 January 2014 10:46, Mohit Anchlia <[email protected]> wrote: > >> I am trying to understand why I should change it? This setting looks like >> the right setting in most cases. Did you see anything out of ordinary in >> that log message I pasted? >> >> On Mon, Jan 13, 2014 at 3:40 PM, David Pilato <[email protected]> wrote: >> >>> No it's not the default setting. >>> min master node = total num of nodes / 2 + 1 >>> >>> You have 5 nodes, 5 / 2 + 1 = 3 >>> >>> >>> >>> -- >>> *David Pilato* | *Technical Advocate* | *Elasticsearch.com* >>> @dadoonet <https://twitter.com/dadoonet> | >>> @elasticsearchfr<https://twitter.com/elasticsearchfr> >>> >>> >>> Le 14 janvier 2014 at 00:06:34, Mohit Anchlia >>> ([email protected]<//[email protected]>) >>> a écrit: >>> >>> Is the default setting 3? How do I go about selecting right >>> configuration setting for this property? >>> >>> On Mon, Jan 13, 2014 at 3:01 PM, David Pilato <[email protected]> wrote: >>> >>>> So I think you should first set minimum master node to 3. See >>>> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-update-settings.html#_discovery >>>> >>>> Probably you need to close your isolated node and restart it. >>>> >>>> >>>> >>>> -- >>>> *David Pilato* | *Technical Advocate* | *Elasticsearch.com* >>>> @dadoonet <https://twitter.com/dadoonet> | >>>> @elasticsearchfr<https://twitter.com/elasticsearchfr> >>>> >>>> >>>> Le 13 janvier 2014 at 23:58:33, Mohit Anchlia >>>> ([email protected]<//[email protected]>) >>>> a écrit: >>>> >>>> I have 5 nodes. I am using all the default settings. >>>> >>>> On Mon, Jan 13, 2014 at 2:32 PM, David Pilato <[email protected]> wrote: >>>> >>>>> It sounds like a split brain issue or something. >>>>> >>>>> How many nodes do you have? >>>>> Did you set minimum master node? >>>>> >>>>> -- >>>>> *David Pilato* | *Technical Advocate* | *Elasticsearch.com* >>>>> @dadoonet <https://twitter.com/dadoonet> | >>>>> @elasticsearchfr<https://twitter.com/elasticsearchfr> >>>>> >>>>> >>>>> Le 13 janvier 2014 at 22:48:32, Mohit Anchlia >>>>> ([email protected]<//[email protected]>) >>>>> a écrit: >>>>> >>>>> Thanks! It looks like one of the nodes even though it was up still >>>>> it went out of cluster. Is there no retry? >>>>> >>>>> >>>>> [2014-01-02 17:55:01,148][INFO ][cluster.service ] [10.80.140.14] >>>>> master {new >>>>> [10.80.140.14][267foffDRhO6zzXlHg3YKA][inet[/10.80.140.14:9300]]{group=group1}, >>>>> previous >>>>> [10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2}}, >>>>> removed >>>>> {[10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2},}, >>>>> reason: zen-disco-master_failed >>>>> ([10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300 >>>>> ]]{group=group2}) >>>>> >>>>> [2014-01-02 17:55:02,158][INFO ][cluster.service ] [10.80.140.14] >>>>> removed >>>>> {[10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2},}, >>>>> reason: >>>>> zen-disco-node_failed([10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2}), >>>>> reason failed to ping, tried [3] times, each with maximum [30s] timeout >>>>> >>>>> >>>>> On Mon, Jan 13, 2014 at 1:34 PM, David Pilato <[email protected]> wrote: >>>>> >>>>>> Yellow means that some replicas are not allocated. >>>>>> If you have only one node and have index set with one replica >>>>>> (default), yellow is the expected state. >>>>>> >>>>>> >>>>>> -- >>>>>> *David Pilato* | *Technical Advocate* | *Elasticsearch.com* >>>>>> @dadoonet <https://twitter.com/dadoonet> | >>>>>> @elasticsearchfr<https://twitter.com/elasticsearchfr> >>>>>> >>>>>> >>>>>> Le 13 janvier 2014 at 22:30:23, Mohit Anchlia >>>>>> ([email protected]<//[email protected]>) >>>>>> a écrit: >>>>>> >>>>>> Cluster state is showing yellow. What's the best way to find why >>>>>> it's in yellow state? >>>>>> >>>>>> It has been up and stable so far except when I looked today it shows >>>>>> state turned into yellow. >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "elasticsearch" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpSSEVTpYA-DB8Hgfyo9WdR4bVRoKtQ9c_PC%2BFZ1aQiKg%40mail.gmail.com >>>>>> . >>>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "elasticsearch" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/elasticsearch/etPan.52d45be9.7c3dbd3d.376c%40MacBook-Air-de-David.local. >>>>>> >>>>>> >>>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "elasticsearch" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpLn_AgMKcTwQuGD-CHHe6ptg%3DRxaNGLPxFn3kroed%3DVQ%40mail.gmail.com. >>>>> >>>>> >>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "elasticsearch" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/elasticsearch/etPan.52d46968.5577f8e1.376c%40MacBook-Air-de-David.local. >>>>> >>>>> >>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "elasticsearch" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/elasticsearch/CAOT3TWrsU-k9u6hir2CWoobhnaOTD_3Bjk3Xf_ZjMDSBkB0DeQ%40mail.gmail.com. >>>> >>>> >>>> For more options, visit https://groups.google.com/groups/opt_out. >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "elasticsearch" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/elasticsearch/etPan.52d47036.77465f01.376c%40MacBook-Air-de-David.local. >>>> >>>> >>>> For more options, visit https://groups.google.com/groups/opt_out. >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elasticsearch" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/elasticsearch/CAOT3TWqbrUh5p2t74Bc_6kL%3DxjAcLGrJzJ6wt_fRQqy6Dj-DKg%40mail.gmail.com >>> . >>> >>> For more options, visit https://groups.google.com/groups/opt_out. >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elasticsearch" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/elasticsearch/etPan.52d47982.5e884adc.376c%40MacBook-Air-de-David.local >>> . >>> >>> For more options, visit https://groups.google.com/groups/opt_out. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/CAOT3TWqx_KLF_p8nJ4m3On-nSvRncY9OdHa8KS1HDjcRHhfQhQ%40mail.gmail.com >> . >> >> For more options, visit https://groups.google.com/groups/opt_out. >> > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/CAEM624aGEw8HqnMP--Fb-U%2BCjB58VBFb1bQenk_RmenNKAmMHg%40mail.gmail.com > . > > For more options, visit https://groups.google.com/groups/opt_out. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAOT3TWrGJOS4%2BA6bm%3DKLm7aTDoQxX8TZGM7CgxVoGAeO%3D-XNLA%40mail.gmail.com. For more options, visit https://groups.google.com/groups/opt_out.
