Alex, It looks like the issue was indeed not with the es itself but with our virtual machine setup. We had to setup so that at least 2x heap space reserved to our machines all the time. It has been running okay for a month.
Thanks, T On Thursday, January 16, 2014 8:50:38 AM UTC-8, Tula wrote: > > Hi, > > I have 3 ubuntu VM's on a private network, each has 64GB ram. I started > ES Beta2 (need it to use term vector feature) on each node with 30GB heap > space and with the following changes in the configuration file: > > discovery.zen.ping.multicast.enabled: false > discovery.zen.ping.unicast.hosts: ["host1","host2","host3"] > bootstrap.mlockall: true > gateway.type: local > gateway.recover_after_nodes: 2 > gateway.expected_nodes: 3 > discovery.zen.minimum_master_nodes: 2 > > Sometimes es processes get killed by OOM killer and when I restart a node > I ran into situation like > > host1,host2,host3 thinking all three nodes are connected and form a > cluster, and the host3 thinks it is all by itself with status red (names > are random) > > host1 and host2 form one cluster and host2 and host3 forms another > cluster. > > Any idea about what I am doing wrong and any suggestions? I will need to > index 20 million documents and have 3 separate indexes with 5 shards 1 > replica. > > Thanks, > T > > > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1265e27a-6a02-40b8-b2d4-286b96c10429%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
