According to Elastic HQ I currently am keeping 12 days of logs - 8 nodes (3 data nodes that can all be master and 5 Logstash) - 399 shards - 59 indices - 1,540,343,998 documents - 380 GB
On Thursday, February 12, 2015 at 4:41:01 PM UTC-5, Mark Walkom wrote: > > Except that is overkill when you only have 3 nodes. > > How much data do you have in the cluster? > > On 13 February 2015 at 01:15, Itamar Syn-Hershko <[email protected] > <javascript:>> wrote: > >> See this: >> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-node.html >> >> Basically, the recommended pattern talks about isolating >> responsibilities. A node should either be a data node, master-eligible >> node, or an external gateway to the cluster (client node) >> >> -- >> >> Itamar Syn-Hershko >> http://code972.com | @synhershko <https://twitter.com/synhershko> >> Freelance Developer & Consultant >> Lucene.NET committer and PMC member >> >> On Thu, Feb 12, 2015 at 4:08 PM, Eric <[email protected] <javascript:>> >> wrote: >> >>> Hello, >>> >>> Currently I have a 3 node ElasticSearch cluster. Each node is a RHEL VM >>> with 16 gig RAM. The basic config is: >>> >>> - All nodes can be master and are data nodes. >>> - 3 shards and 1 replica >>> - 6 different indexes >>> >>> I'm starting to run into issues of ElasticSearch bogging down on >>> searches and is completely freezing sometimes at night. I've dedicated 9 >>> gig to heap size and it says i'm using ~60% of the heap RAM and about 70% >>> of the overall heap. So even though I'm using quite a bit of the heap, I'm >>> not maxed out. I've attached a screenshot of the exact stats from Elastic >>> HQ. I'm averaging around 10,000 events/sec coming into the cluster from 6 >>> different Logstash instances on another server. >>> >>> My question is what can I do to help the stability and speed of my >>> cluster. Currently I'm having issues with 1 node going down and it taking >>> everything else down. The HA portion isn't working very well. I'm debating >>> about either adding 1 more node with the exact same stats or adding 2 more >>> smaller VMs that will act as master nodes only. I didn't know which one was >>> recommended or where I would get the biggest bang for the buck. >>> >>> Any information would be greatly appreciated. >>> >>> Thanks, >>> Eric >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elasticsearch" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected] <javascript:>. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/elasticsearch/354a2326-5532-4239-87ea-f02af64fe71f%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/elasticsearch/354a2326-5532-4239-87ea-f02af64fe71f%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZureLROJMaO7gVprFjA2OmRZA0ZYyH1v%2Bges06u_V__6w%40mail.gmail.com >> >> <https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZureLROJMaO7gVprFjA2OmRZA0ZYyH1v%2Bges06u_V__6w%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/10133d13-ff4d-4798-80e2-4fffabfeb53c%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
