Given you're only on one server you are limited with what you can do. You'd be better off adding another node if you can, maybe someone else can comment on the rest.
Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [email protected] web: www.campaignmonitor.com On 13 May 2014 16:34, Rujuta Deshpande <[email protected]> wrote: > Hi, > > I have set up an elasticsearch cluster with 1 shard and 0 replicas. > My system has 16 GB RAM and I have allocated 8 GB to the ES Max/Min Heap. > > We are indexing a large number of logs everyday and the size of our daily > index is approximately 3,500,000 documents. > > We are using Kibana to query ES and generate reports. Most of our reports > are histograms and hence require heavy facetting. My observation was this - > the dashboards took very long to load (depending on the time limit > selected) and the field cache size being unlimited (by default) started > rising and eventually resulted in an Out of Memory Error. > > I have now restricted the field cache size to 50% of the available heap > memory. Although this does result in reducing this error, there is not much > difference in the performance and search takes long. > > Another observation is that with 1 shard and 0 replicas, my ES node is not > making use of the other CPU cores. I have 4 cores on my system and the CPU% > shown by the top command for the elasticsearch process just barely exceeds > 100%. I believe this indicates that it uses one core in entirety but not > all 4 cores. > > Will increasing the number of shards make better use of the multi-core > architecture and enable parallel search? Also, if so, what is the best way > to get this working? Should I make changes in the ES configuration file and > then re-start the cluster? How does this affect the currently existing > indices? We create 2 indices per day. Now when I increase the number of > shards for the cluster, how will the data in the previously created indices > get distributed among the newly created shards? > > Thanks, > Rujuta > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/fa490e88-87a4-4c9d-aeef-b93f51e7e7e0%40googlegroups.com<https://groups.google.com/d/msgid/elasticsearch/fa490e88-87a4-4c9d-aeef-b93f51e7e7e0%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624YDYQBGjouagbVBWmV7xYFe1JnLrkdkchRLBbQOTgFHTQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
