Hi, I have set up an elasticsearch cluster with 1 shard and 0 replicas. My system has 16 GB RAM and I have allocated 8 GB to the ES Max/Min Heap.
We are indexing a large number of logs everyday and the size of our daily index is approximately 3,500,000 documents. We are using Kibana to query ES and generate reports. Most of our reports are histograms and hence require heavy facetting. My observation was this - the dashboards took very long to load (depending on the time limit selected) and the field cache size being unlimited (by default) started rising and eventually resulted in an Out of Memory Error. I have now restricted the field cache size to 50% of the available heap memory. Although this does result in reducing this error, there is not much difference in the performance and search takes long. Another observation is that with 1 shard and 0 replicas, my ES node is not making use of the other CPU cores. I have 4 cores on my system and the CPU% shown by the top command for the elasticsearch process just barely exceeds 100%. I believe this indicates that it uses one core in entirety but not all 4 cores. Will increasing the number of shards make better use of the multi-core architecture and enable parallel search? Also, if so, what is the best way to get this working? Should I make changes in the ES configuration file and then re-start the cluster? How does this affect the currently existing indices? We create 2 indices per day. Now when I increase the number of shards for the cluster, how will the data in the previously created indices get distributed among the newly created shards? Thanks, Rujuta -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/fa490e88-87a4-4c9d-aeef-b93f51e7e7e0%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
