Because each Elasticsearch workload and hardware plus software is different, you should not expect a serious documentation can always give "one-size-fits-all" advisory. There are many "rules of thumb" and best practices, mostly based on individual experience on individual test cases. The foundation may change in the future, because of other Java version, because of other hardware specs, or on other Elasticsearch versions etc.
For example, "shards should not be bigger than the heap size" - this might be useful in situations where massive indexing is going on for that shard so that segment merging can proceed smoothly, or when you must recover from broken shards. Beside this, it is also not wrong to have huge shards. If you have a finalized index, never recover from replica, or never have to move a shard around, you will not even notice how large your shards are. It always depends on your requirements. Jörg On Fri, May 9, 2014 at 1:26 PM, Jasper Siero <[email protected] > wrote: > We are investigating stability issues we encounter in our ES cluster, and > feel that the ES documentation is lacking import information on this topic. > > The mailing list seems to fill the gap with the "Document about recommended > hardware specs & sharding/index strategy" > > http://elasticsearch-users.115913.n3.nabble.com/Recommended-Hardware-Specs-amp-Sharding-Index-Strate. > .. > > The information in this conversation looks important (for example shards > should not be bigger then the heap size and the heap size shoud be half of > the total memory) but we haven’t found this information in the > documentation. We have two questions: > * Did we overlook this in the official documentation? > * If the official documentation is lacking this important information, what > should we do to make this taken care of? > > Information about our cluster: > > 4 node Elasticsearch cluster > 6 indices in use > Number of active shards 114 > version 0.90.3 > > JVM: > VM name: Java HotSpot(TM) 64-Bit Server VM > VM vendor: Oracle Corporation > VM version: 23.25-b01 > Java version: 1.7.0_25 > > OS: > virtual machine > CentOS 6.5 > 20 GB memory > > CPU: > CPU vendor: Intel > CPU model: Common KVM processor (2099 MHz) > CPU total logical cores: 2 > CPU cache: 4kb > > Cluster-wide storage size: 140 GB > > > > > > -- > View this message in context: > http://elasticsearch-users.115913.n3.nabble.com/Stability-issues-due-to-documention-flaw-ElasticSearch-tp4055631.html > Sent from the ElasticSearch Users mailing list archive at Nabble.com. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/1399634790470-4055631.post%40n3.nabble.com > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGVpWdYuhCD%3DzVoTEJEPxXq-zG8ViFHmm_Qr%3DdAMVJccA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
