Unfortunately, there is no way that we can tell you an optimal number. But there is a way that you can perform some capacity tests, and arrive at usable numbers that you can extrapolate from. The process is very simple:
- Create a single index, with a single shard, on a single production-style machine - Start indexing *real, production-style *data. "Fake" or "dummy" data won't work here, it needs to mimic real-world data - Periodically, run real-world queries that you would expect users to enter - At some point, you'll find that performance is no longer acceptable to you. Perhaps the indexing rate becomes too slow. Or perhaps query latency is too slow. Or perhaps your node just runs out of memory - Write down the number of documents in the shard, and the physical size of the shard Now you know the limit of a single shard given your hardware + queries + data. Using that knowledge, you can extrapolate given your expected search/indexing load, and how many documents you expect to index over the next few years, etc. -Zach On Thursday, March 20, 2014 3:29:47 PM UTC-5, Rajan Bhatt wrote: > > Hello, > > I would appreciate if someone can suggest optimal number of shards per ES > node for optimal performance or any recommended way to arrive at number of > shards given number of core and memory foot print. > > Thanks in advance > Reagards > Rajan > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/94bd5180-1198-4cfd-9b3d-f532d3fea5d2%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
