Part of it is based on knowledge picked up from our customers, part of it is that once you have to start shifting files larger than this around (during reallocation or recovery) it can take excessive time.
There is also a ~2 billion hard limit for documents in a single shard, which is a lucene limit, so you reduce your exposure there too. On 18 March 2015 at 10:29, Andrej Rosenheinrich < [email protected]> wrote: > Hi Mark, > > may I ask what the reason for this recommendation is? > > Thanks, > Andrej > > Am Mittwoch, 18. März 2015 17:50:09 UTC+1 schrieb Mark Walkom: >> >> We recommend shards no larger than 50GB, but as you mention there is no >> exact limit. >> >> On 18 March 2015 at 04:09, Georgi Ivanov <[email protected]> wrote: >> >>> My rules is : 1 primary shard per server. >>> >>> Also make some estimation how big will be the single index/shard >>> >>> I think it is not good if single shard exceed 10 GB, although there is >>> no exact limit. >>> >>> >>> Georgi >>> >>> On Tuesday, March 17, 2015 at 7:00:23 PM UTC+1, John S wrote: >>>> >>>> Hi All, >>>> >>>> Is there any best practices of having on the number of shards for a >>>> cluster? I have a 4 node cluster and used shards of 20. >>>> >>>> During any node failure or other events i doubts since the shards >>>> number is high, replication to new node is taking more time... >>>> >>>> Is there any metrics or formula to be done for number or shards? >>>> >>>> Regards >>>> John >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elasticsearch" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit https://groups.google.com/d/ >>> msgid/elasticsearch/36ef3ed0-870f-41a5-915b-fb3ad919f7a0% >>> 40googlegroups.com >>> <https://groups.google.com/d/msgid/elasticsearch/36ef3ed0-870f-41a5-915b-fb3ad919f7a0%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/385c959b-0132-4d9d-95cd-b4b08fca9c94%40googlegroups.com > <https://groups.google.com/d/msgid/elasticsearch/385c959b-0132-4d9d-95cd-b4b08fca9c94%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8ajDqiyn04JXqH9RcBU8vaOLKZkEdFRYnNWFiV0ihw9Q%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
