I think I would start reading this: http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/kagillion-shards.html <http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/kagillion-shards.html> This http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/user-based.html <http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/user-based.html> and this http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/faking-it.html <http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/faking-it.html>
Actually the full chapter: http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/scale.html <http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/scale.html> :) HTH -- David Pilato | Technical Advocate | Elasticsearch.com @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasticsearchfr> | @scrutmydocs <https://twitter.com/scrutmydocs> > Le 14 janv. 2015 à 02:04, 'Cindy' via elasticsearch > <[email protected]> a écrit : > > Hello > > We are using some other search engine and consider moving to use > Elasticsearch. After done quite a lot reading, I am still not quite sure what > the optimized way should be in our case, especially after I read that the > number of shards can NOT be changed once the index is created. > > In our situation, our product is hosted in cloud environment and has rapid > growing number of users, and each user is given various disk space(several > gigabytes to hundreds gigabytes) to import their datasets. We index these > datasets with fixed number of fields and the fields are all the same for some > purpose. Each user can only search in their own imported datasets for > security reason (segregated). So there is no need to query against the entire > index and query time is much more important than indexing time. Our current > query time is about 10 to 40 ms. > > It's very crucial for us how to scale out horizontally smoothly. > > If everything is added into one index with one type, I worried the > index/search will be getting slower and slower with growing of the size of > the indices. > > So I plan to split the indices to speed up query, and here are some options > Use one index and create a type for each user such that the query from one > user is directly against his own type. But since the number of users can be > over million, can elasticsearch be able to handle million types in one index? > Group users into different indices such that the index/query can be > dispatched to different indices, so a smaller index to query from. But this > means our application has to handle the complexity of horizontal scale out. > > Is any option doable? Any option would you recommend? > > Besides, could you please tell me how many shards one index should have in > best practice? Does too many shards also have performance hit? > > Many thanks, > Cindy > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/a2adcd16-1c7b-4e78-a131-d9ae4d61379b%40googlegroups.com > > <https://groups.google.com/d/msgid/elasticsearch/a2adcd16-1c7b-4e78-a131-d9ae4d61379b%40googlegroups.com?utm_medium=email&utm_source=footer>. > For more options, visit https://groups.google.com/d/optout > <https://groups.google.com/d/optout>. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/28CA132C-0158-430C-97D4-F3E5BBEE162D%40pilato.fr. For more options, visit https://groups.google.com/d/optout.
