Hi, You could get around this by using routing based on customer ID when indexing and searching. This will ensure that all documents belonging to a single customer will be located in the same shard, which means that each search for a specific customer can hit a single shard instead of all 9, which makes it scale better.
Best regards, Christian On Friday, March 20, 2015 at 4:28:55 PM UTC, Vladi Feigin wrote: > > Hello, > > Please share your thoughts > We have one big ES index and 18 shards (9 primary and 9 replicas) > We have thousands of customers and each customer could have millions or as > opposite very small number of documents > We never search across all customers but within a specific customer. In > other words all our queries have a customer id filter. > The big disadvantage of having one big index is we always search the data > of all customers rather than looking in one customer > Obviously it hurts our queries performance. > We're thinking to create multiple indexes : an index per customer. But in > our case it means having hundreds or maybe thousands indexes > In terms of the maintenance is a big overhead > Other approach is create many shards > Could you, please share your experience and thoughts? > What would you recommend in this scenario > Thank you in advance, > Vladi Feigin > > > > > > This message may contain confidential and/or privileged information. > If you are not the addressee or authorized to receive this on behalf of > the addressee you must not use, copy, disclose or take action based on this > message or any information herein. > If you have received this message in error, please advise the sender > immediately by reply email and delete this message. Thank you. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/db03ac4c-bef8-40a4-a109-d37b40c8b463%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
