On Tue, 2015-11-03 at 11:09 +0530, Modassar Ather wrote: > It is around 90GB of index (around 8 million documents) on one shard and > there are 12 such shards. As per my understanding the sharding is required > for this case. Please help me understand if it is not required.
Except for an internal limit of 2 billion documents/shard (or 2 billion unique values in a field in a single shard), there are no requirements as such. Our shards are 900GB / 200M+ documents and works well for our use case, but it all depends on what you are doing. Your heaps are quite large already, so merging into a single shard would probably require a heap so large that your would run into trouble with garbage collection. Your problem seems to be query processing speed. If your machine is not maxed out by many concurrent requests, sharding should help you there: As you have noticed, it allows the search to take advantage of multiple processors. - Toke Eskildsen, State and University Library, Denmark
