Hi, Currently we have index of size 3.5 TB. These index are distributed across 12 shards under two cores. The size of index on each shards are almost equal. We do a delta indexing every week and optimise the index.
The server configuration is as follows. - Solr Version : 6.5.1 - AWS instance type : r5a.16xlarge - CPU(s) : 64 - RAM : 512GB - EBS size : 7 TB (For indexing as well as index optimisation.) - IOPs : 30000 (For faster index optimisation) Can you please help me with following few questions? - What is the ideal index size per shard? - The optimisation takes lot of time and IOPs to complete. Will increasing the number of shards help in reducing the optimisation time and IOPs? - We are planning to reduce each shard index size to 30GB and the entire 3.5 TB index will be distributed across more shards. In this case to almost 70+ shards. Will this help? - Will adding so many new shards increase the search response time and possibly how much? - If we have to increase the shards should we do it on a single larger server or should do it on multiple small servers? Kindly share your thoughts on how best we can use Solr with such a large index size. Best, Modassar
