Hi,
We have solr cluster with 2 shards running 2 nodes on each shard. They are beefy physical boxes with index size of 162 GB , RAM of about 96 GB and around 153M documents. Two times this week we have seen the thread usage spike from the usual 1000 to 4000 on all nodes at the same time and bring down the cluster. We had to divert the traffic(search and update), perform a rolling restart each time, and put them back in. Has anyone faced this issue before? We don't have any other process running on the box that could cause such a huge spike in thread usage on all nodes at the same time. Any pointers appreciated. Thanks A