Hi Shawn, Thanks again for detailed reply.
Regarding auto commit, we discussed lot with our product owners and atlast we are forced to keep it to 1sec and we couldn't increase further. As this itself, sometimes our customers says that they have to refresh their pages for couple of times to get the update from solr. So we can't increase further. We have kept openSearcher, it's as typical configuration & recommended only. Yes. As of now only solr is running in that machine. But intially we were running along with hbase region servers and was working fine. But due to CPU spikes and OS disk cache, we are forced to move solr to separate machine. But just I checked, our solr data folder size is coming only to 17GB. 2 collection has around 5GB and other are have 2 to 3 GB of size. If you say that only 2/3 of total size comes to OS disk cache, in top command VIRT property it's always 28G, which means more than what we have. Why is that... Pls check that top command & GC we used in this doc <https://docs.google.com/document/d/1SaKPbGAKEPP8bSbdvfX52gaLsYWnQfDqfmV802hWIiQ/edit?usp=sharing> Yes we are using solr cloud only. Queries are quiet fast, most of time simple queries with fq. Regarding index, during peak hours, we index around 100 documents in a second in a average. I also shared the CPU utilization in the same doc <https://docs.google.com/document/d/1SaKPbGAKEPP8bSbdvfX52gaLsYWnQfDqfmV802hWIiQ/edit#heading=h.ahsgapiu4829> Regarding release, initially we tried with 6.4.1 and since many discussions over here, mentioned like moving to 6.5.x will solve lot of performance issues etc, so we moved to 6.5.1. We will move to 6.6.3 in near future. Hope I have given enough information. One strange thing is that, CPU and memory spike are not seen when we move to r4.xlarge to r4.2xlarge ( which is 8 core with 60 GB RAM ). But this would not be cost effective. What's making CPU and memory to go high in this new version ( due to doc values )? If I switch off docvalues will CPU & Memory spikes will get reduced ? Let me know... -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
