I know this has been discussed in the past (although not too recently), but the advices in those have failed us, so here we are.
Some basics: We're running Solr 6 (6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - nknize - 2016-04-01 14:41:49), on Java 8 (OpenJDK Runtime Environment (build 1.8.0_72-internal-b15)), on at this point some rather large cloud instances (8 cpu / 40gb ram). Our general cluster layout is 3 nodes per shard, 6 shards, we have three collections, but one primary collection which is heavily used and is resulting in the GC situation we're seeing. There are roughly 55m documents in this collection. Our test load is multiple large, complicated queries which facet across multiple fields. After trying many of the off the shelf configurations (including CMS configurations but excluding G1GC, which we're still taking the warnings about seriously), numerous tweaks, rumors, various instance sizes, and all the rest, most of which regardless of heap size and newspace size resulted in frequent 30+ second STW GCs, we settled on the following configuration which leads to occasional high GCs but mostly stays between 10-20 second STWs every few minutes (which is almost acceptable): -XX:+AggressiveOpts -XX:+UnlockDiagnosticVMOptions -XX:+UseAdaptiveSizePolicy -XX:+UseLargePages -XX:+UseParallelGC -XX:+UseParallelOldGC -XX:MaxGCPauseMillis=15000 -XX:MaxNewSize=12000m -XX:ParGCCardsPerStrideChunk=4096 -XX:ParallelGCThreads=16 -Xms31000m -Xmx31000m Note that HugeTable is working on the instances, and allocates approximately the size of the java instance, and Java doesn't produce the error that indicates that the HugeTable didn't work - getting this working did provide a marginal improvement in performance. Mostly we're wondering if there's something we missed something in the configuration, and if anyone has experienced something similar! Thanks for any help! -- Cas Rusnov, Engineer [image: Manzama Logo] <http://www.manzama.com> Visit our Resource Center <http://www.manzama.com/resource-center/>. US & Canada Office: +1 (541) 306-3271 <+15413063271> | UK Office: +44 (0)203 282 1633 <+4402032821633> | AUS Office: +61 02 9326 6264 <+610293266264> LinkedIn <http://www.linkedin.com/company/manzama>| Twitter <https://twitter.com/ManzamaInc>| Facebook <http://www.facebook.com/manzamainc>| Google + <https://plus.google.com/u/0/b/116326385357563344293/+ManzamaInc/about>| YouTube <https://www.youtube.com/channel/UCgbgt-xWBTxrbQESTVeMMHw>| Pinterest <https://www.pinterest.com/manzama1754/>
