For what it’s worth, I’d suggest you go into a conversation with Azul with a more explicit “I’m looking to buy” approach. I reached out to them with a more “I’m exploring my options” attitude, and never even got a trial. I get the impression their business model involves a fairly expensive (to them) trial process, so they’re looking for more urgency on the part of the client than I was expressing.
Instead, I spent a few weeks analyzing how my specific index allocated memory. This turned out to be quite worthwhile. Armed with that information, I was able to file a few patches (coming in 6.1, perhaps?) that reduced allocations by a pretty decent amount on large indexes. (SOLR-8922, particularly) It also straight-up ruled out certain things Solr supports, because the allocations were just too heavy. (SOLR-9125) I suppose the next thing I’m considering is using multiple JVMs per host, essentially one per shard. This wouldn’t change the allocation rate, but does serve to reduce the worst-case GC pause, since each JVM can have a smaller heap. I’d be trading a little p50 latency for some p90 latency reduction, I’d expect. Of course, that adds a bunch of headache to managing replica locations too. On 6/2/16, 6:30 PM, "Phillip Peleshok" <ppeles...@gmail.com> wrote: >Fantastic! I'm sorry I couldn't find that JIRA before and for getting you >to track it down. > >Yup, I noticed that for the docvalues with the ordinal map and I'm >definitely leveraging all that but I'm hitting the terms limit now and that >ends up pushing me over. I'll see about giving Zing/Azul a try. From all >my readings using theUnsafe seemed a little sketchy ( >http://mishadoff.com/blog/java-magic-part-4-sun-dot-misc-dot-unsafe/) so >I'm glad that seemed to be the point of contention bringing it in and not >anything else. > >Thank you very much for the info, >Phil > >On Thu, Jun 2, 2016 at 6:14 PM, Erick Erickson <erickerick...@gmail.com> >wrote: > >> Basically it never reached consensus, see the discussion at: >> https://issues.apache.org/jira/browse/SOLR-6638 >> >> If you can afford it I've seen people with very good results >> using Zing/Azul, but that can be expensive. >> >> DocValues can help for fields you facet and sort on, >> those essentially move memory into the OS >> cache. >> >> But memory is an ongoing struggle I'm afraid. >> >> Best, >> Erick >> >> On Wed, Jun 1, 2016 at 12:34 PM, Phillip Peleshok <ppeles...@gmail.com> >> wrote: >> > Hey everyone, >> > >> > I've been using Solr for some time now and running into GC issues as most >> > others have. Now I've exhausted all the traditional GC settings >> > recommended by various individuals (ie Shawn Heisey, etc) but neither >> > proved sufficient. The one solution that I've seen that proved useful is >> > Heliosearch and the off-heap implementation. >> > >> > My question is this, why wasn't the off-heap FieldCache implementation ( >> > http://yonik.com/hs-solr-off-heap-fieldcache-performance/) ever rolled >> into >> > Solr when the other HelioSearch improvement were merged? Was there a >> > fundamental design problem or just a matter of time/testing that would be >> > incurred by the move? >> > >> > Thanks, >> > Phil >>
