On 2/5/2018 12:29 PM, mmb1234 wrote:
> Maybe this is the issue: https://github.com/eclipse/jetty.project/issues/2169
Could be. That was one of the Jetty issues I found, and the one that
said 9.4 doesn't have the problem. Your Solr version would be using
Jetty 9.3, not 9.2.
> The custom client code is:
I don't know a lot about all the code you shared. But this caught my eye:
> idleConnectionFuture = executor.scheduleAtFixedRate(() -> {
> cmAsync.closeExpiredConnections();
> cmAsync.closeIdleConnections(1, TimeUnit.SECONDS);
> }, 1, 1, TimeUnit.SECONDS);
You're really going to close connections that have been idle for only
one second? A lot of Solr requests are going to need more than one
second to complete. As I said in the earlier message, I very much
recommend a socket/idle timeout of at least one minute. Make it 30
seconds if one minute seems too long.
> Also /solr/admin/cores takes a very long time to respond (QTime = 300secs+).
> So curl with a timeout also causes an additional CLOSE_WAIT (as expected)
> curl -m 5 'http://<solrhost>:<port>/solr/admin/cores'
It literally takes five minutes for /solr/admin/cores to respond on your
server? That means something is REALLY REALLY wrong. When I hit it on
one of my servers, which has a few dozen cores, the QTime is 39
milliseconds, and I see the response in the browser very quickly.
> Note there are 100's of solr cores on each solr jvm but only few needed to
> be open at any given time in each solr jvm to avoid heap memory bloat.
> solr.xml has this setting therefore because we're not running solr cloud.
> <int name="transientCacheSize">${transientCacheSize:30}</int>
Loading and unloading large cores on a frequent basis is going to make
operations pretty slow. If your index data is too large to have loaded
all at once, perhaps you need more memory or more servers.
> Also <updateLog> was removed from solrconfig.xml because we saw 1000's of
> threads BLOCKED on VersionBucket even with.
> <int
> name="numVersionBuckets">${solr.ulog.numVersionBuckets:655360}</int>
>
> The side effect is lots of merges [ we'll tackle that when solr stops
> die-ing :-) ]
The presence, absence, or configuration of Solr's transaction log is not
going to affect segment merging in any way. Segment merging is a
function of Lucene, not Solr, and Lucene has absolutely no knowledge of
Solr features like the transaction log. I would strongly recommend that
you do NOT disable the transaction log. Doing so might speed things up
a very small amount, but that speedup is NOT a good tradeoff,
considering that you'd be more likely to lose data in the event of a
serious problem.
Lots of segment merges is a result of doing a lot of indexing.
Thanks,
Shawn
