[
https://issues.apache.org/jira/browse/SOLR-4509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225183#comment-14225183
]
Ryan Zezeski commented on SOLR-4509:
------------------------------------
> Ryan Zezeski, what drove your decision to make the sweeper thread interval 5
> seconds (considering what the default should be)?
It had to be shorter than the server connection timeout; 5 seconds was
less than the 60 second server timeout I had.
It seems you are running with this patch nicely. Feel free to modify
it however you see fit.
As for the latency differences that Shawn mentioned, keep in mind I
was running microbenchmarks. It was a tight loop of small queries
running at max throughput and everything was a cache hit. IIRC, faster
CPUs showed less of a gain. Part of this is explained because it
removes at least one syscall for every request. Faster, more modern,
CPUs should context switch more efficiently. If I was doing this
benchmark over again today I would try other variations (low-rate to
avoid any CPU run-queue buildup and focus on latency only) and examine
more performance metrics.
That aside, this patch should avoid the case where a stale conn is
chosen and a new one has to be created as part of the request which
will result in latency outliers. A win in my book.
> Disable HttpClient stale check for performance and fewer spurious connection
> errors.
> ------------------------------------------------------------------------------------
>
> Key: SOLR-4509
> URL: https://issues.apache.org/jira/browse/SOLR-4509
> Project: Solr
> Issue Type: Improvement
> Components: search
> Environment: 5 node SmartOS cluster (all nodes living in same global
> zone - i.e. same physical machine)
> Reporter: Ryan Zezeski
> Assignee: Mark Miller
> Priority: Minor
> Fix For: 5.0, Trunk
>
> Attachments: IsStaleTime.java, SOLR-4509-4_4_0.patch,
> SOLR-4509.patch, SOLR-4509.patch, SOLR-4509.patch, SOLR-4509.patch,
> baremetal-stale-nostale-med-latency.dat,
> baremetal-stale-nostale-med-latency.svg,
> baremetal-stale-nostale-throughput.dat, baremetal-stale-nostale-throughput.svg
>
>
> By disabling the Apache HTTP Client stale check I've witnessed a 2-4x
> increase in throughput and reduction of over 100ms. This patch was made in
> the context of a project I'm leading, called Yokozuna, which relies on
> distributed search.
> Here's the patch on Yokozuna: https://github.com/rzezeski/yokozuna/pull/26
> Here's a write-up I did on my findings:
> http://www.zinascii.com/2013/solr-distributed-search-and-the-stale-check.html
> I'm happy to answer any questions or make changes to the patch to make it
> acceptable.
> ReviewBoard: https://reviews.apache.org/r/28393/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
