Hi Shawn,
It looks like you probably have pointed to the root cause of the issue.
I am using a java client and using HttpClient library directly to fire the
Http get queries. I am not using SolrJ client for firing the queries.
The following is my code
CloseableHttpClient httpclient = HttpClients.createDefault();
HttpGet httpGet = new HttpGet(url);
CloseableHttpResponse response = null;
try {
response = httpclient.execute(httpGet);
} finally {
}
Is a route per httpHost/port combination?
I will try to set the MaxConnPerRoute and maxTotalConn to a larger value
and see if that gives me some benefit.
Thanks
Suresh
On 3/24/17 6:05 AM, "Shawn Heisey" <apa...@elyograg.org> wrote:
>On 3/23/2017 6:10 PM, Suresh Pendap wrote:
>> I performed the test with 1 thread, 10 client threads and 50 client
>> threads. I noticed that as I increased the number of threads, the
>> query latency kept increasing drastically which I was not expecting.
>
>What language and Solr library was the client using? If it's Java and
>SolrJ, then the following will apply. If the client is written in a
>language other than Java, you may find that there are similar default
>settings in the HTTP library:
>
>A dependency called HttpClient is used by SolrJ. The default settings
>for HttpClient are only capable of making *two* simultaneous connections
>to a target server. Further connections will wait until existing
>connections are complete. Unless it is overridden, SolrJ creates the
>HttpClient object with default settings.
>
>If more threads are needed, the SolrClient object must be built with a
>custom HttpClient. Here's some SolrJ code to create a
>multithread-capable client object (300 threads to a single server):
>
> RequestConfig rc = RequestConfig.custom().setConnectTimeout(15000)
> .setSocketTimeout(Const.SOCKET_TIMEOUT).build();
> httpClient = HttpClients.custom().setDefaultRequestConfig(rc)
>
>.setMaxConnPerRoute(300).setMaxConnTotal(5000).disableAutomaticRetries()
> .build();
> client = new HttpSolrClient(serverBaseUrl, httpClient);
>
>I have also placed this code at the following URL. It will expire in
>one month:
>
>http://apaste.info/BpoWY
>
>A similar technique can be used with CloudSolrClient if needed.
>
>It's my opinion that SolrJ needs to create client objects by default
>that are capable of more threads, but I have not yet put any time into
>making it happen.
>
>Thanks,
>Shawn
>
