On 2/16/14 8:23 PM, "Shawn Heisey" <s...@elyograg.org> wrote:
>On 2/14/2014 2:15 PM, Kiran Chitturi wrote:
>> I am using http client 4.3.2 and I have a few questions.
>>
>> 1) I am making lot of concurrent requests to a server. When creating
>>requests, I want to debug and check which ports are newly created by the
>>client. Currently, I can see the ports in netstat as pasted below.
>>
>> tcp4 0 0 172.16.11.138.7575 172.16.11.138.52014
>>ESTABLISHED
>>
>> I enabled the debug logging for MainClientExec class and it gives me
>>the below information. Here it does not exactly tell me what is the
>>source port that is being used for the connection. How can I use this ?
>>Also, whenever MainClientExec prints the below line, does it mean a new
>>connection is created with a new port or an existing port is being used ?
>>
>> 2014-02-14 13:01:01 DEBUG MainClientExec:217 - Opening connection
>>{}->http://172.16.11.138:8983
>>
>> 2) My code looks like this for creating a http Client. I pass the
>>client to
>>SolrJ<https://cwiki.apache.org/confluence/display/solr/Using+SolrJ>
>>which uses this client to make requests. SolrJ makes requests to a Solr
>>server.
>>
>> HttpClientBuilder httpBuilder = HttpClientBuilder.create();
>> Builder socketConfig = SocketConfig.custom();
>> socketConfig.setSoReuseAddress(true);
>> httpBuilder.setDefaultSocketConfig(socketConfig.build());
>> httpBuilder.setMaxConnTotal(100);
>> httpBuilder.setMaxConnPerRoute(100);
>> httpBuilder.disableRedirectHandling();
>> httpBuilder.useSystemProperties();
>> final CloseableHttpClient httpClient = httpBuilder.build();
>>
>> This httpClient created above also throws some exceptions. Can I assume
>>retry is successful and does this happen because I enabled
>>socketReuseAddress ?
>>
>> 2014-02-14 12:53:21,415 - INFO [CloudSolrServer
>>ThreadPool-1-thread-59:RetryExec@93] - I/O exception
>>(java.net.SocketException) caught when processing request: Address
>>already in use
>> 2014-02-14 12:53:21,443 - INFO [CloudSolrServer
>>ThreadPool-1-thread-59:RetryExec@106] - Retrying request
>
>When it comes to the HttpClient side of this, I don't have much idea
>what's going on ... but I do have a little bit of experience with SolrJ.
>
>I have run into a lot of problems when trying to fully migrate Solr to
>HttpClient 4.3, most of which are likely a result of my own
>inexperience. Most of SolrJ's interaction with HttpClient is now
>deprecated.
>
>https://issues.apache.org/jira/browse/SOLR-5604
>
>For my own SolrJ code, I couldn't get it to work when I used
>HttpClientBuilder to make the HttpClient object for SolrJ. I don't
>remember what the exceptions looked like. What I ended up doing to
>avoid deprecations in my own code was using the utilities built into
>SolrJ. I basically kicked the problem upstream. Here's how I build an
>HttpClient in my own SolrJ code now, one that is shared between all my
>HttpSolrServer instances:
>
>ModifiableSolrParams params = new ModifiableSolrParams();
>params.add(HttpClientUtil.PROP_MAX_CONNECTIONS_PER_HOST, "300");
>params.add(HttpClientUtil.PROP_MAX_CONNECTIONS, "5000");
>httpClient = HttpClientUtil.createClient(params);
>
>https://lucene.apache.org/solr/4_6_0/solr-solrj/org/apache/solr/client/sol
>rj/impl/HttpClientUtil.html
>
>Most of the options you have above in your code are available in the
>Solr utility class. Note that the object built under all these layers
>is currently the deprecated SystemDefaultHttpClient class, which is
>modifiable after it is created. The new objects in HttpClient 4.3 are
>NOT modifiable after creation.
Shawn,
Thank you so much for your reply. If I use the 'SystemDefaultHttpClient'
for creating http client, there are many options for configuring the http
client like socket reuse, disabling stale check, etc.. that are not
possible to configure through SolrJ. May be after SOLR-5604, there will be
more options through SolrJ for configuring or one could pass their own
http client with the configured parameters.
If I use Http client from 4.3.x series, do you think there is code inside
SolrJ that would make http client (4.3.2) incompatible with SolrJ?
Currently, my code looks like this:
HttpClientBuilder httpBuilder = HttpClientBuilder.create();
Builder socketConfig = SocketConfig.custom();
socketConfig.setSoReuseAddress(true);
socketConfig.setSoTimeout(10000);
httpBuilder.setDefaultSocketConfig(socketConfig.build());
httpBuilder.setMaxConnTotal(300);
httpBuilder.setMaxConnPerRoute(100);
httpBuilder.disableRedirectHandling();
httpBuilder.useSystemProperties();
LBHttpSolrServer lb = new LBHttpSolrServer(httpClient, parser)
CloudSolrServer server = new CloudSolrServer(zkConnect, lb);
I also faced the issue of stale connections (TIME_WAIT) when using 4.2.3
but not with 4.3.2 (may be because I enabled socket reuse). So far, I
haven't seen any exceptions with 4.3.x when using CSS but I will keep a
note on them so that I can switch to 4.2.3 if 4.3.2 proves to be a PIA
with SolrJ. Please share with me any issues you encounter if you use 4.3.x
client at any point of time. I will keep testing my use of CSS with above
code.
Thanks again,
Kiran.
>
>There's still a lot of work left to do for SOLR-5604. Until that work
>is done, I don't think you'll have much luck using the new classes
>introduced by HttpClient 4.3.x.
>
>Thanks,
>Shawn
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
>For additional commands, e-mail: httpclient-users-h...@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org
