[jira] [Commented] (HTTPCLIENT-1246) Patch for JmxEnabledHttpClient
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487292#comment-14487292 ] Oleg Kalnichevski commented on HTTPCLIENT-1246: --- No, it did not. There is no guarantee it will ever make it into HttpClient unless someone takes this task on. Oleg Patch for JmxEnabledHttpClient -- Key: HTTPCLIENT-1246 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1246 Project: HttpComponents HttpClient Issue Type: New Feature Components: HttpClient Reporter: Mike Boyers Fix For: Future Attachments: jmx-enabled-httpclient.patch Oleg asked me to submit this patch via JIRA. This is an initial version of code to enable JMX for requests/timeouts/etc. I've uploaded the patch as an attachment. Here are my comments about what I have and have not done. This is from an email earlier in the week: What I have at this point is a version of a JmxEnabledHttpClient that implements the HttpClient interface. You can instantiate a JmxEnabledHttpClient given an HttpClient. I have decoupled it from the work I had done previously (Spring-dependent), so it works without any additional dependencies. I probably just need a couple more hours to clean this area up. I did work on it a bit yesterday. But I have not made progress in integrating the stats more closely with the HttpClient codebase. So it doesn't currently have visibility into some of the underlying behavior of HttpClient and some of the mechanisms for retaining and managing statistics could probably be done more cheaply if they were integrated more closely. This is the area I just don't think I'm going to be able to find the time to do, sadly. So what I'm thinking at this point: I'll hand off what I've done (which is functional) and you guys can decide on whether or not to use it as the basis for anything going forward. Even if you decide not to ultimately check it in with the rest of the HttpClient code, it could still be useful to you in the short-term, as it could easily be turned into its own jar and used with the existing HttpClient. I'm pretty sure it records everything we talked about previous except the current pool size. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
Re: SSL socket timeouts with SolrJ usage of HttpClient
On Thu, 2015-04-09 at 10:41 -0400, Karl Wright wrote:
Hi Oleg,
I've been looking at the tickets for potential issues with SSL socket
timeout values not being honored in some versions of HttpClient, and I must
say I'm not clear where things stand.
I have a ManifoldCF user who is seeing socket read timeouts when using
SSL. The stack in that case involves the Solr client library (SolrJ). I
am passing in an HttpClient instance that's already built:
RequestConfig.Builder requestBuilder = RequestConfig.custom()
.setCircularRedirectsAllowed(true)
.setSocketTimeout(socketTimeout)
.setStaleConnectionCheckEnabled(true)
.setExpectContinueEnabled(true)
.setConnectTimeout(connectionTimeout)
.setConnectionRequestTimeout(socketTimeout);
HttpClientBuilder clientBuilder = HttpClients.custom()
.setConnectionManager(connectionManager)
Karl,
If one explicitly assigns an already initialized connection manager
instance basically all connection manager parameters have no effect.
Please try setting default SocketConfig on connection manager directly.
Oleg
.setMaxConnTotal(1)
.disableAutomaticRetries()
.setDefaultRequestConfig(requestBuilder.build())
.setRedirectStrategy(new DefaultRedirectStrategy())
.setSSLSocketFactory(myFactory)
.setRequestExecutor(new HttpRequestExecutor(socketTimeout))
.setDefaultSocketConfig(SocketConfig.custom()
.setTcpNoDelay(true)
.setSoTimeout(socketTimeout)
.build()
);
if (userID != null userID.length() 0 password != null)
{
CredentialsProvider credentialsProvider = new
BasicCredentialsProvider();
Credentials credentials = new UsernamePasswordCredentials(userID,
password);
if (realm != null)
credentialsProvider.setCredentials(new
AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT, realm), credentials);
else
credentialsProvider.setCredentials(AuthScope.ANY, credentials);
clientBuilder.setDefaultCredentialsProvider(credentialsProvider);
}
HttpClient localClient = clientBuilder.build();
It is remotely possible that SolrJ is modifying a parameter in the client,
which I am aware would invalidate the builder-based configuration. So my
question is simple: IF the HttpClient instance is *not* being configured in
SolrJ, would you expect the socket timeout to be honored for SSL requests,
on the current codebase?
yes.
Was there ever a time when that was not true? If
No.
there's a buried default SSL socket timeout value that would be used if
configuration was overridden by setting a parameter, what is that value?
SSL handshake as well as CONNECT message exchange use socket timeout set
by the connection manager. Request level settings apply only once a
connection has been fully established and routed.
Hope this helps.
Oleg
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
Re: SSL socket timeouts with SolrJ usage of HttpClient
Hi Oleg,
Can you verify that the following code is not setting any parameter that
will have no effect because of the context in which I'm setting it? It's
not easy to determine this without looking through much code.
// Initialize standard solr-j.
// First, we need an HttpClient where basic auth is properly set up.
connectionManager = new PoolingHttpClientConnectionManager();
connectionManager.setMaxTotal(1);
connectionManager.setDefaultSocketConfig(SocketConfig.custom()
.setTcpNoDelay(true)
.setSoTimeout(socketTimeout)
.build());
SSLConnectionSocketFactory myFactory;
if (keystoreManager != null)
{
myFactory = new
SSLConnectionSocketFactory(keystoreManager.getSecureSocketFactory(),
SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
}
else
{
// Use the trust everything one
myFactory = new
SSLConnectionSocketFactory(KeystoreManagerFactory.getTrustingSecureSocketFactory(),SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
}
RequestConfig.Builder requestBuilder = RequestConfig.custom()
.setCircularRedirectsAllowed(true)
.setSocketTimeout(socketTimeout)
.setStaleConnectionCheckEnabled(true)
.setExpectContinueEnabled(true)
.setConnectTimeout(connectionTimeout)
.setConnectionRequestTimeout(socketTimeout);
HttpClientBuilder clientBuilder = HttpClients.custom()
.setConnectionManager(connectionManager)
.setMaxConnTotal(1)
.disableAutomaticRetries()
.setDefaultRequestConfig(requestBuilder.build())
.setRedirectStrategy(new DefaultRedirectStrategy())
.setSSLSocketFactory(myFactory)
.setRequestExecutor(new HttpRequestExecutor(socketTimeout));
if (userID != null userID.length() 0 password != null)
{
CredentialsProvider credentialsProvider = new
BasicCredentialsProvider();
Credentials credentials = new UsernamePasswordCredentials(userID,
password);
if (realm != null)
credentialsProvider.setCredentials(new
AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT, realm), credentials);
else
credentialsProvider.setCredentials(AuthScope.ANY, credentials);
clientBuilder.setDefaultCredentialsProvider(credentialsProvider);
}
HttpClient localClient = clientBuilder.build();
I'm also getting warnings now for setStaleConnectionCheckEnabled(true). It
looks like this is now a connection parameter too? Will all of my
setStaleConnectionCheckEnabled(true) calls now be invalid?
Karl
On Thu, Apr 9, 2015 at 3:03 PM, Oleg Kalnichevski ol...@apache.org wrote:
On Thu, 2015-04-09 at 10:41 -0400, Karl Wright wrote:
Hi Oleg,
I've been looking at the tickets for potential issues with SSL socket
timeout values not being honored in some versions of HttpClient, and I
must
say I'm not clear where things stand.
I have a ManifoldCF user who is seeing socket read timeouts when using
SSL. The stack in that case involves the Solr client library (SolrJ). I
am passing in an HttpClient instance that's already built:
RequestConfig.Builder requestBuilder = RequestConfig.custom()
.setCircularRedirectsAllowed(true)
.setSocketTimeout(socketTimeout)
.setStaleConnectionCheckEnabled(true)
.setExpectContinueEnabled(true)
.setConnectTimeout(connectionTimeout)
.setConnectionRequestTimeout(socketTimeout);
HttpClientBuilder clientBuilder = HttpClients.custom()
.setConnectionManager(connectionManager)
Karl,
If one explicitly assigns an already initialized connection manager
instance basically all connection manager parameters have no effect.
Please try setting default SocketConfig on connection manager directly.
Oleg
.setMaxConnTotal(1)
.disableAutomaticRetries()
.setDefaultRequestConfig(requestBuilder.build())
.setRedirectStrategy(new DefaultRedirectStrategy())
.setSSLSocketFactory(myFactory)
.setRequestExecutor(new HttpRequestExecutor(socketTimeout))
.setDefaultSocketConfig(SocketConfig.custom()
.setTcpNoDelay(true)
.setSoTimeout(socketTimeout)
.build()
);
if (userID != null userID.length() 0 password != null)
{
CredentialsProvider credentialsProvider = new
BasicCredentialsProvider();
Credentials credentials = new UsernamePasswordCredentials(userID,
password);
if (realm != null)
credentialsProvider.setCredentials(new
AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT, realm), credentials);
else
credentialsProvider.setCredentials(AuthScope.ANY, credentials);
clientBuilder.setDefaultCredentialsProvider(credentialsProvider);
}
HttpClient localClient = clientBuilder.build();
It is remotely possible that SolrJ is modifying a parameter in the
client,
which I am aware would invalidate the
Re: Addresses that do not Resolve an IP will cause an Error.
The DNS resolver is part of the problem creating a custom DnsResolver does not appear to resolve the issue. If the address is https://facebookcorewwwi.onion/ the server tries to resolve this and fails, no ip address exists. This results in an UnknownHostException which is the correct behavior if i wasn't using a proxy. I am using a custom socket factory that creates TOR socks5 sockets. This address is valid (for tor) and should allow it to create a connection to the server using hostname. Tor will create the connection to the server by hostname since tor addresses do not have an ip address. This is blocked however by the below line which forces the InetSocketAddress to have a null hostname, by constructor. final InetSocketAddress remoteAddress = new InetSocketAddress(address,port); Since there is no easy access to HttpClientConnectionManager it is difficult to make update this for my specific use case. I was wondering if there was a way to allow it to create the socket using the Host rather then the Address? In this case i could create a custom DnsResolver that would correct the issue. Thanks, Lucas On Wed, Apr 1, 2015 at 3:08 AM, Oleg Kalnichevski ol...@apache.org wrote: On Mon, 2015-03-30 at 15:18 -0600, Lucas Sapp wrote: Hello Everyone, I have discovered an issue when I try to reach addresses that do not resolve an IP address but are still valid and can be accessed with a SOCKS/HTTP Proxy. I am using HttpClient 4.3.4. The error seems to occur when it tries to resolve the address prior to connecting the socket. It throws an exception trying to resolve DNS in HttpClientConnectionOperator#Connect. I have verified that by making some changes it will correctly connect to the desired endpoint for SOCKS connections. I was curious to know if there is a better way to handle this, and if not, if there was any interest in including this functionality. Thanks, Lucas Custom DnsResolver might be what you want. Oleg - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
