Hi,
I'm trying to get a multi-threaded implementation of HttpClient.
Basically i have a number of threads that call getHttpData()
concurrently. It seems to work, but upon starting my threads I get a
bunch of exceptions:
Dec 14, 2010 11:32:17 AM
org.apache.http.impl.client.DefaultRequestDirector execute
INFO: I/O exception (java.net.SocketException) caught when processing
request: Connection reset
Dec 14, 2010 11:32:17 AM
org.apache.http.impl.client.DefaultRequestDirector execute
INFO: Retrying request
After this the threads seem to actually retry and succeed in
establishing a connection, but I'd like to find out what's going wrong
here.
It doesn't happen on every run, that's what makes this difficult to
track down. The program is a server, sometimes I can make dozens of
calls without hitting that exception, then suddenly it will display
errors for a large number of requests -- all on one server instance.
My program works like this:
1 call has about 40 sources.
for(String source : sourceList){
SearchThread searchThread = new SearchThread(name, source, id++);
Thread thread = new Thread(searchThread);
thread.start();
}
SearchThread is a simple Runnable that does data =
HttpClientWrapper.getHttpData(url); in its run() body.
Can anyone shed a light on this? Thanks in advance.
Below is the full code for HttpClientWrapper:
=============================
import java.io.BufferedReader;
public class HttpClientWrapper {
private static HttpClient httpClient;
private static HttpClient createHttpClient(){
HttpParams params = new BasicHttpParams();
HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
HttpProtocolParams.setContentCharset(params,
HTTP.DEFAULT_CONTENT_CHARSET);
HttpProtocolParams.setUseExpectContinue(params, true);
HttpConnectionParams.setStaleCheckingEnabled(params, false);
HttpClientParams.setCookiePolicy(params, CookiePolicy.NETSCAPE);
HttpConnectionParams.setConnectionTimeout(params, 15 * 1000);
HttpConnectionParams.setSoTimeout(params, 15 * 1000);
HttpConnectionParams.setSocketBufferSize(params, 8192);
params.setParameter(ConnManagerPNames.MAX_TOTAL_CONNECTIONS, 100);
params.setParameter(ConnManagerPNames.MAX_CONNECTIONS_PER_ROUTE, new
ConnPerRouteBean(100));
SchemeRegistry schReg = new SchemeRegistry();
schReg.register(new Scheme("http",
PlainSocketFactory.getSocketFactory(), 80));
schReg.register(new Scheme("https",
SSLSocketFactory.getSocketFactory(), 443));
ClientConnectionManager conMgr = new
ThreadSafeClientConnManager(params, schReg);
return new DefaultHttpClient(conMgr, params);
}
public static HttpClient getHttpClient(){
if(httpClient == null){
httpClient = createHttpClient();
}
return httpClient;
}
public void shutdown(){
if(httpClient != null &&
httpClient.getConnectionManager() !=null){
httpClient.getConnectionManager().shutdown();
}
}
public static String getHttpData(String url){
HttpGet request;
try {
HttpClient httpClient = getHttpClient();
request = new HttpGet();
request.setURI(new URI(url));
HttpResponse response = httpClient.execute(request);
InputStream input = response.getEntity().getContent();
BufferedReader reader = new BufferedReader(new
InputStreamReader(input), 8192);
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
request.abort();
return sb.toString();
} catch (Exception e){
e.printStackTrace();
}
return null;
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
