I'm using DefaultHttpAsyncClient to connect to approximately 100 remote hosts.
What I'm seeing through Wireshark is that only 19 or 20 requests are made
immediately. As those 20 finish, then the rest of the requests in the queue are
processed gradually.
I want DefaultHttyAsyncClient to be able to initiate all 100 immediately
because these requests remain idle for a very long time, like 20 seconds each)
Looking into the source, I can see PoolingAsyncClientConnectionManager uses
HttpNIOConnPool which the underlying base class shows a max of 20. The
pool.setMaxTotal() I would expect to fix that does not seem to have any effect
when I run it.
Here is a snippet of the code:
DefaultConnectingIOReactor defaultioreactor = new
DefaultConnectingIOReactor();
PoolingAsyncClientConnectionManager pool = new
PoolingAsyncClientConnectionManager(defaultioreactor);
pool.setMaxTotal(200); // This I would expect to enable the
DefaultHttpAsyncClient to immediately connect up to 200 socket channels.
DefaultHttpAsyncClient defaultHttpClient = new
DefaultHttpAsyncClient(pool);
httpClient = defaultHttpClient;
httpClient.getParams()
.setIntParameter(CoreConnectionPNames.SO_TIMEOUT, 60000)
.setIntParameter(CoreConnectionPNames.CONNECTION_TIMEOUT,
60000)
.setBooleanParameter(CoreProtocolPNames.USE_EXPECT_CONTINUE,
true)
.setIntParameter(CoreProtocolPNames.WAIT_FOR_CONTINUE, 500);
defaultHttpClient.getCredentialsProvider().setCredentials(
AuthScope.ANY,
new UsernamePasswordCredentials("user", "password"));
httpClient.start();
targets = ...; //targets is a Map<String,
HttpHost>
CountDownLatch latch = new CountDownLatch(targets.size()); // size
will be approx. 100.
for (Map.Entry<String,HttpHost> SBB : targets.entrySet()) {
String name = SBB.getKey();
HttpHost target = SBB.getValue();
HttpRequestBase targetRequest = // function to create
HttpGet();
SHEFFutureCallback futresponse = // new future callback
object.
httpClient.execute(target, targetRequest, futresponse);
}
And for context, I'm running it on a Windows XP Intel Core2 Duo machine with a
Gigabit NIC.
----------------------------------------
Drew Chen
22516 Gateway Center Dr.
Suite 100
Clarksburg, MD 20871
[email protected]<mailto:[email protected]>
Office: 301-444-2993
Fax: 301-444-2998
