Hi Oleg,
Thanks for the valuable information.
I would love to create a testcase for it.
I'm already a commiter of Apache CXF, can I check in the code directly
into the httpclient? or provide a patch?
Thanks,
James
On Tue, 2008-06-03 at 21:42 +0800, maomaode wrote:
Hi Oleg,
Thanks for the quick response, I'm dev an intranet web
crawling/indexing, previous I used the httpclient 3-x.
we use the nio feature in the new httpclient to increase the performance.
Please note NIO is actually _significantly_ slow compared to classic I/O
_unless_ you are dealing with thousands of simultaneous high latency
connections. If the number of simultaneous connections is less than 500
NIO is almost certainly bound to perform worse than the classic blocking
I/O.
I don't know how much of simultaneous conn httpclient-nio can handle?
for me, I hope as much as the system can handle.
Really a lot. BUT every connection usually takes from 100 to 150KB, so
you have to ensure there is enough memory of the heap to support a great
number of simultaneous connections.
I don't know the max length of the content in our system, I guess 300KB max?
Since the ThrottlingHttpClientHandler can use the ThreadPool, I guess
the workerCount in the DefaultConnectingIOReactor will take NO effect,
is that right?
Pretty much. But I would still recommend to use the same number of I/O
dispatch workers as CPU cores on the system to ensure better CPU
utilization.
Could you please create a test app that reproduces the problem you have
been seeing and post it to this list or to the dev list?
Cheers
Oleg
Thanks,
James
http://people.apache.org/~mmao
On Tue, 2008-06-03 at 20:48 +0800, maomaode wrote:
Hi,
I'm hitting a Out Of Memory error when using ThrottlingHttpClientHandler
<http://hc.apache.org/httpcomponents-core/httpcore-nio/apidocs/org/apache/http/nio/protocol/ThrottlingHttpClientHandler.html>
I gave a Executors.|*newCachedThreadPool
<cid:[email protected]>*(), should I give a fixed thread
pool?
James,
What is it exactly you are trying to do? How many simultaneous
connections are you trying to open? What is the size of the content
buffer? it is 20480 (default)?
And BTW, what's the workerCount means in the |DefaultConnectingIOReactor?
The number of threads I/O reactor uses to dispatch I/O events to the
protocol handlers. This number should be equal to that of CPU cores of
the system.
Hope this helps
Oleg
Thanks in advance,
James
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
