ok, I got it. this stale check is fixed by adding retry if connection is closed by server after response is sent
On Thu, Jul 4, 2013 at 12:41 PM, Oleg Kalnichevski <[email protected]> wrote: > On Thu, 2013-07-04 at 12:22 +0100, Ke Ren wrote: > > I still get NoHttpResponseException after I turned off keepAlive and > stale > > check. Do you mean closing connection every time after a request? > > Absolutely not. In some cases proactive eviction of idle connections > from the connection pool makes stale connection check unnecessary. For > details please refer to > > > http://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html#d5e652 > > > > I feel it > > is more expensive. Besides, the second call was sent now immediately > after > > the first call was replied in milliseconds. I can't see any long idle in > > the gap. > > > > > > Post a wire / context log of the session > > http://hc.apache.org/httpcomponents-client-ga/logging.html > > Oleg > > > On Thu, Jul 4, 2013 at 11:53 AM, Ke Ren <[email protected]> wrote: > > > > > I will try keepAlive off with stale check off. we have the same http > > > client usage like the one in benchmark code. that is only one client > with > > > pool manager shared by all requests. > > > > > > > > > On Thu, Jul 4, 2013 at 11:41 AM, Oleg Kalnichevski <[email protected] > >wrote: > > > > > >> On Thu, 2013-07-04 at 11:19 +0100, Ke Ren wrote: > > >> > Thanks Oleg. 4.2 benchmark code is super useful. I ran this code > there > > >> is > > >> > no big difference between 4.3 and 4.2. I found > STALE_CONNECTION_CHECK is > > >> > false in benchmark test. If it's on it will drop Requests per second > > >> from > > >> > 17k to 14k. > > >> > > > >> > > >> Yes, this is expected. > > >> > > >> > I plugin our http client wrapper on top of apache http client in > apache > > >> > benchmark test with the same settings and it achieved the same. The > > >> problem > > >> > is I still got the same issue in our own loadtest either with our > > >> wrapper > > >> > client or with the client from benchmark. STALE_CONNECTION_CHECK is > > >> true in > > >> > our loadtest because it caused another issue: > > >> > > > >> > rg.apache.http.NoHttpResponseException: The target server failed to > > >> respond > > >> > at > > >> > > > >> > org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:95) > > >> > > > >> > > >> I think you might be better off closing persistent connections that > have > > >> been idle for too long every once in a while rather than doing an > stale > > >> connection check for every request, which is very expensive in terms > of > > >> performance. > > >> > > >> Please also make sure you are using version 4.2.5 of HttpClient > > >> > > >> > Our loadtest is pretty simple. We have a target ec2 instance to > serve > > >> the > > >> > url we are going to hit. Benchmark and our loadtest were tested > against > > >> the > > >> > same instance. In our loadtest, we have a jetty server with jersey > rest > > >> > framework on another m3.xlarge instance. basically it receives > requests > > >> > from pressure test framework and then calls target url. So the > testing > > >> > function is just a http client call inside a rest call. It means > each > > >> > income call will cause a outcome call. > > >> > > > >> > Comparing with benchmark code, the concurrency in our code set by > > >> > maxPerRoute doesn't reflect the real concurrency caused from income > > >> calls. > > >> > I am going to investigate this part. Do you see any other issues? > > >> > > > >> > > >> You are not creating a new instance of HttpClient for each request by > > >> any chance? > > >> > > >> Oleg > > >> > > >> > Thanks, > > >> > > > >> > Ke > > >> > > > >> > > > >> > On Wed, Jul 3, 2013 at 8:59 PM, Oleg Kalnichevski <[email protected] > > > > >> wrote: > > >> > > > >> > > On Wed, 2013-07-03 at 18:59 +0100, Ke Ren wrote: > > >> > > > Hi, > > >> > > > > > >> > > > We are using httpcomponents httpclient 4.2.2 and doing loadtest > with > > >> > > > client. when we increased concurrent requests to 3000 per > second, we > > >> > > found > > >> > > > it took near half of cpu usage on aws ec m3.xlarge instance. We > have > > >> > > config > > >> > > > as the following: > > >> > > > > > >> > > > keepAlive is enabled. socket buffer size is 8 * 1024. > maxConnection: > > >> > > 2000, > > >> > > > maxPerRoute: 2000 > > >> > > > > > >> > > > Today we found apache benchmark test with httpclient 4.3.beta > and > > >> ran it > > >> > > on > > >> > > > the same aws ec2 instance. The outcome is impressive. It > achieved > > >> more > > >> > > than > > >> > > > 16k requests per second and cpu usage is much lower. Its config > is > > >> pretty > > >> > > > simple: max connection is 2000 with max 50 conns per route. same > > >> buffer > > >> > > > size. > > >> > > > > > >> > > > I am just curious if http client 4.3.beta is improved so much > or we > > >> did > > >> > > > wrong config with 4.2.2. > > >> > > > > > >> > > > Thanks in advance > > >> > > > > > >> > > > Ke > > >> > > > > >> > > I suspect wrong configuration. HC 4.3 can be expected to be > marginally > > >> > > faster for smaller messages but the difference can hardly be more > than > > >> > > 15%. There is also a version of benchmark that uses 4.2.5 version > of > > >> > > HttpClient if that can help. > > >> > > > > >> > > > > >> > > > > >> > https://svn.apache.org/repos/asf/httpcomponents/benchmark/httpclient/branches/4.2.x/ > > >> > > > > >> > > Oleg > > >> > > > > >> > > > > >> > > > > >> > > > --------------------------------------------------------------------- > > >> > > To unsubscribe, e-mail: > [email protected] > > >> > > For additional commands, e-mail: > [email protected] > > >> > > > > >> > > > > >> > > >> > > >> > > >> --------------------------------------------------------------------- > > >> To unsubscribe, e-mail: [email protected] > > >> For additional commands, e-mail: [email protected] > > >> > > >> > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
