On Mon, 2012-01-30 at 14:56 +0100, Oleg Kalnichevski wrote: > On Mon, 2012-01-30 at 17:45 +0530, Sadeep Jayasumana wrote: > > Hi, > > > > On Mon, Jan 30, 2012 at 5:15 PM, Oleg Kalnichevski <[email protected]> wrote: > > > > > On Sat, 2012-01-28 at 23:46 +0530, Sadeep Jayasumana wrote: > > > > Hi, > > > > > > > > On Fri, Jan 27, 2012 at 6:50 PM, Oleg Kalnichevski <[email protected]> > > > wrote: > > > > > > > > > On Fri, 2012-01-27 at 15:18 +0530, Sadeep Jayasumana wrote: > > > > > > Hi, > > > > > > > > > > > > I have written an HTTP server using HTTPCore-NIO and currently > > > testing it > > > > > > with httpcore-ab [1]. I have been seeing very good TPS values so > > > far. But > > > > > > when I try to send 5KB HTTP messages with 2500 and concurrency > > > level, 50 > > > > > > messages per thread, I'm getting the following exception several > > > times > > > > > from > > > > > > httpcore-ab and it results in "Failed requests" in the httpcore-ab > > > > > output. > > > > > > No exceptions are displayed from the server side. Hence, I think the > > > > > server > > > > > > is rejecting some socket connections because it's already fully > > > loaded. > > > > > > > > > > > > I'm running the server and load generator on two server machines > > > that are > > > > > > connected with Gigabit Ethernet. Each server has 16 CPU cores, 16 GB > > > > > memory > > > > > > etc. > > > > > > > > > > > > I have increased the socket timeout in both httpcore-ab and server > > > side > > > > > > ListeningIOReactor to 360000. 16 threads are allocated for the > > > reactor. > > > > > > Messages received by the server are processed in a separate worker > > > pool > > > > > > which has ~500 threads. I have also increased the open file limit of > > > my > > > > > OS > > > > > > (Debian) to 65535. > > > > > > > > > > > > Is there any tuning parameter of HTTPCore that I can used to get rid > > > of > > > > > > this exception? Perhaps, something similar to "acceptQueueSize" in > > > Jetty > > > > > > [2]? > > > > > > > > > > > > > > > > Hi Sadeep > > > > > > > > > > A couple of points in no particular logical order. > > > > > > > > > > (1) HttpCore does not depend on a logging toolkit and does not do any > > > > > logging per default. If it ever encounters an abnormal situation, > > > > > instead of logging such an event it throws an exception and > > > > > terminates. > > > > > Quite often this is not desirable, so one can provide an exception > > > > > handler that can log using a a logging toolkit of choice or handle a > > > > > particular type of exception in an application specific way. > > > > > > > > > > If the server did not terminate under with an exception, I kind of > > > > > suspect it was still trying to chew through the load without > > > > > encountering anything abnormal. Unless you have added a custom > > > exception > > > > > handler. > > > > > > > > > > (2) HttpCore does not have a concept of accepted connection queue. As > > > > > soon as the listener accepts an incoming connection its channel gets > > > > > immediately added to the underlying i/o selector. No intermediate > > > > > queuing takes place. > > > > > > > > > > > > > (3) Given that the connection times out on the client, this sounds > > > > > more > > > > > like a client side issue. Was connection really suck in > > > > > Socket#socketConnect for 360000? > > > > > > > > > > > > > I'm using httpcore-ab [1] as the client. I have set socket timeout of > > > using > > > > -t 360000, and changed the connection timeout with the patch shown > > > > below. > > > > But I'm starting to get this error well before 360000 millis past the > > > test > > > > startup. Therefore, I there is no possibility that the connection > > > > attempt > > > > hangs for 360000 millis before printing this exception. I was thinking > > > > whether there are upper bounds for these limits that cause my settings > > > > to > > > > be neglected. > > > > > > > > > > You see. If java.net.ConnectException: Connection timed out is thrown by > > > java.net.PlainSocketImpl.socketConnect before 360000 millis elapse, > > > there must be something wrong with the client side. > > > > > > > Could this be a possible bug in httpcore-ab code? I'm quite sure that it > > doesn't wait 360000 millis before throwing this exception. > > >
Hi Sadeep It did turn out to be an issue with the way HttpCore AB initialized connections. It was basically using an old (Java 1.0) socket constructor whose behavior can differ between OS platforms and possibly between JRE major releases with regards to connect timeout handling. The connect timeout value set in the HttpParams simply had no effect and therefore connections in your tests were timing out before 360000 milliseconds elapsed (probably using some sort of a system default). I committed a fix for the problem http://svn.apache.org/viewvc?rev=1238566&view=rev Please review Oleg --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
