Hi Oleg, On Tue, Jan 31, 2012 at 7:48 PM, Oleg Kalnichevski <[email protected]> wrote:
> On Mon, 2012-01-30 at 14:56 +0100, Oleg Kalnichevski wrote: > > On Mon, 2012-01-30 at 17:45 +0530, Sadeep Jayasumana wrote: > > > Hi, > > > > > > On Mon, Jan 30, 2012 at 5:15 PM, Oleg Kalnichevski <[email protected]> > wrote: > > > > > > > On Sat, 2012-01-28 at 23:46 +0530, Sadeep Jayasumana wrote: > > > > > Hi, > > > > > > > > > > On Fri, Jan 27, 2012 at 6:50 PM, Oleg Kalnichevski < > [email protected]> > > > > wrote: > > > > > > > > > > > On Fri, 2012-01-27 at 15:18 +0530, Sadeep Jayasumana wrote: > > > > > > > Hi, > > > > > > > > > > > > > > I have written an HTTP server using HTTPCore-NIO and currently > > > > testing it > > > > > > > with httpcore-ab [1]. I have been seeing very good TPS values > so > > > > far. But > > > > > > > when I try to send 5KB HTTP messages with 2500 and concurrency > > > > level, 50 > > > > > > > messages per thread, I'm getting the following exception > several > > > > times > > > > > > from > > > > > > > httpcore-ab and it results in "Failed requests" in the > httpcore-ab > > > > > > output. > > > > > > > No exceptions are displayed from the server side. Hence, I > think the > > > > > > server > > > > > > > is rejecting some socket connections because it's already fully > > > > loaded. > > > > > > > > > > > > > > I'm running the server and load generator on two server > machines > > > > that are > > > > > > > connected with Gigabit Ethernet. Each server has 16 CPU cores, > 16 GB > > > > > > memory > > > > > > > etc. > > > > > > > > > > > > > > I have increased the socket timeout in both httpcore-ab and > server > > > > side > > > > > > > ListeningIOReactor to 360000. 16 threads are allocated for the > > > > reactor. > > > > > > > Messages received by the server are processed in a separate > worker > > > > pool > > > > > > > which has ~500 threads. I have also increased the open file > limit of > > > > my > > > > > > OS > > > > > > > (Debian) to 65535. > > > > > > > > > > > > > > Is there any tuning parameter of HTTPCore that I can used to > get rid > > > > of > > > > > > > this exception? Perhaps, something similar to > "acceptQueueSize" in > > > > Jetty > > > > > > > [2]? > > > > > > > > > > > > > > > > > > > Hi Sadeep > > > > > > > > > > > > A couple of points in no particular logical order. > > > > > > > > > > > > (1) HttpCore does not depend on a logging toolkit and does not > do any > > > > > > logging per default. If it ever encounters an abnormal situation, > > > > > > instead of logging such an event it throws an exception and > terminates. > > > > > > Quite often this is not desirable, so one can provide an > exception > > > > > > handler that can log using a a logging toolkit of choice or > handle a > > > > > > particular type of exception in an application specific way. > > > > > > > > > > > > If the server did not terminate under with an exception, I kind > of > > > > > > suspect it was still trying to chew through the load without > > > > > > encountering anything abnormal. Unless you have added a custom > > > > exception > > > > > > handler. > > > > > > > > > > > > (2) HttpCore does not have a concept of accepted connection > queue. As > > > > > > soon as the listener accepts an incoming connection its channel > gets > > > > > > immediately added to the underlying i/o selector. No intermediate > > > > > > queuing takes place. > > > > > > > > > > > > > > > > (3) Given that the connection times out on the client, this > sounds more > > > > > > like a client side issue. Was connection really suck in > > > > > > Socket#socketConnect for 360000? > > > > > > > > > > > > > > > > I'm using httpcore-ab [1] as the client. I have set socket timeout > of > > > > using > > > > > -t 360000, and changed the connection timeout with the patch shown > below. > > > > > But I'm starting to get this error well before 360000 millis past > the > > > > test > > > > > startup. Therefore, I there is no possibility that the connection > attempt > > > > > hangs for 360000 millis before printing this exception. I was > thinking > > > > > whether there are upper bounds for these limits that cause my > settings to > > > > > be neglected. > > > > > > > > > > > > > You see. If java.net.ConnectException: Connection timed out is > thrown by > > > > java.net.PlainSocketImpl.socketConnect before 360000 millis elapse, > > > > there must be something wrong with the client side. > > > > > > > > > > Could this be a possible bug in httpcore-ab code? I'm quite sure that > it > > > doesn't wait 360000 millis before throwing this exception. > > > > > > > Hi Sadeep > > It did turn out to be an issue with the way HttpCore AB initialized > connections. It was basically using an old (Java 1.0) socket constructor > whose behavior can differ between OS platforms and possibly between JRE > major releases with regards to connect timeout handling. The connect > timeout value set in the HttpParams simply had no effect and therefore > connections in your tests were timing out before 360000 milliseconds > elapsed (probably using some sort of a system default). > > I committed a fix for the problem > > http://svn.apache.org/viewvc?rev=1238566&view=rev > > Please review > Many thanks for looking into this. I will review this at my earliest and let you know the outcome. Thanks, Sadeep > > Oleg > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > -- Sadeep Jayasumana**** Email: [email protected]**** Phone: +94-77-2266507
