BindExceptions come when a socket is not in valid state when a call to bind is made. I guess this was obvious.
What's happening is when you put high sleep times, the sockets which were otherwise in timeout state ( 2 MSL) are completely released by the OS. Hence a new call to bind is successful. So much for your concern about ridding yourself of bind exceptions. setReuse allows you to bind to a socket even when it is in a timeout state. Perhaps, the function is there so user does not have to explicitly put "sleep" Could you verify by commenting out most of your code, how many sockets you are able to create concurrently on your local port and then the webserver. On UNIX install lsof ... and then do a lsof | grep TCP Windows, do a netstat and check how many of the sockets are there. Could you throw in more of your code here. This should have been fairly simpler task... -----Original Message----- From: Aaron [mailto:[EMAIL PROTECTED] Sent: Tuesday, November 29, 2005 6:51 PM To: HttpClient User Discussion Subject: Re: HttpClient/TCP experts--how do I rapidly use http connections? Sam, Thanks for the quick response! We're running this tool on a Windows machine, which allows creation of sockets across a range of roughly 4000 ports--we were seeing the BindExceptions after a short duration, so I think that means we were consuming these ports too quickly. Your conjecture that the "OS is doing more work to recycle the port" in the case of setReuseAddress(true) is a sensible theory--can anyone verify? What concerns me, or what I ultimately don't understand, is why in both cases, to rid myself of the BindExceptions or to improve my response times, I have to set relatively high sleep times in my threads. I suppose what's happening in these sleep times, is I'm allowing Windows/Java/HttpClient time to recycle the connection. It seems that when I use sleep times lower than 1s, my response times are noticeably affected or I get those BindExceptions. So is Windows really *that* slow at recycling TCP connections? Then how do high-performance servers work to accept so many incoming connections? Is there a way around this, besides running my tool simultaneously on a 100 machines? Thanks again! Aaron On 11/29/05, Sam Berlin <[EMAIL PROTECTED]> wrote: > Hi Aaaron, > > Assuming you already tried the ulimit suggestion that Rajat mentioned, > the reason this may be happening is because you're really doing a lot > of socket creations. Do the BindExceptions happen immediately, or > after a short bit of the program running? If it's after a while, I > can only guess that the app created so many sockets that it ran out of > local ports to give to a newly created socket. Setting > setReuseAddress to true lets you reuse already-used ports, so it'll > never run out. If the response times are going up when that's set, > it's likely because the OS is doing more work to recycle the port. > All of this is conjecture, of course, but I can't think of any other > reason you'd be seeing what you're seeing. > > Thanks, > Sam > > > On 11/29/05, Aaron <[EMAIL PROTECTED]> wrote: > > In my email (inlined below) you'll see a description of how I am using > > HTTP client to rapidly generate HTTP post requests to a server--for > > performance testing purposes. The tool I'm writing creates a > > multi-threaded HttpClient object (configured w/ > > MultiThreadedConnectionManager) and then kicks off N threads that each > > use this object to execute PostMethods. When any of the N threads > > finishes its request and releases the connection, a new thread takes > > its place and executes a PostMethod. And so on--so that effectively > > my tool is greedily keeping 10 PostMethods executing at any given > > time. > > > > Each thread is executing code that looks like this: > > > > // client thread > > ... > > final long start = sampleTime(); > > myHttpClient.executeMethod(post); > > final long finish = sampleTime(); > > reportResponseTime(finish - start); > > ... > > > > At first I was getting BindExceptions when I configured the tool to > > use N > 5 concurrent threads. I resolved this by registering my own > > http socket factory with HttpClient--*my* socket factory did this: > > > > { > > socket = createSocket(); > > socket.setReuseAddress(true); // *** This is the important change *** > > return socket; > > } > > > > This alleviated my BindExceptions, HOWEVER setting reuseAddress to > > true caused my response times to go up--even for cases where N <= 5. > > That is, I was getting better response times when I didn't set > > reuseAddress. > > > > My response times improve (in all cases) when I introduce a > > significant sleep time (e.g., 1s) at the end of each thread (before it > > allows another thread to wake and execute a post method)--HOWEVER, I > > do not want to throttle my throughput like this. > > > > I suspect that there is some TCP/HttpClient related issues here that I > > do not understand completely. Is there an expert on this that can > > give me some insight into this problem? Why did setReuseAddress(true) > > prevent the BindExceptions, but cause greater response times? How do > > I reuse http connections or execute post methods as fast as possible, > > without effecting my response times? > > > > Thanks for any help!! > > > > Aaron > > > > > > > > On 11/16/05, Rajat Sharma <[EMAIL PROTECTED]> wrote: > > > a) You can try reusing your connections for the subsequent post methods, > > > if this fits in your test scenario. > > > b) Put socket timeout. ttpClient.setTimeout() > > > andHttpClient.setConnectionTimeout() should suffice. > > > c) Bind exceptions can also come since on Unix there is a limit of number > > > of total TCP sockets you can open. This limit is 1024. You can increase > > > this limit > > > by ulimit command. > > > > > > I hope it helps. > > > Raj > > > > > > -----Original Message----- > > > From: Aaron [mailto:[EMAIL PROTECTED] > > > Sent: Wednesday, November 16, 2005 3:25 PM > > > To: [email protected] > > > Subject: rapid http connection use > > > > > > > > > Hi, > > > > > > I am writing a performance testing tool that uses HttpClient; the tool > > > kicks off N threads, and each thread creates a PostMethod and executes > > > it via an HttpClient instance (configured with > > > MultiThreadedHttpConnectionManager). When each request thread > > > finishes, another thread takes its place immediately and > > > creates/executes another post method using the same HttpClient. The > > > result is that I am rapidly creating PostMethods and executing them. > > > (I am ensuring that releaseConnection() is called in each thread.) > > > > > > I am getting repeated java.net.BindExceptions, and I understand from > > > the research that I've done that this can happen if you are creating > > > socket connections so rapidly. This has to do with TCP connections > > > sitting in a CLOSED state for some time after being closed. > > > > > > My tool can be configured to have each request thread sleep some > > > number of milliseconds after releasing its connection. When I > > > configure this sleep time to be high (~1 sec), the BindExceptions go > > > away. I assume this is time needed for the released TCP connection to > > > leave its CLOSED state. However, that this sleep time must be so high > > > is what concerns me. > > > > > > HttpClient experts, can you help me around this issue? How do I > > > dispatch these threads more quickly, without getting BindExceptions? > > > > > > Thanks for any insights-- > > > > > > Aaron > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
