Hi Roland, Thanks for the reply. Actually I am doing a proof of concept for HttpClient and checking whether it can be used for the application during that I tried the example given in HtpClient and ran on my windows desktop connecting to the adjacent desktop. With the netstat command I saw the connections not being reused. I dont know if I am analysing it in a correct way. Every time a new foreign address gets created. I am also attaching the sample code. I tried this example for two connections and saw that connection was not being reused. I was seeing the Established, Finwait2, TIMEWait states in the netstat command. Please let me know if there is some other way of checking I the connection is being reused. I have included my questions below your answers. Could you please have a look and them also?
On Sun, 18 Dec 2005 Roland Weber wrote : >Hi Bharathi, > > > We will use only one httpConnection and MultiThreadedConnection manager > > > for the total application. > >I assume you mean one HttpClient, not one HttpConnection. Yes I meant one HttpClient Only. > > > But inorder that all the threads in case 160 are launched at the same time, > > get back the response in 3 seconds, do i need to add any thing extra in code > > by making the connection persistent. > >We don't know your code, so we really can't tell what you need to add or not. >HttpClient supports connection re-use out of the box if used correctly, but >your other comments suggest that your application is not using it correctly, >or that your environment does not allow connection re-use. For example, if >the server sends back "Connection: close" headers with each response, it is >not possible to re-use the connections. >The most common error that prevents connection re-use is that people create >a new HttpClient object and/or HttpConnectionManager for each request they >send. That was the first thing we asked you. Now that we know you are not >making that mistake, we need more information from you to learn why you are >not seeing connections being re-used. A wire log of two requests being sent >to the same server consecutively (*not* in parallel) would be a good start. >Feel free to remove the request contents from the log, what we need is the >headers and traces from HttpClient code. How do i get this information i mean the request contents from the log and headers and traces. Please guide in this aspect as i am new in this area. > > > The production boxes have 8 cpus. I need the values to be set for the > > maximum connections per host. > >There is a step called "performance tuning" that should be part of any >application deployment into a production environment. You develop a load >test and run it against some initial values for your tuning parameters. >Then you vary some of the tuning parameters and run the load test again >to see if there is an improvement. Repeat this until the performance is >satisfactory. If you can't achieve satisfactory performance though the >hardware is powerful enough, review the application code to identify and >remove bottlenecks. >If you are hitting only one host, set the maximum number of connections >to the same value as the maximum number of connections per host. You've >got 8 CPUs, so there really is no point in starting with less than 8 >connections, as a rough guess. If you find that performance can no longer >be improved by increasing the connections per host, but the CPU load is >still low, then consider vertical scaling of your application. > I can implemet in this way, but we need to decide the hardware requirements intially itself in the project. We cant give it at a later moment. So i need to do a mathematical calculation and decide. If it doesnot support at the last stages then it will be a problem. the boxes are not available now. So could you help me in this aspect. >We know how HttpClient works, but we don't know your environment nor your >application. We don't know how many requests you have to send in a given >timeslice, we don't know how much data is sent with each request, we >don't know how much data is returned by the server with each response, >we don't know how much time the server needs after receiving a request >before it starts to send the response. Neither do we know about possible >design flaws in your application, or peculiarities of your environment. >If you have only one HttpClient, are you supplying individual HttpState >objects for each request or do you share the default state across all >threads? Do the servers use HTTP/1.0 or HTTP/1.1? Are there proxies or >firewalls inbetween? There are a lot of things that could have an impact >on HTTP performance in general, and on HttpClient in particular. We are >in no position to give you any reliable values for tuning HttpClient. We will be using Weblogic 8.1 on the Solaris 9 box. I think it uses http1.1. > > > I alos need a clarifictaion do i need i need to add any thing extra for > > pooling and closing the connection. In the test i did i find the > > connections are not being reused at all. > >This is a specific problem which we can help you to solve. Send us a >wire log (http://jakarta.apache.org/commons/httpclient/logging.html) >and somebody is going to have a look at it. Some code fragments that >show how you are configuring and using HttpClient might also be helpful. > > > But how do i know what are the values to be set. The no of threads > > launching is not controlled by us. It might be 150 or more. But how > > do i decide the max no connections for host and total connections. > > That is, which one is faster, running the same with lesser no of > > connections in pool or having 150 in the connection pool. > >Develop a load test for simulating varying numbers of threads. Consider >limiting the number of threads by some external means. Since you have >no upper bound on the number of threads, you can never guarantee that >there is a connection for every thread available. Besides, connection >re-use is likely to decrease with the number of connections, right? >You should try to simulate a high-load situation as closely as possible, >with real-life behavior for the amount of data being transferred, the >response times from the server, and the frequency in which the threads >generate requests. Then play around with the number of connections. > > > I also want the clarifictaion if the connections are being reused. > >They are not, you told us yourself. We don't know why. >You should fix that before running performance tests. > > > We want the response back in 3 secs,but if the aplication connectivity > > itself is taking 1 minute then it is a big problem. > >You can set timeouts to report errors instead of locking up threads. >You can implement your own connection manager to detect specific >load situations and report them, at least in a development or system >integration environment. You can implement your own connection manager >to create new connections if the delay until an old one is released is >getting too long. You can clog your application with high load, then >send signal 3 to the JVM to generate a core dump, which will tell you >where exactly all the threads are really waiting. (I would try that >before implementing a custom connection manager.) > > >Personally, I think you have some very basic flaw in your application >that prevents the connections from being re-used. Your first step >should be to locate and remove that flaw. But we can't help you if >all you're telling us about the problem is that you have it, while >asking for performance tuning values to scale your application to >an 8 CPU box though it doesn't seem to scale on a single CPU. > >cheers, > Roland > >--------------------------------------------------------------------- >To unsubscribe, e-mail: [EMAIL PROTECTED] >For additional commands, e-mail: [EMAIL PROTECTED] > Thanks & Regards S.S. Bharathi
