If there is another box available you could try a scheme like the following::
Use the other box as a proxy. It presents a synchronous web service to clients, which presumably connect via http. It passes the requests and responses to/from the box that implements the service (doing the heavy lifting) in an asynchronous pattern. It is thus called back by that service with the response to each request. It could use JMS for this purpose, for instance. Each request/response to the implementation box would be correlated with the client http connection on which the response was to be returned to the client. The proxy box would need 2500 or so open sockets (plus a small number for traffic to/from the implementation box) but would not need 2500 threads (unless the proxy used the thread-per-request model that Axis uses). Essentially, the proxy box is bridging between synchronous http and a message queueing system. The implementation box would need just a small number of open sockets and could use any desired threading architecture, such as some kind of thread pool. In general, the synchronization overhead for thousands of threads is a signficant performance hit even if the threads are blocking most of the time. There are efficient architectures for handling many sockets. Google for the Reactor pattern, or consult the Patterns of Software Architecture book (vol 2) or the papers of Doug Schmidt and the ACE group http://www.cs.wustl.edu/~schmidt/ACE.html . Jeff On 8/2/06, Kiggy Pop <[EMAIL PROTECTED]> wrote:
Hi Sunil, Anne et al, I am not designing this system from scratch and the SOAP API with this 25 sec blocking synchronous operation is a requirement. (By "asynchronous exchange" I take it Anne means mean get my server to do a 'WS callback' ... unfortunately that is a not allowed API change.) My server is using Tomcat 5.5 within JBoss and by default Tomcat is using 250 threads in its pool; I have upped this to 2500, have modified Tomcat to use native not Java code for handling the HTTP requests (See http://tomcat.apache.org/tomcat-5.5-doc/apr.html ) & have made TCP/IP stack server modifications. This is giving me around a max of 60 to 80 WS requests per second each lasting 25 seconds with very low CPU utilization on the server. This is running stable in overnight tests with around 1300 Java threads active in the server (mostly Tomcat threads) however I am not sure how to push this rate up further or even if I am going down the right road here. So that is why I am wondering if at a TCP/IP level is there is anything I can do to make communication more efficient e.g. reusing same socket for many and overlapping SOAP requests & responses or maybe UDP not TCP. I thought this would be a common problem but it v hard to find info on the web on this. Thanks, KP Sunil D'Monte <[EMAIL PROTECTED]> wrote: I would second that... asynchronous is definitely the way to go. And I don't think it's a just a matter of the number of sockets at the OS level, it's also a matter of the number of threads your web/app server can run. A heavy-duty app server like weblogic has 15 execute threads by default. Here you're talking about a 100 new requests coming in per second, and each taking 25 seconds to complete. So even if you had 10 weblogic servers in a load-balanced cluster, you would still run out of available threads after just a few seconds ... Regards, Sunil D'Monte Tavant Technologies http://www.tavant.com > -----Original Message----- > From: Anne Thomas Manes [mailto:[EMAIL PROTECTED] > Sent: Wednesday, August 02, 2006 17:18 > To: [email protected] > Subject: Re: Too many sockets being created. Bad architecture? > > My question is : > What is the standard way to architect web services which have > a very long blocking synchronous operations ? > > Don't. Design an asynchronous exchange instead. > > Anne > > On 8/2/06, kk kk wrote: > > Hi, > > > > I am developing a WS which provides an operation that takes > around 25 > > seconds to complete. I can not speed this up nor I can not use a > > polling pattern or a callback to inform the client of the result of > > the request. > > > > I want to make the WS server capable to handle 100 new requests per > > second, so 100 x 25 = 2500 sockets connections will exist > at any point > > in time with my current design !! I am sure this is bad but am not > > sure what is the best way to go to get a scalable solution with a > > single server process. > > > > I am using Axis 1.2 and Tomcat on the server which is a Sun Solaris > > box. My test client is a Axis/Java application though I need to be > > able to support other clients too. > > > > As expected, I can see huge numbers of open sockets and FDs > using unix > > commands like netstat and lsof. > > > > I have been experimenting with TCP/IP kernel setting on the > server and > > using ideas from > > > http://publib.boulder.ibm.com/infocenter/wasinfo/v6r0/index.jsp?topic= > > /com.ibm.websphere.express.doc/info/exp/ae/tprf_tunesolaris.html > > I have used > > ndd -set /dev/tcp tcp_conn_req_max_q 8000 to increase number of > > sockets that the server can handle and this does seem to > allow a 1500+ > > parallel sockets to exist. > > > > My question is : > > What is the standard way to architect web services which > have a very > > long blocking synchronous operations ? > > Assuming I have a small number of clients each making many > WS calls is > > there a way to somehow multiplex many SOAP requests for > eackh client > > down one socket? > > Is there a way to get Axis/Tomcat server to use UDP/IP rather than > > TCP/IP? > > > > I am new to this and all ideas much appreciated. > > > > Cheers, > > KP > > > > > > > > > > > > > > ___________________________________________________________ > > All new Yahoo! Mail "The new Interface is stunning in its > simplicity > > and ease of use." - PC Magazine > > http://uk.docs.yahoo.com/nowyoucan.html > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] ________________________________ All new Yahoo! Mail "The new Interface is stunning in its simplicity and ease of use." - PC Magazine
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
