Re: nbio connector
- Original Message - From: Remy Maucherat [EMAIL PROTECTED] To: Tomcat Developers List [EMAIL PROTECTED] Sent: 2002. január 8. 16:35 Subject: Re: nbio connector , and could use a helping hand here and there. In fact, I already have much of the code for building the HttpServletRequest object, but to keep it really elegant I'll need some help from the Tomcat community. Most notably, much of the code duplications could go away if Catalina code in org.apache.catalina.connectors.http was more reusable (some classes are package-private and some public classes take package-private classes in a constructor argument etc.). I'll return to these issues on this list after I've properly set up the project on SourceForge. Don't use that code, as it is not very efficient / memory intensive, etc. Something more useful would be to write a nbio selector for the HTTP/1.1 connector I'm writing in j-t-c. I assume you're talking about coyote when refering to connector I'm writing in j-t-c. Could you be a bit more specific on that? I finally got the chance to jump on the NBIO HTTP connector implementation, and would like to know what is the current state of the coyote connector (seems like it's heavily in the primary under construction phase), and what is its long term goal? Will binary distributions of Tomcat include it once it's complete? Is it meant to become the default HTTP connector implementation shipped with Tomcat? I'm now studying it for bits that could get reused in a NBIO implementation (that means mostly the response code; request processing algorithms can't be generalized accross blocking and non-blocking approach, though (this asymmetry has its deep reasons and is quite amusing)). Attila. Remy -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] smime.p7s Description: application/pkcs7-signature
RE: nbio connector
In few days we'll have the jk2 working, so you can play with unix domain sockets - that should improve a bit the performance. There are few other optimizations there ( more agressive recycling, etc ) - and we really need help testing it and feedback. ( I have it working on my machine, but I'm still trying to figure how to deal with 4.0 class loaders - if I can't I'll just use the old mechanism ). Unix domain will be faster on systems running Apache and Tomcat on the same box, so faster network IO will still be fine for configuration with separate Webserver and Tomcat (security concerns or load-balancing) Regarding NBIO and Jk2 - I think the Channel abstraction could be enhanced to use poll() ( using APR ). Jk2 is really a whole new study for the good old mod_jk developpers, like many others I'm back to school Professor Costin ;) -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: nbio connector
On Fri, 11 Jan 2002, GOMEZ Henri wrote: Unix domain will be faster on systems running Apache and Tomcat on the same box, so faster network IO will still be fine for configuration with separate Webserver and Tomcat (security concerns or load-balancing) Of course, different issues. For 'lb' you have an extra layer of scalability - you can add more workers to better handle the load. Assuming a normal case, with servlets doing something besides helloworld, the benefits for NBIO on lb will remain small. As I said, I'm +1 on NBIO ( and I love the new java.nio package ), not because of possible improvements on tomcat's performance but because it's cool and the right ( or better ) way to do io. Regarding NBIO and Jk2 - I think the Channel abstraction could be enhanced to use poll() ( using APR ). Jk2 is really a whole new study for the good old mod_jk developpers, What I tried to do is make it easier to read and cleaner - for new and future 'old mod_jk developers'. There is a big change in code organization - but if anyone feels the code is harder to read and maintain then we need to start over again. I personally believe it's going in the right direction, so I'll continue moving. BTW, the new configuration ( based on Location instead of JkMount and without workers.properties ) needs review and feedback. ( the old stuff should still work, but I would like it deprecated ) Costin -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: nbio connector
I believe someone on this list once said something like open source development: have fun or get paid for it. I'm working on NIO solely because it's a fun new technology in Java with a huge potential, and if it happens to turn out useful it's a clear win. :-) NBIO is really an interesting stuff and any works conducted on it is good to make us learn (or relearn) An interesting point is that we ALL (Unix Developpers) used select(), poll() or whatever on our early age (especially up to the time threads API came up. Then we all switch to threads with blocked IO since it's really simpler for developpment. And it's funny to note that on the Windows Developpers with winsocks async mode have often keep the NON-BLOCKED mode. So my advice is continue to play with NBIO, and if bench are good for any parts of Tomcat (or even jakarta), contribute it ASAP. Keep on this works, since the benchs I saw on Matt site were more than interesting on heavily servers. Regards -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: nbio connector
Yes, after Mauricio brought to my attention the existence of a non-JDK NBIO package few days ago. At this point in time my framework was almost complete. On the first glance, the Sandstorm services platform part of the SEDA quite resembles my effort but I'll have to also take a closer look. Attila. - Original Message - From: Daniel Rall [EMAIL PROTECTED] To: Tomcat Developers List [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Sent: 2002. január 10. 0:26 Subject: Re: nbio connector Have you looked at Matt Welsh's SEDA http://www.cs.berkeley.edu/~mdw/proj/sandstorm/? -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] smime.p7s Description: application/pkcs7-signature
Re: nbio connector
El Jueves 10 Enero 2002 06:16, GOMEZ Henri escribió: I believe someone on this list once said something like open source development: have fun or get paid for it. I'm working on NIO solely because it's a fun new technology in Java with a huge potential, and if it happens to turn out useful it's a clear win. :-) NBIO is really an interesting stuff and any works conducted on it is good to make us learn (or relearn) Really, I need to learn all about this, because I am a autodidact programmer. ( I am a commercial enginner working as webmaster) So my advice is continue to play with NBIO, and if bench are good for any parts of Tomcat (or even jakarta), contribute it ASAP. Keep on this works, since the benchs I saw on Matt site were more than interesting on heavily servers. I will try to get NBIO work as a Ajp13 connector, because my site it's very high loaded ( IMHO ), and i NEED to get a better performance. Will be posible publish a bechmarking page? I want to compare my hardware,load, software, OS, JVM , and configuration with others. By example: My site Load: 65000 page views daily. I'm using a Linux Firewall compiled with LinuxVirtualServer to balanced between 2 webservers. Each webServers have a Apache 1.3.20 and a Tomcat 3.3.1 Both are Linux 2.2 I'm using Blackdown JDK 1.3 with green threads and the JIT ShudoJit. The Tomcat Home and the Apache DocumentRoot is hosted on a third server, and accesssed by NFS for the webservers. This third server have a Oracle database to server to both Tomcats. TOMCAT_OPTS=-Xms32m -Xmx256m -Djava.compiler=shudojit I'm using AJP13 balanced between both webservers. in httpd.conf: MaxClients 256 in server.xml: MaxThreads 300 AutoReloading off. Aditional libraries: FreeMarker JDBC Oracle Thin COS Regards -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: nbio connector
On Thu, 10 Jan 2002, Mauricio Nuñez wrote: Keep on this works, since the benchs I saw on Matt site were more than interesting on heavily servers. I will try to get NBIO work as a Ajp13 connector, because my site it's very high loaded ( IMHO ), and i NEED to get a better performance. In few days we'll have the jk2 working, so you can play with unix domain sockets - that should improve a bit the performance. There are few other optimizations there ( more agressive recycling, etc ) - and we really need help testing it and feedback. ( I have it working on my machine, but I'm still trying to figure how to deal with 4.0 class loaders - if I can't I'll just use the old mechanism ). Regarding NBIO and Jk2 - I think the Channel abstraction could be enhanced to use poll() ( using APR ). One thing on my todo list is another attempt at doors, which should be even ligher than nbio. ( I don't expect this to happen soon, unless more people start to help ) Costin -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: nbio connector
- Original Message - From: Remy Maucherat [EMAIL PROTECTED] To: Tomcat Developers List [EMAIL PROTECTED] Sent: 2002. január 8. 17:35 Subject: Re: nbio connector I'm a bit sceptical about the usefulness of the thing, then, since reading and parsing requests headers is by far the fastest part of the current connector (all this is coming from many OptimizeIt runs I did). The new Unless the input comes in slowly due to network conditions, or a deliberate DoS attack attempt. A client that opens many connections and then supplies input one byte per minute will bog down many threads and can cause a DoS with a blocking connector. 1. It needs JDK 1.4 to compile, and I believe Tomcat should not rely on the not-yet-final 1.4 JDK. Find something else. We already have JDK 1.4 flags in the build process, so it's a very bad justification I'm afraid. I wasn't aware of the fact. This argument does not hold then, really. 2. My framework as it stands now is a generic framework for implementing arbitrary TCP/IP services in a non-blocking fashion. It is an extension of nio from JDK 1.4 then ? No. Having non-blocking sockets API is not yet a server framework. You still need to attend to many implementation details: output buffering (and the associated memory management), setting up threads that act as selector loops for accepting, reading and writing, creating and tearing down those threads as load rises/lowers (as it stands now in JDK 1.4, a single thread can manage up to 63 channels so you still need multiple threads, only less than with blocking approach), explicit key cancelling, etc. With the framework, the protocol implementor only has to implement a ProtocolHandlerFactory and ProtocolHandler interface, and need not be concerned with low-level connection management. The whole thing is not a rocket science, and is quite trivial AFTER I've spent several nights on trying out how nio works and resolving surfacing issues -- that's what I want the spare the fellow developers of. That's not true that way. The ASF wants the copyright on all code in its repositories. SO you're free to take our code, but we can't take yours without you donating it. I see, that's really a distinction since in order for you to incorporate the code I have to transfer you the copyright. However, that's the consequence of the ASF policy and not of my licensing model (which is Apache-stlye itself). The I'll have no objections to migrating it into Tomcat source tree. sentence still holds even if it means donating the code later on. Attila. Remy smime.p7s Description: application/pkcs7-signature
Re: nbio connector
- Original Message - From: Remy Maucherat [EMAIL PROTECTED] To: Tomcat Developers List [EMAIL PROTECTED] Sent: 2002. január 8. 19:19 Subject: Re: nbio connector I agree on both the points above. Especially the first one actually, since you can avoid wasting a thread on waiting, and use all of them to do active work if needed. For the second point, Catalina uses a servlet to do the static files serving. I suppose the connector could do some optimization and serve some resources itself, but that's a bit complex, and not as clean as the current solution. Or, the default servlet could check if the response is instanceof non-blocking response and supply it with the appropriate file channel for response body; the response would take care of instructing the connector to transfer the file using the direct filesystem cache-to-socket transferTo() method in its own non-blocking code. This way, the majority of the logic would be inside a connector; the servlet would only have few lines of additional code inside an if(resp instanceof ...) block I like that difference between a fun optimization and a useful optimization :) At the moment, a lot of useful optimization work still has yet to be done. I believe someone on this list once said something like open source development: have fun or get paid for it. I'm working on NIO solely because it's a fun new technology in Java with a huge potential, and if it happens to turn out useful it's a clear win. :-) Attila. Remy -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] smime.p7s Description: application/pkcs7-signature
Re: nbio connector
- Original Message - From: [EMAIL PROTECTED] To: Tomcat Developers List [EMAIL PROTECTED] Sent: 2002. január 8. 20:35 Subject: Re: nbio connector I'm not very sure about nbio - most of the time there's a lot of complexity ( and a different programming model, etc ) - and the benefits seems pretty small. We already spend most of the time in user code ( i.e. servlets), the container overhead is getting smaller and smaller - so improvements on this area are very unlikely to matter. But it's cool - so I'm +1 on it as long as it doesn't mess with the existing APIs ( i.e. it is implemented as a module, etc ). Another reason why I'm inclined to take on it as a separate project - the whole thing is highly experimental now; oh and the JDK is still in beta. Keeping it isolated for some time until it matures a bit lessens the chance it causes any negative interference with Tomcat in its early days. Attila. Costin -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] smime.p7s Description: application/pkcs7-signature
Re: nbio connector
El Miércoles 09 Enero 2002 06:52, Attila Szegedi escribió: - Original Message - From: Remy Maucherat [EMAIL PROTECTED] To: Tomcat Developers List [EMAIL PROTECTED] Sent: 2002. január 8. 17:35 Subject: Re: nbio connector I'm a bit sceptical about the usefulness of the thing, then, since reading and parsing requests headers is by far the fastest part of the current connector (all this is coming from many OptimizeIt runs I did). The new Unless the input comes in slowly due to network conditions, or a deliberate DoS attack attempt. A client that opens many connections and then supplies input one byte per minute will bog down many threads and can cause a DoS with a blocking connector. I agree. connections via modem it's a source of latency. 1. It needs JDK 1.4 to compile, and I believe Tomcat should not rely on the not-yet-final 1.4 JDK. Find something else. We already have JDK 1.4 flags in the build process, so it's a very bad justification I'm afraid. I wasn't aware of the fact. This argument does not hold then, really. The Nbio project by Matt Welsh is focuses on JDK1.3. He is part of the java.nio Team, and i will wait some degree of coherency ;-) between both. Attila. Remy Mauricio -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: nbio connector
Attila Szegedi wrote: (as it stands now in JDK 1.4, a single thread can manage up to 63 channels so you still need multiple threads, only less than with blocking approach), The 63 channel limit is a Windows-only bug in the Beta, designing code around it probably isn't a good idea. -- Christopher St. John [EMAIL PROTECTED] DistribuTopia http://www.distributopia.com -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: nbio connector
Well I didn't want to bring this to the sunlight so soon but since you brought up the issue: I'm developing a generic non-blocking server framework for JDK 1.4. It handles all subtleties of the non-blocking server's life, such as non-blocking pipes for servlet output buffering, thread scaling (since a single selector handles at most 63 channels, you still need multiple threads), channel registration from a different thread than the thread that listens to the selector via a mechanism similar to Swing's invokeAndWait(), etc, etc. Using this framework, all you need to do is implement a single ProtocolHandler interface and focus your attention on the protocol implementation and not on the details of non-blocking I/O. The initial code is practically done, there is only one JUnit test left to pass. I've created a project for it on SourceForge (http://sourceforge.net/projects/nbserver) *today* (see, it's really fresh) and will upload the code there soon. The project is pretty much empty now, but I'll populate the CVS and the introductory website in the following days. It will be distributed under Apache-style license. My primary goal for bringing the code in the public is that I want to build a HTTP/1.1 connector for Tomcat 4.0 based on it, and could use a helping hand here and there. In fact, I already have much of the code for building the HttpServletRequest object, but to keep it really elegant I'll need some help from the Tomcat community. Most notably, much of the code duplications could go away if Catalina code in org.apache.catalina.connectors.http was more reusable (some classes are package-private and some public classes take package-private classes in a constructor argument etc.). I'll return to these issues on this list after I've properly set up the project on SourceForge. I wanted to keep the whole project out of the spotlight until it's up and running, but since you mentioned the issue, I thought you should know about similar efforts. Maybe you will want to take a look at it; I can notify you when the SourceForge project site is ready. Cheers, Attila. - Original Message - From: Mauricio Nuñez [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: 2002. január 8. 0:27 Subject: nbio connector Hi everybody! I'm using Tomcat 3.3 and Apache on Linux, and i want to develop a connector based on nbio (http://www.cs.berkeley.edu/~mdw/proj/java-nbio). My focus is get a multiplex connector based on few threads waiting connections, reducing the overhead on my server. Firstly, i will port the ajp13 interceptor, based on only 2 threads (i guess) The next step will be to migrate to java.nio, when Sun release a stable jdk1.4. Any comment? Can i submit my code here? I need to get a stable and scalable Tomcat on Linux. This work try to get that. Bye Mauricio Nuñez [EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] smime.p7s Description: application/pkcs7-signature
Re: nbio connector
My primary goal for bringing the code in the public is that I want to build a HTTP/1.1 connector for Tomcat 4.0 based on it I don't think you can have a nbio connector with a compliant servlet container, because the J2EE spec mandates at the moment: 1 thread - 1 request. That requirements makes nbio much less interesting over the current since you would need a big thread pool. , and could use a helping hand here and there. In fact, I already have much of the code for building the HttpServletRequest object, but to keep it really elegant I'll need some help from the Tomcat community. Most notably, much of the code duplications could go away if Catalina code in org.apache.catalina.connectors.http was more reusable (some classes are package-private and some public classes take package-private classes in a constructor argument etc.). I'll return to these issues on this list after I've properly set up the project on SourceForge. Don't use that code, as it is not very efficient / memory intensive, etc. Something more useful would be to write a nbio selector for the HTTP/1.1 connector I'm writing in j-t-c. For the connector, nbio is the last optimization which should be done. There are much bigger hotspots than the time spent in java.net in the connector which ships with 4.0.x ;-) Also, why not contributing the thing to the core project (if it manages to get around the problem mentioned above, of course), instead of developing it separately ? Remy -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: nbio connector
- Original Message - From: Remy Maucherat [EMAIL PROTECTED] To: Tomcat Developers List [EMAIL PROTECTED] Sent: 2002. január 8. 16:35 Subject: Re: nbio connector My primary goal for bringing the code in the public is that I want to build a HTTP/1.1 connector for Tomcat 4.0 based on it I don't think you can have a nbio connector with a compliant servlet container, because the J2EE spec mandates at the moment: 1 thread - 1 request. I don't think I conflict with this requirement. In fact, to execute a servlet's service() method still requires a separate thread. In my design, the connector first reads the request line and the headers in a non-blocking fashion, naturally multiplexing up to 63 requests on a single thread. I have a working HTTP processing state machine for that. (You simply HAVE to write state machines in nb world, since you can't encapsulate the algorithm in a single-thread linear flow of methods because of multiplexing). Then it buffers the whole request body using multiplexed non-blocking code. This buffering is done using a special non-blocking pipe (more on this below). After that, it constructs the request and response objects, and fires off container.invoke(req, resp) on a separate thread. The response object also has a non-blocking pipe that to the servlet masquerades as a ServletOutputStream. All this is done with the goal of completely avoiding any thread running container.invoke(req, resp) to EVER block on a socket read or write. A special non-blocking pipe is more-or-less an ordinary pipe: one input stream and one output stream, however it does not have a fixed-size buffer (because that would block when the buffer gets full), instead it manages a growing list of buffers and can hold any size of buffered bytes. It's used both for buffering request body, and for buffering the response. The response pipe is written to by the servlet and read from by the non-blocking code that transfers its data to the socket channel; so in the response phase the pipe buffers that part of the output that the servlet has already generated, but that still couldn't get pushed through the network interface. Thus, it eliminates I/O block in response.getOutputStream().write(...) calls. As an additional memory optimization, it starts buffering in memory, but if the buffered content gets large it switches to buffering in a memory-mapped file (also new in JDK 1.4). That requirements makes nbio much less interesting over the current since you would need a big thread pool. With my design, you still need one thread/request but only for the time required to process container.invoke() (that is, while the filters and servlets run). Since the design also eliminates I/O blocking in those threads, they can complete as quickly as their code otherwise allows, so the overall requirement for threads is substantially lower than in a blocking implementation. , and could use a helping hand here and there. In fact, I already have much of the code for building the HttpServletRequest object, but to keep it really elegant I'll need some help from the Tomcat community. Most notably, much of the code duplications could go away if Catalina code in org.apache.catalina.connectors.http was more reusable (some classes are package-private and some public classes take package-private classes in a constructor argument etc.). I'll return to these issues on this list after I've properly set up the project on SourceForge. Don't use that code, as it is not very efficient / memory intensive, etc. Something more useful would be to write a nbio selector for the HTTP/1.1 connector I'm writing in j-t-c. For the connector, nbio is the last optimization which should be done. There are much bigger hotspots than the time spent in java.net in the connector which ships with 4.0.x ;-) Also, why not contributing the thing to the core project (if it manages to get around the problem mentioned above, of course), instead of developing it separately ? There are several reasons: 1. It needs JDK 1.4 to compile, and I believe Tomcat should not rely on the not-yet-final 1.4 JDK. I know, this can be dealt with using conditional compilation in the build.xml file, as it does now with optional components. 2. My framework as it stands now is a generic framework for implementing arbitrary TCP/IP services in a non-blocking fashion. Having a Tomcat connector built upon it is only one possible application. SMTP, FTP, etc. servers can also be written upon this framework. 3. It would be tedious for me to send in CVS patches over long run. While I have no problem with this approach in general (I have already contributed to both Velocity and Ant by sending patches to committers), on SourceForge I have my own CVS I can commit to right now, and I need this level of control. I don't doubt, that the community would grant me committer status on Tomcat over time if I contribute regularly
Re: nbio connector
I don't think I conflict with this requirement. In fact, to execute a servlet's service() method still requires a separate thread. In my design, the connector first reads the request line and the headers in a non-blocking fashion, naturally multiplexing up to 63 requests on a single thread. I have a working HTTP processing state machine for that. (You simply HAVE to write state machines in nb world, since you can't encapsulate the algorithm in a single-thread linear flow of methods because of multiplexing). Then it buffers the whole request body using multiplexed non-blocking code. This buffering is done using a special non-blocking pipe (more on this below). After that, it constructs the request and response objects, and fires off container.invoke(req, resp) on a separate thread. The response object also has a non-blocking pipe that to the servlet masquerades as a ServletOutputStream. All this is done with the goal of completely avoiding any thread running container.invoke(req, resp) to EVER block on a socket read or write. I'm a bit sceptical about the usefulness of the thing, then, since reading and parsing requests headers is by far the fastest part of the current connector (all this is coming from many OptimizeIt runs I did). The new connector should even be a bit faster for reading. OTOH, the output is extremely slow (not because it's blocking IO, but rather because of really bad char/byte conversion), and nbio won't help that. The GC problems of the current connector is also a very significant issue which would take away all the benefits nbio would bring. It's still very useful work as the final optimization for very high load scenarios. With my design, you still need one thread/request but only for the time required to process container.invoke() In the real world, the servlets and JSPs are the thing which take by far the most time to complete, so I'm not sure you wouldn't end up spending a lot of time in the threaded part. 1. It needs JDK 1.4 to compile, and I believe Tomcat should not rely on the not-yet-final 1.4 JDK. Find something else. We already have JDK 1.4 flags in the build process, so it's a very bad justification I'm afraid. 2. My framework as it stands now is a generic framework for implementing arbitrary TCP/IP services in a non-blocking fashion. It is an extension of nio from JDK 1.4 then ? 3. It would be tedious for me to send in CVS patches over long run. If your contributions are valuable, you would get commit access very quickly, but it's up to you. If the Tomcat HTTP connector comes to life as part of the nbserver project and works fine, I'll have no objections to migrating it into Tomcat source tree. In fact, since I'm publishing the code under Apache-style license, I *CAN NOT* have any objections if it ever gets transferred into Tomcat source tree - either with or without my assistance. :-) That's not true that way. The ASF wants the copyright on all code in its repositories. SO you're free to take our code, but we can't take yours without you donating it. Remy -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: nbio connector
Remy Maucherat wrote: With my design, you still need one thread/request but only for the time required to process container.invoke() In the real world, the servlets and JSPs are the thing which take by far the most time to complete, so I'm not sure you wouldn't end up spending a lot of time in the threaded part. There was a long discussion on the advanced-servlets list a while back. That discussion concluded that the NIO looked most promising for: - Cleanly waiting for persistent connections (which take up a connector thread but don't really need to) You can kinda sorta do this already with the existing java.io classes, but NIO is better. This is a relatively easy win. - Serving static content. It's not just the non-blocking io, serving from memory mapped files is a beguilingly easy (but maybe not always the best) way to improve file serving performance. Using NIO to reduce the time service() needed to tie up a thread is fun, but seemed unlikely to help very much in practice. And of course the optimized char conversion stuff, but that's obviously useful and therefore boring. I'm pretty sure newesh versions of Jetty include a first pass at integrating NIO, so anybody that's interested might want to take a look. There's some fairly bogus (but still kinda interesting) benchmark results using NIO-like routines to serve static files at distributopia.com. (Sun says no NIO-bmarks until it's out of beta, so I used Matt Welsh's NBIO code) -- Christopher St. John [EMAIL PROTECTED] DistribuTopia http://www.distributopia.com -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: nbio connector
IMHO, the main benefit of non-blocking io has to do with improved robustness. If a router or adapter goes off the air while you're in the middle of a 'blocking' read, you are likely to never to see the light of day. This problem is exacerbated for client-side development because it is 'difficult' to set SO_TIMEOUT on a URLConnection. Actually, it's impossible, unless you provide your own URLStreamHandler. And this requires that you start reading sun's java.net source code. Just my two cents. Tom Drake - Original Message - From: Remy Maucherat [EMAIL PROTECTED] To: Tomcat Developers List [EMAIL PROTECTED] Sent: Tuesday, January 08, 2002 8:35 AM Subject: Re: nbio connector | I don't think I conflict with this requirement. In fact, | to execute a servlet's service() method still requires | a separate thread. In my design, the connector first reads | the request line and the headers in a non-blocking fashion, | naturally multiplexing up to 63 requests on a single thread. | I have a working HTTP processing state machine for that. | (You simply HAVE to write state machines in nb world, since | you can't encapsulate the algorithm in a single-thread linear | flow of methods because of multiplexing). Then it buffers the | whole request body using multiplexed non-blocking code. This | buffering is done using a special non-blocking pipe (more on | this below). After that, it constructs the request and response | objects, and fires off container.invoke(req, resp) on a separate | thread. The response object also has a non-blocking pipe that | to the servlet masquerades as a ServletOutputStream. | All this is done with the goal of completely avoiding any thread | running container.invoke(req, resp) to EVER block on a socket read or | write. | | I'm a bit sceptical about the usefulness of the thing, then, since reading | and parsing requests headers is by far the fastest part of the current | connector (all this is coming from many OptimizeIt runs I did). The new | connector should even be a bit faster for reading. OTOH, the output is | extremely slow (not because it's blocking IO, but rather because of really | bad char/byte conversion), and nbio won't help that. The GC problems of the | current connector is also a very significant issue which would take away all | the benefits nbio would bring. | | It's still very useful work as the final optimization for very high load | scenarios. | | With my design, you still need one thread/request but only for the | time required to process container.invoke() | | In the real world, the servlets and JSPs are the thing which take by far the | most time to complete, so I'm not sure you wouldn't end up spending a lot of | time in the threaded part. | | 1. It needs JDK 1.4 to compile, and I believe Tomcat should not rely | on the not-yet-final 1.4 JDK. | | Find something else. We already have JDK 1.4 flags in the build process, so | it's a very bad justification I'm afraid. | | 2. My framework as it stands now is a generic framework | for implementing arbitrary TCP/IP services in a non-blocking fashion. | | It is an extension of nio from JDK 1.4 then ? | | 3. It would be tedious for me to send in CVS patches over long run. | | If your contributions are valuable, you would get commit access very | quickly, but it's up to you. | | If the Tomcat HTTP connector comes to life as part | of the nbserver project and works fine, I'll have no | objections to migrating it into Tomcat source tree. | In fact, since I'm publishing the code under Apache-style license, | I *CAN NOT* have any objections if it ever gets transferred into Tomcat | source tree - | either with or without my assistance. :-) | | That's not true that way. The ASF wants the copyright on all code in its | repositories. SO you're free to take our code, but we can't take yours | without you donating it. | | Remy | | | -- | To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] | For additional commands, e-mail: mailto:[EMAIL PROTECTED] | | | -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: nbio connector
Remy Maucherat wrote: With my design, you still need one thread/request but only for the time required to process container.invoke() In the real world, the servlets and JSPs are the thing which take by far the most time to complete, so I'm not sure you wouldn't end up spending a lot of time in the threaded part. There was a long discussion on the advanced-servlets list a while back. That discussion concluded that the NIO looked most promising for: - Cleanly waiting for persistent connections (which take up a connector thread but don't really need to) You can kinda sorta do this already with the existing java.io classes, but NIO is better. This is a relatively easy win. - Serving static content. It's not just the non-blocking io, serving from memory mapped files is a beguilingly easy (but maybe not always the best) way to improve file serving performance. Using NIO to reduce the time service() needed to tie up a thread is fun, but seemed unlikely to help very much in practice. And of course the optimized char conversion stuff, but that's obviously useful and therefore boring. I agree on both the points above. Especially the first one actually, since you can avoid wasting a thread on waiting, and use all of them to do active work if needed. For the second point, Catalina uses a servlet to do the static files serving. I suppose the connector could do some optimization and serve some resources itself, but that's a bit complex, and not as clean as the current solution. I like that difference between a fun optimization and a useful optimization :) At the moment, a lot of useful optimization work still has yet to be done. I'm pretty sure newesh versions of Jetty include a first pass at integrating NIO, so anybody that's interested might want to take a look. There's some fairly bogus (but still kinda interesting) benchmark results using NIO-like routines to serve static files at distributopia.com. (Sun says no NIO-bmarks until it's out of beta, so I used Matt Welsh's NBIO code) I didn't know. They're annoying ... Remy -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: nbio connector
My opinion on the nbio issue: I completely agree with Remy, the IO is not the issue at least for the current code and the near future. The char-byte conversions are far more expensive. JDK1.4 provides a very nice solution for the char conversions, however my tests show it to be comparable in speed ( i.e I couldn't notice any difference ) with the current 'hack' in tomcat.util.buf ( which uses normal Writer with some recycling and other tricks ). While I would love the clean 1.4 solution, I see no particular need or benefit - so I'll not spend any time on it, but if someone contributes that - I would be +1 to add it to tomcat ( optional, if 1.4 is detected ) Regarding 'sendfile' - as part of j-t-c we'll be using a significant chunk of APR ( the apache portable runtime ) - and it would be easy to add support for sendfile as well. This will work on any platform ( where Apache can run - that's probably = than java at least for servers ). I would rather use the APR code that is available on all VMs and provides the same feature. I'm not very sure about nbio - most of the time there's a lot of complexity ( and a different programming model, etc ) - and the benefits seems pretty small. We already spend most of the time in user code ( i.e. servlets), the container overhead is getting smaller and smaller - so improvements on this area are very unlikely to matter. But it's cool - so I'm +1 on it as long as it doesn't mess with the existing APIs ( i.e. it is implemented as a module, etc ). Costin -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: nbio connector
[EMAIL PROTECTED] wrote: I'm not very sure about nbio - most of the time there's a lot of complexity ( and a different programming model, etc ) Yes. - and the benefits seems pretty small. It depends. Using NIO, you can serve static content as fast or faster than Apache 1.3 in pure Java, which means one less reason to use a separate front-end web server. (Of course, performance isn't the only reason to run Apache as a front end to Tomcat, but it seems to be perceived as the biggest reason) -- Christopher St. John [EMAIL PROTECTED] DistribuTopia http://www.distributopia.com -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
