Re: Challenges for Java hosting
--- Preston L. Bannister [EMAIL PROTECTED] wrote: Again, the main question is which target you are trying to hit. Are you aiming at Java web hosting where a hosting service would offer Java to *all* their customers? Are you aiming at Java web hosting for any (or nearly any) existing Java web application? In the first case you are aiming at the niche where PHP has found great success. On a purely theoretical level, PHP as a solution should not exist. On a practical level, there are all sorts of reasons why (at present) Java is a PITA for web hosting services, and PHP is easy to accomodate. In this problem domain the vast majority of sites are very low usage. In the second case you are aiming at a much smaller niche. Playing games to keep a pool of JVMs with per-user-application in-memory state makes sense. This is almost certainly going to translate to greater trouble and complexity for the hosting service. More trouble means a higher priced service offered less often. If your aim to make Java ubiquitous (which I believe possible) then you want to target the first case. For low-traffic websites the probability of a GC during the process of a single request is low. Also - though I can't say I've looked at the latest GC implementations in detail, the trend (starting with generational GC) in algorithms was to not move long-lived stuff around. Taken together, it is reasonable to assume that GC is not going to have much (or any) impact. This is aiming at the niche where PHP is popular - a rather large niche. What you are looking for is the simplest thing that can possibly work. You don't need JVM enhancements. You don't need to add complexity in terms of managing pools (always fun). You don't need to worry about leaky abstractions. Odds are you end up with something simple and dead reliable for a hosting service to deploy. A hosting service I use has a good scheme with using multiple workers. Each Tomcat is it's own instance, and points to the domain folder where we have the common files and places for classes...shared...commonthe config files. They then have rules we have to follow as far as developing, or they'll let you know you need to change somethingsame would happen for PHP...some page using all the processor. But, other than that, we have our own directory, our own process for the JVM, and we have ssh shell access to the things we need, but overall nothing that could not be handled through a web interface. It works out really well, and we have several domains we support on it. Doesn't seem any more difficult than anything I've done with PHP other than the classes and folder and things and every now and then restarting the TC instance. Wade - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Challenges for Java hosting
Some hints, although I didn't go deeper into it and most of it might still be experimental: 1) Java Isolate API JSP-221 http://java.sys-con.com/read/99716.htm http://jcp.org/en/jsr/detail?id=121 2) Building on top Java Resource Consumption Management API JSR-284 http://jcp.org/en/jsr/detail?id=284 http://research.sun.com/techrep/2003/abstract-124.html http://research.sun.com/projects/barcelona/papers/middleware04.pdf 3) Sun's experimental JVM, implementing parts of it: http://java.sun.com/developer/technicalArticles/Programming/mvm/ https://mvm.dev.java.net/ [you'll need to get the JDK Researcher role for that last one via https://mvm.dev.java.net/] 4) Maybe 1) and 2) are things to discuss with project Harmony? - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Challenges for Java hosting
Interesting discussion. Ideas? Yes, and it's very simple - use fork()! The problem with fork() in the way you propose is Garbage Collection. A JVM can move objects and compact heap space during GC, this presents a problem that moving data unshare's that page. For a native executable Copy-on-Write provides an improvement on resource usage as memory allocation is static once allocated, but between two long running JVMs trying to share pages its not so attractive. The other problem of course is the amount of work a JVM does to load byte code, interpret it and ultimately optimize/compile it into machine code. This has additional memory overhead a native executable just does not have. The only thing in the data segment of the address space is data. With linux (maybe others) the native program (and shared libs) is demand page loaded from the EXE file on the system, the OS can just drop little used code pages knowing it can reload them from a file. While the concept is partially possible with a JVM its just not done. Any solution on this level is only going to be possible through major enhancements at the JVM level against what traditionally Java has been about. ByteCode and GarbageCollection really inhibit traditional approaches to the problem, maybe the GCJ project in the not to distant will be a faster moving more innovative vehicle for a Java like language than Sun Microsystems. So from that standpoint of throwing away ByteCode for CompiledCode, the dynamics of your multiple JVMs become much more like a traditional EXEs. I would envisage an ideal solution not too different from what you are saying. Java code compiled into fully native code and stored on disk, you can then properly share the classes as nature intended between multiple JVMs. Then exactly as you say you have a master JVM in control and could dynamically put web-apps into other JVM processes which could each dynamically tune their thread and memory usage (on the fly) and as a last resort shut down the process to cut it away when its been bad, without disrupting the container environment as a whole. This leads on to pooling JVMs to recycle the process to walk around memory leak problems. With your problem web-app you could make sure he always got his own JVM to run in, for the nicer web-apps you could put more than one into a slave JVM. You could make use of shared mapped pages between processes and file descriptor passing to hand off incoming connections. There is no reason the above can't be implemented on both Win32 and Linux in due course. There are no new ideas here. Darryl - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Challenges for Java hosting
Just some thoughts based on what Tim has mentioned. --- Tim Funk [EMAIL PROTECTED] wrote: I was thinking that too. A big problem with JVM's is memory leaks. I would say a big problem any application is memory leaks. C and C++ have the same types of issues...just a little different along with cleaned up pointers being referenced from other objects no causing access violations as they are now null(0) or junk. Profilers can be ran against both types of projects as long as you can find one for the given platform. Java is a little easier per the profiler interface. The easy solution is to restart tomcat. But that causes a period of downtime due to waiting for a restart. Why not make mod_ajp smarter or create a tomcat launcher where some parent process (apache, or the launcher) listens for requests from the public and uses AJP to have tomcat serve the requests. [The downside is added latenency] I like this idea. Something like this: Server !-- allowWebApplicationContextVM=false means META-INF/Context.xml vm attributes will be ignored or an error raised -- Host vm=true vmpath=...could allow different JVMs other than the Tomcat one to be used for a given Host..obviously the Tomcat version would have to support it allowWebApplicationContextVM=false Context vm=true vmpath=...could allow different JVMs other than the Tomcat one to be used for a given Context/ !-- default to false -- Context vm=false/ /Host /Server The parent process can spawn a new tomcat after some configurable time and kill the old one. By using the cluster code - the sessions can be synced before the old tomcat goes away. With the XML scheme above it would watch each JVM process and restart it after some time. This mechanism could also be used as a way to perform graceful restarts too where an web app upgrade is done and reloading webapps will cause memory leaks. Yes each process could then be killed off regardless of worrying about threads. JVM hooks can be used to start shutting down the process and the admin can give the shutdown/restart command of a given context or host a time limit and if the shutdown fails to take =X amount of time then the process will be killed. The developer will be responsible for only using daemon threads if they spawn their own threads. [Disclaimer: the above ramblings may not be based in reality] I think it is realistic...obviously it would take some reworking and a good amount of timeI mean this may not even be realistic, but it would certainly be more inline with a good hosting solution and make it more flexible. I think using a Tomcat launcher would be best and then filter AJP through the front process like happens now, and that process filter to it's sub processes using AJP because IMO one should be able to only use Tomcat and not have to use both httpd and Tomcat unless they just really want to ,and I think localhost TCP/IP loopback with AJP would be fine as it doesn't have the same overhead as TCP/IP over a network interface and will be handled in memory buffers, but being able to have the Tomcat front end communicate with other Tomcat front ends over AJP so one could define a host be ported through this instance to another instance on another machine would be nice as well so it's all clustered from the Tomcat configuration and all Tomcat basedworkers could be defined in the server configuration for a Host or down to the Context/Application level. I think it would make sense to have AJP not pool local loopback connections, and only pool Network interface connections because they are software based connections anyways and don't really have anything to do with an Ethernet (or what ever HW scheme) connection and a real network connection...they are much fastermaybe only reuse them if they reach a certain count as to help avoid running out of TCP/IP connections on the computer, but reduce that level and not reuse them unless traffic is very heavy. This way local loopback AJP can support more connections...and really how much overhead is avoided by pooling the local loopback connections (obviously it makes a huge difference for real connections)? -Tim Wade - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Challenges for Java hosting
Hey, Java/JSP and Tomcat for german hoster is a very bad story. For two year we start a tomcat 5.0 based spezial tomcat distibution for hosting. The Centaurus Platform has show that effectiv hosting is possible. Problem is to find hoster that use that package. Look at http://centaurus.sourceforge.net/ and see that we have create a very cool tomcat bundle. Centaurus features: Used a patched tomcat 5.0.27 full graphical and console installer package for LINUX/Windows You can have multiple installation profiles Very nice html admin/manager console for one instance Loading new Security Policy on demand without downtime every webapp can register there own policy part (admin can control this integration with rules) Integrate new host from templates without downtime Native Iintegration at os services with Java Service Wrapper Plugin concept for people to extends tomcat core functionality use of Mx4J Http adaptor with authorisation for remote control. At cvs head exist a not final tomcat 5.5 version. Regards Peter - Documentation only exist in german language. Am 06.04.2006 um 23:48 schrieb Leon Rosenberg: On 4/6/06, Preston L. Bannister [EMAIL PROTECTED] wrote: Define lightweight. :) only the basics you need for a webapp. no admin/manager, no clustering, no gadgets. To explain it: Besides large portals with own server farms and millions of hits, I often have small customers which get a dynamical website with some cms etc. The problem I had and still have, is that typical hosting providers (at least in germany) don't offer any support for java webapps, best you can get is support for jsps only which sucks. This is ugly, since the customer pays money for the webapp and asks me afterwards, why should he rent a complete server to host it. Therefore I started to rent servers myself, and re-rent it to the customer. The customer gets the complete package, mail, backup, ftp/ssh access and the webapp. To ensure this, each server has an apache running with two jsp container instances on it, one for production, one for testing versions. The customer pays less than he would pay for professional hosting, and I can refinance the server with 3-4 customers. However, having all test-webapps in one server, and needing to restart it from time to time isn't really cool. I'd prefer to give each customer two instances, which consumes low resources, maybe even multiple tomcat instances in one jvm (is that possible?), and keep them independent from each other. Therefore lightweight. And therefore pre-configured :-) regards Leon If we are talking about a small number of users, with high average utilization, this might be a good solution. In fact this is similar in resource usage to the virtual hosting (i.e. Xen) solutions. For more typical usage, the number of users is large, and the average utilization is low. In this case one (very rarely used) JVM per user is somewhat expensive. Note you could reduce the expense with the same approach of using a fork() of a single image, expecting copy-on-write to radically reduce the real memory use (virtual memory use would be larger). Depends on what target you are trying to hit. The hosting world (by numbers of users) is dominated by very low usage sites. Is this a goal for Java/Tomcat hosting? If you can beat PHP in CGI mode *both* in performance and in resource usage, then you have a pretty compelling solution. If you are fatter or slower - this is going to disinterest a lot of hosting providers. Note that this notion is pretty much a non-starter if Linux does not do copy-on-write with fork(). This was a big deal back in the late 1980's (big Lisp apps forking vi). Don't know if this made it's way into Linux. I'm pretty sure copy-on-write in fork() was in SunOS, but I don't know about Solaris. On 4/6/06, Leon Rosenberg [EMAIL PROTECTED] wrote: isn't it easier to give each user a pre-configured lightweight but own tomcat? leon On 4/6/06, Preston L. Bannister [EMAIL PROTECTED] wrote: Well, that is one definition of real applications. There are other definitions. :) On 4/6/06, Tino Schwarze [EMAIL PROTECTED] wrote: On Thu, Apr 06, 2006 at 09:15:17AM -0700, Preston L. Bannister wrote: You have to consider how (or if) to allow for long-running background threads. Successive requests for the same user will not use the JVM (whether this counts as an advantage or disadvantage is debatable). The JVM isn't going to be optimizing code. The point of using an application server (instead of e.g. PHP) is that it maintains state on the server. You lose this by using fork (). So it's not going to work at all for real applications since your application returns to it's previous state after every request. Bye, Tino. -- --- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Challenges for Java hosting
Henri Gomez [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] 2006/4/6, Remy Maucherat [EMAIL PROTECTED]: Hi, This thread started (for whatever reason) on the private list as part of an unrelated discussion. The point is to see what could be improved to make Tomcat more suitable for shared hosting, which is a very nice goal, but unfortunately with very serious issues. I don't see many improvements possibilities, as I consider the following solutions and problems (where each user would at least need its own vhost): I see vhost as a client virtual server, where the 'client' is not a real / human end user but someone who ask you to host its applications with one or many webapps in it. - Every virtual host gets its own appBase folder. Having its own folder for JARs just won't work (or it means you were able to use the shared folder successfully, which I doubt). What do you mean ? Did we have actually such own folder of vhosts jars with its own classloader ? If not could it be done easily ? In theory, it should be possible to inject a parentClassLoader into the StandardHost to do this. I don't believe that StandardHost exposes this to JMX, so at the moment, you'd need to use introspection. And, I've never actually tried to do this myself ;-). Of course, the patch to enable doing it with JMX is trivial. Actually a client application get its classes from : - jars in common [lib, endorsed, i18n] and webapps lib directory - classes from common [ classes ] and webapps classes directory. Could we have instead a by vhost supplemental directory to get its own jar ? example : vhosta will grab in jars in /home/vhostsa/lib and common [lib, endorsed, i18n] and webapps lib directory vhostb will grab in jars in /home/vhostsb/lib and common [lib, endorsed, i18n] and webapps lib directory. In such case did the classloader for vhosta should be different from the one for vhostb. If you use the TOMCAT_HOME/TOMCAT_BASE stuff, each user can get its own shared folder. - There are still tons of JVMs to manage and monitor, which may be a problem. Why did there should be tons of JVM ? Couldn't we use a single JVM for each vhosts ? As many I prefer give more power and resource to a single JVM that spread thread and memory to many JVM where some could be at some time very less used than others. - If the connector should be shared, with the servlet containers running in separate processes, I don't see how to cross the process barrier except by going back to square one (httpd in front, with AJP and many JVMs/Tomcats, each with its own vhost). That's the current situation now, one Apache 2.x with multiples virtuals servers, each virtual server with its own JK/AJP worker to reach a specific Tomcat. Some general problems for production are: - No self tuning of the JVM. Of course but it's not better to have may JVM present in system with memory and thread/system resources use when for exemple users are only using one of the multiples JVM. Imagine for example users for differents country, said asia, europe and USA. You setup one JVM / Application for ASIA users, one for Europeans users and the third for US users. During the day, you'll see activity on Asian JVM, then European JVM and then on US JVM, but even when only one is really used, the 2 others are still in machine and consume memory and resources. Sad, this power could be better used for the really active JVM. - No actual isolation, throttling capabilities, etc, provided by the JVM. JVM or OS ? - Impossibility to control memory leaks (impossible to force discarding classloaders and all associated class defs and instances, for example). What do you means ? - Hard to do thread management (by that I mean, monitor and recover for threads stuck in loops or deadlocked). Well it's also hard today with a 'dedicated' JVM, the only solution may be to restart the JVM ... Any ideas ? Well if it could be done, I'm sure you're probably the best to imagine such solution. I suppose native code could be used to improve the situation in some areas (although I don't know how to do it ;) ). Native code ? Do you imagine some sort of sub-JVM launching ? Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Challenges for Java hosting
I was thinking that too. A big problem with JVM's is memory leaks. The easy solution is to restart tomcat. But that causes a period of downtime due to waiting for a restart. Why not make mod_ajp smarter or create a tomcat launcher where some parent process (apache, or the launcher) listens for requests from the public and uses AJP to have tomcat serve the requests. [The downside is added latenency] The parent process can spawn a new tomcat after some configurable time and kill the old one. By using the cluster code - the sessions can be synced before the old tomcat goes away. This mechanism could also be used as a way to perform graceful restarts too where an web app upgrade is done and reloading webapps will cause memory leaks. [Disclaimer: the above ramblings may not be based in reality] -Tim Henri Gomez wrote: I suppose native code could be used to improve the situation in some areas (although I don't know how to do it ;) ). Native code ? Do you imagine some sort of sub-JVM launching ? - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Challenges for Java hosting
Very interesting stuff My german is too bad but from what I could see it seems a good candidate. Could this works be ported back into ASF Tomcat 5.5.x ? Or better included in ASF Tomcat ? 2006/4/7, Peter Rossbach [EMAIL PROTECTED]: Hey, Java/JSP and Tomcat for german hoster is a very bad story. For two year we start a tomcat 5.0 based spezial tomcat distibution for hosting. The Centaurus Platform has show that effectiv hosting is possible. Problem is to find hoster that use that package. Look at http://centaurus.sourceforge.net/ and see that we have create a very cool tomcat bundle. Centaurus features: Used a patched tomcat 5.0.27 full graphical and console installer package for LINUX/Windows You can have multiple installation profiles Very nice html admin/manager console for one instance Loading new Security Policy on demand without downtime every webapp can register there own policy part (admin can control this integration with rules) Integrate new host from templates without downtime Native Iintegration at os services with Java Service Wrapper Plugin concept for people to extends tomcat core functionality use of Mx4J Http adaptor with authorisation for remote control. At cvs head exist a not final tomcat 5.5 version. Regards Peter - Documentation only exist in german language. Am 06.04.2006 um 23:48 schrieb Leon Rosenberg: On 4/6/06, Preston L. Bannister [EMAIL PROTECTED] wrote: Define lightweight. :) only the basics you need for a webapp. no admin/manager, no clustering, no gadgets. To explain it: Besides large portals with own server farms and millions of hits, I often have small customers which get a dynamical website with some cms etc. The problem I had and still have, is that typical hosting providers (at least in germany) don't offer any support for java webapps, best you can get is support for jsps only which sucks. This is ugly, since the customer pays money for the webapp and asks me afterwards, why should he rent a complete server to host it. Therefore I started to rent servers myself, and re-rent it to the customer. The customer gets the complete package, mail, backup, ftp/ssh access and the webapp. To ensure this, each server has an apache running with two jsp container instances on it, one for production, one for testing versions. The customer pays less than he would pay for professional hosting, and I can refinance the server with 3-4 customers. However, having all test-webapps in one server, and needing to restart it from time to time isn't really cool. I'd prefer to give each customer two instances, which consumes low resources, maybe even multiple tomcat instances in one jvm (is that possible?), and keep them independent from each other. Therefore lightweight. And therefore pre-configured :-) regards Leon If we are talking about a small number of users, with high average utilization, this might be a good solution. In fact this is similar in resource usage to the virtual hosting (i.e. Xen) solutions. For more typical usage, the number of users is large, and the average utilization is low. In this case one (very rarely used) JVM per user is somewhat expensive. Note you could reduce the expense with the same approach of using a fork() of a single image, expecting copy-on-write to radically reduce the real memory use (virtual memory use would be larger). Depends on what target you are trying to hit. The hosting world (by numbers of users) is dominated by very low usage sites. Is this a goal for Java/Tomcat hosting? If you can beat PHP in CGI mode *both* in performance and in resource usage, then you have a pretty compelling solution. If you are fatter or slower - this is going to disinterest a lot of hosting providers. Note that this notion is pretty much a non-starter if Linux does not do copy-on-write with fork(). This was a big deal back in the late 1980's (big Lisp apps forking vi). Don't know if this made it's way into Linux. I'm pretty sure copy-on-write in fork() was in SunOS, but I don't know about Solaris. On 4/6/06, Leon Rosenberg [EMAIL PROTECTED] wrote: isn't it easier to give each user a pre-configured lightweight but own tomcat? leon On 4/6/06, Preston L. Bannister [EMAIL PROTECTED] wrote: Well, that is one definition of real applications. There are other definitions. :) On 4/6/06, Tino Schwarze [EMAIL PROTECTED] wrote: On Thu, Apr 06, 2006 at 09:15:17AM -0700, Preston L. Bannister wrote: You have to consider how (or if) to allow for long-running background threads. Successive requests for the same user will not use the JVM (whether this counts as an advantage or disadvantage is debatable). The JVM isn't going to be optimizing code. The point of using an application server (instead of e.g. PHP) is that it maintains
Re: Challenges for Java hosting
2006/4/7, Bill Barker [EMAIL PROTECTED]: Henri Gomez [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] 2006/4/6, Remy Maucherat [EMAIL PROTECTED]: Hi, This thread started (for whatever reason) on the private list as part of an unrelated discussion. The point is to see what could be improved to make Tomcat more suitable for shared hosting, which is a very nice goal, but unfortunately with very serious issues. I don't see many improvements possibilities, as I consider the following solutions and problems (where each user would at least need its own vhost): I see vhost as a client virtual server, where the 'client' is not a real / human end user but someone who ask you to host its applications with one or many webapps in it. - Every virtual host gets its own appBase folder. Having its own folder for JARs just won't work (or it means you were able to use the shared folder successfully, which I doubt). What do you mean ? Did we have actually such own folder of vhosts jars with its own classloader ? If not could it be done easily ? In theory, it should be possible to inject a parentClassLoader into the StandardHost to do this. I don't believe that StandardHost exposes this to JMX, so at the moment, you'd need to use introspection. And, I've never actually tried to do this myself ;-). Of course, the patch to enable doing it with JMX is trivial. Good to see it could be possible. Next question will be could we have such functionality added in future 5.5.x versions ? - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Challenges for Java hosting
Hey Henri, no problem. I have talk with Thorsten Kamann the other author and we are happy to contribute the code. Then we can translate the complete docs and finish the tomcat 5.5 integration. Regards Peter Thorsten Am 07.04.2006 um 13:16 schrieb Henri Gomez: Very interesting stuff My german is too bad but from what I could see it seems a good candidate. Could this works be ported back into ASF Tomcat 5.5.x ? Or better included in ASF Tomcat ? 2006/4/7, Peter Rossbach [EMAIL PROTECTED]: Hey, Java/JSP and Tomcat for german hoster is a very bad story. For two year we start a tomcat 5.0 based spezial tomcat distibution for hosting. The Centaurus Platform has show that effectiv hosting is possible. Problem is to find hoster that use that package. Look at http://centaurus.sourceforge.net/ and see that we have create a very cool tomcat bundle. Centaurus features: Used a patched tomcat 5.0.27 full graphical and console installer package for LINUX/Windows You can have multiple installation profiles Very nice html admin/manager console for one instance Loading new Security Policy on demand without downtime every webapp can register there own policy part (admin can control this integration with rules) Integrate new host from templates without downtime Native Iintegration at os services with Java Service Wrapper Plugin concept for people to extends tomcat core functionality use of Mx4J Http adaptor with authorisation for remote control. At cvs head exist a not final tomcat 5.5 version. Regards Peter - Documentation only exist in german language. Am 06.04.2006 um 23:48 schrieb Leon Rosenberg: On 4/6/06, Preston L. Bannister [EMAIL PROTECTED] wrote: Define lightweight. :) only the basics you need for a webapp. no admin/manager, no clustering, no gadgets. To explain it: Besides large portals with own server farms and millions of hits, I often have small customers which get a dynamical website with some cms etc. The problem I had and still have, is that typical hosting providers (at least in germany) don't offer any support for java webapps, best you can get is support for jsps only which sucks. This is ugly, since the customer pays money for the webapp and asks me afterwards, why should he rent a complete server to host it. Therefore I started to rent servers myself, and re-rent it to the customer. The customer gets the complete package, mail, backup, ftp/ssh access and the webapp. To ensure this, each server has an apache running with two jsp container instances on it, one for production, one for testing versions. The customer pays less than he would pay for professional hosting, and I can refinance the server with 3-4 customers. However, having all test-webapps in one server, and needing to restart it from time to time isn't really cool. I'd prefer to give each customer two instances, which consumes low resources, maybe even multiple tomcat instances in one jvm (is that possible?), and keep them independent from each other. Therefore lightweight. And therefore pre-configured :-) regards Leon If we are talking about a small number of users, with high average utilization, this might be a good solution. In fact this is similar in resource usage to the virtual hosting (i.e. Xen) solutions. For more typical usage, the number of users is large, and the average utilization is low. In this case one (very rarely used) JVM per user is somewhat expensive. Note you could reduce the expense with the same approach of using a fork() of a single image, expecting copy-on-write to radically reduce the real memory use (virtual memory use would be larger). Depends on what target you are trying to hit. The hosting world (by numbers of users) is dominated by very low usage sites. Is this a goal for Java/Tomcat hosting? If you can beat PHP in CGI mode *both* in performance and in resource usage, then you have a pretty compelling solution. If you are fatter or slower - this is going to disinterest a lot of hosting providers. Note that this notion is pretty much a non-starter if Linux does not do copy-on-write with fork(). This was a big deal back in the late 1980's (big Lisp apps forking vi). Don't know if this made it's way into Linux. I'm pretty sure copy-on-write in fork() was in SunOS, but I don't know about Solaris. On 4/6/06, Leon Rosenberg [EMAIL PROTECTED] wrote: isn't it easier to give each user a pre-configured lightweight but own tomcat? leon On 4/6/06, Preston L. Bannister [EMAIL PROTECTED] wrote: Well, that is one definition of real applications. There are other definitions. :) On 4/6/06, Tino Schwarze [EMAIL PROTECTED] wrote: On Thu, Apr 06, 2006 at 09:15:17AM -0700, Preston L. Bannister wrote: You have to consider how (or if) to allow for long-running background threads. Successive requests for the same user will not use the JVM (whether this counts as an advantage or disadvantage is debatable). The JVM isn't
Re: Challenges for Java hosting
Hi List, good that you brought up this point. IMHO many people underestimated this subject in both importance _and_ difficulty. We know, shared java hosting can be done in 2 ways: a) one JVM per host b) many hosts in one JVM As Remy noted, the java machine does not provide the necessary isolation for solution (b). Security Managers do their work for security, but not for stabilty. Solution (a) is expensive. These fact facts have been standing for many years. But now, the situation is slightly different! In my opinion, we can use solution (a), if we utilize clustering. The PHP folks say, simply make the server stateless and everything is easy. True, but tomcat does not work that way. Now, with clustering, we could combine both. Consider the following: 1. Set up a couple of tomcat servers (at least 2). I call them 'node' These can sit on a single server 2. Cluster and load-balance these nodes. They should been seen as a single tomcat server . 3. Now use this cluster a in solution (b) above. 4. Each node should be monitored and if a situation arises, that compromises the node's stability, restart that node. (Memory nearly exhausted; endless looping threads; ...) 5. Another node should have the complete state of all applications on the failing node, so no interrupt occurs. 6. precondition: n-1 nodes must be able to operate all vhosts I see one only one enhancement, that must be done before: The cluster _and_ the load-balancer must be vhost-aware. With this, you could achive a RAID-like situation. It's an array of inexpensive tomcats :-) just my two cents R. Am Donnerstag, 6. April 2006 16:12 schrieb Remy Maucherat: Hi, This thread started (for whatever reason) on the private list as part of an unrelated discussion. The point is to see what could be improved to make Tomcat more suitable for shared hosting, which is a very nice goal, but unfortunately with very serious issues. I don't see many improvements possibilities, as I consider the following solutions and problems (where each user would at least need its own vhost): - Every virtual host gets its own appBase folder. Having its own folder for JARs just won't work (or it means you were able to use the shared folder successfully, which I doubt). If you use the TOMCAT_HOME/TOMCAT_BASE stuff, each user can get its own shared folder. - There are still tons of JVMs to manage and monitor, which may be a problem. - If the connector should be shared, with the servlet containers running in separate processes, I don't see how to cross the process barrier except by going back to square one (httpd in front, with AJP and many JVMs/Tomcats, each with its own vhost). Some general problems for production are: - No self tuning of the JVM. - No actual isolation, throttling capabilities, etc, provided by the JVM. - Impossibility to control memory leaks (impossible to force discarding classloaders and all associated class defs and instances, for example). - Hard to do thread management (by that I mean, monitor and recover for threads stuck in loops or deadlocked). Any ideas ? I suppose native code could be used to improve the situation in some areas (although I don't know how to do it ;) ). Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Reinhard Moosauer IT Beratung - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Challenges for Java hosting
Reinhard Moosauer wrote: Now, with clustering, we could combine both. Consider the following: 1. Set up a couple of tomcat servers (at least 2). I call them 'node' These can sit on a single server 2. Cluster and load-balance these nodes. They should been seen as a single tomcat server . 3. Now use this cluster a in solution (b) above. 4. Each node should be monitored and if a situation arises, that compromises the node's stability, restart that node. (Memory nearly exhausted; endless looping threads; ...) 5. Another node should have the complete state of all applications on the failing node, so no interrupt occurs. 6. precondition: n-1 nodes must be able to operate all vhosts I had thought about this solution. However, there are issues, as clustering is quite expensive (if there is more than one node, lol), and is incompatible with many applications. So for starters, clustering should be updated to add an option to not yell when a non-serializable attribute is added to a session, it should only operate with one node and then periodically (not too often, so that applications with do-it-yourself state will not break too often) start a second node, switch the traffic to it, and stop the first one. A similar process could also be done when redeploying things, etc. For regular hosting, this should provide good enough reliability (some state loss, but very decent availability). I don't think it's actually doable to host Java webapps which break every 3 requests (going into infinite loops or deadlocking, etc), however. Another downside is that the server needs to have CPU and memory headroom, since there will be a lot of full replication of sessions (during a node transition). Note: Tim stated the same concept earlier too. Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Challenges for Java hosting
Am Freitag, 7. April 2006 15:12 schrieb Remy Maucherat: Reinhard Moosauer wrote: Now, with clustering, we could combine both. Consider the following: 1. Set up a couple of tomcat servers (at least 2). I call them 'node' These can sit on a single server 2. Cluster and load-balance these nodes. They should been seen as a single tomcat server . 3. Now use this cluster a in solution (b) above. 4. Each node should be monitored and if a situation arises, that compromises the node's stability, restart that node. (Memory nearly exhausted; endless looping threads; ...) 5. Another node should have the complete state of all applications on the failing node, so no interrupt occurs. 6. precondition: n-1 nodes must be able to operate all vhosts I had thought about this solution. However, there are issues, as clustering is quite expensive (if there is more than one node, lol), and is incompatible with many applications. So for starters, clustering should be updated to add an option to not yell when a non-serializable attribute is added to a session, it should only operate with one node and then periodically (not too often, so that applications with do-it-yourself state will not break too often) start a second node, switch the traffic to it, and stop the first one. A similar process could also be done when redeploying things, etc. For regular hosting, this should provide good enough reliability (some state loss, but very decent availability). I don't think it's actually doable to host Java webapps which break every 3 requests (going into infinite loops or deadlocking, etc), however. Ok. But you can kill the webapp with the amok-thread. So we will not have a break every 3 requests. (the thread can be tracked down to the failing webapp. Send a mail with the thread-stack-dump) The ignore-not-serializable-sessions-objects thing would be very nice! Another downside is that the server needs to have CPU and memory headroom, since there will be a lot of full replication of sessions (during a node transition). Maybe it is still much better than the one-vhost-per-vm solution. I think for hosters with thousands of vhosts, this would be quite interesting. Note: Tim stated the same concept earlier too. Rémy What do you think about my suggestion to enhance the cluster? (vhost-node would no more be all-to-all, so you can optimize clusters with more than 2 nodes, which also reduces replication overhead) - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] Reinhard - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Challenges for Java hosting
Reinhard Moosauer wrote: Ok. But you can kill the webapp with the amok-thread. So we will not have a break every 3 requests. (the thread can be tracked down to the failing webapp. Send a mail with the thread-stack-dump) You can't really kill a thread in Java. We used to have a joke at a previous job that the only safe way to kill a thread in Java is System.exit(0). And as I recall, while the javadocs for Thread.stop() detail some of the reasons, there is much more badness possible than even what it states. -Paul - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Challenges for Java hosting
Ideas? Yes, and it's very simple - use fork()! Obviously this is not going to work on Windows, but for the case of Java hosting, we largely don't care. Let's put this in perspective. Quite a few hosting providors (mine included) run PHP in CGI mode. Any less expensive solution is competitive. The notion is very simple. Load a single master instance of the JVM. Pre-load lots of common classes (a list customized by the hosting service). Pre-initialize lots of classes (again a customized list). When a user request comes in, fork() a copy of the master instance (including possibly a chroot()). The child then processes the request and exits. The master instance basically does nothing but sit around and wait for fork() requests. Assuming that fork() on your version of Unix has a good implementation of copy-on-write (something I have not been able to verify easily), then the incremental real memory use should be *very* small. You have to consider how (or if) to allow for long-running background threads. Successive requests for the same user will not use the JVM (whether this counts as an advantage or disadvantage is debatable). The JVM isn't going to be optimizing code. But you get excellent isolation, no concern over memory leaks (and other left-over cruft), and you should (vastly) out-perform PHP in CGI mode. On 4/6/06, Remy Maucherat [EMAIL PROTECTED] wrote: Hi, This thread started (for whatever reason) on the private list as part of an unrelated discussion. The point is to see what could be improved to make Tomcat more suitable for shared hosting, which is a very nice goal, but unfortunately with very serious issues. I don't see many improvements possibilities, as I consider the following solutions and problems (where each user would at least need its own vhost): - Every virtual host gets its own appBase folder. Having its own folder for JARs just won't work (or it means you were able to use the shared folder successfully, which I doubt). If you use the TOMCAT_HOME/TOMCAT_BASE stuff, each user can get its own shared folder. - There are still tons of JVMs to manage and monitor, which may be a problem. - If the connector should be shared, with the servlet containers running in separate processes, I don't see how to cross the process barrier except by going back to square one (httpd in front, with AJP and many JVMs/Tomcats, each with its own vhost). Some general problems for production are: - No self tuning of the JVM. - No actual isolation, throttling capabilities, etc, provided by the JVM. - Impossibility to control memory leaks (impossible to force discarding classloaders and all associated class defs and instances, for example). - Hard to do thread management (by that I mean, monitor and recover for threads stuck in loops or deadlocked). Any ideas ? I suppose native code could be used to improve the situation in some areas (although I don't know how to do it ;) ). Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Challenges for Java hosting
On Thu, Apr 06, 2006 at 09:15:17AM -0700, Preston L. Bannister wrote: You have to consider how (or if) to allow for long-running background threads. Successive requests for the same user will not use the JVM (whether this counts as an advantage or disadvantage is debatable). The JVM isn't going to be optimizing code. The point of using an application server (instead of e.g. PHP) is that it maintains state on the server. You lose this by using fork(). So it's not going to work at all for real applications since your application returns to it's previous state after every request. Bye, Tino. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Challenges for Java hosting
Well, that is one definition of real applications. There are other definitions. :) On 4/6/06, Tino Schwarze [EMAIL PROTECTED] wrote: On Thu, Apr 06, 2006 at 09:15:17AM -0700, Preston L. Bannister wrote: You have to consider how (or if) to allow for long-running background threads. Successive requests for the same user will not use the JVM (whether this counts as an advantage or disadvantage is debatable). The JVM isn't going to be optimizing code. The point of using an application server (instead of e.g. PHP) is that it maintains state on the server. You lose this by using fork(). So it's not going to work at all for real applications since your application returns to it's previous state after every request. Bye, Tino. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Challenges for Java hosting
isn't it easier to give each user a pre-configured lightweight but own tomcat? leon On 4/6/06, Preston L. Bannister [EMAIL PROTECTED] wrote: Well, that is one definition of real applications. There are other definitions. :) On 4/6/06, Tino Schwarze [EMAIL PROTECTED] wrote: On Thu, Apr 06, 2006 at 09:15:17AM -0700, Preston L. Bannister wrote: You have to consider how (or if) to allow for long-running background threads. Successive requests for the same user will not use the JVM (whether this counts as an advantage or disadvantage is debatable). The JVM isn't going to be optimizing code. The point of using an application server (instead of e.g. PHP) is that it maintains state on the server. You lose this by using fork(). So it's not going to work at all for real applications since your application returns to it's previous state after every request. Bye, Tino. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Challenges for Java hosting
I have one suggestion regarding tomcat and security manager, but I don´t know if it fits here. We have a huge problem managing security configuration (i.e. catalina.policy). We have a common base policy and an entry for each virtual host. Sometimes clients put unmanaged libraries that require special permissions. We have to reinitialize the JVM to make these permissions take effect. Thanks Renato --- Remy Maucherat [EMAIL PROTECTED] wrote: Hi, This thread started (for whatever reason) on the private list as part of an unrelated discussion. The point is to see what could be improved to make Tomcat more suitable for shared hosting, which is a very nice goal, but unfortunately with very serious issues. I don't see many improvements possibilities, as I consider the following solutions and problems (where each user would at least need its own vhost): - Every virtual host gets its own appBase folder. Having its own folder for JARs just won't work (or it means you were able to use the shared folder successfully, which I doubt). If you use the TOMCAT_HOME/TOMCAT_BASE stuff, each user can get its own shared folder. - There are still tons of JVMs to manage and monitor, which may be a problem. - If the connector should be shared, with the servlet containers running in separate processes, I don't see how to cross the process barrier except by going back to square one (httpd in front, with AJP and many JVMs/Tomcats, each with its own vhost). Some general problems for production are: - No self tuning of the JVM. - No actual isolation, throttling capabilities, etc, provided by the JVM. - Impossibility to control memory leaks (impossible to force discarding classloaders and all associated class defs and instances, for example). - Hard to do thread management (by that I mean, monitor and recover for threads stuck in loops or deadlocked). Any ideas ? I suppose native code could be used to improve the situation in some areas (although I don't know how to do it ;) ). Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Challenges for Java hosting
Define lightweight. :) If we are talking about a small number of users, with high average utilization, this might be a good solution. In fact this is similar in resource usage to the virtual hosting (i.e. Xen) solutions. For more typical usage, the number of users is large, and the average utilization is low. In this case one (very rarely used) JVM per user is somewhat expensive. Note you could reduce the expense with the same approach of using a fork() of a single image, expecting copy-on-write to radically reduce the real memory use (virtual memory use would be larger). Depends on what target you are trying to hit. The hosting world (by numbers of users) is dominated by very low usage sites. Is this a goal for Java/Tomcat hosting? If you can beat PHP in CGI mode *both* in performance and in resource usage, then you have a pretty compelling solution. If you are fatter or slower - this is going to disinterest a lot of hosting providers. Note that this notion is pretty much a non-starter if Linux does not do copy-on-write with fork(). This was a big deal back in the late 1980's (big Lisp apps forking vi). Don't know if this made it's way into Linux. I'm pretty sure copy-on-write in fork() was in SunOS, but I don't know about Solaris. On 4/6/06, Leon Rosenberg [EMAIL PROTECTED] wrote: isn't it easier to give each user a pre-configured lightweight but own tomcat? leon On 4/6/06, Preston L. Bannister [EMAIL PROTECTED] wrote: Well, that is one definition of real applications. There are other definitions. :) On 4/6/06, Tino Schwarze [EMAIL PROTECTED] wrote: On Thu, Apr 06, 2006 at 09:15:17AM -0700, Preston L. Bannister wrote: You have to consider how (or if) to allow for long-running background threads. Successive requests for the same user will not use the JVM (whether this counts as an advantage or disadvantage is debatable). The JVM isn't going to be optimizing code. The point of using an application server (instead of e.g. PHP) is that it maintains state on the server. You lose this by using fork(). So it's not going to work at all for real applications since your application returns to it's previous state after every request. Bye, Tino. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Challenges for Java hosting
On 4/6/06, Preston L. Bannister [EMAIL PROTECTED] wrote: Define lightweight. :) only the basics you need for a webapp. no admin/manager, no clustering, no gadgets. To explain it: Besides large portals with own server farms and millions of hits, I often have small customers which get a dynamical website with some cms etc. The problem I had and still have, is that typical hosting providers (at least in germany) don't offer any support for java webapps, best you can get is support for jsps only which sucks. This is ugly, since the customer pays money for the webapp and asks me afterwards, why should he rent a complete server to host it. Therefore I started to rent servers myself, and re-rent it to the customer. The customer gets the complete package, mail, backup, ftp/ssh access and the webapp. To ensure this, each server has an apache running with two jsp container instances on it, one for production, one for testing versions. The customer pays less than he would pay for professional hosting, and I can refinance the server with 3-4 customers. However, having all test-webapps in one server, and needing to restart it from time to time isn't really cool. I'd prefer to give each customer two instances, which consumes low resources, maybe even multiple tomcat instances in one jvm (is that possible?), and keep them independent from each other. Therefore lightweight. And therefore pre-configured :-) regards Leon If we are talking about a small number of users, with high average utilization, this might be a good solution. In fact this is similar in resource usage to the virtual hosting (i.e. Xen) solutions. For more typical usage, the number of users is large, and the average utilization is low. In this case one (very rarely used) JVM per user is somewhat expensive. Note you could reduce the expense with the same approach of using a fork() of a single image, expecting copy-on-write to radically reduce the real memory use (virtual memory use would be larger). Depends on what target you are trying to hit. The hosting world (by numbers of users) is dominated by very low usage sites. Is this a goal for Java/Tomcat hosting? If you can beat PHP in CGI mode *both* in performance and in resource usage, then you have a pretty compelling solution. If you are fatter or slower - this is going to disinterest a lot of hosting providers. Note that this notion is pretty much a non-starter if Linux does not do copy-on-write with fork(). This was a big deal back in the late 1980's (big Lisp apps forking vi). Don't know if this made it's way into Linux. I'm pretty sure copy-on-write in fork() was in SunOS, but I don't know about Solaris. On 4/6/06, Leon Rosenberg [EMAIL PROTECTED] wrote: isn't it easier to give each user a pre-configured lightweight but own tomcat? leon On 4/6/06, Preston L. Bannister [EMAIL PROTECTED] wrote: Well, that is one definition of real applications. There are other definitions. :) On 4/6/06, Tino Schwarze [EMAIL PROTECTED] wrote: On Thu, Apr 06, 2006 at 09:15:17AM -0700, Preston L. Bannister wrote: You have to consider how (or if) to allow for long-running background threads. Successive requests for the same user will not use the JVM (whether this counts as an advantage or disadvantage is debatable). The JVM isn't going to be optimizing code. The point of using an application server (instead of e.g. PHP) is that it maintains state on the server. You lose this by using fork(). So it's not going to work at all for real applications since your application returns to it's previous state after every request. Bye, Tino. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Challenges for Java hosting
Write you own SecurityManager. http://java.sun.com/docs/books/tutorial/essential/system/writingSMgr.html I have one suggestion regarding tomcat and security manager, but I don´t know if it fits here. We have a huge problem managing security configuration (i.e. catalina.policy). We have a common base policy and an entry for each virtual host. Sometimes clients put unmanaged libraries that require special permissions. We have to reinitialize the JVM to make these permissions take effect. Thanks Renato --- Remy Maucherat [EMAIL PROTECTED] wrote: Hi, This thread started (for whatever reason) on the private list as part of an unrelated discussion. The point is to see what could be improved to make Tomcat more suitable for shared hosting, which is a very nice goal, but unfortunately with very serious issues. I don't see many improvements possibilities, as I consider the following solutions and problems (where each user would at least need its own vhost): - Every virtual host gets its own appBase folder. Having its own folder for JARs just won't work (or it means you were able to use the shared folder successfully, which I doubt). If you use the TOMCAT_HOME/TOMCAT_BASE stuff, each user can get its own shared folder. - There are still tons of JVMs to manage and monitor, which may be a problem. - If the connector should be shared, with the servlet containers running in separate processes, I don't see how to cross the process barrier except by going back to square one (httpd in front, with AJP and many JVMs/Tomcats, each with its own vhost). Some general problems for production are: - No self tuning of the JVM. - No actual isolation, throttling capabilities, etc, provided by the JVM. - Impossibility to control memory leaks (impossible to force discarding classloaders and all associated class defs and instances, for example). - Hard to do thread management (by that I mean, monitor and recover for threads stuck in loops or deadlocked). Any ideas ? I suppose native code could be used to improve the situation in some areas (although I don't know how to do it ;) ). Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] smime.p7s Description: S/MIME Cryptographic Signature
