RE: in search of more efficient design
There has been changed something in 1.4.1. Can't tell you the bug ID, but a quote from http://java.sun.com/j2se/1.4.1/changes.html: The compiler now releases the pointers to its internal data structures after compilation completes, so that using javac inside a java program will not leak space. -Original Message- From: Chris McCabe [mailto:[EMAIL PROTECTED]] Sent: Thursday, December 19, 2002 11:21 PM To: Tomcat Users List Subject: Re: in search of more efficient design but with 1.4 the memory usage would steadily increase until it reached the max and then it would die with out of memory exceptions. I am hoping this was fixed in 1.4.1, but I have not seen any specific bug report on the problem. -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: in search of more efficient design
That depend very much on the operation system, and the way the vm is implemented. The default behavior for many operating systems is that the memory isn't shrinking. The freed spaces is just free inside the memory of the vm. (The last time I had to deal with solaris back in 98 the normal malloc/free cycles didn't give back the memory to the os. To implement that, you had to use memory mapped files or use special libraries.) -Original Message- From: Rick Fincher [mailto:[EMAIL PROTECTED]] Sent: Thursday, December 19, 2002 11:02 PM To: Tomcat Users List Subject: Re: in search of more efficient design Is this correct? My JVM memory use expands and contracts all the time during a Tomcat run. The JVM will get bigger until it hits the memory limits set up at start time, then it will GC and get smaller, according to the OS anyway. This is on Solaris but I can't imagine that they would make the JVM such a memory pig on other OS's. Early versions maybe, but not now. -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: in search of more efficient design
-Original Message- From: ilasno [mailto:[EMAIL PROTECTED]] Sent: Thursday, December 19, 2002 8:29 PM To: [EMAIL PROTECTED] Subject: in search of more efficient design PREAMBLE - I apologize if this is off-topic. I have been on this search for a month, and have searched the web far and wide, but have mostly found either J2EE design recommendations (higher-level than i am prepared to explore right now) or software vendors trying to sell me load-testing programs. I would appreciate pointers to self-enlightenment sites almost as much as more specific guidance. SHORT VERSION - I am a beginner-intermediate webapp designer, and my first large-scale development is using around 500 mb of memory, with low client load. I am seeking ways to both measure where this extreme resource-usage is stemming from and design tips to bring it down. LONG VERSION - The site I am developing is for a company that wishes to be able to change/update portions of their site through a web-based private section of their site. The result is an extensive admin implemented through a combination of jsp pages (for forms, and feedback) and corresponding servlets (for processing and database updating through db-access modules). The admin section is obviously much bigger and more resource-intensive than the public portion, which just hits the database once-an-hour for updates and displays the current data. Conversely, while the admin section is much larger (many more servlets with larger file sizes, more processing), it is obviously used much less (one or two users per day as opposed to many per day for the public portion). I have steered away from keeping variables global within servlets (including db-access module instances), so that within a doGet or doPost everything i need is created, and then i am assuming when the method ends everything is garbage-collected? is it possible to have that much memory used just to hold 60 or 70 servlets ready for requests? I am wondering if my design is flawed, or a bad idea altogether.. any help or guidance would be greatly appreciated. jesse -- i am deprogrammed -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: in search of more efficient design
Although your question is rather vage I'll try to give some hints on the same vage level. If you are using a JDK 1.2.* and 1.4.1 you should do one of the following: - upgrade to 1.4.1 - use precompiled jsp's - use jikes to complie the pages - Just restart tomcat after all jsp have been compiled This is not a medium term solution, but may be approriate to isolate the resource leaks Make damn shure that you really release all db resources at the end of the methods. Several JDBC drivers are missbehaving if you don't explicitly close all statements, result sets, and connections. Make shure that the closing is executed no matter what happend in the method (Do all closing in a finally block off a try statement). Watch for Objects that are stored variables that are stored outside of methods. (Class/Instance/Session/Context/Application) Although I think you don't like the last answer (please read to the end) I have to provide it: Use a tool like OptimizeIt to find out, where the memory is consumed. The last time I looked, they had 30 days trial version, so you may not need to buy it. -Original Message- From: ilasno [mailto:[EMAIL PROTECTED]] Sent: Thursday, December 19, 2002 8:29 PM To: [EMAIL PROTECTED] Subject: in search of more efficient design (including db-access module instances), so that within a doGet or doPost everything i need is created, and then i am assuming when the method ends everything is garbage-collected? is it possible to have that much memory used just to hold 60 or 70 servlets ready for requests? I am wondering if my design is flawed, or a bad idea altogether.. any help or guidance would be greatly appreciated. jesse -- i am deprogrammed -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: in search of more efficient design
Why use jikes? It would make any difference to the final performance of my application? I tried in vain to use Ant to precompile my JSPs :-( On Fri, 2002-12-20 at 07:17, Ralph Einfeldt wrote: Although your question is rather vage I'll try to give some hints on the same vage level. If you are using a JDK 1.2.* and 1.4.1 you should do one of the following: - upgrade to 1.4.1 - use precompiled jsp's - use jikes to complie the pages - Just restart tomcat after all jsp have been compiled This is not a medium term solution, but may be approriate to isolate the resource leaks Make damn shure that you really release all db resources at the end of the methods. Several JDBC drivers are missbehaving if you don't explicitly close all statements, result sets, and connections. Make shure that the closing is executed no matter what happend in the method (Do all closing in a finally block off a try statement). Watch for Objects that are stored variables that are stored outside of methods. (Class/Instance/Session/Context/Application) Although I think you don't like the last answer (please read to the end) I have to provide it: Use a tool like OptimizeIt to find out, where the memory is consumed. The last time I looked, they had 30 days trial version, so you may not need to buy it. -Original Message- From: ilasno [mailto:[EMAIL PROTECTED]] Sent: Thursday, December 19, 2002 8:29 PM To: [EMAIL PROTECTED] Subject: in search of more efficient design (including db-access module instances), so that within a doGet or doPost everything i need is created, and then i am assuming when the method ends everything is garbage-collected? is it possible to have that much memory used just to hold 60 or 70 servlets ready for requests? I am wondering if my design is flawed, or a bad idea altogether.. any help or guidance would be greatly appreciated. jesse -- i am deprogrammed -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- Felipe Schnack Analista de Sistemas [EMAIL PROTECTED] Cel.: (51)91287530 Linux Counter #281893 Centro Universitário Ritter dos Reis http://www.ritterdosreis.br [EMAIL PROTECTED] Fone/Fax.: (51)32303341 -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: in search of more efficient design
As javac has some memory leaks that affect the cbefore 1.4.1 it's an option to use jikes instead if you can't upgrade to JDK 1.4.1. (There may be many reasons for that). Beside solving the memory problem Jikes will not affect the performance of your application significant. -Original Message- From: Felipe Schnack [mailto:[EMAIL PROTECTED]] Sent: Friday, December 20, 2002 12:06 PM To: Tomcat Users List Subject: RE: in search of more efficient design Why use jikes? It would make any difference to the final performance of my application? If you are using a JDK 1.2.* and 1.4.1 you should do one of the following: - upgrade to 1.4.1 - use precompiled jsp's - use jikes to complie the pages - Just restart tomcat after all jsp have been compiled This is not a medium term solution, but may be approriate to isolate the resource leaks -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: in search of more efficient design
One word... struts -Original Message- From: ilasno [mailto:[EMAIL PROTECTED]] Sent: Thursday, December 19, 2002 1:29 PM To: [EMAIL PROTECTED] Subject: in search of more efficient design PREAMBLE - I apologize if this is off-topic. I have been on this search for a month, and have searched the web far and wide, but have mostly found either J2EE design recommendations (higher-level than i am prepared to explore right now) or software vendors trying to sell me load-testing programs. I would appreciate pointers to self-enlightenment sites almost as much as more specific guidance. SHORT VERSION - I am a beginner-intermediate webapp designer, and my first large-scale development is using around 500 mb of memory, with low client load. I am seeking ways to both measure where this extreme resource-usage is stemming from and design tips to bring it down. LONG VERSION - The site I am developing is for a company that wishes to be able to change/update portions of their site through a web-based private section of their site. The result is an extensive admin implemented through a combination of jsp pages (for forms, and feedback) and corresponding servlets (for processing and database updating through db-access modules). The admin section is obviously much bigger and more resource-intensive than the public portion, which just hits the database once-an-hour for updates and displays the current data. Conversely, while the admin section is much larger (many more servlets with larger file sizes, more processing), it is obviously used much less (one or two users per day as opposed to many per day for the public portion). I have steered away from keeping variables global within servlets (including db-access module instances), so that within a doGet or doPost everything i need is created, and then i am assuming when the method ends everything is garbage-collected? is it possible to have that much memory used just to hold 60 or 70 servlets ready for requests? I am wondering if my design is flawed, or a bad idea altogether.. any help or guidance would be greatly appreciated. jesse -- i am deprogrammed -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: in search of more efficient design
Maybe a bit more than one word would be more helpful. ;-) Instead of having 60 - 70 servlet classes, you could have one servlet with 60 - 70 delegates that are loaded on demand. Struts includes the main servlet, and basically consists of a framework for creating and invoking those delegate classes. I'm not sure, though, whether they're actually loaded (and unloaded) on demand, or whether it loads on startup and keeps them all in memory. I have a feeling that it's half half: loads on demand, but then keeps them in memory until the webapp is restarted. Maybe Craig can comment. It's also worth noting that the memory freed from garbage collecting objects in Java isn't returned to the OS, but kept as the VMs free internal heap space. So just because the VM shows that it's using 500M of space doesn't mean that there are actually 500M of objects in memory. If there was high load at some earlier point that pushed it up to 500, it will stay there until you restart the VM. I don't know if that's relevant to your case, though. -- Tim Moore / Blackboard Inc. / Software Engineer 1899 L Street, NW / 5th Floor / Washington, DC 20036 Phone 202-463-4860 ext. 258 / Fax 202-463-4863 -Original Message- From: Jacob Hookom [mailto:[EMAIL PROTECTED]] Sent: Thursday, December 19, 2002 3:07 PM To: 'Tomcat Users List' Subject: RE: in search of more efficient design One word... struts -Original Message- From: ilasno [mailto:[EMAIL PROTECTED]] Sent: Thursday, December 19, 2002 1:29 PM To: [EMAIL PROTECTED] Subject: in search of more efficient design PREAMBLE - I apologize if this is off-topic. I have been on this search for a month, and have searched the web far and wide, but have mostly found either J2EE design recommendations (higher-level than i am prepared to explore right now) or software vendors trying to sell me load-testing programs. I would appreciate pointers to self-enlightenment sites almost as much as more specific guidance. SHORT VERSION - I am a beginner-intermediate webapp designer, and my first large-scale development is using around 500 mb of memory, with low client load. I am seeking ways to both measure where this extreme resource-usage is stemming from and design tips to bring it down. LONG VERSION - The site I am developing is for a company that wishes to be able to change/update portions of their site through a web-based private section of their site. The result is an extensive admin implemented through a combination of jsp pages (for forms, and feedback) and corresponding servlets (for processing and database updating through db-access modules). The admin section is obviously much bigger and more resource-intensive than the public portion, which just hits the database once-an-hour for updates and displays the current data. Conversely, while the admin section is much larger (many more servlets with larger file sizes, more processing), it is obviously used much less (one or two users per day as opposed to many per day for the public portion). I have steered away from keeping variables global within servlets (including db-access module instances), so that within a doGet or doPost everything i need is created, and then i am assuming when the method ends everything is garbage-collected? is it possible to have that much memory used just to hold 60 or 70 servlets ready for requests? I am wondering if my design is flawed, or a bad idea altogether.. any help or guidance would be greatly appreciated. jesse -- i am deprogrammed -- To unsubscribe, e-mail: mailto:tomcat-user- [EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:tomcat-user- [EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: in search of more efficient design
I thought 1 vs many servlets were brought up a while back (possibly late July), the conclusion was that it was VM dependent if I remember correctly. -Original Message- From: Tim Moore [mailto:[EMAIL PROTECTED]] Sent: Thursday, December 19, 2002 2:17 PM To: Tomcat Users List Subject: RE: in search of more efficient design Maybe a bit more than one word would be more helpful. ;-) Instead of having 60 - 70 servlet classes, you could have one servlet with 60 - 70 delegates that are loaded on demand. Struts includes the main servlet, and basically consists of a framework for creating and invoking those delegate classes. I'm not sure, though, whether they're actually loaded (and unloaded) on demand, or whether it loads on startup and keeps them all in memory. I have a feeling that it's half half: loads on demand, but then keeps them in memory until the webapp is restarted. Maybe Craig can comment. It's also worth noting that the memory freed from garbage collecting objects in Java isn't returned to the OS, but kept as the VMs free internal heap space. So just because the VM shows that it's using 500M of space doesn't mean that there are actually 500M of objects in memory. If there was high load at some earlier point that pushed it up to 500, it will stay there until you restart the VM. I don't know if that's relevant to your case, though. -- Tim Moore / Blackboard Inc. / Software Engineer 1899 L Street, NW / 5th Floor / Washington, DC 20036 Phone 202-463-4860 ext. 258 / Fax 202-463-4863 -Original Message- From: Jacob Hookom [mailto:[EMAIL PROTECTED]] Sent: Thursday, December 19, 2002 3:07 PM To: 'Tomcat Users List' Subject: RE: in search of more efficient design One word... struts -Original Message- From: ilasno [mailto:[EMAIL PROTECTED]] Sent: Thursday, December 19, 2002 1:29 PM To: [EMAIL PROTECTED] Subject: in search of more efficient design PREAMBLE - I apologize if this is off-topic. I have been on this search for a month, and have searched the web far and wide, but have mostly found either J2EE design recommendations (higher-level than i am prepared to explore right now) or software vendors trying to sell me load-testing programs. I would appreciate pointers to self-enlightenment sites almost as much as more specific guidance. SHORT VERSION - I am a beginner-intermediate webapp designer, and my first large-scale development is using around 500 mb of memory, with low client load. I am seeking ways to both measure where this extreme resource-usage is stemming from and design tips to bring it down. LONG VERSION - The site I am developing is for a company that wishes to be able to change/update portions of their site through a web-based private section of their site. The result is an extensive admin implemented through a combination of jsp pages (for forms, and feedback) and corresponding servlets (for processing and database updating through db-access modules). The admin section is obviously much bigger and more resource-intensive than the public portion, which just hits the database once-an-hour for updates and displays the current data. Conversely, while the admin section is much larger (many more servlets with larger file sizes, more processing), it is obviously used much less (one or two users per day as opposed to many per day for the public portion). I have steered away from keeping variables global within servlets (including db-access module instances), so that within a doGet or doPost everything i need is created, and then i am assuming when the method ends everything is garbage-collected? is it possible to have that much memory used just to hold 60 or 70 servlets ready for requests? I am wondering if my design is flawed, or a bad idea altogether.. any help or guidance would be greatly appreciated. jesse -- i am deprogrammed -- To unsubscribe, e-mail: mailto:tomcat-user- [EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:tomcat-user- [EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: in search of more efficient design
-Original Message- From: Jacob Hookom [mailto:[EMAIL PROTECTED]] Sent: Thursday, December 19, 2002 3:51 PM To: 'Tomcat Users List' Subject: RE: in search of more efficient design I thought 1 vs many servlets were brought up a while back (possibly late July), the conclusion was that it was VM dependent if I remember correctly. Probably appserver dependent. The question is really whether the infrequently used classes are unloaded from memory after use, because AFAIK no containers load servlets before they're first invoked unless you specify that they load on startup. With servlets, they may be unloaded after some period of inactivity, but that will be container-dependent and I don't think Tomcat supports that. With struts actions, I believe the answer is no; once an action is loaded, it's in memory until the ActionServlet is destroyed. You could create your own ActionServlet subclass, however, that throws out its action cache after a while. -- Tim Moore / Blackboard Inc. / Software Engineer 1899 L Street, NW / 5th Floor / Washington, DC 20036 Phone 202-463-4860 ext. 258 / Fax 202-463-4863 -Original Message- From: Tim Moore [mailto:[EMAIL PROTECTED]] Sent: Thursday, December 19, 2002 2:17 PM To: Tomcat Users List Subject: RE: in search of more efficient design Maybe a bit more than one word would be more helpful. ;-) Instead of having 60 - 70 servlet classes, you could have one servlet with 60 - 70 delegates that are loaded on demand. Struts includes the main servlet, and basically consists of a framework for creating and invoking those delegate classes. I'm not sure, though, whether they're actually loaded (and unloaded) on demand, or whether it loads on startup and keeps them all in memory. I have a feeling that it's half half: loads on demand, but then keeps them in memory until the webapp is restarted. Maybe Craig can comment. It's also worth noting that the memory freed from garbage collecting objects in Java isn't returned to the OS, but kept as the VMs free internal heap space. So just because the VM shows that it's using 500M of space doesn't mean that there are actually 500M of objects in memory. If there was high load at some earlier point that pushed it up to 500, it will stay there until you restart the VM. I don't know if that's relevant to your case, though. -- Tim Moore / Blackboard Inc. / Software Engineer 1899 L Street, NW / 5th Floor / Washington, DC 20036 Phone 202-463-4860 ext. 258 / Fax 202-463-4863 -Original Message- From: Jacob Hookom [mailto:[EMAIL PROTECTED]] Sent: Thursday, December 19, 2002 3:07 PM To: 'Tomcat Users List' Subject: RE: in search of more efficient design One word... struts -Original Message- From: ilasno [mailto:[EMAIL PROTECTED]] Sent: Thursday, December 19, 2002 1:29 PM To: [EMAIL PROTECTED] Subject: in search of more efficient design PREAMBLE - I apologize if this is off-topic. I have been on this search for a month, and have searched the web far and wide, but have mostly found either J2EE design recommendations (higher-level than i am prepared to explore right now) or software vendors trying to sell me load-testing programs. I would appreciate pointers to self-enlightenment sites almost as much as more specific guidance. SHORT VERSION - I am a beginner-intermediate webapp designer, and my first large-scale development is using around 500 mb of memory, with low client load. I am seeking ways to both measure where this extreme resource-usage is stemming from and design tips to bring it down. LONG VERSION - The site I am developing is for a company that wishes to be able to change/update portions of their site through a web-based private section of their site. The result is an extensive admin implemented through a combination of jsp pages (for forms, and feedback) and corresponding servlets (for processing and database updating through db-access modules). The admin section is obviously much bigger and more resource-intensive than the public portion, which just hits the database once-an-hour for updates and displays the current data. Conversely, while the admin section is much larger (many more servlets with larger file sizes, more processing), it is obviously used much less (one or two users per day as opposed to many per day for the public portion). I have steered away from keeping variables global within servlets (including db-access module instances), so that within a doGet or doPost everything i need is created, and then i am assuming when the method ends everything is garbage-collected? is it possible to have that much memory used just to hold 60 or 70 servlets ready for requests? I am wondering if my design is flawed, or a bad idea altogether.. any help or guidance
Re: in search of more efficient design
From: ilasno [EMAIL PROTECTED] Sent: Thursday, December 19, 2002 11:29 AM Subject: in search of more efficient design SHORT VERSION - I am a beginner-intermediate webapp designer, and my first large-scale development is using around 500 mb of memory, with low client load. I am seeking ways to both measure where this extreme resource-usage is stemming from and design tips to bring it down. Others have mentioned that Java doesn't return memory to the OS after it is GC'd, and this is true. So the question is how is the application using its memory? You can certainly do some simple analysis by running through some use case scenarios of you app, and monitoring GC activity from Java. This will give you some idea where your memory may be going. For example, run your application with the -verbose:gc option and you'll see some information every time the jvm runs garbage collection. What it prints out will be some basic heap information. Start with a small heap (java -Xms32M say), and everytime it needs to expand the heap, it will GC and print out a line. You can also use the java -Xrunhprof options to monitor the heap. But, basically, start the app, and run through some simple, common scenarios (user viewing the site, updating the site, whatever), and see if the memory use surges. If you notice that when a user updates the site your memory jumps from 64MB to 500MB, then that should give you an idea on where to look. Perhaps you could change your code to reuse some object versus recreating them. But, for all I know one of the things that your site does is upload large digital images and you're simply stuck with the memory cost of manipulating them. If you application is reasonably factored without a lot of redundant code, then your servlets per se are probably not the direct culprit in using up your memory. Obviously their actions affect memory, but a servlet is simply an object and doesn't have dramatic overhead in the container. You mention how all of your work is done in the doGet methods, but perhaps there can be some efficiencies gained by using some global Singleton caches. If your application is constantly loading reasonably static data from the DB and then throwing it away, that's just a waster of CPU and memory if it could effectively be loaded once and then used several times. Struts (as was elsewhere suggested) is a fine tool, but if you simply converted your application over to the struts framerwork, and kept your core code the same, then you'd still be using a lot of memory I would bet. I wouldn't throw the baby out with the bathwater and completely reengineer the site right off. You first need to understand where the resources are going so you can decided whether an in place fix is appropriate or the problem is so fundamental that you'd need to restart from scratch. Finally, if it's practical or necessary, you could run the admin features on one machine, and have the production users run on another machine. Since the production users only need access to the database, there's no real requirement that one machine support both the consumers and content producers. Regards, Will Hartung ([EMAIL PROTECTED]) -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: in search of more efficient design
Is this correct? My JVM memory use expands and contracts all the time during a Tomcat run. The JVM will get bigger until it hits the memory limits set up at start time, then it will GC and get smaller, according to the OS anyway. This is on Solaris but I can't imagine that they would make the JVM such a memory pig on other OS's. Early versions maybe, but not now. Rick - Original Message - snip It's also worth noting that the memory freed from garbage collecting objects in Java isn't returned to the OS, but kept as the VMs free internal heap space. snip -- Tim Moore / Blackboard Inc. / Software Engineer 1899 L Street, NW / 5th Floor / Washington, DC 20036 Phone 202-463-4860 ext. 258 / Fax 202-463-4863 -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: in search of more efficient design
-Original Message- From: Rick Fincher [mailto:[EMAIL PROTECTED]] Sent: Thursday, December 19, 2002 5:02 PM To: Tomcat Users List Subject: Re: in search of more efficient design Is this correct? My JVM memory use expands and contracts all the time during a Tomcat run. The JVM will get bigger until it hits the memory limits set up at start time, then it will GC and get smaller, according to the OS anyway. This is on Solaris but I can't imagine that they would make the JVM such a memory pig on other OS's. Early versions maybe, but not now. I could be wrong, but I've seen it repeated often enough on here that I guess I took it as gospel! ;-) Well, at best then, it varies based on the specific JVM you're using. In general, I'd say you can't assume that all of the memory used by the JVM process is actually in use by the application, and you especially can't assume at any point that the GC has freed all of the memory that it could. -- Tim Moore / Blackboard Inc. / Software Engineer 1899 L Street, NW / 5th Floor / Washington, DC 20036 Phone 202-463-4860 ext. 258 / Fax 202-463-4863 Rick - Original Message - snip It's also worth noting that the memory freed from garbage collecting objects in Java isn't returned to the OS, but kept as the VMs free internal heap space. snip -- Tim Moore / Blackboard Inc. / Software Engineer 1899 L Street, NW / 5th Floor / Washington, DC 20036 Phone 202-463-4860 ext. 258 / Fax 202-463-4863 -- To unsubscribe, e-mail: mailto:tomcat-user- [EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: in search of more efficient design
We ran into a memory leak with JDK1.4 on Solaris, and could not figure out exactly where it was occuring. The exact same setup with 1.2.2 can run for weeks with constant memory usage, but with 1.4 the memory usage would steadily increase until it reached the max and then it would die with out of memory exceptions. I am hoping this was fixed in 1.4.1, but I have not seen any specific bug report on the problem. The interesting thing is that if you turn on memory profiling, the leak does not occur, so trying to narrow down the problem is very difficult. Chris Rick Fincher wrote: Is this correct? My JVM memory use expands and contracts all the time during a Tomcat run. The JVM will get bigger until it hits the memory limits set up at start time, then it will GC and get smaller, according to the OS anyway. This is on Solaris but I can't imagine that they would make the JVM such a memory pig on other OS's. Early versions maybe, but not now. Rick - Original Message - snip It's also worth noting that the memory freed from garbage collecting objects in Java isn't returned to the OS, but kept as the VMs free internal heap space. snip -- Tim Moore / Blackboard Inc. / Software Engineer 1899 L Street, NW / 5th Floor / Washington, DC 20036 Phone 202-463-4860 ext. 258 / Fax 202-463-4863 -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- Chris P. McCabe - Principle Engineer Choice Hotels International - Information Technology [EMAIL PROTECTED] - 602-953-4416 -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: in search of more efficient design
i think that's the best answer to such a vague question i've ever seen - i appreciate it. my first move is going to be to utilize your two-machine suggestion, just for a short-term memory relief. then i can focus on analyzing both sides for their resource usage. will update this thread when i know more. thanks all. Will Hartung wrote: From: ilasno [EMAIL PROTECTED] Sent: Thursday, December 19, 2002 11:29 AM Subject: in search of more efficient design SHORT VERSION - I am a beginner-intermediate webapp designer, and my first large-scale development is using around 500 mb of memory, with low client load. I am seeking ways to both measure where this extreme resource-usage is stemming from and design tips to bring it down. Others have mentioned that Java doesn't return memory to the OS after it is GC'd, and this is true. So the question is how is the application using its memory? You can certainly do some simple analysis by running through some use case scenarios of you app, and monitoring GC activity from Java. This will give you some idea where your memory may be going. For example, run your application with the -verbose:gc option and you'll see some information every time the jvm runs garbage collection. What it prints out will be some basic heap information. Start with a small heap (java -Xms32M say), and everytime it needs to expand the heap, it will GC and print out a line. You can also use the java -Xrunhprof options to monitor the heap. But, basically, start the app, and run through some simple, common scenarios (user viewing the site, updating the site, whatever), and see if the memory use surges. If you notice that when a user updates the site your memory jumps from 64MB to 500MB, then that should give you an idea on where to look. Perhaps you could change your code to reuse some object versus recreating them. But, for all I know one of the things that your site does is upload large digital images and you're simply stuck with the memory cost of manipulating them. If you application is reasonably factored without a lot of redundant code, then your servlets per se are probably not the direct culprit in using up your memory. Obviously their actions affect memory, but a servlet is simply an object and doesn't have dramatic overhead in the container. You mention how all of your work is done in the doGet methods, but perhaps there can be some efficiencies gained by using some global Singleton caches. If your application is constantly loading reasonably static data from the DB and then throwing it away, that's just a waster of CPU and memory if it could effectively be loaded once and then used several times. Struts (as was elsewhere suggested) is a fine tool, but if you simply converted your application over to the struts framerwork, and kept your core code the same, then you'd still be using a lot of memory I would bet. I wouldn't throw the baby out with the bathwater and completely reengineer the site right off. You first need to understand where the resources are going so you can decided whether an in place fix is appropriate or the problem is so fundamental that you'd need to restart from scratch. Finally, if it's practical or necessary, you could run the admin features on one machine, and have the production users run on another machine. Since the production users only need access to the database, there's no real requirement that one machine support both the consumers and content producers. Regards, Will Hartung ([EMAIL PROTECTED]) -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- i am deprogrammed -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
