[OT] RE: Making the most of 8G of RAM
From: brien colwell [mailto:[EMAIL PROTECTED] Any program that needs 8gb of RAM to run seems suspect. Large amounts of data should be abstracted through the filesystem and dealt with as files. I have several programs that will cheerfully use 8G, or more, of RAM. In general, they are large simulations or large searches where there is no obvious subset of the search that can be pulled out and handled independently. There's not sufficient locality of reference to page the data to disk in any ordered way. For the one machine I need, it's far cheaper to pay the extra to guarantee Enough RAM To Do The Job than it is to pay for the days of thinking that *might* lead to a way of segmenting the problem. They are not webapps. However, they need 8Gb of RAM to run, and they are not suspect. - Peter - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Making the most of 8G of RAM
From: Stephen Nelson-Smith [mailto:[EMAIL PROTECTED] I have a new machine with 8G of RAM, running a 32 bit 2.6 kernel (CentOS 5). It is running an application inside 4.1.37 with Java 1.4.2. [...] What's the best way to make to most of the RAM? 1) 64-bit OS. 2) 64-bit JVM. 3) Newer Tomcat. Tomcat's own data structures shrank somewhat from 4.x - 5.0 - 5.5. I've not looked at 6 yet. 4) If the developers say the app needs 1.4.2, find out *exactly* why. Does it *need* 1.4.2 or will they only *support* it on 1.4.2? The former is a technical limitation, the latter a business decision that each business can negotiate about (along the lines of OK, pay us more money for support and we'll train someone on the current generation of VMs). - Peter - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Making the most of 8G of RAM
On Mon, Mar 17, 2008 at 11:16:49PM +, Stephen Nelson-Smith wrote: I plan to test running with a newer JVM shortly. I have been told by the developers that the application needs to run under 1.4.2. Whatever they mean by that. If they mean, needs 1.4.2 or newer, that's quite reasonable and you are in good shape for an upgrade. If they mean, we tested it aeons ago when 1.4.2 was current and don't know what it will do with a newer JVM then they should do the testing again with a contemporary version. If they mean, must have 1.4.2 *exactly* or it will croak then your developers are insufficiently trained. Code so haywired that it depends on the quirks of a single version of anything in its environment is too fragile for production use. -- Mark H. Wood, Lead System Programmer [EMAIL PROTECTED] Typically when a software vendor says that a product is intuitive he means the exact opposite. pgpvDTlSSE2Jl.pgp Description: PGP signature
Re: [OT] RE: Making the most of 8G of RAM
Don't forget to leave some space for the OS' buffer cache. If your app.s do a lot of I/O then they can benefit a great deal from having the most commonly accessed storage blocks cached all the time. W.r.t. Tomcat tuning this means you don't try to use up all of physical memory with Tomcat instances. Set the heap maxima so that the total, plus fixed overhead for Tomcat itself and the JVM, leave enough room for the OS and generous caches. You'll have to monitor and retune iteratively to find out how much generous should be. If you see *any* swapping then you probably ought to reduce the maxima until it goes away. Also monitor how much of your heap space is actually used during peak usage periods. Giving your app.s a huge slab of memory is a reasonable first step in exploring performance issues, but you may be unable to make use of all that memory in your app.s. In that case you may want to see if giving some of it back to the OS (see caching above) would give better overall performance. -- Mark H. Wood, Lead System Programmer [EMAIL PROTECTED] Typically when a software vendor says that a product is intuitive he means the exact opposite. pgpiyolwU14ia.pgp Description: PGP signature
Re: Making the most of 8G of RAM
Stephen Nelson-Smith a écrit : Hi, I have a new machine with 8G of RAM, running a 32 bit 2.6 kernel (CentOS 5). It is running an application inside 4.1.37 with Java 1.4.2. Given that we only have a 32 bit address space, is it fair to say that I can't use more than 2G of RAM for my Java virtual machine? I tried to prove this with: export CATALINA_OPTS=-server -Xms2512m -Xmx2512m but so far I've not seen memory usage go over 1G. What's the best way to make to most of the RAM? S. - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] Not fair to say that. The maximum space depends on several factor (see mailing list archives, this has already been discussed). To get most of your 8G machine, I recommend 64bits OS + 64bits jvm. - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Making the most of 8G of RAM
From: Stephen Nelson-Smith [mailto:[EMAIL PROTECTED] Subject: Making the most of 8G of RAM Given that we only have a 32 bit address space, is it fair to say that I can't use more than 2G of RAM for my Java virtual machine? Most 32-bit Linux systems will give you 3GB of virtual space (not RAM - don't confuse the two) per process. Some of this is taken up by various process and thread structures, along with code and non-heap data. Consequently, there's considerably less than 3GB available for the heap. but so far I've not seen memory usage go over 1G. There is (or at least was) a bug in the Sun 1.4.2 JVM that didn't do the arithmetic right when heap sizes were over 2GB due to treating some unsigned values as signed. You really should move up to a much more current JVM. What's the best way to make to most of the RAM? Put in a 64-bit OS and a 64-bit JVM. While you're at it, try running a newer Tomcat. If you must stick with a 32-bit environment, you could run multiple Tomcats, one for each webapp you have, assuming you have more than one. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Making the most of 8G of RAM
On Mon, Mar 17, 2008 at 10:53 PM, Caldarale, Charles R [EMAIL PROTECTED] wrote: but so far I've not seen memory usage go over 1G. There is (or at least was) a bug in the Sun 1.4.2 JVM that didn't do the arithmetic right when heap sizes were over 2GB due to treating some unsigned values as signed. You really should move up to a much more current JVM. I plan to test running with a newer JVM shortly. I have been told by the developers that the application needs to run under 1.4.2. We shall see. Put in a 64-bit OS and a 64-bit JVM. While you're at it, try running a newer Tomcat. Yes indeed. All what I am planning to do. In the meantime I have to get the most out of what I have, while I work out what combination of JVM / Tomcat / OS will work with the application, and while I get the dev team to rewrite code that won't work. If you must stick with a 32-bit environment, you could run multiple Tomcats, one for each webapp you have, assuming you have more than one. How about running eg 3 x xen vms? The machine seems rarely pushed for CPU, and with 3 vms I could load balance across them. - Chuck S. - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Making the most of 8G of RAM
From: Stephen Nelson-Smith [mailto:[EMAIL PROTECTED] Subject: Re: Making the most of 8G of RAM I plan to test running with a newer JVM shortly. I have been told by the developers that the application needs to run under 1.4.2. We shall see. Unless the app uses the keyword enum or the JSPs aren't in packages, I doubt you'll find any problems. How about running eg 3 x xen vms? The machine seems rarely pushed for CPU, and with 3 vms I could load balance across them. Why bother with xen? If you have work enough for three Tomcats, just run them all on one copy of the OS. What does the additional complexity buy you? - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Making the most of 8G of RAM
If one of your webapps could use a faster database or filesystem, you might look into using a ramdisk. Any program that needs 8gb of RAM to run seems suspect. Large amounts of data should be abstracted through the filesystem and dealt with as files. On Mon, Mar 17, 2008 at 4:21 PM, Caldarale, Charles R [EMAIL PROTECTED] wrote: From: Stephen Nelson-Smith [mailto:[EMAIL PROTECTED] Subject: Re: Making the most of 8G of RAM I plan to test running with a newer JVM shortly. I have been told by the developers that the application needs to run under 1.4.2. We shall see. Unless the app uses the keyword enum or the JSPs aren't in packages, I doubt you'll find any problems. How about running eg 3 x xen vms? The machine seems rarely pushed for CPU, and with 3 vms I could load balance across them. Why bother with xen? If you have work enough for three Tomcats, just run them all on one copy of the OS. What does the additional complexity buy you? - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Making the most of 8G of RAM
On Mon, Mar 17, 2008 at 11:21 PM, Caldarale, Charles R [EMAIL PROTECTED] wrote: From: Stephen Nelson-Smith [mailto:[EMAIL PROTECTED] How about running eg 3 x xen vms? The machine seems rarely pushed for CPU, and with 3 vms I could load balance across them. Why bother with xen? If you have work enough for three Tomcats, just run them all on one copy of the OS. What does the additional complexity buy you? We only run one app, but it is heavily used. I couldn't easily point different groups of users to different virtual hosts, but I could fairly easily balance across 3 vms using mod_proxy_ajp. As I write that I realise I could do that on one OS, of course, so I am not sure I see the benefit any more, as you say. - Chuck S. - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Making the most of 8G of RAM
there is a recorded webinar called Inside the Java Virtual Machine http://www.covalent.net/services/training/webinars.html it's one hour spent demystifying all you need to know about the JVM memory usage, once you've understood that, you'll have a very different perspective on -Xmx Filip Stephen Nelson-Smith wrote: Hi, I have a new machine with 8G of RAM, running a 32 bit 2.6 kernel (CentOS 5). It is running an application inside 4.1.37 with Java 1.4.2. Given that we only have a 32 bit address space, is it fair to say that I can't use more than 2G of RAM for my Java virtual machine? I tried to prove this with: export CATALINA_OPTS=-server -Xms2512m -Xmx2512m but so far I've not seen memory usage go over 1G. What's the best way to make to most of the RAM? S. - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
