Hi Axton, Thank you for the detailed response! We just upgraded to 8.1 over the weekend and I was very tempted to make some adjustments before we release the system. I was a good boy and will test my changes in non-prod first and submit a Change Request :)
I have been watching the heap and permsize in JvisualVM. You are right that the permsize isn't even getting close to the 256mb we have configured. I increased it from the default 128mb because I was receiving a PermGen space error from a scheduled BIRT report ( https://communities.bmc.com/people/JMiller/status/29293#comment-21720). It didn't seem to make a difference though so I'll lower it again. After being up a few days it is ~116mb right now. Jason On Thu, Feb 13, 2014 at 8:24 PM, Axton <[email protected]> wrote: > ** > A large heap will mean longer GC times. Set the min to something > reasonable, like 512mb, and the max to whatever you want to spare. Let the > VM manage the size of the heap in that range. Setting the min=max is a bad > practice, though I see/hear many people doing it. I can't rationalize why > this is a good idea and i see negative side effects of doing it. All I > have ever seen a large heap with an equally large min heap size result in > are very long pauses in the application for the very long GC's the JVM has > to do. > > Think the scenario through: > - you create a JVM with 6gb of heap, min and max > - the jvm starts up and allocates all 6gb of heap > - the jvm runs for a very long time and eventually uses up all the heap > - once all the heap is used up it triggers a garbage collection > - the jvm must pause the application to perform a full garbage collection > of 6gb memory > - the pause for the full GC takes 75 seconds, during which time your > application is unresponsive and CPU usage spikes > > If you want to see this in action, use jstat (included with your JRE) to > collect GC and pause statistics. This is the same data you can see > visually using jvisualvm or jconsole. > > If you set the min to 512mb, the max to 6gb, the JVM will allocate > additional memory as the JVM see's fit. The JRE not a broken piece of > software; it knows how to determine a somewhat optimal memory allocation. > You can improve upon their configuration, but not by setting min heap=max > heap. It has more to do with profiling the application, setting targets > for GC cpu utilization, GC timings, and memory utilization, then working > toward those targets by selecting the appropriate GC and setting the > tunables for the particular GC in line with the characteristics of your > application. > > Whoever says setting the min heap equal to the max heap helps performance > is simply parroting what they heard someone else say. If you don't believe > me, ask them why this is a good idea and see if you get back an answer that > has an ounce of reasoning behind it. If someone out there can argue that > point I'd be very interested in hearing your argument. I like to be proven > wrong. The closest thing I've heard to a reasonable argument, from a > person at BMC, is that they suggest this as a one size fits all solution to > help with performance because they tested in a virtual environment where > there were tight contention for physical resources and they project the > majority of their user base is in a similar configuration. I still don't > agree with this though, but it's better than 'the internet said to' or the > 'somebody said it was good,' or the 'I've always done it that way' that I > usually hear. The hypervisor is going to page if the hypervisor is going > to page; I don't see a memory grab changing that, much less helping > matters. If you have VM's where you are overallocated you need to deal > with that issue, but not by trying to grab all the physical resources you > can. > > On the permsize, I don't see any condition where this needs to be > 256mb > for a container running the mid-tier. It simply won't use it. I have yet > to see a midtier that used more than ~200mb. The permanent generation > holds specific things. There are only so many of them to go around in the > mid-tier. If you want to read more, see here: > > https://blogs.oracle.com/jonthecollector/entry/presenting_the_permanent_generation > > Most OS's show far less free memory than is actually free. This is due to > storage subsystem caching. This memory, while not showing free in top or > task manager, is actually available for immediate use. If top shows less > free memory than you set the max heap to it's probably ok. See here: > http://www.linuxatemyram.com/ > http://xteams.oit.ncsu.edu/iso/node/938 > With Windows I try hard to not become any kind of authority so you are on > your own. > > Axton Grams > > > On Thu, Feb 13, 2014 at 2:04 PM, Jason Miller <[email protected]>wrote: > >> ** >> Same here, no issues. Out of the 80+ servers my team supports we only >> have two physicals, both production Remedy DB servers. We are talking with >> our DBAs about possibly moving our older 7.6.04 DB to their VM DB >> cluster since the (physical) server is up for refresh this year. >> >> Currently our two production ITSM 8.x MT web servers have 16 GB of RAM. >> So far this appears to be overkill for just using CM, SRM and WO. I >> expect as we roll out more apps the usage will grow but I wonder if we'll >> really need 16 GB (now that the base host build isn't taking 75% itself). >> >> For our AR 7.6.04, HD 6, everything else custom environment (70% WUTusage) >> our web servers usually have 3 to 4 GB of RAM. >> >> <a bit OT> >> Question for the Java gurus out there... I have learned quite a bit >> about TC, heap, JVM configuration over the last few years but there is >> one thing I haven't been able to figure out (or was given bogus info). >> With the different types of memory is it the heap setting that dictates >> the total max memory that the java process can use? >> >> This is from our new production web server with 16 GB of RAM: >> >> -XX:PermSize=256m >> -XX:MaxPermSize=512m >> -Xms3584m >> -Xmx3584m >> >> Using this will the JVM ever touch the majority of the RAM? The way I >> read that is our Tomcat Java process will never us more than 3.5 GB of RAM. >> Is there another type of memory that will grow beyond the heap? >> >> I used to have Xmx set much higher, to use most of the RAM on the server >> leaving a few gigs for OS and other processes but PS advised us that large >> of a heap will cause performance issues. >> </a bit OT> >> >> >> Jason >> >> >> On Thu, Feb 13, 2014 at 10:12 AM, heverro12 . >> <[email protected]>wrote: >> >>> ** >>> Hey Claire, >>> >>> Our MT sits on a VM and only experienced the known issues with MT. >>> >>> Robert >>> >>> >>> On Thu, Feb 13, 2014 at 9:06 AM, Sanford, Claire < >>> [email protected]> wrote: >>> >>>> ** >>>> >>>> It is in the Readme doc of the Mid-Tier hotfix. It is a 7.6.04 SP4. >>>> *MT_7604SP4_2013NOV07_CU_ALL* >>>> >>>> >>>> >>>> SW00450564 - Mid-Tier running extremely slowly with Tomcat's CPU >>>> usage spiking to 100%...This fix also requires a Tomcat patch to resolve >>>> the Tomcat portion of what causes the CPU spike. According to this >>>> link, https://issues.apache.org/bugzilla/show_bug.cgi?id=52858 , about >>>> the Tomcat defect - it's fixed in 6.0 patch 36 of Tomcat. You need to >>>> have this hotfix bundle or newer in addition to the latest Tomcat 6.0 patch >>>> available. >>>> >>>> >>>> >>>> >>>> >>>> *From:* Action Request System discussion list(ARSList) [mailto: >>>> [email protected]] *On Behalf Of *Chad Wilhelm >>>> *Sent:* Thursday, February 13, 2014 11:57 AM >>>> >>>> *To:* [email protected] >>>> *Subject:* Re: Caught Exception Error and Tomcat >>>> >>>> >>>> >>>> ** >>>> >>>> Hello, >>>> >>>> >>>> >>>> Where is the document located where it mentions patching TC? What >>>> version of TC? What release of Java? We are on build Version 8.1.00 >>>> 201401210137 Hotfix running TC 6 and we are receiving the error. >>>> >>>> >>>> >>>> Thank You, >>>> >>>> Chad Wilhelm >>>> CareTech Solutions >>>> Office: (248) 823-0177 >>>> [image: Description: >>>> cid:[email protected]]<http://www.caretech.com/> >>>> Helping extraordinary people do extraordinary things >>>> >>>> Best in KLAS >>>> >>>> Partial IT Outsourcing 2012 >>>> >>>> Extensive IT Outsourcing >>>> 2008, 2009, 2010 and 2011 >>>> >>>> Best in KLAS Awards: >>>> Software & Services >>>> >>>> www.KLASresearch.com <http://www.klasresearch.com/> >>>> >>>> * [image: Description: Description: KLAS_2013]-----* >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> *From:* Action Request System discussion list(ARSList) [ >>>> mailto:[email protected] <[email protected]>] *On Behalf Of *Jason >>>> Miller >>>> *Sent:* Thursday, February 13, 2014 12:45 PM >>>> *To:* [email protected] >>>> *Subject:* Re: Caught Exception Error and Tomcat >>>> >>>> >>>> >>>> ** >>>> >>>> I am a fan of stripping everything down and building with newer >>>> versions to keep things updated and clean. We have it down to about 45 >>>> minutes (if we are trying to be fast) to uninstall MT, Tomcat, Java -> >>>> resinstall Java, TC, MT, hotfix MT, configure MT/TC, customize login.jsp >>>> pages. It is for 8.1 but I'll strip out any proprietary info and post our >>>> process doc. >>>> >>>> >>>> >>>> Recently when doing this I found our web server used about 75% of 16 GB >>>> of ram *without *our apps installed! No wonder this sever was >>>> unhappy. We just swapped it out with a new build earlier this week and is >>>> only using 20% of 16 GB after being active for almost a week (ITSM 8.0). >>>> That other server is heading towards a rebuild... >>>> >>>> >>>> >>>> Jason >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> On Thu, Feb 13, 2014 at 9:29 AM, Sanford, Claire < >>>> [email protected]> wrote: >>>> >>>> This goes back to the "Caught Exception Error" thread... >>>> >>>> One of the solutions recommended is to load a patch to Tomcat. Reading >>>> the instructions it says I have to uninstall the midtier and then upgrade >>>> tomcat and then reinstall the midtier. That is not a short downtime! >>>> >>>> Question... >>>> >>>> When you upgraded the tomcat, did you have to reinstall the midtier? >>>> >>>> If so, how long did it take? >>>> >>>> What else did you do to your mid-tier server while you were doing the >>>> install (s). >>>> >>>> Thank you! >>>> >>>> >>>> >>>> ITSM 7.6.04 SP2 >>>> ARS 7.6.04 SP3 >>>> Oracle 11.2.0.3.0 - 64bit Production >>>> Win 2008 Server >>>> >>>> Claire Sanford >>>> Information Systems Division >>>> Memorial Hermann Healthcare System >>>> [email protected] >>>> >>>> >>>> >>>> >>>> _______________________________________________________________________________ >>>> UNSUBSCRIBE or access ARSlist Archives at www.arslist.org >>>> "Where the Answers Are, and have been for 20 years" >>>> >>>> >>>> >>>> _ARSlist: "Where the Answers Are" and have been for 20 years_ >>>> >>>> _ARSlist: "Where the Answers Are" and have been for 20 years_ >>>> _ARSlist: "Where the Answers Are" and have been for 20 years_ >>>> >>> >>> _ARSlist: "Where the Answers Are" and have been for 20 years_ >>> >> >> _ARSlist: "Where the Answers Are" and have been for 20 years_ >> > > _ARSlist: "Where the Answers Are" and have been for 20 years_ > _______________________________________________________________________________ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org "Where the Answers Are, and have been for 20 years"
<<inline: image002.jpg>>
<<inline: image001.gif>>
