Typical Monday morning! I thought it was fixed but it isn't, after the reboot the server seemed to start behaving normally again so I left it in the hands of my colleagues and went away for the weekend. According to them about 3 hours after I left it started up again and they've had to monitor it constantly over the weekend restarting a hung CF server on average about every 2 hours....
There wasn't a huge response to my original post but I'll keep this thread updated in case anyone might have some ideas.... Or if, for some unknown reason, I actually find a solution which may be useful to others in future This is a real brain-masher of a problem... > -----Original Message----- > From: Edward Chanter [mailto:[EMAIL PROTECTED] > Sent: Friday, June 17, 2005 12:46 PM > To: CF-Linux > Subject: RE: Memory Usage and JVM - Solution;- boy do I feel dumb...! > > What is it about computers and Occam's Razor anyway?? "The > simplest explanation...." > > I seem to have solved this problem with a cold reboot, shut > it down, left it for a while and then powered up again and so > far it SEEMS to be running normally.... Boy do I feel dumb, I > must have rebooted it a hundred times but didn't once think > that maybe a power-down would make a difference... D'OH D'OH > D'OH D'OH D'OH D'OH Just call me Homer J Simpson! > > Oh well, what would life be without a good lesson from time to time! > > Thanks to anyone who put thought into this for me, I'm sorry > I wasted everyone's time :-) > > Have a nice weekend all, I will! > > -= Ed > > > -----Original Message----- > > From: Ellison, Ben A [mailto:[EMAIL PROTECTED] > > Sent: 16 June 2005 15:18 > > To: CF-Linux > > Subject: RE: Memory Usage and JVM > > > > We ran into a very similar thing... > > > > There was a triple bubble-sort (don't' ask :) on a large > xml data set. > > Something (some cold fusion compare or xml swap or another > > function) inside one of those loops was just eating memory like > > nobody's business. > > > > After we got rid of the sorts, things got better -- but it was just > > eating memory slower. > > > > When we got rid of the XML all together & just dealt with a query > > recordset instead, the problem virtually disappeared. > > > > I can send out a copy of the sort code that was causing the > errors (it > > was kind of inefficient, but not so badly written to explain server > > crashes). > > > > --B > > > > > > -----Original Message----- > > From: Edward Chanter [mailto:[EMAIL PROTECTED] > > Sent: Thursday, June 16, 2005 4:37 AM > > To: CF-Linux > > Subject: Memory Usage and JVM > > > > Dear All, > > > > One of our servers has recently started to exhibit what I can only > > describe as memory leaks. Essentially what happens is that > it will run > > just fine for roungly 2 hours (depending on > > traffic) and then slowly JVM seems to eat up memory. The > machine is a > > little under-spec and a new one is on order but I'm a > little worried > > that it's not the server because this app has been running > well for a > > number of years on both windows and more recently linux. > > > > Trying to get to the bottom of it is proving very difficult > and we've > > streamlined the app and run all the benchmarking we can > think of but > > the problem keeps right on happening. I've followed various > macromedia > > tech notes that seem to deal with this problem but nothing > I've done > > has had any effect on the problem. > > > > The main reason for this email is that I'm seeing something > odd in the > > cfserver.log that I thought I'd share with you in case someone has > > seen something simillar: > > > > <!--- Begin logfile output ---> > > Unexpected Signal : 11 occurred at PC=0x40385580 > > Function=(null)+0x40385580 > > Library=/opt/coldfusionmx/runtime/jre/lib/i386/server/libjvm.so > > > > NOTE: We are unable to locate the function name symbol for the error > > just occurred. Please refer to release documentation for > > possible > > reason and solutions. > > > > > > Current Java thread: > > > > Dynamic libraries: > > 08048000-0804f000 r-xp 00000000 03:41 1393294 > > /opt/coldfusionmx/bin/cfusion 0804f000-08051000 rw-p 00006000 03:41 > > 1393294 /opt/coldfusionmx/bin/cfusion > > 40000000-40016000 r-xp 00000000 03:41 2228234 /lib/ld-2.3.3.so > > 40016000-40017000 r--p 00015000 03:41 2228234 /lib/ld-2.3.3.so > > 40017000-40018000 rw-p 00016000 03:41 2228234 /lib/ld-2.3.3.so > > 40018000-40020000 r-xp 00000000 03:41 738476 > > /opt/coldfusionmx/runtime/jre/lib/i386/native_threads/libhpi.so > > 40020000-40021000 rw-p 00007000 03:41 738476 > > /opt/coldfusionmx/runtime/jre/lib/i386/native_threads/libhpi.so > > 40021000-40025000 rw-s 00000000 03:41 1525214 > > /tmp/hsperfdata_ewar/6403 > > 40025000-40034000 r-xp 00000000 03:41 2228522 > > /lib/libpthread-0.10.so > > 40034000-40035000 r--p 0000e000 03:41 2228522 > > /lib/libpthread-0.10.so > > 40035000-40036000 rw-p 0000f000 03:41 2228522 > > /lib/libpthread-0.10.so > > 40078000-4007a000 r-xp 00000000 03:41 2228352 /lib/libdl-2.3.3.so > > 4007a000-4007b000 r--p 00001000 03:41 2228352 /lib/libdl-2.3.3.so > > 4007b000-4007c000 rw-p 00002000 03:41 2228352 /lib/libdl-2.3.3.so > > 4007d000-40197000 r-xp 00000000 03:41 2228273 /lib/libc-2.3.3.so > > 40197000-40198000 r--p 0011a000 03:41 2228273 /lib/libc-2.3.3.so > > 40198000-4019b000 rw-p 0011b000 03:41 2228273 /lib/libc-2.3.3.so > > 4019d000-40749000 r-xp 00000000 03:41 738478 > > /opt/coldfusionmx/runtime/jre/lib/i386/server/libjvm.so > > 40749000-407a3000 rw-p 005ab000 03:41 738478 > > /opt/coldfusionmx/runtime/jre/lib/i386/server/libjvm.so > > 407b6000-407c8000 r-xp 00000000 03:41 2228356 > /lib/libnsl-2.3.3.so > > 407c8000-407c9000 r--p 00011000 03:41 2228356 > /lib/libnsl-2.3.3.so > > 407c9000-407ca000 rw-p 00012000 03:41 2228356 > /lib/libnsl-2.3.3.so > > 407cc000-407ed000 r-xp 00000000 03:41 2228354 /lib/libm-2.3.3.so > > 407ed000-407ee000 r--p 00020000 03:41 2228354 /lib/libm-2.3.3.so > > 407ee000-407ef000 rw-p 00021000 03:41 2228354 /lib/libm-2.3.3.so > > 407ef000-407f0000 r--s 00000000 03:41 722168 > > /opt/coldfusionmx/lib/webchartsJava2D.jar > > 407f0000-407f3000 r--s 00000000 03:41 1852639 > > /opt/coldfusionmx/runtime/jre/lib/ext/dnsns.jar > > 407f3000-407f7000 r--s 00000000 03:41 722150 > > /opt/coldfusionmx/lib/cf4was_ae.jar > > 407f7000-407f9000 r--s 00000000 03:41 2033196 > > /opt/coldfusionmx/lib/cfx.jar 407f9000-407fc000 r--s 00000000 03:41 > > 2033195 /opt/coldfusionmx/lib/cfmx_bootstrap.jar > > 407fc000-40806000 r-xp 00000000 03:41 2228263 > > /lib/libnss_files-2.3.3.so 40806000-40807000 r--p 00009000 > > 03:41 2228263 /lib/libnss_files-2.3.3.so 40807000-40808000 rw-p > > 0000a000 03:41 2228263 /lib/libnss_files-2.3.3.so 40808000-40818000 > > r-xp 00000000 03:41 738483 > > /opt/coldfusionmx/runtime/jre/lib/i386/libverify.so > > 40818000-4081a000 rw-p 0000f000 03:41 738483 > > /opt/coldfusionmx/runtime/jre/lib/i386/libverify.so > > 4081a000-4083a000 r-xp 00000000 03:41 738484 > > /opt/coldfusionmx/runtime/jre/lib/i386/libjava.so > > 4083a000-4083c000 rw-p 0001f000 03:41 738484 > > /opt/coldfusionmx/runtime/jre/lib/i386/libjava.so > > 4083c000-40850000 r-xp 00000000 03:41 738486 > > /opt/coldfusionmx/runtime/jre/lib/i386/libzip.so > > 40850000-40853000 rw-p 00013000 03:41 738486 > > /opt/coldfusionmx/runtime/jre/lib/i386/libzip.so > > 40853000-421de000 r--s 00000000 03:41 738535 > > /opt/coldfusionmx/runtime/jre/lib/rt.jar > > 42228000-4223e000 r--s 00000000 03:41 738508 > > /opt/coldfusionmx/runtime/jre/lib/sunrsasign.jar > > 4223e000-42319000 r--s 00000000 03:41 738510 > > /opt/coldfusionmx/runtime/jre/lib/jsse.jar > > 42319000-4232a000 r--s 00000000 03:41 738509 > > /opt/coldfusionmx/runtime/jre/lib/jce.jar > > 4232a000-42883000 r--s 00000000 03:41 738534 > > /opt/coldfusionmx/runtime/jre/lib/charsets.jar > > 4492b000-44930000 r--s 00000000 03:41 722149 > > /opt/coldfusionmx/lib/cf4was.jar 65b2d000-65d2d000 r--p > 00000000 03:41 > > 475880 /usr/lib/locale/locale-archive 65d2d000-66025000 > r--s 00000000 > > 03:41 722151 /opt/coldfusionmx/lib/cfusion.jar > 66025000-66041000 r--s > > 00000000 03:41 1852638 > > /opt/coldfusionmx/runtime/jre/lib/ext/sunjce_provider.jar > > 66041000-660fd000 r--s 00000000 03:41 1852640 > > /opt/coldfusionmx/runtime/jre/lib/ext/localedata.jar > > 660fd000-6610a000 r--s 00000000 03:41 1852641 > > /opt/coldfusionmx/runtime/jre/lib/ext/ldapsec.jar > > 6610a000-66183000 r--s 00000000 03:41 722148 > > /opt/coldfusionmx/lib/bcel.jar 66183000-66211000 r--s 00000000 03:41 > > 722172 /opt/coldfusionmx/lib/cfmx-ssl.jar > > 66211000-6623f000 r--s 00000000 03:41 722154 > > /opt/coldfusionmx/lib/flashgateway.jar > > 6623f000-662a8000 r--s 00000000 03:41 722155 > > /opt/coldfusionmx/lib/httpclient.jar > > 662a8000-662d2000 r--s 00000000 03:41 722152 > > /opt/coldfusionmx/lib/collections.jar > > 662d2000-662e2000 r--s 00000000 03:41 722173 > > /opt/coldfusionmx/lib/jakarta-oro-2.0.6.jar > > 662e2000-662f0000 r--s 00000000 03:41 722159 > > /opt/coldfusionmx/lib/ldapbp.jar 662f0000-662f7000 r--s > 00000000 03:41 > > 722157 /opt/coldfusionmx/lib/jaxp-api.jar > > 662f7000-662f8000 r--s 00000000 03:41 722202 > > /opt/coldfusionmx/lib/java2wsdl.jar > > 662f8000-662f9000 r--s 00000000 03:41 722204 > > /opt/coldfusionmx/lib/wsdl2java.jar > > 662f9000-662fa000 r--s 00000000 03:41 1393254 > > /opt/coldfusionmx/runtime/lib/java2wsdl.jar > > 662fa000-662fd000 r--s 00000000 03:41 1393260 > > /opt/coldfusionmx/runtime/lib/jrun-ant-tasks.jar > > 662fd000-662fe000 r--s 00000000 03:41 1393255 > > /opt/coldfusionmx/runtime/lib/jspc.jar > > 662fe000-662ff000 r--s 00000000 03:41 1393279 > > /opt/coldfusionmx/runtime/lib/migrate.jar > > 662ff000-66300000 r--s 00000000 03:41 1393256 > > /opt/coldfusionmx/runtime/lib/sniffer.jar > > 66400000-66420000 r--s 00000000 03:41 722174 > > /opt/coldfusionmx/lib/jdom.jar 66420000-6643f000 r--s 00000000 03:41 > > 722158 /opt/coldfusionmx/lib/ldap.jar 6643f000-66466000 > r--s 00000000 > > 03:41 722160 /opt/coldfusionmx/lib/log4j.jar 66466000-665ec000 r--s > > 00000000 03:41 722161 /opt/coldfusionmx/lib/macromedia_drivers.jar > > 665ec000-66631000 r--s 00000000 03:41 722175 > > /opt/coldfusionmx/lib/mail.jar 66631000-66651000 r--s 00000000 03:41 > > 722162 /opt/coldfusionmx/lib/NetComponents.jar > > 66651000-66b04000 r--s 00000000 03:41 722170 > > /opt/coldfusionmx/lib/tools.jar 66b04000-66bf6000 r--s > 00000000 03:41 > > 722167 /opt/coldfusionmx/lib/webcharts.jar > > 66bf6000-66d6f000 r--s 00000000 03:41 722203 > > /opt/coldfusionmx/lib/webservices.jar > > 66d6f000-66e95000 r--s 00000000 03:41 722169 > > /opt/coldfusionmx/lib/xalan.jar 66e95000-66ec8000 r--s > 00000000 03:41 > > 722153 /opt/coldfusionmx/lib/crimson.jar 66ec8000-66f04000 r--s > > 00000000 > > 03:41 722163 /opt/coldfusionmx/lib/pbclient42RE.jar > > 66f04000-6701f000 r--s 00000000 03:41 722164 > > /opt/coldfusionmx/lib/pbembedded42RE.jar > > 6701f000-67143000 r--s 00000000 03:41 722165 > > /opt/coldfusionmx/lib/pbserver42RE.jar > > 67143000-67184000 r--s 00000000 03:41 722166 > > /opt/coldfusionmx/lib/pbtools42RE.jar > > 67184000-671a6000 r--s 00000000 03:41 722176 > > /opt/coldfusionmx/lib/tt-bytecode.jar > > 671d4000-671e4000 r-xp 00000000 03:41 738489 > > /opt/coldfusionmx/runtime/jre/lib/i386/libnet.so > > 671e4000-671e5000 rw-p 0000f000 03:41 738489 > > /opt/coldfusionmx/runtime/jre/lib/i386/libnet.so > > 671e5000-671f3000 r--s 00000000 03:41 1393264 > > /opt/coldfusionmx/runtime/lib/instutil.jar > > 671f3000-671f4000 r--s 00000000 03:41 1393261 > > /opt/coldfusionmx/runtime/lib/wsdl2java.jar > > 671f4000-671f5000 r--s 00000000 03:41 1393278 > > /opt/coldfusionmx/runtime/lib/xmlscript.jar > > 671f5000-671f6000 r-xp 00000000 03:41 738501 > > /opt/coldfusionmx/runtime/jre/lib/i386/librmi.so > > 671f6000-671f7000 rw-p 00000000 03:41 738501 > > /opt/coldfusionmx/runtime/jre/lib/i386/librmi.so > > 671f7000-671fb000 r-xp 00000000 03:41 738500 > > /opt/coldfusionmx/runtime/jre/lib/i386/libioser12.so > > 671fb000-671fc000 rw-p 00003000 03:41 738500 > > /opt/coldfusionmx/runtime/jre/lib/i386/libioser12.so > > 67300000-6736a000 r--s 00000000 03:41 722156 > > /opt/coldfusionmx/lib/izmado.jar 6736a000-67b49000 r--s > 00000000 03:41 > > 1393280 /opt/coldfusionmx/runtime/lib/jrun.jar > > 67b49000-67b5d000 r-xp 00000000 03:41 722209 > > /opt/coldfusionmx/lib/libcfindex.so > > 67b5d000-67b61000 rw-p 00013000 03:41 722209 > > /opt/coldfusionmx/lib/libcfindex.so > > 67b62000-67b77000 r-xp 00000000 03:41 722211 > > /opt/coldfusionmx/lib/libCFXNeo.so > > 67b77000-67b7c000 rw-p 00014000 03:41 722211 > > /opt/coldfusionmx/lib/libCFXNeo.so > > 67b7c000-67bb0000 r-xp 00000000 03:41 360796 > > /usr/lib/libstdc++-2-libc6.1-1-2.9.0.so > > 67bb0000-67bbc000 rw-p 00033000 03:41 360796 > > /usr/lib/libstdc++-2-libc6.1-1-2.9.0.so > > 67bcb000-67bce000 r-xp 00000000 03:41 2228514 > /lib/libnss_dns-2.3.3.so > > 67bce000-67bcf000 r--p 00002000 > > 03:41 2228514 /lib/libnss_dns-2.3.3.so 67bcf000-67bd0000 > rw-p 00003000 > > 03:41 2228514 /lib/libnss_dns-2.3.3.so > > 67bd0000-67bdf000 r-xp 00000000 03:41 2228523 > > /lib/libresolv-2.3.3.so > > 67bdf000-67be1000 rw-p 0000e000 03:41 2228523 > > /lib/libresolv-2.3.3.so > > 67f00000-68079000 r--s 00000000 03:41 1393262 > > /opt/coldfusionmx/runtime/lib/webservices.jar > > 68079000-6816f000 r--s 00000000 03:41 1393263 > > /opt/coldfusionmx/runtime/lib/jrun-xdoclet.jar > > 6816f000-6829f000 r--s 00000000 03:41 1393258 > > /opt/coldfusionmx/runtime/lib/oem-xdoclet.jar > > 6829f000-68459000 r--s 00000000 03:41 1393250 > > /opt/coldfusionmx/runtime/lib/wsconfig.jar > > 68459000-684ce000 r--s 00000000 03:41 2033191 > > /opt/coldfusionmx/wwwroot/WEB-INF/lib/js.jar > > 68a00000-68a94000 r-xp 00000000 03:41 722212 > > /opt/coldfusionmx/lib/libporting.so > > 68a94000-68ab6000 rw-p 00093000 03:41 722212 > > /opt/coldfusionmx/lib/libporting.so > > 68ab7000-68d08000 r-xp 00000000 03:41 722207 > > /opt/coldfusionmx/lib/liballrsei.so > > 68d08000-68d34000 rw-p 00250000 03:41 722207 > > /opt/coldfusionmx/lib/liballrsei.so > > > > Heap at VM Abort: > > Heap > > PSYoungGen total 19008K, used 6463K [0x44930000, 0x46660000, > > 0x475a0000) > > eden space 4928K, 21% used [0x44930000,0x44a33c60,0x44e00000) > > from space 14080K, 38% used [0x44e00000,0x4534c2d0,0x45bc0000) > > to space 10880K, 0% used [0x45bc0000,0x45bc0000,0x46660000) > > PSOldGen total 178304K, used 145390K [0x475a0000, > 0x523c0000, > > 0x5d930000) > > object space 178304K, 81% used [0x475a0000,0x5039b830,0x523c0000) > > PSPermGen total 17152K, used 17080K [0x5d930000, 0x5e9f0000, > > 0x65930000) > > object space 17152K, 99% used [0x5d930000,0x5e9de130,0x5e9f0000) > > > > Local Time = Thu Jun 16 12:26:14 2005 > > Elapsed Time = 3129 > > # > > # HotSpot Virtual Machine Error : 11 > > # Error ID : 4F530E43505002EF > > # Please report this error at > > # http://java.sun.com/cgi-bin/bugreport.cgi > > # > > # Java VM: Java HotSpot(TM) Server VM (1.4.2-b28 mixed > mode) # space > > 4928K, 53% used space 14080K, 38% used space 10880K, 0% used space > > 178304K, 81% used space 17152K, 99% used# An error report file has > > been saved as hs_err_pid6403.log. > > # Please refer to the file for further information. > > # > > <!--- End logfile output ---> > > > > If anyone has seen this error or has any thoughts on it I would be > > eternally gratefull because this one is tying me knots.... > > > > Thanks in advance. > > > > best wishes, > > > > -= Ed > > > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > Edward Chanter > > Technical Innovations Manager > > Collaborative Connections Ltd. > > > > http://www.cc.uk.com/ > > Phone: +44 (0) 1273 832221 > > Mobile: +44 (0) 7866 365282 > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > > best wishes, > > > > -= Ed > > > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > Edward Chanter > > Technical Innovations Manager > > Collaborative Connections Ltd. > > > > http://www.cc.uk.com/ > > Phone: +44 (0) 1273 832221 > > Mobile: +44 (0) 7866 365282 > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > > > > > > > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Discover CFTicket - The leading ColdFusion Help Desk and Trouble Ticket application http://www.houseoffusion.com/banners/view.cfm?bannerid=48 Message: http://www.houseoffusion.com/lists.cfm/link=i:14:3978 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/14 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:14 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.14 Donations & Support: http://www.houseoffusion.com/tiny.cfm/54
