In which part of your jvm? "Dont know what you mean "
Heap? "huh?" PermGen? "more huh?" Premature ageing of objects? "more huh?" How soon after restart? "Depends on the load. Somtimes can be ok for days and other times like now....a few hours" How often? "Depends on the load" What else happens on the machine? "Nothing." What's the CPU load at the time? "CPU is fine so doesnt seem to be any endless loops or anything like that" How does it "die"? "Exhausts the amount of RAM allocated to the CF server JVM and just stops responding" Have you created and analysed stack traces and heap dumps? "No not yet because i cant make heads or tails of them anyway" etc etc etc I'm afraid there's no magic bullet such as a "my server hangs - show me in which template and why"-button. :-) K lol.....running out of memory :) I need to know what "exactly" is causing it which is why i am wanting request level information _____ From: Kai Koenig [mailto:[email protected]] Sent: Friday, 19 February 2010 1:22 PM To: [email protected] Subject: Re: [cfaussie] Seefusoin Vs Fusion Reactor If you are indeed running out of memory and the machine becomes unresponsive, the first thing I would do would be logging on the JVM level as well as on the CF level (Jrun metrics and/or cfstat). Also thread dumps when it crashes etc. That will already tell you a lot what's going on. Still not on a request level, but information such as "CF server which it seems to be running out of..." is just not enough to even give you a proper detailed advice. Cheers Kai Charlie Thanks for the response. I have always had memory issues with CF8 64 Bit and even now i have allocated 2Gb of RAM to the CF server which it seems to be running out of. Even though you mentioned that viewing the memory usage in the task manager is not very reliable, i do see it increase until it gets to a point where the CF server stops responding and requests begin to time out. I have even seen it get to a point where the CF server stops responding all together and crashes. Because of this i am wanting to try and see where this 2Gb of memory is being consumes and if it is a particular application that is causing the issue. whether it be GC is not happening or if there are persistant requests not being released or what ever....thats what i need to be able to find out. Any suggestions? Steve _____ From: charlie arehart [mailto:[email protected]] Sent: Friday, 19 February 2010 12:51 PM To: [email protected] Subject: RE: [cfaussie] Seefusoin Vs Fusion Reactor Following up my last note, as for determining memory used per request, I would say there is more to that question than just the simple one asked. Besides addressing if FR/SF can do it, I would also ask if it's really what is needed most times. FR/SF cannot track memory per request: ------------------------------------------------ First, no, neither FR nor SF have what no feature to track memory use per request. They operate as servlet filters, watching the requests going in and out of CF. Despite what it may seem, they have no insight into what goes on in the request while it's running (other than taking a stack trace, or tracking SQL statements). Only the CF 8/9 Monitor can: ----------------------------------- For what you want, being able to see memory use per request, the only monitor offering that is the CF 8/9 Enterprise Monitor, using its "start memory tracking" feature. As some here will warn, that's not generally something that most will want to enable on a production server (the "memory tracking", I mean), as it can have substantial overhead (and even add to CF's use of memory)-not always, as some have had no problem. Forewarned is forearmed. And even then, I've found that even with memory tracking turned on, it's still not always possible to determine what requests are really using memory. Is memory per request really the problem? ---------------------------------------------------- Indeed, the fact is (in my experience), the real cause of a lot of use of memory (that should raise an alarm) is not "per request" use. In such cases, when the requests finish, they give up their memory they used to be garbage collected. Now, memory may indeed show as growing even after they finish, but such "released" memory will eventually be garbage collected. Now, if memory is NOT GC'ed, then something is holding on to memory. It's not been released, and that may cause problems over time. The far more common cause of memory that can't be GC'ed is memory used that's living beyond the life of any requests, such as sessions, the application or server scopes, cached queries, and so on. There can also be memory use (not released) due to a true leak, but those are rare. A common one is the var scoping bug. Is an increase in memory really a problem? ---------------------------------------------------- As important, I see some people "sweating" because they see memory rising in various tools. In most cases, it may not be anything to worry about. Memory WILL increase, as I said above, when requests start, use memory, and end. The memory used will be marked to be released, but the JVM may not choose (and often WILL not choose) to do a GC until memory use approaches the total. People freak when they see memory "rising", but it may just be normal. What matters is if it CANNOT do a GC. That, then, is when you'll have problems. I'll add that another reason I see some sweat it when it seems memory is "rising" is that in a tool like SeeFusion or the CF 8/9 Monitor, they may see their memory as being always nearly full. But let me clarify something: if you have min heap < max heap, those tools are (unintentionally) fooling you. They are tracking the percent of used memory to allocated, NOT total memory (max heap). FusionReactor, instead, does track all three individually: used, allocated, and total (using total memory as the top of the graph) and computing the percent of memory used as the ratio of used to total. Since SF and the CF Monitor track the ratio of used to allocated, and the top of their memory graphs is the amount allocated, that can be very misleading. Again, for those who do set min heap=max heap, this is not an issue. That causes the JVM to pre-allocate all the memory requested, and then the top number (allocated) is the same as used (the true point at which memory maxes out) in SF and CF. But back to the point above: with any of these tools (and whether min heap=max or not), you may well see memory rising and approaching the total limit. I'm saying: don't always presume that that's necessarily something to worry about. In fact, all 3 tools have in them a button you can click to request a GC. If you do that, and memory drops like a stone, then it's as I said: this memory that was used, released to be GCed, but the JVM simply hadn't gotten around to it. If that's the case, then don't worry. But sure, if you're getting outofmemory errors that are for sure due to the heap space filling (not all outofmemory errors are about that), then yes you want to find what is holding on to memory. It may not be any one request, but the other uses of memory above. Tracking memory use in Task Manager, other tools -------------------------------------------------------------- Finally, Dan mentioned watching the memory in task manager (or other OS memory monitoring), but that is less reliable, as it tracks more than just heap memory used in the address space. And in fact using those to monitor memory suffer an opposite impact if you DO set min heap=max heap: since the JVM then pre-allocates all the memory, the OS perspective shows that all the memory is used, and you no longer will see it raise/fall with GCs. Really, you need to use one of the 3 monitors (or as Dave mentioned, a Java monitoring tool) to really see the heap memory use. Apologies ------------ I apologize both to those who hate long emails. I've added the headings to try to help. I apologize as well to those who may regard me a tall poppy for prattling on with all this information, and worse, for correcting assertions made by others. I realize it's a risky business, and I really mean no offense. I just spend my day helping people troubleshoot their CF servers, and I'm nearly always having to counter these and similar misperceptions in the community. When you come to understand some of these things, it really helps put a new perspective on CF. It so often gets blamed for problems that are really not its fault, or people spend hours/days digging through code to optimize it, when there's some larger root cause that may have nothing to do with that. I'm just trying to help stop the madness. :-) Hope that's helpful. Feel free to give me feedback, pro or con, on thoughts like these whether on-list of off ([email protected]). /charlie From: [email protected] [mailto:[email protected]] On Behalf Of Steve Onnis Sent: Wednesday, February 17, 2010 7:41 PM To: [email protected] Subject: [cfaussie] Seefusoin Vs Fusion Reactor What i am looking for is something that will show me requests based on memory usage to i can see what is consuming memory and how much of it. I have installed FusionReactor and it doesn't seem to have anything like this..it just tells you what the memory was like when the request was made. I tried to install Seefusion and the install failed and now i cant even seem to run the installer anymore...gives me an error saying "Failed to load java VM library: [java path selected in previous install attempt] (errno = 1930)" message. Can you not run these together? Can anyone recommend a tool that will do this? I need to be able to kill of requests that are consuming memory and free it up. Steve -- You received this message because you are subscribed to the Google Groups "cfaussie" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en. -- You received this message because you are subscribed to the Google Groups "cfaussie" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en. -- You received this message because you are subscribed to the Google Groups "cfaussie" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en. _________________________________________________ Kai Koenig - Ventego Creative Ltd ph: +64 4 476 6781 - mob: +64 21 928 365 / +61 450 132 117 web: http://www.ventego-creative.co.nz <http://www.ventego-creative.co.nz/> blog: http://www.bloginblack.de <http://www.bloginblack.de/> twitter: http://www.twitter.com/agentK -- You received this message because you are subscribed to the Google Groups "cfaussie" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en. -- You received this message because you are subscribed to the Google Groups "cfaussie" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en. _________________________________________________ Kai Koenig - Ventego Creative Ltd ph: +64 4 476 6781 - mob: +64 21 928 365 / +61 450 132 117 web: http://www.ventego-creative.co.nz blog: http://www.bloginblack.de twitter: http://www.twitter.com/agentK -- You received this message because you are subscribed to the Google Groups "cfaussie" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en. -- You received this message because you are subscribed to the Google Groups "cfaussie" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en.
