Re: how to diagnose a potential bottleneck
no, you install FR onto your Railo or CF server, and it integrates with it by installing an FR JAR file. If you follow the manual installation instructions you will get a better idea how it works, On Tue, Dec 2, 2014 at 5:25 AM, Don danfar...@hotmail.com wrote: There is absolutely no restrictions whatsoever on where FR can be installed, it would be pretty useless if that was the case, you can install it on any supported OS, FR does not know or care what you are using that machine for. So how does that work then? You point FR to a domain? i thought it needed more a intimate connection with the server its introspecting. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:359738 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: how to diagnose a potential bottleneck
Doesnt seem anyone noticed my reply so I'll post this again... I noticed a sharp drop in JVM memory during this process ( it all happened in about one minute ) Say from the average JVM Freememory of : 255 Megs Dropping to around : 30 Megs Any thoughts or ideas? It did come back up to normal levels, but the rate of the drop has me concerned. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:359739 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: how to diagnose a potential bottleneck
This does not necessarily mean anything Don. It could simply mean migration from eden to persistent memory where GC recovers in due course. It's the overall pattern of GC recovery that means the most when talking about gross numbers. You should see a saw tooth pattern over time with steep drops as memory is recovered on the heap. Problems arise when the drop does not level off (it recovers less and less memory each time - i.e. a memory leak) or when the top side of the heap is too lean (i.e. the heap is too small for it's overall usage pattern). But keying in on a big dump of objects into perm heap isn't necessarily going to give you information that you can use. Mark Kruger - CFG CF Webtools www.cfwebtools.com www.coldfusionmuse.com O: 402.932.3318 E: mkru...@cfwebtools.com Skype: markakruger -Original Message- From: Don [mailto:danfar...@hotmail.com] Sent: Tuesday, December 02, 2014 9:36 AM To: cf-talk Subject: Re: how to diagnose a potential bottleneck Doesnt seem anyone noticed my reply so I'll post this again... I noticed a sharp drop in JVM memory during this process ( it all happened in about one minute ) Say from the average JVM Freememory of : 255 Megs Dropping to around : 30 Megs Any thoughts or ideas? It did come back up to normal levels, but the rate of the drop has me concerned. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:359740 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: how to diagnose a potential bottleneck
This does not necessarily mean anything Don. It could simply mean migration from eden to persistent memory where GC recovers in due course. It's the overall pattern of GC recovery that means the most when talking about gross numbers. You should see a saw tooth pattern over time with steep drops as memory is recovered on the heap. Problems arise when the drop does not level off (it recovers less and less memory each time - i.e. a memory leak) or when the top side of the heap is too lean (i.e. the heap is too small for it's overall usage pattern). But keying in on a big dump of objects into perm heap isn't necessarily going to give you information that you can use. Yes thanks Mark. I do realize that this may be barking up the wrong tree. But since this site is hosted my hands are tied on a few levels in debugging this strange error. This is really all I have at present to look into. Could also be the hardware isn't up to the task. I'm not seeing many offending pages. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:359741 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: how to diagnose a potential bottleneck
Russ Michaels wrote: doing debugging and load testing on a live production server is generally a bad idea. On general principles, sure... although I'd say that a diagnostic reporter like F-R belongs first and foremost on a production box. To the OP... it sounds like you are stuck, so how good is your error reporting? Assuming errors are being thrown at all (you said the server slowed which doesn't necessarily mean its erroring) are you capturing them for review? If not you should put something in to troll and see what comes up. Maybe something unexpected is hanging up threads? If error trapping is out, what about timing the execution of pages and seeing who the hogs are? variables.CFMstart=gettickcount(); ... stuff ... variables.CFMTook=int(gettickcount()-variables.CFMstart); Put start in Application, stop in OnRequestEnd and output to a very small db record that also includes, say, cgi.script_name and cgi.query_string. Leave it up for an hour or three and see what it finds. -- --m@Robertson-- Janitor, The Robertson Team mysecretbase.com ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:359742 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: how to diagnose a potential bottleneck
Russ Michaels wrote: doing debugging and load testing on a live production server is generally a bad idea. On general principles, sure... although I'd say that a diagnostic reporter like F-R belongs first and foremost on a production box. To the OP... it sounds like you are stuck, so how good is your error reporting? Assuming errors are being thrown at all (you said the server slowed which doesn't necessarily mean its erroring) are you capturing them for review? If not you should put something in to troll and see what comes up. Maybe something unexpected is hanging up threads? If error trapping is out, what about timing the execution of pages and seeing who the hogs are? variables.CFMstart=gettickcount(); ... stuff ... variables.CFMTook=int(gettickcount()-variables.CFMstart); Put start in Application, stop in OnRequestEnd and output to a very small db record that also includes, say, cgi.script_name and cgi.query_string. Leave it up for an hour or three and see what it finds. -- --m@Robertson-- Janitor, The Robertson Team mysecretbase.com Yah, creative little solution there. I will make use of your idea. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:359743 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: how to diagnose a potential bottleneck
If you want code samples on the error dumping stuff let me know. I worked something up for an article years ago but its grown up since. A breadcrumb array kept in the session scope has been key. Stores everything the user does and gets dumped to the error file. --m@-- ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:359744 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: how to diagnose a potential bottleneck
You can also configure CF to log slow pages and if you have enterprise, server monitor can log/alert you to slow/hung requests. On 12/2/14, 10:52 AM, Money Pit wrote: Russ Michaels wrote: doing debugging and load testing on a live production server is generally a bad idea. On general principles, sure... although I'd say that a diagnostic reporter like F-R belongs first and foremost on a production box. To the OP... it sounds like you are stuck, so how good is your error reporting? Assuming errors are being thrown at all (you said the server slowed which doesn't necessarily mean its erroring) are you capturing them for review? If not you should put something in to troll and see what comes up. Maybe something unexpected is hanging up threads? If error trapping is out, what about timing the execution of pages and seeing who the hogs are? variables.CFMstart=gettickcount(); ... stuff ... variables.CFMTook=int(gettickcount()-variables.CFMstart); Put start in Application, stop in OnRequestEnd and output to a very small db record that also includes, say, cgi.script_name and cgi.query_string. Leave it up for an hour or three and see what it finds. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:359745 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: how to diagnose a potential bottleneck
A breadcrumb array kept in the session scope has been key. Stores everything the does and gets dumped to the error file. Am curious as to what you did there. Would you mind sharing how you did that? ( I dont need code ) just your idea(s) explained with some more detail. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:359746 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: how to diagnose a potential bottleneck
You can also configure CF to log slow pages and if you have enterprise, server monitor can log/alert you to slow/hung requests. On 12/2/14, 10:52 AM, Money Pit wrote: Another good suggestion. But I'm not recieving much in the way of information from the host anymore. Seems they might not like me poking around too much I dont know. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:359747 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: how to diagnose a potential bottleneck
Sounds like it's time to move on. On 12/2/14, 1:11 PM, Don wrote: I'm not recieving much in the way of information from the host anymore. Seems they might not like me poking around too much I dont know. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:359748 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: how to diagnose a potential bottleneck
Sounds like it's time to move on. On 12/2/14, 1:11 PM, Don wrote: I'm not recieving much in the way of information from the host anymore. Seems they might not like me poking around too much I dont know. lol, well that's exactly what the developer of their e-commerce software apparently has done. That's why I'm here! ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:359749 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: how to diagnose a potential bottleneck
Sounds like they need a new host. Is that an option? On Tue, Dec 2, 2014 at 4:33 PM, Don danfar...@hotmail.com wrote: Sounds like it's time to move on. On 12/2/14, 1:11 PM, Don wrote: I'm not recieving much in the way of information from the host anymore. Seems they might not like me poking around too much I dont know. lol, well that's exactly what the developer of their e-commerce software apparently has done. That's why I'm here! ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:359750 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: how to diagnose a potential bottleneck
FR is indeed meant to run on a production server, and I was not saying otherwise. But you can also install it on dev to help you with the debugging and load testing as well. I think you are incorrectly assuming that FusionReactor is a debugging and load testing tool, which is not the case. FR is basically a monitor app for CF, it doesn't just fix stuff by itself, other than killing requests which hang around too long. to clarify by debugging I mean as in the cf debugger by load testing I mean tools such as these http://carehart.org/cf411/#loadtest On Tue, Dec 2, 2014 at 6:52 PM, Money Pit websitema...@gmail.com wrote: Russ Michaels wrote: doing debugging and load testing on a live production server is generally a bad idea. On general principles, sure... although I'd say that a diagnostic reporter like F-R belongs first and foremost on a production box. To the OP... it sounds like you are stuck, so how good is your error reporting? Assuming errors are being thrown at all (you said the server slowed which doesn't necessarily mean its erroring) are you capturing them for review? If not you should put something in to troll and see what comes up. Maybe something unexpected is hanging up threads? If error trapping is out, what about timing the execution of pages and seeing who the hogs are? variables.CFMstart=gettickcount(); ... stuff ... variables.CFMTook=int(gettickcount()-variables.CFMstart); Put start in Application, stop in OnRequestEnd and output to a very small db record that also includes, say, cgi.script_name and cgi.query_string. Leave it up for an hour or three and see what it finds. -- --m@Robertson-- Janitor, The Robertson Team mysecretbase.com ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:359751 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: how to diagnose a potential bottleneck
If I had to diagnose this, I would start with making sure that none of the function calls or cfinvokes are endlessly recursing. Then I would make sure none of the database queries are hanging. On Thu, Nov 27, 2014 at 9:03 PM, Don danfar...@hotmail.com wrote: Got this custom e-commerce app that is basically going down daily. Seems to run fine again upon server restart, but eventually it goes down to a crawl. This started happening after some patches to Coldfusion 11 were implemented. Host says it was rolled back, but problems are persisting. I'm seeing a lot use of CFINVOKE accross the application. Does anyone have any ideas on where I could start looking at this issue? ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:359752 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: how to diagnose a potential bottleneck
Russ Michaels wrote: I think you are incorrectly assuming that FusionReactor is a debugging and load testing tool, No not at all. Thats why I called it a 'diagnostic reporter' earlier on. Something I don't need at all for the most part but need desperately when things aren't going right and I haven't figured out why yet. Thx by the way for the load testing tools link. Don (Dan?) wrote Am curious as to what you did there. Would you mind sharing how you did that? ( I dont need code ) just your idea(s) explained with some more detail. Sure, but dumping out the code is probably faster than explaining it :-) in Application.cfm I define a 2D session-scope array if not isdefined. Next I build a variables-scope struct that contains the following cgi scope info: cgi.https, cgi.http_referer, cgi.remote_addr, cgi.request_method, cgi.server_name. You can skip the last one if you are tracking only one domain. Next I define a variables-scope struct containing all form scope fieldnames and their contents, if any exist. After that I define a variables-scope 1D array that contains 5 items in this order: 1. date/time to the second 2. cfid+_+cftoken values 3. The complete url including query string (built for another purpose and re-used here). 4. the form variables struct created above 5. The cgi variables struct created above. Lastly I arrayAppend the 1D array created above to the 2d session-scoped array, which is the actual breadcrumb trail. What you get in the end is a visually compact, chronological display of what the user did, complete with how long they paused between moves, form inputs made etc. There is enough info in this to re-create a genuine user session manually yourself to watch what goes wrong, or just spot a bot up to no good. This is delivered to me in an error report that holds a dump of all desired scopes so this is part of a more comprehensive reference that I can look up as needed. Looking at this I can see a couple of things I could do better. No need to store the cfid/cftoken on every step. Should be at 1st item only and thats it. Also the cgi.server_name is redundant since I am outputting the entire url. Its created in pieces and assembled at the end to minimize need for cflocks, which are named and not session scoped. Execution time is only a few ms, using the timer I noted in the previous post. Forget what it comes out to exactly as its been some time since i put this into play. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:359753 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: how to diagnose a potential bottleneck
Great! And these sessions structures, do you write the data to the DB onSessionEnd ? ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:359754 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: how to diagnose a potential bottleneck
No I don't. It all sits in memory unused. But if an error occurs this info gets dumped out via my error handler where it can be used post-mortem. Generally my error handler loops over a pre-defined list of scopes and dumps them all out. Depending on the setting I've made, the dump is either to a db record (Access memo fields aren't large enough for this) or to a disk file. I prefer a browser-readable disk file - a large query or similar sitting in the variables scope can make these files very large. 500k to 1mb files are common. Dumping out scopes like this has security implications so you need to address that when storing/viewing this info. --m@-- ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:359755 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: how to diagnose a potential bottleneck
No I don't. It all sits in memory unused. But if an error occurs this info gets dumped out via my error handler where it can be used post-mortem. Generally my error handler loops over a pre-defined list of scopes and dumps them all out. Depending on the setting I've made, the dump is either to a db record (Access memo fields aren't large enough for this) or to a disk file. I prefer a browser-readable disk file - a large query or similar sitting in the variables scope can make these files very large. 500k to 1mb files are common. Dumping out scopes like this has security implications so you need to address that when storing/viewing this info. --m@-- I see, so it follows all active users around and reports only upon error. Quite an elegant approach. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:359756 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: how to diagnose a potential bottleneck
I run a financial service site that has very complex pathways running
through it. This has saved me an enormous amount of time trying to figure
out how a series of user actions could uncover some sort of hole in the
code.
This thread got me tinkering and I did some tweaking. For starters when I
wrote it I had not yet switched to jsessionIDs. I think this is all of it
here:
// initialize. reqUID is a UUID that lives as long as each page request.
lock name=#request.reqUID# type=exclusive timeout=10 {
if (not isDefined (session.crumbs)) {
session.crumbs=arrayNew(2);
variables.SessionIDs=arrayNew(1);
variables.SessionIDs[1]=client.cfid;
variables.SessionIDs[2]=client.cftoken;
variables.SessionIDs[3]=session.sessionID;
temp=arrayAppend(session.crumbs,variables.SessionIDs);
}
}
variables.cgiBits=structNew();
structInsert(variables.cgiBits,https,cgi.https);
structInsert(variables.cgiBits,http_referer,cgi.http_referer);
structInsert(variables.cgiBits,remote_addr,cgi.remote_addr);
structInsert(variables.cgiBits,request_method,cgi.request_method);
variables.formBits=;
if (isdefined (form.fieldnames)) {
variables.formBits=structNew();
for (i=1; i lte listlen(form.fieldnames); i=i+1) {
fieldName=listGetAt(form.fieldnames,i);
structInsert(variables.formBits,fieldName,form[fieldname]);
}
}
// zoneNow is now() where time value is corrected to the time zone the site
owner prefers.
// variables.currPage is full current url with query string, built
elsewhere.
variables.crumb=arrayNew(1);
variables.crumb[1]=DateFormat(CreateODBCDate(request.zoneNow),-mm-dd)
TimeFormat(CreateODBCTime(request.zoneNow),HH:mm:ss);
variables.crumb[2]=variables.currPage;
variables.crumb[3]=variables.formBits;
variables.crumb[4]=variables.cgiBits;
lock name=#request.reqUID# type=exclusive timeout=10 {
temp=arrayAppend(session.crumbs,variables.crumb);
}
~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive:
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:359757
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
