Re: Load Testing - what to look for and monitor/report
We're doing performance analysis and looking for bottlenecks ... In that case, you want to test a single server only, at least to start. You'll need to define user paths that are most likely to be followed by users within the application, and define timers for each step of the user path (where a step is a single user action that triggers any server-side response). Then, you'll run a typical performance analysis test (no pauses between requests) for a single user, then scale up by some increment (say, 5 users) until the application becomes unresponsive. The data you collect during this process will primarily be response time data (for each timer). You can then drop that data into a spreadsheet and chart the timer steps that increase more than others, and those are your bottlenecks. If those steps are themselves pretty complex, you can add CFTIMER tags within your page to find out which part of the page is the slower part, as well. ... and also memory leaks since we have a lot of application scoped components and large numbers of objects created per request. That's one of my main concerns actually.. Well, that might require a different test, or at least might affect the point at which the application becomes unresponsive as described above. Proper performance analysis testing can really only happen once you've removed functionality and operation bugs from your application, and memory leaks would fall into that category. That said, it's unlikely you'll run into true memory leaks with CF components - you just might run out of memory, though. Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ http://training.figleaf.com/ Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on GSA Schedule, and provides the highest caliber vendor-authorized instruction at our training centers, online, or onsite. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349864 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Load Testing - what to look for and monitor/report
Anyhow, what I really want to know from you experts is what should I be watching and recording during these load tests? Generally, you're interested in response time and throughput. Also should I pair down from a load balanced solution to a single server to make it easier to debug log files and track memory etc? And then scale up after a specific duration? Or for the second test interval? I don't have much experience with load testing and just want to make sure that this is money well spend in terms of having some actionable data. It's hard to answer this without more information. What is the goal of the load test? What kind of actionable data are you looking for? You can use load tests to answer a couple of different questions. There are two basic types of load test: performance analysis, and capacity planning. In a performance analysis test, you're looking for bottlenecks in your application with the goal of improving application performance. In a capacity planning test, you're trying to identify how many users your application can support as it's currently written. Which one of these are you doing? Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ http://training.figleaf.com/ Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on GSA Schedule, and provides the highest caliber vendor-authorized instruction at our training centers, online, or onsite. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349862 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: Load Testing - what to look for and monitor/report
Hi Dave, We're doing performance analysis and looking for bottlenecks and also memory leaks since we have a lot of application scoped components and large numbers of objects created per request. That's one of my main concerns actually.. Brook -Original Message- From: Dave Watts [mailto:dwa...@figleaf.com] Sent: February-11-12 7:46 AM To: cf-talk Subject: Re: Load Testing - what to look for and monitor/report Anyhow, what I really want to know from you experts is what should I be watching and recording during these load tests? Generally, you're interested in response time and throughput. Also should I pair down from a load balanced solution to a single server to make it easier to debug log files and track memory etc? And then scale up after a specific duration? Or for the second test interval? I don't have much experience with load testing and just want to make sure that this is money well spend in terms of having some actionable data. It's hard to answer this without more information. What is the goal of the load test? What kind of actionable data are you looking for? You can use load tests to answer a couple of different questions. There are two basic types of load test: performance analysis, and capacity planning. In a performance analysis test, you're looking for bottlenecks in your application with the goal of improving application performance. In a capacity planning test, you're trying to identify how many users your application can support as it's currently written. Which one of these are you doing? Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ http://training.figleaf.com/ Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on GSA Schedule, and provides the highest caliber vendor-authorized instruction at our training centers, online, or onsite. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349863 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Load Testing - what to look for and monitor/report
Hey Guys, I am hiring a load testing company, Neustar, to run some load tests on a new cf app and server deployment. I have FusionReactor installed and the platform is CF9 Standard on windows 2008 web server, and the backend is a separate MS SQL box. A beefy one at that J. The server environment is virtualized running on vmware on Dell 610s (in case any one cared..).. Anyhow, what I really want to know from you experts is what should I be watching and recording during these load tests? Also should I pair down from a load balanced solution to a single server to make it easier to debug log files and track memory etc? And then scale up after a specific duration? Or for the second test interval? I don't have much experience with load testing and just want to make sure that this is money well spend in terms of having some actionable data. Thank yo! Brook ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349861 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
