RE: [cfaussie] Re: Slow Server Startup Time
Kai's reply makes me want to cue the music to Dylan's song (covered by many), The answer my friend, is blowin' in the logs [wind]. The answer is blowin' in the logs. Or maybe that's too old for many to get it. :-) Yes, it could be that it's a systemic problem (outofmemory errors, jvm issues, etc.) which might be reported in the logs. Also, check the [cf]/logs/cfserver.log, as on *nix that has some of what is otherwise in the jrun or runtime logs (though they may have info as well). Between that and any console log you may have, it may well give insight into the problem. If this had been a CF8 server, some would point out to make sure you were on JVM 1.6.0_11 or above (because of the JVM class loading bug that existed prior to that), but since you say you're on CF9 we would assume you're on at least _18 (which I think is what 9 installs). But then since you may have implemented this CF9 deployment in a non-standard way (running as an instance on a previously installed CF 8 multiserver deployment, or perhaps as a J2EE deployment), let's at least have you confirm (in the CF Admin settings summary or system information pages) that you are running a JVM update greater than 1.6.0_11. Hope that's helpful. /charlie -Original Message- From: cfaussie@googlegroups.com [mailto:cfaus...@googlegroups.com] On Behalf Of Kai Koenig Sent: Friday, March 26, 2010 11:03 PM To: cfaussie@googlegroups.com Subject: Re: [cfaussie] Re: Slow Server Startup Time Andrew, can you post the startup logs? Cheers Kai On 27/03/2010, at 1:34 PM, BarryC wrote: Is there anything else running on the server, what is the CPU grunt like on the server? The best bet would be to monitor the server and see what resource usage is like on it when you start Coldfusion and then your script. Also, you could set a custom timeout value in your script with the cfsetting tag so it at least starts without timing out. What is your timeout set to? Barry. On Mar 27, 1:24 pm, Andrew Myers am2...@gmail.com wrote: Hi Guys, I'm in the process of configuring a new server from scratch and I'm running into some issues with ColdFusion taking a long time to start up. After a start or restart I get the following error when I try and access my site for a few minutes: Server ErrorThe server encountered an internal error and was unable to complete your request. *Application server is busy. Either there are too many concurrent requests or the server still is starting up.* Then when that finally completes, I am getting a cffile timeout when my app is loading it's ColdSpring config. I have previously installed this on another server without seeing these problems. The server I am seeing the problem on is 64bit with 32Gb RAM (the other one was 32bit with 4Gb RAM) so I am concerned it is slower Both are ubuntu, running CF9 and I did the install of CF myself on both of them. Has anyone seen this before and if so do you have any advice on how to improve (or even debug) this? Regards, Andrew. -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en. -- Kai Koenig - Ventego Creative Ltd ph: +64 4 476 6781 - mob: +64 21 928 365 / +61 450 132 117 web: http://www.ventego-creative.co.nz blog: http://www.bloginblack.de twitter: http://www.twitter.com/agentK Hands-on Regular Expression training @ webDU 2010 http://bloginblack.de/agentk/workshop-befriending-regular-expressions/ Hands-on Flash Catalyst and Flex 4 training @ Webinale 2010 http://bloginblack.de/agentk/workshop-rias-with-flash-catalyst-and- flex-4/ -- -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en. -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en.
Re: [cfaussie] Re: Slow Server Startup Time
From looking at the logs, the MessageBrokerServlet or the CFFormGateway inits are the culprits. Across all your logs, those show a 30-40 delay for something just runs through in either the same or within 1-2 secs on the machines I just had a look at. I'm wondering if it's maybe waiting for some hostname in something to resolve? Also - it seems that /san/lun0... etc is actually pointing to a SAN. Just wondering if your other cf servers in that environment live in the same infrastructure and on the SAN as well - have seen crazy things with SANs or any other network drives happening before where some component of an application was waiting to authenticate against something and all of a sudden things came to a halt. Cheers Kai 27/03 11:06:05 user CFCServlet: init 27/03 11:06:05 user FlashGateway: init 27/03 11:06:05 user MessageBrokerServlet: init 27/03 11:07:22 user CFFormGateway: init 27/03 11:07:22 user CFInternalServlet: init 27/03 11:07:22 user WSRPProducer: init 27/03 11:07:23 user ServerCFCServlet: init 27/03 11:07:23 info Deploying enterprise application JRun 4.0 Internal J2EE Components from: file:/san/lun0/jrun4/lib/jrun-comp.ear 27/03 11:07:23 info Deploying EJB JRunSQLInvoker from: file:/san/ lun0/jrun4/lib/jrun-comp.ear Server cfusion ready (startup time: 87 seconds) -- Kai Koenig - Ventego Creative Ltd ph: +64 4 476 6781 - mob: +64 21 928 365 / +61 450 132 117 web: http://www.ventego-creative.co.nz blog: http://www.bloginblack.de twitter: http://www.twitter.com/agentK Hands-on Regular Expression training @ webDU 2010 http://bloginblack.de/agentk/workshop-befriending-regular-expressions/ Hands-on Flash Catalyst and Flex 4 training @ Webinale 2010 http://bloginblack.de/agentk/workshop-rias-with-flash-catalyst-and-flex-4/ -- -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en.
Re: [cfaussie] Re: Slow Server Startup Time
Hi Andrew, With the startup you can see that cranking up the Message Broker Servlet (ie Livecycle/Blaze) is taking almost a minute, which I find is usually due to some network ports used by LCDS still being open from the last time ColdFusion was run. However CF is usually available to take forwarded requests from the web server when you see: 27/03 11:06:03 user ColdFusionStartUpServlet: ColdFusion: application services are now available Which in your log is only seven seconds after the first startup log message. What happens on line 67 of index.cfm (or is it Application.cfc)? The logs are trying to tell you something there, check it out. As an earlier, ahead-of-its-time draft of the song goes: How many logs must a developer tail, before you can call them a developer? ;-) Cheers, Robin ROBIN HILLIARD Chief Technology Officer ro...@rocketboots.com.au RocketBoots Pty Ltd Level 11 189 Kent Street Sydney NSW 2001 Australia Phone +61 2 9323 2507 Facsimile +61 2 9323 2501 Mobile +61 418 414 341 www.rocketboots.com.au On 28/03/2010, at 10:07 AM, Andrew wrote: Just to add to that, after the initial timeout on index.cfm, if you hit the page again instantly it is fine, and the app works fine and seems to perform as expected. It's just the initial load that are extremely slugging (both starting the server, and initialising the app). -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en. -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en.
[cfaussie] Re: Slow Server Startup Time
Thanks - I think this is worth following up. I also installed jrun on the SAN although I wasn't sure if this was the right thing to do and the more I read about it the more I think maybe it isn't good practice? Is uninstalling on Linux as simple as blowing away the jrun4 dir or do you need to run an uninstaller? I'm ok with manually removing the apache config and startup scripts. Generally speaking is the SAN an appropriate place for web resources? This is where my host has allocated all my web space. Andrew. On Mar 28, 11:55 am, MrBuzzy mrbu...@gmail.com wrote: There is a house in New Orleans, they call the risin' SAN Morning everybody :) Andrew, I noticed your app is stored on 'The SAN' (storage area network); *03/27 11:09:01 Warning [jrpp-0] - Thread: jrpp-0, processing template: /san/lun0/www/index.cfm, completed in 88 seconds, exceeding the 30 second warning limit* Can you deploy the app to a locally attached disk instead? CF startup and compilation is probably slower because of the remote disk activity. I also noticed this (but Kai is faster hehe); 27/03 11:06:05 user FlashGateway: init *27/03 11:06:05 user MessageBrokerServlet: init* 27/03 11:07:22 user CFFormGateway: init 27/03 11:07:22 user CFInternalServlet: init Seems that servlet is taking a long time to be initialised, it could be for the same reason (SAN). (Just a stab in the dark really) HTH. Cheers. On 28 March 2010 10:07, Andrew am2...@gmail.com wrote: Just to add to that, after the initial timeout on index.cfm, if you hit the page again instantly it is fine, and the app works fine and seems to perform as expected. It's just the initial load that are extremely slugging (both starting the server, and initialising the app). -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.comcfaussie%2bunsubscr...@googlegroups.c om . For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en. -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en.
[cfaussie] Re: Slow Server Startup Time
Thanks Kai. I don't have he San on my other machines and to be perfectly honest I am a complete noob when it comes to SANs. Tomorrow ill take it out if the equation and see if it improves the situation. One thing I have noticed on this box is that after ssh'ing to it there is a lenghty delay between entering my username and receiving the password prompt. Not sure if it could be related but if there is an issue with authenticating users perhaps there is something similar with the network drives? On Mar 28, 11:52 am, Kai Koenig k...@koeni.de wrote: From looking at the logs, the MessageBrokerServlet or the CFFormGateway inits are the culprits. Across all your logs, those show a 30-40 delay for something just runs through in either the same or within 1-2 secs on the machines I just had a look at. I'm wondering if it's maybe waiting for some hostname in something to resolve? Also - it seems that /san/lun0... etc is actually pointing to a SAN. Just wondering if your other cf servers in that environment live in the same infrastructure and on the SAN as well - have seen crazy things with SANs or any other network drives happening before where some component of an application was waiting to authenticate against something and all of a sudden things came to a halt. Cheers Kai 27/03 11:06:05 user CFCServlet: init 27/03 11:06:05 user FlashGateway: init 27/03 11:06:05 user MessageBrokerServlet: init 27/03 11:07:22 user CFFormGateway: init 27/03 11:07:22 user CFInternalServlet: init 27/03 11:07:22 user WSRPProducer: init 27/03 11:07:23 user ServerCFCServlet: init 27/03 11:07:23 info Deploying enterprise application JRun 4.0 Internal J2EE Components from: file:/san/lun0/jrun4/lib/jrun-comp.ear 27/03 11:07:23 info Deploying EJB JRunSQLInvoker from: file:/san/ lun0/jrun4/lib/jrun-comp.ear Server cfusion ready (startup time: 87 seconds) -- Kai Koenig - Ventego Creative Ltd ph:+64 4 476 6781- mob:+64 21 928 365/ +61 450 132 117 web:http://www.ventego-creative.co.nz blog:http://www.bloginblack.de twitter:http://www.twitter.com/agentK Hands-on Regular Expression training @ webDU 2010http://bloginblack.de/agentk/workshop-befriending-regular-expressions/ Hands-on Flash Catalyst and Flex 4 training @ Webinale 2010http://bloginblack.de/agentk/workshop-rias-with-flash-catalyst-and-fl... -- On Mar 28, 11:52 am, Kai Koenig k...@koeni.de wrote: From looking at the logs, the MessageBrokerServlet or the CFFormGateway inits are the culprits. Across all your logs, those show a 30-40 delay for something just runs through in either the same or within 1-2 secs on the machines I just had a look at. I'm wondering if it's maybe waiting for some hostname in something to resolve? Also - it seems that /san/lun0... etc is actually pointing to a SAN. Just wondering if your other cf servers in that environment live in the same infrastructure and on the SAN as well - have seen crazy things with SANs or any other network drives happening before where some component of an application was waiting to authenticate against something and all of a sudden things came to a halt. Cheers Kai 27/03 11:06:05 user CFCServlet: init 27/03 11:06:05 user FlashGateway: init 27/03 11:06:05 user MessageBrokerServlet: init 27/03 11:07:22 user CFFormGateway: init 27/03 11:07:22 user CFInternalServlet: init 27/03 11:07:22 user WSRPProducer: init 27/03 11:07:23 user ServerCFCServlet: init 27/03 11:07:23 info Deploying enterprise application JRun 4.0 Internal J2EE Components from: file:/san/lun0/jrun4/lib/jrun-comp.ear 27/03 11:07:23 info Deploying EJB JRunSQLInvoker from: file:/san/ lun0/jrun4/lib/jrun-comp.ear Server cfusion ready (startup time: 87 seconds) -- Kai Koenig - Ventego Creative Ltd ph:+64 4 476 6781- mob:+64 21 928 365/ +61 450 132 117 web:http://www.ventego-creative.co.nz blog:http://www.bloginblack.de twitter:http://www.twitter.com/agentK Hands-on Regular Expression training @ webDU 2010http://bloginblack.de/agentk/workshop-befriending-regular-expressions/ Hands-on Flash Catalyst and Flex 4 training @ Webinale 2010http://bloginblack.de/agentk/workshop-rias-with-flash-catalyst-and-fl... -- -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en.
[cfaussie] Re: Slow Server Startup Time
With a long and distinguished history of not being afraid to make embarrassingly stupid suggestions... On Mar 28, 12:41 pm, Andrew am2...@gmail.com wrote: ... One thing I have noticed on this box is that after ssh'ing to it there is a lenghty delay between entering my username and receiving the password prompt. Not sure if it could be related but if there is an issue with authenticating users perhaps there is something similar with the network drives? Any chance you have a network duplex error? We have a router that fails to conect at full-duplex - regardless of settings after a power interruption. I'd at least be shooting the hosting companay a quick email asking them to ensure that all equipment that you're resources are on and any other network resources out from there to the www are manually checked to be correct. Gavin. -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en.
Re: [cfaussie] Re: Slow Server Startup Time
Hi Andrew, Thanks - I think this is worth following up. I also installed jrun on the SAN although I wasn't sure if this was the right thing to do and the more I read about it the more I think maybe it isn't good practice? ... Generally speaking is the SAN an appropriate place for web resources? This is where my host has allocated all my web space. I can't really speak for the general public and overall best practices on that, but one common reason for using a SAN (for me) is having for instance a cluster of CF servers dealing with the same template/CFC/assets on the SAN. In those instances I would always install Jrun/CF on the local machine resp a local disk and refer to the SAN for the webroot/cfroot etc. Also what I found with SANs (particular in hosting environments) is that the performance you get _dramatically_ varies depending on the connection (LAN, Fibre etc) and on what the other users of the SAN are doing. Real scenario that happened to one of my clients a while ago: another SAN users was running DB backups at a fixed time of the day. At the same time the overall SAN performance decreased so much that all of the CF servers delivering files had issues and we got significant performance issues on the web front end. SAN/Virtualisiation etc. is fine - but one still shares resources and sometimes weird stuff happens. Kai -- Kai Koenig - Ventego Creative Ltd ph: +64 4 476 6781 - mob: +64 21 928 365 / +61 450 132 117 web: http://www.ventego-creative.co.nz blog: http://www.bloginblack.de twitter: http://www.twitter.com/agentK Hands-on Regular Expression training @ webDU 2010 http://bloginblack.de/agentk/workshop-befriending-regular-expressions/ Hands-on Flash Catalyst and Flex 4 training @ Webinale 2010 http://bloginblack.de/agentk/workshop-rias-with-flash-catalyst-and-flex-4/ -- -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en.
