Re: [cfaussie] Re: Slow Server Startup Time
Just a quick follow up question - have you tied CF into Apache or is it just running on the internal HTTP server on port 8300/8500 so far? Also: I think (and that was confirmed by Mark when we briefly discussed that) that MessageBrokerServlet is the culprit. It's a servlet that's used in Remoting/LCDS. I wonder two things: a) did the installer ask you if you wanted to install Blaze DS or LCDS Express (or whatever that LCDS light version is called)? If yes, did you install it? Note: MessageBrokerServlet most likely also comes with a plain CF setup as it's responsible for the /flex2gateway servlet endpoint afaik b) You could try to comment the sections in web.xml that use the MessageBrokerServlet. You'll probably find it in at least 2 instances in there. Try to restart CF and see if it b1) comes up at all b2) makes a difference re start-up times (maybe post the relevant log sections) Cheers Kai Here's a comparison of my new environment, compared to my old one 29/03 11:36:37 user ColdFusionStartUpServlet: init 29/03 11:36:37 user ColdFusionStartUpServlet: ColdFusion: Starting application services 29/03 11:36:37 user ColdFusionStartUpServlet: ColdFusion: VM version = 14.0-b16 29/03 11:38:58 user ColdFusionStartUpServlet: ColdFusion: application services are now available 29/03 11:38:58 user CFMxmlServlet: init 29/03 11:38:58 user CFMxmlServlet: Macromedia Flex Build: 87315.134646 29/03 11:38:59 user CFSwfServlet: init 29/03 11:38:59 user CFCServlet: init 29/03 11:39:00 user FlashGateway: init 29/03 11:39:00 user MessageBrokerServlet: init 29/03 11:41:08 user CFFormGateway: init 29/03 11:41:08 user CFInternalServlet: init 29/03 11:41:08 user WSRPProducer: init 29/03 11:41:08 user ServerCFCServlet: init 29/03 11:41:08 info Deploying enterprise application JRun 4.0 Internal J2EE Components from: file:/opt/jrun4/lib/jrun-comp.ear 29/03 11:41:08 info Deploying EJB JRunSQLInvoker from: file:/opt/ jrun4/lib/jrun-comp.ear Been going 5 mins. Compare this to an environment which works: 15/03 12:06:24 user JSPServlet: init 15/03 12:06:25 user ColdFusionStartUpServlet: init 15/03 12:06:25 user ColdFusionStartUpServlet: ColdFusion: Starting application services 15/03 12:06:25 user ColdFusionStartUpServlet: ColdFusion: VM version = 14.0-b16 15/03 12:06:29 user ColdFusionStartUpServlet: ColdFusion: application services are now available 15/03 12:06:29 user CFMxmlServlet: init 15/03 12:06:29 user CFMxmlServlet: Macromedia Flex Build: 87315.134646 15/03 12:06:29 user CFSwfServlet: init 15/03 12:06:30 user CFCServlet: init 15/03 12:06:31 user FlashGateway: init 15/03 12:06:31 user MessageBrokerServlet: init 15/03 12:06:31 user CFFormGateway: init 15/03 12:06:31 user CFInternalServlet: init 15/03 12:06:31 user WSRPProducer: init 15/03 12:06:32 user ServerCFCServlet: init 15/03 12:06:32 info Deploying enterprise application JRun 4.0 Internal J2EE Components from: file:/opt/jrun4/lib/jrun-comp.ear 15/03 12:06:32 info Deploying EJB JRunSQLInvoker from: file:/opt/ jrun4/lib/jrun-comp.ear 15/03 12:06:51 info JRun Proxy Server listening on *:51020 That last line is missing on my new environment. Could it be that I need to open a firewall port or something? -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en. -- Kai Koenig - Ventego Creative Ltd ph: +64 4 476 6781 - mob: +64 21 928 365 / +61 450 132 117 web: http://www.ventego-creative.co.nz blog: http://www.bloginblack.de twitter: http://www.twitter.com/agentK Hands-on Regular Expression training @ webDU 2010 http://bloginblack.de/agentk/workshop-befriending-regular-expressions/ Hands-on Flash Catalyst and Flex 4 training @ Webinale 2010 http://bloginblack.de/agentk/workshop-rias-with-flash-catalyst-and-flex-4/ -- -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en.
Re: [cfaussie] Re: Slow Server Startup Time
I have also seen incorrctly configured DNS cause slow startups and antivirus. If there is av running on the box try disable it. Paul Kukiel On 29/03/2010, at 8:07 PM, Kai Koenig k...@koeni.de wrote: Just a quick follow up question - have you tied CF into Apache or is it just running on the internal HTTP server on port 8300/8500 so far? Also: I think (and that was confirmed by Mark when we briefly discussed that) that MessageBrokerServlet is the culprit. It's a servlet that's used in Remoting/LCDS. I wonder two things: a) did the installer ask you if you wanted to install Blaze DS or LCDS Express (or whatever that LCDS light version is called)? If yes, did you install it? Note: MessageBrokerServlet most likely also comes with a plain CF setup as it's responsible for the /flex2gateway servlet endpoint afaik b) You could try to comment the sections in web.xml that use the MessageBrokerServlet. You'll probably find it in at least 2 instances in there. Try to restart CF and see if it b1) comes up at all b2) makes a difference re start-up times (maybe post the relevant log sections) Cheers Kai Here's a comparison of my new environment, compared to my old one 29/03 11:36:37 user ColdFusionStartUpServlet: init 29/03 11:36:37 user ColdFusionStartUpServlet: ColdFusion: Starting application services 29/03 11:36:37 user ColdFusionStartUpServlet: ColdFusion: VM version = 14.0-b16 29/03 11:38:58 user ColdFusionStartUpServlet: ColdFusion: application services are now available 29/03 11:38:58 user CFMxmlServlet: init 29/03 11:38:58 user CFMxmlServlet: Macromedia Flex Build: 87315.134646 29/03 11:38:59 user CFSwfServlet: init 29/03 11:38:59 user CFCServlet: init 29/03 11:39:00 user FlashGateway: init 29/03 11:39:00 user MessageBrokerServlet: init 29/03 11:41:08 user CFFormGateway: init 29/03 11:41:08 user CFInternalServlet: init 29/03 11:41:08 user WSRPProducer: init 29/03 11:41:08 user ServerCFCServlet: init 29/03 11:41:08 info Deploying enterprise application JRun 4.0 Internal J2EE Components from: file:/opt/jrun4/lib/jrun-comp.ear 29/03 11:41:08 info Deploying EJB JRunSQLInvoker from: file:/opt/ jrun4/lib/jrun-comp.ear Been going 5 mins. Compare this to an environment which works: 15/03 12:06:24 user JSPServlet: init 15/03 12:06:25 user ColdFusionStartUpServlet: init 15/03 12:06:25 user ColdFusionStartUpServlet: ColdFusion: Starting application services 15/03 12:06:25 user ColdFusionStartUpServlet: ColdFusion: VM version = 14.0-b16 15/03 12:06:29 user ColdFusionStartUpServlet: ColdFusion: application services are now available 15/03 12:06:29 user CFMxmlServlet: init 15/03 12:06:29 user CFMxmlServlet: Macromedia Flex Build: 87315.134646 15/03 12:06:29 user CFSwfServlet: init 15/03 12:06:30 user CFCServlet: init 15/03 12:06:31 user FlashGateway: init 15/03 12:06:31 user MessageBrokerServlet: init 15/03 12:06:31 user CFFormGateway: init 15/03 12:06:31 user CFInternalServlet: init 15/03 12:06:31 user WSRPProducer: init 15/03 12:06:32 user ServerCFCServlet: init 15/03 12:06:32 info Deploying enterprise application JRun 4.0 Internal J2EE Components from: file:/opt/jrun4/lib/jrun-comp.ear 15/03 12:06:32 info Deploying EJB JRunSQLInvoker from: file:/opt/ jrun4/lib/jrun-comp.ear 15/03 12:06:51 info JRun Proxy Server listening on *:51020 That last line is missing on my new environment. Could it be that I need to open a firewall port or something? -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en . -- Kai Koenig - Ventego Creative Ltd ph: +64 4 476 6781 - mob: +64 21 928 365 / +61 450 132 117 web: http://www.ventego-creative.co.nz blog: http://www.bloginblack.de twitter: http://www.twitter.com/agentK Hands-on Regular Expression training @ webDU 2010 http://bloginblack.de/agentk/workshop-befriending-regular-expressions/ Hands-on Flash Catalyst and Flex 4 training @ Webinale 2010 http://bloginblack.de/agentk/workshop-rias-with-flash-catalyst-and-flex-4/ -- -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en . -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en.
RE: [cfaussie] Re: Slow Server Startup Time
Paul, Anyone running an AV on a physical server should be re-considering their user access. AV's should not run on a physical server, if you need to check a file uploaded then you can run a scan, but resident memory checks should be disabled. Av's check every file when resident memory is switched on, and on a server the only files that should be run, should always be checked before running on a server. -Original Message- From: cfaussie@googlegroups.com [mailto:cfaus...@googlegroups.com] On Behalf Of Paul Kukiel Sent: Monday, 29 March 2010 8:23 PM To: cfaussie@googlegroups.com Subject: Re: [cfaussie] Re: Slow Server Startup Time I have also seen incorrctly configured DNS cause slow startups and antivirus. If there is av running on the box try disable it. Paul Kukiel -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en.
Re: [cfaussie] Re: Slow Server Startup Time
Ah - Andrew, this is on Ubuntu Server, yes? I've definitely seen this on Ubuntu Server, both on CF8, and CF9, when shutting down CF and then starting it back up again. Server startup gets stuck on the MessageBrokerServlet init call, for a while. It makes me think it's looking for something, and eventually it ends up timing out. For development, I end up rebooting the whole server - it's faster. Mark On Tue, Mar 30, 2010 at 1:00 PM, Andrew am2...@gmail.com wrote: Fixed the Verity issue. It needed ia32-libs libs installed, because Verity runs in 32 bit. However, I have since re-read the system requirements and it appears Ubuntu is only supported for 32 bit. I am going to try and RHEL and see if the MessageBrokerServlet issue still exists. Andrew. On Mar 29, 11:12 pm, Andrew am2...@gmail.com wrote: I don't believe that there is AV running. Might follow up on the DNS suggestion though. Now I've got another issue. I just went to start verity and found that it wouldn't run. /opt/jrun4/verity/verity-install.out reports this: sh: /opt/jrun4/verity/k2/_ilnx21/bin/vconfig: not found ERROR: Configuration failed to complete successfuly at Mon Mar 29 23:05:46 EST 2010 I'm starting to wonder if I have a dodgy version of the installer. Although I did do the md5sum thing on it before I used it and it passed... Anyone else got Verity running on Linux 64 bit? -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.comcfaussie%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en. -- E: mark.man...@gmail.com T: http://www.twitter.com/neurotic W: www.compoundtheory.com Hands-on ColdFusion ORM Training @ cf.Objective() 2010 www.ColdFusionOrmTraining.com/ -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en.
Re: [cfaussie] Re: Slow Server Startup Time
Hmm, that still looks pretty much the same to me. 45-ish seconds for that is still too high and unusual. I'll try to find something out about those two. Cheers Kai 29/03 10:50:46 user MessageBrokerServlet: init 29/03 10:51:32 user CFFormGateway: init -- Kai Koenig - Ventego Creative Ltd ph: +64 4 476 6781 - mob: +64 21 928 365 / +61 450 132 117 web: http://www.ventego-creative.co.nz blog: http://www.bloginblack.de twitter: http://www.twitter.com/agentK Hands-on Regular Expression training @ webDU 2010 http://bloginblack.de/agentk/workshop-befriending-regular-expressions/ Hands-on Flash Catalyst and Flex 4 training @ Webinale 2010 http://bloginblack.de/agentk/workshop-rias-with-flash-catalyst-and-flex-4/ -- -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en.
RE: [cfaussie] Re: Slow Server Startup Time
Kai's reply makes me want to cue the music to Dylan's song (covered by many), The answer my friend, is blowin' in the logs [wind]. The answer is blowin' in the logs. Or maybe that's too old for many to get it. :-) Yes, it could be that it's a systemic problem (outofmemory errors, jvm issues, etc.) which might be reported in the logs. Also, check the [cf]/logs/cfserver.log, as on *nix that has some of what is otherwise in the jrun or runtime logs (though they may have info as well). Between that and any console log you may have, it may well give insight into the problem. If this had been a CF8 server, some would point out to make sure you were on JVM 1.6.0_11 or above (because of the JVM class loading bug that existed prior to that), but since you say you're on CF9 we would assume you're on at least _18 (which I think is what 9 installs). But then since you may have implemented this CF9 deployment in a non-standard way (running as an instance on a previously installed CF 8 multiserver deployment, or perhaps as a J2EE deployment), let's at least have you confirm (in the CF Admin settings summary or system information pages) that you are running a JVM update greater than 1.6.0_11. Hope that's helpful. /charlie -Original Message- From: cfaussie@googlegroups.com [mailto:cfaus...@googlegroups.com] On Behalf Of Kai Koenig Sent: Friday, March 26, 2010 11:03 PM To: cfaussie@googlegroups.com Subject: Re: [cfaussie] Re: Slow Server Startup Time Andrew, can you post the startup logs? Cheers Kai On 27/03/2010, at 1:34 PM, BarryC wrote: Is there anything else running on the server, what is the CPU grunt like on the server? The best bet would be to monitor the server and see what resource usage is like on it when you start Coldfusion and then your script. Also, you could set a custom timeout value in your script with the cfsetting tag so it at least starts without timing out. What is your timeout set to? Barry. On Mar 27, 1:24 pm, Andrew Myers am2...@gmail.com wrote: Hi Guys, I'm in the process of configuring a new server from scratch and I'm running into some issues with ColdFusion taking a long time to start up. After a start or restart I get the following error when I try and access my site for a few minutes: Server ErrorThe server encountered an internal error and was unable to complete your request. *Application server is busy. Either there are too many concurrent requests or the server still is starting up.* Then when that finally completes, I am getting a cffile timeout when my app is loading it's ColdSpring config. I have previously installed this on another server without seeing these problems. The server I am seeing the problem on is 64bit with 32Gb RAM (the other one was 32bit with 4Gb RAM) so I am concerned it is slower Both are ubuntu, running CF9 and I did the install of CF myself on both of them. Has anyone seen this before and if so do you have any advice on how to improve (or even debug) this? Regards, Andrew. -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en. -- Kai Koenig - Ventego Creative Ltd ph: +64 4 476 6781 - mob: +64 21 928 365 / +61 450 132 117 web: http://www.ventego-creative.co.nz blog: http://www.bloginblack.de twitter: http://www.twitter.com/agentK Hands-on Regular Expression training @ webDU 2010 http://bloginblack.de/agentk/workshop-befriending-regular-expressions/ Hands-on Flash Catalyst and Flex 4 training @ Webinale 2010 http://bloginblack.de/agentk/workshop-rias-with-flash-catalyst-and- flex-4/ -- -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en. -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en.
Re: [cfaussie] Re: Slow Server Startup Time
From looking at the logs, the MessageBrokerServlet or the CFFormGateway inits are the culprits. Across all your logs, those show a 30-40 delay for something just runs through in either the same or within 1-2 secs on the machines I just had a look at. I'm wondering if it's maybe waiting for some hostname in something to resolve? Also - it seems that /san/lun0... etc is actually pointing to a SAN. Just wondering if your other cf servers in that environment live in the same infrastructure and on the SAN as well - have seen crazy things with SANs or any other network drives happening before where some component of an application was waiting to authenticate against something and all of a sudden things came to a halt. Cheers Kai 27/03 11:06:05 user CFCServlet: init 27/03 11:06:05 user FlashGateway: init 27/03 11:06:05 user MessageBrokerServlet: init 27/03 11:07:22 user CFFormGateway: init 27/03 11:07:22 user CFInternalServlet: init 27/03 11:07:22 user WSRPProducer: init 27/03 11:07:23 user ServerCFCServlet: init 27/03 11:07:23 info Deploying enterprise application JRun 4.0 Internal J2EE Components from: file:/san/lun0/jrun4/lib/jrun-comp.ear 27/03 11:07:23 info Deploying EJB JRunSQLInvoker from: file:/san/ lun0/jrun4/lib/jrun-comp.ear Server cfusion ready (startup time: 87 seconds) -- Kai Koenig - Ventego Creative Ltd ph: +64 4 476 6781 - mob: +64 21 928 365 / +61 450 132 117 web: http://www.ventego-creative.co.nz blog: http://www.bloginblack.de twitter: http://www.twitter.com/agentK Hands-on Regular Expression training @ webDU 2010 http://bloginblack.de/agentk/workshop-befriending-regular-expressions/ Hands-on Flash Catalyst and Flex 4 training @ Webinale 2010 http://bloginblack.de/agentk/workshop-rias-with-flash-catalyst-and-flex-4/ -- -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en.
Re: [cfaussie] Re: Slow Server Startup Time
Hi Andrew, With the startup you can see that cranking up the Message Broker Servlet (ie Livecycle/Blaze) is taking almost a minute, which I find is usually due to some network ports used by LCDS still being open from the last time ColdFusion was run. However CF is usually available to take forwarded requests from the web server when you see: 27/03 11:06:03 user ColdFusionStartUpServlet: ColdFusion: application services are now available Which in your log is only seven seconds after the first startup log message. What happens on line 67 of index.cfm (or is it Application.cfc)? The logs are trying to tell you something there, check it out. As an earlier, ahead-of-its-time draft of the song goes: How many logs must a developer tail, before you can call them a developer? ;-) Cheers, Robin ROBIN HILLIARD Chief Technology Officer ro...@rocketboots.com.au RocketBoots Pty Ltd Level 11 189 Kent Street Sydney NSW 2001 Australia Phone +61 2 9323 2507 Facsimile +61 2 9323 2501 Mobile +61 418 414 341 www.rocketboots.com.au On 28/03/2010, at 10:07 AM, Andrew wrote: Just to add to that, after the initial timeout on index.cfm, if you hit the page again instantly it is fine, and the app works fine and seems to perform as expected. It's just the initial load that are extremely slugging (both starting the server, and initialising the app). -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en. -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en.
Re: [cfaussie] Re: Slow Server Startup Time
Hi Andrew, Thanks - I think this is worth following up. I also installed jrun on the SAN although I wasn't sure if this was the right thing to do and the more I read about it the more I think maybe it isn't good practice? ... Generally speaking is the SAN an appropriate place for web resources? This is where my host has allocated all my web space. I can't really speak for the general public and overall best practices on that, but one common reason for using a SAN (for me) is having for instance a cluster of CF servers dealing with the same template/CFC/assets on the SAN. In those instances I would always install Jrun/CF on the local machine resp a local disk and refer to the SAN for the webroot/cfroot etc. Also what I found with SANs (particular in hosting environments) is that the performance you get _dramatically_ varies depending on the connection (LAN, Fibre etc) and on what the other users of the SAN are doing. Real scenario that happened to one of my clients a while ago: another SAN users was running DB backups at a fixed time of the day. At the same time the overall SAN performance decreased so much that all of the CF servers delivering files had issues and we got significant performance issues on the web front end. SAN/Virtualisiation etc. is fine - but one still shares resources and sometimes weird stuff happens. Kai -- Kai Koenig - Ventego Creative Ltd ph: +64 4 476 6781 - mob: +64 21 928 365 / +61 450 132 117 web: http://www.ventego-creative.co.nz blog: http://www.bloginblack.de twitter: http://www.twitter.com/agentK Hands-on Regular Expression training @ webDU 2010 http://bloginblack.de/agentk/workshop-befriending-regular-expressions/ Hands-on Flash Catalyst and Flex 4 training @ Webinale 2010 http://bloginblack.de/agentk/workshop-rias-with-flash-catalyst-and-flex-4/ -- -- You received this message because you are subscribed to the Google Groups cfaussie group. To post to this group, send email to cfaus...@googlegroups.com. To unsubscribe from this group, send email to cfaussie+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/cfaussie?hl=en.
