I don’t think there has been any changes in wotaskd in the last year that would help with this. There have been fixes in mod_webobjects since 2011, but unless you have changed that or updated Apache lately, that would not explain why this just started to happen.
As John Huss said, you should be able to get more debug info out of wotaskd. Chuck On 2/27/2014, 9:52 AM, "John Pollard" wrote: Back to my original problem, these errors are happening about one every 100 requests: Info: <WebObjects Apache Module> new request: /cgi-bin/WebObjects/MPMall.woa/wa/1/63/2392-Gilets-SALE.html Debug: App Name: MPMall.woa/wa/1/63/2392-Gilets-SALE.html (6) Info: V4 URL: /cgi-bin/WebObjects/MPMall.woa/wa/1/63/2392-Gilets-SALE.html Info: tr_selectInstance(): scheduler failed to select instance. Error: Request handling error: The requested application was not found on this server. I can't see logging from wotaskd in terms of making its choices about which instance is next, even for the working requests. I tried passing -_DeploymentDebugging true, when running wotaskd but the only logging I see is Lifebeats coming in. Am I misunderstanding what wotaskd does? The request never makes it to the application. Would there be much point in be rebuilding the latest Wonder wotaskd? The one I have running is about a year old. Or could the WO apache module be at fault, my mod_WebObjects.so file is dated 38 April 2011, apache v. 2.2.25 Many thanks again, John On 27 Feb 2014, at 16:42, John Pollard <j...@pollardweb.com<mailto:j...@pollardweb.com>> wrote: John, a big thank you as this seems to have answered why I couldn't start apps at all. I don't have the LOG path shown, so changed it to one I use for logging and now can fire up apps again. Or removing /tmp/logWebObjects also fixes it. Perhaps this script should test for access to the LOG file and if no access, not launch the apps that way. Many thanks; I don't think this relates to my underlying intermittent "The requested application was not found on this server." but it brings a bit of sanity back! John On 27 Feb 2014, at 15:20, John Huss <johnth...@gmail.com<mailto:johnth...@gmail.com>> wrote: If you're using Wonder's wotaskd (or care to replace the following script) you can turn on logging for instance startup in: /Library/WebObjects/JavaApplications/wotaskd.woa/Contents/Resources/SpawnOfWotaskd.sh #!/bin/sh # To enable logging of instance startup run the command 'touch /tmp/logWebObjects' # Log messages will be written to: LOG=/Library/WebObjects/Logs/SpawnOfWotaskd.log if [ -f /tmp/logWebObjects ]; then echo "************" >>${LOG} echo "date: `date`" >>${LOG} echo "args: $@" >>${LOG} $@ 1>>${LOG} 2>&1 & else $@ 1>/dev/null 2>&1 & fi On Thu, Feb 27, 2014 at 8:34 AM, John Pollard <j...@pollardweb.com<mailto:j...@pollardweb.com>> wrote: I am getting more intermittent app not found on server. Looking at the WO apache module log: A good request sequence logged looks like this: Info: received ->200 Apple Debug: <WebObjects Apache Module> new translate: /ires/8206/GOHWeb/miscRes/gohStyle11.css Debug: <WebObjects Apache Module> translate - DECLINED: /ires/8206/GOHWeb/miscRes/gohStyle11.css Debug: WebObjects_handler declined! /ires/8206/GOHWeb/miscRes/gohStyle11.css Debug: <WebObjects Apache Module> new translate: /favicon.ico Debug: <WebObjects Apache Module> translate - DECLINED: /favicon.ico Debug: WebObjects_handler declined! /favicon.ico Info: New request is GET /cgi-bin/WebObjects/MPMall.woa/1/wa/8/43.html HTTP/1.1^M Info: Sending request to instance number 1, port 2001 Info: Trying to contact MPMall:1 on (2001) Info: attempting to connect to localhost on port 2001 Info: MPMall:1 on (2001) connected [pooled: No] Info: Request GET /cgi-bin/WebObjects/MPMall.woa/1/wa/8/43.html HTTP/1.1^M sent, awaiting response Debug: ac_readConfiguration(): skipped reading config Info: New response: HTTP/1.1 302 Apple WebObjects A failed request logged: Info: received ->200 Apple Debug: <WebObjects Apache Module> new translate: /cgi-bin/WebObjects/MPMall.woa/wa/2/52/3689-Garden-Sprinklers/106568-DOG-SPRINKLER-BRONZE-FINISH.html Info: <WebObjects Apache Module> new request: /cgi-bin/WebObjects/MPMall.woa/wa/2/52/3689-Garden-Sprinklers/106568-DOG-SPRINKLER-BRONZE-FINISH.html Debug: App Name: MPMall.woa/wa/2/52/3689-Garden-Sprinklers/106568-DOG-SPRINKLER-BRONZE-FINISH.html (6) Info: V4 URL: /cgi-bin/WebObjects/MPMall.woa/wa/2/52/3689-Garden-Sprinklers/106568-DOG-SPRINKLER-BRONZE-FINISH.html Info: tr_selectInstance(): scheduler failed to select instance. Error: Request handling error: The requested application was not found on this server. Given these intermittent failures again, I thought I would start a couple of new instances, but they failed to start, with no log output. All I had done was to restart apache after configuring logging to debug. Things are going a bit pear shaped. John On 27 Feb 2014, at 10:34, John Pollard <j...@pollardweb.com<mailto:j...@pollardweb.com>> wrote: For the record, including: -_DeploymentDebugging true when running wotaskd didn't shed any light on why the apps don't start (no log files) A change of SiteConfig.xml to an older auto-saved one didn't help, so that seems fine Using ps to look for apps that wotaskd tries to start gives nothing, so they don't seem to be launching at all, or too briefly to spot with no log output I can run the apps manually as the appserver user. Is there now way for wotaskd to pass on the stdout/stderrror from the apps it tries to launch? On 27 Feb 2014, at 09:38, John Pollard <j...@pollardweb.com<mailto:j...@pollardweb.com>> wrote: Hi Chuck, On 26 Feb 2014, at 22:26, Chuck Hill <ch...@global-village.net<mailto:ch...@global-village.net>> wrote: Hi John, On 2/25/2014, 10:59 PM, "John Pollard" wrote: Thanks for the feedback, really useful. To try to stop the intermittent errors, I decided to restart wotaskd in this way: sudo /etc/init.d/webobjects restart & I think you can just kill it and let it get restarted automatically. I usually do this on OS X, but I think that init.d will do the same-ish thing. I know this works on my Mac dev box, but I just tried it on a test deployment server and it didn't restart after I killed it. /etc/init.d/webobjects uses ps aux | awk '/WOPort 1085/ && !/awk/ {print $2}' to find the process and kills it Somehow this created a cycle of reported app deaths and restarts in JavaMonitior which I only discovered in the early hours, though I don't see a long trail of app log files, suggesting the restarts might not have been happening in reality, or logs couldn't be written. At first I thought more than one wotaskd was starting, but that can’t be as there would be a port conflict. Usually killing wotaskd and letting it restart is clean and does not result in issues like this. wotaskd and JavaMonitor run as appserver and I couldn't see a permissions problem either in SiteConfig.xml or for the app log files When a reboot resulted in the same problem (now really panic), I launched a replacement live app server from a recent baseline and that was ok That seems a little odd. What had changed between the two? No change, it was a recent baseline. The only points were that: - I had been seeing the intermittent: The requested application was not found on this server. - I had tried to restart with sudo /etc/init.d/webobjects restart & (which I have subsequently shown seems to behave ok on another replicated server) One thought, can this ever be a problem?.... a WO server needs to start say 20 apps which is a big load on the system, if too big a load, could wotaskd think the apps failed to start and get into a cycle of restarting. I don't think this is it, but a thought. Yes, that can happen. To mitigate this, set Time Allowed For Startup and check Phased Startup in the Application Settings section of the Application Config page. Thanks, I do have Phased Startup and I have now doubled the startup times from 30 to 60, though I don't think this is it. The key thing seems to be that the apps weren't even getting to the point of producing any log output. You are so helpful Chuck, thank you. John Chuck On 26 Feb 2014, at 00:07, Chuck Hill <ch...@global-village.net<mailto:ch...@global-village.net>> wrote: I usually set the apps to restart once a week. I never restart wotaskd. Chuck On 2/25/2014, 7:09 AM, "John Pollard" wrote: Do WO deployments / wotaskd need any occasional restarts scheduled? Our live server uptime is currently 118 days. Might a wotaskd restart help with the intermittent "The requested application was not found on this server."? On 25 Feb 2014, at 11:50, John Pollard <j...@pollardweb.com<mailto:j...@pollardweb.com>> wrote: Over the past few weeks we have started to notice the occasional: "The requested application was not found on this server." >From experimenting, am I right in thinking this is from the apache WebObjects >module, because I am able to trigger this by stopping wotaskd? Why might the WebObjects apache module not be able to find wotaskd, very occasionally, if this is what is happening? The wotaskd we are running is a Wonder version from March 2013. The deployment is on Amazon Linux. No shortage of ram on the box and running a fairly light load e.g. 20,000 transactions a day across 6 app instances, averaging about 0.05 seconds per request with many seconds idle between request. The WO apache module used is: /usr/lib64/httpd/modules/mod_WebObjects.so -rwxr-xr-x 1 root root 395668 Apr 28 2011 mod_WebObjects.so Apache version: Server version: Apache/2.2.25 (Unix) Server built: Jul 12 2013 01:00:05 Any pointers as to how to track down the intermittent error would be a boost! Thanks John _______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com<mailto:Webobjects-deploy@lists.apple.com>) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/webobjects-deploy/john%40pollardweb.com This email sent to j...@pollardweb.com<mailto:j...@pollardweb.com> _______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com<mailto:Webobjects-deploy@lists.apple.com>) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/webobjects-deploy/chill%40global-village.net This email sent to ch...@global-village.net<mailto:ch...@global-village.net> _______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com<mailto:Webobjects-deploy@lists.apple.com>) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/webobjects-deploy/jpollard%40inrax.com This email sent to jpoll...@inrax.com<mailto:jpoll...@inrax.com> _______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com<mailto:Webobjects-deploy@lists.apple.com>) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/webobjects-deploy/jpollard%40inrax.com This email sent to jpoll...@inrax.com<mailto:jpoll...@inrax.com> _______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com<mailto:Webobjects-deploy@lists.apple.com>) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/webobjects-deploy/john%40pollardweb.com This email sent to j...@pollardweb.com<mailto:j...@pollardweb.com> _______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com<mailto:Webobjects-deploy@lists.apple.com>) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/webobjects-deploy/john%40pollardweb.com This email sent to j...@pollardweb.com<mailto:j...@pollardweb.com> _______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com<mailto:Webobjects-deploy@lists.apple.com>) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/webobjects-deploy/johnthuss%40gmail.com This email sent to johnth...@gmail.com<mailto:johnth...@gmail.com> _______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com<mailto:Webobjects-deploy@lists.apple.com>) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/webobjects-deploy/jpollard%40inrax.com This email sent to jpoll...@inrax.com<mailto:jpoll...@inrax.com> _______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com<mailto:Webobjects-deploy@lists.apple.com>) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/webobjects-deploy/jpollard%40inrax.com This email sent to jpoll...@inrax.com<mailto:jpoll...@inrax.com>
_______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/webobjects-deploy/archive%40mail-archive.com This email sent to arch...@mail-archive.com
