If you're using Wonder's wotaskd (or care to replace the following script) you can turn on logging for instance startup in:
/Library/WebObjects/JavaApplications/wotaskd.woa/Contents/Resources/SpawnOfWotaskd.sh #!/bin/sh # To enable logging of instance startup run the command 'touch /tmp/logWebObjects' # Log messages will be written to: LOG=/Library/WebObjects/Logs/SpawnOfWotaskd.log if [ -f /tmp/logWebObjects ]; then echo "************" >>${LOG} echo "date: `date`" >>${LOG} echo "args: $@" >>${LOG} $@ 1>>${LOG} 2>&1 & else $@ 1>/dev/null 2>&1 & fi On Thu, Feb 27, 2014 at 8:34 AM, John Pollard <j...@pollardweb.com> wrote: > I am getting more intermittent app not found on server. > Looking at the WO apache module log: > > A good request sequence logged looks like this: > Info: received ->200 Apple > Debug: <WebObjects Apache Module> new translate: > /ires/8206/GOHWeb/miscRes/gohStyle11.css > Debug: <WebObjects Apache Module> translate - DECLINED: > /ires/8206/GOHWeb/miscRes/gohStyle11.css > Debug: WebObjects_handler declined! > /ires/8206/GOHWeb/miscRes/gohStyle11.css > Debug: <WebObjects Apache Module> new translate: /favicon.ico > Debug: <WebObjects Apache Module> translate - DECLINED: /favicon.ico > Debug: WebObjects_handler declined! /favicon.ico > Info: New request is GET /cgi-bin/WebObjects/MPMall.woa/1/wa/8/43.html > HTTP/1.1^M > Info: Sending request to instance number 1, port 2001 > Info: Trying to contact MPMall:1 on (2001) > Info: attempting to connect to localhost on port 2001 > Info: MPMall:1 on (2001) connected [pooled: No] > Info: Request GET /cgi-bin/WebObjects/MPMall.woa/1/wa/8/43.html HTTP/1.1^M > sent, awaiting response > Debug: ac_readConfiguration(): skipped reading config > Info: New response: HTTP/1.1 302 Apple WebObjects > > A failed request logged: > > Info: received ->200 Apple > Debug: <WebObjects Apache Module> new translate: > /cgi-bin/WebObjects/MPMall.woa/wa/2/52/3689-Garden-Sprinklers/106568-DOG-SPRINKLER-BRONZE-FINISH.html > Info: <WebObjects Apache Module> new request: > /cgi-bin/WebObjects/MPMall.woa/wa/2/52/3689-Garden-Sprinklers/106568-DOG-SPRINKLER-BRONZE-FINISH.html > Debug: App Name: > MPMall.woa/wa/2/52/3689-Garden-Sprinklers/106568-DOG-SPRINKLER-BRONZE-FINISH.html > (6) > Info: V4 URL: > /cgi-bin/WebObjects/MPMall.woa/wa/2/52/3689-Garden-Sprinklers/106568-DOG-SPRINKLER-BRONZE-FINISH.html > Info: tr_selectInstance(): scheduler failed to select instance. > Error: Request handling error: The requested application was not found on > this server. > > Given these intermittent failures again, I thought I would start a couple > of new instances, but they failed to start, with no log output. All I had > done was to restart apache after configuring logging to debug. > > Things are going a bit pear shaped. > > John > > On 27 Feb 2014, at 10:34, John Pollard <j...@pollardweb.com> wrote: > > For the record, including: > -_DeploymentDebugging true > when running wotaskd didn't shed any light on why the apps don't start (no > log files) > A change of SiteConfig.xml to an older auto-saved one didn't help, so that > seems fine > Using ps to look for apps that wotaskd tries to start gives nothing, so > they don't seem to be launching at all, or too briefly to spot with no log > output > I can run the apps manually as the appserver user. > Is there now way for wotaskd to pass on the stdout/stderrror from the apps > it tries to launch? > > On 27 Feb 2014, at 09:38, John Pollard <j...@pollardweb.com> wrote: > > Hi Chuck, > > On 26 Feb 2014, at 22:26, Chuck Hill <ch...@global-village.net> wrote: > > Hi John, > > On 2/25/2014, 10:59 PM, "John Pollard" wrote: > > Thanks for the feedback, really useful. To try to stop the intermittent > errors, I decided to restart wotaskd in this way: > > sudo /etc/init.d/webobjects restart & > > > I think you can just kill it and let it get restarted automatically. I > usually do this on OS X, but I think that init.d will do the same-ish thing. > > > I know this works on my Mac dev box, but I just tried it on a test > deployment server and it didn't restart after I killed it. > /etc/init.d/webobjects uses > ps aux | awk '/WOPort 1085/ && !/awk/ {print $2}' > to find the process and kills it > > > > Somehow this created a cycle of reported app deaths and restarts in > JavaMonitior which I only discovered in the early hours, though I don't see > a long trail of app log files, suggesting the restarts might not have been > happening in reality, or logs couldn't be written. > > > At first I thought more than one wotaskd was starting, but that can't be > as there would be a port conflict. Usually killing wotaskd and letting it > restart is clean and does not result in issues like this. > > > > wotaskd and JavaMonitor run as appserver and I couldn't see a > permissions problem either in SiteConfig.xml or for the app log files > > When a reboot resulted in the same problem (now really panic), I > launched a replacement live app server from a recent baseline and that was > ok > > > That seems a little odd. What had changed between the two? > > > No change, it was a recent baseline. The only points were that: > - I had been seeing the intermittent: The requested application was not > found on this server. > - I had tried to restart with sudo /etc/init.d/webobjects restart & > (which I have subsequently shown seems to behave ok on another replicated > server) > > > > One thought, can this ever be a problem?.... a WO server needs to start > say 20 apps which is a big load on the system, if too big a load, could > wotaskd think the apps failed to start and get into a cycle of restarting. > I don't think this is it, but a thought. > > > Yes, that can happen. To mitigate this, set Time Allowed For Startup > and check Phased Startup in the Application Settings section of > the Application Config page. > > > Thanks, I do have Phased Startup and I have now doubled the startup times > from 30 to 60, though I don't think this is it. The key thing seems to be > that the apps weren't even getting to the point of producing any log output. > > You are so helpful Chuck, thank you. > John > > > > Chuck > > > > On 26 Feb 2014, at 00:07, Chuck Hill <ch...@global-village.net> wrote: > > I usually set the apps to restart once a week. I never restart wotaskd. > > Chuck > > On 2/25/2014, 7:09 AM, "John Pollard" wrote: > > Do WO deployments / wotaskd need any occasional restarts scheduled? Our > live server uptime is currently 118 days. > Might a wotaskd restart help with the intermittent "The requested > application was not found on this server."? > > On 25 Feb 2014, at 11:50, John Pollard <j...@pollardweb.com> wrote: > > Over the past few weeks we have started to notice the occasional: "The > requested application was not found on this server." > From experimenting, am I right in thinking this is from the apache > WebObjects module, because I am able to trigger this by stopping wotaskd? > Why might the WebObjects apache module not be able to find wotaskd, very > occasionally, if this is what is happening? > The wotaskd we are running is a Wonder version from March 2013. > The deployment is on Amazon Linux. No shortage of ram on the box and > running a fairly light load e.g. 20,000 transactions a day across 6 app > instances, averaging about 0.05 seconds per request with many seconds idle > between request. > The WO apache module used is: /usr/lib64/httpd/modules/mod_WebObjects.so > -rwxr-xr-x 1 root root 395668 Apr 28 2011 mod_WebObjects.so > Apache version: > Server version: Apache/2.2.25 (Unix) > Server built: Jul 12 2013 01:00:05 > Any pointers as to how to track down the intermittent error would be a > boost! > Thanks > John > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com) > Help/Unsubscribe/Update your Subscription: > > https://lists.apple.com/mailman/options/webobjects-deploy/john%40pollardweb.com > This email sent to j...@pollardweb.com > > > > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com) > Help/Unsubscribe/Update your Subscription: > > https://lists.apple.com/mailman/options/webobjects-deploy/chill%40global-village.net > > This email sent to ch...@global-village.net > > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com) > Help/Unsubscribe/Update your Subscription: > > https://lists.apple.com/mailman/options/webobjects-deploy/jpollard%40inrax.com > > This email sent to jpoll...@inrax.com > > > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com) > Help/Unsubscribe/Update your Subscription: > > https://lists.apple.com/mailman/options/webobjects-deploy/jpollard%40inrax.com > > This email sent to jpoll...@inrax.com > > > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com) > Help/Unsubscribe/Update your Subscription: > > https://lists.apple.com/mailman/options/webobjects-deploy/john%40pollardweb.com > > This email sent to j...@pollardweb.com > > > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com) > Help/Unsubscribe/Update your Subscription: > > https://lists.apple.com/mailman/options/webobjects-deploy/john%40pollardweb.com > > This email sent to j...@pollardweb.com > > > > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com) > Help/Unsubscribe/Update your Subscription: > > https://lists.apple.com/mailman/options/webobjects-deploy/johnthuss%40gmail.com > > This email sent to johnth...@gmail.com >
_______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/webobjects-deploy/archive%40mail-archive.com This email sent to arch...@mail-archive.com