Back to my original problem, these errors are happening about one every 100
requests:
Info: <WebObjects Apache Module> new request:
/cgi-bin/WebObjects/MPMall.woa/wa/1/63/2392-Gilets-SALE.html
Debug: App Name: MPMall.woa/wa/1/63/2392-Gilets-SALE.html (6)
Info: V4 URL: /cgi-bin/WebObjects/MPMall.woa/wa/1/63/2392-Gilets-SALE.html
Info: tr_selectInstance(): scheduler failed to select instance.
Error: Request handling error: The requested application was not found on this
server.
I can't see logging from wotaskd in terms of making its choices about which
instance is next, even for the working requests. I tried passing
-_DeploymentDebugging true, when running wotaskd but the only logging I see is
Lifebeats coming in. Am I misunderstanding what wotaskd does?
The request never makes it to the application. Would there be much point in be
rebuilding the latest Wonder wotaskd? The one I have running is about a year
old. Or could the WO apache module be at fault, my mod_WebObjects.so file is
dated 38 April 2011, apache v. 2.2.25
Many thanks again,
John
On 27 Feb 2014, at 16:42, John Pollard <j...@pollardweb.com> wrote:
> John, a big thank you as this seems to have answered why I couldn't start
> apps at all.
> I don't have the LOG path shown, so changed it to one I use for logging and
> now can fire up apps again.
> Or removing /tmp/logWebObjects also fixes it.
> Perhaps this script should test for access to the LOG file and if no access,
> not launch the apps that way.
> Many thanks; I don't think this relates to my underlying intermittent "The
> requested application was not found on this server." but it brings a bit of
> sanity back!
> John
>
> On 27 Feb 2014, at 15:20, John Huss <johnth...@gmail.com> wrote:
>
>> If you're using Wonder's wotaskd (or care to replace the following script)
>> you can turn on logging for instance startup in:
>>
>> /Library/WebObjects/JavaApplications/wotaskd.woa/Contents/Resources/SpawnOfWotaskd.sh
>>
>> #!/bin/sh
>>
>> # To enable logging of instance startup run the command 'touch
>> /tmp/logWebObjects'
>>
>> # Log messages will be written to:
>> LOG=/Library/WebObjects/Logs/SpawnOfWotaskd.log
>>
>> if [ -f /tmp/logWebObjects ]; then
>>
>> echo "************" >>${LOG}
>> echo "date: `date`" >>${LOG}
>> echo "args: $@" >>${LOG}
>> $@ 1>>${LOG} 2>&1 &
>>
>> else
>>
>> $@ 1>/dev/null 2>&1 &
>>
>> fi
>>
>>
>> On Thu, Feb 27, 2014 at 8:34 AM, John Pollard <j...@pollardweb.com> wrote:
>> I am getting more intermittent app not found on server.
>> Looking at the WO apache module log:
>>
>> A good request sequence logged looks like this:
>> Info: received ->200 Apple
>> Debug: <WebObjects Apache Module> new translate:
>> /ires/8206/GOHWeb/miscRes/gohStyle11.css
>> Debug: <WebObjects Apache Module> translate - DECLINED:
>> /ires/8206/GOHWeb/miscRes/gohStyle11.css
>> Debug: WebObjects_handler declined! /ires/8206/GOHWeb/miscRes/gohStyle11.css
>> Debug: <WebObjects Apache Module> new translate: /favicon.ico
>> Debug: <WebObjects Apache Module> translate - DECLINED: /favicon.ico
>> Debug: WebObjects_handler declined! /favicon.ico
>> Info: New request is GET /cgi-bin/WebObjects/MPMall.woa/1/wa/8/43.html
>> HTTP/1.1^M
>> Info: Sending request to instance number 1, port 2001
>> Info: Trying to contact MPMall:1 on (2001)
>> Info: attempting to connect to localhost on port 2001
>> Info: MPMall:1 on (2001) connected [pooled: No]
>> Info: Request GET /cgi-bin/WebObjects/MPMall.woa/1/wa/8/43.html HTTP/1.1^M
>> sent, awaiting response
>> Debug: ac_readConfiguration(): skipped reading config
>> Info: New response: HTTP/1.1 302 Apple WebObjects
>>
>> A failed request logged:
>>
>> Info: received ->200 Apple
>> Debug: <WebObjects Apache Module> new translate:
>> /cgi-bin/WebObjects/MPMall.woa/wa/2/52/3689-Garden-Sprinklers/106568-DOG-SPRINKLER-BRONZE-FINISH.html
>> Info: <WebObjects Apache Module> new request:
>> /cgi-bin/WebObjects/MPMall.woa/wa/2/52/3689-Garden-Sprinklers/106568-DOG-SPRINKLER-BRONZE-FINISH.html
>> Debug: App Name:
>> MPMall.woa/wa/2/52/3689-Garden-Sprinklers/106568-DOG-SPRINKLER-BRONZE-FINISH.html
>> (6)
>> Info: V4 URL:
>> /cgi-bin/WebObjects/MPMall.woa/wa/2/52/3689-Garden-Sprinklers/106568-DOG-SPRINKLER-BRONZE-FINISH.html
>> Info: tr_selectInstance(): scheduler failed to select instance.
>> Error: Request handling error: The requested application was not found on
>> this server.
>>
>> Given these intermittent failures again, I thought I would start a couple of
>> new instances, but they failed to start, with no log output. All I had done
>> was to restart apache after configuring logging to debug.
>>
>> Things are going a bit pear shaped.
>>
>> John
>>
>> On 27 Feb 2014, at 10:34, John Pollard <j...@pollardweb.com> wrote:
>>
>>> For the record, including:
>>> -_DeploymentDebugging true
>>> when running wotaskd didn't shed any light on why the apps don't start (no
>>> log files)
>>> A change of SiteConfig.xml to an older auto-saved one didn't help, so that
>>> seems fine
>>> Using ps to look for apps that wotaskd tries to start gives nothing, so
>>> they don't seem to be launching at all, or too briefly to spot with no log
>>> output
>>> I can run the apps manually as the appserver user.
>>> Is there now way for wotaskd to pass on the stdout/stderrror from the apps
>>> it tries to launch?
>>>
>>> On 27 Feb 2014, at 09:38, John Pollard <j...@pollardweb.com> wrote:
>>>
>>>> Hi Chuck,
>>>>
>>>> On 26 Feb 2014, at 22:26, Chuck Hill <ch...@global-village.net> wrote:
>>>>
>>>>> Hi John,
>>>>>
>>>>> On 2/25/2014, 10:59 PM, "John Pollard" wrote:
>>>>>
>>>>> Thanks for the feedback, really useful. To try to stop the intermittent
>>>>> errors, I decided to restart wotaskd in this way:
>>>>>
>>>>> sudo /etc/init.d/webobjects restart &
>>>>>
>>>>> I think you can just kill it and let it get restarted automatically. I
>>>>> usually do this on OS X, but I think that init.d will do the same-ish
>>>>> thing.
>>>>
>>>> I know this works on my Mac dev box, but I just tried it on a test
>>>> deployment server and it didn't restart after I killed it.
>>>> /etc/init.d/webobjects uses
>>>> ps aux | awk '/WOPort 1085/ && !/awk/ {print $2}'
>>>> to find the process and kills it
>>>>
>>>>>
>>>>>
>>>>> Somehow this created a cycle of reported app deaths and restarts in
>>>>> JavaMonitior which I only discovered in the early hours, though I don't
>>>>> see a long trail of app log files, suggesting the restarts might not have
>>>>> been happening in reality, or logs couldn't be written.
>>>>>
>>>>> At first I thought more than one wotaskd was starting, but that can’t be
>>>>> as there would be a port conflict. Usually killing wotaskd and letting
>>>>> it restart is clean and does not result in issues like this.
>>>>>
>>>>>
>>>>>
>>>>> wotaskd and JavaMonitor run as appserver and I couldn't see a permissions
>>>>> problem either in SiteConfig.xml or for the app log files
>>>>>
>>>>> When a reboot resulted in the same problem (now really panic), I launched
>>>>> a replacement live app server from a recent baseline and that was ok
>>>>>
>>>>> That seems a little odd. What had changed between the two?
>>>>
>>>> No change, it was a recent baseline. The only points were that:
>>>> - I had been seeing the intermittent: The requested application was not
>>>> found on this server.
>>>> - I had tried to restart with sudo /etc/init.d/webobjects restart &
>>>> (which I have subsequently shown seems to behave ok on another replicated
>>>> server)
>>>>
>>>>>
>>>>>
>>>>> One thought, can this ever be a problem?.... a WO server needs to start
>>>>> say 20 apps which is a big load on the system, if too big a load, could
>>>>> wotaskd think the apps failed to start and get into a cycle of
>>>>> restarting. I don't think this is it, but a thought.
>>>>>
>>>>> Yes, that can happen. To mitigate this, set Time Allowed For Startup and
>>>>> check Phased Startup in the Application Settings section of the
>>>>> Application Config page.
>>>>
>>>> Thanks, I do have Phased Startup and I have now doubled the startup times
>>>> from 30 to 60, though I don't think this is it. The key thing seems to be
>>>> that the apps weren't even getting to the point of producing any log
>>>> output.
>>>>
>>>> You are so helpful Chuck, thank you.
>>>> John
>>>>
>>>>>
>>>>>
>>>>> Chuck
>>>>>
>>>>>
>>>>>
>>>>> On 26 Feb 2014, at 00:07, Chuck Hill <ch...@global-village.net> wrote:
>>>>>
>>>>>> I usually set the apps to restart once a week. I never restart wotaskd.
>>>>>>
>>>>>> Chuck
>>>>>>
>>>>>> On 2/25/2014, 7:09 AM, "John Pollard" wrote:
>>>>>>
>>>>>> Do WO deployments / wotaskd need any occasional restarts scheduled? Our
>>>>>> live server uptime is currently 118 days.
>>>>>> Might a wotaskd restart help with the intermittent "The requested
>>>>>> application was not found on this server."?
>>>>>>
>>>>>> On 25 Feb 2014, at 11:50, John Pollard <j...@pollardweb.com> wrote:
>>>>>>
>>>>>> Over the past few weeks we have started to notice the occasional: "The
>>>>>> requested application was not found on this server."
>>>>>> From experimenting, am I right in thinking this is from the apache
>>>>>> WebObjects module, because I am able to trigger this by stopping wotaskd?
>>>>>> Why might the WebObjects apache module not be able to find wotaskd, very
>>>>>> occasionally, if this is what is happening?
>>>>>> The wotaskd we are running is a Wonder version from March 2013.
>>>>>> The deployment is on Amazon Linux. No shortage of ram on the box and
>>>>>> running a fairly light load e.g. 20,000 transactions a day across 6 app
>>>>>> instances, averaging about 0.05 seconds per request with many seconds
>>>>>> idle between request.
>>>>>> The WO apache module used is: /usr/lib64/httpd/modules/mod_WebObjects.so
>>>>>> -rwxr-xr-x 1 root root 395668 Apr 28 2011 mod_WebObjects.so
>>>>>> Apache version:
>>>>>> Server version: Apache/2.2.25 (Unix)
>>>>>> Server built: Jul 12 2013 01:00:05
>>>>>> Any pointers as to how to track down the intermittent error would be a
>>>>>> boost!
>>>>>> Thanks
>>>>>> John
>>>>>> _______________________________________________
>>>>>> Do not post admin requests to the list. They will be ignored.
>>>>>> Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com)
>>>>>> Help/Unsubscribe/Update your Subscription:
>>>>>> https://lists.apple.com/mailman/options/webobjects-deploy/john%40pollardweb.com
>>>>>> This email sent to j...@pollardweb.com
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Do not post admin requests to the list. They will be ignored.
>>>>>> Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com)
>>>>>> Help/Unsubscribe/Update your Subscription:
>>>>>> https://lists.apple.com/mailman/options/webobjects-deploy/chill%40global-village.net
>>>>>>
>>>>>> This email sent to ch...@global-village.net
>>>>>> _______________________________________________
>>>>>> Do not post admin requests to the list. They will be ignored.
>>>>>> Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com)
>>>>>> Help/Unsubscribe/Update your Subscription:
>>>>>> https://lists.apple.com/mailman/options/webobjects-deploy/jpollard%40inrax.com
>>>>>>
>>>>>> This email sent to jpoll...@inrax.com
>>>>>
>>>>> _______________________________________________
>>>>> Do not post admin requests to the list. They will be ignored.
>>>>> Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com)
>>>>> Help/Unsubscribe/Update your Subscription:
>>>>> https://lists.apple.com/mailman/options/webobjects-deploy/jpollard%40inrax.com
>>>>>
>>>>> This email sent to jpoll...@inrax.com
>>>>
>>>> _______________________________________________
>>>> Do not post admin requests to the list. They will be ignored.
>>>> Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com)
>>>> Help/Unsubscribe/Update your Subscription:
>>>> https://lists.apple.com/mailman/options/webobjects-deploy/john%40pollardweb.com
>>>>
>>>> This email sent to j...@pollardweb.com
>>>
>>> _______________________________________________
>>> Do not post admin requests to the list. They will be ignored.
>>> Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com)
>>> Help/Unsubscribe/Update your Subscription:
>>> https://lists.apple.com/mailman/options/webobjects-deploy/john%40pollardweb.com
>>>
>>> This email sent to j...@pollardweb.com
>>
>>
>> _______________________________________________
>> Do not post admin requests to the list. They will be ignored.
>> Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com)
>> Help/Unsubscribe/Update your Subscription:
>> https://lists.apple.com/mailman/options/webobjects-deploy/johnthuss%40gmail.com
>>
>> This email sent to johnth...@gmail.com
>>
>> _______________________________________________
>> Do not post admin requests to the list. They will be ignored.
>> Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com)
>> Help/Unsubscribe/Update your Subscription:
>> https://lists.apple.com/mailman/options/webobjects-deploy/jpollard%40inrax.com
>>
>> This email sent to jpoll...@inrax.com
>
> _______________________________________________
> Do not post admin requests to the list. They will be ignored.
> Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com)
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/webobjects-deploy/jpollard%40inrax.com
>
> This email sent to jpoll...@inrax.com
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-deploy mailing list (Webobjects-deploy@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/webobjects-deploy/archive%40mail-archive.com
This email sent to arch...@mail-archive.com
