While writing my last reply, I thought of a few things to test. I modified my
loop of ns_httpget calls to access the template.adp file directly, instead of
members.html thus completely avoiding the ns_adp_parse call.
The result, I still recieved some pages of 0 size. So the problem suddently
looked like it may not be an ns_adp_parse error but something else, though it
still had the possibility of being so since there are ns_adp_include calls in
template.adp, and template.adp itself is an adp that needs to be parsed.
So I modify template.adp to template2.adp and I put some ns_logs to track the
progress of each request and made some minor modifactions to give each each
request a unique url for easier tracking.
So my loop on the server I send the requests is now:
<%
set count 0
while {$count<2000} {
incr count
set page [ns_httpget "http://www.mysebsite.com/template2.adp?count=$count";]
ns_log notice "page $count size [string length $page]"
if {[string length $page]<2} { break }
}
%>
where $count is just an incremental counter and then template2.adp consists of
only the following....
alpha
<%
ns_log notice "[ns_conn peeraddr] connected to template for [ns_conn url]"
ns_log notice "[ns_conn peeraddr] connected to template for [ns_conn url]
setting form vars"
set_form_variables 0
ns_log notice "[ns_conn peeraddr] connected to template for [ns_conn url] set
$count"
%>
beta
The result..... some pages with a returned $page size of 0
Some sample log lines from the server which sends the requests:
[06/Jun/2009:00:13:07][31912.2948537232][-default:387-] Notice: page 954 size 13
[06/Jun/2009:00:13:07][31912.2948537232][-default:387-] Notice: page 955 size 13
[06/Jun/2009:00:13:07][31912.2948537232][-default:387-] Notice: page 956 size 0
And from the server serving the pages:
[05/Jun/2009:23:19:02][31792.3058563984][-default:50-] Notice: 70.85.172.155
connected to template for /template2.adp
[05/Jun/2009:23:19:02][31792.3058563984][-default:50-] Notice: 70.85.172.155
connected to template for /template2.adp setting form vars
[05/Jun/2009:23:19:02][31792.3058563984][-default:50-] Notice: 70.85.172.155
connected to template for /template2.adp set 954
[05/Jun/2009:23:19:02][31792.3060997008][-default:44-] Notice: 70.85.172.155
connected to template for /template2.adp
[05/Jun/2009:23:19:02][31792.3060997008][-default:44-] Notice: 70.85.172.155
connected to template for /template2.adp setting form vars
[05/Jun/2009:23:19:02][31792.3060997008][-default:44-] Notice: 70.85.172.155
connected to template for /template2.adp set 955
So on request 956 nothing was returned, not the word 'alpha' or 'beta' and the
code in <% %> never gets executed to make the log entries.
I'm going to get my much needed beauty sleep now and continue this in the
morning, testing non-adp file requests. My suspisions are heavy on the side
that something is happening when the file is sent to the parser, since when a
.html request is made, the filter still logs the access with ns_log (the page
request also appears in the webstats log with a file size of 0).
Whatever the problem I am fairly certain the issue must have something to do
with the physical server's settings, since such a problem with Aolserver itself
would have surely been noticed by others.
_Peter
> Date: Fri, 5 Jun 2009 17:01:23 -0700
> From: [email protected]
> Subject: Re: [AOLSERVER] ns_adp_parse issue
> To: [email protected]
>
> I would try testing on a page from a local server so you can verify that
> the server is returning what you expect.
>
> Also, log the actual content when it is not what you expect. you have
> access to the $page value, maybe it is an error message.
>
> Obviously something is failing, but nothing you have shown us narrows
> the location of the possible problem. Is it the adp, the configuration
> of adps or the error log, or is the foreign server screwing up, or your
> network. The fact that the code works most of the time means you
> probably need to look at every step and the value of every step.
>
> That said, my guess something is going wrong with the ns_httpget
> operation, maybe it gets oveloaded, times out or something else.
>
> tom jackson
>
> On Fri, 2009-06-05 at 20:56 +0000, Francesco Petrarch wrote:
> > Thanks for the suggestions, however they have not provided any
> > positive results.
> >
> > ns_adp_ctl stricterror 1 - did not yield any additional errors/notices
> > (nothing is written to the log files when this happens)
> > ns_param StackSize 2048000 - only succeeding in causing aolserver to
> > crash with "could not allocate memory' errors after a day of running
> >
> > For interest sake, the way I test this is from a different server I
> > have this little script:
> >
> > <%
> > set count 0
> > while {$count<1000} {
> > incr count
> > set page [ns_httpget http://www.mysite.com/members.html]
> > ns_log notice "page $count size [string length $page]"
> > if {[string length $page]<1000} { break }
> > }
> > %>
> >
> > And some sample lines form that logfile are:
> >
> > [05/Jun/2009:16:26:47][31912.2950486928][-default:394-] Notice: page
> > 86 size 38660
> > [05/Jun/2009:16:26:47][31912.2950486928][-default:394-] Notice: page
> > 87 size 38660
> > [05/Jun/2009:16:26:47][31912.2950486928][-default:394-] Notice: page
> > 88 size 38660
> > [05/Jun/2009:16:26:48][31912.2950486928][-default:394-] Notice: page
> > 89 size 38660
> > [05/Jun/2009:16:26:48][31912.2950486928][-default:394-] Notice: page
> > 90 size 38660
> > [05/Jun/2009:16:26:48][31912.2950486928][-default:394-] Notice: page
> > 91 size 38660
> > [05/Jun/2009:16:26:48][31912.2950486928][-default:394-] Notice: page
> > 92 size 38660
> > [05/Jun/2009:16:26:48][31912.2950486928][-default:394-] Notice: page
> > 93 size 38660
> > [05/Jun/2009:16:26:48][31912.2950486928][-default:394-] Notice: page
> > 94 size 87
> >
> > When I didn't return the javascript on a failure the size would be 0
> > not 87.
> >
> > I have yet to try Dossy's suggestion of using the startpage because I
> > would rather try and find out what the issue actually is incase there
> > are other issues involved with it. Plus since the javascript is
> > currently a working bandaid fix I am in no hurry, the only real
> > downside with my hack is a search engine may index it.
> >
> > What puzzels me is that, as I mentioned, these 0 sized ns_adp_parse
> > calls seem to happen at random, when the page it happens on is
> > reloaded things work fine, so all the input data is the same.
> > Combined with the fact when I attempt to call ns_adp_parse multiple
> > times when it does return a 0 sized string, all successive calls also
> > return a 0 sized string leads me to think something in the
> > ns_adp_parse functionality is failing without throwing an error...
> > maybe a lock contention somewhere? I even put a ns_sleep 1 between
> > ns_adp_parse calls but even after 10 seconds it returns a 0 length
> > string.
> >
> > I will now be digging into the Aolserver code itself to see what I can
> > find.... logging to the log file between every line of code if I must
> > to find out where this is failing.
> >
> > Thanks for the help so far,
> >
> > _Peter
> >
> >
> >
> > > Date: Fri, 29 May 2009 09:06:11 -0700
> > > From: [email protected]
> > > Subject: Re: [AOLSERVER] ns_adp_parse issue
> > > To: [email protected]
> > >
> > > Maybe there is an error which is silently caught in ns_adp_parse.
> > > Try this before sourcing your adp:
> > >
> > > ns_adp_ctl stricterror 1
> > >
> > > tom jackson
> > >
> > > On Fri, 2009-05-29 at 14:48 +0000, Francesco Petrarch wrote:
> > > > I appear to be having a problem with ns_adp_parse and ns_adp_eval.
> > I
> > > > currently run the latest code 4.5.1, upgraded from 3.13 with Jerry
> > > > Asher's patches for unix sockets for multiple hosts. Everything
> > > > worked fine on 3.13 but now I have intermittent issues with 4.5.1
> > (I
> > > > had acutally upgraded in November and have the issue with all 4.5
> > > > updates since then).
> > > >
> > > > My website serves about 140 million requests a month, about 3
> > million
> > > > of those are .html requests.
> > > > I don't use ACS, everything is custom coded and my CMS is based on
> > the
> > > > following filter:
> > > >
> > > >
> > > > ns_register_filter preauth GET /*.html decode_url
> > > > ns_register_filter preauth POST /*.html decode_url
> > > > proc decode_url { why } {
> > > > ns_return 200 "text/html" [ns_adp_parse
> > > > -file /www/website/template.adp]
> > > > return filter_return
> > > > }
> > > >
> > > >
> > > > Essentially, every .html page goes through this filter, and the
> > > > file /www/website/template.adp is a graphical template/layout for
> > the
> > > > site and dynamically includes the content for the give [ns_conn
> > url].
> > > > As I mentioned this worked fine for 3.13
> > > >
> > > > However, seemly at random times (for a random connection),
> > > > [ns_adp_parse -file /www/website/template.adp] will return an
> > empty
> > > > string. I have modified the filter to reattempt the ns_adp_parse
> > if
> > > > the returned value was an empty string (ie: [string length
> > $page]==0)
> > > > and logged the progress, but on a connection where ns_adp_parse
> > > > returns the empty string it appears to always return an empty
> > string.
> > > > ns_adp_eval also returns an empty string when this happens. With
> > the
> > > > above filter the visitor to the website would see just a blank
> > screen
> > > > because of course the content was just "". To be exact here, I say
> > > > empty string, to be more accurate it returns $page where [string
> > > > length $page]==0
> > > >
> > > > My current little fix is this:
> > > >
> > > >
> > > > ns_register_filter preauth GET /*.html decode_url
> > > > ns_register_filter preauth POST /*.html decode_url
> > > > proc decode_url { why } {
> > > > set page [ns_adp_parse -file /www/website/template.adp]
> > > > set size [string length $page]
> > > > if {$size==0} {
> > > > ns_log notice "Page [ns_conn url] ns_adp_parse $size in size for
> > > > visitor from [ns_conn peeraddr]"
> > > > set page "<body onload='window.location.reload()'>"
> > > > }
> > > > ns_return 200 "text/html" $page
> > > > return filter_return
> > > > }
> > > >
> > > >
> > > > As you can see, not so much a fix as just creating a little
> > javascript
> > > > to reload the page. At least the visitor to the website is no
> > longer
> > > > presented with a blank page, but not as ideal as an actual fix to
> > the
> > > > empty string issue.
> > > >
> > > > I have thought that this may be an issue with threads or some
> > limit
> > > > with handles to databases or maybe the adp parser (I really don't
> > know
> > > > how that works). However, given that this issue was not present on
> > > > 3.13, and no errors are generated I am at a loss. I have also
> > tried
> > > > messing with the config, changing just about everything to see if
> > it
> > > > made any difference, it didn't.
> > > >
> > > > I will be continuing to try and debug this to confirm
> > > > that /www/website/template.adp is being parsed and so on, but
> > before I
> > > > got too involved I figured I'd try this list to see if anyone
> > knows of
> > > > similar issues.
> > > >
> > > > Thanks for your help,
> > > >
> > > > _Peter
> > > >
> > > >
> > > >
> > > >
> > ______________________________________________________________________
> > > > Create a cool, new character for your Windows Live™ Messenger.
> > Check
> > > > it out
> > > >
> > > > --
> > > > AOLserver - http://www.aolserver.com/
> > > >
> > > >
> > > >
> > > > To Remove yourself from this list, simply send an email to
> > <[email protected]> with the
> > > > body of "SIGNOFF AOLSERVER" in the email message. You can leave
> > the Subject: field of your email blank.
> > > >
> > > >
> > >
> > >
> > > --
> > > AOLserver - http://www.aolserver.com/
> > >
> > > To Remove yourself from this list, simply send an email to
> > <[email protected]> with the
> > > body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> > Subject: field of your email blank.
> >
> >
> > ______________________________________________________________________
> > Windows Live helps you keep up with all your friends, in one place.
> >
> > --
> > AOLserver - http://www.aolserver.com/
> >
> >
> >
> > To Remove yourself from this list, simply send an email to
> > <[email protected]> with the
> > body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> > Subject: field of your email blank.
> >
> >
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to
> <[email protected]> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
> field of your email blank.
_________________________________________________________________
Attention all humans. We are your photos. Free us.
http://go.microsoft.com/?linkid=9666046
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
<[email protected]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
field of your email blank.
