I understand that on this issue my problem solving skills have been less then
perfect and have offered some incorrect assumptions, however, my last example
included in my previous message used exactly 1 server, not 2 and the problem
was 100% repeatable.
My original reasoning for using 2 servers was to test everything from the
connecting machine to the server and everything inbetween. As I was able to
narrow down problems I was able to reproduce the problem as simply as possible.
Once again.... when a filter uses a ns_adp_abort (or as I've tested now
ns_adp_break) the next time the same thread is used and asked to parse an adp,
the adp is not parsed, however, that thread can still properly serve static
content. I have now simplified this down to just the filter by putting the
ns_adp_abort into the filter. With all error reporting turned on, this time
the very helpful error message was generated (also, catching this error does
not help):
[07/Jul/2009:12:19:16][16953.3073735568][-default:0-] Error: Tcl exception:
while executing
"ns_adp_abort"
(procedure "decode_url" line 6)
invoked from within
"decode_url preauth"
Here is the filter:
ns_register_filter preauth GET /*.html decode_url
ns_register_filter preauth POST /*.html decode_url
ns_register_filter preauth HEAD /*.html decode_url
proc decode_url { why } {
set_form_variables 0
if {$count==1} {
ns_log notice "Thread Redirecting to Google and ns_adp_abort"
ns_returnredirect http://www.google.com/
ns_adp_abort
} {
set page [ns_adp_parse "Hello"]
ns_log notice "Thread with data size [string length $page]"
ns_return 200 "text/html" $page
}
return filter_return
}
You can access this anyway you'd like, from a web browser, wget, lynx, or, you
can use my script which requests the page a whole bunch of times which I have
included before.
I recommend watching the server.log as you access pages, since the server log
will tell you the thread numbers.
You can request
http://www.YOURWEBSITE.com/page.html?count=2
as much as you'd like, but once you request
http://www.YOURWEBSITE.com/page.html?count=1
that very helpful error is generated, and the page redirects properly.
Make a note of the thread number and continue requesting
http://www.YOURWEBSITE.com/page.html?count=2
Watch the server.log, when that thread number is used again, the adp is not
parsed**. If the thread is used to server a static file, it serves fine,
however, if it is ever asked to serve an adp in the future it will fail**.
**Just a note here... there is something strange here with when the thread may
be used again to properly serve an adp. I haven't looked into it too far but
it appears that if a request is made through the filter (ie, in this case a
.html file) the adp is NOT parsed and the thread is NOT cleaned. However, if
an adp is called directly, such as http://www.MYSITE.com/page.adp (ie, an adp
is requested but not through the filter) then the adp is NOT parsed, but the
thread IS cleaned and will then parse the next time it is asked to parse an
adp, filter or not. (next time as in the next time the thread is used, it will
continue to always not parse adps until it is closed/dropped/released, whatever
that terminology would be)
The difficulty in finally coming to this conclusion was of course the apparent
random nature of the problem, since the problem was generated by one request,
yet did not materialize until another request x seconds later, which must be an
adp, it was difficult to narrow down. I was only able to find it after setting
up a new physical server, running my tests which all worked and thinking it had
something to do with my server setup, yet when I went started testing of the
site on the new server discovered the same problem.
Now you can debate whether or not ns_adp_abort / ns_adp_break should be called
in a filter, or in an adp called from a filter, or whether my content
management / template system is a good or bad method of serving content. But
until then,
Is there a way to do something like ns_adp_unabort?
Is there something besides ns_adp_abort and ns_adb_break that will return all
the way up the callstack?
Is there a way to expire a thread before it's max_connections are met?
Does anyone have any other suggestions?
_Peter
> Date: Tue, 7 Jul 2009 09:37:48 -0700
> From: t...@rmadilo.com
> Subject: Re: [AOLSERVER] ns_adp_parse issue
> To: AOLSERVER@LISTSERV.AOL.COM
>
> Francesco,
>
> Okay here is the thing: you keep using two servers to diagnose an API
> which doesn't appear in any of the code you are using. Just use one page
> on the server that you think does not work. You don't need to include
> any other page, parse any other page or do anything fancy. If you do,
> you will have no idea where the problem is or what it is. You could keep
> guessing for years until you do this, and it is a waste of your time and
> everyone else's time.
>
> But even if your three page/two server setup could narrow down the
> problem, the fact is that the problem doesn't always happen. If it
> doesn't happen every time, the failure must be due to specific
> conditions or configurations, or a bug in your code.
>
> Right now you have no idea what is going on, and neither do I, but your
> debugging method cannot ever work to find out. My opinion is learn to
> use telnet to request a simple page which reproduces the behavior.
>
> My opinion is that it is just a programming bug and has nothing to do
> with ns_adp_abort, but you are not going to find the bug by using
> ns_httpget, and you may not find it using a regular browser or command
> line client like wget of curl.
>
> tom jackson
>
> On Mon, 2009-07-06 at 21:04 +0000, Francesco Petrarch wrote:
> > Ok, I got this, this time I really do, and it should be easily
> > replicated by anyone.
> >
> > I have this filter:
> >
> > ns_register_filter preauth GET /*.html decode_url
> > ns_register_filter preauth POST /*.html decode_url
> > ns_register_filter preauth HEAD /*.html decode_url
> > proc decode_url { why } {
> > set page [ns_adp_parse -file /www/website/template.adp]
> > ns_return 200 "text/html" "$page"
> > return filter_return
> > }
> >
> > template.adp then does a [ns_conn url] etc etc to serve the correct
> > content into the template. However, if template.adp does a
> > ns_adp_abort itself (or any of the files which may be ns_adp_parsed or
> > ns_adp_included etc) then the thread gets "corrupted".
> >
> >
> > Here is a simple way on how to replicate this with just a single
> > server.
> >
> > First create a filter:
> >
> > ns_register_filter preauth GET /*.html decode_url
> > ns_register_filter preauth POST /*.html decode_url
> > ns_register_filter preauth HEAD /*.html decode_url
> > proc decode_url { why } {
> > set page [ns_adp_parse -file /PATHTOWWWFILES/testpage.html]
> > ns_return 200 "text/html" "$page"
> > return filter_return
> > }
> >
> >
> > Second create the file testpage.adp
> >
> > <%
> > set ret ""
> > set max 200
> > set count 0
> > while {$count<$max} {
> > incr count
> > set page [ns_httpget
> > "http://www.YOURWEBSITE.com/testpage.html?count=$count";]
> >
> > if {[string length $page]!=0} {
> > if {$count==50} {
> > append ret "First 50 requests ok<br>"
> > }
> > if {$count==$max} {
> > append ret "Request Max ok<br>"
> > }
> > } {
> > append ret "request $count had filesize [string length
> > $page]<br>"
> > }
> > }
> > ns_puts $ret
> > %>
> >
> >
> > Finally create testpage.html (note the last line Testdata, so we have
> > some text on the page)
> >
> > <%
> > set_form_variables 0
> > if {$count==50} {
> > # ns_adp_abort
> > }
> > %>
> > Testdata
> >
> >
> > Now, in your webbrowser go to http://www.YOURSITE.com/testpage.adp,
> > after a few seconds you should see a webpage like
> >
> > First 50 requests ok
> > Request Max ok
> >
> > You can reload this a number of times and always see the same thing.
> > Now edit testpage.html by removing the comment on the ns_adp_abort
> > line
> > Then reload http://www.YOURSITE.com/testpage.adp
> >
> > You will see something like this:
> >
> > request 50 had filesize 0
> > request 113 had filesize 0
> > request 176 had filesize 0
> > Request Max ok
> >
> > Keep reloading for lots of fun!
> >
> > I have tried modifying the filter to account for a redirect and
> > returning filter_break or filter_ok but with no luck. What is
> > interesting to note is that you can put ns_log notices in the filter,
> > and the requests that return 0 in size do get into the filter, but the
> > ns_adp_parse just will just return an empty string. I am thinking
> > that this is because 'ns_adp_abort' is still set for the thread?
> >
> > I am uncertain if this is a bug. If there something other than
> > ns_adp_abort I could use after a ns_returnredirect which will return
> > all the way back up to the filter? I believe ns_adp_return will just
> > return 1 level up and ns_adp_break gives the same "problem" of also
> > breaking future connections. Or is there a way I can ns_adp_unabort
> > at the start of the filter?
> >
> > _Peter
> >
> >
> >
> >
> > ______________________________________________________________________
> > Date: Mon, 6 Jul 2009 10:42:28 -0700
> > From: t...@rmadilo.com
> > Subject: Re: [AOLSERVER] ns_adp_parse issue
> > To: AOLSERVER@LISTSERV.AOL.COM
> >
> > Did I miss something?
> >
> > Your script below does not contain a redirect and does not contain
> > ns_adp_abort. Why do you think this script tells you anything about
> > your problem?
> >
> > But one thing seems apparent: this adp script should return nothing.
> > It is just a script, it doesn't produce content. If no content is
> > produced, then it is working.
> >
> > On Sun, Jul 5, 2009 at 10:32 PM, Francesco Petrarch
> > <f_petra...@hotmail.com> wrote:
> > Hummmm, on further testing I guess I was pretty much
> > completely wrong.... I guess I spoke too soon.....
> >
> > I have tried to turn on all error logging but have come up
> > with nothing. I am able to intentionally kill a thread,
> > though I am not entirely sure how :S
> >
> > There are some pages I can request that will kill an
> > additional ADP thread, at first I thought this was simply the
> > ns_returnredirect or the ns_adp_abort but seems to be
> > something else since I can't duplicate it with all
> > redirects/aborts (although it does happen each time these
> > specific pages are requested). For example, the profile.html
> > page redirects to the login/signup page if you are not logged
> > in and if it does so causes "the problem"
> >
> > Here is the latest.....
> >
> > I have a script at http://SERVERA/test .adp
> >
> > <%
> > set count 0
> > while {$count<20000} {
> > incr count
> > set page [ns_httpget "http://SERVERB/file.html?count=
> > $count"]
> >
> > ns_log notice "page $count size [string length $page]"
> >
> > if {[string length $page]<2} { break }
> > }
> > %>
> >
> > it basically requests a file 20000 times from SERVERB (which
> > also hosts the aforementioned profile.html page) and stops if
> > the file size is small. The file file.html contains just a
> > single word "test". If .html is not in "ns_section
> > ns/server/serverb/adp" then "the problem" does not happen, it
> > only occurs when file.html may or may not need the parser.
> > In the event that .html is not in the ns_section and is
> > static, fastpath settings do not seem to make a difference,
> > everything works.
> >
> > Now.... from my observations, it appears that when a file such
> > as the aforementioned profile.html page is requested and the
> > redirect happens, that thread becomes contaminated. The next
> > time that contaminated thread is used for an adp "the problem"
> > appears. It appears as though the ns_adp_abort value is still
> > set and adps can not be parsed, even if the file.html file has
> > no <% %> tags, if .html is set as a file to be parsed then an
> > empty string is returned, however, just once. Once this
> > thread is closed again it seems fixed for future requests.
> > However, this "contamination" is only cleaned when the thread
> > tries to serve/parse an adp, if it's serving static content
> > the contamination is not removed. Going back to the filters I
> > had originally I believe the request still comes in for the
> > page with the adp, the filter can log info, but ns_adp_parse
> > just returns an empty string.
> >
> > Anyways, I'll be digging further into this, trying to see what
> > exactly is "contaminating" the thread as it appears now. To
> > repeat, this "problem" is repeatable everytime profile.html is
> > requested, and if it is related to ns_adp_abort or
> > ns_adp_redirect there must be other variables to the problem
> > since I can not seem to duplicate the problem with just those
> > functions. Also I do not use ns_perms, I simply set a cookie
> > once someone is logged in.
> >
> > I don't suppose there is a ns_adp_unabort
> >
> > _Peter
> >
> >
> >
> >
> >
> > ______________________________________________________________
> > Date: Sun, 5 Jul 2009 18:50:53 -0700
> > From: t...@rmadilo.com
> >
> > Subject: Re: [AOLSERVER] ns_adp_parse issue
> > To: AOLSERVER@LISTSERV.AOL.COM
> >
> >
> >
> >
> >
> > On Sun, Jul 5, 2009 at 6:13 PM, Francesco Petrarch
> > <f_petra...@hotmail.com> wrote:
> > It's been awhile since I have looked into this and I
> > think I have tracked it down. In short, I now believe
> > that ns_adp_abort aborts all current threads/adps, not
> > just the one currently executing the ns_adp_abort
> > line. Therefore, when these "blank screens" were
> > appearing on my site to VISITOR A it was because
> > another visitor (VISITOR B) had visited a page which
> > called ns_adp_abort (for example after a
> > ns_returnredirect) before the VISITOR A thread
> > finished.
> >
> > Does that make sense? Can someone with more
> > experience in the code take a look and let me know?
> >
> >
> > No, makes no sense. This is essentially impossible. Each
> > thread executes pretty much independently of all others. The
> > problem probably has to do with the adp configuration...error
> > without signaling an error, which is a configuration which is
> > necessary to keep complex sites up if one component is not
> > working.
> >
> > tom jackson
> >
> >
> >
> >
> > --
> > AOLserver - http://www.aolserver.com/
> >
> >
> > To Remove yourself from this list, simply send an email to
> > <lists...@listserv.aol.com> with the
> >
> > body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> > Subject: field of your email blank.
> >
> >
> >
> > ______________________________________________________________
> > Windows Live helps you keep up with all your friends, in one
> > place.
> >
> > --
> > AOLserver - http://www.aolserver.com/
> >
> >
> > To Remove yourself from this list, simply send an email to
> > <lists...@listserv.aol.com> with the
> > body of "SIGNOFF AOLSERVER" in the email message. You can
> > leave the Subject: field of your email blank.
> >
> >
> >
> >
> > --
> > AOLserver - http://www.aolserver.com/
> >
> >
> > To Remove yourself from this list, simply send an email to
> > <lists...@listserv.aol.com> with the
> > body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> > Subject: field of your email blank.
> >
> >
> >
> > ______________________________________________________________________
> > Attention all humans. We are your photos. Free us.
> >
> > --
> > AOLserver - http://www.aolserver.com/
> >
> >
> >
> > To Remove yourself from this list, simply send an email to
> > <lists...@listserv.aol.com> with the
> > body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> > Subject: field of your email blank.
> >
> >
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to
> <lists...@listserv.aol.com> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
> field of your email blank.
_________________________________________________________________
Attention all humans. We are your photos. Free us.
http://go.microsoft.com/?linkid=9666046
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
<lists...@listserv.aol.com> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
field of your email blank.
