Re: [AOLSERVER] ns_adp_parse issue
Francesco,
Okay here is the thing: you keep using two servers to diagnose an API
which doesn't appear in any of the code you are using. Just use one page
on the server that you think does not work. You don't need to include
any other page, parse any other page or do anything fancy. If you do,
you will have no idea where the problem is or what it is. You could keep
guessing for years until you do this, and it is a waste of your time and
everyone else's time.
But even if your three page/two server setup could narrow down the
problem, the fact is that the problem doesn't always happen. If it
doesn't happen every time, the failure must be due to specific
conditions or configurations, or a bug in your code.
Right now you have no idea what is going on, and neither do I, but your
debugging method cannot ever work to find out. My opinion is learn to
use telnet to request a simple page which reproduces the behavior.
My opinion is that it is just a programming bug and has nothing to do
with ns_adp_abort, but you are not going to find the bug by using
ns_httpget, and you may not find it using a regular browser or command
line client like wget of curl.
tom jackson
On Mon, 2009-07-06 at 21:04 +, Francesco Petrarch wrote:
Ok, I got this, this time I really do, and it should be easily
replicated by anyone.
I have this filter:
ns_register_filter preauth GET /*.html decode_url
ns_register_filter preauth POST /*.html decode_url
ns_register_filter preauth HEAD /*.html decode_url
proc decode_url { why } {
set page [ns_adp_parse -file /www/website/template.adp]
ns_return 200 text/html $page
return filter_return
}
template.adp then does a [ns_conn url] etc etc to serve the correct
content into the template. However, if template.adp does a
ns_adp_abort itself (or any of the files which may be ns_adp_parsed or
ns_adp_included etc) then the thread gets corrupted.
Here is a simple way on how to replicate this with just a single
server.
First create a filter:
ns_register_filter preauth GET /*.html decode_url
ns_register_filter preauth POST /*.html decode_url
ns_register_filter preauth HEAD /*.html decode_url
proc decode_url { why } {
set page [ns_adp_parse -file /PATHTOWWWFILES/testpage.html]
ns_return 200 text/html $page
return filter_return
}
Second create the file testpage.adp
%
set ret
set max 200
set count 0
while {$count$max} {
incr count
set page [ns_httpget
http://www.YOURWEBSITE.com/testpage.html?count=$count;]
if {[string length $page]!=0} {
if {$count==50} {
append ret First 50 requests okbr
}
if {$count==$max} {
append ret Request Max okbr
}
} {
append ret request $count had filesize [string length
$page]br
}
}
ns_puts $ret
%
Finally create testpage.html (note the last line Testdata, so we have
some text on the page)
%
set_form_variables 0
if {$count==50} {
# ns_adp_abort
}
%
Testdata
Now, in your webbrowser go to http://www.YOURSITE.com/testpage.adp,
after a few seconds you should see a webpage like
First 50 requests ok
Request Max ok
You can reload this a number of times and always see the same thing.
Now edit testpage.html by removing the comment on the ns_adp_abort
line
Then reload http://www.YOURSITE.com/testpage.adp
You will see something like this:
request 50 had filesize 0
request 113 had filesize 0
request 176 had filesize 0
Request Max ok
Keep reloading for lots of fun!
I have tried modifying the filter to account for a redirect and
returning filter_break or filter_ok but with no luck. What is
interesting to note is that you can put ns_log notices in the filter,
and the requests that return 0 in size do get into the filter, but the
ns_adp_parse just will just return an empty string. I am thinking
that this is because 'ns_adp_abort' is still set for the thread?
I am uncertain if this is a bug. If there something other than
ns_adp_abort I could use after a ns_returnredirect which will return
all the way back up to the filter? I believe ns_adp_return will just
return 1 level up and ns_adp_break gives the same problem of also
breaking future connections. Or is there a way I can ns_adp_unabort
at the start of the filter?
_Peter
__
Date: Mon, 6 Jul 2009 10:42:28 -0700
From: t...@rmadilo.com
Subject: Re: [AOLSERVER] ns_adp_parse issue
To: AOLSERVER@LISTSERV.AOL.COM
Did I miss something?
Your script below does not contain a redirect and does not contain
ns_adp_abort
Re: [AOLSERVER] ns_adp_parse issue
Francesco Petrarch wrote:
Ok, I got this, this time I really do, and it should be easily
replicated by anyone.
I have this filter:
ns_register_filter preauth GET /*.html decode_url
ns_register_filter preauth POST /*.html decode_url
ns_register_filter preauth HEAD /*.html decode_url
proc decode_url { why } {
set page [ns_adp_parse -file /www/website/template.adp]
ns_return 200 text/html $page
return filter_return
}
Your problem has nothing to do with ns_httpget.
The problem is that the ADP execution context (which is tied to the tcl
interpreter executing the request) does not get reset until the adp
request completes, which normally happens when the output is flushed at
the end of an adp page. Since you are calling the adp page within a
filter, the full adp page handler is never executed, so the cleanup
never happens, and the adp interpreter is in the aborted state, which
means that it never executes any code.
You can use ns_adp_exception state to get the current exception state
of the adp interpreter. And you can call ns_adp_return to set the
exception state to return which I believe will get turned into ok
the next time through. For example:
proc decode_url { why } {
set page [ns_adp_parse -file /www/website/template.adp]
ns_return 200 text/html $page
ns_adp_exception state
ns_log notice adp exception state is $state
catch {ns_adp_return}
return filter_return
}
It appears that adp code is only expected to be run from a handler, not
from a filter. Perhaps a reset subcommand should be added to
ns_adp_ctl to expose NsAdpReset at the tcl level for cases like this.
-J
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] ns_adp_parse issue
I understand that on this issue my problem solving skills have been less then
perfect and have offered some incorrect assumptions, however, my last example
included in my previous message used exactly 1 server, not 2 and the problem
was 100% repeatable.
My original reasoning for using 2 servers was to test everything from the
connecting machine to the server and everything inbetween. As I was able to
narrow down problems I was able to reproduce the problem as simply as possible.
Once again when a filter uses a ns_adp_abort (or as I've tested now
ns_adp_break) the next time the same thread is used and asked to parse an adp,
the adp is not parsed, however, that thread can still properly serve static
content. I have now simplified this down to just the filter by putting the
ns_adp_abort into the filter. With all error reporting turned on, this time
the very helpful error message was generated (also, catching this error does
not help):
[07/Jul/2009:12:19:16][16953.3073735568][-default:0-] Error: Tcl exception:
while executing
ns_adp_abort
(procedure decode_url line 6)
invoked from within
decode_url preauth
Here is the filter:
ns_register_filter preauth GET /*.html decode_url
ns_register_filter preauth POST /*.html decode_url
ns_register_filter preauth HEAD /*.html decode_url
proc decode_url { why } {
set_form_variables 0
if {$count==1} {
ns_log notice Thread Redirecting to Google and ns_adp_abort
ns_returnredirect http://www.google.com/
ns_adp_abort
} {
set page [ns_adp_parse Hello]
ns_log notice Thread with data size [string length $page]
ns_return 200 text/html $page
}
return filter_return
}
You can access this anyway you'd like, from a web browser, wget, lynx, or, you
can use my script which requests the page a whole bunch of times which I have
included before.
I recommend watching the server.log as you access pages, since the server log
will tell you the thread numbers.
You can request
http://www.YOURWEBSITE.com/page.html?count=2
as much as you'd like, but once you request
http://www.YOURWEBSITE.com/page.html?count=1
that very helpful error is generated, and the page redirects properly.
Make a note of the thread number and continue requesting
http://www.YOURWEBSITE.com/page.html?count=2
Watch the server.log, when that thread number is used again, the adp is not
parsed**. If the thread is used to server a static file, it serves fine,
however, if it is ever asked to serve an adp in the future it will fail**.
**Just a note here... there is something strange here with when the thread may
be used again to properly serve an adp. I haven't looked into it too far but
it appears that if a request is made through the filter (ie, in this case a
.html file) the adp is NOT parsed and the thread is NOT cleaned. However, if
an adp is called directly, such as http://www.MYSITE.com/page.adp (ie, an adp
is requested but not through the filter) then the adp is NOT parsed, but the
thread IS cleaned and will then parse the next time it is asked to parse an
adp, filter or not. (next time as in the next time the thread is used, it will
continue to always not parse adps until it is closed/dropped/released, whatever
that terminology would be)
The difficulty in finally coming to this conclusion was of course the apparent
random nature of the problem, since the problem was generated by one request,
yet did not materialize until another request x seconds later, which must be an
adp, it was difficult to narrow down. I was only able to find it after setting
up a new physical server, running my tests which all worked and thinking it had
something to do with my server setup, yet when I went started testing of the
site on the new server discovered the same problem.
Now you can debate whether or not ns_adp_abort / ns_adp_break should be called
in a filter, or in an adp called from a filter, or whether my content
management / template system is a good or bad method of serving content. But
until then,
Is there a way to do something like ns_adp_unabort?
Is there something besides ns_adp_abort and ns_adb_break that will return all
the way up the callstack?
Is there a way to expire a thread before it's max_connections are met?
Does anyone have any other suggestions?
_Peter
Date: Tue, 7 Jul 2009 09:37:48 -0700
From: t...@rmadilo.com
Subject: Re: [AOLSERVER] ns_adp_parse issue
To: AOLSERVER@LISTSERV.AOL.COM
Francesco,
Okay here is the thing: you keep using two servers to diagnose an API
which doesn't appear in any of the code you are using. Just use one page
on the server that you think does not work. You don't need to include
any other page, parse any other page or do anything fancy. If you do,
you will have no idea where the problem is or what it is. You could keep
guessing for years until you do this, and it is a waste of your time and
everyone else's time.
But even if your three page/two server
Re: [AOLSERVER] ns_adp_parse issue
Got it. you were close
ns_adp_return set the status to return, but still caused the next adp to fail,
so taking your code I altered it a bit...
proc decode_url { why } {
set page [ns_adp_parse -file /www/spells/template2.adp]
ns_return 200 text/html $page
ns_adp_exception state
ns_log notice adp exception state is $state
if {$state==abort} {
catch {ns_adp_return}
set page [ns_adp_parse Hello]
ns_adp_exception state
ns_log notice adp exception state is $state
}
return filter_return
}
I added a ns_adp_parse on the case an abort was called, this then fails the
parse, but the ns_adp_exception gets set to ok, and future connections work.
Now that this is fixed I will have to find something else to do with my spare
time.
Thanks for everyone's help on this,
_Peter
Date: Tue, 7 Jul 2009 11:54:52 -0700
From: dv...@diphi.com
Subject: Re: [AOLSERVER] ns_adp_parse issue
To: AOLSERVER@LISTSERV.AOL.COM
Francesco Petrarch wrote:
Ok, I got this, this time I really do, and it should be easily
replicated by anyone.
I have this filter:
ns_register_filter preauth GET /*.html decode_url
ns_register_filter preauth POST /*.html decode_url
ns_register_filter preauth HEAD /*.html decode_url
proc decode_url { why } {
set page [ns_adp_parse -file /www/website/template.adp]
ns_return 200 text/html $page
return filter_return
}
Your problem has nothing to do with ns_httpget.
The problem is that the ADP execution context (which is tied to the tcl
interpreter executing the request) does not get reset until the adp
request completes, which normally happens when the output is flushed at
the end of an adp page. Since you are calling the adp page within a
filter, the full adp page handler is never executed, so the cleanup
never happens, and the adp interpreter is in the aborted state, which
means that it never executes any code.
You can use ns_adp_exception state to get the current exception state
of the adp interpreter. And you can call ns_adp_return to set the
exception state to return which I believe will get turned into ok
the next time through. For example:
proc decode_url { why } {
set page [ns_adp_parse -file /www/website/template.adp]
ns_return 200 text/html $page
ns_adp_exception state
ns_log notice adp exception state is $state
catch {ns_adp_return}
return filter_return
}
It appears that adp code is only expected to be run from a handler, not
from a filter. Perhaps a reset subcommand should be added to
ns_adp_ctl to expose NsAdpReset at the tcl level for cases like this.
-J
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
_
We are your photos. Share us now with Windows Live Photos.
http://go.microsoft.com/?linkid=9666047
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] ns_adp_parse issue
Jeff Rogers wrote:
proc decode_url { why } {
set page [ns_adp_parse -file /www/website/template.adp]
ns_return 200 text/html $page
ns_adp_exception state
ns_log notice adp exception state is $state
catch {ns_adp_return}
return filter_return
}
Whoops, this won't work. The ns_adp_return will let the adp interpreter
run *after* the next eval, but the next time just exits. (an additional
piece of debugging code was hiding this from me initially)
I think a better approach (lacking ns_adp_ctl reset) is
proc decode_url { why } {
set page [ns_adp_parse -file /www/website/template.adp]
ns_return 200 text/html $page
ns_adp_exception state
ns_log notice adp exception state is $state
# reset the adp state
set dn [open /dev/null w]
ns_adp_ctl chan $dn
catch { ns_adp_ctl chan }
close $dn
return filter_return
}
Of course, I had to dig around in the code a bit to figure out how to
call NsAdpReset, but on the plus side the AOLserver code is much more
pleasant to dig around in than many other programs.
-J
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] ns_adp_parse issue
Jeff,
Thanks for that bit of help. As far as I can tell everything seems to
work correctly. I setup a slightly different test to demonstrate the
behavior.
Register filters in the private library:
proc ::test_adp_abort {arg why} {
switch -exact -- $arg {
file {
ns_return 200 text/plain [ns_adp_parse -file test.adp]
}
info {
ns_adp_exception adp_state
ns_return 200 text/plain ns_adp_exception_state = $adp_state
}
}
return filter_return
}
ns_register_filter preauth GET /adp-file ::test_adp_abort file
ns_register_filter preauth GET /adp-info ::test_adp_abort info
Place test.adp in the pageroot:
%
ns_return 200 text/plain hi
ns_adp_abort
ns_log Notice After abort, I shouldn't be running
%
Visit urls in a particular order:
1. use ns_adp_parse -file to load test.adp
http://localhost:8000/adp-file
(browser returns hi)
Error log shows an error:
Error: Tcl exception:
while executing
ns_adp_abort
invoked from within
ns_adp_parse -file test.adp
(file arm line 2)
invoked from within
switch -exact -- $arg {
file {
ns_return 200 text/plain [ns_adp_parse -file test.adp]
}
info {
ns_adp_exception adp_state
ns_...
(procedure ::test_adp_abort line 3)
invoked from within
::test_adp_abort file preauth
---
2. Now check the adp status
http://localhost:8000/adp-info
(browser returns ns_adp_exception_state = abort)
Nothing in the error logs
3. Revisit adp-file
http://localhost:8000/adp-file
(browser returns server error)
Error log has:
Error: Tcl exception:
while executing
ns_adp_parse -file test.adp
(file arm line 2)
invoked from within
switch -exact -- $arg {
file {
ns_return 200 text/plain [ns_adp_parse -file test.adp]
}
info {
ns_adp_exception adp_state
ns_...
(procedure ::test_adp_abort line 3)
invoked from within
::test_adp_abort file preauth
Notice that now ns_adp_parse is returning the error, not the call to
ns_adp_abort.
4. Directly access test.adp:
http://localhost:8000/test.adp
(server returns hi)
5. Revisit adp-info
http://localhost:8000/adp-info
(server returns ns_adp_exception_state = ok)
*** Analysis ***
Working exactly like Jeff said. One thing to notice is that direct
access of the test.adp file resets the state.
Why does this happen? Using ns_adp_abort inside a tcl context, or using
it outside of an adp context. Even ns_adp_parse is outside of the adp
context, there is no toplevel handler.
The basic problem is trying to reuse a directly accessed adp as input to
ns_adp_parse. This doesn't work in general, although you might find some
cases where it does work.
tom jackson
On Tue, 2009-07-07 at 13:08 -0700, Jeff Rogers wrote:
Jeff Rogers wrote:
proc decode_url { why } {
set page [ns_adp_parse -file /www/website/template.adp]
ns_return 200 text/html $page
ns_adp_exception state
ns_log notice adp exception state is $state
catch {ns_adp_return}
return filter_return
}
Whoops, this won't work. The ns_adp_return will let the adp interpreter
run *after* the next eval, but the next time just exits. (an additional
piece of debugging code was hiding this from me initially)
I think a better approach (lacking ns_adp_ctl reset) is
proc decode_url { why } {
set page [ns_adp_parse -file /www/website/template.adp]
ns_return 200 text/html $page
ns_adp_exception state
ns_log notice adp exception state is $state
# reset the adp state
set dn [open /dev/null w]
ns_adp_ctl chan $dn
catch { ns_adp_ctl chan }
close $dn
return filter_return
}
Of course, I had to dig around in the code a bit to figure out how to
call NsAdpReset, but on the plus side the AOLserver code is much more
pleasant to dig around in than many other programs.
-J
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] ns_adp_parse issue
Did I miss something?
Your script below does not contain a redirect and does not contain
ns_adp_abort. Why do you think this script tells you anything about your
problem?
But one thing seems apparent: this adp script should return nothing. It is
just a script, it doesn't produce content. If no content is produced, then
it is working.
On Sun, Jul 5, 2009 at 10:32 PM, Francesco Petrarch
f_petra...@hotmail.comwrote:
Hu, on further testing I guess I was pretty much completely wrong
I guess I spoke too soon.
I have tried to turn on all error logging but have come up with nothing. I
am able to intentionally kill a thread, though I am not entirely sure how :S
There are some pages I can request that will kill an additional ADP thread,
at first I thought this was simply the ns_returnredirect or the ns_adp_abort
but seems to be something else since I can't duplicate it with all
redirects/aborts (although it does happen each time these specific pages are
requested). For example, the profile.html page redirects to the
login/signup page if you are not logged in and if it does so causes the
problem
Here is the latest.
I have a script at http://SERVERA/test .adp
%
set count 0
while {$count2} {
incr count
set page [ns_httpget http://SERVERB/file.html?count=$count;]
ns_log notice page $count size [string length $page]
if {[string length $page]2} { break }
}
%
it basically requests a file 2 times from SERVERB (which also hosts the
aforementioned profile.html page) and stops if the file size is small. The
file file.html contains just a single word test. If .html is not in
ns_section ns/server/serverb/adp then the problem does not happen, it
only occurs when file.html may or may not need the parser. In the event
that .html is not in the ns_section and is static, fastpath settings do not
seem to make a difference, everything works.
Now from my observations, it appears that when a file such as the
aforementioned profile.html page is requested and the redirect happens, that
thread becomes contaminated. The next time that contaminated thread is used
for an adp the problem appears. It appears as though the ns_adp_abort
value is still set and adps can not be parsed, even if the file.html file
has no % % tags, if .html is set as a file to be parsed then an empty
string is returned, however, just once. Once this thread is closed again it
seems fixed for future requests. However, this contamination is only
cleaned when the thread tries to serve/parse an adp, if it's serving static
content the contamination is not removed. Going back to the filters I had
originally I believe the request still comes in for the page with the adp,
the filter can log info, but ns_adp_parse just returns an empty string.
Anyways, I'll be digging further into this, trying to see what exactly is
contaminating the thread as it appears now. To repeat, this problem is
repeatable everytime profile.html is requested, and if it is related to
ns_adp_abort or ns_adp_redirect there must be other variables to the problem
since I can not seem to duplicate the problem with just those functions.
Also I do not use ns_perms, I simply set a cookie once someone is logged
in.
I don't suppose there is a ns_adp_unabort
_Peter
--
Date: Sun, 5 Jul 2009 18:50:53 -0700
From: t...@rmadilo.com
Subject: Re: [AOLSERVER] ns_adp_parse issue
To: AOLSERVER@LISTSERV.AOL.COM
On Sun, Jul 5, 2009 at 6:13 PM, Francesco Petrarch f_petra...@hotmail.com
wrote:
It's been awhile since I have looked into this and I think I have tracked
it down. In short, I now believe that ns_adp_abort aborts all current
threads/adps, not just the one currently executing the ns_adp_abort line.
Therefore, when these blank screens were appearing on my site to VISITOR A
it was because another visitor (VISITOR B) had visited a page which called
ns_adp_abort (for example after a ns_returnredirect) before the VISITOR A
thread finished.
Does that make sense? Can someone with more experience in the code take a
look and let me know?
No, makes no sense. This is essentially impossible. Each thread executes
pretty much independently of all others. The problem probably has to do with
the adp configuration...error without signaling an error, which is a
configuration which is necessary to keep complex sites up if one component
is not working.
tom jackson
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
--
Windows Live helps you keep up with all your friends, in one
place.http://go.microsoft.com/?linkid=9660824
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists
Re: [AOLSERVER] ns_adp_parse issue
Sorry, I guess I wasn't clear. That script runs on SERVERA just to do a lot of
http requests on SERVERB and stops when SERVERB returns something of 0 size
(less than 2 actually). Monitoring the aolserver/log/server.log file on
SERVERA will show the progress of requests and monitoring the access.log file
on SERVERB will show the progress of responces.
When a bad page is requested on SERVERB, the bad page processes normally
and without any detected errors, however, upon exit seems to corrupt a thread
which then causes the problem the next time a request is made and the thread
needs to parse an adp.
Since all of these so called bad pages I have found use a ns_returnredirect
followed by a ns_adp_abort I assumed that that was the culprit, however, it
seems that something else is required to cause the problem as well since it
does not happen with all redirect/aborts, just with some, and always with the
same ones.
I haven't had time to look into it more since last night, but I plan on
narrowing down the problem this evening once I have time. Currently the site
in question with these bad pages is about 12 years old, version built on
version and as you can no doubt imagine is now quite bloated and complex in
some areas, so instead of posting 1000 lines of code which may contain an
issue, I'll wait till I cannarrow it down a bit.
_Peter
Date: Mon, 6 Jul 2009 10:42:28 -0700
From: t...@rmadilo.com
Subject: Re: [AOLSERVER] ns_adp_parse issue
To: AOLSERVER@LISTSERV.AOL.COM
Did I miss something?
Your script below does not contain a
redirect and does not contain ns_adp_abort. Why do you think this
script tells you anything about your problem?
But one thing
seems apparent: this adp script should return nothing. It is just a
script, it doesn't produce content. If no content is produced, then it
is working.
On Sun, Jul 5, 2009 at 10:32 PM, Francesco Petrarch f_petra...@hotmail.com
wrote:
Hu, on further testing I guess I was pretty much completely wrong I
guess I spoke too soon.
I have tried to turn on all error logging but have come up with nothing. I am
able to intentionally kill a thread, though I am not entirely sure how :S
There are some pages I can request that will kill an additional ADP thread, at
first I thought this was simply the ns_returnredirect or the ns_adp_abort but
seems to be something else since I can't duplicate it with all redirects/aborts
(although it does happen each time these specific pages are requested). For
example, the profile.html page redirects to the login/signup page if you are
not logged in and if it does so causes the problem
Here is the latest.
I have a script at http://SERVERA/test .adp
%
set count 0
while {$count2} {
incr count
set page [ns_httpget http://SERVERB/file.html?count=$count;]
ns_log notice page $count size [string length $page]
if {[string length $page]2} { break }
}
%
it basically requests a file 2 times from SERVERB (which also hosts the
aforementioned profile.html page) and stops if the file size is small. The
file file.html contains just a single word test. If .html is not in
ns_section ns/server/serverb/adp then the problem does not happen, it only
occurs when file.html may or may not need the parser. In the event that .html
is not in the ns_section and is static, fastpath settings do not seem to make a
difference, everything works.
Now from my observations, it appears that when a file such as the
aforementioned profile.html page is requested and the redirect happens, that
thread becomes contaminated. The next time that contaminated thread is used
for an adp the problem appears. It appears as though the ns_adp_abort value
is still set and adps can not be parsed, even if the file.html file has no %
% tags, if .html is set as a file to be parsed then an empty string is
returned, however, just once. Once this thread is closed again it seems fixed
for future requests. However, this contamination is only cleaned when the
thread tries to serve/parse an adp, if it's serving static content the
contamination is not removed. Going back to the filters I had originally I
believe the request still comes in for the page with the adp, the filter can
log info, but ns_adp_parse just returns an empty string.
Anyways, I'll be digging further into this, trying to see what exactly is
contaminating the thread as it appears now. To repeat, this problem is
repeatable everytime profile.html is requested, and if it is related to
ns_adp_abort or ns_adp_redirect there must be other variables to the problem
since I can not seem to duplicate the problem with just those functions. Also
I do not use ns_perms, I simply set a cookie once someone is logged in.
I don't suppose there is a ns_adp_unabort
_Peter
Date: Sun, 5 Jul 2009 18:50:53 -0700
From: t...@rmadilo.com
Subject: Re: [AOLSERVER] ns_adp_parse issue
To: AOLSERVER@LISTSERV.AOL.COM
On Sun, Jul 5, 2009
Re: [AOLSERVER] ns_adp_parse issue
Ok, I got this, this time I really do, and it should be easily replicated by
anyone.
I have this filter:
ns_register_filter preauth GET /*.html decode_url
ns_register_filter preauth POST /*.html decode_url
ns_register_filter preauth HEAD /*.html decode_url
proc decode_url { why } {
set page [ns_adp_parse -file /www/website/template.adp]
ns_return 200 text/html $page
return filter_return
}
template.adp then does a [ns_conn url] etc etc to serve the correct content
into the template. However, if template.adp does a ns_adp_abort itself (or any
of the files which may be ns_adp_parsed or ns_adp_included etc) then the thread
gets corrupted.
Here is a simple way on how to replicate this with just a single server.
First create a filter:
ns_register_filter preauth GET /*.html decode_url
ns_register_filter preauth POST /*.html decode_url
ns_register_filter preauth HEAD /*.html decode_url
proc decode_url { why } {
set page [ns_adp_parse -file /PATHTOWWWFILES/testpage.html]
ns_return 200 text/html $page
return filter_return
}
Second create the file testpage.adp
%
set ret
set max 200
set count 0
while {$count$max} {
incr count
set page [ns_httpget http://www.YOURWEBSITE.com/testpage.html?count=$count;]
if {[string length $page]!=0} {
if {$count==50} {
append ret First 50 requests okbr
}
if {$count==$max} {
append ret Request Max okbr
}
} {
append ret request $count had filesize [string length $page]br
}
}
ns_puts $ret
%
Finally create testpage.html (note the last line Testdata, so we have some text
on the page)
%
set_form_variables 0
if {$count==50} {
# ns_adp_abort
}
%
Testdata
Now, in your webbrowser go to http://www.YOURSITE.com/testpage.adp, after a few
seconds you should see a webpage like
First 50 requests ok
Request Max ok
You can reload this a number of times and always see the same thing.
Now edit testpage.html by removing the comment on the ns_adp_abort line
Then reload http://www.YOURSITE.com/testpage.adp
You will see something like this:
request 50 had filesize 0
request 113 had filesize 0
request 176 had filesize 0
Request Max ok
Keep reloading for lots of fun!
I have tried modifying the filter to account for a redirect and returning
filter_break or filter_ok but with no luck. What is interesting to note is
that you can put ns_log notices in the filter, and the requests that return 0
in size do get into the filter, but the ns_adp_parse just will just return an
empty string. I am thinking that this is because 'ns_adp_abort' is still set
for the thread?
I am uncertain if this is a bug. If there something other than ns_adp_abort I
could use after a ns_returnredirect which will return all the way back up to
the filter? I believe ns_adp_return will just return 1 level up and
ns_adp_break gives the same problem of also breaking future connections. Or
is there a way I can ns_adp_unabort at the start of the filter?
_Peter
Date: Mon, 6 Jul 2009 10:42:28 -0700
From: t...@rmadilo.com
Subject: Re: [AOLSERVER] ns_adp_parse issue
To: AOLSERVER@LISTSERV.AOL.COM
Did I miss something?
Your script below does not contain a
redirect and does not contain ns_adp_abort. Why do you think this
script tells you anything about your problem?
But one thing
seems apparent: this adp script should return nothing. It is just a
script, it doesn't produce content. If no content is produced, then it
is working.
On Sun, Jul 5, 2009 at 10:32 PM, Francesco Petrarch f_petra...@hotmail.com
wrote:
Hu, on further testing I guess I was pretty much completely wrong I
guess I spoke too soon.
I have tried to turn on all error logging but have come up with nothing. I am
able to intentionally kill a thread, though I am not entirely sure how :S
There are some pages I can request that will kill an additional ADP thread, at
first I thought this was simply the ns_returnredirect or the ns_adp_abort but
seems to be something else since I can't duplicate it with all redirects/aborts
(although it does happen each time these specific pages are requested). For
example, the profile.html page redirects to the login/signup page if you are
not logged in and if it does so causes the problem
Here is the latest.
I have a script at http://SERVERA/test .adp
%
set count 0
while {$count2} {
incr count
set page [ns_httpget http://SERVERB/file.html?count=$count;]
ns_log notice page $count size [string length $page]
if {[string length $page]2} { break }
}
%
it basically requests a file 2 times from SERVERB (which also hosts the
aforementioned profile.html page) and stops if the file size is small. The
file file.html contains just a single word test. If .html is not in
ns_section ns/server/serverb/adp then the problem does not happen, it only
occurs when file.html may or may not need the parser. In the event that .html
is not in the ns_section and is static, fastpath settings do not seem to make
Re: [AOLSERVER] ns_adp_parse issue
It's been awhile since I have looked into this and I think I have tracked it down. In short, I now believe that ns_adp_abort aborts all current threads/adps, not just the one currently executing the ns_adp_abort line. Therefore, when these blank screens were appearing on my site to VISITOR A it was because another visitor (VISITOR B) had visited a page which called ns_adp_abort (for example after a ns_returnredirect) before the VISITOR A thread finished. Does that make sense? Can someone with more experience in the code take a look and let me know? Thanks, _Peter Date: Wed, 10 Jun 2009 00:08:36 +0200 From: neum...@wu.ac.at Subject: Re: [AOLSERVER] ns_adp_parse issue To: AOLSERVER@LISTSERV.AOL.COM Francesco Petrarch schrieb: Thanks for the suggestions, however they have not provided any positive results. ns_param StackSize 2048000 - only succeeding in causing aolserver to crash with could not allocate memory' errors after a day of running i would not give up fast on that. What ist your stack-size setting, when the system crashes? Set it to a value between your current value and 2mb. From your other posting i conclude, that it normally crashes more frequent, since you mentioned ... estimate about 100 times from 100,000 page requests a day right now.. The could not allocate memory shows that you are most likely running a 32bit version which gets a size close to 2GB. You could reduce resource consumption (e.g. less connection threads) or compile with 64 bit. best regards -gustaf neumann -- Univ.Prof. Dr. Gustaf Neumann Institute of Information Systems and New Media WU Vienna Augasse 2-6, A-1090 Vienna, AUSTRIA -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. _ Create a cool, new character for your Windows Live™ Messenger. http://go.microsoft.com/?linkid=9656621 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] ns_adp_parse issue
On Sun, Jul 5, 2009 at 6:13 PM, Francesco Petrarch f_petra...@hotmail.comwrote: It's been awhile since I have looked into this and I think I have tracked it down. In short, I now believe that ns_adp_abort aborts all current threads/adps, not just the one currently executing the ns_adp_abort line. Therefore, when these blank screens were appearing on my site to VISITOR A it was because another visitor (VISITOR B) had visited a page which called ns_adp_abort (for example after a ns_returnredirect) before the VISITOR A thread finished. Does that make sense? Can someone with more experience in the code take a look and let me know? No, makes no sense. This is essentially impossible. Each thread executes pretty much independently of all others. The problem probably has to do with the adp configuration...error without signaling an error, which is a configuration which is necessary to keep complex sites up if one component is not working. tom jackson -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] ns_adp_parse issue
Hu, on further testing I guess I was pretty much completely wrong I
guess I spoke too soon.
I have tried to turn on all error logging but have come up with nothing. I am
able to intentionally kill a thread, though I am not entirely sure how :S
There are some pages I can request that will kill an additional ADP thread, at
first I thought this was simply the ns_returnredirect or the ns_adp_abort but
seems to be something else since I can't duplicate it with all redirects/aborts
(although it does happen each time these specific pages are requested). For
example, the profile.html page redirects to the login/signup page if you are
not logged in and if it does so causes the problem
Here is the latest.
I have a script at http://SERVERA/test .adp
%
set count 0
while {$count2} {
incr count
set page [ns_httpget http://SERVERB/file.html?count=$count;]
ns_log notice page $count size [string length $page]
if {[string length $page]2} { break }
}
%
it basically requests a file 2 times from SERVERB (which also hosts the
aforementioned profile.html page) and stops if the file size is small. The
file file.html contains just a single word test. If .html is not in
ns_section ns/server/serverb/adp then the problem does not happen, it only
occurs when file.html may or may not need the parser. In the event that .html
is not in the ns_section and is static, fastpath settings do not seem to make a
difference, everything works.
Now from my observations, it appears that when a file such as the
aforementioned profile.html page is requested and the redirect happens, that
thread becomes contaminated. The next time that contaminated thread is used
for an adp the problem appears. It appears as though the ns_adp_abort value
is still set and adps can not be parsed, even if the file.html file has no %
% tags, if .html is set as a file to be parsed then an empty string is
returned, however, just once. Once this thread is closed again it seems fixed
for future requests. However, this contamination is only cleaned when the
thread tries to serve/parse an adp, if it's serving static content the
contamination is not removed. Going back to the filters I had originally I
believe the request still comes in for the page with the adp, the filter can
log info, but ns_adp_parse just returns an empty string.
Anyways, I'll be digging further into this, trying to see what exactly is
contaminating the thread as it appears now. To repeat, this problem is
repeatable everytime profile.html is requested, and if it is related to
ns_adp_abort or ns_adp_redirect there must be other variables to the problem
since I can not seem to duplicate the problem with just those functions. Also
I do not use ns_perms, I simply set a cookie once someone is logged in.
I don't suppose there is a ns_adp_unabort
_Peter
Date: Sun, 5 Jul 2009 18:50:53 -0700
From: t...@rmadilo.com
Subject: Re: [AOLSERVER] ns_adp_parse issue
To: AOLSERVER@LISTSERV.AOL.COM
On Sun, Jul 5, 2009 at 6:13 PM, Francesco Petrarch f_petra...@hotmail.com
wrote:
It's been awhile since I have looked into this and I think I have tracked it
down. In short, I now believe that ns_adp_abort aborts all current
threads/adps, not just the one currently executing the ns_adp_abort line.
Therefore, when these blank screens were appearing on my site to VISITOR A it
was because another visitor (VISITOR B) had visited a page which called
ns_adp_abort (for example after a ns_returnredirect) before the VISITOR A
thread finished.
Does that make sense? Can someone with more experience in the code take a look
and let me know?
No, makes no sense. This is essentially impossible. Each thread executes pretty
much independently of all others. The problem probably has to do with the adp
configuration...error without signaling an error, which is a configuration
which is necessary to keep complex sites up if one component is not working.
tom jackson
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
_
Windows Live helps you keep up with all your friends, in one place.
http://go.microsoft.com/?linkid=9660826
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] ns_adp_parse issue
Francesco Petrarch schrieb: Thanks for the suggestions, however they have not provided any positive results. ns_param StackSize 2048000 - only succeeding in causing aolserver to crash with could not allocate memory' errors after a day of running i would not give up fast on that. What ist your stack-size setting, when the system crashes? Set it to a value between your current value and 2mb. From your other posting i conclude, that it normally crashes more frequent, since you mentioned ... estimate about 100 times from 100,000 page requests a day right now.. The could not allocate memory shows that you are most likely running a 32bit version which gets a size close to 2GB. You could reduce resource consumption (e.g. less connection threads) or compile with 64 bit. best regards -gustaf neumann -- Univ.Prof. Dr. Gustaf Neumann Institute of Information Systems and New Media WU Vienna Augasse 2-6, A-1090 Vienna, AUSTRIA -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] ns_adp_parse issue
Thanks for the suggestions, however they have not provided any positive
results.
ns_adp_ctl stricterror 1 - did not yield any additional errors/notices (nothing
is written to the log files when this happens)
ns_param StackSize 2048000 - only succeeding in causing aolserver to crash with
could not allocate memory' errors after a day of running
For interest sake, the way I test this is from a different server I have this
little script:
%
set count 0
while {$count1000} {
incr count
set page [ns_httpget http://www.mysite.com/members.html]
ns_log notice page $count size [string length $page]
if {[string length $page]1000} { break }
}
%
And some sample lines form that logfile are:
[05/Jun/2009:16:26:47][31912.2950486928][-default:394-] Notice: page 86 size
38660
[05/Jun/2009:16:26:47][31912.2950486928][-default:394-] Notice: page 87 size
38660
[05/Jun/2009:16:26:47][31912.2950486928][-default:394-] Notice: page 88 size
38660
[05/Jun/2009:16:26:48][31912.2950486928][-default:394-] Notice: page 89 size
38660
[05/Jun/2009:16:26:48][31912.2950486928][-default:394-] Notice: page 90 size
38660
[05/Jun/2009:16:26:48][31912.2950486928][-default:394-] Notice: page 91 size
38660
[05/Jun/2009:16:26:48][31912.2950486928][-default:394-] Notice: page 92 size
38660
[05/Jun/2009:16:26:48][31912.2950486928][-default:394-] Notice: page 93 size
38660
[05/Jun/2009:16:26:48][31912.2950486928][-default:394-] Notice: page 94 size 87
When I didn't return the javascript on a failure the size would be 0 not 87.
I have yet to try Dossy's suggestion of using the startpage because I would
rather try and find out what the issue actually is incase there are other
issues involved with it. Plus since the javascript is currently a working
bandaid fix I am in no hurry, the only real downside with my hack is a search
engine may index it.
What puzzels me is that, as I mentioned, these 0 sized ns_adp_parse calls seem
to happen at random, when the page it happens on is reloaded things work fine,
so all the input data is the same. Combined with the fact when I attempt to
call ns_adp_parse multiple times when it does return a 0 sized string, all
successive calls also return a 0 sized string leads me to think something in
the ns_adp_parse functionality is failing without throwing an error... maybe a
lock contention somewhere? I even put a ns_sleep 1 between ns_adp_parse calls
but even after 10 seconds it returns a 0 length string.
I will now be digging into the Aolserver code itself to see what I can find
logging to the log file between every line of code if I must to find out where
this is failing.
Thanks for the help so far,
_Peter
Date: Fri, 29 May 2009 09:06:11 -0700
From: t...@rmadilo.com
Subject: Re: [AOLSERVER] ns_adp_parse issue
To: AOLSERVER@LISTSERV.AOL.COM
Maybe there is an error which is silently caught in ns_adp_parse.
Try this before sourcing your adp:
ns_adp_ctl stricterror 1
tom jackson
On Fri, 2009-05-29 at 14:48 +, Francesco Petrarch wrote:
I appear to be having a problem with ns_adp_parse and ns_adp_eval. I
currently run the latest code 4.5.1, upgraded from 3.13 with Jerry
Asher's patches for unix sockets for multiple hosts. Everything
worked fine on 3.13 but now I have intermittent issues with 4.5.1 (I
had acutally upgraded in November and have the issue with all 4.5
updates since then).
My website serves about 140 million requests a month, about 3 million
of those are .html requests.
I don't use ACS, everything is custom coded and my CMS is based on the
following filter:
ns_register_filter preauth GET /*.html decode_url
ns_register_filter preauth POST /*.html decode_url
proc decode_url { why } {
ns_return 200 text/html [ns_adp_parse
-file /www/website/template.adp]
return filter_return
}
Essentially, every .html page goes through this filter, and the
file /www/website/template.adp is a graphical template/layout for the
site and dynamically includes the content for the give [ns_conn url].
As I mentioned this worked fine for 3.13
However, seemly at random times (for a random connection),
[ns_adp_parse -file /www/website/template.adp] will return an empty
string. I have modified the filter to reattempt the ns_adp_parse if
the returned value was an empty string (ie: [string length $page]==0)
and logged the progress, but on a connection where ns_adp_parse
returns the empty string it appears to always return an empty string.
ns_adp_eval also returns an empty string when this happens. With the
above filter the visitor to the website would see just a blank screen
because of course the content was just . To be exact here, I say
empty string, to be more accurate it returns $page where [string
length $page]==0
My current little fix is this:
ns_register_filter preauth GET /*.html decode_url
ns_register_filter preauth POST /*.html
Re: [AOLSERVER] ns_adp_parse issue
I would try testing on a page from a local server so you can verify that
the server is returning what you expect.
Also, log the actual content when it is not what you expect. you have
access to the $page value, maybe it is an error message.
Obviously something is failing, but nothing you have shown us narrows
the location of the possible problem. Is it the adp, the configuration
of adps or the error log, or is the foreign server screwing up, or your
network. The fact that the code works most of the time means you
probably need to look at every step and the value of every step.
That said, my guess something is going wrong with the ns_httpget
operation, maybe it gets oveloaded, times out or something else.
tom jackson
On Fri, 2009-06-05 at 20:56 +, Francesco Petrarch wrote:
Thanks for the suggestions, however they have not provided any
positive results.
ns_adp_ctl stricterror 1 - did not yield any additional errors/notices
(nothing is written to the log files when this happens)
ns_param StackSize 2048000 - only succeeding in causing aolserver to
crash with could not allocate memory' errors after a day of running
For interest sake, the way I test this is from a different server I
have this little script:
%
set count 0
while {$count1000} {
incr count
set page [ns_httpget http://www.mysite.com/members.html]
ns_log notice page $count size [string length $page]
if {[string length $page]1000} { break }
}
%
And some sample lines form that logfile are:
[05/Jun/2009:16:26:47][31912.2950486928][-default:394-] Notice: page
86 size 38660
[05/Jun/2009:16:26:47][31912.2950486928][-default:394-] Notice: page
87 size 38660
[05/Jun/2009:16:26:47][31912.2950486928][-default:394-] Notice: page
88 size 38660
[05/Jun/2009:16:26:48][31912.2950486928][-default:394-] Notice: page
89 size 38660
[05/Jun/2009:16:26:48][31912.2950486928][-default:394-] Notice: page
90 size 38660
[05/Jun/2009:16:26:48][31912.2950486928][-default:394-] Notice: page
91 size 38660
[05/Jun/2009:16:26:48][31912.2950486928][-default:394-] Notice: page
92 size 38660
[05/Jun/2009:16:26:48][31912.2950486928][-default:394-] Notice: page
93 size 38660
[05/Jun/2009:16:26:48][31912.2950486928][-default:394-] Notice: page
94 size 87
When I didn't return the javascript on a failure the size would be 0
not 87.
I have yet to try Dossy's suggestion of using the startpage because I
would rather try and find out what the issue actually is incase there
are other issues involved with it. Plus since the javascript is
currently a working bandaid fix I am in no hurry, the only real
downside with my hack is a search engine may index it.
What puzzels me is that, as I mentioned, these 0 sized ns_adp_parse
calls seem to happen at random, when the page it happens on is
reloaded things work fine, so all the input data is the same.
Combined with the fact when I attempt to call ns_adp_parse multiple
times when it does return a 0 sized string, all successive calls also
return a 0 sized string leads me to think something in the
ns_adp_parse functionality is failing without throwing an error...
maybe a lock contention somewhere? I even put a ns_sleep 1 between
ns_adp_parse calls but even after 10 seconds it returns a 0 length
string.
I will now be digging into the Aolserver code itself to see what I can
find logging to the log file between every line of code if I must
to find out where this is failing.
Thanks for the help so far,
_Peter
Date: Fri, 29 May 2009 09:06:11 -0700
From: t...@rmadilo.com
Subject: Re: [AOLSERVER] ns_adp_parse issue
To: AOLSERVER@LISTSERV.AOL.COM
Maybe there is an error which is silently caught in ns_adp_parse.
Try this before sourcing your adp:
ns_adp_ctl stricterror 1
tom jackson
On Fri, 2009-05-29 at 14:48 +, Francesco Petrarch wrote:
I appear to be having a problem with ns_adp_parse and ns_adp_eval.
I
currently run the latest code 4.5.1, upgraded from 3.13 with Jerry
Asher's patches for unix sockets for multiple hosts. Everything
worked fine on 3.13 but now I have intermittent issues with 4.5.1
(I
had acutally upgraded in November and have the issue with all 4.5
updates since then).
My website serves about 140 million requests a month, about 3
million
of those are .html requests.
I don't use ACS, everything is custom coded and my CMS is based on
the
following filter:
ns_register_filter preauth GET /*.html decode_url
ns_register_filter preauth POST /*.html decode_url
proc decode_url { why } {
ns_return 200 text/html [ns_adp_parse
-file /www/website/template.adp]
return filter_return
}
Essentially, every .html page goes through this filter, and the
file /www/website/template.adp is a graphical template/layout for
the
site and dynamically includes the content for the give [ns_conn
url].
As I mentioned this worked
Re: [AOLSERVER] ns_adp_parse issue
On 6/5/09 4:56 PM, Francesco Petrarch wrote: What puzzels me is that, as I mentioned, these 0 sized ns_adp_parse calls seem to happen at random, when the page it happens on is reloaded things work fine, so all the input data is the same. Do you call ns_adp_trunc *anywhere* in your code, ever? -- Dossy Shiobara | do...@panoptic.com | http://dossy.org/ Panoptic Computer Network | http://panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] ns_adp_parse issue
Nope, never used that function. Date: Fri, 5 Jun 2009 20:10:34 -0400 From: do...@panoptic.com Subject: Re: [AOLSERVER] ns_adp_parse issue To: AOLSERVER@LISTSERV.AOL.COM On 6/5/09 4:56 PM, Francesco Petrarch wrote: What puzzels me is that, as I mentioned, these 0 sized ns_adp_parse calls seem to happen at random, when the page it happens on is reloaded things work fine, so all the input data is the same. Do you call ns_adp_trunc *anywhere* in your code, ever? -- Dossy Shiobara | do...@panoptic.com | http://dossy.org/ Panoptic Computer Network | http://panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. _ Create a cool, new character for your Windows Live™ Messenger. http://go.microsoft.com/?linkid=9656621 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] ns_adp_parse issue
The $page value is empty, when I
ns_log (content=$page)
I get
(content=)
in the logs
Whether I have that script which ns_httpget on the same physcial machine,
serving from the same site, or on a different physical server, I still get the
error with ns_adp_parse.
However, if I have a script which just calls ns_adp_parse thousands of times in
a loop, it seems to work everytime.
So again, it seems to be something which exists only for a given connection,
but happens fairly rarely, I'm trying to collect some stats on exactly how
often, I would estimate about 100 times from 100,000 page requests a day right
now.
My original hope was that someone would be able to say oh it's this setting
or know of a bug or by my description could see something which was an obvious
problem. I guess that means more digging for me.
_Peter
Date: Fri, 5 Jun 2009 17:01:23 -0700
From: t...@rmadilo.com
Subject: Re: [AOLSERVER] ns_adp_parse issue
To: AOLSERVER@LISTSERV.AOL.COM
I would try testing on a page from a local server so you can verify that
the server is returning what you expect.
Also, log the actual content when it is not what you expect. you have
access to the $page value, maybe it is an error message.
Obviously something is failing, but nothing you have shown us narrows
the location of the possible problem. Is it the adp, the configuration
of adps or the error log, or is the foreign server screwing up, or your
network. The fact that the code works most of the time means you
probably need to look at every step and the value of every step.
That said, my guess something is going wrong with the ns_httpget
operation, maybe it gets oveloaded, times out or something else.
tom jackson
On Fri, 2009-06-05 at 20:56 +, Francesco Petrarch wrote:
Thanks for the suggestions, however they have not provided any
positive results.
ns_adp_ctl stricterror 1 - did not yield any additional errors/notices
(nothing is written to the log files when this happens)
ns_param StackSize 2048000 - only succeeding in causing aolserver to
crash with could not allocate memory' errors after a day of running
For interest sake, the way I test this is from a different server I
have this little script:
%
set count 0
while {$count1000} {
incr count
set page [ns_httpget http://www.mysite.com/members.html]
ns_log notice page $count size [string length $page]
if {[string length $page]1000} { break }
}
%
And some sample lines form that logfile are:
[05/Jun/2009:16:26:47][31912.2950486928][-default:394-] Notice: page
86 size 38660
[05/Jun/2009:16:26:47][31912.2950486928][-default:394-] Notice: page
87 size 38660
[05/Jun/2009:16:26:47][31912.2950486928][-default:394-] Notice: page
88 size 38660
[05/Jun/2009:16:26:48][31912.2950486928][-default:394-] Notice: page
89 size 38660
[05/Jun/2009:16:26:48][31912.2950486928][-default:394-] Notice: page
90 size 38660
[05/Jun/2009:16:26:48][31912.2950486928][-default:394-] Notice: page
91 size 38660
[05/Jun/2009:16:26:48][31912.2950486928][-default:394-] Notice: page
92 size 38660
[05/Jun/2009:16:26:48][31912.2950486928][-default:394-] Notice: page
93 size 38660
[05/Jun/2009:16:26:48][31912.2950486928][-default:394-] Notice: page
94 size 87
When I didn't return the javascript on a failure the size would be 0
not 87.
I have yet to try Dossy's suggestion of using the startpage because I
would rather try and find out what the issue actually is incase there
are other issues involved with it. Plus since the javascript is
currently a working bandaid fix I am in no hurry, the only real
downside with my hack is a search engine may index it.
What puzzels me is that, as I mentioned, these 0 sized ns_adp_parse
calls seem to happen at random, when the page it happens on is
reloaded things work fine, so all the input data is the same.
Combined with the fact when I attempt to call ns_adp_parse multiple
times when it does return a 0 sized string, all successive calls also
return a 0 sized string leads me to think something in the
ns_adp_parse functionality is failing without throwing an error...
maybe a lock contention somewhere? I even put a ns_sleep 1 between
ns_adp_parse calls but even after 10 seconds it returns a 0 length
string.
I will now be digging into the Aolserver code itself to see what I can
find logging to the log file between every line of code if I must
to find out where this is failing.
Thanks for the help so far,
_Peter
Date: Fri, 29 May 2009 09:06:11 -0700
From: t...@rmadilo.com
Subject: Re: [AOLSERVER] ns_adp_parse issue
To: AOLSERVER@LISTSERV.AOL.COM
Maybe there is an error which is silently caught in ns_adp_parse.
Try this before sourcing your adp:
ns_adp_ctl stricterror 1
tom jackson
On Fri, 2009-05-29 at 14:48 +, Francesco Petrarch
Re: [AOLSERVER] ns_adp_parse issue
While writing my last reply, I thought of a few things to test. I modified my
loop of ns_httpget calls to access the template.adp file directly, instead of
members.html thus completely avoiding the ns_adp_parse call.
The result, I still recieved some pages of 0 size. So the problem suddently
looked like it may not be an ns_adp_parse error but something else, though it
still had the possibility of being so since there are ns_adp_include calls in
template.adp, and template.adp itself is an adp that needs to be parsed.
So I modify template.adp to template2.adp and I put some ns_logs to track the
progress of each request and made some minor modifactions to give each each
request a unique url for easier tracking.
So my loop on the server I send the requests is now:
%
set count 0
while {$count2000} {
incr count
set page [ns_httpget http://www.mysebsite.com/template2.adp?count=$count;]
ns_log notice page $count size [string length $page]
if {[string length $page]2} { break }
}
%
where $count is just an incremental counter and then template2.adp consists of
only the following
alpha
%
ns_log notice [ns_conn peeraddr] connected to template for [ns_conn url]
ns_log notice [ns_conn peeraddr] connected to template for [ns_conn url]
setting form vars
set_form_variables 0
ns_log notice [ns_conn peeraddr] connected to template for [ns_conn url] set
$count
%
beta
The result. some pages with a returned $page size of 0
Some sample log lines from the server which sends the requests:
[06/Jun/2009:00:13:07][31912.2948537232][-default:387-] Notice: page 954 size 13
[06/Jun/2009:00:13:07][31912.2948537232][-default:387-] Notice: page 955 size 13
[06/Jun/2009:00:13:07][31912.2948537232][-default:387-] Notice: page 956 size 0
And from the server serving the pages:
[05/Jun/2009:23:19:02][31792.3058563984][-default:50-] Notice: 70.85.172.155
connected to template for /template2.adp
[05/Jun/2009:23:19:02][31792.3058563984][-default:50-] Notice: 70.85.172.155
connected to template for /template2.adp setting form vars
[05/Jun/2009:23:19:02][31792.3058563984][-default:50-] Notice: 70.85.172.155
connected to template for /template2.adp set 954
[05/Jun/2009:23:19:02][31792.3060997008][-default:44-] Notice: 70.85.172.155
connected to template for /template2.adp
[05/Jun/2009:23:19:02][31792.3060997008][-default:44-] Notice: 70.85.172.155
connected to template for /template2.adp setting form vars
[05/Jun/2009:23:19:02][31792.3060997008][-default:44-] Notice: 70.85.172.155
connected to template for /template2.adp set 955
So on request 956 nothing was returned, not the word 'alpha' or 'beta' and the
code in % % never gets executed to make the log entries.
I'm going to get my much needed beauty sleep now and continue this in the
morning, testing non-adp file requests. My suspisions are heavy on the side
that something is happening when the file is sent to the parser, since when a
.html request is made, the filter still logs the access with ns_log (the page
request also appears in the webstats log with a file size of 0).
Whatever the problem I am fairly certain the issue must have something to do
with the physical server's settings, since such a problem with Aolserver itself
would have surely been noticed by others.
_Peter
Date: Fri, 5 Jun 2009 17:01:23 -0700
From: t...@rmadilo.com
Subject: Re: [AOLSERVER] ns_adp_parse issue
To: AOLSERVER@LISTSERV.AOL.COM
I would try testing on a page from a local server so you can verify that
the server is returning what you expect.
Also, log the actual content when it is not what you expect. you have
access to the $page value, maybe it is an error message.
Obviously something is failing, but nothing you have shown us narrows
the location of the possible problem. Is it the adp, the configuration
of adps or the error log, or is the foreign server screwing up, or your
network. The fact that the code works most of the time means you
probably need to look at every step and the value of every step.
That said, my guess something is going wrong with the ns_httpget
operation, maybe it gets oveloaded, times out or something else.
tom jackson
On Fri, 2009-06-05 at 20:56 +, Francesco Petrarch wrote:
Thanks for the suggestions, however they have not provided any
positive results.
ns_adp_ctl stricterror 1 - did not yield any additional errors/notices
(nothing is written to the log files when this happens)
ns_param StackSize 2048000 - only succeeding in causing aolserver to
crash with could not allocate memory' errors after a day of running
For interest sake, the way I test this is from a different server I
have this little script:
%
set count 0
while {$count1000} {
incr count
set page [ns_httpget http://www.mysite.com/members.html]
ns_log notice page $count size [string length $page]
if {[string length $page]1000} { break
Re: [AOLSERVER] ns_adp_parse issue
Maybe there is an error which is silently caught in ns_adp_parse.
Try this before sourcing your adp:
ns_adp_ctl stricterror 1
tom jackson
On Fri, 2009-05-29 at 14:48 +, Francesco Petrarch wrote:
I appear to be having a problem with ns_adp_parse and ns_adp_eval. I
currently run the latest code 4.5.1, upgraded from 3.13 with Jerry
Asher's patches for unix sockets for multiple hosts. Everything
worked fine on 3.13 but now I have intermittent issues with 4.5.1 (I
had acutally upgraded in November and have the issue with all 4.5
updates since then).
My website serves about 140 million requests a month, about 3 million
of those are .html requests.
I don't use ACS, everything is custom coded and my CMS is based on the
following filter:
ns_register_filter preauth GET /*.html decode_url
ns_register_filter preauth POST /*.html decode_url
proc decode_url { why } {
ns_return 200 text/html [ns_adp_parse
-file /www/website/template.adp]
return filter_return
}
Essentially, every .html page goes through this filter, and the
file /www/website/template.adp is a graphical template/layout for the
site and dynamically includes the content for the give [ns_conn url].
As I mentioned this worked fine for 3.13
However, seemly at random times (for a random connection),
[ns_adp_parse -file /www/website/template.adp] will return an empty
string. I have modified the filter to reattempt the ns_adp_parse if
the returned value was an empty string (ie: [string length $page]==0)
and logged the progress, but on a connection where ns_adp_parse
returns the empty string it appears to always return an empty string.
ns_adp_eval also returns an empty string when this happens. With the
above filter the visitor to the website would see just a blank screen
because of course the content was just . To be exact here, I say
empty string, to be more accurate it returns $page where [string
length $page]==0
My current little fix is this:
ns_register_filter preauth GET /*.html decode_url
ns_register_filter preauth POST /*.html decode_url
proc decode_url { why } {
set page [ns_adp_parse -file /www/website/template.adp]
set size [string length $page]
if {$size==0} {
ns_log notice Page [ns_conn url] ns_adp_parse $size in size for
visitor from [ns_conn peeraddr]
set page body onload='window.location.reload()'
}
ns_return 200 text/html $page
return filter_return
}
As you can see, not so much a fix as just creating a little javascript
to reload the page. At least the visitor to the website is no longer
presented with a blank page, but not as ideal as an actual fix to the
empty string issue.
I have thought that this may be an issue with threads or some limit
with handles to databases or maybe the adp parser (I really don't know
how that works). However, given that this issue was not present on
3.13, and no errors are generated I am at a loss. I have also tried
messing with the config, changing just about everything to see if it
made any difference, it didn't.
I will be continuing to try and debug this to confirm
that /www/website/template.adp is being parsed and so on, but before I
got too involved I figured I'd try this list to see if anyone knows of
similar issues.
Thanks for your help,
_Peter
__
Create a cool, new character for your Windows Live™ Messenger. Check
it out
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] ns_adp_parse issue
On 5/29/09 10:48 AM, Francesco Petrarch wrote: However, seemly at random times (for a random connection), [ns_adp_parse -file /www/website/template.adp] will return an empty string. Are you scouring your server log for any uncaught Tcl errors? Without knowing anything about your application code, if ns_adp_parse is consistently returning an empty string for a particular connection, then there's some application code that is connection-sensitive (thus, simply retrying the ns_adp_parse yields the same result consistently) that is probably resulting in an uncaught Tcl error that causes ns_adp_parse to return an empty string. FYI, instead of the preauth filter, might I suggest you use this instead in your server's configuration .tcl: ns_section ns/server/$server/adp ns_param startpage /www/website/template.adp It may not give you the same level of control as you are getting with the ns_adp_parse approach, but it might work better. I also notice you don't register the same filter for HEAD requests. Was this intentional? -- Dossy Shiobara | do...@panoptic.com | http://dossy.org/ Panoptic Computer Network | http://panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
