Re: [AOLSERVER] missing access log entries

2009-04-16 Thread Tom Jackson
You can try the "diff" link to the parent at the following two links. The second one is older, but each patch is to different files. But please note that my queue.c file (first patch below) is not identical to CVS HEAD, so the patch line numbers will not match up. On Fri, 2009-04-17 at 00:32 +04

Re: [AOLSERVER] missing access log entries

2009-04-16 Thread Alexey Pechnikov
Hello! On Monday 13 April 2009 09:12:20 Tom Jackson wrote: > Attached is a patch based upon the original code, not my last attempt, > and more info at: > > http://www.junom.com/gitweb/gitweb.perl?p=aolserver.git;a=commit;h=7ab44 I need to build AOL with complete resolved subj problem. Which patch

Re: [AOLSERVER] missing access log entries

2009-04-16 Thread Tom Jackson
On Mon, 2009-04-13 at 13:49 -0400, Jim Davidson wrote: > Hi, > > A bit old but let me try to be helpful here... > > > On Apr 3, 2009, at 11:45 PM, Tom Jackson wrote: Jim, Thanks for adding some back story. Over the weekend I spent more time looking into this problem and basically rewrote

Re: [AOLSERVER] missing access log entries

2009-04-15 Thread Tom Jackson
Jim, I'm thinking along the same lines. The while {1} thing is similar to a try block, but it handles the special case where there is really only on exception, on error or a successful return, fast-forward to traces...actually check for an error then maybe do traces. What I'm not sure about is s

Re: [AOLSERVER] missing access log entries

2009-04-15 Thread Jim Davidson
I'm wondering if there needs to be some more specific exception flags stored in the connection structure to handle all these cases and some C and Tcl api's to access/modify the same. It seems you're doing the good work of rationalizing all the error conditions that had been pretty confused

Re: [AOLSERVER] missing access log entries

2009-04-15 Thread Tom Jackson
Gustaf, Thanks for testing on a production server. I haven't tested this with the authorization module, but it seems like it should work okay. The internal server loop bug that I tracked down yesterday led me to another function which contains code very similar to the new ConnAuthorize function,

Re: [AOLSERVER] missing access log entries

2009-04-15 Thread Jim Davidson
Hi, I haven't been able to test myself but the changes make sense to me too. Thanks Tom! -Jim On Apr 15, 2009, at 6:52 AM, Gustaf Neumann wrote: Dear Tom, your rewrite of the two functions below look fine to me. The structure is much clearer now, results of the authorization handling are

Re: [AOLSERVER] missing access log entries

2009-04-15 Thread Gustaf Neumann
Dear Tom, your rewrite of the two functions below look fine to me. The structure is much clearer now, results of the authorization handling are handled now consistently. I am useing these functions since a few days on one of my servers and found nothing unusual. Many thanks to Tom! Andrew, did y

Re: [AOLSERVER] missing access log entries

2009-04-14 Thread Jim Davidson
Hi, Yes -- I think in general, the meaning of "status" with regards to a connection processing (or a filter that may preempt it) was that "a valid HTTP response was sent..."If you look deep in the rat's nest of connio.c, you'll find: int Ns_WriteConn(Ns_Conn *conn, char *buf, int len)

Re: [AOLSERVER] missing access log entries

2009-04-13 Thread Tom Jackson
On Mon, 2009-04-13 at 13:49 -0400, Jim Davidson wrote: > > On Apr 3, 2009, at 11:45 PM, Tom Jackson wrote: > > > > * Filters seem uniquely superior when the url pattern determines which > > code should run. Unfortunately filters are many times registered to / > > *, > > the global pattern. This i

Re: [AOLSERVER] missing access log entries

2009-04-13 Thread Jim Davidson
Hi, A bit old but let me try to be helpful here... On Apr 3, 2009, at 11:45 PM, Tom Jackson wrote: On Fri, 2009-04-03 at 16:20 -0600, Jim Davidson wrote: HI, Sorry -- I wrote the original and goofy code leading to this confusion... Thanks for monitoring our head scratching! I have a few

Re: [AOLSERVER] missing access log entries

2009-04-12 Thread Tom Jackson
Gustaf, Yep, you are right. Looking at this again, I noticed a few additional things which might help get this working as expected. For one, the log trace is registered with Ns_RegisterServerTrace. The documentation with this function says: * Register a connection trace procedure. Traces

Re: [AOLSERVER] missing access log entries

2009-04-12 Thread Gustaf Neumann
Tom Jackson schrieb: Hey, I just noticed that my patch to queue.c misses one case. If an authorization proc returns NS_ERROR, a 500 response is returned, but no trace filters run. This seems like a bug, ... i was as well looking at the result of Ns_AuthorizeRequest, and not at the result of a

Re: [AOLSERVER] missing access log entries

2009-04-11 Thread Tom Jackson
Hey, I just noticed that my patch to queue.c misses one case. If an authorization proc returns NS_ERROR, a 500 response is returned, but no trace filters run. This seems like a bug, but further indicates to me that a decision was made to return an error response and log errors, but the code didn'

Re: [AOLSERVER] missing access log entries

2009-04-11 Thread Tom Jackson
Gustaf, Here is the blowhard answer (the short answers are below): I agree that there must be some additional code paths which haven't been checked. I'll try to go through what I discovered based on your questions, but if I miss something ask again. One nice thing about the AOLserver code is tha

Re: [AOLSERVER] missing access log entries

2009-04-10 Thread Gustaf Neumann
Tom Jackson schrieb: Andrew, Okay, great! The changes should probably be divided into two patches: good idea. While the changes in queue.c are somewhat non-obvious the changes related to adprequest.c and adpeval.c are more clear to me. They address the original problem. i was not able to f

Re: [AOLSERVER] missing access log entries

2009-04-10 Thread Tom Jackson
Andrew, Okay, great! The changes should probably be divided into two patches: queue.c is a behavior change and doesn't address any bugs, just returns a 500 response if possible, and allows trace filters to run. My guess is that this patch is backwards compatible with all code because someone would

Re: [AOLSERVER] missing access log entries

2009-04-09 Thread Andrew Steets
Hi Tom, sorry to go dark for so long. It was operator error. I was in a hurry and I don't think I restarted the server after I installed the patched version of the server. I checked again just now and everything works as expected. -Andrew On Mon, Apr 6, 2009 at 4:14 PM, Tom Jackson wrote: >

Re: [AOLSERVER] missing access log entries

2009-04-06 Thread Tom Jackson
Andrew, Hmmm, well without knowing how you tested this, I can't help much. I created a few test adp pages. I tested them before my changes to identify the problems. Here is an example set of pages: include.adp: <% ns_adp_puts "before include" ns_adp_include test-ns-return.adp ns_adp_puts "after

Re: [AOLSERVER] missing access log entries

2009-04-06 Thread Andrew Steets
I got a chance to test this out this morning. I don't understand what it is supposed to fix. I still don't get access log entries when ns_adp_abort is called. On Sun, Apr 5, 2009 at 12:52 PM, Tom Jackson wrote: > The attached patch fixes ns_adp_break, it differs from the previous > patch by one

Re: [AOLSERVER] missing access log entries

2009-04-05 Thread Tom Jackson
The attached patch fixes ns_adp_break, it differs from the previous patch by one line in adpeval.c tom jackson On Sat, 2009-04-04 at 16:25 -0500, Andrew Steets wrote: > Hi Tom, > > Attachments seem to work ok on this list. -- AOLserver - http://www.aolserver.com/ To Remove yourself from thi

Re: [AOLSERVER] missing access log entries

2009-04-04 Thread Tom Jackson
Andrew, If the connection has been closed, nothing will be sent. But anyone who uses streaming output will have to live with the possibility that an error will occur mid-response. I don't recommend using streaming output for many reasons, but the most obvious is the one you mention: you (the gener

Re: [AOLSERVER] missing access log entries

2009-04-04 Thread Andrew Steets
Hi Tom, Attachments seem to work ok on this list. I don't think we can return 500 internal server error after Ns_ConnRunRequest has been invoked as it may have already sent an http response code via streaming output or ns_returnxxx. -Andrew On Sat, Apr 4, 2009 at 12:00 PM, Tom Jackson wrote: >

Re: [AOLSERVER] missing access log entries

2009-04-04 Thread Tom Jackson
Here is a test patch for the ns_adp_abort issue. The patch enables sending an error message in the case of an actual error during adp processing, or after a postauth filter (preauth errors already allow this behavior). Also, logging is enabled in all cases. If an error occurs, a 500 response is

Re: [AOLSERVER] missing access log entries

2009-04-03 Thread Tom Jackson
On Fri, 2009-04-03 at 16:20 -0600, Jim Davidson wrote: > HI, > > Sorry -- I wrote the original and goofy code leading to this > confusion... > Thanks for monitoring our head scratching! I have a few random thoughts, and I'm not sure they add up to a particular conclusion or not. But here goe

Re: [AOLSERVER] missing access log entries

2009-04-03 Thread Jim Davidson
HI, Sorry -- I wrote the original and goofy code leading to this confusion... Based on this discussion I think it's NOT working as intended. If there was a time I thought ns_adp_abort should mean "return NS_ERROR for connection status and thus no-traces..." I've long since forgotten t

Re: [AOLSERVER] missing access log entries

2009-04-03 Thread Tom Jackson
After reviewing the code from version 4.0 and up, it looks like the intended behavior is to not log an access log entry if an error occurs. In fact, the request was not serviced, so an entry in the access log would be incorrect, the correct log entry would be a 500 internal server error. I just

Re: [AOLSERVER] missing access log entries

2009-04-03 Thread Andrew Steets
This is what I suggested a few emails ago :-) > I looked a little deeper into the source. The confusion seems to > arise in NsAdpFlush() which is run at the end of all ADP processing. > The code there is smart enough to recognize when an abort exception > has been signalled; it sets the TCL resul

Re: [AOLSERVER] missing access log entries

2009-04-03 Thread Andrew Steets
My original concern was with the access logging proc, which happens to be run as a trace filter. I think that the access log entries should be generated regardless of whether or not ns_adp_abort is called. I don't care too much about anything else that was installed as a trace filter. Do you agr

Re: [AOLSERVER] missing access log entries

2009-04-03 Thread Tom Jackson
Hey, Hopefully this is my last post on this subject, I think I actually found the bug. The bug is in NsAdpFlush from nsd/adprequest.c: 214- */ 215- 216-Tcl_ResetResult(interp); 217:if (itPtr->adp.exception == ADP_ABORT) { 218-Tcl_SetResult(interp, "adp flush disabled: adp aborte

Re: [AOLSERVER] missing access log entries

2009-04-03 Thread Tom Jackson
Andrew, I wasted a little more time looking at the actual code. My impression is that everything is working as expected. If there is an error in a postauth filter or in adp processing (registered proc), trace filters are skipped. Until about 4.5, errors during preauth also skipped trace filters. N

Re: [AOLSERVER] missing access log entries

2009-04-02 Thread Tom Jackson
Andrew, Here is a link to the tiny bit of code which handles every request: http://junom.com/gitweb/gitweb.perl?p=aolserver.git;a=blob;f=aolserver/nsd/queue.c;hb=HEAD#l585 >From my reading of this, the problem must be that ns_adp_abort is not returning NS_OK or NS_FILTER_RETURN. It probably shou

Re: [AOLSERVER] missing access log entries

2009-04-02 Thread Tom Jackson
The trace filters are filters which run at the end of a connection. Here is the basic order of processing: preauth filters (frfr) first register, first run auth proc (one only) postauth filters (frfr) registered proc (one only) trace filters (frlr) first to register, last to run There is also ano

Re: [AOLSERVER] missing access log entries

2009-04-02 Thread Andrew Steets
What was the original purpose of "trace" filters? At the C API level there is a distinction between between a trace filter and a cleanup callback, but it doesn't look like you can register a cleanup proc from TCL. Maybe this was mistakenly omitted? The cleanup procs run unconditionally. It seem

Re: [AOLSERVER] missing access log entries

2009-04-02 Thread Tom Jackson
Gustaf, You may be "using" traces but not realize it, it sounds like ns_adp_abort isn't don't what was originally intended. I wouldn't worry about an runtime error caused during running traces, it would be an error to even use ns_adp_abort in a trace filter because the connection is already finis

Re: [AOLSERVER] missing access log entries

2009-04-02 Thread Gustaf Neumann
Andrew Steets schrieb: The patch I sent earlier seems to fulfill these needs, but I am worried about corner cases where LogTrace (from the nslog module) could blow up. Nothing about the state of the Conn * seems to be guaranteed when the ConnCleanup callbacks are called. Dear Andrew, i thin

Re: [AOLSERVER] missing access log entries

2009-03-29 Thread Tom Jackson
Hey again, I just noticed ns_adp_break: http://rmadilo.com/files/nsapi/ns_adp_break.html In contrast to ns_adp_abort. ns_adp_break flushes all output to the client then aborts further adp processing. So my guess is that you use ns_adp_break when you want to use the current ADP buffer, and you

Re: [AOLSERVER] missing access log entries

2009-03-29 Thread Tom Jackson
On Fri, 2009-03-27 at 09:56 +0100, Gustaf Neumann wrote: > Andrew Steets schrieb: > > As Scott suggested, we should probably log everything, at least for > > some reasonable value of "everything." Even if you switch the access > > log trace to the cleanup callback, you still don't get access entri

Re: [AOLSERVER] missing access log entries

2009-03-27 Thread Gustaf Neumann
Andrew Steets schrieb: I'm inclined to agree with you that the current behavior is a bug, but it raises the question: should there be such a function that says "this connection wasn't handled, don't even log it" - or, should ALL connections always be logged, even if it's aborted? As Scott

Re: [AOLSERVER] missing access log entries

2009-03-26 Thread Alexey Pechnikov
Hello! On Friday 27 March 2009 01:21:38 Dossy Shiobara wrote: > Why are you using ns_adp_abort here instead of ns_adp_return? = $ cat test.adp <% ns_adp_puts "Test ADP" ns_adp_include test.inc %> $ cat test.inc <% ns_adp_puts "Test inc" ns_return 200 {text/html; charset=utf-8

Re: [AOLSERVER] missing access log entries

2009-03-26 Thread Andrew Steets
This thread has turned into an interesting discussion of what may be a fairly useful ADP programming idiom, but I don't want to focus too much on ns_adp_abort. There are other cases that will cause the ADP to run without generating an access log entry. For example, if you call one of the ns_retur

Re: [AOLSERVER] missing access log entries

2009-03-26 Thread Tom Jackson
By a strange coincidence I needed had a similar issue with Tcl (tcl pages). I did a ns_returnredirect way deep into an application. I was hoping to abort further execution of Tcl code, but by design, script execution continues. I considered that only throwing an error would unwind everything cor

Re: [AOLSERVER] missing access log entries

2009-03-26 Thread Dossy Shiobara
On 3/26/09 5:12 PM, Alexey Pechnikov wrote: How about ADP or file is included into ADP with code ns_return 200 {text/html; charset=utf-8} {some message} ns_adp_abort or ns_returnfile 200 [ns_guesstype $fname] $fname ns_adp_abort This request _must be_ logged. ns_adp_break is not correct here

Re: [AOLSERVER] missing access log entries

2009-03-26 Thread Alexey Pechnikov
Hello! On Thursday 26 March 2009 21:40:05 Dossy Shiobara wrote: > I wonder - should this be the documented known behavior of ns_adp_abort > vs. ns_adp_return?  i.e., abort indicates that the connection is > intentionally terminated, not logged, etc. vs. ns_adp_return which halts > ADP processing b

Re: [AOLSERVER] missing access log entries

2009-03-26 Thread Andrew Steets
On Thu, Mar 26, 2009 at 1:40 PM, Dossy Shiobara wrote: > I wonder - should this be the documented known behavior of ns_adp_abort vs. > ns_adp_return?  i.e., abort indicates that the connection is intentionally > terminated, not logged, etc. vs. ns_adp_return which halts ADP processing > but contin

Re: [AOLSERVER] missing access log entries

2009-03-26 Thread Tom Jackson
On Thu, 2009-03-26 at 14:40 -0400, Dossy Shiobara wrote: > On 3/26/09 1:31 PM, Andrew Steets wrote: > > Hello, > > > > There are certain cases where connections probably ought to generate > > access log entries but do not. Specifically if an ADP exits via > > ns_adp_abort no access log entry will

Re: [AOLSERVER] missing access log entries

2009-03-26 Thread Scott Goodwin
All connections should be logged as requests that came from clients along with details on how the server responds. Some indication that the connection was aborted should be made in the log, perhaps with a count of how many bytes were transferred. In cases where no response is going to be se

Re: [AOLSERVER] missing access log entries

2009-03-26 Thread Dossy Shiobara
On 3/26/09 1:31 PM, Andrew Steets wrote: Hello, There are certain cases where connections probably ought to generate access log entries but do not. Specifically if an ADP exits via ns_adp_abort no access log entry will be generated, but data may have been returned to the client. This seems lik

[AOLSERVER] missing access log entries

2009-03-26 Thread Andrew Steets
Hello, There are certain cases where connections probably ought to generate access log entries but do not. Specifically if an ADP exits via ns_adp_abort no access log entry will be generated, but data may have been returned to the client. This seems like a bug. The access log callback is regist