Gustaf, I'm going to ask that the patch be removed and replaced with a module. I've already written one which does the same thing.
We are in a bad habit in this community of letting anything happen at the least possible cost to those who modify core code. This is a perfect example of the situation. We now have two new subcommands for ns_conn. Some would phrase it as only two. But, for instance, most commands have like four total subcommands. Regardless of the number, we have just added two undocumented commands to the core code. We have a major problem with documentation here, and we know why: developers are not required (not even encouraged!) to do any work documenting new code or features. They don't have to provide any examples which work with AOLserver. All that they have to do is to commit a patch. This easy five minute job now requires everyone else to do work. Second, the patch does nothing for AOLserver without at least figuring out where to get the ::thread package. Again, there is no documentation on where to get it, how to install it or use it. But the example code which was referenced below is part of a much larger project: OpenACS, and a very recent version of it. In fact, it is part of OpenACS that isn't even necessary for the core features, and is only there for optimization as you have pointed out. The examples are written in xotcl, which I have tried to figure out for a year or more with little success. Third, some technical issues: New features which don't require core changes should be done as modules, unless there is an obvious close relationship to the core functions. In this case, the new feature is so foreign to the normal workings of AOLserver that the code should be isolated from the core until it is fully understood. Since you didn't write the original code, I assume that you can't really explain all the details of what is going on and why. I spent a lot of time looking at this new feature and seeing if there was anything similar in AOLserver. What I discovered was quite the opposite. There is not a single example where two conns (conns, not sockets) are returning data to clients in the same thread. It is possible that the AOL team once considered this, but none of the core API support this in any way. Each is written to work in a single thread (or block) and contains an internal event loop (in C). It is impossible to imagine that the core could be rearranged to support a generic multiplexing of client returns. When you survey how many different types of returns there are, you will immediately understand why. The easiest way to put it is that reading client data is essentially a binary read until passed to a connection thread. Once the connection thread has all the binary data, it can do conversions as needed. AOLserver request model is a pipeline which is specialized per request. We can consider this new feature as creating a branch where the original pipeline continues (like after ns_conn close), and another branch sends data to the client. But AOLserver has no control over the new branch. It could fail and we would never know. There are no logs of failures. Actually, not quite true, it looks like if the delivery fails for some reason, or if you don't call [ns_conn contentsentlength $filelength], you get zero bytes in the access.log. On the surface, the patch appears (as described) to do something like ns_returnfile. All the patch does is to make a copy of the connection and wrap the sock in a Tcl_Channel. It doesn't create a worker thread, doesn't pass the conn to this thread, it doesn't write headers or open the static file. There is no way to signal for shutdown and to a thread join. All of this work has to be done by the developer, including debugging, testing, and everything else. So what we have is a new way for everyone to start doing things in different ways, wondering why it doesn't work, complaining about why it doesn't work the way they expect, etc. Some will probably offer their own patches to make it work better. When we accept the patch, we allow you to transfer all this mess to the AOLserver community. Now we have to answer all these concerns, you don't. This also short circuits discussion about the best way to do this in AOLserver, and what to recommend to new users. Why? We just had another good discussion about the performance of AOLserver for static content. We discovered that the model for AOLserver works very well in this context, so much so that we are kind of scrathing our heads over any need to think too hard about performance improvements. There really are better things to work on. There seems to be an obsession among new developers that performance tweaking is the first thing to be considered when writing code. They think about a few lines of code, one syscall vs another. But when you read the AOLserver sources, it really starts to dawn on you: it ain't easy. Fortunately it is already written, so why not use it to best advantage? Part of that being done for you is the module system. If new functionality can be added without patching the core it does two important things: it provides examples of how to do it, and proves (as in tests) the sound design of the core. If developers are encouraged to provide patches instead of modules it works against these goals. Anyway, I fully support the code in module form. I don't completely understand the performance or security impact, but as a module, users can take it or not by their own choice. If I had to do a similar task, I would probably create a unique code and create a symlink to the actual file, then redirect the user to that file using another instance of AOLserver. The other instance would check the age of the symlink and serve the file if under some configured amount. If the entire file gets served, remove the link. This way, you can fully leverage the REST style, and it will likely release your expensive connection thread even faster than the current example. Using the code as a module (minus the ability to transfer the channel), here is an example of use (the string bgwrite could easily be changed to anything else, I chose it before I realized that the code doesn't do any bgwriting.): # Get nContentSent (should be 0) set contentsentlength [ns_bgwrite contentsentlength] # Set nContentSent (to zero) ns_bgwrite contentsentlength $contentsentlength # Dup sock and wrap in Tcl_Channel set channel [ns_bgwrite channel] # File to return: set dir [file dirname [info script]] set image sns-thumb.jpg set file [file join $dir $image] # Get length of file: set contentLength [file size $file] # Get content type: set contentType [ns_guesstype $file] # Create last modified header data: set lastModified [ns_httptime [file mtime $file]] # Create header: set header "HTTP/1.1 200 OK Last-Modified: $lastModified MIME-Version: 1.0 Date: [ns_httptime [ns_time]] Server: AOLserver/4.5.0 Content-Type: $contentType Content-Length: $contentLength Connection: close " # Open file to send: set inFd [open $file r] # Configure channels: fconfigure $inFd -translation binary fconfigure $channel -translation binary # put headers (could block forever): puts $channel $header # fcopy file to channel. fcopy $inFd $channel -command [list close $channel] #^^in a conn thread, this only sends the first 4096 bytes # while in background mode (caused by -command switch) # The following still happens regardless of fcopy status: # Set nContentSent so log file will look normal: ns_bgwrite contentsentlength $contentLength # Log something ns_log Notice "File $file has contentlength = $contentLength" # No need to return, as [ns_bgwrite channel] has effect of [ns_conn close]. # Traces will still run on this thread # what happens to keepalive? ### Log File Messages: # error.log [-default:0-] Notice: File /web/nsd45/servers/jnm/pages/sns-thumb.jpg has contentlength = 28672 [-default:0-] Notice: running ::traceFilter, sleeping for '1' seconds # access.log 192.168.1.102 - - [01/Oct/2007:06:58:23 -0700] "GET /nsbgwrite.tcl HTTP/1.0" 200 28672 "" "" # access.log without setting nContentSent 192.168.1.102 - - [30/Sep/2007:22:00:25 -0700] "GET /nsbgwrite.tcl HTTP/1.0" 200 0 "" "" tom jackson On Saturday 29 September 2007 13:55, Gustaf Neumann wrote: > Tom Jackson schrieb: > > Okay, after looking further into this patch, I see that it doesn't > > actually add any functionality to AOLserver. It looks like you would have > > to install a newer version of OpenACS to use this. > > as i wrote in my earlier mail, the patch is simple and small and adds > just two > subcommands to ns_conn. Thanks to dossy, the patch is commited to cvs. > head. > > The applications to "ns_conn channel" are on the tcl layer and > are quite simple to use. Look into the xotcl-core package > (xotcl-core/tcl/bgdelivery-procs.tcl) in the openacs cvs repository. > With the patch, xotcl-core and libthread, one can replace e.g. > ns_returnfile 200 $mime_type $filename > by > ad_returnfile_background 200 $mime_type $filename > in cr_write_content in acs-content-repository/tcl/revision-procs.tcl > to use the background delivery for content sent from the > content repository (e.g. file store). -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
