Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-10-02 Thread Gustaf Neumann
Hi everybody, Tom Jackson schrieb: Gustaf, I'm going to ask that the patch be removed and replaced with a module. i got the - wrong - impression that you (Tom) revised your proposal to change the patch into a module, when you realized, that the patch is NOT implementing background delivery,

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-10-02 Thread Vasiljevic Zoran
On 02.10.2007, at 19:51, Tom Jackson wrote: web/navi $ ./bin/nsd -f -t sample-config.tcl [02/Oct/2007:10:37:45][6858.690331232][-main-] Notice: nsmain: Tcl version: 8.4.14 [02/Oct/2007:10:37:45][6858.690331232][-main-] Fatal: NsTclInitObjs: sizeof(int) sizeof(long) Aborted This has been

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-10-02 Thread Stephen Deasey
On 10/2/07, Tom Jackson [EMAIL PROTECTED] wrote: ... it would be nice if new code followed AOLserver coding norms. Maybe you can get naviserver to take out their code. The module I wrote at least compiled against their server... You're using symbols declared in nsd/nsd.h, which is private to

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-10-02 Thread Tom Jackson
Stephen, Thanks for pointing that out. I was wondering about that. I have posted new module code which avoids the private symbols. But this removes the contentsentlength option. I still wonder what the difference between dup'ing and not dup'ing is for (spliceout vs. non-spliceout). In the

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-10-02 Thread Stephen Deasey
On 10/3/07, Tom Jackson [EMAIL PROTECTED] wrote: Anyway, I don't really know if this will work as the original, I was able to fcopy in background a short text file, but larger than 4096 bytes just gets that amount according to wget. In foreground/blocking mode, fcopy returns larger files.

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-10-02 Thread Tom Jackson
Stephen, Thanks, that worked. The -command was executed logging 'okay' + number of bytes written. But the ns_log statements following vwait are not executed. I assume that is expected? I posted the updated script at: http://rmadilo.com/files/nsbgwrite/nsbgwrite.tcl tom jackson On Tuesday

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-10-01 Thread Tom Jackson
Gustaf, I'm going to ask that the patch be removed and replaced with a module. I've already written one which does the same thing. We are in a bad habit in this community of letting anything happen at the least possible cost to those who modify core code. This is a perfect example of the

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-10-01 Thread Andrew Piskorski
On Mon, Oct 01, 2007 at 09:40:19AM -0700, Tom Jackson wrote: Gustaf, I'm going to ask that the patch be removed and replaced with a module. Why? What possible harm does including Gustaf's patch in the stock server cause, rather than loading it as an additional module? Regardless of the

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-10-01 Thread Dossy Shiobara
On 2007.10.01, Andrew Piskorski [EMAIL PROTECTED] wrote: And just where is Gustaf SUPPOSED to add docs for his new feature anyway? I will try to update doc/ns_conn.n soon and include the two new ns_conn subcommands that Gustaf's patch adds. And AOLserver should probably be changed somtime to

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-10-01 Thread Jeff Rogers
Andrew Piskorski wrote: What's the actual problem with Gustaf's code? You've obviously read and thought about it, Tom (which I have not), but so far I see a lot of theoretical hand wavy complaints from you, but little solid criticism of the actual code. I also think the code belongs on a

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-10-01 Thread Vasiljevic Zoran
On 01.10.2007, at 21:04, Tom Jackson wrote: tclsock.c-766-/* tclsock.c-767- * Pass a dup of the socket to the callback thread, allowing tclsock.c-768- * this thread's cleanup to close the current socket. It's tclsock.c-769- * not possible to simply register the channel

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-09-30 Thread Dossy Shiobara
On 2007.09.28, Gustaf Neumann [EMAIL PROTECTED] wrote: I have asked dossy for commit permissions to add my patches to the aolserver on Sept 22, 2006 and sent him the patches as well, but my impression was that there was very little or no interest. Sorry, I totally forgot to respond to your

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-09-29 Thread Gustaf Neumann
Tom Jackson schrieb: Okay, after looking further into this patch, I see that it doesn't actually add any functionality to AOLserver. It looks like you would have to install a newer version of OpenACS to use this. as i wrote in my earlier mail, the patch is simple and small and adds just

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-09-29 Thread John Buckman
When comparing lighthttpd vs aolserver, notice that aolserver only does worse than lighthttpd for large files, and on the same file system/hardware. Thus, the difference in benchmarks is not likely to be the access logs or disk. Lighthttpd is *not* using the system call to send a file to a

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-09-28 Thread Tom Jackson
When I look at the patch, it seems to me that this could be put into a module. The new C level command doesn't need to be a static command, it uses only external functions and variables (Ns_*, Tcl_*). Can we work togeather to get a module instead of a patch? I'll work up a module file today

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-09-28 Thread Dossy Shiobara
On 2007.09.28, John Buckman [EMAIL PROTECTED] wrote: The 404 handler approach is very clever, that'll do exactly what I need, thanks! Dossy: not sure if it's even worth benchmarking this, as this approach will yield the static-file-speeds, which are amazing, except in the rare

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-09-28 Thread Tom Jackson
Okay, after looking further into this patch, I see that it doesn't actually add any functionality to AOLserver. It looks like you would have to install a newer version of OpenACS to use this. I have the stubs in for an AOLserver C module, but I'm not sure about a few things. Two commands

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-09-28 Thread Gustaf Neumann
Dossy Shiobara schrieb: On 2007.09.27, Jeff Rogers [EMAIL PROTECTED] wrote: [...] It seems at first glance that it would make more sense to hand the task of writing to the connection back to the driver thread once the connection thread is done with it, [...] I don't know if the change

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-09-28 Thread John Buckman
On Sep 28, 2007, at 8:43 AM, John Buckman wrote: My solution to that problem was simply caching in the filesystem and serving static files. The way this works in a multi-server environment is that the custom 404 handler figures out the request was for /photo/123/axbcgsfdt.jpg and just grabs

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-09-27 Thread Tom Jackson
I was looking at lighttpd performance (at http://trac.lighttpd.net/trac/wiki/Docs%3APerformance ) Considering how well AOLserver stands up to lighttpd, my question was why does lighttpd do better? For sending small files AOLserver is slightly better, but then performance goes down. One

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-09-27 Thread Dossy Shiobara
On 2007.09.27, Tom Jackson [EMAIL PROTECTED] wrote: Another reason might be that lighttpd doesn't write an access.log by default. I wonder if you can turn this off in AOLserver? Yes, you can. If you don't load the nslog module, you turn off access logging. -- Dossy -- Dossy Shiobara

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-09-27 Thread Jeff Rogers
Tom Jackson wrote: I was looking at lighttpd performance (at http://trac.lighttpd.net/trac/wiki/Docs%3APerformance ) Considering how well AOLserver stands up to lighttpd, my question was why does lighttpd do better? I was wondering if it had something to do with how aolserver does its

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-09-27 Thread Dossy Shiobara
On 2007.09.27, John Buckman [EMAIL PROTECTED] wrote: Lighthttpd is *not* using the system call to send a file to a socket (I forget the name) as this call was taken out of the Linux kernel, I believe with 2.4. I remember reading a note about this from Linus, that the performance for

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-09-27 Thread Tom Jackson
On Thursday 27 September 2007 09:55, John Buckman wrote: When comparing lighthttpd vs aolserver, notice that aolserver only does worse than lighthttpd for large files, and on the same file system/hardware. Thus, the difference in benchmarks is not likely to be the access logs or disk. Yeah,

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-09-27 Thread Dossy Shiobara
On 2007.09.27, Jeff Rogers [EMAIL PROTECTED] wrote: [...] It seems at first glance that it would make more sense to hand the task of writing to the connection back to the driver thread once the connection thread is done with it, [...] Since this is such an obvious change, would I be correct

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-09-27 Thread Tom Jackson
On Thursday 27 September 2007 11:08, Dossy Shiobara wrote: It might be possible to push static file processing further up the chain into the DriverThread and get better performance on larger static files--or, have one dedicated I/O thread separate from the main driver thread to handle async

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-09-27 Thread John Buckman
Hmm--I'm not sure what you're thinking of or referring to, but the common optimization is to use sendfile(2), which seems to be alive and well in the 2.6 tree. Whoops, you're right, I just remembered a sysadmin email about sendfile not existing on the kernel we're using. Hmm... Sendfile

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-09-27 Thread Dossy Shiobara
On 2007.09.28, Bas Scheffers [EMAIL PROTECTED] wrote: I would suspect Linux being faster on a box like this than OS X (BSD) because of better threading support. Or am I wrong in that assumption? I'm only guessing, but I doubt there would be a significant performance difference between OS X's

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-09-27 Thread Bas Scheffers
On 28/09/2007, at 9:28 AM, Dossy Shiobara wrote: Yes! Finally, someone else who uses the 404-handler-as-request- processor pattern! Indeed, you can't beat static file serving performance. And, My first inspiration for this came way back in the last century, from working with Vignette

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-09-27 Thread Bas Scheffers
On 28/09/2007, at 3:38 AM, Dossy Shiobara wrote: I bet with just a few minutes of tweaking and tuning, we can get between 4k-8k simple dynamic req/sec out of your hardware. Hear, hear. I just blasted my brand-spanking-new Quad 2.6 Mac Pro using ab. I was testing my ns_session, so it was

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-09-27 Thread Bas Scheffers
On 28/09/2007, at 5:04 AM, John Buckman wrote: I've been looking at C-caching of Tcl dynamic content, with dirty cache support. For example, replacing the Tcl code that returns a user's uploaded photo with C code. I wrote C code to do this, and got 14k/second vs 240/s for the same tcl

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-09-27 Thread Rusty Brooks
Yeah, I do this too. To dirty the cache you can just delete the file. I do regular rounds to delete old files. I don't use the 404 though, that's a neat idea. Instead I register a proc that ns_returnfiles the cache file if it exists, otherwise it makes it and then returns it. Bas

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-09-27 Thread Dossy Shiobara
On 2007.09.28, Bas Scheffers [EMAIL PROTECTED] wrote: My solution to that problem was simply caching in the filesystem and serving static files. The way this works in a multi-server environment is that the custom 404 handler figures out the request was for /photo/123/axbcgsfdt.jpg and

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-09-27 Thread Dossy Shiobara
On 2007.09.27, Rusty Brooks [EMAIL PROTECTED] wrote: Yeah, I do this too. To dirty the cache you can just delete the file. I do regular rounds to delete old files. I don't use the 404 though, that's a neat idea. Instead I register a proc that ns_returnfiles the cache file if it exists,

[AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-09-25 Thread John Buckman
I did some benchmarks of aolserver vs lighthttpd on plain files of various sizes, on my 8cpu 64 bit server. For 3 runs, with a 4k text file, the lighthttpd stats were 15103.87/ s, 14845.20/s and 15307.17/s, vs (as Dossy reports http://dossy.org/ archives/000517.html) aolserver's 15237.00/s.

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-09-25 Thread Dossy Shiobara
On 2007.09.25, John Buckman [EMAIL PROTECTED] wrote: I did some benchmarks of aolserver vs lighthttpd on plain files of various sizes, on my 8cpu 64 bit server. Thanks! This is all fantastic information to have! www64:/b# ab -c 5 -n 5 http://images.bookmooch.com/x.txt Since your

Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks

2007-09-25 Thread John Buckman
www64:/b# ab -c 5 -n 5 http://images.bookmooch.com/x.txt Since your machine is a 2-CPU 8-core box, could you try runs with -c 8 and -c 16? It may have no effect, but it'd be nice to know that for sure. I get more-or-less the same numbers at -c 8, -c 16 and -c 32, varying about 100