caching includes parse tree

Any thought into parsing the results of the includes filter (offsets, etc.). In our environment, parsing the includes files is a huge performance hit. We are willing to help in any way.

Re: caching includes parse tree

Glenn wrote: For files where server-side includes are used for page fragment reuse rather than complicated server-side conditional processing, this could be an easy win, and a bit more flexible than the XBitHack. In our environment, we have several includes on a page, only one of which is

Bug in mod_cache?

MCacheMinObjectSize 0 MCacheMaxObjectSize 65536 MCacheMaxStreamingBuffer 65536 -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: Bug in mod_cache?

] == '/') { return DECLINED; } I also ask, why? url's ending in / are 80%+ of our requests. -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: [STATUS] (httpd-2.0) Wed Nov 19 23:45:14 EST 2003

Rodent of Unusual Size wrote: * Replace some of the mutex locking in the worker MPM with atomic operations for higher concurrency. server/mpm/worker/fdqueue.c 1.24, 1.25 +1: brianp Is anyone working on this right now? If not, this is of some interest to me. -- Brian Akins

2.0 atomics in worket mpm

André Malo wrote: * Brian Akins [EMAIL PROTECTED] wrote: Rodent of Unusual Size wrote: * Replace some of the mutex locking in the worker MPM with atomic operations for higher concurrency. server/mpm/worker/fdqueue.c 1.24, 1.25 +1: brianp Is anyone working on this right now

Global keepalive count possible?

) -After create connection, check this variable, if less than max, then process normally, increment variable, and register a cleanup on connection pool. This cleanup will decrement the variable. Thoughts? -- Brian Akins Senior Systems Engineer CNN Internet Technologies

[PATCH 25137] atomics in worker mpm

Backported from 2.1. Stable for me in various loads. -- Brian Akins Senior Systems Engineer CNN Internet Technologies --- /home/bakins/src/httpd-2.0.48/server/mpm/worker/fdqueue.c 2003-11-06 08:16:03.0 -0500 +++ fdqueue.c 2003-09-28 23:58:41.0 -0400 @@ -57,26 +57,40

Re: [PATCH 25137] atomics in worker mpm

squirrelly before. YMMV. -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Stupid question about pools

Does the request pool get cleaned up after every request? IE, If I register a cleanup on r-pool, will it get ran after every request? -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: Stupid question about pools

to make sure. I assume the same is true for connections? Every time a connection is closed, the registered cleanups would get ran. -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: [PATCH 25137] atomics in worker mpm

Jeff Trawick wrote: Brian Akins wrote: Backported from 2.1. Stable for me in various loads. without the non-portable atomic code available and enabled in APR, this is going to hurt performance, right? (more mutex operations performed in mainline path for the unlucky who use the non-portable

Re: [PATCH 25137] atomics in worker mpm

. All: Please correct me if I am wrong. I'm sure you will ;) -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: UseCanonicalName Off *surprise*

is: UseCanonicalName On Listen 80 Listen 8080 ServerName www.domain.com:80 So redirects, no matter what port they came in one, get redirected to port 80. This was our desired effect. -- Brian Akins Senior Systems Engineer CNN Internet Technologies

FD_SETSIZE comparison

). Is this just not true elsewhere? Can someone explain? -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Save brigade and buckets

. -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: Save brigade and buckets

by a random server, For my uses it actually would help alot if I created my own longlived pool. Thanks. -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Custom bucket types

It would be cool if there was a way to create custom bucket types. It looks like all we would have to do is expand how the built in buckets declare themselves. Any comments. I'm willing to play with it.. -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: [PATCH] Location block speed up

How wrong would it be to have a map_to_storage that simply filled out r-finfo and returned OK (or the appropriate error) always? IE, no directory or file walk. From what I understand, Directory and File directives would be useless, but Location would still work. Is this correct? -- Brian

Hook order

Is there a doc somewhere that describes the order the hooks are called in apache 2? I.e, 1. ap_hook_pre_connection 2. ap_hook_process_connection N. ap_hook_log_transaction TIA -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: fstat?

Brian Akins wrote: More of an apr question, Any reason there is no equivalent to fstat? Never mind. I found it. obviously it's apr_file_info_get -- Brian Akins Senior Systems Engineer CNN Internet Technologies

adding output filters in quickhandler

Possible bug: If I add an output filter in the quickhandler for something like /foo/ and later mod_dir changes it to /foo/index.html, my output filter is never ran. Is this why mod_cache cannot cache things that end in /. This just seems broken... -- Brian Akins Senior Systems Engineer CNN

Re: 2.0.48 worker mpm on RH3 NPTL results

In our testing RH3 was horrible -- worse than 2.1. However, a system with a stock 2.6 kernel with glic2.3.3 with nptl performed comfortable better than 2.1 YMMV -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: FileSystem v.s. Other Resources [was configurable Location?]

virtual-dynamic-handler /Location Alias /normal/stuff/dyn /dynamic-stuff How would all this affect this situation? IE, we would still need to know that /normal/stuff/dyn/1/2/test.html became /dynamic-stuff/1/2/test.html -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: FileSystem v.s. Other Resources [was configurable Location?]

Joshua Slive wrote: On Mon, 9 Feb 2004, Brian Akins wrote: Location /dynamic-stuff SetHandler virtual-dynamic-handler /Location Alias /normal/stuff/dyn /dynamic-stuff How would all this affect this situation? IE, we would still need to know that /normal/stuff/dyn/1/2/test.html became

Re: apache 2.0.48 mod_userdir over nfs mount

UHISWdev UHISWdev wrote: Thanks to all kind folks who replied - that worked a treat! Does anyone know why sendfile would fail over ncpfs? Sendfile does not work on anything but local storage. -- Brian Akins Senior Systems Engineer CNN Internet Technologies

SetEnv and other modules

environment setter... -- Brian Akins Senior Systems Engineer CNN Internet Technologies

worker error

[Tue Apr 06 12:16:10 2004] [crit] [Tue Apr 06 12:16:10 2004] file worker.c, line 708, assertion rv == 0 || !csd failed Can someone interpret? 2.0.49 on Linux -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: worker error

and all is well now. Is this portable? -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: Request for feedback - UseCanonicalPort

it not care about the ServerName, but use a selected port. -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: expires in redirects

Joe Orton wrote: On Mon, May 10, 2004 at 11:02:21AM -0400, Brian Akins wrote: Any reason why expires set by mod_expires are not added to redirected requests? Should I hack up my own expires that does? Are you using 2.0.49? There's a fix for setting Expires on error responses

expires in redirects

Any reason why expires set by mod_expires are not added to redirected requests? Should I hack up my own expires that does? -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: [STATUS] (httpd-2.0) Wed May 5 23:45:14 EDT 2004

that, early in the request, changes r-parsed_uri.port to the correct port. This works and should not be affected by the proposed changes, but I'm unsure if it's screwing something else up. This is a little confusing. Is there a better way to do this? -- Brian Akins Senior Systems Engineer

Re: [STATUS] (httpd-2.0) Wed May 5 23:45:14 EDT 2004

Mads Toftum wrote: Wouldn't adding the port to ServerName be what you need? ServerName example.com:80 Iirc it is even suggested in the docs. AFAIK, that only works when usecanonical names is on. Otherwise, it just uses what the client sends. -- Brian Akins Senior Systems Engineer CNN

{PATCH] Re: x86_64 atomics and linux

Joe Orton wrote: On Wed, Jun 02, 2004 at 03:40:52PM -0400, Brian Akins wrote: AFAIK, the linux x86 atomic stuff can be used unchanged on Linux x86_64. This is based on my digging in the kernel source. All the functions apr uses are identical. This is already done for APR HEAD

Errors from handlers and EOS in filters

Here's a hypothetical situation: Say proxy_http gets a read error halfway through a request. It returns r-status = HTTP_BAD_GATEWAY. Do the output filters (deflate, cache, etc) receive an EOS bucket? -- Brian Akins Senior Systems Engineer CNN Internet Technologies

mod_status hook

How do I register a mod_status hook. Looking at the code, I figure I register a hook using ap_hook_status_hook (if I try anything else, I get a complier warning). However I get undefined symbol: ap_hook_status_hook when I try to run. Obviously, I'm missing something... -- Brian Akins Senior

Re: mod_status hook

Edward Rudd wrote: APR_OPTIONAL_HOOK(status_hook,my_status_hook, NULL,NULL,APR_HOOK_MIDDLE); Thanks! I had to add another argument: APR_OPTIONAL_HOOK(ap, status_hook, my_status_hook, NULL, NULL, APR_HOOK_MIDDLE); -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: mod_status hook

21 Apr 20 11:31 libaprutil-0.so - libaprutil-0.so.0.9.5 lrwxrwxrwx1 bakins root 21 Apr 20 11:31 libaprutil-0.so.0 - libaprutil-0.so.0.9.5 -rwxr-xr-x1 bakins root 736314 Apr 20 11:31 libaprutil-0.so.0.9.5 ... -- Brian Akins Senior Systems Engineer CNN Internet

Some Oprofile results

libaprutil-0.so.0.9.5file_bucket_read -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: Some Oprofile results

Bill Stoddard wrote: I submitted a patch to rework ap_rgetline/core_input_filter a year or so ago that provided some performance improvements. If you are inclined, it might be interesting to dig that patch out and see how it profiles. Sure. Repost-it and/or send it to me. -- Brian Akins Senior

Re: Some Oprofile results

. mod_url_cache (I couldn't come up with a better name) can fill up two Gigabit interfaces on our Opterons and still have ~25% CPU idle time. -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: Some Oprofile results

read. Also, using a filesystem and sendfile is much faster than serving from RAM, plus multiple processes can share the same cache. mod_cache doesn't do that. All of these are based on my observartions of our environment. -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: Some Oprofile results

(ctx-b); 65812 0.7953 :if (APR_STATUS_IS_EAGAIN(rv)) { :rv = APR_SUCCESS; :} -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: mod_cache: allowing urls ending in / to be cached

this filter, so it never gets cached. -- Brian Akins Senior Systems Engineer CNN Internet Technologies

pre_read ?

Is there a hook that gets called after a connection is recieved that is only called once per connection, not per request? -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: [PATCH] mod_cache fixes

much, but when you are doing 20k+ requests/sec it makes a difference. -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: [PATCH] mod_cache fixes: #8

directories. Trust me, it does speed things up a bit |-- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: [PATCH] mod_cache fixes: #8

, etc.) and thats it. Only useful when you know a lot about the content being served, which is the case for a reverse proxy usually. -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: [PATCH] mod_cache fixes: #3

== ENABLE_SENDFILE_OFF) ? 0 : APR_SENDFILE_ENABLED) #endif , 0, r-pool)) != APR_SUCCESS) { -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: [PATCH] mod_cache fixes: #3

. -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: [PATCH] mod_cache fixes: #8

are not sure it exists? -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: [PATCH] mod_cache fixes: #8

Justin Erenkrantz wrote: The other bottleneck I looked at was MD5 as the on-disk naming scheme. I think MD5 is a poor choice here because it's not very fast. Ideally, switching to a variant of the times-33 hash might work out better. *shrug* How to handle collisions? -- Brian Akins Senior

Re: [PATCH] mod_cache fixes: #8

ap_scan_script_header_er which is fairly slow. and possibly bypassing or at least reworking input filters. And the prebuilt headers will need to be properly aligned in memory. If using disk cache, this obviously does not matter. -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: [PATCH] mod_cache fixes: #8

+Accept-Encoding%3A+gzip+Cookie%3A+Special%3DSomeValue Should be pretty fast (?) if the urlencode was effecient. -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: [PATCH] mod_cache fixes: #8

if someone cracks this and gets a url... -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: mod_cache performance

that does several dozen hits per day using squid :) On an OS that supports sendfile, a disk based cache will almost always bury a memory based one. -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: [PATCH] mod_cache fixes: #8

the top 10% be a good thing? Also, if the defaults are reasonable, what difference does it make? -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: [PATCH] event driven MPM

are interested in this. -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: mod_cache performance

Eli Marmor wrote: Graham Leggett wrote: Brian Akins wrote: On an OS that supports sendfile, a disk based cache will almost always bury a memory based one. Quite probably. But on a system without a disk, chances are it won't. :( It will. Unless mod_disk_cache + ram-disk

Re: mod_cache performance

.). -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: mod_cache performance

Bill Stoddard wrote: mod_mem_cache is broken then. It used to kick the pants off of 'no cache' and mod_disk_cache. If mod_disk_cache was patched to use sendfile, it will perform better. -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: mod_cache performance

. -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: mod_cache performance

Justin Erenkrantz wrote: That brings it in line with mod_disk_cache in maxing out my network. Time to craft some better tests or find a faster network... -- justin I can probably help with the latter :) Can you send me details of your setup and I'll try to test later this week. -- Brian Akins

Re: mod_cache performance

Justin Erenkrantz wrote: --On Tuesday, August 3, 2004 8:11 AM -0400 Brian Akins [EMAIL PROTECTED] wrote: Under load, squid will always use 100% of the CPU. This is because it uses poll/select. Ouch. That sucks. (But, httpd uses poll - so why does that force 100% CPU usage?) httpd blocks

mod_dir and mod_cache

I think I missed the answer to this: Has the feature that prevents mod_cache from caching urls ending in / (as related to mod_dir) been fixed? If so, will this make it into 2.0? -- Brian Akins Senior Systems Engineer CNN Internet Technologies

[PATCH] mod_disk_cache metadata

This speeds up writing and reading the metadata by using a struct rather that all the calls to ap_cache_usec2hex Note: I haven't actually tested this in mod_cache. Very similar code works very well elsewhere. -- Brian Akins Senior Systems Engineer CNN Internet Technologies --- httpd-2.0.50

[PATCH] mod_disk cached fixed

Sorry about this, but the last patch had a mistake in the writev -- Brian Akins Senior Systems Engineer CNN Internet Technologies --- httpd-2.0.50.old/modules/experimental/mod_disk_cache.c 2004-02-09 15:53:16.0 -0500 +++ httpd-2.0.50/modules/experimental/mod_disk_cache.c 2004-08-04

Re: [PATCH] mod_disk cached fixed

for the current way as well.You could do a quick check to make sure the numbers look resonable, I suppose. -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: [PATCH] mod_disk cached fixed

= ap_register_output_filter(CACHE_IN, cache_in_filter, NULL, AP_FTYPE_CONTENT_SET+1); Notice the plus in the second. -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: [PATCH] mod_disk cached fixed

].*Nav no-gzip and just vary on no-gzip (1 or 0), but this may be hard to do just using headers... -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: [PATCH] mod_disk cached fixed

. -- Brian Akins Senior Systems Engineer CNN Internet Technologies

apr_off_t on linux i386

apr_off_t gets defined as a long on linux i386 and an off_t on x86_64. apr_off_t used as a byte counter rolls over fairly quickly on i386. Suggestions? -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: apr_off_t on linux i386

Joe Orton wrote: On Wed, Aug 11, 2004 at 09:23:36AM -0400, Brian Akins wrote: apr_off_t gets defined as a long on linux i386 and an off_t on x86_64. apr_off_t used as a byte counter rolls over fairly quickly on i386. Suggestions? Stick with x86_64 :) Wish i could :) Or use HEAD where

Re: [PATCH] event driven MPM

Paul Querna wrote: My basic question for the list is, are we better off modifying the Worker MPM, or should we create a new 'event' MPM for now? My $.02 worht: Make an event MPM that is experimental. That way people can use tried and true worker, or try the fancy new event one. -- Brian

mod_deflate and no-gzip

set no-gzip. THis seems broken. If mod_deflate is called, if should send a Vary. -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: BUG? buffer logs and signals

Jeff Trawick wrote: if not appropriate to run the normal file cleanups at child exit, then mod_log_config could have a child exit hook to flush buffered log files Does child_exit exist in 2.0? -- Brian Akins Senior Systems Engineer CNN Internet Technologies

Re: BUG? buffer logs and signals

William A. Rowe, Jr. wrote: At 10:37 AM 8/23/2004, Brian Akins wrote: Jeff Trawick wrote: if not appropriate to run the normal file cleanups at child exit, then mod_log_config could have a child exit hook to flush buffered log files Does child_exit exist in 2.0? plog

Re: [feature request] defaultports directive for improved loadbalancing/NAT support

DECLINED; } registered like: ap_hook_translate_name(port, NULL, NULL, APR_HOOK_FIRST); -- Brian Akins Lead Systems Engineer CNN Internet Operations

Re: Time for 2.0.51 and 2.1.0

.. -- Brian Akins Lead Systems Engineer CNN Internet Operations

multiple host headers

Why do we merge multiple Host headers? I am getting wierd things like this for headers_in host: www.cnn.com, www.cnn.com This may be correct, but it caught me by surprise! -- Brian Akins Lead Systems Engineer CNN Internet Operations

EOS bucket data?

Can an EOS bucket ever have data a filter needs to read? -- Brian Akins Lead Systems Engineer CNN Internet Technologies

Re: EOS bucket data?

Cliff Woolley wrote: On Fri, 24 Sep 2004, Brian Akins wrote: Can an EOS bucket ever have data a filter needs to read? Nope. :) Thanks.. That's what the APR_BUCKET_IS_METADATA flag (or whatever we ended up calling it) is supposed to tell you. So read from any thing !APR_BUCKET_IS_METADATA

buffered logs + piped logs

If I enable buffered logs and use a log program (like rotate logs), it seems I loose whatever was in the buffer when I do a stop. Is the pipe being closed before the buffer is being flushed? -- Brian Akins Lead Systems Engineer CNN Internet Technologies

Re: buffered logs + piped logs

sockets. I didn't want to replicate someone else's work. I couldn't find anything on google. Any suggestions (doen's have to use unix sockets)? -- Brian Akins Lead Systems Engineer CNN Internet Technologies

Re: buffered logs + piped logs

Paul Querna wrote: mod_log_spread: http://www.backhand.org/mod_log_spread/ Unfortunantly, not an option here :( I'm looking for something to implement before the first Tuesday in November... -- Brian Akins Lead Systems Engineer CNN Internet Technologies

custom vhost module

: I need to be able to have port based virtual hosts that matches on multiple ports. Ie, ports 8080-8083 match one VirtualHost, while ports 9090-9093 match another, etc. This does not seem to be supported by current vhost chooser. -- Brian Akins Lead Systems Engineer CNN Internet Technologies

Re: custom vhost module

Glenn Strauss wrote: On Tue, Sep 28, 2004 at 09:29:15AM -0400, Brian Akins wrote: Have you tried giving a list of addresses and ports? VirtualHost 1.2.3.4:8080 1.2.3.4:8081 1.2.3.4:8082 1.2.3.4:8083 ... /VirtualHost I need to be able to use *:8080 *:8081, but that's not allowed... -- Brian Akins

OT. What I found on a large site today

2004 14:09:30 GMT Transfer-Encoding: chunked Content-Type: text/html Also, http://uptime.netcraft.com/up/graph/?host=www.cnn.com Not an endorsement, just an observation. -- Brian Akins Lead Systems Engineer CNN Internet Technologies

Re: svn commit: r440337 - in /httpd/httpd/trunk: ./ include/ modules/arch/netware/ modules/experimental/ modules/generators/ modules/http/ modules/mappers/ modules/proxy/ modules/ssl/ server/ server/m

, it gets a symbol rather than disappearing after compile. -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies

Re: mod_cache responsibilities vs mod_xxx_cache provider responsibilities

. -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies

Re: mod_cache responsibilities vs mod_xxx_cache provider responsibilities

Niklas Edmundsson wrote: If I remember correctly the code in 2.2.3 only does whole-file revalidation, No, it can have a stale handle that it makes fresh if it gets a 304. -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies

Re: mod_cache responsibilities vs mod_xxx_cache provider responsibilities

rsyncing the mirrored content. I don't know your environment, but was just what I cam up with off the top of my head. -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies

Re: mod_cache responsibilities vs mod_xxx_cache provider responsibilities

. Very good ideas that just need a little refinement. -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies

Re: mod_cache responsibilities vs mod_xxx_cache provider responsibilities

Issac Goldstand wrote: I can see how other tracking information (like how often the cached entity is accessed, last access time, etc) would be useful, Also, those statistics could be updated asynchronously by using a queue so that statistics doesn't slow down a busy web server. -- Brian

Re: mod_slotmem

Oden Eriksson wrote: onsdag 30 augusti 2006 10:37 skrev Brian Akins: With all the talk of a generic scoreboard, here's something I whipped up that allows any other module to have some amount of memory per worker slot. We have a different module in-house at CNN which does something similar

Re: [patch 09/16] simplify array and table serialization

Davi Arnaut wrote: Simplify the array and table serialization code, separating it from the underlying I/O operations. Probably faster to just put every thing in an iovec (think writev). -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies

Re: mod_cache responsibilities vs mod_xxx_cache provider responsibilities

because we have a really quick way to find the size and remove the oldest expired objects first. Every cache store gets recorded in SQLite with info about the object (size, mtime, expire time, url, key, etc.). Makes it trivial tow write cron jobs to do cache management. -- Brian Akins Chief

Re: [patch 09/16] simplify array and table serialization

Davi Arnaut wrote: On 20/09/2006, at 10:16, Brian Akins wrote: Davi Arnaut wrote: Simplify the array and table serialization code, separating it from the underlying I/O operations. Probably faster to just put every thing in an iovec (think writev). Probably no, apr_brigade_writev

Re: mod_cache responsibilities vs mod_xxx_cache provider responsibilities

structures. -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies

  1   2   3   4   5   >