Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
Chris Darroch wrote: I've been working with the 2.4 authn/z stuff a bit lately and what I keep tripping over is that the default authorization merge rule uses OR logic. For example, if I enable mod_access_compat and put in a traditional: I wonder if anyone would offer a fastfeather talk next week on wed or thurs - it's only 15 minutes - to introduce what's upcoming in 2.4? Bill
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
William A. Rowe, Jr. wrote: I've been working with the 2.4 authn/z stuff a bit lately and what I keep tripping over is that the default authorization merge rule uses OR logic. For example, if I enable mod_access_compat and put in a traditional: I wonder if anyone would offer a fastfeather talk next week on wed or thurs - it's only 15 minutes - to introduce what's upcoming in 2.4? I won't be there, but here's a recap of the issue for discussion. (Caveat: I may be missing something important!) With 2.2 and prior versions, one can do something like: Directory /htdocs Require valid-user /Directory Directory /htdocs/admin Require user admin /Directory The logic which is then applied is: 1) For all requests under /htdocs, except those under /htdocs/admin, require any valid user. 2) For all requests under /htdocs/admin, require the admin user. With 2.4, unless I'm missing something, the same configuration produces the logic: 1) For all requests under /htdocs, except those under /htdocs/admin, require any valid user. 2) For all requests under /htdocs/admin, require any valid user OR require the user admin. Of course this grants any valid user access. To get the old behaviour, you seem to need to add AuthzMergeRules Off to the second Directory. I just tested versions of this configuration with 2.2 and 2.4 and I think I'm describing the situation correctly. Assuming I am, I fear this will surprise a lot of people who think they've secured their systems after upgrading. It certainly caught me short. Perhaps the default AuthzMergeRules setting should be Off rather than On, at least when merging across configuration blocks? Chris. -- GPG Key ID: 366A375B GPG Key Fingerprint: 485E 5041 17E1 E2BB C263 E4DE C8E3 FA36 366A 375B
Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?]
... if we had a config finalize, modules who were prepared to declare their config (e.g. mod_vhost declaring the per-host directory merges completed) then as-root, we can finish these out, opening logs with full privileges. Other merges will happen at run time (or be optimized when we can accomplish this) per-request. So does a setup like this make it possible for the processes/thread handling the request to change to the correct UID/GID before reading/writing files? Just something that popped into my head when reading this. -- ~Jorge
Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?]
Jorge Schrauwen wrote: ... if we had a config finalize, modules who were prepared to declare their config (e.g. mod_vhost declaring the per-host directory merges completed) then as-root, we can finish these out, opening logs with full privileges. Other merges will happen at run time (or be optimized when we can accomplish this) per-request. So does a setup like this make it possible for the processes/thread handling the request to change to the correct UID/GID before reading/writing files? Just something that popped into my head when reading this. No. Once the httpd engine finishes the config phase altogether, we continue to drop from root to the desired UID/GID and that process must not have the privilege to change these again. The request engine ... which is the container where exploits are targeted, must remain secure.
2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
On 4/3/2008 at 8:06 AM, in message [EMAIL PROTECTED], Jim Jagielski [EMAIL PROTECTED] wrote: Another good topic of discussion: Time for a 2.4 release? I wouldn't mind pushing that along and get some of the feature-set of 2.4 out before we do too much ripping with the inevitable delays associated with that :) Please let's get 2.4 out. It would be great to finally have the new Authz configuration logic see the light of day along with other functionality that has been sitting around for a while. Brad
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
-Ursprüngliche Nachricht- Von: Jim Jagielski Gesendet: Donnerstag, 3. April 2008 16:07 An: dev@httpd.apache.org Betreff: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?]) Another good topic of discussion: Time for a 2.4 release? I wouldn't mind pushing that along and get some of the feature-set of 2.4 out before we do too much ripping with the inevitable delays associated with that :) I know that I am always devils advocate and a brakeman regarding this, but we should keep in mind the following: 1. After the rewrite of the authz code to a provider based model we still fail the basic authz tests in the perl framework. This is a clear showstopper and needs to be fixed first. Yes, I also should have a had a more closer look on what Brad (no blame game intended against anyone as I failed to do proper review back then) did there in the past to highlight issues earlier, but my gut feeling tells me that there are still some surprises in this code regarding bugs and the configuration syntax. 2. My feeling regarding the usage of 2.2 is that since about 6 month we are getting track as commercial 3rd parties now supply modules for httpd 2.2. This means that will have to maintain one more stable branch for quite some time and to be honest currently we effectively manage only one since 1.3 and also 2.0 seem to be pretty abandoned. I am not quite sure if we have enough resources to do this. Regards Rüdiger
2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
Another good topic of discussion: Time for a 2.4 release? I wouldn't mind pushing that along and get some of the feature-set of 2.4 out before we do too much ripping with the inevitable delays associated with that :)
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
On Thu, Apr 03, 2008 at 10:06:50AM -0400, Jim Jagielski wrote: Time for a 2.4 release? I wouldn't mind pushing that along and get some of the feature-set of 2.4 out before we do too much ripping with the inevitable delays associated with that :) Is there really enough news in trunk to warrant the overhead of maintaining yet another branch? tbh. I'd much rather see work going towards 3.x ;) vh Mads Toftum -- http://soulfood.dk
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
Plüm wrote:
2. My feeling regarding the usage of 2.2 is that since about 6 month we are
getting
track as commercial 3rd parties now supply modules for httpd 2.2. This means
that
will have to maintain one more stable branch for quite some time and to be
honest
currently we effectively manage only one since 1.3 and also 2.0 seem to be
pretty
abandoned. I am not quite sure if we have enough resources to do this.
Perhaps 1.3 is maintained, perhaps not, just depending on if there
are enough developers who care to provide minor patches. But it
really shouldn't be a distraction.
Once folks accept 2.2 as a clean replacement for 2.0 (we are essentially
there now, I think, perhaps after one or two more point-bumps), 2.0 should
fall off the radar entirely.
This 2.2 becomes stable, so we are really only talking about supporting
a 2.2 and a 2.4 for a while, then 2.4 and a 3.0.
I'd -1 a 2.4.0 release today, because nobody has even bothered to make
a candidate for 2.3-dev. Auth logic changes break most if not all third
party auth modules (broke an auth feature in mod_ftp). Not talking about
commercial modules but every third party auth extension out there.
The only thing I'd like to change before we push 2.3-dev onto our module
developer community is to enhance the methods logic (64 is now insufficient
when you add DAV + FTP + {whatever} into the method stew) and drop the
Limit directive already for the Method directive. These have some
subtle auth-changes that a few auth module authors need to compensate for.
(The third party auth modules that won't change are the ones that are
already broken for Limit ... who won't have to do anything special in
support of the Method directive.)
I'll commit the Method changes before the hackathon so that folks can
corner me in person, dev@ list feedback will also be welcome as always.
Just to prove I'm not being a prick with respect to Lua support, I'm working
on Method as an add in module ;-)
Bill
Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?]
On 4/2/08 5:50 PM, William A. Rowe, Jr. [EMAIL PROTECTED] wrote: ixnay on the run-time intensive, slow down the server sorts of changes. httpd continues to become slower as it becomes more powerful. I know you are the first one to raise your hand and point out when we are doing too much processing for too simple a request. Isn't this what modules are for? Perhaps you could elaborate? Yes, there is always a fine line between configuration and performance, I suppose. Basically, at the heart of what I'm imagining is everything is per_dir (no real per-server configs) and the fancy configuration stuff really boils down to per-dir config merges. The parser or language needs to be small and quick and do most of it's heavy lifting in post-config (parse and cache the tree, don't do the full string parsing at run time). Sorta how mod_rewrite works now... The If/Else stuff would probably be able to do this.. -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?]
On 4/2/08 5:56 PM, William A. Rowe, Jr. [EMAIL PROTECTED] wrote:
I'm pondering this... if we drop per-server ... yet retain the ability
for authors to factor their config info into related config sections...
Yes... Bcs what IO am imagining is something like what I've posted before:
If HTTP_HEADER{'Host'} == www.cnn.com and Port == 8080
DocumentRoot /www/cnn
ServerAdmin [EMAIL PROTECTED]
etc
If Location == /my/prefix
Foo bar
/If
/If
Maybe no need for Directory, location, etc, either...
--
Brian Akins
Chief Operations Engineer
Turner Digital Media Technologies
Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?]
On 4/2/08 6:07 PM, William A. Rowe, Jr. [EMAIL PROTECTED] wrote: we can finish these out, opening logs with full privileges. Other merges will happen at run time (or be optimized when we can accomplish this) per-request. We already fake per-dir logs with the env stuff in mod_log_config. -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
On 4/3/08 10:47 AM, William A. Rowe, Jr. [EMAIL PROTECTED] wrote: I'll commit the Method If HTTP_Method == GET ... /If ;) -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?]
Akins, Brian wrote:
On 4/2/08 5:56 PM, William A. Rowe, Jr. [EMAIL PROTECTED] wrote:
I'm pondering this... if we drop per-server ... yet retain the ability
for authors to factor their config info into related config sections...
Yes... Bcs what IO am imagining is something like what I've posted before:
If HTTP_HEADER{'Host'} == www.cnn.com and Port == 8080
DocumentRoot /www/cnn
ServerAdmin [EMAIL PROTECTED]
etc
If Location == /my/prefix
Foo bar
/If
/If
Maybe no need for Directory, location, etc, either...
horribly inefficient, on a 10-fold order of magnitude.
But that *doesn't* mean I don't want it... simply not to replace directory,
file, location or method. Keep in mind you wouldn't have your ErrorLog
opened at startup time, as this is too variant (unless the ErrorLog code
became really very clever - although dynamic patterns in the ErrorLog file
name could also make that impossible) ... but where you are very cautious
about your log file directive, and it's not a mass vhosting server, this
really wouldn't prove to be an issue.
Bill
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
Akins, Brian wrote: On 4/3/08 10:47 AM, William A. Rowe, Jr. [EMAIL PROTECTED] wrote: I'll commit the Method If HTTP_Method == GET ... /If Slow
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
On 4/3/08 11:38 AM, William A. Rowe, Jr. [EMAIL PROTECTED] wrote:
If HTTP_Method == GET
...
/If
Slow
Not if the parsing is done at config time and HTTP_Method is handle by a
provider. Some pseudo code:
At config time, the parser would do something like:
parse_provider *prov;
void *data;
prov = ap_lookup_provider(config_parse, HTTP_Method, 0.1);
data = prov-init(conf_pool, HTTP_Method, TOKEN_EQUAL, GET)
/*the provider may do something like*/
typedef struct {
parse_token token;
int method_number;
} method_data;
void *method_init(apr_pool_t *pool, const char *key, parse_token token,
const char *arg) {
method_data *data = apr_pcalloc(pool, sizeof(method_data));
data-token = token; /*need to check if we only handle === or something
*/
if(strcacecmp(arg, GET)) {
data-method_number = M_GET;
}
return data;
/*the parser stores this data with the node*/
At run time, then when running this node from the cached parse tree, it may
call something like:
node-prov-exec(r, nod-data)
/*the provider runs something like*/
int method_exec(reuqest_rec *r, method_data *data) {
if(data-method_number == r-method_number)
return 1;
return 0;
}
Very rough draft. But this is not necessarily slow... ;)
--
Brian Akins
Chief Operations Engineer
Turner Digital Media Technologies
Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?]
On Thu, 03 Apr 2008 11:22:00 -0400
Akins, Brian [EMAIL PROTECTED] wrote:
If HTTP_HEADER{'Host'} == www.cnn.com and Port == 8080
DocumentRoot /www/cnn
ServerAdmin [EMAIL PROTECTED]
etc
That basically comes out of what I committed this morning.
Well, up to a point: it only applies to the per-dir config.
Maybe no need for Directory, location, etc, either...
I'm thinking more of displacing tortuous mod_rewrite-based
configs than any of the old containers (except possibly
the much-misunderstood-and-abused Limit).
--
Nick Kew
Application Development with Apache - the Apache Modules Book
http://www.apachetutor.org/
Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?]
On 4/3/08 11:38 AM, William A. Rowe, Jr. [EMAIL PROTECTED] wrote: But that *doesn't* mean I don't want it... simply not to replace directory, file, location or method. Keep in mind you wouldn't have your ErrorLog opened at startup time, as this is too variant Unless I'm mistaken, there is nothing that really stops us from making all log related directives from being per-dir (assuming they have real names at startup). Using mod_log_config as an example, it opens all of it's logs as root at start, and we are able to select one at run time via env. That could just as easily be done via a per-dir merge. (Note: I am ignorant of the stuff between the errolog directive until it gets to the actually logging stuff, so I may be way off. ) -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
On Thu, Apr 3, 2008 at 8:53 AM, Akins, Brian [EMAIL PROTECTED] wrote: Very rough draft. But this is not necessarily slow... ;) Right. Even then, the user/admin may be willing to burn CPU cycles anyway to get a simpler config. Plus, if they were to use mod_rewrite, they've already blown a huge chunk of CPU cycles! =) -- justin
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
On Thu, Apr 3, 2008 at 4:31 PM, Mads Toftum [EMAIL PROTECTED] wrote: On Thu, Apr 03, 2008 at 10:06:50AM -0400, Jim Jagielski wrote: Time for a 2.4 release? I wouldn't mind pushing that along and get some of the feature-set of 2.4 out before we do too much ripping with the inevitable delays associated with that :) Is there really enough news in trunk to warrant the overhead of maintaining yet another branch? tbh. I'd much rather see work going towards 3.x ;) vh Mads Toftum -- http://soulfood.dk I'm wondering the same. -- ~Jorge
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
Nick Kew wrote: Limit is of course a crusty old relative. Limit is unrelated, it's fundamentally borked (directive must know it is participating in a limit-ed section, cannot overly multiple limit-ed sections because that directive has never created a conf section, and there is no exception thrown when a non-participating directive is nested within a limit). This is why I'm not 'fixing' Limit ... there are simply too many broken configs out there. Bill
Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?]
Akins, Brian wrote: On 4/3/08 11:38 AM, William A. Rowe, Jr. [EMAIL PROTECTED] wrote: But that *doesn't* mean I don't want it... simply not to replace directory, file, location or method. Keep in mind you wouldn't have your ErrorLog opened at startup time, as this is too variant Unless I'm mistaken, there is nothing that really stops us from making all log related directives from being per-dir (assuming they have real names at startup). Using mod_log_config as an example, it opens all of it's logs as root at start, and we are able to select one at run time via env. That could just as easily be done via a per-dir merge. (Note: I am ignorant of the stuff between the errolog directive until it gets to the actually logging stuff, so I may be way off. ) You are right; provided that we pre-merge SOME of the layers of dir configs (e.g. we pre-merge all of the vhost-related dir configs). And remember a merge of A+null or null+A is fast. So if we break up a very very big dir config structure into three, e.g. those directives that are very frequently tweaked, those which are sometimes tweaked and those we don't expect to change at all (at least, not after server-merges at startup) the overall hit of dir_merge (now three different functions for this hypothetical modules) will be much less.
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
Justin Erenkrantz wrote: On Thu, Apr 3, 2008 at 8:53 AM, Akins, Brian [EMAIL PROTECTED] wrote: Very rough draft. But this is not necessarily slow... ;) Right. Even then, the user/admin may be willing to burn CPU cycles anyway to get a simpler config. Plus, if they were to use mod_rewrite, they've already blown a huge chunk of CPU cycles! =) -- justin Yes! I'm not against offering slow features :) I'm only antagonistic towards replacing the fast ones, from today. FYI - Files and Directory should be entirely moved out of core into our default filesystem provider module. Only host/location/method should be part of the core (well, host perhaps in the http layer). The If logic doesn't even apply when that module isn't loaded, I'd hope. Those admins who refuse to let their junior admins use that directive should have a level of control over their outward facing heavily-loaded machines :)
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
On Thu, 03 Apr 2008 11:13:31 -0500 William A. Rowe, Jr. [EMAIL PROTECTED] wrote: The If logic doesn't even apply when that module isn't loaded, I'd hope. Those admins who refuse to let their junior admins use that directive should have a level of control over their outward facing heavily-loaded machines :) The If logic is approximately cloned from Files, and just differs in what it evaluates. If we're talking about any substantial config changes, then the whole location_walk and merges should be on the table. But before that, we need a vision of where we're going, and how to get there without breaking what we've got. I don't see any such vision in this discussion. -- Nick Kew Application Development with Apache - the Apache Modules Book http://www.apachetutor.org/
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
On 4/3/2008 at 8:23 AM, in message [EMAIL PROTECTED], Plüm, Rüdiger, VF-Group [EMAIL PROTECTED] wrote: -Ursprüngliche Nachricht- Von: Jim Jagielski Gesendet: Donnerstag, 3. April 2008 16:07 An: dev@httpd.apache.org Betreff: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?]) Another good topic of discussion: Time for a 2.4 release? I wouldn't mind pushing that along and get some of the feature-set of 2.4 out before we do too much ripping with the inevitable delays associated with that :) I know that I am always devils advocate and a brakeman regarding this, but we should keep in mind the following: 1. After the rewrite of the authz code to a provider based model we still fail the basic authz tests in the perl framework. This is a clear showstopper and needs to be fixed first. Yes, I also should have a had a more closer look on what Brad (no blame game intended against anyone as I failed to do proper review back then) did there in the past to highlight issues earlier, but my gut feeling tells me that there are still some surprises in this code regarding bugs and the configuration syntax. It wouldn't surprise me, which is why we need to get a 2.3-beta out there for testing. I've tried to make sure that the holes where filled with the authz refactor, but it is likely that something will be missing until more wide spread testing is done. The perl framework problems were discussed on the list a couple of months ago http://mail-archives.apache.org/mod_mbox/httpd-dev/200801.mbox/[EMAIL PROTECTED] The current tests don't apply anymore because authz has moved from hook based to provider based with logic operators added. If we need to rework something outside of the tests themselves, then that needs to be identified. As far as breaking existing authz modules, it is a similar situation when authn moved from hooks to providers in 2.2. All of the standard Apache authz modules have already been refactored. This issue is third parties will have to refactor their authz modules for 2.4. Brad
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
Nick Kew wrote: But before that, we need a vision of where we're going, and how to get there without breaking what we've got. * server_conf goes away. Modules have zero or more conf sections, essentially today's misnamed dir_conf, which are initialized and merged as they are today. Either using simple-wrappers or custom. * we lose nothing, any module can rerun the conf merge conditionally (see mod_proxy today, or method, or the new if implementation). * modules may pre-merge things to optimize them. E.g. mod_vhost will pre-merge all of our VirtualHosts such that there is no run time penalty for vhosts. * particular merges should remain, as they are today, as a modular assemblage of features. If your server will never serve files, there is no reason to compile in mod_filesystem e.g. Files and Dir mergers. No mod_vhost? Then no VirtualHost support. *** there must be sufficient hooks so that a lua, perl or m4 wrapper can participate in parsing the config. TBD if that exists today. I don't see any such vision in this discussion. Go back to my post from last night. Not saying there's agreement, just some vision. I know folks are thinking wow, I want to put my favorite [Lua] language parser into httpd core! Well that's fine for some users, but at the very same time, you have a huge crowd of perl users, mod_macro users, etc. Based on httpd's design, you are SUPPOSED to be able to plug whichever wrapper you like into the core. Overloading the core is taking httpd into an entirely different direction. Just look at the effort we've already expended is splitting cache from proxy, auth components from one another, proxy capabilities from one monolithic module. For that matter, plug in an XML syntax parser. We aren't quite there because we don't have the concept of named-arguments. Fix that across the entire spectrum of modules. E.g. instead of TAKE2, we have some tuple for alias such as (TAKE2, uri-path, file-path, NULL) which will facilitate more deliberate parsers and conf authoring. And that is without breaking today's syntax. Bill
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
William A. Rowe, Jr. wrote: I'd -1 a 2.4.0 release today, because nobody has even bothered to make a candidate for 2.3-dev. Auth logic changes break most if not all third party auth modules (broke an auth feature in mod_ftp). Not talking about commercial modules but every third party auth extension out there. I've been working with the 2.4 authn/z stuff a bit lately and what I keep tripping over is that the default authorization merge rule uses OR logic. For example, if I enable mod_access_compat and put in a traditional: Location /foo Order deny,allow Deny from all /Location it doesn't take effect, because the default top-level Directory contains Require all granted and since that succeeds for all requests, everything else is short-circuited by the OR merge logic. So at a minimum I seem to have to put in an AuthzMergeRules Off to get things to DWIM. I fear that might trip up a significant percentage of people upgrading ... perhaps a AuthzMergeRules Off in the default httpd.conf would be sufficient, but my experience with mod_authz_dbd suggested that I needed to put it in a lot of places to get stuff working the way I intended (e.g., the example config in the mod_authz_dbd manual page in the trunk docs). Chris. -- GPG Key ID: 366A375B GPG Key Fingerprint: 485E 5041 17E1 E2BB C263 E4DE C8E3 FA36 366A 375B
Re: 2.4 (Was: Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?])
On Apr 3, 2008, at 12:32 PM, Brad Nicholes wrote: It wouldn't surprise me, which is why we need to get a 2.3-beta out there for testing. That would be good as well... that way we can determine how solid the existing impl is, so when the new stuff is added we know the old stuff is still good :)
Re: Dynamic configuration for the hackathon?
On Mar 31, 2008, at 13:31, Paul Querna wrote:
Just look at SSLRequire, Rewrite*, MPM Process/Thread Management,
Filter chaining, large Auth{N,Z} chains, and more.
Imagine them not sucking.
That would be lovely. Truly.
--
Happiness isn't something you experience; it's something you remember.
Oscar Levant
Re: Dynamic configuration for the hackathon?
On Mar 31, 2008, at 13:46, Issac Goldstand wrote: Make mod_wombat a standard module and part of the default moduleset May I request, if mod_wombat becomes a standard module, that it be given a name not quite so calculated to make the newbie disable it without a second glance. I mean, wombats are cute and all, and it's a delightful name, but nobody knows what it means. -- Whatever you do will be insignificant, but it is very important that you do it. Mahatma Ghandi
Re: Dynamic configuration for the hackathon?
On 4/2/08 8:44 AM, Rich Bowen [EMAIL PROTECTED] wrote: May I request, if mod_wombat becomes a standard module, that it be given a name not quite so calculated to make the newbie disable it without a second glance. I think the reason wombat was chosen is because mod_lua was taken. In reality, I suppose there should be a base lua module, then a configuration module based upon it and a hook handler module based upon it as well. -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: Dynamic configuration for the hackathon?
There's a couple of conflicting demands by our users, and spending time on the users mailing list and on the IRC channel is a great way to see this first-hand. They don't want to learn a new syntax. And they want a new syntax that lets them do what they mean. And they're very frustrated when they can't do things that are obviously what everyone wants, and have to resort to mod_rewrite. Now, I'm very fond of mod_rewrite, because, after all, I wrote a book, and royalties from that book buy me my toys. But the time spent answering the same questions again and again on IRC lead me to believe that there are certain features that would gain us a lot of appreciation from our users - the folks who, at least to me, make this stuff worth doing. Easy for me to say, since I just document what the rest of you guys actually do. I recognize that it looks easy from the cheap seats. Virtual hosts are just one example of something that's amazingly hard (for beginners) to get right, and very easy to get wrong. And when it's wrong, it's unpredictably wrong, in ways that are hard to anticipate. On Mar 26, 2008, at 09:06, Nick Kew wrote: There seems to be a demand for dynamic per-request configuration, as evidenced by the number of users hacking it with mod_rewrite, and the other very limited tools available. Modern mod_rewrite usage commonly looks like programming, but it's not designed as a programming language. Result: confused and frustrated users. ... if [some-per-request-expr] Directives applicable per-request (anything, subject to per-directive context checking) /if ++1. This would solve an enormous number of the user problems that we experience on the help channels, which, I have to assume, are but the tip of the iceberg of problems experienced out there in the real world. It's amazing, sometimes, what absurd lengths folks have to go to, to solve the problems that they've created. A noble objective: render mod_rewrite obsolete :-) ++1 -- What the world needs is more geniuses with humility, there are so few of us left. Oscar Levant
Re: Dynamic configuration for the hackathon?
Issac Goldstand wrote: We're not talking about fresh users, we're talking about existing users. Fresh users have to deal with one learning curve or another anyway. I'm not talking about fresh users either. Matt
Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?]
Akins, Brian wrote: The biggest problems I have with current system are: -Every module does things differently Within limits this will remain true. But we are missing a host of very trivial simplifications for the casual module developer, and reinvent the same wheel module after module. I'm going to propose some changes to the default TAKE[0-9]+ macro and support functions to illustrate how much more useful these can be, and offer some new functions that make it really simple to * define constraints (an int, sure, but in what range?) * tag default-value, and error out if a non-default value is replaced by the user (e.g. LogLevel debug LogLevel error later on) -No real per-request configuration. Some modules use env to do some of this. ixnay on the run-time intensive, slow down the server sorts of changes. httpd continues to become slower as it becomes more powerful. I know you are the first one to raise your hand and point out when we are doing too much processing for too simple a request. Isn't this what modules are for? Perhaps you could elaborate? -I have to write a good bit of code before a module is configurable. (I'm lazy. Very lazy.) Agreed - see my first point. One interesting point; why do we keep per-server and per-dir sections? Perhaps it's time for a single simpler-to-use mechanic which can represent either or both (essentially deprecate per-server in 3.0). If we do what niq suggests (which, if we stick with current config system is fine) is that it just adds another layer on top of all the existing issues. With lua, for example you could make modules Lua modules... Maybe could do same in perl?? That's why I'm not a fan of all lua all the time. But to do what we all want, I bet we'll need to refine the config API, and simplify it for adding pluggable semantic engines (lua, perl, simply 'sed' etc). My opinion (which is worthless, I know) is to pick one way and do it. Lua is easy to learn and satisfies most (all?) of our requirements. And if Brian M. and I can learn it, anyone can ;) Most importantly, if I am building an embedded httpd, I don't want all of this extra crap. httpd was famous for how lean it could be, are we all ready to throw out that advantage?
Re: Dynamic configuration for the hackathon?
Graham Leggett wrote: Jim Jagielski wrote: This reminds me: a serf BOF or session would, I think, go over quite well :) A question that has been on my mind for a bit, is what does serf intend to replace, and why is it better?. The impression I have so far is that somehow what we have now is suboptimal, and that serf somehow does it better, but exactly what is suboptimal, and exactly how serf does it better has yet to be clarified. Someone please consider offering a title/abstract/speaker/bio on a fast paced introduction to the advantages of serf as a 15 minute FFT preso, so that we can all understand the answers to Graham's questions. [EMAIL PROTECTED] - for either wed or thurs afternoon.
Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?]
William A. Rowe, Jr. wrote: -I have to write a good bit of code before a module is configurable. (I'm lazy. Very lazy.) Agreed - see my first point. One interesting point; why do we keep per-server and per-dir sections? Perhaps it's time for a single simpler-to-use mechanic which can represent either or both (essentially deprecate per-server in 3.0). I'm pondering this... if we drop per-server ... yet retain the ability for authors to factor their config info into related config sections... ... we could divide the frequently-merged and infrequently-merged options into two or more groups; the overall merge calls would run more quickly, but the flexibility would be greatly enhanced. Override-able -document_root, anyone? Bill
Re: Configuration Issues to Address [was Re: Dynamic configuration for the hackathon?]
William A. Rowe, Jr. wrote: William A. Rowe, Jr. wrote: -I have to write a good bit of code before a module is configurable. (I'm lazy. Very lazy.) Agreed - see my first point. One interesting point; why do we keep per-server and per-dir sections? Perhaps it's time for a single simpler-to-use mechanic which can represent either or both (essentially deprecate per-server in 3.0). I'm pondering this... if we drop per-server ... yet retain the ability for authors to factor their config info into related config sections... ... we could divide the frequently-merged and infrequently-merged options into two or more groups; the overall merge calls would run more quickly, but the flexibility would be greatly enhanced. Override-able -document_root, anyone? And taking this one step further - addressing issues such as the fact that we all want certain merged-sections (e.g. vhosts) to be run as root early in the process, but be flexible enough to override a bit later on... ... if we had a config finalize, modules who were prepared to declare their config (e.g. mod_vhost declaring the per-host directory merges completed) then as-root, we can finish these out, opening logs with full privileges. Other merges will happen at run time (or be optimized when we can accomplish this) per-request. It's not looking trivial, but it seems doable.
Re: Dynamic configuration for the hackathon?
On Mon, Mar 31, 2008 at 11:15 AM, Roy T. Fielding [EMAIL PROTECTED] wrote: Unfortunately, after last year's experience of being the only server person around who wasn't working on a Joost release, *hides* I decided to delay my arrival until Tuesday rather than attend the hackathon. Please have fun, finish the server, and let me know what happened when I arrive; For other reasons, I too won't arrive until Tuesday morning. I look forward to seeing what happens on Monday. =P -- justin
Re: Dynamic configuration for the hackathon?
On Mon, Mar 31, 2008 at 7:41 PM, William A. Rowe, Jr. [EMAIL PROTECTED] wrote: I sympathize, but this doesn't reflect the addition of lua blocks... those blocks can be trivially implemented as a loadable module ;-) As I grok it, the point would be throw out our ridiculous config syntax entirely (or at best write a compatibility module or a converter to the new format) and expose a real config API (hello providers! *chuckle*) and then build a pure LUA-based config format on top of that. I'd be definitely curious to see what would come of that - 'cuz really it can't be much worse than the garbage we're stuck with now. -- justin
Re: Dynamic configuration for the hackathon?
On Mon, 31 Mar 2008 11:15:38 -0700 Roy T. Fielding [EMAIL PROTECTED] wrote: On Mar 31, 2008, at 10:39 AM, Paul Querna wrote: Just read the mod_rewrite docs: http://httpd.apache.org/docs/2.2/rewrite/rewrite_tech.html#InternalAPI They are already exposing internals to users'. Not, erm, yer average user ... Embed Lua. That is orthogonal to the question I'm addressing. You're going for more powerful; I'm going for more usable! Unfortunately, after last year's experience of being the only server person around who wasn't working on a Joost release, ... or tied up in training events ... I decided to delay my arrival until Tuesday rather than attend the hackathon. Heh. For the first time, I have the hackathon and no conflicting committment! Please have fun, finish the server, and let me know what happened when I arrive; I'll try to squeeze it into my keynote for Friday. ;-) Please sir, we broke it! -- Nick Kew Application Development with Apache - the Apache Modules Book http://www.apachetutor.org/
Re: Dynamic configuration for the hackathon?
William A. Rowe, Jr. wrote:
-0.5 PLEASE not in the core. Make mod_wombat a standard module and
part of the default moduleset, whatever, but let's not make more core
dependencies, please?!?
-0.99 - agreed. Perl is perfectly happy having perl blocks as modular
behaviors... I've noticed a trend in the last few years of building on the
core (and folks rightfully accused me of growing mod_proxy core when new
directives are rightfully part of mod_proxy_{whatever}).
Let's focus on keeping it in useful pieces, even if they are built by
default.
-0.99 - agree with wrowe.
I looked at the option of supporting different configuration providers a
while back, I was researching the idea that configuration could be
stored in LDAP instead of flat files, and it seemed pretty
straightforward (didn't have time to explore further).
What I did find is that it is quite possible to support pluggable
configuration options, it isn't difficult: your configuration module
must somehow render config lines which are pumped into the core. How
those lines come into existence, whether sourced from flat file, rows in
a database, or rendered by a programming language, is up to you.
One further thing PLEASE not the default. I don't have the time in my
day to learn a whole programming language just to configure my webserver.
The problem that needs solving is to find an acceptable tradeoff between
allowing the mod_rewrite fans to do complex stuff, while at the same
time letting us mere mortals do the simple stuff as well without any
drama. Please don't lose sight of that.
Regards,
Graham
--
smime.p7s
Description: S/MIME Cryptographic Signature
Re: Dynamic configuration for the hackathon?
On Tue 01 Apr 2008, Paul Querna wrote:
William A. Rowe, Jr. wrote:
-0.99 - agreed. Perl is perfectly happy having perl blocks as modular
behaviors... I've noticed a trend in the last few years of building on
the core (and folks rightfully accused me of growing mod_proxy core when
new directives are rightfully part of mod_proxy_{whatever}).
Yes, but the root of the problem even with perl blocks is that they
have almost no way to change the behavior of existing modules -- like
you can via configuration -- and instead for the most part, they
reimplment entire C modules in Perl, or any other language, rather than
binding to the configuration, and change that based on some other inputs.
I disagree. In the mod_perl API you can almost entirely configure a single
request. You can hook maptostorage and add there directives for other modules
via $r-add_config. Anything that can be configured in .htaccess or
Location can be done that way as well. One can even change DocumentRoot
(prefork-only) for a single request. The PerlMapToStorageHandler can then
decide whether to skip the standard maptostorage (directory walk and file
walk) or not. It would be good for such a perl-configured request to be able
to skip the location-walk that follows maptostorage. But if the admin wants
to do that he can simply avoid location blocks.
You cannot add virtual servers on the fly or redirect error_log. But I don't
expect that to become feasible in the new config language since those are
things that are done at server startup.
Then the existing configuration file, a new lua system, or anything
else, could be written in terms of that, rather than the current system
where each language reinvents the modules it wants to control.
I agree that a configuration language like lua is good but it doesn't
necessarily have to be in core. It can be a module as mod_perl shows.
Torsten
Re: Dynamic configuration for the hackathon?
Torsten Foertsch wrote:
On Tue 01 Apr 2008, Paul Querna wrote:
William A. Rowe, Jr. wrote:
-0.99 - agreed. Perl is perfectly happy having perl blocks as modular
behaviors... I've noticed a trend in the last few years of building on
the core (and folks rightfully accused me of growing mod_proxy core when
new directives are rightfully part of mod_proxy_{whatever}).
Yes, but the root of the problem even with perl blocks is that they
have almost no way to change the behavior of existing modules -- like
you can via configuration -- and instead for the most part, they
reimplment entire C modules in Perl, or any other language, rather than
binding to the configuration, and change that based on some other inputs.
I disagree. In the mod_perl API you can almost entirely configure a single
request. You can hook maptostorage and add there directives for other modules
via $r-add_config. Anything that can be configured in .htaccess or
Location can be done that way as well. One can even change DocumentRoot
(prefork-only) for a single request. The PerlMapToStorageHandler can then
decide whether to skip the standard maptostorage (directory walk and file
walk) or not. It would be good for such a perl-configured request to be able
to skip the location-walk that follows maptostorage. But if the admin wants
to do that he can simply avoid location blocks.
You cannot add virtual servers on the fly or redirect error_log. But I don't
expect that to become feasible in the new config language since those are
things that are done at server startup.
I don't think we're even talking about on-the-fly stuff for the Lua
parser engine, so Perl can do everything that Lua can. As people have
mentioned, though, Perl is bloated (but +1 for mp2.1 to include plugs
for this new config API :))
Then the existing configuration file, a new lua system, or anything
else, could be written in terms of that, rather than the current system
where each language reinvents the modules it wants to control.
I agree that a configuration language like lua is good but it doesn't
necessarily have to be in core. It can be a module as mod_perl shows.
I think if we take Paul's idea of a new pluggable API, we'll have a lot
of happy people. We can have a lua implementation, a perl
implementation, and IMHO we should retain some C implementation to the
crappy config that people (users) have now and are used to working with
(and bear in mind that there are STILL folks out there in 1.3-land
because of what we did to the httpd-2 config).
It would make everyone happy by having a C implementation that can be
used immediately without any new dependencies (and looks just like the
existing engine does), a Lua implementation which can come
out-of-the-box with mod_wombat (and we can even make that the ASF
recommended approach effective immediately as long as we keep
everything compatible at least until httpd-3 is ready), and mod_perl
(and friends) can provide other interfaces.
My EUR 0.02,
Issac
Re: Dynamic configuration for the hackathon?
Can we 1st determine exactly what pain-point we're trying to solve here? Or, at least, prioritize them? It seems to me that if the main issue is runtime re-configuration during the request/response phases, then that really forces us into something which must be very lean, mean and FAST. By extension, this also provides to us a way to really clean up and standardize our current config mess. If however, our focus would be to make our config files cleaner and prettier, then I fear that by the time that trickles down to how that interacts with the actual runtime processing, we'll get nasty performance. I'd prefer optimum runtime and let that drive how it gets exposed to the admin, rather than the reverse... And then we can see if that pain is worth it :)
Re: Dynamic configuration for the hackathon?
On 4/1/08 5:35 AM, Issac Goldstand [EMAIL PROTECTED] wrote: I don't think we're even talking about on-the-fly stuff for the Lua parser engine, so Perl can do everything that Lua can. I am. The biggest problems I have with current system are: -Every module does things differently -No real per-request configuration. Some modules use env to do some of this. -I have to write a good bit of code before a module is configurable. (I'm lazy. Very lazy.) If we do what niq suggests (which, if we stick with current config system is fine) is that it just adds another layer on top of all the existing issues. With lua, for example you could make modules Lua modules... Maybe could do same in perl?? My opinion (which is worthless, I know) is to pick one way and do it. Lua is easy to learn and satisfies most (all?) of our requirements. And if Brian M. and I can learn it, anyone can ;) -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: Dynamic configuration for the hackathon?
On Apr 1, 2008, at 2:17 AM, Justin Erenkrantz wrote: On Mon, Mar 31, 2008 at 7:41 PM, William A. Rowe, Jr. [EMAIL PROTECTED] wrote: I sympathize, but this doesn't reflect the addition of lua blocks... those blocks can be trivially implemented as a loadable module ;-) As I grok it, the point would be throw out our ridiculous config syntax entirely (or at best write a compatibility module or a converter to the new format) and expose a real config API (hello providers! *chuckle*) and then build a pure LUA-based config format on top of that. I'd be definitely curious to see what would come of that - 'cuz really it can't be much worse than the garbage we're stuck with now. -- justin Agreed... historically, of course, that alone has not be reason enough. I recall many many discussions, esp with Dean, about the overhead of folding such a complex parser into httpd compared to having that done externally (m4 anyone? :) ) and out of process...
Re: Dynamic configuration for the hackathon?
IMO, we work best when we feel empowered to scratch our itches... As we've also seen, sometimes all it takes is someone to create a rough framework of an implementation for people to get excited by it and jump right on in, adding, extending and tuning it. This reminds me: a serf BOF or session would, I think, go over quite well :)
Re: Dynamic configuration for the hackathon?
On Apr 1, 2008, at 5:21 AM, Torsten Foertsch wrote: You cannot add virtual servers on the fly Hmmm let's see now. If we have a default Vhost that all non-matched name-based hosts get directed to configured, then a mod_perl based handler can be adjusted to look at the Server header and do different stuff based on what it's set to... envision a simple dynamic mapping.
Re: Dynamic configuration for the hackathon?
On Mar 31, 2008, at 2:15 PM, Roy T. Fielding wrote: Users like mod_rewrite for many reasons, but I think mostly because it does so many things that almost every Apache hosting provider needs to have it installed and usable in .htaccess Except for web hosting companies, and some special cases, at least in my experience people avoid .htaccess files like the plague. :)
Re: Dynamic configuration for the hackathon?
Akins, Brian wrote: My opinion (which is worthless, I know) is to pick one way and do it. Lua is easy to learn and satisfies most (all?) of our requirements. And if Brian M. and I can learn it, anyone can ;) The trouble is, if I want to solve problem A (configure the server), and I find out that before I can solve problem A (configure the server) I need to first solve problem B (learn a new language), that is a big incentive to just ignore the new server and stick with httpd v2. It makes no difference in the world how easy Lua is to learn, no learning required will always beat easy to learn hands down every single time. The Apache config already supports the concepts of a global config, and a per-request config (.htaccess), these basic things could easily form the building blocks of a pluggable config mechanism, in whatever language you like. Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
Re: Dynamic configuration for the hackathon?
Let me try and summarize this then: Problem: The httpd configuration is to static for some users (e.g. large host providers) they want to have a more dynamic system. Where they can configure things on a request basis and add vhosts and such without restarting httpd. Solutions propose: - lua in the core or atleast in a module - mod_perl Downside: - perl isn't very easy and userfriendly - overhead of having this be re-evaluated a lot - lue - users to lazy to learn new language I'm sure I missed a lot since I seem to be missing some older messages so feel free to ignore this add to it or whatever. -- ~Jorge
Re: Dynamic configuration for the hackathon?
On Apr 1, 2008, at 9:34 AM, Graham Leggett wrote: Jim Jagielski wrote: This reminds me: a serf BOF or session would, I think, go over quite well :) A question that has been on my mind for a bit, is what does serf intend to replace, and why is it better?. The impression I have so far is that somehow what we have now is suboptimal, and that serf somehow does it better, but exactly what is suboptimal, and exactly how serf does it better has yet to be clarified. Basically, with serf it's easier to get away from the 1 request/response cycle being tied to a specific thread/process. serf provides the async aspects for free and abstracts it out for us.
Re: Dynamic configuration for the hackathon?
On 4/1/08 9:40 AM, Torsten Foertsch [EMAIL PROTECTED] wrote:
I know one can do that. But a VHost has a server_rec, maybe a separate
error_log and access_log, etc. Those cannot be created at request time. That
is what I meant.
Well
I was hacking around with the idea that the selection of vhost is a hook,
rather than how it is now. Basically, what I did was protect the vhost list
with a mutex, so you can add/delete (well not fully delete) vhosts. Combine
that with a global logger (ie, piped logger to syslog-ng) and the access
and error logs are created on the fly as well.
In the end, it was way too hacky and I haven't thought about it much in well
over a year.
In pseudo config, like niq is suggesting, you could have something like:
If HTTP_HEADER{Host} =~ cnn\.com$ || TCPPort == 8080
#cnn specific stuff here...
DocumentRoot /htdocs/cnn
CutomLog |/usr/bin/logger cnn my_format
ErrorLog /var/log/cnn.error
/If
H
Would just have to move a bunch of stuff to per_dir and make sure all of our
directory config creation and merging is very fast...
This IF/else stuff could be done in a module and not in core for testing
purposes. Not quite where I'd like the config to be, but this may actually
be obtainable in near future.
--
Brian Akins
Chief Operations Engineer
Turner Digital Media Technologies
Re: Dynamic configuration for the hackathon?
On Tue 01 Apr 2008, Jim Jagielski wrote: On Apr 1, 2008, at 5:21 AM, Torsten Foertsch wrote: You cannot add virtual servers on the fly Hmmm let's see now. If we have a default Vhost that all non-matched name-based hosts get directed to configured, then a mod_perl based handler can be adjusted to look at the Server header and do different stuff based on what it's set to... envision a simple dynamic mapping. I know one can do that. But a VHost has a server_rec, maybe a separate error_log and access_log, etc. Those cannot be created at request time. That is what I meant. Torsten
Re: Dynamic configuration for the hackathon?
Jim Jagielski wrote: This reminds me: a serf BOF or session would, I think, go over quite well :) A question that has been on my mind for a bit, is what does serf intend to replace, and why is it better?. The impression I have so far is that somehow what we have now is suboptimal, and that serf somehow does it better, but exactly what is suboptimal, and exactly how serf does it better has yet to be clarified. Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
Re: Dynamic configuration for the hackathon?
On Apr 1, 2008, at 9:40 AM, Torsten Foertsch wrote: On Tue 01 Apr 2008, Jim Jagielski wrote: On Apr 1, 2008, at 5:21 AM, Torsten Foertsch wrote: You cannot add virtual servers on the fly Hmmm let's see now. If we have a default Vhost that all non- matched name-based hosts get directed to configured, then a mod_perl based handler can be adjusted to look at the Server header and do different stuff based on what it's set to... envision a simple dynamic mapping. I know one can do that. But a VHost has a server_rec, maybe a separate error_log and access_log, etc. Those cannot be created at request time. That is what I meant. Gotcha.
Re: Dynamic configuration for the hackathon?
Jorge Schrauwen wrote: Solutions propose: - lua in the core or atleast in a module - mod_perl mod_perl exists already. We're looking to replace it because... (see below) Downside: - perl isn't very easy and userfriendly I think that the downside is the fact that perl interpreters suck up RAM, not the easiness factor. It's probably not significantly easier/harder than lua, but it's big and clunky Issac
Re: Dynamic configuration for the hackathon?
On Tue, Apr 1, 2008 at 4:13 PM, Issac Goldstand [EMAIL PROTECTED] wrote: Jorge Schrauwen wrote: Solutions propose: - lua in the core or atleast in a module - mod_perl mod_perl exists already. We're looking to replace it because... (see below) I'm quite aware that it exists, I ment that it is a possible solution to the problem. (I'm actually using it... although restarts are still needed to load the new vhosts) Downside: - perl isn't very easy and userfriendly I think that the downside is the fact that perl interpreters suck up RAM, not the easiness factor. It's probably not significantly easier/harder than lua, but it's big and clunky Well perl did scare the *bleep* out of my class mates when they where looking what I was doing during the break. Then again, I do most of my system scripts in perl ^^ Issac -- ~Jorge
Re: Dynamic configuration for the hackathon?
On Apr 1, 2008, at 9:36 AM, Jorge Schrauwen wrote: Let me try and summarize this then: Problem: The httpd configuration is to static for some users (e.g. large host providers) they want to have a more dynamic system. IMO, based on feedback from people I've dealt with with my Covalent/SpringSource hat on, the biggest problem for these dynamic hosts is adding/changing/deleting SSL vhosts more than anything else :)
Re: Dynamic configuration for the hackathon?
Jim Jagielski wrote: On Apr 1, 2008, at 9:36 AM, Jorge Schrauwen wrote: Let me try and summarize this then: Problem: The httpd configuration is to static for some users (e.g. large host providers) they want to have a more dynamic system. IMO, based on feedback from people I've dealt with with my Covalent/SpringSource hat on, the biggest problem for these dynamic hosts is adding/changing/deleting SSL vhosts more than anything else :) Actually, wearing my company's hat, our need is actually more for the dynamic config (including vhosts if we could plug one in via a config API) that can be modified at run-time (and not only after [graceful] reloads). Issac
Re: Dynamic configuration for the hackathon?
On Apr 1, 2008, at 10:32 AM, Matthew M. Burke wrote: Graham Leggett wrote: The trouble is, if I want to solve problem A (configure the server), and I find out that before I can solve problem A (configure the server) I need to first solve problem B (learn a new language), that is a big incentive to just ignore the new server and stick with httpd v2. It makes no difference in the world how easy Lua is to learn, no learning required will always beat easy to learn hands down every single time. That's just marketing! Configuring the server is not a no learning required activity, particularly if you are using mod_rewrite! Matt Graham's point is, I think, that for most setups, although ugly, the config does not require a lot of thought. Compare, for example, a typical Apache config to sendmail's sendmail.cf file :)
Re: Dynamic configuration for the hackathon?
Matthew M. Burke wrote: Graham Leggett wrote: The trouble is, if I want to solve problem A (configure the server), and I find out that before I can solve problem A (configure the server) I need to first solve problem B (learn a new language), that is a big incentive to just ignore the new server and stick with httpd v2. It makes no difference in the world how easy Lua is to learn, no learning required will always beat easy to learn hands down every single time. That's just marketing! Configuring the server is not a no learning required activity, particularly if you are using mod_rewrite! We're not talking about fresh users, we're talking about existing users. Fresh users have to deal with one learning curve or another anyway. Issac
Re: Dynamic configuration for the hackathon?
Graham Leggett wrote: The trouble is, if I want to solve problem A (configure the server), and I find out that before I can solve problem A (configure the server) I need to first solve problem B (learn a new language), that is a big incentive to just ignore the new server and stick with httpd v2. It makes no difference in the world how easy Lua is to learn, no learning required will always beat easy to learn hands down every single time. That's just marketing! Configuring the server is not a no learning required activity, particularly if you are using mod_rewrite! Matt
Re: Dynamic configuration for the hackathon?
On Tue 01 Apr 2008, Akins, Brian wrote:
In pseudo config, like niq is suggesting, you could have something like:
If HTTP_HEADER{Host} =~ cnn\.com$ || TCPPort == 8080
#cnn specific stuff here...
DocumentRoot /htdocs/cnn
CutomLog |/usr/bin/logger cnn my_format
ErrorLog /var/log/cnn.error
/If
I don't like that. I think there are security considerations why logfiles are
opened from the parent process as root. But with other logging mechanisms
that provide write-only semantics it is good. In my setup the apache logs to
a named pipe to a process outside the chroot.
To really create vhosts on the fly I think a new hook in the MPM would be good
that is called from a configuration provider module. It reconfigures the
parent apache and does a graceful restart. This way almost anything can be
reconfigured. A question is whether the provider should send changes to the
apache or a complete new config. In the former case we'd need something like
UnListen localhost:80
CloseErrorLog ...
DeleteVirtualHost localhost:80
In the end the server_rec would go away. We have one server with a list of
loaded modules, a PidFile and an AcceptMutex that is listening on a list of
ports. The rest is configurable this way:
if localport==443 and localaddr=1.2.3.4
SSLCertificateFile ...
Protocol http # expecting HTTP to be spoken on the wire
if header_in{Host}=~cnn\.com
Timeout 10
ErrorLog ...
/if
/if
Or rather the request is passed to the config module that checks localport and
localaddr and issues the SSLCertificateFile directive. Then it checks the
Host-header ...
As for dynamic request configuration, I'd wish some configuration provider
with intelligent conftree caching. That provider then implements a language
as it likes, LUA, Perl, Apache-style if.../if, ... It generates a list of
directives that is compiled into a conftree.
As I understood it the main problem with the current mod_rewrite based config
is that it is too complex. The new language has to watch out not to end at
the same place. One thing that I think is messy is the use of subprocess_env
to pass information from module to module and even from administrator to
module: no-gzip, force-gzip, downgrade-1.0, nokeepalive, redirect-carefully
etc.
Torsten
Re: Dynamic configuration for the hackathon?
On 4/1/08 11:21 AM, Torsten Foertsch [EMAIL PROTECTED] wrote:
On Tue 01 Apr 2008, Akins, Brian wrote:
In pseudo config, like niq is suggesting, you could have something like:
If HTTP_HEADER{Host} =~ cnn\.com$ || TCPPort == 8080
#cnn specific stuff here...
DocumentRoot /htdocs/cnn
CutomLog |/usr/bin/logger cnn my_format
ErrorLog /var/log/cnn.error
/If
I don't like that. I think there are security considerations why logfiles are
opened from the parent process as root. But with other logging mechanisms
that provide write-only semantics it is good. In my setup the apache logs to
a named pipe to a process outside the chroot.
The example wasn't about logs, it was just an example of how, if you wanted,
could do virtual hosts using niq's if/else style. Nothing says that the
logs wouldn't be opened in parent by root. If log config was per dir (which
we emulate with env variables) this would just work.
As I understood it the main problem with the current mod_rewrite based config
is that it is too complex. The new language has to watch out not to end at
the same place. One thing that I think is messy is the use of subprocess_env
to pass information from module to module and even from administrator to
module: no-gzip, force-gzip, downgrade-1.0, nokeepalive, redirect-carefully
etc.
+1 to all that...
--
Brian Akins
Chief Operations Engineer
Turner Digital Media Technologies
Re: Dynamic configuration for the hackathon?
Jim Jagielski wrote: I'd prefer optimum runtime and let that drive how it gets exposed to the admin, rather than the reverse... And then we can see if that pain is worth it :) +1 to this as a guiding principle. I know our administrators would, above all else, like a standard way to build up configuration files from templates. Perhaps that could be mod_macro packaged in the main distribution, or some kind of configulator utility, or yet something else. Dynamic (re)configuration issues are a lesser concern for them, I think; it would be great if whatever is implemented could be largely optional so it can be loaded only if necessary. Good luck at the hackathon! Chris. -- GPG Key ID: 366A375B GPG Key Fingerprint: 485E 5041 17E1 E2BB C263 E4DE C8E3 FA36 366A 375B
Re: Dynamic configuration for the hackathon?
On Tue, Apr 1, 2008 at 6:20 AM, Jim Jagielski [EMAIL PROTECTED] wrote: IMO, we work best when we feel empowered to scratch our itches... As we've also seen, sometimes all it takes is someone to create a rough framework of an implementation for people to get excited by it and jump right on in, adding, extending and tuning it. This reminds me: a serf BOF or session would, I think, go over quite well :) Lieven and I are going to be focusing on cutting a serf release during the Hackathon. I promised that we'd get a new serf release out before Subversion 1.5 final comes out. =P If folks are interested in a serf BOF or 'beer session' at the bar, I'm sure we could swing something. =) -- justin
Re: Dynamic configuration for the hackathon?
Akins, Brian wrote:
On 3/26/08 9:06 AM, Nick Kew [EMAIL PROTECTED] wrote:
There seems to be a demand for dynamic per-request configuration,
as evidenced by the number of users hacking it with mod_rewrite,
and the other very limited tools available. Modern mod_rewrite
usage commonly looks like programming, but it's not designed as
a programming language. Result: confused and frustrated users.
This is what I had in mind when I suggested having Lua blocks of code. No
need to invent a new language when a perfectly fine one exists...
+1, and embed Lua in the core, and a dozen problems just like this are
solved.
Every complicated 'directive' is trying to be a programing language in
the config file, but they aren't.
Just look at SSLRequire, Rewrite*, MPM Process/Thread Management, Filter
chaining, large Auth{N,Z} chains, and more.
Imagine them not sucking.
No offense intended to the respective authors of each -- they are all
complicated things, and I've helped prolong their lives.
I'll be at the hackathon :-)
-Paul
Re: Dynamic configuration for the hackathon?
Nick Kew wrote: On Thu, 27 Mar 2008 08:17:01 -0400 Akins, Brian [EMAIL PROTECTED] wrote: On 3/27/08 3:58 AM, Torsten Foertsch [EMAIL PROTECTED] wrote: So I was going to reimplement it based on mod_wombat some time this year. The nice thing about lua, in addition to being lightweight, is that most of the url mapping/rewriting can be simple lua statements. Lua fixups if string.match(r.uri, '/something') then r.filename = '/www/that/path' end /Lua Fine for users who want to hack their own server. Like Perl. But r.filename is the kind of innards we really don't want to expose to the typical mod_rewrite user! I disagree. Just read the mod_rewrite docs: http://httpd.apache.org/docs/2.2/rewrite/rewrite_tech.html#InternalAPI They are already exposing internals to users'. Users want customization. We should just do it right, and stop hacking around the central problem. Expose the structures. Embed Lua. -Paul
Re: Dynamic configuration for the hackathon?
Paul Querna wrote:
Akins, Brian wrote:
On 3/26/08 9:06 AM, Nick Kew [EMAIL PROTECTED] wrote:
There seems to be a demand for dynamic per-request configuration,
as evidenced by the number of users hacking it with mod_rewrite,
and the other very limited tools available. Modern mod_rewrite
usage commonly looks like programming, but it's not designed as
a programming language. Result: confused and frustrated users.
This is what I had in mind when I suggested having Lua blocks of
code. No
need to invent a new language when a perfectly fine one exists...
+1, and embed Lua in the core, and a dozen problems just like this are
solved.
-0.5 PLEASE not in the core. Make mod_wombat a standard module and
part of the default moduleset, whatever, but let's not make more core
dependencies, please?!?
If I get voted down (which is still pretty likely especially if all the
pro-lua'ers will be at the hackathon, whereas I won't :)) at least
consider trying to limit the embedded interpreter to the parent process
and preventing it from being inherited by the children, if possible (to
remove completely unnecessary bloating)
Every complicated 'directive' is trying to be a programing language in
the config file, but they aren't.
Just look at SSLRequire, Rewrite*, MPM Process/Thread Management, Filter
chaining, large Auth{N,Z} chains, and more.
Imagine them not sucking.
Agreed. That's why we have modules like mod_wombat and mod_perl which
give you *better* directives. More flexible directives. And in
addition, the learning curve to learn to use these powerful directives
is still optional (ok - rewrite's a pretty damn big curve itself, but
many of the other above items aren't anywhere near as bad).
Issac
Re: Dynamic configuration for the hackathon?
On 3/31/08 1:46 PM, Issac Goldstand [EMAIL PROTECTED] wrote:
if possible (to
remove completely unnecessary bloating)
Lua != perl
Lua perl (size wise by an order of magnitude)
And in
addition, the learning curve to learn to use these powerful directives
is still optional
I disagree. It's hard currently bcs every module does their config
differently... Some use dir... blocks, some don't. Some support
expressions ($stuff, $ENV{foo}), some don't, etc. With Lua (or magic slim
interpreter) module author just writes a bit of glue code, no more
complicated than the current module boot-strap stuff, and it all just
works.
--
Brian Akins
Chief Operations Engineer
Turner Digital Media Technologies
Re: Dynamic configuration for the hackathon?
On 3/31/08 1:39 PM, Paul Querna [EMAIL PROTECTED] wrote: We should just do it right, and stop hacking around the central problem. Expose the structures. Embed Lua. +1, but you already knew that... Also, mod_wombat, as such, goes away if Lua is embedded. We may have a module that sits on top of the embedded lua to give mod_wombat like features (handlers). Lua is not perl. Comparisons between embedding lua (the core of mod_wombat, not the handler stuff to mod_perl is just not the same. -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: Dynamic configuration for the hackathon?
Issac Goldstand wrote: Paul Querna wrote: Akins, Brian wrote: On 3/26/08 9:06 AM, Nick Kew [EMAIL PROTECTED] wrote: There seems to be a demand for dynamic per-request configuration, as evidenced by the number of users hacking it with mod_rewrite, and the other very limited tools available. Modern mod_rewrite usage commonly looks like programming, but it's not designed as a programming language. Result: confused and frustrated users. This is what I had in mind when I suggested having Lua blocks of code. No need to invent a new language when a perfectly fine one exists... +1, and embed Lua in the core, and a dozen problems just like this are solved. -0.5 PLEASE not in the core. Make mod_wombat a standard module and part of the default moduleset, whatever, but let's not make more core dependencies, please?!? Why not? I look at it this way: We are already making up a fake language in our configuration file. If I get voted down (which is still pretty likely especially if all the pro-lua'ers will be at the hackathon, whereas I won't :)) at least No decisions will be made at the hackathon that will not be reflected on the mailing list. consider trying to limit the embedded interpreter to the parent process and preventing it from being inherited by the children, if possible (to remove completely unnecessary bloating) I think the right approach is to first change the internal configuration API. Make it a real API, not a series of callbacks with filepointers and strings in them. Once we have that, we can write language bindings for all of them, and all languages are on an equal footing. My personal belief is that Lua should be compiled into the core, and enabled by default. This does not really mean a hard dependency. I would *prefer* that it can still be compiled out, and yes, while it would be hard to configure without a 'configuration provider', alternatives could be written in any language, or even with XML input as others have wanted for years, since we now fixed configuration to be an API. -Paul
Re: Dynamic configuration for the hackathon?
Paul Querna wrote: Issac Goldstand wrote: I think the right approach is to first change the internal configuration API. Make it a real API, not a series of callbacks with filepointers and strings in them. Once we have that, we can write language bindings for all of them, and all languages are on an equal footing. My personal belief is that Lua should be compiled into the core, and enabled by default. This does not really mean a hard dependency. I would *prefer* that it can still be compiled out, and yes, while it would be hard to configure without a 'configuration provider', alternatives could be written in any language, or even with XML input as others have wanted for years, since we now fixed configuration to be an API. That, I'd happily +1 Issac
Re: Dynamic configuration for the hackathon?
On Mar 31, 2008, at 10:39 AM, Paul Querna wrote: Just read the mod_rewrite docs: http://httpd.apache.org/docs/2.2/rewrite/rewrite_tech.html#InternalAPI They are already exposing internals to users'. Users want customization. We should just do it right, and stop hacking around the central problem. Expose the structures. Embed Lua. Users like mod_rewrite for many reasons, but I think mostly because it does so many things that almost every Apache hosting provider needs to have it installed and usable in .htaccess, and thus is one of the few solutions that can be taught in a FAQ to powerless users). I like customizations, but I also like servers that stay up forever and do most of their configuration-based thinking outside the critical path of request processing. My worry about Lua is that run-time procedural customizations are far less efficient than precompiled regex tables. In other words, we need both, though we'd be better served if we can do everything in a single, sensible syntax. Unfortunately, after last year's experience of being the only server person around who wasn't working on a Joost release, I decided to delay my arrival until Tuesday rather than attend the hackathon. Please have fun, finish the server, and let me know what happened when I arrive; I'll try to squeeze it into my keynote for Friday. ;-) Roy
Re: Dynamic configuration for the hackathon?
Issac Goldstand wrote:
Paul Querna wrote:
Akins, Brian wrote:
On 3/26/08 9:06 AM, Nick Kew [EMAIL PROTECTED] wrote:
There seems to be a demand for dynamic per-request configuration,
as evidenced by the number of users hacking it with mod_rewrite,
and the other very limited tools available. Modern mod_rewrite
usage commonly looks like programming, but it's not designed as
a programming language. Result: confused and frustrated users.
This is what I had in mind when I suggested having Lua blocks of
code. No
need to invent a new language when a perfectly fine one exists...
+1, and embed Lua in the core, and a dozen problems just like this are
solved.
-0.5 PLEASE not in the core. Make mod_wombat a standard module and
part of the default moduleset, whatever, but let's not make more core
dependencies, please?!?
-0.99 - agreed. Perl is perfectly happy having perl blocks as modular
behaviors... I've noticed a trend in the last few years of building on the
core (and folks rightfully accused me of growing mod_proxy core when new
directives are rightfully part of mod_proxy_{whatever}).
Let's focus on keeping it in useful pieces, even if they are built by
default.
Re: Dynamic configuration for the hackathon?
William A. Rowe, Jr. wrote:
Issac Goldstand wrote:
Paul Querna wrote:
Akins, Brian wrote:
On 3/26/08 9:06 AM, Nick Kew [EMAIL PROTECTED] wrote:
There seems to be a demand for dynamic per-request configuration,
as evidenced by the number of users hacking it with mod_rewrite,
and the other very limited tools available. Modern mod_rewrite
usage commonly looks like programming, but it's not designed as
a programming language. Result: confused and frustrated users.
This is what I had in mind when I suggested having Lua blocks of
code. No
need to invent a new language when a perfectly fine one exists...
+1, and embed Lua in the core, and a dozen problems just like this
are solved.
-0.5 PLEASE not in the core. Make mod_wombat a standard module and
part of the default moduleset, whatever, but let's not make more core
dependencies, please?!?
-0.99 - agreed. Perl is perfectly happy having perl blocks as modular
behaviors... I've noticed a trend in the last few years of building on the
core (and folks rightfully accused me of growing mod_proxy core when new
directives are rightfully part of mod_proxy_{whatever}).
Yes, but the root of the problem even with perl blocks is that they
have almost no way to change the behavior of existing modules -- like
you can via configuration -- and instead for the most part, they
reimplment entire C modules in Perl, or any other language, rather than
binding to the configuration, and change that based on some other inputs.
The few that can change behaviors, ie via ENV vars, are special cases in
every modules case, and do not make a difference in the larger picture.
The core of what I want is a better configuration API.
Then the existing configuration file, a new lua system, or anything
else, could be written in terms of that, rather than the current system
where each language reinvents the modules it wants to control.
-Paul
Re: Dynamic configuration for the hackathon?
Paul Querna wrote: Then the existing configuration file, a new lua system, or anything else, could be written in terms of that, rather than the current system where each language reinvents the modules it wants to control. I sympathize, but this doesn't reflect the addition of lua blocks... those blocks can be trivially implemented as a loadable module ;-)
Re: Dynamic configuration for the hackathon?
Paul Querna wrote:
William A. Rowe, Jr. wrote:
Issac Goldstand wrote:
Paul Querna wrote:
Akins, Brian wrote:
On 3/26/08 9:06 AM, Nick Kew [EMAIL PROTECTED] wrote:
There seems to be a demand for dynamic per-request configuration,
as evidenced by the number of users hacking it with mod_rewrite,
and the other very limited tools available. Modern mod_rewrite
usage commonly looks like programming, but it's not designed as
a programming language. Result: confused and frustrated users.
This is what I had in mind when I suggested having Lua blocks of
code. No
need to invent a new language when a perfectly fine one exists...
+1, and embed Lua in the core, and a dozen problems just like this
are solved.
-0.5 PLEASE not in the core. Make mod_wombat a standard module and
part of the default moduleset, whatever, but let's not make more core
dependencies, please?!?
-0.99 - agreed. Perl is perfectly happy having perl blocks as modular
behaviors... I've noticed a trend in the last few years of building on
the
core (and folks rightfully accused me of growing mod_proxy core when new
directives are rightfully part of mod_proxy_{whatever}).
Yes, but the root of the problem even with perl blocks is that they
have almost no way to change the behavior of existing modules -- like
you can via configuration -- and instead for the most part, they
reimplment entire C modules in Perl, or any other language, rather than
binding to the configuration, and change that based on some other inputs.
The few that can change behaviors, ie via ENV vars, are special cases in
every modules case, and do not make a difference in the larger picture.
Not completely true. mod_perl provides several ways to bind to the
configuration. Consider this bit of Perl magic - this is some mp1 code,
running in Perl-land, not in a Perl block, though it could as easily
be run there...
package Apache::ReadConfig;
no strict;
$Location{$UploadScript} = {
Options = '+ExecCGI',
PerlHeaderParserHandler = $namespace.::upload_jit_handler,
};
$Location{$UploadMeter} = {
Options = '+ExecCGI',
PerlHeaderParserHandler = $namespace.::meter_jit_handler,
};
$Location{$UploadForm} = {
Options = '+ExecCGI',
PerlHeaderParserHandler = $namespace.::form_jit_handler,
};
Granted, you're still limited to the underlying config parser, so I'm
not arguing with the new config API idea (at least as I understood it
yesterday), but this should be treated as a pluggable config engine
which everyone can access; not as a switch from one config parser to
another.
Issac
Re: Dynamic configuration for the hackathon?
On Wed 26 Mar 2008, Akins, Brian wrote: There seems to be a demand for dynamic per-request configuration, as evidenced by the number of users hacking it with mod_rewrite, and the other very limited tools available. Modern mod_rewrite usage commonly looks like programming, but it's not designed as a programming language. Result: confused and frustrated users. This is what I had in mind when I suggested having Lua blocks of code. No need to invent a new language when a perfectly fine one exists... As Issac pointed out something similar can be done with Perl blocks at the cost of having mod_perl in core. Those are not evaluated evaluated per-request. But based on mod_perl there is Apache2::Translation that does per-request configuration. It hooks uri translation, maptostorage and fixup to do the job. Again it needs a perl interpreter in core and hence doesn't work well with threaded MPMs. So I was going to reimplement it based on mod_wombat some time this year. I just wanted to add these $0.02 to the discussion. Torsten
Re: Dynamic configuration for the hackathon?
I used to use mod_macro, then I moved to mod_perl but like you said. mod_perl is great (well, more okay than great) for dynamic configurations that change/get generated on start and not per request. A new more flexible alternative would be awsome. Jorge (on vacation) On Thu, Mar 27, 2008 at 8:58 AM, Torsten Foertsch [EMAIL PROTECTED] wrote: On Wed 26 Mar 2008, Akins, Brian wrote: There seems to be a demand for dynamic per-request configuration, as evidenced by the number of users hacking it with mod_rewrite, and the other very limited tools available. Modern mod_rewrite usage commonly looks like programming, but it's not designed as a programming language. Result: confused and frustrated users. This is what I had in mind when I suggested having Lua blocks of code. No need to invent a new language when a perfectly fine one exists... As Issac pointed out something similar can be done with Perl blocks at the cost of having mod_perl in core. Those are not evaluated evaluated per-request. But based on mod_perl there is Apache2::Translation that does per-request configuration. It hooks uri translation, maptostorage and fixup to do the job. Again it needs a perl interpreter in core and hence doesn't work well with threaded MPMs. So I was going to reimplement it based on mod_wombat some time this year. I just wanted to add these $0.02 to the discussion. Torsten -- ~Jorge
Re: Dynamic configuration for the hackathon?
On Thu, 27 Mar 2008 08:17:01 -0400 Akins, Brian [EMAIL PROTECTED] wrote: On 3/27/08 3:58 AM, Torsten Foertsch [EMAIL PROTECTED] wrote: So I was going to reimplement it based on mod_wombat some time this year. The nice thing about lua, in addition to being lightweight, is that most of the url mapping/rewriting can be simple lua statements. Lua fixups if string.match(r.uri, '/something') then r.filename = '/www/that/path' end /Lua Fine for users who want to hack their own server. Like Perl. But r.filename is the kind of innards we really don't want to expose to the typical mod_rewrite user! And if the more complicated modules had a little lua glue: if string.match(r.uri, '/something') then mod_cache:cacheable( r ) end A fine recipe for users shooting themselves in the foot, PHP-style. How would you propose to make that work without hackage to existing modules? If one were so inclined, the entire configuration could be lua. Just define and register these functions that need to run per request. That'll go alongside DrBacchus's You can do everything with mod_rewrite :-) -- Nick Kew Application Development with Apache - the Apache Modules Book http://www.apachetutor.org/
Re: Dynamic configuration for the hackathon?
On 3/27/08 9:00 AM, Nick Kew [EMAIL PROTECTED] wrote: /Lua Fine for users who want to hack their own server. Like Perl. Every play with lighttpd? It's almost the same way... Of course typical lighthttpd user is a hacker. But r.filename is the kind of innards we really don't want to expose to the typical mod_rewrite user! We already expose a lot. You can, indirectly set r-filename with mod_rewrite currently. And if the more complicated modules had a little lua glue: if string.match(r.uri, '/something') then mod_cache:cacheable( r ) end A fine recipe for users shooting themselves in the foot, PHP-style. How would you propose to make that work without hackage to existing modules? I don't. Hence the glue. If one were so inclined, the entire configuration could be lua. Just define and register these functions that need to run per request. That'll go alongside DrBacchus's You can do everything with mod_rewrite :-) If you had lua (or whatever) in configs, you don't need mod_rewrite, alias, etc... Anyway, back to your suggestion of If's and per request configs, I'm +1, especially if it's easily extensible and doesn't parse the tree on every single request. -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: Dynamic configuration for the hackathon?
On 3/26/08 9:06 AM, Nick Kew [EMAIL PROTECTED] wrote: There seems to be a demand for dynamic per-request configuration, as evidenced by the number of users hacking it with mod_rewrite, and the other very limited tools available. Modern mod_rewrite usage commonly looks like programming, but it's not designed as a programming language. Result: confused and frustrated users. This is what I had in mind when I suggested having Lua blocks of code. No need to invent a new language when a perfectly fine one exists... -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: Dynamic configuration for the hackathon?
Akins, Brian wrote: On 3/26/08 9:06 AM, Nick Kew [EMAIL PROTECTED] wrote: There seems to be a demand for dynamic per-request configuration, as evidenced by the number of users hacking it with mod_rewrite, and the other very limited tools available. Modern mod_rewrite usage commonly looks like programming, but it's not designed as a programming language. Result: confused and frustrated users. This is what I had in mind when I suggested having Lua blocks of code. No need to invent a new language when a perfectly fine one exists... FWIW, it's done with Perl blocks too (I do some funky things that way), BUT I'm not sure if those are parsed per-request as I think Nick is suggesting. Also, many times people don't want to bloat their processes with a fully-fleged interpreter (again, I'm building on my mod_perl experience here - I know that the shared Perl objects are pretty clunky, and not sure if mod_wombat looks the same). Right now it doesn't look like I'll be in Amsterdam... Issac
Re: Dynamic configuration for the hackathon?
On Wed, 26 Mar 2008 15:39:53 +0200 Issac Goldstand [EMAIL PROTECTED] wrote: Akins, Brian wrote: On 3/26/08 9:06 AM, Nick Kew [EMAIL PROTECTED] wrote: There seems to be a demand for dynamic per-request configuration, as evidenced by the number of users hacking it with mod_rewrite, and the other very limited tools available. Modern mod_rewrite usage commonly looks like programming, but it's not designed as a programming language. Result: confused and frustrated users. This is what I had in mind when I suggested having Lua blocks of code. No need to invent a new language when a perfectly fine one exists... I'm not talking about inventing a new language. Those who want one have some options already, as noted below ... FWIW, it's done with Perl blocks too (I do some funky things that way), BUT I'm not sure if those are parsed per-request as I think Nick is suggesting. Neither am I, FWIW. Also, many times people don't want to bloat their processes with a fully-fleged interpreter That is much more of a consideration. As I said, the basic idea is to provide a much simpler rationalisation for the kind of things people struggle to do with mod_rewrite et al. (again, I'm building on my mod_perl experience here - I know that the shared Perl objects are pretty clunky, and not sure if mod_wombat looks the same). AFAICT mod_wombat just provides Lua bindings (some of them stubs) for hooks exported from the core. Nothing for configuration. -- Nick Kew Application Development with Apache - the Apache Modules Book http://www.apachetutor.org/
Re: Dynamic configuration for the hackathon?
On 3/26/08 9:53 AM, Nick Kew [EMAIL PROTECTED] wrote:
I'm not talking about inventing a new language. Those who want one
have some options already, as noted below ...
Right. I was just throwing it out there, so to speak. I'm not opposed to
what you are saying, just wondering if we would/should take it to the next
level.
As to your suggestion:
So basically, the per_dir merge would use this mechanism instead of what it
does now (file walk, location walk) (or in addition to??)
Something like:
If Directory == /www/stuff and Remote_IP =~ 10.189.
SetEnv coolstuff
Elsif HTTP_Host == www.domain.com or Local_Port == 8080
Set something different
Elsif ENV{blah} =~ foo or Cookie{baz} == iamset
foo bar
Else
something completely different
/endif
(Horrible, example I know). If it were easy to extend the expresions (ie, I
want to implement (Cache == yes/no) and stuff like ENV{key} were made to
work, I'm all for it.
It *should* be fairly easy to test this out with the current system (ala
Proxy blocks).
--
Brian Akins
Chief Operations Engineer
Turner Digital Media Technologies
Re: Dynamic configuration for the hackathon?
On Wed, 26 Mar 2008 10:15:05 -0400
Akins, Brian [EMAIL PROTECTED] wrote:
As to your suggestion:
So basically, the per_dir merge would use this mechanism instead of
what it does now (file walk, location walk) (or in addition to??)
Something like:
If Directory == /www/stuff and Remote_IP =~ 10.189.
SetEnv coolstuff
Elsif HTTP_Host == www.domain.com or Local_Port == 8080
Set something different
Elsif ENV{blah} =~ foo or Cookie{baz} == iamset
foo bar
Else
something completely different
/endif
Sort-of. There's a question of ordering: merge-config happens
before some of the vars we'd like to make available are available,
so we'd have a bit more work to make Cookie{baz} work (as opposed
to parsing a CGI-style HTTP_COOKIE variable). But that's basically
the kind of thing.
Oh, and there's no inherent reason it shouldn't apply per-host
config too.
(Horrible, example I know). If it were easy to extend the expresions
(ie, I want to implement (Cache == yes/no) and stuff like ENV{key}
were made to work, I'm all for it.
Straightforward: conditions on headers, method (obsoletes Limit),
request line, env, CGI vars. With the option to disable conditional
stuff for speed. Higher-level stuff like *evaluating* caching or
cookie headers ... maybe sometime, but one could argue that's the
point where Perl or Lua makes more sense.
It *should* be fairly easy to test this out with the current system
(ala Proxy blocks).
Hopefully, yes.
Oh, and since ap_expr is a prerequisite for this, it would be great
if folks could review at least the API part (ap_expr.h) of what
I posted earlier.
--
Nick Kew
Application Development with Apache - the Apache Modules Book
http://www.apachetutor.org/
Re: Dynamic configuration for the hackathon?
On 3/26/08 10:31 AM, Nick Kew [EMAIL PROTECTED] wrote:
Straightforward: conditions on headers, method (obsoletes Limit),
request line, env, CGI vars. With the option to disable conditional
stuff for speed.
In mod_include, we parse into a tree on every request. For the
configuration, we should probably just parse it at startup and run it on
every request.
Also, currently, ap_expr is string specific, it would be nice if this was
provider based. Not sure of the exact interface, but it would be extendable
for other types of comparisons, for example.
typedef struct {
apr_status_t (*init_expr)(apr_pool_t *p, const char *lvalue, const char
*rvalue, int expr, void**data);
apr_status_t (*eval_expr)(reuqest_rec *r, void *data);
} ap_expr_provider_t;
So this expresssion, at startup:
If Remote_IP =~ 10.189.
...
/endif
Would call the provider registered for Remote_IP as:
provider-init_expr(conf-pool, Remote_IP, 10.189., AP_EXPR_REGEX,
data);
The provider would construct what ever struct it needs, in this case, to do
partial ip address matching, shove that into a struct and return it via the
data argument;
And then, at request time, we would run:
provider-eval_expr(r, data)
Where data is what was returned by init. This returns basically true or
false.
The string stuff could be easily integrated and provided by default. The
nice thing, is all of this could be then used in mod_include as well, as
well as any other modules that use ap_expr.
Thoughts?
--
Brian Akins
Chief Operations Engineer
Turner Digital Media Technologies
Re: Dynamic configuration for the hackathon?
On 3/26/08 12:42 PM, Akins, Brian [EMAIL PROTECTED] wrote: Thoughts? Of course, it will not work exactly as I have said because we have to take stuff like variable substitution into account, etc. Was just thinking out loud... -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: Dynamic configuration for the hackathon?
On Wed, 26 Mar 2008 12:42:51 -0400
Akins, Brian [EMAIL PROTECTED] wrote:
On 3/26/08 10:31 AM, Nick Kew [EMAIL PROTECTED] wrote:
Straightforward: conditions on headers, method (obsoletes Limit),
request line, env, CGI vars. With the option to disable conditional
stuff for speed.
In mod_include, we parse into a tree on every request. For the
configuration, we should probably just parse it at startup and run
it on every request.
Indeed - hence the parse/eval separation in the proposed API.
Also, currently, ap_expr is string specific, it would be nice if this
was provider based. Not sure of the exact interface, but it would be
extendable for other types of comparisons, for example.
Well, we always start from a string. Later when it's tokenised
we can, and indeed do, dispatch to a provider (in mod_include's
case, functions called handle_foo for keyword foo).
typedef struct {
apr_status_t (*init_expr)(apr_pool_t *p, const char *lvalue, const
char *rvalue, int expr, void**data);
apr_status_t (*eval_expr)(reuqest_rec *r, void *data);
} ap_expr_provider_t;
That's no use at top level, because
So this expresssion, at startup:
If Remote_IP =~ 10.189.
...
/endif
Would call the provider registered for Remote_IP as:
we have to parse a string before we have Remote_IP.
Once we have that, sure, our evaluation function can dispatch
to the Remote_IP handler.
You seem to be looking a little further than my proposal went.
Which is kind-of why it would be good to hackathonise this:-)
--
Nick Kew
Application Development with Apache - the Apache Modules Book
http://www.apachetutor.org/
Re: Dynamic configuration for the hackathon?
On 3/26/08 1:14 PM, Nick Kew [EMAIL PROTECTED] wrote: we have to parse a string before we have Remote_IP. Once we have that, sure, our evaluation function can dispatch to the Remote_IP handler. Of course. I was getting ahead of my self... You seem to be looking a little further than my proposal went. Which is kind-of why it would be good to hackathonise this:-) True. Let me digest this some more... -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
