Re: FileSystem v.s. Other Resources [was configurable Location?]
Joshua Slive wrote: On Mon, 9 Feb 2004, Brian Akins wrote: Location /dynamic-stuff SetHandler virtual-dynamic-handler /Location Alias /normal/stuff/dyn /dynamic-stuff How would all this affect this situation? IE, we would still need to know that /normal/stuff/dyn/1/2/test.html became /dynamic-stuff/1/2/test.html Alias maps URLs to the filesystem. The second argument must be a filesystem path, and should not be mapped against any Location. Joshua. OK. So in my example, in the proposed system, I should use Rewrite? So the alias would become: RewriteRule /normal/stuff/dyn(.*) /dynamic-stuff/$1 I am agreement with the need to not do uneccesarry stats, but I do'nt want to have to write my own alias style routine... -- Brian Akins Senior Systems Engineer CNN Internet Technologies
Re: FileSystem v.s. Other Resources [was configurable Location?]
William A. Rowe, Jr. wrote: At 04:34 PM 2/6/2004, [EMAIL PROTECTED] wrote: /* this info-handler won't deal with the filename * so null the filename to ensure no file is served. */ r-filename = ; \ What if you have something like: Location /dynamic-stuff SetHandler virtual-dynamic-handler /Location Alias /normal/stuff/dyn /dynamic-stuff How would all this affect this situation? IE, we would still need to know that /normal/stuff/dyn/1/2/test.html became /dynamic-stuff/1/2/test.html -- Brian Akins Senior Systems Engineer CNN Internet Technologies
Re: FileSystem v.s. Other Resources [was configurable Location?]
On Mon, 9 Feb 2004, Brian Akins wrote: Location /dynamic-stuff SetHandler virtual-dynamic-handler /Location Alias /normal/stuff/dyn /dynamic-stuff How would all this affect this situation? IE, we would still need to know that /normal/stuff/dyn/1/2/test.html became /dynamic-stuff/1/2/test.html Alias maps URLs to the filesystem. The second argument must be a filesystem path, and should not be mapped against any Location. Joshua.
Re: FileSystem v.s. Other Resources [was configurable Location?]
You hit the nail on the head - Alias (and mod_rewrite) cause us the greatest grief in fixing this set of issues. *IF* they all are parsed in the translate name phase we are fine, since map_to_storage is run after translate_name is done. If they are not handled up front we have problems (some translations can occur in the fixup phase, IIRC, which is wrong IMHO.) The other risk, that we reset filename because we might hit the core handler, could be much cleaner if we just failed 500 right out of the core handler any time that dir_walk isn't run against the resource. I recall some arguments to that fix in 2.0 - I'd like to force the hand on that issue in 2.2. Unsetting the r-filename would become unnecessary in this trivial example. Bill At 08:08 AM 2/9/2004, Brian Akins wrote: William A. Rowe, Jr. wrote: At 04:34 PM 2/6/2004, [EMAIL PROTECTED] wrote: /* this info-handler won't deal with the filename * so null the filename to ensure no file is served. */ r-filename = ; \ What if you have something like: Location /dynamic-stuff SetHandler virtual-dynamic-handler /Location Alias /normal/stuff/dyn /dynamic-stuff How would all this affect this situation? IE, we would still need to know that /normal/stuff/dyn/1/2/test.html became /dynamic-stuff/1/2/test.html
Re: FileSystem v.s. Other Resources [was configurable Location?]
* William A. Rowe, Jr. [EMAIL PROTECTED] wrote: You hit the nail on the head - Alias (and mod_rewrite) cause us the greatest grief in fixing this set of issues. *IF* they all are parsed in the translate name phase we are fine, since map_to_storage is run after translate_name is done. If they are not handled up front we have problems (some translations can occur in the fixup phase, IIRC, which is wrong IMHO.) No. Alias translations and RewriteRules in servercontext are handled in the translate_name hook. Fixup rewrites (-internal redirects) or mod_alias-redirects in directory context are only triggered by per-directory configuration, which would be skipped in that case. The only issue in mod_rewrite could be the Type/Handler forcing which occurs also in fixup, where is the right place for such things (to force something). nd
Re: FileSystem v.s. Other Resources [was configurable Location?]
William A. Rowe, Jr. wrote:
At 04:34 PM 2/6/2004, [EMAIL PROTECTED] wrote:
Joshua Slive wrote:
And perhaps I'm going way off in left field here, but why should this be
user-configurable at all? Shouldn't the (for example) server-status
handler know itself that it is a virtual handler, and therefore indicate
that the directory-walk should be skipped?
I like it! assuming we tweak it to be the module's responsibility to declare this property, vs. strictly the handler's (too late)
Modules can do that today with some very trivial code...
I think I see a problem. No doubt it could be made to work with a simple tweak.
SetHandler in the location container sets the handler field in the core dir
conf during location walk.
map_to_storage runs, but r-handler hasn't been updated due to the SetHandler
yet. virtual_map_location doesn't know that it owns this URI.
An unnecessary directory walk happens.
core_override_type runs during the fixups phase and sets r-handler to the
handler specified in the Location block from the handler field in the core
dir config. It's too late to affect the directory walk.
Another nit is that every module with virtual content would repeat essentially
the same code, other than checking the unique handler name of course, and add to
mainline path length on every request by increasing the number of map_to_storage
routines. We would depend on each module zapping r-filename. This doesn't
bother me for 2.0 if we can save a directory walk.
Greg
static int virtual_map_location(request_rec *r)
{
int access_status;
/* Test that SetHandler forced to this content
* otherwise skip it.
*/
if (strncmp(r-handler, info-handler) != 0)
return DECLINED;
/* this info-handler won't deal with the filename
* so null the filename to ensure no file is served.
*/
r-filename = ;
/* Don't let the core map_to_storage hook handle this,
* We don't need directory/file_walk. See mod_proxy.c
* if our own custom Restrictions blocks were needed.
*/
return OK;
}
static void register_hooks(apr_pool_t *p)
{
...
/* Suppress default dir_walk behavior, our module's
* content is virtual.
*/
ap_hook_map_to_storage(virtual_map_location, NULL, NULL, APR_HOOK_MIDDLE);
}
Re: FileSystem v.s. Other Resources [was configurable Location?]
At 01:37 PM 2/9/2004, André Malo wrote: * William A. Rowe, Jr. [EMAIL PROTECTED] wrote: You hit the nail on the head - Alias (and mod_rewrite) cause us the greatest grief in fixing this set of issues. *IF* they all are parsed in the translate name phase we are fine, since map_to_storage is run after translate_name is done. If they are not handled up front we have problems (some translations can occur in the fixup phase, IIRC, which is wrong IMHO.) No. Alias translations and RewriteRules in servercontext are handled in the translate_name hook. Fixup rewrites (-internal redirects) or mod_alias-redirects in directory context are only triggered by per-directory configuration, which would be skipped in that case. Ahhh of course. The only issue in mod_rewrite could be the Type/Handler forcing which occurs also in fixup, where is the right place for such things (to force something). Don't they result in internal redirects? In that case there is nothing wrong when the internal redirect starts a new request cycle with the modified uri/filename. Bill
Re: FileSystem v.s. Other Resources [was configurable Location?]
* William A. Rowe, Jr. [EMAIL PROTECTED] wrote: The only issue in mod_rewrite could be the Type/Handler forcing which occurs also in fixup, where is the right place for such things (to force something). Don't they result in internal redirects? In that case there is nothing wrong when the internal redirect starts a new request cycle with the modified uri/filename. Type or handler forcing just does ap_set_content_type or sets r-handler respectively. (like ForceType does). nd
Re: FileSystem v.s. Other Resources [was configurable Location?]
At 02:11 PM 2/9/2004, [EMAIL PROTECTED] wrote: William A. Rowe, Jr. wrote: At 04:34 PM 2/6/2004, [EMAIL PROTECTED] wrote: Joshua Slive wrote: And perhaps I'm going way off in left field here, but why should this be user-configurable at all? Shouldn't the (for example) server-status handler know itself that it is a virtual handler, and therefore indicate that the directory-walk should be skipped? I like it! assuming we tweak it to be the module's responsibility to declare this property, vs. strictly the handler's (too late) Modules can do that today with some very trivial code... I think I see a problem. No doubt it could be made to work with a simple tweak. SetHandler in the location container sets the handler field in the core dir conf during location walk. ??? I'm not seeing that, the dir/location walk doesn't set up r-handler at all. You can use the SetHandler directive in location or dir contexts. I suppose if it is set in a location, we set r-handler it in the translate_name phase. Even if it's overridden in a Directory block, the Location block it points at would still override it. So this change seems harmless. map_to_storage runs, but r-handler hasn't been updated due to the SetHandler yet. virtual_map_location doesn't know that it owns this URI. An unnecessary directory walk happens. Yes - the module needs to determine ownership in the translate_name phase. core_override_type runs during the fixups phase and sets r-handler to the handler specified in the Location block from the handler field in the core dir config. It's too late to affect the directory walk. Another nit is that every module with virtual content would repeat essentially the same code, other than checking the unique handler name of course, and add to mainline path length on every request by increasing the number of map_to_storage routines. We would depend on each module zapping r-filename. This doesn't bother me for 2.0 if we can save a directory walk. I guess I'm really concerned, after some of the problems and confusion this thread has revealed, about changing the behavior at all in 2.0. If we can get our own core not a file handlers to behave nicely (by fixing r-handler back in the translate_name phase) I would feel better about a minor change. That's a pretty restrictive change that would only affect, mod_info, mod_status etc. Other authors could do likewise if they understand the ramifications. But it seems after reading all of the comments, we aught to make some appropriate design changes in 2.2 so this entire solution is robust without breaking existing modules and possibly introducing security issues. After all, Apache 2.0 should be stable, and we should spend some time breaking the server in 2.1-dev with the eye to releasing a faster, more optimized 2.2. And yup - *every* handler author aught to make the appropriate change, because only they understand what's up with their content, and if it should be walked by dir and file sections, or not. We know mod_info and mod_status don't need to be, so we can code that ourselves. Bill
Re: FileSystem v.s. Other Resources [was configurable Location?]
William A. Rowe, Jr. wrote:
At 02:11 PM 2/9/2004, [EMAIL PROTECTED] wrote:
William A. Rowe, Jr. wrote:
Modules can do that today with some very trivial code...
I think I see a problem. No doubt it could be made to work with a simple tweak.
SetHandler in the location container sets the handler field in the core dir conf during location walk.
??? I'm not seeing that, the dir/location walk doesn't set up r-handler at all.
right. From merge_core_dir_configs():
if (new-handler) {
conf-handler = new-handler;
}
...so currently it only lives in config land until the fixups phase if I'm
reading the code correctly. That's the problem.
You can use the SetHandler directive in location or dir contexts. I suppose
if it is set in a location, we set r-handler it in the translate_name phase.
sounds good
Even if it's overridden in a Directory block, the Location block it
points at would still override it. So this change seems harmless.
I'll buy it
map_to_storage runs, but r-handler hasn't been updated due to the SetHandler yet. virtual_map_location doesn't know that it owns this URI.
An unnecessary directory walk happens.
Yes - the module needs to determine ownership in the translate_name phase.
core_override_type runs during the fixups phase and sets r-handler to the handler specified in the Location block from the handler field in the core dir config. It's too late to affect the directory walk.
Another nit is that every module with virtual content would repeat essentially the same code, other than checking the unique handler name of course, and add to mainline path length on every request by increasing the number of map_to_storage routines. We would depend on each module zapping r-filename. This doesn't bother me for 2.0 if we can save a directory walk.
I guess I'm really concerned, after some of the problems and confusion this
thread has revealed, about changing the behavior at all in 2.0.
Maybe you're right.
If we can get
our own core not a file handlers to behave nicely (by fixing r-handler back
in the translate_name phase) I would feel better about a minor change. That's
a pretty restrictive change that would only affect, mod_info, mod_status etc.
Other authors could do likewise if they understand the ramifications.
The performance of mod_info and mod_status doesn't bother me. Seeing the system
calls they generate in a trace does, but I'll get over it. Isn't mod_dav
virtual content though from the core's point of view (pls excuse my ignorance)?
mod_proxy is, but it already has working special case code scattered
throughout the core. (hmmm, might be some hints as to where the hits are for a
more general solution)
But it seems after reading all of the comments, we aught to make some
appropriate design changes in 2.2 so this entire solution is robust without
breaking existing modules and possibly introducing security issues. After
all, Apache 2.0 should be stable, and we should spend some time breaking
the server in 2.1-dev with the eye to releasing a faster, more optimized 2.2.
Amen
And yup - *every* handler author aught to make the appropriate change,
because only they understand what's up with their content, and if it should
be walked by dir and file sections, or not. We know mod_info and mod_status
don't need to be, so we can code that ourselves.
right
Greg
Re: FileSystem v.s. Other Resources [was configurable Location?]
At 04:34 PM 2/6/2004, [EMAIL PROTECTED] wrote:
Joshua Slive wrote:
I do, however, agree that doing a directory-walk on virtual resources is
not nice. But my opinion is that virtualness is a property of the
resource, and hence should be designated when selecting the resource.
That is why I suggested changing SetHandler rather than Location.
And perhaps I'm going way off in left field here, but why should this be
user-configurable at all? Shouldn't the (for example) server-status
handler know itself that it is a virtual handler, and therefore indicate
that the directory-walk should be skipped?
I like it! assuming we tweak it to be the module's responsibility to declare this
property, vs. strictly the handler's (too late).
Modules can do that today with some very trivial code...
static int virtual_map_location(request_rec *r)
{
int access_status;
/* Test that SetHandler forced to this content
* otherwise skip it.
*/
if (strncmp(r-handler, info-handler) != 0)
return DECLINED;
/* this info-handler won't deal with the filename
* so null the filename to ensure no file is served.
*/
r-filename = ;
/* Don't let the core map_to_storage hook handle this,
* We don't need directory/file_walk. See mod_proxy.c
* if our own custom Restrictions blocks were needed.
*/
return OK;
}
static void register_hooks(apr_pool_t *p)
{
...
/* Suppress default dir_walk behavior, our module's
* content is virtual.
*/
ap_hook_map_to_storage(virtual_map_location, NULL, NULL, APR_HOOK_MIDDLE);
}
Re: FileSystem v.s. Other Resources [was configurable Location?]
William A. Rowe, Jr. wrote: At 12:17 PM 2/5/2004, Joshua Slive wrote: I do, however, agree that doing a directory-walk on virtual resources is not nice. But my opinion is that virtualness is a property of the resource, and hence should be designated when selecting the resource. That is why I suggested changing SetHandler rather than Location. And perhaps I'm going way off in left field here, but why should this be user-configurable at all? Shouldn't the (for example) server-status handler know itself that it is a virtual handler, and therefore indicate that the directory-walk should be skipped? For example, yes. But on the other hand, what prevents someone from removing the server-status handler in the fixups phase and tricking us into serving a file. sounds like we're getting into defense mechanisms against hypothetical malicious modules... a loosing battle IMO I'd like to see the default handler entirely detached from the request if the dir_walk code has been bypassed. if it's zero cost/dirt cheap in mainline code, I don't mind. Greg
Re: FileSystem v.s. Other Resources [was configurable Location?]
At 09:47 AM 2/6/2004, [EMAIL PROTECTED] wrote: William A. Rowe, Jr. wrote: At 12:17 PM 2/5/2004, Joshua Slive wrote: I do, however, agree that doing a directory-walk on virtual resources is not nice. But my opinion is that virtualness is a property of the resource, and hence should be designated when selecting the resource. That is why I suggested changing SetHandler rather than Location. And perhaps I'm going way off in left field here, but why should this be user-configurable at all? Shouldn't the (for example) server-status handler know itself that it is a virtual handler, and therefore indicate that the directory-walk should be skipped? For example, yes. But on the other hand, what prevents someone from removing the server-status handler in the fixups phase and tricking us into serving a file. sounds like we're getting into defense mechanisms against hypothetical malicious modules... a loosing battle IMO Malice? Nah, defending against badly written or inherently insecure modules. But much more to the discussion at hand... ...will administrators grok when Location virtual /foo is a safe bet? Will it proliferate in example configurations in the wild, in unsafe ways? I think we can all presume it will. Look at our frustrations with users that have essentially open proxy configs, which they set up by following examples that proliferate out there. Just playing defense here. Bill
Re: FileSystem v.s. Other Resources [was configurable Location?]
Greg Marr wrote: I'm only changing Location ... Directory is unaffected. Well, that's not entirely true. The Directory is affected indirectly, because it no longer applies. The behavior currently is: it applies to everything it matches. This would change it to: it applies to everything it matches unless it also matches a Location, in which case it doesn't apply. OK, got it. Thanks for the clear explanation. This could possibly be handled by checking for overlaps between Location and Directory at config time, especially if they are sorted in least-to-most specific order as wrowe suggests. But my current favorite idea is Joshua's suggestion for the module to declare that the content is virtual, pure and simple. In that case, any such overlaps are an admin screwup IMO. GET /server-status/index.html doesn't make any sense to me even if such a directory exists. Greg
Re: FileSystem v.s. Other Resources [was configurable Location?]
Let's do this in 2.1 by splitting out the file system, and if the filesystem module isn't handling a request, it won't be serving content but also won't be invoking the directory walk or stat-ing files. this all sounds kinda interesting, and similar to the way auth has been set up in 2.1 - more directives and modules, but more flexibility and power. Oh last observation - it should become (in 2.1) nearly impossible for folks to just bork around with the contents of r-filename and r-finfo, first by stripping them from the request rec, and second by providing an API to the filesystem module that lets another module link into another file. That API would prevent module authors from bypassing the filesystem module's internal security. this has come up before, and I'd love to see an API that prevents accidental disagreement between r-filename and r-finfo (for one). IIRC, there was supposed to be some discussion at the hackathon about this, but it sounds like I didn't miss it :) --Geoff
Re: FileSystem v.s. Other Resources [was configurable Location?]
Thanks for the feedback, Will. William A. Rowe, Jr. wrote: At 03:39 PM 2/4/2004, [EMAIL PROTECTED] wrote: But then if I play devil's advocate, someone could see the new directive and turn it on when it's not appropriate and cause Bad Things to happen. Mainly I'm looking for comments on whether this should be configurable or not. Yes, I'm one who will agree with your devil's position :) I know the problem you are trying to solve, great! and changing the behavior of Directory isn't quite the solution... I'm only changing Location ... Directory is unaffected. Issue 1: Walking the filesystem (the stat aspect of Directory and File) for something that doesn't live there is stupid :) We agree, and this must be fixed in 2.1 cool! Issue 2: Matching directories when something is outside of the filesystem is simply wasteful, and here too we agree. The problem is that you want to add layers of additional directives, which would change the behavior of Directory or Location , only Location , in a way that IMO is consistent with the existing documentation, but not the existing code of course. in ways that would allow the user to seriously compromise what they had explicitly configured for security. This is unwise. Let's look at the two issues above; Effect/Issue 1: Bypassing the filesystem canonicalization would be very bad on certain platforms such as windows, depending on case sensitivity, etc. It would also bypass *user configured* options such as avoiding symlinks. only for Location If the admin is telling us the content is outside the filesystem, I don't think filesystem canonicalization or symlinks are relevant. Effect/Issue 2: Bypassing the directory block walk would ignore the very Directory sections that the user explicitly put in their config. This too is bad. This is only for Location . There is no change to the behavior of Directory . There is only one way that sounds kinda religious...in software there are generally a number of ways to solve a problem :-) to avoid these consequences, and still save ourselves the pain of the filesystem walk and Directory block handling... I'm only changing Location === Break out file handling as a separate component === I've proposed in the past the simple directive FilesystemMount /path/to/files At first glance it looks like a key difference is that you are approaching it by saying that it's OK to have Location map to a real filesystem and providing config to explicitly specify the mapping. I'm advocating going the other way and allowing Location to be treated as if it is really outside of the filesystem. A complication to treating Location as if it's outside the filesystem is the doc that states Location / is a quick way to apply a config to the whole server. It will indeed do that today, unfortunately. I believe another quick way to get the same result is to put your config directives outside of any container. If that's true, I would prefer to see that doc for Location / removed. If the change to the behavior of Location is configurable, this is not an issue. If it's not configurable, then we probably should treat Location / as if it's in the filesystem - a special case for backwards compatibility. I'd like to study your comments more after my coffee kicks in :-) Obviously you've given a lot of thought to this issue so I want to see if I can understand where you're coming from. Greg
Re: FileSystem v.s. Other Resources [was configurable Location?]
I just want to add a couple notes here on what I see as user-expectations. The problem is that you want to add layers of additional directives, which would change the behavior of Directory or Location , only Location , in a way that IMO is consistent with the existing documentation, but not the existing code of course. [...] A complication to treating Location as if it's outside the filesystem is the doc that states Location / is a quick way to apply a config to the whole server. It will indeed do that today, unfortunately. I believe another quick way to get the same result is to put your config directives outside of any container. If that's true, I would prefer to see that doc for Location / removed. If the change to the behavior of Location is configurable, this is not an issue. If it's not configurable, then we probably should treat Location / as if it's in the filesystem - a special case for backwards compatibility. I wrote the particular doc you quoted earlier, and I meant it more as a recommendation (for security reasons) than a documentation of how apache works. The true description of how apache is supposed to function is here: http://httpd.apache.org/docs-2.0/sections.html (But it is a little vauge, because my knowledge of the internals of configuration merging was not sufficient to clarify it the last time I went through.) Many people do, in fact, use Location for filesystem-related things. See the examples in the mod_dav docs for one: http://httpd.apache.org/docs-2.0/mod/mod_dav.html Although I don't think this is esthetically correct, it is not particularly dangerous. Using Location to provide *extra* access is okay, because if you circumvent the Location using filesystem tricks, you just lose the access. Using Location to *restrict* access is not safe, because the restrictions may be circumvented. The other important thing to note is that Location is often used for a reason that is particular to apache configuration merging: those sections are evaluated last. That means that Location / Order deny,allow Deny from all Allow from local.net /Location has a very different effect than the same directives placed in Directory / because it will be evaluated after all Directory and Files sections which could potentially override directives. I'm not saying that this is a strictly correct use of Location, but it is very useful and there is no other easy way to accomplish this apply this directive after everything else effect. I do, however, agree that doing a directory-walk on virtual resources is not nice. But my opinion is that virtualness is a property of the resource, and hence should be designated when selecting the resource. That is why I suggested changing SetHandler rather than Location. Joshua.
Re: FileSystem v.s. Other Resources [was configurable Location?]
At 10:22 AM 2/5/2004, [EMAIL PROTECTED] wrote: Thanks for the feedback, Will. William A. Rowe, Jr. wrote: At 03:39 PM 2/4/2004, [EMAIL PROTECTED] wrote: But then if I play devil's advocate, someone could see the new directive and turn it on when it's not appropriate and cause Bad Things to happen. Mainly I'm looking for comments on whether this should be configurable or not. Yes, I'm one who will agree with your devil's position :) I know the problem you are trying to solve, and changing the behavior of Directory isn't quite the solution... I'm only changing Location ... Directory is unaffected. Well, that's not entirely true. The Directory is affected indirectly, because it no longer applies. The behavior currently is: it applies to everything it matches. This would change it to: it applies to everything it matches unless it also matches a Location, in which case it doesn't apply.
Re: FileSystem v.s. Other Resources [was configurable Location?]
I do, however, agree that doing a directory-walk on virtual resources is not nice. But my opinion is that virtualness is a property of the resource, and hence should be designated when selecting the resource. That is why I suggested changing SetHandler rather than Location. And perhaps I'm going way off in left field here, but why should this be user-configurable at all? Shouldn't the (for example) server-status handler know itself that it is a virtual handler, and therefore indicate that the directory-walk should be skipped? Joshua.
Re: FileSystem v.s. Other Resources [was configurable Location?]
At 10:43 AM 2/5/2004, Greg Marr wrote: At 10:22 AM 2/5/2004, [EMAIL PROTECTED] wrote: Thanks for the feedback, Will. William A. Rowe, Jr. wrote: At 03:39 PM 2/4/2004, [EMAIL PROTECTED] wrote: But then if I play devil's advocate, someone could see the new directive and turn it on when it's not appropriate and cause Bad Things to happen. Mainly I'm looking for comments on whether this should be configurable or not. Yes, I'm one who will agree with your devil's position :) I know the problem you are trying to solve, and changing the behavior of Directory isn't quite the solution... I'm only changing Location ... Directory is unaffected. Well, that's not entirely true. The Directory is affected indirectly, because it no longer applies. The behavior currently is: it applies to everything it matches. This would change it to: it applies to everything it matches unless it also matches a Location, in which case it doesn't apply. Right - I understand. I'm saying (as you pointed out) that it's too dangerous :) Defies the principle of least surprise. They configure Dir blocks only to have them ignored. The only safe way to do what you are suggesting, is to yank the default file handler from under the request anytime the user has overridden the Dir checks. Simplest way to do that is bind them together as a filesystem module. Oh, the same problems (of yanking Dir out from under the user) would be true of the CGI handler, as well as third party modules that do things with the filesystem. It would be important that they could explicitly enable the module to add Dir handling. Which brings us back to square one. I'd actually argued a long time ago that a module that handles requests outside of the filesystem should be responsible for bypassing the Dir handler itself. For example, mod_proxy satisfies the map_to_storage hook on it's own, therefore never hitting Dir . Bill
Re: FileSystem v.s. Other Resources [was configurable Location?]
At 09:22 AM 2/5/2004, [EMAIL PROTECTED] wrote: Effect/Issue 1: Bypassing the filesystem canonicalization would be very bad on certain platforms such as windows, depending on case sensitivity, etc. It would also bypass *user configured* options such as avoiding symlinks. only for Location If the admin is telling us the content is outside the filesystem, I don't think filesystem canonicalization or symlinks are relevant. But the question is how the admin communicates that to us. You are suggesting we allow Location virtual /not/a/file to say it's outside of the file system. *However* are you preventing a file from being served? Guess that is the gist of my concern. Effect/Issue 2: Bypassing the directory block walk would ignore the very Directory sections that the user explicitly put in their config. This too is bad. This is only for Location . There is no change to the behavior of Directory . to avoid these consequences, and still save ourselves the pain of the filesystem walk and Directory block handling... I'm only changing Location Wait - so you say. But the effect is to disable Directory block handling, am I confused on this point??? If you ignore valid Directory blocks because somewhere else the admin says Wait - this location is virtual! then are you also preventing those files from being served? If not this is what I suggest is a violation of the principal of least surprise. === Break out file handling as a separate component === I've proposed in the past the simple directive FilesystemMount /path/to/files At first glance it looks like a key difference is that you are approaching it by saying that it's OK to have Location map to a real filesystem and providing config to explicitly specify the mapping. I'm advocating going the other way and allowing Location to be treated as if it is really outside of the filesystem. Here's what I'm suggesting; 1. Everything in Apache is at some location. Location is a predictable factor because every URI becomes a location. A Location block will always override all other configurations, location blocks are always applied as least-to-most significant if they are absolute, and then pattern matching location blocks are applied after that. 2. Some things in Apache are in the Filesystem, some are in JK, some are in an application (a cgi with PATH_INFO). These aren't always 1:1 to the filesystem, except in the case of filesystem-based modules such as our default handler and the current cgi modules. 3. If something is in the Filesystem, then Directory and Files should be applied. If something is not in the filesystem, than no, it should not, and that other backend (e.g. proxy) should have it's own container blocks to deal with the resource (e.g. Proxy blocks.) Overloading can become a very confusing a dangerous game for admins to properly configure their servers. 4. If something is not in the Filesystem, there should be no way that our own CGI handler or default file handler should be allowed to handle the request. If Apache 5. Whatever resource (Filesystem or what have you) that claims to own the resource for config block processing should be the only available handler to deal with the resource on the back end. Jumping from one resource context in the configuration phase to another resource resource in the handler phase is very dangerous. Beyond what I've detailed above, I'm flexible about how it works/what the various directives are, etc. Bill
FileSystem v.s. Other Resources [was configurable Location?]
At 03:39 PM 2/4/2004, [EMAIL PROTECTED] wrote: But then if I play devil's advocate, someone could see the new directive and turn it on when it's not appropriate and cause Bad Things to happen. Mainly I'm looking for comments on whether this should be configurable or not. Yes, I'm one who will agree with your devil's position :) I know the problem you are trying to solve, and changing the behavior of Directory isn't quite the solution... Issue 1: Walking the filesystem (the stat aspect of Directory and File) for something that doesn't live there is stupid :) We agree, and this must be fixed in 2.1 Issue 2: Matching directories when something is outside of the filesystem is simply wasteful, and here too we agree. The problem is that you want to add layers of additional directives, which would change the behavior of Directory or Location , in ways that would allow the user to seriously compromise what they had explicitly configured for security. This is unwise. Let's look at the two issues above; Effect/Issue 1: Bypassing the filesystem canonicalization would be very bad on certain platforms such as windows, depending on case sensitivity, etc. It would also bypass *user configured* options such as avoiding symlinks. Effect/Issue 2: Bypassing the directory block walk would ignore the very Directory sections that the user explicitly put in their config. This too is bad. There is only one way to avoid these consequences, and still save ourselves the pain of the filesystem walk and Directory block handling... === Break out file handling as a separate component === I've proposed in the past the simple directive FilesystemMount /path/to/files Consider the existing directive, DocumentRoot, as shorthand for; Location / FilesystemMount /old/root/ /Location Also consider the Alias /foo /path/to/bar directive as shorthand for; Location /foo FilesystemMount /path/to/bar /Location But wait, there is more :) We have a default handler. Simply modify it to be the default_file_handler. Insert it ONLY if the appropriate FilesystemMount directive exists in the particular location block. What happens to requests that don't have a FilesystemMount block, and don't have another handler (e.g. SetHandler info-handler or whatever)? These requests would error out with a 500 - there is no way for the server to handle them. What happens to our old default handler (inserted at all times?) It becomes a simple error-500 handler if nothing else intercepts the request. Note that the exact semantics could change (my naming isn't all that great either) - but the key detail is that mount directives within a given Location are much safer and easier to follow than arbitrary Mount src dst directives which may be order and/or length dependent, creating confusion for the user. So in short, veto on the original proposal to give users gun to point at foot. +1 on the concepts that we quit interrogating the filesystem when the user wants to serve non-files, and quit walking directory blocks for things that do not live in directories. Let's do this in 2.1 by splitting out the file system, and if the filesystem module isn't handling a request, it won't be serving content but also won't be invoking the directory walk or stat-ing files. Oh last observation - it should become (in 2.1) nearly impossible for folks to just bork around with the contents of r-filename and r-finfo, first by stripping them from the request rec, and second by providing an API to the filesystem module that lets another module link into another file. That API would prevent module authors from bypassing the filesystem module's internal security. If they want different behavior, they can plug in their own backend handler to deal with the whole request process. Bill
