Re: 3.0, the 2011 thread.
On Wed, Jun 15, 2011 at 3:01 PM, Paul Querna p...@querna.org wrote: I think we have all joked on and off about 3.0 for... well about 8 years now. At least! I think there are exciting things happening in C however. I love C, but unless we can come up with something radical, it's hard to see a way out of the prison it creates. That realisation led me to hacking mostly on functional-oriented servers. I'll try to explain why - in case any of those thoughts are useful here too :) I like the things you've pointed out, but they seem relatively cosmetic. Things like the parser, async, event and portability frameworks are really cool - but hardly fundamental. Anyone could use those, in any language - it's not a real leap in the field. Similarly, SPDY, websockets, COMET and so on are ultra-cool - but are still potential bolt-ons to almost any kind of webserver. It sucks that we don't do them well, but doing them better won't fundamentally change the market or the pressures on adoption. Today webservers are almost entirely network I/O bound - disk seek and CPU speeds are pretty great these days, way faster than is really neccessary. In a properly architected set-up, end-user delay is really about the limitations of TCP. You can multiplex and use keepalives as much as you want, you'll eventually realise that the size of the world and speed of light mean that this inevitably ends up being slow without a lot of distributed endpoints. But we have some cool secret sauce to help fix that. I think the best architectural thing about Apache is buckets and brigades. Using a list structure to represent portions of differently-generated content like that is great. Imagine how much better wordpress would run if PHP represented the php scripts as a some dynamic buckets intermingled with some static file io buckets (and even repeated when in loops). There'd be a lot less data to copy around. Now imagine a backend that could identify the dynamic buckets and, by forbidding side effects, parallellise work on them - a bucket as a message in message-passing system of co-routines, for example. Imagine that in turn feeding into a set of co-routine filters. That's fundamentally different - it parallelises content generation, but it's really really hard to do in C. Then next, imagine a backend that could identify the static buckets and re-order them so that they come first - it could understand things like XML and Javascript and intelligently front-load your transfer so that the content we have ready goes first, while the dynamic stuff is being built. It's a real layer-8-aware scheduler and content re-compiler. Again it's really really hard to do in C - but imagine the benefits of a server layer that really understood how to model and re-order content. These are the kinds of transform that make a webservers job as optimal as it can be. Network data is the most expensive part of any modern web application, in terms of both time and money, so the ecosystem faces huge economic pressure to make these as optimal as possible over time. Things like SPDY are just the first generation. It'd be cool if Apache 3.0 could do those things - we have some great building blocks and experience - but it feels like a language with support for first-order functions and co-routines would be better at it. Again, I'm just thinking out loud :) -- Colm
Re: Listen syntax RFE
On Mon, Feb 8, 2010 at 3:55 PM, Mark Watts m.wa...@eris.qinetiq.com wrote: I often need to configure httpd to Listen on more than one IP an a range, and more than one port. Are these being used with IP-based virtual hosts? It might be easier to create an option that allows each such vhost to be an implicit Listen .. as that avoids duplication of config. -- Colm
Re: [VOTE] 1.3.42 release candidate
Just a reminder; Intention is to release 1.3.42 on Tuesday. If anyone has strong feelings, make them known :-) Nóirín has kindly translated the announcement into English; http://people.apache.org/~noirin/Announcement1.3.txt 2010/1/27 Colm MacCárthaigh c...@allcosts.net: 2010/1/27 Colm MacCárthaigh c...@allcosts.net: On Wed, Jan 27, 2010 at 10:15 PM, Jeff Trawick traw...@gmail.com wrote: 2010/1/27 Colm MacCárthaigh c...@allcosts.net: I still can't figure out where the repos for dev/dist or dist/ are hosted. None of the docs seem up to date. https://dist.apache.org/repos/dist/release/httpd/ https://dist.apache.org/repos/dist/dev/httpd/ Cool, thanks! I'll update the docs :-) And now up at ... http://httpd.apache.org/dev/dist/ -- Colm -- Colm -- Colm
Re: [VOTE] 1.3.42 release candidate
On Wed, Jan 27, 2010 at 3:40 AM, Sander Temme scte...@apache.org wrote: Why don't we do this: roll the same tag with the docs fixes as you indicate immediately above; sign, hash and put them up on dev/dist. Then call 72 hours. We have a quick look to see if smoke emerges and, if not, we can release early next week. That would also give us the opportunity to align PRC. Thoughts? Sounds good to me, I've rerolled, and uploaded to; http://people.apache.org/~colm/1.3.42/ I still can't figure out where the repos for dev/dist or dist/ are hosted. None of the docs seem up to date. -- Colm
Re: [VOTE] 1.3.42 release candidate
2010/1/27 Colm MacCárthaigh c...@allcosts.net: On Wed, Jan 27, 2010 at 10:15 PM, Jeff Trawick traw...@gmail.com wrote: 2010/1/27 Colm MacCárthaigh c...@allcosts.net: I still can't figure out where the repos for dev/dist or dist/ are hosted. None of the docs seem up to date. https://dist.apache.org/repos/dist/release/httpd/ https://dist.apache.org/repos/dist/dev/httpd/ Cool, thanks! I'll update the docs :-) And now up at ... http://httpd.apache.org/dev/dist/ -- Colm -- Colm
Re: [VOTE] 1.3.42 release candidate
On Wed, Jan 27, 2010 at 12:43 AM, Sander Temme scte...@apache.org wrote: On Jan 8, 2010, at 4:29 AM, Colm MacCárthaigh wrote: There is a 1.3.42 release candidate for testing, and voting, at; What happened to this, besides making Slashdot? I transited the atlantic twice. I actually wasted about 2 days and 7 EC2 instances trying to document how many build problems there were on modern linux distros due to the glibc/dash problems ... to try and come up with a coherent here's how to build, run, and test ... but it's a complete mess. There are technically enough binding votes for release now, though there is still the outstanding with the bundled docs tree (which ironically turned out to be due to my using dash for testing!). Unless there are any vetoes in the next 2 days, I'd be inclined to release as-is, with the docs tree rerolled to fix includes. It is *definitely* worth never making another release again imo, patches are far less burden than this show! -- Colm
Re: Apache devs attending FOSDEM 2010?
On Mon, Jan 11, 2010 at 3:02 PM, Martin Langhoff martin.langh...@gmail.com wrote: are any developers of apache (core, modules) coming to FOSDEM in Brussels? I'll probably be there on the 7th. Any year I've gone there's usually been a few httpd people around. -- Colm
Re: [VOTE] 1.3.42 release candidate
On Sat, Jan 9, 2010 at 11:48 AM, Rainer Jung rainer.j...@kippdata.de wrote: There's one other manual related thing I wasn't thinking enough about before writing my last mail: the 1.3.42 manual contains SSI #include virtual statements, the 1.3.41 version not. I (wildly) guess the script expand.pl is usually used during releasing to fix the manual, so that the pages caqn also be read without running them though the web server. At least the contents of expand.pl look like it recursively goes through a directory and resolves all #include virtual in the files. I ran expand.pl, I'll look into what the hell it's doing :-) -- Colm
[VOTE] 1.3.42 release candidate
There is a 1.3.42 release candidate for testing, and voting, at; http://people.apache.org/~colm/1.3.42/ As per the changes, there are two updates; *) SECURITY: CVE-2010-0010 (cve.mitre.org) mod_proxy: Prevent chunk-size integer overflow on platforms where sizeof(int) sizeof(long). Reported by Adam Zabrocki. [Colm MacCárthaigh *) Protect logresolve from mismanaged DNS records that return blank/null hostnames. [Jim Jagielski] Notes; this is intended as the final release of Apache httpd 1.3, which has reached end of life. Security updates may continue to be provided by another means (see the CHANGES file for details). Apache httpd 1.3's ./configure script does not work with some versions of dash. Please change the hash-bang line to execute a bourne-compatible shell, such as /bin/bash on platforms affected. Many thanks in advance for your help and testing. -- Colm
Re: [VOTE] 1.3.42 release candidate
On Fri, Jan 8, 2010 at 5:44 PM, Sander Temme scte...@apache.org wrote: On Jan 8, 2010, at 9:41 AM, Sander Temme wrote: On Jan 8, 2010, at 4:29 AM, Colm MacCárthaigh wrote: There is a 1.3.42 release candidate for testing, and voting, at; http://people.apache.org/~colm/1.3.42/ Not seeing gpg sigs or md5s on the tarballs. Didn't we used to do that back then? Never mind, two sips of coffee later I found the .asc files. Maybe by the end of my cup I'll see hashes. I've uploaded hashes now too :-) -- Colm
Re: [VOTE] 1.3.42 release candidate
On Fri, Jan 8, 2010 at 7:50 PM, Brian Havard brian.hav...@gmail.com wrote: Colm MacCárthaigh wrote: There is a 1.3.42 release candidate for testing, and voting, at; http://people.apache.org/~colm/1.3.42/ I just tried building on a fairly stock Ubuntu Karmic system and ran into this compile error: gcc -c -I../os/unix -I../include -DLINUX=22 -DHAVE_SET_DUMPABLE -DUSE_HSREGEX -DNO_DL_NEEDED `../apaci` htpasswd.c htpasswd.c:101: error: conflicting types for ‘getline’ /usr/include/stdio.h:651: note: previous declaration of ‘getline’ was here So getline() is a system provided function here. I used the top level configure with /bin/sh changed to /bin/bash as mentioned elsewhere, no options. On fixing this I found that htdigest.c and logrotate.c have the same problem. The obvious fix would be to namespace protect these getline implementations, Thanks for the report - disinclined to fix anything that isn't a regression from 1.3.41 though. This is a security release, for upgraders only - so presumably people who've managed to work around these problems. Making life easy for new users is a deliberate non-goal :-) 2.2 is much much better for new users. -- Colm
Re: Should ap_get_server_name() behave like get_server_name_for_url() for literal IPv6 addresses?
On Tue, Dec 22, 2009 at 9:54 PM, Stefan Fritsch s...@sfritsch.de wrote: mod_rewrite incorrectly omits the brackets around literal IPv6 addresses in redirects. Similar issues have been fixed at other places in the code already. In server/core.c there is get_server_name_for_url(), a wrapper around ap_get_server_name, which adds the brackets. IPv6 addresses do have brackets in URLs, because colon is a port separator too, ... mod_proxy_http: via header (should have brackets) ssl_engine_kernel.c: error message url (should have brackets) mod_rewrite: redirects (should have brackets) logging (doesn't matter) SERVER_NAME variable Redirects should have brackets yes ... but I don't think that it's the job of the SERVER_NAME variable to anticipate potential use in a url, it's just a representation of an IPv6 address. Introducing braces would break reverse DNS lookups, for one thing. I guess many scripts don't have a special case for IPv6 literals. I think having the brackets in the SERVER_NAME variable would be better. Is there some spec how this should be? I think encoding as per RFC3513 Section 2.2 is best :-) -- Colm
Re: [VOTE] 1.3.42 release candidate
On Fri, Jan 8, 2010 at 6:58 PM, Rainer Jung rainer.j...@kippdata.de wrote: File htdocs/manual/misc/FAQ.html was previously 150KB and now is only 5KB. I didn't yet load it from the web server itself, but it seems some generation step was missing. The file installed by make install has the same problem. links -src now returns the gzipped body, as we have mod_deflate enabled these days. I've fixed up the other nits and rerolled, tarballs with new checksums/signatures in the same place now. I'll fix up the Announce language too, waiting for PRC input on that too. -- Colm
Re: [VOTE] Formal deprecation of 1.3.x branch
Based on this thread, I'll going to commit an EOL notice later today (1.3.x is still CTR) and language nits and so on can be fixed with patches/commits as appropriate, and if anyone feels strongly they can revert :-) On Wed, Jan 6, 2010 at 1:28 AM, Jorge Schrauwen jorge.schrau...@gmail.com wrote: On Wed, Jan 6, 2010 at 12:30 AM, William A. Rowe Jr. wr...@rowe-clan.net wrote: Res wrote: On Tue, 5 Jan 2010, Jorge Schrauwen wrote: On Mon, Jan 4, 2010 at 11:03 PM, Rich Bowen rbo...@rcbowen.com wrote: Speaking from the community that provides end-user support for these products, a big +1 on that proposal. Sadly, questions will keep on showing up for a long time :( I agree, however if it is EOL'd (yes Jeff, I agree this is likely more of an appropriate term :) ) you more or less can advise the requestor they are using a very old and unsupported version and that they should use the current stable version 2.2.xx These questions aren't a bother on an established 1.3 server, we all have to support legacy systems. What is frightening is the number of users who are clearly deploying their httpd server with 1.3, for the first time, and trying to learn that. Almost want to suggest to them that they go take a community college class on COBOL while they are at it, since they are clearly trying to improve their technical knowledge. Heh You could just put a anti troll like system where you ask a question only someone who's used 1.3 for a long time would know before letting them download. New users wouldn't be able to answer it and would be pointed to 2.2 branch ;) /sarc It's understandable that you can't just magicly update a legacy 1.3 to a 2.x series without breaking a whole lot of stuff. But's sadly as you say, a lot users seem to be new to httpd and still grab 1.3 for some insane reason! Maybe adding some form of color coding (for the non-colorblind) on the download page, green - current, orange/yellow - old stable, red - EOL Then again that would probably not have a big effect :( I also think punch cards COBAL course! That will teach em to proof read there code before hitting compile ;) /me off to more studying Jorge -- Colm
ITR 1.3.x Friday
I don't think there's much interest in merging any of the patches that are currently in the STATUS, so I'm intending to a roll a 1.3.x release Friday for testing/voting. With a release next week. Unfortunately testing 1.3.x is more painful than it used to be, and so far I haven't managed to get 1.3.x to pass a clean suite of tests on *any* platform - but I can at least make sure that there are no regressions. With some massaging I can get to 96% coverage on Linux and FreeBSD, and 92% coverage on Solaris. Another interesting problem is that 1.3.x's configure/apaci doesn't actually work with dash - so the tarball refuses to build out of the box on Ubuntu and Debian. Nice. -- Colm
Re: [VOTE] Formal deprecation of 1.3.x branch
On Tue, Jan 5, 2010 at 8:55 PM, Rich Bowen rbo...@rcbowen.com wrote: On Jan 5, 2010, at 15:31 , Jorge Schrauwen wrote: +1 (non-binding) There are still to many questions about the 1.3 branch on the support channels IMHO One hopes that a formal EOL statement will be the encouragement that most of these folks need to move into the new century. If one of the goals here is to reduce the support nuisance and help folk out, should we also ask the Apache PR team for help? I'm sure they'd be willing to help publicise the change After 15 years of community support ... mumble mumble ... marking version 1 of Apache as end of life with final release ...mumble mumble ... 10 years of successful Apache 2 ... mumble mumble .. please use 2.2.x . -- Colm
Re: ITR 1.3.x Friday
On Wed, Jan 6, 2010 at 11:28 AM, Graham Leggett minf...@sharp.fm wrote: It sounds to me like it may be a bit of a pandora's box making another release. Is it not possible to just make a patch available? It sure is a PITA, but I think it would be more professional to get a final release out there - if only because it's what upstream packagers and users are used to, many will have scripts and systems oriented around a tarball. We can give them one more, and a warning that it's the last - so they have some time to rewrite their systems. That said, if it turns out to be so much of a PITA that we can't muster 3 +1's for a release ... I'll just post a patch. -- Colm
Re: ITR 1.3.x Friday
On Wed, Jan 6, 2010 at 12:20 PM, Rainer Jung rainer.j...@kippdata.de wrote: So there's only one source update and the update of the mime.types. Is that worth a release? There's a pending security fix too that will be committed before the roll. -- Colm
[VOTE] Formal deprecation of 1.3.x branch
Observers of the commits list may have noticed some small cleanups to the 1.3.x branch earlier today. There are currently a number of several years-old backport/patch proposals in there too, including two marked as release show-stoppers (neither actually stopped the show, when last we had a release). I've reviewed and tested the relatively low-hanging fruit, and cleaned up the backport proposals to be functional patches. If you really really think any of those patches are important, now is a good time to review/test/vote! Because ... stealing an idea from wrowe@ ... how about we formally deprecate the 1.3.x branch? Make one more release, but attach a notice to the effect that it will be the final release, and that in future we'll be distributing security updates by other means :-) Patch; Index: README === --- README (revision 895672) +++ README (working copy) @@ -14,9 +14,16 @@ The Latest Version -- - Details of the latest version can be found on the Apache HTTP - server project page under http://httpd.apache.org/. + As of January 2010, support for version 1.3 of Apache has been + formally deprecated. Updates will be available as patches, from + http://www.apache.org/dist/httpd/patches/ + Users are encouraged to upgrade to version 2 of Apache httpd, + and information on upgrading is available at; + + http://httpd.apache.org/docs/2.0/upgrading.html + http://httpd.apache.org/docs/2.2/upgrading.html + Documentation - Index: src/CHANGES === --- src/CHANGES (revision 895795) +++ src/CHANGES (working copy) @@ -1,6 +1,22 @@ Changes with Apache 1.3.42 + *) INFORMATIONAL: Please note that this release is considered + the final release of the Apache 1.3.x branch. Future + updates will be distributed as patches, via + http://www.apache.org/dist/httpd/patches/ + + Apache 1.3.x users who wish to avail of security releases + are advised to use Apache 2 or higher. Please see; + + http://httpd.apache.org/docs/2.0/upgrading.html + http://httpd.apache.org/docs/2.2/upgrading.html + + for information on upgrading. On behalf of the Apache + httpd group; thank you to everyone who helped make + Apache 1.3.x the most successful, and most used, + webserver software on the planet. + Changes with Apache 1.3.41 *) SECURITY: CVE-2007-6388 (cve.mitre.org) -- Colm
Re: Adopting mod_remoteip to modules/metadata/ ?
On Wed, Apr 1, 2009 at 12:45 AM, William A. Rowe, Jr. wr...@rowe-clan.netwrote: I have essentially finished mod_remoteip at this point and am looking to find out the interest level of adopting this as a core module into trunk (modules/metadata/ appears to be the most appropriate target)? +1 :-) RemoteIPTrustedProxy 192.168.0. localhost/8 I don't think permitting hostname/number is a good idea, because a hostname can map to multiple IPs, and it gets confusing, it's non-standard :-) Right now the code just does a single lookup, and uses that - so where there are multiple A/ records we'll have random behaviour. I'm not sure that I think hard-coding RFC1918 addresses is a good idea. RFC3330 is more authoratitive for a start, and there's no correspending code for IPv6 (e.g. the documentation prefixes, link-local scope, site-local scope, and so on). Either way, I think it's inaccurate to assume that there is anything special about RFC1918 space, the user should be forced to configure those ranges. Many networks have differing levels of trust and overlapping usages for that space, and many others use globally-unique IPs in private contexts. Looks cool though. We might need to add some docs warning peolpe that the address might no longer match the address family of the actual socket. -- Colm
Re: Untouchable header fields ?
On Sat, Mar 28, 2009 at 2:27 PM, Paul Querna p...@querna.org wrote: I believe there have been bug reports/requests to allow changing the Server header, but no one has complained about Date: before. Heh, I made it modifiable, and provided a patch iirc , but noone else liked it :-) http://www.maths.tcd.ie/~dwmalone/p/login06.pdf for details on the use-case (it's a fun read). -- Colm
Re: [VOTE] Release Apache HTTP server 2.2.11
On Sat, Dec 6, 2008 at 4:30 PM, Ruediger Pluem [EMAIL PROTECTED] wrote: +/-1 [ ] Release httpd-2.2.11 as GA +1 , tested on EC2 with Ubuntu and Redhat :-) -- Colm
Adding purge/invalidation to mod_cache
For one reason or another, I need to be able to invalidate cache entries in some mod_cache caches. There's no good standard for this, WCIP/BEEP went nowhere afaict, but I want to keep things simple. The way Squid handles this is by implementing a non-standard PURGE HTTP method, so I've taken the same approach here - in a very basic patch. The patch actually works, you can call; PURGE /foo HTTP/1.1 Host: example.org Accept: foo/bar and it will actually go and remove the item from whatever cache its in, with the entity being negotiated as per the Accept headers. But before I move on to tidying it up to return better codes, work behind ACLs/auth, handle the uncached case better and add a PurgeEnable directive (or similar) - is this style of invalidation something suitable for httpd at all? or would people prefer to see something more like mod_proxy_balancer's full-scale html interface for doing this kind of thing at run-time? -- Colm purge.patch Description: Binary data
Re: Adding purge/invalidation to mod_cache
On Fri, May 30, 2008 at 10:35 AM, Dirk-Willem van Gulik [EMAIL PROTECTED] wrote: something more like mod_proxy_balancer's full-scale html interface for doing this kind of thing at run-time? No, totally nut - bot I am much more interested in a deeper, programmatic, interface - which I can hook up to things like spread. Esp. if that 'bus' can also contain pro-active warnings of one cache to the others that it has spotted something going out of date or whatever. However the current cache_remove_url_filter() backing is not quite ideal - with respect to purging 'recursively' into the Vary-ants. But I guess that is a problem which can be solved totally separately. Yep, Squid will delete all variations of an entity when you use Accept: */*, that isn't easy with our current approach, but I'll see what I can do - it would be nice. -- Colm
Re: Adding purge/invalidation to mod_cache
On Fri, May 30, 2008 at 1:59 PM, Akins, Brian [EMAIL PROTECTED] wrote:
How we handle purge:
Oh that reminds me, a long time ago, I wrote htcacheadmin - a generic
command line utility for administering mod_disk_cache caches. Which is
how I /used/ to handle this situation. (I've attached the source, but
it's *very* stale). If I find the time next week, I'll get this
working again, I think it might be useful to others!
--
Colm
/* Copyright 2001-2005 The Apache Software Foundation or its licensors, as
* applicable.
*
* Licensed under the Apache License, Version 2.0 (the License);
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an AS IS BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
/*
* htcacheadmin.c: a utility to allow administrators to track down urls
* in their caches, and perform actions on that basis.
*
* Contributed by Colm MacCarthaigh colm stdlib.net
* 11 Aug 2005
*/
#include apr.h
#include apr_lib.h
#include apr_strings.h
#include apr_file_io.h
#include apr_file_info.h
#include apr_pools.h
#include apr_md5.h
#include apr_getopt.h
#include apr_date.h
#include apr_uri.h
#if APR_HAVE_UNISTD_H
#include unistd.h
#endif
#if APR_HAVE_STDLIB_H
#include stdlib.h
#endif
/* mod_disk_cache.c extract start */
#define VARY_FORMAT_VERSION 3
#define DISK_FORMAT_VERSION 4
typedef struct
{
/* Indicates the format of the header struct stored on-disk. */
apr_uint32_t format;
/* The HTTP status code returned for this response. */
int status;
/* The size of the entity name that follows. */
apr_size_t name_len;
/* The number of times we've cached this entity. */
apr_size_t entity_version;
/* Miscellaneous time values. */
apr_time_t date;
apr_time_t expire;
apr_time_t request_time;
apr_time_t response_time;
} disk_cache_info_t;
/* mod_disk_cache.c extract end */
/* cache_util.c extract started */
static void cache_hash(const char *it, char *val, int ndepth, int nlength)
{
apr_md5_ctx_t context;
unsigned char digest[16];
char tmp[22];
int i, k, d;
unsigned int x;
static const char enc_table[64] =
ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789_@;
apr_md5_init(context);
apr_md5_update(context, (const unsigned char *) it, strlen(it));
apr_md5_final(digest, context);
/* encode 128 bits as 22 characters, using a modified uuencoding
* the encoding is 3 bytes - 4 characters* i.e. 128 bits is
* 5 x 3 bytes + 1 byte - 5 * 4 characters + 2 characters
*/
for (i = 0, k = 0; i 15; i += 3) {
x = (digest[i] 16) | (digest[i + 1] 8) | digest[i + 2];
tmp[k++] = enc_table[x 18];
tmp[k++] = enc_table[(x 12) 0x3f];
tmp[k++] = enc_table[(x 6) 0x3f];
tmp[k++] = enc_table[x 0x3f];
}
/* one byte left */
x = digest[15];
tmp[k++] = enc_table[x 2]; /* use up 6 bits */
tmp[k++] = enc_table[(x 4) 0x3f];
/* now split into directory levels */
for (i = k = d = 0; d ndepth; ++d) {
memcpy(val[i], tmp[k], nlength);
k += nlength;
val[i + nlength] = '/';
i += nlength + 1;
}
memcpy(val[i], tmp[k], 22 - k);
val[i + 22 - k] = '\0';
}
/* cache_util.c extract end */
static apr_file_t *errfile; /* stderr file handle */
/* short program name as called */
static const char *shortname = htcacheadmin;
/*
* usage info
*/
#define NL APR_EOL_STR
static void usage(void)
{
apr_file_printf(errfile,
%s -- utility for administration of the disk cache.NL
Usage: %s [-gq] [-h HOSTNAME] -c ROOT -d DIRLEVEL -l DIRLENGTH URL NL
%s [-emHRbD] [-E DATE] HEADERFILE NL
NL
Options: NL
NL
Locate mode: NL
NL
-c Use ROOT as the root directory for the cache. Must be identical NL
to the CacheRoot setting for Apache.NL
NL
-d Use DIRLEVEL as the number of subdirectories mod_disk_cache NL
creates for cached files. Must be identical to the NL
CacheDirLevels setting for Apache. NL
Re: [ANNOUNCE] Apache 2.0.44 Released
On Tue, Jan 21, 2003 at 03:07:36PM +0100, Jeroen Massar wrote: First, congrats with 2.0.44 ;) *) Introduce the EnableSendfile directive, allowing users of NFS shares to disable sendfile mechanics when they either fail outright or provide intermitantly corrupted data. PR [William Rowe] Shouldn't this also be noted as the fix for: IPv6 enabled version doesn't output any data on some NIC's and linux It's one form of fix :) I've been running for a few weeks now with both sendfile just off, and the TCP_CORK patch/hack I posted, turning sendfile off is probably more reliable, but in cases of heavy load the latter makes more sense. ftp.heanet.ie is currently running the TCP_CORK patch for a reference case, www.heanet.ie is running with sendfile disabled. As mentioned in the following threads: http://groups.yahoo.com/group/new-httpd/message/41447 http://groups.yahoo.com/group/new-httpd/message/41485 The general fix with 2.0.43 was to use --disable-sendfile too. This option is thus also meant for Linux users who have a NIC which behaves strange when sendfile() is enabled along with IPv6. Also I can't find any mention of it in the docs (http://httpd.apache.org/docs-2.0/) It's probably worth documenting better, I'll submit a docs patch later unless someone beats me to it. -- [EMAIL PROTECTED]PubKey: [EMAIL PROTECTED] Web: http://devnull.redbrick.dcu.ie/
Re: bug in mod_cgi (sends 200 instead of 500)
On Fri, Jan 17, 2003 at 04:36:58PM +1100, Stas Bekman wrote: Consider this mod_cgi script: #!/usr/bin/perl -w print Content-type: text/plain\n\n; print no_such_func(); print Shouldn't be printed; httpd.conf: ScriptAlias /cgi-bin/ /home/httpd/2.0/perl/ The error is correctly logged: [Fri Jan 17 16:31:03 2003] [error] [client 127.0.0.1] Undefined subroutine main::no_such_func called at /home/httpd/2.0/perl/500.pl line 6. Though the client gets the response code 200! That's correct behaviour :) If the CGI outputs valid headers and a blank newline, then it's considered valid. Any subsequent errors or return status within the CGI have no effect. -- [EMAIL PROTECTED]PubKey: [EMAIL PROTECTED] Web: http://devnull.redbrick.dcu.ie/
Re: Mass Vhosting SuExec (was Re: [PATCH] remove hardcoding of suexec log location)
On Wed, Jan 01, 2003 at 10:43:18PM +, Thom May wrote: * Aaron Bannert ([EMAIL PROTECTED]) wrote : The log is generated from the suexec binary, not httpd, right? Then we can't use a directive to control it and it needs to be hardcoded for safety. The other issue for suexec is mass vhosting; this has somewhat different needs, and mostly results in ISPs patching suexec to do what they need, which seems like a bad thing unless the ISPs can sucessfully audit the resulting codebase. It's a very bad thing, because in 99.99% of cases it's completely unneccessary! The real problem is that mass vhosting generates large numbers of document roots; covering them all with one docroot compiled into suexec can result in, eg, /home being set as the docroot. /home is a very bad docroot, and I'd question the reasons behind hosting virtual hosts in /home, the usual reason is for FTP/shell access, but that can be solved with symlinks, or just setting the homedir elsewhere. In general for sites with virtual hosts that need SuexecUserGroup I set docroot to $prefix/vhosts, and put them all in there, problem solved :) Compiling with a list of document roots sounds good in principal, but we on average add a site an hour, recompiling suexec every hour isn't particularily practical, and the configure args would be several miles long :-) Every hour! Youch, but are you adding a VirtualHost and restarting apache every hour ? If not, how are you mapping those URI's and how are you associating them with a username/group ? If you are, and if this is common, there is some limited justification for getting suexec to support such situations. But against that is the reality that in order to support it suexec would have to parse every single configuration file, determine which VirtualHost blocks have SuexecUserGroup directives and remember their Docroot, that's an awful lot of work for something that's exec'd for every CGI and is security critical. As has been pointed out, if people really want to use /home, then mod_rewrite [+mod_proxy +mod_userdir] is probably a much better way to go. It seems to me that a different binary would be the best path; suexec-mass-vhost or whatever. it needs to be able to work correctly with mod_vhost_alias, and it potentially needs to be able to take docroot arguments from httpd.conf. suexec will never work correctly with vhost_alias, or mod_rewrite :) How would you tell it what username/group to use ? The whole point of these approaches to virtual hosting is to simplify a long list of domains into one ruleset. That's not compatible with having a different username/group for each one. Then again, if you wanted to have them all run as the same user, there've been (now incomplete) patches to allow SuexecUserGroup to work in Directory blocks, see: http://nagoya.apache.org/bugzilla/show_bug.cgi?id=9038 for one. Aswell as the mass vhost stuff there's still the outstanding issue of making SHELLCMD work with suexec , see: http://redbrick.dcu.ie/~colmmacc/patches/proc.patch from the STATUS. Also maybe change the default permissions of the suexec binary to 4750 and advise people to chgrp it to the group httpd runs as. -- [EMAIL PROTECTED]PubKey: [EMAIL PROTECTED] Web: http://devnull.redbrick.dcu.ie/
Re: mpm perchild and mod_php4
On Sun, Dec 08, 2002 at 10:57:32PM +0100, Jochen Kächelin wrote: Will perchild work on the next release or are there any other workarounds to secure php-scripts (module) such as suEXEC and cgis? if you *need* mod_php instances that run as seperate users the easiest solution is to reverse proxy instances of Apache running on different ports (and ideally, only listening on localhost), each running as the desired user. To make the most of it use mod_rewrite and mod_proxy, and try to avoid proxying non-dynamic content for the sake of efficiency. There are some disadvantges, the only access you have to the originating IP address is HTTP_X_FORWARDED_FOR, it's a lot more management and can complicate scripts if they rely on thigns like SERVER_NAME. -- [EMAIL PROTECTED]PubKey: [EMAIL PROTECTED] Web: http://devnull.redbrick.dcu.ie/
Re: Linux + TCP_CORK + IPv6 = Broken [PATCH]
On Thu, Dec 05, 2002 at 07:16:02AM +, Joe Orton wrote:
On Wed, Dec 04, 2002 at 11:38:06PM +, Colm MacCárthaigh wrote:
On Wed, Dec 04, 2002 at 11:05:21PM +, Joe Orton wrote:
Colm, can you try running the apr/test/sendfile binary with your
machines?
It was one of the first things I tried when I was debugging,
unfortunately, it doesnt seem to support IPv6 from the client side
at all, I get:
colmmacc@byron:~/apr/test$ ./sendfile client blocking orion.ipv6.heanet.ie
Creating a test file...
apr_sockaddr_info_get()-670005/No address associated with hostname
Oh, sorry, I forgot to mention, you have to change the family =
APR_INET line to family = APR_INET6 to make it work over IPv6.
O.k., I've done that, and yes it works for me. sendfile.c
creates and deletes the file it sends though, it also only
serves it once, and the bug only appears when you do it twice.
In order to see the bug sendfile.c would have to try and
send the same file down two different sockets. One after
another, and monitor the second one. I patched it to fix
this (patch is below), but it isnt displaying the problem.
There are a few slight differences in what it's doing compared
to httpd though. Firstly, it's the client end that's sending
the file here, which might be relevant.
The way I can reliably reproduce the bug, is :
./configure
make
make install
dd bs=1 count=256 if=/dev/zero of=/usr/local/apache2/htdocs/index.html
/usr/local/apache2/bin/apachectl start
[ go to other machine ]
telnet [v6addr] 80
GET / HTTP/1.1
Host: madeup.tld
observe response
telnet [v6addr] 80
GET / HTTP/1.1
Host: madeup.tld
observe lack of response
Index: sendfile.c
===
RCS file: /home/cvspublic/apr/test/sendfile.c,v
retrieving revision 1.23
diff -u -r1.23 sendfile.c
--- sendfile.c 20 Nov 2002 03:50:22 - 1.23
+++ sendfile.c 5 Dec 2002 10:40:39 -
@@ -222,9 +222,9 @@
int family;
apr_sockaddr_t *destsa;
-family = APR_INET;
+family = APR_INET6;
apr_setup(p, sock, family);
-create_testfile(p, TESTFILE);
+/* create_testfile(p, TESTFILE); */
rv = apr_file_open(f, TESTFILE, APR_READ, 0, p);
if (rv != APR_SUCCESS) {
@@ -492,13 +492,13 @@
printf(client: apr_socket_sendfile() worked as expected!\n);
-rv = apr_file_remove(TESTFILE, p);
+/*rv = apr_file_remove(TESTFILE, p);
if (rv != APR_SUCCESS) {
fprintf(stderr, apr_file_remove()-%d/%s\n,
rv,
apr_strerror(rv, buf, sizeof buf));
exit(1);
-}
+} */
return 0;
}
@@ -752,6 +752,8 @@
host = argv[3];
}
if (!strcmp(argv[2], blocking)) {
+ client(BLK, host);
+ sleep(10);
return client(BLK, host);
}
else if (!strcmp(argv[2], timeout)) {
--
[EMAIL PROTECTED]PubKey: [EMAIL PROTECTED]
Web: http://devnull.redbrick.dcu.ie/
Re: Linux + TCP_CORK + IPv6 = Broken [PATCH]
On Thu, Dec 05, 2002 at 11:30:29AM +, Joe Orton wrote: On Thu, Dec 05, 2002 at 10:54:53AM +, Colm MacCárthaigh wrote: .. telnet [v6addr] 80 GET / HTTP/1.1 Host: madeup.tld observe lack of response Have you verified with tcpdump/ethereal etc that this hang is because the server is not sending the packets? What does netstat -t show for this connection? That test works fine for me here between two 2.4.18-based boxes - are you using vanilla unpatched 2.4.18? I'm using vanilla 2.4.18, from Debian kernel-source-2.4.18 , one machine has the broadcom bcm5700 module, and the the other has the intel e1000 module. Just to make doubly sure netstat -t shows tcp 0504 2001:770:18:2:206:5b:80 2001:770:18:1:201:33128 ESTABLISHED ethereal tells me that the first request is happening fine, it sees the GET and then returns the 200. the second request however doesnt do this, it sends 4 HTTP 200's before issueing 2 FIN/ACK's and an RST. I'm seeing the same thing from the client side, but no output in telnet. So it is sending *something*, just a very broken something. -- [EMAIL PROTECTED]PubKey: [EMAIL PROTECTED] Web: http://devnull.redbrick.dcu.ie/
Re: Linux + TCP_CORK + IPv6 = Broken [PATCH]
On Thu, Dec 05, 2002 at 04:17:53PM +, Joe Orton wrote: To summarize some off-list dicussion - the kernel guys have said that using sendfile on IPv6 sockets may trigger bugs in cards which do hardware TCP checksumming for card/driver/OS combinations which support that. (since the cards have to know about IPv6, and that probably doesn't get tested much). Cool :-) Just to point out, IPv6 + sendfile() works fine , IPv6 + sendfile() + TCP_CORK = badness. For me anyway :-) -- [EMAIL PROTECTED]PubKey: [EMAIL PROTECTED] Web: http://devnull.redbrick.dcu.ie/
Re: Linux + TCP_CORK + IPv6 = Broken
On Tue, Dec 03, 2002 at 11:33:27PM +0100, Jeroen Massar wrote: Colm MacCarthaigh [mailto:[EMAIL PROTECTED]] wrote: On Mon, Dec 02, 2002 at 02:48:53PM -0500, Jeff Trawick wrote: Colm MacCarthaigh [EMAIL PROTECTED] writes: Linux (2.4.18 and 2.4.19, for me anyway) with apache versions 2.0.40 to 2.0.43 (that I've tested anyways) is broken with TCP_CORK and IPv6. Bizarrely v6 requests will work the first few times and then start failing, typically you just wont get a response from the server. Though strace shows that it did try to send one. Another 'fun' thing: - Linux 2.4.18 - Apache 2.0.43 (preforked) - Debian unstable My tests were Linux 2.4.18, Apache 2.0.43 (worker), with Debian Stable http://www.sixxs.net - IPv4 works - IPv6 doesn't work ouch https://www.sixxs.net - IPv4 works - IPv6 works O.k., now that is interesting, another thing I've noticed is that when the encoding is chunked, I can't replicate the problem. Presumably the problem is related to how much data is trying to be sent down the socket at once. So I have currently installed a default reroute on http://www.sixxs.net for IPv6 requests to redirect them to the https:// variant. Try http://www.ipv6.sixxs.net/news/ for example to not get redirected (only a IPv6 non-ssl / request redirs). btw www.ipv4.sixxs.net has only A, www.ipv6.sixxs.net has only . The IPv6 doesn't work actually means: request comes in, answer comes out partially. A first request gets returned correctly any pipelined request coming in next fail. That's why the / works (as it's only 1 answer, not pipelined) some other pages without pictures show this too. Odd that IPv4 and SSL'd IPv4/IPv6 do work. makes sense if it's related the size of the writes Another thing is that I do got it working on other boxes with 2.4.18 :) Same build of apache on those, other NIC's though: 2.4.18 - purgatory.unfix.org (HTTP and HTTPS IPv4 and IPv6 ofcourse) 2.4.19-pre5-ac3 - games.concepts.nl (HTTP IPv4 and IPv6) They're outputting chunked encoding, which is why I think they work. Oddness all around, but maybe that problem is related? Definitely, one slight difference though, when I'm testing the first time I connect I can make any number of requests safely. Ie. the entire socket seems clean. The second one I create just won't give me responses from the server. -- [EMAIL PROTECTED]PubKey: [EMAIL PROTECTED] Web: http://devnull.redbrick.dcu.ie/
Re: Linux + TCP_CORK + IPv6 = Broken [PATCH]
On Wed, Dec 04, 2002 at 07:12:04AM -0500, Jeff Trawick wrote:
Colm MacCárthaigh [EMAIL PROTECTED] writes:
O.k., now that is interesting, another thing I've noticed is that when
the encoding is chunked, I can't replicate the problem. Presumably the
problem is related to how much data is trying to be sent down the
socket at once.
Well, note that Apache uses different socket APIs for dynamic content
vs. static content, and chunking is only used for dynamic content.
That should be related to the breakage.
Also, the other poster mentioned that SSL works over IPv6 but non-SSL
doesn't on one box. That also corresponds to the same difference in
socket API usage.
We can't use sendfile for any dynamic content. We can't use sendfile
for encrypted static or encrypted dynamic content. So it really seems
that sendfile is key to the issue.
Looks very much like that is the case.
But I'm confused because you already disabled sendfile usage and that
didn't bring some amount of happiness. Also, the path where we use
sendfile is the path where we use TCP_CORK so how can there be a
problem if we don't use sendfile()?
double-checking where TCP_CORK is used:
1) APR_TCP_NOPUSH_FLAG is TCP_CORK on Linux
2) APR_TCP_NOPUSH socket option controls APR_TCP_NOPUSH_FLAG
3) apr_sendfile() for Linux is the only code that dorks with
APR_TCP_NOPUSH (and thus TCP_CORK)
Here's what I see , when I use --disable-sendfile, this is a successful
request:
poll([{fd=8, events=POLLIN, revents=POLLIN}], 1, 30) = 1
read(8, GET / HTTP/1.1\r\n, 8000) = 16
gettimeofday({1039003895, 580353}, NULL) = 0
poll([{fd=8, events=POLLIN, revents=POLLIN}], 1, 30) = 1
read(8, Host: www.heanet.ie\r\n, 8000) = 21
poll([{fd=8, events=POLLIN, revents=POLLIN}], 1, 30) = 1
read(8, \r\n, 8000) = 2
stat64(/usr/local/apache/htdocs/, {st_mode=S_IFDIR|S_ISGID|0755,
st_size=4096, ...}) = 0
brk(0x814c000) = 0x814c000
stat64(/usr/local/apache/htdocs/index.html, {st_mode=S_IFREG|0644,
st_size=256, ...}) = 0
open(/usr/local/apache/htdocs/index.html, O_RDONLY) = 9
brk(0x814f000) = 0x814f000
read(8, 0x8146a30, 8000)= -1 EAGAIN (Resource
temporarily unavailable)
setsockopt(8, SOL_TCP, TCP_NODELAY, [0], 4) = 0
setsockopt(8, SOL_TCP, TCP_CORK, [1], 4) = 0
writev(8, [{HTTP/1.1 200 OK\r\nDate: Wed, 04 D..., 248}], 1) = 248
sendfile(8, 9, [0], 256)= 256
setsockopt(8, SOL_TCP, TCP_CORK, [0], 4) = 0
setsockopt(8, SOL_TCP, TCP_NODELAY, [1], 4) = 0
write(4, 2001:770:18:1:201:2ff:fef0:f281 ..., 90) = 90
poll(^C
unfinished ...
and this is an unsuccessful one:
poll([{fd=8, events=POLLIN, revents=POLLIN}], 1, 30) = 1
read(8, GET / HTTP/1.1\r\n, 8000) = 16
gettimeofday({1039003973, 629255}, NULL) = 0
poll([{fd=8, events=POLLIN, revents=POLLIN}], 1, 30) = 1
read(8, Host: www.heanet.ie\r\n, 8000) = 21
poll([{fd=8, events=POLLIN, revents=POLLIN}], 1, 30) = 1
read(8, \r\n, 8000) = 2
stat64(/usr/local/apache/htdocs/, {st_mode=S_IFDIR|S_ISGID|0755,
st_size=4096, ...}) = 0
brk(0x814c000) = 0x814c000
stat64(/usr/local/apache/htdocs/index.html, {st_mode=S_IFREG|0644,
st_size=256, ...}) = 0
open(/usr/local/apache/htdocs/index.html, O_RDONLY) = 9
brk(0x814f000) = 0x814f000
read(8, 0x8146a30, 8000)= -1 EAGAIN (Resource
temporarily unavailable)
setsockopt(8, SOL_TCP, TCP_NODELAY, [0], 4) = 0
setsockopt(8, SOL_TCP, TCP_CORK, [1], 4) = 0
writev(8, [{HTTP/1.1 200 OK\r\nDate: Wed, 04 D..., 248}], 1) = 248
sendfile(8, 9, [0], 256)= 256
setsockopt(8, SOL_TCP, TCP_CORK, [0], 4) = 0
setsockopt(8, SOL_TCP, TCP_NODELAY, [1], 4) = 0
write(4, 2001:770:18:1:201:2ff:fef0:f281 ..., 90) = 90
poll([{fd=8, events=POLLIN, revents=POLLIN}], 1, 15000) = 1
read(8, , 8000) = 0
gettimeofday({1039003977, 946871}, NULL) = 0
shutdown(8, 1 /* send */) = 0
poll([{fd=8, events=POLLIN, revents=POLLIN|POLLHUP}], 1, 2000) = 1
read(8, , 512)= 0
close(8)= 0
read(5, 0xbb77, 1) = -1 EAGAIN (Resource
temporarily unavailable)
close(9)= 0
accept(3, unfinished ...
So I still don't understand why disabling sendfile usage didn't avoid
the problem, unless it isn't specific to TCP_CORK after all.
Looks like the --disable-sendfile doesnt work, it's still using
sendfile. Reading the configure.in tells me you meant
--without-sendfile ;-) Which does do as would be expected, the
problem goes away.
I've written a patch, to confirm that it is really
TCP_CORK, applied it, and tested it, and when I use the new
--disable-ipv6-tcp-cork option, everything is nice and happy :-)
Wether I use sendfile or not.
My tests (and patch) were based on apr and apr-util from CVS ,
with the 2.0.43 codebase, because CVS seems broken right now.
Index
Re: Linux + TCP_CORK + IPv6 = Broken [PATCH]
On Wed, Dec 04, 2002 at 12:25:49PM +, Colm MacCárthaigh wrote:
My tests (and patch) were based on apr and apr-util from CVS ,
with the 2.0.43 codebase, because CVS seems broken right now.
stupid pre-test patch, here's the real one:
Index: configure.in
===
RCS file: /home/cvspublic/apr/configure.in,v
retrieving revision 1.506
diff -u -r1.506 configure.in
--- configure.in2 Dec 2002 16:07:09 - 1.506
+++ configure.in4 Dec 2002 12:29:41 -
@@ -1770,6 +1770,16 @@
echo ${nl}Checking for IPv6 Networking support...
dnl Start of checking for IPv6 support...
+use_ipv6_tcp_cork=1
+AC_ARG_ENABLE(ipv6-tcp-cork,
+ [ --disable-ipv6-tcp-cork Disable TCP_CORK with IPv6.],
+ [ if test $enableval = no; then
+use_ipv6_tcp_cork=0
+ fi ],
+ [ use_ipv6_tcp_cork=1 ] )
+
+AC_SUBST(use_ipv6_tcp_cork)
+
AC_ARG_ENABLE(ipv6,
[ --disable-ipv6 Disable IPv6 support in APR.],
[ if test $enableval = no; then
Index: include/apr.h.in
===
RCS file: /home/cvspublic/apr/include/apr.h.in,v
retrieving revision 1.117
diff -u -r1.117 apr.h.in
--- include/apr.h.in22 Oct 2002 12:37:40 - 1.117
+++ include/apr.h.in4 Dec 2002 12:29:41 -
@@ -171,6 +171,11 @@
*/
#define APR_TCP_NOPUSH_FLAG @apr_tcp_nopush_flag@
+/* Should we use corked TCP with IPv6 ? (this seems to be broken on
+ * linux
+ */
+#define APR_USE_IPV6_TCP_CORK @use_ipv6_tcp_cork@
+
/* Is the TCP_NODELAY socket option inherited from listening sockets?
*/
#define APR_TCP_NODELAY_INHERITED @tcp_nodelay_inherited@
Index: network_io/unix/sockopt.c
===
RCS file: /home/cvspublic/apr/network_io/unix/sockopt.c,v
retrieving revision 1.63
diff -u -r1.63 sockopt.c
--- network_io/unix/sockopt.c 20 Nov 2002 03:50:21 - 1.63
+++ network_io/unix/sockopt.c 4 Dec 2002 12:29:42 -
@@ -259,7 +259,12 @@
return APR_ENOTIMPL;
#endif
}
+#if APR_USE_IPV6_TCP_CORK
if (opt APR_TCP_NOPUSH) {
+#else
+if (opt APR_TCP_NOPUSH sock-remote_addr-sa.sin.sin_family != APR_INET6) {
+#endif
+
#if APR_TCP_NOPUSH_FLAG
if (apr_is_option_set(sock-netmask, APR_TCP_NOPUSH) != on) {
int optlevel = IPPROTO_TCP;
--
[EMAIL PROTECTED]PubKey: [EMAIL PROTECTED]
Web: http://devnull.redbrick.dcu.ie/
Re: Linux + TCP_CORK + IPv6 = Broken [PATCH]
On Wed, Dec 04, 2002 at 08:21:36AM -0500, Jeff Trawick wrote: Colm MacCárthaigh [EMAIL PROTECTED] writes: On Wed, Dec 04, 2002 at 12:25:49PM +, Colm MacCárthaigh wrote: My tests (and patch) were based on apr and apr-util from CVS , with the 2.0.43 codebase, because CVS seems broken right now. stupid pre-test patch, here's the real one: patch looks basically right to me... probably is exactly right... unless somebody beats me to it I'll play with it and commit it... nag me directly if it isn't committed in 36 hours or so :) Cool :-) as David Reid reminded, the flag needs to default to 0 on the right Linux boxen... unless/until we get specific info, I plan to tweak your patch to default it to --disable when building on Linux 2.4.x... relatively few users who would encounter the problem are going to be as able as you to dig to the bottom of it, and the symptoms mentioned in subsequent bug reports may not trigger the right questions on the part of the developers... Well, all I can say for certain is 2.4.18 is broken. In the interest of tying up loose ends, I'm still concerned with your observation that --disable-sendfile didn't do the right thing... did you make distclean before re-configuring? The problem there was that --disable-sendfile isnt an option configure knows anything about, the right one is --without-sendfile, which does work, and does fix the problem. :-) -- [EMAIL PROTECTED]PubKey: [EMAIL PROTECTED] Web: http://devnull.redbrick.dcu.ie/
Re: Linux + TCP_CORK + IPv6 = Broken [PATCH]
On Wed, Dec 04, 2002 at 02:29:29PM -, David Reid wrote: In the interest of tying up loose ends, I'm still concerned with your observation that --disable-sendfile didn't do the right thing... did you make distclean before re-configuring? The problem there was that --disable-sendfile isnt an option configure knows anything about, the right one is --without-sendfile, which does work, and does fix the problem. :-) However, that will disable sendfile on IPv4 as well won't it? Can you just confirm that it does for both v4 and v6? Thanks. yep, --without-sendfile completely disables sendfile , so IPv4 won't try to use it, or TCP_CORK, which is a bit of a loss (for IPv4). The patch will just disable TCP_CORK for IPv6, but continue to use sendfile in both cases. -- [EMAIL PROTECTED]PubKey: [EMAIL PROTECTED] Web: http://devnull.redbrick.dcu.ie/
Re: Linux + TCP_CORK + IPv6 = Broken [PATCH]
On Wed, Dec 04, 2002 at 11:05:21PM +, Joe Orton wrote: On Wed, Dec 04, 2002 at 08:21:36AM -0500, Jeff Trawick wrote: Colm MacCárthaigh [EMAIL PROTECTED] writes: On Wed, Dec 04, 2002 at 12:25:49PM +, Colm MacCárthaigh wrote: My tests (and patch) were based on apr and apr-util from CVS , with the 2.0.43 codebase, because CVS seems broken right now. stupid pre-test patch, here's the real one: patch looks basically right to me... probably is exactly right... unless somebody beats me to it I'll play with it and commit it... nag me directly if it isn't committed in 36 hours or so :) It would seem prudent to confirm this is a kernel bug before adding a workaround to APR. APR's test/sendfile works fine here over IPv6 between two 2.4.18-based kernels (RHL 8.0), and is doing basically the same thing as httpd, albeit without the TCP_NODELAY - maybe that makes a difference. TCP_CORK and TCP_NODELAY are mutually exclusive anyway, so it wouldnt make any difference :) As for it's status as a kernel bug, I'm more and more certain that it is, I don't think the kernel is recycling a file descriptor properly (this is purely based on kdb, I havnt started trawling source yet). I'm more familiar with the apache source (grep -c colmmacc CHANGES ;) , it's a while since I played with the Linux kernel internals. Colm, can you try running the apr/test/sendfile binary with your machines? It was one of the first things I tried when I was debugging, unfortunately, it doesnt seem to support IPv6 from the client side at all, I get: colmmacc@byron:~/apr/test$ ./sendfile client blocking orion.ipv6.heanet.ie Creating a test file... apr_sockaddr_info_get()-670005/No address associated with hostname colmmacc@byron:~/apr/test$ host -t orion.ipv6.heanet.ie orion.ipv6.heanet.ie2001:770:18:2:206:5BFF:FE8D:2402 colmmacc@byron:~/apr/test$ ./sendfile client blocking 2001:770:18:2:206:5BFF:FE8D:2402 Creating a test file... apr_sockaddr_info_get()-670009/Address family for hostname not supported Although, the server process does seem to be listening on IPv6. tcp0 0 :::8021 :::*LISTEN -- [EMAIL PROTECTED]PubKey: [EMAIL PROTECTED] Web: http://devnull.redbrick.dcu.ie/
Re: cvs commit: httpd-2.0/modules/http http_request.c
On Thu, Jun 20, 2002 at 05:30:52PM -0400, Greg Ames wrote: We ought to be able to have our cake and eat it too. It is easy to turn off keepalives based on the original type passed to ap_die. I think you're saying there's now a dependency that r-status matches the original type on exit, right? Maybe I'm reading you wrong, but I've always noticed quiet a few situations in which the status gets changed depending on how the error is handled. Off the top of my head one is when the $PREFIX/error directory is inaccessible , you can end up with HTTP 403's instead of 404's, 401's .. et al. The latter being really annoying because the WWW-Authenticate: header is present, and the request would work just fine, if it only had a 401 status. -- [EMAIL PROTECTED]PubKey: [EMAIL PROTECTED] Web: http://devnull.redbrick.dcu.ie/
2.0.38-39 lockup problem ?
I havnt tracked down a cause yet, but this happens with the 2.0.38 tarball, the 2.0.39 one, and the current HEAD. I'm running Solaris 2.8 on sparc, colmmacc@prodigy (~) $ uname -a SunOS prodigy 5.8 Generic_108528-13 sun4u sparc SUNW,Ultra-4 it's a nice and beefy E450. I was running 2.0.36 with my patches for mod_cgid and mod_include applied (now in 2.0.39) , no problems whatsoever, I used: ./configure --prefix=/local/web \ --enable-ssl \ --with-ssl=/usr/local/ssl \ --enable-module=unique_id \ --enable-module=env \ --enable-mods-shared=MAX \ --enable-rewrite \ --enable-info --enable-proxy --enable-proxy-ftp --enable-proxy-http \ --enable-cgid \ --enable-suexec --with-suexec-caller=www --with-suexec-docroot=/local/web \ --with-mpm=worker \ --with-suexec-bin=/local/web/bin/suexec \ --with-suexec-logfile=/local/web/logs/suexec_log \ --with-suexec-userdir=public_html \ --with-suexec-uidmin=100 --with-suexec-gidmin=100 --with-suexec-umask=077 first of all, with worker: Compiled in modules: core.c mod_access.c mod_auth.c mod_include.c mod_log_config.c mod_env.c mod_setenvif.c mod_proxy.c proxy_connect.c proxy_ftp.c proxy_http.c mod_ssl.c worker.c http_core.c mod_mime.c mod_status.c mod_autoindex.c mod_asis.c mod_info.c mod_suexec.c mod_cgid.c mod_negotiation.c mod_dir.c mod_imap.c mod_actions.c mod_userdir.c mod_alias.c mod_rewrite.c mod_so.c load averages: 3.40, 3.47, 3.45 07:19:47 217 processes: 177 sleeping, 2 running, 2 zombie, 34 stopped, 2 on cpu CPU states: 0.0% idle, 91.2% user, 8.8% kernel, 0.0% iowait, 0.0% swap Memory: 2048M real, 31M free, 2009M swap in use, 1524M swap free PID USERNAME THR PRI NICE SIZE RES STATETIMECPU COMMAND 18978 www5 00 474M 415M cpu/3 91:43 45.04% httpd 9847 www4 00 1126M 961M run279:03 44.73% httpd happened within a few hours of having it running, took a look in /proc//fd , it didnt have anything too big open, truss -p revealed a very quick stream of brk()'s in case it was a worker mpm problem, recompiled last night with prefork Compiled in modules: core.c mod_access.c mod_auth.c mod_include.c mod_log_config.c mod_env.c mod_setenvif.c mod_proxy.c proxy_connect.c proxy_ftp.c proxy_http.c mod_ssl.c prefork.c http_core.c mod_mime.c mod_status.c mod_autoindex.c mod_asis.c mod_info.c mod_suexec.c mod_cgi.c mod_negotiation.c mod_dir.c mod_imap.c mod_actions.c mod_userdir.c mod_alias.c mod_rewrite.c mod_so.c and : load averages: 19.79, 21.14, 19.19 01:43:00 291 processes: 229 sleeping, 19 running, 3 zombie, 38 stopped, 2 on cpu CPU states: 0.0% idle, 95.9% user, 4.1% kernel, 0.0% iowait, 0.0% swap Memory: 2048M real, 729M free, 1126M swap in use, 2393M swap free PID USERNAME THR PRI NICE SIZE RES STATETIMECPU COMMAND 28884 www3 220 52M 47M run 4:12 6.89% httpd 17682 www3 230 52M 47M run 4:12 6.57% httpd 511 www3 220 52M 47M run 4:09 6.56% httpd 29712 www3 320 52M 48M cpu/14:16 6.50% httpd 23149 www3 230 52M 47M run 4:11 6.45% httpd 26294 www3 320 53M 48M run 4:16 6.42% httpd 522 www3 230 52M 47M run 4:13 6.41% httpd 518 www3 230 52M 47M run 4:13 6.38% httpd 26313 www3 320 53M 49M run 4:18 6.37% httpd 29663 www3 330 52M 47M run 4:10 6.35% httpd 515 www3 230 53M 48M run 4:19 6.34% httpd 514 www3 320 52M 47M run 4:13 6.33% httpd 521 www3 320 53M 47M run 4:15 6.17% httpd 519 www3 320 52M 47M run 4:14 6.07% httpd 520 www3 320 51M 46M run 4:08 6.07% httpd once again, truss reveals a whole big pile of brk()'s, I wasnt around to kill these, so I didnt to see much more. So I'm now rebuilding without SSL, because I had some ssl related problems months ago. But Just to make the list aware that there may be issues. Is anyone else on Solaris seeing anything like this ? seems odd, 2.0.36 was running without any problems at all. I don't think disabling ssl will fix it, so I'm going to wait for it again, and try and gather as much as I can. -- [EMAIL PROTECTED]PubKey: [EMAIL PROTECTED] Web: http://devnull.redbrick.dcu.ie/
Re: 2.0.38-39 lockup problem ?
On Wed, Jun 19, 2002 at 01:32:00AM -0700, Justin Erenkrantz wrote: On Wed, Jun 19, 2002 at 09:26:46AM +0100, Colm MacCárthaigh wrote: I havnt tracked down a cause yet, but this happens with the 2.0.38 tarball, the 2.0.39 one, and the current HEAD. I'm running Solaris 2.8 on sparc, If you can, please run pstack on the running processes and report back where the process is. -- justin will do, I've abondonded the rebuild without ssl, I just want this to happen again. Generally takes 3-4 hours to pop up, expect feedback later. -- [EMAIL PROTECTED]PubKey: [EMAIL PROTECTED] Web: http://devnull.redbrick.dcu.ie/
suexec related patches for PR 7810, 7791, 8291, 9038
Since there have been some changes to the affected source files and multiple problems presented themselves in unixd.c, my patches to make suexec + [ mod_include | mod_userdir | mod_cgid ] work were getting stale. So I've rediffed them against CVS. I also had a good look through all of the suexec bugs, I'm using the patches on a production system now with over 2000 shell users (redbrick.dcu.ie) and it's proving stable. Anyway, I think they fix these : PR 7810 - suexec + mod_userdir + mod_cgid needed fixing (also it's currently insecure by default, this really needs to be fixed) PR 7791 - malformed arguments array passed to suexec PR 8291 - mod_include + suexec exec cmd not working PR 9038 - really a duplicate of 7810 Some notes: 1: http://redbrick.dcu.ie/~colmmacc/patches/mod_cgid.patch 2: http://redbrick.dcu.ie/~colmmacc/patches/unixd.patch 3: http://redbrick.dcu.ie/~colmmacc/patches/mod_include.patch patch 1 (mod_cgid.c)fixes 7810/9039/mod_cgid, it just works. patch 2 (unixd.c) fixes 7791 and 8291 patch 3 (mod_include.c) makes patch 2 secure. (otherwise include file=some.cgi runs as the server user) Other Patches: These are against 2.0.36, but should apply to CVS. Whilst trawling code for patch 2 I noticed that in srclib/apr/threadproc/unix/proc.c shell commands get executed as: shell -c argv0 argv1 argv2 I believe it should be: shell -c argv0 argv1 .. I initially fixed the suexec problem this way ... because shell -c suexec user group ... would never work (at least with my /bin/sh), but fixing it such that shell -c 'suexec user group ... ' leads to simple exploits like : !--#exec cmd=somecmd ; evilcmd-- being trivial. I used the code in patch 4 (proc.c) to fix this for me though (for the general non-suexec case) ... it might be desireable anyway , just to have exec cmd work in general. 4: http://redbrick.dcu.ie/~colmmacc/patches/proc.patch And finally , a whole bundle of patches related to the comment in the STATUS file: * PR#1120: suexec suexec does not parse arguments to #exec cmd I decided to make this work, for my own amusement. The result is rather convoluted though , but if anyone is interested in resolving this issue, it's there. Basically just define a trusted system shell at buildtime and have suexec allow it be used .. and have unixd.c detect shellcmd's and warp what suexec gets sent on that basis. It's at: http://redbrick.dcu.ie/~colmmacc/patches/suexec-shell.patch All of the patches are proving useful to us at least, but I would say that a patch to mod_cgid should be a matter of priority for the next release of apache, as it is currently a security hole. -- [EMAIL PROTECTED]PubKey: [EMAIL PROTECTED] Web: http://devnull.redbrick.dcu.ie/
Re: [PATCH] exec cmd working with suexec
On Tue, May 21, 2002 at 01:03:03AM +0100, Colm MacCárthaigh wrote:
The following patch reverts to the previous and documented
bevahiour (exhibited by 1.3). Currently !--exec cmd
does not work with suexec enabled as the proc.c will try to
run : shell -c suexec uid gid ... so on.
snip patch
I should have mentioned, this patch resolves PR 8291. Also
after applying this patch, SSI + suexec works fully except
that
!--#include file=some.cgi--
will run the cgi as the webserver user, and not the suexec
user. Obviously a security hole. include virtual is not
affected, Nor is the fsize directive. (they all use
subrequests)
The patch below fixes it by using the ap_sub_req_lookup_uri
instead of ap_sub_req_lookup_file. Since the functionality of
_uri is a superset of _file (afaict) this means that nothing
valid in an include file directive will break. But it
also means that some of what was previously invalid will
(query string etc). Since include file has deprecated
support anyway .. I'm not sure how big a deal it is that
mod_include wouldnt be enforcing what include file should
only be able to do.
Colm
Index: modules/filters/mod_include.c
===
RCS file: /home/cvspublic/httpd-2.0/modules/filters/mod_include.c,v
retrieving revision 1.222
diff -u -r1.222 mod_include.c
--- modules/filters/mod_include.c 17 May 2002 11:33:09 - 1.222
+++ modules/filters/mod_include.c 22 May 2002 09:26:39 -
-1263,7 +1263,7
in parsed file %s;
}
else {
-rr = ap_sub_req_lookup_file(parsed_string, r, f-next);
+rr = ap_sub_req_lookup_uri(parsed_string, r, f-next);
}
}
else {
[PATCH] exec cmd working with suexec
The following patch reverts to the previous and documented
bevahiour (exhibited by 1.3). Currently !--exec cmd
does not work with suexec enabled as the proc.c will try to
run : shell -c suexec uid gid ... so on.
Index: os/unix/unixd.c
===
RCS file: /home/cvspublic/httpd-2.0/os/unix/unixd.c,v
retrieving revision 1.52
diff -u -r1.52 unixd.c
--- os/unix/unixd.c 17 May 2002 11:33:10 - 1.52
+++ os/unix/unixd.c 20 May 2002 23:59:53 -
-357,6 +357,15
newargs[2] = execgroup;
newargs[3] = apr_pstrdup(p, progname);
+/*
+** using a shell to execute suexec makes no sense thus
+** we force everything to be APR_PROGRAM, and never
+** APR_SHELLCMD
+*/
+if(apr_procattr_cmdtype_set(attr, APR_PROGRAM) != APR_SUCCESS) {
+return APR_EGENERAL;
+}
+
i = 0;
do {
newargs[i + 4] = args[i];
[PATCH] suexec + mod_userdir + mod_cgid
A few weeks ago I opened Bug 7810, which was suexec not working
with mod_userdir with either mod_cgi or mod_cgid. Anyway, Justin
kindly applied a modified patch of mine to fix mod_cgi .. up
until today I have been simply too busy to look at it .. but
here's a patch that makes mod_cgid work. The bug is already
closed, but this patch shuold fix all of the issues that are in
it.
It's two changes, send and receive the entire suexec_cfg struc-
ture to/from cgid, and send the mod_userdir_user note. Also, just
to note that current beahaviour is that :
--enable-cgid --with-mpm=[something threaded] \
--enable-suexec ...
results in a build of apache which is fully functional , with all
userdir CGI working .. but not suexec'd. Which in certain
environments is a major security hole.
- Colm
Index: modules/generators/mod_cgid.c
===
RCS file: /home/cvspublic/httpd-2.0/modules/generators/mod_cgid.c,v
retrieving revision 1.129
diff -u -r1.129 mod_cgid.c
--- modules/generators/mod_cgid.c 17 May 2002 11:33:09 - 1.129
+++ modules/generators/mod_cgid.c 20 May 2002 02:15:44 -
-329,16 +329,8
if (rc != sizeof(int)) {
return 1;
}
-rc = read(fd, suexec_cfg-ugid.uid, sizeof(uid_t));
-if (rc != sizeof(uid_t)) {
-return 1;
-}
-rc = read(fd, suexec_cfg-ugid.gid, sizeof(gid_t));
-if (rc != sizeof(gid_t)) {
-return 1;
-}
-rc = read(fd, suexec_cfg-active, sizeof(int));
-if (rc != sizeof(int)) {
+rc = read(fd, suexec_cfg, sizeof(*suexec_cfg));
+if (rc != sizeof(*suexec_cfg)) {
return 1;
}
dconf[i] = (void *)suexec_cfg;
-379,12 +371,20
}
#endif
#endif
-/* For right now, just make the notes table. At some point we will need
- * to actually fill this out, but for now we just don't want suexec to
- * seg fault.
- */
+
+/* basic notes table to avoid seg faults */
r-notes = apr_table_make(r-pool, 1);
+/* mod_userdir requires the mod_userdir_user note */
+rc = read(fd, len, sizeof(len));
+if (len) {
+data = apr_pcalloc(r-pool, len + 1); /* last byte is '\0' */
+rc = read(fd, data, len);
+if(rc != len) {
+ return 1;
+}
+ apr_table_set(r-notes,mod_userdir_user, data);
+}
return 0;
}
-441,9 +441,7
suexec_mod);
write(fd, suexec_mod-module_index, sizeof(int));
-write(fd, suexec_cfg-ugid.uid, sizeof(uid_t));
-write(fd, suexec_cfg-ugid.gid, sizeof(gid_t));
-write(fd, suexec_cfg-active, sizeof(int));
+write(fd, suexec_cfg, sizeof(*suexec_cfg));
}
#if 0
-483,6 +481,16
}
#endif
#endif
+ /* send a minimal notes table */
+ data = (char *) apr_table_get(r-notes, mod_userdir_user);
+ if(data != NULL) {
+ len = strlen(data);
+ write(fd, len, sizeof(len));
+ write(fd, data, len);
+ } else {
+ len = 0;
+ write(fd, len, sizeof(len));
+ }
}
static void daemon_signal_handler(int sig)
Re: FWD: patch to resolve bug 7810 (userdir + suexec)
On Sun, Apr 28, 2002 at 10:55:59AM -0700, Justin Erenkrantz wrote:
This is a message from the person who originally filed PR 7810.
I am not familiar enough with mod_cgid to verify his patch or
to get suexec working with worker. -- justin
Just to clarify , the patch below doesnt fix it at all :-), it's
really just an addendum to the userdir patch to the ugid
structure that was committed a few days ago.
Actually, I'm not entirely sure why mod_cgid insists on reading/writing
the elements individually, it'd be (slgihtly) more efficient and
just as effective to simply:
send_req(...
write(fd, suexec_cfg, sizeof(suexec_cfg));
get_req(...
rc = read(fd, suexec_cfg, sizeof(suexec_cfg));
if(rc != sizeof(suexec_cfg) {
return 1;
}
which would be a bit more future-proofed, but anyway ..
As I said, I'm continuing to look into the problem, I'm not entirely
certain if it is a mod_cgid problem. By the time send_req is being
called (send_req is where the handler actually resends the request
accross the local socket) suexec_cfg.active is not 1, so the
server/cgid end of the equation is quiet correct to not try and
use suexec.
Additionaly the uid and gid are simply set to 1 (which I think is
their initial default, and the userdir flag to random memory), the
suexec_cfg stuff never seems to get set at any stage. It seems
that by the time the mod_cgid handler has been called the problem
has already occurred.
It seems that somehow changing mod_cgi - mod_cgid, or prefork - worker
means that the mod_suexec cfg stuff never happens. I'm not familiar
at all with the codebase which is why this is all taking me so long,
but afaict the only place in the code that ever does set active to 1
is set_suexec_ugid in mod_suexec.c, that doesnt make much sense to
me because this is a configuration handler. Function pointers and
code divergency make it pretty hard to track down how exactly
(with prefork and mod_cgi) active ever gets set to 1, because I'm
pretty sure this is the step that's failing.
I'm fairly certain from gleaning over the mod_cgid code that in
order for it to work correctly for userdir, it will probably
need to do a chdir to the correct directory before constructing
the child process, but that's a comparitively trivial problem.
Anyway, that's simply a summary of what I have to date.
Anyway, this patch is definitely something that's needed at some stage :
Index: mod_cgid.c
===
RCS file: /home/cvspublic/httpd-2.0/modules/generators/mod_cgid.c,v
retrieving revision 1.128
diff -u -r1.128 mod_cgid.c
--- mod_cgid.c 22 Apr 2002 08:08:38 - 1.128
+++ mod_cgid.c 28 Apr 2002 13:14:55 -
@@ -337,6 +337,10 @@
if (rc != sizeof(gid_t)) {
return 1;
}
+rc = read(fd, suexec_cfg-ugid.userdir, sizeof(int));
+if (rc != sizeof(int)) {
+return 1;
+}
rc = read(fd, suexec_cfg-active, sizeof(int));
if (rc != sizeof(int)) {
return 1;
@@ -443,6 +447,7 @@
write(fd, suexec_mod-module_index, sizeof(int));
write(fd, suexec_cfg-ugid.uid, sizeof(uid_t));
write(fd, suexec_cfg-ugid.gid, sizeof(gid_t));
+write(fd, suexec_cfg-ugid.userdir, sizeof(int));
write(fd, suexec_cfg-active, sizeof(int));
}
it's just to integrate the change you made to the ugid structure
to mod_cgid.
--
[EMAIL PROTECTED][EMAIL PROTECTED]
