Re: svn commit: r1896976 - /httpd/httpd/branches/2.4.x/STATUS
On Mon, Jan 17, 2022 at 02:16:32PM +0100, Ruediger Pluem wrote: > > > On 1/17/22 1:38 PM, Eric Covener wrote: > > On Mon, Jan 17, 2022 at 7:28 AM Ruediger Pluem wrote: > >> > >> > >> > >> On 1/16/22 10:35 PM, William A Rowe Jr wrote: > >>> 4 day ago, you all saw this. 6 years ago, you all started using this on > >>> trunk. > >>> > >>> Don't know what I can to do help this along and honor the library > >>> author's wishes for > >>> all of us to walk away from the dead fork, and use the maintained > >>> fork. It's whatever > >>> it is, I'm out of here and removing the backport application branch, > >>> whoever 3rd upvotes > >>> this be prepared to apply this for us all, thanks. > >> > >> With regards to the de-optimization / stack usage probably stupid question: > >> Can't we use the TLS features that several compilers offer? > >> e.g. see at the end of the page at > >> https://stackoverflow.com/questions/18298280/how-to-declare-a-variable-as-thread-local-portably > >> > >> If we have no thread_local we could fall back to the current unoptimized > >> implementation? > > > > I more identified with Joe's comment, if we had an _ex version we > > could pass pools for our own usage. > > If some other heavy regex hitter third-party module finds an issue > > (like mod_security) it might degrade a little bit but they can always > > take advantage of it too. > > > > Also, what about alloca()? > > > > Trivia but I should add that in an integration between two third-party > > mods I recently ripped out apr_threadkey stuff due to a crash after > > some OS updates that nobody can explain. Fortunately the problematic > > API had been extended with userdata and it was no longer absolutely > > necessary. It was our only use of these API's so I was glad to > > simplify and just remove them and move on. > > Net, I would be hesitant to introduce something too new. > > From a quick glance apr_threadkey seems to use pthread_*specific for which I > found postings that state that it is slow compared to > the compiler implemented thread locals. OTOH __thread alike stuff does not > seem to offer any cleanup at thread exit which would > not allow us to free any resource that was created for that thread :-(. Hence > I guess providing a pool to the call remains the > only solution here. Maybe use thread-local storage to set a pointer to an apr_pool associated with the thread, and set that only for short-lived (or otherwise appropriately scoped) threads? Cheers, Glenn
Re: Apache 2.2 MPM Worker Virtual Memory Usage
68 - - - r-x-- libaprutil-1.so.0.2.11 b7dda000 4 - - - rw--- libaprutil-1.so.0.2.11 b7ddb000 132 - - - r-x-- libm-2.3.4.so b7dfc000 4 - - - r libm-2.3.4.so b7dfd000 4 - - - rw--- libm-2.3.4.so b7dfe000 60 - - - r-x-- libz.so.1.2.1.2 b7e0d000 4 - - - rw--- libz.so.1.2.1.2 b7e0e000 4 - - - rw---[ anon ] b7e0f000 60 - - - r-x-- libresolv-2.3.4.so b7e1e000 4 - - - r libresolv-2.3.4.so b7e1f000 4 - - - rw--- libresolv-2.3.4.so b7e2 8 - - - rw---[ anon ] b7e22000 128 - - - r-x-- libk5crypto.so.3.0 b7e42000 4 - - - rw--- libk5crypto.so.3.0 b7e43000 8 - - - r-x-- libcom_err.so.2.1 b7e45000 4 - - - rw--- libcom_err.so.2.1 b7e46000 396 - - - r-x-- libkrb5.so.3.2 b7ea9000 8 - - - rw--- libkrb5.so.3.2 b7eab000 76 - - - r-x-- libgssapi_krb5.so.2.2 b7ebe000 4 - - - rw--- libgssapi_krb5.so.2.2 b7ebf000 848 - - - r-x-- libcrypto.so.0.9.7a b7f93000 72 - - - rw--- libcrypto.so.0.9.7a b7fa5000 16 - - - rw---[ anon ] b7fa9000 196 - - - r-x-- libssl.so.0.9.7a b7fda000 12 - - - rw--- libssl.so.0.9.7a b7fe4000 4 - - - r-x-- mod_bwlimited.so b7fe5000 4 - - - rw--- mod_bwlimited.so b7fe6000 4 - - - r-x-- mod_auth_passthrough.so b7fe7000 4 - - - rw--- mod_auth_passthrough.so b7fe8000 88 - - - r-x-- ld-2.3.4.so b7ffe000 4 - - - r ld-2.3.4.so b7fff000 4 - - - rw--- ld-2.3.4.so bfff9000 20 - - - rw---[ stack ] e000 4 - - - -[ anon ] --- --- --- --- total kB 61316 - - - Ruediger Pluem wrote: What is your setting for ThreadsPerChild? On my Linux each thread consumes 8MB of virtual memory (I assume for stack and other thread private data) as shown by pmap. This can sum up to a large amount of memory. Regards Rüdiger On 10/21/2007 12:29 PM, Apache Dev wrote: I tried asking this on the users mailing list. But, I think it is beyond that list. There appears to be a significant difference between Apache 2.2 MPM Worker and MPM Perfork virtual memory usage. As well as between Apache 2.2 MPM Worker and Apache 1.3 virtual memory usage. This can become an issue in a VPS (virtual private server) environment where resources are more constrained. Privvmpages is limited in this environment. The impact here is that some VPS users simply cannot run the Worker MPM. Which is unfortunate. I am seeing 280MB vs 5.8MB of VM usage per process. You could argue that worker is supposed to use more virtual memory, as running multiple threads per process it actually uses less. But, that is not the case. Total VPS privvmpages is 2.1GB vs 358MB. What is really interesting here is why is it so much higher. You would expect some increase but it looks like most (if not all) of the virtual memory of each of the 7 worker processes is not shared. 7 * 280MB = 2GB. Which means it can't be code. I don't see how it could be this much code anyway. So then what is it? (Conversely most of the 2.2 perfork virtual memory is shared code. 150 * 5.8MB = 850MB,which is more than 358MB for the entire VPS.) Is the code building some kind of large local process database? Anyway to turn it off? Note I do not have PHP or any other programmatic modules loaded. See below for configure info. Thanks...
httpd-2.1.3-beta under a large DDOS attack ... not good.
*long but interesting, I hope* I had the displeasure of coping with a large DDOS attack this weekend and tested out how apache 2.1.3-beta did. It didn't do very well at all. I realize this list is for discussion of changes to the source code and related issues but I'm hoping this is still appropriate and would be interested to get feedback. The attack was from a botnet comprising, at any one moment, over 6000 unique IPs. New IPs were adding themselves fairly constantly, a dozen every few minutes at least. They were also rolling off. The clients were all windows XP and 2k machines, and judging from the ping time to many of them, many were dialup or on other dynamic IPs. They were getting their current attack targets from a php program on the webserver of a rooted box, not from an IRC type control system. The zombie army had a rather unique attack. They would send multiple syn packets in order to try to open a connection on the web server. syn_cookies coped ok with this. If they succeeded in opening a connection to port 80 they would then send one random character at a time, about one second apart. Each character would come in its own tcp packet - so the tcp PSH flag was set. If you had an infinitely powerful web server you would therefore soon be handling over 10,000 to 50,000 active connections, all doing nothing much - why more than the number of unique ips? because the zombies were also doing this in parallel, so one zombie could hold open more than one connection. At the same time, they were also flooding our IP with fragmented ping packets of 1480 bytes each in order to choke up our port. About 9/10ths of the traffic by volume was fragmented pings and about 9/10ths of traffic by number of packets was syn or 1 or 2 byte data packets along with associated RSTs and so on. The hardware on the machine coped ok with all this - about 60mbit incoming traffic and linux 2.4 with NAPIfied latest e1000.o - but one cpu was pretty much flat out picking up packets from the card. Unfortunately, if more than 3000 IPs were added to netfilter as DROPs then the box would start to fall behind (overruns on the card). So blacklisting all bad ips was a non-starter. Even picking out bad ips wasn't so easy as they look initially like a normal open request. Ok, so how did the latest apache source cope with this, using the mpm-worker module? Not too good. I tried a number of different ways. The first thing I did was reduce the Timeout to 1 second. In this way I hoped to fast-drop any connections that were dribbling characters. Unfortunately, zombies sending characters 1 per second meant that apache did not drop the connection fast enough, a zombie could keep a slot open for 5-10 seconds until it got kicked. Still, my apache configured for 6000 max clients, spread over 60 httpds with 100 threads each, the server-status would soon show 4000-5000 active connections and could still serve legit requests (hardly difficult - I was serving with 302 redirects with mod_rewrite to an unmolested IP address in order to move users). HOWEVER, even though server-status showed the config was stable like this - as many new connections coming in as old ones dying off, and although the server was functional, the memory on the box was being consumed at a crazy rate. Within 40 seconds, over 1gb of physical memory had vanished all sucked down by apache.. and unless all processes were immediately killed, the box would move into swap space and become totally unresponsive. (the box was an SMP xeon with 2gb of memory). So, I had to kill and restart apache every 50 seconds. It also did not matter what I did with RequestsPerChild, 1 or 50, I was not getting memory back. Lastly, no logging of 408 (timeout) errors were happening. I could telnet to apache and sit and wait and get kicked after 1 second, and get no 408 log line. Apart from that issue, apache was crashing, especially if I tried to config for more than 5000 clients. I would either receive out of memory errors when doing thread creates, or other bad error messages relating to one child, that would cause the entire server to shut down. Here is an example of the scoreboard: notice, the server has been up for just 20 seconds: Current Time: Sunday, 13-Mar-2005 14:54:39 EST Restart Time: Sunday, 13-Mar-2005 14:54:18 EST Parent Server Generation: 0 Server uptime: 20 seconds 3598 requests currently being processed, 987 idle workers RR..C..CCRRRCRRR.RC.R.R..RRRCCCR..CRR.R.CRRR CRR.RR.CRC.CCRCC..CCR..CC.CC.CCCC..CCR.C.R.RR.R.CC.C CC..CC..CC.CR.C.R....RR.R.CC.CCRC..C.CR..C...CC..C.C ..C.CCCRRCRCCRRRCRCRRRCCRCCCRRCC RRCCRCCCRCCRRRCCRRCCRRCRCRCR RCCCRRR_RRCRCC_CRCCRCRRRCRCCRRCRCRCRRCCRCCRC Here is an example of the memory usage: total used free sharedbuffers cached Mem: 20638921301084 762808
Build for win32 without VC++?
I had originally posted this on news:comp.infosystems.www.server.ms- windows and news:alt.apache.configuration but wanted to try with the mailing list, also. Has anybody ever tried (not to mention succeeded!) to build a version of Apache for Win32 (definitely version 2) using FOSS tools and completely bypassing MS Visual Studio? I don't mean for Cygwin, but native Win32? Maybe I am being a bit paranoid here, but I don't want to use Cygwin and add an extra layer of interface. On the face of it, this makes it to me less secure, more avenues for attack by intruders. Plus efficiency suffers, IMHO. Any comments on this being wrong most readily accepted. It irritates me that I have to spend hundreds of $ to build a FOSS tool. Also, I want SSL/HTTPS capability and it seems, with the latest caution (I mean, come on, RSA's patent lapsed in the year 2000, the US government relaxed its encryption export laws, the code is distributed with the SSL source) of the Apache developers, that the only way to get this on Win32 is to either build it with MS VC++ or to download prebuilt versions from someone I don't know. Plus, if I can build Apache from source, I can keep up with source code security updates like the one for 2.0.51. Up to now, I have gotten the apr component to build using straight GCC after converting the .DSW and .DSP files into makefiles. Running into some brick walls, though. If someone else has solved all of this and has published it on the Web, a URL would be most appreciated. If not, I'll slog along and publish it myself when I get a web site up. Any other comments most appreciated.
Re: New Module: mod_auth_mysql_digest
torsdagen den 7 november 2002 01.19 skrev Greg Stein: On Mon, Nov 04, 2002 at 07:21:32AM -0500, Jeff Trawick wrote: Justin Erenkrantz [EMAIL PROTECTED] writes: --On Friday, November 01, 2002 16:02:45 -0800 Rob Emanuele [EMAIL PROTECTED] wrote: So I took the mod_auth_digest code and munged it to use mysql for authentication. Swell. It works great and we're heavily using it. If you'd be willing to write this as an auth provider, I'm sure we might be interested in integrating it in our tree. -- justin absolutely +1 here, too. Considering the popularity of MySQL-based auth, I think this would be a great addition to httpd. See: http://www.securityspace.com/s_survey/data/man.200210/apachemods.html Cool, no one's cooking up a mod_log_mysql for apache2? http://modules.apache.org/ http://modules.apache.org/search and do a search for mod Or do a search for mysql Fun place to look around. -Glenn
Re: [VOTE] Location of aaa rewrite
Please vote: [X] Check in aaa rewrite to 2.0. [ ] Check in aaa rewrite to 2.1. Adoption of Apache 2 so far has been low enough that Apache 2 may still be considered to be in the early adopter phase. Breaking compatibility is to be avoided when possible, but is allowable when necessary, and the aaa rewrite is necessary. Hopefully, early adopters read release notes. :-) -Glenn
Re: 2.0.40 (UNIX), mysterious SSL connections to self
On Thu, Aug 29, 2002 at 09:21:27PM -0400, Jeff Trawick wrote: [EMAIL PROTECTED] [EMAIL PROTECTED] writes: Looking into the matter, I discovered that this error is logged whenever a connection from localhost to localhost:443 is made. The errors ALWAYS occur after apachectl startssl; they never occur with apachectl start. I'd bet that this is idle server maintenance (taking down a child process which isn't necessary). The non-SSL path doesn't log an error for a dummy connect but evidently the SSL path does. The connect to itself isn't a bug, though it would be nice to avoid the SSL complaints. Try swapping the order of your listen statements to see if that gets rid of the complaints (i.e., if the dummy connect to wake up a child is then done to the non-SSL socket). My solution to the complaints was to use piped error logs and have a simple Perl script as the first script in a pipeline. The Perl script's only job was to remove those error messages and then pass the log line on to cronolog. The reason for taking such a measure was because the server farm was behind a pair of commercial load balancers which made TCP connections to port 80 and port 443 ** every second ** to health check that the servers were alive and accepting connections within a reasonable response time. Then, it shut down the connection, without attempting any SSL negotiation. So every second, every server was logging two SSL failure messages (from each of the redundant load balancers). Talk about noise in the logs! It would be Real Nice (tm) if these sort of SSL error messages weren't reported unless some sort of data was exchanged above the connection level. Only in those cases would the SSL error message be correct that SSL negotiation failed, as opposed to not even having started. -Glenn
Re: daedalus is running httpd-2.0.pre40
On Tue, Jul 23, 2002 at 11:03:03AM -0400, Greg Ames wrote: johannes m. richter wrote: If you read this whole thread, you'll see that I'm not the only one who likes having current -std.conf files available. They worked this way for ages. I don't recall seeing any complaints about this behavior until yesterday. .. People seem to want their conf-directories to be really clean - which is can understand. But i also understand it might be useful to have a current default configuration file around. IMHO the documentation dir SERVERROOT/manual/conf/ perhaps?) would be a good place for this. SERVERROOT/manual/conf-default/ ? That might make it clearer that this is just for the default configs. Opinions? As another lurker on the list, I'd like to weigh in on having the template conf files in conf/sample and named after the release which created it, i.e. conf/sample/httpd.conf-2.0.40 (You could also use conf/dist/* or conf/template/* or ...) This avoids cluttering conf and keeps the template configs close at hand for reference. Apache conf files are large and they tend to change subtly between releases. It is really nice to be able to compare differences. Personally, I use the default conf files and comment out what I don't want and comment in my changes explaining the deviation from the distributed defaults. -Glenn
Re: suexec ScritpAlias/ScriptAliasMatch
I am just starting to look at the server code and have been playing with the suexec code. Want to try to pass additional parameters to suexec, in paticular the value of the ScriptAlias or ScriptAliasMatch (after regex) for the VirtualHost. Believe I have found where suexec is called but unable to figure out where the ScriptAlias/Match value is or how to pass it. Any help greatly appreciated. Why modify the httpd code? Why not just (carefully) modify the suEXEC code? suEXEC requires that Apache chdir into the directory with the target script before starting suEXEC. Therefore, getcwd() will return the absolute path equivalent to ScriptAlias or ScriptAliasMatch. (They'll be identical if there are no symlinks in the path.) Anyway, what are you planning on doing with the ScriptAlias or ScriptAliasMatch value in suEXEC? BTW, if you want a quick way to pass additional params to suEXEC, you could use SetEnv in Apache. Then, suEXEC would have the value available to it until the part of suEXEC near the end of the code which purges the environment of all but known good CGI vars. Of course, you should never blindly trust environment values. Same thing with params passed to an suid program such as suEXEC. That's why you should have a really, really good reason for needing to pass additional params to suEXEC. The current parameters (target uid, target gid, and target program) are all rigorously checked by suEXEC before being used. If you plan on using any new info from the environment or from an additional param, then make sure it is properly validated before you use it! -Glenn
Q: licensing suEXEC modifications under GPL?
Hi. I sent this over three weeks ago to the human response email address at apache.org (and pinged them again a week ago), but have yet to receive a response. Would someone on his list please direct me to whom I should contact? A synopsis is that I am requesting that Apache release suEXEC under the GPL. Notwithstanding your opinions on licensing in general, I hope that you'll agree that there are large benefits to releasing *security* software under the GPL. Thank you. Glenn Strauss --- Greetings. suEXEC is a program that I know well, love, and use on numerous servers. Thank you very much! To extend its usage, I have largely rewritten it and am in the process of unit testing. Among many other things, I have modularized it and have added an option (off by default) for suEXEC to safely run in a group environment (with group write privileges). I would like to make my heavily modified version available under the GNU Public License and am writing to ask for permission and advice. (I am not a lawyer and the gnu.org website indicates that the Apache Software License v1.1 is not compatible with the GPL, but does not give specific reasons.) I am hoping that the ASF will consider the merits of suEXEC as a standalone program that may be used with other webservers and will consider releasing suEXEC under the GPL. I have searched through the newsgroup at http://www.apachelabs.org/ and have turned up a number of (not-so-friendly) discussions relating to licensing (GPL, PHP, GZIP, PCRE, ...), so I have an idea where the ASF stands on this issue, but I am asking anyway. Please direct me to where I might find more information on: a) what steps I might take to release my modifications under the GPL with proper attribution to the Apache Software Foundation. b) if the Apache Software Foundation would specifically release suEXEC under the GPL or another GPL compatible license. It is a standalone program and so this would not affect anything else licensed under the Apache Software License. c) other suggestions Your assistance is greatly appreciated. Thank you. Glenn Strauss PS: Yes, I have read: http://www.apache.org/foundation/preFAQ.html#licence The FAQ answer contains a mistake: there is no section 6 in the Apache Software License v1.1 * You have questions about the Apache licence. If you have sent mail about a licence issue or question, please review the online licence at the URL listed below. The gist of the licence is that you may use, modify, and/or [re]distribute the Apache software as-is. As long as you do not change the software, you may re-distribute it and call it Apache. If you alter the software in any way, other than tailoring the configuration files or making it compilable on your platform, you may only refer to it as being based upon Apache. In all cases, altered or not, you must include attribution as described in sections 3 and 6 of the licence. Let us know if you have further questions.
