Re: Apache proxy engineering specs
William A. Rowe, Jr. wrote: Shaw, Dan wrote: 1. Does a apache proxy server create stateful or stateless connection upon request? It might help to clarify right off that RFC 2616 defines HTTP/1.1 and prior as stateless protocols. Dan, maybe you mean persistent rather than stateful? At least for the reverse proxy setup, which I believe is what you're considering, mod_proxy does implement persistent connections to the server, if configured to do so. Search for smax on the mod_proxy documentation page. In fact I've found the connections to be rather too persistent in some cases (see my messages in the mod_proxy: is smax=0 allowed thread). 2. On the transport layer does the proxy server stream data or send ACK back to originating request and then continue with sending request from proxy to end server/point? I believe that it streams the data with a buffer size set by the ProxyIOBufferSize directive. Or something like that. But I have a vague recollection that there is some sort of issue with the content-length header, i.e. you don't know what it should say until all the data has been sent. Regards, Phil.
Re: mod_proxy: Is smax=0 allowed ?
Phil Endecott wrote:
Phil Endecott wrote:
I'm going to have another look at the source, and see whether I can
change the sentinel value for smax not specified from 0 to -1, so
that 0 can really mean zero.
I have tried the following:
Index: proxy_util.c
===
--- proxy_util.c(revision 573391)
+++ proxy_util.c(working copy)
@@ -1752,7 +1752,7 @@
if (worker-hmax == 0 || worker-hmax mpm_threads) {
worker-hmax = mpm_threads;
}
-if (worker-smax == 0 || worker-smax worker-hmax) {
+if (worker-smax == -1 || worker-smax worker-hmax) {
worker-smax = worker-hmax;
}
/* Set min to be lower then smax */
Index: mod_proxy.c
===
--- mod_proxy.c (revision 573391)
+++ mod_proxy.c (working copy)
@@ -1263,6 +1263,7 @@
}
PROXY_COPY_CONF_PARAMS(worker, conf);
+worker-smax = -1;
for (i = 0; i arr-nelts; i++) {
const char *err = set_worker_param(cmd-pool, worker, elts[i].key,
elts[i].val);
This does seem to result in smax=0 being used, but it breaks something;
as far as I can see the requests never reach the backend. I have taken
the source from trunk and am loading the module into my 2.2.4 server
OK, I've tried again with the same patch applied to the 2.2.4 source,
and it seems to be somewhat working; I think I've seen the backend drop
to zero connections.
I would much appreciate some input from someone who knows their way
around this code.
Please!
Regards,
Phil.
Re: mod_proxy: Is smax=0 allowed ?
Phil Endecott wrote:
[EMAIL PROTECTED] wrote:
Seems like you are right about the smax=0.
Luckily for you, according to the documentation you may control the persistency
of the backend connection with the environment variable proxy-nokeepalive
Well, I can use that to switch of persistency, i.e. I can get smax=0
ttl=0. I don't think I can use that to get smax=0 ttl=60, can I?
I'm going to have another look at the source, and see whether I can
change the sentinel value for smax not specified from 0 to -1, so
that 0 can really mean zero.
I have tried the following:
Index: proxy_util.c
===
--- proxy_util.c(revision 573391)
+++ proxy_util.c(working copy)
@@ -1752,7 +1752,7 @@
if (worker-hmax == 0 || worker-hmax mpm_threads) {
worker-hmax = mpm_threads;
}
-if (worker-smax == 0 || worker-smax worker-hmax) {
+if (worker-smax == -1 || worker-smax worker-hmax) {
worker-smax = worker-hmax;
}
/* Set min to be lower then smax */
Index: mod_proxy.c
===
--- mod_proxy.c (revision 573391)
+++ mod_proxy.c (working copy)
@@ -1263,6 +1263,7 @@
}
PROXY_COPY_CONF_PARAMS(worker, conf);
+worker-smax = -1;
for (i = 0; i arr-nelts; i++) {
const char *err = set_worker_param(cmd-pool, worker, elts[i].key,
elts[i].val);
This does seem to result in smax=0 being used, but it breaks something;
as far as I can see the requests never reach the backend. I have taken
the source from trunk and am loading the module into my 2.2.4 server; I
suppose that's far from certain to work. Or maybe the APR reslist is
not designed to work with smax=0.
I would much appreciate some input from someone who knows their way
around this code.
Regards,
Phil.
mod_proxy: Is smax=0 allowed ?
Dear Experts,
According to the mod_proxy docs, parameters to ProxyPass include:
smax max Upto the Soft Maximum number of connections will be
created on demand.
Any connections above smax are subject to a time to live
or ttl.
So, am I allowed to set smax=0 ? In my case, I don't want any
permanent connections to the backend server; when the site is being
used connections should be created and can persist for a while and then
time out. So I tried smax=0 ttl=60. But I have not seen any
connections being dropped. So I looked at the source, and in
proxy_util.c there is some code that does this:
if (worker-smax == 0 || worker-smax worker-hmax) {
worker-smax = worker-hmax;
}
So it looks like 0 is a sentinel for no value specified, and the
default of the hard maximum is applied in that case. Is this a correct
understanding of the code? If so, is this the desired behaviour? Is
it possible to keep no permanent connections to the backend server?
Many thanks for any advice.
Regards,
Phil.
(I'm subscribed to the digest so I'll see any replies sooner if you Cc: me)
Re: auth dbd pgsql issues
Chris Darroch wrote: Phil Endecott wrote: OK; my experience seems to be that in this respect 2.2.4 has regressed compared to 2.2.3 (though I may have been lucky in some way with my 2.2.3 setup) and certainly compared to 2.0.x + the 3rd-party mod_auth_pgsql. I don't know if this affects how the issue is prioritised for inclusion in future versions. I would love to see working authn_dbd ASAP. We have mod_authn_dbd + mod_dbd working but we use the trunk version of mod_dbd.c. It's a drop-in replacement for 2.2.x's mod_dbd.c; you can just copy it into 2.2.4 and recompile. $ svn cat http://svn.apache.org/repos/asf/httpd/httpd/trunk/modules/database/mod_dbd.c mod_dbd.c $ apxs2 -c mod_dbd.c # apxs2 -i mod_dbd.la # /etc/init.d/apache2 restart ..and it works. (Well, for all of 5 minutes so far.) I'm also pleased to see that performance is quite reasonable. I had assumed that I would need to add some sort of credentials caching, and have posted about this here before. But it looks as if, with a persistent connection, PostgreSQL responds very quickly and takes very little CPU to do so. Many thanks for your help. Phil.
Re: auth dbd pgsql issues
Hi Chris, thanks for your reply. Chris Darroch wrote: Phil Endecott wrote: http://marc.info/?l=apache-httpd-usersm=118765132424174w=2 My guess is that I'm encountering the known issues with DBD described here: http://marc.info/?l=apache-httpd-devm=116742014418304w=2 Am I right in thinking that this is fixed in the trunk but not in 2.2.4? What about 2.2.5/6 - I don't see anything in the 2.2.5 changelog. I think you're right about the problem you're encountering; the patches for 2.2.x await a third vote and so they're not in expected in 2.2.5/6, as it stands at the moment. OK; my experience seems to be that in this respect 2.2.4 has regressed compared to 2.2.3 (though I may have been lucky in some way with my 2.2.3 setup) and certainly compared to 2.0.x + the 3rd-party mod_auth_pgsql. I don't know if this affects how the issue is prioritised for inclusion in future versions. I would love to see working authn_dbd ASAP. - authn_dbd_password() uses the error message Error looking up %s in database for 3 different errors. It would be really great to have different messages in each case. I'd suggest opening a Bugzilla report and, if possible, attaching a patch file with the revised messages you'd like to see (and please add the keyword PatchAvailable in this case). done; http://issues.apache.org/bugzilla/show_bug.cgi?id=43210 - The mod_authn_dbd docs (http://httpd.apache.org/docs/2.2/mod/mod_authn_dbd.html) claim DBD drivers recognise both stdio-like %s and native syntax. Is this accurate? It seems that the postgresql DBD driver does some type magic based on the character after the %, which wouldn't be possible with the postgresql $1 syntax. Maybe $1 only works for strings (which would be OK for usernames, of course). (Does it correctly count the number of parameters if I use $1?) I seem to recall some type magic in this driver, but I'm not particularly familiar with it. If you encounter problems, I'd again suggest opening a Bugzilla report. I haven't actually got as far as encountering any problems with this since dbd is not working at all :-( However, see the documentation for apr_dbd_prepare() at http://apr.apache.org/docs/apr-util/trunk/group___a_p_r___util___d_b_d.html#gcf21412447c4357c64d1e9200a0f5eec To specify parameters of the prepared query, use %s, %d etc. (see below for full list) in place of database specific parameter syntax Looking at the source it's pretty clear that it understands only %letter synatx, not $n or even MySQL % syntax. So I've filed a bug on the mod_authn_dbd documentation: http://issues.apache.org/bugzilla/show_bug.cgi?id=43211 Regards, Phil.
auth dbd pgsql issues
Dear Experts, I am trying to track down a broken postgresql authn issue which I have described here: http://marc.info/?l=apache-httpd-usersm=118765132424174w=2 My guess is that I'm encountering the known issues with DBD described here: http://marc.info/?l=apache-httpd-devm=116742014418304w=2 Am I right in thinking that this is fixed in the trunk but not in 2.2.4? What about 2.2.5/6 - I don't see anything in the 2.2.5 changelog. Anyway, I have been looking at the source, and a few issues thoughts have occurred to me: - In mod_authn_dbd.c, a couple of global variables are used to point to the dbd_acquire and dbd_prepare functions. Am I right in thinking that this means you can have only one dbd driver for authn? So you can't for example, have postgresql in one virtual host and mysql in another? (Is this already a limitation of dbd?) This isn't a problem for me, unless these variables get set to NULL by a virtual host with no DBDriver directive. Maybe DBDriver should only appear in the global configuration? Is it certain that these variables are set before the server first forks? - It looks as if, when a new db connection is created, all prepared statements are prepared on the new connection. However, when a new prepared statement is created, it is not prepared on any existing connections. This is fine as long as all prepared statements are declared before any connections are established. Correct? - authn_dbd_password() uses the error message Error looking up %s in database for 3 different errors. It would be really great to have different messages in each case. - The mod_authn_dbd docs (http://httpd.apache.org/docs/2.2/mod/mod_authn_dbd.html) claim DBD drivers recognise both stdio-like %s and native syntax. Is this accurate? It seems that the postgresql DBD driver does some type magic based on the character after the %, which wouldn't be possible with the postgresql $1 syntax. Maybe $1 only works for strings (which would be OK for usernames, of course). (Does it correctly count the number of parameters if I use $1?) - The mod_dbd docs (http://httpd.apache.org/docs/2.2/mod/mod_dbd.html) say that DBDPersist can be 0 or 1; this should be on or off. Many thanks for any feedback. If the answer is this is fixed in 2.2.6, and that's due soon, I'll just wait. Regards, Phil.
PostgreSQL authentication in 2.2; caching?
Dear Experts, A few days ago I posted a question with this subject on the users lists, but have got no replies: http://thread.gmane.org/gmane.comp.apache.user/69944 The summary is: - Currently I'm using 2.0 with a 3rd-party mod_auth_pgsql. - In 2.2 I would presumably use the core DB-backed auth framework. - As far as I am aware, the core DB-backed auth modules don't implement caching of credentials. The 2.0 mod_auth_pgsql that I'm currently using does have this, and it's essential for my application. - There is a mod_authn_cache at http://mod-auth.sourceforge.net/docs/mod_authn_cache/ but it looks a bit abandoned; does anyone know anything about it? - Ideally I would also want the cache to be flushed when the database is changed; this could be done using PostgreSQL's asynchronous notification feature. I imagine that I would have to do this myself. I mentioned this here ages ago when I first considered moving to 2.2: http://thread.gmane.org/gmane.comp.apache.devel/23903/focus=23904 If anyone would be interested in helping me implement this I would love to hear from them. Any thoughts anyone? Regards, Phil.
Compiling a C++ module with g++ on Solaris
Dear All, I'm the author of Anyterm (http://anyterm.org/) which uses an Apache module written in C++. This works OK on Linux. However, a user has attempted to compile it on Solaris and it fails at run time. Specifically, the first time that any C++ memory allocation stuff happens - it creates a std::string in the post_config hook function - it segfaults deep inside libstdc++. This user is using g++ 3.4.3 and the Sun linker on Solaris 10 x86. I'm curous to know if any readers have any experience compiling C++ Apache modules on this platform, and whether there are any gotchas to look out for. Is it possible that there is some libstdc++ initialisation that hasn't happened? I could imagine that this would require special support from the linker or the dlopen stuff, and that that behaves differently with Sun's libc and linker than on Linux. Any ideas? He has tried with 2.0.5? and 2.2.? with the same result. Cheers, Phil.
Re: Accessing POST request body from a module
I am porting some CGI code to run as a module. It is all pretty
straightforward, except that I can't work out how to get at a POST
request's body.
Thanks to Michael for this:
while (ap_get_brigade(r-input_filters, brigade, AP_MODE_READBYTES,
APR_BLOCK_READ, len) == APR_SUCCESS) {
apr_brigade_flatten(brigade, buf, len);
apr_brigade_cleanup(brigade);
tlen += len;
if (tlen == count_bytes || !len) {
break;
}
buf += len;
len = count_bytes - tlen;
}
That looks good and I've tried something like it, but it seems to
segfault as soon as ap_get_brigade is called. I'm guessing that I need
to create the brigade first, but apr_brigade_create has a second
mysterious parameter that I can't work out. Michael, maybe you just
need to post the five lines before those??
BTW, is there any better documentation for this than
http://docx.webperf.org/group__APR__Util__Bucket__Brigades.html?
Nick Kew wrote:
If you had a usable-looking email address. I could reply with
exactly what you need
I'm sorry that it doesn't look useable to you, but I can assure you that
it does work; it's not even filtered. It would be quite hard to
subscribe to this list with a non-functioning address, I think.
Regards,
--Phil.
Accessing POST request body from a module
Dear All, I am porting some CGI code to run as a module. It is all pretty straightforward, except that I can't work out how to get at a POST request's body. The CGI code reads from stdin, which doesn't seem to be the right thing to do in the module. I'm probably missing something obvious; maybe I'm not searching for the right keywords. Can someone point me in the right direction? Many Thanks, --Phil.
Re: AP_INIT_TAKE1 in 2.2 and C++
Thanks for the replies. The patch to ap_config.h that Joe Orton posted is reported to fix the problem. Can this be included in the next release? Cheers, --Phil.
AP_INIT_TAKE1 in 2.2 and C++
Dear All,
I'm the author of Anyterm (http://anyterm.org), which includes an Apache
module. Although I haven't yet tried to compile it for Apache 2.2 some
of my users have, and they're having problems. See
http://anyterm.org/forums/viewtopic.php?pid=768#p768. It looks as if
the definition of AP_INIT_TAKE1 has changed at some point, and now looks
like this:
# define AP_INIT_TAKE1(directive, func, mconfig, where, help) \
{ directive, { .take1=func }, mconfig, where, TAKE1, help }
That {.take1=func} syntax was a new one on me. A quick test suggests
that it is legal in C but not in C++, and I compile my module with g++.
I can see that this definition is selected by a #if, with a regular
definition without the whacky syntax also available. So, would it be OK
to have this alternative more conventional declaration when __cplusplus
is defined? Is there something I can do before #including the Apache
headers to make it all work?
Cheers,
--Phil.
p.s. Either I'm being dense, or there isn't a Doxygenated source tree
linked anywhere from the httpd web pages. apr.apache.org links to one
for apr only. Did there used to be, or am I imagining it?
mod_auth[nz]_dbd password caching
Dear All, First of all, congratulation on the release of 2.2. I use mod_auth_pgsql at http://anyterm.org/my.html, and found a problem earlier in the year. To get reasonable performance you need to use the module's caching mechanism, but this cache is not flushed or updated when the database changes. So things don't work properly when the user changes their password. I started to think about fixing it myself but quickly realised that both the database and authentication frameworks were changing in 2.1+ and decided to wait before doing anything. I now see that 2.2 has mod_auth[nz]_dbd - great! However, as far as I can see from http://httpd.apache.org/docs/2.2/mod/mod_authn_dbd.html and the source, this new module doesn't do any caching. Is this true? To get the sort of performance that I need for my site I really need in-memory caching of passwords, but I also need to solve mod_auth_pgsql's non-updating problem. The solution to this is to use PostgreSQL's asynchronous notification mechanism: the module issues a LISTEN command and is then notified when the password table changes. I don't know if the APR DB interface has any support for this (it doesn't seem to be documented at all at http://apr.apache.org/docs/apr-util/modules.html); even if it does, it is not portable to other databases. Has anyone looked at this? If no-one is working on this and you think it would be a useful feature to add, I may be able to write something with a bit of help. Cheers, --Phil.
APR_FORCE_ATOMIC_GENERIC setting in Mandrake 10.1
Dear Apache Experts, I'm the author of Anyterm, an Apache2 module and Javascript thing that creates a box on a web page that behaves like a shell using XmlHttpRequest. (http://chezphil.org/anyterm/) I've had a couple of reports from Mandrake 10.1 users who have got the following error while compiling: /home/bravo/tmp/ccqUjIBR.s: Assembler messages: /home/bravo/tmp/ccqUjIBR.s:30: Error: bad register name `%dil' My code uses apr_atomic_cas(). It seems that on my Debian system, apr-0/apr.h defines APR_FORCE_ATOMIC_GENERIC as 1, while on the systems that show the error it is defined as 0. Can someone explain what should be happening here? Under what circumstances should APR_FORCE_ATOMIC_GENERIC be set? My guess is that people who are distributing binary packages (eg Mandrake) should be forcing it to 1 with a configure option since they don't know the precise details of the machine on which it will run. Or, is it my responsibility to set it? Many thanks for any suggestions you can offer. I can try to provide more details if that would be useful but I have never seen the error happen first hand. Regards, Phil.
Re: APR_FORCE_ATOMIC_GENERIC setting in Mandrake 10.1
Phil Endecott wrote: I've had a couple of reports from Mandrake 10.1 users who have got the following error while compiling: /home/bravo/tmp/ccqUjIBR.s: Assembler messages: /home/bravo/tmp/ccqUjIBR.s:30: Error: bad register name `%dil' My code uses apr_atomic_cas(). I've been investigating this further, and I think that the problem is with the asm version of apr_atomic_cas that you use. It looks like you have the same code in the current trunk, in atomic.c, as in the released version's .h that I have: asm volatile (lock; cmpxchgl %1, %2 : =a (prev) : r (with), m (*(mem)), 0(cmp) : memory, cc); I've asked about this on the gcc mailing list and they confirm that r should be q. q is an x86-specific thing that indicates the subset of x86 / AMD64 registers that can be used with this instruction; using r you risk gcc choosing an inappropriate register. (I realise now that the APR list would be a better place to mention this - are the appropriate people also listening here?) Regards, Phil.
